Initial commit: video effects processing system

Add S-expression based video effects pipeline with modular effect definitions, constructs, and recipe files. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-19 12:34:45 +00:00
commit 406cc7c0c7
171 changed files with 13406 additions and 0 deletions
--- a/libs/plan
+++ b/libs/plan
@@ -0,0 +1,68 @@
+Exactly. You're describing a DAG of pipelines that can branch and merge:                                                                                                                                                                                                                                
+                                                                                                                                                                                                                                                                                                          
+  audio-a ─→ analyze ─→ plan-a ─┐                                                                                                                                                                                                                                                                         
+                                 ├─→ combine ─→ final                                                                                                                                                                                                                                                     
+  audio-b ─→ analyze ─→ plan-b ─┘                                                                                                                                                                                                                                                                         
+                                                                                                                                                                                                                                                                                                          
+  videos ─→ analyze ─────────────┴─→ (shared by both plans)                                                                                                                                                                                                                                               
+                                                                                                                                                                                                                                                                                                          
+  Each node is independently cacheable. Parallel branches run in tandem.                                                                                                                                                                                                                                  
+                                                                                                                                                                                                                                                                                                          
+  A clean syntax might be:                                                                                                                                                                                                                                                                                
+                                                                                                                                                                                                                                                                                                          
+  (recipe "multi-track-video"                                                                                                                                                                                                                                                                             
+    :encoding (...)                                                                                                                                                                                                                                                                                       
+                                                                                                                                                                                                                                                                                                          
+    ;; Sources (stage 0 - always available)                                                                                                                                                                                                                                                               
+    (def audio-a (source "track1.mp3"))                                                                                                                                                                                                                                                                   
+    (def audio-b (source "track2.mp3"))                                                                                                                                                                                                                                                                   
+    (def videos (source-glob "videos/*.mp4"))                                                                                                                                                                                                                                                             
+                                                                                                                                                                                                                                                                                                          
+    ;; Analysis stages (run in parallel, cached by input hash)                                                                                                                                                                                                                                            
+    (stage :analyze-a                                                                                                                                                                                                                                                                                     
+      (def beats-a (-> audio-a (analyze beats))))                                                                                                                                                                                                                                                         
+                                                                                                                                                                                                                                                                                                          
+    (stage :analyze-b                                                                                                                                                                                                                                                                                     
+      (def beats-b (-> audio-b (analyze beats))))                                                                                                                                                                                                                                                         
+                                                                                                                                                                                                                                                                                                          
+    (stage :analyze-videos                                                                                                                                                                                                                                                                                
+      (def video-infos (-> videos (analyze-each video-info))))                                                                                                                                                                                                                                            
+                                                                                                                                                                                                                                                                                                          
+    ;; Planning stages (depend on analysis, explicit deps)                                                                                                                                                                                                                                                
+    (stage :plan-a :requires [:analyze-a :analyze-videos]                                                                                                                                                                                                                                                 
+      (def segments-a (make-segments :beats beats-a :video-infos video-infos)))                                                                                                                                                                                                                           
+                                                                                                                                                                                                                                                                                                          
+    (stage :plan-b :requires [:analyze-b :analyze-videos]                                                                                                                                                                                                                                                 
+      (def segments-b (make-segments :beats beats-b :video-infos video-infos)))                                                                                                                                                                                                                           
+                                                                                                                                                                                                                                                                                                          
+    ;; Render stages (can parallelize)                                                                                                                                                                                                                                                                    
+    (stage :render-a :requires [:plan-a]                                                                                                                                                                                                                                                                  
+      (def rendered-a (-> segments-a (sequence))))                                                                                                                                                                                                                                                        
+                                                                                                                                                                                                                                                                                                          
+    (stage :render-b :requires [:plan-b]                                                                                                                                                                                                                                                                  
+      (def rendered-b (-> segments-b (sequence))))                                                                                                                                                                                                                                                        
+                                                                                                                                                                                                                                                                                                          
+    ;; Final combine                                                                                                                                                                                                                                                                                      
+    (stage :output :requires [:render-a :render-b]                                                                                                                                                                                                                                                        
+      (-> (list rendered-a rendered-b)                                                                                                                                                                                                                                                                    
+          (concat)                                                                                                                                                                                                                                                                                        
+          (crossfade :duration 2)                                                                                                                                                                                                                                                                         
+          (mux audio-a audio-b))))                                                                                                                                                                                                                                                                        
+                                                                                                                                                                                                                                                                                                          
+  What this gives you:                                                                                                                                                                                                                                                                                    
+                                                                                                                                                                                                                                                                                                          
+  1. Explicit data availability - :requires declares what's available                                                                                                                                                                                                                                     
+  2. Parallel execution - :analyze-a and :analyze-b run simultaneously                                                                                                                                                                                                                                    
+  3. Granular caching - each stage output cached by its inputs' hashes                                                                                                                                                                                                                                    
+  4. Flexible composition - add more tracks, branches, merge points as needed                                                                                                                                                                                                                             
+  5. Clear errors - referencing beats-a before :analyze-a is a compile error                                                                                                                                                                                                                              
+                                                                                                                                                                                                                                                                                                          
+  Changes needed to sexp system:                                                                                                                                                                                                                                                                          
+                                                                                                                                                                                                                                                                                                          
+  1. stage form with :requires dependency declaration                                                                                                                                                                                                                                                     
+  2. Stage scheduler that builds execution DAG                                                                                                                                                                                                                                                            
+  3. Cache layer keyed by stage + input hashes                                                                                                                                                                                                                                                            
+  4. Dict iteration (keys, for-each) for generic constructs                                                                                                                                                                                                                                               
+                                                                                                                                                                                                                                                                                                          
+  Want to prototype this direction?                                                                                                                                                                                                                                                                       
+