26 Commits

Author SHA1 Message Date
giles
48018d09b7 Add CI/CD workflow
Some checks failed
Build and Deploy / build-and-deploy (push) Failing after 1m2s
GPU Worker CI/CD / test (push) Failing after 1m4s
GPU Worker CI/CD / deploy (push) Has been skipped
2026-02-06 10:45:13 +00:00
giles
0534081e44 Fix HLS validation for multi-resolution output
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
MultiResolutionHLSOutput creates files in subdirectories:
- original/playlist.m3u8 instead of stream.m3u8
- original/segment_*.ts instead of segment_*.ts

The validation now checks both paths.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 21:16:58 +00:00
giles
2f56ffc472 Fix live HLS streaming with dynamic quality playlist URLs
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
The problem: HLS.js caches quality playlist URLs from the master playlist.
Even when we update the master playlist CID, HLS.js keeps polling the same
static quality CID URL, so it never sees new segments.

The fix:
- Store quality-level CIDs in database (quality_playlists JSONB column)
- Generate master playlist with dynamic URLs (/runs/{id}/quality/{name}/playlist.m3u8)
- Add quality endpoint that fetches LATEST CID from database
- HLS.js now polls our dynamic endpoints which return fresh content

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 21:07:29 +00:00
giles
43d73c7bf7 Fix segment deletion and add progress callback
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- Remove stream_dir deletion in finally block to prevent IPFS upload failures
- Add on_progress callback to StreamInterpreter for real-time progress updates
- Task now sends progress updates to Celery state during rendering

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 20:13:27 +00:00
giles
d5f30035da Fix async event loop conflict in resolve_asset
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
When running with --pool=solo, there may already be a running event loop.
Use thread pool to run async coroutines when a loop is already running.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:52:50 +00:00
giles
ed617fcdd6 Fix lazy audio path resolution for GPU streaming
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
Audio playback path was being resolved during parsing when database
may not be ready, causing fallback to non-existent path. Now resolves
lazily when stream starts, matching how audio analyzer works.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 11:32:04 +00:00
giles
ef4bc24eda Use GPUVideoSource for hardware-accelerated video decoding
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- CIDVideoSource now uses GPUVideoSource when GPU is available
- Enables CUDA hardware decoding for video sources
- Should significantly improve rendering performance

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 01:03:16 +00:00
giles
0bd8ee71c7 Fix MP4 mux for web playback: add faststart and genpts
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- Add -movflags +faststart to move moov atom to start
- Add -fflags +genpts for proper timestamp generation
- Fixes jerky playback and video/audio desync

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:32:45 +00:00
giles
9151d2c2a8 Update IPFS playlist CID in database during streaming for live HLS
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- Add on_playlist_update callback to IPFSHLSOutput
- Pass callback through StreamInterpreter to output
- Update database with playlist CID as segments are created
- Enables live HLS redirect to IPFS before rendering completes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:23:49 +00:00
giles
ed5ef2bf39 Add ipfs_playlist_cid to pending_runs and fail-fast on DB errors
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- Add ipfs_playlist_cid column to pending_runs schema with migration
- Add pool guards to critical database functions (RuntimeError if not initialized)
- Add update_pending_run_playlist() function for streaming
- Update streaming task to save playlist CID to DB for HLS redirect
- Change database error handling from warning to raising exception

Errors should fail fast and explicitly, not be silently swallowed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:02:18 +00:00
giles
86830019ad Add IPFS HLS streaming and GPU optimizations
- Add IPFSHLSOutput class that uploads segments to IPFS as they're created
- Update streaming task to use IPFS HLS output for distributed streaming
- Add /ipfs-stream endpoint to get IPFS playlist URL
- Update /stream endpoint to redirect to IPFS when available
- Add GPU persistence mode (STREAMING_GPU_PERSIST=1) to keep frames on GPU
- Add hardware video decoding (NVDEC) support for faster video processing
- Add GPU-accelerated primitive libraries: blending_gpu, color_ops_gpu, geometry_gpu
- Add streaming_gpu module with GPUFrame class for tracking CPU/GPU data location
- Add Dockerfile.gpu for building GPU-enabled worker image

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 20:23:16 +00:00
giles
a57be27907 Add live video streaming for in-progress renders 2026-02-03 00:27:19 +00:00
giles
487acdd606 Fix global declaration placement in streaming task 2026-02-03 00:16:49 +00:00
giles
6b2991bf24 Fix database event loop conflicts in streaming task 2026-02-03 00:14:42 +00:00
giles
3ec045c533 Add _stream_time and skip() to CIDVideoSource 2026-02-03 00:07:33 +00:00
giles
3bff130e57 Add path property to CIDVideoSource 2026-02-03 00:06:10 +00:00
giles
414cbddd66 Fix VideoSource import path 2026-02-03 00:02:23 +00:00
giles
89b2fd3d2e Add debug logging to resolve_asset 2026-02-03 00:00:30 +00:00
giles
d8360e5945 Use Celery task logger for debug 2026-02-02 23:56:13 +00:00
giles
44066e9bdd Use warnings for debug output 2026-02-02 23:54:18 +00:00
giles
32474380fa Add module load debug 2026-02-02 23:46:22 +00:00
giles
5bd055f031 Use print for debug output 2026-02-02 23:44:34 +00:00
giles
cbdae70b66 Add more debug logging for primitive override 2026-02-02 23:41:05 +00:00
giles
e6dd6e851c Add debug logging for CID video source resolution
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 23:34:08 +00:00
giles
d20eef76ad Fix completed runs not appearing in list + add purge-failed endpoint
- Update save_run_cache to also update actor_id, recipe, inputs on conflict
- Add logging for actor_id when saving runs to run_cache
- Add admin endpoint DELETE /runs/admin/purge-failed to delete all failed runs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 23:24:39 +00:00
giles
bb458aa924 Replace batch DAG system with streaming architecture
- Remove legacy_tasks.py, hybrid_state.py, render.py
- Remove old task modules (analyze, execute, execute_sexp, orchestrate)
- Add streaming interpreter from test repo
- Add sexp_effects with primitives and video effects
- Add streaming Celery task with CID-based asset resolution
- Support both CID and friendly name references for assets
- Add .dockerignore to prevent local clones from conflicting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 19:10:11 +00:00