- Remove stream_dir deletion in finally block to prevent IPFS upload failures
- Add on_progress callback to StreamInterpreter for real-time progress updates
- Task now sends progress updates to Celery state during rendering
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When running with --pool=solo, there may already be a running event loop.
Use thread pool to run async coroutines when a loop is already running.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Audio playback path was being resolved during parsing when database
may not be ready, causing fallback to non-existent path. Now resolves
lazily when stream starts, matching how audio analyzer works.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- CIDVideoSource now uses GPUVideoSource when GPU is available
- Enables CUDA hardware decoding for video sources
- Should significantly improve rendering performance
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add -movflags +faststart to move moov atom to start
- Add -fflags +genpts for proper timestamp generation
- Fixes jerky playback and video/audio desync
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add on_playlist_update callback to IPFSHLSOutput
- Pass callback through StreamInterpreter to output
- Update database with playlist CID as segments are created
- Enables live HLS redirect to IPFS before rendering completes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ipfs_playlist_cid column to pending_runs schema with migration
- Add pool guards to critical database functions (RuntimeError if not initialized)
- Add update_pending_run_playlist() function for streaming
- Update streaming task to save playlist CID to DB for HLS redirect
- Change database error handling from warning to raising exception
Errors should fail fast and explicitly, not be silently swallowed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add IPFSHLSOutput class that uploads segments to IPFS as they're created
- Update streaming task to use IPFS HLS output for distributed streaming
- Add /ipfs-stream endpoint to get IPFS playlist URL
- Update /stream endpoint to redirect to IPFS when available
- Add GPU persistence mode (STREAMING_GPU_PERSIST=1) to keep frames on GPU
- Add hardware video decoding (NVDEC) support for faster video processing
- Add GPU-accelerated primitive libraries: blending_gpu, color_ops_gpu, geometry_gpu
- Add streaming_gpu module with GPUFrame class for tracking CPU/GPU data location
- Add Dockerfile.gpu for building GPU-enabled worker image
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update save_run_cache to also update actor_id, recipe, inputs on conflict
- Add logging for actor_id when saving runs to run_cache
- Add admin endpoint DELETE /runs/admin/purge-failed to delete all failed runs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove legacy_tasks.py, hybrid_state.py, render.py
- Remove old task modules (analyze, execute, execute_sexp, orchestrate)
- Add streaming interpreter from test repo
- Add sexp_effects with primitives and video effects
- Add streaming Celery task with CID-based asset resolution
- Support both CID and friendly name references for assets
- Add .dockerignore to prevent local clones from conflicting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove execute_level() from tasks/execute.py (defined but never called)
- Remove render_dog_from_cat() from legacy_tasks.py (test convenience, never used)
- Remove duplicate file_hash() from legacy_tasks.py, import from cache_manager
- Remove unused hashlib import
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix all registry lookups to use "cid" instead of "hash" key
- app/routers/recipes.py: asset and effect resolution
- tasks/execute_sexp.py: effect config lookups
- server_legacy.py references (now deleted)
- Prefer IPFS CID over local hash in cache operations
- cache_service.py: import_from_ipfs, upload_content
- orchestrate.py: plan caching
- legacy_tasks.py: node hash tracking
Remove ~7800 lines of dead code:
- server_legacy.py: replaced by modular app/ structure
- tasks/*_cid.py: unused refactoring only imported by server_legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation
This completes the transition to CID-only content addressing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload endpoint returns both CID and content_hash
- Cache manager handles both SHA3-256 hashes and IPFS CIDs
- get_by_cid() fetches from IPFS if not cached locally
- Execute tasks support :cid in addition to :hash
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Execute COMPOUND nodes with combined FFmpeg filter chain
- Handle TRANSFORM, RESIZE, SEGMENT filters in chain
- Migrate orchestrator to S-expression recipes (remove YAML)
- Update API endpoints to use recipe_sexp parameter
- Extract analysis nodes from recipe for dynamic analysis
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Recipe service now only handles S-expressions
- Removed yaml import and all YAML parsing code
- Plans are just node outputs - cached by content hash
- Run service looks up plans from cache, falls back to legacy dir
Code is data. Everything is S-expressions.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Plans now go through cache_manager.put() for IPFS pinning
- Returns plan_cache_id and plan_ipfs_cid in result
- Plan S-expression is content-addressed like everything else
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Orchestrator saves plan as {plan_id}.sexp (content-addressed)
- Also saves {run_id}.sexp for easy lookup by run
- Falls back to JSON for legacy plans without to_sexp_string()
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add format detection that correctly handles ; comments
- Import artdag.sexp parser/compiler with YAML fallback
- Add execute_step_sexp and run_plan_sexp Celery tasks
- Update recipe upload to handle both S-expr and YAML formats
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements HybridStateManager providing fast local Redis operations
with background IPNS sync for eventual consistency across L1 nodes.
- hybrid_state.py: Centralized state management (cache, claims, analysis, plans, runs)
- Updated execute_cid.py, analyze_cid.py, orchestrate_cid.py to use state manager
- Background IPNS sync (configurable interval, disabled by default)
- Atomic claiming with Redis SETNX for preventing duplicate work
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Complete pipeline with everything on IPFS:
- register_input_cid / register_recipe_cid
- generate_plan_cid (stores plan on IPFS)
- execute_plan_from_cid (fetches plan from IPFS)
- run_recipe_cid (full pipeline, returns output CID)
- run_from_local (convenience: local files → IPFS → run)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changed default from /data/cache to ~/.artdag/cache for local runs.
Docker sets CACHE_DIR=/data/cache via environment variable.
Files updated:
- tasks/analyze.py
- tasks/orchestrate.py
- app/config.py
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Simplified step execution where:
- Steps receive CIDs, produce CIDs
- No local cache management (IPFS handles it)
- Minimal Redis: just claims + cache_id→CID mapping
- Temp workspace for execution, cleaned up after
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
RunStatus now stores:
- plan_id, plan_name for linking to execution plan
- step_results for per-step execution status
- all_outputs for all artifacts from all steps
Plan visualization:
- Shows human-readable step names from recipe structure
- Video/audio artifact preview on node click
- Outputs list with links to cached artifacts
- Stats reflect actual execution status (completed/cached/pending)
Execution:
- Step results include outputs list with cache_ids
- run_plan returns all outputs from all steps
- Support for completed_by_other status
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New files:
- claiming.py - Redis Lua scripts for atomic task claiming
- tasks/analyze.py - Analysis Celery task
- tasks/execute.py - Step execution with IPFS-backed cache
- tasks/orchestrate.py - Plan orchestration (run_plan, run_recipe)
New API endpoints (/api/v2/):
- POST /api/v2/plan - Generate execution plan
- POST /api/v2/execute - Execute a plan
- POST /api/v2/run-recipe - Full 3-phase pipeline
- GET /api/v2/run/{run_id} - Get run status
Features:
- Hash-based task claiming prevents duplicate work
- Parallel execution within dependency levels
- IPFS-backed cache for durability
- Integration with artdag planning module
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
