Commit Graph

15 Commits

Author SHA1 Message Date
giles
3116a70c3e Fix IPFS upload: sync instead of background task
The background IPFS upload task was running on workers that don't have
the file locally, causing uploads to fail silently. Now uploads go to
IPFS synchronously so the IPFS CID is available immediately.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 21:17:22 +00:00
giles
86830019ad Add IPFS HLS streaming and GPU optimizations
- Add IPFSHLSOutput class that uploads segments to IPFS as they're created
- Update streaming task to use IPFS HLS output for distributed streaming
- Add /ipfs-stream endpoint to get IPFS playlist URL
- Update /stream endpoint to redirect to IPFS when available
- Add GPU persistence mode (STREAMING_GPU_PERSIST=1) to keep frames on GPU
- Add hardware video decoding (NVDEC) support for faster video processing
- Add GPU-accelerated primitive libraries: blending_gpu, color_ops_gpu, geometry_gpu
- Add streaming_gpu module with GPUFrame class for tracking CPU/GPU data location
- Add Dockerfile.gpu for building GPU-enabled worker image

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 20:23:16 +00:00
giles
d20eef76ad Fix completed runs not appearing in list + add purge-failed endpoint
- Update save_run_cache to also update actor_id, recipe, inputs on conflict
- Add logging for actor_id when saving runs to run_cache
- Add admin endpoint DELETE /runs/admin/purge-failed to delete all failed runs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 23:24:39 +00:00
gilesb
8bf6f87c2a Implement ownership model for all cached content deletion
- cache_service.delete_content: Remove user's ownership link first,
  only delete actual file if no other owners remain

- cache_manager.discard_activity_outputs_only: Check if outputs and
  intermediates are used by other activities before deleting

- run_service.discard_run: Now cleans up run outputs/intermediates
  (only if not shared by other runs)

- home.py clear_user_data: Use ownership model for effects and media
  deletion instead of directly deleting files

The ownership model ensures:
1. Multiple users can "own" the same cached content
2. Deleting removes the user's ownership link (item_types entry)
3. Actual files only deleted when no owners remain (garbage collection)
4. Shared intermediates between runs are preserved

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 20:02:27 +00:00
gilesb
ee8719ac0b Fix media friendly names, metadata display, output recording, and plan display
- Add friendly name display to media detail and list pages
- Unpack nested meta fields to top level for template access
- Fix output_cid mismatch: use IPFS CID consistently between cache and database
- Add dual-indexing in cache_manager to map both IPFS CID and local hash
- Fix plan display: accept IPFS CIDs (Qm..., bafy...) not just 64-char hashes
- Add friendly names to recipe listing
- Add recipe upload button and handler to recipes list
- Add debug logging to recipe listing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:21:39 +00:00
gilesb
585c75e846 Fix item visibility bugs and add effects web UI
- Fix recipe filter to allow owner=None (S-expression compiled recipes)
- Fix media uploads to use category (video/image/audio) not MIME type
- Fix IPFS imports to detect and store correct media type
- Add Effects navigation link between Recipes and Media
- Create effects list and detail templates with upload functionality
- Add cache/not_found.html template (was missing)
- Add type annotations to service classes
- Add tests for item visibility and effects web UI (30 tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:01:54 +00:00
gilesb
60344b34f4 Fix registry lookups to use cid, remove dead legacy code
- Fix all registry lookups to use "cid" instead of "hash" key
  - app/routers/recipes.py: asset and effect resolution
  - tasks/execute_sexp.py: effect config lookups
  - server_legacy.py references (now deleted)
- Prefer IPFS CID over local hash in cache operations
  - cache_service.py: import_from_ipfs, upload_content
  - orchestrate.py: plan caching
  - legacy_tasks.py: node hash tracking

Remove ~7800 lines of dead code:
- server_legacy.py: replaced by modular app/ structure
- tasks/*_cid.py: unused refactoring only imported by server_legacy

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 09:09:40 +00:00
gilesb
92d26b2b72 Rename content_hash/output_hash to cid throughout
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation

This completes the transition to CID-only content addressing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:02:44 +00:00
gilesb
494a2a8650 Add IPFS CID support for asset lookup
- Upload endpoint returns both CID and content_hash
- Cache manager handles both SHA3-256 hashes and IPFS CIDs
- get_by_cid() fetches from IPFS if not cached locally
- Execute tasks support :cid in addition to :hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:36:18 +00:00
gilesb
921d81421a Fix cache method name: get_content_path -> get_by_content_hash
The L1CacheManager uses get_by_content_hash not get_content_path.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 16:36:29 +00:00
gilesb
27cbb0a85c Store MIME type in database during upload
Detect actual MIME type from file content and store it instead of
generic "media" type. This enables proper media categorization
and filtering in the UI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 16:18:56 +00:00
giles
e8501de466 Fix list_media to use get_user_items instead of list_cache_items
list_cache_items doesn't accept actor_id parameter.
Use get_user_items which properly filters by actor_id and item_type.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:10:49 +00:00
giles
854396680f Refactor storage: remove Redis duplication, use proper data tiers
- Recipes: Now content-addressed only (cache + IPFS), removed Redis storage
- Runs: Completed runs stored in PostgreSQL, Redis only for task_id mapping
- Add list_runs_by_actor() to database.py for paginated run queries
- Add list_by_type() to cache_manager for filtering by node_type
- Fix upload endpoint to return size and filename fields
- Fix recipe run endpoint with proper DAG input binding
- Fix get_run_service() dependency to pass database module

Storage architecture:
- Redis: Ephemeral only (sessions, task mappings with TTL)
- PostgreSQL: Permanent records (completed runs, metadata)
- Cache: Content-addressed files (recipes, media, outputs)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:05:31 +00:00
giles
8591faf0fc Fix CacheService list_media and recipe inputs type
- Add list_media method to CacheService
- Change recipe run inputs from List to dict

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:28:27 +00:00
giles
adc876dbd6 Add modular app structure for L1 server refactoring
Phase 2 of the full modernization:
- App factory pattern with create_app()
- Settings via dataclass with env vars
- Dependency injection container
- Router stubs for auth, storage, api, recipes, cache, runs
- Service layer stubs for run, recipe, cache
- Repository layer placeholder

Routes are stubs that import from legacy server.py during migration.
Next: Migrate each router fully with templates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:08:08 +00:00