init_db() was creating new pools without checking if one already exists,
causing "too many clients already" errors under load. Added early return
if pool is already initialized and set explicit pool limits (min=2, max=10).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add IPFSHLSOutput class that uploads segments to IPFS as they're created
- Update streaming task to use IPFS HLS output for distributed streaming
- Add /ipfs-stream endpoint to get IPFS playlist URL
- Update /stream endpoint to redirect to IPFS when available
- Add GPU persistence mode (STREAMING_GPU_PERSIST=1) to keep frames on GPU
- Add hardware video decoding (NVDEC) support for faster video processing
- Add GPU-accelerated primitive libraries: blending_gpu, color_ops_gpu, geometry_gpu
- Add streaming_gpu module with GPUFrame class for tracking CPU/GPU data location
- Add Dockerfile.gpu for building GPU-enabled worker image
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update save_run_cache to also update actor_id, recipe, inputs on conflict
- Add logging for actor_id when saving runs to run_cache
- Add admin endpoint DELETE /runs/admin/purge-failed to delete all failed runs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add plan_cid column to pending_runs table schema
- Add update_pending_run_plan() function to save plan_cid
- Update get_pending_run() to return plan_cid
- Save plan_cid right after storing plan to IPFS (before execution)
- Plan is now available even if run fails
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Track when items are being saved to database to diagnose
why recipes show 0 in stats but effects show correctly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add plan_cid column to run_cache schema
- Store DAG JSON to IPFS during execute_dag task
- Return plan_cid in run status and list APIs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- discard_run now deletes from run_cache and pending_runs tables
- Add delete_run_cache() and delete_pending_run() database functions
- Previously clear-data only cleared Redis, leaving DB cache intact
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add friendly_names table with unique constraints per actor
- Create NamingService with HMAC-signed timestamp version IDs
- Version IDs use base32-crockford encoding, always increase alphabetically
- Name normalization: spaces/underscores to dashes, lowercase, strip special chars
- Format: "my-effect 01hw3x9k" (space separator ensures uniqueness)
- Integrate naming into recipe, effect, and media uploads
- Resolve friendly names to CIDs during DAG execution
- Update effects UI to display friendly names
- Add 30 tests covering normalization, parsing, and service structure
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation
This completes the transition to CID-only content addressing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Handle dict inputs ({"node": "id"}) when building DAG edges
- Add normalize_inputs() to convert dict inputs to node IDs for steps
- Fix _parse_inputs to use _json.loads (correct import alias)
- Add SOURCE/EFFECT/SEQUENCE colors to node color maps
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Parse inputs JSON string to list in list_runs_by_actor
- Add status field to completed runs
- Prevents "136 files" display when inputs is a string
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
JSONB columns may return strings in some cases - explicitly parse
inputs field to ensure it's always a list.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Store pending runs in PostgreSQL for durability across restarts
- Add recovery method for orphaned runs
- Increase Celery result_expires to 7 days
- Add task_reject_on_worker_lost for automatic re-queuing
- Add logging to recipe list to debug filter issues
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Recipes: Now content-addressed only (cache + IPFS), removed Redis storage
- Runs: Completed runs stored in PostgreSQL, Redis only for task_id mapping
- Add list_runs_by_actor() to database.py for paginated run queries
- Add list_by_type() to cache_manager for filtering by node_type
- Fix upload endpoint to return size and filename fields
- Fix recipe run endpoint with proper DAG input binding
- Fix get_run_service() dependency to pass database module
Storage architecture:
- Redis: Ephemeral only (sessions, task mappings with TTL)
- PostgreSQL: Permanent records (completed runs, metadata)
- Cache: Content-addressed files (recipes, media, outputs)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add activity_id column to l2_shares table
- Store activity_id when publishing to L2
- Link to /activities/{activity_id} instead of /assets/{name}
- Falls back to asset link if no activity_id
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Database: Add description field, remove unique constraint to allow
multiple configs of same provider type
- UI: Main page shows provider types as cards with counts
- UI: Per-type page (/storage/type/{type}) for managing configs
- API: Add get_user_storage_by_type() for filtered queries
- Form: Add description field for distinguishing configs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Prepares L1 for distributed storage integration with user-provided
storage backends (Pinata, web3.storage, local). The storage config
is synced from L2 or can be configured locally per actor.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add run_cache table for fast run_id -> output lookup
- compute_run_id() computes deterministic run_id from inputs + recipe
- create_run checks L1 cache then L2 before running Celery
- If output exists on L2 but not L1, pulls from IPFS
- Saves run results to cache on completion
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Database: Use DISTINCT ON to deduplicate items by content_hash
- Database: Count unique content_hashes in count_user_items
- Server: Fix media card link from /ui/cache to /cache
- Server: Use /raw endpoint for image thumbnails
- Server: Add seen_hashes dedup in media list iteration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace all load_cache_meta/save_cache_meta calls with database functions
- Update get_user_cache_hashes to async, query both DB and legacy JSON
- Replace get_user_from_cookie with get_user_context_from_cookie throughout
- Update all endpoints to use UserContext (ctx) instead of plain username
- Update render_page calls to use ctx.actor_id
- Add high-level database helper functions:
- save_item_metadata, load_item_metadata, update_item_metadata
- save_l2_share, get_user_items, count_user_items
- Keep legacy JSON functions for backwards compatibility during migration
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Multi-user support for cache items:
- item_types now includes actor_id (@user@server format)
- l2_shares now includes actor_id
- Same cache item can be owned by multiple users with different types
- Deleting severs user's connection, not the actual file
- Cache files only removed when no users reference them
- Added has_remaining_references() and cleanup_orphaned_cache_item()
- Updated all CRUD functions to include actor_id parameter
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add PostgreSQL database for cache metadata storage with schema for
cache_items, item_types, pin_reasons, and l2_shares tables
- Add IPFS integration as durable backing store (local cache as hot storage)
- Add postgres and ipfs services to docker-compose.yml
- Update cache_manager to upload to IPFS and track CIDs
- Rename all config references to recipe throughout server.py
- Update API endpoints: /configs/* -> /recipes/*
- Update models: ConfigStatus -> RecipeStatus, ConfigRunRequest -> RecipeRunRequest
- Update UI tabs and pages to show Recipes instead of Configs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
