- Update save_run_cache to also update actor_id, recipe, inputs on conflict
- Add logging for actor_id when saving runs to run_cache
- Add admin endpoint DELETE /runs/admin/purge-failed to delete all failed runs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add /runs/stream POST endpoint for streaming recipes
- Accepts recipe_sexp, sources_sexp, audio_sexp
- Submits to run_stream Celery task
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove legacy_tasks.py, hybrid_state.py, render.py
- Remove old task modules (analyze, execute, execute_sexp, orchestrate)
- Add streaming interpreter from test repo
- Add sexp_effects with primitives and video effects
- Add streaming Celery task with CID-based asset resolution
- Support both CID and friendly name references for assets
- Add .dockerignore to prevent local clones from conflicting
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Celery task "succeeds" (no exception) but may return {"success": False}.
Now we check the task result's success field AND output_cid before
marking run as completed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Failed runs were not showing in UI/CLI because list_runs only
included runs with status "pending" or "running", excluding "failed".
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- SOURCE nodes with :input true now resolve CID from input_hashes
- Tries multiple name formats: exact, lowercase-dashes, lowercase-underscores
- Only return "completed" status for runs with actual output
- Add integration tests for SOURCE CID resolution
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major changes:
- Add execute_recipe task that uses S-expression planner
- Recipe S-expression unfolds into plan S-expression with code-addressed cache IDs
- Cache IDs computed from Merkle tree of plan structure (before execution)
- Add ipfs_client.add_string() for storing S-expression plans
- Update run_service.create_run() to use execute_recipe when recipe_sexp available
- Add _sexp_to_steps() to parse S-expression plans for UI visualization
- Plan endpoint now returns both sexp content and parsed steps
The code-addressed hashing means each plan step's cache_id is:
sha3_256({node_type, config, sorted(input_cache_ids)})
This creates deterministic "buckets" for computation results computed
entirely from the plan structure, enabling automatic cache reuse.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The CLI expects {"steps": [...]} but DAG format stores {"nodes": {...}}.
Added _dag_to_steps() to convert between formats, including topological
sorting so sources appear first.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Compiler now generates SHA3-256 hashes for node IDs
- Each hash includes type, config, and input hashes (Merkle tree)
- Same plan = same hashes = automatic cache reuse
Cache changes:
- Remove index.json - filesystem IS the index
- Files at {cache_dir}/{hash}/output.* are source of truth
- Per-node metadata.json for optional stats (not an index)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Media: Only count video/image/audio/unknown types, not effects/recipes
- Effects: Use database count_user_items instead of filesystem scan
- Recipes: Use database count_user_items instead of loading all recipes
This ensures stats reflect user ownership via item_types table,
and prevents effects from being double-counted as media.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Call get_cache_manager() to get the cache manager instance
before using it in effects and media deletion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- cache_service.delete_content: Remove user's ownership link first,
only delete actual file if no other owners remain
- cache_manager.discard_activity_outputs_only: Check if outputs and
intermediates are used by other activities before deleting
- run_service.discard_run: Now cleans up run outputs/intermediates
(only if not shared by other runs)
- home.py clear_user_data: Use ownership model for effects and media
deletion instead of directly deleting files
The ownership model ensures:
1. Multiple users can "own" the same cached content
2. Deleting removes the user's ownership link (item_types entry)
3. Actual files only deleted when no owners remain (garbage collection)
4. Shared intermediates between runs are preserved
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload: Create item_types entry to track user-effect relationship
- List: Query item_types for user's effects instead of scanning filesystem
- Delete: Remove ownership link, only delete files if orphaned (garbage collect)
This matches the ownership model used by recipes and media, where multiple
users can "own" the same cached content through item_types entries.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload now creates item_types entry linking user to recipe
- List queries item_types for user's recipes (not all cached)
- Delete removes item_types entry (not the file)
- File only deleted when no users own it (garbage collection)
This allows multiple users to "own" the same recipe CID.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Same reasoning as the list fix: the owner field from recipe content
could be spoofed. For L1, any authenticated user can delete recipes.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The delete_recipe() returns (success, error) tuple but
clear-data wasn't checking the result, so failed deletes
weren't being reported.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add plan_cid column to run_cache schema
- Store DAG JSON to IPFS during execute_dag task
- Return plan_cid in run status and list APIs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- discard_run now deletes from run_cache and pending_runs tables
- Add delete_run_cache() and delete_pending_run() database functions
- Previously clear-data only cleared Redis, leaving DB cache intact
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The owner field from recipe content could be spoofed to hide recipes
from users or make recipes appear to belong to someone else.
For L1, all recipes in cache are now visible to authenticated users.
Ownership tracking should use the naming service or cache metadata,
not untrusted data from recipe content.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The upload endpoint accepts both YAML and S-expression recipes, but
get_recipe only tried to parse S-expression. Now it detects the format
and parses accordingly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All API endpoints now use require_auth or get_current_user which handle
both Authorization header (for CLI) and cookies (for browser). Previously
many endpoints only checked cookies via get_user_from_cookie.
Changed files:
- runs.py: list_runs, run_detail, run_plan, run_artifacts, plan_node_detail, ui_discard_run
- recipes.py: list_recipes, get_recipe, ui_discard_recipe
- storage.py: list_storage, add_storage_form, delete_storage, test_storage, storage_type_page
- cache.py: get_cached, list_media, get_metadata_form, update_metadata_htmx
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- DELETE /api/clear-data clears all user L1 data
- Deletes runs, recipes, effects, and media/cache items
- Preserves storage provider configurations
- Returns counts of deleted items and any errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add pagination to effects list with infinite scroll
- Refactor home stats into reusable get_user_stats function
- Add /api/stats endpoint for CLI/API clients
- Add has_more flag to recipes listing
- Add JSON API support to storage type page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add friendly name display to media detail and list pages
- Unpack nested meta fields to top level for template access
- Fix output_cid mismatch: use IPFS CID consistently between cache and database
- Add dual-indexing in cache_manager to map both IPFS CID and local hash
- Fix plan display: accept IPFS CIDs (Qm..., bafy...) not just 64-char hashes
- Add friendly names to recipe listing
- Add recipe upload button and handler to recipes list
- Add debug logging to recipe listing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add friendly_names table with unique constraints per actor
- Create NamingService with HMAC-signed timestamp version IDs
- Version IDs use base32-crockford encoding, always increase alphabetically
- Name normalization: spaces/underscores to dashes, lowercase, strip special chars
- Format: "my-effect 01hw3x9k" (space separator ensures uniqueness)
- Integrate naming into recipe, effect, and media uploads
- Resolve friendly names to CIDs during DAG execution
- Update effects UI to display friendly names
- Add 30 tests covering normalization, parsing, and service structure
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix list_by_type to return node_id (IPFS CID) instead of local hash
- Fix effects count on home page (count from _effects/ directory)
- Add nav_counts to all page templates (recipes, effects, runs, media, storage)
- Add editable metadata section to cache/media detail page
- Show more metadata on recipe detail page (ID, IPFS CID, step count)
- Update tests for new list_by_type behavior
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Make recipe name clickable link to recipe page on run detail
- Fix step count to use plan.steps length as fallback
- Add nav_counts support to base template for showing counts in brackets
- Add get_nav_counts helper in dependencies
- Pass nav_counts on home page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add /effects/{cid}/publish endpoint to publish effects to L2
- Add Share to L2 button to effects detail page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Template uses inp.cid and input.cid but router created previews with
'hash' key. Fixed both input_previews and run_inputs to use 'cid'.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Effects card to home page grid (now 3 columns)
- Add stats["effects"] count from cache.list_by_type('effect')
- Add tests for home page effects display
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Template runs/detail.html expects artifact.cid but code provided
artifact.hash, causing UndefinedError when viewing run details.
- Change run_service.get_run_artifacts to return 'cid' key
- Change runs.py router inline artifact creation to use 'cid' key
- Add regression tests for artifact data structure
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix recipe filter to allow owner=None (S-expression compiled recipes)
- Fix media uploads to use category (video/image/audio) not MIME type
- Fix IPFS imports to detect and store correct media type
- Add Effects navigation link between Recipes and Media
- Create effects list and detail templates with upload functionality
- Add cache/not_found.html template (was missing)
- Add type annotations to service classes
- Add tests for item visibility and effects web UI (30 tests)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Testing setup:
- Add pyproject.toml with mypy and pytest configuration
- Add requirements-dev.txt for development dependencies
- Create tests/ directory with test fixtures
- Add 17 unit tests for DAG transformation pipeline
Type annotations:
- Add app/types.py with TypedDict definitions for node configs
- Add typed helper functions: transform_node, build_input_name_mapping,
bind_inputs, prepare_dag_for_execution
- Refactor run_recipe to use the new typed helpers
Regression tests for today's bugs:
- test_effect_cid_key_not_effect_hash: Verifies CID uses 'cid' key
- test_source_cid_binding_persists: Verifies bound CIDs in final DAG
Run tests with: pytest tests/ -v
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The EFFECT executor looks for config["cid"] or config["hash"],
but the transformation was setting config["effect_hash"].
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix all registry lookups to use "cid" instead of "hash" key
- app/routers/recipes.py: asset and effect resolution
- tasks/execute_sexp.py: effect config lookups
- server_legacy.py references (now deleted)
- Prefer IPFS CID over local hash in cache operations
- cache_service.py: import_from_ipfs, upload_content
- orchestrate.py: plan caching
- legacy_tasks.py: node hash tracking
Remove ~7800 lines of dead code:
- server_legacy.py: replaced by modular app/ structure
- tasks/*_cid.py: unused refactoring only imported by server_legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The recipe upload was returning the SHA3-256 hash from artdag's cache
instead of the IPFS CID, causing recipe lookups to fail.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation
This completes the transition to CID-only content addressing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload endpoint returns both CID and content_hash
- Cache manager handles both SHA3-256 hashes and IPFS CIDs
- get_by_cid() fetches from IPFS if not cached locally
- Execute tasks support :cid in addition to :hash
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload effects to IPFS, return CID instead of content hash
- Fetch effects from IPFS if not in local cache
- Keep local cache for fast worker access
- Support both :cid (new) and :hash (legacy) in recipes
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- New /effects/upload endpoint for uploading effect files
- Parses PEP 723 dependencies and @-tag metadata
- Lists, gets, and deletes effects by content hash
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Execute COMPOUND nodes with combined FFmpeg filter chain
- Handle TRANSFORM, RESIZE, SEGMENT filters in chain
- Migrate orchestrator to S-expression recipes (remove YAML)
- Update API endpoints to use recipe_sexp parameter
- Extract analysis nodes from recipe for dynamic analysis
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Show ipfs://... link next to recipe S-expression header
- Links to ipfs.io gateway for viewing the raw S-expression
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
