The artdag Cache object doesn't persist state across process restarts,
so cache.get(node_id) returns None even when files exist on disk.
Now we check the filesystem directly at {cache_dir}/nodes/{node_id}/output.*
when the in-memory cache lookup fails but we have a valid node_id from
the Redis index.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When the Celery worker can't find source content in the local cache,
fetch it from IPFS using the CID. This ensures workers can execute
DAGs even when they don't share the same filesystem as the web server.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Debug why recipes are not found in cache after upload.
Logs now show each step of put() and get_by_cid().
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Media: Only count video/image/audio/unknown types, not effects/recipes
- Effects: Use database count_user_items instead of filesystem scan
- Recipes: Use database count_user_items instead of loading all recipes
This ensures stats reflect user ownership via item_types table,
and prevents effects from being double-counted as media.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Track when items are being saved to database to diagnose
why recipes show 0 in stats but effects show correctly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Call get_cache_manager() to get the cache manager instance
before using it in effects and media deletion.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- cache_service.delete_content: Remove user's ownership link first,
only delete actual file if no other owners remain
- cache_manager.discard_activity_outputs_only: Check if outputs and
intermediates are used by other activities before deleting
- run_service.discard_run: Now cleans up run outputs/intermediates
(only if not shared by other runs)
- home.py clear_user_data: Use ownership model for effects and media
deletion instead of directly deleting files
The ownership model ensures:
1. Multiple users can "own" the same cached content
2. Deleting removes the user's ownership link (item_types entry)
3. Actual files only deleted when no owners remain (garbage collection)
4. Shared intermediates between runs are preserved
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload: Create item_types entry to track user-effect relationship
- List: Query item_types for user's effects instead of scanning filesystem
- Delete: Remove ownership link, only delete files if orphaned (garbage collect)
This matches the ownership model used by recipes and media, where multiple
users can "own" the same cached content through item_types entries.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Upload now creates item_types entry linking user to recipe
- List queries item_types for user's recipes (not all cached)
- Delete removes item_types entry (not the file)
- File only deleted when no users own it (garbage collection)
This allows multiple users to "own" the same recipe CID.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Same reasoning as the list fix: the owner field from recipe content
could be spoofed. For L1, any authenticated user can delete recipes.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The dual-indexing code was calling file_hash(source_path) after
cache.put(move=True) had already moved the file, causing
"No such file or directory" errors on upload.
Now computes local_hash before the move operation.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The delete_recipe() returns (success, error) tuple but
clear-data wasn't checking the result, so failed deletes
weren't being reported.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add plan_cid column to run_cache schema
- Store DAG JSON to IPFS during execute_dag task
- Return plan_cid in run status and list APIs
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- discard_run now deletes from run_cache and pending_runs tables
- Add delete_run_cache() and delete_pending_run() database functions
- Previously clear-data only cleared Redis, leaving DB cache intact
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Rename misleading ipfs_cid variable to content_cid
- Detect IPFS CIDs by prefix (Qm or bafy) instead of truthy check
- Add clearer logging to show whether IPFS or local hash is used
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The owner field from recipe content could be spoofed to hide recipes
from users or make recipes appear to belong to someone else.
For L1, all recipes in cache are now visible to authenticated users.
Ownership tracking should use the naming service or cache metadata,
not untrusted data from recipe content.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The upload endpoint accepts both YAML and S-expression recipes, but
get_recipe only tried to parse S-expression. Now it detects the format
and parses accordingly.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All API endpoints now use require_auth or get_current_user which handle
both Authorization header (for CLI) and cookies (for browser). Previously
many endpoints only checked cookies via get_user_from_cookie.
Changed files:
- runs.py: list_runs, run_detail, run_plan, run_artifacts, plan_node_detail, ui_discard_run
- recipes.py: list_recipes, get_recipe, ui_discard_recipe
- storage.py: list_storage, add_storage_form, delete_storage, test_storage, storage_type_page
- cache.py: get_cached, list_media, get_metadata_form, update_metadata_htmx
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- DELETE /api/clear-data clears all user L1 data
- Deletes runs, recipes, effects, and media/cache items
- Preserves storage provider configurations
- Returns counts of deleted items and any errors
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add pagination to effects list with infinite scroll
- Refactor home stats into reusable get_user_stats function
- Add /api/stats endpoint for CLI/API clients
- Add has_more flag to recipes listing
- Add JSON API support to storage type page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add friendly name display to media detail and list pages
- Unpack nested meta fields to top level for template access
- Fix output_cid mismatch: use IPFS CID consistently between cache and database
- Add dual-indexing in cache_manager to map both IPFS CID and local hash
- Fix plan display: accept IPFS CIDs (Qm..., bafy...) not just 64-char hashes
- Add friendly names to recipe listing
- Add recipe upload button and handler to recipes list
- Add debug logging to recipe listing
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add friendly_names table with unique constraints per actor
- Create NamingService with HMAC-signed timestamp version IDs
- Version IDs use base32-crockford encoding, always increase alphabetically
- Name normalization: spaces/underscores to dashes, lowercase, strip special chars
- Format: "my-effect 01hw3x9k" (space separator ensures uniqueness)
- Integrate naming into recipe, effect, and media uploads
- Resolve friendly names to CIDs during DAG execution
- Update effects UI to display friendly names
- Add 30 tests covering normalization, parsing, and service structure
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix list_by_type to return node_id (IPFS CID) instead of local hash
- Fix effects count on home page (count from _effects/ directory)
- Add nav_counts to all page templates (recipes, effects, runs, media, storage)
- Add editable metadata section to cache/media detail page
- Show more metadata on recipe detail page (ID, IPFS CID, step count)
- Update tests for new list_by_type behavior
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Make recipe name clickable link to recipe page on run detail
- Fix step count to use plan.steps length as fallback
- Add nav_counts support to base template for showing counts in brackets
- Add get_nav_counts helper in dependencies
- Pass nav_counts on home page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add /effects/{cid}/publish endpoint to publish effects to L2
- Add Share to L2 button to effects detail page
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Template uses inp.cid and input.cid but router created previews with
'hash' key. Fixed both input_previews and run_inputs to use 'cid'.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Effects card to home page grid (now 3 columns)
- Add stats["effects"] count from cache.list_by_type('effect')
- Add tests for home page effects display
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Bug: Two get_by_cid methods existed in L1CacheManager. The second
definition shadowed the first, breaking recipe lookup because the
comprehensive method (using find_by_cid) was hidden.
- Remove duplicate get_by_cid method (lines 470-494)
- Add regression test to ensure only one get_by_cid exists
- Add tests for template variables and recipe visibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Template runs/detail.html expects artifact.cid but code provided
artifact.hash, causing UndefinedError when viewing run details.
- Change run_service.get_run_artifacts to return 'cid' key
- Change runs.py router inline artifact creation to use 'cid' key
- Add regression tests for artifact data structure
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix recipe filter to allow owner=None (S-expression compiled recipes)
- Fix media uploads to use category (video/image/audio) not MIME type
- Fix IPFS imports to detect and store correct media type
- Add Effects navigation link between Recipes and Media
- Create effects list and detail templates with upload functionality
- Add cache/not_found.html template (was missing)
- Add type annotations to service classes
- Add tests for item visibility and effects web UI (30 tests)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix UserContext l2_server field not found error by tracking
master branch instead of pinned commit.
- Update art-common to track master branch
- Add tests for UserContext l2_server field
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Effects commonly use numpy and opencv for video processing.
Without these, effects fail with confusing "Unknown effect" error.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Pin artdag to commit that uses IPFS_API instead of IPFS_GATEWAY
for consistent IPFS access in Docker.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
All IPFS operations now use IPFS_API (multiaddr format) instead of
separate IPFS_GATEWAY config. This fixes effect loading in Docker where
the gateway URL defaulted to localhost.
- Add tests for effect loading and IPFS configuration
- Simplify docker-compose.yml (remove IPFS_GATEWAY)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Testing setup:
- Add pyproject.toml with mypy and pytest configuration
- Add requirements-dev.txt for development dependencies
- Create tests/ directory with test fixtures
- Add 17 unit tests for DAG transformation pipeline
Type annotations:
- Add app/types.py with TypedDict definitions for node configs
- Add typed helper functions: transform_node, build_input_name_mapping,
bind_inputs, prepare_dag_for_execution
- Refactor run_recipe to use the new typed helpers
Regression tests for today's bugs:
- test_effect_cid_key_not_effect_hash: Verifies CID uses 'cid' key
- test_source_cid_binding_persists: Verifies bound CIDs in final DAG
Run tests with: pytest tests/ -v
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The EFFECT executor looks for config["cid"] or config["hash"],
but the transformation was setting config["effect_hash"].
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix all registry lookups to use "cid" instead of "hash" key
- app/routers/recipes.py: asset and effect resolution
- tasks/execute_sexp.py: effect config lookups
- server_legacy.py references (now deleted)
- Prefer IPFS CID over local hash in cache operations
- cache_service.py: import_from_ipfs, upload_content
- orchestrate.py: plan caching
- legacy_tasks.py: node hash tracking
Remove ~7800 lines of dead code:
- server_legacy.py: replaced by modular app/ structure
- tasks/*_cid.py: unused refactoring only imported by server_legacy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The recipe upload was returning the SHA3-256 hash from artdag's cache
instead of the IPFS CID, causing recipe lookups to fail.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation
This completes the transition to CID-only content addressing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
