Commit Graph

93 Commits

Author SHA1 Message Date
gilesb
f1b90fe65d Add debug logging for recipe filtering
Shows owner and actor_id values when filtering recipes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 15:30:24 +00:00
gilesb
4b5066c525 Fix clear-data missing username arg for discard_run
The discard_run service method requires username as the third argument.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 15:02:09 +00:00
gilesb
280dddebd0 Fix authentication to support both header and cookie auth
All API endpoints now use require_auth or get_current_user which handle
both Authorization header (for CLI) and cookies (for browser). Previously
many endpoints only checked cookies via get_user_from_cookie.

Changed files:
- runs.py: list_runs, run_detail, run_plan, run_artifacts, plan_node_detail, ui_discard_run
- recipes.py: list_recipes, get_recipe, ui_discard_recipe
- storage.py: list_storage, add_storage_form, delete_storage, test_storage, storage_type_page
- cache.py: get_cached, list_media, get_metadata_form, update_metadata_htmx

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:56:52 +00:00
gilesb
79a74df2bb Add clear-data API endpoint
- DELETE /api/clear-data clears all user L1 data
- Deletes runs, recipes, effects, and media/cache items
- Preserves storage provider configurations
- Returns counts of deleted items and any errors

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:34:56 +00:00
gilesb
7d24ba4dd7 Add pagination and API improvements
- Add pagination to effects list with infinite scroll
- Refactor home stats into reusable get_user_stats function
- Add /api/stats endpoint for CLI/API clients
- Add has_more flag to recipes listing
- Add JSON API support to storage type page

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:22:09 +00:00
gilesb
ee8719ac0b Fix media friendly names, metadata display, output recording, and plan display
- Add friendly name display to media detail and list pages
- Unpack nested meta fields to top level for template access
- Fix output_cid mismatch: use IPFS CID consistently between cache and database
- Add dual-indexing in cache_manager to map both IPFS CID and local hash
- Fix plan display: accept IPFS CIDs (Qm..., bafy...) not just 64-char hashes
- Add friendly names to recipe listing
- Add recipe upload button and handler to recipes list
- Add debug logging to recipe listing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:21:39 +00:00
gilesb
19634a4ac5 Add friendly names system for recipes, effects, and media
- Add friendly_names table with unique constraints per actor
- Create NamingService with HMAC-signed timestamp version IDs
- Version IDs use base32-crockford encoding, always increase alphabetically
- Name normalization: spaces/underscores to dashes, lowercase, strip special chars
- Format: "my-effect 01hw3x9k" (space separator ensures uniqueness)
- Integrate naming into recipe, effect, and media uploads
- Resolve friendly names to CIDs during DAG execution
- Update effects UI to display friendly names
- Add 30 tests covering normalization, parsing, and service structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:02:17 +00:00
gilesb
98ca2a6c81 Fix recipe listing, effects count, and add nav counts to all pages
- Fix list_by_type to return node_id (IPFS CID) instead of local hash
- Fix effects count on home page (count from _effects/ directory)
- Add nav_counts to all page templates (recipes, effects, runs, media, storage)
- Add editable metadata section to cache/media detail page
- Show more metadata on recipe detail page (ID, IPFS CID, step count)
- Update tests for new list_by_type behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 13:30:11 +00:00
gilesb
9bb1c4278e Add recipe link, fix step count, add nav counts
- Make recipe name clickable link to recipe page on run detail
- Fix step count to use plan.steps length as fallback
- Add nav_counts support to base template for showing counts in brackets
- Add get_nav_counts helper in dependencies
- Pass nav_counts on home page

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:53:00 +00:00
gilesb
73cc3e7c2f Add Share to L2 button for effects
- Add /effects/{cid}/publish endpoint to publish effects to L2
- Add Share to L2 button to effects detail page

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:30:38 +00:00
gilesb
991db29e27 Fix input preview hash -> cid key mismatch
Template uses inp.cid and input.cid but router created previews with
'hash' key. Fixed both input_previews and run_inputs to use 'cid'.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:27:05 +00:00
gilesb
2cc7d88d2e Add effects count to home page dashboard
- Add Effects card to home page grid (now 3 columns)
- Add stats["effects"] count from cache.list_by_type('effect')
- Add tests for home page effects display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:19:06 +00:00
gilesb
977d9a9258 Fix artifact dict key mismatch (hash -> cid)
Template runs/detail.html expects artifact.cid but code provided
artifact.hash, causing UndefinedError when viewing run details.

- Change run_service.get_run_artifacts to return 'cid' key
- Change runs.py router inline artifact creation to use 'cid' key
- Add regression tests for artifact data structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:05:13 +00:00
gilesb
585c75e846 Fix item visibility bugs and add effects web UI
- Fix recipe filter to allow owner=None (S-expression compiled recipes)
- Fix media uploads to use category (video/image/audio) not MIME type
- Fix IPFS imports to detect and store correct media type
- Add Effects navigation link between Recipes and Media
- Create effects list and detail templates with upload functionality
- Add cache/not_found.html template (was missing)
- Add type annotations to service classes
- Add tests for item visibility and effects web UI (30 tests)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:01:54 +00:00
gilesb
56009c391d Add testing infrastructure and refactor DAG transformation
Testing setup:
- Add pyproject.toml with mypy and pytest configuration
- Add requirements-dev.txt for development dependencies
- Create tests/ directory with test fixtures
- Add 17 unit tests for DAG transformation pipeline

Type annotations:
- Add app/types.py with TypedDict definitions for node configs
- Add typed helper functions: transform_node, build_input_name_mapping,
  bind_inputs, prepare_dag_for_execution
- Refactor run_recipe to use the new typed helpers

Regression tests for today's bugs:
- test_effect_cid_key_not_effect_hash: Verifies CID uses 'cid' key
- test_source_cid_binding_persists: Verifies bound CIDs in final DAG

Run tests with: pytest tests/ -v

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 09:37:06 +00:00
gilesb
0ba1d6e82d Fix effect CID resolution: use 'cid' not 'effect_hash'
The EFFECT executor looks for config["cid"] or config["hash"],
but the transformation was setting config["effect_hash"].

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 09:24:15 +00:00
gilesb
19e74a097d Add debug logging to trace input binding
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 09:15:45 +00:00
gilesb
60344b34f4 Fix registry lookups to use cid, remove dead legacy code
- Fix all registry lookups to use "cid" instead of "hash" key
  - app/routers/recipes.py: asset and effect resolution
  - tasks/execute_sexp.py: effect config lookups
  - server_legacy.py references (now deleted)
- Prefer IPFS CID over local hash in cache operations
  - cache_service.py: import_from_ipfs, upload_content
  - orchestrate.py: plan caching
  - legacy_tasks.py: node hash tracking

Remove ~7800 lines of dead code:
- server_legacy.py: replaced by modular app/ structure
- tasks/*_cid.py: unused refactoring only imported by server_legacy

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 09:09:40 +00:00
gilesb
8e7228fc38 Fix recipe_id to use IPFS CID instead of local hash
The recipe upload was returning the SHA3-256 hash from artdag's cache
instead of the IPFS CID, causing recipe lookups to fail.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:58:51 +00:00
gilesb
faa54b2e85 Add database migration for content_hash -> cid rename
- Add DO block to migrate existing columns:
  - cache_items.content_hash -> cid
  - item_types.content_hash -> cid
  - l2_shares.content_hash -> cid
  - storage_pins.content_hash -> cid
  - run_cache.output_hash -> output_cid
- Fix duplicate key bug in upload response

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:18:23 +00:00
gilesb
92d26b2b72 Rename content_hash/output_hash to cid throughout
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation

This completes the transition to CID-only content addressing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:02:44 +00:00
gilesb
494a2a8650 Add IPFS CID support for asset lookup
- Upload endpoint returns both CID and content_hash
- Cache manager handles both SHA3-256 hashes and IPFS CIDs
- get_by_cid() fetches from IPFS if not cached locally
- Execute tasks support :cid in addition to :hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:36:18 +00:00
gilesb
c7c7c90909 Store effects in IPFS instead of local cache
- Upload effects to IPFS, return CID instead of content hash
- Fetch effects from IPFS if not in local cache
- Keep local cache for fast worker access
- Support both :cid (new) and :hash (legacy) in recipes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:28:05 +00:00
gilesb
8e1c08abdc Add effects upload endpoint
- New /effects/upload endpoint for uploading effect files
- Parses PEP 723 dependencies and @-tag metadata
- Lists, gets, and deletes effects by content hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 06:37:24 +00:00
gilesb
8e0b473925 Add COMPOUND node execution and S-expression API
- Execute COMPOUND nodes with combined FFmpeg filter chain
- Handle TRANSFORM, RESIZE, SEGMENT filters in chain
- Migrate orchestrator to S-expression recipes (remove YAML)
- Update API endpoints to use recipe_sexp parameter
- Extract analysis nodes from recipe for dynamic analysis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 01:26:26 +00:00
gilesb
3599f3779b Add IPFS link to recipe S-expression display
- Show ipfs://... link next to recipe S-expression header
- Links to ipfs.io gateway for viewing the raw S-expression

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:36:14 +00:00
gilesb
b686ce75f8 Remove YAML support - S-expressions only
- Recipe service now only handles S-expressions
- Removed yaml import and all YAML parsing code
- Plans are just node outputs - cached by content hash
- Run service looks up plans from cache, falls back to legacy dir

Code is data. Everything is S-expressions.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:33:54 +00:00
gilesb
65a8170192 Use native S-expression for recipe/plan display
- Display recipe's original S-expression when available (code is data)
- Fall back to generating S-expression from plan for legacy JSON
- Run service now prefers .sexp plan files over .json
- Add get_run_plan_sexp() for direct S-expression access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:26:05 +00:00
gilesb
f554122b07 Replace plan JSON with colored S-expression display
- Add plan_to_sexp() to convert plan to S-expression format
- Syntax highlighting for S-expressions:
  - Pink: special forms (plan, recipe, def, ->)
  - Blue: primitives (source, effect, sequence, etc.)
  - Purple: keywords (:input, :name, etc.)
  - Green: strings
  - Yellow: parentheses
  - Gray: comments

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:23:12 +00:00
gilesb
82d94f6e0e Add inline media previews for runs list and detail page
- Run card shows thumbnail previews for inputs and output
- Run detail shows output media inline (image/video/audio)
- Add audio detection (MP3, FLAC, OGG, WAV) to detect_media_type
- Add debug logging for recipe count on home page
- Add console.log debugging for DAG elements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:20:26 +00:00
gilesb
5c3558e1ba Fix DAG visualization and step link handling
- Handle dict inputs ({"node": "id"}) when building DAG edges
- Add normalize_inputs() to convert dict inputs to node IDs for steps
- Fix _parse_inputs to use _json.loads (correct import alias)
- Add SOURCE/EFFECT/SEQUENCE colors to node color maps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:05:50 +00:00
gilesb
e91803f71d Add runs and storage counts to home page stats
Home page now shows:
- Execution runs count
- Recipes count
- Media files count
- Storage providers count

All stats require authentication.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:56:11 +00:00
gilesb
5e6ed38ca4 Improve recipe listing - check both uploader and owner
- Add better debug logging for recipe filtering
- Check both uploader and owner fields when filtering by actor_id
- This handles S-expression recipes that use 'owner' field

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:54:56 +00:00
gilesb
48faf7ee43 Fix media type detection using magic bytes
Use detect_media_type() with magic bytes instead of mimetypes.guess_type()
which requires file extensions. Cache files are stored by content hash
without extensions, so magic byte detection is needed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:53:40 +00:00
gilesb
0a82158622 Support multiple input name formats for recipe run
Variable inputs can now be referenced by:
- Node ID (e.g., source_4)
- Config name (e.g., "Second Video")
- Snake case (e.g., second_video, second-video)
- Node name from def binding

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:42:52 +00:00
gilesb
fe8e65881d Map owner to uploader for S-expression recipes
S-expression recipes use 'owner' field while YAML uses 'uploader'.
Normalize to 'uploader' so recipe listing filter works for both formats.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:40:50 +00:00
gilesb
8e70a9b9f2 Add interactive plan node selection with media previews
- Click plan nodes (in DAG or list) to see details in side panel
- URL updates to #node-{id} for direct linking
- Node detail panel shows: type, status, inputs, output, config
- Inputs can be clicked to navigate to that node
- Inputs tab now shows media previews (image/video/audio)
- Steps include config data for display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:30:43 +00:00
gilesb
d73592fbe2 Fix DAG visualization node IDs
Add "id" field to plan steps so edges connect correctly.
Previously steps only had "name" but dag_elements looked for "id",
causing edges to reference non-existent nodes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:21:33 +00:00
gilesb
24b6b4af28 Give clear error when sexp module not available
Instead of falling through to YAML parsing (which gives confusing errors),
return a clear message that artdag.sexp is required but not installed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:17:55 +00:00
gilesb
0ead728fde Fix dag_elements undefined in run detail template
Add dag_elements to the get_run endpoint render call to match
what the detail.html template expects.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:15:52 +00:00
gilesb
f7dbb952ab Preserve recipe name in run service and templates
- Pass recipe_name through create_run to display friendly names
- Update templates to show name instead of hash
- Fall back to truncated hash if no name available

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:09:33 +00:00
gilesb
e59a50c000 Add S-expression recipe support
- Add format detection that correctly handles ; comments
- Import artdag.sexp parser/compiler with YAML fallback
- Add execute_step_sexp and run_plan_sexp Celery tasks
- Update recipe upload to handle both S-expr and YAML formats

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:08:53 +00:00
gilesb
9df78f771d Fix run detail: add username, total_steps, recipe_name
- Extract username from actor_id format (@user@server)
- Set total_steps and executed from recipe nodes
- Use recipe name for display instead of hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:23:08 +00:00
gilesb
9a8e26e79c Compute step_count in recipe service
Check multiple locations for nodes (nodes, dag.nodes, pipeline, steps)
and compute step_count for display in recipe list.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:09:34 +00:00
gilesb
145c69f21b Add artifacts to run detail view
Build artifacts list from run output_hash and detect media type
for display in the artifacts tab.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:08:06 +00:00
gilesb
ccf28bd351 Load recipe to show plan in run detail view
When viewing a run, try to load the recipe by hash to display
the plan/steps in the UI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:05:41 +00:00
gilesb
50ea0f1491 Fix run detail: default to JSON, only HTML if browser requests it
API clients like Python requests send Accept: */* which wasn't
matching wants_json(). Switch to checking wants_html() instead
so API clients get JSON by default.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:03:53 +00:00
gilesb
95ffe9fa69 Fix run detail page and recipe name
- Render HTML template for run detail (not just JSON)
- Get recipe name from pending_runs instead of hardcoding "dag"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 21:56:08 +00:00
gilesb
aacf1ceb77 Add error logging for pending run creation 2026-01-11 21:45:29 +00:00
gilesb
59de1cf6b5 Add _ensure_inputs_list to handle legacy Redis data format
Inputs stored in old Redis format are JSON strings - this helper
ensures they're always returned as lists regardless of source.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 21:27:49 +00:00