Commit Graph

48 Commits

Author SHA1 Message Date
gilesb
faa54b2e85 Add database migration for content_hash -> cid rename
- Add DO block to migrate existing columns:
  - cache_items.content_hash -> cid
  - item_types.content_hash -> cid
  - l2_shares.content_hash -> cid
  - storage_pins.content_hash -> cid
  - run_cache.output_hash -> output_cid
- Fix duplicate key bug in upload response

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:18:23 +00:00
gilesb
92d26b2b72 Rename content_hash/output_hash to cid throughout
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation

This completes the transition to CID-only content addressing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:02:44 +00:00
gilesb
494a2a8650 Add IPFS CID support for asset lookup
- Upload endpoint returns both CID and content_hash
- Cache manager handles both SHA3-256 hashes and IPFS CIDs
- get_by_cid() fetches from IPFS if not cached locally
- Execute tasks support :cid in addition to :hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:36:18 +00:00
gilesb
c7c7c90909 Store effects in IPFS instead of local cache
- Upload effects to IPFS, return CID instead of content hash
- Fetch effects from IPFS if not in local cache
- Keep local cache for fast worker access
- Support both :cid (new) and :hash (legacy) in recipes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 07:28:05 +00:00
gilesb
8e1c08abdc Add effects upload endpoint
- New /effects/upload endpoint for uploading effect files
- Parses PEP 723 dependencies and @-tag metadata
- Lists, gets, and deletes effects by content hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 06:37:24 +00:00
gilesb
8e0b473925 Add COMPOUND node execution and S-expression API
- Execute COMPOUND nodes with combined FFmpeg filter chain
- Handle TRANSFORM, RESIZE, SEGMENT filters in chain
- Migrate orchestrator to S-expression recipes (remove YAML)
- Update API endpoints to use recipe_sexp parameter
- Extract analysis nodes from recipe for dynamic analysis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 01:26:26 +00:00
gilesb
3599f3779b Add IPFS link to recipe S-expression display
- Show ipfs://... link next to recipe S-expression header
- Links to ipfs.io gateway for viewing the raw S-expression

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:36:14 +00:00
gilesb
65a8170192 Use native S-expression for recipe/plan display
- Display recipe's original S-expression when available (code is data)
- Fall back to generating S-expression from plan for legacy JSON
- Run service now prefers .sexp plan files over .json
- Add get_run_plan_sexp() for direct S-expression access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:26:05 +00:00
gilesb
f554122b07 Replace plan JSON with colored S-expression display
- Add plan_to_sexp() to convert plan to S-expression format
- Syntax highlighting for S-expressions:
  - Pink: special forms (plan, recipe, def, ->)
  - Blue: primitives (source, effect, sequence, etc.)
  - Purple: keywords (:input, :name, etc.)
  - Green: strings
  - Yellow: parentheses
  - Gray: comments

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:23:12 +00:00
gilesb
82d94f6e0e Add inline media previews for runs list and detail page
- Run card shows thumbnail previews for inputs and output
- Run detail shows output media inline (image/video/audio)
- Add audio detection (MP3, FLAC, OGG, WAV) to detect_media_type
- Add debug logging for recipe count on home page
- Add console.log debugging for DAG elements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:20:26 +00:00
gilesb
5c3558e1ba Fix DAG visualization and step link handling
- Handle dict inputs ({"node": "id"}) when building DAG edges
- Add normalize_inputs() to convert dict inputs to node IDs for steps
- Fix _parse_inputs to use _json.loads (correct import alias)
- Add SOURCE/EFFECT/SEQUENCE colors to node color maps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:05:50 +00:00
gilesb
e91803f71d Add runs and storage counts to home page stats
Home page now shows:
- Execution runs count
- Recipes count
- Media files count
- Storage providers count

All stats require authentication.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:56:11 +00:00
gilesb
48faf7ee43 Fix media type detection using magic bytes
Use detect_media_type() with magic bytes instead of mimetypes.guess_type()
which requires file extensions. Cache files are stored by content hash
without extensions, so magic byte detection is needed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:53:40 +00:00
gilesb
0a82158622 Support multiple input name formats for recipe run
Variable inputs can now be referenced by:
- Node ID (e.g., source_4)
- Config name (e.g., "Second Video")
- Snake case (e.g., second_video, second-video)
- Node name from def binding

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:42:52 +00:00
gilesb
8e70a9b9f2 Add interactive plan node selection with media previews
- Click plan nodes (in DAG or list) to see details in side panel
- URL updates to #node-{id} for direct linking
- Node detail panel shows: type, status, inputs, output, config
- Inputs can be clicked to navigate to that node
- Inputs tab now shows media previews (image/video/audio)
- Steps include config data for display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:30:43 +00:00
gilesb
d73592fbe2 Fix DAG visualization node IDs
Add "id" field to plan steps so edges connect correctly.
Previously steps only had "name" but dag_elements looked for "id",
causing edges to reference non-existent nodes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:21:33 +00:00
gilesb
24b6b4af28 Give clear error when sexp module not available
Instead of falling through to YAML parsing (which gives confusing errors),
return a clear message that artdag.sexp is required but not installed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:17:55 +00:00
gilesb
0ead728fde Fix dag_elements undefined in run detail template
Add dag_elements to the get_run endpoint render call to match
what the detail.html template expects.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:15:52 +00:00
gilesb
f7dbb952ab Preserve recipe name in run service and templates
- Pass recipe_name through create_run to display friendly names
- Update templates to show name instead of hash
- Fall back to truncated hash if no name available

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:09:33 +00:00
gilesb
e59a50c000 Add S-expression recipe support
- Add format detection that correctly handles ; comments
- Import artdag.sexp parser/compiler with YAML fallback
- Add execute_step_sexp and run_plan_sexp Celery tasks
- Update recipe upload to handle both S-expr and YAML formats

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:08:53 +00:00
gilesb
9df78f771d Fix run detail: add username, total_steps, recipe_name
- Extract username from actor_id format (@user@server)
- Set total_steps and executed from recipe nodes
- Use recipe name for display instead of hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:23:08 +00:00
gilesb
145c69f21b Add artifacts to run detail view
Build artifacts list from run output_hash and detect media type
for display in the artifacts tab.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:08:06 +00:00
gilesb
ccf28bd351 Load recipe to show plan in run detail view
When viewing a run, try to load the recipe by hash to display
the plan/steps in the UI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:05:41 +00:00
gilesb
50ea0f1491 Fix run detail: default to JSON, only HTML if browser requests it
API clients like Python requests send Accept: */* which wasn't
matching wants_json(). Switch to checking wants_html() instead
so API clients get JSON by default.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:03:53 +00:00
gilesb
95ffe9fa69 Fix run detail page and recipe name
- Render HTML template for run detail (not just JSON)
- Get recipe name from pending_runs instead of hardcoding "dag"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 21:56:08 +00:00
gilesb
cca8f74d3c Add debug logging for recipe node structure 2026-01-11 21:24:17 +00:00
gilesb
a6dd470623 Fix recipe steps display and DAG visualization
- Check multiple locations for nodes: dag.nodes, recipe.nodes, pipeline, steps
- Add dagre layout libraries for cytoscape DAG visualization
- Fix inputs parsing when stored as JSON string in Redis

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 20:21:24 +00:00
gilesb
ab2f65e14d Add client download link and build tarball during deployment
- Added /download/client endpoint to serve the CLI client tarball
- Added "Client" link to navigation in base template
- Created build-client.sh script to clone and package the client
- Updated Dockerfile to run build-client.sh during container build

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:08:55 +00:00
gilesb
0ddeb5ba94 Resolve registry asset/effect references when running recipes
SOURCE nodes with config.asset now get content_hash from registry.
EFFECT nodes with config.effect now get effect_hash from registry.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 18:22:36 +00:00
gilesb
156014a1f3 Add Share to L2 buttons for media, recipes, and runs
- Updated cache detail button from "Publish to IPFS" to "Share to L2"
- Added Share to L2 button on recipe detail page
- Added Share to L2 button on run detail page
- Created /recipes/{id}/publish endpoint
- Created /runs/{id}/publish endpoint (publishes run output)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 18:21:14 +00:00
gilesb
088af1611b Transform recipe DAG format to artdag library format
The recipe YAML uses:
- type, id, output

But artdag library expects:
- node_type, node_id, output_id, metadata

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 18:14:44 +00:00
gilesb
3acaacce1f Fix recipe detail to display DAG nodes as steps
The template expects recipe.steps and recipe.yaml but the recipe
data has dag.nodes. Convert nodes (list or dict) to steps format
and add YAML dump for source display.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 18:12:08 +00:00
gilesb
970faa3fa0 Fix run recipe: optional fields in RunStatus, list->dict nodes
- Made recipe and inputs optional in RunStatus model
- Convert DAG nodes from list format to dict format when running recipes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:50:36 +00:00
gilesb
b57745098e Fix run_recipe to handle dict return from create_run
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:43:31 +00:00
gilesb
2e9ba46f19 Use UserContext from artdag_common, remove duplicate
- Updated requirements.txt to use art-common@11aa056 with l2_server field
- All routers now import UserContext from artdag_common
- Removed duplicate UserContext from auth_service.py
- dependencies.py sets l2_server from settings on user context

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:38:02 +00:00
gilesb
3373be285e Fix recipe list links and home page recipe count
- Template used recipe.id but service returns recipe.recipe_id
- Add recipe count to home page stats

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:26:00 +00:00
gilesb
a6fe88c277 Fix recipe upload to accept file uploads instead of JSON
The CLI client sends multipart file uploads but the server expected JSON.
Changed the /recipes/upload endpoint to accept UploadFile and return
the additional fields (name, version, variable_inputs, fixed_inputs)
that the client expects.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:13:06 +00:00
gilesb
697fa7c64d Show actual media count on home page 2026-01-11 16:55:14 +00:00
giles
854396680f Refactor storage: remove Redis duplication, use proper data tiers
- Recipes: Now content-addressed only (cache + IPFS), removed Redis storage
- Runs: Completed runs stored in PostgreSQL, Redis only for task_id mapping
- Add list_runs_by_actor() to database.py for paginated run queries
- Add list_by_type() to cache_manager for filtering by node_type
- Fix upload endpoint to return size and filename fields
- Fix recipe run endpoint with proper DAG input binding
- Fix get_run_service() dependency to pass database module

Storage architecture:
- Redis: Ephemeral only (sessions, task mappings with TTL)
- PostgreSQL: Permanent records (completed runs, metadata)
- Cache: Content-addressed files (recipes, media, outputs)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:05:31 +00:00
giles
8591faf0fc Fix CacheService list_media and recipe inputs type
- Add list_media method to CacheService
- Change recipe run inputs from List to dict

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:28:27 +00:00
giles
4f011a66ff Fix recipe run endpoint to accept JSON body
Use Pydantic model for inputs parameter.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:22:13 +00:00
giles
3f77c24699 Enable markdown tables and fenced_code extensions
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 13:20:48 +00:00
giles
f0db4f4ea6 Restore home page with README display
Instead of redirecting to /runs, show the home page with stats and README.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 12:54:50 +00:00
giles
d19d6d6e42 Fix RecipeService initialization with cache manager
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 12:34:39 +00:00
giles
29da91e01a Refactor to modular app factory architecture
- Replace 6847-line monolithic server.py with 26-line entry point
- All routes now in app/routers/ using Jinja2 templates
- Add plan_node.html template for step details
- Add plan node route to runs router with cache_id lookup
- Backup old server as server_legacy.py

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 11:48:24 +00:00
giles
945fb3b413 Fix analysis display for recipe-based runs
Add get_run_analysis() to RunService to load per-input analysis from
CACHE_DIR/analysis/{hash}.json files. Update runs router and template
to display tempo, beats, energy, and beat timeline visualization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:05:40 +00:00
giles
022f88bf0c Complete L1 router and template migration
- Full implementation of runs, recipes, cache routers with templates
- Auth and storage routers fully migrated
- Jinja2 templates for all L1 pages
- Service layer for auth and storage

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:49:37 +00:00
giles
adc876dbd6 Add modular app structure for L1 server refactoring
Phase 2 of the full modernization:
- App factory pattern with create_app()
- Settings via dataclass with env vars
- Dependency injection container
- Router stubs for auth, storage, api, recipes, cache, runs
- Service layer stubs for run, recipe, cache
- Repository layer placeholder

Routes are stubs that import from legacy server.py during migration.
Next: Migrate each router fully with templates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:08:08 +00:00