Commit Graph

46 Commits

Author SHA1 Message Date
giles
48018d09b7 Add CI/CD workflow
Some checks failed
Build and Deploy / build-and-deploy (push) Failing after 1m2s
GPU Worker CI/CD / test (push) Failing after 1m4s
GPU Worker CI/CD / deploy (push) Has been skipped
2026-02-06 10:45:13 +00:00
giles
2f56ffc472 Fix live HLS streaming with dynamic quality playlist URLs
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
The problem: HLS.js caches quality playlist URLs from the master playlist.
Even when we update the master playlist CID, HLS.js keeps polling the same
static quality CID URL, so it never sees new segments.

The fix:
- Store quality-level CIDs in database (quality_playlists JSONB column)
- Generate master playlist with dynamic URLs (/runs/{id}/quality/{name}/playlist.m3u8)
- Add quality endpoint that fetches LATEST CID from database
- HLS.js now polls our dynamic endpoints which return fresh content

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 21:07:29 +00:00
giles
4647dd52c8 Add IPFS proxy for live HLS streaming - rewrites playlist URLs 2026-02-04 21:00:07 +00:00
giles
81dc40534c Add progress metadata to ipfs-stream endpoint
Returns progress, frame, total_frames from Celery task state
so clients can display rendering progress.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 20:16:17 +00:00
giles
d6c575760b Add Delete and Run Again buttons to run detail page
- Add "Run Again" button that reruns the recipe with same parameters
- Add "Delete" button with confirmation to delete run and artifacts
- Consolidate result display into single #action-result span
- Implement POST /runs/rerun/{recipe_id} endpoint

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 17:18:16 +00:00
giles
9a8a701492 Fix GPU encoding black frames and improve debug logging
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- Add CUDA sync before encoding to ensure RGB->NV12 kernel completes
- Add debug logging for frame data validation (sum check)
- Handle GPUFrame objects in GPUHLSOutput.write()
- Fix cv2.resize for CuPy arrays (use cupyx.scipy.ndimage.zoom)
- Fix fused pipeline parameter ordering (geometric first, color second)
- Add raindrop-style ripple with random position/freq/decay/amp
- Generate final VOD playlist with #EXT-X-ENDLIST

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 16:33:12 +00:00
giles
baf79f453f Fix IPFS stream caching - return live playlist CID from database
- Return ipfs_playlist_cid from pending_runs while task is running
- Add Cache-Control: no-cache headers to prevent browser/CDN caching
- Fix streaming clients getting stale playlist CIDs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 12:32:56 +00:00
giles
0bd8ee71c7 Fix MP4 mux for web playback: add faststart and genpts
Some checks are pending
GPU Worker CI/CD / test (push) Waiting to run
GPU Worker CI/CD / deploy (push) Blocked by required conditions
- Add -movflags +faststart to move moov atom to start
- Add -fflags +genpts for proper timestamp generation
- Fixes jerky playback and video/audio desync

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-04 00:32:45 +00:00
giles
9096824444 Redirect /hls/stream.m3u8 to IPFS playlist when available
When the local HLS playlist doesn't exist, check for IPFS playlist
CID in pending/completed run and redirect to the IPFS gateway.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 22:09:19 +00:00
giles
09d5359725 Re-enable GPU queue routing after image rebuild
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 20:38:37 +00:00
giles
4930eb99ad Temp: disable GPU queue for testing IPFS HLS streaming
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 20:25:50 +00:00
giles
86830019ad Add IPFS HLS streaming and GPU optimizations
- Add IPFSHLSOutput class that uploads segments to IPFS as they're created
- Update streaming task to use IPFS HLS output for distributed streaming
- Add /ipfs-stream endpoint to get IPFS playlist URL
- Update /stream endpoint to redirect to IPFS when available
- Add GPU persistence mode (STREAMING_GPU_PERSIST=1) to keep frames on GPU
- Add hardware video decoding (NVDEC) support for faster video processing
- Add GPU-accelerated primitive libraries: blending_gpu, color_ops_gpu, geometry_gpu
- Add streaming_gpu module with GPUFrame class for tracking CPU/GPU data location
- Add Dockerfile.gpu for building GPU-enabled worker image

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 20:23:16 +00:00
giles
a57be27907 Add live video streaming for in-progress renders 2026-02-03 00:27:19 +00:00
giles
9302283e86 Add admin token support to runs list endpoint 2026-02-03 00:23:14 +00:00
giles
2d520cf256 Allow admin token auth for stream endpoint
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 23:37:57 +00:00
giles
2081092ce8 Add ADMIN_TOKEN for admin endpoints and allow token-based auth
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 23:28:19 +00:00
giles
d20eef76ad Fix completed runs not appearing in list + add purge-failed endpoint
- Update save_run_cache to also update actor_id, recipe, inputs on conflict
- Add logging for actor_id when saving runs to run_cache
- Add admin endpoint DELETE /runs/admin/purge-failed to delete all failed runs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 23:24:39 +00:00
giles
581da68b3b Add streaming endpoint to runs router
- Add /runs/stream POST endpoint for streaming recipes
- Accepts recipe_sexp, sources_sexp, audio_sexp
- Submits to run_stream Celery task

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-02 19:18:47 +00:00
gilesb
280dddebd0 Fix authentication to support both header and cookie auth
All API endpoints now use require_auth or get_current_user which handle
both Authorization header (for CLI) and cookies (for browser). Previously
many endpoints only checked cookies via get_user_from_cookie.

Changed files:
- runs.py: list_runs, run_detail, run_plan, run_artifacts, plan_node_detail, ui_discard_run
- recipes.py: list_recipes, get_recipe, ui_discard_recipe
- storage.py: list_storage, add_storage_form, delete_storage, test_storage, storage_type_page
- cache.py: get_cached, list_media, get_metadata_form, update_metadata_htmx

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:56:52 +00:00
gilesb
ee8719ac0b Fix media friendly names, metadata display, output recording, and plan display
- Add friendly name display to media detail and list pages
- Unpack nested meta fields to top level for template access
- Fix output_cid mismatch: use IPFS CID consistently between cache and database
- Add dual-indexing in cache_manager to map both IPFS CID and local hash
- Fix plan display: accept IPFS CIDs (Qm..., bafy...) not just 64-char hashes
- Add friendly names to recipe listing
- Add recipe upload button and handler to recipes list
- Add debug logging to recipe listing

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 14:21:39 +00:00
gilesb
98ca2a6c81 Fix recipe listing, effects count, and add nav counts to all pages
- Fix list_by_type to return node_id (IPFS CID) instead of local hash
- Fix effects count on home page (count from _effects/ directory)
- Add nav_counts to all page templates (recipes, effects, runs, media, storage)
- Add editable metadata section to cache/media detail page
- Show more metadata on recipe detail page (ID, IPFS CID, step count)
- Update tests for new list_by_type behavior

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 13:30:11 +00:00
gilesb
991db29e27 Fix input preview hash -> cid key mismatch
Template uses inp.cid and input.cid but router created previews with
'hash' key. Fixed both input_previews and run_inputs to use 'cid'.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:27:05 +00:00
gilesb
977d9a9258 Fix artifact dict key mismatch (hash -> cid)
Template runs/detail.html expects artifact.cid but code provided
artifact.hash, causing UndefinedError when viewing run details.

- Change run_service.get_run_artifacts to return 'cid' key
- Change runs.py router inline artifact creation to use 'cid' key
- Add regression tests for artifact data structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 12:05:13 +00:00
gilesb
92d26b2b72 Rename content_hash/output_hash to cid throughout
Refactor to use IPFS CID as the primary content identifier:
- Update database schema: content_hash -> cid, output_hash -> output_cid
- Update all services, routers, and tasks to use cid terminology
- Update HTML templates to display CID instead of hash
- Update cache_manager parameter names
- Update README documentation

This completes the transition to CID-only content addressing.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 08:02:44 +00:00
gilesb
3599f3779b Add IPFS link to recipe S-expression display
- Show ipfs://... link next to recipe S-expression header
- Links to ipfs.io gateway for viewing the raw S-expression

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:36:14 +00:00
gilesb
65a8170192 Use native S-expression for recipe/plan display
- Display recipe's original S-expression when available (code is data)
- Fall back to generating S-expression from plan for legacy JSON
- Run service now prefers .sexp plan files over .json
- Add get_run_plan_sexp() for direct S-expression access

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:26:05 +00:00
gilesb
f554122b07 Replace plan JSON with colored S-expression display
- Add plan_to_sexp() to convert plan to S-expression format
- Syntax highlighting for S-expressions:
  - Pink: special forms (plan, recipe, def, ->)
  - Blue: primitives (source, effect, sequence, etc.)
  - Purple: keywords (:input, :name, etc.)
  - Green: strings
  - Yellow: parentheses
  - Gray: comments

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:23:12 +00:00
gilesb
82d94f6e0e Add inline media previews for runs list and detail page
- Run card shows thumbnail previews for inputs and output
- Run detail shows output media inline (image/video/audio)
- Add audio detection (MP3, FLAC, OGG, WAV) to detect_media_type
- Add debug logging for recipe count on home page
- Add console.log debugging for DAG elements

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:20:26 +00:00
gilesb
5c3558e1ba Fix DAG visualization and step link handling
- Handle dict inputs ({"node": "id"}) when building DAG edges
- Add normalize_inputs() to convert dict inputs to node IDs for steps
- Fix _parse_inputs to use _json.loads (correct import alias)
- Add SOURCE/EFFECT/SEQUENCE colors to node color maps

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-12 00:05:50 +00:00
gilesb
48faf7ee43 Fix media type detection using magic bytes
Use detect_media_type() with magic bytes instead of mimetypes.guess_type()
which requires file extensions. Cache files are stored by content hash
without extensions, so magic byte detection is needed.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:53:40 +00:00
gilesb
8e70a9b9f2 Add interactive plan node selection with media previews
- Click plan nodes (in DAG or list) to see details in side panel
- URL updates to #node-{id} for direct linking
- Node detail panel shows: type, status, inputs, output, config
- Inputs can be clicked to navigate to that node
- Inputs tab now shows media previews (image/video/audio)
- Steps include config data for display

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:30:43 +00:00
gilesb
d73592fbe2 Fix DAG visualization node IDs
Add "id" field to plan steps so edges connect correctly.
Previously steps only had "name" but dag_elements looked for "id",
causing edges to reference non-existent nodes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:21:33 +00:00
gilesb
0ead728fde Fix dag_elements undefined in run detail template
Add dag_elements to the get_run endpoint render call to match
what the detail.html template expects.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:15:52 +00:00
gilesb
f7dbb952ab Preserve recipe name in run service and templates
- Pass recipe_name through create_run to display friendly names
- Update templates to show name instead of hash
- Fall back to truncated hash if no name available

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:09:33 +00:00
gilesb
9df78f771d Fix run detail: add username, total_steps, recipe_name
- Extract username from actor_id format (@user@server)
- Set total_steps and executed from recipe nodes
- Use recipe name for display instead of hash

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:23:08 +00:00
gilesb
145c69f21b Add artifacts to run detail view
Build artifacts list from run output_hash and detect media type
for display in the artifacts tab.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:08:06 +00:00
gilesb
ccf28bd351 Load recipe to show plan in run detail view
When viewing a run, try to load the recipe by hash to display
the plan/steps in the UI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:05:41 +00:00
gilesb
50ea0f1491 Fix run detail: default to JSON, only HTML if browser requests it
API clients like Python requests send Accept: */* which wasn't
matching wants_json(). Switch to checking wants_html() instead
so API clients get JSON by default.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:03:53 +00:00
gilesb
95ffe9fa69 Fix run detail page and recipe name
- Render HTML template for run detail (not just JSON)
- Get recipe name from pending_runs instead of hardcoding "dag"

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 21:56:08 +00:00
gilesb
156014a1f3 Add Share to L2 buttons for media, recipes, and runs
- Updated cache detail button from "Publish to IPFS" to "Share to L2"
- Added Share to L2 button on recipe detail page
- Added Share to L2 button on run detail page
- Created /recipes/{id}/publish endpoint
- Created /runs/{id}/publish endpoint (publishes run output)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 18:21:14 +00:00
gilesb
970faa3fa0 Fix run recipe: optional fields in RunStatus, list->dict nodes
- Made recipe and inputs optional in RunStatus model
- Convert DAG nodes from list format to dict format when running recipes

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:50:36 +00:00
gilesb
2e9ba46f19 Use UserContext from artdag_common, remove duplicate
- Updated requirements.txt to use art-common@11aa056 with l2_server field
- All routers now import UserContext from artdag_common
- Removed duplicate UserContext from auth_service.py
- dependencies.py sets l2_server from settings on user context

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 17:38:02 +00:00
giles
29da91e01a Refactor to modular app factory architecture
- Replace 6847-line monolithic server.py with 26-line entry point
- All routes now in app/routers/ using Jinja2 templates
- Add plan_node.html template for step details
- Add plan node route to runs router with cache_id lookup
- Backup old server as server_legacy.py

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 11:48:24 +00:00
giles
945fb3b413 Fix analysis display for recipe-based runs
Add get_run_analysis() to RunService to load per-input analysis from
CACHE_DIR/analysis/{hash}.json files. Update runs router and template
to display tempo, beats, energy, and beat timeline visualization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:05:40 +00:00
giles
022f88bf0c Complete L1 router and template migration
- Full implementation of runs, recipes, cache routers with templates
- Auth and storage routers fully migrated
- Jinja2 templates for all L1 pages
- Service layer for auth and storage

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:49:37 +00:00
giles
adc876dbd6 Add modular app structure for L1 server refactoring
Phase 2 of the full modernization:
- App factory pattern with create_app()
- Settings via dataclass with env vars
- Dependency injection container
- Router stubs for auth, storage, api, recipes, cache, runs
- Service layer stubs for run, recipe, cache
- Repository layer placeholder

Routes are stubs that import from legacy server.py during migration.
Next: Migrate each router fully with templates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:08:08 +00:00