Commit Graph

55 Commits

Author SHA1 Message Date
gilesb
63ec895514 Update README with config endpoints
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 03:20:04 +00:00
gilesb
4a99866602 Add config/recipe support for DAG-based jobs
- Add PyYAML dependency for parsing config files
- Add Pydantic models: VariableInput, FixedInput, ConfigStatus, ConfigRunRequest
- Add Redis storage functions for configs
- Add config YAML parsing with variable and fixed input detection
- Add config API endpoints: upload, list, get, delete, run
- Add config UI: Configs tab, list page, detail page with run form
- Add HTMX endpoints for config operations
- Add pinning on publish: configs and their fixed inputs are pinned
  when runs from configs are published to L2
- Clean up debug logging in cache_manager

Config YAML format supports:
- Fixed inputs: resolve asset hashes from registry
- Variable inputs: marked with `input: true`, filled at run time
- DAG definition with nodes and edges
- Registry of assets and effects

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 03:17:50 +00:00
gilesb
f23a721816 Use local pinned metadata for deletion checks instead of L2 API
- Add is_pinned(), pin(), _load_meta(), _save_meta() to L1CacheManager
- Update can_delete() and can_discard_activity() to check local pinned status
- Update run deletion endpoints (API and UI) to check pinned metadata
- Remove L2 shared check fallback from run deletion
- Fix L2SharedChecker to return True on error (safer - prevents accidental deletion)
- Update tests for new pinned behavior

When items are published to L2, the publish flow marks them as pinned
locally. This ensures items remain non-deletable even if L2 is unreachable,
and both outputs AND inputs of published runs are protected.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 02:44:18 +00:00
gilesb
f61c82dd51 Add PYTHONPATH to Dockerfile for worker imports 2026-01-08 02:21:05 +00:00
gilesb
01f6db0621 Add debug logging to cache lookup 2026-01-08 02:11:37 +00:00
gilesb
3e12596dbf Fix cache lookup to work across processes
cache_manager.py:
- get_by_content_hash() now tries direct cache.get(content_hash)
  since uploads use content_hash as node_id
- This works even if cache index hasn't been reloaded

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 02:02:47 +00:00
gilesb
13b93ab17c Add delete button for runs in UI
- Add delete_html section to run detail page with delete button
- Add /ui/runs/{run_id}/discard HTMX endpoint
- Failed runs can always be deleted without restrictions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:53:26 +00:00
gilesb
6142c8bbab Allow failed runs to always be deleted
Failed runs have no output to protect, so they can be deleted
without checking L2 shared status or activity records.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:48:23 +00:00
gilesb
0e4f596401 Fix render_effect to find uploads in new cache location
The legacy render_effect task was looking for inputs only at
CACHE_DIR/{hash}, but uploads now go through cache_manager which
stores them in CACHE_DIR/nodes/{node_id}/. Now uses
cache_manager.get_by_content_hash() which checks both locations.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:47:23 +00:00
gilesb
ff195a7ce5 Add multi-step DAG execution support
tasks.py:
- Import artdag DAG, Node, Engine, Executor
- Register executors for effect:dog, effect:identity, SOURCE
- Add execute_dag task for running full DAG workflows
- Add build_effect_dag helper for simple effect-to-DAG conversion

server.py:
- Add use_dag and dag_json fields to RunRequest
- Update create_run to support DAG mode
- Handle both legacy render_effect and new execute_dag result formats
- Import new tasks (execute_dag, build_effect_dag)

The DAG engine executes nodes in topological order with automatic
caching. This enables multi-step pipelines like: source -> effect1 ->
effect2 -> output.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:45:29 +00:00
gilesb
034c7542c4 Fix cache listing to include files from new structure
- Update list_all() to scan cache_dir for legacy files directly
  (old files stored as CACHE_DIR/{hash}, not CACHE_DIR/legacy/)
- Update cache listing endpoints to use cache_manager.list_all()
  instead of iterating CACHE_DIR.iterdir() directly
- This ensures uploaded files appear in the cache UI regardless
  of whether they're in the old or new cache structure

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:37:17 +00:00
gilesb
a97c6309d5 Fix deletion rules: runs deletable, cache items protected
- Run deletion: Handle legacy runs without activity records by
  checking L2 shared status directly (instead of failing)
- Cache deletion: Check Redis runs in addition to activity store
  to prevent deleting inputs/outputs that belong to runs
- Add find_runs_using_content() helper to check if content_hash
  is used as input or output of any run

This fixes the inverted deletion logic where runs couldn't be
deleted but their cache items could.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:29:03 +00:00
gilesb
f8ec42b445 Refactor cache access to use cache_manager consistently
- Remove symlink hack from cache_file() - no longer needed
- Add get_cache_path() helper for content_hash lookups
- Update all CACHE_DIR / content_hash patterns to use cache_manager
- Fix cache_manager.get_by_content_hash() to check path.exists()
- Fix legacy path lookup (cache_dir not legacy_dir)
- Update upload endpoint to use cache_manager.put()

This ensures cache lookups work correctly for both legacy files
(stored directly in CACHE_DIR) and new files (stored in nodes/).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:21:11 +00:00
gilesb
26768b5942 Fix cache file serving after cache_manager integration
- Update get_cached endpoint to check cache_manager first, then legacy
- Update get_cached_mp4 endpoint similarly
- Make cache_file() create symlink in legacy location for backward compat

Fixes issue where newly cached outputs weren't being served because
they were stored in new location but served from legacy location.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 01:08:40 +00:00
gilesb
ea60afb624 Force pip cache bust on deploy
- Add CACHEBUST build arg to Dockerfile
- Pass timestamp to --build-arg in deploy.sh
- Ensures artdag is re-fetched from GitHub on each deploy

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 00:59:20 +00:00
gilesb
e4fd5eb010 Integrate artdag cache with deletion rules
- Add cache_manager.py with L1CacheManager wrapping artdag Cache
- Add L2SharedChecker for checking published status via L2 API
- Update server.py to use cache_manager for storage
- Update DELETE /cache/{content_hash} to enforce deletion rules
- Add DELETE /runs/{run_id} endpoint for discarding runs
- Record activities when runs complete for deletion tracking
- Add comprehensive tests for cache manager

Deletion rules enforced:
- Cannot delete items published to L2
- Cannot delete inputs/outputs of runs
- Can delete orphaned items
- Runs can only be discarded if no items are shared

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 00:51:18 +00:00
gilesb
9a399cfcee Add clean URLs with content negotiation and pagination for L1
- Add content negotiation (HTML for browsers, JSON for APIs)
- Clean URLs: /runs, /cache, /run/{id}, /cache/{hash}/detail
- Auth routes: /login, /logout, /register with clean URLs
- Add render_page() helper for consistent page layout
- Add infinite scroll pagination for HTML views
- Add API pagination with ?page=1&limit=20
- Redirect old /ui/* routes to clean URLs
- Update nav links to use clean URLs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 22:55:44 +00:00
gilesb
b2e4f81edf Add cache pinning and discard functionality
- Pin items when published (published, input_to_published)
- Discard endpoint (DELETE /cache/{hash}) refuses pinned items
- UI shows pin status and discard button on cache detail
- Run publish also pins output and all inputs for provenance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 21:36:52 +00:00
gilesb
a3a37c568a Add metadata editing UI to cache detail page
- Added /ui/cache/{hash}/meta-form endpoint for HTMX form
- Origin selector (self vs external URL)
- Description and tags fields
- Publish to L2 with asset name
- Update on L2 for already-published items

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 21:22:48 +00:00
gilesb
5f58bf117e Increase L2 publish timeout to 30 seconds
L2 needs to call back to L1 to fetch run details, which can take
longer than 10 seconds especially with network latency.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 21:13:44 +00:00
gilesb
3436a22a94 Fix iOS video playback in list views
Use video_src_for_request() for iOS MP4 transcoding in:
- ui_runs partial
- ui_cache_list partial
- ui_run_partial (also added request parameter)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 20:58:40 +00:00
gilesb
1e95badddb Update UI to use Tailwind CSS with dark theme
- Replace custom CSS with Tailwind CSS via CDN
- Consistent dark theme across all pages
- Responsive layouts with mobile-friendly breakpoints
- Updated: home, login/register, runs list, cache list, run detail, cache detail
- Modern card-based UI with hover states and status badges

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 20:44:11 +00:00
gilesb
a0a719cda5 fix: add ffmpeg to Docker image for video transcoding
Required for iOS MKV→MP4 on-the-fly transcoding.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 20:06:44 +00:00
gilesb
a5187c0f45 feat: iOS video support with on-the-fly MKV→MP4 transcoding
- Add /cache/{hash}/mp4 endpoint that transcodes MKV to MP4
- First request transcodes (using ffmpeg), result cached as {hash}.mp4
- Subsequent requests serve cached MP4 directly
- Detect iOS devices and use MP4 endpoint for video src
- Works with Cloudflare caching for efficiency

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 20:04:30 +00:00
gilesb
809d8e452b fix: add playsinline for iOS video support
iOS Safari requires playsinline attribute for inline video playback.

Note: MKV videos still won't play on iOS - only MP4/H.264 is supported.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:57:27 +00:00
gilesb
f9e22171de feat: add L1_PUBLIC_URL config for many-to-many L1/L2 support
- Add L1_PUBLIC_URL environment variable
- UI publish sends L1 URL to L2 so it knows where to fetch run data
- Update .env.example with new config

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:40:59 +00:00
gilesb
223568df3f fix: use env_file for L2 server config
- Add env_file support for L2_SERVER and L2_DOMAIN
- Fix typo in L2_SERVER URL (ros-ash -> rose-ash)
- Add .env.example with required variables

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:29:40 +00:00
gilesb
917146a7f3 feat: add publish to L2 button in run detail web UI
- Add POST /ui/publish-run/{run_id} endpoint for UI publishing
- Add publish form with HTMX to run detail page
- Shows publish button for completed runs with outputs
- Uses existing L2 /registry/record-run endpoint

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:12:46 +00:00
gilesb
85f349a7ab feat: add L1-L2 publish integration endpoints
- Add POST /cache/{hash}/publish to publish cache items to L2
- Add PATCH /cache/{hash}/republish to sync metadata updates
- Validates origin is set before publishing
- Tracks publish status in cache metadata (to_l2, asset_name, timestamps)
- Forwards auth token to L2 for authentication

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:08:03 +00:00
gilesb
bfa7568db4 feat: add cache metadata, folders, and collections API
- Update save_cache_meta() to support updates (not just create)
- Add GET/PATCH /cache/{hash}/meta endpoints for metadata management
- Add user data storage for folders/collections (per-user .json files)
- Add folder CRUD endpoints (/user/folders)
- Add collection CRUD endpoints (/user/collections)
- Add cache list filtering by folder, collection, and tags
- Support origin tracking (self vs external URL)
- Support tags, description, and organization metadata

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 18:57:38 +00:00
gilesb
0cbfd87711 feat: per-user cache visibility and login protection
- Run detail page requires login and ownership check
- Cache uploads require login and track uploader metadata
- Cache list filtered to show only user's own files (uploaded or from runs)
- Cache detail view requires login and ownership check
- Add helper functions for cache metadata and user hash lookup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 18:18:16 +00:00
gilesb
e01db2ed26 fix: require login to view runs and cache in UI
- Runs list now requires login, shows only user's own runs
- Cache list now requires login to view

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 18:02:34 +00:00
gilesb
fb354d8afe fix: match both username formats in runs filter
Filter now matches both plain username and ActivityPub format
(@user@domain) to support runs created before and after the
actor ID change.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:56:54 +00:00
gilesb
0b50f5ea95 feat: add cache browsing UI with tabs for runs and cache
- Add tabbed navigation between Runs and Cache views
- Add /ui/cache-list endpoint to list cached files with thumbnails
- Show media type, file size, and preview for each cached item
- Sort cache items by modification time (newest first)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:41:55 +00:00
gilesb
618e3b1e04 feat: enhance provenance with infrastructure, actor ID, and commit tracking
- Add infrastructure field to RunStatus model
- Store infrastructure (software/hardware) from task result
- Format username as ActivityPub actor ID (@user@domain)
- Display owner, effects commit, and infrastructure in UI provenance section
- Add artdag commit tracking for identity effect
- Include infrastructure in raw JSON provenance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:26:07 +00:00
giles
c7074a9261 deploy 2026-01-07 17:16:43 +00:00
gilesb
c7ee59968a fix: identity effect links to GitHub, others to rose-ash
- Identity effect is part of artdag package on GitHub
- Other effects (dog) link to rose-ash effects repo with commit hash
- Added effect_url field to RunStatus for proper URL storage
- Extract repo_url from provenance instead of building it

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:55:59 +00:00
gilesb
631571ed88 feat: add authentication to L1 server
- Runs require auth token (verified with L2)
- Store username with each run
- UI login/register/logout via L2
- Filter runs by logged-in user
- Cookie-based auth for UI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:43:56 +00:00
gilesb
0c7e43e069 feat: link effect to specific git commit for provenance
- Capture effects repo commit hash at render time
- Store effects_commit in run record
- Effect URLs now link to exact commit, not main branch
- Include commit in raw JSON provenance

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:29:40 +00:00
gilesb
5f7b6c3031 feat: show raw provenance JSON on detail page
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:24:29 +00:00
gilesb
62891f57fe feat: add cache view page with media display
- /ui/cache/{hash} shows image/video inline
- File details: hash, type, size
- Download button
- All cache links now go to view page

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:23:42 +00:00
gilesb
30050f7a65 style: left-justify layout, increase text sizes
- Runs/cards max-width 900px, left-aligned
- Media boxes left-justified, max 400px each
- Increased font sizes throughout

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:20:38 +00:00
gilesb
c03a26dbf2 feat: add detail page with provenance and linkable hashes
- Click run to see full detail page
- Effect name links to git.rose-ash.com source
- Input/output hashes link to cache downloads
- Full provenance: effect, inputs, output, timestamps

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:18:44 +00:00
gilesb
73908b318d feat: responsive side-by-side input/output layout
- Desktop: input left, output right
- Mobile (<600px): stacked vertically

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:15:32 +00:00
gilesb
dc9c13ffd9 feat: add HTML home page with nav to UI and docs
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:13:56 +00:00
gilesb
56d6f53695 feat: show input media in UI
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:12:40 +00:00
gilesb
5fbe9cbc45 feat: add HTMX web UI for viewing runs and results
- /ui endpoint shows runs list
- Auto-detects image vs video content
- HTMX polling for running jobs
- Dark theme

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:06:07 +00:00
gilesb
e6184b8bd3 fix: use cache for input files instead of hardcoded paths 2026-01-07 13:42:36 +00:00
giles
32b6aef1ea refactor: standalone docker-compose for L1 celery stack 2026-01-07 13:39:04 +00:00
gilesb
2258c461af feat: clone effects repo in Docker, use EFFECTS_PATH env var 2026-01-07 13:16:55 +00:00