Commit Graph

155 Commits

Author SHA1 Message Date
giles
17b92c77ef Fix async function call for load_plan_for_run_with_fallback
Use await directly instead of asyncio.to_thread() since the
function is already async.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 11:31:27 +00:00
giles
64ef9396d6 Add /help routes to display README documentation
Provides /help index and /help/{doc_name} routes to view
L1 server and Common library READMEs in the web UI.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 11:30:36 +00:00
giles
255d44fbf6 Remove redundant documentation UI routes
/docs now correctly points to FastAPI's Swagger API docs.
README files can be viewed directly in the git repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 11:21:16 +00:00
giles
de9fbaed4a Add debug logging for plan loading 2026-01-11 10:24:58 +00:00
giles
c68c0cedba Fix plan node detail to generate plan from recipe if not cached
- Add load_plan_for_run_with_fallback() that generates plan from recipe
  when not found in cache
- Share this helper between plan page and node detail endpoint
- Removes code duplication

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 10:23:03 +00:00
giles
45eb9c0b1c Fix plan loading and add inline DAG view for recipes
- Add load_plan_for_run() helper that tries plan_id first, then matches by inputs
- Fix "plan not found" error when clicking plan nodes
- Add inline DAG visualization to recipe detail page with tabs (DAG View / YAML Source)
- Recipe page now uses render_page_with_cytoscape for proper DAG rendering

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 10:09:07 +00:00
giles
28349ad944 Add documentation routes and update README
- Update README with comprehensive documentation covering IPFS-primary mode,
  3-phase execution, storage providers, and all API endpoints
- Add /docs routes to serve markdown documentation as styled HTML
- Include common library documentation in web interface

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 09:58:31 +00:00
gilesb
ca8bfd8705 Add hybrid state manager for distributed L1 coordination
Implements HybridStateManager providing fast local Redis operations
with background IPNS sync for eventual consistency across L1 nodes.

- hybrid_state.py: Centralized state management (cache, claims, analysis, plans, runs)
- Updated execute_cid.py, analyze_cid.py, orchestrate_cid.py to use state manager
- Background IPNS sync (configurable interval, disabled by default)
- Atomic claiming with Redis SETNX for preventing duplicate work

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 09:36:14 +00:00
giles
f11cec9d48 Add SPA-style navigation for plan nodes
- Add /run/{run_id}/plan/node/{step_id} endpoint for node details
- Node click updates URL without full page reload (pushState)
- Browser back/forward works correctly
- Refreshing page preserves selected node via ?node= parameter
- Node details loaded via fetch with partial HTML response

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 09:30:27 +00:00
gilesb
0634142576 Add ARTDAG_CLUSTER_KEY to docker-compose
Reads from environment or .env file.
Must be same on server and workers for consistent cache_ids.

Generate with: openssl rand -hex 32

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 09:17:41 +00:00
giles
c145d4a427 Add IPFS_PRIMARY mode UI support
- Add output_ipfs_cid field to RunStatus model
- Handle output_cid from IPFS-primary task results
- Add /ipfs/{cid} redirect route to IPFS gateway
- Add /ipfs/{cid}/raw to fetch and serve IPFS content
- Show IPFS output in run detail when output_hash unavailable
- Display step CIDs on plan page for IPFS_PRIMARY runs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:32:27 +00:00
gilesb
68c2e45541 Add IPFS_PRIMARY environment variable for server
When IPFS_PRIMARY=true:
- /api/run-recipe uses run_recipe_cid task
- Recipe registered on IPFS before execution
- Input CIDs fetched from cache manager
- Everything flows through IPFS, no local cache

Usage in docker-compose:
  environment:
    - IPFS_PRIMARY=true

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:27:24 +00:00
giles
791fe31483 Fix double-encoded JSON in Plan JSON display
Parse nested plan_json field before displaying to avoid escaped newlines.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:19:44 +00:00
gilesb
25f7213741 Add IPFS-primary orchestration
Complete pipeline with everything on IPFS:
- register_input_cid / register_recipe_cid
- generate_plan_cid (stores plan on IPFS)
- execute_plan_from_cid (fetches plan from IPFS)
- run_recipe_cid (full pipeline, returns output CID)
- run_from_local (convenience: local files → IPFS → run)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:18:34 +00:00
gilesb
92d154f524 Add IPFS-primary analysis task
- Fetches input from IPFS by CID
- Stores analysis JSON on IPFS
- Returns analysis_cid
- Redis cache: input_hash:features → analysis CID

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:15:55 +00:00
giles
1869a76157 Fix CACHE_DIR default path mismatch across files
Changed default from /data/cache to ~/.artdag/cache for local runs.
Docker sets CACHE_DIR=/data/cache via environment variable.

Files updated:
- tasks/analyze.py
- tasks/orchestrate.py
- app/config.py

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:14:39 +00:00
giles
8fd9acce28 Add step list with cache links and Plan JSON to plan page
- Show numbered step list with cache_id links for each completed step
- Add collapsible "Show Plan JSON" section with full plan data
- Steps show type, name, status badge, and clickable output link

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:12:39 +00:00
giles
945fb3b413 Fix analysis display for recipe-based runs
Add get_run_analysis() to RunService to load per-input analysis from
CACHE_DIR/analysis/{hash}.json files. Update runs router and template
to display tempo, beats, energy, and beat timeline visualization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:05:40 +00:00
giles
29d8d06d76 Fix plan artifacts display for completed runs
- Add get_run_plan() and get_run_artifacts() methods to RunService
- Merge step_results into plan steps to show cache_id per step
- Display output hash links under each completed step
- Use cache manager for artifact path lookups
- Fix RunService constructor to accept database param

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 08:01:12 +00:00
giles
ae68eeef52 Add Plan JSON view to run detail page
Shows collapsible JSON of execution plan below steps list.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:57:56 +00:00
giles
022f88bf0c Complete L1 router and template migration
- Full implementation of runs, recipes, cache routers with templates
- Auth and storage routers fully migrated
- Jinja2 templates for all L1 pages
- Service layer for auth and storage

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:49:37 +00:00
gilesb
383dbf6e03 Add IPFS-primary execute_step_cid implementation
Simplified step execution where:
- Steps receive CIDs, produce CIDs
- No local cache management (IPFS handles it)
- Minimal Redis: just claims + cache_id→CID mapping
- Temp workspace for execution, cleaned up after

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:42:34 +00:00
giles
adc876dbd6 Add modular app structure for L1 server refactoring
Phase 2 of the full modernization:
- App factory pattern with create_app()
- Settings via dataclass with env vars
- Dependency injection container
- Router stubs for auth, storage, api, recipes, cache, runs
- Service layer stubs for run, recipe, cache
- Repository layer placeholder

Routes are stubs that import from legacy server.py during migration.
Next: Migrate each router fully with templates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 07:08:08 +00:00
giles
3e24156c99 Fix undefined completed_count variable in plan visualization
Added completed_count stat for steps executed during run (vs cached_count for
steps already in cache). Also fixed pending_count calculation to account for
completed steps.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 01:24:22 +00:00
giles
b9f8b5538b Use orchestrated run_recipe for all recipe runs, remove v2 prefix
- UI recipe run now uses tasks.orchestrate.run_recipe (3-phase)
- Deterministic run_id via compute_run_id for cache deduplication
- Check for already-completed runs before starting
- Rename /api/v2/* endpoints to /api/* (plan, execute, run-recipe, run)
- All recipe runs now go through: analyze → plan → execute

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 01:07:14 +00:00
giles
41ceae1e6c Fix plan visualization to show completed status and artifact links
- Check cache_manager.has_content(cache_id) to determine if step is cached
- Show green border for cached nodes in completed runs
- Display artifact preview (video) when clicking on cached nodes
- Add "View" button to access cached artifacts directly
- Simplify node data structure (hasCached instead of outputs array)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 01:03:11 +00:00
gilesb
3db606bf15 Plan-based caching with artifact playback in UI
RunStatus now stores:
- plan_id, plan_name for linking to execution plan
- step_results for per-step execution status
- all_outputs for all artifacts from all steps

Plan visualization:
- Shows human-readable step names from recipe structure
- Video/audio artifact preview on node click
- Outputs list with links to cached artifacts
- Stats reflect actual execution status (completed/cached/pending)

Execution:
- Step results include outputs list with cache_ids
- run_plan returns all outputs from all steps
- Support for completed_by_other status

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 00:20:19 +00:00
giles
36bf0cd3f7 Generate execution plans on-the-fly for plan visualization
Instead of only showing cached plans, now attempts to generate
a plan from the recipe if no cached plan is found. This works
for recipe-based runs even if they were started via the legacy
execute_dag path.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 23:33:38 +00:00
giles
b498d6fe4b Enable Celery events for Flower monitoring
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 23:27:36 +00:00
giles
e7d3a5ed6c Add DAG visualization for runs and recipes
- Add Cytoscape.js infrastructure for interactive DAG visualization
- New run sub-pages: /run/{id}/plan, /run/{id}/analysis, /run/{id}/artifacts
- Plan page shows execution DAG with cached/pending status
- Analysis page displays tempo, beats, energy per input
- Artifacts page lists all cached items with thumbnails
- New /recipe/{id}/dag page for recipe structure visualization
- Add sub-tabs navigation to run detail page
- Add JSON API endpoints for future WebSocket support
- Architecture designed for real-time updates (global Cytoscape instance)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 23:27:28 +00:00
gilesb
1feecc8c05 Add Flower for Celery monitoring
Flower provides a web UI to monitor:
- Active/completed/failed tasks
- Worker status and stats
- Task history and details

Accessible on port 5555.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 22:06:17 +00:00
gilesb
e56b4792c7 Show all inputs with names on run detail page
Previously only displayed the first input. Now shows all inputs with
their names from the recipe (variable_inputs and fixed_inputs).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 19:48:07 +00:00
gilesb
9a3d6949fc Store and use activity_id for L2 links
- Add activity_id column to l2_shares table
- Store activity_id when publishing to L2
- Link to /activities/{activity_id} instead of /assets/{name}
- Falls back to asset link if no activity_id

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 18:56:23 +00:00
gilesb
3d45efa90b Link to activity instead of asset in "View on L2"
After publishing, link to /activities/{activity_id} instead of
/assets/{asset_name} so user sees the published run.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 18:38:07 +00:00
gilesb
837f9a6442 Add defensive check for null asset in publish
Prevents NoneType subscript error if L2 returns null asset.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 15:08:03 +00:00
gilesb
eaff758395 Rename tasks.py to legacy_tasks.py to avoid conflict with tasks/ package
The tasks/ directory for 3-phase execution was shadowing the old tasks.py.
Renamed to legacy_tasks.py and updated all imports.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 12:05:27 +00:00
gilesb
f7890dd1ad Add 3-phase execution with IPFS cache and hash-based task claiming
New files:
- claiming.py - Redis Lua scripts for atomic task claiming
- tasks/analyze.py - Analysis Celery task
- tasks/execute.py - Step execution with IPFS-backed cache
- tasks/orchestrate.py - Plan orchestration (run_plan, run_recipe)

New API endpoints (/api/v2/):
- POST /api/v2/plan - Generate execution plan
- POST /api/v2/execute - Execute a plan
- POST /api/v2/run-recipe - Full 3-phase pipeline
- GET /api/v2/run/{run_id} - Get run status

Features:
- Hash-based task claiming prevents duplicate work
- Parallel execution within dependency levels
- IPFS-backed cache for durability
- Integration with artdag planning module

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 11:44:00 +00:00
gilesb
7d05011daa Add Storage link to L1 navigation bars
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 02:44:52 +00:00
gilesb
2a4f24b7ee Add storage engine configuration to L1, matching L2 implementation
- Copy storage_providers.py from L2 (Pinata, web3.storage, NFT.Storage,
  Infura, Filebase, Storj, local storage providers)
- Add storage management endpoints: GET/POST/PATCH/DELETE /storage
- Add provider-specific pages at /storage/type/{provider_type}
- Include connection testing via POST /storage/{id}/test
- Add HTML UI pages with dark theme matching L2

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 02:41:50 +00:00
gilesb
5ce28abe52 Phase 2: Multiple storage configs per type with new UI structure
- Database: Add description field, remove unique constraint to allow
  multiple configs of same provider type
- UI: Main page shows provider types as cards with counts
- UI: Per-type page (/storage/type/{type}) for managing configs
- API: Add get_user_storage_by_type() for filtered queries
- Form: Add description field for distinguishing configs

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 00:52:28 +00:00
gilesb
58b3240430 Store IPFS CID in database for DAG outputs
Properly unpack cache_manager.put() tuple to get IPFS CID and store
it in PostgreSQL via database.create_cache_item(). This fixes the
"Output has no IPFS CID - cannot publish" error when publishing
from L1 to L2.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 23:35:50 +00:00
gilesb
5cb8d4ec17 Add storage_backends table for user storage config
Prepares L1 for distributed storage integration with user-provided
storage backends (Pinata, web3.storage, local). The storage config
is synced from L2 or can be configured locally per actor.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 23:22:28 +00:00
gilesb
2cfe0f2bd2 Update README with /auth/revoke-user documentation
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 22:23:05 +00:00
gilesb
e485384d2f Add /auth/revoke-user to revoke all tokens for a user
- Track user tokens in Redis set (artdag:user_tokens:{username})
- Register token when user authenticates via /auth
- Add /auth/revoke-user endpoint to revoke all user tokens
- L2 calls this on logout to revoke scoped tokens

Fixes logout not working across L1/L2 (scoped tokens differ from L2 token).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 22:22:08 +00:00
gilesb
9a83722273 Document authentication and token revocation in README
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 18:21:36 +00:00
gilesb
cd60016558 Add token revocation for federated logout
- Add revoke_token() and is_token_revoked() functions using Redis
- Check revocation in get_verified_user_context()
- Add /auth/revoke endpoint for L2 to call on logout
- Revoked tokens stored with 30-day expiry (matching token lifetime)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 18:07:50 +00:00
gilesb
ce4d38b943 Update Web UI routes in README
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:56:02 +00:00
gilesb
38e988586f Pass L1_PUBLIC_URL when verifying tokens with L2
L2 now requires L1 servers to identify themselves for authorization.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:43:19 +00:00
gilesb
9a51d66aec Add /auth endpoint for iOS Safari cross-subdomain auth
iOS Safari blocks shared cookies. Now L2 can redirect to
L1/auth?auth_token=xxx after login, and L1 sets its own
first-party cookie.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 17:06:10 +00:00
gilesb
304f3ad56f Remove login links from L1 - L1 doesn't know where L2 is
L1 can't redirect users to login because it doesn't know which L2
server they use. Users must log in directly at their L2 server,
and the shared cookie will authenticate them on L1.

- Replace "Log in" links with "Not logged in" text
- Remove /login and /register routes
- Keep /logout to just clear cookie and redirect home
- Remove unused DEFAULT_L2_SERVER config

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-09 16:52:32 +00:00