Add documentation routes and update README

- Update README with comprehensive documentation covering IPFS-primary mode,
  3-phase execution, storage providers, and all API endpoints
- Add /docs routes to serve markdown documentation as styled HTML
- Include common library documentation in web interface

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

README.md

@@ -1,212 +1,312 @@
-# Art Celery
+# Art DAG L1 Server
 
-L1 rendering server for the Art DAG system. Manages distributed rendering jobs via Celery workers.
+L1 rendering server for the Art DAG system. Manages distributed rendering jobs via Celery workers with content-addressable caching and optional IPFS integration.
+
+## Features
+
+- **3-Phase Execution**: Analyze → Plan → Execute pipeline for recipe-based rendering
+- **Content-Addressable Caching**: SHA3-256 hashed content with deduplication
+- **IPFS Integration**: Optional IPFS-primary mode for distributed storage
+- **Storage Providers**: S3, IPFS, and local storage backends
+- **DAG Visualization**: Interactive graph visualization of execution plans
+- **SPA-Style Navigation**: Smooth URL-based navigation without full page reloads
+- **L2 Federation**: Publish outputs to ActivityPub registry
 
 ## Dependencies
 
 - **artdag** (GitHub): Core DAG execution engine
 - **artdag-effects** (rose-ash): Effect implementations
+- **artdag-common**: Shared templates and middleware
 - **Redis**: Message broker, result backend, and run persistence
+- **PostgreSQL**: Metadata storage
+- **IPFS** (optional): Distributed content storage
 
-## Setup
+## Quick Start
 
 ```bash
-# Install Redis
+# Install dependencies
-sudo apt install redis-server
-
-# Install Python dependencies
 pip install -r requirements.txt
 
+# Start Redis
+redis-server
+
 # Start a worker
-celery -A celery_app worker --loglevel=info
+celery -A celery_app worker --loglevel=info -E
 
 # Start the L1 server
 python server.py
 ```
 
-## Web UI
+## Docker Swarm Deployment
 
-The server provides a web interface at the root URL:
+```bash
+docker stack deploy -c docker-compose.yml artdag
+```
+
+The stack includes:
+- **redis**: Message broker (Redis 7)
+- **postgres**: Metadata database (PostgreSQL 16)
+- **ipfs**: IPFS node (Kubo)
+- **l1-server**: FastAPI web server
+- **l1-worker**: Celery workers (2 replicas)
+- **flower**: Celery task monitoring
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `HOST` | `0.0.0.0` | Server bind address |
+| `PORT` | `8000` | Server port |
+| `REDIS_URL` | `redis://localhost:6379/5` | Redis connection |
+| `DATABASE_URL` | `postgresql://artdag:artdag@localhost:5432/artdag` | PostgreSQL connection |
+| `CACHE_DIR` | `~/.artdag/cache` | Local cache directory |
+| `IPFS_API` | `/dns/localhost/tcp/5001` | IPFS API multiaddr |
+| `IPFS_GATEWAY_URL` | `https://ipfs.io/ipfs` | Public IPFS gateway |
+| `IPFS_PRIMARY` | `false` | Enable IPFS-primary mode |
+| `L1_PUBLIC_URL` | `http://localhost:8100` | Public URL for redirects |
+| `L2_SERVER` | - | L2 ActivityPub server URL |
+| `L2_DOMAIN` | - | L2 domain for federation |
+| `ARTDAG_CLUSTER_KEY` | - | Cluster key for trust domains |
+
+### IPFS-Primary Mode
+
+When `IPFS_PRIMARY=true`, all content is stored on IPFS:
+- Input files are added to IPFS on upload
+- Analysis results stored as JSON on IPFS
+- Execution plans stored on IPFS
+- Step outputs pinned to IPFS
+- Local cache becomes a read-through cache
+
+This enables distributed execution across multiple L1 nodes sharing the same IPFS network.
+
+## Web UI
 
 | Path | Description |
 |------|-------------|
 | `/` | Home page with server info |
 | `/runs` | View and manage rendering runs |
-| `/run/{id}` | Run detail page |
+| `/run/{id}` | Run detail with tabs: Plan, Analysis, Artifacts |
+| `/run/{id}/plan` | Interactive DAG visualization |
+| `/run/{id}/analysis` | Audio/video analysis data |
+| `/run/{id}/artifacts` | Cached step outputs |
 | `/recipes` | Browse and run available recipes |
 | `/recipe/{id}` | Recipe detail page |
+| `/recipe/{id}/dag` | Recipe DAG visualization |
 | `/media` | Browse cached media files |
+| `/storage` | Manage storage providers |
 | `/auth` | Receive auth token from L2 |
-| `/auth/revoke` | Revoke a specific token |
-| `/auth/revoke-user` | Revoke all tokens for a user (called by L2 on logout) |
 | `/logout` | Log out |
 | `/download/client` | Download CLI client |
 
-## Authentication
+## API Reference
-
-L1 servers authenticate users via L2 (the ActivityPub registry). No shared secrets are required.
-
-### Configuration
-
-```bash
-export L1_PUBLIC_URL=https://celery-artdag.rose-ash.com
-```
-
-### How it works
-
-1. User clicks "Attach" on L2's Renderers page
-2. L2 creates a **scoped token** bound to this specific L1
-3. User is redirected to L1's `/auth?auth_token=...`
-4. L1 calls L2's `/auth/verify` to validate the token
-5. L2 checks: token valid, not revoked, scope matches this L1
-6. L1 sets a local cookie and records the token
-
-### Token revocation
-
-When a user logs out of L2, L2 calls `/auth/revoke-user` on all attached L1s. L1 maintains a Redis-based token tracking and revocation system:
-
-- Tokens registered per-user when authenticating (`artdag:user_tokens:{username}`)
-- `/auth/revoke-user` revokes all tokens for a username
-- Revoked token hashes stored in Redis with 30-day expiry
-- Every authenticated request checks the revocation list
-- Revoked tokens are immediately rejected
-
-### Security
-
-- **Scoped tokens**: Tokens are bound to a specific L1. A stolen token can't be used on other L1 servers.
-- **L2 verification**: L1 verifies every token with L2, which checks its revocation table.
-- **No shared secrets**: L1 doesn't need L2's JWT secret.
-
-## API
 
 Interactive docs: http://localhost:8100/docs
 
-### Endpoints
+### Runs
 
 | Method | Path | Description |
 |--------|------|-------------|
-| GET | `/` | Server info |
 | POST | `/runs` | Start a rendering run |
-| GET | `/runs` | List all runs |
+| GET | `/runs` | List all runs (paginated) |
 | GET | `/runs/{run_id}` | Get run status |
 | DELETE | `/runs/{run_id}` | Delete a run |
-| GET | `/cache` | List cached content hashes |
+| GET | `/api/run/{run_id}` | Get run as JSON |
-| GET | `/cache/{hash}` | Download cached content |
+| GET | `/api/run/{run_id}/plan` | Get execution plan JSON |
-| DELETE | `/cache/{hash}` | Delete cached content |
+| GET | `/api/run/{run_id}/analysis` | Get analysis data JSON |
-| POST | `/cache/import?path=` | Import local file to cache |
-| POST | `/cache/upload` | Upload file to cache |
-| GET | `/assets` | List known assets |
-| POST | `/configs/upload` | Upload a config YAML |
-| GET | `/configs` | List configs |
-| GET | `/configs/{id}` | Get config details |
-| DELETE | `/configs/{id}` | Delete a config |
-| POST | `/configs/{id}/run` | Run a config |
 
-### Configs
+### Recipes
 
-Configs are YAML files that define reusable DAG pipelines. They can have:
+| Method | Path | Description |
-- **Fixed inputs**: Assets with pre-defined content hashes
+|--------|------|-------------|
-- **Variable inputs**: Placeholders filled at run time
+| POST | `/recipes/upload` | Upload recipe YAML |
+| GET | `/recipes` | List recipes (paginated) |
+| GET | `/recipes/{recipe_id}` | Get recipe details |
+| DELETE | `/recipes/{recipe_id}` | Delete recipe |
+| POST | `/recipes/{recipe_id}/run` | Execute recipe |
+
+### Cache
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/cache/{hash}` | Get cached content (with preview) |
+| GET | `/cache/{hash}/raw` | Download raw content |
+| GET | `/cache/{hash}/mp4` | Get MP4 video |
+| GET | `/cache/{hash}/meta` | Get content metadata |
+| PATCH | `/cache/{hash}/meta` | Update metadata |
+| POST | `/cache/{hash}/publish` | Publish to L2 |
+| DELETE | `/cache/{hash}` | Delete from cache |
+| POST | `/cache/import?path=` | Import local file |
+| POST | `/cache/upload` | Upload file |
+| GET | `/media` | Browse media gallery |
+
+### IPFS
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/ipfs/{cid}` | Redirect to IPFS gateway |
+| GET | `/ipfs/{cid}/raw` | Fetch raw content from IPFS |
+
+### Storage Providers
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/storage` | List storage providers |
+| POST | `/storage` | Add provider (form) |
+| POST | `/storage/add` | Add provider (JSON) |
+| GET | `/storage/{id}` | Get provider details |
+| PATCH | `/storage/{id}` | Update provider |
+| DELETE | `/storage/{id}` | Delete provider |
+| POST | `/storage/{id}/test` | Test connection |
+| GET | `/storage/type/{type}` | Get form for provider type |
+
+### 3-Phase API
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | `/api/plan` | Generate execution plan |
+| POST | `/api/execute` | Execute a plan |
+| POST | `/api/run-recipe` | Full pipeline (analyze+plan+execute) |
+
+### Authentication
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/auth` | Receive auth token from L2 |
+| GET | `/logout` | Log out |
+| POST | `/auth/revoke` | Revoke a specific token |
+| POST | `/auth/revoke-user` | Revoke all user tokens |
+
+## 3-Phase Execution
+
+Recipes are executed in three phases:
+
+### Phase 1: Analyze
+Extract features from input files:
+- **Audio/Video**: Tempo, beat times, energy levels
+- Results cached by content hash
+
+### Phase 2: Plan
+Generate an execution plan:
+- Parse recipe YAML
+- Resolve dependencies between steps
+- Compute cache IDs for each step
+- Skip already-cached steps
+
+### Phase 3: Execute
+Run the plan level by level:
+- Steps at each level run in parallel
+- Results cached with content-addressable hashes
+- Progress tracked in Redis
+
+## Recipe Format
+
+Recipes define reusable DAG pipelines:
 
-Example config:
 ```yaml
-name: my-effect
+name: beat-sync
 version: "1.0"
-description: "Apply effect to user image"
+description: "Synchronize video to audio beats"
 
-registry:
+inputs:
-  effects:
+  video:
-    dog:
+    type: video
-      hash: "abc123..."
+    description: "Source video"
+  audio:
+    type: audio
+    description: "Audio track"
 
-dag:
+steps:
-  nodes:
+  - id: analyze_audio
-    - id: user_image
+    type: ANALYZE
-      type: SOURCE
+    inputs: [audio]
-      config:
+    config:
-        input: true        # Variable input
+      features: [beats, energy]
-        name: "input_image"
 
-    - id: apply_dog
+  - id: sync_video
-      type: EFFECT
+    type: BEAT_SYNC
-      config:
+    inputs: [video, analyze_audio]
-        effect: dog
+    config:
-      inputs:
+      mode: stretch
-        - user_image
 
-  output: apply_dog
+output: sync_video
 ```
 
-### Start a run
-
-```bash
-curl -X POST http://localhost:8100/runs \
-  -H "Content-Type: application/json" \
-  -d '{"recipe": "dog", "inputs": ["33268b6e..."], "output_name": "my-output"}'
-```
-
-### Check run status
-
-```bash
-curl http://localhost:8100/runs/{run_id}
-```
-
-### Delete a run
-
-```bash
-curl -X DELETE http://localhost:8100/runs/{run_id} \
-  -H "Authorization: Bearer <token>"
-```
-
-Note: Failed runs can always be deleted. Completed runs can only be deleted if their outputs haven't been published to L2.
-
-### Delete cached content
-
-```bash
-curl -X DELETE http://localhost:8100/cache/{hash} \
-  -H "Authorization: Bearer <token>"
-```
-
-Note: Items that are inputs/outputs of runs, or published to L2, cannot be deleted.
-
 ## Storage
 
-- **Cache**: `~/.artdag/cache/` (content-addressed files)
+### Local Cache
-- **Runs**: Redis db 5, keys `artdag:run:*` (persists across restarts)
+- Location: `~/.artdag/cache/` (or `CACHE_DIR`)
+- Content-addressed by SHA3-256 hash
+- Subdirectories: `plans/`, `analysis/`
+
+### Redis
+- Database 5 (configurable via `REDIS_URL`)
+- Keys:
+  - `artdag:run:*` - Run state
+  - `artdag:recipe:*` - Recipe definitions
+  - `artdag:revoked:*` - Token revocation
+  - `artdag:user_tokens:*` - User token tracking
+
+### PostgreSQL
+- Content metadata
+- Storage provider configurations
+- Provenance records
+
+## Authentication
+
+L1 servers authenticate via L2 (ActivityPub registry). No shared secrets required.
+
+### Flow
+1. User clicks "Attach" on L2's Renderers page
+2. L2 creates a scoped token bound to this L1
+3. User redirected to L1's `/auth?auth_token=...`
+4. L1 calls L2's `/auth/verify` to validate
+5. L1 sets local cookie and records token
+
+### Token Revocation
+- Tokens tracked per-user in Redis
+- L2 calls `/auth/revoke-user` on logout
+- Revoked hashes stored with 30-day expiry
+- Every request checks revocation list
 
 ## CLI Usage
 
 ```bash
-# Render cat through dog effect
+# Quick render (effect mode)
 python render.py dog cat --sync
 
-# Render cat through identity effect
+# Submit async
-python render.py identity cat --sync
-
-# Submit async (don't wait)
 python render.py dog cat
+
+# Run a recipe
+curl -X POST http://localhost:8100/recipes/beat-sync/run \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer <token>" \
+  -d '{"inputs": {"video": "abc123...", "audio": "def456..."}}'
 ```
 
 ## Architecture
 
 ```
-server.py (L1 Server - FastAPI)
+L1 Server (FastAPI)
     │
-    ├── POST /runs → Submit job
+    ├── Web UI (Jinja2 + HTMX + Tailwind)
-    │       │
-    │       ▼
-    │   celery_app.py (Celery broker)
-    │       │
-    │       ▼
-    │   tasks.py (render_effect task)
-    │       │
-    │       ├── artdag (GitHub) - DAG execution
-    │       └── artdag-effects (rose-ash) - Effects
-    │               │
-    │               ▼
-    │           Output + Provenance
     │
-    └── GET /cache/{hash} → Retrieve output
+    ├── POST /runs → Celery tasks
+    │       │
+    │       └── celery_app.py
+    │           ├── tasks/analyze.py    (Phase 1)
+    │           ├── tasks/execute.py    (Phase 3 steps)
+    │           └── tasks/orchestrate.py (Full pipeline)
+    │
+    ├── cache_manager.py
+    │       │
+    │       ├── Local filesystem (CACHE_DIR)
+    │       ├── IPFS (ipfs_client.py)
+    │       └── S3/Storage providers
+    │
+    └── database.py (PostgreSQL metadata)
 ```
 
 ## Provenance

server.py

@@ -6316,6 +6316,190 @@ async def download_client():
     )
 
 
+# ============================================================================
+# Documentation Routes
+# ============================================================================
+
+# Documentation paths
+DOCS_DIR = Path(__file__).parent
+COMMON_DOCS_DIR = Path(__file__).parent.parent / "common"
+
+DOCS_MAP = {
+    "l1": DOCS_DIR / "README.md",
+    "common": COMMON_DOCS_DIR / "README.md",
+}
+
+
+def render_markdown(content: str) -> str:
+    """Convert markdown to HTML with basic styling."""
+    import re
+
+    # Escape HTML first
+    content = content.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
+
+    # Code blocks (``` ... ```)
+    def code_block_replace(match):
+        lang = match.group(1) or ""
+        code = match.group(2)
+        return f'<pre class="bg-gray-800 p-4 rounded-lg overflow-x-auto text-sm"><code class="language-{lang}">{code}</code></pre>'
+    content = re.sub(r'```(\w*)\n(.*?)```', code_block_replace, content, flags=re.DOTALL)
+
+    # Inline code
+    content = re.sub(r'`([^`]+)`', r'<code class="bg-gray-700 px-1 rounded text-sm">\1</code>', content)
+
+    # Headers
+    content = re.sub(r'^### (.+)$', r'<h3 class="text-lg font-semibold text-white mt-6 mb-2">\1</h3>', content, flags=re.MULTILINE)
+    content = re.sub(r'^## (.+)$', r'<h2 class="text-xl font-bold text-white mt-8 mb-3 border-b border-gray-700 pb-2">\1</h2>', content, flags=re.MULTILINE)
+    content = re.sub(r'^# (.+)$', r'<h1 class="text-2xl font-bold text-white mb-4">\1</h1>', content, flags=re.MULTILINE)
+
+    # Bold and italic
+    content = re.sub(r'\*\*([^*]+)\*\*', r'<strong class="font-semibold">\1</strong>', content)
+    content = re.sub(r'\*([^*]+)\*', r'<em>\1</em>', content)
+
+    # Links
+    content = re.sub(r'\[([^\]]+)\]\(([^)]+)\)', r'<a href="\2" class="text-blue-400 hover:underline">\1</a>', content)
+
+    # Tables
+    def table_replace(match):
+        lines = match.group(0).strip().split('\n')
+        if len(lines) < 2:
+            return match.group(0)
+
+        header = lines[0]
+        rows = lines[2:] if len(lines) > 2 else []
+
+        header_cells = [cell.strip() for cell in header.split('|')[1:-1]]
+        header_html = ''.join(f'<th class="px-4 py-2 text-left border-b border-gray-600">{cell}</th>' for cell in header_cells)
+
+        rows_html = ''
+        for row in rows:
+            cells = [cell.strip() for cell in row.split('|')[1:-1]]
+            cells_html = ''.join(f'<td class="px-4 py-2 border-b border-gray-700">{cell}</td>' for cell in cells)
+            rows_html += f'<tr class="hover:bg-gray-700">{cells_html}</tr>'
+
+        return f'<table class="w-full text-sm mb-4"><thead><tr class="bg-gray-700">{header_html}</tr></thead><tbody>{rows_html}</tbody></table>'
+
+    content = re.sub(r'(\|[^\n]+\|\n)+', table_replace, content)
+
+    # Bullet points
+    content = re.sub(r'^- (.+)$', r'<li class="ml-4 list-disc">\1</li>', content, flags=re.MULTILINE)
+    content = re.sub(r'(<li[^>]*>.*</li>\n?)+', r'<ul class="mb-4">\g<0></ul>', content)
+
+    # Paragraphs (lines not starting with < or whitespace)
+    lines = content.split('\n')
+    result = []
+    in_paragraph = False
+    for line in lines:
+        stripped = line.strip()
+        if not stripped:
+            if in_paragraph:
+                result.append('</p>')
+                in_paragraph = False
+            result.append('')
+        elif stripped.startswith('<'):
+            if in_paragraph:
+                result.append('</p>')
+                in_paragraph = False
+            result.append(line)
+        else:
+            if not in_paragraph:
+                result.append('<p class="mb-4 text-gray-300">')
+                in_paragraph = True
+            result.append(line)
+    if in_paragraph:
+        result.append('</p>')
+    content = '\n'.join(result)
+
+    return content
+
+
+@app.get("/docs", response_class=HTMLResponse)
+async def docs_index(request: Request):
+    """Documentation index page."""
+    user = await get_optional_user(request)
+
+    html = f"""<!DOCTYPE html>
+<html class="dark">
+<head>
+    <title>Documentation - Art DAG L1</title>
+    <script src="https://cdn.tailwindcss.com"></script>
+    <script>tailwind.config = {{ darkMode: 'class' }}</script>
+</head>
+<body class="bg-gray-900 text-gray-100 min-h-screen">
+    <nav class="bg-gray-800 border-b border-gray-700 px-6 py-4">
+        <div class="flex items-center justify-between">
+            <a href="/" class="text-xl font-bold text-white">Art DAG L1</a>
+            <div class="flex items-center gap-4">
+                <a href="/runs" class="text-gray-300 hover:text-white">Runs</a>
+                <a href="/recipes" class="text-gray-300 hover:text-white">Recipes</a>
+                <a href="/media" class="text-gray-300 hover:text-white">Media</a>
+                <a href="/docs" class="text-white font-semibold">Docs</a>
+            </div>
+        </div>
+    </nav>
+    <main class="max-w-4xl mx-auto p-8">
+        <h1 class="text-3xl font-bold mb-8">Documentation</h1>
+        <div class="grid gap-4">
+            <a href="/docs/l1" class="block p-6 bg-gray-800 rounded-lg hover:bg-gray-700 transition">
+                <h2 class="text-xl font-semibold text-white mb-2">L1 Server (Celery)</h2>
+                <p class="text-gray-400">Distributed rendering server with Celery workers, IPFS integration, and 3-phase execution.</p>
+            </a>
+            <a href="/docs/common" class="block p-6 bg-gray-800 rounded-lg hover:bg-gray-700 transition">
+                <h2 class="text-xl font-semibold text-white mb-2">Common Library</h2>
+                <p class="text-gray-400">Shared components: Jinja2 templates, middleware, content negotiation, and utilities.</p>
+            </a>
+        </div>
+    </main>
+</body>
+</html>"""
+    return HTMLResponse(html)
+
+
+@app.get("/docs/{doc_name}", response_class=HTMLResponse)
+async def docs_page(doc_name: str, request: Request):
+    """Render a markdown documentation file as HTML."""
+    if doc_name not in DOCS_MAP:
+        raise HTTPException(404, f"Documentation '{doc_name}' not found")
+
+    doc_path = DOCS_MAP[doc_name]
+    if not doc_path.exists():
+        raise HTTPException(404, f"Documentation file not found: {doc_path}")
+
+    content = doc_path.read_text()
+    html_content = render_markdown(content)
+
+    html = f"""<!DOCTYPE html>
+<html class="dark">
+<head>
+    <title>{doc_name.upper()} - Art DAG Documentation</title>
+    <script src="https://cdn.tailwindcss.com"></script>
+    <script>tailwind.config = {{ darkMode: 'class' }}</script>
+</head>
+<body class="bg-gray-900 text-gray-100 min-h-screen">
+    <nav class="bg-gray-800 border-b border-gray-700 px-6 py-4">
+        <div class="flex items-center justify-between">
+            <a href="/" class="text-xl font-bold text-white">Art DAG L1</a>
+            <div class="flex items-center gap-4">
+                <a href="/runs" class="text-gray-300 hover:text-white">Runs</a>
+                <a href="/recipes" class="text-gray-300 hover:text-white">Recipes</a>
+                <a href="/media" class="text-gray-300 hover:text-white">Media</a>
+                <a href="/docs" class="text-white font-semibold">Docs</a>
+            </div>
+        </div>
+    </nav>
+    <main class="max-w-4xl mx-auto p-8">
+        <div class="mb-4">
+            <a href="/docs" class="text-blue-400 hover:underline">&larr; Back to Documentation</a>
+        </div>
+        <article class="prose prose-invert max-w-none">
+            {html_content}
+        </article>
+    </main>
+</body>
+</html>"""
+    return HTMLResponse(html)
+
+
 # ============================================================================
 # 3-Phase Execution API (Analyze → Plan → Execute)
 # ============================================================================