Art Celery

L1 rendering server for the Art DAG system. Manages distributed rendering jobs via Celery workers.

Dependencies

• artdag (GitHub): Core DAG execution engine
• artdag-effects (rose-ash): Effect implementations
• Redis: Message broker, result backend, and run persistence

Setup

# Install Redis
sudo apt install redis-server

# Install Python dependencies
pip install -r requirements.txt

# Start a worker
celery -A celery_app worker --loglevel=info

# Start the L1 server
python server.py

L1 Server API

Interactive docs: http://localhost:8100/docs

Endpoints

Method	Path	Description
GET	/	Server info
POST	/runs	Start a rendering run
GET	/runs	List all runs
GET	/runs/{run_id}	Get run status
DELETE	/runs/{run_id}	Delete a run
GET	/cache	List cached content hashes
GET	/cache/{hash}	Download cached content
DELETE	/cache/{hash}	Delete cached content
POST	/cache/import?path=	Import local file to cache
POST	/cache/upload	Upload file to cache
GET	/assets	List known assets
POST	/configs/upload	Upload a config YAML
GET	/configs	List configs
GET	/configs/{id}	Get config details
DELETE	/configs/{id}	Delete a config
POST	/configs/{id}/run	Run a config

Configs

Configs are YAML files that define reusable DAG pipelines. They can have:

• Fixed inputs: Assets with pre-defined content hashes
• Variable inputs: Placeholders filled at run time

Example config:

name: my-effect
version: "1.0"
description: "Apply effect to user image"

registry:
  effects:
    dog:
      hash: "abc123..."

dag:
  nodes:
    - id: user_image
      type: SOURCE
      config:
        input: true        # Variable input
        name: "input_image"

    - id: apply_dog
      type: EFFECT
      config:
        effect: dog
      inputs:
        - user_image

  output: apply_dog

Start a run

curl -X POST http://localhost:8100/runs \
  -H "Content-Type: application/json" \
  -d '{"recipe": "dog", "inputs": ["33268b6e..."], "output_name": "my-output"}'

Check run status

curl http://localhost:8100/runs/{run_id}

Delete a run

curl -X DELETE http://localhost:8100/runs/{run_id} \
  -H "Authorization: Bearer <token>"

Note: Failed runs can always be deleted. Completed runs can only be deleted if their outputs haven't been published to L2.

Delete cached content

curl -X DELETE http://localhost:8100/cache/{hash} \
  -H "Authorization: Bearer <token>"

Note: Items that are inputs/outputs of runs, or published to L2, cannot be deleted.

Storage

• Cache: ~/.artdag/cache/ (content-addressed files)
• Runs: Redis db 5, keys artdag:run:* (persists across restarts)

CLI Usage

# Render cat through dog effect
python render.py dog cat --sync

# Render cat through identity effect
python render.py identity cat --sync

# Submit async (don't wait)
python render.py dog cat

Architecture

server.py (L1 Server - FastAPI)
    │
    ├── POST /runs → Submit job
    │       │
    │       ▼
    │   celery_app.py (Celery broker)
    │       │
    │       ▼
    │   tasks.py (render_effect task)
    │       │
    │       ├── artdag (GitHub) - DAG execution
    │       └── artdag-effects (rose-ash) - Effects
    │               │
    │               ▼
    │           Output + Provenance
    │
    └── GET /cache/{hash} → Retrieve output

Provenance

Every render produces a provenance record:

{
  "task_id": "celery-task-uuid",
  "rendered_at": "2026-01-07T...",
  "rendered_by": "@giles@artdag.rose-ash.com",
  "output": {"name": "...", "content_hash": "..."},
  "inputs": [...],
  "effects": [...],
  "infrastructure": {
    "software": {"name": "infra:artdag", "content_hash": "..."},
    "hardware": {"name": "infra:giles-hp", "content_hash": "..."}
  }
}