feat: distributed rendering with Celery
- celery_app.py: Celery configuration with Redis broker - tasks.py: render_effect task with full provenance tracking - render.py: CLI for submitting render jobs - Successfully renders cat → dog with provenance chain 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
72
README.md
Normal file
72
README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Art Celery
|
||||
|
||||
Distributed rendering for the Art DAG system using Celery.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **artdag** (GitHub): Core DAG execution engine
|
||||
- **artdag-effects** (rose-ash): Effect implementations
|
||||
- **Redis**: Message broker and result backend
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
# Install Redis
|
||||
sudo apt install redis-server
|
||||
|
||||
# Install Python dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Start a worker
|
||||
celery -A celery_app worker --loglevel=info
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
# Render cat through dog effect
|
||||
python render.py dog cat --sync
|
||||
|
||||
# Render cat through identity effect
|
||||
python render.py identity cat --sync
|
||||
|
||||
# Submit async (don't wait)
|
||||
python render.py dog cat
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
render.py (CLI)
|
||||
│
|
||||
▼
|
||||
celery_app.py (Celery broker)
|
||||
│
|
||||
▼
|
||||
tasks.py (render_effect task)
|
||||
│
|
||||
├── artdag (GitHub) - DAG execution
|
||||
└── artdag-effects (rose-ash) - Effect implementations
|
||||
│
|
||||
▼
|
||||
Provenance JSON + Output file
|
||||
```
|
||||
|
||||
## Provenance
|
||||
|
||||
Every render produces a provenance record:
|
||||
|
||||
```json
|
||||
{
|
||||
"task_id": "celery-task-uuid",
|
||||
"rendered_at": "2026-01-07T...",
|
||||
"rendered_by": "@giles@artdag.rose-ash.com",
|
||||
"output": {"name": "...", "content_hash": "..."},
|
||||
"inputs": [...],
|
||||
"effects": [...],
|
||||
"infrastructure": {
|
||||
"software": {"name": "infra:artdag", "content_hash": "..."},
|
||||
"hardware": {"name": "infra:giles-hp", "content_hash": "..."}
|
||||
}
|
||||
}
|
||||
```
|
||||
Reference in New Issue
Block a user