Refactor storage: remove Redis duplication, use proper data tiers

- Recipes: Now content-addressed only (cache + IPFS), removed Redis storage
- Runs: Completed runs stored in PostgreSQL, Redis only for task_id mapping
- Add list_runs_by_actor() to database.py for paginated run queries
- Add list_by_type() to cache_manager for filtering by node_type
- Fix upload endpoint to return size and filename fields
- Fix recipe run endpoint with proper DAG input binding
- Fix get_run_service() dependency to pass database module

Storage architecture:
- Redis: Ephemeral only (sessions, task mappings with TTL)
- PostgreSQL: Permanent records (completed runs, metadata)
- Cache: Content-addressed files (recipes, media, outputs)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
giles
2026-01-11 14:05:31 +00:00
parent 8591faf0fc
commit 854396680f
8 changed files with 965 additions and 264 deletions

View File

@@ -1132,6 +1132,34 @@ async def get_run_by_output(output_hash: str) -> Optional[dict]:
return None
async def list_runs_by_actor(actor_id: str, offset: int = 0, limit: int = 20) -> List[dict]:
"""List completed runs for a user, ordered by creation time (newest first)."""
async with pool.acquire() as conn:
rows = await conn.fetch(
"""
SELECT run_id, output_hash, ipfs_cid, provenance_cid, recipe, inputs, actor_id, created_at
FROM run_cache
WHERE actor_id = $1
ORDER BY created_at DESC
LIMIT $2 OFFSET $3
""",
actor_id, limit, offset
)
return [
{
"run_id": row["run_id"],
"output_hash": row["output_hash"],
"ipfs_cid": row["ipfs_cid"],
"provenance_cid": row["provenance_cid"],
"recipe": row["recipe"],
"inputs": row["inputs"],
"actor_id": row["actor_id"],
"created_at": row["created_at"].isoformat() if row["created_at"] else None,
}
for row in rows
]
# ============ Storage Backends ============
async def get_user_storage(actor_id: str) -> List[dict]: