rose-ash/plans/agent-briefings/artdag-loop.md

artdag-on-sx loop agent (single agent, queue-driven)

Role: iterates plans/artdag-on-sx.md forever. Content-addressed dataflow DAG engine — the SX heart of art-dag's Analyze→Plan→Execute pipeline: dependency analysis, scheduling, content-addressed memoization, incremental recompute, composable s-expression effects. Media kernels (JAX) and storage (IPFS) stay opaque — this is the engine, not the pixels. One feature per commit.

description: artdag-on-sx queue loop
subagent_type: general-purpose
run_in_background: true
isolation: worktree

Prerequisites — check before starting

1. lib/datalog/ present (Analyze, Phase 2) — public API lib/datalog/datalog.sx.
2. lib/persist/ present (memo cache / result store, Phase 4) — public API.

Pre-flight:

ls /root/rose-ash/lib/datalog/datalog.sx
ls /root/rose-ash/lib/persist/ 2>/dev/null

Phase 1 (DAG model + content addressing) needs neither and can start immediately. Datalog is needed at Phase 2, persist at Phase 4 — if a dependency is missing when you reach its phase, record a Blockers entry and work earlier phases / other boxes.

Prompt

You are the sole background agent working /root/rose-ash/plans/artdag-on-sx.md, in an isolated git worktree on branch loops/artdag, forever, one commit per feature. Push to origin/loops/artdag after every commit. Never touch main or architecture.

Restart baseline — check before iterating

1. Read plans/artdag-on-sx.md — Roadmap + Progress log + Blockers.
2. Run the pre-flight; record gaps in Blockers (per-phase, not all-or-nothing).
3. ls lib/artdag/ — pick up from the most advanced file. No dir → Phase 1.
4. If lib/artdag/tests/*.sx exist, run them via bash lib/artdag/conformance.sh (SX_SERVER=/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe). Green before new work.
5. Read the existing Python engine for design lineage — repo top-level artdag/core/ (DAG, effects, analysis, planning) and artdag/l1/sexp_effects/. Understand the 3-phase model + the effect language. Do not import or port its code — this is a fresh SX implementation.

The queue

Phase order per plans/artdag-on-sx.md:

• Phase 1 — DAG model + content addressing (node id = digest of (op, sorted input-ids, params); identical subgraphs share an id); validate + topo order. No substrate deps — start here.
• Phase 2 — Analyze on Datalog: deps/dependents/reachable + dirty-closure propagation
• Phase 3 — Plan: topological batches + parallelism cap; incremental (dirty-only)
• Phase 4 — Execute: effect interpreter (perform per node) + content-addressed memo on lib/persist/ + incremental recompute (only the dirty closure)
• Phase 5 — Optimize: DCE + CSE (free from content ids) + adjacent-op fusion; result-preserving (optional: rule-based via maude-on-sx)
• Phase 6 — Federation: shared content-addressed cache across instances (L2-style), trust-gated result import, invalidation

Within a phase, pick the checkbox with the best tests-per-effort ratio. Every iteration: implement → test → commit → tick [ ] → Progress log → push → next.

artdag-specific gotchas

• Content addressing is the whole game. A node's id must be a deterministic structural digest of its spec (op + sorted input ids + params). Get this stable and order-insensitive on inputs where the op is commutative, or sharing/caching silently misses. Never let a clock, randomness, or insertion order into the id.
• Media ops are opaque. In tests a node op is a pure SX function over inputs; never compute real media. Production kernels (JAX) and storage (IPFS) are injected adapters behind an interface — depend on the interface, not the impl.
• Analyze is the acl/relations reachability shape — recursive Datalog. Read lib/acl/engine.sx / lib/relations/ for the worked shape; re-derive, don't import.
• Incremental = recompute only the dirty closure. The Phase 4 keystone test asserts that re-running after one leaf change recomputes only the transitive dependents and cache-hits everything else (count the recomputes). If it recomputes more, the dirty closure (Phase 2) or the memo key (Phase 4) is wrong.
• The memo key is the content-id, not the node identity. Two different DAGs that contain the same subgraph share cache slots — that's the point. Don't key on position or DAG-local node index.
• Effects are composable s-expressions (art-dag's sexp_effects) — Phase 5 optimization rewrites the effect DAG; keep ops as data you can fuse/eliminate, not opaque closures, so the optimizer can reason about them.
• persist stores results by reference where large — for big blobs, persist holds the ref/CID and the bytes live in the content-addressed store (see plans/persist-on-sx.md); the engine caches the result handle, not megabytes.

Ground rules (hard)

• Scope: only lib/artdag/** and plans/artdag-on-sx.md. Do not edit spec/, hosts/, shared/, lib/datalog/** / lib/persist/** (read-only — import public APIs), other lib/<lang>/, or the top-level artdag/ Python engine (read-only, design lineage only).
• Don't modify the substrates. Datalog/persist issues → failing test + Blockers.
• NEVER call sx_build (600s watchdog). Broken sx_server.exe → Blockers, stop.
• SX files: sx-tree MCP tools ONLY; sx_validate after every edit; file: not path:. Never Edit/Read/Write on .sx.
• Worktree: commit, then push origin/loops/artdag. Never main/architecture.
• Commits: one feature per commit (artdag: dirty-closure propagation + 6 tests).
• Plan file: Progress log (newest first) + tick boxes every commit.
• Blocked 2 iterations on one issue → Blockers entry, move on.

General gotchas (all loops)

• SX do = R7RS iteration; use begin for multi-expr sequences.
• cond/when/let clauses evaluate only the last expr — wrap multiples in begin.
• let is parallel — nest lets when one binding references an earlier one.
• env-bind! creates a binding; env-set! mutates an existing one.
• Namespace-prefix guest helpers (artdag/…) — short/host-colliding names get shadowed.
• Shell heredoc || gets eaten — escape or use case.

Style

• No comments in .sx unless non-obvious. No new planning docs — update the plan.
• Short, factual commit messages. One feature per iteration. Commit. Log. Push. Next.

Go. Read the plan and the existing artdag/ engine for design lineage, then start Phase 1 (DAG model + content addressing) — it needs no substrate dependency. Find the first unchecked [ ], implement it.