rose-ash/plans/agent-briefings/artdag-loop.md

# artdag-on-sx loop agent (single agent, queue-driven)

Role: iterates `plans/artdag-on-sx.md` forever. **Content-addressed dataflow DAG
engine** — the SX heart of art-dag's Analyze→Plan→Execute pipeline: dependency
analysis, scheduling, content-addressed memoization, incremental recompute,
composable s-expression effects. Media kernels (JAX) and storage (IPFS) stay opaque
— this is the *engine*, not the pixels. One feature per commit.

```
description: artdag-on-sx queue loop
subagent_type: general-purpose
run_in_background: true
isolation: worktree
```

## Prerequisites — check before starting

1. **`lib/datalog/` present** (Analyze, Phase 2) — public API `lib/datalog/datalog.sx`.
2. **`lib/persist/` present** (memo cache / result store, Phase 4) — public API.

**Pre-flight:**
```
ls /root/rose-ash/lib/datalog/datalog.sx
ls /root/rose-ash/lib/persist/ 2>/dev/null
```
Phase 1 (DAG model + content addressing) needs neither and can start immediately.
Datalog is needed at Phase 2, persist at Phase 4 — if a dependency is missing when
you reach its phase, record a Blockers entry and work earlier phases / other boxes.

## Prompt

You are the sole background agent working `/root/rose-ash/plans/artdag-on-sx.md`,
in an isolated git worktree on branch `loops/artdag`, forever, one commit per
feature. Push to `origin/loops/artdag` after every commit. Never touch `main` or
`architecture`.

## Restart baseline — check before iterating

1. Read `plans/artdag-on-sx.md` — Roadmap + Progress log + Blockers.
2. Run the pre-flight; record gaps in Blockers (per-phase, not all-or-nothing).
3. `ls lib/artdag/` — pick up from the most advanced file. No dir → Phase 1.
4. If `lib/artdag/tests/*.sx` exist, run them via `bash lib/artdag/conformance.sh`
   (`SX_SERVER=/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe`). Green
   before new work.
5. **Read the existing Python engine for design lineage** — repo top-level
   `artdag/core/` (DAG, effects, analysis, planning) and `artdag/l1/sexp_effects/`.
   Understand the 3-phase model + the effect language. **Do not import or port its
   code** — this is a fresh SX implementation.

## The queue

Phase order per `plans/artdag-on-sx.md`:

- **Phase 1** — DAG model + **content addressing** (node id = digest of
  `(op, sorted input-ids, params)`; identical subgraphs share an id); validate +
  topo order. *No substrate deps — start here.*
- **Phase 2** — Analyze on **Datalog**: deps/dependents/reachable + **dirty-closure**
  propagation
- **Phase 3** — Plan: topological **batches** + parallelism cap; incremental (dirty-only)
- **Phase 4** — Execute: effect interpreter (`perform` per node) + **content-addressed
  memo** on `lib/persist/` + **incremental recompute** (only the dirty closure)
- **Phase 5** — Optimize: DCE + CSE (free from content ids) + adjacent-op fusion;
  result-preserving (optional: rule-based via `maude-on-sx`)
- **Phase 6** — Federation: shared content-addressed cache across instances
  (L2-style), trust-gated result import, invalidation

Within a phase, pick the checkbox with the best tests-per-effort ratio.
Every iteration: implement → test → commit → tick `[ ]` → Progress log → push → next.

## artdag-specific gotchas

- **Content addressing is the whole game.** A node's id must be a *deterministic
  structural digest* of its spec (op + sorted input ids + params). Get this stable
  and order-insensitive on inputs where the op is commutative, or sharing/caching
  silently misses. Never let a clock, randomness, or insertion order into the id.
- **Media ops are opaque.** In tests a node op is a pure SX function over inputs;
  never compute real media. Production kernels (JAX) and storage (IPFS) are injected
  adapters behind an interface — depend on the interface, not the impl.
- **Analyze is the acl/relations reachability shape** — recursive Datalog. Read
  `lib/acl/engine.sx` / `lib/relations/` for the worked shape; re-derive, don't import.
- **Incremental = recompute only the dirty closure.** The Phase 4 keystone test
  asserts that re-running after one leaf change recomputes *only* the transitive
  dependents and cache-hits everything else (count the recomputes). If it recomputes
  more, the dirty closure (Phase 2) or the memo key (Phase 4) is wrong.
- **The memo key is the content-id, not the node identity.** Two different DAGs that
  contain the same subgraph share cache slots — that's the point. Don't key on
  position or DAG-local node index.
- **Effects are composable s-expressions** (art-dag's `sexp_effects`) — Phase 5
  optimization rewrites the effect DAG; keep ops as data you can fuse/eliminate, not
  opaque closures, so the optimizer can reason about them.
- **persist stores results by reference where large** — for big blobs, persist holds
  the ref/CID and the bytes live in the content-addressed store (see
  `plans/persist-on-sx.md`); the engine caches the *result handle*, not megabytes.

## Ground rules (hard)

- **Scope:** only `lib/artdag/**` and `plans/artdag-on-sx.md`. Do **not** edit
  `spec/`, `hosts/`, `shared/`, `lib/datalog/**` / `lib/persist/**` (read-only —
  import public APIs), other `lib/<lang>/`, or the top-level `artdag/` Python engine
  (read-only, design lineage only).
- **Don't modify the substrates.** Datalog/persist issues → failing test + Blockers.
- **NEVER call `sx_build`** (600s watchdog). Broken `sx_server.exe` → Blockers, stop.
- **SX files:** `sx-tree` MCP tools ONLY; `sx_validate` after every edit; `file:` not
  `path:`. Never `Edit`/`Read`/`Write` on `.sx`.
- **Worktree:** commit, then push `origin/loops/artdag`. Never `main`/`architecture`.
- **Commits:** one feature per commit (`artdag: dirty-closure propagation + 6 tests`).
- **Plan file:** Progress log (newest first) + tick boxes every commit.
- **Blocked 2 iterations on one issue → Blockers entry, move on.**

## General gotchas (all loops)

- SX `do` = R7RS iteration; use `begin` for multi-expr sequences.
- `cond`/`when`/`let` clauses evaluate only the last expr — wrap multiples in `begin`.
- `let` is parallel — nest `let`s when one binding references an earlier one.
- `env-bind!` creates a binding; `env-set!` mutates an existing one.
- Namespace-prefix guest helpers (`artdag/…`) — short/host-colliding names get shadowed.
- Shell heredoc `||` gets eaten — escape or use `case`.

## Style

- No comments in `.sx` unless non-obvious. No new planning docs — update the plan.
- Short, factual commit messages. One feature per iteration. Commit. Log. Push. Next.

Go. Read the plan and the existing `artdag/` engine for design lineage, then start
Phase 1 (DAG model + content addressing) — it needs no substrate dependency. Find
the first unchecked `[ ]`, implement it.