rose-ash/plans/agent-briefings/datalog-loop.md

datalog-on-sx loop agent (single agent, queue-driven)

Role: iterates plans/datalog-on-sx.md forever. Bottom-up Datalog with stratified negation, aggregation, magic sets, body arithmetic. Companion to the Prolog implementation; shares unification, owns its own evaluator (fixpoint, not DFS). One feature per commit.

description: datalog-on-sx queue loop
subagent_type: general-purpose
run_in_background: true
isolation: worktree

Prompt

You are the sole background agent working /root/rose-ash/plans/datalog-on-sx.md. You run in an isolated git worktree on branch loops/datalog. You work the plan's roadmap forever, one commit per feature. Push to origin/loops/datalog after every commit.

Restart baseline — check before iterating

1. Read plans/datalog-on-sx.md — Roadmap + Progress log tell you where you are. Phases 1–10, all [ ] until something ships.
2. ls lib/datalog/ — if the directory does not exist, you are at Phase 1. Create it on the first code commit.
3. If lib/datalog/tests/*.sx exist, run them via the epoch protocol against sx_server.exe. They must be green before new work.
4. If lib/datalog/scoreboard.json exists (Phase 3 onwards), that is your starting number — read it each iteration and attack the worst failure mode you can plausibly fix in < a day.
5. Check ## Blockers in the plan — items there are not for you to fix, only to work around or wait on.

The queue

Work in phase order per plans/datalog-on-sx.md:

• Phase 1 — tokenizer + parser (facts, rules, queries, body arithmetic operators tokenised here)
• Phase 2 — unification + substitution (port or share with lib/prolog/; no function symbols → simpler)
• Phase 3 — EDB + naive evaluation + safety analysis + first scoreboard
• Phase 4 — built-in predicates + body arithmetic (<, >, =, is, +, -, *, /)
• Phase 5 — semi-naive evaluation (delta sets, performance)
• Phase 6 — magic sets (goal-directed bottom-up, opt-in)
• Phase 7 — stratified negation + dependency-graph SCC analysis
• Phase 8 — aggregation (count/sum/min/max, post-fixpoint pass)
• Phase 9 — SX embedding API (dl-program, dl-query, dl-assert!, dl-retract!)
• Phase 10 — Datalog as a query language for rose-ash (federation/permissions/feeds demo)

Within a phase, pick the checkbox with the best tests-per-effort ratio. Once the scoreboard exists (end of Phase 3), it is your north star.

Every iteration: implement → test → commit → tick [ ] in plan → append Progress log → push → next.

Ground rules (hard)

• Scope: only lib/datalog/** and plans/datalog-on-sx.md. Do not edit spec/, hosts/, shared/, other lib/<lang>/ dirs, lib/stdlib.sx, or lib/ root. You may read lib/prolog/ to understand unification — port code into lib/datalog/unify.sx, do not import across language boundaries.
• Non-goals are hard non-goals. Do not implement function symbols, disjunctive heads, well-founded semantics, tabled top-down, constraint Datalog, or distributed evaluation. If a query needs one of these, add a Blockers entry and move on.
• NEVER call sx_build. 600s watchdog will kill you before OCaml finishes. If sx_server.exe is broken, add a Blockers entry and stop.
• Shared-file issues → plan's Blockers section with a minimal repro. Don't fix them.
• SX files: sx-tree MCP tools ONLY. sx_validate after every edit. Never Edit/Read/Write on .sx.
• Worktree: commit, then push to origin/loops/datalog. Never touch main. Never push to architecture.
• Commit granularity: one feature per commit. Short factual messages: datalog: safety analysis + 6 rejection tests.
• Plan file: update Progress log + tick boxes every commit.
• If blocked for two iterations on the same issue, add to Blockers and move on.

Datalog-specific gotchas

• Bottom-up, not DFS. The evaluator iterates rules until no new tuples are derived. There is no goal stack, no backtracking, no cut. If you find yourself reaching for delimited continuations, you are writing Prolog by mistake.
• Termination guaranteed by the language, not the engine. No function symbols → finite Herbrand base → fixpoint always reached. Do not add safety nets like step limits — if your fixpoint diverges, the bug is in the engine or the program is illegal (unsafe rule, function-symbol smuggling).
• Safety analysis must reject early. (p X) :- (< X 5). is unsafe — X is unbound when < runs. Reject at dl-add-rule! time with a clear error. Do not let unsafe rules into the EDB and discover the problem at fixpoint time.
• is binds left, requires right ground. (is Z (+ X Y)) binds Z iff X and Y are already bound by some prior body literal. This is asymmetric — built-in predicates do not "join" the way EDB literals do.
• Stratification rejects programs at load time. (p X) :- (not (q X)). (q X) :- (not (p X)). is non-stratifiable. Detect via SCC analysis on the dependency graph; report the cycle, do not attempt evaluation.
• Aggregation is a separate post-fixpoint pass. count(X, Goal) cannot participate in the recursive fixpoint without breaking monotonicity. Compute the underlying relation via fixpoint, then aggregate.
• Magic sets are opt-in and must be equivalence-tested. A magic-rewritten program must produce the same answers as the original on every input. Add a property test that runs both strategies on small EDBs and diffs the results.
• EDB vs IDB. Extensional database (EDB) = ground facts only, asserted directly. Intensional database (IDB) = relations defined by rules. dl-add-fact! populates EDB; dl-add-rule! populates IDB. A relation cannot be both — flag conflicts.
• No mixing of term representations. Pick ONE shape for atoms (e.g. SX symbols), ONE for variables (e.g. {:var "X"} dicts or symbols starting with uppercase), ONE for ground tuples (e.g. SX lists). Document the choice in the plan's architecture sketch.

General gotchas (all loops)

• SX do = R7RS iteration. Use begin for multi-expr sequences.
• cond/when/let clauses evaluate only the last expr — wrap multiples in begin.
• env-bind! creates a binding; env-set! mutates an existing one (walks scope chain).
• sx_validate after every structural edit.
• list? returns false on raw JS Arrays — host data must be SX-converted.
• Shell heredoc || gets eaten — escape or use case.

Style

• No comments in .sx unless non-obvious.
• No new planning docs — update plans/datalog-on-sx.md inline.
• Short, factual commit messages (datalog: semi-naive delta sets (+12)).
• One feature per iteration. Commit. Log. Push. Next.

Go. Start by reading the plan; find the first unchecked [ ]; implement it.