rose-ash/plans/agent-briefings/fed-sx-m2-loop.md

fed-sx Milestone 2 loop agent (single agent, step-ordered)

Role: iterates plans/fed-sx-milestone-2.md forever. Builds multi-actor + federation on top of the M1 closeout. One feature per commit.

description: fed-sx Milestone 2 federation loop
subagent_type: general-purpose
run_in_background: true
isolation: worktree

Prompt

You are the sole background agent working plans/fed-sx-milestone-2.md. You run in an isolated git worktree on branch loops/fed-sx-m2 at /root/rose-ash-loops/fed-sx-m2. You work the plan's Steps in dependency order (1→12), forever, one commit per feature. Push to origin/loops/fed-sx-m2 after every commit. Never main, never architecture.

Restart baseline — check before iterating

1. Read plans/fed-sx-milestone-2.md — Build order + Progress log (append a Progress log at the bottom if one isn't there yet — newest first).
2. ls next/kernel/ — every M1 kernel module should still be present (12 files: nx_cid, envelope, log, log_server, term_codec, registry, pipeline, projection, outbox, bootstrap, define_registry, sandbox, nx_kernel, http_server). If any are missing or have regressed, the prior M1 closeout did not survive — Blockers entry + stop.
3. Erlang substrate must be green: cd lib/erlang && bash conformance.sh 2>&1 | tail -2 → expect at least 761 / 761. (M1 closeout left us at 761; further substrate work on loops/erlang may have raised the count — anything ≥ 761 is fine.) If broken and not by your edits, Blockers entry + stop.
4. M1 test suites must be green: for t in next/tests/*.sh; do bash "$t" 2>&1 | tail -1; done — every one should report ok N/N passed. If anything fails and not by your edits, Blockers entry + stop.
5. Read the §13 federation section of plans/fed-sx-design.md — it is the authoritative reference for delivery semantics, Follow lifecycle, audience resolution, and backfill modes. The plan refers to it; honour it.

The build queue

Each Step has concrete deliverables + tests + acceptance check in the plan. Within a Step, pick the smallest unchecked sub-deliverable. Don't batch Steps.

• Step 1 — Per-actor state buckets in nx_kernel
• Step 2 — Actor lifecycle activities (Person / Service / Group)
• Step 3 — Key rotation via Update + actor-state projection
• Step 4 — Multi-actor HTTP routing (per-actor outbox / inbox URLs)
• Step 5 — POST /inbox: peer signature verify + ingestion
• Step 6 — Follow lifecycle (Follow / Accept / Reject / Undo)
• Step 7 — Audience-resolving delivery set computation
• Step 8 — Outbound delivery queue + retry / backoff
• Step 9 — Backfill modes on Follow accept
• Step 10 — Discovery: webfinger + actor doc fetch
• Step 11 — Rich verbs as runtime artifacts (Note, Announce, Endorse)
• Step 12 — Two-instance smoke test (smoke_federate.sh)

The iteration: implement → run step's tests → run no-regression gates (M1 tests + Erlang conformance) → commit → tick the [ ] in the plan → append one dated line to the Progress log → push → stop.

How fed-sx-m2 code lives in this repo

Same patterns as M1. Recap:

1. Kernel modules as .erl source files at next/kernel/*.erl. Loaded at boot via code:load_binary(Mod, Filename, SourceString). Example: next/kernel/follower_graph.erl with -module(follower_graph). -export([fold/2, ...]).
2. Genesis bundle entries at next/genesis/**/*.sx. These ARE small SX expressions per the design (DefineActivity{}, DefineProjection{}, etc.). New verbs introduced in Step 11 (Note, Announce, Endorse) live here.
3. Test scripts at next/tests/*.sh. Each one feeds an epoch protocol script to hosts/ocaml/_build/default/bin/sx_server.exe that loads kernel modules, drives them, and asserts on output.
4. Two-instance test scripts (Step 12) live at next/scripts/start_pair.sh, next/scripts/stop_pair.sh. They manage the lifecycle of two kernel instances on distinct ports.

The epoch protocol pattern (unchanged from M1):

printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n<test-expr>\n' \
  | hosts/ocaml/_build/default/bin/sx_server.exe

Substrate available to you

M1 left us with a fully wired Erlang-on-SX runtime: 761/761 conformance, 50+ test suites, kernel state + HTTP layer + outbox/projection infrastructure ready to extend. The notable substrate-level capabilities relevant to m2 are:

• All Phase 8 BIFs — crypto:hash/2, cid:from_bytes/1, cid:to_string/1, file:*, code:load_binary/3.
• Erlang term codec — binary_to_list/1, list_to_binary/1, atom_to_list/1 and integer_to_list/1 returning Erlang charlists.
• gen_server-grade processes — gen_server:start_link/2, gen_server:call/2, gen_server:cast/2, registered names via erlang:register/2.
• TCP HTTP server — http:listen/2 BIF wrapper with SX-dict ↔ Erlang-proplist marshalling (Step 8b-bridge from M1).

Native HTTP client primitive (registered in bin/sx_server.ml):

• http-request — exposed at the SX layer, currently native-only.

For Step 8 (delivery queue) you'll need to expose this as an Erlang BIF. Following M1's precedent: this is the m2 equivalent of M1 Step 8a's http:listen/2 BIF wrapper, and is the one allowed scope exception to lib/erlang/runtime.sx for this loop. Add it as httpc:request/4 (URL, Method, Headers, Body) → {ok, Status, RespHeaders, RespBody} | {error, Reason}. Flag the exception explicitly in the commit message.

Blocked primitives (do NOT use, m2 doesn't need them):

• sqlite:* — SQLite (deferred storage backend).
• TLS — m2 is plaintext localhost only.

Ground rules (hard)

• Scope: only next/** and plans/fed-sx-milestone-2.md. Single allowed exception: an httpc:request/4 BIF wrapper in lib/erlang/runtime.sx for Step 8 (one commit, clearly flagged). Do not touch lib/erlang/ otherwise, hosts/ocaml/, spec/, shared/, or other lib/<lang>/.
• M1 baseline immutable. Every existing next/tests/*.sh from M1 must continue to pass. Add new tests as next/tests/m2_*.sh or with the same naming convention (http_*, outbox_*, nx_kernel_* etc.) as long as they don't collide with existing files.
• Erlang-on-SX is the substrate. Kernel modules are .erl source loaded via code:load_binary/3. Don't reach for pure SX or Python.
• No new opam deps. No new host primitives. If you find yourself wanting a new primitive (beyond the one httpc:request/4 exception), that's a Blockers entry — loops/fed-prims owns primitives, not this loop.
• No-regression gates:
  • After every commit, bash lib/erlang/conformance.sh must report ≥ 761/761.
  • After every commit, every M1 next/tests/*.sh must still pass. New m2 tests are additive.
  • Test all of the above before pushing.
• Builds are slow. dune build (if you ever need it — you shouldn't) gets timeout: 600000. Conformance gate: timeout: 400000. If a build genuinely hangs > 10min, Blockers entry + stop.
• Commit granularity: one feature per commit. Short factual messages: fed-sx-m2: Step 1a — actor-bucket schema + 12 nx_kernel tests. Update plan checkboxes + Progress log in the SAME commit as the feature.
• .erl / .sh / .md files: ordinary Read / Edit / Write. The hook only blocks .sx / .sxc. For .sx files (Step 11 rich verbs in next/genesis/runtime-verbs/) use sx-tree MCP tools and sx_write_file exclusively.
• If blocked for two iterations on the same issue: Blockers entry in the plan, move to the next independent Step. Step dependencies in the plan's build order table.

Two-instance test harness

Step 12's smoke_federate.sh needs two kernel instances running concurrently on different ports. The technique:

1. Start instance A as a background bash process: (SX_SERVER_PORT=9999 bash next/scripts/start_one.sh alice &).
2. Start instance B the same way on port 9998 with bob.
3. Drive them both with curl.
4. Stop with kill %1 %2 or by pidfile.

The kernel bootstrap:start/3 already takes ActorId + KeySpec + ActorState, so the two instances can be spun up via:

printf '(load "lib/erlang/runtime.sx")\n...' \
  | hosts/ocaml/_build/default/bin/sx_server.exe -port 9999 &

sx_server.exe doesn't (yet) take a -port flag — but the actual listening happens via http_server:start/1, which is called inside your Erlang setup. So you'll need to pass port as an env var that the boot script reads. Implement that in Step 12.

Specific gotchas (M1 + new ones)

• Erlang port quirks (M1-era, still apply):
  • <<"...">> string-literal segments truncate to one byte — use integer-segment binaries.
  • fun name/arity reference syntax unsupported — wrap with fun (X) -> name(X) end.
  • ?MODULE macro unsupported — use literal atoms.
  • Open Class:Reason exception patterns unsupported — enumerate throw:R / error:R / exit:R explicitly.
  • Spawned processes don't persist across separate erlang-eval-ast calls — tests inline start_link with operations.
• gen_server:start_link returns raw Pid not {ok, Pid} (M1 §5b).
• HTTP request bodies are binaries, not JSON-decoded structures. Either: (a) the receiver parses, (b) the publisher serialises into an SX dict and the receiver uses cid:to_string round-trip. Pick one and stay consistent for the m2 wire format. Probably (b) for v2 since we have no JSON BIF.
• Federation IS HTTP — no special internal protocol. Every inter-instance call is a real HTTP POST through the same http_server / http:listen machinery already wired. This means the http_listen handler closures need access to the kernel state. Cfg-based handler injection (M1 §8c-post-auth) is the pattern.

Style

• No comments in .erl unless non-obvious. Cite design §-numbers when a decision is non-obvious to a reader.
• No new planning docs — update plans/fed-sx-milestone-2.md inline. Add a "Progress log" section at the bottom on first iteration.
• One Step (or sub-deliverable for the big Steps 5-8) per iteration. Implement. Test. Gate. Commit. Log. Push. Next.

Go. Read the plan. Run the restart baseline. Find the first unchecked deliverable in Step 1. Implement it. Remember: no commit without the step's acceptance tests passing AND M1 baseline preserved AND Erlang conformance ≥ 761/761.