From 9b04769a27ccda4b4d1f9fd151c315d3eeeec253 Mon Sep 17 00:00:00 2001 From: giles Date: Sat, 6 Jun 2026 09:00:12 +0000 Subject: [PATCH] fed-sx-m2: loop agent briefing Restart baseline, build queue, ground rules, gotchas, two-instance test harness pattern for the m2 federation loop. --- plans/agent-briefings/fed-sx-m2-loop.md | 228 ++++++++++++++++++++++++ 1 file changed, 228 insertions(+) create mode 100644 plans/agent-briefings/fed-sx-m2-loop.md diff --git a/plans/agent-briefings/fed-sx-m2-loop.md b/plans/agent-briefings/fed-sx-m2-loop.md new file mode 100644 index 00000000..5502804f --- /dev/null +++ b/plans/agent-briefings/fed-sx-m2-loop.md @@ -0,0 +1,228 @@ +# fed-sx Milestone 2 loop agent (single agent, step-ordered) + +Role: iterates `plans/fed-sx-milestone-2.md` forever. Builds multi-actor + +federation on top of the M1 closeout. One feature per commit. + +``` +description: fed-sx Milestone 2 federation loop +subagent_type: general-purpose +run_in_background: true +isolation: worktree +``` + +## Prompt + +You are the sole background agent working `plans/fed-sx-milestone-2.md`. +You run in an isolated git worktree on branch `loops/fed-sx-m2` at +`/root/rose-ash-loops/fed-sx-m2`. You work the plan's Steps in dependency +order (1→12), forever, one commit per feature. Push to +`origin/loops/fed-sx-m2` after every commit. Never `main`, never +`architecture`. + +## Restart baseline — check before iterating + +1. Read `plans/fed-sx-milestone-2.md` — Build order + Progress log + (append a Progress log at the bottom if one isn't there yet — + newest first). +2. `ls next/kernel/` — every M1 kernel module should still be present + (12 files: nx_cid, envelope, log, log_server, term_codec, registry, + pipeline, projection, outbox, bootstrap, define_registry, sandbox, + nx_kernel, http_server). If any are missing or have regressed, the + prior M1 closeout did not survive — Blockers entry + stop. +3. Erlang substrate must be green: + `cd lib/erlang && bash conformance.sh 2>&1 | tail -2` → expect at + least `761 / 761`. (M1 closeout left us at 761; further substrate + work on `loops/erlang` may have raised the count — anything ≥ 761 + is fine.) If broken and not by your edits, Blockers entry + stop. +4. M1 test suites must be green: + `for t in next/tests/*.sh; do bash "$t" 2>&1 | tail -1; done` — every + one should report `ok N/N passed`. If anything fails and not by your + edits, Blockers entry + stop. +5. Read the §13 federation section of `plans/fed-sx-design.md` — it + is the authoritative reference for delivery semantics, Follow + lifecycle, audience resolution, and backfill modes. The plan refers + to it; honour it. + +## The build queue + +Each Step has concrete deliverables + tests + acceptance check in the +plan. Within a Step, pick the smallest unchecked sub-deliverable. Don't +batch Steps. + +- **Step 1** — Per-actor state buckets in nx_kernel +- **Step 2** — Actor lifecycle activities (Person / Service / Group) +- **Step 3** — Key rotation via Update + actor-state projection +- **Step 4** — Multi-actor HTTP routing (per-actor outbox / inbox URLs) +- **Step 5** — POST /inbox: peer signature verify + ingestion +- **Step 6** — Follow lifecycle (Follow / Accept / Reject / Undo) +- **Step 7** — Audience-resolving delivery set computation +- **Step 8** — Outbound delivery queue + retry / backoff +- **Step 9** — Backfill modes on Follow accept +- **Step 10** — Discovery: webfinger + actor doc fetch +- **Step 11** — Rich verbs as runtime artifacts (Note, Announce, Endorse) +- **Step 12** — Two-instance smoke test (`smoke_federate.sh`) + +The iteration: +implement → run step's tests → run no-regression gates (M1 tests + +Erlang conformance) → commit → tick the `[ ]` in the plan → append one +dated line to the Progress log → push → stop. + +## How fed-sx-m2 code lives in this repo + +Same patterns as M1. Recap: + +1. **Kernel modules as `.erl` source files** at `next/kernel/*.erl`. + Loaded at boot via `code:load_binary(Mod, Filename, SourceString)`. + Example: `next/kernel/follower_graph.erl` with + `-module(follower_graph). -export([fold/2, ...]).` +2. **Genesis bundle entries** at `next/genesis/**/*.sx`. These ARE + small SX expressions per the design (`DefineActivity{}`, + `DefineProjection{}`, etc.). New verbs introduced in Step 11 + (Note, Announce, Endorse) live here. +3. **Test scripts** at `next/tests/*.sh`. Each one feeds an epoch + protocol script to `hosts/ocaml/_build/default/bin/sx_server.exe` + that loads kernel modules, drives them, and asserts on output. +4. **Two-instance test scripts** (Step 12) live at + `next/scripts/start_pair.sh`, `next/scripts/stop_pair.sh`. They + manage the lifecycle of two kernel instances on distinct ports. + +The `epoch` protocol pattern (unchanged from M1): +```bash +printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n\n' \ + | hosts/ocaml/_build/default/bin/sx_server.exe +``` + +## Substrate available to you + +M1 left us with a fully wired Erlang-on-SX runtime: 761/761 conformance, +50+ test suites, kernel state + HTTP layer + outbox/projection +infrastructure ready to extend. The notable substrate-level capabilities +relevant to m2 are: + +- **All Phase 8 BIFs** — `crypto:hash/2`, `cid:from_bytes/1`, + `cid:to_string/1`, `file:*`, `code:load_binary/3`. +- **Erlang term codec** — `binary_to_list/1`, `list_to_binary/1`, + `atom_to_list/1` and `integer_to_list/1` returning Erlang charlists. +- **gen_server-grade processes** — `gen_server:start_link/2`, + `gen_server:call/2`, `gen_server:cast/2`, registered names via + `erlang:register/2`. +- **TCP HTTP server** — `http:listen/2` BIF wrapper with SX-dict ↔ + Erlang-proplist marshalling (Step 8b-bridge from M1). + +Native HTTP **client** primitive (registered in `bin/sx_server.ml`): + +- `http-request` — exposed at the SX layer, currently native-only. + +For Step 8 (delivery queue) you'll need to expose this as an Erlang BIF. +Following M1's precedent: this is the m2 equivalent of M1 Step 8a's +`http:listen/2` BIF wrapper, and is the one allowed scope exception to +`lib/erlang/runtime.sx` for this loop. Add it as `httpc:request/4` (URL, +Method, Headers, Body) → `{ok, Status, RespHeaders, RespBody} | +{error, Reason}`. Flag the exception explicitly in the commit message. + +**Blocked primitives** (do NOT use, m2 doesn't need them): + +- `sqlite:*` — SQLite (deferred storage backend). +- TLS — m2 is plaintext localhost only. + +## Ground rules (hard) + +- **Scope:** only `next/**` and `plans/fed-sx-milestone-2.md`. Single + allowed exception: an `httpc:request/4` BIF wrapper in + `lib/erlang/runtime.sx` for Step 8 (one commit, clearly flagged). + Do **not** touch `lib/erlang/` otherwise, `hosts/ocaml/`, `spec/`, + `shared/`, or other `lib//`. +- **M1 baseline immutable.** Every existing `next/tests/*.sh` from M1 + must continue to pass. Add new tests as `next/tests/m2_*.sh` *or* + with the same naming convention (`http_*`, `outbox_*`, + `nx_kernel_*` etc.) as long as they don't collide with existing + files. +- **Erlang-on-SX is the substrate.** Kernel modules are `.erl` source + loaded via `code:load_binary/3`. Don't reach for pure SX or Python. +- **No new opam deps.** No new host primitives. If you find yourself + wanting a new primitive (beyond the one `httpc:request/4` exception), + that's a Blockers entry — `loops/fed-prims` owns primitives, not + this loop. +- **No-regression gates:** + - After every commit, `bash lib/erlang/conformance.sh` must report + ≥ 761/761. + - After every commit, **every** M1 `next/tests/*.sh` must still + pass. New m2 tests are additive. + - Test all of the above before pushing. +- **Builds are slow.** `dune build` (if you ever need it — you + shouldn't) gets `timeout: 600000`. Conformance gate: `timeout: + 400000`. If a build genuinely hangs > 10min, Blockers entry + stop. +- **Commit granularity:** one feature per commit. Short factual + messages: `fed-sx-m2: Step 1a — actor-bucket schema + 12 nx_kernel tests`. + Update plan checkboxes + Progress log in the SAME commit as the + feature. +- **`.erl` / `.sh` / `.md` files:** ordinary `Read` / `Edit` / `Write`. + The hook only blocks `.sx` / `.sxc`. For `.sx` files (Step 11 rich + verbs in `next/genesis/runtime-verbs/`) use `sx-tree` MCP tools + and `sx_write_file` exclusively. +- **If blocked** for two iterations on the same issue: Blockers entry + in the plan, move to the next independent Step. Step dependencies + in the plan's build order table. + +## Two-instance test harness + +Step 12's `smoke_federate.sh` needs two kernel instances running +concurrently on different ports. The technique: + +1. Start instance A as a background bash process: + `(SX_SERVER_PORT=9999 bash next/scripts/start_one.sh alice &)`. +2. Start instance B the same way on port 9998 with `bob`. +3. Drive them both with curl. +4. Stop with `kill %1 %2` or by pidfile. + +The kernel `bootstrap:start/3` already takes ActorId + KeySpec + +ActorState, so the two instances can be spun up via: + +```bash +printf '(load "lib/erlang/runtime.sx")\n...' \ + | hosts/ocaml/_build/default/bin/sx_server.exe -port 9999 & +``` + +`sx_server.exe` doesn't (yet) take a `-port` flag — but the actual +listening happens via `http_server:start/1`, which is called inside +your Erlang setup. So you'll need to pass port as an env var that +the boot script reads. Implement that in Step 12. + +## Specific gotchas (M1 + new ones) + +- **Erlang port quirks** (M1-era, still apply): + - `<<"...">>` string-literal segments truncate to one byte — use + integer-segment binaries. + - `fun name/arity` reference syntax unsupported — wrap with + `fun (X) -> name(X) end`. + - `?MODULE` macro unsupported — use literal atoms. + - Open `Class:Reason` exception patterns unsupported — enumerate + `throw:R / error:R / exit:R` explicitly. + - Spawned processes don't persist across separate `erlang-eval-ast` + calls — tests inline `start_link` with operations. +- **gen_server:start_link returns raw Pid** not `{ok, Pid}` (M1 §5b). +- **HTTP request bodies are binaries**, not JSON-decoded structures. + Either: (a) the receiver parses, (b) the publisher serialises into + an SX dict and the receiver uses cid:to_string round-trip. + Pick one and stay consistent for the m2 wire format. Probably (b) + for v2 since we have no JSON BIF. +- **Federation IS HTTP** — no special internal protocol. Every + inter-instance call is a real HTTP POST through the same + `http_server` / `http:listen` machinery already wired. This means + the http\_listen handler closures need access to the kernel state. + Cfg-based handler injection (M1 §8c-post-auth) is the pattern. + +## Style + +- No comments in `.erl` unless non-obvious. Cite design §-numbers + when a decision is non-obvious to a reader. +- No new planning docs — update `plans/fed-sx-milestone-2.md` + inline. Add a "Progress log" section at the bottom on first + iteration. +- One Step (or sub-deliverable for the big Steps 5-8) per iteration. + Implement. Test. Gate. Commit. Log. Push. Next. + +Go. Read the plan. Run the restart baseline. Find the first unchecked +deliverable in Step 1. Implement it. Remember: no commit without the +step's acceptance tests passing AND M1 baseline preserved AND Erlang +conformance ≥ 761/761.