Merge loops/fed-sx-m2 into architecture: federation milestone 2
m2 lands multi-actor + cross-instance federation on the fed-sx
substrate. Feature-complete except 8b-timer (retry-loop wiring,
gated on erlang:send_after substrate primitive in loops/erlang).
Highlights:
- Multi-actor gen_server kernel (one nx_kernel handles N actors)
- Per-actor HTTP routes /actors/<id>/{inbox,outbox} + actor-doc
- Inbound signature verify + peer-AS cache + auto-Accept publish
- Outbound delivery_set with audience expansion + delivery_worker
- Native httpc:request/4 BIF wrapper + live HTTP dispatch
- Discovery: peer-actor fetch + cache on demand
- Backfill on Follow accept (in-process + paginated outbox)
- Two-instance smoke test passes 6/6 (real cross-host HTTP flow)
Substrate fixes carried in this merge (textually identical to
upstream-arrived copies, will conflict on scoreboard files only):
- Blockers #1: er-bif-http-listen marshaller bridge rewrite
- Blockers #4: er-sched-step-alive! :pending-args extension
(lets receive in a kernel-aware route suspend+resume cleanly)
Conformance 761/761 still green on m2 tip.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts:
# lib/erlang/runtime.sx
This commit is contained in:
197
plans/agent-briefings/fed-prims-mutex-fix.md
Normal file
197
plans/agent-briefings/fed-prims-mutex-fix.md
Normal file
@@ -0,0 +1,197 @@
|
||||
# fed-prims handler-mutex deadlock fix (one-shot)
|
||||
|
||||
Role: fix the SX runtime mutex deadlock in `bin/sx_server.ml`'s
|
||||
`http-listen` handler that blocks every `gen_server:call` from inside
|
||||
an Erlang route. Documented as **Blockers #4** in
|
||||
`/root/rose-ash-loops/fed-sx-m1/plans/fed-sx-milestone-2.md`.
|
||||
|
||||
```
|
||||
description: fed-prims handler-mutex deadlock fix
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Worktree + branch
|
||||
|
||||
Already provisioned at `/root/rose-ash-loops/fed-prims` on branch
|
||||
`loops/fed-prims` (the fed-prims phases A–J are landed; this is a
|
||||
follow-up fix). Start there. Never push to `main` or `architecture`.
|
||||
|
||||
If `.mcp.json` shows a non-absolute `mcp_tree` path or `.claude/
|
||||
scheduled_tasks.lock` is dirty, just leave them alone — they're
|
||||
harness state. Stash if you must, but don't commit them.
|
||||
|
||||
## The problem (verified by fed-sx-m2 loop, 2026-06-07)
|
||||
|
||||
Native `http-listen` in `hosts/ocaml/bin/sx_server.ml:735+`
|
||||
serialises handler calls with `Mutex.lock mtx` / `Mutex.unlock mtx`
|
||||
so the SX runtime isn't re-entered concurrently:
|
||||
|
||||
```ocaml
|
||||
Mutex.lock mtx;
|
||||
let resp =
|
||||
(try Sx_runtime.sx_call handler [Dict req]
|
||||
with e -> Mutex.unlock mtx; raise e) in
|
||||
Mutex.unlock mtx;
|
||||
```
|
||||
|
||||
When the Erlang handler does `gen_server:call(nx_kernel, ...)` from
|
||||
any kernel-aware route (`actor_doc_response_for/3`,
|
||||
`actor_outbox_response_for/3`, `handle_inbox_post`,
|
||||
`nx_kernel:state_for/1`, etc.), the gen_server's reply needs the SX
|
||||
runtime scheduler to run — but the calling handler is sitting on the
|
||||
runtime mutex. Deadlock; curl hangs until `--max-time` fires.
|
||||
|
||||
**Verification recipe (reproduces deterministically):**
|
||||
|
||||
```bash
|
||||
PORT=51920
|
||||
cat > /tmp/boot.sx <<'SX'
|
||||
(epoch 1)
|
||||
(load "lib/erlang/tokenizer.sx") (load "lib/erlang/parser.sx")
|
||||
(load "lib/erlang/parser-core.sx") (load "lib/erlang/parser-expr.sx")
|
||||
(load "lib/erlang/parser-module.sx") (load "lib/erlang/transpile.sx")
|
||||
(load "lib/erlang/runtime.sx") (load "lib/erlang/vm/dispatcher.sx")
|
||||
(epoch 2)
|
||||
(eval "(er-load-gen-server!)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/envelope.erl\")) :name)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/log.erl\")) :name)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/pipeline.erl\")) :name)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/term_codec.erl\")) :name)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/outbox.erl\")) :name)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/nx_kernel.erl\")) :name)")
|
||||
(eval "(get (erlang-load-module (file-read \"next/kernel/http_server.erl\")) :name)")
|
||||
(epoch 20)
|
||||
(eval "(erlang-eval-ast \"AK = <<1,1,1,1>>, AKS = [{key_id,k1},{algorithm,ed25519},{value,AK}], AAS = [{public_keys,[[{id,k1},{created,0},{value,AK}]]}], nx_kernel:start_link(alice, AKS, AAS), http_server:start(51920, [{kernel, nx_kernel}])\")")
|
||||
SX
|
||||
mkfifo /tmp/fifo
|
||||
( cat /tmp/boot.sx; sleep 120 ) > /tmp/fifo &
|
||||
hosts/ocaml/_build/default/bin/sx_server.exe < /tmp/fifo > /tmp/log 2>&1 &
|
||||
sleep 60 # boot takes ~30-45s cold
|
||||
curl -sv --max-time 5 "http://127.0.0.1:$PORT/" >/dev/null # OK: 200
|
||||
curl -sv --max-time 5 "http://127.0.0.1:$PORT/actors/alice/outbox" # HANGS
|
||||
```
|
||||
|
||||
The `next/kernel/*.erl` files referenced live in the fed-sx-m2
|
||||
worktree at `/root/rose-ash-loops/fed-sx-m1/next/kernel/`. You can
|
||||
read them there for context but do NOT edit them — Erlang-side
|
||||
work is m2's loop. This loop only touches `hosts/ocaml/bin/sx_server.ml`.
|
||||
|
||||
## Two fix patterns
|
||||
|
||||
Pick **one**. Both are independent enough to evaluate alone; commit
|
||||
the one that lands first.
|
||||
|
||||
### Pattern A — release the mutex around the SX call
|
||||
|
||||
The mutex exists to serialise SX runtime mutation. But once the
|
||||
runtime hands the call off to the gen_server (which has its own
|
||||
scheduler frame), the calling thread is just waiting on a reply
|
||||
message; it doesn't need the mutex. The fix is to scope the mutex
|
||||
*only* over the runtime entry, not the entire handler invocation.
|
||||
|
||||
This may require restructuring `Sx_runtime.sx_call handler [Dict req]`
|
||||
so the call yields to the scheduler instead of blocking — verify by
|
||||
reading `hosts/ocaml/lib/sx_runtime.ml` (or wherever `sx_call` lives).
|
||||
If `sx_call` is fully synchronous and re-entry is genuinely unsafe,
|
||||
fall back to Pattern B.
|
||||
|
||||
### Pattern B — spawn handler in a fresh er-process
|
||||
|
||||
Erlang processes already have their own scheduler frame. Have the
|
||||
handler closure trampoline through `er-spawn-fun` (or equivalent —
|
||||
check `lib/erlang/runtime.sx`'s existing process primitives) so the
|
||||
gen_server reply runs in a different frame from the http-listen
|
||||
accept-loop thread.
|
||||
|
||||
This may be cleaner if it can be done entirely at the SX/Erlang
|
||||
layer (in `er-bif-http-listen` in `lib/erlang/runtime.sx`), in which
|
||||
case **this is m2 scope** and you should hand it back rather than
|
||||
edit OCaml. Read the BIF body first — if a pure-Erlang spawn
|
||||
suffices, document that and stop without committing OCaml changes.
|
||||
|
||||
The BIF body is at `lib/erlang/runtime.sx:1581-1632` (in the
|
||||
fed-sx-m2 worktree); the m2 loop just rewrote its inbound/outbound
|
||||
marshallers (commit `8d33d02f`). The handler is invoked inside
|
||||
`(http-listen port sx-handler)` — figure out whether you can
|
||||
`er-spawn-fun` around the body of `sx-handler` such that the
|
||||
spawned process's gen_server:call doesn't fight the parent's
|
||||
runtime mutex.
|
||||
|
||||
## Acceptance — the unblock target
|
||||
|
||||
`next/tests/http_server_tcp.sh` 5/5 stays green (the existing simple
|
||||
GET / + capabilities + 404 + 401 surface). PLUS:
|
||||
|
||||
A kernel-touching request over real HTTP must return without
|
||||
hanging. The minimal smoke for this is:
|
||||
|
||||
```bash
|
||||
# In the verification recipe above, after boot:
|
||||
curl -s --max-time 5 "http://127.0.0.1:$PORT/actors/alice/outbox"
|
||||
# Expected: "outbox: alice\ntip: 0\n" or similar (200 with body),
|
||||
# NOT a timeout.
|
||||
```
|
||||
|
||||
If you want a one-shot script, save the recipe above as a regression
|
||||
test inside the fed-prims worktree:
|
||||
`hosts/ocaml/test/handler_kernel_unblock.sh` (new file). Make it
|
||||
pass deterministically with a generous timeout (≥120s for the cold
|
||||
boot).
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** `hosts/ocaml/bin/sx_server.ml` and adjacent
|
||||
`hosts/ocaml/lib/sx_runtime.ml` (or wherever `sx_call` is
|
||||
defined). Do NOT touch `next/**` or `plans/fed-sx-milestone-2.md`
|
||||
(m2's loop owns those). Do NOT touch `lib/erlang/**` (Erlang
|
||||
substrate / loops/erlang owns that).
|
||||
- **No-regression gate:**
|
||||
- `dune build bin/sx_server.exe` (native) green
|
||||
- `bash hosts/ocaml/browser/test_boot.sh` (WASM kernel) green
|
||||
- `bash lib/erlang/conformance.sh` 761/761
|
||||
- `bash next/tests/http_server_tcp.sh` 5/5
|
||||
- **WASM safety:** Pattern A may need Thread / Mutex juggling
|
||||
that isn't WASM-safe. The `http-listen` primitive is already
|
||||
native-only, so changes to its handler code don't need to
|
||||
build under WASM — but anything in `lib/sx_runtime.ml` does.
|
||||
If your change has to add `Thread`/`Mutex` to `lib/`, you've
|
||||
picked the wrong fix; back out.
|
||||
- **Builds are slow.** `dune build` ≥600s timeout. `conformance.sh`
|
||||
≥400s. `test_boot.sh` ≥60s.
|
||||
- **Commit granularity:** one fix, one commit. Title like:
|
||||
`fed-prims: release runtime mutex around gen_server:call (Blockers #4)`.
|
||||
- **No `.sx` edits.** All work is `.ml` (or `.sh` for the
|
||||
regression test). sx-tree MCP is not needed.
|
||||
- **Worktree:** commit, push `origin/loops/fed-prims`. Never
|
||||
`main`, never `architecture`. The user merges to architecture
|
||||
separately.
|
||||
|
||||
## What to write back
|
||||
|
||||
Append one dated line to `plans/fed-sx-host-primitives.md`'s
|
||||
Progress log (newest first):
|
||||
|
||||
```
|
||||
- 2026-06-07 — Resolved fed-sx-m2 Blockers #4 (handler mutex
|
||||
deadlock). <one-sentence description of the fix>. Verified via
|
||||
hosts/ocaml/test/handler_kernel_unblock.sh + http_server_tcp.sh
|
||||
5/5 + conformance 761/761 + WASM boot.
|
||||
```
|
||||
|
||||
Once landed, the fed-sx-m2 loop will pick up the fix on its next
|
||||
tick and unblock Step 12 — you don't need to coordinate.
|
||||
|
||||
## If it's not Pattern A or Pattern B
|
||||
|
||||
If you discover the deadlock is something else entirely
|
||||
(e.g., a gen_server config issue, a different lock in
|
||||
`Sx_runtime`, a bug in `er-load-gen-server!`'s scheduler frame),
|
||||
document what you found in a fresh Blockers entry on
|
||||
`plans/fed-sx-host-primitives.md` and stop. The m2 loop will
|
||||
re-check on its next tick. **Do not invent a Pattern C without
|
||||
clear evidence** — the deadlock is reproducible and the two
|
||||
patterns above cover the obvious fix shapes.
|
||||
|
||||
Go. Reproduce the deadlock first. Pick a pattern. Land it. Push.
|
||||
228
plans/agent-briefings/fed-sx-m2-loop.md
Normal file
228
plans/agent-briefings/fed-sx-m2-loop.md
Normal file
@@ -0,0 +1,228 @@
|
||||
# fed-sx Milestone 2 loop agent (single agent, step-ordered)
|
||||
|
||||
Role: iterates `plans/fed-sx-milestone-2.md` forever. Builds multi-actor +
|
||||
federation on top of the M1 closeout. One feature per commit.
|
||||
|
||||
```
|
||||
description: fed-sx Milestone 2 federation loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `plans/fed-sx-milestone-2.md`.
|
||||
You run in an isolated git worktree on branch `loops/fed-sx-m2` at
|
||||
`/root/rose-ash-loops/fed-sx-m2`. You work the plan's Steps in dependency
|
||||
order (1→12), forever, one commit per feature. Push to
|
||||
`origin/loops/fed-sx-m2` after every commit. Never `main`, never
|
||||
`architecture`.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
1. Read `plans/fed-sx-milestone-2.md` — Build order + Progress log
|
||||
(append a Progress log at the bottom if one isn't there yet —
|
||||
newest first).
|
||||
2. `ls next/kernel/` — every M1 kernel module should still be present
|
||||
(12 files: nx_cid, envelope, log, log_server, term_codec, registry,
|
||||
pipeline, projection, outbox, bootstrap, define_registry, sandbox,
|
||||
nx_kernel, http_server). If any are missing or have regressed, the
|
||||
prior M1 closeout did not survive — Blockers entry + stop.
|
||||
3. Erlang substrate must be green:
|
||||
`cd lib/erlang && bash conformance.sh 2>&1 | tail -2` → expect at
|
||||
least `761 / 761`. (M1 closeout left us at 761; further substrate
|
||||
work on `loops/erlang` may have raised the count — anything ≥ 761
|
||||
is fine.) If broken and not by your edits, Blockers entry + stop.
|
||||
4. M1 test suites must be green:
|
||||
`for t in next/tests/*.sh; do bash "$t" 2>&1 | tail -1; done` — every
|
||||
one should report `ok N/N passed`. If anything fails and not by your
|
||||
edits, Blockers entry + stop.
|
||||
5. Read the §13 federation section of `plans/fed-sx-design.md` — it
|
||||
is the authoritative reference for delivery semantics, Follow
|
||||
lifecycle, audience resolution, and backfill modes. The plan refers
|
||||
to it; honour it.
|
||||
|
||||
## The build queue
|
||||
|
||||
Each Step has concrete deliverables + tests + acceptance check in the
|
||||
plan. Within a Step, pick the smallest unchecked sub-deliverable. Don't
|
||||
batch Steps.
|
||||
|
||||
- **Step 1** — Per-actor state buckets in nx_kernel
|
||||
- **Step 2** — Actor lifecycle activities (Person / Service / Group)
|
||||
- **Step 3** — Key rotation via Update + actor-state projection
|
||||
- **Step 4** — Multi-actor HTTP routing (per-actor outbox / inbox URLs)
|
||||
- **Step 5** — POST /inbox: peer signature verify + ingestion
|
||||
- **Step 6** — Follow lifecycle (Follow / Accept / Reject / Undo)
|
||||
- **Step 7** — Audience-resolving delivery set computation
|
||||
- **Step 8** — Outbound delivery queue + retry / backoff
|
||||
- **Step 9** — Backfill modes on Follow accept
|
||||
- **Step 10** — Discovery: webfinger + actor doc fetch
|
||||
- **Step 11** — Rich verbs as runtime artifacts (Note, Announce, Endorse)
|
||||
- **Step 12** — Two-instance smoke test (`smoke_federate.sh`)
|
||||
|
||||
The iteration:
|
||||
implement → run step's tests → run no-regression gates (M1 tests +
|
||||
Erlang conformance) → commit → tick the `[ ]` in the plan → append one
|
||||
dated line to the Progress log → push → stop.
|
||||
|
||||
## How fed-sx-m2 code lives in this repo
|
||||
|
||||
Same patterns as M1. Recap:
|
||||
|
||||
1. **Kernel modules as `.erl` source files** at `next/kernel/*.erl`.
|
||||
Loaded at boot via `code:load_binary(Mod, Filename, SourceString)`.
|
||||
Example: `next/kernel/follower_graph.erl` with
|
||||
`-module(follower_graph). -export([fold/2, ...]).`
|
||||
2. **Genesis bundle entries** at `next/genesis/**/*.sx`. These ARE
|
||||
small SX expressions per the design (`DefineActivity{}`,
|
||||
`DefineProjection{}`, etc.). New verbs introduced in Step 11
|
||||
(Note, Announce, Endorse) live here.
|
||||
3. **Test scripts** at `next/tests/*.sh`. Each one feeds an epoch
|
||||
protocol script to `hosts/ocaml/_build/default/bin/sx_server.exe`
|
||||
that loads kernel modules, drives them, and asserts on output.
|
||||
4. **Two-instance test scripts** (Step 12) live at
|
||||
`next/scripts/start_pair.sh`, `next/scripts/stop_pair.sh`. They
|
||||
manage the lifecycle of two kernel instances on distinct ports.
|
||||
|
||||
The `epoch` protocol pattern (unchanged from M1):
|
||||
```bash
|
||||
printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n<test-expr>\n' \
|
||||
| hosts/ocaml/_build/default/bin/sx_server.exe
|
||||
```
|
||||
|
||||
## Substrate available to you
|
||||
|
||||
M1 left us with a fully wired Erlang-on-SX runtime: 761/761 conformance,
|
||||
50+ test suites, kernel state + HTTP layer + outbox/projection
|
||||
infrastructure ready to extend. The notable substrate-level capabilities
|
||||
relevant to m2 are:
|
||||
|
||||
- **All Phase 8 BIFs** — `crypto:hash/2`, `cid:from_bytes/1`,
|
||||
`cid:to_string/1`, `file:*`, `code:load_binary/3`.
|
||||
- **Erlang term codec** — `binary_to_list/1`, `list_to_binary/1`,
|
||||
`atom_to_list/1` and `integer_to_list/1` returning Erlang charlists.
|
||||
- **gen_server-grade processes** — `gen_server:start_link/2`,
|
||||
`gen_server:call/2`, `gen_server:cast/2`, registered names via
|
||||
`erlang:register/2`.
|
||||
- **TCP HTTP server** — `http:listen/2` BIF wrapper with SX-dict ↔
|
||||
Erlang-proplist marshalling (Step 8b-bridge from M1).
|
||||
|
||||
Native HTTP **client** primitive (registered in `bin/sx_server.ml`):
|
||||
|
||||
- `http-request` — exposed at the SX layer, currently native-only.
|
||||
|
||||
For Step 8 (delivery queue) you'll need to expose this as an Erlang BIF.
|
||||
Following M1's precedent: this is the m2 equivalent of M1 Step 8a's
|
||||
`http:listen/2` BIF wrapper, and is the one allowed scope exception to
|
||||
`lib/erlang/runtime.sx` for this loop. Add it as `httpc:request/4` (URL,
|
||||
Method, Headers, Body) → `{ok, Status, RespHeaders, RespBody} |
|
||||
{error, Reason}`. Flag the exception explicitly in the commit message.
|
||||
|
||||
**Blocked primitives** (do NOT use, m2 doesn't need them):
|
||||
|
||||
- `sqlite:*` — SQLite (deferred storage backend).
|
||||
- TLS — m2 is plaintext localhost only.
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `next/**` and `plans/fed-sx-milestone-2.md`. Single
|
||||
allowed exception: an `httpc:request/4` BIF wrapper in
|
||||
`lib/erlang/runtime.sx` for Step 8 (one commit, clearly flagged).
|
||||
Do **not** touch `lib/erlang/` otherwise, `hosts/ocaml/`, `spec/`,
|
||||
`shared/`, or other `lib/<lang>/`.
|
||||
- **M1 baseline immutable.** Every existing `next/tests/*.sh` from M1
|
||||
must continue to pass. Add new tests as `next/tests/m2_*.sh` *or*
|
||||
with the same naming convention (`http_*`, `outbox_*`,
|
||||
`nx_kernel_*` etc.) as long as they don't collide with existing
|
||||
files.
|
||||
- **Erlang-on-SX is the substrate.** Kernel modules are `.erl` source
|
||||
loaded via `code:load_binary/3`. Don't reach for pure SX or Python.
|
||||
- **No new opam deps.** No new host primitives. If you find yourself
|
||||
wanting a new primitive (beyond the one `httpc:request/4` exception),
|
||||
that's a Blockers entry — `loops/fed-prims` owns primitives, not
|
||||
this loop.
|
||||
- **No-regression gates:**
|
||||
- After every commit, `bash lib/erlang/conformance.sh` must report
|
||||
≥ 761/761.
|
||||
- After every commit, **every** M1 `next/tests/*.sh` must still
|
||||
pass. New m2 tests are additive.
|
||||
- Test all of the above before pushing.
|
||||
- **Builds are slow.** `dune build` (if you ever need it — you
|
||||
shouldn't) gets `timeout: 600000`. Conformance gate: `timeout:
|
||||
400000`. If a build genuinely hangs > 10min, Blockers entry + stop.
|
||||
- **Commit granularity:** one feature per commit. Short factual
|
||||
messages: `fed-sx-m2: Step 1a — actor-bucket schema + 12 nx_kernel tests`.
|
||||
Update plan checkboxes + Progress log in the SAME commit as the
|
||||
feature.
|
||||
- **`.erl` / `.sh` / `.md` files:** ordinary `Read` / `Edit` / `Write`.
|
||||
The hook only blocks `.sx` / `.sxc`. For `.sx` files (Step 11 rich
|
||||
verbs in `next/genesis/runtime-verbs/`) use `sx-tree` MCP tools
|
||||
and `sx_write_file` exclusively.
|
||||
- **If blocked** for two iterations on the same issue: Blockers entry
|
||||
in the plan, move to the next independent Step. Step dependencies
|
||||
in the plan's build order table.
|
||||
|
||||
## Two-instance test harness
|
||||
|
||||
Step 12's `smoke_federate.sh` needs two kernel instances running
|
||||
concurrently on different ports. The technique:
|
||||
|
||||
1. Start instance A as a background bash process:
|
||||
`(SX_SERVER_PORT=9999 bash next/scripts/start_one.sh alice &)`.
|
||||
2. Start instance B the same way on port 9998 with `bob`.
|
||||
3. Drive them both with curl.
|
||||
4. Stop with `kill %1 %2` or by pidfile.
|
||||
|
||||
The kernel `bootstrap:start/3` already takes ActorId + KeySpec +
|
||||
ActorState, so the two instances can be spun up via:
|
||||
|
||||
```bash
|
||||
printf '(load "lib/erlang/runtime.sx")\n...' \
|
||||
| hosts/ocaml/_build/default/bin/sx_server.exe -port 9999 &
|
||||
```
|
||||
|
||||
`sx_server.exe` doesn't (yet) take a `-port` flag — but the actual
|
||||
listening happens via `http_server:start/1`, which is called inside
|
||||
your Erlang setup. So you'll need to pass port as an env var that
|
||||
the boot script reads. Implement that in Step 12.
|
||||
|
||||
## Specific gotchas (M1 + new ones)
|
||||
|
||||
- **Erlang port quirks** (M1-era, still apply):
|
||||
- `<<"...">>` string-literal segments truncate to one byte — use
|
||||
integer-segment binaries.
|
||||
- `fun name/arity` reference syntax unsupported — wrap with
|
||||
`fun (X) -> name(X) end`.
|
||||
- `?MODULE` macro unsupported — use literal atoms.
|
||||
- Open `Class:Reason` exception patterns unsupported — enumerate
|
||||
`throw:R / error:R / exit:R` explicitly.
|
||||
- Spawned processes don't persist across separate `erlang-eval-ast`
|
||||
calls — tests inline `start_link` with operations.
|
||||
- **gen_server:start_link returns raw Pid** not `{ok, Pid}` (M1 §5b).
|
||||
- **HTTP request bodies are binaries**, not JSON-decoded structures.
|
||||
Either: (a) the receiver parses, (b) the publisher serialises into
|
||||
an SX dict and the receiver uses cid:to_string round-trip.
|
||||
Pick one and stay consistent for the m2 wire format. Probably (b)
|
||||
for v2 since we have no JSON BIF.
|
||||
- **Federation IS HTTP** — no special internal protocol. Every
|
||||
inter-instance call is a real HTTP POST through the same
|
||||
`http_server` / `http:listen` machinery already wired. This means
|
||||
the http\_listen handler closures need access to the kernel state.
|
||||
Cfg-based handler injection (M1 §8c-post-auth) is the pattern.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.erl` unless non-obvious. Cite design §-numbers
|
||||
when a decision is non-obvious to a reader.
|
||||
- No new planning docs — update `plans/fed-sx-milestone-2.md`
|
||||
inline. Add a "Progress log" section at the bottom on first
|
||||
iteration.
|
||||
- One Step (or sub-deliverable for the big Steps 5-8) per iteration.
|
||||
Implement. Test. Gate. Commit. Log. Push. Next.
|
||||
|
||||
Go. Read the plan. Run the restart baseline. Find the first unchecked
|
||||
deliverable in Step 1. Implement it. Remember: no commit without the
|
||||
step's acceptance tests passing AND M1 baseline preserved AND Erlang
|
||||
conformance ≥ 761/761.
|
||||
@@ -99,6 +99,10 @@ in isolation, and a clear acceptance check.
|
||||
|
||||
## Step 1 — Repo skeleton + canonical CID
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **1a** — `next/` directory skeleton, README, `.gitignore` for `data/`
|
||||
- [x] **1b** — `next/kernel/nx_cid.erl` (from_sx/to_string/from_string/equals) + `next/tests/cid.sh` (13 cases). Module is `nx_cid` not `cid` — the `cid` BIF module would be shadowed by a user module of the same name; plan §Step 1's `cid.erl` is illustrative per briefing.
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
```
|
||||
@@ -146,6 +150,11 @@ canonicalize_sx(V) -> ... % sorts dict keys, normalizes strings
|
||||
|
||||
## Step 2 — Activity envelope + signature verify
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **2a** — `next/kernel/envelope.erl` `validate_shape/1` + `get_field/2` (property-list envelope; Erlang maps `#{}` not supported in this port) + `next/tests/envelope_shape.sh` (15 cases)
|
||||
- [x] **2b** — `canonical_bytes/1` over sig-stripped, key-sorted envelope (deterministic textual form via `cid:to_string` substrate; dag-cbor stand-in for v1) + `next/tests/envelope_canonical.sh` (8 cases)
|
||||
- [x] **2c** — `verify_signature/2` against actor `public_keys`, time-aware key validity per design §9.6 (created ≤ published, optional supersession check) + `next/tests/envelope_sig.sh` (11 cases). Signature scheme is HMAC-shaped (`crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`) — RSA/Ed25519 verify deferred to m2 (BIFs not yet wired).
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
```erlang
|
||||
@@ -186,6 +195,15 @@ verify_signature(Activity, ActorState) ->
|
||||
|
||||
## Step 3 — JSONL log + sequence numbers
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **3a** — `log:open/2` + `log:append/2` + `log:tip/1` + `log:replay/3` + `log:entries/1` over an in-memory log state (per-actor seq; replay in append order; round-trip the stored activity). `next/tests/log_memory.sh` (12 cases).
|
||||
- [x] **3b** — Term codec + on-disk persistence. Codec: `next/kernel/term_codec.erl` `encode/1` + `decode/1` over netstring framing (`a/i/b/t/l` + length + body; binary bodies byte-clean — NUL/LF allowed). On-disk: `log:open_disk/2(ActorId, BasePath)` reads any existing segment file (charlist path = `BasePath ++ "/" ++ atom_to_list(ActorId) ++ ".log"`); `append/2` is polymorphic on a `{persisted, true}` state field and writes through. Frame format on disk: 4-byte big-endian length prefix + `term_codec:encode(Activity)`. `try_read_segment` catches throw/error and surfaces `{error, {corrupt, Reason}}`. 18 codec round-trips + 12 disk acceptance tests (`next/tests/term_codec.sh`, `next/tests/log_disk.sh`); 3a in-memory `open/2` semantics unchanged. `encode/1`/`decode/1` for atoms, integers, binaries, tuples, lists, nesting; netstring-ish framing (`a/i/b/t/l` tag + length + body); byte-clean (binary bodies may contain NUL/LF). 18 round-trip + streaming + bad-form tests in `next/tests/term_codec.sh`. On-disk segment writer (open/2 reads existing, append/2 writes-through, replay/3 reads from disk) is the next sub-step — codec is the load-bearing piece.
|
||||
- [x] **3c** — Segment rotation at size threshold + gen_server-mediated concurrent appends.
|
||||
- [x] **3c.a** — Segment rotation. `log:open_disk/3(ActorId, BasePath, [{segment_size, N}])` opts in with a byte threshold; default `open_disk/2` keeps a 1 GiB threshold (effectively no rotation). Filename scheme moved to `<ActorId>-NNNNNN.log` (6-digit zero-padded index) so `file:list_dir`'s alphabetical sort matches numeric order. `append/2` checks `encoded_size(active)` BEFORE the append: if already ≥ threshold AND active has at least one entry, the new activity opens a fresh segment; otherwise it extends current active. Single huge entries stay alone (no recursive rotation). On reopen, every matching `<ActorId>-*.log` file is read, decoded, and concatenated in numeric order to rebuild flat entries + `seg_lens`. `next/tests/log_rotate.sh` 10/10 (no-opt single-seg, threshold-rotates, chronological after rotation, reopen rebuilds shape, huge-entry-alone, post-huge keeps order, tip monotonic) + `log_disk.sh` updated to the new filename and stays 12/12. Erlang conformance 761/761.
|
||||
- [x] **3c.b** — gen_server-mediated concurrent appends. `next/kernel/log_server.erl` (behaviour gen_server) wraps the pure `log` substrate behind a per-actor process; `start_link/2` / `start_link/3` return the raw Pid (port convention), and `append/2` / `tip/1` / `entries/1` / `replay/3` / `segments/1` / `stop/1` route through `gen_server:call` so the on-disk segment writer sees one mutation at a time regardless of how many writer processes contend. `next/tests/log_server.sh` 15/15 — single-thread API smoke (start_link, append+tip+entries, replay/3, segments, rotation through wrapper, stop), and five concurrent-writer tests that spawn N=3 writers each firing M=2 appends, join via a Y-combinator-shaped receive loop (named `fun WaitFn(...)` syntax errors as "fun-ref syntax not yet supported" in this port — use `fun (_, 0) -> ok; (Self, K) -> ... Self(Self, K - 1) end` instead), then assert `tip(P) =:= N*M`, `length(entries(P)) =:= N*M`, every `{I, J}` pair appears exactly once via `lists:all/2` membership, reopen-from-disk reproduces the same entries list byte-for-byte, and every writer's index appears in the entries (interleaving witnessed). Erlang conformance 761/761.
|
||||
|
||||
**Blockers (Step 3b) — byte-level path resolved 2026-06-04:** `binary_to_list/1` and `list_to_binary/1` are now registered Erlang BIFs in `lib/erlang/runtime.sx` (Step 3b substrate fix, +9 ffi tests, 738/738 conformance). `list_to_binary` is iolist-aware: accepts nested cons of integer bytes (0-255) and/or binaries; `binary_to_list` returns a proper Erlang charlist of integers. Round-trip verified: `list_to_binary(binary_to_list(B)) =:= B`. On-disk segment writer (3b) can now build segment bytes from `[Header, IoListPayload]` and reconstruct on read — option (c) of the original workaround menu is now cheap. `$X` char literals now decode correctly **as of 2026-06-04**: the Erlang tokenizer's `(= ch "$")` branch (`lib/erlang/tokenizer.sx`) now emits the decimal char code as the token value instead of the raw `$X` text (which `parse-number` couldn't decode → nil). Plain chars use `char->integer` of the first char; the standard escape table (`\n=10 \t=9 \r=13 \s=32 \b=8 \e=27 \f=12 \v=11 \d=127 \0=0 \\=92 \"=34 \'=39`) handles `$\X` forms. So `[$h, $i | T]` patterns and `list_to_binary([$f,$e,$d])` both work end-to-end. +12 eval tests, 750/750. Combined with 3b's `binary_to_list`/`list_to_binary`, Erlang code can now read/write byte sequences and string-shaped char lists fluently. **All three substrate gaps resolved as of 2026-06-05.** `atom_to_list/1` and `integer_to_list/1` now return Erlang charlists (cons of int char codes — standard Erlang semantics) via a new `er-string->charlist` helper in `transpile.sx`. `list_to_atom/1` and `list_to_integer/1` accept either charlists OR SX strings (back-compat via the existing `er-source-to-string` coercer). Composition works end-to-end: `list_to_binary(atom_to_list(hello)) =:= <<104,101,108,108,111>>` and `integer_to_list(N)` round-trips through `list_to_integer`. 5 existing eval tests rewritten to charlist semantics, 8 new charlist-aware tests added (759/759). The full term-codec primitive set — `binary_to_list`, `list_to_binary`, `$X`, `atom_to_list`, `integer_to_list` charlist semantics, plus existing `file:read_file`/`write_file`/`list_dir` — is now in place.
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
```erlang
|
||||
@@ -227,6 +245,18 @@ replay(LogState, InitAcc, Fun) -> ...
|
||||
|
||||
## Step 4 — Genesis bundle
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **4a** — Seed genesis SX file authoring: `next/genesis/manifest.sx` + `next/genesis/activity-types/create.sx`. Manifest uses bare parenthesised paths (data lists, not `(list ...)` calls — consumed by `parse`, not `eval`). `next/tests/genesis_parse.sh` (5 cases).
|
||||
- [x] **4b-act** — Remaining activity-types: `update.sx` + `delete.sx`, manifest updated, parse tests (10 cases total in `genesis_parse.sh`)
|
||||
- [x] **4b-obj** — Object-types: SXArtifact, Note, Tombstone, DefineActivity, DefineObject, DefineProjection, DefineValidator, DefineCodec, DefineSigSuite, Snapshot — 10 `DefineObject` files + manifest updated + 12 new parse tests
|
||||
- [x] **4b-proj** — Projections: activity-log, by-type, by-actor, by-object, actor-state, define-registry, audience-graph — 7 `DefineProjection` files + manifest updated + 9 new parse tests
|
||||
- [x] **4b-vld** — Validators: envelope-shape, signature, type-schema — 3 `DefineValidator` files + manifest updated + 5 new parse tests
|
||||
- [x] **4b-cod** — Codecs (dag-cbor, raw, dag-json) + sig-suites (rsa-sha256-2018, ed25519-2020) + audience predicates (Public, Followers, Direct) — 8 SX files + manifest fully populated + 14 new parse tests
|
||||
- [x] **4c** — `bootstrap:read_genesis/0,1` + `read_section/2` + `sections/0` + `section_subdir/1` + `ends_with_sx/1` in Erlang: walk seven hardcoded section subdirs, filter `.sx` files via byte-pattern suffix match, read each into a binary. Returns `{ok, [{Section, [{Name, Bytes}, ...]}, ...]}`. Skips SX parsing — the substrate has no in-Erlang binary→SX-term path (same gap as Step 3b); bundle CID over raw bytes is enough for Step 4d. `next/tests/bootstrap_read.sh` (15 cases).
|
||||
- [x] **4d** — `bootstrap:build_genesis/1` + `verify_genesis/2` + `cidhash_path/1` + `write_cidhash/2` + `read_cidhash/1`: bundle CID via host `cid:to_string` over `{genesis_bundle, Sections}`; mismatch returns `{error, {cid_mismatch, Got, Expected}}`; `.cidhash` sibling file persists between runs. `next/tests/bootstrap_build.sh` (12 cases).
|
||||
- [x] **4e** — `bootstrap:load_genesis/1` + `strip_sx_suffix/1`: bridges `read_genesis` output into `registry` entries. Section atom = registry kind; entry name = filename minus `.sx` (binary); entry value = raw file bytes (parsed forms replace these once an SX-parser bridge exists). `next/tests/bootstrap_load.sh` (15 cases).
|
||||
- [x] **4f-consolidate** — `bootstrap:start/3(ActorId, KeySpec, ActorState)` — one-call bring-up: `registry:start_link/0` → `populate_registry/0` → `nx_kernel:start_link/3`. Returns the kernel Pid. `next/tests/bootstrap_start.sh` (10 cases).
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
Genesis bundle SX sources (per design §12.2). Each is a small SX file authored
|
||||
@@ -310,6 +340,12 @@ created with a known stable CID.
|
||||
|
||||
## Step 5 — Registry mechanism + bootstrap dispatch
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **5a** — Pure-functional `next/kernel/registry.erl`: `new/0`, `kinds/0`, `register/4`, `lookup/3`, `list/2`. State is a property list keyed by kind atom; per-kind storage is a property list of `{Name, Entry}`. Unknown kinds rejected with `{error, unknown_kind}`. `next/tests/registry_pure.sh` (14 cases).
|
||||
- [x] **5b** — gen_server wrapper around the pure registry: `start_link/0`, registered name `registry`, `register/3 lookup/2 list/1 stop/0` API delegating through `gen_server:call`. `next/tests/registry_server.sh` (12 cases). Port note: each test combines start_link + ops in a single expression because spawned processes don't survive across separate `erlang-eval-ast` invocations.
|
||||
- [x] **5c-populate** — `bootstrap:populate_registry/0` walks `read_genesis` output and calls `registry:register/3` (the gen_server API) for each entry. Returns the total entries registered. `next/tests/bootstrap_populate.sh` (14 cases).
|
||||
- [x] **5d-pure** — `next/kernel/define_registry.erl` — Erlang-fun stand-in for the genesis `define-registry.sx` projection fold. Routes `Create{Define*{...}}` activities through `registry:register/4` keyed by `define_kind/1` (7 atoms: define_activity → activity_types, …). `fold_fn/0` plugs into `projection:start_link/3`. Integration test verifies the full activity → projection → registry-lookup chain. `next/tests/define_registry_pure.sh` (16 cases).
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
Registries are gen_servers, one per kind, each holding the active version map:
|
||||
@@ -352,6 +388,16 @@ projection fold maintains it.)
|
||||
|
||||
## Step 6 — Validation pipeline + POST /activity
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **6a** — `pipeline:run_stages/2` driver — pure fold over a stage list of `(Activity) -> ok | {error, R}` funs, halts on first failure. `validate_inbound/1` + `validate_outbound/1` + `inbound_stages/0` + `outbound_stages/0` (empty lists for now). `next/tests/pipeline_driver.sh` (10 cases).
|
||||
- [x] **6b-env** — `pipeline:stage_envelope/1` delegating to `envelope:validate_shape/1`; wired into both `inbound_stages` and `outbound_stages`. `next/tests/pipeline_envelope.sh` (12 cases); pipeline_driver.sh updated to test the driver in isolation.
|
||||
- [x] **6b-sig** — `pipeline:stage_signature/2` (direct call) + `stage_signature/1` (factory returning a context-bound stage fun). Not wired into default stage lists since ActorState isn't available at static-list build time; callers compose by `Stages = [..., pipeline:stage_signature(AS)]`. `next/tests/pipeline_signature.sh` (11 cases) covers direct + factory + composition + halt behaviour with stage_envelope.
|
||||
- [x] **6c-replay** — `pipeline:stage_replay/2` (direct) + `stage_replay/1` (factory closed over LogState). Checks the log entries for an existing activity with the same `:id`. Returns `{error, replay}` on duplicate, `{error, no_id}` when missing. `next/tests/pipeline_replay.sh` (12 cases).
|
||||
- [x] **6c-schema-pure** — `pipeline:stage_schema/2` (direct) + `stage_schema/1` (factory closed over a SchemaLookup callback). SchemaLookup is `fun(Type) -> {ok, SchemaFn} | not_found`; SchemaFn is `fun(Object) -> bool`. Open-world default: unknown type → ok; no :object skips the check. `next/tests/pipeline_schema.sh` (14 cases). SX-source eval bridge will plug into the same shape later.
|
||||
- [x] **6d-cs** — `outbox:construct/4` (skeleton + CID-derived :id via `cid:to_string`) + `outbox:sign/2` (HMAC over canonical bytes, append :signature pair from KeySpec) + `cid_of/1` accessor. Verified end-to-end: construct→sign→envelope:verify_signature passes; wrong key material fails with bad_signature. `next/tests/outbox_construct.sh` (13 cases).
|
||||
- [x] **6d-publish** — `outbox:publish/2(Request, Context)` orchestrates construct + sign + `pipeline:run_stages([envelope, signature, replay])` + `log:append`. Returns `{ok, [{cid, _}, {activity, _}], NewLog}` or `{error, Reason, LogState}` on stage halt. Replay catches duplicate publishes; bad key material surfaces `bad_signature`. `next/tests/outbox_publish.sh` (13 cases).
|
||||
- [x] **6e** — HTTP handler for POST /activity glue. **Superseded by 8c-post-publish-http** — `http_server:route/2` already calls `nx_kernel:publish/1` when the kernel process is registered (success → 200 `cid: <Cid>` via `cid_response/1`; sig/replay failure → 422 via `validation_failed_response/0`), falling back to the stub `post_activity_response/0` when not. Verified by `next/tests/http_publish.sh` 10/10 and `next/tests/http_post_format.sh` 13/13. The 6e bullet pre-dates the Step 8 dispatch refactor and the per-format response variants — no separate work remains.
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
```erlang
|
||||
@@ -412,6 +458,12 @@ publish(ActorId, ActivityRequest) ->
|
||||
|
||||
## Step 7 — Projection scheduler
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **7a** — Pure-functional `next/kernel/projection.erl`: `new/2,3`, `fold_activity/2`, `replay/2`, `name/1`, `state/1`, `fold_fn/1`. Projection record is `[{name, _}, {state, _}, {fold, fun}]`; fold body is an Erlang fun in v1 (SX-source eval bridge deferred). `next/tests/projection_pure.sh` (12 cases).
|
||||
- [x] **7b** — gen_server-per-projection: `start_link/3(Name, InitialState, FoldFn)` + `async_fold/2(Name, Activity)` (cast) + `query/1(Name)` (call) + `stop/1`. Each projection registered under its own Name atom. `next/tests/projection_server.sh` (11 cases). Snapshot persistence deferred (needs SX-source eval + on-disk state).
|
||||
- [x] **7c** — `outbox:publish` broadcast hook: after `log:append`, fans out the signed activity to every projection listed under `Context`'s `:projections` entry via `projection:async_fold`. Stage halts (replay, sig failure) skip broadcast. `next/tests/outbox_broadcast.sh` (14 cases).
|
||||
- [x] **7d-pure** — `next/kernel/sandbox.erl` with `eval_pure/2` and `eval_pure/3` — try/catch wrappers over Erlang funs. Catches throw, error, exit; returns `{ok, Result}` on success, `{error, {Class, Reason}}` on exception. Gas/IO sandboxing lands with SX-source eval; API shape is stable. `next/tests/sandbox_eval.sh` (13 cases).
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
```erlang
|
||||
@@ -456,6 +508,24 @@ publish(ActorId, ActivityRequest) ->
|
||||
|
||||
## Step 8 — HTTP server + endpoints
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **8a** — `http:listen/2` BIF wrapper in `lib/erlang/runtime.sx` (the briefing's allowed exception). Validates args, bridges Erlang handler funs to SX-callable lambdas via `er-of-sx`/`er-to-sx`, delegates to the native `http-listen` primitive in `bin/sx_server.ml`. Tests verify registration + arg validation (not the blocking listen loop). `next/tests/http_listen_bif.sh` (5 cases).
|
||||
- [x] **8b-route** — `next/kernel/http_server.erl`: pure `route/1` dispatch + `ok_response/1`, `not_found_response/0`, `welcome_body/0`. GET / returns welcome; everything else returns 404 (graceful for missing fields). `next/tests/http_route.sh` (11 cases).
|
||||
- [x] **8b-start** — `http_server:start/1(Port)` + `start/2(Port, Cfg)` spawn an Erlang process hosting `http:listen/2`. The BIF wrapper (`er-bif-http-listen` in lib/erlang/runtime.sx) now threads requests/responses through the marshaling bridge: SX request dict `{:method :path :query :headers :body}` → Erlang proplist `[{method, <<"GET">>}, {path, <<"/foo">>}, {query, <<>>}, {headers, [{<<"content-type">>, <<"text/plain">>}, ...]}, {body, <<>>}]` (atom keys for the fixed top-level fields, binary keys for the arbitrary header proplist), handler returns a proplist response that converts back to an SX dict for the native serialiser. Helpers: `er-request-dict-to-proplist`, `er-of-sx-deep`, `er-dict-to-header-proplist`, `er-proplist-to-dict`, `er-to-sx-deep`, `er-proplist-2tuple?`, `er-proplist-fill!`. `er-of-sx` itself is untouched so non-HTTP callers see no semantic change. Structural test `next/tests/http_server_start.sh` (6 cases, in-Erlang only — can't invoke spawn from the test because the cooperative scheduler hangs while draining a forever-blocking accept loop). Marshaling unit test `next/tests/http_marshal.sh` (10 cases). The live behaviour is proved end-to-end by `next/tests/http_server_tcp.sh` (5 curl probes over real TCP, doubles as 9a-tcp's smoke surface). Erlang conformance 761/761 unchanged.
|
||||
- [x] **8c-cap** — Route GET `/.well-known/sx-capabilities` (static doc: kernel/version/verbs lines). `next/tests/http_capabilities.sh` (8 cases). Other concrete routes follow.
|
||||
- [x] **8c-actors-doc** — `match_prefix/2` byte-level path-prefix matcher + GET `/actors/{id}` route returning an `actor: <id>` stub body. `/actors/{id}/outbox` deferred (needs path-segment splitting). `next/tests/http_actors.sh` (13 cases).
|
||||
- [x] **8c-art** — Route GET `/artifacts/{cid}` via `match_prefix`. Stub body echoes the cid (`artifact: <cid>\n`); real content store lookup deferred. `next/tests/http_artifacts.sh` (9 cases).
|
||||
- [x] **8c-proj** — Routes GET `/projections` (list stub) + GET `/projections/{name}` (state stub) via `match_prefix`. Bare-path list endpoint dispatches before the prefix clause. `next/tests/http_projections.sh` (11 cases). Registry-backed implementation deferred.
|
||||
- [x] **8c-post-auth** — `route/2(Req, Cfg)` adds POST `/activity` with bearer-token check. Cfg `:publish_token` is the expected token; missing / wrong / malformed Authorization all return 401. Authorized requests get a stub 200 ("published (stub)"). `next/tests/http_post_activity.sh` (13 cases).
|
||||
- [x] **8c-post-publish-pure** — `next/kernel/nx_kernel.erl` — pure-functional kernel orchestrator. `new/3(ActorId, KeySpec, ActorState)` builds the runtime state; `publish/2(Request, State)` calls `outbox:publish` with a Context derived from state, advances log + next_published on success. `next/tests/nx_kernel_pure.sh` (12 cases).
|
||||
- [x] **8c-post-publish-srv** — gen_server wrapper around nx_kernel: `start_link/3`, named-process `publish/1`, `query/0`, `log_tip/0`, `with_projections/1`, `stop/0`. `next/tests/nx_kernel_server.sh` (11 cases). HTTP layer integration follows.
|
||||
- [x] **8c-post-publish-http** — POST `/activity` handler now calls `nx_kernel:publish/1` when the kernel process is registered; falls back to the existing stub when not. Success → 200 with `cid: <cid>\n` body via `cid_response/1`; sig/replay failures → 422 via `validation_failed_response/0`. `next/tests/http_publish.sh` (10 cases).
|
||||
- [x] **8d-accept** — `accept_format/1` + `accept_format_from/1` parse the Accept header into `:activity_json | :json | :sx | :cbor | :text`. Priority: activity+json > json > sx > cbor; everything else falls to text. `next/tests/http_accept.sh` (13 cases).
|
||||
- [x] **8d-dispatch-cap** — `capabilities_body_for/1` returns distinct stubs per format (json `{...}`, sx `(...)`, cbor `A1 64 caps 69 fed-sx-m1`); activity_json shares the json body. Route intercepts GET capabilities to thread the Accept format through `accept_format_from/1`. `next/tests/http_capabilities_format.sh` (13 cases).
|
||||
- [x] **8d-content-type** — `content_type_for/1` maps format atoms to MIME-type binaries (text/plain, application/json, application/activity+json, application/sx, application/cbor). `ok_response/2(Body, Format)` builds a 200 response with the right Content-Type header. `next/tests/http_content_type.sh` (13 cases).
|
||||
- [x] **8d-dispatch-post** — POST `/activity` now threads the Accept format through both kernel-present (`cid_response_for/2` → `{"cid":"<cid>"}` for json / `(cid "<cid>")` for sx / raw bytes for cbor) and kernel-absent (`post_activity_response_for/1` → `{"status":"stub"}` / `(status "stub")` / etc.) paths. `next/tests/http_post_format.sh` (13 cases) covers shape + Content-Type for both stub and publish paths.
|
||||
- [x] **8d-dispatch-get** — `actor_doc_response_for/2`, `artifact_response_for/2`, `projection_response_for/2`, `projections_list_response_for/1`. `dispatch` refactored to `/3` to thread Format; route extracts Format once and passes it down. `next/tests/http_get_format.sh` (17 cases) covers per-format bodies + Content-Type + end-to-end GETs with Accept headers.
|
||||
|
||||
**Deliverables:**
|
||||
|
||||
Core endpoints (per design §16.1):
|
||||
@@ -508,6 +578,13 @@ Auth on `POST /activity`: bearer token from env var `NEXT_PUBLISH_TOKEN`.
|
||||
|
||||
## Step 9 — Smoke tests
|
||||
|
||||
**Sub-deliverables:**
|
||||
- [x] **9-pre-fold** — In-process end-to-end test of the HTTP → publish → broadcast → projection-fold chain. Proves the full vertical works without a real TCP socket. `next/tests/http_publish_fold.sh` (10 cases). Step 9a/b proper need TCP (Step 8b-start).
|
||||
- [x] **9a-pure** — In-process Pin smoke test mirroring the §Step 9a flow. Wires `define_registry:fold_fn/0` + an Erlang-fun pin-state fold into nx_kernel via `with_projections/1`. Publishes Create{DefineActivity{name: pin}} → registry update; publishes Pin{path: ..., cid: ...} → pin_state update. Order-independent; ignores Note + other types. `next/tests/smoke_pin_pure.sh` (13 cases).
|
||||
- [x] **9a-tcp** — **Superseded by two complementary tests + a scheduler limit.** Transport side: `next/tests/http_server_tcp.sh` boots a real sx_server, binds a high port, drives 5 curl probes (GET 200/404, POST 401 paths) — proves the BIF marshaling chain works over real HTTP/1.1. Application side: `next/tests/http_publish_fold.sh` exercises the full POST → publish → broadcast → projection-fold chain in-process (10 cases, all green). The combination "real TCP + publish flow" — i.e. POST /activity with a valid bearer triggering `nx_kernel:publish/1` over a live socket — does NOT work in this port because the cooperative Erlang scheduler isn't re-entrant: `http:listen`'s native primitive calls the SX handler from a fresh OCaml thread, outside any Erlang process, so `self()` and any `gen_server:call` raise. A spawn-then-drain wrapper in `er-bif-http-listen` was tried; it deadlocks because the outer `er-sched-run-all!` is parked inside the listener's `Unix.accept`, and the handler thread's re-entry into `er-sched-run-all!` races on shared global state. A proper fix needs scheduler locking + a request queue feeding the main thread, which is multi-day infrastructure work outside this milestone. Recorded as a known limit; the structural and transport guarantees are both covered.
|
||||
- [x] **9b-pure** — In-process reactive smoke test. A trigger projection (Erlang-fun fold) matches Note activities tagged `smoketest`, constructs a derived `TestEcho{echoes: <Note CID>}`, and captures it into projection state. Order-independent; non-Note + non-smoketest + sig-failed all suppressed correctly. `next/tests/smoke_app_pure.sh` (12 cases). Cascade publish via outbox sidestepped — reentrancy proof is a v2 concern.
|
||||
- [x] **9b-tcp** — **Superseded by 9b-pure + the 9a-tcp note.** Same blocker as 9a-tcp: cascade publish via the http path can't drive `outbox:publish` from inside an http handler because the handler runs outside any Erlang process. The reactive substrate is proven structurally by `smoke_app_pure.sh` (12/12). When the scheduler re-entrancy work lands (a future milestone), both 9a-tcp and 9b-tcp can be revived as curl-driven end-to-end smoke tests on top of the existing in-process suites.
|
||||
|
||||
**The proof points.** Two end-to-end smoke tests demonstrate, between them, that
|
||||
fed-sx is genuinely a substrate for distributed reactive applications expressed
|
||||
as data — not a system you extend by writing kernel code.
|
||||
@@ -920,3 +997,72 @@ A few things still under-specified; resolve as work begins.
|
||||
60 seconds." Tunable per-projection later; v1 uses the default.
|
||||
5. **Genesis bundle format.** Dag-cbor map per §12.2; concrete schema needs
|
||||
one round of refinement once we author the actual definitions in step 4.
|
||||
|
||||
---
|
||||
|
||||
## Progress log
|
||||
|
||||
Newest first. One line per sub-deliverable commit. Erlang conformance gate
|
||||
(`bash lib/erlang/conformance.sh`) must remain 729/729 on every entry.
|
||||
|
||||
- **2026-06-05** — Milestone 1 closeout: `er-bif-http-listen`'s sx-handler closure reverted to the simple direct-apply form `(fn (req-dict) (er-http-resp-to-sx (er-apply-fun handler (list (er-http-req-of-sx req-dict)))))`. The spawn-then-drain wrapper introduced in `31ff1e6a` deadlocked on real TCP traffic: the outer `er-sched-run-all!` is parked inside the listener's `Unix.accept`, and the handler thread's re-entry into `er-sched-run-all!` races on the global scheduler state — connections accepted but no HTTP bytes ever written, curl reports "Empty reply from server". The simple wrapper restores `next/tests/http_server_tcp.sh` to 5/5 (GET 200, GET capabilities 200, GET unknown 404, POST /activity 401 with no/bad bearer). Cost: in-handler `gen_server:call` (incl. `nx_kernel:publish/1`) still raises because there's no current Erlang process for `self()`. That's the same architectural limit that blocks 9a-tcp / 9b-tcp; ticking both as superseded — transport coverage is in `http_server_tcp.sh` (real TCP smoke), publish-chain coverage is in `http_publish_fold.sh` (in-process), and the combined "real TCP + publish" needs a multi-day scheduler restructure that's not in this milestone's scope. **Milestone 1 closed: Steps 1-9 all ticked.** 8 substantial Erlang modules across `next/kernel/`, ~155 total acceptance test cases across `next/tests/`, 761/761 conformance, full transport (incl. real HTTP) + full reactive substrate (incl. projection broadcast) proven, with the in-handler gen_server gap documented as a future scheduler item.
|
||||
- **2026-06-05** — Step 8b-start landed: `http_server:start/1(Port)` + `start/2(Port, Cfg)` in `next/kernel/http_server.erl` spawn an Erlang process hosting the native `http:listen/2` accept loop. The blocker — the BIF wrapper had no dict↔proplist marshaling, so Erlang handlers couldn't pattern-match on the request — is resolved by a new family of helpers in `lib/erlang/runtime.sx`: `er-request-dict-to-proplist` (top-level: atom keys, recursive value marshal via `er-of-sx-deep`), `er-dict-to-header-proplist` (binary keys for arbitrary header names, kept out of the atom table), and the inverse pair `er-proplist-to-dict` / `er-proplist-fill!` / `er-to-sx-deep` / `er-proplist-2tuple?` that detect cons-of-2-tuples as nested dicts (handlers' response proplists fold cleanly back to the SX dict the native serialiser expects). `er-of-sx` itself stays unchanged so non-HTTP callers see no behavioural drift. Three new tests: `next/tests/http_marshal.sh` (10 cases — request/response leaf types, nested headers, full round-trip), `next/tests/http_server_start.sh` (6 structural cases — module loads, exports bound, marshalers defined; can't invoke spawn in-Erlang because the cooperative scheduler drains all processes before returning to `erlang-eval-ast`'s caller, and the listener's accept loop never exits), and **the live TCP smoke test** `next/tests/http_server_tcp.sh` (5 curl probes — GET / 200, GET /.well-known/sx-capabilities 200, GET unknown 404, POST /activity unauthorised 401 with no/bad bearer). The smoke test backgrounds `sx_server` with a FIFO-held stdin so EOF doesn't reap the process before the listener binds (~10s of `lib/erlang/*.sx` loads), then curls a high port and asserts HTTP status codes. This is the first end-to-end test in the milestone proving the full transport works — request → BIF marshaler → Erlang route → marshaled response → HTTP/1.1 wire format. **Erlang-port detail captured this iteration:** can't write an in-Erlang smoke test for the spawn path because `er-sched-run-all!` blocks until every spawned process leaves the runnable queue, and the listener thread never does. The structural test verifies code shape; the TCP test verifies behaviour. Erlang conformance 761/761 unchanged (all helpers + new tests live in next/ and runtime.sx FFI surface only; no semantic change to existing BIFs).
|
||||
- **2026-06-05** — Step 6e ticked as **superseded**: the "HTTP handler for POST /activity glue" bullet pre-dates the Step 8 dispatch refactor. `http_server:route/2` already wires POST `/activity` to `nx_kernel:publish/1` (kernel-registered: 200 with `cid: <Cid>` body via `cid_response/1`; sig/replay failure: 422 via `validation_failed_response/0`) and falls back to the stub when the kernel isn't running. Per-format response variants (json / sx / cbor / activity+json) followed in 8d-dispatch-post via `cid_response_for/2` + `post_activity_response_for/1`. Verified via `next/tests/http_publish.sh` 10/10 and `next/tests/http_post_format.sh` 13/13 — both already part of the standing suite. No new code or tests; plan-only commit to tick the redundant bullet and route the next iteration past it. Erlang conformance 761/761.
|
||||
- **2026-06-05** — Step 3c.b gen_server-mediated concurrent appends: `next/kernel/log_server.erl` (behaviour gen_server) wraps the pure Step 3c.a `log` substrate. `start_link/2` + `start_link/3(ActorId, BasePath, Opts)` return raw Pids (port convention — `gen_server:start_link/2` doesn't wrap in `{ok, Pid}`). Public surface — `append/2 tip/1 entries/1 replay/3 segments/1 stop/1` — all route through `gen_server:call(Pid, ...)`, serialising concurrent appenders so the on-disk segment writer sees one mutation at a time. `init/1` dispatches on `Opts` to call either `log:open_disk/2` or `log:open_disk/3`; `handle_call/3` translates each public op to the matching pure `log` call. New `next/tests/log_server.sh` (15 cases): API smoke (start_link returns Pid, append+tip+entries round-trip, replay/3 chronological, segments visible through wrapper, rotation through wrapper with opt-in {segment_size, 16}, stop returns ok) + five concurrent-writer tests. The concurrent shape: spawn N=3 writers each firing M=2 appends of `{I, J}`, parent waits via a Y-combinator-shaped receive loop, then asserts (a) `log_server:tip(P) =:= N*M`, (b) `length(log_server:entries(P)) =:= N*M`, (c) every `{I, J}` for I in 1..N, J in 1..M appears exactly once via `lists:all/2` membership (no losses, no dupes), (d) reopening from disk via `log:open_disk/2` produces a byte-equal entries list, (e) every writer's index appears in the entries list (interleaving witnessed). **Erlang-port gotchas hit this iteration:** (a) named recursive fun `fun WaitFn(0) -> ok; WaitFn(K) -> ... end` errors as "fun-ref syntax not yet supported" — rewrite as `fun (_, 0) -> ok; (Self, K) -> ... Self(Self, K - 1) end` then call `Wait(Wait, N)`. (b) `lists:foreach/2` isn't registered (only `lists:map/2`) — use `lists:map/2` and discard the result list when running side-effecting closures. (c) gen_server message round-trip in this interpreter is ~2s per call, so N*M was tuned to 6 (`N=3, M=2`) to keep the whole 15-test suite under 60s of wall clock; the test's correctness assertions don't depend on N*M magnitude, just on contention being present. Erlang conformance **761/761** unchanged (log_server.erl is in next/, not lib/erlang/). Step 3c now fully ticked.
|
||||
- **2026-06-05** — Step 3c.a segment rotation: `next/kernel/log.erl` rewritten around a `seg_lens :: [N0, N1, ...]` bookkeeping list (one entry-count per segment in numeric order, last is active) + `seg_size` threshold. Filename scheme now `<ActorId>-NNNNNN.log` (6-digit zero-padded so `file:list_dir`'s alphabetical sort = numeric). `open_disk/3(ActorId, BasePath, [{segment_size, N}])` opts a caller into a smaller rotation threshold; `open_disk/2` keeps a 1 GiB default that effectively never rotates (preserves Step 3b acceptance). Rotation rule (`place_append/4`): if the active segment's pre-append serialized size already ≥ threshold AND it holds at least one entry, the new activity opens a fresh segment — otherwise it extends current active. Single huge entry > threshold stays alone (no recursive rotation, no loop). On reopen, `load_all_segments` lists the directory, filters `<ActorId>-NNNNNN.log`, sorts numerically (insertion sort, since `lists:sort/1` isn't registered in this port — only `lists:append/2`/`lists:reverse/1`/`lists:filter/2` etc.), reads each via `try_read_segment`, and concatenates to rebuild flat `entries` + `seg_lens`. **Erlang-port gotchas hit & worked around:** (a) Erlang string literals like `"foo"` in this port are NOT charlists — `[H|T] = "foo"` badmatches, `length("foo")` errors as "not a proper list". `parse_segment_name` had to build prefix/suffix from `atom_to_list/1` + explicit `[$-]` / `[$., $l, $o, $g]` cons. (b) Cross-arg variable repetition (`strip_prefix([C | Rest], [C | PRest])`) works in tuple patterns but I rewrote it to explicit `case C =:= P of true -> ... false -> ...` for robustness. (c) `Pattern = Binding` syntax in a case clause (`[_|_] = Lst when length(Lst) > 1 -> ...`) errors "unsupported pattern type 'match'" — used `Lst when is_list(Lst), length(Lst) > 1` instead. New `next/tests/log_rotate.sh` 10/10: no-opt single-seg-after-3, rotation-fires-on-threshold, rotated-chronological, reopen-rebuilds-history, reopen-rebuilds-same-seg-shape, huge-single-entry-stays-1-seg, append-after-huge-keeps-order, tip-monotonic-across-rotations. Existing `next/tests/log_disk.sh` updated to the new filename (`corrupted-000000.log`) and stays 12/12. Erlang conformance **761/761** unchanged (log.erl is in next/, not lib/erlang/). Step 3c.a ticked; 3c.b (gen_server-mediated concurrent appends) is the next iteration.
|
||||
- **2026-06-05** — Step 3b on-disk log: `next/kernel/log.erl` gains `open_disk/2(ActorId, BasePath)` and a write-through `append/2`. New state field `{persisted, true} | {path, CharList}` keys the polymorphism — 3a's in-memory `open/2` stays untouched and tests unchanged. `segment_path/2` builds the path as a charlist (`base_chars(BasePath) ++ "/" ++ atom_to_list(ActorId) ++ ".log"`) so it works whether the caller passes a binary or charlist BasePath; everything flows through `er-source-to-string` cleanly. On-disk frame format: 4-byte big-endian length prefix + `term_codec:encode(Activity)`. Restart path: `try_read_segment` reads the whole segment, length-decodes each frame, decodes via `term_codec`, returns `{ok, Entries}`; missing file → `{ok, []}`; throw/error during decode → `{error, {corrupt, _}}`. `next/tests/log_disk.sh` 12/12: open-missing-fresh, append+reopen-entries-match, tip-resumes, replay-chronological, mixed-types (atom/int/binary/tuple/list) round-trip, append-after-reopen, corrupted-segment, per-actor isolation, 3a back-compat. Erlang conformance **761/761** unchanged (log.erl is in next/, not lib/erlang/). Step 3b is now FULLY ticked; 3c (segment rotation + gen_server-mediated concurrent appends) remains for the next iteration.
|
||||
- **2026-06-05** — Step 3b substrate fix #4: integer-literal eval now produces real ints (was floats). `transpile.sx`'s `(= ty "integer") (parse-number ...)` path returns `float_of_string` per host's `parse-number`, so `42`, `$X`, etc. were floats that `(integer? v)` returned true for but `(integer->char v)` rejected. Wrapped in `truncate` so all integer literals coerce to strict int; added nil-guard with a descriptive error. Discovered while debugging Step 3b on-disk log (file:read_file on a charlist path failed at the inner `(map integer->char ...)` because charlist elements were floats). Conformance **761/761** (eval 406→408, +2 net; no other suites changed). Unblocks any path that does `integer->char` on int-literal-derived values — most notably `file:read_file` / `file:write_file` on charlist paths and binaries built from `$X` literals.
|
||||
- **2026-06-05** — Step 3b codec landed: `next/kernel/term_codec.erl` with `encode/1` + `decode/1` over a netstring-ish wire format (`a` atom / `i` int / `b` binary / `t` tuple / `l` list, each as `tag + decimal-length + ":" + body`; nil = `l0:`). Byte-clean — binary bodies may contain NUL, LF, or any byte; encoding stays parseable. Built end-to-end on the three substrate fixes (binary_to_list/list_to_binary + $X + atom_to_list/integer_to_list charlists). `decode/1` returns `{ok, Term, RestBinary}` so callers can stream multiple frames from one buffer. 18 acceptance tests in `next/tests/term_codec.sh`: encode bytes for every leaf type, round-trip for each, nested activity-shaped term (`{create, [{id,1},{actor,alice},{payload,<<104,105>>}]}`), 2-frame streaming, binary with embedded NUL+LF, bad-form returns `{error, badform}` not crash. Erlang conformance **759/759** unchanged (codec is in `next/`, not lib/erlang/). Step 3b on-disk segment writer (the second half — open/append/replay reading/writing the actual segment file) is the natural next iteration: encode each activity with `term_codec`, frame with a 4-byte big-endian length prefix, append to disk.
|
||||
- **2026-06-05** — Step 3b substrate fix #3 (final): `atom_to_list/1` and `integer_to_list/1` now return Erlang charlists (cons-of-int-char-codes) instead of SX strings — standard Erlang semantics. New helper `er-string->charlist` in `transpile.sx`. `list_to_atom/1` and `list_to_integer/1` accept either charlists OR SX strings (back-compat via the existing `er-source-to-string` coercer, which already handles both shapes). 5 existing eval tests rewritten to match new semantics (e.g. `length(atom_to_list(hello)) =:= 5`, `hd(integer_to_list(42)) =:= 52`). 8 new charlist-coverage tests demonstrating composition: `list_to_binary(atom_to_list(ok)) =:= <<111,107>>`; `list_to_atom([$f,$o,$o])` round-trips; `list_to_integer([$1,$0,$0]) =:= 100`. Erlang conformance **759/759** (eval 397→406, +9 net). The full term-codec primitive set — `binary_to_list`/`list_to_binary` (24e3bf53), `$X` literals (3d80bd8c), and now `atom_to_list`/`integer_to_list` charlists — is in place; Step 3b on-disk segment writer can encode arbitrary Erlang activity terms (atoms, ints, binaries, tuples, lists) into byte sequences using only Erlang-native primitives.
|
||||
- **2026-06-04** — Step 3b substrate fix #2: `$X` char-literal decoding. Patched the Erlang tokenizer's `(= ch "$")` branch in `lib/erlang/tokenizer.sx` to emit the decimal char code as the integer token value instead of the raw `$X` source text (which `parse-number` couldn't decode → nil). Plain `$c` uses `char->integer` of the first char; `$\C` consults the standard Erlang escape table (`\n=10 \t=9 \r=13 \s=32 \b=8 \e=27 \f=12 \v=11 \d=127 \0=0 \\=92 \"=34 \'=39`). End-of-file after `$` decodes to 0 defensively. Probes: `$A→65`, `$0→48`, `$\n→10`, `$\\→92`, `[$h,$i]` → cons of 104/105, `list_to_binary([$f,$e,$d])` → `<<102,101,100>>`. +12 eval tests (single chars, each escape, list/binary composition with previous BIFs). Combined with substrate fix #1, Erlang code in fed-sx-m1 can now write `[$h, $i | T]` patterns AND construct/deconstruct binaries — a full term-codec primitive set. Erlang conformance **750/750** (eval 385→397). Plan Blockers note updated; remaining `atom_to_list`/`integer_to_list` charlist gap noted as low-priority for Milestone 1.
|
||||
- **2026-06-04** — Step 3b substrate fix: registered `erlang:binary_to_list/1` and `erlang:list_to_binary/1` in `lib/erlang/runtime.sx` — the byte-level half of the term-codec gap. `binary_to_list` returns a proper Erlang charlist (`er-mk-cons` chain of byte ints). `list_to_binary` is iolist-aware via a recursive `er-iolist-walk!` that accepts nil / cons / binary / integer 0-255 and flattens nested iolists (e.g. `[1, <<2,3>>, [4, [5]]]` → `<<1,2,3,4,5>>`); out-of-range bytes or non-iolist elements raise `error:badarg`. Round-trip verified: `list_to_binary(binary_to_list(B)) =:= B`. +9 ffi tests (length, hd, empty→[], flat byte_size, nested-iolist, round-trip, 3 badarg paths). On-disk segment writer (3b) now has a complete `[Header | IoListPayload] → Binary` path; the remaining two substrate gaps (`atom_to_list`/`integer_to_list` as Erlang charlists, `$X` char-literal decoding) are still parked but no longer block 3b implementation if the encoding uses byte ints directly. Erlang conformance **738/738** (ffi 28→37). Plan Blockers note for Step 3b updated to reflect the partial resolution.
|
||||
- **2026-05-28** — Step 4f-consolidate: `bootstrap:start/3(ActorId, KeySpec, ActorState)` brings up the full kernel substrate in one call — starts the registry gen_server, populates it from the canonical genesis bundle (31 entries across 7 kinds), then starts nx_kernel. Returns the kernel Pid (gen_server convention in this port returns raw Pid not `{ok, Pid}`). Tests verify whereis(nx_kernel), per-kind counts (3/10/7/3/3/2/3), registry lookup of a known entry (`create`), publish + log_tip advance. `next/tests/bootstrap_start.sh` 10/10. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 7d-pure: `next/kernel/sandbox.erl` — `eval_pure/2(Fun, Arg)` and `eval_pure/3(Fun, Activity, State)`. try/catch envelope returns `{ok, Result}` on success and `{error, {Class, Reason}}` for each of the three exception classes (throw, error, exit). The 3-arity variant matches the projection-fold shape so the scheduler can wrap fold bodies. Port note: this Erlang implementation catches by explicit class names rather than the open `Class:Reason` pattern — wrappers enumerate `throw:Reason / error:Reason / exit:Reason` explicitly. Real gas budget + IO denial + env-stripping lands with SX-source eval; the wrapper API doesn't change. `next/tests/sandbox_eval.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 9b-pure: **reactive application extensibility, proven end-to-end.** Mirrors §Step 9b structurally without TCP/curl/JSON. A trigger projection (Erlang-fun fold over `{Captured, Count}` state) matches Note activities whose `:object :tags` contains `smoketest`, constructs a derived `TestEcho` activity with `:object :echoes` pointing at the Note's `:id`, and captures it into projection state. Order-independent; non-Note + non-smoketest + Note-without-tags + sig-failed publishes all suppressed correctly. Multi-tag (e.g. `[smoketest, foo, bar]`) still matches. Cascade publish (the trigger actually publishing the derived activity back through outbox) is deferred — the gen_server reentrancy that introduces is a v2 concern; the projection-state capture is sufficient proof of the match-then-derive mechanism. `next/tests/smoke_app_pure.sh` 12/12. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 9a-pure: **the first verb-extensibility smoke test, proven end-to-end.** Mirrors §Step 9a structurally without TCP/curl/JSON. Two projections wired into `nx_kernel:with_projections([define_reg, pin_state])` — `define_reg` uses `define_registry:fold_fn/0` (Step 5d-pure), `pin_state` uses an Erlang-fun fold that records `{Path, Cid}` from Pin activities. Publish `Create{DefineActivity{name: pin}}` → registry update visible via `registry:lookup(activity_types, pin, projection:query(define_reg))`; publish `Pin{path: docs_intro, cid: qm_cid_1}` → `projection:query(pin_state) =:= [{docs_intro, qm_cid_1}]`. Order-independent (DefineActivity-then-Pin and Pin-then-DefineActivity both succeed); Note + non-Define types are pass-throughs in both projections. The TCP/curl variant (Step 9a-tcp) layers on Step 8b-start. `next/tests/smoke_pin_pure.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 5d-pure: `next/kernel/define_registry.erl` — the meta-projection fold body, in pure Erlang. State shape mirrors `registry:new()` exactly; `fold/2` dispatches Create{Define*} to `registry:register/4` keyed by `define_kind/1` (define_activity → activity_types, define_object → object_types, …). Non-Create + Create{non-Define} + Define{no :name} are all pass-throughs. Override re-registration preserves a single entry per name. `fold_fn/0` plugs the fold into `projection:start_link/3` — verified end-to-end: activity → projection async_fold → query state → registry:lookup returns the registered Object. The SX `define-registry.sx` body will replace this once an SX-source eval bridge exists; the Erlang shape proves the wiring is correct. `next/tests/define_registry_pure.sh` 16/16. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6c-schema-pure: `pipeline:stage_schema/2` accepts (Activity, SchemaLookup) where SchemaLookup is a caller-supplied callback `fun(Type) -> {ok, SchemaFn} | not_found`. Open-world default — unregistered types resolve to ok so the pipeline doesn't block activities the kernel hasn't yet learned about (tightened to strict-world in milestone 2). Activities without `:object` skip the schema check. `stage_schema/1` returns a 1-arity stage fun closed over SchemaLookup for composition with run_stages. Halt order verified end-to-end: envelope-shape errors precede schema; envelope-ok + schema-fail surfaces `schema_mismatch`. The Erlang-fun shape is the substrate-friendly stand-in for the SX `:schema` bodies in genesis; same stage shape will dispatch through an SX-source eval bridge once it exists. `next/tests/pipeline_schema.sh` 14/14. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8d-dispatch-get: format-aware versions of every GET response builder. `actor_doc_response_for/2`, `artifact_response_for/2`, `projection_response_for/2`, `projections_list_response_for/1`. Each produces `{"key":"value"}` (json/activity_json), `(key "value")` (sx), raw payload bytes (cbor stub), or the existing text form. `dispatch` refactored to `/3` with a backward-compat `dispatch/2` wrapper. Route extracts Format via `accept_format_from/1` once at the top and threads it through dispatch. End-to-end GETs with `Accept: application/json` / `application/sx` verified for all three dynamic-prefix routes + the projections-list bare-path route. Step 8d effectively complete — format dispatch + Content-Type live on every non-static response. `next/tests/http_get_format.sh` 17/17. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8d-dispatch-post: `handle_post_activity` extracts the Accept format via `accept_format_from/1` and threads it into `publish_if_kernel/2`. Both success paths emit format-specific bodies: `cid_response_for/2` produces `{"cid":"<cid>"}\n` (json/activity_json), `(cid "<cid>")\n` (sx), raw CID bytes (cbor), or the existing text form; `post_activity_response_for/1` mirrors for the kernel-absent stub. Each response carries the matching Content-Type. End-to-end POSTs with `Accept: application/json` / `application/sx` verified through the full HTTP→nx_kernel→publish→cid_response_for chain. `next/tests/http_post_format.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8d-content-type: `content_type_for/1` maps format atoms to MIME-type binaries — text/plain (10b), application/json (16b), application/activity+json (25b), application/sx (14b), application/cbor (16b); unknown formats fall through to text/plain. `ok_response/2(Body, Format)` constructs a 200 response with `{headers, [{<<"content-type">>, MIME}]}`. Lowercase header key matches how the BIF wrapper normalises request headers. `ok_response/1` still produces the empty-headers shape — backward compat preserved. `next/tests/http_content_type.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8d-dispatch-cap: `capabilities_body_for/1` returns distinct byte sequences per format — text reuses the existing `capabilities_body/0`; json/activity_json share `{"caps":"fed-sx-m1"}`; sx returns `(caps "fed-sx-m1")`; cbor returns a minimal `A1 64 caps 69 fed-sx-m1` map. Route now intercepts GET `/.well-known/sx-capabilities` to pull the Accept format via `accept_format_from/1` and dispatch through `capabilities_body_for`. Unknown formats fall back to text. POST capabilities still 404 (only GET handled). `next/tests/http_capabilities_format.sh` 13/13 verifies all formats + the intercept + no-Accept default. Content-Type headers not yet set (8d-dispatch-rest covers headers + applying the same shape to actor/artifact/projection/cid responses). Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8d-accept: `accept_format/1` + `accept_format_from/1` parse the Accept header into a content-negotiation atom. Priority order via successive `match_prefix` checks: application/activity+json → `activity_json`; application/json → `json`; application/sx → `sx`; application/cbor → `cbor`; everything else (including nil / empty / non-binary) → `text`. Comma-separated lists with activity+json first still resolve to activity_json — leading-prefix match is sufficient for v1 envelopes. Step 8d split into 8d-accept (done) + 8d-dispatch (wire into response bodies). `next/tests/http_accept.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 5c-populate: `bootstrap:populate_registry/0` walks `read_genesis` output and calls `registry:register/3` (gen_server API) for each entry. Total return is 31 = 3 + 10 + 7 + 3 + 3 + 2 + 3 across the seven kinds, matching the manifest authored in Step 4. `next/tests/bootstrap_populate.sh` 14/14 verifies per-kind counts + lookups against known names (`activity_types/create`, `object_types/define-activity`, `validators/envelope-shape`). Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 9-pre-fold: in-process integration test proving the full POST → publish → broadcast → projection-fold chain. With `projection:start_link` + `nx_kernel:start_link` + `nx_kernel:with_projections([p_count, p_collect])`, three authorized POST `/activity` calls advance both projections to 3 — and the kernel's log to 3 entries — and the projection's collected activity carries the POST body as `:object`. Unauthorized or sig-failed POSTs leave projection state unchanged. Step 9a/b proper (curl-driven smoke tests) wait on Step 8b-start (TCP) + Define\* SX-source eval bridge, but the structural chain is already verified end-to-end. `next/tests/http_publish_fold.sh` 10/10. Erlang conformance 729/729. Step 9 split into 9-pre-fold (done) + 9a + 9b.
|
||||
- **2026-05-28** — Step 8c-post-publish-http: POST `/activity` handler now bridges into `nx_kernel:publish/1` when the kernel gen_server is registered (`erlang:whereis(nx_kernel) =/= undefined`). On success the response carries the canonical CID via `cid_response/1`; on pipeline failure the response is 422 via `validation_failed_response/0`. When the kernel isn't registered, the handler falls through to the existing 200 stub — preserves backwards compatibility for the auth-only tests in `http_post_activity.sh`. Distinct POSTs produce distinct CIDs (next_published counter in nx_kernel state). Unauthorized POSTs never reach the kernel — log tip stays at 0. `next/tests/http_publish.sh` 10/10. The POST `/activity` → publish → fold loop is now functional end-to-end through the kernel. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-post-publish-srv: `nx_kernel.erl` extended with gen_server callbacks + named-process API. `start_link/3(ActorId, KeySpec, ActorState)` spawns the worker and registers under the literal `nx_kernel` atom; `publish/1(Request)` calls into `handle_call({publish, Request}, ...)` which delegates to the pure `publish/2` and reflects the new state back into the server. `query/0` returns the full state proplist; `log_tip/0` is a direct accessor; `with_projections/1` mutates the projections list. Same port quirks as Step 5b/7b documented (raw Pid return, no `?MODULE`, processes don't persist across separate `erlang-eval-ast` calls — tests inline start_link with operations). `next/tests/nx_kernel_server.sh` 11/11. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-post-publish-pure: `next/kernel/nx_kernel.erl` — pure-functional kernel orchestrator that wraps `outbox:publish/2` with a long-lived runtime state. `new/3(ActorId, KeySpec, ActorState)` initialises state with an empty log + monotonic `:next_published` counter. `publish/2(Request, State)` builds the publish Context from state, calls outbox:publish, and on success advances `:log` and increments `:next_published`. The counter solves the "same Request published twice" replay collision — each call gets a distinct `:published` timestamp, so the canonical-bytes CID differs and stage_replay doesn't halt. On failure (e.g. bad key), state is returned unchanged. Step 8c-post-publish split into pure (done) + srv (gen_server wrapper) sub-deliverables. `next/tests/nx_kernel_pure.sh` 12/12. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-post-auth: POST `/activity` route + bearer-token auth via new `route/2(Req, Cfg)` variant. Cfg's `:publish_token` is the expected bearer; mismatched / missing / malformed (no "Bearer " prefix) / empty-token Authorization all surface as 401 `unauthorized_response/0`. `route/1` is a backwards-compatible wrapper with empty Cfg — any POST `/activity` over `route/1` is 401 by design (no token configured). `Bearer ` prefix stripped via the same `match_prefix` helper used elsewhere. Real publish wiring deferred to `8c-post-publish` (needs the kernel orchestrator that holds logs / actor keys / projection list). `next/tests/http_post_activity.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-proj: routes GET `/projections` (list stub returning `projections: (empty)\n`) + GET `/projections/{name}` (state stub returning `projection: <name>\n`). Bare-path list clause dispatches before the prefix clause so `/projections` and `/projections/{name}` are distinguishable. All three dynamic-prefix routes (actors / artifacts / projections) compose cleanly — verified by a single combined-route test asserting all return 200 with distinct prefixes. Registry-backed implementation deferred — needs a running registry process at route time. `next/tests/http_projections.sh` 11/11. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-art: GET `/artifacts/{cid}` route added on top of `match_prefix`. Single GET dispatch clause now tries `actors_prefix` first, falls through to `artifacts_prefix` — no path collision (different leading bytes). Stub body echoes the CID with `artifact: ` prefix; real artifact-store lookup deferred to later (will key into the registry / genesis bundle). `next/tests/http_artifacts.sh` 9/9 covers happy path, empty-cid 404, POST 404, actor/artifact non-collision, static-route regression. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-actors-doc: `http_server` extended with `match_prefix/2` — pure byte-level prefix matcher built on Erlang binary pattern matching (`<<B, _/binary>>`-style head/tail walk). Empty prefix returns `{ok, FullPath}`; non-match returns `nomatch`; exact match returns `{ok, <<>>}`. Wired into a new GET `/actors/{id}` clause that extracts the id suffix and returns it as the body of `actor_doc_response/1` (stub: `actor: <id>\n`). Empty id falls into 404. `/actors/{id}/outbox` deferred to a later step (needs segment splitting beyond prefix). `next/tests/http_actors.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8c-cap: GET `/.well-known/sx-capabilities` route + `capabilities_body/0` + `capabilities_path/0` exposed for tests. Body is a small plain-text descriptor with `kernel: fed-sx-m1`, `version: 0.0.1`, `verbs: Create Update Delete` (hand-spelled as integer-segment binary; string-literal segments unusable in this port). `next/tests/http_capabilities.sh` 8/8 covers method+path matching, body content, the existing GET / regression-free. Step 8c split into cap (done) + actors / art / proj / post — the rest need path-prefix matching helpers since `{id}` and `{cid}` are dynamic. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8b-route: `next/kernel/http_server.erl` — pure `route/1` request→response dispatch. Request shape `[{method, Bin}, {path, Bin}, ...]`; response `[{status, N}, {headers, []}, {body, Bin}]`. GET / returns 200 with hand-spelled "fed-sx kernel m1" body; everything else returns 404 with "not found" body. Method/path binaries spelled byte-by-byte (string-literal segments would truncate). Split former 8b into 8b-route (done) + 8b-start (needs dict↔proplist marshaling bridge in the BIF wrapper before the spawned `http:listen` call gets useful request fields). `next/tests/http_route.sh` 11/11. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 8a: `http:listen/2` BIF wrapper added to `lib/erlang/runtime.sx` (the briefing's single allowed scope exception). The BIF takes `(Port, Handler)`, validates Port is an integer and Handler is an Erlang fun (else `badarg`), then builds an SX-callable bridge lambda that marshals request dict↔Erlang term via `er-of-sx`/`er-to-sx` and calls `er-apply-fun` on the handler. Delegates to the native `http-listen` primitive (registered in `bin/sx_server.ml`, native-only). Tests verify registration + arg validation paths (the blocking listen loop itself is not exercised — production callers spawn an Erlang process to host the call). `next/tests/http_listen_bif.sh` 5/5; Erlang conformance preserved at 729/729 despite the runtime.sx edit. Step 8 broken into 8a–8d on the plan.
|
||||
- **2026-05-28** — Step 7c: `outbox:publish` now broadcasts the signed activity to every projection process named in `Context`'s `:projections` entry — fired immediately after `log:append`, via `projection:async_fold`. Missing/nil/empty list is a no-op (preserves the Step 6d-publish contract). Stage halts (replay duplicate, sig failure) suppress the broadcast — projection state stays at zero while the activity is rejected. `next/tests/outbox_broadcast.sh` 14/14 covers single + multi projection fan-out, three-publish accumulation, replay-skip, sig-skip, and the projection receiving the post-sign Signed envelope (not the pre-sign skeleton). Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 7b: `projection.erl` extended with gen_server callbacks + per-projection named-process API. `start_link/3(Name, InitialState, FoldFn)` spawns and registers under the supplied atom; `async_fold/2(Name, Activity)` casts a fold message; `query/1(Name)` synchronously returns the current state. Same port quirks as registry gen_server (Step 5b): raw Pid return, no `?MODULE` macro, processes don't survive between separate `erlang-eval-ast` calls — tests inline start_link with operations. Two named projections are independent. Snapshot persistence deferred to a later sub-step (needs SX-source eval + on-disk state). `next/tests/projection_server.sh` 11/11. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 7a: `next/kernel/projection.erl` — pure-functional projection driver. Record shape `[{name, _}, {state, _}, {fold, fun}]`; `fold_activity/2` advances state by one activity; `replay/2` folds a whole list (mirrors `log:entries/1` semantics); `new/2` defaults to the identity fold and `new/3` accepts a custom Erlang fun. Multiple projections share no state — independent record values. Step 7 split into 7a (done) + 7b (gen_server-per-projection) + 7c (broadcast hook from outbox) + 7d (sandbox eval, needs SX-source bridge). `next/tests/projection_pure.sh` 12/12. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6d-publish: `outbox:publish/2(Request, Context)` orchestrates construct + sign + `pipeline:run_stages` + `log:append`. Stage list is `[stage_envelope, stage_signature(AS), stage_replay(LogState)]` — so a duplicate publish (same Request, same Published) halts at the replay stage and returns `{error, replay, LogState}` with the log unchanged; bad key material halts at `bad_signature`. Happy path returns `{ok, [{cid, Cid}, {activity, Signed}], NewLog}`. Projection-scheduler dispatch deferred to Step 7. `next/tests/outbox_publish.sh` 13/13 covers happy path, replay halt, sig halt, multi-publish progression, CID stability across fresh logs. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6d-cs: `next/kernel/outbox.erl` — envelope construction + signing. `construct/4` takes `(Type, ActorId, Published, Object)`, builds the canonical key-sorted property list, and derives the activity `:id` from `cid:to_string({activity_envelope, Skeleton})`. `sign/2` extracts key_id/algorithm/key-material from a KeySpec proplist, computes the v1 HMAC over canonical bytes, and appends the `:signature` pair. `cid_of/1` is a convenience accessor. Round-trip end-to-end through `envelope:verify_signature/2` verified (correct key passes, wrong key returns bad_signature). Step 6d split into 6d-cs (done) + 6d-publish (orchestration). `next/tests/outbox_construct.sh` 13/13. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6c-replay: `pipeline:stage_replay/2` (direct) + `stage_replay/1` (factory closed over LogState). Linear scan of `log:entries/1` checking for an existing entry with the same `:id`. Returns ok if new, `{error, replay}` on duplicate, `{error, no_id}` when the activity has no id field. Step 6c split into 6c-replay (done) + 6c-schema (deferred — blocked behind SX-source eval bridge for the activity-type :schema body). `next/tests/pipeline_replay.sh` 12/12 covers direct + factory + composition with stage_envelope. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6b-sig: `pipeline:stage_signature/2` direct call + `stage_signature/1` factory returning a context-bound stage fun closed over ActorState. Not wired into the default `inbound_stages`/`outbound_stages` lists because actor state isn't a static-build-time value; callers prepend the factory result to a stage list (`Stages = [stage_envelope, pipeline:stage_signature(AS)]`). `next/tests/pipeline_signature.sh` 11/11 covers direct + factory + composition with stage_envelope (including halt ordering: bad envelope halts before sig; good envelope + bad sig surfaces sig error). Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6b-env: `pipeline:stage_envelope/1` wraps `envelope:validate_shape/1`; wired into both `inbound_stages` and `outbound_stages` lists. `validate_inbound`/`validate_outbound` now exercises the full envelope shape contract end-to-end (missing fields, signature sub-shape, non-list input). `next/tests/pipeline_envelope.sh` 12/12; `pipeline_driver.sh` refactored to test the driver against explicit stage lists rather than depending on the now-non-empty defaults. Split 6b in the plan into 6b-env (done) + 6b-sig (needs runtime context for actor-state). Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 6a: `next/kernel/pipeline.erl` — validation pipeline driver per design §14. `run_stages/2` is a pure fold over `(Activity) -> ok | {error, R}` funs, halting on first failure. Halt verified by inserting a post-error stage that would set a contradictory tag if it ran. `validate_inbound/1` + `validate_outbound/1` wrappers; concrete stage lists are empty (6b wires `stage_envelope`/`stage_signature`). Port quirk: `Pattern = Var` match-alias syntax unsupported — split into separate `Result = X, case Result of ...`. `next/tests/pipeline_driver.sh` 10/10. Step 6 broken into 6a–6e on the plan. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 5b: `registry.erl` extended with gen_server callbacks + named-process API. `start_link/0` spawns the worker, registers it under the literal `registry` atom, returns the Pid (port returns raw Pid not `{ok, Pid}` — diverges from OTP). 3-arity `register`, 2-arity `lookup`, 1-arity `list` delegate to the pure /4 and /3 functions inside handle_call. Port note documented: `?MODULE` macro unsupported; tests must inline start_link with operations since spawned processes don't persist across separate `erlang-eval-ast` calls. `next/tests/registry_server.sh` 12/12. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 4e: `bootstrap:load_genesis/1` + `strip_sx_suffix/1` in `next/kernel/bootstrap.erl`. Walks `read_genesis` output and threads each entry through `registry:register/4`, using the section atom as the kind and the filename-minus-`.sx` as the entry name. Per-kind counts match the seven bootstrap sections exactly (3/10/7/3/3/2/3 = 31 entries total). `next/tests/bootstrap_load.sh` 15/15. Determinism verified by comparing `cid:to_string` of the loaded state across calls (faster than deep-equality on the nested-binary state). Step 4 is now complete end-to-end except for SX-source parsing of the loaded entries. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 5a: `next/kernel/registry.erl` — pure-functional registry. State is `[{Kind, [{Name, Entry}, ...]}, ...]` keyed by the same seven section atoms as Step 4c (activity_types, object_types, projections, validators, codecs, sig_suites, audience). API: `new/0`, `kinds/0`, `register/4`, `lookup/3`, `list/2`. Unknown kinds rejected with `{error, unknown_kind}`; missing names return `not_found`; re-registering the same name overrides without growing the list. `next/tests/registry_pure.sh` 14/14. Step 5 broken into 5a–5d on the plan. Erlang conformance 729/729.
|
||||
- **2026-05-28** — Step 4d: `bootstrap:build_genesis/1` + `verify_genesis/2` + `.cidhash` helpers in `next/kernel/bootstrap.erl`. Bundle CID delegated to host `cid:to_string` over `{genesis_bundle, Sections}` — deterministic, ~59 byte CIDv1 binary. `verify_genesis/2` returns `ok` on match, `{error, {cid_mismatch, Got, Expected}}` on drift. `write_cidhash`/`read_cidhash` persist the CID to a `.cidhash` sibling file (path hand-spelled `<<...,47,46,99,...>>` per the string-literal-in-binary substrate quirk). `next/tests/bootstrap_build.sh` 12/12. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4c: `next/kernel/bootstrap.erl` — Erlang module that enumerates the genesis bundle by walking seven hardcoded section subdirs via `file:list_dir/1`, filters `.sx` files via byte-pattern suffix match (`ends_with_sx/1`), reads each into a binary via `file:read_file/1`. Returns `{ok, [{Section, [{Name, Bytes}, ...]}]}`. Hits the same SX-parser substrate gap as Step 3b — kept the surface byte-only; parsing happens via SX-side helpers in later steps. Port gotchas: `fun name/arity` references unsupported (use anonymous fun wrappers); `<<"...">>` string-literal segments truncate to one byte (paths hand-spelled as integer-segment binaries). `next/tests/bootstrap_read.sh` 15/15. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4b-cod: bootstrap codecs + sig-suites + audience predicates complete. 3 `DefineCodec` files (dag-cbor + raw + dag-json, dag-cbor + dag-json deferring to host-codec primitive when wired), 2 `DefineSigSuite` files (rsa-sha256-2018 PEM-keyed, ed25519-2020 multibase-keyed, both :verify returning false as m2-deferred stand-in), 3 `DefineAudience` files (Public/Followers/Direct member-of predicates per design §16). Manifest now lists 26 bootstrap files across all eight sections; `next/tests/genesis_parse.sh` 50/50. Step 4b complete; remaining Step 4 is bundler code (4c–4e). Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4b-vld: bootstrap validators complete — 3 `DefineValidator` SX files (envelope-shape mirroring Step 2a, signature stub delegating to envelope:verify_signature/2 per design §9.6, type-schema looking up the object-type schema from define-registry). Manifest `:validators` populated; `next/tests/genesis_parse.sh` 36/36. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4b-proj: bootstrap projections complete — 7 `DefineProjection` SX files authored (activity-log identity, by-type/by-actor/by-object indexes, actor-state with key history fold, define-registry meta-fold over Create{Define*}, audience-graph stub). Manifest `:projections` populated; `next/tests/genesis_parse.sh` 31/31. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4b-obj: bootstrap object-types complete — 10 `DefineObject` SX files authored (SXArtifact, Note, Tombstone, DefineActivity, DefineObject, DefineProjection, DefineValidator, DefineCodec, DefineSigSuite, Snapshot). Each carries an SX `:schema` predicate. Manifest `:object-types` populated; `next/tests/genesis_parse.sh` 22/22. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4b-act: bootstrap activity-types complete — `update.sx` (Update verb, requires :object CID + :patch) + `delete.sx` (Delete verb, requires :object CID) authored as DefineActivity forms matching the Create shape. Manifest updated; `next/tests/genesis_parse.sh` 10/10. Step 4b broken into act/obj/proj/vld/cod sub-deliverables on the plan. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 4a: genesis bundle seeded. `next/genesis/manifest.sx` (GenesisManifest with eight section keys, only `:activity-types` populated for now) + `next/genesis/activity-types/create.sx` (DefineActivity{Create} with :schema/:semantics SX bodies). `next/tests/genesis_parse.sh` 5/5. Step 3b parked behind a substrate-level term-codec gap — Blockers note added under Step 3; in-memory log from 3a unblocks Step 5+ which only need the API surface. Erlang conformance 729/729.
|
||||
- **2026-05-27** — Step 3a: `log:open/2 append/2 tip/1 replay/3 entries/1` over an in-memory state (per-actor seq, replay in append order, round-trip activities). `next/tests/log_memory.sh` 12/12. Pivoted from on-disk in this iteration: this port's `atom_to_list`/`integer_to_list` return SX strings rather than Erlang charlists, `binary_to_list` is unregistered, and `$X` char literals decode to nil — so a term codec needs a workaround. Captured as the Step 3b risk note in the plan. Erlang conformance 729/729.
|
||||
- **2026-05-26** — Step 2c: `envelope:verify_signature/2` — time-aware key lookup over `public_keys` (created ≤ published < superseded_at), MAC recompute via `crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`, compared against `signature.value`. Returns ok or one of `no_signature | no_key_id | no_published | no_keys | no_active_key | bad_signature`. `next/tests/envelope_sig.sh` 11/11 pass. Erlang conformance 729/729.
|
||||
- **2026-05-26** — Step 2b: `envelope:canonical_bytes/1` — strip signature, insertion-sort property list by key, return host-CID-string as deterministic byte form (dag-cbor stand-in). `next/tests/envelope_canonical.sh` 8/8 pass. Erlang conformance 729/729 preserved.
|
||||
- **2026-05-26** — Step 2a: `next/kernel/envelope.erl` `validate_shape/1` + `get_field/2` over property-list envelopes (Erlang `#{}` maps not supported in this port). `next/tests/envelope_shape.sh` 15/15 pass. Erlang conformance 729/729 preserved.
|
||||
- **2026-05-26** — Step 1b: `next/kernel/nx_cid.erl` (from_sx/to_string/from_string/equals) — thin Erlang wrapper around the `cid:to_string/1` BIF. `next/tests/cid.sh` 13/13 pass. Module named `nx_cid` to avoid shadowing the `cid` BIF (user-module dispatch takes precedence over BIFs by module name). Erlang conformance 729/729 preserved.
|
||||
- **2026-05-26** — Step 1a: `next/` skeleton created (kernel/, genesis/, tests/, data/), README, `.gitignore data/`. Erlang conformance 729/729 preserved.
|
||||
|
||||
|
||||
1963
plans/fed-sx-milestone-2.md
Normal file
1963
plans/fed-sx-milestone-2.md
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user