fed-sx-m1: refresh next/README with module map, test inventory, substrate gaps + resume order

next/README.md

@@ -2,33 +2,154 @@
 
 Single-instance, single-actor fed-sx server built as Erlang-on-SX modules.
 See `plans/fed-sx-design.md` for the architecture and
-`plans/fed-sx-milestone-1.md` for the build plan.
+`plans/fed-sx-milestone-1.md` for the build plan + per-step progress log.
+
+## Status
+
+Both Step 9 smoke proof points are functional **in-process**:
+
+- **9a-pure (verb extensibility)** — `Create{DefineActivity{Pin}}` registers Pin
+  at runtime; subsequent `Pin{path, cid}` activities fold into a pin-state
+  projection. Zero kernel code between definition and use.
+  See `next/tests/smoke_pin_pure.sh`.
+- **9b-pure (reactive application)** — A trigger projection matches Notes
+  tagged `smoketest` and derives a `TestEcho` carrying the source CID.
+  See `next/tests/smoke_app_pure.sh`.
+
+The remaining `9a-tcp` / `9b-tcp` deliverables layer TCP transport on top — see
+*Substrate gaps* below.
 
 ## Layout
 
 ```
 next/
-├── kernel/      Erlang-on-SX kernel modules (.erl, hot-loaded via code:load_binary/3)
-├── genesis/     SX source files for the genesis bootstrap bundle (DefineActivity, ...)
+├── kernel/      Erlang-on-SX kernel modules (.erl)
+├── genesis/     SX source files for the bootstrap bundle
 ├── tests/       Bash test scripts driving sx_server.exe via the epoch protocol
 └── data/        Runtime state — gitignored
-    ├── log/         per-actor JSONL outboxes
-    ├── objects/     CID-addressed artifacts on disk
-    ├── snapshots/   projection snapshots
-    ├── indexes/     derived projection index files
-    └── keys/        actor signing keys + bearer tokens
 ```
 
+## Module map
+
+| Module                | Role                                                                   |
+|-----------------------|------------------------------------------------------------------------|
+| `nx_cid.erl`          | Canonical CID wrapper around the host `cid:to_string` BIF              |
+| `envelope.erl`        | Activity envelope shape, canonical bytes, time-aware sig verify        |
+| `log.erl`             | Per-actor in-memory append log (open / append / tip / replay / entries) |
+| `registry.erl`        | Pure-functional + gen_server-wrapped registry keyed by Kind             |
+| `pipeline.erl`        | Validation driver + stage_envelope/signature/replay/schema             |
+| `projection.erl`      | Pure projection driver + gen_server-per-projection wrapper             |
+| `outbox.erl`          | Envelope construct + sign + publish orchestrator + broadcast            |
+| `bootstrap.erl`       | Genesis read/build/verify/load + one-call `start/3` kernel bring-up    |
+| `define_registry.erl` | Meta-projection fold for `Create{Define*}` → registry                  |
+| `sandbox.erl`         | `eval_pure/2,3` try/catch envelope for projection folds                |
+| `nx_kernel.erl`       | Long-lived runtime orchestrator (state + gen_server)                    |
+| `http_server.erl`     | route/1,2 + format-aware GET + POST + Accept header content negotiation |
+
+## Genesis bundle
+
+`next/genesis/` contains 31 SX files across 7 sections, all consumed as data
+(read + serialised by `bootstrap:populate_registry`, not eval'd):
+
+- 3 activity-types — Create, Update, Delete
+- 10 object-types — SXArtifact, Note, Tombstone, 6 Define* meta-types, Snapshot
+- 7 projections — activity-log, by-type, by-actor, by-object, actor-state,
+  define-registry, audience-graph
+- 3 validators — envelope-shape, signature, type-schema
+- 3 codecs — dag-cbor, raw, dag-json
+- 2 sig-suites — rsa-sha256-2018, ed25519-2020
+- 3 audience predicates — Public, Followers, Direct
+
+`manifest.sx` is the bundle root, listed in dependency-friendly order.
+
+## Tests
+
+43 test suites, ~560+ assertions. Each script drives `sx_server.exe` via the
+epoch protocol — loads the Erlang substrate, loads relevant kernel modules
+via `code:load_binary` / `erlang-load-module`, then exercises behaviour
+through `erlang-eval-ast`.
+
+Conventions:
+
+- Scripts marked `_pure.sh` exercise pure-functional state.
+- Scripts marked `_server.sh` (or no suffix) exercise gen_server APIs and
+  must inline `start_link` with operations — the Erlang-on-SX scheduler
+  doesn't preserve spawned processes across separate `erlang-eval-ast`
+  invocations.
+- `smoke_*_pure.sh` are end-to-end smoke tests demonstrating the §Step 9
+  proof points without TCP / curl / JSON.
+
+The Erlang-on-SX conformance gate (`bash lib/erlang/conformance.sh`, **729 /
+729**) is the no-regression contract — every commit on `loops/fed-sx-m1`
+preserves it.
+
 ## Substrate
 
-The kernel is Erlang-on-SX. Each `.erl` source file is hot-loaded at boot via
-`code:load_binary(Mod, Filename, SourceString)` (Erlang Phase 7 BIF). The
-underlying SX runtime provides the host primitives the kernel calls into:
-`crypto:*`, `cid:*`, `file:*`, `code:*`, and (Step 8) `http:listen/2`.
-
-Tests drive the kernel via the epoch protocol:
+Each `.erl` source file is hot-loaded at boot via
+`code:load_binary(Mod, Filename, SourceString)` (Phase 7 BIF). Tests drive
+the runtime via the epoch protocol:
 
 ```bash
 printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n<test-expr>\n' \
   | hosts/ocaml/_build/default/bin/sx_server.exe
 ```
+
+The kernel calls into these host primitives: `crypto:hash/2`,
+`cid:from_bytes/1`, `cid:to_string/1`, `file:read_file/1`, `file:write_file/2`,
+`file:delete/1`, `file:list_dir/1`, `code:load_binary/3`, plus `http:listen/2`
+(the briefing's allowed scope exception, added to `lib/erlang/runtime.sx`).
+
+### Substrate gaps (parked work)
+
+These three gaps block the remaining unchecked deliverables:
+
+1. **Term codec** (`3b`/`3c`) — `atom_to_list`/`integer_to_list` return
+   SX-strings (an opaque OCaml-string type), not Erlang charlists;
+   `binary_to_list`/`list_to_binary` are unregistered; `$X` char literals
+   decode to `nil` in `parse-number`. Net effect: no in-Erlang term ↔ binary
+   round-trip path. Blocks on-disk log persistence.
+
+2. **SX-source eval bridge** — There's no BIF that lets Erlang call into the
+   SX evaluator on a parsed source string. Blocks evaluating the `:schema` /
+   `:fold` / `:predicate` / `:verify` bodies from the genesis bundle. Erlang-fun
+   stand-ins (`pipeline:stage_schema`, `define_registry:fold`, etc.) prove the
+   API shapes; the bridge would let bundle bodies dispatch through them
+   unchanged.
+
+3. **Dict ↔ proplist marshalling for `http:listen/2`** — The native
+   `http-listen` primitive calls the handler with an SX dict; the BIF
+   wrapper's bridge would need to marshal that to / from an Erlang proplist.
+   Blocks `Step 8b-start` (actual TCP listening with working route dispatch).
+   The briefing allowed the BIF *wrapper* as a single scope exception; further
+   in-place modifications need agent approval.
+
+### Bringing up the kernel
+
+For tests, `bootstrap:start/3(ActorId, KeySpec, ActorState)` is the
+one-call boot:
+
+```erlang
+KM = <<1,2,3,4>>,
+KS = [{key_id, k1}, {algorithm, ed25519}, {value, KM}],
+AS = [{public_keys, [[{id, k1}, {created, 0}, {value, KM}]]}],
+Pid = bootstrap:start(alice, KS, AS),
+%% nx_kernel + registry populated; you now have a kernel.
+```
+
+The HTTP layer (`http_server`) and `nx_kernel:publish/1` flow through the
+same in-process gen_servers; `http_publish_fold.sh` is the end-to-end proof
+the chain works.
+
+## What's next (when work resumes)
+
+In priority order:
+
+1. **8b-bridge** — extend `er-bif-http-listen` with dict ↔ proplist marshalling
+   so requests reach `route/1` shaped correctly.
+2. **8b-start** — `http_server:start/1` spawns a process hosting `http:listen/2`.
+3. **9a-tcp / 9b-tcp** — replace the in-process smoke scripts with curl-driven
+   versions hitting the running server.
+4. **Term codec / on-disk log** — needs either a new BIF or a temp-file
+   workaround; current in-memory log keeps everything functional otherwise.
+5. **SX-source eval bridge** — unlocks real `:schema` / `:fold` body
+   evaluation from the genesis bundle.