# next — fed-sx Milestone 1 kernel

Single-instance, single-actor fed-sx server built as Erlang-on-SX modules.
See `plans/fed-sx-design.md` for the architecture and
`plans/fed-sx-milestone-1.md` for the build plan + per-step progress log.

## Status

Both Step 9 smoke proof points are functional **in-process**:

- **9a-pure (verb extensibility)** — `Create{DefineActivity{Pin}}` registers Pin
  at runtime; subsequent `Pin{path, cid}` activities fold into a pin-state
  projection. Zero kernel code between definition and use.
  See `next/tests/smoke_pin_pure.sh`.
- **9b-pure (reactive application)** — A trigger projection matches Notes
  tagged `smoketest` and derives a `TestEcho` carrying the source CID.
  See `next/tests/smoke_app_pure.sh`.

The remaining `9a-tcp` / `9b-tcp` deliverables layer TCP transport on top — see
*Substrate gaps* below.

## Layout

```
next/
├── kernel/      Erlang-on-SX kernel modules (.erl)
├── genesis/     SX source files for the bootstrap bundle
├── tests/       Bash test scripts driving sx_server.exe via the epoch protocol
└── data/        Runtime state — gitignored
```

## Module map

| Module                | Role                                                                   |
|-----------------------|------------------------------------------------------------------------|
| `nx_cid.erl`          | Canonical CID wrapper around the host `cid:to_string` BIF              |
| `envelope.erl`        | Activity envelope shape, canonical bytes, time-aware sig verify        |
| `log.erl`             | Per-actor in-memory append log (open / append / tip / replay / entries) |
| `registry.erl`        | Pure-functional + gen_server-wrapped registry keyed by Kind             |
| `pipeline.erl`        | Validation driver + stage_envelope/signature/replay/schema             |
| `projection.erl`      | Pure projection driver + gen_server-per-projection wrapper             |
| `outbox.erl`          | Envelope construct + sign + publish orchestrator + broadcast            |
| `bootstrap.erl`       | Genesis read/build/verify/load + one-call `start/3` kernel bring-up    |
| `define_registry.erl` | Meta-projection fold for `Create{Define*}` → registry                  |
| `sandbox.erl`         | `eval_pure/2,3` try/catch envelope for projection folds                |
| `nx_kernel.erl`       | Long-lived runtime orchestrator (state + gen_server)                    |
| `http_server.erl`     | route/1,2 + format-aware GET + POST + Accept header content negotiation |

## Genesis bundle

`next/genesis/` contains 31 SX files across 7 sections, all consumed as data
(read + serialised by `bootstrap:populate_registry`, not eval'd):

- 3 activity-types — Create, Update, Delete
- 10 object-types — SXArtifact, Note, Tombstone, 6 Define* meta-types, Snapshot
- 7 projections — activity-log, by-type, by-actor, by-object, actor-state,
  define-registry, audience-graph
- 3 validators — envelope-shape, signature, type-schema
- 3 codecs — dag-cbor, raw, dag-json
- 2 sig-suites — rsa-sha256-2018, ed25519-2020
- 3 audience predicates — Public, Followers, Direct

`manifest.sx` is the bundle root, listed in dependency-friendly order.

## Tests

43 test suites, ~560+ assertions. Each script drives `sx_server.exe` via the
epoch protocol — loads the Erlang substrate, loads relevant kernel modules
via `code:load_binary` / `erlang-load-module`, then exercises behaviour
through `erlang-eval-ast`.

Conventions:

- Scripts marked `_pure.sh` exercise pure-functional state.
- Scripts marked `_server.sh` (or no suffix) exercise gen_server APIs and
  must inline `start_link` with operations — the Erlang-on-SX scheduler
  doesn't preserve spawned processes across separate `erlang-eval-ast`
  invocations.
- `smoke_*_pure.sh` are end-to-end smoke tests demonstrating the §Step 9
  proof points without TCP / curl / JSON.

The Erlang-on-SX conformance gate (`bash lib/erlang/conformance.sh`, **729 /
729**) is the no-regression contract — every commit on `loops/fed-sx-m1`
preserves it.

## Substrate

Each `.erl` source file is hot-loaded at boot via
`code:load_binary(Mod, Filename, SourceString)` (Phase 7 BIF). Tests drive
the runtime via the epoch protocol:

```bash
printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n<test-expr>\n' \
  | hosts/ocaml/_build/default/bin/sx_server.exe
```

The kernel calls into these host primitives: `crypto:hash/2`,
`cid:from_bytes/1`, `cid:to_string/1`, `file:read_file/1`, `file:write_file/2`,
`file:delete/1`, `file:list_dir/1`, `code:load_binary/3`, plus `http:listen/2`
(the briefing's allowed scope exception, added to `lib/erlang/runtime.sx`).

### Substrate gaps (parked work)

These three gaps block the remaining unchecked deliverables:

1. **Term codec** (`3b`/`3c`) — **substrate fixes #1 + #2 done 2026-06-04:**
   `erlang:binary_to_list/1` and `erlang:list_to_binary/1` are registered
   in `lib/erlang/runtime.sx` (`list_to_binary` is iolist-aware); the
   tokenizer's `$X` branch now emits the decimal char code, so `[$h, $i | T]`
   patterns and `list_to_binary([$f,$e,$d])` work end-to-end. 750/750
   conformance, +9 ffi + +12 eval tests. Step 3b on-disk segment writer
   has a complete byte-level term ↔ binary path. Still parked (low priority
   for Milestone 1): `atom_to_list`/`integer_to_list` return SX-strings
   (an opaque OCaml-string type), not Erlang charlists — only blocks code
   that wants charlist arithmetic on atom/integer names.

2. **SX-source eval bridge** — There's no BIF that lets Erlang call into the
   SX evaluator on a parsed source string. Blocks evaluating the `:schema` /
   `:fold` / `:predicate` / `:verify` bodies from the genesis bundle. Erlang-fun
   stand-ins (`pipeline:stage_schema`, `define_registry:fold`, etc.) prove the
   API shapes; the bridge would let bundle bodies dispatch through them
   unchanged.

3. **Dict ↔ proplist marshalling for `http:listen/2`** — The native
   `http-listen` primitive calls the handler with an SX dict; the BIF
   wrapper's bridge would need to marshal that to / from an Erlang proplist.
   Blocks `Step 8b-start` (actual TCP listening with working route dispatch).
   The briefing allowed the BIF *wrapper* as a single scope exception; further
   in-place modifications need agent approval.

### Bringing up the kernel

For tests, `bootstrap:start/3(ActorId, KeySpec, ActorState)` is the
one-call boot:

```erlang
KM = <<1,2,3,4>>,
KS = [{key_id, k1}, {algorithm, ed25519}, {value, KM}],
AS = [{public_keys, [[{id, k1}, {created, 0}, {value, KM}]]}],
Pid = bootstrap:start(alice, KS, AS),
%% nx_kernel + registry populated; you now have a kernel.
```

The HTTP layer (`http_server`) and `nx_kernel:publish/1` flow through the
same in-process gen_servers; `http_publish_fold.sh` is the end-to-end proof
the chain works.

## What's next (when work resumes)

In priority order:

1. **8b-bridge** — extend `er-bif-http-listen` with dict ↔ proplist marshalling
   so requests reach `route/1` shaped correctly.
2. **8b-start** — `http_server:start/1` spawns a process hosting `http:listen/2`.
3. **9a-tcp / 9b-tcp** — replace the in-process smoke scripts with curl-driven
   versions hitting the running server.
4. **Term codec / on-disk log** — needs either a new BIF or a temp-file
   workaround; current in-memory log keeps everything functional otherwise.
5. **SX-source eval bridge** — unlocks real `:schema` / `:fold` body
   evaluation from the genesis bundle.