rose-ash/plans/fed-sx-host-primitives.md

# fed-sx host primitives — `hosts/ocaml/`

The single blocker between Erlang Phase 8 (FFI mechanism — done) and starting
fed-sx Milestone 1: the SX OCaml runtime exposes no crypto / CID / HTTP host
primitives for the Phase 8 BIF wrappers to call. This plan adds exactly that
surface, pure-OCaml where it must stay WASM-safe, native-only where it can't.

Reference: `plans/fed-sx-milestone-1.md` (build steps 1-8),
`plans/erlang-on-sx.md` Blockers ("SX runtime lacks platform primitives …").

## The hard constraint — WASM boundary

`hosts/ocaml/lib/` is the `sx` library. `hosts/ocaml/browser/dune` links it
with `(modes byte js wasm)`. **Anything added to `lib/sx_primitives.ml` must
compile under `js_of_ocaml` AND `wasm_of_ocaml`.** Therefore:

- **Pure OCaml only** for hash / CBOR / CID / Ed25519 / RSA. No `digestif`,
  no `mirage-crypto`, no C stubs, no `Unix` dependency in these primitives.
  (None of those libs are even installed — the switch has only
  re/unix/yojson/otfm/js_of_ocaml. Pure OCaml is both required and hermetic.)
- **HTTP server is native-only**: it needs sockets/threads. Register it in
  `bin/sx_server.ml` via `Sx_primitives.register` (precedent: `eval-in-env` at
  `bin/sx_server.ml:721`), **not** in the shared lib. It must never enter the
  WASM build.
- **`file-list-dir`** uses `Sys.readdir` (stdlib, WASM-stubbed) — safe in lib,
  but the fed-sx server is native anyway; native registration is acceptable too.

**Every phase must prove the WASM build still links** (`sx_build target="wasm"`
or `bash hosts/ocaml/browser/test_boot.sh`) before its commit. A broken WASM
browser kernel is a hard regression and fails the phase.

## Primitive surface (what fed-sx Milestone 1 actually needs)

Mapped to `plans/fed-sx-milestone-1.md` build steps:

| Primitive (SX name) | Signature | fed-sx step | Host |
|---|---|---|---|
| `crypto-sha256` | `(bytes) -> hex-string` | 1, 2 | lib (pure) |
| `crypto-sha512` | `(bytes) -> hex-string` | 2 | lib (pure) |
| `crypto-sha3-256` | `(bytes) -> hex-string` | 1 (CID default) | lib (pure) |
| `cbor-encode` | `(sx-value) -> bytes` (dag-cbor, deterministic) | 1 | lib (pure) |
| `cbor-decode` | `(bytes) -> sx-value` | 1 (round-trip tests) | lib (pure) |
| `cid-from-bytes` | `(codec multihash-bytes) -> cid-string` | 1 | lib (pure) |
| `cid-from-sx` | `(sx-value) -> cid-string` (canonicalize→cbor→sha→mh→cidv1) | 1 | lib (pure) |
| `ed25519-verify` | `(pubkey-32 msg sig-64) -> bool` | 2 | lib (pure) |
| `rsa-sha256-verify` | `(der-spki msg sig) -> bool` (PKCS#1 v1.5) | 2 | lib (pure) |
| `file-list-dir` | `(path) -> (list string)` | 3 | lib/native |
| `http-listen` | `(port handler-fn) -> never` (handler: req-dict→resp-dict) | 8 | **native only** |

Deferred (not Milestone 1): `httpc-request` (HTTP client — federation is v2),
`sqlite-*` (Milestone 1 is file-on-disk; sqlite is v2 indexes).

## Registration pattern (established)

`lib/sx_primitives.ml`:
```ocaml
register "crypto-sha256" (fun args ->
  match args with
  | [String s] -> String (Sha2.sha256_hex s)
  | _ -> raise (Eval_error "crypto-sha256: (bytes)"))
```
Errors: `raise (Eval_error "name: shape")`. Byte strings are OCaml `string`
(SX `String`). Lists are `Pair`/`Nil` per `sx_types.ml`. Native-only prims go in
`bin/sx_server.ml` the same way.

## Phasing — one feature per loop iteration

Dependency order. Each phase: implement → `dune build` (ocaml) → **WASM build
check** → tests → commit → tick box → Progress-log line → push.

### Phase A — SHA-2 (sha256 + sha512), pure OCaml ✅ DONE
- New `lib/sx_sha2.ml` (or inline in primitives if small): SHA-256 + SHA-512.
- Primitives `crypto-sha256`, `crypto-sha512` → lowercase hex string.
- Tests (`bin/run_tests.ml` or a dedicated `bin/test_crypto.ml`): NIST vectors —
  `""`, `"abc"`, the 896-bit message, a 1MB "a" repetition.
  - sha256("") = `e3b0c442…b7852b855`; sha256("abc") = `ba7816bf…f20015ad`
  - sha512("abc") = `ddaf35a1…2a9ac94f…`
- **Acceptance:** vectors pass; WASM build links; OCaml conformance unchanged.

### Phase B — SHA-3 / Keccak-256, pure OCaml ✅ DONE
- Keccak-f[1600] + SHA3-256 padding. Primitive `crypto-sha3-256`.
- Tests: sha3-256("") = `a7ffc6f8…0f8434a`; sha3-256("abc") = `3a985da7…11431532`.
- **Acceptance:** NIST SHA-3 vectors pass; WASM links.

### Phase C — dag-cbor encoder + decoder, pure OCaml ✅ DONE
- RFC 8949 deterministic subset (RFC 8742 dag-cbor): unsigned/negative ints,
  byte strings, text strings, arrays, maps with **keys sorted by
  length-then-bytewise**, bool, null, tag 42 (CID link). No floats unless a
  fed-sx shape needs them (defer; document).
- SX↔CBOR mapping: `Integer`→int, `String`→text str, `Bool`, `Nil`→null,
  `Pair/Nil`→array, `Dict`→map (sorted keys), keyword/symbol→text str.
- Primitives `cbor-encode`, `cbor-decode`. Round-trip property tests + RFC 8949
  appendix-A vectors + a "reordered dict keys → identical bytes" determinism test.
- **Acceptance:** vectors + round-trip + determinism pass; WASM links.

### Phase D — CID computation, pure OCaml ✅ DONE
- Multihash (sha2-256 = 0x12, sha3-256 = 0x16; varint code + varint len + digest).
- CIDv1 = `0x01 || codec-varint || multihash`. Codecs: dag-cbor 0x71, raw 0x55.
- Multibase base32 lower (`b` prefix, RFC 4648 no-pad).
- Primitives `cid-from-bytes` (codec, raw mh bytes), `cid-from-sx`
  (canonicalize → cbor-encode → sha2-256 → multihash → cidv1 → base32).
- Tests: known IPFS CIDs — cross-check against `ipfs` CLI if present, else the
  fixed vectors for `{}` dag-cbor and `"abc"` raw (hardcode expected strings).
  Determinism: same SX value (whitespace/comment/key-order variants) → same CID.
- **Acceptance:** matches reference CIDs; determinism holds; WASM links. Satisfies
  fed-sx Milestone 1 Step 1.

### Phase E — Ed25519 verify, pure OCaml ✅ DONE
- Curve25519/edwards25519 field arith (mod 2^255-19), point decompress,
  SHA-512-based verify per RFC 8032 §5.1.7. (Reuse Phase A sha512.)
- Primitive `ed25519-verify (pubkey msg sig) -> bool`. Bad-length args → false,
  not exception (verify is total).
- Tests: RFC 8032 §7.1 vectors (TEST 1-4 + the 1024-byte one). Tampered msg/sig
  → false. Wrong-length key → false.
- **Acceptance:** all RFC 8032 vectors pass; WASM links. Satisfies fed-sx Step 2
  (Ed25519 sig-suite).

### Phase F — RSA-SHA256 verify (PKCS#1 v1.5), pure OCaml ✅ DONE
- Minimal pure-OCaml bignum (only need modexp + DER parse). Parse SPKI DER →
  (n, e). RSASSA-PKCS1-v1_5 verify with SHA-256 (Phase A).
- Primitive `rsa-sha256-verify (der-spki msg sig) -> bool`.
- Tests: a generated 2048-bit keypair's signature (vectors hardcoded in the test
  from a one-off openssl run, documented in a comment), tamper → false.
- **Acceptance:** vector verifies; tamper fails; WASM links. Satisfies fed-sx
  Step 2 (rsa-sha256-2018 sig-suite). **Lower priority** than E — Ed25519 is the
  modern default; RSA can land after the HTTP phase if time-boxed.

### Phase G — `file-list-dir`, native-safe ✅ DONE
- `Sys.readdir` → sorted SX list of names (no `.`/`..`). Errors → `enoent`/
  `enotdir` classified like the existing `file-read` error mapping.
- Tests: list a known dir, missing dir → error, file-not-dir → error.
- **Acceptance:** passes; WASM build still links (Sys.readdir is stubbed there).
  Satisfies fed-sx Step 3 segment replay.

### Phase H — HTTP/1.1 server, **native-only** (`bin/sx_server.ml`) ✅ DONE
- Minimal threaded HTTP/1.1: accept loop (`Unix` + `Thread`), parse request
  line + headers + body (Content-Length), build an SX request dict
  `{:method :path :query :headers :body}`, call the SX handler callable, take an
  SX response dict `{:status :headers :body}`, write it. Connection: close
  (keep-alive optional, defer). Bind `127.0.0.1:<port>`.
- Primitive `http-listen (port handler) -> never-returns` registered ONLY in
  `bin/sx_server.ml`. Document that it is absent from the WASM kernel.
- Tests: `bin/test_http.sh` — start a server on a port with a tiny SX echo
  handler in a subprocess, `curl` GET/POST/404/headers, assert responses, kill.
- **Acceptance:** curl test script green; WASM build untouched (prim not in lib).
  Satisfies fed-sx Step 8 transport.

### Phase I — handoff ✅ DONE
- Flip the `plans/erlang-on-sx.md` Blockers entry "SX runtime lacks platform
  primitives …" to **RESOLVED**, listing the exact SX primitive names so the
  Erlang loop can one-line-wire its blocked Phase 8 BIFs (`crypto:hash/2`,
  `cid:from_bytes/1`, `cid:to_string/1`, `file:list_dir/1`, plus note
  `httpc`/`sqlite` still deferred). **Do not edit `lib/erlang/`** — that wiring
  is the Erlang loop's job; this phase only updates the blocker text + this
  plan's "Handoff" section with the primitive→BIF mapping.
- **Acceptance:** blocker text updated; fed-sx Milestone 1 Steps 1-3 + 8
  prerequisites all green.

## Scope (hard)

- **Edit only:** `hosts/ocaml/lib/**`, `hosts/ocaml/bin/**`, this plan file.
- **Do NOT edit:** `lib/erlang/**` (Erlang loop owns BIF wiring), `spec/`,
  `lib/` root, other `lib/<lang>/`, `plans/erlang-on-sx.md` *except* the one
  Blockers entry in Phase I.
- **Pure OCaml for lib primitives.** No new opam deps. If a phase seems to need
  one, stop and add a Blockers entry instead.
- **Prove WASM every phase.** No commit without `test_boot.sh` (or wasm build)
  green.
- **Never push to `main` or `architecture`.** Branch `loops/fed-prims`, push
  `origin/loops/fed-prims`.
- One feature per commit. Short factual messages: `fed-prims: SHA-256 + 4 NIST
  vectors`. Tick the box, append a dated Progress-log line (newest first).
- **Never call `sx_build` with no timeout-awareness** — OCaml builds are slow;
  use the MCP `sx_build target="ocaml"` / `target="wasm"` tools or
  `dune build` with a generous timeout. If the build hangs >10min, Blockers +
  stop.

## Build & test reference

```bash
cd hosts/ocaml && dune build bin/sx_server.exe 2>&1 | tail   # native
bash hosts/ocaml/browser/test_boot.sh                        # WASM links + boots
cd hosts/ocaml && dune exec bin/run_tests.exe 2>&1 | tail     # OCaml unit tests
SX_SERVER=hosts/ocaml/_build/default/bin/sx_server.exe \
  timeout 400 bash lib/erlang/conformance.sh 2>&1 | tail -3   # no-regression gate
```

A primitive is reachable from SX via the epoch protocol:
```bash
printf '(epoch 1)\n(crypto-sha256 "abc")\n' | \
  hosts/ocaml/_build/default/bin/sx_server.exe
```

## Handoff (Phase I fills this in)

| SX primitive | Erlang Phase 8 BIF it unblocks |
|---|---|
| `crypto-sha256` / `crypto-sha512` / `crypto-sha3-256` | `crypto:hash/2` |
| `cid-from-bytes` / `cid-from-sx` | `cid:from_bytes/1`, `cid:to_string/1` |
| `ed25519-verify` / `rsa-sha256-verify` | `crypto:verify` / sig-suites |
| `file-list-dir` | `file:list_dir/1` |
| `http-listen` | fed-sx kernel `http:listen/2` (Milestone 1 Step 8) |

**Status: DELIVERED (Phases A–H, 2026-05-18).** All primitives are
registered and reachable from SX (`(eval "(crypto-sha256 \"abc\")")`
via the epoch protocol). Signatures the Erlang loop can one-line-wire:

- `(crypto-sha256 bytes) -> hex-string` — also `crypto-sha512`,
  `crypto-sha3-256`. lib (`Sx_sha2`/`Sx_sha3`), WASM-safe.
- `(cbor-encode value) -> bytes` / `(cbor-decode bytes) -> value` —
  deterministic dag-cbor, lib (`Sx_cbor`), WASM-safe.
- `(cid-from-bytes codec mh-bytes) -> cid-string` /
  `(cid-from-sx value) -> cid-string` — lib (`Sx_cid`), WASM-safe.
- `(ed25519-verify pk msg sig) -> bool` /
  `(rsa-sha256-verify spki msg sig) -> bool` — total (bad input →
  false), lib (`Sx_ed25519`/`Sx_rsa`), WASM-safe.
- `(file-list-dir path) -> (list string)` — sorted, lib, WASM-stubbed.
- `(http-listen port handler) -> never` — **NATIVE ONLY**
  (`bin/sx_server.ml`); absent from the WASM kernel by design.

Still **deferred** (not Milestone 1, not provided here): `httpc-request`
(HTTP client / federation v2), `sqlite-*` (v2 indexes). The Erlang loop
should leave `httpc`/`sqlite` BIFs blocked with that note.

## Progress log

_Newest first._

- 2026-05-18 — Phase I: handoff. `erlang-on-sx.md` Blockers gained one
  RESOLVED entry (no "SX runtime lacks…" entry pre-existed; it read
  "_(none yet)_") mapping every delivered primitive → its Phase 8 BIF,
  with httpc/sqlite explicitly left deferred. Handoff section here
  filled with signatures + native/WASM notes. Doc-only (no lib/erlang/
  edits); Erlang 530/530 unchanged. **fed-sx Milestone 1 Steps 1-3 + 8
  prerequisites all green — plan complete (Phases A–I done).**
- 2026-05-18 — Phase H: `http-listen` primitive in `bin/sx_server.ml`
  (NATIVE ONLY — Unix sockets + Thread per connection, Mutex around
  the shared-runtime handler call; HTTP/1.1, Connection: close;
  req {:method :path :query :headers :body} → resp {:status :headers
  :body}). Test `bin/test_http.sh`: curl GET+query / POST+body / 404
  / custom header — 6/6. NOT in lib, so WASM kernel untouched (boot
  green); run_tests 4897 unchanged; Erlang 530/530. Satisfies fed-sx
  Milestone 1 Step 8 transport.
- 2026-05-18 — Phase G: `file-list-dir` primitive in
  `lib/sx_primitives.ml` (Sys.readdir → sorted names, no "."/"..";
  Sys_error prefixed like file-read, msg carries enoent/enotdir).
  4 tests: sorted listing, missing dir, not-a-dir, arity. WASM boot
  green (Sys.readdir stubbed there); Erlang 530/530; run_tests +4.
  Satisfies fed-sx Step 3 segment replay.
- 2026-05-18 — Phase F: pure-OCaml `lib/sx_rsa.ml` (self-contained
  bignum modexp, minimal DER SPKI reader, RFC 8017 §8.2.2 PKCS#1
  v1.5 verify with SHA-256 DigestInfo prefix). Primitive
  `rsa-sha256-verify` total. 5 tests on a fixed RSA-2048 vector
  (one-off python-cryptography keygen, hardcoded): valid, tampered
  msg/sig, garbage SPKI, non-string. WASM boot green with new lib
  module; Erlang 530/530; run_tests +5. Satisfies fed-sx Step 2
  (rsa-sha256-2018 sig-suite).
- 2026-05-18 — Phase E: pure-OCaml `lib/sx_ed25519.ml` (minimal
  base-2^26 bignum, edwards25519 extended-coord points, RFC 8032
  §5.1.7 cofactorless verify reusing Phase-A sha512). Primitive
  `ed25519-verify` is total (bad/short/non-string args → false).
  8 tests: RFC 8032 §7.1 TEST 1-3 (re-derived independently via
  python-cryptography), tampered msg/sig, wrong-length, non-string.
  WASM boot green with new lib module; Erlang 530/530; run_tests +8.
  Satisfies fed-sx Milestone 1 Step 2 (Ed25519 sig-suite).
- 2026-05-18 — Phase D: pure-OCaml `lib/sx_cid.ml` (unsigned-varint,
  multihash, CIDv1, multibase base32-lower), primitives `cid-from-bytes`
  / `cid-from-sx` (cbor→sha2-256→mh→cidv1, dag-cbor codec 0x71). 5 tests:
  raw "abc"=bafkreif2pall7d…, raw ""=bafkreihdwdcefg…, dag-cbor {}=
  bafyreigbtj4x7i… (all match canonical IPFS CIDs; no `ipfs` CLI so
  vectors independently derived in Python), key-order determinism. WASM
  boot green with new lib module; Erlang 530/530; run_tests +5.
- 2026-05-18 — Phase C: pure-OCaml `lib/sx_cbor.ml` (dag-cbor encode/
  decode), primitives `cbor-encode`/`cbor-decode`. RFC 8949 Appendix-A
  vectors, length-then-bytewise key sort + order-invariance determinism,
  decode∘encode round-trip (30 tests). Floats unsupported (raise, no
  fed-sx shape needs them); tag-42 decode = inner-item passthrough.
  WASM boot green with new lib module; Erlang 530/530; run_tests +30.
- 2026-05-18 — Phase B: pure-OCaml `lib/sx_sha3.ml` (Keccak-f[1600] +
  SHA-3 pad, domain 0x06), primitive `crypto-sha3-256`. 4 NIST FIPS 202
  vectors pass (empty/abc/896-bit + 1600-bit 0xa3 multi-block). WASM boot
  green with new lib module; Erlang conformance 530/530; run_tests +4.
- 2026-05-18 — Phase A: pure-OCaml `lib/sx_sha2.ml` (SHA-256 + SHA-512),
  primitives `crypto-sha256`/`crypto-sha512`. 7 NIST FIPS 180-4 vectors pass
  (empty/abc/896-bit/1M-'a' for sha256; empty/abc/896-bit for sha512). WASM
  boot green with new lib module; Erlang conformance 530/530 unchanged.

## Blockers

- _(none yet)_