rose-ash/plans/fed-sx-host-primitives.md

fed-sx host primitives — hosts/ocaml/

The single blocker between Erlang Phase 8 (FFI mechanism — done) and starting fed-sx Milestone 1: the SX OCaml runtime exposes no crypto / CID / HTTP host primitives for the Phase 8 BIF wrappers to call. This plan adds exactly that surface, pure-OCaml where it must stay WASM-safe, native-only where it can't.

Reference: plans/fed-sx-milestone-1.md (build steps 1-8), plans/erlang-on-sx.md Blockers ("SX runtime lacks platform primitives …").

The hard constraint — WASM boundary

hosts/ocaml/lib/ is the sx library. hosts/ocaml/browser/dune links it with (modes byte js wasm). Anything added to lib/sx_primitives.ml must compile under js_of_ocaml AND wasm_of_ocaml. Therefore:

• Pure OCaml only for hash / CBOR / CID / Ed25519 / RSA. No digestif, no mirage-crypto, no C stubs, no Unix dependency in these primitives. (None of those libs are even installed — the switch has only re/unix/yojson/otfm/js_of_ocaml. Pure OCaml is both required and hermetic.)
• HTTP server is native-only: it needs sockets/threads. Register it in bin/sx_server.ml via Sx_primitives.register (precedent: eval-in-env at bin/sx_server.ml:721), not in the shared lib. It must never enter the WASM build.
• file-list-dir uses Sys.readdir (stdlib, WASM-stubbed) — safe in lib, but the fed-sx server is native anyway; native registration is acceptable too.

Every phase must prove the WASM build still links (sx_build target="wasm" or bash hosts/ocaml/browser/test_boot.sh) before its commit. A broken WASM browser kernel is a hard regression and fails the phase.

Primitive surface (what fed-sx Milestone 1 actually needs)

Mapped to plans/fed-sx-milestone-1.md build steps:

Primitive (SX name)	Signature	fed-sx step	Host
crypto-sha256	(bytes) -> hex-string	1, 2	lib (pure)
crypto-sha512	(bytes) -> hex-string	2	lib (pure)
crypto-sha3-256	(bytes) -> hex-string	1 (CID default)	lib (pure)
cbor-encode	(sx-value) -> bytes (dag-cbor, deterministic)	1	lib (pure)
cbor-decode	(bytes) -> sx-value	1 (round-trip tests)	lib (pure)
cid-from-bytes	(codec multihash-bytes) -> cid-string	1	lib (pure)
cid-from-sx	(sx-value) -> cid-string (canonicalize→cbor→sha→mh→cidv1)	1	lib (pure)
ed25519-verify	(pubkey-32 msg sig-64) -> bool	2	lib (pure)
rsa-sha256-verify	(der-spki msg sig) -> bool (PKCS#1 v1.5)	2	lib (pure)
file-list-dir	(path) -> (list string)	3	lib/native
http-listen	(port handler-fn) -> never (handler: req-dict→resp-dict)	8	native only

Deferred (not Milestone 1): httpc-request (HTTP client — federation is v2), sqlite-* (Milestone 1 is file-on-disk; sqlite is v2 indexes).

Registration pattern (established)

lib/sx_primitives.ml:

register "crypto-sha256" (fun args ->
  match args with
  | [String s] -> String (Sha2.sha256_hex s)
  | _ -> raise (Eval_error "crypto-sha256: (bytes)"))

Errors: raise (Eval_error "name: shape"). Byte strings are OCaml string (SX String). Lists are Pair/Nil per sx_types.ml. Native-only prims go in bin/sx_server.ml the same way.

Phasing — one feature per loop iteration

Dependency order. Each phase: implement → dune build (ocaml) → WASM build check → tests → commit → tick box → Progress-log line → push.

Phase A — SHA-2 (sha256 + sha512), pure OCaml ✅ DONE

• New lib/sx_sha2.ml (or inline in primitives if small): SHA-256 + SHA-512.
• Primitives crypto-sha256, crypto-sha512 → lowercase hex string.
• Tests (bin/run_tests.ml or a dedicated bin/test_crypto.ml): NIST vectors — "", "abc", the 896-bit message, a 1MB "a" repetition.
  • sha256("") = e3b0c442…b7852b855; sha256("abc") = ba7816bf…f20015ad
  • sha512("abc") = ddaf35a1…2a9ac94f…
• Acceptance: vectors pass; WASM build links; OCaml conformance unchanged.

Phase B — SHA-3 / Keccak-256, pure OCaml ✅ DONE

• Keccak-f[1600] + SHA3-256 padding. Primitive crypto-sha3-256.
• Tests: sha3-256("") = a7ffc6f8…0f8434a; sha3-256("abc") = 3a985da7…11431532.
• Acceptance: NIST SHA-3 vectors pass; WASM links.

Phase C — dag-cbor encoder + decoder, pure OCaml ✅ DONE

• RFC 8949 deterministic subset (RFC 8742 dag-cbor): unsigned/negative ints, byte strings, text strings, arrays, maps with keys sorted by length-then-bytewise, bool, null, tag 42 (CID link). No floats unless a fed-sx shape needs them (defer; document).
• SX↔CBOR mapping: Integer→int, String→text str, Bool, Nil→null, Pair/Nil→array, Dict→map (sorted keys), keyword/symbol→text str.
• Primitives cbor-encode, cbor-decode. Round-trip property tests + RFC 8949 appendix-A vectors + a "reordered dict keys → identical bytes" determinism test.
• Acceptance: vectors + round-trip + determinism pass; WASM links.

Phase D — CID computation, pure OCaml ✅ DONE

• Multihash (sha2-256 = 0x12, sha3-256 = 0x16; varint code + varint len + digest).
• CIDv1 = 0x01 || codec-varint || multihash. Codecs: dag-cbor 0x71, raw 0x55.
• Multibase base32 lower (b prefix, RFC 4648 no-pad).
• Primitives cid-from-bytes (codec, raw mh bytes), cid-from-sx (canonicalize → cbor-encode → sha2-256 → multihash → cidv1 → base32).
• Tests: known IPFS CIDs — cross-check against ipfs CLI if present, else the fixed vectors for {} dag-cbor and "abc" raw (hardcode expected strings). Determinism: same SX value (whitespace/comment/key-order variants) → same CID.
• Acceptance: matches reference CIDs; determinism holds; WASM links. Satisfies fed-sx Milestone 1 Step 1.

Phase E — Ed25519 verify, pure OCaml ✅ DONE

• Curve25519/edwards25519 field arith (mod 2^255-19), point decompress, SHA-512-based verify per RFC 8032 §5.1.7. (Reuse Phase A sha512.)
• Primitive ed25519-verify (pubkey msg sig) -> bool. Bad-length args → false, not exception (verify is total).
• Tests: RFC 8032 §7.1 vectors (TEST 1-4 + the 1024-byte one). Tampered msg/sig → false. Wrong-length key → false.
• Acceptance: all RFC 8032 vectors pass; WASM links. Satisfies fed-sx Step 2 (Ed25519 sig-suite).

Phase F — RSA-SHA256 verify (PKCS#1 v1.5), pure OCaml ✅ DONE

• Minimal pure-OCaml bignum (only need modexp + DER parse). Parse SPKI DER → (n, e). RSASSA-PKCS1-v1_5 verify with SHA-256 (Phase A).
• Primitive rsa-sha256-verify (der-spki msg sig) -> bool.
• Tests: a generated 2048-bit keypair's signature (vectors hardcoded in the test from a one-off openssl run, documented in a comment), tamper → false.
• Acceptance: vector verifies; tamper fails; WASM links. Satisfies fed-sx Step 2 (rsa-sha256-2018 sig-suite). Lower priority than E — Ed25519 is the modern default; RSA can land after the HTTP phase if time-boxed.

Phase G — file-list-dir, native-safe ✅ DONE

• Sys.readdir → sorted SX list of names (no ./..). Errors → enoent/ enotdir classified like the existing file-read error mapping.
• Tests: list a known dir, missing dir → error, file-not-dir → error.
• Acceptance: passes; WASM build still links (Sys.readdir is stubbed there). Satisfies fed-sx Step 3 segment replay.

Phase H — HTTP/1.1 server, native-only (bin/sx_server.ml)

• Minimal threaded HTTP/1.1: accept loop (Unix + Thread), parse request line + headers + body (Content-Length), build an SX request dict {:method :path :query :headers :body}, call the SX handler callable, take an SX response dict {:status :headers :body}, write it. Connection: close (keep-alive optional, defer). Bind 127.0.0.1:<port>.
• Primitive http-listen (port handler) -> never-returns registered ONLY in bin/sx_server.ml. Document that it is absent from the WASM kernel.
• Tests: bin/test_http.sh — start a server on a port with a tiny SX echo handler in a subprocess, curl GET/POST/404/headers, assert responses, kill.
• Acceptance: curl test script green; WASM build untouched (prim not in lib). Satisfies fed-sx Step 8 transport.

Phase I — handoff

• Flip the plans/erlang-on-sx.md Blockers entry "SX runtime lacks platform primitives …" to RESOLVED, listing the exact SX primitive names so the Erlang loop can one-line-wire its blocked Phase 8 BIFs (crypto:hash/2, cid:from_bytes/1, cid:to_string/1, file:list_dir/1, plus note httpc/sqlite still deferred). Do not edit lib/erlang/ — that wiring is the Erlang loop's job; this phase only updates the blocker text + this plan's "Handoff" section with the primitive→BIF mapping.
• Acceptance: blocker text updated; fed-sx Milestone 1 Steps 1-3 + 8 prerequisites all green.

Scope (hard)

• Edit only: hosts/ocaml/lib/**, hosts/ocaml/bin/**, this plan file.
• Do NOT edit: lib/erlang/** (Erlang loop owns BIF wiring), spec/, lib/ root, other lib/<lang>/, plans/erlang-on-sx.md except the one Blockers entry in Phase I.
• Pure OCaml for lib primitives. No new opam deps. If a phase seems to need one, stop and add a Blockers entry instead.
• Prove WASM every phase. No commit without test_boot.sh (or wasm build) green.
• Never push to main or architecture. Branch loops/fed-prims, push origin/loops/fed-prims.
• One feature per commit. Short factual messages: fed-prims: SHA-256 + 4 NIST vectors. Tick the box, append a dated Progress-log line (newest first).
• Never call sx_build with no timeout-awareness — OCaml builds are slow; use the MCP sx_build target="ocaml" / target="wasm" tools or dune build with a generous timeout. If the build hangs >10min, Blockers + stop.

Build & test reference

cd hosts/ocaml && dune build bin/sx_server.exe 2>&1 | tail   # native
bash hosts/ocaml/browser/test_boot.sh                        # WASM links + boots
cd hosts/ocaml && dune exec bin/run_tests.exe 2>&1 | tail     # OCaml unit tests
SX_SERVER=hosts/ocaml/_build/default/bin/sx_server.exe \
  timeout 400 bash lib/erlang/conformance.sh 2>&1 | tail -3   # no-regression gate

A primitive is reachable from SX via the epoch protocol:

printf '(epoch 1)\n(crypto-sha256 "abc")\n' | \
  hosts/ocaml/_build/default/bin/sx_server.exe

Handoff (Phase I fills this in)

SX primitive	Erlang Phase 8 BIF it unblocks
crypto-sha256 / crypto-sha512 / crypto-sha3-256	crypto:hash/2
cid-from-bytes / cid-from-sx	cid:from_bytes/1, cid:to_string/1
ed25519-verify / rsa-sha256-verify	crypto:verify / sig-suites
file-list-dir	file:list_dir/1
http-listen	fed-sx kernel http:listen/2 (Milestone 1 Step 8)

Progress log

Newest first.

• 2026-05-18 — Phase G: file-list-dir primitive in lib/sx_primitives.ml (Sys.readdir → sorted names, no "."/".."; Sys_error prefixed like file-read, msg carries enoent/enotdir). 4 tests: sorted listing, missing dir, not-a-dir, arity. WASM boot green (Sys.readdir stubbed there); Erlang 530/530; run_tests +4. Satisfies fed-sx Step 3 segment replay.
• 2026-05-18 — Phase F: pure-OCaml lib/sx_rsa.ml (self-contained bignum modexp, minimal DER SPKI reader, RFC 8017 §8.2.2 PKCS#1 v1.5 verify with SHA-256 DigestInfo prefix). Primitive rsa-sha256-verify total. 5 tests on a fixed RSA-2048 vector (one-off python-cryptography keygen, hardcoded): valid, tampered msg/sig, garbage SPKI, non-string. WASM boot green with new lib module; Erlang 530/530; run_tests +5. Satisfies fed-sx Step 2 (rsa-sha256-2018 sig-suite).
• 2026-05-18 — Phase E: pure-OCaml lib/sx_ed25519.ml (minimal base-2^26 bignum, edwards25519 extended-coord points, RFC 8032 §5.1.7 cofactorless verify reusing Phase-A sha512). Primitive ed25519-verify is total (bad/short/non-string args → false). 8 tests: RFC 8032 §7.1 TEST 1-3 (re-derived independently via python-cryptography), tampered msg/sig, wrong-length, non-string. WASM boot green with new lib module; Erlang 530/530; run_tests +8. Satisfies fed-sx Milestone 1 Step 2 (Ed25519 sig-suite).
• 2026-05-18 — Phase D: pure-OCaml lib/sx_cid.ml (unsigned-varint, multihash, CIDv1, multibase base32-lower), primitives cid-from-bytes / cid-from-sx (cbor→sha2-256→mh→cidv1, dag-cbor codec 0x71). 5 tests: raw "abc"=bafkreif2pall7d…, raw ""=bafkreihdwdcefg…, dag-cbor {}= bafyreigbtj4x7i… (all match canonical IPFS CIDs; no ipfs CLI so vectors independently derived in Python), key-order determinism. WASM boot green with new lib module; Erlang 530/530; run_tests +5.
• 2026-05-18 — Phase C: pure-OCaml lib/sx_cbor.ml (dag-cbor encode/ decode), primitives cbor-encode/cbor-decode. RFC 8949 Appendix-A vectors, length-then-bytewise key sort + order-invariance determinism, decode∘encode round-trip (30 tests). Floats unsupported (raise, no fed-sx shape needs them); tag-42 decode = inner-item passthrough. WASM boot green with new lib module; Erlang 530/530; run_tests +30.
• 2026-05-18 — Phase B: pure-OCaml lib/sx_sha3.ml (Keccak-f[1600] + SHA-3 pad, domain 0x06), primitive crypto-sha3-256. 4 NIST FIPS 202 vectors pass (empty/abc/896-bit + 1600-bit 0xa3 multi-block). WASM boot green with new lib module; Erlang conformance 530/530; run_tests +4.
• 2026-05-18 — Phase A: pure-OCaml lib/sx_sha2.ml (SHA-256 + SHA-512), primitives crypto-sha256/crypto-sha512. 7 NIST FIPS 180-4 vectors pass (empty/abc/896-bit/1M-'a' for sha256; empty/abc/896-bit for sha512). WASM boot green with new lib module; Erlang conformance 530/530 unchanged.

Blockers

• (none yet)