# fed-sx Milestone 1 — Kernel + Registries + Pin Smoke Test Concrete implementation plan for the smallest fed-sx that proves the architecture works end-to-end. Reference: `plans/fed-sx-design.md`. Prerequisite: Erlang-on-SX Phases 7 (hot reload) + 8 (FFI BIFs). ## Goal Ship a single-instance, single-actor fed-sx server that: 1. Boots from a verified genesis bundle. 2. Accepts and durably appends signed activities via `POST /activity`. 3. Folds them into projections in real time. 4. Serves AP-standard endpoints (actor, outbox, artifacts, capabilities). 5. Demonstrates **two extensibility proof-points** end-to-end with zero kernel code changes between definition and use: - **Verb extensibility** (§5 meta-level): publish `DefineActivity{Pin}` + `DefineProjection{pin-state}`, then publish a `Pin` activity, observe it validated and projected. - **Reactive application extensibility** (§§18-19): publish `DefineSubscription{Topic}` + `Subscribe{topic: smoketest}` + `DefineTrigger{when: that subscription, then: publish TestEcho}`, then publish a tagged Note, observe the subscription match, the trigger fire, and the derived activity appear in the outbox. Federation, multi-actor, advanced verbs, IPFS, browser UI, operator dashboard are **explicitly v2**. ## Non-goals (what milestone 1 deliberately does NOT do) - **Federation.** No `POST /inbox` from peers, no `Follow`, no delivery queue, no webfinger discovery flow. Single instance only. - **Multi-actor.** Single domain actor (`acct:next@next.rose-ash.com`). - **IPFS / S3 storage backends.** Files on disk only. - **Advanced verbs.** No `Endorse`, `Supersede`, `Test`, `Build`, `Compose`, `Note`, `Announce`. Only the four bootstrap verbs (`Create`, `Update`, `Delete`) plus a defined-from-the-log `Pin` for the smoke test. (`Announce` deferred — no use case until federation exists.) - **Browser UI.** Curl-shaped API only. - **Operator dashboard, quarantine UX.** Logs only. - **Performance work.** Functional correctness first; perf when measured. - **Cross-host conformance test corpus.** Only the OCaml/Erlang-on-SX host runs fed-sx in v1; conformance suite for other hosts is v2. ## Architecture summary ``` POST /activity │ ▼ ┌──────────────────────────┐ │ HTTP server (Erlang-on-SX)│ └─────────────┬─────────────┘ │ ┌─────────────▼──────────────┐ │ Validation pipeline driver │ │ (envelope→sig→schema→...) │ └─────────────┬──────────────┘ │ ┌─────────────▼──────────────┐ │ Log append (JSONL segment) │ ← canonical └─────────────┬──────────────┘ │ ┌─────────────▼──────────────┐ │ Projection workers │ ← gen_server per │ (fold scheduler) │ projection └─────────────────────────────┘ │ ▼ Projection state (queryable via HTTP) Native primitives (Erlang-on-SX BIFs from Phase 8): crypto:* cid:* fs:* http:* sqlite:* Genesis bundle (binary-embedded SX): activity-types object-types projections validators codecs sig-suites ``` ## Build order Eight steps in dependency order. Each step has concrete deliverables, testable in isolation, and a clear acceptance check. | Step | Title | Depends on | |------|-------|------------| | **1** | Repo skeleton + canonical CID computation | Phase 8 (cid BIFs) | | **2** | Activity envelope + signature verify | Phase 8 (crypto BIFs) | | **3** | JSONL log + sequence numbers | Phase 8 (fs BIFs) | | **4** | Genesis bundle (SX sources + bundling + CID verification) | Step 1 | | **5** | Registry mechanism + bootstrap-projection dispatch | Steps 2, 4 | | **6** | Validation pipeline driver + `POST /activity` | Steps 2, 3, 5 | | **7** | Projection scheduler (gen_server per projection) | Steps 5, 6 | | **8** | HTTP server, AP endpoints, projection queries | Steps 6, 7 | | **9** | Smoke tests (Pin verb + reactive application) | Steps 1-8 | --- ## Step 1 — Repo skeleton + canonical CID **Sub-deliverables:** - [x] **1a** — `next/` directory skeleton, README, `.gitignore` for `data/` - [x] **1b** — `next/kernel/nx_cid.erl` (from_sx/to_string/from_string/equals) + `next/tests/cid.sh` (13 cases). Module is `nx_cid` not `cid` — the `cid` BIF module would be shadowed by a user module of the same name; plan §Step 1's `cid.erl` is illustrative per briefing. **Deliverables:** ``` next/ ├── README.md # what this is ├── kernel/ # Erlang-on-SX │ └── (empty for now) ├── genesis/ # core SX bootstrap definitions │ └── (empty for now) ├── tests/ # smoke test scripts │ └── (empty for now) └── data/ # gitignored runtime state ├── log/ ├── objects/ ├── snapshots/ ├── indexes/ └── keys/ ``` Plus one Erlang-on-SX module: ```erlang % next/kernel/cid.erl -module(cid). -export([from_sx/1, to_string/1, from_string/1, equals/2]). from_sx(SxValue) -> Cbor = cid:cbor_encode(canonicalize_sx(SxValue)), Hash = crypto:sha2_256(Cbor), cid:from_bytes(<<"raw">>, Hash). % defaults to dag-cbor codec canonicalize_sx(V) -> ... % sorts dict keys, normalizes strings ``` **Tests:** - Same SX value → same CID across multiple invocations. - Different SX values → different CIDs. - Whitespace/comment differences in source → identical CIDs (parsed AST identical). - Reordered dict keys → identical CIDs (sorted-key canonicalization). - Cross-host parity (just OCaml host for v1, but write the test so adding hosts is mechanical). **Acceptance:** `bash next/tests/cid.sh` passes 10+ cases. --- ## Step 2 — Activity envelope + signature verify **Sub-deliverables:** - [x] **2a** — `next/kernel/envelope.erl` `validate_shape/1` + `get_field/2` (property-list envelope; Erlang maps `#{}` not supported in this port) + `next/tests/envelope_shape.sh` (15 cases) - [x] **2b** — `canonical_bytes/1` over sig-stripped, key-sorted envelope (deterministic textual form via `cid:to_string` substrate; dag-cbor stand-in for v1) + `next/tests/envelope_canonical.sh` (8 cases) - [x] **2c** — `verify_signature/2` against actor `public_keys`, time-aware key validity per design §9.6 (created ≤ published, optional supersession check) + `next/tests/envelope_sig.sh` (11 cases). Signature scheme is HMAC-shaped (`crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`) — RSA/Ed25519 verify deferred to m2 (BIFs not yet wired). **Deliverables:** ```erlang % next/kernel/envelope.erl -module(envelope). -export([validate_shape/1, canonical_bytes/1, verify_signature/2]). % Envelope shape per design §3.1: % #{id, type, actor, published, to, cc, audience_extras, % object | target | origin | result, % capabilities_required, proofs, signature} validate_shape(Activity) -> ok | {error, Reason}. canonical_bytes(Activity) -> % Strip signature, canonicalize via dag-cbor, return bytes for sig coverage Stripped = maps:remove(signature, Activity), cid:cbor_encode(canonicalize_for_sig(Stripped)). verify_signature(Activity, ActorState) -> % Time-aware: find key with id == sig.key_id that was active at published % Per design §9.6 ... ``` **Tests:** - Envelope shape: required fields present (id, type, actor, published, signature) - Envelope shape: type is a known activity-type or unknown-but-string - Envelope shape: signature has key_id, algorithm, value - Sig verify: valid RSA-SHA256 signature against published key → ok - Sig verify: valid Ed25519 signature → ok - Sig verify: tampered envelope → fail - Sig verify: key superseded before activity timestamp → fail - Sig verify: key superseded after activity timestamp → ok (historical valid) **Acceptance:** `bash next/tests/envelope.sh` passes 15+ cases. --- ## Step 3 — JSONL log + sequence numbers **Sub-deliverables:** - [x] **3a** — `log:open/2` + `log:append/2` + `log:tip/1` + `log:replay/3` + `log:entries/1` over an in-memory log state (per-actor seq; replay in append order; round-trip the stored activity). `next/tests/log_memory.sh` (12 cases). - [ ] **3b** — *Parked behind substrate gap (see Blockers below).* Term codec + on-disk persistence: serializer/parser writing each activity as a JSONL-style line; restart-resumes-tip from the segment file. - [ ] **3c** — Segment rotation at size threshold + gen_server-mediated concurrent appends. **Blockers (Step 3b):** The Erlang port returns SX strings (an opaque OCaml-string type) from `atom_to_list/1` and `integer_to_list/1`, rejects them from `++`/list pattern matching, and does not register `binary_to_list`/`list_to_binary`. `$X` character literals decode to `nil` in `parse-number`. Net effect: there is no in-Erlang path from an arbitrary term to a byte sequence (or back) that doesn't go through a temp-file round-trip through the filesystem. Workaround paths: (a) add a `term_to_binary`/`binary_to_term` BIF in a separate substrate loop, (b) accept a filesystem-mediated SX-string→binary helper and live with the O(N) IO cost, (c) restrict the on-disk format to a binary-only encoding with a per-instance atom-id table for atoms (introduces an extra durability dependency). Decision to defer; revisit once a downstream Step (5–8) forces the issue or a substrate BIF arrives. In-memory log from 3a is sufficient to unblock Step 5+ which consume the API surface. **Deliverables:** ```erlang % next/kernel/log.erl -module(log). -export([open/1, append/2, read_segment/2, tip/1, replay/3]). % Per design §15.2: per-actor outbox, segments cap ~64MB, % format = JSONL (one canonical JSON-LD activity per line) open(ActorId) -> BasePath = log_path_for_actor(ActorId), fs:mkdir_p(BasePath), {ok, #{base => BasePath, current => current_segment(BasePath), seq => next_seq(BasePath)}}. append(LogState, Activity) -> Json = jsonld:encode(Activity), Path = current_segment_path(LogState), Line = <>, fs:append_file(Path, Line), NewSeq = LogState#{seq := LogState.seq + 1}, rotate_if_needed(NewSeq). % replay/3 calls Fun(Activity, Acc) for every activity in chronological order replay(LogState, InitAcc, Fun) -> ... ``` **Tests:** - Append + read back gives identical activity (round-trip). - Sequence numbers monotonic and gap-free per actor. - Segment rotation at size threshold. - Replay visits all activities in append order across multiple segments. - Restart preserves tip pointer (seq number resumes correctly). - Concurrent appends (using gen_server-mediated access) are serialized correctly. **Acceptance:** `bash next/tests/log.sh` passes 10+ cases. --- ## Step 4 — Genesis bundle **Sub-deliverables:** - [x] **4a** — Seed genesis SX file authoring: `next/genesis/manifest.sx` + `next/genesis/activity-types/create.sx`. Manifest uses bare parenthesised paths (data lists, not `(list ...)` calls — consumed by `parse`, not `eval`). `next/tests/genesis_parse.sh` (5 cases). - [x] **4b-act** — Remaining activity-types: `update.sx` + `delete.sx`, manifest updated, parse tests (10 cases total in `genesis_parse.sh`) - [x] **4b-obj** — Object-types: SXArtifact, Note, Tombstone, DefineActivity, DefineObject, DefineProjection, DefineValidator, DefineCodec, DefineSigSuite, Snapshot — 10 `DefineObject` files + manifest updated + 12 new parse tests - [x] **4b-proj** — Projections: activity-log, by-type, by-actor, by-object, actor-state, define-registry, audience-graph — 7 `DefineProjection` files + manifest updated + 9 new parse tests - [x] **4b-vld** — Validators: envelope-shape, signature, type-schema — 3 `DefineValidator` files + manifest updated + 5 new parse tests - [x] **4b-cod** — Codecs (dag-cbor, raw, dag-json) + sig-suites (rsa-sha256-2018, ed25519-2020) + audience predicates (Public, Followers, Direct) — 8 SX files + manifest fully populated + 14 new parse tests - [x] **4c** — `bootstrap:read_genesis/0,1` + `read_section/2` + `sections/0` + `section_subdir/1` + `ends_with_sx/1` in Erlang: walk seven hardcoded section subdirs, filter `.sx` files via byte-pattern suffix match, read each into a binary. Returns `{ok, [{Section, [{Name, Bytes}, ...]}, ...]}`. Skips SX parsing — the substrate has no in-Erlang binary→SX-term path (same gap as Step 3b); bundle CID over raw bytes is enough for Step 4d. `next/tests/bootstrap_read.sh` (15 cases). - [x] **4d** — `bootstrap:build_genesis/1` + `verify_genesis/2` + `cidhash_path/1` + `write_cidhash/2` + `read_cidhash/1`: bundle CID via host `cid:to_string` over `{genesis_bundle, Sections}`; mismatch returns `{error, {cid_mismatch, Got, Expected}}`; `.cidhash` sibling file persists between runs. `next/tests/bootstrap_build.sh` (12 cases). - [x] **4e** — `bootstrap:load_genesis/1` + `strip_sx_suffix/1`: bridges `read_genesis` output into `registry` entries. Section atom = registry kind; entry name = filename minus `.sx` (binary); entry value = raw file bytes (parsed forms replace these once an SX-parser bridge exists). `next/tests/bootstrap_load.sh` (15 cases). **Deliverables:** Genesis bundle SX sources (per design §12.2). Each is a small SX file authored by hand for the bootstrap set: ``` next/genesis/ ├── manifest.sx # bundle root: lists all definitions ├── activity-types/ │ ├── create.sx # DefineActivity{name: "Create", ...} │ ├── update.sx │ └── delete.sx ├── object-types/ │ ├── sx-artifact.sx │ ├── note.sx │ ├── tombstone.sx │ ├── define-activity.sx # DefineObject for the Define* meta types │ ├── define-object.sx │ ├── define-projection.sx │ ├── define-validator.sx │ ├── define-codec.sx │ ├── define-sig-suite.sx │ └── snapshot.sx ├── projections/ │ ├── activity-log.sx # identity projection │ ├── by-type.sx │ ├── by-actor.sx │ ├── by-object.sx │ ├── actor-state.sx │ ├── define-registry.sx # the chicken-and-egg projection │ └── audience-graph.sx ├── validators/ │ ├── envelope-shape.sx │ ├── signature.sx │ └── type-schema.sx ├── codecs/ │ ├── dag-cbor.sx # delegates to cid:cbor_encode/decode BIFs │ ├── raw.sx │ └── dag-json.sx ├── sig-suites/ │ ├── rsa-sha256-2018.sx │ └── ed25519-2020.sx └── audience/ ├── public.sx ├── followers.sx └── direct.sx ``` Plus a build-time bundler: ```erlang % next/kernel/bootstrap.erl -module(bootstrap). -export([build_genesis/1, verify_genesis/1, load_genesis/1]). build_genesis(SourceDir) -> % Walk SourceDir, parse each .sx file, build a single dag-cbor bundle, % compute its CID, write bundle.cbor + CID to data/genesis/ ... verify_genesis(BundlePath) -> % Compute CID of the bundle as loaded; compare to expected (hardcoded % in the kernel binary). Mismatch → halt. ... load_genesis(BundlePath) -> % Parse the bundle, register all definitions in the in-memory registry ... ``` **Tests:** - All genesis SX files parse cleanly. - Bundle CID is deterministic (rebuild same sources → same CID). - Bundle reload reproduces the exact same registry state. - Tampered bundle → `verify_genesis` returns `{error, cid_mismatch}`. **Acceptance:** `bash next/tests/bootstrap.sh` passes; `next/data/genesis/bundle.cbor` created with a known stable CID. --- ## Step 5 — Registry mechanism + bootstrap dispatch **Sub-deliverables:** - [x] **5a** — Pure-functional `next/kernel/registry.erl`: `new/0`, `kinds/0`, `register/4`, `lookup/3`, `list/2`. State is a property list keyed by kind atom; per-kind storage is a property list of `{Name, Entry}`. Unknown kinds rejected with `{error, unknown_kind}`. `next/tests/registry_pure.sh` (14 cases). - [x] **5b** — gen_server wrapper around the pure registry: `start_link/0`, registered name `registry`, `register/3 lookup/2 list/1 stop/0` API delegating through `gen_server:call`. `next/tests/registry_server.sh` (12 cases). Port note: each test combines start_link + ops in a single expression because spawned processes don't survive across separate `erlang-eval-ast` invocations. - [x] **5c-populate** — `bootstrap:populate_registry/0` walks `read_genesis` output and calls `registry:register/3` (the gen_server API) for each entry. Returns the total entries registered. `next/tests/bootstrap_populate.sh` (14 cases). - [ ] **5d** — define-registry projection fold integration: incoming `Create{Define*}` activities are routed through the projection scheduler (Step 7) and update the registry. **Deliverables:** Registries are gen_servers, one per kind, each holding the active version map: ```erlang % next/kernel/registry.erl -module(registry). -behaviour(gen_server). -export([start_link/0, lookup/2, register/3, list/1]). % Internal state: % #{activity_types => #{Name => #{cid, schema_fn, semantics_fn, supersedes}}, % object_types => ..., % projections => ..., % validators => ..., % codecs => ..., % sig_suites => ..., % ...} lookup(Kind, Name) -> {ok, Entry} | {error, not_found}. register(Kind, Name, Entry) -> ok | {error, Reason}. list(Kind) -> [#{name, cid}]. ``` The `define-registry` projection's fold updates this gen_server's state when new `Define*` activities arrive. (Bootstrapping circle resolved: at startup, `bootstrap:load_genesis/1` populates the registry directly; from then on, the projection fold maintains it.) **Tests:** - After genesis load, `registry:list(activity_types)` returns Create/Update/Delete. - `registry:lookup(activity_types, "Create")` returns the schema and semantics. - A new `DefineActivity{name: "Pin"}` activity (synthesised, hand-signed for the test) routes through the projection fold, ends up in the registry. - Lookup never caches across activities (verified by introducing a new definition mid-test and confirming the next lookup sees it). **Acceptance:** `bash next/tests/registry.sh` passes 10+ cases. --- ## Step 6 — Validation pipeline + POST /activity **Sub-deliverables:** - [x] **6a** — `pipeline:run_stages/2` driver — pure fold over a stage list of `(Activity) -> ok | {error, R}` funs, halts on first failure. `validate_inbound/1` + `validate_outbound/1` + `inbound_stages/0` + `outbound_stages/0` (empty lists for now). `next/tests/pipeline_driver.sh` (10 cases). - [x] **6b-env** — `pipeline:stage_envelope/1` delegating to `envelope:validate_shape/1`; wired into both `inbound_stages` and `outbound_stages`. `next/tests/pipeline_envelope.sh` (12 cases); pipeline_driver.sh updated to test the driver in isolation. - [x] **6b-sig** — `pipeline:stage_signature/2` (direct call) + `stage_signature/1` (factory returning a context-bound stage fun). Not wired into default stage lists since ActorState isn't available at static-list build time; callers compose by `Stages = [..., pipeline:stage_signature(AS)]`. `next/tests/pipeline_signature.sh` (11 cases) covers direct + factory + composition + halt behaviour with stage_envelope. - [x] **6c-replay** — `pipeline:stage_replay/2` (direct) + `stage_replay/1` (factory closed over LogState). Checks the log entries for an existing activity with the same `:id`. Returns `{error, replay}` on duplicate, `{error, no_id}` when missing. `next/tests/pipeline_replay.sh` (12 cases). - [ ] **6c-schema** — `stage_activity_schema/1` (registry lookup of activity-type, evaluate :schema body) — blocked behind SX-source eval bridge. - [x] **6d-cs** — `outbox:construct/4` (skeleton + CID-derived :id via `cid:to_string`) + `outbox:sign/2` (HMAC over canonical bytes, append :signature pair from KeySpec) + `cid_of/1` accessor. Verified end-to-end: construct→sign→envelope:verify_signature passes; wrong key material fails with bad_signature. `next/tests/outbox_construct.sh` (13 cases). - [x] **6d-publish** — `outbox:publish/2(Request, Context)` orchestrates construct + sign + `pipeline:run_stages([envelope, signature, replay])` + `log:append`. Returns `{ok, [{cid, _}, {activity, _}], NewLog}` or `{error, Reason, LogState}` on stage halt. Replay catches duplicate publishes; bad key material surfaces `bad_signature`. `next/tests/outbox_publish.sh` (13 cases). - [ ] **6e** — HTTP handler for POST /activity glue (depends on Step 8 http server) **Deliverables:** ```erlang % next/kernel/pipeline.erl -module(pipeline). -export([validate_inbound/1, validate_outbound/1]). % Per design §14, run stages in order, halt on first failure. validate_inbound(Activity) -> Stages = [ fun stage_envelope/1, fun stage_signature/1, fun stage_replay/1, fun stage_audience/1, fun stage_activity_schema/1, fun stage_object_schema/1, fun stage_content_validators/1, fun stage_capabilities/1, fun stage_trust/1 ], run_stages(Activity, Stages). validate_outbound(Activity) -> % Subset of inbound stages (no replay, no trust check; auth done at HTTP layer) ... ``` ```erlang % next/kernel/outbox.erl -module(outbox). -export([publish/2]). publish(ActorId, ActivityRequest) -> Activity = construct_envelope(ActorId, ActivityRequest), Signed = sig:sign(Activity, ActorId), case pipeline:validate_outbound(Signed) of ok -> log:append(actor_log(ActorId), Signed), projection:async_fold(Signed), {ok, #{cid => cid:from_sx(Signed), ap_id => maps:get(id, Signed)}}; {error, Reason} -> {error, Reason} end. ``` **Tests:** - Valid activity through full pipeline → appended to log. - Bad envelope → 400, not in log. - Bad signature → 401, not in log. - Replayed activity → 200 duplicate, not re-appended. - Schema violation (e.g. Create with no object) → 422. - Activity logged before projection completes (async). **Acceptance:** `bash next/tests/pipeline.sh` passes 15+ cases. --- ## Step 7 — Projection scheduler **Sub-deliverables:** - [x] **7a** — Pure-functional `next/kernel/projection.erl`: `new/2,3`, `fold_activity/2`, `replay/2`, `name/1`, `state/1`, `fold_fn/1`. Projection record is `[{name, _}, {state, _}, {fold, fun}]`; fold body is an Erlang fun in v1 (SX-source eval bridge deferred). `next/tests/projection_pure.sh` (12 cases). - [x] **7b** — gen_server-per-projection: `start_link/3(Name, InitialState, FoldFn)` + `async_fold/2(Name, Activity)` (cast) + `query/1(Name)` (call) + `stop/1`. Each projection registered under its own Name atom. `next/tests/projection_server.sh` (11 cases). Snapshot persistence deferred (needs SX-source eval + on-disk state). - [x] **7c** — `outbox:publish` broadcast hook: after `log:append`, fans out the signed activity to every projection listed under `Context`'s `:projections` entry via `projection:async_fold`. Stage halts (replay, sig failure) skip broadcast. `next/tests/outbox_broadcast.sh` (14 cases). - [ ] **7d** — `sandbox:eval_pure/2` (Erlang sandbox-mode caller — gas budget + IO denial) once an SX-source eval bridge exists. **Deliverables:** ```erlang % next/kernel/projection.erl -module(projection). -export([start_link/1, async_fold/1, query/2, snapshot/1]). -behaviour(gen_server). % One gen_server per active projection. State: % #{cid, name, fold_fn, current_state, log_tip, % snapshot_dir, last_snapshot_at} % async_fold/1 broadcasts a new activity to every projection gen_server; % each folds it into its own state. Failures (gas, sandbox violation) % tag the activity but don't affect log durability. % query/2 returns current state (or state-as-of) % snapshot/1 forces a snapshot now (also runs periodically) ``` ```erlang % next/kernel/sandbox.erl -module(sandbox). -export([eval_pure/2, eval_crypto/2, eval_effectful/3]). % eval_pure runs an SX function in pure mode: no IO platform, gas budget, % deterministic. Used by projection folds, validators, audience predicates. % Wrapper over the SX runtime evaluator with a stripped platform. ``` **Tests:** - New activity → all projections fold it concurrently. - Projection fold completes within gas budget. - Gas-exhausting fold → activity tagged, projection state unchanged, no kernel crash. - Sandbox violation (fold tries IO) → same handling. - Snapshot create + reload → state matches. - Snapshot CID stable across kernel restarts. **Acceptance:** `bash next/tests/projection.sh` passes 15+ cases. --- ## Step 8 — HTTP server + endpoints **Sub-deliverables:** - [x] **8a** — `http:listen/2` BIF wrapper in `lib/erlang/runtime.sx` (the briefing's allowed exception). Validates args, bridges Erlang handler funs to SX-callable lambdas via `er-of-sx`/`er-to-sx`, delegates to the native `http-listen` primitive in `bin/sx_server.ml`. Tests verify registration + arg validation (not the blocking listen loop). `next/tests/http_listen_bif.sh` (5 cases). - [x] **8b-route** — `next/kernel/http_server.erl`: pure `route/1` dispatch + `ok_response/1`, `not_found_response/0`, `welcome_body/0`. GET / returns welcome; everything else returns 404 (graceful for missing fields). `next/tests/http_route.sh` (11 cases). - [ ] **8b-start** — `start/1(Port)` spawns an Erlang process hosting `http:listen/2`, requires the dict↔proplist marshaling bridge in the BIF wrapper. - [x] **8c-cap** — Route GET `/.well-known/sx-capabilities` (static doc: kernel/version/verbs lines). `next/tests/http_capabilities.sh` (8 cases). Other concrete routes follow. - [x] **8c-actors-doc** — `match_prefix/2` byte-level path-prefix matcher + GET `/actors/{id}` route returning an `actor: ` stub body. `/actors/{id}/outbox` deferred (needs path-segment splitting). `next/tests/http_actors.sh` (13 cases). - [x] **8c-art** — Route GET `/artifacts/{cid}` via `match_prefix`. Stub body echoes the cid (`artifact: \n`); real content store lookup deferred. `next/tests/http_artifacts.sh` (9 cases). - [x] **8c-proj** — Routes GET `/projections` (list stub) + GET `/projections/{name}` (state stub) via `match_prefix`. Bare-path list endpoint dispatches before the prefix clause. `next/tests/http_projections.sh` (11 cases). Registry-backed implementation deferred. - [x] **8c-post-auth** — `route/2(Req, Cfg)` adds POST `/activity` with bearer-token check. Cfg `:publish_token` is the expected token; missing / wrong / malformed Authorization all return 401. Authorized requests get a stub 200 ("published (stub)"). `next/tests/http_post_activity.sh` (13 cases). - [x] **8c-post-publish-pure** — `next/kernel/nx_kernel.erl` — pure-functional kernel orchestrator. `new/3(ActorId, KeySpec, ActorState)` builds the runtime state; `publish/2(Request, State)` calls `outbox:publish` with a Context derived from state, advances log + next_published on success. `next/tests/nx_kernel_pure.sh` (12 cases). - [x] **8c-post-publish-srv** — gen_server wrapper around nx_kernel: `start_link/3`, named-process `publish/1`, `query/0`, `log_tip/0`, `with_projections/1`, `stop/0`. `next/tests/nx_kernel_server.sh` (11 cases). HTTP layer integration follows. - [x] **8c-post-publish-http** — POST `/activity` handler now calls `nx_kernel:publish/1` when the kernel process is registered; falls back to the existing stub when not. Success → 200 with `cid: \n` body via `cid_response/1`; sig/replay failures → 422 via `validation_failed_response/0`. `next/tests/http_publish.sh` (10 cases). - [x] **8d-accept** — `accept_format/1` + `accept_format_from/1` parse the Accept header into `:activity_json | :json | :sx | :cbor | :text`. Priority: activity+json > json > sx > cbor; everything else falls to text. `next/tests/http_accept.sh` (13 cases). - [x] **8d-dispatch-cap** — `capabilities_body_for/1` returns distinct stubs per format (json `{...}`, sx `(...)`, cbor `A1 64 caps 69 fed-sx-m1`); activity_json shares the json body. Route intercepts GET capabilities to thread the Accept format through `accept_format_from/1`. `next/tests/http_capabilities_format.sh` (13 cases). - [x] **8d-content-type** — `content_type_for/1` maps format atoms to MIME-type binaries (text/plain, application/json, application/activity+json, application/sx, application/cbor). `ok_response/2(Body, Format)` builds a 200 response with the right Content-Type header. `next/tests/http_content_type.sh` (13 cases). - [x] **8d-dispatch-post** — POST `/activity` now threads the Accept format through both kernel-present (`cid_response_for/2` → `{"cid":""}` for json / `(cid "")` for sx / raw bytes for cbor) and kernel-absent (`post_activity_response_for/1` → `{"status":"stub"}` / `(status "stub")` / etc.) paths. `next/tests/http_post_format.sh` (13 cases) covers shape + Content-Type for both stub and publish paths. - [x] **8d-dispatch-get** — `actor_doc_response_for/2`, `artifact_response_for/2`, `projection_response_for/2`, `projections_list_response_for/1`. `dispatch` refactored to `/3` to thread Format; route extracts Format once and passes it down. `next/tests/http_get_format.sh` (17 cases) covers per-format bodies + Content-Type + end-to-end GETs with Accept headers. **Deliverables:** Core endpoints (per design §16.1): ``` GET /actors/ # actor doc GET /actors//outbox # OrderedCollection GET /actors//outbox?page=true # OrderedCollectionPage POST /activity # publish (auth: bearer token) GET /artifacts/ # CID-addressed artifact GET /artifacts//raw GET /projections # list of projections GET /projections/ # full state GET /projections/?at= # time-travel GET /projections// # indexed lookup GET /define-registry GET /.well-known/sx-capabilities GET /.well-known/webfinger ``` ```erlang % next/kernel/http_server.erl -module(http_server). -export([start/1, route/1]). start(Port) -> http:listen(Port, fun ?MODULE:route/1). route(Request) -> {Status, Headers, Body}. ``` Content negotiation per `Accept`: - `application/activity+json` (default) - `application/cbor` (dag-cbor) - `application/json` (compact, no @context expansion) - `application/sx` Auth on `POST /activity`: bearer token from env var `NEXT_PUBLISH_TOKEN`. **Tests:** - Each endpoint returns expected shape for known artifact. - Content negotiation: same artifact in 4 representations. - 404 for unknown artifact CID. - 401 for `POST /activity` without token. - Pagination: outbox with > 50 activities returns OrderedCollectionPage. **Acceptance:** `bash next/tests/http.sh` passes 20+ cases. --- ## Step 9 — Smoke tests **Sub-deliverables:** - [x] **9-pre-fold** — In-process end-to-end test of the HTTP → publish → broadcast → projection-fold chain. Proves the full vertical works without a real TCP socket. `next/tests/http_publish_fold.sh` (10 cases). Step 9a/b proper need TCP (Step 8b-start). - [ ] **9a** — Pin smoke test (TCP-driven, curl) — needs Step 8b-start + Define\* SX-source eval. - [ ] **9b** — Reactive smoke test (TCP-driven, curl) — needs DefineSubscription / DefineTrigger eval. **The proof points.** Two end-to-end smoke tests demonstrate, between them, that fed-sx is genuinely a substrate for distributed reactive applications expressed as data — not a system you extend by writing kernel code. - **9a — Pin smoke test (`next/tests/smoke_pin.sh`)** — verb extensibility: defining a new activity type and projection at runtime via `Define*` artifacts. Verifies the meta-level (§5). - **9b — Reactive application smoke test (`next/tests/smoke_app.sh`)** — application extensibility: defining a new subscription type, subscribing, registering a trigger, and observing the full reactive loop fire end-to-end without kernel code changes. Verifies §§18-19. Both must pass for milestone 1 acceptance. ### Step 9a — Pin smoke test **Test script:** `next/tests/smoke_pin.sh` ```bash #!/usr/bin/env bash set -euo pipefail # 0. Start a fresh fed-sx kernel (background) ./next/scripts/start.sh fresh sleep 2 TOKEN=$(cat next/data/keys/publish.token) # 1. Verify actor exists curl -s http://localhost:9999/actors/next | jq -e '.type == "Person"' # 2. Verify outbox has actor's first Create{Person} curl -s http://localhost:9999/actors/next/outbox?page=true \ | jq -e '.orderedItems | length == 1 and .[0].type == "Create"' # 3. Verify Pin is NOT a known activity type curl -s http://localhost:9999/define-registry?kind=activity_types \ | jq -e '.[] | select(.name == "Pin") | length == 0' || exit 1 # 4. Publish DefineActivity{name: "Pin", schema: ..., semantics: ...} PIN_DEF=$(cat <<'JSON' { "type": "Create", "object": { "type": "DefineActivity", "name": "Pin", "schema": "(fn (act) (and (string? (-> act :object :path)) (cid? (-> act :object :cid))))", "semantics": "(fn (state act) (assoc-in state [:pins (-> act :object :path)] (-> act :object :cid)))" } } JSON ) curl -s -X POST http://localhost:9999/activity \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/activity+json" \ -d "$PIN_DEF" | jq -e '.cid' > /dev/null # 5. Verify Pin IS now a known activity type curl -s http://localhost:9999/define-registry?kind=activity_types \ | jq -e '.[] | select(.name == "Pin") | length == 1' # 6. Also publish a DefineProjection{name: "pin-state"} that folds Pin into state PIN_PROJ=$(cat <<'JSON' { "type": "Create", "object": { "type": "DefineProjection", "name": "pin-state", "initial-state": "{}", "fold": "(fn (state act) (if (= (:type act) \"Pin\") (assoc state (-> act :object :path) (-> act :object :cid)) state))" } } JSON ) curl -s -X POST http://localhost:9999/activity \ -H "Authorization: Bearer $TOKEN" \ -d "$PIN_PROJ" | jq -e '.cid' # 7. Now publish a Pin activity PIN=$(cat <<'JSON' { "type": "Pin", "object": { "type": "PinSpec", "path": "/docs/intro", "cid": "bafyreigh2akiscaildc3xqxx4xqxx4xqxx4xqxx4xqxx4xqxx4xqxx4xqxxe" } } JSON ) curl -s -X POST http://localhost:9999/activity \ -H "Authorization: Bearer $TOKEN" \ -d "$PIN" | jq -e '.cid' # 8. Verify Pin appears in outbox curl -s http://localhost:9999/actors/next/outbox?page=true \ | jq -e '.orderedItems | map(select(.type == "Pin")) | length == 1' # 9. Verify pin-state projection has the entry sleep 1 # allow async projection curl -s http://localhost:9999/projections/pin-state \ | jq -e '."/docs/intro" == "bafyreigh2akiscaildc3xqxx4xqxx4xqxx4xqxx4xqxx4xqxx4xqxx4xqxxe"' # 10. Negative test: publish a malformed Pin (missing path) → expect 422 BAD_PIN='{"type": "Pin", "object": {"cid": "bafy..."}}' HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" -X POST http://localhost:9999/activity \ -H "Authorization: Bearer $TOKEN" -d "$BAD_PIN") [[ "$HTTP_STATUS" == "422" ]] || { echo "expected 422, got $HTTP_STATUS"; exit 1; } # 11. Restart kernel; verify state recovers ./next/scripts/stop.sh ./next/scripts/start.sh sleep 2 curl -s http://localhost:9999/projections/pin-state \ | jq -e '."/docs/intro" == "bafyreigh2akiscaildc3xqxx4xqxx4xqxx4xqxx4xqxx4xqxx4xqxxe"' echo "✓ Pin smoke test passed — verb extensibility demonstrated end-to-end" ``` **Acceptance for 9a:** smoke test exits 0. The whole flow happens with **zero fed-sx kernel code changes** between defining the verb and using it. ### Step 9b — Reactive application smoke test **The bigger proof point.** Demonstrates that fed-sx supports distributed reactive applications composed of `DefineSubscription` + `DefineTrigger` + `DefineProjection` — the application model from §§18-19. The test runs on a single instance (federation is v2), so the "subscriber" and "publisher" are the same actor. That's intentional — milestone 1 proves the mechanism; milestone 2 spreads it across instances. **Test script:** `next/tests/smoke_app.sh` ```bash #!/usr/bin/env bash set -euo pipefail # Assumes 9a has already run (fresh kernel optional; can run alongside). TOKEN=$(cat next/data/keys/publish.token) BASE=http://localhost:9999 # 1. Verify "Topic" subscription type and "Subscribe" verb are NOT yet defined. curl -s "$BASE/define-registry?kind=subscription_types" \ | jq -e 'map(select(.name == "Topic")) | length == 0' # 2. Publish DefineSubscription{name: "Topic", ...} TOPIC_DEF=$(cat <<'JSON' { "type": "Create", "object": { "type": "DefineSubscription", "name": "Topic", "schema": "(fn (sub) (string? (-> sub :tag)))", "match": "(fn (sub act) (and (= (:type act) \"Note\") (member? (-> sub :tag) (or (-> act :object :tags) (list)))))", "delivery": "{:default :push :modes (list :push :pull)}" } } JSON ) curl -s -X POST "$BASE/activity" \ -H "Authorization: Bearer $TOKEN" -d "$TOPIC_DEF" | jq -e '.cid' # 3. Verify Topic IS now a known subscription type. curl -s "$BASE/define-registry?kind=subscription_types" \ | jq -e 'map(select(.name == "Topic")) | length == 1' # 4. Subscribe to the "smoketest" topic. SUBSCRIBE=$(cat <<'JSON' { "type": "Subscribe", "object": {"type": "Topic", "tag": "smoketest"} } JSON ) SUB_CID=$(curl -s -X POST "$BASE/activity" \ -H "Authorization: Bearer $TOKEN" -d "$SUBSCRIBE" | jq -r '.cid') # 5. Verify subscriptions projection has the new entry. sleep 1 curl -s "$BASE/projections/subscriptions" \ | jq -e '.["https://next.rose-ash.com/actors/next"] | map(select(.type == "Topic")) | length == 1' # 6. Define a projection that records matched activities (per-application # namespace would happen via DefineApplication in v1.x; for v1 the # projection is global to the actor). TOPIC_PROJ=$(cat <<'JSON' { "type": "Create", "object": { "type": "DefineProjection", "name": "topic-events", "initial-state": "{}", "fold": "(fn (state act) (if (and (= (:type act) \"Note\") (member? \"smoketest\" (or (-> act :object :tags) (list)))) (assoc-in state [(:cid act)] act) state))" } } JSON ) curl -s -X POST "$BASE/activity" \ -H "Authorization: Bearer $TOKEN" -d "$TOPIC_PROJ" | jq -e '.cid' # 7. Define a trigger: when a Topic{smoketest} subscription matches, publish # a TestEcho activity. We need an "Echo" activity type first. ECHO_DEF=$(cat <<'JSON' { "type": "Create", "object": { "type": "DefineActivity", "name": "TestEcho", "schema": "(fn (act) (cid? (-> act :object :echoes)))", "semantics": "(fn (state act) state)" } } JSON ) curl -s -X POST "$BASE/activity" \ -H "Authorization: Bearer $TOKEN" -d "$ECHO_DEF" | jq -e '.cid' TRIGGER=$(cat <= 1" curl -s "$BASE/define-registry?kind=triggers" \ | jq -e 'map(select(.name == "echo-on-smoketest")) | length == 1' echo "✓ Reactive application smoke test passed — Subscribe + Trigger + Projection demonstrated end-to-end" ``` **What this proves (and what it doesn't):** Proves: - `DefineSubscription` + `Subscribe` mechanism works end-to-end. - Subscription's `match-fn` evaluates correctly in pure mode against inbound activities. - `DefineTrigger` fires on subscription matches. - Trigger's `then-sx` can publish derived activities (the `:publish` result). - Cascade-depth metadata propagates correctly. - Subscription state, trigger registration, and projection state all survive kernel restart (snapshot + log replay). - The full reactive application loop works without any kernel code changes between defining the components and exercising them. Does NOT prove (deferred to milestone 2+): - Cross-instance subscriptions (federation). - Trigger `:effect` results calling effectful primitives. - `DefineApplication` bundle install/update/fork. - Per-application namespace isolation. - Cascade prevention against malicious cascading from peer instances. **Acceptance for 9b:** smoke test exits 0. Like 9a, **zero fed-sx kernel code changes** between defining the application components and observing them operate. --- ## Acceptance criteria for milestone 1 All of: 1. **Each step's test suite passes** (`bash next/tests/.sh`). 2. **Both smoke tests pass** (`bash next/tests/smoke_pin.sh` and `bash next/tests/smoke_app.sh`). 3. **Erlang-on-SX baseline preserved** — adding fed-sx kernel modules in `next/kernel/*.erl` doesn't break Phase 1-8 conformance. 4. **Restart durability** — kill the kernel mid-write, restart, projections resume from snapshot, no log corruption. 5. **Manual Mastodon poke** — point a Mastodon account at `https://next.rose-ash.com/actors/next` and verify the actor doc fetches and webfinger discovery works (read-only AP interop, no follow). ## What lands when This is the work-order an agent (or human) follows. Steps 1-3 can be done in parallel after the Erlang Phase 8 BIFs land. Steps 4-7 are sequential. Step 8 can start in parallel with step 7. Step 9 is the integration test. ``` Phase 7+8 (loops/erlang) ───┐ │ ▼ ┌─── Step 1 ──┬─── Step 2 ──┬─── Step 3 │ │ │ └─────────────┼─── Step 4 ──┴────┐ │ │ └─── Step 5 ───────┤ │ Step 6 ─────┤ │ Step 7 ─────┤ │ Step 8 ─────┤ │ Step 9 ─────┘ ``` Estimated effort if done by a focused agent loop, one feature per iteration: ~30-50 commits across all 9 steps. Could plausibly be a `loops/fed-sx` workstream once Phase 7+8 are done. ## What's deferred to milestone 2 - **Federation** (the second-biggest piece). `POST /inbox`, Follow lifecycle, delivery queue, backfill, capability negotiation between peers. Whole of design §13. - **Multi-actor** with per-user OAuth and capability tokens. Design §9.5. - **IPFS storage backend** as a `DefineStorage` entry. Design §15.3. - **Browser client + operator dashboard** (probably in Elm-on-SX or similar). - **Rich verbs**: `Endorse`, `Supersede`, `Test`, `Build`, `Compose`, `Note`, `Announce`. All defined as `DefineActivity` artifacts, federated. - **Cross-host conformance** — Python/JS/Haskell hosts running fed-sx. Design §11.8. - **OpenTimestamps proofs** as a `DefineProof` entry. - **Performance work** — JIT-compiled folds, snapshot acceleration, federation batching. Milestone 2 unlocks "real federation between two fed-sx instances." Milestone 3 is the rose-ash port (blog, market, events, federation, account, orders) as fed-sx applications. --- ## Appendix A: open questions for milestone 1 A few things still under-specified; resolve as work begins. 1. **HTTP server library.** Does the Phase 8 `http:listen/2` BIF wrap an existing OCaml HTTP server (the sx.rose-ash.com one) or something simpler? Implementation choice deferred to Phase 8. 2. **JSON-LD library.** AP wire format requires JSON-LD canonicalization for signature coverage. Either pull a library or write a minimal subset for the shapes we actually use. Probably the latter — our envelope is well-defined. 3. **Bearer token rotation.** v1 uses a single env-var token. Token rotation without restart needs registry-style mgmt; can wait. 4. **Snapshot rate limits.** Default in design is "every 1000 activities or 60 seconds." Tunable per-projection later; v1 uses the default. 5. **Genesis bundle format.** Dag-cbor map per §12.2; concrete schema needs one round of refinement once we author the actual definitions in step 4. --- ## Progress log Newest first. One line per sub-deliverable commit. Erlang conformance gate (`bash lib/erlang/conformance.sh`) must remain 729/729 on every entry. - **2026-05-28** — Step 8d-dispatch-get: format-aware versions of every GET response builder. `actor_doc_response_for/2`, `artifact_response_for/2`, `projection_response_for/2`, `projections_list_response_for/1`. Each produces `{"key":"value"}` (json/activity_json), `(key "value")` (sx), raw payload bytes (cbor stub), or the existing text form. `dispatch` refactored to `/3` with a backward-compat `dispatch/2` wrapper. Route extracts Format via `accept_format_from/1` once at the top and threads it through dispatch. End-to-end GETs with `Accept: application/json` / `application/sx` verified for all three dynamic-prefix routes + the projections-list bare-path route. Step 8d effectively complete — format dispatch + Content-Type live on every non-static response. `next/tests/http_get_format.sh` 17/17. Erlang conformance 729/729. - **2026-05-28** — Step 8d-dispatch-post: `handle_post_activity` extracts the Accept format via `accept_format_from/1` and threads it into `publish_if_kernel/2`. Both success paths emit format-specific bodies: `cid_response_for/2` produces `{"cid":""}\n` (json/activity_json), `(cid "")\n` (sx), raw CID bytes (cbor), or the existing text form; `post_activity_response_for/1` mirrors for the kernel-absent stub. Each response carries the matching Content-Type. End-to-end POSTs with `Accept: application/json` / `application/sx` verified through the full HTTP→nx_kernel→publish→cid_response_for chain. `next/tests/http_post_format.sh` 13/13. Erlang conformance 729/729. - **2026-05-28** — Step 8d-content-type: `content_type_for/1` maps format atoms to MIME-type binaries — text/plain (10b), application/json (16b), application/activity+json (25b), application/sx (14b), application/cbor (16b); unknown formats fall through to text/plain. `ok_response/2(Body, Format)` constructs a 200 response with `{headers, [{<<"content-type">>, MIME}]}`. Lowercase header key matches how the BIF wrapper normalises request headers. `ok_response/1` still produces the empty-headers shape — backward compat preserved. `next/tests/http_content_type.sh` 13/13. Erlang conformance 729/729. - **2026-05-28** — Step 8d-dispatch-cap: `capabilities_body_for/1` returns distinct byte sequences per format — text reuses the existing `capabilities_body/0`; json/activity_json share `{"caps":"fed-sx-m1"}`; sx returns `(caps "fed-sx-m1")`; cbor returns a minimal `A1 64 caps 69 fed-sx-m1` map. Route now intercepts GET `/.well-known/sx-capabilities` to pull the Accept format via `accept_format_from/1` and dispatch through `capabilities_body_for`. Unknown formats fall back to text. POST capabilities still 404 (only GET handled). `next/tests/http_capabilities_format.sh` 13/13 verifies all formats + the intercept + no-Accept default. Content-Type headers not yet set (8d-dispatch-rest covers headers + applying the same shape to actor/artifact/projection/cid responses). Erlang conformance 729/729. - **2026-05-28** — Step 8d-accept: `accept_format/1` + `accept_format_from/1` parse the Accept header into a content-negotiation atom. Priority order via successive `match_prefix` checks: application/activity+json → `activity_json`; application/json → `json`; application/sx → `sx`; application/cbor → `cbor`; everything else (including nil / empty / non-binary) → `text`. Comma-separated lists with activity+json first still resolve to activity_json — leading-prefix match is sufficient for v1 envelopes. Step 8d split into 8d-accept (done) + 8d-dispatch (wire into response bodies). `next/tests/http_accept.sh` 13/13. Erlang conformance 729/729. - **2026-05-28** — Step 5c-populate: `bootstrap:populate_registry/0` walks `read_genesis` output and calls `registry:register/3` (gen_server API) for each entry. Total return is 31 = 3 + 10 + 7 + 3 + 3 + 2 + 3 across the seven kinds, matching the manifest authored in Step 4. `next/tests/bootstrap_populate.sh` 14/14 verifies per-kind counts + lookups against known names (`activity_types/create`, `object_types/define-activity`, `validators/envelope-shape`). Erlang conformance 729/729. - **2026-05-28** — Step 9-pre-fold: in-process integration test proving the full POST → publish → broadcast → projection-fold chain. With `projection:start_link` + `nx_kernel:start_link` + `nx_kernel:with_projections([p_count, p_collect])`, three authorized POST `/activity` calls advance both projections to 3 — and the kernel's log to 3 entries — and the projection's collected activity carries the POST body as `:object`. Unauthorized or sig-failed POSTs leave projection state unchanged. Step 9a/b proper (curl-driven smoke tests) wait on Step 8b-start (TCP) + Define\* SX-source eval bridge, but the structural chain is already verified end-to-end. `next/tests/http_publish_fold.sh` 10/10. Erlang conformance 729/729. Step 9 split into 9-pre-fold (done) + 9a + 9b. - **2026-05-28** — Step 8c-post-publish-http: POST `/activity` handler now bridges into `nx_kernel:publish/1` when the kernel gen_server is registered (`erlang:whereis(nx_kernel) =/= undefined`). On success the response carries the canonical CID via `cid_response/1`; on pipeline failure the response is 422 via `validation_failed_response/0`. When the kernel isn't registered, the handler falls through to the existing 200 stub — preserves backwards compatibility for the auth-only tests in `http_post_activity.sh`. Distinct POSTs produce distinct CIDs (next_published counter in nx_kernel state). Unauthorized POSTs never reach the kernel — log tip stays at 0. `next/tests/http_publish.sh` 10/10. The POST `/activity` → publish → fold loop is now functional end-to-end through the kernel. Erlang conformance 729/729. - **2026-05-28** — Step 8c-post-publish-srv: `nx_kernel.erl` extended with gen_server callbacks + named-process API. `start_link/3(ActorId, KeySpec, ActorState)` spawns the worker and registers under the literal `nx_kernel` atom; `publish/1(Request)` calls into `handle_call({publish, Request}, ...)` which delegates to the pure `publish/2` and reflects the new state back into the server. `query/0` returns the full state proplist; `log_tip/0` is a direct accessor; `with_projections/1` mutates the projections list. Same port quirks as Step 5b/7b documented (raw Pid return, no `?MODULE`, processes don't persist across separate `erlang-eval-ast` calls — tests inline start_link with operations). `next/tests/nx_kernel_server.sh` 11/11. Erlang conformance 729/729. - **2026-05-28** — Step 8c-post-publish-pure: `next/kernel/nx_kernel.erl` — pure-functional kernel orchestrator that wraps `outbox:publish/2` with a long-lived runtime state. `new/3(ActorId, KeySpec, ActorState)` initialises state with an empty log + monotonic `:next_published` counter. `publish/2(Request, State)` builds the publish Context from state, calls outbox:publish, and on success advances `:log` and increments `:next_published`. The counter solves the "same Request published twice" replay collision — each call gets a distinct `:published` timestamp, so the canonical-bytes CID differs and stage_replay doesn't halt. On failure (e.g. bad key), state is returned unchanged. Step 8c-post-publish split into pure (done) + srv (gen_server wrapper) sub-deliverables. `next/tests/nx_kernel_pure.sh` 12/12. Erlang conformance 729/729. - **2026-05-28** — Step 8c-post-auth: POST `/activity` route + bearer-token auth via new `route/2(Req, Cfg)` variant. Cfg's `:publish_token` is the expected bearer; mismatched / missing / malformed (no "Bearer " prefix) / empty-token Authorization all surface as 401 `unauthorized_response/0`. `route/1` is a backwards-compatible wrapper with empty Cfg — any POST `/activity` over `route/1` is 401 by design (no token configured). `Bearer ` prefix stripped via the same `match_prefix` helper used elsewhere. Real publish wiring deferred to `8c-post-publish` (needs the kernel orchestrator that holds logs / actor keys / projection list). `next/tests/http_post_activity.sh` 13/13. Erlang conformance 729/729. - **2026-05-28** — Step 8c-proj: routes GET `/projections` (list stub returning `projections: (empty)\n`) + GET `/projections/{name}` (state stub returning `projection: \n`). Bare-path list clause dispatches before the prefix clause so `/projections` and `/projections/{name}` are distinguishable. All three dynamic-prefix routes (actors / artifacts / projections) compose cleanly — verified by a single combined-route test asserting all return 200 with distinct prefixes. Registry-backed implementation deferred — needs a running registry process at route time. `next/tests/http_projections.sh` 11/11. Erlang conformance 729/729. - **2026-05-28** — Step 8c-art: GET `/artifacts/{cid}` route added on top of `match_prefix`. Single GET dispatch clause now tries `actors_prefix` first, falls through to `artifacts_prefix` — no path collision (different leading bytes). Stub body echoes the CID with `artifact: ` prefix; real artifact-store lookup deferred to later (will key into the registry / genesis bundle). `next/tests/http_artifacts.sh` 9/9 covers happy path, empty-cid 404, POST 404, actor/artifact non-collision, static-route regression. Erlang conformance 729/729. - **2026-05-28** — Step 8c-actors-doc: `http_server` extended with `match_prefix/2` — pure byte-level prefix matcher built on Erlang binary pattern matching (`<>`-style head/tail walk). Empty prefix returns `{ok, FullPath}`; non-match returns `nomatch`; exact match returns `{ok, <<>>}`. Wired into a new GET `/actors/{id}` clause that extracts the id suffix and returns it as the body of `actor_doc_response/1` (stub: `actor: \n`). Empty id falls into 404. `/actors/{id}/outbox` deferred to a later step (needs segment splitting beyond prefix). `next/tests/http_actors.sh` 13/13. Erlang conformance 729/729. - **2026-05-28** — Step 8c-cap: GET `/.well-known/sx-capabilities` route + `capabilities_body/0` + `capabilities_path/0` exposed for tests. Body is a small plain-text descriptor with `kernel: fed-sx-m1`, `version: 0.0.1`, `verbs: Create Update Delete` (hand-spelled as integer-segment binary; string-literal segments unusable in this port). `next/tests/http_capabilities.sh` 8/8 covers method+path matching, body content, the existing GET / regression-free. Step 8c split into cap (done) + actors / art / proj / post — the rest need path-prefix matching helpers since `{id}` and `{cid}` are dynamic. Erlang conformance 729/729. - **2026-05-28** — Step 8b-route: `next/kernel/http_server.erl` — pure `route/1` request→response dispatch. Request shape `[{method, Bin}, {path, Bin}, ...]`; response `[{status, N}, {headers, []}, {body, Bin}]`. GET / returns 200 with hand-spelled "fed-sx kernel m1" body; everything else returns 404 with "not found" body. Method/path binaries spelled byte-by-byte (string-literal segments would truncate). Split former 8b into 8b-route (done) + 8b-start (needs dict↔proplist marshaling bridge in the BIF wrapper before the spawned `http:listen` call gets useful request fields). `next/tests/http_route.sh` 11/11. Erlang conformance 729/729. - **2026-05-28** — Step 8a: `http:listen/2` BIF wrapper added to `lib/erlang/runtime.sx` (the briefing's single allowed scope exception). The BIF takes `(Port, Handler)`, validates Port is an integer and Handler is an Erlang fun (else `badarg`), then builds an SX-callable bridge lambda that marshals request dict↔Erlang term via `er-of-sx`/`er-to-sx` and calls `er-apply-fun` on the handler. Delegates to the native `http-listen` primitive (registered in `bin/sx_server.ml`, native-only). Tests verify registration + arg validation paths (the blocking listen loop itself is not exercised — production callers spawn an Erlang process to host the call). `next/tests/http_listen_bif.sh` 5/5; Erlang conformance preserved at 729/729 despite the runtime.sx edit. Step 8 broken into 8a–8d on the plan. - **2026-05-28** — Step 7c: `outbox:publish` now broadcasts the signed activity to every projection process named in `Context`'s `:projections` entry — fired immediately after `log:append`, via `projection:async_fold`. Missing/nil/empty list is a no-op (preserves the Step 6d-publish contract). Stage halts (replay duplicate, sig failure) suppress the broadcast — projection state stays at zero while the activity is rejected. `next/tests/outbox_broadcast.sh` 14/14 covers single + multi projection fan-out, three-publish accumulation, replay-skip, sig-skip, and the projection receiving the post-sign Signed envelope (not the pre-sign skeleton). Erlang conformance 729/729. - **2026-05-28** — Step 7b: `projection.erl` extended with gen_server callbacks + per-projection named-process API. `start_link/3(Name, InitialState, FoldFn)` spawns and registers under the supplied atom; `async_fold/2(Name, Activity)` casts a fold message; `query/1(Name)` synchronously returns the current state. Same port quirks as registry gen_server (Step 5b): raw Pid return, no `?MODULE` macro, processes don't survive between separate `erlang-eval-ast` calls — tests inline start_link with operations. Two named projections are independent. Snapshot persistence deferred to a later sub-step (needs SX-source eval + on-disk state). `next/tests/projection_server.sh` 11/11. Erlang conformance 729/729. - **2026-05-28** — Step 7a: `next/kernel/projection.erl` — pure-functional projection driver. Record shape `[{name, _}, {state, _}, {fold, fun}]`; `fold_activity/2` advances state by one activity; `replay/2` folds a whole list (mirrors `log:entries/1` semantics); `new/2` defaults to the identity fold and `new/3` accepts a custom Erlang fun. Multiple projections share no state — independent record values. Step 7 split into 7a (done) + 7b (gen_server-per-projection) + 7c (broadcast hook from outbox) + 7d (sandbox eval, needs SX-source bridge). `next/tests/projection_pure.sh` 12/12. Erlang conformance 729/729. - **2026-05-28** — Step 6d-publish: `outbox:publish/2(Request, Context)` orchestrates construct + sign + `pipeline:run_stages` + `log:append`. Stage list is `[stage_envelope, stage_signature(AS), stage_replay(LogState)]` — so a duplicate publish (same Request, same Published) halts at the replay stage and returns `{error, replay, LogState}` with the log unchanged; bad key material halts at `bad_signature`. Happy path returns `{ok, [{cid, Cid}, {activity, Signed}], NewLog}`. Projection-scheduler dispatch deferred to Step 7. `next/tests/outbox_publish.sh` 13/13 covers happy path, replay halt, sig halt, multi-publish progression, CID stability across fresh logs. Erlang conformance 729/729. - **2026-05-28** — Step 6d-cs: `next/kernel/outbox.erl` — envelope construction + signing. `construct/4` takes `(Type, ActorId, Published, Object)`, builds the canonical key-sorted property list, and derives the activity `:id` from `cid:to_string({activity_envelope, Skeleton})`. `sign/2` extracts key_id/algorithm/key-material from a KeySpec proplist, computes the v1 HMAC over canonical bytes, and appends the `:signature` pair. `cid_of/1` is a convenience accessor. Round-trip end-to-end through `envelope:verify_signature/2` verified (correct key passes, wrong key returns bad_signature). Step 6d split into 6d-cs (done) + 6d-publish (orchestration). `next/tests/outbox_construct.sh` 13/13. Erlang conformance 729/729. - **2026-05-28** — Step 6c-replay: `pipeline:stage_replay/2` (direct) + `stage_replay/1` (factory closed over LogState). Linear scan of `log:entries/1` checking for an existing entry with the same `:id`. Returns ok if new, `{error, replay}` on duplicate, `{error, no_id}` when the activity has no id field. Step 6c split into 6c-replay (done) + 6c-schema (deferred — blocked behind SX-source eval bridge for the activity-type :schema body). `next/tests/pipeline_replay.sh` 12/12 covers direct + factory + composition with stage_envelope. Erlang conformance 729/729. - **2026-05-28** — Step 6b-sig: `pipeline:stage_signature/2` direct call + `stage_signature/1` factory returning a context-bound stage fun closed over ActorState. Not wired into the default `inbound_stages`/`outbound_stages` lists because actor state isn't a static-build-time value; callers prepend the factory result to a stage list (`Stages = [stage_envelope, pipeline:stage_signature(AS)]`). `next/tests/pipeline_signature.sh` 11/11 covers direct + factory + composition with stage_envelope (including halt ordering: bad envelope halts before sig; good envelope + bad sig surfaces sig error). Erlang conformance 729/729. - **2026-05-28** — Step 6b-env: `pipeline:stage_envelope/1` wraps `envelope:validate_shape/1`; wired into both `inbound_stages` and `outbound_stages` lists. `validate_inbound`/`validate_outbound` now exercises the full envelope shape contract end-to-end (missing fields, signature sub-shape, non-list input). `next/tests/pipeline_envelope.sh` 12/12; `pipeline_driver.sh` refactored to test the driver against explicit stage lists rather than depending on the now-non-empty defaults. Split 6b in the plan into 6b-env (done) + 6b-sig (needs runtime context for actor-state). Erlang conformance 729/729. - **2026-05-28** — Step 6a: `next/kernel/pipeline.erl` — validation pipeline driver per design §14. `run_stages/2` is a pure fold over `(Activity) -> ok | {error, R}` funs, halting on first failure. Halt verified by inserting a post-error stage that would set a contradictory tag if it ran. `validate_inbound/1` + `validate_outbound/1` wrappers; concrete stage lists are empty (6b wires `stage_envelope`/`stage_signature`). Port quirk: `Pattern = Var` match-alias syntax unsupported — split into separate `Result = X, case Result of ...`. `next/tests/pipeline_driver.sh` 10/10. Step 6 broken into 6a–6e on the plan. Erlang conformance 729/729. - **2026-05-28** — Step 5b: `registry.erl` extended with gen_server callbacks + named-process API. `start_link/0` spawns the worker, registers it under the literal `registry` atom, returns the Pid (port returns raw Pid not `{ok, Pid}` — diverges from OTP). 3-arity `register`, 2-arity `lookup`, 1-arity `list` delegate to the pure /4 and /3 functions inside handle_call. Port note documented: `?MODULE` macro unsupported; tests must inline start_link with operations since spawned processes don't persist across separate `erlang-eval-ast` calls. `next/tests/registry_server.sh` 12/12. Erlang conformance 729/729. - **2026-05-28** — Step 4e: `bootstrap:load_genesis/1` + `strip_sx_suffix/1` in `next/kernel/bootstrap.erl`. Walks `read_genesis` output and threads each entry through `registry:register/4`, using the section atom as the kind and the filename-minus-`.sx` as the entry name. Per-kind counts match the seven bootstrap sections exactly (3/10/7/3/3/2/3 = 31 entries total). `next/tests/bootstrap_load.sh` 15/15. Determinism verified by comparing `cid:to_string` of the loaded state across calls (faster than deep-equality on the nested-binary state). Step 4 is now complete end-to-end except for SX-source parsing of the loaded entries. Erlang conformance 729/729. - **2026-05-28** — Step 5a: `next/kernel/registry.erl` — pure-functional registry. State is `[{Kind, [{Name, Entry}, ...]}, ...]` keyed by the same seven section atoms as Step 4c (activity_types, object_types, projections, validators, codecs, sig_suites, audience). API: `new/0`, `kinds/0`, `register/4`, `lookup/3`, `list/2`. Unknown kinds rejected with `{error, unknown_kind}`; missing names return `not_found`; re-registering the same name overrides without growing the list. `next/tests/registry_pure.sh` 14/14. Step 5 broken into 5a–5d on the plan. Erlang conformance 729/729. - **2026-05-28** — Step 4d: `bootstrap:build_genesis/1` + `verify_genesis/2` + `.cidhash` helpers in `next/kernel/bootstrap.erl`. Bundle CID delegated to host `cid:to_string` over `{genesis_bundle, Sections}` — deterministic, ~59 byte CIDv1 binary. `verify_genesis/2` returns `ok` on match, `{error, {cid_mismatch, Got, Expected}}` on drift. `write_cidhash`/`read_cidhash` persist the CID to a `.cidhash` sibling file (path hand-spelled `<<...,47,46,99,...>>` per the string-literal-in-binary substrate quirk). `next/tests/bootstrap_build.sh` 12/12. Erlang conformance 729/729. - **2026-05-27** — Step 4c: `next/kernel/bootstrap.erl` — Erlang module that enumerates the genesis bundle by walking seven hardcoded section subdirs via `file:list_dir/1`, filters `.sx` files via byte-pattern suffix match (`ends_with_sx/1`), reads each into a binary via `file:read_file/1`. Returns `{ok, [{Section, [{Name, Bytes}, ...]}]}`. Hits the same SX-parser substrate gap as Step 3b — kept the surface byte-only; parsing happens via SX-side helpers in later steps. Port gotchas: `fun name/arity` references unsupported (use anonymous fun wrappers); `<<"...">>` string-literal segments truncate to one byte (paths hand-spelled as integer-segment binaries). `next/tests/bootstrap_read.sh` 15/15. Erlang conformance 729/729. - **2026-05-27** — Step 4b-cod: bootstrap codecs + sig-suites + audience predicates complete. 3 `DefineCodec` files (dag-cbor + raw + dag-json, dag-cbor + dag-json deferring to host-codec primitive when wired), 2 `DefineSigSuite` files (rsa-sha256-2018 PEM-keyed, ed25519-2020 multibase-keyed, both :verify returning false as m2-deferred stand-in), 3 `DefineAudience` files (Public/Followers/Direct member-of predicates per design §16). Manifest now lists 26 bootstrap files across all eight sections; `next/tests/genesis_parse.sh` 50/50. Step 4b complete; remaining Step 4 is bundler code (4c–4e). Erlang conformance 729/729. - **2026-05-27** — Step 4b-vld: bootstrap validators complete — 3 `DefineValidator` SX files (envelope-shape mirroring Step 2a, signature stub delegating to envelope:verify_signature/2 per design §9.6, type-schema looking up the object-type schema from define-registry). Manifest `:validators` populated; `next/tests/genesis_parse.sh` 36/36. Erlang conformance 729/729. - **2026-05-27** — Step 4b-proj: bootstrap projections complete — 7 `DefineProjection` SX files authored (activity-log identity, by-type/by-actor/by-object indexes, actor-state with key history fold, define-registry meta-fold over Create{Define*}, audience-graph stub). Manifest `:projections` populated; `next/tests/genesis_parse.sh` 31/31. Erlang conformance 729/729. - **2026-05-27** — Step 4b-obj: bootstrap object-types complete — 10 `DefineObject` SX files authored (SXArtifact, Note, Tombstone, DefineActivity, DefineObject, DefineProjection, DefineValidator, DefineCodec, DefineSigSuite, Snapshot). Each carries an SX `:schema` predicate. Manifest `:object-types` populated; `next/tests/genesis_parse.sh` 22/22. Erlang conformance 729/729. - **2026-05-27** — Step 4b-act: bootstrap activity-types complete — `update.sx` (Update verb, requires :object CID + :patch) + `delete.sx` (Delete verb, requires :object CID) authored as DefineActivity forms matching the Create shape. Manifest updated; `next/tests/genesis_parse.sh` 10/10. Step 4b broken into act/obj/proj/vld/cod sub-deliverables on the plan. Erlang conformance 729/729. - **2026-05-27** — Step 4a: genesis bundle seeded. `next/genesis/manifest.sx` (GenesisManifest with eight section keys, only `:activity-types` populated for now) + `next/genesis/activity-types/create.sx` (DefineActivity{Create} with :schema/:semantics SX bodies). `next/tests/genesis_parse.sh` 5/5. Step 3b parked behind a substrate-level term-codec gap — Blockers note added under Step 3; in-memory log from 3a unblocks Step 5+ which only need the API surface. Erlang conformance 729/729. - **2026-05-27** — Step 3a: `log:open/2 append/2 tip/1 replay/3 entries/1` over an in-memory state (per-actor seq, replay in append order, round-trip activities). `next/tests/log_memory.sh` 12/12. Pivoted from on-disk in this iteration: this port's `atom_to_list`/`integer_to_list` return SX strings rather than Erlang charlists, `binary_to_list` is unregistered, and `$X` char literals decode to nil — so a term codec needs a workaround. Captured as the Step 3b risk note in the plan. Erlang conformance 729/729. - **2026-05-26** — Step 2c: `envelope:verify_signature/2` — time-aware key lookup over `public_keys` (created ≤ published < superseded_at), MAC recompute via `crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`, compared against `signature.value`. Returns ok or one of `no_signature | no_key_id | no_published | no_keys | no_active_key | bad_signature`. `next/tests/envelope_sig.sh` 11/11 pass. Erlang conformance 729/729. - **2026-05-26** — Step 2b: `envelope:canonical_bytes/1` — strip signature, insertion-sort property list by key, return host-CID-string as deterministic byte form (dag-cbor stand-in). `next/tests/envelope_canonical.sh` 8/8 pass. Erlang conformance 729/729 preserved. - **2026-05-26** — Step 2a: `next/kernel/envelope.erl` `validate_shape/1` + `get_field/2` over property-list envelopes (Erlang `#{}` maps not supported in this port). `next/tests/envelope_shape.sh` 15/15 pass. Erlang conformance 729/729 preserved. - **2026-05-26** — Step 1b: `next/kernel/nx_cid.erl` (from_sx/to_string/from_string/equals) — thin Erlang wrapper around the `cid:to_string/1` BIF. `next/tests/cid.sh` 13/13 pass. Module named `nx_cid` to avoid shadowing the `cid` BIF (user-module dispatch takes precedence over BIFs by module name). Erlang conformance 729/729 preserved. - **2026-05-26** — Step 1a: `next/` skeleton created (kernel/, genesis/, tests/, data/), README, `.gitignore data/`. Erlang conformance 729/729 preserved.