From 28168b16aa3d7232609616202bd1ebeedacdfa9c Mon Sep 17 00:00:00 2001 From: giles Date: Wed, 27 May 2026 07:18:04 +0000 Subject: [PATCH] =?UTF-8?q?fed-sx-m1:=20Step=204a=20=E2=80=94=20genesis=20?= =?UTF-8?q?manifest=20+=20Create=20activity-type=20seed=20+=205=20parse=20?= =?UTF-8?q?tests;=20Step=203b=20parked=20(substrate=20term-codec=20gap)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- next/genesis/activity-types/create.sx | 15 ++++++ next/genesis/manifest.sx | 27 ++++++++++ next/tests/genesis_parse.sh | 71 +++++++++++++++++++++++++++ plans/fed-sx-milestone-1.md | 12 ++++- 4 files changed, 124 insertions(+), 1 deletion(-) create mode 100644 next/genesis/activity-types/create.sx create mode 100644 next/genesis/manifest.sx create mode 100755 next/tests/genesis_parse.sh diff --git a/next/genesis/activity-types/create.sx b/next/genesis/activity-types/create.sx new file mode 100644 index 00000000..feb8d5c0 --- /dev/null +++ b/next/genesis/activity-types/create.sx @@ -0,0 +1,15 @@ +;; next/genesis/activity-types/create.sx +;; +;; Bootstrap definition of the Create verb per design §3 and §12.2. +;; Read as data by the bundler (bootstrap.erl) — never evaluated as +;; code. The :schema and :semantics bodies are SX source; the +;; validation pipeline (Step 6) and projection scheduler (Step 7) +;; evaluate them at the appropriate times. + +(DefineActivity + :name "Create" + :doc "Publish a new object. Required for actor onboarding and for\n every Define* meta-activity. The activity's :object holds\n the canonical content of the published object." + :schema (fn + (act) + (and (not (nil? (-> act :object))) (string? (-> act :object :type)))) + :semantics (fn (state act) state)) diff --git a/next/genesis/manifest.sx b/next/genesis/manifest.sx new file mode 100644 index 00000000..bd62daa3 --- /dev/null +++ b/next/genesis/manifest.sx @@ -0,0 +1,27 @@ +;; next/genesis/manifest.sx +;; +;; Genesis bundle root per design §12.2. Lists every definition file +;; that gets packed into the bundle. The bundler (bootstrap.erl) +;; walks this manifest, reads each referenced file, parses its +;; top-level form, and inserts it into the bundle dict at the +;; appropriate section path. +;; +;; The bundle CID is the content-address of the resulting dag-cbor +;; (or v1 stand-in) blob over the assembled dict. That CID is +;; baked into the kernel at build time and re-verified on startup +;; per design §12.3. +;; +;; Section values are bare parenthesised paths (data lists, not +;; function calls) — the manifest is consumed by `parse`, not +;; `eval`. Empty sections are written as `()`. + +(GenesisManifest + :version "0.0.1" + :kernel-version "1.0.0-m1" + :activity-types ("activity-types/create.sx") + :object-types () + :projections () + :validators () + :codecs () + :sig-suites () + :audience ()) diff --git a/next/tests/genesis_parse.sh b/next/tests/genesis_parse.sh new file mode 100755 index 00000000..d10f1ac6 --- /dev/null +++ b/next/tests/genesis_parse.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# next/tests/genesis_parse.sh — Step 4a acceptance test. +# +# Confirms the seed genesis SX files parse cleanly and have the +# expected top-level head form. The bundler (Step 4c+) consumes +# these forms directly as data. 5 cases. + +set -uo pipefail +cd "$(git rev-parse --show-toplevel)" + +SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}" +if [ ! -x "$SX_SERVER" ]; then + SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe" +fi +if [ ! -x "$SX_SERVER" ]; then + echo "ERROR: sx_server.exe not found." >&2 + exit 1 +fi + +VERBOSE="${1:-}" +PASS=0; FAIL=0; ERRORS="" +TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT + +cat > "$TMPFILE" <<'EPOCHS' +(epoch 10) +(eval "(first (parse (file-read \"next/genesis/manifest.sx\")))") +(epoch 11) +(eval "(first (parse (file-read \"next/genesis/activity-types/create.sx\")))") +(epoch 12) +(eval "(first (get (apply dict (rest (parse (file-read \"next/genesis/manifest.sx\")))) :activity-types))") +(epoch 13) +(eval "(get (apply dict (rest (parse (file-read \"next/genesis/activity-types/create.sx\")))) :name)") +(epoch 14) +(eval "(get (apply dict (rest (parse (file-read \"next/genesis/manifest.sx\")))) :version)") +EPOCHS + +OUTPUT=$(timeout 30 "$SX_SERVER" < "$TMPFILE" 2>/dev/null) + +check() { + local epoch="$1" desc="$2" expected="$3" + local actual + actual=$(echo "$OUTPUT" | awk -v e="$epoch" ' + $0 ~ "^\\(ok-len " e " " { getline; print; exit } + $0 ~ "^\\(ok " e " " { print; exit } + $0 ~ "^\\(error " e " " { print; exit } + ') + [ -z "$actual" ] && actual="" + if echo "$actual" | grep -qF -- "$expected"; then + PASS=$((PASS+1)) + [ "$VERBOSE" = "-v" ] && echo " ok $desc" + else + FAIL=$((FAIL+1)) + ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual +" + fi +} + +check 10 "manifest.sx head form" "GenesisManifest" +check 11 "create.sx head form" "DefineActivity" +check 12 "manifest lists create.sx" "activity-types/create.sx" +check 13 "create.sx name is Create" "Create" +check 14 "manifest version present" "0.0.1" + +TOTAL=$((PASS+FAIL)) +if [ $FAIL -eq 0 ]; then + echo "ok $PASS/$TOTAL next/tests/genesis_parse.sh passed" +else + echo "FAIL $PASS/$TOTAL passed, $FAIL failed:" + echo "$ERRORS" +fi +[ $FAIL -eq 0 ] diff --git a/plans/fed-sx-milestone-1.md b/plans/fed-sx-milestone-1.md index be7a3d9f..1a00e992 100644 --- a/plans/fed-sx-milestone-1.md +++ b/plans/fed-sx-milestone-1.md @@ -197,9 +197,11 @@ verify_signature(Activity, ActorState) -> **Sub-deliverables:** - [x] **3a** — `log:open/2` + `log:append/2` + `log:tip/1` + `log:replay/3` + `log:entries/1` over an in-memory log state (per-actor seq; replay in append order; round-trip the stored activity). `next/tests/log_memory.sh` (12 cases). -- [ ] **3b** — Term codec + on-disk persistence: serializer/parser writing each activity as a JSONL-style line; restart-resumes-tip from the segment file. Blocker risk: `atom_to_list`/`integer_to_list` return SX strings (not Erlang charlists) and there is no `binary_to_list`, so a term-to-binary path needs a workaround. +- [ ] **3b** — *Parked behind substrate gap (see Blockers below).* Term codec + on-disk persistence: serializer/parser writing each activity as a JSONL-style line; restart-resumes-tip from the segment file. - [ ] **3c** — Segment rotation at size threshold + gen_server-mediated concurrent appends. +**Blockers (Step 3b):** The Erlang port returns SX strings (an opaque OCaml-string type) from `atom_to_list/1` and `integer_to_list/1`, rejects them from `++`/list pattern matching, and does not register `binary_to_list`/`list_to_binary`. `$X` character literals decode to `nil` in `parse-number`. Net effect: there is no in-Erlang path from an arbitrary term to a byte sequence (or back) that doesn't go through a temp-file round-trip through the filesystem. Workaround paths: (a) add a `term_to_binary`/`binary_to_term` BIF in a separate substrate loop, (b) accept a filesystem-mediated SX-string→binary helper and live with the O(N) IO cost, (c) restrict the on-disk format to a binary-only encoding with a per-instance atom-id table for atoms (introduces an extra durability dependency). Decision to defer; revisit once a downstream Step (5–8) forces the issue or a substrate BIF arrives. In-memory log from 3a is sufficient to unblock Step 5+ which consume the API surface. + **Deliverables:** ```erlang @@ -241,6 +243,13 @@ replay(LogState, InitAcc, Fun) -> ... ## Step 4 — Genesis bundle +**Sub-deliverables:** +- [x] **4a** — Seed genesis SX file authoring: `next/genesis/manifest.sx` + `next/genesis/activity-types/create.sx`. Manifest uses bare parenthesised paths (data lists, not `(list ...)` calls — consumed by `parse`, not `eval`). `next/tests/genesis_parse.sh` (5 cases). +- [ ] **4b** — Author remaining activity-types (Update / Delete) + object-types (DefineActivity, DefineObject, DefineProjection, …) + projections, validators, codecs, sig-suites, audience predicates +- [ ] **4c** — `bootstrap:read_genesis/1` in Erlang: walk the manifest, file-read each referenced .sx, return parsed forms +- [ ] **4d** — `bootstrap:build_genesis/1` + `bootstrap:verify_genesis/1`: compute bundle CID over the read forms via the host `cid:to_string` substrate; verify against a stored `bundle.cidhash` +- [ ] **4e** — `bootstrap:load_genesis/1`: register parsed definitions into the in-memory registry (depends on Step 5) + **Deliverables:** Genesis bundle SX sources (per design §12.2). Each is a small SX file authored @@ -942,6 +951,7 @@ A few things still under-specified; resolve as work begins. Newest first. One line per sub-deliverable commit. Erlang conformance gate (`bash lib/erlang/conformance.sh`) must remain 729/729 on every entry. +- **2026-05-27** — Step 4a: genesis bundle seeded. `next/genesis/manifest.sx` (GenesisManifest with eight section keys, only `:activity-types` populated for now) + `next/genesis/activity-types/create.sx` (DefineActivity{Create} with :schema/:semantics SX bodies). `next/tests/genesis_parse.sh` 5/5. Step 3b parked behind a substrate-level term-codec gap — Blockers note added under Step 3; in-memory log from 3a unblocks Step 5+ which only need the API surface. Erlang conformance 729/729. - **2026-05-27** — Step 3a: `log:open/2 append/2 tip/1 replay/3 entries/1` over an in-memory state (per-actor seq, replay in append order, round-trip activities). `next/tests/log_memory.sh` 12/12. Pivoted from on-disk in this iteration: this port's `atom_to_list`/`integer_to_list` return SX strings rather than Erlang charlists, `binary_to_list` is unregistered, and `$X` char literals decode to nil — so a term codec needs a workaround. Captured as the Step 3b risk note in the plan. Erlang conformance 729/729. - **2026-05-26** — Step 2c: `envelope:verify_signature/2` — time-aware key lookup over `public_keys` (created ≤ published < superseded_at), MAC recompute via `crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`, compared against `signature.value`. Returns ok or one of `no_signature | no_key_id | no_published | no_keys | no_active_key | bad_signature`. `next/tests/envelope_sig.sh` 11/11 pass. Erlang conformance 729/729. - **2026-05-26** — Step 2b: `envelope:canonical_bytes/1` — strip signature, insertion-sort property list by key, return host-CID-string as deterministic byte form (dag-cbor stand-in). `next/tests/envelope_canonical.sh` 8/8 pass. Erlang conformance 729/729 preserved.