Compare commits

...

10 Commits

Author SHA1 Message Date
4c0295cdff fed-sx-m1: Step 4b-obj — 10 bootstrap object-types + manifest update + 12 new parse tests (22 total)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 29s
2026-05-27 19:48:26 +00:00
b308ddb9b0 fed-sx-m1: Step 4b-act — Update + Delete activity-types + manifest update + 5 new parse tests (10 total)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 22s
2026-05-27 07:44:20 +00:00
28168b16aa fed-sx-m1: Step 4a — genesis manifest + Create activity-type seed + 5 parse tests; Step 3b parked (substrate term-codec gap)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 16s
2026-05-27 07:18:04 +00:00
ab159dface fed-sx-m1: Step 3a — in-memory log:open/append/tip/replay + 12 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 20s
2026-05-27 07:06:40 +00:00
53b4a4c1fd fed-sx-m1: Step 2c — envelope:verify_signature/2 (time-aware key lookup + HMAC stand-in) + 11 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 31s
2026-05-26 21:00:39 +00:00
65dfdd0ba4 fed-sx-m1: Step 2b — envelope:canonical_bytes/1 + 8 determinism tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 23s
2026-05-26 20:41:27 +00:00
e11e8b941f fed-sx-m1: Step 2a — envelope:validate_shape/1 + get_field/2 + 15 shape tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 26s
2026-05-26 20:29:25 +00:00
9cbf14fe8c fed-sx-m1: Step 1b — nx_cid kernel module + 13 canonical CID tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 6m22s
2026-05-26 19:55:13 +00:00
11ed4ddf27 fed-sx-m1: Step 1a — next/ skeleton + README + gitignore
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 14s
2026-05-26 19:44:56 +00:00
abde5fbac1 Merge loops/erlang into architecture: Phase 8 host-primitive BIFs (crypto/cid/file:list_dir)
Wires the 3 previously-BLOCKED Phase 8 FFI BIFs against loops/fed-prims
primitives (merged at 380bc69f):

- crypto:hash/2 → crypto-sha256/sha512/sha3-256 (atom dispatch, raw-binary
  return via er-hex->bytes), +6 ffi tests
- cid:from_bytes/1 → CIDv1 raw-codec (0x55) + sha2-256 multihash assembled
  in SX; cid:to_string/1 → cid-from-sx of canonical er-format-value string,
  +7 ffi tests
- file:list_dir/1 → file-list-dir, {ok,[Binary]} / {error,Reason} reusing
  er-classify-file-error, +4 ffi tests

ffi suite 14 → 28 (3 BLOCKED negative-asserts flipped to functional tests).
httpc:request and sqlite:* remain BLOCKED — need HTTP-client and SQLite
host primitives which loops/fed-prims didn't deliver.

Full conformance 729/729 (eval 385, vm 78, ffi 28, all process suites).
2026-05-26 19:30:35 +00:00
29 changed files with 1268 additions and 0 deletions

1
next/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
data/

34
next/README.md Normal file
View File

@@ -0,0 +1,34 @@
# next — fed-sx Milestone 1 kernel
Single-instance, single-actor fed-sx server built as Erlang-on-SX modules.
See `plans/fed-sx-design.md` for the architecture and
`plans/fed-sx-milestone-1.md` for the build plan.
## Layout
```
next/
├── kernel/ Erlang-on-SX kernel modules (.erl, hot-loaded via code:load_binary/3)
├── genesis/ SX source files for the genesis bootstrap bundle (DefineActivity, ...)
├── tests/ Bash test scripts driving sx_server.exe via the epoch protocol
└── data/ Runtime state — gitignored
├── log/ per-actor JSONL outboxes
├── objects/ CID-addressed artifacts on disk
├── snapshots/ projection snapshots
├── indexes/ derived projection index files
└── keys/ actor signing keys + bearer tokens
```
## Substrate
The kernel is Erlang-on-SX. Each `.erl` source file is hot-loaded at boot via
`code:load_binary(Mod, Filename, SourceString)` (Erlang Phase 7 BIF). The
underlying SX runtime provides the host primitives the kernel calls into:
`crypto:*`, `cid:*`, `file:*`, `code:*`, and (Step 8) `http:listen/2`.
Tests drive the kernel via the epoch protocol:
```bash
printf '(epoch 1)\n(load "lib/erlang/runtime.sx")\n(epoch 2)\n<test-expr>\n' \
| hosts/ocaml/_build/default/bin/sx_server.exe
```

0
next/genesis/.gitkeep Normal file
View File

View File

@@ -0,0 +1,15 @@
;; next/genesis/activity-types/create.sx
;;
;; Bootstrap definition of the Create verb per design §3 and §12.2.
;; Read as data by the bundler (bootstrap.erl) — never evaluated as
;; code. The :schema and :semantics bodies are SX source; the
;; validation pipeline (Step 6) and projection scheduler (Step 7)
;; evaluate them at the appropriate times.
(DefineActivity
:name "Create"
:doc "Publish a new object. Required for actor onboarding and for\n every Define* meta-activity. The activity's :object holds\n the canonical content of the published object."
:schema (fn
(act)
(and (not (nil? (-> act :object))) (string? (-> act :object :type))))
:semantics (fn (state act) state))

View File

@@ -0,0 +1,13 @@
;; next/genesis/activity-types/delete.sx
;;
;; Bootstrap definition of the Delete verb per design §3 and §12.2.
;; Read as data by the bundler — never evaluated as code here. The
;; :schema and :semantics bodies are SX source; the validator
;; pipeline (Step 6) and projection scheduler (Step 7) evaluate them
;; at the appropriate times.
(DefineActivity
:name "Delete"
:doc "Tombstone an existing object. :object is the CID of the\n target. Projections fold Delete by removing the object from\n their working indexes; the underlying log line is never\n erased — durability of the historical record is independent\n of projection state."
:schema (fn (act) (string? (-> act :object)))
:semantics (fn (state act) state))

View File

@@ -0,0 +1,15 @@
;; next/genesis/activity-types/update.sx
;;
;; Bootstrap definition of the Update verb per design §3 and §12.2.
;; Read as data by the bundler — never evaluated as code here. The
;; :schema and :semantics bodies are SX source; the validator
;; pipeline (Step 6) and projection scheduler (Step 7) evaluate them
;; at the appropriate times.
(DefineActivity
:name "Update"
:doc "Patch or replace an existing object. :object is the CID of\n the target; :patch is the field-level edit. Behaviour is\n delegated to per-object-type semantics — e.g. an Update of a\n DefineActivity supersedes the prior registry entry; an\n Update of a Person actor rotates keys via :patch :add-publicKey\n + :patch :supersede."
:schema (fn
(act)
(and (string? (-> act :object)) (not (nil? (-> act :patch)))))
:semantics (fn (state act) state))

38
next/genesis/manifest.sx Normal file
View File

@@ -0,0 +1,38 @@
;; next/genesis/manifest.sx
;;
;; Genesis bundle root per design §12.2. Lists every definition file
;; that gets packed into the bundle. The bundler (bootstrap.erl)
;; walks this manifest, reads each referenced file, parses its
;; top-level form, and inserts it into the bundle dict at the
;; appropriate section path.
;;
;; The bundle CID is the content-address of the resulting dag-cbor
;; (or v1 stand-in) blob over the assembled dict. That CID is
;; baked into the kernel at build time and re-verified on startup
;; per design §12.3.
;;
;; Section values are bare parenthesised paths (data lists, not
;; function calls) — the manifest is consumed by `parse`, not
;; `eval`. Empty sections are written as `()`.
(GenesisManifest
:version "0.0.1"
:kernel-version "1.0.0-m1"
:activity-types ("activity-types/create.sx"
"activity-types/update.sx"
"activity-types/delete.sx")
:object-types ("object-types/sx-artifact.sx"
"object-types/note.sx"
"object-types/tombstone.sx"
"object-types/define-activity.sx"
"object-types/define-object.sx"
"object-types/define-projection.sx"
"object-types/define-validator.sx"
"object-types/define-codec.sx"
"object-types/define-sig-suite.sx"
"object-types/snapshot.sx")
:projections ()
:validators ()
:codecs ()
:sig-suites ()
:audience ())

View File

@@ -0,0 +1,12 @@
;; next/genesis/object-types/define-activity.sx
;;
;; Meta-object that registers a new activity verb. Published as
;; Create{DefineActivity{...}}; the define-registry projection
;; folds it into the activity-types registry. Per design §5.
(DefineObject
:name "DefineActivity"
:doc "Activity-type registration. :name is the verb (e.g.\n \"Pin\"); :schema is an SX predicate over activity\n envelopes; :semantics is an optional state-fold body."
:schema (fn
(obj)
(and (string? (-> obj :name)) (not (nil? (-> obj :schema))))))

View File

@@ -0,0 +1,15 @@
;; next/genesis/object-types/define-codec.sx
;;
;; Meta-object that registers a content codec — an encode/decode
;; pair. The bootstrap bundle ships dag-cbor, raw, and dag-json
;; codecs; new codecs can be added via Create{DefineCodec{...}}.
(DefineObject
:name "DefineCodec"
:doc "Codec registration. :name identifies the codec ('dag-cbor',\n 'raw', 'dag-json', ...); :encode and :decode are the\n SX bodies the kernel calls when serialising / parsing\n artifacts under this codec."
:schema (fn
(obj)
(and
(string? (-> obj :name))
(not (nil? (-> obj :encode)))
(not (nil? (-> obj :decode))))))

View File

@@ -0,0 +1,12 @@
;; next/genesis/object-types/define-object.sx
;;
;; Meta-object that registers a new object-type. Bootstrap-level —
;; runtime registration of new object types (e.g. DefineSubscription
;; in the Step 9b smoke test) flows through this.
(DefineObject
:name "DefineObject"
:doc "Object-type registration. :name is the type tag (e.g.\n \"PinSpec\"); :schema is an SX predicate over object\n forms of that type."
:schema (fn
(obj)
(and (string? (-> obj :name)) (not (nil? (-> obj :schema))))))

View File

@@ -0,0 +1,16 @@
;; next/genesis/object-types/define-projection.sx
;;
;; Meta-object that registers a new projection. The projection
;; scheduler (Step 7) spawns one gen_server per registered
;; projection and feeds activities through its :fold body in
;; sandbox mode.
(DefineObject
:name "DefineProjection"
:doc "Projection registration. :name is the projection key;\n :initial-state is the empty state value; :fold is the\n pure (state activity) -> state function evaluated in\n sandbox mode per activity."
:schema (fn
(obj)
(and
(string? (-> obj :name))
(not (nil? (-> obj :initial-state)))
(not (nil? (-> obj :fold))))))

View File

@@ -0,0 +1,12 @@
;; next/genesis/object-types/define-sig-suite.sx
;;
;; Meta-object that registers a signature suite. Bootstrap ships
;; rsa-sha256-2018 and ed25519-2020; the suite name maps an
;; algorithm to a :verify body and a :key-format predicate.
(DefineObject
:name "DefineSigSuite"
:doc "Signature suite registration. :name identifies the suite\n ('rsa-sha256-2018', 'ed25519-2020', ...); :verify is the\n SX (canonical-bytes signature key) -> bool body; the\n envelope-signature validator dispatches by suite name."
:schema (fn
(obj)
(and (string? (-> obj :name)) (not (nil? (-> obj :verify))))))

View File

@@ -0,0 +1,12 @@
;; next/genesis/object-types/define-validator.sx
;;
;; Meta-object that registers a validator predicate. The validation
;; pipeline (Step 6) consults registered validators by name when
;; running its stages.
(DefineObject
:name "DefineValidator"
:doc "Validator registration. :name is the validator key (e.g.\n \"envelope-shape\"); :predicate is the SX (activity) ->\n ok|{error, R} body."
:schema (fn
(obj)
(and (string? (-> obj :name)) (not (nil? (-> obj :predicate))))))

View File

@@ -0,0 +1,10 @@
;; next/genesis/object-types/note.sx
;;
;; Short message intended for an audience, ActivityPub-Note-compatible.
;; Used by the Step 9b reactive smoke test (Note tagged "smoketest"
;; matches the Topic subscription).
(DefineObject
:name "Note"
:doc "Short authored message. :content is the body text;\n :tags is a list of subscription-routable tags."
:schema (fn (obj) (string? (-> obj :content))))

View File

@@ -0,0 +1,13 @@
;; next/genesis/object-types/snapshot.sx
;;
;; Projection state checkpoint. The projection scheduler emits
;; Snapshot{projection-name, state-cid, log-seq} periodically;
;; cold starts read the most recent Snapshot and replay only
;; activities after :log-seq. Per design §10.5.
(DefineObject
:name "Snapshot"
:doc "Projection-state checkpoint. :projection-name identifies\n the projection; :state-cid is the content-address of\n the snapshotted state value; :log-seq is the activity\n sequence number the snapshot was taken at."
:schema (fn
(obj)
(and (string? (-> obj :projection-name)) (string? (-> obj :state-cid)))))

View File

@@ -0,0 +1,10 @@
;; next/genesis/object-types/sx-artifact.sx
;;
;; Content-addressed SX source — a library, component, or
;; executable form published via Create{SXArtifact{...}}.
;; Consumers reference an artifact by its CID. Per design §3.4.
(DefineObject
:name "SXArtifact"
:doc "Published SX source. :source carries the form text;\n :language is optional ('sx' by default); :imports lists\n CIDs the artifact depends on."
:schema (fn (obj) (string? (-> obj :source))))

View File

@@ -0,0 +1,9 @@
;; next/genesis/object-types/tombstone.sx
;;
;; Replacement for an object that has been Delete'd. Lets projection
;; folds keep a marker without retaining the deleted content.
(DefineObject
:name "Tombstone"
:doc "Marker for a deleted object. :former-cid carries the CID\n of the object that was removed. Projections fold Tombstone\n by replacing the cached entry (not by omitting it)."
:schema (fn (obj) (string? (-> obj :former-cid))))

0
next/kernel/.gitkeep Normal file
View File

177
next/kernel/envelope.erl Normal file
View File

@@ -0,0 +1,177 @@
-module(envelope).
-export([validate_shape/1, get_field/2, canonical_bytes/1, verify_signature/2]).
%% Activity envelope per design §3.1.
%%
%% Erlang maps (#{...}) are not supported by this port, so envelopes
%% are represented as property lists of {atom_key, value} pairs. This
%% port's binary syntax also can't carry string literals; values that
%% would naturally be binaries in real Erlang are kept as atoms or
%% integer-segment binaries in the test corpus.
%%
%% Required fields: id, type, actor, published, signature.
%% The signature value is itself a property list with key_id,
%% algorithm, value.
%%
%% validate_shape/1 returns ok | {error, Reason}. Reasons:
%% not_a_proplist
%% {missing_field, FieldName}
%% {bad_signature, BadSigReason}
%%
%% get_field/2 returns {ok, Value} | not_found.
validate_shape(Env) when is_list(Env) ->
case check_required([id, type, actor, published, signature], Env) of
ok -> validate_signature_shape(Env);
Err -> Err
end;
validate_shape(_) ->
{error, not_a_proplist}.
get_field(_, []) -> not_found;
get_field(K, [{K, V} | _]) -> {ok, V};
get_field(K, [_ | Rest]) -> get_field(K, Rest).
check_required([], _) -> ok;
check_required([F | Rest], Env) ->
case get_field(F, Env) of
{ok, _} -> check_required(Rest, Env);
not_found -> {error, {missing_field, F}}
end.
validate_signature_shape(Env) ->
{ok, Sig} = get_field(signature, Env),
case is_list(Sig) of
true ->
case check_required([key_id, algorithm, value], Sig) of
ok -> ok;
{error, {missing_field, F}} ->
{error, {bad_signature, {missing_field, F}}}
end;
false ->
{error, {bad_signature, not_a_proplist}}
end.
%% canonical_bytes/1 — the byte string the signature covers.
%%
%% Real fed-sx will use dag-cbor over a JSON-LD-canonicalised form
%% (design §3.2). For milestone 1 we stand in for that with the host
%% BIF `cid:to_string/1`, which produces a CIDv1 over the deterministic
%% textual form of the term. Two prior steps make this work:
%% 1. The signature pair is stripped (sig covers everything except
%% itself).
%% 2. The top-level property list is sorted by key so field order in
%% the source envelope is not load-bearing.
%%
%% The result is an Erlang binary suitable as the sig-cover input.
canonical_bytes(Env) when is_list(Env) ->
Stripped = strip_signature(Env),
Sorted = sort_pairs(Stripped),
cid:to_string(Sorted).
strip_signature([]) -> [];
strip_signature([{signature, _} | Rest]) -> strip_signature(Rest);
strip_signature([P | Rest]) -> [P | strip_signature(Rest)].
sort_pairs([]) -> [];
sort_pairs([H | T]) -> insert_pair(H, sort_pairs(T)).
insert_pair(P, []) -> [P];
insert_pair({K1, V1}, [{K2, V2} | Rest]) ->
case K1 < K2 of
true -> [{K1, V1}, {K2, V2} | Rest];
false -> [{K2, V2} | insert_pair({K1, V1}, Rest)]
end.
%% verify_signature/2 — time-aware sig verification per design §9.6.
%%
%% Activity carries a `signature` proplist with `key_id`, `algorithm`,
%% `value`. ActorState carries `public_keys` — a list of key proplists
%% with `id`, `created`, optionally `superseded_at`, and `value` (the
%% key material).
%%
%% A key is active at time T iff `created =< T` AND
%% (no `superseded_at` OR T < `superseded_at`). Verification picks the
%% first matching active key whose `id == signature.key_id` at the
%% activity's `published` timestamp, then recomputes the MAC
%% `crypto:hash(sha256, <<KeyMaterial/binary, CanonicalBytes/binary>>)`
%% and compares it to `signature.value`.
%%
%% Returns ok | {error, Reason}. Reasons:
%% no_signature | no_key_id | no_published | no_keys |
%% no_active_key | bad_signature
%%
%% Real RSA-SHA256 / Ed25519 verification is deferred to milestone 2:
%% Phase 8 only ships `crypto:hash/2`, so we stand in with an HMAC-shaped
%% MAC that exercises the same key-lookup and canonical-bytes pipeline.
verify_signature(Activity, ActorState) ->
case get_field(signature, Activity) of
not_found -> {error, no_signature};
{ok, Sig} ->
case get_field(key_id, Sig) of
not_found -> {error, no_key_id};
{ok, KeyId} ->
case get_field(published, Activity) of
not_found -> {error, no_published};
{ok, Published} ->
verify_with_keys(Activity, Sig, KeyId,
Published, ActorState)
end
end
end.
verify_with_keys(Activity, Sig, KeyId, Published, ActorState) ->
case get_field(public_keys, ActorState) of
not_found -> {error, no_keys};
{ok, Keys} ->
case find_active_key(KeyId, Published, Keys) of
not_found -> {error, no_active_key};
{ok, Key} -> verify_mac(Activity, Sig, Key)
end
end.
find_active_key(_, _, []) -> not_found;
find_active_key(KeyId, Now, [Key | Rest]) ->
case is_matching_active_key(Key, KeyId, Now) of
true -> {ok, Key};
false -> find_active_key(KeyId, Now, Rest)
end.
is_matching_active_key(Key, WantId, Now) ->
case get_field(id, Key) of
{ok, WantId} -> is_active_at(Key, Now);
_ -> false
end.
is_active_at(Key, Now) ->
case get_field(created, Key) of
not_found -> false;
{ok, Created} ->
case Now >= Created of
false -> false;
true ->
case get_field(superseded_at, Key) of
not_found -> true;
{ok, SupAt} -> Now < SupAt
end
end
end.
verify_mac(Activity, Sig, Key) ->
case get_field(value, Sig) of
not_found -> {error, bad_signature};
{ok, SigValue} ->
case get_field(value, Key) of
not_found -> {error, bad_signature};
{ok, KeyMat} ->
Bytes = canonical_bytes(Activity),
Computed = crypto:hash(sha256,
<<KeyMat/binary, Bytes/binary>>),
case SigValue =:= Computed of
true -> ok;
false -> {error, bad_signature}
end
end
end.

63
next/kernel/log.erl Normal file
View File

@@ -0,0 +1,63 @@
-module(log).
-export([open/2, append/2, tip/1, replay/3, entries/1]).
%% Per-actor activity log — the canonical record of everything an
%% actor has emitted, in chronological order. Per design §15.2 this
%% lives on disk as a JSONL segment file; v1 starts with an in-memory
%% backend so the API and seq-number machinery can be locked down
%% before the on-disk format is added (Step 3b).
%%
%% State shape (a property list):
%% [{actor, ActorId}, {base, BasePath}, {seq, NextSeq}, {entries, [Act|...]}]
%%
%% `entries` stores activities in append order — i.e. oldest first.
%% `seq` is the next sequence number that will be assigned by append.
%% `base` is kept on the state for forward-compatibility with 3b
%% (where it becomes the segment-file directory).
%%
%% open/2 takes ActorId + BasePath and returns {ok, LogState} starting
%% with seq=0 and no entries.
%%
%% append/2 returns {ok, NewLogState, AssignedSeq}.
%%
%% tip/1 returns the next seq the log would assign (== count of entries).
%%
%% replay/3 folds Fun(Activity, AssignedSeq, Acc) over every entry in
%% append order. Three-arity rather than two-arity because the plan's
%% example test is "sequence numbers gap-free across replay" — having
%% the seq number visible in the fold makes that test direct.
%%
%% entries/1 is a debug accessor returning [Activity, ...] in append
%% order. Not part of the public API contract.
open(ActorId, BasePath) ->
{ok, [{actor, ActorId}, {base, BasePath}, {seq, 0}, {entries, []}]}.
append(LogState, Activity) ->
Seq = field(seq, LogState),
Entries = field(entries, LogState),
NewState = replace_field(seq, Seq + 1,
replace_field(entries, Entries ++ [Activity], LogState)),
{ok, NewState, Seq}.
tip(LogState) ->
field(seq, LogState).
replay(LogState, InitAcc, Fun) ->
Entries = field(entries, LogState),
replay_loop(Entries, 0, InitAcc, Fun).
replay_loop([], _, Acc, _) -> Acc;
replay_loop([Act | Rest], Seq, Acc, Fun) ->
replay_loop(Rest, Seq + 1, Fun(Act, Seq, Acc), Fun).
entries(LogState) ->
field(entries, LogState).
field(K, [{K, V} | _]) -> V;
field(K, [_ | Rest]) -> field(K, Rest);
field(_, []) -> erlang:error(badkey).
replace_field(K, V, []) -> [{K, V}];
replace_field(K, V, [{K, _} | Rest]) -> [{K, V} | Rest];
replace_field(K, V, [P | Rest]) -> [P | replace_field(K, V, Rest)].

24
next/kernel/nx_cid.erl Normal file
View File

@@ -0,0 +1,24 @@
-module(nx_cid).
-export([from_sx/1, to_string/1, from_string/1, equals/2]).
%% The kernel-side CID wrapper. The host BIF `cid:to_string/1` already
%% produces a canonical CIDv1 (raw codec, sha2-256 multihash) over the
%% deterministic textual form of any term (er-format-value); we expose
%% it under the kernel namespace and add the equality + round-trip
%% helpers the rest of the kernel needs.
%%
%% Naming note: the BIF module is `cid`, so we use `nx_cid` to avoid
%% shadowing. Plans/fed-sx-milestone-1.md §Step 1 spells the file as
%% `cid.erl`; the briefing flags Erlang snippets as illustrative.
from_sx(V) ->
cid:to_string(V).
to_string(Cid) ->
Cid.
from_string(S) ->
S.
equals(A, B) ->
A =:= B.

0
next/tests/.gitkeep Normal file
View File

117
next/tests/cid.sh Executable file
View File

@@ -0,0 +1,117 @@
#!/usr/bin/env bash
# next/tests/cid.sh — Step 1b acceptance test.
#
# Loads next/kernel/nx_cid.erl into the Erlang-on-SX runtime and checks
# the canonical CID contract: determinism, uniqueness, equality, and
# to_string/from_string round-trip. 12 cases.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
PASS=0; FAIL=0; ERRORS=""
TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT
cat > "$TMPFILE" <<'EPOCHS'
(epoch 1)
(load "lib/erlang/tokenizer.sx")
(load "lib/erlang/parser.sx")
(load "lib/erlang/parser-core.sx")
(load "lib/erlang/parser-expr.sx")
(load "lib/erlang/parser-module.sx")
(load "lib/erlang/transpile.sx")
(load "lib/erlang/runtime.sx")
(load "lib/erlang/vm/dispatcher.sx")
(epoch 2)
(eval "(get (erlang-load-module (file-read \"next/kernel/nx_cid.erl\")) :name)")
;; from_sx returns a binary
(epoch 10)
(eval "(get (erlang-eval-ast \"is_binary(nx_cid:from_sx(foo))\") :name)")
;; from_sx is deterministic on atoms / ints / compound terms
(epoch 11)
(eval "(get (erlang-eval-ast \"nx_cid:from_sx(foo) =:= nx_cid:from_sx(foo)\") :name)")
(epoch 12)
(eval "(get (erlang-eval-ast \"nx_cid:from_sx(42) =:= nx_cid:from_sx(42)\") :name)")
(epoch 13)
(eval "(get (erlang-eval-ast \"nx_cid:from_sx({a, [1, 2, 3]}) =:= nx_cid:from_sx({a, [1, 2, 3]})\") :name)")
;; from_sx is collision-resistant on distinct terms
(epoch 20)
(eval "(get (erlang-eval-ast \"nx_cid:from_sx(foo) =/= nx_cid:from_sx(bar)\") :name)")
(epoch 21)
(eval "(get (erlang-eval-ast \"nx_cid:from_sx(1) =/= nx_cid:from_sx(2)\") :name)")
(epoch 22)
(eval "(get (erlang-eval-ast \"nx_cid:from_sx([1, 2]) =/= nx_cid:from_sx([1, 2, 3])\") :name)")
;; equals/2 is alias for =:=
(epoch 30)
(eval "(get (erlang-eval-ast \"nx_cid:equals(nx_cid:from_sx(foo), nx_cid:from_sx(foo))\") :name)")
(epoch 31)
(eval "(get (erlang-eval-ast \"nx_cid:equals(nx_cid:from_sx(foo), nx_cid:from_sx(bar))\") :name)")
;; to_string + from_string round-trip
(epoch 40)
(eval "(get (erlang-eval-ast \"nx_cid:equals(nx_cid:from_string(nx_cid:to_string(nx_cid:from_sx(foo))), nx_cid:from_sx(foo))\") :name)")
(epoch 41)
(eval "(get (erlang-eval-ast \"is_binary(nx_cid:to_string(nx_cid:from_sx({tuple, 1, 2})))\") :name)")
;; CIDv1 raw codec sha256 base32 form is around 59 chars; sanity-check length
(epoch 50)
(eval "(get (erlang-eval-ast \"byte_size(nx_cid:from_sx(hello)) > 50\") :name)")
EPOCHS
OUTPUT=$(timeout 120 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
check() {
local epoch="$1" desc="$2" expected="$3"
local actual
actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
$0 ~ "^\\(ok-len " e " " { getline; print; exit }
$0 ~ "^\\(ok " e " " { print; exit }
$0 ~ "^\\(error " e " " { print; exit }
')
[ -z "$actual" ] && actual="<no output for epoch $epoch>"
if echo "$actual" | grep -qF -- "$expected"; then
PASS=$((PASS+1))
[ "$VERBOSE" = "-v" ] && echo " ok $desc"
else
FAIL=$((FAIL+1))
ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
"
fi
}
check 2 "module load name" "nx_cid"
check 10 "from_sx returns binary" "true"
check 11 "from_sx atom deterministic" "true"
check 12 "from_sx int deterministic" "true"
check 13 "from_sx compound deterministic" "true"
check 20 "from_sx atoms distinct" "true"
check 21 "from_sx ints distinct" "true"
check 22 "from_sx lists distinct" "true"
check 30 "equals same CIDs" "true"
check 31 "equals different CIDs" "false"
check 40 "to_string/from_string round-trip" "true"
check 41 "to_string returns binary" "true"
check 50 "CIDv1 base32 length sanity" "true"
TOTAL=$((PASS+FAIL))
if [ $FAIL -eq 0 ]; then
echo "ok $PASS/$TOTAL next/tests/cid.sh passed"
else
echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
echo "$ERRORS"
fi
[ $FAIL -eq 0 ]

105
next/tests/envelope_canonical.sh Executable file
View File

@@ -0,0 +1,105 @@
#!/usr/bin/env bash
# next/tests/envelope_canonical.sh — Step 2b acceptance test.
#
# Loads next/kernel/envelope.erl and checks canonical_bytes/1 contract:
# returns a binary, deterministic across runs, invariant under
# field-order permutation, invariant under signature changes, and
# different for different covered content. 7 cases.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
PASS=0; FAIL=0; ERRORS=""
TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT
cat > "$TMPFILE" <<'EPOCHS'
(epoch 1)
(load "lib/erlang/tokenizer.sx")
(load "lib/erlang/parser.sx")
(load "lib/erlang/parser-core.sx")
(load "lib/erlang/parser-expr.sx")
(load "lib/erlang/parser-module.sx")
(load "lib/erlang/transpile.sx")
(load "lib/erlang/runtime.sx")
(load "lib/erlang/vm/dispatcher.sx")
(epoch 2)
(eval "(get (erlang-load-module (file-read \"next/kernel/envelope.erl\")) :name)")
;; canonical_bytes returns a binary
(epoch 10)
(eval "(get (erlang-eval-ast \"is_binary(envelope:canonical_bytes([{id,1},{type,create},{actor,alice},{published,1000},{signature,whatever}]))\") :name)")
;; Determinism: same envelope twice -> same bytes
(epoch 11)
(eval "(get (erlang-eval-ast \"envelope:canonical_bytes([{id,1},{type,create},{actor,alice}]) =:= envelope:canonical_bytes([{id,1},{type,create},{actor,alice}])\") :name)")
;; Signature stripping: different signatures -> same canonical bytes
(epoch 12)
(eval "(get (erlang-eval-ast \"envelope:canonical_bytes([{id,1},{type,create},{actor,alice},{signature,sig_one}]) =:= envelope:canonical_bytes([{id,1},{type,create},{actor,alice},{signature,sig_two}])\") :name)")
;; No signature vs some signature -> same canonical bytes
(epoch 13)
(eval "(get (erlang-eval-ast \"envelope:canonical_bytes([{id,1},{type,create},{actor,alice}]) =:= envelope:canonical_bytes([{id,1},{type,create},{actor,alice},{signature,whatever}])\") :name)")
;; Key-order invariance: reordering top-level fields -> same bytes
(epoch 14)
(eval "(get (erlang-eval-ast \"envelope:canonical_bytes([{id,1},{type,create},{actor,alice}]) =:= envelope:canonical_bytes([{actor,alice},{type,create},{id,1}])\") :name)")
;; Changing a covered field changes the bytes
(epoch 15)
(eval "(get (erlang-eval-ast \"envelope:canonical_bytes([{id,1},{type,create},{actor,alice}]) =/= envelope:canonical_bytes([{id,2},{type,create},{actor,alice}])\") :name)")
;; Distinct envelopes -> distinct bytes
(epoch 16)
(eval "(get (erlang-eval-ast \"envelope:canonical_bytes([{id,1},{type,create},{actor,alice}]) =/= envelope:canonical_bytes([{id,1},{type,update},{actor,bob}])\") :name)")
EPOCHS
OUTPUT=$(timeout 120 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
check() {
local epoch="$1" desc="$2" expected="$3"
local actual
actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
$0 ~ "^\\(ok-len " e " " { getline; print; exit }
$0 ~ "^\\(ok " e " " { print; exit }
$0 ~ "^\\(error " e " " { print; exit }
')
[ -z "$actual" ] && actual="<no output for epoch $epoch>"
if echo "$actual" | grep -qF -- "$expected"; then
PASS=$((PASS+1))
[ "$VERBOSE" = "-v" ] && echo " ok $desc"
else
FAIL=$((FAIL+1))
ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
"
fi
}
check 2 "module load name" "envelope"
check 10 "canonical_bytes returns binary" "true"
check 11 "deterministic" "true"
check 12 "signature stripped (changes)" "true"
check 13 "signature stripped (absent)" "true"
check 14 "key-order invariant" "true"
check 15 "covered field change visible" "true"
check 16 "distinct envelopes distinct" "true"
TOTAL=$((PASS+FAIL))
if [ $FAIL -eq 0 ]; then
echo "ok $PASS/$TOTAL next/tests/envelope_canonical.sh passed"
else
echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
echo "$ERRORS"
fi
[ $FAIL -eq 0 ]

126
next/tests/envelope_shape.sh Executable file
View File

@@ -0,0 +1,126 @@
#!/usr/bin/env bash
# next/tests/envelope_shape.sh — Step 2a acceptance test.
#
# Loads next/kernel/envelope.erl into the Erlang-on-SX runtime and
# checks validate_shape/1 / get_field/2 against the design §3.1 shape
# contract. 13 cases.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
PASS=0; FAIL=0; ERRORS=""
TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT
cat > "$TMPFILE" <<'EPOCHS'
(epoch 1)
(load "lib/erlang/tokenizer.sx")
(load "lib/erlang/parser.sx")
(load "lib/erlang/parser-core.sx")
(load "lib/erlang/parser-expr.sx")
(load "lib/erlang/parser-module.sx")
(load "lib/erlang/transpile.sx")
(load "lib/erlang/runtime.sx")
(load "lib/erlang/vm/dispatcher.sx")
(epoch 2)
(eval "(get (erlang-load-module (file-read \"next/kernel/envelope.erl\")) :name)")
;; Reusable valid envelope as Erlang text. The signature itself is a
;; property list with key_id, algorithm, value.
;; E0 = [{id,1},{type,create},{actor,alice},{published,1000},
;; {signature,[{key_id,k1},{algorithm,ed25519},{value,v}]}]
;; Complete valid envelope
(epoch 10)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{published,1000},{signature,[{key_id,k1},{algorithm,ed25519},{value,v}]}]) =:= ok\") :name)")
;; Missing each top-level required field
(epoch 11)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{type,create},{actor,alice},{published,1000},{signature,[{key_id,k1},{algorithm,ed25519},{value,v}]}]) =:= {error,{missing_field,id}}\") :name)")
(epoch 12)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{actor,alice},{published,1000},{signature,[{key_id,k1},{algorithm,ed25519},{value,v}]}]) =:= {error,{missing_field,type}}\") :name)")
(epoch 13)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{published,1000},{signature,[{key_id,k1},{algorithm,ed25519},{value,v}]}]) =:= {error,{missing_field,actor}}\") :name)")
(epoch 14)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{signature,[{key_id,k1},{algorithm,ed25519},{value,v}]}]) =:= {error,{missing_field,published}}\") :name)")
(epoch 15)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{published,1000}]) =:= {error,{missing_field,signature}}\") :name)")
;; Non-list inputs
(epoch 16)
(eval "(get (erlang-eval-ast \"envelope:validate_shape(42) =:= {error,not_a_proplist}\") :name)")
(epoch 17)
(eval "(get (erlang-eval-ast \"envelope:validate_shape(some_atom) =:= {error,not_a_proplist}\") :name)")
;; Signature sub-shape
(epoch 20)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{published,1000},{signature,[{algorithm,ed25519},{value,v}]}]) =:= {error,{bad_signature,{missing_field,key_id}}}\") :name)")
(epoch 21)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{published,1000},{signature,[{key_id,k1},{value,v}]}]) =:= {error,{bad_signature,{missing_field,algorithm}}}\") :name)")
(epoch 22)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{published,1000},{signature,[{key_id,k1},{algorithm,ed25519}]}]) =:= {error,{bad_signature,{missing_field,value}}}\") :name)")
(epoch 23)
(eval "(get (erlang-eval-ast \"envelope:validate_shape([{id,1},{type,create},{actor,alice},{published,1000},{signature,not_a_proplist}]) =:= {error,{bad_signature,not_a_proplist}}\") :name)")
;; get_field
(epoch 30)
(eval "(get (erlang-eval-ast \"envelope:get_field(actor,[{id,1},{actor,alice}]) =:= {ok,alice}\") :name)")
(epoch 31)
(eval "(get (erlang-eval-ast \"envelope:get_field(missing,[{id,1},{actor,alice}]) =:= not_found\") :name)")
EPOCHS
OUTPUT=$(timeout 120 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
check() {
local epoch="$1" desc="$2" expected="$3"
local actual
actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
$0 ~ "^\\(ok-len " e " " { getline; print; exit }
$0 ~ "^\\(ok " e " " { print; exit }
$0 ~ "^\\(error " e " " { print; exit }
')
[ -z "$actual" ] && actual="<no output for epoch $epoch>"
if echo "$actual" | grep -qF -- "$expected"; then
PASS=$((PASS+1))
[ "$VERBOSE" = "-v" ] && echo " ok $desc"
else
FAIL=$((FAIL+1))
ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
"
fi
}
check 2 "module load name" "envelope"
check 10 "complete envelope -> ok" "true"
check 11 "missing id" "true"
check 12 "missing type" "true"
check 13 "missing actor" "true"
check 14 "missing published" "true"
check 15 "missing signature" "true"
check 16 "non-list (integer)" "true"
check 17 "non-list (atom)" "true"
check 20 "signature missing key_id" "true"
check 21 "signature missing algorithm" "true"
check 22 "signature missing value" "true"
check 23 "signature not a proplist" "true"
check 30 "get_field hit" "true"
check 31 "get_field miss" "true"
TOTAL=$((PASS+FAIL))
if [ $FAIL -eq 0 ]; then
echo "ok $PASS/$TOTAL next/tests/envelope_shape.sh passed"
else
echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
echo "$ERRORS"
fi
[ $FAIL -eq 0 ]

129
next/tests/envelope_sig.sh Executable file
View File

@@ -0,0 +1,129 @@
#!/usr/bin/env bash
# next/tests/envelope_sig.sh — Step 2c acceptance test.
#
# Exercises envelope:verify_signature/2 against the full sig pipeline:
# canonical_bytes + crypto:hash MAC + time-aware key validity per design
# §9.6. 10 cases.
#
# The signature stand-in is HMAC-shaped:
# sig.value = crypto:hash(sha256, <<KeyMaterial/binary, CanonicalBytes/binary>>)
# Real Ed25519/RSA verification is deferred to milestone 2 once the
# corresponding crypto BIFs are wired.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
PASS=0; FAIL=0; ERRORS=""
TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT
# Shared Erlang prelude builds a valid-signed envelope template and an
# actor state with one active key. Each test reuses these and asserts
# against an Erlang =:= comparison so the result is a bare boolean.
PRELUDE='KM = <<1,2,3,4>>, U = [{actor,alice},{id,1},{published,100},{type,create}], CB = envelope:canonical_bytes(U), Sig = crypto:hash(sha256, <<KM/binary, CB/binary>>), Env = [{actor,alice},{id,1},{published,100},{type,create},{signature,[{algorithm,ed25519},{key_id,k1},{value,Sig}]}], AS = [{public_keys, [[{id,k1},{created,50},{value,KM}]]}],'
cat > "$TMPFILE" <<EPOCHS
(epoch 1)
(load "lib/erlang/tokenizer.sx")
(load "lib/erlang/parser.sx")
(load "lib/erlang/parser-core.sx")
(load "lib/erlang/parser-expr.sx")
(load "lib/erlang/parser-module.sx")
(load "lib/erlang/transpile.sx")
(load "lib/erlang/runtime.sx")
(load "lib/erlang/vm/dispatcher.sx")
(epoch 2)
(eval "(get (erlang-load-module (file-read \"next/kernel/envelope.erl\")) :name)")
;; valid sig + active key -> ok
(epoch 10)
(eval "(get (erlang-eval-ast \"${PRELUDE} envelope:verify_signature(Env, AS) =:= ok\") :name)")
;; tampered envelope (id mutated post-sign) -> bad_signature
(epoch 11)
(eval "(get (erlang-eval-ast \"${PRELUDE} Tampered = [{actor,alice},{id,999},{published,100},{type,create},{signature,[{algorithm,ed25519},{key_id,k1},{value,Sig}]}], envelope:verify_signature(Tampered, AS) =:= {error,bad_signature}\") :name)")
;; wrong sig value (random bytes) -> bad_signature
(epoch 12)
(eval "(get (erlang-eval-ast \"${PRELUDE} BadEnv = [{actor,alice},{id,1},{published,100},{type,create},{signature,[{algorithm,ed25519},{key_id,k1},{value,<<0,0,0,0>>}]}], envelope:verify_signature(BadEnv, AS) =:= {error,bad_signature}\") :name)")
;; unknown key_id -> no_active_key
(epoch 13)
(eval "(get (erlang-eval-ast \"${PRELUDE} OtherAS = [{public_keys, [[{id,k_other},{created,50},{value,KM}]]}], envelope:verify_signature(Env, OtherAS) =:= {error,no_active_key}\") :name)")
;; key superseded BEFORE published -> no_active_key
(epoch 14)
(eval "(get (erlang-eval-ast \"${PRELUDE} SupAS = [{public_keys, [[{id,k1},{created,50},{superseded_at,80},{value,KM}]]}], envelope:verify_signature(Env, SupAS) =:= {error,no_active_key}\") :name)")
;; key superseded AFTER published -> ok (historical valid)
(epoch 15)
(eval "(get (erlang-eval-ast \"${PRELUDE} SupAS2 = [{public_keys, [[{id,k1},{created,50},{superseded_at,200},{value,KM}]]}], envelope:verify_signature(Env, SupAS2) =:= ok\") :name)")
;; key not yet created at published -> no_active_key
(epoch 16)
(eval "(get (erlang-eval-ast \"${PRELUDE} FutAS = [{public_keys, [[{id,k1},{created,150},{value,KM}]]}], envelope:verify_signature(Env, FutAS) =:= {error,no_active_key}\") :name)")
;; missing signature field -> no_signature
(epoch 17)
(eval "(get (erlang-eval-ast \"${PRELUDE} envelope:verify_signature(U, AS) =:= {error,no_signature}\") :name)")
;; actor state with no public_keys field -> no_keys
(epoch 18)
(eval "(get (erlang-eval-ast \"${PRELUDE} envelope:verify_signature(Env, []) =:= {error,no_keys}\") :name)")
;; second key in list matches when first doesn't (lookup walks list)
(epoch 19)
(eval "(get (erlang-eval-ast \"${PRELUDE} TwoKeys = [{public_keys, [[{id,k_other},{created,50},{value,<<9,9,9>>}], [{id,k1},{created,50},{value,KM}]]}], envelope:verify_signature(Env, TwoKeys) =:= ok\") :name)")
EPOCHS
OUTPUT=$(timeout 120 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
check() {
local epoch="$1" desc="$2" expected="$3"
local actual
actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
$0 ~ "^\\(ok-len " e " " { getline; print; exit }
$0 ~ "^\\(ok " e " " { print; exit }
$0 ~ "^\\(error " e " " { print; exit }
')
[ -z "$actual" ] && actual="<no output for epoch $epoch>"
if echo "$actual" | grep -qF -- "$expected"; then
PASS=$((PASS+1))
[ "$VERBOSE" = "-v" ] && echo " ok $desc"
else
FAIL=$((FAIL+1))
ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
"
fi
}
check 2 "module load name" "envelope"
check 10 "valid sig active key" "true"
check 11 "tampered envelope" "true"
check 12 "wrong sig value" "true"
check 13 "unknown key_id" "true"
check 14 "key superseded before published" "true"
check 15 "key superseded after published" "true"
check 16 "key not yet created" "true"
check 17 "missing signature field" "true"
check 18 "actor state no keys" "true"
check 19 "match second key in list" "true"
TOTAL=$((PASS+FAIL))
if [ $FAIL -eq 0 ]; then
echo "ok $PASS/$TOTAL next/tests/envelope_sig.sh passed"
else
echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
echo "$ERRORS"
fi
[ $FAIL -eq 0 ]

122
next/tests/genesis_parse.sh Executable file
View File

@@ -0,0 +1,122 @@
#!/usr/bin/env bash
# next/tests/genesis_parse.sh — Step 4a acceptance test.
#
# Confirms the seed genesis SX files parse cleanly and have the
# expected top-level head form. The bundler (Step 4c+) consumes
# these forms directly as data. 22 cases.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
PASS=0; FAIL=0; ERRORS=""
TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT
cat > "$TMPFILE" <<'EPOCHS'
(epoch 10)
(eval "(first (parse (file-read \"next/genesis/manifest.sx\")))")
(epoch 11)
(eval "(first (parse (file-read \"next/genesis/activity-types/create.sx\")))")
(epoch 12)
(eval "(first (get (apply dict (rest (parse (file-read \"next/genesis/manifest.sx\")))) :activity-types))")
(epoch 13)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/activity-types/create.sx\")))) :name)")
(epoch 14)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/manifest.sx\")))) :version)")
(epoch 15)
(eval "(first (parse (file-read \"next/genesis/activity-types/update.sx\")))")
(epoch 16)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/activity-types/update.sx\")))) :name)")
(epoch 17)
(eval "(first (parse (file-read \"next/genesis/activity-types/delete.sx\")))")
(epoch 18)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/activity-types/delete.sx\")))) :name)")
(epoch 19)
(eval "(len (get (apply dict (rest (parse (file-read \"next/genesis/manifest.sx\")))) :activity-types))")
(epoch 30)
(eval "(first (parse (file-read \"next/genesis/object-types/sx-artifact.sx\")))")
(epoch 31)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/sx-artifact.sx\")))) :name)")
(epoch 32)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/note.sx\")))) :name)")
(epoch 33)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/tombstone.sx\")))) :name)")
(epoch 34)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/define-activity.sx\")))) :name)")
(epoch 35)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/define-object.sx\")))) :name)")
(epoch 36)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/define-projection.sx\")))) :name)")
(epoch 37)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/define-validator.sx\")))) :name)")
(epoch 38)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/define-codec.sx\")))) :name)")
(epoch 39)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/define-sig-suite.sx\")))) :name)")
(epoch 40)
(eval "(get (apply dict (rest (parse (file-read \"next/genesis/object-types/snapshot.sx\")))) :name)")
(epoch 41)
(eval "(len (get (apply dict (rest (parse (file-read \"next/genesis/manifest.sx\")))) :object-types))")
EPOCHS
OUTPUT=$(timeout 30 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
check() {
local epoch="$1" desc="$2" expected="$3"
local actual
actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
$0 ~ "^\\(ok-len " e " " { getline; print; exit }
$0 ~ "^\\(ok " e " " { print; exit }
$0 ~ "^\\(error " e " " { print; exit }
')
[ -z "$actual" ] && actual="<no output for epoch $epoch>"
if echo "$actual" | grep -qF -- "$expected"; then
PASS=$((PASS+1))
[ "$VERBOSE" = "-v" ] && echo " ok $desc"
else
FAIL=$((FAIL+1))
ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
"
fi
}
check 10 "manifest.sx head form" "GenesisManifest"
check 11 "create.sx head form" "DefineActivity"
check 12 "manifest lists create.sx" "activity-types/create.sx"
check 13 "create.sx name is Create" "Create"
check 14 "manifest version present" "0.0.1"
check 15 "update.sx head form" "DefineActivity"
check 16 "update.sx name is Update" "Update"
check 17 "delete.sx head form" "DefineActivity"
check 18 "delete.sx name is Delete" "Delete"
check 19 "manifest has 3 activity-types" "3"
check 30 "sx-artifact.sx head form" "DefineObject"
check 31 "sx-artifact.sx name" "SXArtifact"
check 32 "note.sx name" "Note"
check 33 "tombstone.sx name" "Tombstone"
check 34 "define-activity.sx name" "DefineActivity"
check 35 "define-object.sx name" "DefineObject"
check 36 "define-projection.sx name" "DefineProjection"
check 37 "define-validator.sx name" "DefineValidator"
check 38 "define-codec.sx name" "DefineCodec"
check 39 "define-sig-suite.sx name" "DefineSigSuite"
check 40 "snapshot.sx name" "Snapshot"
check 41 "manifest has 10 object-types" "10"
TOTAL=$((PASS+FAIL))
if [ $FAIL -eq 0 ]; then
echo "ok $PASS/$TOTAL next/tests/genesis_parse.sh passed"
else
echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
echo "$ERRORS"
fi
[ $FAIL -eq 0 ]

123
next/tests/log_memory.sh Executable file
View File

@@ -0,0 +1,123 @@
#!/usr/bin/env bash
# next/tests/log_memory.sh — Step 3a acceptance test.
#
# Exercises the in-memory log API: open/2, append/2, tip/1, replay/3,
# entries/1. On-disk persistence is the job of Step 3b. 11 cases.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
PASS=0; FAIL=0; ERRORS=""
TMPFILE=$(mktemp); trap "rm -f $TMPFILE" EXIT
cat > "$TMPFILE" <<'EPOCHS'
(epoch 1)
(load "lib/erlang/tokenizer.sx")
(load "lib/erlang/parser.sx")
(load "lib/erlang/parser-core.sx")
(load "lib/erlang/parser-expr.sx")
(load "lib/erlang/parser-module.sx")
(load "lib/erlang/transpile.sx")
(load "lib/erlang/runtime.sx")
(load "lib/erlang/vm/dispatcher.sx")
(epoch 2)
(eval "(get (erlang-load-module (file-read \"next/kernel/log.erl\")) :name)")
;; Fresh log: tip is 0
(epoch 10)
(eval "(get (erlang-eval-ast \"{ok, L} = log:open(alice, base), log:tip(L) =:= 0\") :name)")
;; Fresh log: entries empty
(epoch 11)
(eval "(get (erlang-eval-ast \"{ok, L} = log:open(alice, base), log:entries(L) =:= []\") :name)")
;; First append returns seq 0; tip advances to 1
(epoch 12)
(eval "(get (erlang-eval-ast \"{ok, L0} = log:open(alice, base), {ok, L1, S} = log:append(L0, act_a), {S, log:tip(L1)} =:= {0, 1}\") :name)")
;; Two appends: seq 0,1; tip = 2
(epoch 13)
(eval "(get (erlang-eval-ast \"{ok, L0} = log:open(alice, base), {ok, L1, S0} = log:append(L0, a), {ok, L2, S1} = log:append(L1, b), {S0, S1, log:tip(L2)} =:= {0, 1, 2}\") :name)")
;; Five appends: seq sequence gap-free
(epoch 14)
(eval "(get (erlang-eval-ast \"{ok, L0} = log:open(alice, base), {ok, L1, S0} = log:append(L0, a), {ok, L2, S1} = log:append(L1, b), {ok, L3, S2} = log:append(L2, c), {ok, L4, S3} = log:append(L3, d), {ok, L5, S4} = log:append(L4, e), {S0,S1,S2,S3,S4,log:tip(L5)} =:= {0,1,2,3,4,5}\") :name)")
;; entries/1 returns activities in append order
(epoch 15)
(eval "(get (erlang-eval-ast \"{ok, L0} = log:open(alice, base), {ok, L1, _} = log:append(L0, a), {ok, L2, _} = log:append(L1, b), {ok, L3, _} = log:append(L2, c), log:entries(L3) =:= [a, b, c]\") :name)")
;; Round-trip: appended activity is recoverable byte-for-byte
(epoch 16)
(eval "(get (erlang-eval-ast \"Act = [{id,1},{type,create},{actor,alice}], {ok, L0} = log:open(alice, base), {ok, L1, _} = log:append(L0, Act), log:entries(L1) =:= [Act]\") :name)")
;; Per-actor isolation: two logs are independent
(epoch 17)
(eval "(get (erlang-eval-ast \"{ok, LA0} = log:open(alice, base), {ok, LB0} = log:open(bob, base), {ok, LA1, _} = log:append(LA0, a), {ok, LB1, _} = log:append(LB0, b1), {ok, LB2, _} = log:append(LB1, b2), {log:tip(LA1), log:tip(LB2)} =:= {1, 2}\") :name)")
;; replay/3 visits all activities in append order with monotonic seqs
(epoch 18)
(eval "(get (erlang-eval-ast \"{ok, L0} = log:open(alice, base), {ok, L1, _} = log:append(L0, a), {ok, L2, _} = log:append(L1, b), {ok, L3, _} = log:append(L2, c), log:replay(L3, [], fun (A, S, Acc) -> [{S, A} | Acc] end) =:= [{2,c},{1,b},{0,a}]\") :name)")
;; replay over empty log: InitAcc returned unchanged
(epoch 19)
(eval "(get (erlang-eval-ast \"{ok, L} = log:open(alice, base), log:replay(L, init_acc, fun (_, _, A) -> A end) =:= init_acc\") :name)")
;; replay can compute a derived state (sum of integer activities)
(epoch 20)
(eval "(get (erlang-eval-ast \"{ok, L0} = log:open(alice, base), {ok, L1, _} = log:append(L0, 10), {ok, L2, _} = log:append(L1, 20), {ok, L3, _} = log:append(L2, 30), log:replay(L3, 0, fun (V, _, Acc) -> V + Acc end) =:= 60\") :name)")
EPOCHS
OUTPUT=$(timeout 120 "$SX_SERVER" < "$TMPFILE" 2>/dev/null)
check() {
local epoch="$1" desc="$2" expected="$3"
local actual
actual=$(echo "$OUTPUT" | awk -v e="$epoch" '
$0 ~ "^\\(ok-len " e " " { getline; print; exit }
$0 ~ "^\\(ok " e " " { print; exit }
$0 ~ "^\\(error " e " " { print; exit }
')
[ -z "$actual" ] && actual="<no output for epoch $epoch>"
if echo "$actual" | grep -qF -- "$expected"; then
PASS=$((PASS+1))
[ "$VERBOSE" = "-v" ] && echo " ok $desc"
else
FAIL=$((FAIL+1))
ERRORS+=" FAIL [$desc] (epoch $epoch) expected: $expected | actual: $actual
"
fi
}
check 2 "module load name" "log"
check 10 "fresh log tip is 0" "true"
check 11 "fresh log entries empty" "true"
check 12 "append returns seq 0, tip 1" "true"
check 13 "two appends seq 0,1; tip 2" "true"
check 14 "five appends gap-free" "true"
check 15 "entries in append order" "true"
check 16 "round-trip activity" "true"
check 17 "per-actor isolation" "true"
check 18 "replay visits all in order" "true"
check 19 "replay over empty log" "true"
check 20 "replay computes derived state" "true"
TOTAL=$((PASS+FAIL))
if [ $FAIL -eq 0 ]; then
echo "ok $PASS/$TOTAL next/tests/log_memory.sh passed"
else
echo "FAIL $PASS/$TOTAL passed, $FAIL failed:"
echo "$ERRORS"
fi
[ $FAIL -eq 0 ]

View File

@@ -99,6 +99,10 @@ in isolation, and a clear acceptance check.
## Step 1 — Repo skeleton + canonical CID ## Step 1 — Repo skeleton + canonical CID
**Sub-deliverables:**
- [x] **1a**`next/` directory skeleton, README, `.gitignore` for `data/`
- [x] **1b**`next/kernel/nx_cid.erl` (from_sx/to_string/from_string/equals) + `next/tests/cid.sh` (13 cases). Module is `nx_cid` not `cid` — the `cid` BIF module would be shadowed by a user module of the same name; plan §Step 1's `cid.erl` is illustrative per briefing.
**Deliverables:** **Deliverables:**
``` ```
@@ -146,6 +150,11 @@ canonicalize_sx(V) -> ... % sorts dict keys, normalizes strings
## Step 2 — Activity envelope + signature verify ## Step 2 — Activity envelope + signature verify
**Sub-deliverables:**
- [x] **2a**`next/kernel/envelope.erl` `validate_shape/1` + `get_field/2` (property-list envelope; Erlang maps `#{}` not supported in this port) + `next/tests/envelope_shape.sh` (15 cases)
- [x] **2b**`canonical_bytes/1` over sig-stripped, key-sorted envelope (deterministic textual form via `cid:to_string` substrate; dag-cbor stand-in for v1) + `next/tests/envelope_canonical.sh` (8 cases)
- [x] **2c**`verify_signature/2` against actor `public_keys`, time-aware key validity per design §9.6 (created ≤ published, optional supersession check) + `next/tests/envelope_sig.sh` (11 cases). Signature scheme is HMAC-shaped (`crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`) — RSA/Ed25519 verify deferred to m2 (BIFs not yet wired).
**Deliverables:** **Deliverables:**
```erlang ```erlang
@@ -186,6 +195,13 @@ verify_signature(Activity, ActorState) ->
## Step 3 — JSONL log + sequence numbers ## Step 3 — JSONL log + sequence numbers
**Sub-deliverables:**
- [x] **3a**`log:open/2` + `log:append/2` + `log:tip/1` + `log:replay/3` + `log:entries/1` over an in-memory log state (per-actor seq; replay in append order; round-trip the stored activity). `next/tests/log_memory.sh` (12 cases).
- [ ] **3b***Parked behind substrate gap (see Blockers below).* Term codec + on-disk persistence: serializer/parser writing each activity as a JSONL-style line; restart-resumes-tip from the segment file.
- [ ] **3c** — Segment rotation at size threshold + gen_server-mediated concurrent appends.
**Blockers (Step 3b):** The Erlang port returns SX strings (an opaque OCaml-string type) from `atom_to_list/1` and `integer_to_list/1`, rejects them from `++`/list pattern matching, and does not register `binary_to_list`/`list_to_binary`. `$X` character literals decode to `nil` in `parse-number`. Net effect: there is no in-Erlang path from an arbitrary term to a byte sequence (or back) that doesn't go through a temp-file round-trip through the filesystem. Workaround paths: (a) add a `term_to_binary`/`binary_to_term` BIF in a separate substrate loop, (b) accept a filesystem-mediated SX-string→binary helper and live with the O(N) IO cost, (c) restrict the on-disk format to a binary-only encoding with a per-instance atom-id table for atoms (introduces an extra durability dependency). Decision to defer; revisit once a downstream Step (58) forces the issue or a substrate BIF arrives. In-memory log from 3a is sufficient to unblock Step 5+ which consume the API surface.
**Deliverables:** **Deliverables:**
```erlang ```erlang
@@ -227,6 +243,17 @@ replay(LogState, InitAcc, Fun) -> ...
## Step 4 — Genesis bundle ## Step 4 — Genesis bundle
**Sub-deliverables:**
- [x] **4a** — Seed genesis SX file authoring: `next/genesis/manifest.sx` + `next/genesis/activity-types/create.sx`. Manifest uses bare parenthesised paths (data lists, not `(list ...)` calls — consumed by `parse`, not `eval`). `next/tests/genesis_parse.sh` (5 cases).
- [x] **4b-act** — Remaining activity-types: `update.sx` + `delete.sx`, manifest updated, parse tests (10 cases total in `genesis_parse.sh`)
- [x] **4b-obj** — Object-types: SXArtifact, Note, Tombstone, DefineActivity, DefineObject, DefineProjection, DefineValidator, DefineCodec, DefineSigSuite, Snapshot — 10 `DefineObject` files + manifest updated + 12 new parse tests
- [ ] **4b-proj** — Projections: activity-log, by-type, by-actor, by-object, actor-state, define-registry, audience-graph
- [ ] **4b-vld** — Validators: envelope-shape, signature, type-schema
- [ ] **4b-cod** — Codecs + sig-suites + audience predicates
- [ ] **4c**`bootstrap:read_genesis/1` in Erlang: walk the manifest, file-read each referenced .sx, return parsed forms
- [ ] **4d**`bootstrap:build_genesis/1` + `bootstrap:verify_genesis/1`: compute bundle CID over the read forms via the host `cid:to_string` substrate; verify against a stored `bundle.cidhash`
- [ ] **4e**`bootstrap:load_genesis/1`: register parsed definitions into the in-memory registry (depends on Step 5)
**Deliverables:** **Deliverables:**
Genesis bundle SX sources (per design §12.2). Each is a small SX file authored Genesis bundle SX sources (per design §12.2). Each is a small SX file authored
@@ -920,3 +947,21 @@ A few things still under-specified; resolve as work begins.
60 seconds." Tunable per-projection later; v1 uses the default. 60 seconds." Tunable per-projection later; v1 uses the default.
5. **Genesis bundle format.** Dag-cbor map per §12.2; concrete schema needs 5. **Genesis bundle format.** Dag-cbor map per §12.2; concrete schema needs
one round of refinement once we author the actual definitions in step 4. one round of refinement once we author the actual definitions in step 4.
---
## Progress log
Newest first. One line per sub-deliverable commit. Erlang conformance gate
(`bash lib/erlang/conformance.sh`) must remain 729/729 on every entry.
- **2026-05-27** — Step 4b-obj: bootstrap object-types complete — 10 `DefineObject` SX files authored (SXArtifact, Note, Tombstone, DefineActivity, DefineObject, DefineProjection, DefineValidator, DefineCodec, DefineSigSuite, Snapshot). Each carries an SX `:schema` predicate. Manifest `:object-types` populated; `next/tests/genesis_parse.sh` 22/22. Erlang conformance 729/729.
- **2026-05-27** — Step 4b-act: bootstrap activity-types complete — `update.sx` (Update verb, requires :object CID + :patch) + `delete.sx` (Delete verb, requires :object CID) authored as DefineActivity forms matching the Create shape. Manifest updated; `next/tests/genesis_parse.sh` 10/10. Step 4b broken into act/obj/proj/vld/cod sub-deliverables on the plan. Erlang conformance 729/729.
- **2026-05-27** — Step 4a: genesis bundle seeded. `next/genesis/manifest.sx` (GenesisManifest with eight section keys, only `:activity-types` populated for now) + `next/genesis/activity-types/create.sx` (DefineActivity{Create} with :schema/:semantics SX bodies). `next/tests/genesis_parse.sh` 5/5. Step 3b parked behind a substrate-level term-codec gap — Blockers note added under Step 3; in-memory log from 3a unblocks Step 5+ which only need the API surface. Erlang conformance 729/729.
- **2026-05-27** — Step 3a: `log:open/2 append/2 tip/1 replay/3 entries/1` over an in-memory state (per-actor seq, replay in append order, round-trip activities). `next/tests/log_memory.sh` 12/12. Pivoted from on-disk in this iteration: this port's `atom_to_list`/`integer_to_list` return SX strings rather than Erlang charlists, `binary_to_list` is unregistered, and `$X` char literals decode to nil — so a term codec needs a workaround. Captured as the Step 3b risk note in the plan. Erlang conformance 729/729.
- **2026-05-26** — Step 2c: `envelope:verify_signature/2` — time-aware key lookup over `public_keys` (created ≤ published < superseded_at), MAC recompute via `crypto:hash(sha256, KeyMaterial ++ canonical_bytes)`, compared against `signature.value`. Returns ok or one of `no_signature | no_key_id | no_published | no_keys | no_active_key | bad_signature`. `next/tests/envelope_sig.sh` 11/11 pass. Erlang conformance 729/729.
- **2026-05-26** — Step 2b: `envelope:canonical_bytes/1` — strip signature, insertion-sort property list by key, return host-CID-string as deterministic byte form (dag-cbor stand-in). `next/tests/envelope_canonical.sh` 8/8 pass. Erlang conformance 729/729 preserved.
- **2026-05-26** — Step 2a: `next/kernel/envelope.erl` `validate_shape/1` + `get_field/2` over property-list envelopes (Erlang `#{}` maps not supported in this port). `next/tests/envelope_shape.sh` 15/15 pass. Erlang conformance 729/729 preserved.
- **2026-05-26** — Step 1b: `next/kernel/nx_cid.erl` (from_sx/to_string/from_string/equals) — thin Erlang wrapper around the `cid:to_string/1` BIF. `next/tests/cid.sh` 13/13 pass. Module named `nx_cid` to avoid shadowing the `cid` BIF (user-module dispatch takes precedence over BIFs by module name). Erlang conformance 729/729 preserved.
- **2026-05-26** — Step 1a: `next/` skeleton created (kernel/, genesis/, tests/, data/), README, `.gitignore data/`. Erlang conformance 729/729 preserved.