From f6d584629e62abc2a19a35f7d507e0f57650146c Mon Sep 17 00:00:00 2001 From: giles Date: Sat, 4 Jul 2026 01:17:50 +0000 Subject: [PATCH] =?UTF-8?q?W14:=20env-parity=20ledger=20=E2=80=94=20runner?= =?UTF-8?q?-only=20bindings=20vs=20fresh=20sx=5Fserver=20(test-only)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Section-B audit, all verified live over the epoch protocol. Runner-only bindings absent from production: values, call-with-values (run_tests.ml :1131/:1140), contains-char? (rt.ml:728 + rt.js:85), trim-right (JS runner ONLY — absent even from the OCaml runner), sha3-256 (rt.ml:745 + rt.js:88; production's real primitive is crypto-sha3-256). Consequences pinned: (canonical-serialize 42) on a fresh server errors "Undefined symbol: contains-char?" — content addressing broken for ANY number outside the runners. And BOTH runners' sha3-256 are FAKE stubs (OCaml: Hashtbl.hash), so every test-computed CID differs from production. scripts/test-env-parity.sh is a bidirectional ledger: MUST_HAVE bindings going missing fail; a KNOWN_DRIFT binding APPEARING also fails with instructions to move it to MUST_HAVE and flip the consequence pin — the ledger cannot rot silently in either direction. 7/7 green. Test-only: no semantics edits, no push. Co-Authored-By: Claude Fable 5 --- plans/agent-briefings/sx-gate-loop.md | 22 +++++- scripts/test-env-parity.sh | 100 ++++++++++++++++++++++++++ 2 files changed, 119 insertions(+), 3 deletions(-) create mode 100755 scripts/test-env-parity.sh diff --git a/plans/agent-briefings/sx-gate-loop.md b/plans/agent-briefings/sx-gate-loop.md index 0d69a69b..3ce4f562 100644 --- a/plans/agent-briefings/sx-gate-loop.md +++ b/plans/agent-briefings/sx-gate-loop.md @@ -59,9 +59,11 @@ Pin each confirmed-and-fixed finding with a minimal repro. Add suites to conformance — "housekeeping" was a mislabel from F-15's tag) ### B. Runner/production env unification -- [ ] Audit runner-only bindings (`values`/`call-with-values` F7/K42, JS - fake sha3/equal?/apply/env-set! shims JS5) — inventory + failing pin - that a fresh `sx_server` reproduces the drift +- [x] Audit runner-only bindings — inventory + bidirectional ledger in + `scripts/test-env-parity.sh` (KNOWN_DRIFT: values, call-with-values, + contains-char?, trim-right, sha3-256; consequence pin: + canonical-serialize broken on server; BOTH runners' sha3-256 are FAKE + stubs → test CIDs ≠ production CIDs) ### C. Harness honesty - [ ] K19 — MCP `mcp_tree.ml` harness primitive table drift vs `sx_primitives` @@ -83,6 +85,20 @@ Pin each confirmed-and-fixed finding with a minimal repro. Add suites to ## Progress log (newest first) +- 2026-07-04 — **Section B: env-parity audit + ledger**. Probed a fresh + `sx_server` over the epoch protocol (`deps-check` + live eval). Confirmed + runner-only drift: `values`/`call-with-values` (run_tests.ml:1131/1140), + `contains-char?` (rt.ml:728 + rt.js:85), `trim-right` (**JS runner only** + — absent even from the OCaml runner), `sha3-256` (rt.ml:745 + rt.js:88). + Consequence verified live: `(canonical-serialize 42)` on the server → + `Undefined symbol: contains-char?` (content addressing broken for ANY + number outside the runners). **Worse than the finding**: BOTH runners' + `sha3-256` are FAKE stubs (OCaml uses `Hashtbl.hash`!) while production + has real `crypto-sha3-256` — every CID computed in tests differs from + production CIDs. Authored `scripts/test-env-parity.sh` as a bidirectional + ledger: MUST_HAVE regressions fail; a KNOWN_DRIFT binding *appearing* + also fails (forces ledger + consequence-pin update when W5/W7/W12 land + fixes). 7/7 green. Test-only. - 2026-07-04 — **S4 error-page-cache pin (item A.7) — section A COMPLETE**. Extended `scripts/test-protocol-gate.sh` with an HTTP-mode case: fresh `sx_server.exe --http ` (timeout-bounded, own PID killed at diff --git a/scripts/test-env-parity.sh b/scripts/test-env-parity.sh new file mode 100755 index 00000000..3babf652 --- /dev/null +++ b/scripts/test-env-parity.sh @@ -0,0 +1,100 @@ +#!/bin/bash +# test-env-parity.sh — W14 section-B ledger: runner env vs production env. +# +# The review (F7, K42, JS5, core.md "canonical.sx depends on test-runner-only +# helpers") found bindings that exist ONLY in the test runners, so suites +# pass against an environment production never provides. Rule (PLAN.md W14): +# "if the spec needs it, it's a kernel primitive; if not, the test can't +# have it." +# +# This script is a LEDGER, not a wish: it asserts today's confirmed drift +# stays exactly as recorded. Both directions fail loudly: +# - a MUST_HAVE going missing on the server -> regression, fix the kernel +# - a KNOWN_DRIFT binding appearing on the server -> the fix landed; +# move it to MUST_HAVE and update the consequence pins below. +# +# Confirmed inventory (2026-07-04, all verified live over the epoch protocol): +# +# binding OCaml runner JS runner fresh sx_server +# values real (rt.ml:1131) ? ABSENT +# call-with-values real (rt.ml:1140) ? ABSENT +# contains-char? real (rt.ml:728) real (:85) ABSENT +# trim-right ABSENT real (:87) ABSENT +# sha3-256 FAKE Hashtbl.hash FAKE stub ABSENT (real = crypto-sha3-256) +# +# Consequences (pinned in section 3): +# - (canonical-serialize 42) on a fresh server errors "Undefined symbol: +# contains-char?" -> content addressing broken for ANY number outside +# the test runners. +# - every CID computed inside run_tests uses a FAKE hash, so test CIDs +# never equal production CIDs (crypto-sha3-256 is real SHA3). +# +# Each probe spawns its OWN timeout-bounded sx_server.exe. No shared process. +set -uo pipefail + +cd "$(dirname "$0")/.." +SERVER=hosts/ocaml/_build/default/bin/sx_server.exe + +if [[ ! -x "$SERVER" ]]; then + echo "SKIP: $SERVER not built (run sx_build target=ocaml first)" >&2 + exit 2 +fi + +pass=0 +fail=0 + +# deps_unresolved EXPR -> prints the (unresolved ...) list for EXPR on a fresh server +deps_unresolved() { + printf '(epoch 1)\n(deps-check "%s")\n' "$1" \ + | timeout 60 "$SERVER" 2>/dev/null \ + | grep -o ':unresolved ([^)]*)' || true +} + +# --- Section 1: MUST_HAVE — spec-needed bindings production must provide --- +MUST_HAVE_EXPR='(list (equal? 1 1) (apply + (list 1 2)) (contains? {:a 1} :a) (crypto-sha3-256 \"x\") (split \"a-b\" \"-\"))' +unres=$(deps_unresolved "$MUST_HAVE_EXPR") +if [[ -z "$unres" || "$unres" == ':unresolved ()' ]]; then + echo "PASS: MUST_HAVE core bindings all resolve on fresh sx_server" + pass=$((pass+1)) +else + echo "FAIL: MUST_HAVE binding missing on fresh sx_server: $unres" + fail=$((fail+1)) +fi + +# --- Section 2: KNOWN_DRIFT — runner-only bindings, asserted ABSENT ------- +# If one of these starts resolving, its kernel fix landed: move it to +# MUST_HAVE above and update the consequence pin in section 3. +for name in values call-with-values contains-char? trim-right sha3-256; do + unres=$(deps_unresolved "($name)") + if grep -q -- "$name" <<<"$unres"; then + echo "PASS: KNOWN_DRIFT '$name' still absent on fresh sx_server (ledger accurate)" + pass=$((pass+1)) + else + echo "FAIL: KNOWN_DRIFT '$name' now RESOLVES on fresh sx_server — fix landed?" + echo " Update this ledger: move '$name' to MUST_HAVE and revisit section 3." + fail=$((fail+1)) + fi +done + +# --- Section 3: consequence pin — canonical.sx on the production server --- +# Current reality: canonical-serialize of ANY number errors on a fresh +# server because canonical-number calls runner-only contains-char?. +out=$(printf '(epoch 1)\n(load "spec/canonical.sx")\n(epoch 2)\n(eval "(canonical-serialize 42)")\n' \ + | timeout 60 "$SERVER" 2>&1) +if grep -q 'error 2 .*contains-char?' <<<"$out"; then + echo "PASS: consequence pin — canonical-serialize on numbers still broken on server (as recorded)" + pass=$((pass+1)) +elif grep -q '^(ok 2 ' <<<"$out"; then + echo "FAIL: consequence pin — canonical-serialize 42 now WORKS on the server." + echo " The canonical-helpers fix landed: flip this pin to assert success" + echo " and pin the exact canonical form + CID stability." + fail=$((fail+1)) +else + echo "FAIL: consequence pin — unexpected server output:" + sed 's/^/ /' <<<"$out" + fail=$((fail+1)) +fi + +echo +echo "env-parity: $pass passed, $fail failed" +[[ $fail -eq 0 ]]