W14: env-parity ledger — runner-only bindings vs fresh sx_server (test-only)
Section-B audit, all verified live over the epoch protocol. Runner-only bindings absent from production: values, call-with-values (run_tests.ml :1131/:1140), contains-char? (rt.ml:728 + rt.js:85), trim-right (JS runner ONLY — absent even from the OCaml runner), sha3-256 (rt.ml:745 + rt.js:88; production's real primitive is crypto-sha3-256). Consequences pinned: (canonical-serialize 42) on a fresh server errors "Undefined symbol: contains-char?" — content addressing broken for ANY number outside the runners. And BOTH runners' sha3-256 are FAKE stubs (OCaml: Hashtbl.hash), so every test-computed CID differs from production. scripts/test-env-parity.sh is a bidirectional ledger: MUST_HAVE bindings going missing fail; a KNOWN_DRIFT binding APPEARING also fails with instructions to move it to MUST_HAVE and flip the consequence pin — the ledger cannot rot silently in either direction. 7/7 green. Test-only: no semantics edits, no push. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
100
scripts/test-env-parity.sh
Executable file
100
scripts/test-env-parity.sh
Executable file
@@ -0,0 +1,100 @@
|
||||
#!/bin/bash
|
||||
# test-env-parity.sh — W14 section-B ledger: runner env vs production env.
|
||||
#
|
||||
# The review (F7, K42, JS5, core.md "canonical.sx depends on test-runner-only
|
||||
# helpers") found bindings that exist ONLY in the test runners, so suites
|
||||
# pass against an environment production never provides. Rule (PLAN.md W14):
|
||||
# "if the spec needs it, it's a kernel primitive; if not, the test can't
|
||||
# have it."
|
||||
#
|
||||
# This script is a LEDGER, not a wish: it asserts today's confirmed drift
|
||||
# stays exactly as recorded. Both directions fail loudly:
|
||||
# - a MUST_HAVE going missing on the server -> regression, fix the kernel
|
||||
# - a KNOWN_DRIFT binding appearing on the server -> the fix landed;
|
||||
# move it to MUST_HAVE and update the consequence pins below.
|
||||
#
|
||||
# Confirmed inventory (2026-07-04, all verified live over the epoch protocol):
|
||||
#
|
||||
# binding OCaml runner JS runner fresh sx_server
|
||||
# values real (rt.ml:1131) ? ABSENT
|
||||
# call-with-values real (rt.ml:1140) ? ABSENT
|
||||
# contains-char? real (rt.ml:728) real (:85) ABSENT
|
||||
# trim-right ABSENT real (:87) ABSENT
|
||||
# sha3-256 FAKE Hashtbl.hash FAKE stub ABSENT (real = crypto-sha3-256)
|
||||
#
|
||||
# Consequences (pinned in section 3):
|
||||
# - (canonical-serialize 42) on a fresh server errors "Undefined symbol:
|
||||
# contains-char?" -> content addressing broken for ANY number outside
|
||||
# the test runners.
|
||||
# - every CID computed inside run_tests uses a FAKE hash, so test CIDs
|
||||
# never equal production CIDs (crypto-sha3-256 is real SHA3).
|
||||
#
|
||||
# Each probe spawns its OWN timeout-bounded sx_server.exe. No shared process.
|
||||
set -uo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
SERVER=hosts/ocaml/_build/default/bin/sx_server.exe
|
||||
|
||||
if [[ ! -x "$SERVER" ]]; then
|
||||
echo "SKIP: $SERVER not built (run sx_build target=ocaml first)" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
pass=0
|
||||
fail=0
|
||||
|
||||
# deps_unresolved EXPR -> prints the (unresolved ...) list for EXPR on a fresh server
|
||||
deps_unresolved() {
|
||||
printf '(epoch 1)\n(deps-check "%s")\n' "$1" \
|
||||
| timeout 60 "$SERVER" 2>/dev/null \
|
||||
| grep -o ':unresolved ([^)]*)' || true
|
||||
}
|
||||
|
||||
# --- Section 1: MUST_HAVE — spec-needed bindings production must provide ---
|
||||
MUST_HAVE_EXPR='(list (equal? 1 1) (apply + (list 1 2)) (contains? {:a 1} :a) (crypto-sha3-256 \"x\") (split \"a-b\" \"-\"))'
|
||||
unres=$(deps_unresolved "$MUST_HAVE_EXPR")
|
||||
if [[ -z "$unres" || "$unres" == ':unresolved ()' ]]; then
|
||||
echo "PASS: MUST_HAVE core bindings all resolve on fresh sx_server"
|
||||
pass=$((pass+1))
|
||||
else
|
||||
echo "FAIL: MUST_HAVE binding missing on fresh sx_server: $unres"
|
||||
fail=$((fail+1))
|
||||
fi
|
||||
|
||||
# --- Section 2: KNOWN_DRIFT — runner-only bindings, asserted ABSENT -------
|
||||
# If one of these starts resolving, its kernel fix landed: move it to
|
||||
# MUST_HAVE above and update the consequence pin in section 3.
|
||||
for name in values call-with-values contains-char? trim-right sha3-256; do
|
||||
unres=$(deps_unresolved "($name)")
|
||||
if grep -q -- "$name" <<<"$unres"; then
|
||||
echo "PASS: KNOWN_DRIFT '$name' still absent on fresh sx_server (ledger accurate)"
|
||||
pass=$((pass+1))
|
||||
else
|
||||
echo "FAIL: KNOWN_DRIFT '$name' now RESOLVES on fresh sx_server — fix landed?"
|
||||
echo " Update this ledger: move '$name' to MUST_HAVE and revisit section 3."
|
||||
fail=$((fail+1))
|
||||
fi
|
||||
done
|
||||
|
||||
# --- Section 3: consequence pin — canonical.sx on the production server ---
|
||||
# Current reality: canonical-serialize of ANY number errors on a fresh
|
||||
# server because canonical-number calls runner-only contains-char?.
|
||||
out=$(printf '(epoch 1)\n(load "spec/canonical.sx")\n(epoch 2)\n(eval "(canonical-serialize 42)")\n' \
|
||||
| timeout 60 "$SERVER" 2>&1)
|
||||
if grep -q 'error 2 .*contains-char?' <<<"$out"; then
|
||||
echo "PASS: consequence pin — canonical-serialize on numbers still broken on server (as recorded)"
|
||||
pass=$((pass+1))
|
||||
elif grep -q '^(ok 2 ' <<<"$out"; then
|
||||
echo "FAIL: consequence pin — canonical-serialize 42 now WORKS on the server."
|
||||
echo " The canonical-helpers fix landed: flip this pin to assert success"
|
||||
echo " and pin the exact canonical form + CID stability."
|
||||
fail=$((fail+1))
|
||||
else
|
||||
echo "FAIL: consequence pin — unexpected server output:"
|
||||
sed 's/^/ /' <<<"$out"
|
||||
fail=$((fail+1))
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "env-parity: $pass passed, $fail failed"
|
||||
[[ $fail -eq 0 ]]
|
||||
Reference in New Issue
Block a user