W14: env-parity ledger — runner-only bindings vs fresh sx_server (test-only)
Section-B audit, all verified live over the epoch protocol. Runner-only bindings absent from production: values, call-with-values (run_tests.ml :1131/:1140), contains-char? (rt.ml:728 + rt.js:85), trim-right (JS runner ONLY — absent even from the OCaml runner), sha3-256 (rt.ml:745 + rt.js:88; production's real primitive is crypto-sha3-256). Consequences pinned: (canonical-serialize 42) on a fresh server errors "Undefined symbol: contains-char?" — content addressing broken for ANY number outside the runners. And BOTH runners' sha3-256 are FAKE stubs (OCaml: Hashtbl.hash), so every test-computed CID differs from production. scripts/test-env-parity.sh is a bidirectional ledger: MUST_HAVE bindings going missing fail; a KNOWN_DRIFT binding APPEARING also fails with instructions to move it to MUST_HAVE and flip the consequence pin — the ledger cannot rot silently in either direction. 7/7 green. Test-only: no semantics edits, no push. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -59,9 +59,11 @@ Pin each confirmed-and-fixed finding with a minimal repro. Add suites to
|
|||||||
conformance — "housekeeping" was a mislabel from F-15's tag)
|
conformance — "housekeeping" was a mislabel from F-15's tag)
|
||||||
|
|
||||||
### B. Runner/production env unification
|
### B. Runner/production env unification
|
||||||
- [ ] Audit runner-only bindings (`values`/`call-with-values` F7/K42, JS
|
- [x] Audit runner-only bindings — inventory + bidirectional ledger in
|
||||||
fake sha3/equal?/apply/env-set! shims JS5) — inventory + failing pin
|
`scripts/test-env-parity.sh` (KNOWN_DRIFT: values, call-with-values,
|
||||||
that a fresh `sx_server` reproduces the drift
|
contains-char?, trim-right, sha3-256; consequence pin:
|
||||||
|
canonical-serialize broken on server; BOTH runners' sha3-256 are FAKE
|
||||||
|
stubs → test CIDs ≠ production CIDs)
|
||||||
|
|
||||||
### C. Harness honesty
|
### C. Harness honesty
|
||||||
- [ ] K19 — MCP `mcp_tree.ml` harness primitive table drift vs `sx_primitives`
|
- [ ] K19 — MCP `mcp_tree.ml` harness primitive table drift vs `sx_primitives`
|
||||||
@@ -83,6 +85,20 @@ Pin each confirmed-and-fixed finding with a minimal repro. Add suites to
|
|||||||
|
|
||||||
## Progress log (newest first)
|
## Progress log (newest first)
|
||||||
|
|
||||||
|
- 2026-07-04 — **Section B: env-parity audit + ledger**. Probed a fresh
|
||||||
|
`sx_server` over the epoch protocol (`deps-check` + live eval). Confirmed
|
||||||
|
runner-only drift: `values`/`call-with-values` (run_tests.ml:1131/1140),
|
||||||
|
`contains-char?` (rt.ml:728 + rt.js:85), `trim-right` (**JS runner only**
|
||||||
|
— absent even from the OCaml runner), `sha3-256` (rt.ml:745 + rt.js:88).
|
||||||
|
Consequence verified live: `(canonical-serialize 42)` on the server →
|
||||||
|
`Undefined symbol: contains-char?` (content addressing broken for ANY
|
||||||
|
number outside the runners). **Worse than the finding**: BOTH runners'
|
||||||
|
`sha3-256` are FAKE stubs (OCaml uses `Hashtbl.hash`!) while production
|
||||||
|
has real `crypto-sha3-256` — every CID computed in tests differs from
|
||||||
|
production CIDs. Authored `scripts/test-env-parity.sh` as a bidirectional
|
||||||
|
ledger: MUST_HAVE regressions fail; a KNOWN_DRIFT binding *appearing*
|
||||||
|
also fails (forces ledger + consequence-pin update when W5/W7/W12 land
|
||||||
|
fixes). 7/7 green. Test-only.
|
||||||
- 2026-07-04 — **S4 error-page-cache pin (item A.7) — section A COMPLETE**.
|
- 2026-07-04 — **S4 error-page-cache pin (item A.7) — section A COMPLETE**.
|
||||||
Extended `scripts/test-protocol-gate.sh` with an HTTP-mode case: fresh
|
Extended `scripts/test-protocol-gate.sh` with an HTTP-mode case: fresh
|
||||||
`sx_server.exe --http <random-port>` (timeout-bounded, own PID killed at
|
`sx_server.exe --http <random-port>` (timeout-bounded, own PID killed at
|
||||||
|
|||||||
100
scripts/test-env-parity.sh
Executable file
100
scripts/test-env-parity.sh
Executable file
@@ -0,0 +1,100 @@
|
|||||||
|
#!/bin/bash
|
||||||
|
# test-env-parity.sh — W14 section-B ledger: runner env vs production env.
|
||||||
|
#
|
||||||
|
# The review (F7, K42, JS5, core.md "canonical.sx depends on test-runner-only
|
||||||
|
# helpers") found bindings that exist ONLY in the test runners, so suites
|
||||||
|
# pass against an environment production never provides. Rule (PLAN.md W14):
|
||||||
|
# "if the spec needs it, it's a kernel primitive; if not, the test can't
|
||||||
|
# have it."
|
||||||
|
#
|
||||||
|
# This script is a LEDGER, not a wish: it asserts today's confirmed drift
|
||||||
|
# stays exactly as recorded. Both directions fail loudly:
|
||||||
|
# - a MUST_HAVE going missing on the server -> regression, fix the kernel
|
||||||
|
# - a KNOWN_DRIFT binding appearing on the server -> the fix landed;
|
||||||
|
# move it to MUST_HAVE and update the consequence pins below.
|
||||||
|
#
|
||||||
|
# Confirmed inventory (2026-07-04, all verified live over the epoch protocol):
|
||||||
|
#
|
||||||
|
# binding OCaml runner JS runner fresh sx_server
|
||||||
|
# values real (rt.ml:1131) ? ABSENT
|
||||||
|
# call-with-values real (rt.ml:1140) ? ABSENT
|
||||||
|
# contains-char? real (rt.ml:728) real (:85) ABSENT
|
||||||
|
# trim-right ABSENT real (:87) ABSENT
|
||||||
|
# sha3-256 FAKE Hashtbl.hash FAKE stub ABSENT (real = crypto-sha3-256)
|
||||||
|
#
|
||||||
|
# Consequences (pinned in section 3):
|
||||||
|
# - (canonical-serialize 42) on a fresh server errors "Undefined symbol:
|
||||||
|
# contains-char?" -> content addressing broken for ANY number outside
|
||||||
|
# the test runners.
|
||||||
|
# - every CID computed inside run_tests uses a FAKE hash, so test CIDs
|
||||||
|
# never equal production CIDs (crypto-sha3-256 is real SHA3).
|
||||||
|
#
|
||||||
|
# Each probe spawns its OWN timeout-bounded sx_server.exe. No shared process.
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
SERVER=hosts/ocaml/_build/default/bin/sx_server.exe
|
||||||
|
|
||||||
|
if [[ ! -x "$SERVER" ]]; then
|
||||||
|
echo "SKIP: $SERVER not built (run sx_build target=ocaml first)" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
pass=0
|
||||||
|
fail=0
|
||||||
|
|
||||||
|
# deps_unresolved EXPR -> prints the (unresolved ...) list for EXPR on a fresh server
|
||||||
|
deps_unresolved() {
|
||||||
|
printf '(epoch 1)\n(deps-check "%s")\n' "$1" \
|
||||||
|
| timeout 60 "$SERVER" 2>/dev/null \
|
||||||
|
| grep -o ':unresolved ([^)]*)' || true
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- Section 1: MUST_HAVE — spec-needed bindings production must provide ---
|
||||||
|
MUST_HAVE_EXPR='(list (equal? 1 1) (apply + (list 1 2)) (contains? {:a 1} :a) (crypto-sha3-256 \"x\") (split \"a-b\" \"-\"))'
|
||||||
|
unres=$(deps_unresolved "$MUST_HAVE_EXPR")
|
||||||
|
if [[ -z "$unres" || "$unres" == ':unresolved ()' ]]; then
|
||||||
|
echo "PASS: MUST_HAVE core bindings all resolve on fresh sx_server"
|
||||||
|
pass=$((pass+1))
|
||||||
|
else
|
||||||
|
echo "FAIL: MUST_HAVE binding missing on fresh sx_server: $unres"
|
||||||
|
fail=$((fail+1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- Section 2: KNOWN_DRIFT — runner-only bindings, asserted ABSENT -------
|
||||||
|
# If one of these starts resolving, its kernel fix landed: move it to
|
||||||
|
# MUST_HAVE above and update the consequence pin in section 3.
|
||||||
|
for name in values call-with-values contains-char? trim-right sha3-256; do
|
||||||
|
unres=$(deps_unresolved "($name)")
|
||||||
|
if grep -q -- "$name" <<<"$unres"; then
|
||||||
|
echo "PASS: KNOWN_DRIFT '$name' still absent on fresh sx_server (ledger accurate)"
|
||||||
|
pass=$((pass+1))
|
||||||
|
else
|
||||||
|
echo "FAIL: KNOWN_DRIFT '$name' now RESOLVES on fresh sx_server — fix landed?"
|
||||||
|
echo " Update this ledger: move '$name' to MUST_HAVE and revisit section 3."
|
||||||
|
fail=$((fail+1))
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# --- Section 3: consequence pin — canonical.sx on the production server ---
|
||||||
|
# Current reality: canonical-serialize of ANY number errors on a fresh
|
||||||
|
# server because canonical-number calls runner-only contains-char?.
|
||||||
|
out=$(printf '(epoch 1)\n(load "spec/canonical.sx")\n(epoch 2)\n(eval "(canonical-serialize 42)")\n' \
|
||||||
|
| timeout 60 "$SERVER" 2>&1)
|
||||||
|
if grep -q 'error 2 .*contains-char?' <<<"$out"; then
|
||||||
|
echo "PASS: consequence pin — canonical-serialize on numbers still broken on server (as recorded)"
|
||||||
|
pass=$((pass+1))
|
||||||
|
elif grep -q '^(ok 2 ' <<<"$out"; then
|
||||||
|
echo "FAIL: consequence pin — canonical-serialize 42 now WORKS on the server."
|
||||||
|
echo " The canonical-helpers fix landed: flip this pin to assert success"
|
||||||
|
echo " and pin the exact canonical form + CID stability."
|
||||||
|
fail=$((fail+1))
|
||||||
|
else
|
||||||
|
echo "FAIL: consequence pin — unexpected server output:"
|
||||||
|
sed 's/^/ /' <<<"$out"
|
||||||
|
fail=$((fail+1))
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "env-parity: $pass passed, $fail failed"
|
||||||
|
[[ $fail -eq 0 ]]
|
||||||
Reference in New Issue
Block a user