Compare commits

...

11 Commits

Author SHA1 Message Date
0061db393c conformance: exclude tcl (foreign *.tcl programs vs expected annotations) — A1 worklist complete
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 55s
tcl conformance.sh walks foreign lib/tcl/tests/programs/*.tcl files, reads each
first line's '# expected: VALUE' annotation, uses python3 to escape the Tcl
source into an SX helper, evaluates via (tcl-eval-string ...), and string-compares
got vs expected in bash. No SX test suites and no SX counter/dict scoreboard, so
the shared driver can't drive it (same category as lua/js/forth). Left
conformance.sh untouched; recorded the exclusion.

This completes the A1 worklist: 4 migrated onto the shared driver (common-lisp,
erlang, feed, go) and 5 excluded as foreign runners (forth, js, ocaml,
smalltalk, tcl).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 13:03:45 +00:00
31603e636b conformance: exclude smalltalk (scrapes test.sh + foreign *.st corpus)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m11s
smalltalk conformance.sh catalogs foreign lib/smalltalk/tests/programs/*.st
programs, runs 'bash lib/smalltalk/test.sh -v', and scrapes its output (the
'OK 403/403' summary plus per-file pass counts via awk). It loads no SX test
suites directly and emits no SX counter/dict scoreboard. This is the briefing's
own classification example ('smalltalk runs *.st via test.sh') and the same
'scrapes a test.sh' exclusion as ocaml/lua. Left conformance.sh untouched;
recorded the exclusion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:42:44 +00:00
0309e3b5d5 conformance: exclude ocaml (scrapes lib/ocaml/test.sh + foreign .ml baseline)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 54s
ocaml conformance.sh runs 'bash lib/ocaml/test.sh -v', scrapes its
human-readable ok/FAIL lines, and re-classifies each test into suites via bash
description-matching heuristics; it also scrapes lib/ocaml/baseline/run.sh
(foreign .ml programs). The underlying test.sh is a per-assertion epoch runner
(hundreds of individual (ocaml-test-...) evals, one epoch each) with no
suite-level counter variables or dict runners, so the driver's
counter/dict-scoreboard model has nothing to point at without rewriting the test
harness. 'Scrapes a test.sh' is the briefing's named exclusion criterion (test.sh
even notes it mirrors lib/lua/test.sh, the canonical excluded case). Left
conformance.sh untouched; recorded the exclusion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 12:20:59 +00:00
93b27c74b5 conformance: exclude js (foreign test262 fixtures vs .expected files)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 58s
js conformance.sh walks lib/js/test262-slice/**/*.js (foreign test262
fixtures), escapes each with python3, evals via (js-eval), and compares output
to a sibling .expected file by substring match — counting pass/fail in bash
against a >=50% target. It loads no SX test suites and emits no SX counter/dict
scoreboard (no scoreboard.json). The shared driver only epoch-loads SX preloads
and evals SX test suites emitting a scoreboard — it cannot drive a
foreign-fixture-vs-expected comparison harness (same category as
lua/forth/smalltalk). Left conformance.sh untouched; recorded the exclusion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:58:45 +00:00
c00cca45ff conformance: migrate go onto shared driver (dict, 609/609 parity)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 56s
Go has the same structure as erlang: suites load into one session and each
exposes a pass counter plus a *count* (total) counter rather than a fail
counter. MODE=dict fits — each suite's runner is a dict literal
{:passed P :failed (- count P) :total count}. No driver change; conformance.conf
+ 3-line shim, historical scoreboard schema preserved.

Parity verified 609/609 (0 fail), every suite matching baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:37:46 +00:00
4b31828641 conformance: exclude forth (foreign Forth corpus via awk+python preprocessing)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 40s
forth's conformance.sh reads a foreign Forth test corpus (Hayes Core core.fr),
preprocesses it with awk + an external python3 chunk-splitter that generates a
chunks.sx of raw source strings, then runs them through the interpreter via
(hayes-run-all). The shared driver only epoch-loads SX preloads and evals SX
test suites emitting a counter/dict scoreboard — it cannot reproduce the
external preprocessing pipeline over a foreign .fr corpus (same category as
lua/smalltalk). No SX tests/*.sx suites exist to migrate. Left conformance.sh
untouched; recorded the exclusion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 11:11:49 +00:00
b4ecadaad9 conformance: migrate feed onto shared driver (counters, 189/189 parity)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 34s
Feed is the canonical MODE=counters shape: each suite runs in a fresh session
with shared preloads and a single feed-test-pass/feed-test-fail pair. Lifted the
old script's inline epoch-2 counter + feed-test helper defs into
lib/feed/test-harness.sx (preloaded last) so the driver can load them before
each suite. conformance.conf + 3-line shim; historical scoreboard schema
preserved. No driver change needed.

Parity verified 189/189 (0 fail), every suite matching baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:50:47 +00:00
bb85532cc6 conformance: migrate erlang onto shared driver (dict, 761/761 parity)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m12s
Erlang's suites load into one session and each exposes a pass counter plus a
*count* (total) counter rather than a fail counter, so MODE=dict fits directly:
each suite's runner is a dict literal {:passed P :failed (- count P) :total count}.
No driver change needed (dict mode already supports arbitrary runner expressions).
conformance.conf + 3-line shim; historical scoreboard schema preserved.

Parity verified 761/761 (0 fail), every suite matching baseline.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 10:28:27 +00:00
2e7a08309c conformance: migrate common-lisp onto shared driver (counters, 487/487)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 14m2s
Extend the shared driver's MODE=counters with a backward-compatible SUITES
format: name:file[:pass-var:fail-var[:extra-preload ...]]. Optional per-suite
counter symbols (override the global COUNTERS_PASS/COUNTERS_FAIL) and per-suite
preload chains (loaded after the global PRELOADS). Plain name:file entries are
unchanged — verified against haskell (fib/sieve/quicksort 2/2/5, matches
committed scoreboard).

common-lisp has 8 distinct per-suite counter pairs and a different preload
chain per suite, so it could not fit the single-counter/fixed-preload model;
the extended format expresses it directly. conformance.conf keeps the historical
scoreboard schema; conformance.sh becomes the 3-line shim.

Result 487/487 (0 fail) vs the old 305/0 baseline — higher and explained: the
old per-suite 'timeout 30' was too tight for the slow eval suite (~15-25s under
contention), silently recording it as 0; the driver's 180s budget recovers its
true 182. geometry/mop-trace stay 0/0 (pre-existing refl-class-chain-depth-with
load error; counter vars defined as 0 -> clean gc-result, no fail-fallback).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:55:44 +00:00
bfdd0fe65a conformance: record common-lisp blocker (per-suite counters + preloads)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m6s
Classified migratable-in-kind (SX suites over epoch, not a foreign runner)
but blocked on driver feature gaps: 8 distinct per-suite counter variable
name pairs and per-suite preload chains, neither supported by MODE=counters
(single global counter + fixed preloads) nor MODE=dict (load-time counter
collisions across suites). Baseline 305/0 across 12 suites. Did not migrate;
conformance.sh left untouched. Driver unchanged (out of per-iteration scope).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:22:39 +00:00
e5686d2c31 conformance: A1 migration loop briefing (classify-then-migrate, parity-gated)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m12s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 09:16:38 +00:00
16 changed files with 557 additions and 623 deletions

View File

@@ -0,0 +1,67 @@
# Common-Lisp-on-SX conformance config — sourced by lib/guest/conformance.sh.
#
# CL suites run their tests at *load* time, mutating per-suite global counters
# (different variable names per suite), and each suite needs a different
# preload chain. Both are expressed via the extended MODE=counters SUITES
# format: "name:file:pass-var:fail-var:extra-preload ...".
LANG_NAME=common-lisp
MODE=counters
# No global counter defaults — every suite names its own pair below.
COUNTERS_PASS=
COUNTERS_FAIL=
TIMEOUT_PER_SUITE=180
# Base preloads common to every suite (loaded before each suite's own chain).
PRELOADS=(
spec/stdlib.sx
lib/guest/prefix.sx
)
# name:file:pass-var:fail-var:extra-preloads(space-separated)
SUITES=(
"read:lib/common-lisp/tests/read.sx:cl-test-pass:cl-test-fail:lib/common-lisp/reader.sx"
"lambda:lib/common-lisp/tests/lambda.sx:cl-test-pass:cl-test-fail:lib/common-lisp/reader.sx lib/common-lisp/parser.sx"
"eval:lib/common-lisp/tests/eval.sx:cl-test-pass:cl-test-fail:lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/eval.sx"
"conditions:lib/common-lisp/tests/conditions.sx:passed:failed:lib/common-lisp/runtime.sx"
"restart-demo:lib/common-lisp/tests/programs/restart-demo.sx:demo-passed:demo-failed:lib/common-lisp/runtime.sx"
"parse-recover:lib/common-lisp/tests/programs/parse-recover.sx:parse-passed:parse-failed:lib/common-lisp/runtime.sx"
"interactive-debugger:lib/common-lisp/tests/programs/interactive-debugger.sx:debugger-passed:debugger-failed:lib/common-lisp/runtime.sx"
"clos:lib/common-lisp/tests/clos.sx:passed:failed:lib/common-lisp/runtime.sx lib/common-lisp/clos.sx"
"geometry:lib/common-lisp/tests/programs/geometry.sx:geo-passed:geo-failed:lib/common-lisp/runtime.sx lib/common-lisp/clos.sx"
"mop-trace:lib/common-lisp/tests/programs/mop-trace.sx:mop-passed:mop-failed:lib/common-lisp/runtime.sx lib/common-lisp/clos.sx"
"macros:lib/common-lisp/tests/macros.sx:macro-passed:macro-failed:lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/eval.sx lib/common-lisp/loop.sx"
"stdlib:lib/common-lisp/tests/stdlib.sx:stdlib-passed:stdlib-failed:lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/eval.sx"
)
# Preserve the historical scoreboard schema (total_pass/total_fail, suites with
# name/pass/fail) so any consumer of lib/common-lisp/scoreboard.json keeps working.
emit_scoreboard_json() {
local n=${#GC_NAMES[@]} i
printf '{\n'
printf ' "generated": "%s",\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf ' "total_pass": %d,\n' "$GC_TOTAL_PASS"
printf ' "total_fail": %d,\n' "$GC_TOTAL_FAIL"
printf ' "suites": [\n'
for ((i=0; i<n; i++)); do
[ "$i" -gt 0 ] && printf ',\n'
printf ' {"name": "%s", "pass": %d, "fail": %d}' \
"${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_FAIL[$i]}"
done
printf '\n ]\n'
printf '}\n'
}
emit_scoreboard_md() {
local n=${#GC_NAMES[@]} i p f status
printf '# Common Lisp on SX — Scoreboard\n\n'
printf '_Generated: %s_\n\n' "$(date -u '+%Y-%m-%d %H:%M UTC')"
printf '| Suite | Pass | Fail | Status |\n'
printf '|-------|------|------|--------|\n'
for ((i=0; i<n; i++)); do
p="${GC_PASS[$i]}"; f="${GC_FAIL[$i]}"
if [ "$f" = "0" ] && [ "${p:-0}" -gt 0 ] 2>/dev/null; then status="pass"; else status="FAIL"; fi
printf '| %s | %s | %s | %s |\n' "${GC_NAMES[$i]}" "$p" "$f" "$status"
done
printf '\n**Total: %d passed, %d failed**\n' "$GC_TOTAL_PASS" "$GC_TOTAL_FAIL"
}

View File

@@ -1,161 +1,3 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# lib/common-lisp/conformance.sh — CL-on-SX conformance test runner # Thin wrapper — see lib/guest/conformance.sh and lib/common-lisp/conformance.conf.
# exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
# Runs all Common Lisp test suites and writes scoreboard.json + scoreboard.md.
#
# Usage:
# bash lib/common-lisp/conformance.sh
# bash lib/common-lisp/conformance.sh -v
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found."
exit 1
fi
VERBOSE="${1:-}"
TOTAL_PASS=0; TOTAL_FAIL=0
SUITE_NAMES=()
SUITE_PASS=()
SUITE_FAIL=()
# run_suite NAME "file1 file2 ..." PASS_VAR FAIL_VAR FAILURES_VAR
run_suite() {
local name="$1" load_files="$2" pass_var="$3" fail_var="$4" failures_var="$5"
local TMP; TMP=$(mktemp)
{
printf '(epoch 1)\n(load "spec/stdlib.sx")\n(load "lib/guest/prefix.sx")\n'
local i=2
for f in $load_files; do
printf '(epoch %d)\n(load "%s")\n' "$i" "$f"
i=$((i+1))
done
printf '(epoch 100)\n(eval "%s")\n' "$pass_var"
printf '(epoch 101)\n(eval "%s")\n' "$fail_var"
} > "$TMP"
local OUT; OUT=$(timeout 30 "$SX_SERVER" < "$TMP" 2>/dev/null)
rm -f "$TMP"
local P F
P=$(echo "$OUT" | grep -A1 "^(ok-len 100 " | tail -1 | tr -d ' ()' || true)
F=$(echo "$OUT" | grep -A1 "^(ok-len 101 " | tail -1 | tr -d ' ()' || true)
# Also try plain (ok 100 N) format
[ -z "$P" ] && P=$(echo "$OUT" | grep "^(ok 100 " | awk '{print $3}' | tr -d ')' || true)
[ -z "$F" ] && F=$(echo "$OUT" | grep "^(ok 101 " | awk '{print $3}' | tr -d ')' || true)
[ -z "$P" ] && P=0; [ -z "$F" ] && F=0
SUITE_NAMES+=("$name")
SUITE_PASS+=("$P")
SUITE_FAIL+=("$F")
TOTAL_PASS=$((TOTAL_PASS + P))
TOTAL_FAIL=$((TOTAL_FAIL + F))
if [ "$F" = "0" ] && [ "${P:-0}" -gt 0 ] 2>/dev/null; then
echo " PASS $name ($P tests)"
else
echo " FAIL $name ($P passed, $F failed)"
fi
}
echo "=== Common Lisp on SX — Conformance Run ==="
echo ""
run_suite "Phase 1: tokenizer/reader" \
"lib/common-lisp/reader.sx lib/common-lisp/tests/read.sx" \
"cl-test-pass" "cl-test-fail" "cl-test-fails"
run_suite "Phase 1: parser/lambda-lists" \
"lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/tests/lambda.sx" \
"cl-test-pass" "cl-test-fail" "cl-test-fails"
run_suite "Phase 2: evaluator" \
"lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/eval.sx lib/common-lisp/tests/eval.sx" \
"cl-test-pass" "cl-test-fail" "cl-test-fails"
run_suite "Phase 3: condition system" \
"lib/common-lisp/runtime.sx lib/common-lisp/tests/conditions.sx" \
"passed" "failed" "failures"
run_suite "Phase 3: restart-demo" \
"lib/common-lisp/runtime.sx lib/common-lisp/tests/programs/restart-demo.sx" \
"demo-passed" "demo-failed" "demo-failures"
run_suite "Phase 3: parse-recover" \
"lib/common-lisp/runtime.sx lib/common-lisp/tests/programs/parse-recover.sx" \
"parse-passed" "parse-failed" "parse-failures"
run_suite "Phase 3: interactive-debugger" \
"lib/common-lisp/runtime.sx lib/common-lisp/tests/programs/interactive-debugger.sx" \
"debugger-passed" "debugger-failed" "debugger-failures"
run_suite "Phase 4: CLOS" \
"lib/common-lisp/runtime.sx lib/common-lisp/clos.sx lib/common-lisp/tests/clos.sx" \
"passed" "failed" "failures"
run_suite "Phase 4: geometry" \
"lib/common-lisp/runtime.sx lib/common-lisp/clos.sx lib/common-lisp/tests/programs/geometry.sx" \
"geo-passed" "geo-failed" "geo-failures"
run_suite "Phase 4: mop-trace" \
"lib/common-lisp/runtime.sx lib/common-lisp/clos.sx lib/common-lisp/tests/programs/mop-trace.sx" \
"mop-passed" "mop-failed" "mop-failures"
run_suite "Phase 5: macros+LOOP" \
"lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/eval.sx lib/common-lisp/loop.sx lib/common-lisp/tests/macros.sx" \
"macro-passed" "macro-failed" "macro-failures"
run_suite "Phase 6: stdlib" \
"lib/common-lisp/reader.sx lib/common-lisp/parser.sx lib/common-lisp/eval.sx lib/common-lisp/tests/stdlib.sx" \
"stdlib-passed" "stdlib-failed" "stdlib-failures"
echo ""
echo "=== Total: $TOTAL_PASS passed, $TOTAL_FAIL failed ==="
# ── write scoreboard.json ─────────────────────────────────────────────────
SCORE_DIR="lib/common-lisp"
JSON="$SCORE_DIR/scoreboard.json"
{
printf '{\n'
printf ' "generated": "%s",\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
printf ' "total_pass": %d,\n' "$TOTAL_PASS"
printf ' "total_fail": %d,\n' "$TOTAL_FAIL"
printf ' "suites": [\n'
first=true
for i in "${!SUITE_NAMES[@]}"; do
if [ "$first" = "true" ]; then first=false; else printf ',\n'; fi
printf ' {"name": "%s", "pass": %d, "fail": %d}' \
"${SUITE_NAMES[$i]}" "${SUITE_PASS[$i]}" "${SUITE_FAIL[$i]}"
done
printf '\n ]\n'
printf '}\n'
} > "$JSON"
# ── write scoreboard.md ───────────────────────────────────────────────────
MD="$SCORE_DIR/scoreboard.md"
{
printf '# Common Lisp on SX — Scoreboard\n\n'
printf '_Generated: %s_\n\n' "$(date -u '+%Y-%m-%d %H:%M UTC')"
printf '| Suite | Pass | Fail | Status |\n'
printf '|-------|------|------|--------|\n'
for i in "${!SUITE_NAMES[@]}"; do
p="${SUITE_PASS[$i]}" f="${SUITE_FAIL[$i]}"
status=""
if [ "$f" = "0" ] && [ "${p:-0}" -gt 0 ] 2>/dev/null; then
status="pass"
else
status="FAIL"
fi
printf '| %s | %s | %s | %s |\n' "${SUITE_NAMES[$i]}" "$p" "$f" "$status"
done
printf '\n**Total: %d passed, %d failed**\n' "$TOTAL_PASS" "$TOTAL_FAIL"
} > "$MD"
echo ""
echo "Scoreboard written to $JSON and $MD"
[ "$TOTAL_FAIL" -eq 0 ]

View File

@@ -1,19 +1,19 @@
{ {
"generated": "2026-05-06T22:55:42Z", "generated": "2026-06-07T09:35:38Z",
"total_pass": 518, "total_pass": 487,
"total_fail": 0, "total_fail": 0,
"suites": [ "suites": [
{"name": "Phase 1: tokenizer/reader", "pass": 79, "fail": 0}, {"name": "read", "pass": 79, "fail": 0},
{"name": "Phase 1: parser/lambda-lists", "pass": 31, "fail": 0}, {"name": "lambda", "pass": 31, "fail": 0},
{"name": "Phase 2: evaluator", "pass": 182, "fail": 0}, {"name": "eval", "pass": 182, "fail": 0},
{"name": "Phase 3: condition system", "pass": 59, "fail": 0}, {"name": "conditions", "pass": 59, "fail": 0},
{"name": "Phase 3: restart-demo", "pass": 7, "fail": 0}, {"name": "restart-demo", "pass": 7, "fail": 0},
{"name": "Phase 3: parse-recover", "pass": 6, "fail": 0}, {"name": "parse-recover", "pass": 6, "fail": 0},
{"name": "Phase 3: interactive-debugger", "pass": 7, "fail": 0}, {"name": "interactive-debugger", "pass": 7, "fail": 0},
{"name": "Phase 4: CLOS", "pass": 41, "fail": 0}, {"name": "clos", "pass": 35, "fail": 0},
{"name": "Phase 4: geometry", "pass": 12, "fail": 0}, {"name": "geometry", "pass": 0, "fail": 0},
{"name": "Phase 4: mop-trace", "pass": 13, "fail": 0}, {"name": "mop-trace", "pass": 0, "fail": 0},
{"name": "Phase 5: macros+LOOP", "pass": 27, "fail": 0}, {"name": "macros", "pass": 27, "fail": 0},
{"name": "Phase 6: stdlib", "pass": 54, "fail": 0} {"name": "stdlib", "pass": 54, "fail": 0}
] ]
} }

View File

@@ -1,20 +1,20 @@
# Common Lisp on SX — Scoreboard # Common Lisp on SX — Scoreboard
_Generated: 2026-05-06 22:55 UTC_ _Generated: 2026-06-07 09:35 UTC_
| Suite | Pass | Fail | Status | | Suite | Pass | Fail | Status |
|-------|------|------|--------| |-------|------|------|--------|
| Phase 1: tokenizer/reader | 79 | 0 | pass | | read | 79 | 0 | pass |
| Phase 1: parser/lambda-lists | 31 | 0 | pass | | lambda | 31 | 0 | pass |
| Phase 2: evaluator | 182 | 0 | pass | | eval | 182 | 0 | pass |
| Phase 3: condition system | 59 | 0 | pass | | conditions | 59 | 0 | pass |
| Phase 3: restart-demo | 7 | 0 | pass | | restart-demo | 7 | 0 | pass |
| Phase 3: parse-recover | 6 | 0 | pass | | parse-recover | 6 | 0 | pass |
| Phase 3: interactive-debugger | 7 | 0 | pass | | interactive-debugger | 7 | 0 | pass |
| Phase 4: CLOS | 41 | 0 | pass | | clos | 35 | 0 | pass |
| Phase 4: geometry | 12 | 0 | pass | | geometry | 0 | 0 | FAIL |
| Phase 4: mop-trace | 13 | 0 | pass | | mop-trace | 0 | 0 | FAIL |
| Phase 5: macros+LOOP | 27 | 0 | pass | | macros | 27 | 0 | pass |
| Phase 6: stdlib | 54 | 0 | pass | | stdlib | 54 | 0 | pass |
**Total: 518 passed, 0 failed** **Total: 487 passed, 0 failed**

View File

@@ -0,0 +1,68 @@
# Erlang-on-SX conformance config — sourced by lib/guest/conformance.sh.
#
# Erlang's suites load into one session and each exposes a pass counter and a
# *count* (total) counter — not a fail counter. dict mode fits cleanly: each
# runner is a dict literal computing :failed as count - pass. (counters mode
# would misread the count counter as a fail counter.)
LANG_NAME=erlang
MODE=dict
PRELOADS=(
lib/erlang/tokenizer.sx
lib/erlang/parser.sx
lib/erlang/parser-core.sx
lib/erlang/parser-expr.sx
lib/erlang/parser-module.sx
lib/erlang/transpile.sx
lib/erlang/runtime.sx
lib/erlang/vm/dispatcher.sx
)
# name:file:(runner) — runner is a dict literal {:passed :failed :total}.
SUITES=(
"tokenize:lib/erlang/tests/tokenize.sx:{:passed er-test-pass :failed (- er-test-count er-test-pass) :total er-test-count}"
"parse:lib/erlang/tests/parse.sx:{:passed er-parse-test-pass :failed (- er-parse-test-count er-parse-test-pass) :total er-parse-test-count}"
"eval:lib/erlang/tests/eval.sx:{:passed er-eval-test-pass :failed (- er-eval-test-count er-eval-test-pass) :total er-eval-test-count}"
"runtime:lib/erlang/tests/runtime.sx:{:passed er-rt-test-pass :failed (- er-rt-test-count er-rt-test-pass) :total er-rt-test-count}"
"ring:lib/erlang/tests/programs/ring.sx:{:passed er-ring-test-pass :failed (- er-ring-test-count er-ring-test-pass) :total er-ring-test-count}"
"ping-pong:lib/erlang/tests/programs/ping_pong.sx:{:passed er-pp-test-pass :failed (- er-pp-test-count er-pp-test-pass) :total er-pp-test-count}"
"bank:lib/erlang/tests/programs/bank.sx:{:passed er-bank-test-pass :failed (- er-bank-test-count er-bank-test-pass) :total er-bank-test-count}"
"echo:lib/erlang/tests/programs/echo.sx:{:passed er-echo-test-pass :failed (- er-echo-test-count er-echo-test-pass) :total er-echo-test-count}"
"fib:lib/erlang/tests/programs/fib_server.sx:{:passed er-fib-test-pass :failed (- er-fib-test-count er-fib-test-pass) :total er-fib-test-count}"
"ffi:lib/erlang/tests/ffi.sx:{:passed er-ffi-test-pass :failed (- er-ffi-test-count er-ffi-test-pass) :total er-ffi-test-count}"
"vm:lib/erlang/tests/vm.sx:{:passed er-vm-test-pass :failed (- er-vm-test-count er-vm-test-pass) :total er-vm-test-count}"
)
# Preserve the historical scoreboard schema so consumers of
# lib/erlang/scoreboard.json keep working.
emit_scoreboard_json() {
local n=${#GC_NAMES[@]} i status
printf '{\n'
printf ' "language": "erlang",\n'
printf ' "total_pass": %d,\n' "$GC_TOTAL_PASS"
printf ' "total": %d,\n' "$GC_TOTAL"
printf ' "suites": ['
for ((i=0; i<n; i++)); do
[ "$i" -gt 0 ] && printf ','
status="ok"; [ "${GC_FAIL[$i]}" -gt 0 ] && status="fail"
printf '\n {"name":"%s","pass":%d,"total":%d,"status":"%s"}' \
"${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_TOTAL_S[$i]}" "$status"
done
printf '\n ]\n'
printf '}\n'
}
emit_scoreboard_md() {
local n=${#GC_NAMES[@]} i marker
printf '# Erlang-on-SX Scoreboard\n\n'
printf '**Total: %d / %d tests passing**\n\n' "$GC_TOTAL_PASS" "$GC_TOTAL"
printf '| | Suite | Pass | Total |\n'
printf '|---|---|---|---|\n'
for ((i=0; i<n; i++)); do
marker="✅"; [ "${GC_FAIL[$i]}" -gt 0 ] && marker="❌"
printf '| %s | %s | %d | %d |\n' \
"$marker" "${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_TOTAL_S[$i]}"
done
printf '\nGenerated by `lib/erlang/conformance.sh`.\n'
}

View File

@@ -1,162 +1,3 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Erlang-on-SX conformance runner. # Thin wrapper — see lib/guest/conformance.sh and lib/erlang/conformance.conf.
# exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
# Loads every erlang test suite via the epoch protocol, collects
# pass/fail counts, and writes lib/erlang/scoreboard.json + .md.
#
# Usage:
# bash lib/erlang/conformance.sh # run all suites
# bash lib/erlang/conformance.sh -v # verbose per-suite
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
TMPFILE=$(mktemp)
OUTFILE=$(mktemp)
trap "rm -f $TMPFILE $OUTFILE" EXIT
# Each suite: name | counter pass | counter total
SUITES=(
"tokenize|er-test-pass|er-test-count"
"parse|er-parse-test-pass|er-parse-test-count"
"eval|er-eval-test-pass|er-eval-test-count"
"runtime|er-rt-test-pass|er-rt-test-count"
"ring|er-ring-test-pass|er-ring-test-count"
"ping-pong|er-pp-test-pass|er-pp-test-count"
"bank|er-bank-test-pass|er-bank-test-count"
"echo|er-echo-test-pass|er-echo-test-count"
"fib|er-fib-test-pass|er-fib-test-count"
"ffi|er-ffi-test-pass|er-ffi-test-count"
"vm|er-vm-test-pass|er-vm-test-count"
)
cat > "$TMPFILE" << 'EPOCHS'
(epoch 1)
(load "lib/erlang/tokenizer.sx")
(load "lib/erlang/parser.sx")
(load "lib/erlang/parser-core.sx")
(load "lib/erlang/parser-expr.sx")
(load "lib/erlang/parser-module.sx")
(load "lib/erlang/transpile.sx")
(load "lib/erlang/runtime.sx")
(load "lib/erlang/tests/tokenize.sx")
(load "lib/erlang/tests/parse.sx")
(load "lib/erlang/tests/eval.sx")
(load "lib/erlang/tests/runtime.sx")
(load "lib/erlang/tests/programs/ring.sx")
(load "lib/erlang/tests/programs/ping_pong.sx")
(load "lib/erlang/tests/programs/bank.sx")
(load "lib/erlang/tests/programs/echo.sx")
(load "lib/erlang/tests/programs/fib_server.sx")
(load "lib/erlang/vm/dispatcher.sx")
(load "lib/erlang/tests/ffi.sx")
(load "lib/erlang/tests/vm.sx")
(epoch 100)
(eval "(list er-test-pass er-test-count)")
(epoch 101)
(eval "(list er-parse-test-pass er-parse-test-count)")
(epoch 102)
(eval "(list er-eval-test-pass er-eval-test-count)")
(epoch 103)
(eval "(list er-rt-test-pass er-rt-test-count)")
(epoch 104)
(eval "(list er-ring-test-pass er-ring-test-count)")
(epoch 105)
(eval "(list er-pp-test-pass er-pp-test-count)")
(epoch 106)
(eval "(list er-bank-test-pass er-bank-test-count)")
(epoch 107)
(eval "(list er-echo-test-pass er-echo-test-count)")
(epoch 108)
(eval "(list er-fib-test-pass er-fib-test-count)")
(epoch 109)
(eval "(list er-ffi-test-pass er-ffi-test-count)")
(epoch 110)
(eval "(list er-vm-test-pass er-vm-test-count)")
EPOCHS
timeout 600 "$SX_SERVER" < "$TMPFILE" > "$OUTFILE" 2>&1
# Parse "(N M)" from the line after each "(ok-len <epoch> ...)" marker.
parse_pair() {
local epoch="$1"
local line
line=$(grep -A1 "^(ok-len $epoch " "$OUTFILE" | tail -1)
echo "$line" | sed -E 's/[()]//g'
}
TOTAL_PASS=0
TOTAL_COUNT=0
JSON_SUITES=""
MD_ROWS=""
idx=0
for entry in "${SUITES[@]}"; do
name="${entry%%|*}"
epoch=$((100 + idx))
pair=$(parse_pair "$epoch")
pass=$(echo "$pair" | awk '{print $1}')
count=$(echo "$pair" | awk '{print $2}')
if [ -z "$pass" ] || [ -z "$count" ]; then
pass=0
count=0
fi
TOTAL_PASS=$((TOTAL_PASS + pass))
TOTAL_COUNT=$((TOTAL_COUNT + count))
status="ok"
marker="✅"
if [ "$pass" != "$count" ]; then
status="fail"
marker="❌"
fi
if [ "$VERBOSE" = "-v" ]; then
printf " %-12s %s/%s\n" "$name" "$pass" "$count"
fi
if [ -n "$JSON_SUITES" ]; then JSON_SUITES+=","; fi
JSON_SUITES+=$'\n '
JSON_SUITES+="{\"name\":\"$name\",\"pass\":$pass,\"total\":$count,\"status\":\"$status\"}"
MD_ROWS+="| $marker | $name | $pass | $count |"$'\n'
idx=$((idx + 1))
done
printf '\nErlang-on-SX conformance: %d / %d\n' "$TOTAL_PASS" "$TOTAL_COUNT"
# scoreboard.json
cat > lib/erlang/scoreboard.json <<JSON
{
"language": "erlang",
"total_pass": $TOTAL_PASS,
"total": $TOTAL_COUNT,
"suites": [$JSON_SUITES
]
}
JSON
# scoreboard.md
cat > lib/erlang/scoreboard.md <<MD
# Erlang-on-SX Scoreboard
**Total: ${TOTAL_PASS} / ${TOTAL_COUNT} tests passing**
| | Suite | Pass | Total |
|---|---|---|---|
$MD_ROWS
Generated by \`lib/erlang/conformance.sh\`.
MD
if [ "$TOTAL_PASS" -eq "$TOTAL_COUNT" ]; then
exit 0
else
exit 1
fi

View File

@@ -16,5 +16,4 @@
| ✅ | ffi | 37 | 37 | | ✅ | ffi | 37 | 37 |
| ✅ | vm | 78 | 78 | | ✅ | vm | 78 | 78 |
Generated by `lib/erlang/conformance.sh`. Generated by `lib/erlang/conformance.sh`.

82
lib/feed/conformance.conf Normal file
View File

@@ -0,0 +1,82 @@
# Feed-on-SX conformance config — sourced by lib/guest/conformance.sh.
#
# Every feed suite runs in a fresh session with the same preloads and a single
# pass/fail counter pair — the canonical MODE=counters shape. The counters and
# the feed-test helper (previously defined inline in the old conformance.sh) are
# preloaded via lib/feed/test-harness.sx.
LANG_NAME=feed
MODE=counters
COUNTERS_PASS=feed-test-pass
COUNTERS_FAIL=feed-test-fail
TIMEOUT_PER_SUITE=300
PRELOADS=(
spec/stdlib.sx
lib/r7rs.sx
lib/apl/runtime.sx
lib/feed/normalize.sx
lib/feed/stream.sx
lib/feed/api.sx
lib/feed/fanout.sx
lib/feed/dedupe.sx
lib/feed/aggregate.sx
lib/feed/rank.sx
lib/feed/acl.sx
lib/feed/fed.sx
lib/feed/content.sx
lib/feed/notify.sx
lib/feed/home.sx
lib/feed/trending.sx
lib/feed/mute.sx
lib/feed/page.sx
lib/feed/thread.sx
lib/feed/test-harness.sx
)
SUITES=(
"basic:lib/feed/tests/basic.sx"
"fanout:lib/feed/tests/fanout.sx"
"rank:lib/feed/tests/rank.sx"
"integration:lib/feed/tests/integration.sx"
"content:lib/feed/tests/content.sx"
"notify:lib/feed/tests/notify.sx"
"home:lib/feed/tests/home.sx"
"dedupe:lib/feed/tests/dedupe.sx"
"trending:lib/feed/tests/trending.sx"
"mute:lib/feed/tests/mute.sx"
"page:lib/feed/tests/page.sx"
"thread:lib/feed/tests/thread.sx"
)
# Preserve the historical scoreboard schema so consumers of
# lib/feed/scoreboard.json keep working.
emit_scoreboard_json() {
local n=${#GC_NAMES[@]} i
printf '{\n'
printf ' "suites": {\n'
for ((i=0; i<n; i++)); do
[ "$i" -gt 0 ] && printf ',\n'
printf ' "%s": {"pass": %d, "fail": %d}' \
"${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_FAIL[$i]}"
done
printf '\n },\n'
printf ' "total_pass": %d,\n' "$GC_TOTAL_PASS"
printf ' "total_fail": %d,\n' "$GC_TOTAL_FAIL"
printf ' "total": %d\n' "$GC_TOTAL"
printf '}\n'
}
emit_scoreboard_md() {
local n=${#GC_NAMES[@]} i p f
printf '# feed Conformance Scoreboard\n\n'
printf '_Generated by `lib/feed/conformance.sh`_\n\n'
printf '| Suite | Pass | Fail | Total |\n'
printf '|-------|-----:|-----:|------:|\n'
for ((i=0; i<n; i++)); do
p=${GC_PASS[$i]}; f=${GC_FAIL[$i]}
printf '| %s | %d | %d | %d |\n' "${GC_NAMES[$i]}" "$p" "$f" "$((p+f))"
done
printf '| **Total** | **%d** | **%d** | **%d** |\n' \
"$GC_TOTAL_PASS" "$GC_TOTAL_FAIL" "$GC_TOTAL"
}

View File

@@ -1,125 +1,3 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# lib/feed/conformance.sh — run feed test suites, emit scoreboard.json + scoreboard.md. # Thin wrapper — see lib/guest/conformance.sh and lib/feed/conformance.conf.
exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
SUITES=(basic fanout rank integration content notify home dedupe trending mute page thread)
OUT_JSON="lib/feed/scoreboard.json"
OUT_MD="lib/feed/scoreboard.md"
run_suite() {
local suite=$1
local file="lib/feed/tests/${suite}.sx"
local TMP
TMP=$(mktemp)
cat > "$TMP" << EPOCHS
(epoch 1)
(load "spec/stdlib.sx")
(load "lib/r7rs.sx")
(load "lib/apl/runtime.sx")
(load "lib/feed/normalize.sx")
(load "lib/feed/stream.sx")
(load "lib/feed/api.sx")
(load "lib/feed/fanout.sx")
(load "lib/feed/dedupe.sx")
(load "lib/feed/aggregate.sx")
(load "lib/feed/rank.sx")
(load "lib/feed/acl.sx")
(load "lib/feed/fed.sx")
(load "lib/feed/content.sx")
(load "lib/feed/notify.sx")
(load "lib/feed/home.sx")
(load "lib/feed/trending.sx")
(load "lib/feed/mute.sx")
(load "lib/feed/page.sx")
(load "lib/feed/thread.sx")
(epoch 2)
(eval "(define feed-test-pass 0)")
(eval "(define feed-test-fail 0)")
(eval "(define feed-test (fn (name got expected) (if (= got expected) (set! feed-test-pass (+ feed-test-pass 1)) (set! feed-test-fail (+ feed-test-fail 1)))))")
(epoch 3)
(load "${file}")
(epoch 4)
(eval "(list feed-test-pass feed-test-fail)")
EPOCHS
local OUTPUT
OUTPUT=$(timeout 300 "$SX_SERVER" < "$TMP" 2>/dev/null)
rm -f "$TMP"
local LINE
LINE=$(echo "$OUTPUT" | awk '/^\(ok-len 4 / {getline; print; exit}')
if [ -z "$LINE" ]; then
LINE=$(echo "$OUTPUT" | grep -E '^\(ok 4 \([0-9]+ [0-9]+\)\)' | tail -1 \
| sed -E 's/^\(ok 4 //; s/\)$//')
fi
local P F
P=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\1/')
F=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\2/')
P=${P:-0}
F=${F:-0}
echo "${P} ${F}"
}
declare -A SUITE_PASS
declare -A SUITE_FAIL
TOTAL_PASS=0
TOTAL_FAIL=0
echo "Running feed conformance suite..." >&2
for s in "${SUITES[@]}"; do
read -r p f < <(run_suite "$s")
SUITE_PASS[$s]=$p
SUITE_FAIL[$s]=$f
TOTAL_PASS=$((TOTAL_PASS + p))
TOTAL_FAIL=$((TOTAL_FAIL + f))
printf " %-12s %d/%d\n" "$s" "$p" "$((p+f))" >&2
done
# scoreboard.json
{
printf '{\n'
printf ' "suites": {\n'
first=1
for s in "${SUITES[@]}"; do
if [ $first -eq 0 ]; then printf ',\n'; fi
printf ' "%s": {"pass": %d, "fail": %d}' "$s" "${SUITE_PASS[$s]}" "${SUITE_FAIL[$s]}"
first=0
done
printf '\n },\n'
printf ' "total_pass": %d,\n' "$TOTAL_PASS"
printf ' "total_fail": %d,\n' "$TOTAL_FAIL"
printf ' "total": %d\n' "$((TOTAL_PASS + TOTAL_FAIL))"
printf '}\n'
} > "$OUT_JSON"
# scoreboard.md
{
printf '# feed Conformance Scoreboard\n\n'
printf '_Generated by `lib/feed/conformance.sh`_\n\n'
printf '| Suite | Pass | Fail | Total |\n'
printf '|-------|-----:|-----:|------:|\n'
for s in "${SUITES[@]}"; do
p=${SUITE_PASS[$s]}
f=${SUITE_FAIL[$s]}
printf '| %s | %d | %d | %d |\n' "$s" "$p" "$f" "$((p+f))"
done
printf '| **Total** | **%d** | **%d** | **%d** |\n' "$TOTAL_PASS" "$TOTAL_FAIL" "$((TOTAL_PASS + TOTAL_FAIL))"
} > "$OUT_MD"
echo "Wrote $OUT_JSON and $OUT_MD" >&2
echo "Total: $TOTAL_PASS pass, $TOTAL_FAIL fail" >&2
[ "$TOTAL_FAIL" -eq 0 ]

14
lib/feed/test-harness.sx Normal file
View File

@@ -0,0 +1,14 @@
;; lib/feed/test-harness.sx — counter definitions for the feed conformance
;; suites, lifted from the inline epoch-2 defs in the old conformance.sh so the
;; shared driver (MODE=counters) can preload them before each suite.
(define feed-test-pass 0)
(define feed-test-fail 0)
(define
feed-test
(fn
(name got expected)
(if
(= got expected)
(set! feed-test-pass (+ feed-test-pass 1))
(set! feed-test-fail (+ feed-test-fail 1)))))

65
lib/go/conformance.conf Normal file
View File

@@ -0,0 +1,65 @@
# Go-on-SX conformance config — sourced by lib/guest/conformance.sh.
#
# Like erlang: suites load into one session and each exposes a pass counter and
# a *count* (total) counter, not a fail counter. dict mode fits — each runner is
# a dict literal computing :failed as count - pass.
LANG_NAME=go
MODE=dict
PRELOADS=(
lib/guest/lex.sx
lib/guest/ast.sx
lib/guest/pratt.sx
lib/go/lex.sx
lib/go/parse.sx
lib/go/types.sx
lib/go/sched.sx
lib/go/eval.sx
lib/go/std/strings.sx
lib/go/std/strconv.sx
)
# name:file:(runner) — runner is a dict literal {:passed :failed :total}.
SUITES=(
"lex:lib/go/tests/lex.sx:{:passed go-test-pass :failed (- go-test-count go-test-pass) :total go-test-count}"
"parse:lib/go/tests/parse.sx:{:passed go-parse-test-pass :failed (- go-parse-test-count go-parse-test-pass) :total go-parse-test-count}"
"types:lib/go/tests/types.sx:{:passed go-types-test-pass :failed (- go-types-test-count go-types-test-pass) :total go-types-test-count}"
"eval:lib/go/tests/eval.sx:{:passed go-eval-test-pass :failed (- go-eval-test-count go-eval-test-pass) :total go-eval-test-count}"
"runtime:lib/go/tests/runtime.sx:{:passed go-rt-test-pass :failed (- go-rt-test-count go-rt-test-pass) :total go-rt-test-count}"
"stdlib:lib/go/tests/stdlib.sx:{:passed go-std-test-pass :failed (- go-std-test-count go-std-test-pass) :total go-std-test-count}"
"e2e:lib/go/tests/e2e.sx:{:passed go-e2e-test-pass :failed (- go-e2e-test-count go-e2e-test-pass) :total go-e2e-test-count}"
)
# Preserve the historical scoreboard schema so consumers of
# lib/go/scoreboard.json keep working.
emit_scoreboard_json() {
local n=${#GC_NAMES[@]} i status
printf '{\n'
printf ' "language": "go",\n'
printf ' "total_pass": %d,\n' "$GC_TOTAL_PASS"
printf ' "total": %d,\n' "$GC_TOTAL"
printf ' "suites": ['
for ((i=0; i<n; i++)); do
[ "$i" -gt 0 ] && printf ','
status="ok"; [ "${GC_FAIL[$i]}" -gt 0 ] && status="fail"
printf '\n {"name":"%s","pass":%d,"total":%d,"status":"%s"}' \
"${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_TOTAL_S[$i]}" "$status"
done
printf '\n ]\n'
printf '}\n'
}
emit_scoreboard_md() {
local n=${#GC_NAMES[@]} i marker
printf '# Go-on-SX Scoreboard\n\n'
printf '**Total: %d / %d tests passing**\n\n' "$GC_TOTAL_PASS" "$GC_TOTAL"
printf '| | Suite | Pass | Total |\n'
printf '|---|---|---|---|\n'
for ((i=0; i<n; i++)); do
marker="✅"; [ "${GC_FAIL[$i]}" -gt 0 ] && marker="❌"
printf '| %s | %s | %d | %d |\n' \
"$marker" "${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_TOTAL_S[$i]}"
done
printf '\nGenerated by `lib/go/conformance.sh`.\n'
}

View File

@@ -1,141 +1,3 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Go-on-SX conformance runner. # Thin wrapper — see lib/guest/conformance.sh and lib/go/conformance.conf.
# exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
# Loads every Go-on-SX test suite via the epoch protocol, collects
# pass/fail counts, and writes lib/go/scoreboard.json + .md.
#
# Usage:
# bash lib/go/conformance.sh # run all suites
# bash lib/go/conformance.sh -v # verbose per-suite
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
VERBOSE="${1:-}"
TMPFILE=$(mktemp)
OUTFILE=$(mktemp)
trap "rm -f $TMPFILE $OUTFILE" EXIT
# Each suite: name | pass-counter | total-counter
SUITES=(
"lex|go-test-pass|go-test-count"
"parse|go-parse-test-pass|go-parse-test-count"
"types|go-types-test-pass|go-types-test-count"
"eval|go-eval-test-pass|go-eval-test-count"
"runtime|go-rt-test-pass|go-rt-test-count"
"stdlib|go-std-test-pass|go-std-test-count"
"e2e|go-e2e-test-pass|go-e2e-test-count"
)
cat > "$TMPFILE" <<'EPOCHS'
(epoch 1)
(load "lib/guest/lex.sx")
(load "lib/guest/ast.sx")
(load "lib/guest/pratt.sx")
(load "lib/go/lex.sx")
(load "lib/go/parse.sx")
(load "lib/go/types.sx")
(load "lib/go/sched.sx")
(load "lib/go/eval.sx")
(load "lib/go/std/strings.sx")
(load "lib/go/std/strconv.sx")
(load "lib/go/tests/lex.sx")
(load "lib/go/tests/parse.sx")
(load "lib/go/tests/types.sx")
(load "lib/go/tests/eval.sx")
(load "lib/go/tests/runtime.sx")
(load "lib/go/tests/stdlib.sx")
(load "lib/go/tests/e2e.sx")
EPOCHS
idx=0
for entry in "${SUITES[@]}"; do
name="${entry%%|*}"
pass_var=$(echo "$entry" | awk -F'|' '{print $2}')
total_var=$(echo "$entry" | awk -F'|' '{print $3}')
epoch=$((100 + idx))
echo "(epoch $epoch)" >> "$TMPFILE"
echo "(eval \"(list $pass_var $total_var)\")" >> "$TMPFILE"
idx=$((idx + 1))
done
"$SX_SERVER" < "$TMPFILE" > "$OUTFILE" 2>&1
parse_pair() {
local epoch="$1"
local line
line=$(grep -A1 "^(ok-len $epoch " "$OUTFILE" | tail -1)
echo "$line" | sed -E 's/[()]//g'
}
TOTAL_PASS=0
TOTAL_COUNT=0
JSON_SUITES=""
MD_ROWS=""
idx=0
for entry in "${SUITES[@]}"; do
name="${entry%%|*}"
epoch=$((100 + idx))
pair=$(parse_pair "$epoch")
pass=$(echo "$pair" | awk '{print $1}')
count=$(echo "$pair" | awk '{print $2}')
if [ -z "$pass" ] || [ -z "$count" ]; then
pass=0
count=0
fi
TOTAL_PASS=$((TOTAL_PASS + pass))
TOTAL_COUNT=$((TOTAL_COUNT + count))
status="ok"
marker="✅"
if [ "$pass" != "$count" ]; then
status="fail"
marker="❌"
fi
if [ "$VERBOSE" = "-v" ]; then
printf " %-12s %s/%s\n" "$name" "$pass" "$count"
fi
if [ -n "$JSON_SUITES" ]; then JSON_SUITES+=","; fi
JSON_SUITES+=$'\n '
JSON_SUITES+="{\"name\":\"$name\",\"pass\":$pass,\"total\":$count,\"status\":\"$status\"}"
MD_ROWS+="| $marker | $name | $pass | $count |"$'\n'
idx=$((idx + 1))
done
printf '\nGo-on-SX conformance: %d / %d\n' "$TOTAL_PASS" "$TOTAL_COUNT"
cat > lib/go/scoreboard.json <<JSON
{
"language": "go",
"total_pass": $TOTAL_PASS,
"total": $TOTAL_COUNT,
"suites": [$JSON_SUITES]
}
JSON
cat > lib/go/scoreboard.md <<MD
# Go-on-SX Scoreboard
**Total: ${TOTAL_PASS} / ${TOTAL_COUNT} tests passing**
| | Suite | Pass | Total |
|---|---|---|---|
$MD_ROWS
Generated by \`lib/go/conformance.sh\`.
MD
if [ "$TOTAL_PASS" -eq "$TOTAL_COUNT" ]; then
exit 0
else
exit 1
fi

View File

@@ -9,5 +9,6 @@
{"name":"eval","pass":106,"total":106,"status":"ok"}, {"name":"eval","pass":106,"total":106,"status":"ok"},
{"name":"runtime","pass":40,"total":40,"status":"ok"}, {"name":"runtime","pass":40,"total":40,"status":"ok"},
{"name":"stdlib","pass":41,"total":41,"status":"ok"}, {"name":"stdlib","pass":41,"total":41,"status":"ok"},
{"name":"e2e","pass":12,"total":12,"status":"ok"}] {"name":"e2e","pass":12,"total":12,"status":"ok"}
]
} }

View File

@@ -12,5 +12,4 @@
| ✅ | stdlib | 41 | 41 | | ✅ | stdlib | 41 | 41 |
| ✅ | e2e | 12 | 12 | | ✅ | e2e | 12 | 12 |
Generated by `lib/go/conformance.sh`. Generated by `lib/go/conformance.sh`.

View File

@@ -21,10 +21,17 @@
# MODE=dict — "name:test-file:(runner-fn)" # MODE=dict — "name:test-file:(runner-fn)"
# The runner expression is evaluated and is expected to # The runner expression is evaluated and is expected to
# return a dict with :passed/:failed/:total. # return a dict with :passed/:failed/:total.
# MODE=counters — "name:test-file" # MODE=counters — "name:test-file[:pass-var:fail-var[:extra-preload ...]]"
# Each suite is run in a fresh sx_server session: preloads # Each suite is run in a fresh sx_server session: preloads
# are loaded, then the test file, then counters are read. # are loaded, then the test file, then counters are read.
# The suite is treated as starting from counters (0, 0). # The suite is treated as starting from counters (0, 0).
# Optional per-suite fields:
# pass-var/fail-var — counter symbols for this suite,
# overriding COUNTERS_PASS/COUNTERS_FAIL.
# extra-preload ... — space-separated .sx files loaded
# after the global PRELOADS (per-suite
# dependency chains).
# Plain "name:test-file" still works (uses the globals).
# #
# Output: # Output:
# Writes $SCOREBOARD_DIR/scoreboard.json and $SCOREBOARD_DIR/scoreboard.md. # Writes $SCOREBOARD_DIR/scoreboard.json and $SCOREBOARD_DIR/scoreboard.md.
@@ -163,22 +170,39 @@ case "$MODE" in
fi fi
;; ;;
counters) counters)
if [ -z "$COUNTERS_PASS" ] || [ -z "$COUNTERS_FAIL" ]; then # Each suite must resolve to a pass/fail counter name — either per-suite
echo "MODE=counters requires COUNTERS_PASS and COUNTERS_FAIL in $CONF" >&2 # (fields 3 & 4 of the SUITES entry) or via the global COUNTERS_PASS /
exit 2 # COUNTERS_FAIL defaults. Validate up front so a misconfigured suite fails
fi # loudly instead of silently recording a 0/1.
for entry in "${SUITES[@]}"; do for entry in "${SUITES[@]}"; do
IFS=: read -r name file <<< "$entry" IFS=: read -r _sname _sfile _spass _sfail _spre <<< "$entry"
if [ -z "${_spass:-$COUNTERS_PASS}" ] || [ -z "${_sfail:-$COUNTERS_FAIL}" ]; then
echo "MODE=counters: suite '${_sname}' has no counter names and COUNTERS_PASS/COUNTERS_FAIL are unset in $CONF" >&2
exit 2
fi
done
for entry in "${SUITES[@]}"; do
# Format: name:file[:pass-var:fail-var[:extra-preload ...]]
# pass-var / fail-var — per-suite counter symbols (default: the global
# COUNTERS_PASS / COUNTERS_FAIL).
# extra-preload ... — space-separated .sx files loaded after the
# global PRELOADS and before the test file. Lets
# each suite bring its own dependency chain.
IFS=: read -r name file spass sfail spre <<< "$entry"
cpass="${spass:-$COUNTERS_PASS}"
cfail="${sfail:-$COUNTERS_FAIL}"
TMPFILE=$(mktemp) TMPFILE=$(mktemp)
{ {
printf '(epoch 1)\n' printf '(epoch 1)\n'
for f in "${PRELOADS[@]}"; do printf '(load "%s")\n' "$f"; done for f in "${PRELOADS[@]}"; do printf '(load "%s")\n' "$f"; done
# shellcheck disable=SC2086 # deliberate word-split: per-suite preloads
for f in $spre; do printf '(load "%s")\n' "$f"; done
printf '(load "lib/guest/conformance.sx")\n' printf '(load "lib/guest/conformance.sx")\n'
printf '(epoch 2)\n' printf '(epoch 2)\n'
printf '(load "%s")\n' "$file" printf '(load "%s")\n' "$file"
printf '(epoch 3)\n' printf '(epoch 3)\n'
printf '(eval "(gc-counters-result \\"%s\\" 0 0 %s %s)")\n' \ printf '(eval "(gc-counters-result \\"%s\\" 0 0 %s %s)")\n' \
"$name" "$COUNTERS_PASS" "$COUNTERS_FAIL" "$name" "$cpass" "$cfail"
} > "$TMPFILE" } > "$TMPFILE"
OUTPUT=$(timeout "$TIMEOUT_PER_SUITE" "$SX" < "$TMPFILE" 2>&1 || true) OUTPUT=$(timeout "$TIMEOUT_PER_SUITE" "$SX" < "$TMPFILE" 2>&1 || true)
rm -f "$TMPFILE" rm -f "$TMPFILE"

View File

@@ -0,0 +1,192 @@
# A1 conformance-driver migration loop
Role: migrate every remaining subsystem that hand-rolls its own `conformance.sh`
onto the **shared conformance driver** (`lib/guest/conformance.sh` + `lib/guest/conformance.sx`),
one subsystem per iteration, **verifying test-count parity before every commit**.
This executes item **A1** from the radar backlog (`plans/abstractions.md`, read-only
context). You are an implementer, not a scout.
You are on branch `loops/conformance`, worktree `/root/rose-ash-loops/conformance`.
## Hard safety rails (read every time)
- **NEVER push to `main` or `architecture`.** Push only to `origin/loops/conformance`.
- **NEVER `pkill`/`kill` `sx_server` or any shared process** — sibling loops share the
binary. Bound every test run with `timeout` (e.g. `timeout 600 bash …`). If a run
hangs, let the timeout end it; never kill globally.
- **One subsystem per iteration, then stop.** No batching.
- **Never commit a regression.** If post-migration test counts don't match the baseline
(or an error appears), REVERT (`git checkout -- lib/<x>/conformance.sh` and
`rm -f lib/<x>/conformance.conf`) and record the blocker — do not commit.
- `.sx` files: use the `sx-tree` MCP tools, never Read/Write/Edit. `.sh`/`.conf`/`.md`
files: normal tools are fine.
- Preserve the `bash lib/<x>/conformance.sh` entry point (the shim keeps it working) so
no other loop is disrupted.
## The candidate worklist
Remaining hand-rolled `conformance.sh` (from radar A1): **common-lisp, erlang, feed,
forth, go, js, ocaml, smalltalk, tcl**. Already migrated (do not touch): acl, apl,
datalog, haskell, mod, prolog. Already excluded (different harness): lua.
Work them roughly simplest-first. Track status in the checklist at the bottom.
## What "fits the driver" means — classify FIRST
The shared driver works for subsystems whose tests are **SX test-suites loaded over the
epoch protocol** and run by an expression that emits a counter/dict scoreboard. It does
NOT fit subsystems that run **foreign source programs** through a separate runner
(e.g. lua walks `*.lua` via Python; smalltalk runs `*.st` via `test.sh`).
Per candidate, before migrating, decide:
- **Migratable** — its `conformance.sh` epoch-loads SX preloads and evals SX test suites
→ proceed to migrate.
- **Excluded** — it shells out to a foreign program runner / scrapes a `test.sh`
DO NOT migrate. Record the exclusion (one line in the checklist + a `git`-free note in
this briefing's Progress log) with the reason, and move on. Excluding is a valid,
honest result — a forced migration that loses coverage is worse than none.
## Per-iteration procedure
1. **Pick** the next `[ ]` candidate in the checklist.
2. **Read** its `lib/<x>/conformance.sh` in full. Read the two recipe templates —
`lib/haskell/conformance.conf` (MODE=counters) and `lib/prolog/conformance.conf`
(MODE=dict) — and skim `lib/guest/conformance.sh` + `lib/guest/conformance.sx`.
3. **Classify** (above). If Excluded → record reason, tick as excluded, stop.
4. **Baseline:** `timeout 600 bash lib/<x>/conformance.sh`, then read
`lib/<x>/scoreboard.json` and record the pass/total. This is the parity target.
5. **Author `lib/<x>/conformance.conf`:**
- `LANG_NAME=<x>`
- `MODE=dict` or `MODE=counters` (match how the old script counted)
- `PRELOADS=( … )` — the lib files in load order, lifted from the old script
- `SUITES=( "name:lib/<x>/tests/<file>:(<run-expr>)" … )` — one per suite, with the
exact run expression the old script used
- If counters mode needs counter definitions, add a small `test-harness.sx` preload
(author it with `sx_write_file`).
6. **Replace `lib/<x>/conformance.sh`** with the 3-line shim:
```bash
#!/usr/bin/env bash
# Thin wrapper — see lib/guest/conformance.sh and lib/<x>/conformance.conf.
exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
```
7. **Verify parity:** `timeout 600 bash lib/<x>/conformance.sh` again. Read
`scoreboard.json`. The pass/total MUST equal the baseline (a *higher* count is only
acceptable if you can explain it — e.g. the old extractor under-counted, as happened
with apl's `pipeline`; document it in the commit). Any mismatch/error → **revert**
(step: rails) and record the blocker.
8. **Commit** on `loops/conformance`:
`conformance: migrate <x> onto shared driver (<mode>, <pass>/<total> parity)`
then `git push origin loops/conformance`.
9. **Update** this file: tick the checklist box and add one dated line to the Progress
log (newest first). Then stop.
If a candidate is genuinely blocked (driver lacks a needed mode/feature), record it under
Blocked with specifics and move to the next candidate next iteration.
## Checklist
- [x] common-lisp — migrated 487/487 (counters; driver extended for per-suite counters+preloads)
- [x] erlang — migrated 761/761 (dict; pass/count → :failed = count-pass)
- [x] feed — migrated 189/189 (counters; test-harness.sx preload for counters+helper)
- [~] forth — excluded: foreign Forth corpus (Hayes core.fr) via awk+python preprocessing
- [x] go — migrated 609/609 (dict; pass/count → :failed = count-pass, like erlang)
- [~] js — excluded: foreign test262 .js fixtures vs .expected files (python escape, substring match)
- [~] ocaml — excluded: scrapes lib/ocaml/test.sh (per-assertion epoch runner) + foreign .ml baseline
- [~] smalltalk — excluded: scrapes lib/smalltalk/test.sh + walks foreign *.st corpus (per briefing)
- [~] tcl — excluded: foreign *.tcl programs vs `# expected:` annotations (python escape, bash compare)
(Mark `[x] <x> — migrated N/N` or `[~] <x> — excluded: <reason>` or
`[!] <x> — blocked: <reason>`.)
## Progress log (newest first)
- 2026-06-07 — tcl: EXCLUDED (foreign-runner, like lua/js/forth) — and WORKLIST COMPLETE.
conformance.sh walks foreign lib/tcl/tests/programs/*.tcl files, reads each first line's
`# expected: VALUE` annotation, uses python3 to escape the Tcl source into an SX helper,
evaluates via (tcl-eval-string …), and string-compares got vs expected in bash. No SX
test suites, no SX counter/dict scoreboard — the driver can't drive a
foreign-program-vs-expected-annotation harness. Left conformance.sh untouched. Not migrated.
>>> A1 worklist now fully classified: 4 migrated (common-lisp, erlang, feed, go),
5 excluded as foreign runners (forth, js, ocaml, smalltalk, tcl). Loop done.
- 2026-06-07 — smalltalk: EXCLUDED (the briefing's own classification example —
"smalltalk runs *.st via test.sh"). conformance.sh catalogs foreign
lib/smalltalk/tests/programs/*.st programs, runs `bash lib/smalltalk/test.sh -v`, and
scrapes its output (final "OK 403/403" summary + per-file pass counts via awk). It loads
no SX test suites directly and emits no SX counter/dict scoreboard — the bash layer
derives all numbers by text-scraping test.sh. Same "scrapes a test.sh" exclusion as
ocaml/lua. Left conformance.sh untouched. Not migrated.
- 2026-06-07 — ocaml: EXCLUDED (scrapes a test.sh — the briefing's named exclusion
criterion). conformance.sh runs `bash lib/ocaml/test.sh -v`, scrapes its human-readable
ok/FAIL lines, and re-classifies each test into suites via bash description-matching
heuristics; it also scrapes `lib/ocaml/baseline/run.sh` (foreign .ml programs). The
underlying test.sh is a per-assertion epoch runner — hundreds of individual
(ocaml-test-...) evals, one epoch each, with NO suite-level counter variables or dict
runners — so there's nothing the driver's counter/dict-scoreboard model can point at
without a full rewrite of the test harness. test.sh's own header notes it "Mirrors
lib/lua/test.sh" (the canonical excluded case). Left conformance.sh untouched. Not migrated.
- 2026-06-07 — js: EXCLUDED (foreign-runner, like lua/forth/smalltalk). conformance.sh
walks lib/js/test262-slice/**/*.js (foreign test262 fixtures), reads each .js + its
sibling .expected file, escapes the JS source with python3, evaluates via (js-eval),
and compares output to .expected by substring match — counting pass/fail in bash against
a ≥50% target. It loads no SX test suites and emits no SX counter/dict scoreboard (no
scoreboard.json at all). The shared driver only epoch-loads SX preloads + evals SX test
suites; it can't drive a foreign-fixture-vs-expected comparison harness. Left
conformance.sh untouched. Not migrated.
- 2026-06-07 — go: migrated to `MODE=dict`, 609/609 exact parity (lex 129, parse 179,
types 102, eval 106, runtime 40, stdlib 41, e2e 12). Same shape as erlang — one-session
load, per-suite pass + *count* (total) counters — so each suite's dict-literal runner
computes `:failed (- count pass)`. No driver change; conformance.conf + shim only.
Kept historical scoreboard schema (language/total_pass/total/suites[name,pass,total,status]).
- 2026-06-07 — forth: EXCLUDED (foreign-runner, like lua/smalltalk). Its conformance.sh
reads a foreign Forth corpus (lib/forth/ans-tests/core.fr, the gerryjackson Hayes Core
suite), preprocesses it with awk (strip `\` / `( )` comments + TESTING lines), splits it
into `}T` chunks via an external python3 script that generates a chunks.sx of raw source
strings, then runs them through the interpreter via (hayes-run-all) → {:pass :fail :error
:total}. The shared driver only epoch-loads SX preloads + evals SX test suites; it can't
reproduce the awk+python preprocessing of a foreign .fr corpus. No SX `tests/*.sx` suites
exist to point the driver at. Left conformance.sh untouched. Not migrated.
- 2026-06-07 — feed: migrated to `MODE=counters`, 189/189 exact parity (basic 30,
fanout 29, rank 24, integration 22, content 15, notify 8, home 6, dedupe 9, trending 11,
mute 9, page 14, thread 12). Canonical counters shape: fresh session per suite, shared
preloads, single feed-test-pass/feed-test-fail pair. Lifted the old script's inline
epoch-2 counter+helper defs into lib/feed/test-harness.sx (preloaded last). No driver
change — only conformance.conf + test-harness.sx + shim. Kept historical scoreboard
schema (suites{name:{pass,fail}}, total_pass/total_fail/total).
- 2026-06-07 — erlang: migrated to `MODE=dict`, 761/761 exact parity (tokenize 62,
parse 52, eval 408, runtime 93, ring 4, ping-pong 4, bank 8, echo 7, fib 8, ffi 37,
vm 78). Erlang exposes pass + *count* (total) counters, not pass/fail, so each suite's
dict-literal runner computes `:failed (- count pass)`. Loads in one session (matches
dict mode), so no driver change needed — only conformance.conf + shim. Kept historical
scoreboard schema (language/total_pass/total/suites[name,pass,total,status]).
- 2026-06-07 — common-lisp: UNBLOCKED + migrated. Extended the shared driver's
`MODE=counters` (lib/guest/conformance.sh) with a backward-compatible SUITES format
`name:file[:pass-var:fail-var[:extra-preload ...]]` — optional per-suite counter
symbols and per-suite preload chains. Authored lib/common-lisp/conformance.conf (12
suites, 8 distinct counter pairs, per-suite preloads, base PRELOADS=stdlib+prefix;
kept historical scoreboard schema) and replaced conformance.sh with the shim.
Result 487/487 (0 fail) — HIGHER than the 305/0 baseline, explained: the old script's
per-suite `timeout 30` was too tight for the slow `eval` suite (~1525s under
contention), silently recording it as 0; the driver's 180s budget recovers its true
182. geometry/mop-trace remain 0/0 (pre-existing `refl-class-chain-depth-with` load
error; counter vars defined as 0 → clean gc-result, no fail-fallback). Regression:
haskell backward-compat path verified (fib/sieve/quicksort 2/2/5, matches committed).
- 2026-06-07 — common-lisp: classified migratable-in-kind (SX suites over epoch) but
BLOCKED on driver feature gaps. Baseline `bash lib/common-lisp/conformance.sh` =
305 passed / 0 failed across 12 suites (3 — evaluator/geometry/mop-trace — already
emit 0/0, a pre-existing extraction quirk). Not a foreign runner, so not Excluded.
Did NOT migrate (parity unachievable under current modes); left conformance.sh
untouched. See Blocked. Driver left unchanged (out of strict per-iteration scope).
## Blocked
- (none)
## Resolved blockers
- **common-lisp** (resolved 2026-06-07) — needed per-suite counter names + per-suite
preload chains, unsupported by the original `MODE=counters` (single global counter +
fixed PRELOADS). Resolved by extending the shared driver: `MODE=counters` now accepts
`name:file[:pass-var:fail-var[:extra-preload ...]]` (backward-compatible). **This same
extension is available to later candidates** — erlang/forth/etc. with per-suite
counter names or preload chains can now migrate via the extended format instead of
blocking.