conformance: migrate common-lisp onto shared driver (counters, 487/487)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 14m2s

Extend the shared driver's MODE=counters with a backward-compatible SUITES
format: name:file[:pass-var:fail-var[:extra-preload ...]]. Optional per-suite
counter symbols (override the global COUNTERS_PASS/COUNTERS_FAIL) and per-suite
preload chains (loaded after the global PRELOADS). Plain name:file entries are
unchanged — verified against haskell (fib/sieve/quicksort 2/2/5, matches
committed scoreboard).

common-lisp has 8 distinct per-suite counter pairs and a different preload
chain per suite, so it could not fit the single-counter/fixed-preload model;
the extended format expresses it directly. conformance.conf keeps the historical
scoreboard schema; conformance.sh becomes the 3-line shim.

Result 487/487 (0 fail) vs the old 305/0 baseline — higher and explained: the
old per-suite 'timeout 30' was too tight for the slow eval suite (~15-25s under
contention), silently recording it as 0; the driver's 180s budget recovers its
true 182. geometry/mop-trace stay 0/0 (pre-existing refl-class-chain-depth-with
load error; counter vars defined as 0 -> clean gc-result, no fail-fallback).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-07 09:55:44 +00:00
parent bfdd0fe65a
commit 2e7a08309c
6 changed files with 152 additions and 214 deletions

View File

@@ -85,7 +85,7 @@ Blocked with specifics and move to the next candidate next iteration.
## Checklist
- [!] common-lisp — blocked: needs per-suite counter names + per-suite preloads (see Blocked)
- [x] common-lisp — migrated 487/487 (counters; driver extended for per-suite counters+preloads)
- [ ] erlang
- [ ] feed
- [ ] forth
@@ -100,6 +100,18 @@ Blocked with specifics and move to the next candidate next iteration.
## Progress log (newest first)
- 2026-06-07 — common-lisp: UNBLOCKED + migrated. Extended the shared driver's
`MODE=counters` (lib/guest/conformance.sh) with a backward-compatible SUITES format
`name:file[:pass-var:fail-var[:extra-preload ...]]` — optional per-suite counter
symbols and per-suite preload chains. Authored lib/common-lisp/conformance.conf (12
suites, 8 distinct counter pairs, per-suite preloads, base PRELOADS=stdlib+prefix;
kept historical scoreboard schema) and replaced conformance.sh with the shim.
Result 487/487 (0 fail) — HIGHER than the 305/0 baseline, explained: the old script's
per-suite `timeout 30` was too tight for the slow `eval` suite (~1525s under
contention), silently recording it as 0; the driver's 180s budget recovers its true
182. geometry/mop-trace remain 0/0 (pre-existing `refl-class-chain-depth-with` load
error; counter vars defined as 0 → clean gc-result, no fail-fallback). Regression:
haskell backward-compat path verified (fib/sieve/quicksort 2/2/5, matches committed).
- 2026-06-07 — common-lisp: classified migratable-in-kind (SX suites over epoch) but
BLOCKED on driver feature gaps. Baseline `bash lib/common-lisp/conformance.sh` =
305 passed / 0 failed across 12 suites (3 — evaluator/geometry/mop-trace — already
@@ -109,21 +121,14 @@ Blocked with specifics and move to the next candidate next iteration.
## Blocked
- **common-lisp** — the shared driver's two modes can't reproduce its 305/0 breakdown:
1. **Per-suite counter variable names.** Old script reads 8 distinct pairs across
its 12 suites: `cl-test-pass`/`cl-test-fail` (read, lambda, eval), `passed`/`failed`
(conditions, clos), `demo-passed`/`demo-failed`, `parse-passed`/`parse-failed`,
`debugger-passed`/`debugger-failed`, `geo-passed`/`geo-failed`,
`mop-passed`/`mop-failed`, `macro-passed`/`macro-failed`, `stdlib-passed`/`stdlib-failed`.
`MODE=counters` supports only one global `COUNTERS_PASS`/`COUNTERS_FAIL`.
2. **Per-suite preload chains.** Each suite loads a different file set
(e.g. read: `reader.sx`; clos: `runtime.sx clos.sx`; macros: `reader parser eval loop`).
`MODE=counters` loads one fixed `PRELOADS` set before every suite.
`MODE=dict` also fails: these tests run at *load* time mutating globals (no
`-tests-run!` runner fns), and dict mode loads all suites into one session — the
shared `passed`/`failed` counters used by both conditions and clos would collide.
**Unblock path (driver enhancement, deferred — out of this loop's per-iteration scope):**
extend `MODE=counters` with optional per-suite counter names and per-suite preloads
in the `SUITES` entry format (backward-compatible with the `name:file` shape haskell
uses). Same gap likely affects other remaining candidates; worth a one-time driver
change before resuming migrations.
- (none)
## Resolved blockers
- **common-lisp** (resolved 2026-06-07) — needed per-suite counter names + per-suite
preload chains, unsupported by the original `MODE=counters` (single global counter +
fixed PRELOADS). Resolved by extending the shared driver: `MODE=counters` now accepts
`name:file[:pass-var:fail-var[:extra-preload ...]]` (backward-compatible). **This same
extension is available to later candidates** — erlang/forth/etc. with per-suite
counter names or preload chains can now migrate via the extended format instead of
blocking.