giles
58dcff2639
Extracted the duplicated conformance plumbing into a single driver:
- lib/guest/conformance.sx — two helper fns that emit (gc-result NAME P F T)
lines for the bash side to grep: gc-dict-result for runners returning
a {:passed :failed :total} dict, and gc-counters-result for guests that
bump a global pass/fail counter from a test file load.
- lib/guest/conformance.sh — config-driven bash driver. Sources a per-lang
conf, locates sx_server, runs sx_server in either single-session "dict"
mode (one preload + many suite evals) or per-suite "counters" mode
(fresh sx_server per suite, with shared preloads). Aggregates and writes
scoreboard.{json,md} via per-lang emit_scoreboard_* functions.
- Ported lib/prolog/conformance.sh and lib/haskell/conformance.sh down to
one-line wrappers that exec the shared driver against their .conf file.
Verification:
- Prolog: 590/590 — diff vs baseline is timestamp-only.
- Haskell: 156/156 — significantly higher than the 0/18 in baseline. The
old conformance.sh was buggy (its `(ok-len 3 ...)` grep never matched,
defaulting every program to 0 pass / 1 fail). Updated baseline to the
true count; no actual test regressed. Plan baseline cell updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
40 lines
1.1 KiB
Markdown
40 lines
1.1 KiB
Markdown
# Prolog scoreboard
|
|
|
|
**590 / 590 passing** (0 failure(s)).
|
|
Generated 2026-05-06T22:23:38+00:00.
|
|
|
|
| Suite | Passed | Total | Status |
|
|
|-------|--------|-------|--------|
|
|
| parse | 25 | 25 | ok |
|
|
| unify | 47 | 47 | ok |
|
|
| clausedb | 14 | 14 | ok |
|
|
| solve | 62 | 62 | ok |
|
|
| operators | 19 | 19 | ok |
|
|
| dynamic | 11 | 11 | ok |
|
|
| findall | 11 | 11 | ok |
|
|
| term_inspect | 14 | 14 | ok |
|
|
| append | 6 | 6 | ok |
|
|
| reverse | 6 | 6 | ok |
|
|
| member | 7 | 7 | ok |
|
|
| nqueens | 6 | 6 | ok |
|
|
| family | 10 | 10 | ok |
|
|
| atoms | 34 | 34 | ok |
|
|
| query_api | 16 | 16 | ok |
|
|
| iso_predicates | 29 | 29 | ok |
|
|
| meta_predicates | 25 | 25 | ok |
|
|
| list_predicates | 33 | 33 | ok |
|
|
| meta_call | 15 | 15 | ok |
|
|
| set_predicates | 15 | 15 | ok |
|
|
| char_predicates | 27 | 27 | ok |
|
|
| io_predicates | 24 | 24 | ok |
|
|
| assert_rules | 15 | 15 | ok |
|
|
| string_agg | 25 | 25 | ok |
|
|
| advanced | 21 | 21 | ok |
|
|
| compiler | 17 | 17 | ok |
|
|
| cross_validate | 17 | 17 | ok |
|
|
| integration | 20 | 20 | ok |
|
|
| hs_bridge | 19 | 19 | ok |
|
|
|
|
Run `bash lib/prolog/conformance.sh` to refresh. Override the binary
|
|
with `SX_SERVER=path/to/sx_server.exe bash …`.
|