GUEST: step 1 — lib/guest/conformance.{sx,sh} config-driven driver
Extracted the duplicated conformance plumbing into a single driver:
- lib/guest/conformance.sx — two helper fns that emit (gc-result NAME P F T)
lines for the bash side to grep: gc-dict-result for runners returning
a {:passed :failed :total} dict, and gc-counters-result for guests that
bump a global pass/fail counter from a test file load.
- lib/guest/conformance.sh — config-driven bash driver. Sources a per-lang
conf, locates sx_server, runs sx_server in either single-session "dict"
mode (one preload + many suite evals) or per-suite "counters" mode
(fresh sx_server per suite, with shared preloads). Aggregates and writes
scoreboard.{json,md} via per-lang emit_scoreboard_* functions.
- Ported lib/prolog/conformance.sh and lib/haskell/conformance.sh down to
one-line wrappers that exec the shared driver against their .conf file.
Verification:
- Prolog: 590/590 — diff vs baseline is timestamp-only.
- Haskell: 156/156 — significantly higher than the 0/18 in baseline. The
old conformance.sh was buggy (its `(ok-len 3 ...)` grep never matched,
defaulting every program to 0 pass / 1 fail). Updated baseline to the
true count; no actual test regressed. Plan baseline cell updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
40
lib/guest/conformance.sx
Normal file
40
lib/guest/conformance.sx
Normal file
@@ -0,0 +1,40 @@
|
||||
;; lib/guest/conformance.sx — shared helpers for the guest conformance driver.
|
||||
;;
|
||||
;; The bash driver lib/guest/conformance.sh loads this file and then for each
|
||||
;; suite emits an (eval "...") form whose result is a tagged list:
|
||||
;;
|
||||
;; (gc-result NAME PASSED FAILED TOTAL)
|
||||
;;
|
||||
;; The driver greps these from sx_server's output and aggregates them.
|
||||
;;
|
||||
;; Two suite shapes are supported:
|
||||
;;
|
||||
;; :dict — runner expression returns a dict with :passed/:failed/:total.
|
||||
;; (gc-dict-result "parse" (pl-parse-tests-run!))
|
||||
;;
|
||||
;; :counters — runner has no return value, mutates pass/fail global counters.
|
||||
;; (gc-counters-result NAME P0 F0 PASS FAIL)
|
||||
;; where P0/F0 are the counters captured BEFORE the suite ran
|
||||
;; and PASS/FAIL are the counters AFTER.
|
||||
|
||||
(define
|
||||
gc-dict-result
|
||||
(fn
|
||||
(name r)
|
||||
(list
|
||||
(quote gc-result)
|
||||
name
|
||||
(get r :passed)
|
||||
(get r :failed)
|
||||
(get r :total))))
|
||||
|
||||
(define
|
||||
gc-counters-result
|
||||
(fn
|
||||
(name p0 f0 p1 f1)
|
||||
(list
|
||||
(quote gc-result)
|
||||
name
|
||||
(- p1 p0)
|
||||
(- f1 f0)
|
||||
(- (+ p1 f1) (+ p0 f0)))))
|
||||
Reference in New Issue
Block a user