From 0309e3b5d5f7929a7e2e5ea007ad29238458a90c Mon Sep 17 00:00:00 2001 From: giles Date: Sun, 7 Jun 2026 12:20:59 +0000 Subject: [PATCH] conformance: exclude ocaml (scrapes lib/ocaml/test.sh + foreign .ml baseline) ocaml conformance.sh runs 'bash lib/ocaml/test.sh -v', scrapes its human-readable ok/FAIL lines, and re-classifies each test into suites via bash description-matching heuristics; it also scrapes lib/ocaml/baseline/run.sh (foreign .ml programs). The underlying test.sh is a per-assertion epoch runner (hundreds of individual (ocaml-test-...) evals, one epoch each) with no suite-level counter variables or dict runners, so the driver's counter/dict-scoreboard model has nothing to point at without rewriting the test harness. 'Scrapes a test.sh' is the briefing's named exclusion criterion (test.sh even notes it mirrors lib/lua/test.sh, the canonical excluded case). Left conformance.sh untouched; recorded the exclusion. Co-Authored-By: Claude Opus 4.8 (1M context) --- plans/agent-briefings/conformance-loop.md | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/plans/agent-briefings/conformance-loop.md b/plans/agent-briefings/conformance-loop.md index 0c8b7c0f..a745742e 100644 --- a/plans/agent-briefings/conformance-loop.md +++ b/plans/agent-briefings/conformance-loop.md @@ -91,7 +91,7 @@ Blocked with specifics and move to the next candidate next iteration. - [~] forth — excluded: foreign Forth corpus (Hayes core.fr) via awk+python preprocessing - [x] go — migrated 609/609 (dict; pass/count → :failed = count-pass, like erlang) - [~] js — excluded: foreign test262 .js fixtures vs .expected files (python escape, substring match) -- [ ] ocaml +- [~] ocaml — excluded: scrapes lib/ocaml/test.sh (per-assertion epoch runner) + foreign .ml baseline - [ ] smalltalk - [ ] tcl @@ -100,6 +100,15 @@ Blocked with specifics and move to the next candidate next iteration. ## Progress log (newest first) +- 2026-06-07 — ocaml: EXCLUDED (scrapes a test.sh — the briefing's named exclusion + criterion). conformance.sh runs `bash lib/ocaml/test.sh -v`, scrapes its human-readable + ok/FAIL lines, and re-classifies each test into suites via bash description-matching + heuristics; it also scrapes `lib/ocaml/baseline/run.sh` (foreign .ml programs). The + underlying test.sh is a per-assertion epoch runner — hundreds of individual + (ocaml-test-...) evals, one epoch each, with NO suite-level counter variables or dict + runners — so there's nothing the driver's counter/dict-scoreboard model can point at + without a full rewrite of the test harness. test.sh's own header notes it "Mirrors + lib/lua/test.sh" (the canonical excluded case). Left conformance.sh untouched. Not migrated. - 2026-06-07 — js: EXCLUDED (foreign-runner, like lua/forth/smalltalk). conformance.sh walks lib/js/test262-slice/**/*.js (foreign test262 fixtures), reads each .js + its sibling .expected file, escapes the JS source with python3, evaluates via (js-eval),