conformance: A1 migration loop briefing (classify-then-migrate, parity-gated)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

plans/agent-briefings/conformance-loop.md

@@ -0,0 +1,107 @@
+# A1 conformance-driver migration loop
+
+Role: migrate every remaining subsystem that hand-rolls its own `conformance.sh`
+onto the **shared conformance driver** (`lib/guest/conformance.sh` + `lib/guest/conformance.sx`),
+one subsystem per iteration, **verifying test-count parity before every commit**.
+This executes item **A1** from the radar backlog (`plans/abstractions.md`, read-only
+context). You are an implementer, not a scout.
+
+You are on branch `loops/conformance`, worktree `/root/rose-ash-loops/conformance`.
+
+## Hard safety rails (read every time)
+
+- **NEVER push to `main` or `architecture`.** Push only to `origin/loops/conformance`.
+- **NEVER `pkill`/`kill` `sx_server` or any shared process** — sibling loops share the
+  binary. Bound every test run with `timeout` (e.g. `timeout 600 bash …`). If a run
+  hangs, let the timeout end it; never kill globally.
+- **One subsystem per iteration, then stop.** No batching.
+- **Never commit a regression.** If post-migration test counts don't match the baseline
+  (or an error appears), REVERT (`git checkout -- lib/<x>/conformance.sh` and
+  `rm -f lib/<x>/conformance.conf`) and record the blocker — do not commit.
+- `.sx` files: use the `sx-tree` MCP tools, never Read/Write/Edit. `.sh`/`.conf`/`.md`
+  files: normal tools are fine.
+- Preserve the `bash lib/<x>/conformance.sh` entry point (the shim keeps it working) so
+  no other loop is disrupted.
+
+## The candidate worklist
+
+Remaining hand-rolled `conformance.sh` (from radar A1): **common-lisp, erlang, feed,
+forth, go, js, ocaml, smalltalk, tcl**. Already migrated (do not touch): acl, apl,
+datalog, haskell, mod, prolog. Already excluded (different harness): lua.
+
+Work them roughly simplest-first. Track status in the checklist at the bottom.
+
+## What "fits the driver" means — classify FIRST
+
+The shared driver works for subsystems whose tests are **SX test-suites loaded over the
+epoch protocol** and run by an expression that emits a counter/dict scoreboard. It does
+NOT fit subsystems that run **foreign source programs** through a separate runner
+(e.g. lua walks `*.lua` via Python; smalltalk runs `*.st` via `test.sh`).
+
+Per candidate, before migrating, decide:
+- **Migratable** — its `conformance.sh` epoch-loads SX preloads and evals SX test suites
+  → proceed to migrate.
+- **Excluded** — it shells out to a foreign program runner / scrapes a `test.sh` →
+  DO NOT migrate. Record the exclusion (one line in the checklist + a `git`-free note in
+  this briefing's Progress log) with the reason, and move on. Excluding is a valid,
+  honest result — a forced migration that loses coverage is worse than none.
+
+## Per-iteration procedure
+
+1. **Pick** the next `[ ]` candidate in the checklist.
+2. **Read** its `lib/<x>/conformance.sh` in full. Read the two recipe templates —
+   `lib/haskell/conformance.conf` (MODE=counters) and `lib/prolog/conformance.conf`
+   (MODE=dict) — and skim `lib/guest/conformance.sh` + `lib/guest/conformance.sx`.
+3. **Classify** (above). If Excluded → record reason, tick as excluded, stop.
+4. **Baseline:** `timeout 600 bash lib/<x>/conformance.sh`, then read
+   `lib/<x>/scoreboard.json` and record the pass/total. This is the parity target.
+5. **Author `lib/<x>/conformance.conf`:**
+   - `LANG_NAME=<x>`
+   - `MODE=dict` or `MODE=counters` (match how the old script counted)
+   - `PRELOADS=( … )` — the lib files in load order, lifted from the old script
+   - `SUITES=( "name:lib/<x>/tests/<file>:(<run-expr>)" … )` — one per suite, with the
+     exact run expression the old script used
+   - If counters mode needs counter definitions, add a small `test-harness.sx` preload
+     (author it with `sx_write_file`).
+6. **Replace `lib/<x>/conformance.sh`** with the 3-line shim:
+   ```bash
+   #!/usr/bin/env bash
+   # Thin wrapper — see lib/guest/conformance.sh and lib/<x>/conformance.conf.
+   exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
+   ```
+7. **Verify parity:** `timeout 600 bash lib/<x>/conformance.sh` again. Read
+   `scoreboard.json`. The pass/total MUST equal the baseline (a *higher* count is only
+   acceptable if you can explain it — e.g. the old extractor under-counted, as happened
+   with apl's `pipeline`; document it in the commit). Any mismatch/error → **revert**
+   (step: rails) and record the blocker.
+8. **Commit** on `loops/conformance`:
+   `conformance: migrate <x> onto shared driver (<mode>, <pass>/<total> parity)`
+   then `git push origin loops/conformance`.
+9. **Update** this file: tick the checklist box and add one dated line to the Progress
+   log (newest first). Then stop.
+
+If a candidate is genuinely blocked (driver lacks a needed mode/feature), record it under
+Blocked with specifics and move to the next candidate next iteration.
+
+## Checklist
+
+- [ ] common-lisp
+- [ ] erlang
+- [ ] feed
+- [ ] forth
+- [ ] go
+- [ ] js
+- [ ] ocaml
+- [ ] smalltalk
+- [ ] tcl
+
+(Mark `[x] <x> — migrated N/N` or `[~] <x> — excluded: <reason>` or
+`[!] <x> — blocked: <reason>`.)
+
+## Progress log (newest first)
+
+- (none yet)
+
+## Blocked
+
+- (none yet)