rose-ash/plans/abstractions.md

Abstraction Radar — backlog

Maintained by the read-only radar loop (see plans/agent-briefings/radar-loop.md). Detection only — implementation is a separate, coordinated step owned by the relevant subsystem loop, never by radar.

AHA gate to reach Proposed: ≥3 real consumers · all past Phase 2 & API-stable · structurally identical (file:line evidence) · a natural home (usually NOT lib/guest). Anything short → Watching (what's missing) or Rejected (why).

---

Last scan

• Date: 2026-06-06 (radar loop, pass 2)
• Subsystem set discovered: loop worktrees acl, erlang, fed-prims, fed-sx-m1, feed, flow, go, kernel, mod, ocaml, persist, radar, ruby, search, sx-vm-extensions; main-repo lib/* incl. merged feed + substrates (apl, common-lisp, datalog, erlang, forth, go, haskell, hyperscript, js, lua, minikanren, ocaml, prolog, scheme, smalltalk, tcl) + lib/guest. Actively looping (tmux): acl, fed-sx-m1, feed, flow, mod, persist, search (+ radar).
• New since pass 1: worktrees kernel (empty/unset — not yet a repo) and ocaml (lib/ocaml/baseline only). Both early-stage, pre–Phase 2 → out of proposal scope.
• Re-enumerate every pass; new loops (e.g. a future commerce/identity) auto-join.

---

Proposed (cleared the gate)

A1 · Adopt the shared conformance driver across subsystems

• Pattern: every subsystem hand-rolls a near-identical conformance.sh (epoch-load → eval → scoreboard emit) and an inline <x>-test name got expected pass/fail counter.
• Consumers (≥3, overwhelming): 15 lib/*/conformance.sh — apl, feed, datalog, flow, mod, lua, erlang, forth, go, common-lisp, haskell, js, ocaml, prolog, smalltalk, tcl.
• Home: lib/guest — the one legitimate exception (the shared driver lib/guest/conformance.sh + lib/guest/conformance.sx already exist; modes dict and counters).
• Status: IN PROGRESS — 4 adopters. prolog (dict), haskell (counters), apl (dict), and datalog (dict), newly migrated this pass — all now 3-line exec shims into lib/guest/conformance.sh with a conformance.conf (lib/datalog/conformance.conf: 11 PRELOADS + 11 SUITES). The apl migration earlier surfaced a latent bug: the old awk extractor under-counted pipeline (40 vs the real 152 assertions); true apl total is 562, not 450 — evidence that adopting the driver also improves correctness.
• Not a target (different harness shape): lua/conformance.sh is a Python runner (lib/lua/conformance.py) that walks real *.lua source files via lua-eval-ast and classifies pass/fail/timeout — it does not run SX deftest suites with a counter/dict scoreboard, so the shared driver does not fit. Excluded, not pending.
• Remaining hand-rolled candidates (~120–220 lines each): common-lisp, erlang, feed, forth, go, js, ocaml, smalltalk, tcl — each its OWN loop's migration when quiescent.
• Action: each remaining subsystem's OWN loop migrates when quiescent — add a conformance.conf (+ a test-harness.sx preload defining its counters) and replace conformance.sh with the 1-line exec shim (exec bash …/guest/conformance.sh …/conformance.conf "$@"). Recipe template: lib/haskell/conformance.conf (counters) or lib/prolog/conformance.conf (dict). Keep the bash lib/X/conformance.sh entry point so no loop is disrupted.
• Priority: HIGH (15 consumers, low risk, interface-preserving, additive).

---

Watching (real but not yet through the gate)

W1 · Federation scaffold (merge / ingest / backfill / trust-gate)

• Consumer count NOW met (4): feed/lib/feed/fed.sx (60L/7 defs), search/lib/search/fed.sx (16L/1), mod/lib/mod/fed.sx (145L/19), acl/lib/acl/federation.sx (61L/5).
• BUT structural identity unverified — and the size spread (16→145 lines) is a red flag. search/fed is just relabel+merge of an index; mod/fed is 19 defs; these may be superficially similar (all "federation") not structurally identical.
• Next pass (highest-value thread): read all 4 side by side, extract the common shape (if any) with file:line evidence — trust-gate? injected transport? merge vs ingest split? — before this can move to Proposed. Do NOT propose on the name alone.
• Home when ripe: fed-sx (NOT lib/guest).

W2 · Per-viewer visibility / permission filter

• 2 shipped consumers, same shape — filter <injected-permit> <ranked/candidate stream>:
  • feed/lib/feed/acl.sx:27 feed/visible = (feed/filter stream (fn (a) (permit? viewer a))), capstone at :34 (stream → ACL → rank → top-N). permit? injected, sig (viewer activity)→bool.
  • search/lib/search/fed.sx:16 aclFilter permit docs = filter permit docs; topNTfIdfAcl n permit ts idx = take n (aclFilter permit (rankTfIdf ts idx)). permit injected, sig DocId→Bool (viewer baked in by caller).
• NOT a consumer: mod/lib/mod/policy.sx is moderation policy (reviewer actions), no per-viewer read filter. So mod won't be the 3rd.
• Missing: (a) only 2 consumers, need ≥3; (b) the two interfaces diverge — feed passes (viewer, item), search bakes the viewer in — so any shared form must pick a convention; (c) both already inject the predicate, and the filter body is literally one line (filter permit xs). Leaning toward: the predicate's home is acl-on-sx (permit?), and the one-line filter is too thin to extract.
• Home when ripe: delegate permit? to acl-on-sx; do NOT extract the filter. Re-check if a 3rd genuine per-viewer read filter ships (e.g. events/commerce).

W3 · Collection helpers (group-by, dedupe-by-key, stable top-N, distinct-order)

• feed built all of these on APL primitives. search/commerce/events will want group-by / top-N.
• Missing: ≥3 stable consumers; and these likely belong in the substrate (APL/Haskell already expose grade/sort/unique) rather than a shared lib. Watch.

W4 · In-memory store fakes → persist-on-sx

• Not an abstraction to extract — a migration target. Every subsystem fakes its store with a mutable list (feed/-log, flow store, mod audit, …).
• Owner: persist-on-sx (in progress). Tracked there, listed here for visibility.

---

Rejected (considered, declined — do not re-propose)

• "Continuous auto-implementing abstractor loop." Rejected at design time: an agent writing across lib/<x>/** breaks the worktree isolation that makes the fleet safe, and is rewarded for manufacturing premature/wrong abstractions. The radar is read-only by design. (This file is the alternative.)
• Dumping app-domain plumbing into lib/guest. Rejected: lib/guest is for language-implementation plumbing. App patterns route to acl/fed-sx/persist/ substrate/host instead (see the routing rule in the briefing).