rose-ash/plans/abstractions.md

Abstraction Radar — backlog

Maintained by the read-only radar loop (see plans/agent-briefings/radar-loop.md). Detection only — implementation is a separate, coordinated step owned by the relevant subsystem loop, never by radar.

AHA gate to reach Proposed: ≥3 real consumers · all past Phase 2 & API-stable · structurally identical (file:line evidence) · a natural home (usually NOT lib/guest). Anything short → Watching (what's missing) or Rejected (why).

---

Last scan

• Date: 2026-06-06 (radar loop, pass 2)
• Subsystem set discovered: loop worktrees acl, erlang, fed-prims, fed-sx-m1, feed, flow, go, kernel, mod, ocaml, persist, radar, ruby, search, sx-vm-extensions; main-repo lib/* incl. merged feed + substrates (apl, common-lisp, datalog, erlang, forth, go, haskell, hyperscript, js, lua, minikanren, ocaml, prolog, scheme, smalltalk, tcl) + lib/guest. Actively looping (tmux): acl, fed-sx-m1, feed, flow, mod, persist, search (+ radar).
• New since pass 1: worktrees kernel (empty/unset — not yet a repo) and ocaml (lib/ocaml/baseline only). Both early-stage, pre–Phase 2 → out of proposal scope.
• Re-enumerate every pass; new loops (e.g. a future commerce/identity) auto-join.

---

Proposed (cleared the gate)

A1 · Adopt the shared conformance driver across subsystems

• Pattern: every subsystem hand-rolls a near-identical conformance.sh (epoch-load → eval → scoreboard emit) and an inline <x>-test name got expected pass/fail counter.
• Consumers (≥3, overwhelming): 15 lib/*/conformance.sh — apl, feed, datalog, flow, mod, lua, erlang, forth, go, common-lisp, haskell, js, ocaml, prolog, smalltalk, tcl.
• Home: lib/guest — the one legitimate exception (the shared driver lib/guest/conformance.sh + lib/guest/conformance.sx already exist; modes dict and counters).
• Status: IN PROGRESS — 4 adopters. prolog (dict), haskell (counters), apl (dict), and datalog (dict), newly migrated this pass — all now 3-line exec shims into lib/guest/conformance.sh with a conformance.conf (lib/datalog/conformance.conf: 11 PRELOADS + 11 SUITES). The apl migration earlier surfaced a latent bug: the old awk extractor under-counted pipeline (40 vs the real 152 assertions); true apl total is 562, not 450 — evidence that adopting the driver also improves correctness.
• Not a target (different harness shape): lua/conformance.sh is a Python runner (lib/lua/conformance.py) that walks real *.lua source files via lua-eval-ast and classifies pass/fail/timeout — it does not run SX deftest suites with a counter/dict scoreboard, so the shared driver does not fit. Excluded, not pending.
• Remaining hand-rolled candidates (~120–220 lines each): common-lisp, erlang, feed, forth, go, js, ocaml, smalltalk, tcl — each its OWN loop's migration when quiescent.
• Action: each remaining subsystem's OWN loop migrates when quiescent — add a conformance.conf (+ a test-harness.sx preload defining its counters) and replace conformance.sh with the 1-line exec shim (exec bash …/guest/conformance.sh …/conformance.conf "$@"). Recipe template: lib/haskell/conformance.conf (counters) or lib/prolog/conformance.conf (dict). Keep the bash lib/X/conformance.sh entry point so no loop is disrupted.
• Priority: HIGH (15 consumers, low risk, interface-preserving, additive).

---

Watching (real but not yet through the gate)

W1 · Federation scaffold (merge / ingest / backfill / trust-gate)

• FAILS the structural-identity gate (deep-dived 2026-06-06, all 4 read). Consumer count is met (4) but they are superficially similar, not structurally identical — the federated unit and merge op differ fundamentally:

  Subsystem (file)	Federated unit	Merge op	Trust gate	Injected transport
  feed (fed.sx:14,18,40)	activity streams	dedupe by (actor verb object)	none (visibility via permit? separately)	send-fn, fetch-fn
  search (fed.sx:8)	inverted indices	relabel DocId peer*1000+local + union posting lists	none	none (pure merge fn)
  mod (fed.sx:11-14,99)	moderation decisions	advisory-list vs applied-list; bind iff mod/trusted?	yes — runtime list mod/trusted? peer scope	mock outbox / fed-send!
  acl (federation.sx:43,56)	Datalog delegate facts	pull facts, gate by trust/level_covers rule, re-saturate	yes — Datalog rule at query time	transport dict

• The ONLY real commonality is the injection seam, not extractable code: every one says "the real transport is fed-sx's job; inject send-fn/fetch-fn/transport/ outbox and mock it in tests." That is an architectural convention the fleet already follows, and the trust gate (where present) is implemented two incompatible ways (runtime list vs declarative rule). No shared merge, no shared trust mechanism.

• Disposition: do NOT extract a shared "federation lib." When fed-sx ships its real transport, these 4 become its consumers (wiring send-fn/fetch-fn/transport to it) — that work belongs to each subsystem's loop + the fed-sx loop, not a cross-cutting extraction. Stop re-proposing on the shared name. Home: fed-sx.

W2 · Per-viewer visibility / permission filter

• 2 shipped consumers, same shape — filter <injected-permit> <ranked/candidate stream>:
  • feed/lib/feed/acl.sx:27 feed/visible = (feed/filter stream (fn (a) (permit? viewer a))), capstone at :34 (stream → ACL → rank → top-N). permit? injected, sig (viewer activity)→bool.
  • search/lib/search/fed.sx:16 aclFilter permit docs = filter permit docs; topNTfIdfAcl n permit ts idx = take n (aclFilter permit (rankTfIdf ts idx)). permit injected, sig DocId→Bool (viewer baked in by caller).
• NOT a consumer: mod/lib/mod/policy.sx is moderation policy (reviewer actions), no per-viewer read filter. So mod won't be the 3rd.
• Missing: (a) only 2 consumers, need ≥3; (b) the two interfaces diverge — feed passes (viewer, item), search bakes the viewer in — so any shared form must pick a convention; (c) both already inject the predicate, and the filter body is literally one line (filter permit xs). Leaning toward: the predicate's home is acl-on-sx (permit?), and the one-line filter is too thin to extract.
• Home when ripe: delegate permit? to acl-on-sx; do NOT extract the filter. Re-check if a 3rd genuine per-viewer read filter ships (e.g. events/commerce).

W3 · Collection helpers (group-by, dedupe-by-key, stable top-N, distinct-order)

• feed built all of these on APL primitives. search/commerce/events will want group-by / top-N.
• Missing: ≥3 stable consumers; and these likely belong in the substrate (APL/Haskell already expose grade/sort/unique) rather than a shared lib. Watch.

W4 · In-memory store fakes → persist-on-sx

• Not an abstraction to extract — a migration target. Every subsystem fakes its store with a mutable list (feed/-log, flow store, mod audit, …).
• Owner: persist-on-sx (in progress). Tracked there, listed here for visibility.

---

Rejected (considered, declined — do not re-propose)

• "Continuous auto-implementing abstractor loop." Rejected at design time: an agent writing across lib/<x>/** breaks the worktree isolation that makes the fleet safe, and is rewarded for manufacturing premature/wrong abstractions. The radar is read-only by design. (This file is the alternative.)
• Dumping app-domain plumbing into lib/guest. Rejected: lib/guest is for language-implementation plumbing. App patterns route to acl/fed-sx/persist/ substrate/host instead (see the routing rule in the briefing).