Compare commits

..

5 Commits

Author SHA1 Message Date
9437f99e28 acl: hardening suite (+25) — diamonds, cycles, validation, audit save/restore
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 35s
New adversarial/cross-phase coverage: diamond resource+group hierarchies
(deny wins per path), chain inheritance + leaf deny, cycle termination,
multi-peer delegation, fact validation, audit snapshot/restore round-trip.
Adds acl-validate-facts/acl-facts-valid? (schema) and acl-audit-snapshot/
restore!/copy (audit). Fixed acl-audit-restore! rebuilding the live log via
map (append! silently no-ops on map-derived lists).

Suite is prover-free: a substrate JIT bug loops the recursive proof
reconstructor on deep chains in warm processes (documented in Blockers);
acl-permit? is unaffected. 145/145.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 22:44:28 +00:00
40be9cd074 acl: Phase 4 federation (trust-gated delegation, revocation) + 31 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 10m25s
federation.sx adds peer/trust/delegate/level_covers facts and one engine
rule: delegated grants apply only when local trust covers the action,
re-checked every query (non-transitive, fail-safe). Local/inherited deny
overrides federated grants; delegation composes with group and resource
inheritance. acl-revoke!/acl-fed-assert! propagate retraction/assertion;
mock fed-sx transport for tests. Federated proofs reconstruct via the
existing explainer. Roadmap complete: 120/120.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:54:34 +00:00
15c97119e4 acl: Phase 3 explanation + audit, 35 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 40s
explain.sx reconstructs a canonical proof tree (first-rule, first-solution)
by goal-directed search over the saturated db, since Datalog keeps no
provenance; depth-capped for cyclic safety. acl-explain returns
{:allowed? :proof :reason} with the blocking eff_deny proof on denial.
audit.sx is an append-only decision log (monotonic seq, disk serializer).
api gains acl/explain, acl/audit, acl/audit-tail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:47:07 +00:00
9261d69cc5 acl: Phase 2 inheritance (groups, resource trees, roles) + 30 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 36s
eff_grant/eff_deny derived relations inherit through member_of (group +
role membership) and child_of (resource hierarchy); role_grant confers
role capabilities. Deny-overrides via stratified negation, deny
authoritative across the inheritance closure. Cyclic membership
terminates. Phase 1 suite unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:36:24 +00:00
fe47334e52 acl: Phase 1 direct grants + deny-overrides, 24 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 56s
Datalog ACL layer (schema/facts/engine/api) over lib/datalog/. Direct
grant permits unless explicit deny names same (S,A,R) — deny-overrides
via stratified negation. Conformance wrapper + scoreboard.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:32:13 +00:00
63 changed files with 2135 additions and 3942 deletions

45
lib/acl/api.sx Normal file
View File

@@ -0,0 +1,45 @@
;; lib/acl/api.sx — public ACL surface over an implicit current db.
;;
;; Callers load a fact set once, then issue decisions without threading the db
;; through every call. The current db is module state; (acl/load! facts) rebuilds
;; it. This is the boundary the rest of rose-ash imports.
(define acl-current-db nil)
;; Replace the current fact base. Rebuilds the Datalog db under the active
;; ruleset (see lib/acl/engine.sx).
(define
acl/load!
(fn
(facts)
(do (set! acl-current-db (acl-build-db facts)) acl-current-db)))
;; Ensure a db exists, building an empty one on first use.
(define
acl-ensure-db!
(fn
()
(do
(when
(= acl-current-db nil)
(set! acl-current-db (acl-build-db (list))))
acl-current-db)))
;; Public decision against the current db (pure, no logging).
(define
acl/permit?
(fn (subj act res) (acl-permit? (acl-ensure-db!) subj act res)))
;; Decision-with-proof against the current db. See lib/acl/explain.sx.
(define
acl/explain
(fn (subj act res) (acl-explain (acl-ensure-db!) subj act res)))
;; Audited decision: logs the outcome to the append-only audit log and returns
;; the boolean. See lib/acl/audit.sx.
(define
acl/audit
(fn (subj act res) (acl-audit-decide! (acl-ensure-db!) subj act res)))
;; Recent audited decisions (chronological).
(define acl/audit-tail (fn (n) (acl-audit-tail n)))

110
lib/acl/audit.sx Normal file
View File

@@ -0,0 +1,110 @@
;; lib/acl/audit.sx — append-only decision log.
;;
;; Every decision routed through acl-audit-decide! is appended to an in-memory
;; log with a monotonic sequence number (no wall-clock — deterministic and
;; testable; a host can stamp time at the serializer boundary). The log is
;; append-only: there is no mutate or delete, only append, tail, clear,
;; snapshot/restore, and serialize-for-disk.
(define acl-audit-log (list))
(define acl-audit-seq 0)
;; Copy a list into a fresh, append!-able list. `map`/`rest`-derived lists are
;; NOT extensible by append! in this runtime (it silently no-ops), so the live
;; log must always be a list built with `list` + `append!`.
(define
acl-audit-copy
(fn
(xs)
(let
((fresh (list)))
(do (for-each (fn (e) (append! fresh e)) xs) fresh))))
(define
acl-audit-clear!
(fn
()
(do (set! acl-audit-log (list)) (set! acl-audit-seq 0) nil)))
;; Append a decision record. Returns the record.
(define
acl-audit-record!
(fn
(subj act res allowed?)
(let
((entry {:allowed? allowed? :act act :subj subj :res res :seq acl-audit-seq}))
(do
(set! acl-audit-seq (+ acl-audit-seq 1))
(append! acl-audit-log entry)
entry))))
;; Decide against db, log the outcome, and return the boolean. This is the
;; audited path; acl-permit? remains the pure, side-effect-free decision.
(define
acl-audit-decide!
(fn
(db subj act res)
(let
((allowed? (acl-permit? db subj act res)))
(do (acl-audit-record! subj act res allowed?) allowed?))))
(define acl-audit-count (fn () (len acl-audit-log)))
;; Most recent n entries (in chronological order). n >= log size returns all.
(define
acl-audit-tail
(fn
(n)
(let
((total (len acl-audit-log)))
(if
(<= total n)
acl-audit-log
(acl-audit-drop acl-audit-log (- total n))))))
(define
acl-audit-drop
(fn
(xs k)
(if (<= k 0) xs (acl-audit-drop (rest xs) (- k 1)))))
;; Structured snapshot for save/restore — a {:seq :entries} value carrying a
;; copy of the log (so later appends don't mutate a held snapshot).
(define acl-audit-snapshot (fn () {:seq acl-audit-seq :entries (acl-audit-copy acl-audit-log)}))
;; Replace the live log from a snapshot. Restores both entries and the seq
;; counter so subsequent records continue numbering correctly. The log is
;; rebuilt as a fresh append!-able list (see acl-audit-copy).
(define
acl-audit-restore!
(fn
(snap)
(do
(set! acl-audit-log (acl-audit-copy (get snap :entries)))
(set! acl-audit-seq (get snap :seq))
nil)))
;; Serialize the whole log to a disk-ready string: one record per line,
;; "seq\tsubj\tact\tres\tallowed?". A host writes this; structured reload is via
;; snapshot/restore.
(define
acl-audit-serialize
(fn
()
(reduce
(fn
(acc e)
(str
acc
(get e :seq)
"\t"
(get e :subj)
"\t"
(get e :act)
"\t"
(get e :res)
"\t"
(get e :allowed?)
"\n"))
""
acl-audit-log)))

32
lib/acl/conformance.conf Normal file
View File

@@ -0,0 +1,32 @@
# ACL conformance config — sourced by lib/guest/conformance.sh.
LANG_NAME=acl
MODE=dict
PRELOADS=(
lib/datalog/tokenizer.sx
lib/datalog/parser.sx
lib/datalog/unify.sx
lib/datalog/db.sx
lib/datalog/builtins.sx
lib/datalog/aggregates.sx
lib/datalog/strata.sx
lib/datalog/eval.sx
lib/datalog/api.sx
lib/datalog/magic.sx
lib/acl/schema.sx
lib/acl/facts.sx
lib/acl/engine.sx
lib/acl/explain.sx
lib/acl/audit.sx
lib/acl/federation.sx
lib/acl/api.sx
)
SUITES=(
"direct:lib/acl/tests/direct.sx:(acl-direct-tests-run!)"
"inherit:lib/acl/tests/inherit.sx:(acl-inherit-tests-run!)"
"explain:lib/acl/tests/explain.sx:(acl-explain-tests-run!)"
"fed:lib/acl/tests/fed.sx:(acl-fed-tests-run!)"
"harden:lib/acl/tests/harden.sx:(acl-harden-tests-run!)"
)

3
lib/acl/conformance.sh Executable file
View File

@@ -0,0 +1,3 @@
#!/usr/bin/env bash
# Thin wrapper — see lib/guest/conformance.sh and lib/acl/conformance.conf.
exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"

72
lib/acl/engine.sx Normal file
View File

@@ -0,0 +1,72 @@
;; lib/acl/engine.sx — ACL ruleset + decision reducer over lib/datalog/.
;;
;; The engine is a thin layer: it owns the permit ruleset (SX data rules) and
;; reduces a (subject, action, resource) decision to a Datalog query against a
;; db built from EDB facts. The rule engine itself is Datalog's.
;;
;; Policy — inheritance + federation with deny-overrides:
;;
;; eff_grant(S,A,R) :- grant(S,A,R). ; direct
;; eff_grant(S,A,R) :- member_of(S,G), eff_grant(G,A,R). ; group/role chain
;; eff_grant(S,A,R) :- child_of(R,P), eff_grant(S,A,P). ; resource tree
;; eff_grant(S,A,R) :- member_of(S,Role), role_grant(Role,A,R). ; role expansion
;; eff_grant(S,A,R) :- delegate(Peer,S,A,R), ; federated grant
;; trust(Peer,L), level_covers(L,A).
;;
;; eff_deny(S,A,R) :- deny(S,A,R). ; direct
;; eff_deny(S,A,R) :- member_of(S,G), eff_deny(G,A,R). ; group chain
;; eff_deny(S,A,R) :- child_of(R,P), eff_deny(S,A,P). ; resource tree
;;
;; permit(S,A,R) :- eff_grant(S,A,R), not eff_deny(S,A,R).
;;
;; DENY-OVERRIDES: an effective deny anywhere in the inheritance closure of
;; (S,A,R) defeats any effective grant — including federated grants. Deny
;; inherits through the *same* group and resource chains as grant, so a
;; group-level or ancestor-resource deny is authoritative for members/
;; descendants. This is the principled, fail-safe reading of "deny wins".
;;
;; FEDERATION — non-transitive trust: a peer's `delegate` fact only grants if a
;; *local* `trust(Peer, L)` exists AND that level `level_covers` the action.
;; Trust is re-checked on every query (it is a body literal), never baked in at
;; fact-ingestion time, so revoking trust or narrowing a level takes effect
;; immediately on the next decision.
;;
;; Termination & stratification:
;; - eff_grant/eff_deny recurse only over member_of and child_of, which are
;; EDB relations with no function symbols, so the closure is finite (cyclic
;; membership/containment just reaches a fixpoint, never loops). The
;; federation rule is non-recursive.
;; - permit negates eff_deny; neither eff_grant nor eff_deny depends on
;; permit, so the program is stratifiable (permit sits in a higher stratum).
(define
acl-rules
(quote
((eff_grant S A R <- (grant S A R))
(eff_grant S A R <- (member_of S G) (eff_grant G A R))
(eff_grant S A R <- (child_of R P) (eff_grant S A P))
(eff_grant S A R <- (member_of S Role) (role_grant Role A R))
(eff_grant
S
A
R
<-
(delegate Peer S A R)
(trust Peer L)
(level_covers L A))
(eff_deny S A R <- (deny S A R))
(eff_deny S A R <- (member_of S G) (eff_deny G A R))
(eff_deny S A R <- (child_of R P) (eff_deny S A P))
(permit S A R <- (eff_grant S A R) {:neg (eff_deny S A R)}))))
;; Build a Datalog db from a list of EDB facts under the ACL ruleset.
(define acl-build-db (fn (facts) (dl-program-data facts acl-rules)))
;; Core decision: does the db permit subject S to perform action A on
;; resource R? Reduces to a ground Datalog query on the derived `permit`
;; relation — non-empty result means permitted.
(define
acl-permit?
(fn
(db subj act res)
(> (len (dl-query db (list (quote permit) subj act res))) 0)))

125
lib/acl/explain.sx Normal file
View File

@@ -0,0 +1,125 @@
;; lib/acl/explain.sx — proof-tree reconstruction over the saturated db.
;;
;; lib/datalog/ records derived facts but not their provenance, so the proof is
;; reconstructed here by goal-directed search over the *saturated* db: for a
;; ground goal we find the first ACL rule (in rule order) whose body holds, take
;; the first solution binding its remaining variables, and recurse on each body
;; literal. Negated literals are recorded as verified `:neg-ok` leaves.
;;
;; CANONICAL DERIVATION: the Datalog derivation graph is a DAG (a fact may hold
;; many ways). We pick ONE canonical proof — first matching rule, first solution
;; — matching the rule order in lib/acl/engine.sx (direct/EDB rules first). A
;; depth cap guards against pathological cyclic data producing unbounded search.
;;
;; A proof node is one of:
;; {:fact <lit> :via "edb"} — base EDB fact
;; {:fact <lit> :rule <head> :body (<node|negleaf> ...)} — derived
;; {:neg-ok <lit>} — negation verified to fail
;; {:fact <lit> :truncated true} — depth cap hit
(define acl-proof-max-depth 64)
;; Substitute a body literal, descending into {:neg ...} dicts (dl-apply-subst
;; does not recurse into dicts, which would leak the neg's free vars).
(define
acl-subst-lit
(fn
(lit s)
(if
(and (dict? lit) (has-key? lit :neg))
{:neg (dl-apply-subst (get lit :neg) s)}
(dl-apply-subst lit s))))
(define
acl-lit-edb?
(fn
(lit)
(and
(list? lit)
(> (len lit) 0)
(symbol? (first lit))
(has-key? acl-edb-arity (symbol->string (first lit))))))
(define
acl-subst-zip!
(fn
(d ks vs)
(when
(> (len ks) 0)
(do
(dict-set! d (symbol->string (first ks)) (first vs))
(acl-subst-zip! d (rest ks) (rest vs))))))
;; Bind a rule head's variables to a ground goal's arguments (positional).
(define
acl-bind-head
(fn
(head goal)
(let
((d {}))
(do (acl-subst-zip! d (rest head) (rest goal)) d))))
(define
acl-subst-union
(fn
(a b)
(let
((d {}))
(do
(for-each (fn (k) (dict-set! d k (get a k))) (keys a))
(for-each (fn (k) (dict-set! d k (get b k))) (keys b))
d))))
(define acl-prove (fn (db goal) (acl-prove-d db goal 0)))
(define
acl-prove-d
(fn
(db goal depth)
(cond
((> depth acl-proof-max-depth) {:truncated true :fact goal})
((acl-lit-edb? goal)
(if (> (len (dl-query db goal)) 0) {:via "edb" :fact goal} nil))
(else (acl-prove-rules db goal acl-rules depth)))))
(define
acl-prove-rules
(fn
(db goal rules depth)
(if
(= (len rules) 0)
nil
(let
((p (dl-rule-from-list (first rules))))
(if
(= (first (get p :head)) (first goal))
(let
((hs (acl-bind-head (get p :head) goal)))
(let
((qbody (map (fn (l) (acl-subst-lit l hs)) (get p :body))))
(let
((sols (dl-query db qbody)))
(if
(> (len sols) 0)
(acl-prove-build db goal p hs (first sols) depth)
(acl-prove-rules db goal (rest rules) depth)))))
(acl-prove-rules db goal (rest rules) depth))))))
(define
acl-prove-build
(fn
(db goal p hs sol depth)
(let ((full (acl-subst-union hs sol))) {:body (map (fn (l) (let ((g (acl-subst-lit l full))) (if (and (dict? g) (has-key? g :neg)) {:neg-ok (get g :neg)} (acl-prove-d db g (+ depth 1))))) (get p :body)) :rule (get p :head) :fact goal})))
;; Public decision-with-proof. Returns:
;; {:allowed? <bool> :proof <node|nil> :reason <eff_deny proof|nil>}
;; When permitted, :proof is the permit derivation. When denied, :proof is nil
;; and :reason carries the blocking eff_deny proof if one exists (an explicit or
;; inherited deny), else nil (simply no grant).
(define
acl-explain
(fn
(db subj act res)
(let
((proof (acl-prove db (list (quote permit) subj act res))))
(if (= proof nil) {:allowed? false :proof nil :reason (acl-prove db (list (quote eff_deny) subj act res))} {:allowed? true :proof proof :reason nil}))))

47
lib/acl/facts.sx Normal file
View File

@@ -0,0 +1,47 @@
;; lib/acl/facts.sx — EDB fact constructors.
;;
;; Each constructor returns a Datalog fact tuple (a list whose head is the
;; predicate symbol). These are the only shapes lib/acl/engine.sx feeds to
;; lib/datalog/.
;; Phase 1: actor/resource/grant/deny.
;; Phase 2: member_of (subject -> group/role), child_of (resource -> parent),
;; role_grant (role -> action,resource capability).
;; Phase 4: peer/trust/delegate/level_covers (federation).
(define acl-actor (fn (id kind) (list (quote actor) id kind)))
(define acl-resource-fact (fn (id kind) (list (quote resource) id kind)))
(define acl-grant (fn (subj act res) (list (quote grant) subj act res)))
(define acl-deny (fn (subj act res) (list (quote deny) subj act res)))
;; subject S is a member of group/role G (one hop; transitivity is derived).
(define acl-member-of (fn (subj grp) (list (quote member_of) subj grp)))
;; resource R is a child of parent P (one hop; transitivity is derived).
(define acl-child-of (fn (res parent) (list (quote child_of) res parent)))
;; role confers capability (act on res) to every member of the role.
(define
acl-role-grant
(fn (role act res) (list (quote role_grant) role act res)))
;; --- federation ---
;; a known peer instance at addr, of some kind (e.g. peer).
(define acl-peer (fn (addr kind) (list (quote peer) addr kind)))
;; local trust in a peer at a named level. Gates delegated grants at query time.
(define acl-trust (fn (peer level) (list (quote trust) peer level)))
;; a peer asserts that subject S may A on R. Only takes effect if local trust in
;; that peer covers action A (see level_covers).
(define
acl-delegate
(fn (peer subj act res) (list (quote delegate) peer subj act res)))
;; local policy: trust `level` authorises delegated grants for action `act`.
(define
acl-level-covers
(fn (level act) (list (quote level_covers) level act)))

61
lib/acl/federation.sx Normal file
View File

@@ -0,0 +1,61 @@
;; lib/acl/federation.sx — cross-instance ACL facts + revocation.
;;
;; fed-sx replicates ACL facts between instances; this module models the local
;; side. A peer's authority arrives as `delegate(Peer, S, A, R)` facts, which
;; only take effect when a local `trust(Peer, L)` and `level_covers(L, A)`
;; authorise them (enforced by the engine rule, re-checked every query). The
;; actual network transport is fed-sx's job and is mocked in tests as a dict.
;;
;; Trust is NOT transitive: trusting peer α does not extend to peers α trusts.
;; Only delegate facts that α itself asserts, and that local trust covers, flow.
;; Mock fed-sx pull: `transport` is a dict mapping a peer address (its string
;; name) to the list of delegate facts that peer asserts. Returns the facts for
;; `addr`, or an empty list if the peer is unknown / unreachable.
(define
acl-fed-fetch
(fn
(transport addr)
(let
((k (if (symbol? addr) (symbol->string addr) addr)))
(if (has-key? transport k) (get transport k) (list)))))
;; Gather delegate facts from every peer in `addrs` via the transport.
(define
acl-fed-collect
(fn
(transport addrs)
(let
((acc (list)))
(do
(for-each
(fn
(addr)
(for-each
(fn (f) (append! acc f))
(acl-fed-fetch transport addr)))
addrs)
acc))))
;; Build a db from local facts plus delegate facts pulled from `peers`. Local
;; facts must include the `trust`/`level_covers` policy; replicated delegate
;; facts are gated against it by the engine rule at query time.
(define
acl-fed-build-db
(fn
(local-facts transport peers)
(let
((all (list)))
(do
(for-each (fn (f) (append! all f)) local-facts)
(for-each
(fn (f) (append! all f))
(acl-fed-collect transport peers))
(acl-build-db all)))))
;; Propagated revocation: retract a replicated fact (e.g. a peer's delegate, or
;; local trust) from a live db. The next decision re-saturates and reflects it.
(define acl-revoke! (fn (db fact) (do (dl-retract! db fact) db)))
;; Propagated assertion: ingest a newly replicated fact into a live db.
(define acl-fed-assert! (fn (db fact) (do (dl-assert! db fact) db)))

71
lib/acl/schema.sx Normal file
View File

@@ -0,0 +1,71 @@
;; lib/acl/schema.sx — ACL sorts and EDB predicate vocabulary.
;;
;; Datalog is untyped; this module is the schema-as-data layer. It declares
;; the subject/resource/action sorts and the arity of every EDB predicate the
;; ACL engine recognises, plus light validators. Facts that pass these checks
;; are well-formed inputs to lib/acl/engine.sx.
(define acl-subject-kinds (quote (user group role service)))
(define acl-resource-kinds (quote (page post thread peer)))
;; Actions are open-ended (a grant may name any action symbol), but these are
;; the platform's well-known verbs.
(define acl-actions (quote (read edit comment moderate federate)))
;; EDB predicate name -> arity.
;; Phase 1: actor/resource/grant/deny.
;; Phase 2: member_of (subject->group/role), child_of (resource->parent),
;; role_grant (role->action,resource).
;; Phase 4: peer (addr->kind), trust (peer->level),
;; delegate (peer->subj,action,resource), level_covers (level->action).
(define acl-edb-arity {:role_grant 3 :child_of 2 :trust 2 :peer 2 :actor 2 :level_covers 2 :delegate 4 :member_of 2 :deny 3 :grant 3 :resource 2})
(define
acl-member?
(fn
(x xs)
(cond
((= (len xs) 0) false)
((= (first xs) x) true)
(else (acl-member? x (rest xs))))))
(define acl-subject-kind? (fn (k) (acl-member? k acl-subject-kinds)))
(define acl-resource-kind? (fn (k) (acl-member? k acl-resource-kinds)))
(define acl-known-action? (fn (a) (acl-member? a acl-actions)))
;; A fact is a list whose head is a predicate symbol. Valid when the predicate
;; is known and the argument count matches the declared arity.
(define
acl-fact-valid?
(fn
(f)
(and
(list? f)
(> (len f) 0)
(symbol? (first f))
(let
((pred (symbol->string (first f))))
(and
(has-key? acl-edb-arity pred)
(= (- (len f) 1) (get acl-edb-arity pred)))))))
;; Return the sublist of facts that fail acl-fact-valid?. Empty list means the
;; whole set is well-formed. acl-build-db stays lenient (Datalog accepts any
;; tuple, and custom action symbols are allowed); callers opt in to checking.
(define
acl-validate-facts
(fn
(facts)
(let
((bad (list)))
(do
(for-each
(fn (f) (when (not (acl-fact-valid? f)) (append! bad f)))
facts)
bad))))
(define
acl-facts-valid?
(fn (facts) (= (len (acl-validate-facts facts)) 0)))

14
lib/acl/scoreboard.json Normal file
View File

@@ -0,0 +1,14 @@
{
"lang": "acl",
"total_passed": 145,
"total_failed": 0,
"total": 145,
"suites": [
{"name":"direct","passed":24,"failed":0,"total":24},
{"name":"inherit","passed":30,"failed":0,"total":30},
{"name":"explain","passed":35,"failed":0,"total":35},
{"name":"fed","passed":31,"failed":0,"total":31},
{"name":"harden","passed":25,"failed":0,"total":25}
],
"generated": "2026-06-06T22:43:27+00:00"
}

11
lib/acl/scoreboard.md Normal file
View File

@@ -0,0 +1,11 @@
# acl scoreboard
**145 / 145 passing** (0 failure(s)).
| Suite | Passed | Total | Status |
|-------|--------|-------|--------|
| direct | 24 | 24 | ok |
| inherit | 30 | 30 | ok |
| explain | 35 | 35 | ok |
| fed | 31 | 31 | ok |
| harden | 25 | 25 | ok |

170
lib/acl/tests/direct.sx Normal file
View File

@@ -0,0 +1,170 @@
;; lib/acl/tests/direct.sx — Phase 1: direct grants + deny-overrides.
(define acl-dt-pass 0)
(define acl-dt-fail 0)
(define acl-dt-failures (list))
(define
acl-dt-check!
(fn
(name got expected)
(if
(= got expected)
(set! acl-dt-pass (+ acl-dt-pass 1))
(do
(set! acl-dt-fail (+ acl-dt-fail 1))
(append!
acl-dt-failures
(str name "\n expected: " expected "\n got: " got))))))
;; A small fixture used by most cases: alice can read page1, is denied edit on
;; page1, and a service may federate peer1.
(define
acl-dt-fixture
(fn
()
(acl-build-db
(list
(acl-actor (quote alice) (quote user))
(acl-actor (quote svc1) (quote service))
(acl-resource-fact (quote page1) (quote page))
(acl-resource-fact (quote peer1) (quote peer))
(acl-grant (quote alice) (quote read) (quote page1))
(acl-grant (quote alice) (quote edit) (quote page1))
(acl-deny (quote alice) (quote edit) (quote page1))
(acl-grant (quote svc1) (quote federate) (quote peer1))))))
(define
acl-dt-run-all!
(fn
()
(let
((db (acl-dt-fixture)))
(do
(acl-dt-check!
"direct grant permits"
(acl-permit? db (quote alice) (quote read) (quote page1))
true)
(acl-dt-check!
"service grant permits federate"
(acl-permit? db (quote svc1) (quote federate) (quote peer1))
true)
(acl-dt-check!
"missing action denied"
(acl-permit? db (quote alice) (quote comment) (quote page1))
false)
(acl-dt-check!
"missing resource denied"
(acl-permit? db (quote alice) (quote read) (quote page2))
false)
(acl-dt-check!
"missing subject denied"
(acl-permit? db (quote bob) (quote read) (quote page1))
false)
(acl-dt-check!
"wrong subject for service grant denied"
(acl-permit? db (quote alice) (quote federate) (quote peer1))
false)
(acl-dt-check!
"grant plus deny -> deny wins"
(acl-permit? db (quote alice) (quote edit) (quote page1))
false)
(acl-dt-check!
"deny alone still denies"
(acl-permit?
(acl-build-db
(list (acl-deny (quote alice) (quote read) (quote page1))))
(quote alice)
(quote read)
(quote page1))
false)
(acl-dt-check!
"deny on edit does not block read"
(acl-permit? db (quote alice) (quote read) (quote page1))
true)
(acl-dt-check!
"empty db denies"
(acl-permit?
(acl-build-db (list))
(quote alice)
(quote read)
(quote page1))
false)
(let
((db2 (acl-build-db (list (acl-grant (quote a) (quote read) (quote r)) (acl-grant (quote b) (quote read) (quote r)) (acl-deny (quote b) (quote read) (quote r))))))
(do
(acl-dt-check!
"subject a allowed"
(acl-permit? db2 (quote a) (quote read) (quote r))
true)
(acl-dt-check!
"subject b denied by override"
(acl-permit? db2 (quote b) (quote read) (quote r))
false)))
(let
((db3 (acl-build-db (list (acl-actor (quote editors) (quote role)) (acl-grant (quote editors) (quote edit) (quote post1))))))
(acl-dt-check!
"role subject direct grant"
(acl-permit? db3 (quote editors) (quote edit) (quote post1))
true))
(do
(acl/load!
(list
(acl-grant (quote carol) (quote moderate) (quote thread1))))
(acl-dt-check!
"api permit via current db"
(acl/permit? (quote carol) (quote moderate) (quote thread1))
true)
(acl-dt-check!
"api deny via current db"
(acl/permit? (quote carol) (quote read) (quote thread1))
false))
(do
(acl/load! (list))
(acl-dt-check!
"api reload clears prior grants"
(acl/permit? (quote carol) (quote moderate) (quote thread1))
false))
(acl-dt-check!
"schema grant arity valid"
(acl-fact-valid? (acl-grant (quote x) (quote read) (quote y)))
true)
(acl-dt-check!
"schema bad arity invalid"
(acl-fact-valid? (list (quote grant) (quote x)))
false)
(acl-dt-check!
"schema unknown predicate invalid"
(acl-fact-valid? (list (quote frobnicate) (quote x)))
false)
(acl-dt-check!
"schema subject kind known"
(acl-subject-kind? (quote service))
true)
(acl-dt-check!
"schema resource kind unknown"
(acl-resource-kind? (quote galaxy))
false)
(acl-dt-check!
"schema known action"
(acl-known-action? (quote moderate))
true)
(acl-dt-check!
"grant constructor shape"
(acl-grant (quote u) (quote read) (quote p))
(list (quote grant) (quote u) (quote read) (quote p)))
(acl-dt-check!
"actor constructor shape"
(acl-actor (quote u) (quote user))
(list (quote actor) (quote u) (quote user)))))))
(define
acl-direct-tests-run!
(fn
()
(do
(set! acl-dt-pass 0)
(set! acl-dt-fail 0)
(set! acl-dt-failures (list))
(acl-dt-run-all!)
{:failures acl-dt-failures :total (+ acl-dt-pass acl-dt-fail) :passed acl-dt-pass :failed acl-dt-fail})))

316
lib/acl/tests/explain.sx Normal file
View File

@@ -0,0 +1,316 @@
;; lib/acl/tests/explain.sx — Phase 3: proof correctness + audit completeness.
(define acl-et-pass 0)
(define acl-et-fail 0)
(define acl-et-failures (list))
;; Name-based deep equality. The host `=` compares symbols by interned
;; identity, which is unstable across substitution/saturation; comparing by
;; name (as the datalog suite does) makes structural assertions deterministic.
(define
acl-et-eq?
(fn
(a b)
(cond
((and (list? a) (list? b))
(and (= (len a) (len b)) (acl-et-eq-l? a b 0)))
((and (dict? a) (dict? b))
(let
((ka (keys a)) (kb (keys b)))
(and (= (len ka) (len kb)) (acl-et-eq-d? a b ka 0))))
((and (symbol? a) (symbol? b))
(= (symbol->string a) (symbol->string b)))
(else (= a b)))))
(define
acl-et-eq-l?
(fn
(a b i)
(cond
((>= i (len a)) true)
((not (acl-et-eq? (nth a i) (nth b i))) false)
(else (acl-et-eq-l? a b (+ i 1))))))
(define
acl-et-eq-d?
(fn
(a b ka i)
(cond
((>= i (len ka)) true)
((let ((k (nth ka i))) (not (acl-et-eq? (get a k) (get b k))))
false)
(else (acl-et-eq-d? a b ka (+ i 1))))))
(define
acl-et-check!
(fn
(name got expected)
(if
(acl-et-eq? got expected)
(set! acl-et-pass (+ acl-et-pass 1))
(do
(set! acl-et-fail (+ acl-et-fail 1))
(append!
acl-et-failures
(str name "\n expected: " expected "\n got: " got))))))
;; --- proof-tree walkers ---
;; True if EDB fact `target` appears as a base leaf anywhere in the proof.
(define
acl-et-has-leaf?
(fn
(node target)
(cond
((= node nil) false)
((and (dict? node) (has-key? node :via))
(acl-et-eq? (get node :fact) target))
((and (dict? node) (has-key? node :body))
(acl-et-any-leaf? (get node :body) target))
(else false))))
(define
acl-et-any-leaf?
(fn
(nodes target)
(cond
((= (len nodes) 0) false)
((acl-et-has-leaf? (first nodes) target) true)
(else (acl-et-any-leaf? (rest nodes) target)))))
;; True if the proof records a verified negation (deny did not fire).
(define
acl-et-has-negok?
(fn
(node)
(cond
((= node nil) false)
((and (dict? node) (has-key? node :neg-ok)) true)
((and (dict? node) (has-key? node :body))
(acl-et-any-negok? (get node :body)))
(else false))))
(define
acl-et-any-negok?
(fn
(nodes)
(cond
((= (len nodes) 0) false)
((acl-et-has-negok? (first nodes)) true)
(else (acl-et-any-negok? (rest nodes))))))
(define
acl-et-run-all!
(fn
()
(do
(let
((db (acl-build-db (list (acl-grant (quote u) (quote read) (quote p))))))
(let
((e (acl-explain db (quote u) (quote read) (quote p))))
(do
(acl-et-check! "direct: allowed?" (get e :allowed?) true)
(acl-et-check!
"direct: proof root fact"
(get (get e :proof) :fact)
(list (quote permit) (quote u) (quote read) (quote p)))
(acl-et-check!
"direct: grant leaf present"
(acl-et-has-leaf?
(get e :proof)
(list (quote grant) (quote u) (quote read) (quote p)))
true)
(acl-et-check!
"direct: negation verified"
(acl-et-has-negok? (get e :proof))
true)
(acl-et-check!
"direct: reason nil when allowed"
(get e :reason)
nil))))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-member-of (quote team) (quote org)) (acl-grant (quote org) (quote read) (quote doc))))))
(let
((e (acl-explain db (quote alice) (quote read) (quote doc))))
(do
(acl-et-check! "group: allowed?" (get e :allowed?) true)
(acl-et-check!
"group: member_of alice leaf"
(acl-et-has-leaf?
(get e :proof)
(list (quote member_of) (quote alice) (quote team)))
true)
(acl-et-check!
"group: member_of team leaf"
(acl-et-has-leaf?
(get e :proof)
(list (quote member_of) (quote team) (quote org)))
true)
(acl-et-check!
"group: grant org leaf at base"
(acl-et-has-leaf?
(get e :proof)
(list (quote grant) (quote org) (quote read) (quote doc)))
true))))
(let
((db (acl-build-db (list (acl-child-of (quote sec) (quote book)) (acl-grant (quote u) (quote read) (quote book))))))
(let
((e (acl-explain db (quote u) (quote read) (quote sec))))
(do
(acl-et-check! "resource: allowed?" (get e :allowed?) true)
(acl-et-check!
"resource: child_of leaf"
(acl-et-has-leaf?
(get e :proof)
(list (quote child_of) (quote sec) (quote book)))
true)
(acl-et-check!
"resource: grant on parent leaf"
(acl-et-has-leaf?
(get e :proof)
(list (quote grant) (quote u) (quote read) (quote book)))
true))))
(let
((db (acl-build-db (list (acl-member-of (quote bob) (quote editor)) (acl-role-grant (quote editor) (quote edit) (quote page1))))))
(let
((e (acl-explain db (quote bob) (quote edit) (quote page1))))
(do
(acl-et-check! "role: allowed?" (get e :allowed?) true)
(acl-et-check!
"role: member_of leaf"
(acl-et-has-leaf?
(get e :proof)
(list (quote member_of) (quote bob) (quote editor)))
true)
(acl-et-check!
"role: role_grant leaf"
(acl-et-has-leaf?
(get e :proof)
(list
(quote role_grant)
(quote editor)
(quote edit)
(quote page1)))
true))))
(let
((db (acl-build-db (list (acl-grant (quote u) (quote edit) (quote p)) (acl-deny (quote u) (quote edit) (quote p))))))
(let
((e (acl-explain db (quote u) (quote edit) (quote p))))
(do
(acl-et-check! "deny: not allowed" (get e :allowed?) false)
(acl-et-check! "deny: no proof" (get e :proof) nil)
(acl-et-check!
"deny: reason root is eff_deny"
(get (get e :reason) :fact)
(list (quote eff_deny) (quote u) (quote edit) (quote p)))
(acl-et-check!
"deny: reason has deny leaf"
(acl-et-has-leaf?
(get e :reason)
(list (quote deny) (quote u) (quote edit) (quote p)))
true))))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-grant (quote alice) (quote read) (quote doc)) (acl-deny (quote team) (quote read) (quote doc))))))
(let
((e (acl-explain db (quote alice) (quote read) (quote doc))))
(do
(acl-et-check!
"inherited deny: not allowed"
(get e :allowed?)
false)
(acl-et-check!
"inherited deny: reason has member_of leaf"
(acl-et-has-leaf?
(get e :reason)
(list (quote member_of) (quote alice) (quote team)))
true)
(acl-et-check!
"inherited deny: reason has group deny leaf"
(acl-et-has-leaf?
(get e :reason)
(list (quote deny) (quote team) (quote read) (quote doc)))
true))))
(let
((db (acl-build-db (list))))
(let
((e (acl-explain db (quote u) (quote read) (quote p))))
(do
(acl-et-check! "no grant: not allowed" (get e :allowed?) false)
(acl-et-check! "no grant: proof nil" (get e :proof) nil)
(acl-et-check! "no grant: reason nil" (get e :reason) nil))))
(let
((db (acl-build-db (list (acl-grant (quote u) (quote read) (quote p)) (acl-deny (quote u) (quote edit) (quote p))))))
(do
(acl-audit-clear!)
(acl-et-check! "audit: starts empty" (acl-audit-count) 0)
(acl-et-check!
"audit decide allowed returns true"
(acl-audit-decide! db (quote u) (quote read) (quote p))
true)
(acl-et-check!
"audit decide denied returns false"
(acl-audit-decide! db (quote u) (quote edit) (quote p))
false)
(acl-audit-decide! db (quote u) (quote comment) (quote p))
(acl-et-check!
"audit: count after three decisions"
(acl-audit-count)
3)
(acl-et-check!
"audit: tail size respects n"
(len (acl-audit-tail 2))
2)
(acl-et-check!
"audit: tail returns most recent"
(get (first (acl-audit-tail 1)) :act)
(quote comment))
(acl-et-check!
"audit: first record seq is 0"
(get (first (acl-audit-tail 3)) :seq)
0)
(acl-et-check!
"audit: allowed flag recorded"
(get (first (acl-audit-tail 3)) :allowed?)
true)
(acl-et-check!
"audit: serialize line count"
(len (acl-et-lines (acl-audit-serialize)))
3)
(acl-audit-clear!)
(acl-et-check!
"audit: clear resets count"
(acl-audit-count)
0))))))
;; count newline-terminated lines in a serialized log
(define acl-et-lines (fn (s) (acl-et-count-nl s 0 0)))
(define
acl-et-count-nl
(fn
(s i n)
(if
(>= i (len s))
(if (= n 0) (list) (acl-et-rangelist n))
(acl-et-count-nl
s
(+ i 1)
(if (= (slice s i (+ i 1)) "\n") (+ n 1) n)))))
(define
acl-et-rangelist
(fn
(n)
(if
(<= n 0)
(list)
(cons n (acl-et-rangelist (- n 1))))))
(define
acl-explain-tests-run!
(fn
()
(do
(set! acl-et-pass 0)
(set! acl-et-fail 0)
(set! acl-et-failures (list))
(acl-et-run-all!)
{:failures acl-et-failures :total (+ acl-et-pass acl-et-fail) :passed acl-et-pass :failed acl-et-fail})))

273
lib/acl/tests/fed.sx Normal file
View File

@@ -0,0 +1,273 @@
;; lib/acl/tests/fed.sx — Phase 4: federation (peer trust, delegation,
;; cross-instance chains, revocation). fed-sx transport is mocked as a dict.
(define acl-ft-pass 0)
(define acl-ft-fail 0)
(define acl-ft-failures (list))
;; Name-based deep equality (host `=` compares symbols by unstable interned
;; identity; see lib/acl/tests/explain.sx).
(define
acl-ft-eq?
(fn
(a b)
(cond
((and (list? a) (list? b))
(and (= (len a) (len b)) (acl-ft-eq-l? a b 0)))
((and (symbol? a) (symbol? b))
(= (symbol->string a) (symbol->string b)))
(else (= a b)))))
(define
acl-ft-eq-l?
(fn
(a b i)
(cond
((>= i (len a)) true)
((not (acl-ft-eq? (nth a i) (nth b i))) false)
(else (acl-ft-eq-l? a b (+ i 1))))))
(define
acl-ft-check!
(fn
(name got expected)
(if
(acl-ft-eq? got expected)
(set! acl-ft-pass (+ acl-ft-pass 1))
(do
(set! acl-ft-fail (+ acl-ft-fail 1))
(append!
acl-ft-failures
(str name "\n expected: " expected "\n got: " got))))))
;; proof leaf walker (federated proofs reconstruct through the engine rule).
(define
acl-ft-has-leaf?
(fn
(node target)
(cond
((= node nil) false)
((and (dict? node) (has-key? node :via))
(acl-ft-eq? (get node :fact) target))
((and (dict? node) (has-key? node :body))
(acl-ft-any-leaf? (get node :body) target))
(else false))))
(define
acl-ft-any-leaf?
(fn
(nodes target)
(cond
((= (len nodes) 0) false)
((acl-ft-has-leaf? (first nodes) target) true)
(else (acl-ft-any-leaf? (rest nodes) target)))))
(define acl-ft-p? (fn (db s a r) (acl-permit? db s a r)))
;; A standard federation fixture: local trusts peer alpha at "readonly", which
;; covers read+comment. alpha delegates several capabilities to alice.
(define
acl-ft-fixture
(fn
()
(acl-build-db
(list
(acl-trust (quote alpha) (quote readonly))
(acl-level-covers (quote readonly) (quote read))
(acl-level-covers (quote readonly) (quote comment))
(acl-delegate (quote alpha) (quote alice) (quote read) (quote doc))
(acl-delegate (quote alpha) (quote alice) (quote edit) (quote doc))))))
(define
acl-ft-run-all!
(fn
()
(do
(let
((db (acl-ft-fixture)))
(do
(acl-ft-check!
"trusted delegate, level covers action -> permit"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true)
(acl-ft-check!
"trusted delegate, level does NOT cover action -> deny"
(acl-ft-p? db (quote alice) (quote edit) (quote doc))
false)
(acl-ft-check!
"delegated but action class uncovered (comment has no delegate)"
(acl-ft-p? db (quote alice) (quote comment) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-level-covers (quote readonly) (quote read)) (acl-delegate (quote beta) (quote bob) (quote read) (quote doc))))))
(acl-ft-check!
"untrusted peer delegate -> deny"
(acl-ft-p? db (quote bob) (quote read) (quote doc))
false))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote readonly)) (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc))))))
(acl-ft-check!
"trust but no level_covers -> deny"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
false))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc)) (acl-delegate (quote beta) (quote bob) (quote read) (quote doc))))))
(do
(acl-ft-check!
"trust is per-peer: alpha's delegate applies"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true)
(acl-ft-check!
"trust not transitive: beta's delegate does not apply"
(acl-ft-p? db (quote bob) (quote read) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc)) (acl-deny (quote alice) (quote read) (quote doc))))))
(acl-ft-check!
"local deny overrides federated grant"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
false))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-member-of (quote alice) (quote team)) (acl-delegate (quote alpha) (quote team) (quote read) (quote doc))))))
(acl-ft-check!
"federated grant to group reaches member"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-child-of (quote sec) (quote book)) (acl-delegate (quote alpha) (quote u) (quote read) (quote book))))))
(acl-ft-check!
"federated grant on parent resource reaches child"
(acl-ft-p? db (quote u) (quote read) (quote sec))
true))
(let
((transport {:gamma (list (acl-delegate (quote gamma) (quote carol) (quote read) (quote post))) :alpha (list (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc)))}))
(do
(acl-ft-check!
"fetch known peer returns its delegates"
(len (acl-fed-fetch transport (quote alpha)))
1)
(acl-ft-check!
"fetch unknown peer returns empty"
(len (acl-fed-fetch transport (quote delta)))
0)
(acl-ft-check!
"collect across peers"
(len
(acl-fed-collect transport (list (quote alpha) (quote gamma))))
2)
(let
((db (acl-fed-build-db (list (acl-trust (quote alpha) (quote readonly)) (acl-trust (quote gamma) (quote readonly)) (acl-level-covers (quote readonly) (quote read))) transport (list (quote alpha) (quote gamma)))))
(do
(acl-ft-check!
"fed-build-db: alpha delegate permits"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true)
(acl-ft-check!
"fed-build-db: gamma delegate permits"
(acl-ft-p? db (quote carol) (quote read) (quote post))
true)
(acl-ft-check!
"fed-build-db: untrusted action still denied"
(acl-ft-p? db (quote alice) (quote edit) (quote doc))
false)))))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc))))))
(do
(acl-ft-check!
"before revoke: permitted"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true)
(acl-revoke!
db
(acl-delegate
(quote alpha)
(quote alice)
(quote read)
(quote doc)))
(acl-ft-check!
"after delegate revoked: denied"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc))))))
(do
(acl-ft-check!
"before trust revoke: permitted"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true)
(acl-revoke! db (acl-trust (quote alpha) (quote full)))
(acl-ft-check!
"after trust revoked: denied"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote alice) (quote read) (quote doc))))))
(do
(acl-ft-check!
"delegate without trust: denied"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
false)
(acl-fed-assert! db (acl-trust (quote alpha) (quote full)))
(acl-ft-check!
"trust ingested then re-checked: permitted"
(acl-ft-p? db (quote alice) (quote read) (quote doc))
true)))
(let
((db (acl-ft-fixture)))
(let
((e (acl-explain db (quote alice) (quote read) (quote doc))))
(do
(acl-ft-check! "federated proof allowed?" (get e :allowed?) true)
(acl-ft-check!
"federated proof has delegate leaf"
(acl-ft-has-leaf?
(get e :proof)
(list
(quote delegate)
(quote alpha)
(quote alice)
(quote read)
(quote doc)))
true)
(acl-ft-check!
"federated proof has trust leaf"
(acl-ft-has-leaf?
(get e :proof)
(list (quote trust) (quote alpha) (quote readonly)))
true)
(acl-ft-check!
"federated proof has level_covers leaf"
(acl-ft-has-leaf?
(get e :proof)
(list (quote level_covers) (quote readonly) (quote read)))
true))))
(acl-ft-check!
"schema delegate arity valid"
(acl-fact-valid?
(acl-delegate (quote p) (quote s) (quote a) (quote r)))
true)
(acl-ft-check!
"schema trust arity valid"
(acl-fact-valid? (acl-trust (quote p) (quote l)))
true)
(acl-ft-check!
"schema peer arity valid"
(acl-fact-valid? (acl-peer (quote p) (quote peer)))
true)
(acl-ft-check!
"schema level_covers arity valid"
(acl-fact-valid? (acl-level-covers (quote l) (quote read)))
true)
(acl-ft-check!
"schema delegate bad arity invalid"
(acl-fact-valid? (list (quote delegate) (quote p) (quote s)))
false))))
(define
acl-fed-tests-run!
(fn
()
(do
(set! acl-ft-pass 0)
(set! acl-ft-fail 0)
(set! acl-ft-failures (list))
(acl-ft-run-all!)
{:failures acl-ft-failures :total (+ acl-ft-pass acl-ft-fail) :passed acl-ft-pass :failed acl-ft-fail})))

228
lib/acl/tests/harden.sx Normal file
View File

@@ -0,0 +1,228 @@
;; lib/acl/tests/harden.sx — adversarial / cross-phase hardening.
;;
;; Diamond hierarchies, conflict resolution where deny must win through every
;; path, chain inheritance, cycle termination, multi-peer delegation, fact
;; validation, and audit save/restore.
;;
;; PROVER-FREE BY DESIGN: this suite calls only acl-permit? (which runs in
;; compiled Datalog, safe at any depth) plus pure data ops — never acl-explain /
;; acl-prove-d. The SX-side proof reconstructor recurses, and once the kernel
;; JIT-compiles it (after the explain/fed suites warm the process) it loops on
;; chains deeper than ~3 (substrate JIT bug — see plan Blockers). Proof
;; reconstruction is covered by tests/explain.sx (and federated proofs by
;; tests/fed.sx), both of which stay under the warm-process depth threshold.
(define acl-hd-pass 0)
(define acl-hd-fail 0)
(define acl-hd-failures (list))
(define
acl-hd-check!
(fn
(name got expected)
(if
(= got expected)
(set! acl-hd-pass (+ acl-hd-pass 1))
(do
(set! acl-hd-fail (+ acl-hd-fail 1))
(append!
acl-hd-failures
(str name "\n expected: " expected "\n got: " got))))))
(define acl-hd-p? (fn (db s a r) (acl-permit? db s a r)))
(define
acl-hd-run-all!
(fn
()
(do
(let
((grant-deny (acl-build-db (list (acl-child-of (quote r) (quote p1)) (acl-child-of (quote r) (quote p2)) (acl-grant (quote u) (quote read) (quote p1)) (acl-deny (quote u) (quote read) (quote p2)))))
(both-grant
(acl-build-db
(list
(acl-child-of (quote r) (quote p1))
(acl-child-of (quote r) (quote p2))
(acl-grant (quote u) (quote read) (quote p1))
(acl-grant (quote u) (quote read) (quote p2))))))
(do
(acl-hd-check!
"diamond resource: grant+deny parents -> deny wins"
(acl-hd-p? grant-deny (quote u) (quote read) (quote r))
false)
(acl-hd-check!
"diamond resource: both grant -> permit"
(acl-hd-p? both-grant (quote u) (quote read) (quote r))
true)
(acl-hd-check!
"diamond resource: deny does not leak to other parent"
(acl-hd-p? grant-deny (quote u) (quote read) (quote p1))
true)))
(let
((grant-deny (acl-build-db (list (acl-member-of (quote alice) (quote g1)) (acl-member-of (quote alice) (quote g2)) (acl-grant (quote g1) (quote read) (quote doc)) (acl-deny (quote g2) (quote read) (quote doc)))))
(both-grant
(acl-build-db
(list
(acl-member-of (quote alice) (quote g1))
(acl-member-of (quote alice) (quote g2))
(acl-grant (quote g1) (quote read) (quote doc))
(acl-grant (quote g2) (quote read) (quote doc))))))
(do
(acl-hd-check!
"diamond group: grant+deny groups -> deny wins"
(acl-hd-p? grant-deny (quote alice) (quote read) (quote doc))
false)
(acl-hd-check!
"diamond group: both grant -> permit"
(acl-hd-p? both-grant (quote alice) (quote read) (quote doc))
true)))
(let
((chain (acl-build-db (list (acl-member-of (quote a0) (quote a1)) (acl-member-of (quote a1) (quote a2)) (acl-member-of (quote a2) (quote a3)) (acl-member-of (quote a3) (quote a4)) (acl-grant (quote a4) (quote read) (quote res)))))
(chain-deny
(acl-build-db
(list
(acl-member-of (quote a0) (quote a1))
(acl-member-of (quote a1) (quote a2))
(acl-member-of (quote a2) (quote a3))
(acl-member-of (quote a3) (quote a4))
(acl-grant (quote a4) (quote read) (quote res))
(acl-deny (quote a0) (quote read) (quote res))))))
(do
(acl-hd-check!
"chain: top-group grant reaches leaf member"
(acl-hd-p? chain (quote a0) (quote read) (quote res))
true)
(acl-hd-check!
"chain: intermediate also covered"
(acl-hd-p? chain (quote a2) (quote read) (quote res))
true)
(acl-hd-check!
"chain: leaf-member deny overrides top grant"
(acl-hd-p? chain-deny (quote a0) (quote read) (quote res))
false)
(acl-hd-check!
"chain: deny on leaf does not block sibling level"
(acl-hd-p? chain-deny (quote a1) (quote read) (quote res))
true)))
(let
((self-member (acl-build-db (list (acl-member-of (quote a) (quote a)) (acl-grant (quote a) (quote read) (quote r)))))
(self-child
(acl-build-db
(list
(acl-child-of (quote r) (quote r))
(acl-grant (quote u) (quote read) (quote r)))))
(two-cycle
(acl-build-db
(list
(acl-member-of (quote x) (quote y))
(acl-member-of (quote y) (quote x))
(acl-grant (quote y) (quote read) (quote r))))))
(do
(acl-hd-check!
"self-membership cycle terminates and grants"
(acl-hd-p? self-member (quote a) (quote read) (quote r))
true)
(acl-hd-check!
"self-child cycle terminates and grants"
(acl-hd-p? self-child (quote u) (quote read) (quote r))
true)
(acl-hd-check!
"two-node membership cycle terminates"
(acl-hd-p? two-cycle (quote x) (quote read) (quote r))
true)))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-member-of (quote alice) (quote team)) (acl-delegate (quote alpha) (quote team) (quote read) (quote doc)) (acl-deny (quote alice) (quote read) (quote doc))))))
(acl-hd-check!
"federated group grant, local member deny -> deny wins"
(acl-hd-p? db (quote alice) (quote read) (quote doc))
false))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote bob) (quote read) (quote doc)) (acl-delegate (quote beta) (quote bob) (quote read) (quote doc))))))
(acl-hd-check!
"two peers delegate, one trusted -> permit"
(acl-hd-p? db (quote bob) (quote read) (quote doc))
true))
(let
((db (acl-build-db (list (acl-trust (quote alpha) (quote full)) (acl-trust (quote beta) (quote full)) (acl-level-covers (quote full) (quote read)) (acl-delegate (quote alpha) (quote bob) (quote read) (quote doc)) (acl-delegate (quote beta) (quote bob) (quote read) (quote doc))))))
(acl-hd-check!
"two peers both trusted -> permit"
(acl-hd-p? db (quote bob) (quote read) (quote doc))
true))
(let
((empty (acl-build-db (list))))
(acl-hd-check!
"empty db: nothing permitted"
(acl-hd-p? empty (quote u) (quote read) (quote r))
false))
(do
(acl-hd-check!
"validate: clean set has no bad facts"
(len
(acl-validate-facts
(list
(acl-grant (quote u) (quote read) (quote p))
(acl-member-of (quote u) (quote g))
(acl-delegate (quote pe) (quote u) (quote read) (quote p)))))
0)
(acl-hd-check!
"validate: facts-valid? true on clean set"
(acl-facts-valid?
(list (acl-grant (quote u) (quote read) (quote p))))
true)
(acl-hd-check!
"validate: surfaces wrong-arity and unknown predicate"
(len
(acl-validate-facts
(list
(acl-grant (quote u) (quote read) (quote p))
(list (quote grant) (quote u))
(list (quote bogus) (quote x) (quote y)))))
2)
(acl-hd-check!
"validate: empty set is valid"
(acl-facts-valid? (list))
true))
(let
((db (acl-build-db (list (acl-grant (quote u) (quote read) (quote p)) (acl-deny (quote u) (quote edit) (quote p))))))
(do
(acl-audit-clear!)
(acl-audit-decide! db (quote u) (quote read) (quote p))
(acl-audit-decide! db (quote u) (quote edit) (quote p))
(let
((snap (acl-audit-snapshot)))
(do
(acl-audit-clear!)
(acl-hd-check!
"audit: cleared count is 0"
(acl-audit-count)
0)
(acl-audit-restore! snap)
(acl-hd-check!
"audit: restored count"
(acl-audit-count)
2)
(acl-hd-check!
"audit: restored last act"
(get (first (acl-audit-tail 1)) :act)
(quote edit))
(acl-audit-decide! db (quote u) (quote comment) (quote p))
(acl-hd-check!
"audit: seq continues after restore"
(get (first (acl-audit-tail 1)) :seq)
2)
(acl-hd-check!
"audit: snapshot is an immutable copy"
(len (get snap :entries))
2)
(acl-audit-clear!))))))))
(define
acl-harden-tests-run!
(fn
()
(do
(set! acl-hd-pass 0)
(set! acl-hd-fail 0)
(set! acl-hd-failures (list))
(acl-hd-run-all!)
{:failures acl-hd-failures :total (+ acl-hd-pass acl-hd-fail) :passed acl-hd-pass :failed acl-hd-fail})))

202
lib/acl/tests/inherit.sx Normal file
View File

@@ -0,0 +1,202 @@
;; lib/acl/tests/inherit.sx — Phase 2: inheritance (groups, resource trees,
;; role expansion) with deny-overrides.
(define acl-it-pass 0)
(define acl-it-fail 0)
(define acl-it-failures (list))
(define
acl-it-check!
(fn
(name got expected)
(if
(= got expected)
(set! acl-it-pass (+ acl-it-pass 1))
(do
(set! acl-it-fail (+ acl-it-fail 1))
(append!
acl-it-failures
(str name "\n expected: " expected "\n got: " got))))))
(define acl-it-p? (fn (db s a r) (acl-permit? db s a r)))
(define
acl-it-run-all!
(fn
()
(do
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-grant (quote team) (quote read) (quote doc))))))
(do
(acl-it-check!
"group grant reaches member"
(acl-it-p? db (quote alice) (quote read) (quote doc))
true)
(acl-it-check!
"group grant: non-member excluded"
(acl-it-p? db (quote bob) (quote read) (quote doc))
false)
(acl-it-check!
"group grant: wrong action"
(acl-it-p? db (quote alice) (quote edit) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-member-of (quote team) (quote org)) (acl-member-of (quote org) (quote company)) (acl-grant (quote company) (quote read) (quote doc))))))
(do
(acl-it-check!
"deep nested group grant reaches leaf member"
(acl-it-p? db (quote alice) (quote read) (quote doc))
true)
(acl-it-check!
"intermediate group also covered"
(acl-it-p? db (quote team) (quote read) (quote doc))
true)
(acl-it-check!
"mid group org covered"
(acl-it-p? db (quote org) (quote read) (quote doc))
true)))
(let
((db (acl-build-db (list (acl-member-of (quote a) (quote b)) (acl-member-of (quote b) (quote a)) (acl-grant (quote b) (quote read) (quote r))))))
(do
(acl-it-check!
"cyclic membership terminates and grants"
(acl-it-p? db (quote a) (quote read) (quote r))
true)
(acl-it-check!
"cyclic membership covers both"
(acl-it-p? db (quote b) (quote read) (quote r))
true)))
(let
((db (acl-build-db (list (acl-child-of (quote sec) (quote chap)) (acl-child-of (quote chap) (quote book)) (acl-grant (quote u) (quote read) (quote book))))))
(do
(acl-it-check!
"parent grant reaches direct child"
(acl-it-p? db (quote u) (quote read) (quote chap))
true)
(acl-it-check!
"parent grant reaches deep descendant"
(acl-it-p? db (quote u) (quote read) (quote sec))
true)
(acl-it-check!
"parent grant covers parent itself"
(acl-it-p? db (quote u) (quote read) (quote book))
true)
(acl-it-check!
"child grant does not climb to parent"
(acl-it-p?
(acl-build-db
(list
(acl-child-of (quote sec) (quote book))
(acl-grant (quote u) (quote read) (quote sec))))
(quote u)
(quote read)
(quote book))
false)))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-child-of (quote post1) (quote board)) (acl-grant (quote team) (quote comment) (quote board))))))
(do
(acl-it-check!
"group + resource: member on child resource"
(acl-it-p? db (quote alice) (quote comment) (quote post1))
true)
(acl-it-check!
"group + resource: member on parent resource"
(acl-it-p? db (quote alice) (quote comment) (quote board))
true)))
(let
((db (acl-build-db (list (acl-member-of (quote bob) (quote editor)) (acl-role-grant (quote editor) (quote edit) (quote page1)) (acl-role-grant (quote editor) (quote read) (quote page1))))))
(do
(acl-it-check!
"role confers edit to member"
(acl-it-p? db (quote bob) (quote edit) (quote page1))
true)
(acl-it-check!
"role confers read to member"
(acl-it-p? db (quote bob) (quote read) (quote page1))
true)
(acl-it-check!
"role: capability not in tuple denied"
(acl-it-p? db (quote bob) (quote moderate) (quote page1))
false)
(acl-it-check!
"role: non-member excluded"
(acl-it-p? db (quote eve) (quote edit) (quote page1))
false)))
(let
((db (acl-build-db (list (acl-member-of (quote bob) (quote editor)) (acl-child-of (quote draft) (quote page1)) (acl-role-grant (quote editor) (quote edit) (quote page1))))))
(acl-it-check!
"role grant flows to child resource"
(acl-it-p? db (quote bob) (quote edit) (quote draft))
true))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-grant (quote team) (quote read) (quote doc)) (acl-deny (quote alice) (quote read) (quote doc))))))
(acl-it-check!
"explicit deny beats inherited group allow"
(acl-it-p? db (quote alice) (quote read) (quote doc))
false))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-grant (quote alice) (quote read) (quote doc)) (acl-deny (quote team) (quote read) (quote doc))))))
(do
(acl-it-check!
"group deny inherits and overrides direct grant"
(acl-it-p? db (quote alice) (quote read) (quote doc))
false)
(acl-it-check!
"group deny: another member also blocked"
(acl-it-p? db (quote team) (quote read) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-child-of (quote sec) (quote book)) (acl-grant (quote u) (quote read) (quote sec)) (acl-deny (quote u) (quote read) (quote book))))))
(acl-it-check!
"ancestor deny overrides descendant grant"
(acl-it-p? db (quote u) (quote read) (quote sec))
false))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-grant (quote team) (quote read) (quote doc)) (acl-grant (quote team) (quote edit) (quote doc)) (acl-deny (quote alice) (quote edit) (quote doc))))))
(do
(acl-it-check!
"deny on edit leaves inherited read intact"
(acl-it-p? db (quote alice) (quote read) (quote doc))
true)
(acl-it-check!
"deny on edit blocks edit"
(acl-it-p? db (quote alice) (quote edit) (quote doc))
false)))
(let
((db (acl-build-db (list (acl-member-of (quote alice) (quote team)) (acl-deny (quote team) (quote read) (quote doc))))))
(acl-it-check!
"inherited deny, no grant: denied"
(acl-it-p? db (quote alice) (quote read) (quote doc))
false))
(let
((db (acl-build-db (list (acl-child-of (quote a) (quote root)) (acl-child-of (quote b) (quote root)) (acl-grant (quote u) (quote read) (quote root)) (acl-deny (quote u) (quote read) (quote a))))))
(do
(acl-it-check!
"deny on sibling a blocks a"
(acl-it-p? db (quote u) (quote read) (quote a))
false)
(acl-it-check!
"deny on sibling a leaves b permitted"
(acl-it-p? db (quote u) (quote read) (quote b))
true)
(acl-it-check!
"root itself still permitted"
(acl-it-p? db (quote u) (quote read) (quote root))
true)))
(let
((db (acl-build-db (list (acl-grant (quote x) (quote read) (quote y))))))
(acl-it-check!
"direct grant under inheritance ruleset"
(acl-it-p? db (quote x) (quote read) (quote y))
true)))))
(define
acl-inherit-tests-run!
(fn
()
(do
(set! acl-it-pass 0)
(set! acl-it-fail 0)
(set! acl-it-failures (list))
(acl-it-run-all!)
{:failures acl-it-failures :total (+ acl-it-pass acl-it-fail) :passed acl-it-pass :failed acl-it-fail})))

View File

@@ -1,63 +0,0 @@
# APL conformance config — sourced by lib/guest/conformance.sh.
LANG_NAME=apl
MODE=counters
COUNTERS_PASS=apl-test-pass
COUNTERS_FAIL=apl-test-fail
TIMEOUT_PER_SUITE=300
PRELOADS=(
spec/stdlib.sx
lib/r7rs.sx
lib/apl/runtime.sx
lib/apl/tokenizer.sx
lib/apl/parser.sx
lib/apl/transpile.sx
lib/apl/test-harness.sx
)
SUITES=(
"structural:lib/apl/tests/structural.sx"
"operators:lib/apl/tests/operators.sx"
"dfn:lib/apl/tests/dfn.sx"
"tradfn:lib/apl/tests/tradfn.sx"
"valence:lib/apl/tests/valence.sx"
"programs:lib/apl/tests/programs.sx"
"system:lib/apl/tests/system.sx"
"idioms:lib/apl/tests/idioms.sx"
"eval-ops:lib/apl/tests/eval-ops.sx"
"pipeline:lib/apl/tests/pipeline.sx"
)
emit_scoreboard_json() {
local n=${#GC_NAMES[@]} i sep
printf '{\n'
printf ' "suites": {\n'
for ((i=0; i<n; i++)); do
sep=","; [ $i -eq $((n-1)) ] && sep=""
printf ' "%s": {"pass": %d, "fail": %d}%s\n' \
"${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_FAIL[$i]}" "$sep"
done
printf ' },\n'
printf ' "total_pass": %d,\n' "$GC_TOTAL_PASS"
printf ' "total_fail": %d,\n' "$GC_TOTAL_FAIL"
printf ' "total": %d\n' "$GC_TOTAL"
printf '}\n'
}
emit_scoreboard_md() {
local n=${#GC_NAMES[@]} i
printf '# APL Conformance Scoreboard\n\n'
printf '_Generated by `lib/apl/conformance.sh`_\n\n'
printf '| Suite | Pass | Fail | Total |\n'
printf '|-------|-----:|-----:|------:|\n'
for ((i=0; i<n; i++)); do
printf '| %s | %d | %d | %d |\n' \
"${GC_NAMES[$i]}" "${GC_PASS[$i]}" "${GC_FAIL[$i]}" "${GC_TOTAL_S[$i]}"
done
printf '| **Total** | **%d** | **%d** | **%d** |\n' "$GC_TOTAL_PASS" "$GC_TOTAL_FAIL" "$GC_TOTAL"
printf '\n'
printf '## Notes\n\n'
printf '%s\n' '- Suites use the standard `apl-test name got expected` framework loaded against `lib/apl/runtime.sx` + `lib/apl/transpile.sx`.'
printf '%s\n' '- `lib/apl/tests/parse.sx` and `lib/apl/tests/scalar.sx` use their own self-contained frameworks and are excluded from this scoreboard.'
}

View File

@@ -1,5 +1,116 @@
#!/usr/bin/env bash
# lib/apl/conformance.sh — APL conformance via the shared guest driver.
# Config lives in lib/apl/conformance.conf (MODE=counters). Override the binary
# with SX_SERVER=path/to/sx_server.exe bash lib/apl/conformance.sh
exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"
# lib/apl/conformance.sh — run APL test suites, emit scoreboard.json + scoreboard.md.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
SUITES=(structural operators dfn tradfn valence programs system idioms eval-ops pipeline)
OUT_JSON="lib/apl/scoreboard.json"
OUT_MD="lib/apl/scoreboard.md"
run_suite() {
local suite=$1
local file="lib/apl/tests/${suite}.sx"
local TMP
TMP=$(mktemp)
cat > "$TMP" << EPOCHS
(epoch 1)
(load "spec/stdlib.sx")
(load "lib/r7rs.sx")
(load "lib/apl/runtime.sx")
(load "lib/apl/tokenizer.sx")
(load "lib/apl/parser.sx")
(load "lib/apl/transpile.sx")
(epoch 2)
(eval "(define apl-test-pass 0)")
(eval "(define apl-test-fail 0)")
(eval "(define apl-test (fn (name got expected) (if (= got expected) (set! apl-test-pass (+ apl-test-pass 1)) (set! apl-test-fail (+ apl-test-fail 1)))))")
(epoch 3)
(load "${file}")
(epoch 4)
(eval "(list apl-test-pass apl-test-fail)")
EPOCHS
local OUTPUT
OUTPUT=$(timeout 300 "$SX_SERVER" < "$TMP" 2>/dev/null)
rm -f "$TMP"
local LINE
LINE=$(echo "$OUTPUT" | awk '/^\(ok-len 4 / {getline; print; exit}')
if [ -z "$LINE" ]; then
LINE=$(echo "$OUTPUT" | grep -E '^\(ok 4 \([0-9]+ [0-9]+\)\)' | tail -1 \
| sed -E 's/^\(ok 4 //; s/\)$//')
fi
local P F
P=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\1/')
F=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\2/')
P=${P:-0}
F=${F:-0}
echo "${P} ${F}"
}
declare -A SUITE_PASS
declare -A SUITE_FAIL
TOTAL_PASS=0
TOTAL_FAIL=0
echo "Running APL conformance suite..." >&2
for s in "${SUITES[@]}"; do
read -r p f < <(run_suite "$s")
SUITE_PASS[$s]=$p
SUITE_FAIL[$s]=$f
TOTAL_PASS=$((TOTAL_PASS + p))
TOTAL_FAIL=$((TOTAL_FAIL + f))
printf " %-12s %d/%d\n" "$s" "$p" "$((p+f))" >&2
done
# scoreboard.json
{
printf '{\n'
printf ' "suites": {\n'
first=1
for s in "${SUITES[@]}"; do
if [ $first -eq 0 ]; then printf ',\n'; fi
printf ' "%s": {"pass": %d, "fail": %d}' "$s" "${SUITE_PASS[$s]}" "${SUITE_FAIL[$s]}"
first=0
done
printf '\n },\n'
printf ' "total_pass": %d,\n' "$TOTAL_PASS"
printf ' "total_fail": %d,\n' "$TOTAL_FAIL"
printf ' "total": %d\n' "$((TOTAL_PASS + TOTAL_FAIL))"
printf '}\n'
} > "$OUT_JSON"
# scoreboard.md
{
printf '# APL Conformance Scoreboard\n\n'
printf '_Generated by `lib/apl/conformance.sh`_\n\n'
printf '| Suite | Pass | Fail | Total |\n'
printf '|-------|-----:|-----:|------:|\n'
for s in "${SUITES[@]}"; do
p=${SUITE_PASS[$s]}
f=${SUITE_FAIL[$s]}
printf '| %s | %d | %d | %d |\n' "$s" "$p" "$f" "$((p+f))"
done
printf '| **Total** | **%d** | **%d** | **%d** |\n' "$TOTAL_PASS" "$TOTAL_FAIL" "$((TOTAL_PASS + TOTAL_FAIL))"
printf '\n'
printf '## Notes\n\n'
printf '%s\n' '- Suites use the standard `apl-test name got expected` framework loaded against `lib/apl/runtime.sx` + `lib/apl/transpile.sx`.'
printf '%s\n' '- `lib/apl/tests/parse.sx` and `lib/apl/tests/scalar.sx` use their own self-contained frameworks and are excluded from this scoreboard.'
} > "$OUT_MD"
echo "Wrote $OUT_JSON and $OUT_MD" >&2
echo "Total: $TOTAL_PASS pass, $TOTAL_FAIL fail" >&2
[ "$TOTAL_FAIL" -eq 0 ]

View File

@@ -9,9 +9,9 @@
"system": {"pass": 13, "fail": 0},
"idioms": {"pass": 64, "fail": 0},
"eval-ops": {"pass": 14, "fail": 0},
"pipeline": {"pass": 152, "fail": 0}
"pipeline": {"pass": 40, "fail": 0}
},
"total_pass": 562,
"total_pass": 450,
"total_fail": 0,
"total": 562
"total": 450
}

View File

@@ -13,8 +13,8 @@ _Generated by `lib/apl/conformance.sh`_
| system | 13 | 0 | 13 |
| idioms | 64 | 0 | 64 |
| eval-ops | 14 | 0 | 14 |
| pipeline | 152 | 0 | 152 |
| **Total** | **562** | **0** | **562** |
| pipeline | 40 | 0 | 40 |
| **Total** | **450** | **0** | **450** |
## Notes

View File

@@ -1,15 +0,0 @@
; lib/apl/test-harness.sx — counters + assertion fn for the shared conformance
; driver (lib/guest/conformance.sh, MODE=counters). Loaded as a PRELOAD so each
; suite starts from a fresh 0/0; suites call (apl-test name got expected).
(define apl-test-pass 0)
(define apl-test-fail 0)
(define
apl-test
(fn
(name got expected)
(if
(= got expected)
(set! apl-test-pass (+ apl-test-pass 1))
(set! apl-test-fail (+ apl-test-fail 1)))))

View File

@@ -1,38 +0,0 @@
; feed/acl — per-viewer visibility filtering. The same candidate stream yields
; different timelines for different viewers, so ACL is applied per request and
; pre-ACL timelines are never cached.
;
; permit? is injected: (permit? viewer activity) -> bool. Wire a real acl-sx
; predicate here; feed/permit-acl? is a self-contained default that reads an
; optional :visible-to allowlist on the activity.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem?), lib/feed/rank.sx (feed/top).
; default permit: actor always sees own activity; absent/nil :visible-to is
; public; otherwise viewer must be in the allowlist.
(define
feed/permit-acl?
(fn
(viewer a)
(or
(equal? viewer (get a :actor))
(let
((allowed (get a :visible-to nil)))
(if (= allowed nil) true (feed/-elem? viewer allowed))))))
(define feed/permit-public? (fn (viewer a) true))
; filter a stream to what viewer may read
(define
feed/visible
(fn
(stream viewer permit?)
(feed/filter stream (fn (a) (permit? viewer a)))))
; the capstone: candidate stream -> ACL for viewer -> rank -> top-N
(define
feed/timeline
(fn
(stream viewer permit? score-fn n)
(feed/top (feed/visible stream viewer permit?) score-fn n)))

View File

@@ -1,62 +0,0 @@
; feed/aggregate — group-by / counting via key-reduce. Keys must be strings
; (dict keys), so composite keys (actor, day) are joined into one string.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx.
; group activities into a dict: key-string -> (list of activities), order-preserving
(define
feed/group-by
(fn
(stream key-fn)
(reduce
(fn
(g a)
(let
((k (key-fn a)))
(assoc g k (append (get g k (list)) (list a)))))
{}
(feed/items stream))))
; key-string -> count
(define
feed/group-count
(fn
(stream key-fn)
(reduce
(fn
(g a)
(let
((k (key-fn a)))
(assoc g k (+ (get g k 0) 1))))
{}
(feed/items stream))))
; --- composite keys ---------------------------------------------------------
(define feed/day (fn (at window) (floor (/ at window))))
; (actor, day-bucket) -> "actor#day"
(define
feed/actor-day-key
(fn
(window)
(fn
(a)
(string-append
(get a :actor)
"#"
(number->string (feed/day (get a :at) window))))))
(define
feed/by-actor-day
(fn (stream window) (feed/group-count stream (feed/actor-day-key window))))
; per-actor activity counts
(define
feed/actor-counts
(fn (stream) (feed/group-count stream feed/actor)))
; per-object activity counts (engagement)
(define
feed/object-counts
(fn (stream) (feed/group-count stream feed/object)))

View File

@@ -1,24 +0,0 @@
; feed/api — ergonomic API over the stream layer for non-APL callers.
; A single mutable activity log; post appends, all returns it as a stream.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx (loaded by harness).
(define feed/-log (list))
; post — normalize then append. Returns the stored activity.
(define
feed/post
(fn
(raw)
(let
((a (feed/normalize raw)))
(begin (set! feed/-log (append feed/-log (list a))) a))))
; all — the whole log as a stream (insertion order)
(define feed/all (fn () (feed/stream feed/-log)))
; reset! — clear the log (test hygiene)
(define feed/reset! (fn () (begin (set! feed/-log (list)) nil)))
; size — number of posted activities
(define feed/size (fn () (len feed/-log)))

View File

@@ -1,125 +0,0 @@
#!/usr/bin/env bash
# lib/feed/conformance.sh — run feed test suites, emit scoreboard.json + scoreboard.md.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
SUITES=(basic fanout rank integration content notify home dedupe trending mute page thread)
OUT_JSON="lib/feed/scoreboard.json"
OUT_MD="lib/feed/scoreboard.md"
run_suite() {
local suite=$1
local file="lib/feed/tests/${suite}.sx"
local TMP
TMP=$(mktemp)
cat > "$TMP" << EPOCHS
(epoch 1)
(load "spec/stdlib.sx")
(load "lib/r7rs.sx")
(load "lib/apl/runtime.sx")
(load "lib/feed/normalize.sx")
(load "lib/feed/stream.sx")
(load "lib/feed/api.sx")
(load "lib/feed/fanout.sx")
(load "lib/feed/dedupe.sx")
(load "lib/feed/aggregate.sx")
(load "lib/feed/rank.sx")
(load "lib/feed/acl.sx")
(load "lib/feed/fed.sx")
(load "lib/feed/content.sx")
(load "lib/feed/notify.sx")
(load "lib/feed/home.sx")
(load "lib/feed/trending.sx")
(load "lib/feed/mute.sx")
(load "lib/feed/page.sx")
(load "lib/feed/thread.sx")
(epoch 2)
(eval "(define feed-test-pass 0)")
(eval "(define feed-test-fail 0)")
(eval "(define feed-test (fn (name got expected) (if (= got expected) (set! feed-test-pass (+ feed-test-pass 1)) (set! feed-test-fail (+ feed-test-fail 1)))))")
(epoch 3)
(load "${file}")
(epoch 4)
(eval "(list feed-test-pass feed-test-fail)")
EPOCHS
local OUTPUT
OUTPUT=$(timeout 300 "$SX_SERVER" < "$TMP" 2>/dev/null)
rm -f "$TMP"
local LINE
LINE=$(echo "$OUTPUT" | awk '/^\(ok-len 4 / {getline; print; exit}')
if [ -z "$LINE" ]; then
LINE=$(echo "$OUTPUT" | grep -E '^\(ok 4 \([0-9]+ [0-9]+\)\)' | tail -1 \
| sed -E 's/^\(ok 4 //; s/\)$//')
fi
local P F
P=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\1/')
F=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\2/')
P=${P:-0}
F=${F:-0}
echo "${P} ${F}"
}
declare -A SUITE_PASS
declare -A SUITE_FAIL
TOTAL_PASS=0
TOTAL_FAIL=0
echo "Running feed conformance suite..." >&2
for s in "${SUITES[@]}"; do
read -r p f < <(run_suite "$s")
SUITE_PASS[$s]=$p
SUITE_FAIL[$s]=$f
TOTAL_PASS=$((TOTAL_PASS + p))
TOTAL_FAIL=$((TOTAL_FAIL + f))
printf " %-12s %d/%d\n" "$s" "$p" "$((p+f))" >&2
done
# scoreboard.json
{
printf '{\n'
printf ' "suites": {\n'
first=1
for s in "${SUITES[@]}"; do
if [ $first -eq 0 ]; then printf ',\n'; fi
printf ' "%s": {"pass": %d, "fail": %d}' "$s" "${SUITE_PASS[$s]}" "${SUITE_FAIL[$s]}"
first=0
done
printf '\n },\n'
printf ' "total_pass": %d,\n' "$TOTAL_PASS"
printf ' "total_fail": %d,\n' "$TOTAL_FAIL"
printf ' "total": %d\n' "$((TOTAL_PASS + TOTAL_FAIL))"
printf '}\n'
} > "$OUT_JSON"
# scoreboard.md
{
printf '# feed Conformance Scoreboard\n\n'
printf '_Generated by `lib/feed/conformance.sh`_\n\n'
printf '| Suite | Pass | Fail | Total |\n'
printf '|-------|-----:|-----:|------:|\n'
for s in "${SUITES[@]}"; do
p=${SUITE_PASS[$s]}
f=${SUITE_FAIL[$s]}
printf '| %s | %d | %d | %d |\n' "$s" "$p" "$f" "$((p+f))"
done
printf '| **Total** | **%d** | **%d** | **%d** |\n' "$TOTAL_PASS" "$TOTAL_FAIL" "$((TOTAL_PASS + TOTAL_FAIL))"
} > "$OUT_MD"
echo "Wrote $OUT_JSON and $OUT_MD" >&2
echo "Total: $TOTAL_PASS pass, $TOTAL_FAIL fail" >&2
[ "$TOTAL_FAIL" -eq 0 ]

View File

@@ -1,68 +0,0 @@
; feed/content — TF-IDF relevance over activity :tags. Rare tags carry more
; signal, so an activity matching an uncommon tag ranks above one matching a
; common tag. Composes with rank.sx: feed/tfidf-score is just another scorer.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-distinct), lib/feed/rank.sx (feed/rank).
; document frequency: tag -> number of activities whose :tags contain it
; (a tag repeated within one activity counts once toward df)
(define
feed/tag-df
(fn
(stream)
(reduce
(fn
(df a)
(reduce
(fn (d t) (assoc d t (+ (get d t 0) 1)))
df
(feed/-distinct (get a :tags))))
{}
(feed/items stream))))
; inverse document frequency: tag -> log(N / df)
(define
feed/tag-idf
(fn
(stream)
(let
((n (feed/count stream)) (df (feed/tag-df stream)))
(reduce
(fn (idf t) (assoc idf t (log (/ n (get df t)))))
{}
(keys df)))))
; term frequency within one activity: tag -> occurrence count
(define
feed/-tf
(fn
(a)
(reduce
(fn (tf t) (assoc tf t (+ (get tf t 0) 1)))
{}
(get a :tags))))
; relevance of an activity to a query (list of tags) given precomputed idf:
; sum over query tags of tf(tag in activity) * idf(tag in corpus)
(define
feed/tfidf-score
(fn
(idf query)
(fn
(a)
(let
((tf (feed/-tf a)))
(reduce
(fn
(acc t)
(+ acc (* (get tf t 0) (get idf t 0))))
0
query)))))
; rank a stream by relevance to query tags (idf computed over the stream itself)
(define
feed/by-relevance
(fn
(stream query)
(feed/rank stream (feed/tfidf-score (feed/tag-idf stream) query))))

View File

@@ -1,76 +0,0 @@
; feed/dedupe — collapse duplicate items, keeping first occurrence per key.
; Each verb may want its own key (see briefing): "alice posted X" keys on
; (actor verb object) — distinct per actor; "alice liked X / bob liked X"
; collapse on (verb object) so the cross-actor likes fold into one.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem? lives in fanout.sx).
; generic: dedupe a stream by key-fn, first occurrence wins (stable)
(define
feed/-dedup-by
(fn
(items key-fn)
(get
(reduce
(fn
(st x)
(let
((k (key-fn x)))
(if (feed/-elem? k (get st :seen)) st {:seen (append (get st :seen) (list k)) :out (append (get st :out) (list x))})))
{:seen (list) :out (list)}
items)
:out)))
(define
feed/dedupe
(fn
(stream key-fn)
(feed/stream (feed/-dedup-by (feed/items stream) key-fn))))
; --- keys -------------------------------------------------------------------
(define
feed/activity-key
(fn (a) (list (get a :actor) (get a :verb) (get a :object))))
; collapse cross-actor duplicates of the same verb+object (e.g. likes)
(define feed/collapse-key (fn (a) (list (get a :verb) (get a :object))))
; per-receiver inbox key — one inbox event per (receiver, actor, verb, object)
(define
feed/event-key
(fn
(ev)
(let
((a (get ev :activity)))
(list (get ev :to) (get a :actor) (get a :verb) (get a :object)))))
; verbs whose duplicates collapse across actors (reactions, not authorship).
; rebindable: callers can (set! feed/collapse-verbs ...) to tune the policy.
(define
feed/collapse-verbs
(list "like" "favourite" "follow" "boost" "repost"))
; per-verb key: collapse-verbs fold on (verb object); the rest key on
; (actor verb object).
(define
feed/smart-key
(fn
(a)
(if
(feed/-elem? (get a :verb) feed/collapse-verbs)
(feed/collapse-key a)
(feed/activity-key a))))
; --- ready-made dedupers ----------------------------------------------------
(define feed/dedupe-activities (fn (s) (feed/dedupe s feed/activity-key)))
(define feed/dedupe-collapse (fn (s) (feed/dedupe s feed/collapse-key)))
; verb-aware: reactions collapse cross-actor, posts stay distinct per actor
(define feed/dedupe-smart (fn (s) (feed/dedupe s feed/smart-key)))
; dedupe an inbox: at most one event per receiver per (actor verb object)
(define feed/dedupe-inbox (fn (inbox) (feed/dedupe inbox feed/event-key)))

View File

@@ -1,114 +0,0 @@
; feed/fanout — THE SHOWCASE. Fan activities out to followers via the APL outer
; product (∘.×). activities ∘.× audience → an (activity × follower) matrix of
; inbox events; flatten to a vector; guard-keep only real follow edges.
;
; Requires: lib/apl/runtime.sx, lib/feed/normalize.sx, lib/feed/stream.sx.
;
; NOTE: apl-outer's combiner result is run through (if (scalar? r) (disclose r) r).
; A bare dict counts as a scalar (shape ()) and disclose nils it — so the combiner
; must (enclose ...) its event dict; apl-outer then discloses it back intact.
; --- graph: {followee -> (list of followers)} -------------------------------
(define feed/followers (fn (graph user) (get graph user (list))))
; build a graph from (follower followee) edges: "follower follows followee"
(define
feed/follow-graph
(fn
(edges)
(reduce
(fn
(g e)
(let
((follower (first e)) (followee (nth e 1)))
(assoc
g
followee
(append (feed/followers g followee) (list follower)))))
{}
edges)))
; --- helpers ----------------------------------------------------------------
; unwrap an apl-scalar (has :ravel) back to its value; pass activities through
(define
feed/-val
(fn
(x)
(if (and (= (type-of x) "dict") (has-key? x :ravel)) (disclose x) x)))
(define feed/-elem? (fn (x lst) (some (fn (y) (equal? x y)) lst)))
(define
feed/-distinct
(fn
(lst)
(if
(= (len lst) 0)
(list)
(get (apl-unique (make-array (list (len lst)) lst)) :ravel))))
; rank-2 matrix -> rank-1 stream of its ravel
(define feed/-flatten (fn (arr) (feed/stream (get arr :ravel))))
; distinct receivers across the whole graph, sorted for determinism
; (dict key order is unspecified, so sort to pin audience/recipient ordering)
(define
feed/audience
(fn
(graph)
(sort
(feed/-distinct
(reduce
(fn (acc k) (append acc (feed/followers graph k)))
(list)
(keys graph))))))
; --- the outer product ------------------------------------------------------
; one (activity, follower) inbox event, enclosed so apl-outer keeps the dict
(define feed/-mk-event (fn (a f) (enclose {:activity (feed/-val a) :to (feed/-val f)})))
; keep events where :to actually follows the activity's actor
(define
feed/-edge?
(fn
(graph)
(fn
(ev)
(feed/-elem?
(get ev :to)
(feed/followers graph (get (get ev :activity) :actor))))))
; fanout — activities ∘.× audience, flatten, guard-keep real edges
(define
feed/fanout
(fn
(stream graph)
(let
((matrix (apl-outer feed/-mk-event stream (feed/stream (feed/audience graph)))))
(feed/filter (feed/-flatten matrix) (feed/-edge? graph)))))
; --- inbox queries ----------------------------------------------------------
(define
feed/inbox-for
(fn
(inbox user)
(feed/filter inbox (fn (ev) (equal? (get ev :to) user)))))
(define
feed/recipients
(fn
(inbox)
(feed/-distinct (map (fn (ev) (get ev :to)) (feed/items inbox)))))
; the activities (unwrapped) destined for a user
(define
feed/inbox-activities
(fn
(inbox user)
(map
(fn (ev) (get ev :activity))
(feed/items (feed/inbox-for inbox user)))))

View File

@@ -1,60 +0,0 @@
; feed/fed — federation. Outbound: a local post fans out, then splits into local
; vs remote inboxes; remote events are handed to an injected send-fn. Inbound:
; peer activities merge into the local stream, deduped. Backfill: pull peer
; history via an injected fetch-fn and merge.
;
; remote? / send-fn / fetch-fn are injected so real fed-sx transport wires in here
; without feed depending on it.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx,
; lib/feed/dedupe.sx.
; --- merge / ingest ---------------------------------------------------------
(define
feed/merge
(fn (s1 s2) (feed/stream (append (feed/items s1) (feed/items s2)))))
; merge a peer stream into local, dropping (actor verb object) duplicates
(define
feed/ingest
(fn (local peer) (feed/dedupe-activities (feed/merge local peer))))
; --- inbound ----------------------------------------------------------------
; peer pushes raw activities to the local inbox; normalize + ingest
(define
feed/inbound
(fn
(local raw-activities)
(feed/ingest local (feed/stream (map feed/normalize raw-activities)))))
; backfill on subscribe: pull peer history via fetch-fn, normalize, ingest
(define
feed/backfill
(fn (local fetch-fn peer-id) (feed/inbound local (fetch-fn peer-id))))
; --- outbound ---------------------------------------------------------------
; split an inbox into local vs remote deliveries by viewer-id predicate
(define feed/partition-inbox (fn (inbox remote?) {:local (feed/filter inbox (fn (ev) (not (remote? (get ev :to))))) :remote (feed/filter inbox (fn (ev) (remote? (get ev :to))))}))
; fan a stream out over the graph, then partition by locality
(define
feed/federate
(fn
(stream graph remote?)
(feed/partition-inbox (feed/fanout stream graph) remote?)))
; deliver: hand each remote event to send-fn, return the local inbox to enqueue
(define
feed/deliver
(fn
(stream graph remote? send-fn)
(let
((parts (feed/federate stream graph remote?)))
(begin
(for-each
(fn (ev) (send-fn (get ev :to) (get ev :activity)))
(feed/items (get parts :remote)))
(get parts :local)))))

View File

@@ -1,23 +0,0 @@
; feed/home — the capstone. A user's home timeline is the whole pipeline as one
; line: fan all activities out over the follow graph, take the events landing in
; the viewer's inbox, dedupe cross-posts, apply the viewer's ACL, rank, take N.
;
; Requires: fanout.sx, dedupe.sx, acl.sx (feed/timeline), rank.sx, stream.sx.
; the activities in a user's inbox, as a stream
(define
feed/inbox-stream
(fn (inbox user) (feed/stream (feed/inbox-activities inbox user))))
; fanout ∘ inbox ∘ dedupe ∘ ACL ∘ rank ∘ take
(define
feed/home
(fn
(stream graph viewer permit? score-fn n)
(feed/timeline
(feed/dedupe-activities
(feed/inbox-stream (feed/fanout stream graph) viewer))
viewer
permit?
score-fn
n)))

View File

@@ -1,44 +0,0 @@
; feed/mute — viewer-controlled filtering. ACL (acl.sx) is author-controlled
; visibility; mute is the reader's own preference: hide muted actors or tags.
; Like ACL it is per-viewer and applied per request, never cached.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem?).
; drop activities authored by a muted actor
(define
feed/mute-actors
(fn
(stream actors)
(feed/filter
stream
(fn (a) (not (feed/-elem? (get a :actor) actors))))))
; drop activities carrying any muted tag
(define
feed/mute-tags
(fn
(stream tags)
(feed/filter
stream
(fn (a) (not (some (fn (t) (feed/-elem? t tags)) (get a :tags)))))))
; drop activities about a muted object (thread mute)
(define
feed/mute-objects
(fn
(stream objects)
(feed/filter
stream
(fn (a) (not (feed/-elem? (get a :object) objects))))))
; apply a viewer preference bag: {:mute-actors (...) :mute-tags (...) :mute-objects (...)}
(define
feed/apply-prefs
(fn
(stream prefs)
(feed/mute-objects
(feed/mute-tags
(feed/mute-actors stream (get prefs :mute-actors (list)))
(get prefs :mute-tags (list)))
(get prefs :mute-objects (list)))))

View File

@@ -1,31 +0,0 @@
; feed/normalize — coerce arbitrary input into the canonical activity record.
; An activity is a small dict {:actor :verb :object :at :tags}; a stream is an
; APL vector of such dicts (see stream.sx). Extra keys on the raw input survive
; (e.g. :visible-to for ACL, peer metadata for federation) — :tags is the
; flexible bag but the record is not closed.
(define feed/activity-keys (list :actor :verb :object :at :tags))
(define
feed/normalize
(fn
(raw)
(let
((d (if (= (type-of raw) "dict") raw {})))
(merge d {:actor (get d :actor "") :object (get d :object nil) :at (get d :at 0) :tags (let ((t (get d :tags (list)))) (if (list? t) t (list t))) :verb (get d :verb "post")}))))
(define
feed/activity
(fn (actor verb object at tags) (feed/normalize {:actor actor :object object :at at :tags tags :verb verb})))
(define feed/actor (fn (a) (get a :actor)))
(define feed/verb (fn (a) (get a :verb)))
(define feed/object (fn (a) (get a :object)))
(define feed/at (fn (a) (get a :at)))
(define feed/tags (fn (a) (get a :tags)))
(define
feed/activity?
(fn
(a)
(and (= (type-of a) "dict") (has-key? a :actor) (has-key? a :verb))))

View File

@@ -1,45 +0,0 @@
; feed/notify — a notification feed is a thin layer over a recipient's inbox:
; the events directed at a user, optionally verb-filtered, and a digest that
; collapses "alice, bob and 1 other liked X" by (verb, object).
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/inbox-for, feed/-elem?).
; all inbox events for a user (their raw notifications)
(define feed/notifications (fn (inbox user) (feed/inbox-for inbox user)))
; restrict to notification-worthy verbs (e.g. (list "like" "reply" "follow"))
(define
feed/notify-verbs
(fn
(inbox user verbs)
(feed/filter
(feed/inbox-for inbox user)
(fn (ev) (feed/-elem? (get (get ev :activity) :verb) verbs)))))
; group key "verb|object" — deterministic, sortable
(define
feed/-notify-key
(fn
(ev)
(let
((a (get ev :activity)))
(string-append (get a :verb) "|" (get a :object)))))
; digest: one entry per (verb, object) with the distinct actors and a count,
; ordered by key for determinism.
(define
feed/notify-digest
(fn
(inbox user)
(let
((events (feed/items (feed/inbox-for inbox user))))
(let
((groups (reduce (fn (g ev) (let ((a (get ev :activity)) (k (feed/-notify-key ev))) (let ((cur (get g k {:object (get a :object) :actors (list) :verb (get a :verb)}))) (assoc g k (assoc cur :actors (append (get cur :actors) (list (get a :actor)))))))) {} events)))
(map
(fn
(k)
(let
((grp (get groups k)))
(assoc grp :count (len (get grp :actors)))))
(sort (keys groups)))))))

View File

@@ -1,50 +0,0 @@
; feed/page — pagination. Offset/limit for indexed access, and cursor-based
; (by :at) for recency feeds, which is stable under inserts: a cursor is the
; :at of the last item seen, and the next page is the newest items older than it.
;
; Requires: lib/feed/stream.sx (feed/recent, feed/take, feed/filter).
; --- offset / limit ---------------------------------------------------------
(define
feed/page
(fn
(stream offset limit)
(feed/stream (take (drop (feed/items stream) offset) limit))))
(define
feed/page-count
(fn (stream limit) (ceil (/ (feed/count stream) limit))))
; --- cursor (recency feeds) -------------------------------------------------
; activities strictly older than cursor (scroll down / load older)
(define
feed/before
(fn
(stream cursor)
(feed/filter stream (fn (a) (< (get a :at) cursor)))))
; activities strictly newer than cursor (load newer / "N new posts")
(define
feed/after
(fn
(stream cursor)
(feed/filter stream (fn (a) (> (get a :at) cursor)))))
; one page: the `limit` newest activities older than cursor, newest first
(define
feed/page-before
(fn
(stream cursor limit)
(feed/take (feed/recent (feed/before stream cursor)) limit)))
; cursor to fetch the next (older) page: :at of the last item of a page,
; or nil when the page is empty (end of feed)
(define
feed/next-cursor
(fn
(page)
(let
((items (feed/items page)))
(if (= (len items) 0) nil (get (last items) :at)))))

View File

@@ -1,92 +0,0 @@
; feed/rank — scoring + ranking. Scorers are (activity -> number). Ranking is a
; stable two-pass grade-down: first by :at descending (the tiebreak), then by
; score descending — so ties resolve by recency, then by input order. Fully
; deterministic on ties.
;
; Requires: lib/apl/runtime.sx, lib/feed/normalize.sx, lib/feed/stream.sx.
; --- scorers ----------------------------------------------------------------
; recency: half-life decay. score = 0.5 ^ (age / half-life). at==now -> 1.0.
(define
feed/recency
(fn
(now half-life)
(fn (a) (expt 0.5 (/ (- now (get a :at)) half-life)))))
; velocity: how many of this actor's activities fall in (at-window, at] —
; a burst of recent activity scores higher.
(define
feed/velocity
(fn
(stream window)
(fn
(a)
(len
(filter
(fn
(b)
(and
(equal? (get b :actor) (get a :actor))
(<= (get b :at) (get a :at))
(> (get b :at) (- (get a :at) window))))
(feed/items stream))))))
; engagement: how many activities in the stream touch this activity's :object
(define
feed/engagement
(fn
(stream)
(fn
(a)
(len
(filter
(fn (b) (equal? (get b :object) (get a :object)))
(feed/items stream))))))
; composite: weighted sum. parts = (list (list weight scorer) ...)
(define
feed/composite
(fn
(parts)
(fn
(a)
(reduce
(fn (acc p) (+ acc (* (first p) ((nth p 1) a))))
0
parts))))
; --- ranking ----------------------------------------------------------------
; stable reorder of items by key-fn, descending (grade-down is stable)
(define
feed/-desc-by
(fn
(items key-fn)
(let
((keys (make-array (list (len items)) (map key-fn items))))
(let
((order (get (apl-grade-down keys) :ravel)))
(map (fn (i) (nth items (- i 1))) order)))))
; rank by score descending; ties -> :at descending -> input order
(define
feed/rank
(fn
(stream score-fn)
(let
((by-at (feed/-desc-by (feed/items stream) feed/at)))
(feed/stream (feed/-desc-by by-at score-fn)))))
; attach a :score to each activity (for inspection / debugging)
(define
feed/with-scores
(fn
(stream score-fn)
(feed/stream
(map (fn (a) (assoc a :score (score-fn a))) (feed/items stream)))))
; top-N ranked timeline
(define
feed/top
(fn (stream score-fn n) (feed/take (feed/rank stream score-fn) n)))

View File

@@ -1,19 +0,0 @@
{
"suites": {
"basic": {"pass": 30, "fail": 0},
"fanout": {"pass": 29, "fail": 0},
"rank": {"pass": 24, "fail": 0},
"integration": {"pass": 22, "fail": 0},
"content": {"pass": 15, "fail": 0},
"notify": {"pass": 8, "fail": 0},
"home": {"pass": 6, "fail": 0},
"dedupe": {"pass": 9, "fail": 0},
"trending": {"pass": 11, "fail": 0},
"mute": {"pass": 9, "fail": 0},
"page": {"pass": 14, "fail": 0},
"thread": {"pass": 12, "fail": 0}
},
"total_pass": 189,
"total_fail": 0,
"total": 189
}

View File

@@ -1,19 +0,0 @@
# feed Conformance Scoreboard
_Generated by `lib/feed/conformance.sh`_
| Suite | Pass | Fail | Total |
|-------|-----:|-----:|------:|
| basic | 30 | 0 | 30 |
| fanout | 29 | 0 | 29 |
| rank | 24 | 0 | 24 |
| integration | 22 | 0 | 22 |
| content | 15 | 0 | 15 |
| notify | 8 | 0 | 8 |
| home | 6 | 0 | 6 |
| dedupe | 9 | 0 | 9 |
| trending | 11 | 0 | 11 |
| mute | 9 | 0 | 9 |
| page | 14 | 0 | 14 |
| thread | 12 | 0 | 12 |
| **Total** | **189** | **0** | **189** |

View File

@@ -1,75 +0,0 @@
; feed/stream — a stream is an APL vector (rank-1 array) whose ravel holds
; activity dicts. Operations lift APL primitives onto this shape: filter via
; compress (/), sort via grade (⍋), take via ↑, reverse via ⌽.
;
; Requires: lib/apl/runtime.sx, lib/feed/normalize.sx (loaded by harness).
(define feed/stream (fn (acts) (make-array (list (len acts)) acts)))
(define feed/items (fn (s) (get s :ravel)))
(define feed/count (fn (s) (len (get s :ravel))))
(define feed/empty (feed/stream (list)))
(define feed/empty? (fn (s) (= (feed/count s) 0)))
; filter — bool mask ∘ compress. pred : activity -> truthy
(define
feed/filter
(fn
(s pred)
(let
((items (get s :ravel)))
(let
((mask (make-array (list (len items)) (map (fn (a) (if (pred a) 1 0)) items))))
(apl-compress mask s)))))
; sort-by — ascending, stable on ties (grade-up is stable). key-fn : activity -> number
(define
feed/sort-by
(fn
(s key-fn)
(let
((items (get s :ravel)))
(let
((keys (make-array (list (len items)) (map key-fn items))))
(let
((order (get (apl-grade-up keys) :ravel)))
(feed/stream (map (fn (i) (nth items (- i 1))) order)))))))
(define feed/sort-by-at (fn (s) (feed/sort-by s feed/at)))
; newest-first: ascending sort then reverse (⌽)
(define feed/recent (fn (s) (apl-reverse (feed/sort-by-at s))))
; take N (↑), clamped to stream length so it never over-takes/pads
(define
feed/take
(fn
(s n)
(let
((c (feed/count s)))
(if (>= n c) s (apl-take (apl-scalar n) s)))))
(define feed/reverse (fn (s) (apl-reverse s)))
; common predicates
(define
feed/by-actor
(fn (s actor) (feed/filter s (fn (a) (equal? (get a :actor) actor)))))
(define
feed/by-verb
(fn (s verb) (feed/filter s (fn (a) (equal? (get a :verb) verb)))))
(define
feed/by-object
(fn
(s object)
(feed/filter s (fn (a) (equal? (get a :object) object)))))
; activities at or after timestamp t
(define
feed/since
(fn (s t) (feed/filter s (fn (a) (>= (get a :at) t)))))

View File

@@ -1,118 +0,0 @@
; Phase 1 — normalize, stream ops, api. Uses the feed-test harness
; (feed-test name got expected) provided by conformance.sh.
; ---------- normalize ----------
(feed-test
"normalize default actor"
(feed/actor (feed/normalize {}))
"")
(feed-test
"normalize default verb"
(feed/verb (feed/normalize {}))
"post")
(feed-test
"normalize default at"
(feed/at (feed/normalize {}))
0)
(feed-test
"normalize default object"
(feed/object (feed/normalize {}))
nil)
(feed-test
"normalize default tags"
(feed/tags (feed/normalize {}))
(list))
(feed-test
"normalize keeps actor"
(feed/actor (feed/normalize {:actor "alice"}))
"alice")
(feed-test
"normalize keeps verb"
(feed/verb (feed/normalize {:verb "like"}))
"like")
(feed-test
"normalize scalar tag -> list"
(feed/tags (feed/normalize {:tags "x"}))
(list "x"))
(feed-test
"normalize list tags kept"
(feed/tags (feed/normalize {:tags (list "a" "b")}))
(list "a" "b"))
(feed-test
"activity constructor at"
(feed/at (feed/activity "a" "post" "o" 5 (list)))
5)
(feed-test
"activity? on activity"
(feed/activity? (feed/normalize {:actor "a"}))
true)
(feed-test "activity? on number" (feed/activity? 5) false)
(feed-test "activity? on bare dict" (feed/activity? {:foo 1}) false)
; ---------- stream ----------
(define
S
(feed/stream
(list
(feed/activity "alice" "post" "p1" 30 (list))
(feed/activity "bob" "like" "p1" 10 (list))
(feed/activity "alice" "post" "p2" 20 (list)))))
(feed-test "stream count" (feed/count S) 3)
(feed-test "stream items len" (len (feed/items S)) 3)
(feed-test
"sort-by-at actors asc"
(map feed/actor (feed/items (feed/sort-by-at S)))
(list "bob" "alice" "alice"))
(feed-test
"recent newest first"
(map feed/at (feed/items (feed/recent S)))
(list 30 20 10))
(feed-test
"take 2 of recent"
(feed/count (feed/take (feed/recent S) 2))
2)
(feed-test
"take clamps past end"
(feed/count (feed/take S 10))
3)
(feed-test
"by-actor alice count"
(feed/count (feed/by-actor S "alice"))
2)
(feed-test
"by-verb like actor"
(map feed/actor (feed/items (feed/by-verb S "like")))
(list "bob"))
(feed-test
"by-object p1 count"
(feed/count (feed/by-object S "p1"))
2)
(feed-test
"since 20 count"
(feed/count (feed/since S 20))
2)
(feed-test
"reverse ats"
(map feed/at (feed/items (feed/reverse S)))
(list 20 10 30))
(feed-test "empty? on empty" (feed/empty? feed/empty) true)
(feed-test
"empty? on filtered-out"
(feed/empty? (feed/by-actor S "zzz"))
true)
; ---------- api ----------
(feed/reset!)
(feed/post {:actor "x" :at 1 :verb "post"})
(feed/post {:actor "y" :at 2 :verb "like"})
(feed-test "api size after posts" (feed/size) 2)
(feed-test "api all count" (feed/count (feed/all)) 2)
(feed-test
"post returns normalized verb"
(feed/verb (feed/post {:actor "z"}))
"post")
(feed-test "api size after third post" (feed/size) 3)

View File

@@ -1,85 +0,0 @@
; Follow-up — TF-IDF content ranking over :tags. (feed-test name got expected)
(define
corpus
(feed/stream
(list
(feed/normalize {:actor "u" :object "o1" :at 10 :tags (list "cats" "funny")})
(feed/normalize {:actor "u" :object "o2" :at 20 :tags (list "cats" "news")})
(feed/normalize {:actor "u" :object "o3" :at 30 :tags (list "politics" "news")})
(feed/normalize {:actor "u" :object "o4" :at 40 :tags (list "cats")}))))
; ---------- document frequency ----------
(feed-test "df cats" (get (feed/tag-df corpus) "cats") 3)
(feed-test "df news" (get (feed/tag-df corpus) "news") 2)
(feed-test "df funny" (get (feed/tag-df corpus) "funny") 1)
(feed-test "df politics" (get (feed/tag-df corpus) "politics") 1)
(feed-test "df full" (feed/tag-df corpus) {:news 2 :funny 1 :politics 1 :cats 3})
; ---------- inverse document frequency ----------
(feed-test
"idf news = log(4/2)"
(get (feed/tag-idf corpus) "news")
(log 2))
(feed-test
"idf funny = log(4/1)"
(get (feed/tag-idf corpus) "funny")
(log 4))
(feed-test
"rarer tag has higher idf"
(>
(get (feed/tag-idf corpus) "funny")
(get (feed/tag-idf corpus) "cats"))
true)
; ---------- tf-idf scoring ----------
(define idf (feed/tag-idf corpus))
(feed-test
"score query funny on o1"
((feed/tfidf-score idf (list "funny")) (feed/normalize {:actor "u" :object "x" :tags (list "cats" "funny")}))
(log 4))
(feed-test
"score query funny on non-match"
((feed/tfidf-score idf (list "funny")) (feed/normalize {:actor "u" :object "x" :tags (list "cats")}))
0)
(feed-test
"unknown query tag scores 0"
((feed/tfidf-score idf (list "zzz")) (feed/normalize {:actor "u" :object "x" :tags (list "cats")}))
0)
; ---------- ranking by relevance ----------
; query news: o2,o3 match (score log2), o1,o4 don't (0); ties break by :at desc
(feed-test
"by-relevance news order"
(map
(fn (a) (get a :object))
(feed/items (feed/by-relevance corpus (list "news"))))
(list "o3" "o2" "o4" "o1"))
; query funny: only o1 matches -> ranks first
(feed-test
"by-relevance funny first"
(get
(nth (feed/items (feed/by-relevance corpus (list "funny"))) 0)
:object)
"o1")
; query (cats news): o2 carries both tags -> highest combined tf-idf
(feed-test
"by-relevance cats+news top"
(get
(nth
(feed/items (feed/by-relevance corpus (list "cats" "news")))
0)
:object)
"o2")
(feed-test
"by-relevance preserves count"
(feed/count (feed/by-relevance corpus (list "cats")))
4)

View File

@@ -1,56 +0,0 @@
; Follow-up — verb-aware (smart) dedupe. (feed-test name got expected)
; reactions (like/follow) collapse cross-actor; posts stay distinct per actor
(define
M
(feed/stream
(list
(feed/activity "alice" "like" "X" 1 (list))
(feed/activity "bob" "like" "X" 2 (list))
(feed/activity "alice" "post" "P" 3 (list))
(feed/activity "bob" "post" "P" 4 (list))
(feed/activity "alice" "follow" "C" 5 (list))
(feed/activity "bob" "follow" "C" 6 (list))))) ; collapses
(feed-test
"smart dedupe total"
(feed/count (feed/dedupe-smart M))
4)
(feed-test
"smart keeps both posts"
(feed/count (feed/by-verb (feed/dedupe-smart M) "post"))
2)
(feed-test
"smart collapses likes to one"
(feed/count (feed/by-verb (feed/dedupe-smart M) "like"))
1)
(feed-test
"smart collapses follows to one"
(feed/count (feed/by-verb (feed/dedupe-smart M) "follow"))
1)
(feed-test
"collapsed like keeps first actor"
(map feed/actor (feed/items (feed/by-verb (feed/dedupe-smart M) "like")))
(list "alice"))
; contrast: plain activity dedupe keeps cross-actor likes distinct
(feed-test
"activity dedupe keeps both likes"
(feed/count (feed/by-verb (feed/dedupe-activities M) "like"))
2)
; contrast: blanket collapse folds the two posts (same verb+object) too
(feed-test
"collapse dedupe folds posts"
(feed/count (feed/by-verb (feed/dedupe-collapse M) "post"))
1)
; smart-key dispatch
(feed-test
"smart-key reaction -> (verb object)"
(feed/smart-key (feed/activity "alice" "like" "X" 0 (list)))
(list "like" "X"))
(feed-test
"smart-key post -> (actor verb object)"
(feed/smart-key (feed/activity "alice" "post" "P" 0 (list)))
(list "alice" "post" "P"))

View File

@@ -1,187 +0,0 @@
; Phase 2 — fanout via outer product + dedupe. (feed-test name got expected)
; ---------- graph ----------
; edges: (follower followee). bob,carol follow alice; carol,dave follow bob.
(define
G
(feed/follow-graph
(list
(list "bob" "alice")
(list "carol" "alice")
(list "carol" "bob")
(list "dave" "bob"))))
(feed-test "followers alice" (feed/followers G "alice") (list "bob" "carol"))
(feed-test "followers bob" (feed/followers G "bob") (list "carol" "dave"))
(feed-test "followers unknown" (feed/followers G "zzz") (list))
(feed-test "audience distinct" (feed/audience G) (list "bob" "carol" "dave"))
; ---------- fanout ----------
(define
S
(feed/stream
(list
(feed/activity "alice" "post" "p1" 10 (list))
(feed/activity "alice" "post" "p2" 20 (list))
(feed/activity "bob" "like" "p1" 30 (list)))))
(define IB (feed/fanout S G))
(feed-test "fanout total edges" (feed/count IB) 6)
(feed-test
"inbox bob count"
(feed/count (feed/inbox-for IB "bob"))
2)
(feed-test
"inbox carol count"
(feed/count (feed/inbox-for IB "carol"))
3)
(feed-test
"inbox dave count"
(feed/count (feed/inbox-for IB "dave"))
1)
(feed-test
"inbox alice (follows none)"
(feed/count (feed/inbox-for IB "alice"))
0)
(feed-test
"recipients order"
(feed/recipients IB)
(list "bob" "carol" "dave"))
(feed-test
"bob inbox objects"
(map (fn (a) (get a :object)) (feed/inbox-activities IB "bob"))
(list "p1" "p2"))
(feed-test
"dave inbox objects"
(map (fn (a) (get a :object)) (feed/inbox-activities IB "dave"))
(list "p1"))
(feed-test
"dave inbox verb"
(map (fn (a) (get a :verb)) (feed/inbox-activities IB "dave"))
(list "like"))
; empty graph → no audience → no edges
(feed-test
"empty graph fanout"
(feed/count (feed/fanout S {}))
0)
; actor nobody follows produces no edges
(define
Sghost
(feed/stream (list (feed/activity "ghost" "post" "g1" 5 (list)))))
(feed-test
"unfollowed actor fanout"
(feed/count (feed/fanout Sghost G))
0)
; ---------- high fanout (popular actor) ----------
(define
Gstar
(feed/follow-graph
(list
(list "u1" "star")
(list "u2" "star")
(list "u3" "star")
(list "u4" "star")
(list "u5" "star"))))
(define
Sstar
(feed/stream (list (feed/activity "star" "post" "s1" 1 (list)))))
(feed-test
"star fanout count"
(feed/count (feed/fanout Sstar Gstar))
5)
(feed-test "star audience size" (len (feed/audience Gstar)) 5)
; ---------- mutual follow ----------
(define Gmut (feed/follow-graph (list (list "a" "b") (list "b" "a"))))
(define
Smut
(feed/stream
(list
(feed/activity "a" "post" "pa" 1 (list))
(feed/activity "b" "post" "pb" 2 (list)))))
(define IBmut (feed/fanout Smut Gmut))
(feed-test "mutual total" (feed/count IBmut) 2)
(feed-test
"mutual a gets pb"
(map (fn (x) (get x :object)) (feed/inbox-activities IBmut "a"))
(list "pb"))
(feed-test
"mutual b gets pa"
(map (fn (x) (get x :object)) (feed/inbox-activities IBmut "b"))
(list "pa"))
; ---------- dedupe ----------
(define
Sdup2
(feed/stream
(list
(feed/activity "alice" "post" "p1" 1 (list))
(feed/activity "alice" "post" "p1" 9 (list))
(feed/activity "alice" "post" "p2" 2 (list)))))
(feed-test
"dedupe-activities collapses dup"
(feed/count (feed/dedupe-activities Sdup2))
2)
(feed-test
"dedupe-activities keeps distinct"
(map
(fn (a) (get a :object))
(feed/items (feed/dedupe-activities Sdup2)))
(list "p1" "p2"))
(define
Slikes
(feed/stream
(list
(feed/activity "alice" "like" "X" 1 (list))
(feed/activity "bob" "like" "X" 2 (list))
(feed/activity "carol" "like" "Y" 3 (list)))))
(feed-test
"collapse cross-actor likes"
(feed/count (feed/dedupe-collapse Slikes))
2)
(feed-test
"collapse keeps distinct objects"
(map
(fn (a) (get a :object))
(feed/items (feed/dedupe-collapse Slikes)))
(list "X" "Y"))
(feed-test
"activity-key shape"
(feed/activity-key (feed/activity "a" "post" "o" 0 (list)))
(list "a" "post" "o"))
(feed-test
"collapse-key shape"
(feed/collapse-key (feed/activity "a" "like" "o" 0 (list)))
(list "like" "o"))
; cross-post: alice posts p1 twice → bob's inbox has it twice → dedupe-inbox → once
(define
Scross
(feed/stream
(list
(feed/activity "alice" "post" "p1" 1 (list))
(feed/activity "alice" "post" "p1" 5 (list)))))
(define IBcross (feed/fanout Scross G))
(feed-test
"cross-post raw bob count"
(feed/count (feed/inbox-for IBcross "bob"))
2)
(feed-test
"cross-post deduped bob count"
(feed/count (feed/inbox-for (feed/dedupe-inbox IBcross) "bob"))
1)
(feed-test
"dedupe-inbox keeps distinct receivers"
(feed/count (feed/dedupe-inbox IBcross))
2)

View File

@@ -1,73 +0,0 @@
; Follow-up — feed/home capstone pipeline. (feed-test name got expected)
; alice follows star and bob (edges: follower followee)
(define
G
(feed/follow-graph (list (list "alice" "star") (list "alice" "bob"))))
; star posts s1 then s2; bob posts b1; star re-posts s1 (cross-post dup);
; zoe posts z1 (alice does NOT follow zoe)
(define
S
(feed/stream
(list
(feed/activity "star" "post" "s1" 10 (list))
(feed/activity "star" "post" "s2" 20 (list))
(feed/activity "bob" "post" "b1" 15 (list))
(feed/activity "star" "post" "s1" 5 (list))
(feed/activity "zoe" "post" "z1" 30 (list)))))
(define rec (feed/recency 100 10))
(feed-test
"home count (deduped, followed only)"
(feed/count (feed/home S G "alice" feed/permit-public? rec 10))
3)
(feed-test
"home order by recency"
(map
(fn (a) (get a :object))
(feed/items (feed/home S G "alice" feed/permit-public? rec 10)))
(list "s2" "b1" "s1"))
(feed-test
"home excludes unfollowed zoe"
(feed/-elem?
"z1"
(map
(fn (a) (get a :object))
(feed/items (feed/home S G "alice" feed/permit-public? rec 10))))
false)
(feed-test
"home top-2"
(map
(fn (a) (get a :object))
(feed/items (feed/home S G "alice" feed/permit-public? rec 2)))
(list "s2" "b1"))
(feed-test
"home dedupes cross-post (one s1)"
(len
(filter
(fn (o) (equal? o "s1"))
(map
(fn (a) (get a :object))
(feed/items
(feed/home S G "alice" feed/permit-public? rec 10)))))
1)
; ACL applied per-viewer in the home pipeline
(define
Sacl
(feed/stream
(list (feed/normalize {:actor "star" :object "pub" :at 20}) (feed/normalize {:actor "star" :object "sec" :visible-to (list "carol") :at 25}))))
(define Gacl (feed/follow-graph (list (list "alice" "star"))))
(feed-test
"home hides activity alice not permitted"
(map
(fn (a) (get a :object))
(feed/items (feed/home Sacl Gacl "alice" feed/permit-acl? rec 10)))
(list "pub"))

View File

@@ -1,155 +0,0 @@
; Phase 4 — visibility (ACL) + federation, and the end-to-end timeline.
; (feed-test name got expected)
; ---------- ACL visibility ----------
; pub: public. sec: bob, allows carol. dm: frank, allows dave.
(define
C
(feed/stream
(list
(feed/normalize {:actor "alice" :object "pub" :at 10})
(feed/normalize {:actor "bob" :object "sec" :visible-to (list "carol") :at 20})
(feed/normalize {:actor "frank" :object "dm" :visible-to (list "dave") :at 30}))))
(feed-test
"public visible to anyone"
(feed/count (feed/visible C "zoe" feed/permit-acl?))
1)
(feed-test
"carol sees allowlisted + public"
(feed/count (feed/visible C "carol" feed/permit-acl?))
2)
(feed-test
"dave sees dm + public"
(feed/count (feed/visible C "dave" feed/permit-acl?))
2)
(feed-test
"author always sees own private"
(feed/count (feed/visible C "frank" feed/permit-acl?))
2)
(feed-test
"permit-public? lets all through"
(feed/count (feed/visible C "zoe" feed/permit-public?))
3)
(feed-test
"visible objects for dave"
(map
(fn (a) (get a :object))
(feed/items (feed/visible C "dave" feed/permit-acl?)))
(list "pub" "dm"))
; per-viewer: same stream, different timelines
(feed-test
"zoe timeline differs from carol"
(not
(=
(feed/count (feed/visible C "zoe" feed/permit-acl?))
(feed/count (feed/visible C "carol" feed/permit-acl?))))
true)
; ---------- federation: merge / ingest ----------
(define
L
(feed/stream
(list
(feed/activity "alice" "post" "p1" 10 (list))
(feed/activity "alice" "post" "p2" 20 (list)))))
(define
P
(feed/stream
(list
(feed/activity "alice" "post" "p2" 20 (list))
(feed/activity "peer" "post" "p9" 25 (list)))))
(feed-test "merge concatenates" (feed/count (feed/merge L P)) 4)
(feed-test
"ingest dedupes overlap"
(feed/count (feed/ingest L P))
3)
(feed-test
"inbound normalizes + ingests"
(feed/count (feed/inbound L (list {:actor "peer" :object "p9" :at 25} {:actor "alice" :object "p1" :at 10})))
3)
; backfill via injected fetch-fn
(define peer-history (fn (peer-id) (list {:actor peer-id :object "h1" :at 1} {:actor peer-id :object "h2" :at 2})))
(feed-test
"backfill merges peer history"
(feed/count (feed/backfill L peer-history "remote"))
4)
(feed-test
"backfill objects present"
(map
(fn (a) (get a :object))
(feed/items
(feed/by-actor (feed/backfill L peer-history "remote") "remote")))
(list "h1" "h2"))
; ---------- federation: outbound partition ----------
; bob (local), alice@remote + carol@remote (remote) follow star
(define
Gf
(feed/follow-graph
(list
(list "bob" "star")
(list "alice@remote" "star")
(list "carol@remote" "star"))))
(define
Sf
(feed/stream (list (feed/activity "star" "post" "s1" 1 (list)))))
(define
remote?
(fn (id) (feed/-elem? id (list "alice@remote" "carol@remote"))))
(define parts (feed/federate Sf Gf remote?))
(feed-test "local deliveries" (feed/count (get parts :local)) 1)
(feed-test "remote deliveries" (feed/count (get parts :remote)) 2)
(feed-test
"local recipient is bob"
(feed/recipients (get parts :local))
(list "bob"))
; deliver: send-fn receives each remote event, local inbox returned
(define sent (list))
(define send-fn (fn (to act) (set! sent (append sent (list to)))))
(define local-inbox (feed/deliver Sf Gf remote? send-fn))
(feed-test "deliver returns local inbox" (feed/count local-inbox) 1)
(feed-test "deliver sent to both remotes" (len sent) 2)
(feed-test "deliver remote targets" sent (list "alice@remote" "carol@remote"))
; ---------- end-to-end: federated, ACL-filtered, ranked timeline ----------
(define
base
(feed/stream
(list
(feed/normalize {:actor "alice" :object "a1" :at 100})
(feed/normalize {:actor "bob" :object "b1" :visible-to (list "carol") :at 90})
(feed/normalize {:actor "eve" :object "e1" :visible-to (list "dave") :at 80}))))
(define federated (feed/inbound base (list {:actor "peer" :object "x1" :at 110})))
(define rec (feed/recency 120 10))
(define
carol-tl
(feed/timeline federated "carol" feed/permit-acl? rec 3))
; eve's :visible-to excludes carol -> filtered out; peer/alice public, bob allows carol
(feed-test "carol federated timeline count" (feed/count carol-tl) 3)
(feed-test
"carol timeline order (recency)"
(map (fn (a) (get a :object)) (feed/items carol-tl))
(list "x1" "a1" "b1"))
(feed-test
"eve dm excluded from carol"
(feed/-elem? "e1" (map (fn (a) (get a :object)) (feed/items carol-tl)))
false)
(feed-test
"dave sees eve dm not bob"
(map
(fn (a) (get a :object))
(feed/items
(feed/timeline federated "dave" feed/permit-acl? rec 5)))
(list "x1" "a1" "e1"))

View File

@@ -1,68 +0,0 @@
; Follow-up — viewer mute/block filtering. (feed-test name got expected)
(define
S
(feed/stream
(list
(feed/normalize {:actor "alice" :object "P1" :at 1 :tags (list "news")})
(feed/normalize {:actor "bob" :object "P2" :at 2 :tags (list "spam")})
(feed/normalize {:actor "alice" :object "P3" :at 3 :tags (list "cats")})
(feed/normalize {:actor "carol" :object "P4" :at 4 :tags (list "news" "spam")}))))
; ---------- mute actors ----------
(feed-test
"mute bob drops his post"
(map
(fn (a) (get a :object))
(feed/items (feed/mute-actors S (list "bob"))))
(list "P1" "P3" "P4"))
(feed-test
"mute alice drops two"
(feed/count (feed/mute-actors S (list "alice")))
2)
(feed-test
"mute nobody keeps all"
(feed/count (feed/mute-actors S (list)))
4)
; ---------- mute tags ----------
(feed-test
"mute spam tag drops two"
(map
(fn (a) (get a :object))
(feed/items (feed/mute-tags S (list "spam"))))
(list "P1" "P3"))
(feed-test
"mute news+cats leaves spam-only"
(map
(fn (a) (get a :object))
(feed/items (feed/mute-tags S (list "news" "cats"))))
(list "P2"))
; ---------- mute objects ----------
(feed-test
"mute object P3 (thread mute)"
(feed/count (feed/mute-objects S (list "P3")))
3)
; ---------- combined prefs ----------
(feed-test
"apply-prefs actors + tags"
(map
(fn (a) (get a :object))
(feed/items (feed/apply-prefs S {:mute-actors (list "bob") :mute-tags (list "cats")})))
(list "P1" "P4"))
(feed-test
"apply-prefs empty keeps all"
(feed/count (feed/apply-prefs S {}))
4)
(feed-test
"apply-prefs all three filters"
(map
(fn (a) (get a :object))
(feed/items (feed/apply-prefs S {:mute-objects (list "P3") :mute-actors (list "carol") :mute-tags (list "spam")})))
(list "P1"))

View File

@@ -1,69 +0,0 @@
; Follow-up — notification feed over an inbox. (feed-test name got expected)
; an inbox is a stream of {:to receiver :activity act} events
(define mk-ev (fn (to act) {:activity act :to to}))
(define
IB
(feed/stream
(list
(mk-ev "alice" (feed/activity "bob" "like" "P" 10 (list)))
(mk-ev "alice" (feed/activity "carol" "like" "P" 20 (list)))
(mk-ev "alice" (feed/activity "dave" "reply" "Q" 30 (list)))
(mk-ev "bob" (feed/activity "eve" "like" "R" 40 (list))))))
; ---------- raw notifications ----------
(feed-test
"alice notification count"
(feed/count (feed/notifications IB "alice"))
3)
(feed-test
"bob notification count"
(feed/count (feed/notifications IB "bob"))
1)
(feed-test
"zoe no notifications"
(feed/count (feed/notifications IB "zoe"))
0)
; ---------- verb filtering ----------
(feed-test
"alice likes only"
(feed/count (feed/notify-verbs IB "alice" (list "like")))
2)
(feed-test
"alice replies only"
(feed/count (feed/notify-verbs IB "alice" (list "reply")))
1)
(feed-test
"alice like+reply"
(feed/count (feed/notify-verbs IB "alice" (list "like" "reply")))
3)
(feed-test
"alice follow (none)"
(feed/count (feed/notify-verbs IB "alice" (list "follow")))
0)
; ---------- digest ----------
(define dig (feed/notify-digest IB "alice"))
(feed-test "digest group count" (len dig) 2)
(feed-test
"digest sorted by key (like|P before reply|Q)"
(map (fn (g) (get g :object)) dig)
(list "P" "Q"))
(feed-test
"like group actors"
(get (nth dig 0) :actors)
(list "bob" "carol"))
(feed-test "like group count" (get (nth dig 0) :count) 2)
(feed-test "like group verb" (get (nth dig 0) :verb) "like")
(feed-test "reply group count" (get (nth dig 1) :count) 1)
(feed-test
"reply group actors"
(get (nth dig 1) :actors)
(list "dave"))
(feed-test "empty digest for zoe" (feed/notify-digest IB "zoe") (list))

View File

@@ -1,86 +0,0 @@
; Follow-up — pagination (offset + cursor). (feed-test name got expected)
; ---------- offset / limit ----------
(define
O
(feed/stream
(list
(feed/activity "u" "post" "o1" 1 (list))
(feed/activity "u" "post" "o2" 2 (list))
(feed/activity "u" "post" "o3" 3 (list))
(feed/activity "u" "post" "o4" 4 (list))
(feed/activity "u" "post" "o5" 5 (list)))))
(feed-test
"page 1"
(map
(fn (a) (get a :object))
(feed/items (feed/page O 0 2)))
(list "o1" "o2"))
(feed-test
"page 2"
(map
(fn (a) (get a :object))
(feed/items (feed/page O 2 2)))
(list "o3" "o4"))
(feed-test
"page 3 (partial)"
(map
(fn (a) (get a :object))
(feed/items (feed/page O 4 2)))
(list "o5"))
(feed-test
"page past end empty"
(feed/count (feed/page O 10 2))
0)
(feed-test "page-count 5/2 = 3" (feed/page-count O 2) 3)
(feed-test "page-count 5/5 = 1" (feed/page-count O 5) 1)
; ---------- cursor (recency) ----------
(define
R
(feed/stream
(list
(feed/activity "u" "post" "a" 50 (list))
(feed/activity "u" "post" "b" 40 (list))
(feed/activity "u" "post" "c" 30 (list))
(feed/activity "u" "post" "d" 20 (list))
(feed/activity "u" "post" "e" 10 (list)))))
(define p1 (feed/page-before R 100 2))
(feed-test
"cursor page 1 newest first"
(map (fn (a) (get a :object)) (feed/items p1))
(list "a" "b"))
(feed-test "next cursor after page 1" (feed/next-cursor p1) 40)
(define p2 (feed/page-before R (feed/next-cursor p1) 2))
(feed-test
"cursor page 2"
(map (fn (a) (get a :object)) (feed/items p2))
(list "c" "d"))
(feed-test "next cursor after page 2" (feed/next-cursor p2) 20)
(define p3 (feed/page-before R (feed/next-cursor p2) 2))
(feed-test
"cursor page 3 (partial)"
(map (fn (a) (get a :object)) (feed/items p3))
(list "e"))
(feed-test
"empty page nil cursor"
(feed/next-cursor (feed/page-before R 5 2))
nil)
(feed-test
"after cursor loads newer"
(map
(fn (a) (get a :object))
(feed/items (feed/recent (feed/after R 30))))
(list "a" "b"))
(feed-test
"before cursor count"
(feed/count (feed/before R 30))
2)

View File

@@ -1,160 +0,0 @@
; Phase 3 — aggregation + ranking. (feed-test name got expected)
; ---------- aggregation ----------
(define
A
(feed/stream
(list
(feed/activity "alice" "post" "p1" 5 (list))
(feed/activity "alice" "post" "p2" 15 (list))
(feed/activity "bob" "post" "p3" 25 (list))
(feed/activity "alice" "like" "p1" 35 (list)))))
(feed-test "actor-counts" (feed/actor-counts A) {:alice 3 :bob 1})
(feed-test "object-counts" (feed/object-counts A) {:p2 1 :p3 1 :p1 2})
(feed-test
"group-by actor alice len"
(len (get (feed/group-by A feed/actor) "alice"))
3)
(feed-test
"group-count empty"
(feed/group-count feed/empty feed/actor)
{})
; day bucketing
(define
D
(feed/stream
(list
(feed/activity "alice" "post" "p1" 5 (list))
(feed/activity "alice" "post" "p2" 8 (list))
(feed/activity "alice" "post" "p3" 12 (list)))))
(feed-test "feed/day floor" (feed/day 12 10) 1)
(feed-test "feed/day same bucket" (feed/day 8 10) 0)
(feed-test "by-actor-day" (feed/by-actor-day D 10) {:alice#0 2 :alice#1 1})
; ---------- recency ----------
(define rec (feed/recency 100 10))
(feed-test
"recency at=now -> 1"
(rec (feed/activity "x" "post" "o" 100 (list)))
1)
(feed-test
"recency age=hl -> .5"
(rec (feed/activity "x" "post" "o" 90 (list)))
0.5)
(feed-test
"recency age=2hl -> .25"
(rec (feed/activity "x" "post" "o" 80 (list)))
0.25)
; ---------- velocity ----------
(define vel (feed/velocity D 10))
(feed-test
"velocity burst (at=12)"
(vel (feed/activity "alice" "post" "z" 12 (list)))
3)
(feed-test
"velocity mid (at=8)"
(vel (feed/activity "alice" "post" "z" 8 (list)))
2)
(feed-test
"velocity first (at=5)"
(vel (feed/activity "alice" "post" "z" 5 (list)))
1)
(feed-test
"velocity other actor"
(vel (feed/activity "bob" "post" "z" 12 (list)))
0)
; ---------- engagement ----------
(define eng (feed/engagement A))
(feed-test
"engagement p1"
(eng (feed/activity "x" "post" "p1" 0 (list)))
2)
(feed-test
"engagement p2"
(eng (feed/activity "x" "post" "p2" 0 (list)))
1)
; ---------- composite ----------
(define
cmp1
(feed/composite (list (list 2 (fn (a) (get a :at))))))
(feed-test
"composite single part"
(cmp1 (feed/activity "x" "post" "o" 5 (list)))
10)
(define
cmp2
(feed/composite
(list
(list 2 (fn (a) (get a :at)))
(list 3 (fn (a) 1)))))
(feed-test
"composite two parts"
(cmp2 (feed/activity "x" "post" "o" 5 (list)))
13)
; ---------- ranking ----------
(define
R
(feed/stream
(list
(feed/activity "u" "post" "oC" 80 (list))
(feed/activity "u" "post" "oA" 100 (list))
(feed/activity "u" "post" "oB" 90 (list)))))
(feed-test
"rank by recency objects"
(map (fn (a) (get a :object)) (feed/items (feed/rank R rec)))
(list "oA" "oB" "oC"))
(feed-test
"top-2 by recency"
(map (fn (a) (get a :object)) (feed/items (feed/top R rec 2)))
(list "oA" "oB"))
(feed-test "top-2 count" (feed/count (feed/top R rec 2)) 2)
; constant score -> tiebreak by :at descending
(define
T
(feed/stream
(list
(feed/activity "u" "post" "f" 10 (list))
(feed/activity "u" "post" "g" 30 (list))
(feed/activity "u" "post" "h" 20 (list)))))
(feed-test
"tiebreak at-desc"
(map
(fn (a) (get a :object))
(feed/items (feed/rank T (fn (a) 0))))
(list "g" "h" "f"))
; equal score AND equal :at -> stable input order
(define
E
(feed/stream
(list
(feed/activity "u" "post" "first" 50 (list))
(feed/activity "u" "post" "second" 50 (list)))))
(feed-test
"stable equal-key input order"
(map
(fn (a) (get a :object))
(feed/items (feed/rank E (fn (a) 0))))
(list "first" "second"))
(feed-test
"with-scores attaches score"
(get (nth (feed/items (feed/with-scores R rec)) 1) :score)
1)
(feed-test "rank preserves count" (feed/count (feed/rank A rec)) 4)

View File

@@ -1,49 +0,0 @@
; Follow-up — conversation threading via :reply-to closure. (feed-test name got expected)
(define
S
(feed/stream
(list
(feed/normalize {:actor "a" :object "root" :at 1})
(feed/normalize {:actor "b" :object "r1" :at 2 :verb "reply" :reply-to "root"})
(feed/normalize {:actor "c" :object "r2" :at 3 :verb "reply" :reply-to "root"})
(feed/normalize {:actor "d" :object "r3" :at 4 :verb "reply" :reply-to "r1"})
(feed/normalize {:actor "e" :object "x" :at 5}))))
; ---------- direct replies ----------
(feed-test "direct replies to root" (feed/reply-count S "root") 2)
(feed-test "direct replies to r1" (feed/reply-count S "r1") 1)
(feed-test "no replies to r3" (feed/reply-count S "r3") 0)
(feed-test
"replies objects to root"
(map (fn (a) (get a :object)) (feed/items (feed/replies S "root")))
(list "r1" "r2"))
; ---------- thread closure ----------
(feed-test
"thread objects root (transitive)"
(feed/thread-objects S "root")
(list "root" "r1" "r2" "r3"))
(feed-test
"thread root chronological"
(map (fn (a) (get a :object)) (feed/items (feed/thread S "root")))
(list "root" "r1" "r2" "r3"))
(feed-test "thread size root" (feed/thread-size S "root") 4)
(feed-test
"thread excludes unrelated x"
(feed/-elem?
"x"
(map (fn (a) (get a :object)) (feed/items (feed/thread S "root"))))
false)
; ---------- sub-thread ----------
(feed-test
"thread from r1 (sub-tree)"
(map (fn (a) (get a :object)) (feed/items (feed/thread S "r1")))
(list "r1" "r3"))
(feed-test "thread size r1" (feed/thread-size S "r1") 2)
(feed-test "leaf thread is itself" (feed/thread-size S "r3") 1)
(feed-test "unrelated thread is itself" (feed/thread-size S "x") 1)

View File

@@ -1,82 +0,0 @@
; Follow-up — trending objects/actors by recent activity. (feed-test name got expected)
; window (50,100]: X@60,X@70 (a), Y@80 (b), Z@90 (c); W@40 is too old
(define
S
(feed/stream
(list
(feed/activity "a" "post" "X" 60 (list))
(feed/activity "a" "post" "X" 70 (list))
(feed/activity "b" "post" "Y" 80 (list))
(feed/activity "c" "post" "Z" 90 (list))
(feed/activity "d" "post" "W" 40 (list)))))
; ---------- trending objects ----------
(feed-test
"trending count (3 in window)"
(len (feed/trending S 100 50 10))
3)
(feed-test
"trending top object"
(get
(nth (feed/trending S 100 50 10) 0)
:object)
"X")
(feed-test
"trending top count"
(get
(nth (feed/trending S 100 50 10) 0)
:count)
2)
(feed-test
"trending order (count desc, key asc tiebreak)"
(map
(fn (e) (get e :object))
(feed/trending S 100 50 10))
(list "X" "Y" "Z"))
(feed-test
"trending top-2"
(map
(fn (e) (get e :object))
(feed/trending S 100 50 2))
(list "X" "Y"))
(feed-test
"old object W excluded"
(feed/-elem?
"W"
(map
(fn (e) (get e :object))
(feed/trending S 100 50 10)))
false)
(feed-test
"narrow window keeps only newest"
(map
(fn (e) (get e :object))
(feed/trending S 100 15 10))
(list "Z"))
(feed-test
"empty window -> nothing"
(feed/trending S 100 5 10)
(list))
; ---------- trending actors ----------
(feed-test
"trending actor top"
(get
(nth (feed/trending-actors S 100 50 10) 0)
:actor)
"a")
(feed-test
"trending actor count"
(get
(nth (feed/trending-actors S 100 50 10) 0)
:count)
2)
(feed-test
"trending actors order"
(map
(fn (e) (get e :actor))
(feed/trending-actors S 100 50 10))
(list "a" "b" "c"))

View File

@@ -1,59 +0,0 @@
; feed/thread — conversation threading. A reply carries :reply-to <parent-object>
; (normalize preserves it). A thread is the transitive closure over :reply-to from
; a root object: root + replies + replies-to-replies, gathered chronologically.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem?, feed/-distinct).
; direct replies to an object
(define
feed/replies
(fn
(stream object)
(feed/filter stream (fn (a) (equal? (get a :reply-to) object)))))
(define
feed/reply-count
(fn (stream object) (feed/count (feed/replies stream object))))
; iterate f from x until the result stops growing (set-closure fixpoint)
(define
feed/-fixpoint
(fn
(f x)
(let
((nx (f x)))
(if (= (len nx) (len x)) x (feed/-fixpoint f nx)))))
; the set of object-ids in the thread rooted at `root`
(define
feed/thread-objects
(fn
(stream root)
(let
((all (feed/items stream)))
(feed/-fixpoint
(fn
(acc)
(feed/-distinct
(append
acc
(map
(fn (a) (get a :object))
(filter (fn (a) (feed/-elem? (get a :reply-to) acc)) all)))))
(list root)))))
; the full thread as a chronological stream (root + all descendants)
(define
feed/thread
(fn
(stream root)
(let
((objs (feed/thread-objects stream root)))
(feed/sort-by-at
(feed/filter stream (fn (a) (feed/-elem? (get a :object) objs)))))))
; how many activities are in the thread (root counts as 1)
(define
feed/thread-size
(fn (stream root) (feed/count (feed/thread stream root))))

View File

@@ -1,42 +0,0 @@
; feed/trending — what's hot right now: objects (or actors) ranked by activity
; count within a recency window. Deterministic: count descending, ties broken by
; key ascending (entries are pre-sorted by key, then stable grade-down by count).
;
; Requires: lib/feed/stream.sx, lib/feed/aggregate.sx (object/actor-counts),
; lib/feed/rank.sx (feed/-desc-by).
; activities within (now-window, now]
(define
feed/-recent
(fn
(stream now window)
(feed/filter
stream
(fn (a) (and (<= (get a :at) now) (> (get a :at) (- now window)))))))
; counts dict -> top-N entries {label key, :count n}, count desc, key asc
(define
feed/-top-counts
(fn
(counts label n)
(let
((entries (map (fn (k) (assoc {:count (get counts k)} label k)) (sort (keys counts)))))
(take (feed/-desc-by entries (fn (e) (get e :count))) n))))
; top-N trending objects in the window
(define
feed/trending
(fn
(stream now window n)
(feed/-top-counts
(feed/object-counts (feed/-recent stream now window))
:object n)))
; top-N most active actors in the window
(define
feed/trending-actors
(fn
(stream now window n)
(feed/-top-counts
(feed/actor-counts (feed/-recent stream now window))
:actor n)))

View File

@@ -1,639 +0,0 @@
# Abstraction Radar — backlog
Maintained by the read-only `radar` loop (see `plans/agent-briefings/radar-loop.md`).
Detection only — implementation is a separate, coordinated step owned by the
relevant subsystem loop, never by radar.
**AHA gate to reach _Proposed_:** ≥3 real consumers · all past Phase 2 & API-stable ·
structurally identical (file:line evidence) · a natural home (usually NOT lib/guest).
Anything short → _Watching_ (what's missing) or _Rejected_ (why).
---
## Last scan
- **Date:** 2026-06-07 (radar loop, pass 38)
- **Pass 38 — migration plan DRAFTED (planning loop worklist complete).** All 5 specs
written under `loops/migration:plans/migration/` (host-readiness, strangler-shadow-
harness, slice-01-blog, data-migration, slice-sequencing); loop added a 6th revealed
thread `open-questions.md` (digest for humans) then is end-of-worklist. **Decision point
for the operator: review the plan + decide whether to start an IMPLEMENTATION loop**
(first target per the plan: `lib/host` Phase 1 + multi-`Set-Cookie` fix → slice-01-blog
1a). Branch `loops/migration` is local/un-pushed (per operator's no-push preference).
No new radar candidate; A1 at 13; fed-sx still on deadlock.
- **Date:** 2026-06-07 (radar loop, pass 37)
- **Pass 37 — migration plan 4/5 specs done.** Long-pole shipped: `data-migration.md`
(Postgres → persist via **genesis-import** — seed each stream with current DB state as
initial events). Only `slice-sequencing.md` left; loop self-pacing fine. No new radar
candidate; events (iCal import) + content (sanitize, 799/799) incremental; A1 at 13.
- **Date:** 2026-06-07 (radar loop, pass 36)
- **Pass 36 — migration planning loop healthy + productive.** Self-pacing restored (now
schedules its own ~20min wake-ups). Shipped 2 more specs (3/5 threads): strangler-shadow-
harness (Caddy handle-per-route + offline-replay shadow-diff at the `content/blocks`
facade) and slice-01-blog (GET /<slug>/; **found blog already has `Post.sx_content` +
lexical→SX pipeline** — a real head-start). data-migration + slice-sequencing pending.
No new radar candidate; A1 steady at 13; fed-sx still on deadlock.
- **Date:** 2026-06-07 (radar loop, pass 35)
- **Pass 35 — quiet for findings; ops note.** The migration PLANNING loop had completed
host-readiness and **stalled idle ~1hr** (self-paced `/loop` didn't re-fire after one
iteration). Nudged it to continue its worklist (now on strangler-shadow-harness) +
schedule its own next wake-up. No new radar candidate; events/content incremental;
A1 steady at 13; fed-sx still on the deadlock reproducer.
- **Date:** 2026-06-07 (radar loop, pass 34)
- **Pass 34 — quiet, no new finding.** Minimal churn: migration planning loop still on
host-readiness (next thread pending, self-paced); maude scoreboard refresh; fed-sx
grinding the fed-prims deadlock; A1 adopters steady at 13. Nothing new to discover.
- **Date:** 2026-06-07 (radar loop, pass 33)
- **Pass 33 — host-layer story clarified (refines the migration strategy).** `dream` =
**Dream-on-SX**: OCaml's Dream web framework on the SX CEK, and the project owner's
**confirmed decision to move rose-ash OFF Quart onto Dream** as the ergonomic HTTP front
door over the native SX server (router/session/middleware/cors/csrf/auth/ws/html/json —
16 modules). So the host layer is: **host-on-sx native server (Phases 1-3, carries it
now) → Dream-on-SX framework front door (gated on ocaml-on-sx Phases 1-5) + host-persist
(done) + fed-sx (AP transport).** The migration PLANNING loop (new, tmux `migration`,
commit-only) is now the owner of refining this — it already shipped `host-readiness.md`
pinning the near-term gate to **`lib/host` (unbuilt) + a multi-`Set-Cookie` primitive
fix** (`sx_server.ml:735`). NOTE: `plans/rose-ash-on-sx-migration.md` under-specified the
framework layer (said "host-on-sx HTTP host"); the Dream-over-Quart decision + the
native→Dream sequence is the correction — the planning loop will fold it into its specs.
`maude` at Phase 5 (rewriting-logic substrate). Radar tracks; planning loop details.
- **Date:** 2026-06-07 (radar loop, pass 32)
- **Pass 32 — A1 DONE.** `loops/conformance` merged to architecture (`db76cc8c`); 13 adopters
now on the shared driver; radar spot-checked common-lisp = 487/487 green post-merge →
coordination flag CLEARED. A1 moved to a new **Done** section. New nascent subsystems
`dream` + `maude` (0 files), `fed-prims` resumed (mutex-deadlock fix). The idle
`a1-conformance` loop can be retired (worklist complete).
- **Date:** 2026-06-07 (radar loop, pass 31)
- **Pass 31 — A1 conformance loop WORKLIST COMPLETE.** tcl excluded (foreign `*.tcl`); final:
4 migrated (common-lisp/erlang/feed/go) + 5 excluded (forth/js/ocaml/smalltalk/tcl). A1 =
**12 on shared driver + 6 excluded**; only the parity-gated merge to architecture remains.
commerce shipped a refund saga on flow (2nd flow use) + finished Phase 5 → going quiescent.
relations building graph algos (all-paths) — still unconsumed (W9 unchanged).
- **Date:** 2026-06-07 (radar loop, pass 30)
- **Pass 30:** conformance loop near done — `ocaml` + `smalltalk` excluded (both foreign
`test.sh`/corpus runners, as predicted). Tally: 4 migrated, 4 excluded, **tcl only** left.
Next A1 milestone = the `loops/conformance`→architecture merge under adopter-parity. No
new candidate; relations/artdag steady (no new W9 delegation).
- **Date:** 2026-06-07 (radar loop, pass 29)
- **Pass 29:** conformance loop excluded `js` (test262 fixtures) → 4 migrated + 2 excluded,
3 remain (ocaml/smalltalk/tcl). New subsystems advancing fast: `relations` → Phase 4
federation, `artdag` → Phase 6 federation → both fold into W1 (now 7 federation modules,
theme-not-shape holds) and W9 (relations past Phase 2 but not yet consumed by anyone).
- **Date:** 2026-06-07 (radar loop, pass 28)
- **Pass 28 — fleet expanding again.** Conformance loop: `go` migrated 609/609; **`forth`
excluded** (foreign Forth corpus — classify-then-exclude working). 4 migrated +1 excluded
on the branch; js/ocaml/smalltalk/tcl remain. **2 new subsystems:** `relations` (Phase 1,
parent/child rel facts → new W9 nascent watch) and `artdag` (nascent, 0 files). `events`
MERGED to architecture (its persist+flow adoption now integrated — W4/W8 landed). Briefing
commit hints more incoming: `dream`, `host`, +5 language chisels.
- **Date:** 2026-06-07 (radar loop, passes 2627)
- **Passes 2627 (routine tracking):** conformance loop steady at ~1 migration/iteration —
erlang 761/761, then feed 189/189. A1 = 8 on architecture + 3 on the branch; 6 remain.
W4 still gated (host-persist adapter not landed); no new subsystem; app loops on
incremental domain work (commerce Phase 5 payment envelope, content/events/identity/fed-sx).
Nothing new to discover; merge-time adopter-parity flag still open.
- **Date:** 2026-06-07 (radar loop, pass 25)
- **Pass 25:** A1 → **8 adopters** (events via its own loop) + common-lisp 487/487 on the
conformance branch. The conformance loop **extended the shared `lib/guest` driver**
(per-suite counters/preloads) to do it → raised a **coordination flag in A1**: verify the
branch is non-regressive against all 8 adopters before merging to architecture. commerce
drafting Phase 5 provider-neutral payment envelope. No new candidate; A1 advancing fast.
- **Date:** 2026-06-07 (radar loop, pass 24)
- **Pass 24 — three real updates.** (1) **A1 → 7 adopters** (search migrated, counters mode
— corrects the earlier exclusion). (2) The dedicated `conformance` loop ran its 1st
iteration: refused to force-migrate common-lisp (parity gate worked) and surfaced a
**driver feature-gap** (per-suite counters + preloads) gating the complex multi-suite
candidates → A1 now splits simple-now vs gated-on-driver-enhancement. (3) **W8 commerce
is LIVE** ("order lifecycle as a durable flow-on-sx flow, Phase 3 done") → 2 live flow
consumers. events shipped TZ/DST; mod reverted its extraction note (declined on re-read).
- **Date:** 2026-06-07 (radar loop, pass 23)
- **Pass 23 — trigger fired (empty streak ends at 1922).** commerce recorded a Phase 3
**flow-integration design** (order saga as a flow-on-sx flow, payment suspended until
webhook resume) → 2nd durable-flow consumer; **W8 broadened** from "delivery" to
"externally-resumed orchestration on lib/flow." events made its federation transport
**fed-sx-ready** (injected) → reinforces W1's 5/5 inject-fed-sx seam. acl left tmux
(now fully quiescent). host-persist adapter still not landed (W4 migration still gated).
- **Empty-discovery streak: passes 1922** (last verified pass 22). Fleet at steady state —
active loops (content CvRDT, events recurrence/reschedule, identity grant-mgmt, fed-sx
outbox internals) are building *inside* their domains, not cross-cutting infra. Census
exhausted (p17); all gates re-tested (W1 p18, W2 p19). No new candidate clears any gate.
- **Radar is now trigger-driven.** The next substantive pass needs one of: **(a)** a new
subsystem worktree spawning (auto-joins scan), or **(b)** host-persist's durable adapter
landing → unblocks the W4 acl/mod→persist/log migration, or **(c)** a quiescent
subsystem (acl/mod/search/commerce, static ~916 passes) resuming. Polling ~hourly until
one fires; will tighten cadence then.
- **Date:** 2026-06-07 (radar loop, pass 20)
- **Pass 20 — honest empty pass.** 3 new census recurrences since p17 (normalize/index ×2,
query ×3) — all **name collisions** (same noun, domain-specific op), added to the table.
Recorded the meta-pattern: the fleet shares vocabulary, not structure. Most subsystems
quiescent (acl/mod/search/commerce static ~9-15 passes = API-stable); only events/
identity/content/fed-sx still committing domain features. No new gate-clearer.
- **Date:** 2026-06-07 (radar loop, pass 19)
- **Pass 19 — honest empty pass.** Scanned 10 active subsystems. content/index.sx is a
blog index/tag-cloud listing (presentation, not full-text search — no search reinvention)
and content/multi-doc indexing adds no per-viewer filter. **W2 re-tested: still 2**
(feed, search) — acl's `permit?`-like matches are its own authZ *engine* (the home),
not a downstream read filter. No new candidate cleared any gate.
- **Date:** 2026-06-07 (radar loop, pass 18)
- **Pass 18 — W1 gate re-test.** events shipped Phase 4 federation (5th consumer): a 5th
divergent merge (sorted agenda + `:origin` provenance), trust-gate = runtime list
membership (shares mod's mechanism, not acl's). Reinforces W1's "theme not shape" — but
the **inject-fed-sx-transport seam is now 5/5**, strengthening "all are fed-sx
consumers-in-waiting." Trust sub-pattern refined: mod+events (runtime set) vs acl (rule).
- **Date:** 2026-06-07 (radar loop, pass 17)
- **Pass 17 — filename census declared EXHAUSTED** (see the Census-status table above).
Examined the last unswept ≥2 recurrences (schema/engine = acl⇄mod substrate twins;
catalog/batch = name collisions; store = divergent). No new candidate. Incremental churn
elsewhere (content 621/621, identity PAR, events reminders). Future passes pivot from
censusing to re-testing gates as consumers mature.
- **Date:** 2026-06-07 (radar loop, pass 16)
- **Pass 16:** events started Phase 3 — **durable notification delivery on `lib/flow`**
(new W8: at-least-once + idempotency exemplar; fed-sx/mod roll their own outbox). The two
`notify.sx` (feed vs events) are a name collision (read-side digest vs delivery), noted
in W8. Substrate-adoption story deepening: app domains now consume persist (content/
commerce/events), flow (events), commerce (events), acl-authZ (identity).
- **Date:** 2026-06-07 (radar loop, pass 15)
- **Pass 15:** added the **scanning-method note** above after `query.sx` again proved to
be merged-lib copies (lib/prolog + lib/persist in every worktree). Corrected census
surfaced `wire`×2 (content+mod) → Rejected (shared role, divergent structure: generic SX
serializer vs bespoke pipe-format under a Prolog-env string-prim constraint). events↔
commerce integration appeared (paid tickets); acl/mod/search quiescent ~7 passes (now
API-stable). No new gate-clearer.
- **Date:** 2026-06-07 (radar loop, pass 14)
- **Pass 14:** filename census flagged `snapshot`×?? — but the `*/lib/persist/snapshot.sx`
copies are just the merged `lib/persist` in each worktree, NOT consumers (same artifact
as `lib/feed/rank.sx` everywhere). The one distinct file, `content/snapshot.sx`,
reimplements persist's projection-checkpoint on raw KV instead of using `persist/snapshot`
→ new W7 (persist-adoption nudge). `audit`×3 = the W4 fakes (acl/mod/identity), known.
- **Date:** 2026-06-07 (radar loop, pass 13)
- **Pass 13 — honest re-test, no gate-clearer.** Re-tested the two longest-waiting gates
against the maturing app-domain loops: **W2** (per-viewer visibility) still 2 consumers
(feed, search) — commerce/content/events/identity add no per-viewer read filter; **W3**
(pagination) still 2 (feed, search) — `content/page.sx` is an HTML wrapper, not
pagination (filename collision, noted in W3). Incremental churn only elsewhere.
- **Date:** 2026-06-07 (radar loop, pass 12)
- **Pass 12:** `events` shipped **transactional booking on persist** (3rd live persist
consumer) using `persist/append-expect` (optimistic-concurrency CAS, lock-free capacity
safety). W4 ledger now shows a persist feature-ladder append → append-once → append-expect
that the hand-rolled fakes can't match. No new candidate; W4 reinforced.
- **Date:** 2026-06-07 (radar loop, pass 11)
- **Pass 11 — W4 sharpened with a consumer ledger.** commerce built an **order ledger on
persist** (2nd live exemplar; uses `persist/append-once` for webhook idempotency) and
identity a **grant audit ledger** (in-memory Erlang fake, gated on an Erlang↔persist
bridge). The append-only monotonic-seq event-log pattern is now validated across 4
domains, 2 live on persist + 3 fakes flagged for adoption. See W4 table.
- **Date:** 2026-06-07 (radar loop, pass 10)
- **Pass 10:** commerce/content/events/identity advancing (content 238/238). Probed a
shape outside the routing table — **guarded lifecycle state machines** (mod/lifecycle +
identity/membership) → new W6: shared *design principle*, divergent *structure*
(SX transition-table vs Erlang gen_server), NOT an extraction target. No gate-clearer.
- **Date:** 2026-06-07 (radar loop, pass 9)
- **Pass 9:** `commerce` + `content` reached Phase 2 (`content` 162/162). **Key find:
`content` built its op log directly on `persist/log`** (backend-injected, append+replay-
to-seq) — the live reference exemplar for W4 (see W4). `events` MONTHLY RRULE,
`identity` OAuth2 auth-code + PKCE, search boolean-filtered ranked. A1 still 6 adopters.
- **Date:** 2026-06-06 (radar loop, pass 8)
- **Pass 8 — fleet expanded by 4 app-domain loops** (the briefing's anticipated
`commerce`/`identity` arrivals, auto-picked up by dynamic discovery). All early-stage,
**pre-Phase-2 → moving targets, none count toward any gate yet**:
- `commerce` (Phase 1: `api/cart/catalog/price`). Its "per-line audit" is a cost
*breakdown view* (`api.sx:44`), **not** an append-only decision log → NOT a W4
consumer.
- `events` (Phase 1: `calendar.sx`, RRULE expansion).
- `identity` (early: `session/token`). Defers authZ to acl (`token.sx:15`) — reinforces
W2's "delegate `permit?` to acl-on-sx" routing; identity = authN, acl = authZ.
- `content` (just-started: `block.sx`).
These are the future consumers W2/W3 are waiting on — re-check their per-viewer filters
/ pagination once each clears Phase 2. No new gate-clearer this pass.
- **Pass 7:** **A1 jumped 4→6 adopters**`acl` + `mod` migrated to the shared
conformance driver (first app-domain adopters; proves it generalizes past substrates).
`host-persist` closed its blob-adapter blocker (durable storage adapter now landing →
W4 migration path opening). search shipped proximity/NEAR; flow + persist quiescent.
- **Pass 6:** new worktree **`host-persist`** (active — building persist's durable host
adapter); `feed` went quiescent (left tmux). acl shipped hardening (+25), fed-sx-m1 at
Step 6c. **mod loop independently wrote a shared-plumbing note** (`mod-on-sx.md`,
538b8a53) corroborating W4/W5 — folded its claims + home disagreements into W1/W4/W5.
No new gate-clearer (audit log still 2 consumers), but consumers are now API-stable.
- **Pass 5:** search (+highlight/snippet) and fed-sx-m1 (+follower_graph) moved; rest
unchanged. Filename census: `api`×6, `fed`×3, then `schema/rank/query/page/explain/
engine/batch/audit`×2. Examined the ×6 `api.sx` → Rejected (shared name, divergent
structure incl. implicit-vs-explicit-state contract). rank/batch/engine all ≤2 +
substrate/domain-divergent → no new gate-clearer.
- **Pass 4:** no churn vs pass 3 (same worktrees/tmux/HEADs/adopters). Swept audit+explain
surfaces: acl/mod share an append-only-log shape (→ sharpened W4 with persist/log API
evidence) and a proof-explain shape (→ new W5, substrate-bound). No new gate-clearer.
- **Pass 3 (earlier today):** subsystem set + tmux + A1 adopters (4) all unchanged vs pass 2. Loops
advanced: acl shipped Phase 4 federation; search shipped Phase 4 + pagination; feed
shipped pagination/threading; mod at Ext 19 (capstone); persist did a worked acl-grants
migration (W4). New shape found: offset/limit pagination → folded into W3.
- **Subsystem set discovered:** loop worktrees `acl, erlang, fed-prims, fed-sx-m1,
feed, flow, go, kernel, mod, ocaml, persist, radar, ruby, search,
sx-vm-extensions`; main-repo `lib/*` incl. merged `feed` + substrates (`apl,
common-lisp, datalog, erlang, forth, go, haskell, hyperscript, js, lua, minikanren,
ocaml, prolog, scheme, smalltalk, tcl`) + `lib/guest`.
Actively looping (tmux): `acl, fed-sx-m1, feed, flow, mod, persist, search`
(+ radar).
- **New since pass 1:** worktrees `kernel` (empty/unset — not yet a repo) and `ocaml`
(`lib/ocaml/baseline` only). Both early-stage, prePhase 2 → out of proposal scope.
- Re-enumerate every pass; new loops (e.g. a future `commerce`/`identity`) auto-join.
**Census status (pass 17): EXHAUSTED.** Every own-namespace filename recurring ≥2× has
been examined and dispositioned — further filename-censusing is low-yield until new
subsystems/modules appear. Map:
| filename | owners | verdict |
|---|---|---|
| `api` ×10 | all | Rejected — shared role, divergent state contract |
| `fed`/`federation` | feed/search/mod/acl(+content) | W1 — theme not shape |
| `audit` ×3 | acl/mod/identity | W4 — append-only log → persist/log |
| `page` ×3 | feed/search (pagination) + content (HTML wrapper) | W3 + collision noted |
| `explain` ×2 | acl/mod | W5 — proof tree, substrate-bound |
| `snapshot` ×2 | persist(facet) + content(reinvents) | W7 |
| `wire` ×2 | content(SX serializer) / mod(pipe-format) | Rejected — divergent |
| `schema`,`engine` ×2 | acl/mod | substrate-twin parallels (Datalog vs Prolog); only audit (W4) is liftable |
| `catalog`,`batch` ×2 | commerce/persist, mod/persist | name collisions, unrelated |
| `normalize` ×2 | content(tree-prune)/feed(record-coerce) | name collision (pass 20) |
| `index` ×2 | content(listing)/search(inverted index) | name collision (pass 20) |
| `query` ×3 | content(doc-block)/search(bool AST)/persist(stream-read) | 3-way name collision (pass 20) |
| `store` ×2 | content(on persist) / flow(workflow records) | related concept, divergent |
| `rank` ×2 | feed/search | different domains (activities vs docs), ≤2 |
**acl⇄mod are structural twins** (decision engine over a logic substrate, Datalog vs
Prolog) — they parallel across engine/schema/explain/audit/fed, but only the *audit log*
is substrate-agnostic and liftable (→ W4); the rest are substrate-idiomatic. Next passes:
re-test gates (W2/W3/W8) as consumers mature, watch new modules — not re-census.
**Meta-pattern (pass 20):** new module names keep *recurring* but the operations keep
*colliding* — same noun, domain-specific op (normalize, index, query, catalog, batch,
notify, page, store all proved to be collisions). This is *why* genuine extraction
candidates are rare: the fleet shares vocabulary, not structure. The real shared assets
are the **substrate subsystems** (persist, flow, acl, fed-sx) that app domains *adopt*
(W1/W2/W4/W7/W8), not hand-rolled libs to extract.
**Scanning-method note (learned the hard way, passes 5/12/14/15):** a filename census
for *cross-subsystem* recurrence MUST restrict to each subsystem's OWN namespace —
`X/lib/X/*.sx` — never `X/lib/*/`. The merged substrate libs (`lib/prolog`, `lib/persist`,
`lib/feed`, `lib/datalog`, …) are checked out inside *every* worktree, so a naive census
reports e.g. `query.sx`/`snapshot.sx`/`rank.sx` ×N as phantom recurrences that are really
one merged file copied N times. Correct one-liner:
`for w in <subsystems>; do for f in $w/lib/$w/*.sx; do basename $f .sx; done; done | sort | uniq -c | sort -rn`.
---
## Done
### A1 · Shared conformance driver — ✅ COMPLETE (merged `db76cc8c`, pass 32)
Full closed loop: radar detected it → dedicated `conformance` loop implemented it
(classify-then-migrate-or-exclude, hard parity gate) → **merged to architecture**
(`db76cc8c Merge loops/conformance into architecture: A1 conformance-driver migration`)
→ radar spot-verified post-merge (**common-lisp 487/487 green** on architecture — exercises
the new per-suite-counters/preloads driver feature, the riskiest change). Final state:
- **13 on the shared driver:** acl, apl, common-lisp, datalog, erlang, events, feed, go,
haskell, mod, prolog, relations, search.
- **6 correctly excluded** (foreign-program runners — a legitimately different harness):
forth, js, ocaml, smalltalk, tcl, lua.
- The shared driver gained per-suite counters + per-suite preloads (backward-compatible);
spot-check confirms existing adopters unaffected. Coordination flag CLEARED.
Detail of the migration arc retained under the original entry below.
## Proposed (cleared the gate)
_(empty — A1 graduated to Done, pass 32.)_
### A1 · Adopt the shared conformance driver across subsystems
- **Pattern:** every subsystem hand-rolls a near-identical `conformance.sh`
(epoch-load → eval → scoreboard emit) and an inline `<x>-test name got expected`
pass/fail counter.
- **Consumers (≥3, overwhelming):** 15 `lib/*/conformance.sh` — `apl, feed, datalog,
flow, mod, lua, erlang, forth, go, common-lisp, haskell, js, ocaml, prolog,
smalltalk, tcl`.
- **Home:** `lib/guest` — the one legitimate exception (the shared driver
`lib/guest/conformance.sh` + `lib/guest/conformance.sx` already exist; modes
`dict` and `counters`).
- **Status: IN PROGRESS — 6 adopters (pass 7).** `prolog` (dict), `haskell` (counters),
`apl` (dict), `datalog` (dict), and **`acl` (dict) + `mod` (dict), newly migrated this
pass** — all 3-line exec shims into `lib/guest/conformance.sh` with a `conformance.conf`.
**acl + mod are the first *app-domain* adopters** (not language substrates) — strong
evidence the driver generalizes beyond the substrate layer, which was the open question.
The `apl` migration earlier *surfaced a latent bug*: the old awk extractor
under-counted `pipeline` (40 vs the real 152 assertions); true apl total is **562**,
not 450 — evidence that adopting the driver also improves correctness.
- **Not a target (different harness shape):** `lua/conformance.sh` is a Python runner
(`lib/lua/conformance.py`) that walks real `*.lua` source files via `lua-eval-ast`
and classifies pass/fail/timeout — it does not run SX `deftest` suites with a
counter/dict scoreboard, so the shared driver does not fit. Excluded, not pending.
- **Remaining hand-rolled candidates (~120220 lines each):** `common-lisp, erlang,
feed, forth, go, js, ocaml, smalltalk, tcl` — now being worked by the dedicated
`conformance` loop (above). (`lua` excluded: walks real `*.lua` files via Python.
`smalltalk` likely excludes too — runs `*.st` via its own `test.sh`. `search` was
thought to be excluded but DID migrate via counters mode — see the 7-adopter note.)
- **Action:** each remaining subsystem's OWN loop migrates when quiescent — add a
`conformance.conf` (+ a `test-harness.sx` preload defining its counters) and
replace `conformance.sh` with the 1-line exec shim
(`exec bash …/guest/conformance.sh …/conformance.conf "$@"`). Recipe template:
`lib/haskell/conformance.conf` (counters) or `lib/prolog/conformance.conf` (dict).
Keep the `bash lib/X/conformance.sh` entry point so no loop is disrupted.
- **Priority: HIGH** (15 consumers, low risk, interface-preserving, additive).
- **8 adopters on architecture** (pass 25): acl, apl, datalog, **events**, haskell, mod,
prolog, search — `events` migrated via its OWN loop; `search` via counters mode (which
corrects the earlier "search excluded" note). **+4 on the `loops/conformance` branch:
`common-lisp` 487/487, `erlang` 761/761, `feed` 189/189, `go` 609/609** — pending merge.
**5 EXCLUDED — all foreign-runner harnesses** (correctly, not force-migrated): `forth`
(Hayes core.fr via awk+python), `js` (test262 `.js`/`.expected`), `ocaml` (scrapes
`test.sh` + `.ml` baseline), `smalltalk` (scrapes `test.sh` + `*.st` corpus), `tcl`
(foreign `*.tcl` vs `# expected:` annotations).
- **✅ CONFORMANCE LOOP WORKLIST COMPLETE (pass 31).** Final A1 picture:
- **12 on the shared driver:** acl, apl, datalog, events, haskell, mod, prolog, search
(on architecture) + common-lisp, erlang, feed, go (on `loops/conformance`, pending merge).
- **6 correctly excluded** (foreign-program runners — testing a language impl against an
external corpus is legitimately a different harness): forth, js, ocaml, smalltalk, tcl, lua.
- **Honest finding:** the driver's reach is narrower than the raw "15 conformance.sh"
count implied — language substrates that run real `.lua/.st/.ml/.tcl/.js/.fr` programs
*should* keep their foreign runners. ~half migrate, ~half don't, and that's correct.
- **One step left:** merge `loops/conformance` → architecture under the **adopter-parity
check** (the coordination flag above — the shared `lib/guest` driver change must be
proven non-regressive against all existing adopters first). The loop is now idle.
- **NOW IN PROGRESS — dedicated loop (2026-06-07).** A human-triggered `conformance` loop
(worktree `/root/rose-ash-loops/conformance`, branch `loops/conformance`, tmux session
`a1-conformance`, briefing `plans/agent-briefings/conformance-loop.md`) is working the
remaining candidates (common-lisp, erlang, feed, forth, go, js, ocaml, smalltalk, tcl)
one per iteration, **classify-then-migrate-or-exclude with a hard test-count parity gate**
(reverts on any mismatch; never pushes to main/architecture). Radar tracks; it implements.
- **Driver-capability boundary found (pass 24, first iteration).** The loop did NOT
force-migrate `common-lisp` (baseline 305/0 across 12 suites) — the shared driver can't
reproduce it: `MODE=counters` supports only ONE global pass/fail counter pair + ONE fixed
preload set, but common-lisp needs **per-suite counter names** (8 distinct pairs) and
**per-suite preload chains**. It logged a precise blocker + unblock path (extend the
`SUITES` entry format with optional per-suite counters/preloads) and moved on.
- **Driver gap RESOLVED next iteration (pass 25) — but it touched the shared driver.** The
loop extended `lib/guest/conformance.sh` (+38 lines: optional per-suite counters + per-suite
preloads in the `SUITES` format, backward-compatible) and then migrated common-lisp at
**487/487** (above the 305 baseline — likely another extractor under-count correction, à la
apl's `pipeline`). The parity gate held throughout.
- **⚠ COORDINATION FLAG (radar): the `loops/conformance` branch now carries a change to the
SHARED `lib/guest` driver** used by all 8 adopters. It's additive by design, but **before
this branch merges to `architecture`, re-run the existing adopters' suites under the new
driver to confirm zero regression** (acl/apl/datalog/events/haskell/mod/prolog/search).
This is the one cross-cutting risk in an otherwise per-subsystem-isolated effort — surfaced
here so the merge is gated on adopter-parity, not assumed.
---
## Watching (real but not yet through the gate)
### W1 · Federation scaffold (merge / ingest / backfill / trust-gate)
- **FAILS the structural-identity gate (deep-dived 2026-06-06, all 4 read).** Consumer
count is met (4) but they are *superficially* similar, not structurally identical —
the federated unit and merge op differ fundamentally:
| Subsystem (file) | Federated unit | Merge op | Trust gate | Injected transport |
|---|---|---|---|---|
| feed (`fed.sx:14,18,40`) | activity streams | dedupe by `(actor verb object)` | none (visibility via `permit?` separately) | `send-fn`, `fetch-fn` |
| search (`fed.sx:8`) | inverted indices | relabel DocId `peer*1000+local` + union posting lists | none | none (pure merge fn) |
| mod (`fed.sx:11-14,99`) | moderation decisions | advisory-list vs applied-list; bind iff `mod/trusted?` | **yes — runtime list** `mod/trusted? peer scope` | mock outbox / `fed-send!` |
| acl (`federation.sx:43,56`) | Datalog delegate facts | pull facts, gate by `trust`/`level_covers` rule, re-saturate | **yes — Datalog rule** at query time | `transport` dict |
| events (`federation.sx`) | calendar agendas | fold trusted peers' agendas into one sorted agenda + `:origin` provenance | **yes — runtime list** `ev/trusts?` (peer-id ∈ trust-set) | injected behind `ev/peer-agenda` |
- **The ONLY real commonality is the injection seam** (now 5/5, pass 18), not extractable
code: every one says "the real transport is `fed-sx`'s job; inject `send-fn`/`fetch-fn`/
`transport`/`peer-agenda` and mock it in tests." That is an architectural *convention the
fleet already follows*. The merge op diverges 5 ways (dedupe / index-union / advisory /
fact-saturation / agenda-sort). The trust gate, where present, splits: **mod + events use
a runtime trust-set membership check; acl uses a declarative Datalog rule** — so even the
trust sub-pattern is 2-of-3, and the membership check is a trivial one-liner (below the
extraction threshold). No shared merge, no single shared trust mechanism.
- **Disposition:** do NOT extract a shared "federation lib." When `fed-sx` ships its
real transport, these 4 become its *consumers* (wiring `send-fn`/`fetch-fn`/`transport`
to it) — that work belongs to each subsystem's loop + the `fed-sx` loop, not a
cross-cutting extraction. Stop re-proposing on the shared name. Home: `fed-sx`.
- **Now 7 federation modules (pass 29):** + `relations` (Phase 4: erel trust-gating,
peer_rel/trust, fed-sx mock transport — Datalog-rule trust like acl) and `artdag`
(Phase 6: content-addressed cache + trust + **invalidation** — a merge shape unlike any
other). Each new one reinforces "theme not shape": 7 divergent merges, all sharing only
the inject-fed-sx-transport seam. Verdict unchanged — they're fed-sx consumers-in-waiting.
- **Narrower sub-claim (mod note, pass 6; refined pass 18):** mod asserts the *fed
trust/outbox* shape shares between mod+acl. Radar evidence refines this: the trust gate
splits by mechanism, not by subsystem pair — **mod + events** both use a runtime
trust-set membership check (`mod/trusted?`, `ev/trusts?`), while **acl** uses a Datalog
rule. So a "trust-set membership" helper has 2 consumers (mod, events) — but it's a
one-line `member?` and the merge it gates diverges, so still not worth extracting.
Resolve at the architecture-merge point if a heavier shared trust-set surface emerges.
### W2 · Per-viewer visibility / permission filter
- **2 shipped consumers, same shape** — `filter <injected-permit> <ranked/candidate stream>`:
- `feed/lib/feed/acl.sx:27` `feed/visible = (feed/filter stream (fn (a) (permit? viewer a)))`,
capstone at `:34` (stream → ACL → rank → top-N). `permit?` injected, sig `(viewer activity)→bool`.
- `search/lib/search/fed.sx:16` `aclFilter permit docs = filter permit docs`;
`topNTfIdfAcl n permit ts idx = take n (aclFilter permit (rankTfIdf ts idx))`.
`permit` injected, sig `DocId→Bool` (viewer baked in by caller).
- **NOT a consumer:** `mod/lib/mod/policy.sx` is moderation policy (reviewer actions),
no per-viewer read filter. So mod won't be the 3rd.
- **Missing:** (a) only 2 consumers, need ≥3; (b) the two interfaces *diverge* —
feed passes `(viewer, item)`, search bakes the viewer in — so any shared form must
pick a convention; (c) both already **inject** the predicate, and the filter body is
literally one line (`filter permit xs`). Leaning toward: the predicate's home is
`acl-on-sx` (`permit?`), and the one-line filter is too thin to extract.
- **Home when ripe:** delegate `permit?` to `acl-on-sx`; do NOT extract the filter.
Re-check if a 3rd genuine per-viewer read filter ships (e.g. events/commerce).
### W3 · Collection helpers (group-by, dedupe-by-key, stable top-N, distinct-order, offset/limit page)
- feed built all of these on APL primitives. search/commerce/events will want
group-by / top-N.
- **NEW (2026-06-06): offset/limit pagination shipped in 2 subsystems, identical shape**
`take limit (drop offset xs)`:
- `feed/lib/feed/page.sx:9` `feed/page` (offset/limit window over a stream).
- `search/lib/search/page.sx:9` `paginate off lim docs = take lim (drop off docs)`.
- NOT a 3rd: `persist/lib/persist/query.sx:5` has a *since-cursor* for incremental log
consumption — resumable-stream semantics, not result windowing. Different shape.
- feed *also* has cursor-by-`:at` recency pagination (`page.sx:21-44`); search has no
cursor. So only the plain offset/limit window is shared, and it is a literal 1-liner.
- **Missing:** ≥3 stable consumers; AND every item here is collection math that belongs
in the **substrate** (APL/Haskell already expose grade/sort/unique/take/drop), not a
shared lib. A 1-line `take/drop` window is far below the extraction threshold. Watch;
revisit only if a non-substrate subsystem needs the same windowing without take/drop.
- **Filename-collision caution (pass 13):** `content/lib/content/page.sx` is an **HTML
page wrapper** (full HTML5 doc), NOT pagination — do not count it as a 3rd pagination
consumer. `page.sx` now means two unrelated things across the fleet. Re-tested pass 13:
pagination still only feed + search (2).
### W4 · In-memory store fakes → `persist-on-sx`
- Not an abstraction to extract — a migration target. Every subsystem fakes its
store with a mutable list (`feed/-log`, flow store, mod audit, …).
- **Owner:** `persist-on-sx` (in progress). Tracked there, listed here for visibility.
- **Concrete instance (file:line, found pass 4): the append-only decision/audit log.**
`acl/lib/acl/audit.sx` and `mod/lib/mod/audit.sx` are the SAME hand-rolled shape, and
`persist/lib/persist/log.sx` (the persist *log facet*) already implements it durably:
| role | acl/audit.sx | mod/audit.sx | persist/log.sx (target) |
|---|---|---|---|
| log var | `acl-audit-log` :9 | `mod/*audit-log*` :10 | backend stream |
| monotonic seq | `acl-audit-seq` :10 | `mod/*audit-seq*` :11 | per-stream high-water :1 |
| append (auto-seq) | `acl-audit-decide!` | commit :32 | `persist/append` :17 |
| count | `acl-audit-count` :51 | `mod/audit-count` :44 | `persist/count` :12 |
| read-all oldest-first | snapshot/tail :73 | `mod/audit-all` :43 | `persist/read` :29 |
| read seq≥from | — | by-seq | `persist/read-from` :31 |
Both deliberately use a monotonic seq with **no wall-clock** (deterministic/testable) —
identical to persist/log's design. Action when persist's host adapter lands: acl + mod
loops swap their in-memory log for `persist/log`. 2 consumers today; not a new lib —
the home already exists. Belongs to acl/mod loops × persist loop, not an extraction.
- **Cross-loop corroboration (pass 6):** the mod loop independently reached the same
conclusion — `mod/plans/mod-on-sx.md` (commit 538b8a53): *"mod-sx (Prolog) and acl-sx
(Datalog) converged on the same module shape … only the audit log + fed trust/outbox
shapes truly share; extract at the architecture-merge point, refactoring both consumers
atomically, not unilaterally from a loop branch."* Confirms the shape AND the
do-not-extract-unilaterally stance.
- **Home disagreement to resolve at merge:** mod's note proposes lifting the audit-log
primitives into **`lib/guest/`**. Radar routing disagrees: a durable append-only log is
a **`persist-on-sx`** concern (the log facet already exists), not language-impl plumbing.
Hold the line — `lib/guest` is lexer/parser/AST/HM/test-runner, not an event log.
- **Migration is becoming concrete:** new `host-persist` loop (worktree + tmux, pass 6)
is building the durable-storage host adapter persist was blocked on — once it lands,
acl/mod can actually swap to `persist/log`.
- **LIVE REFERENCE EXEMPLAR (pass 9): `content` already does it right.** `content`
(Phase 2 complete, 162/162) built its op log directly on `persist/log` instead of
faking it — `content/lib/content/store.sx`: backend injected via `(persist/open)`
("content knows nothing about which backend", :10); append op as event
`persist/append b (content/-stream doc-id) …` (:20); read `persist/read` (:36);
`persist/last-seq` (:47); **version = replay op stream up to a seq**
(filter `persist/event-seq ev <= seq`, :61). "The op log is the source of truth …
the materialised doc is a cache, never primary state."
This proves the W4 target is real, not hypothetical: acl + mod's hand-rolled
monotonic-seq logs should adopt exactly content's `persist/log` pattern.
- **Consumer ledger of the append-only monotonic-seq event log (pass 11):**
| consumer | what | backing | note |
|---|---|---|---|
| content (`store.sx`) | doc op log | **persist/log ✓ live** | plain append + replay-to-seq |
| commerce (`ledger.sx`) | order ledger | **persist/log ✓ live** | `persist/append-once` — idempotent, webhook-replay-safe :40,58 |
| events (`booking.sx`) | booking roster | **persist/log ✓ live** | `persist/append-expect` — optimistic-concurrency CAS, capacity-safe, lock-free |
| acl (`audit.sx`) | decision log | in-memory fake (SX) | migrate directly when host adapter lands |
| mod (`audit.sx`) | decision log | in-memory fake (SX) | migrate directly |
| identity (`audit.sx`) | grant ledger | in-memory fake (**Erlang**) | `{Seq,Subject,Action}`; needs an **Erlang↔persist bridge** first — author scoped it out until persist lands ("queryable semantics identical") |
- **Two takeaways:** (1) the pattern is **validated across domains** — CRDT doc ops,
financial orders, event bookings, rule decisions, OAuth grants all reduce to the same
append-only monotonic-seq stream; (2) migrating to `persist/log` is strictly *better*
than the fakes — persist exposes a **feature ladder the fakes don't have**:
`append` (content) → `append-once`/idempotency (commerce) → `append-expect`/optimistic-
concurrency (events). Every fake would have to reinvent a weaker version of these.
This is an **adoption** item (the home already exists), NOT a new extraction — owned by
persist/host-persist × each consumer loop. The SX fakes (acl, mod) migrate directly;
the Erlang fake (identity) is gated on an Erlang↔persist bridge.
### W5 · Proof-tree explanation over a logic-program derivation
- `acl/lib/acl/explain.sx` (reconstructs a canonical proof by goal-directed search over a
saturated Datalog db) and `mod/lib/mod/explain.sx` (renders a Prolog-style proof tree
goal-by-goal with proved/unproved marks + unification bindings) are the same *idea*.
- **Missing / disposition:** only 2 consumers, and they sit on **different substrates**
(acl→`lib/datalog`, mod→`lib/prolog`). Proof reconstruction/rendering is logic-engine
machinery → it belongs in each **substrate** (datalog/prolog), not a shared app lib.
Watch; revisit only if a 3rd logic-backed subsystem reimplements proof explanation.
- **Cross-loop note (pass 6):** mod's note calls `mod/proof-goals` (re-query-each-goal)
generic and proposes lifting it into **`lib/guest/`**. Radar caveat: proof-tree
reconstruction *is* engine-agnostic logic machinery, but `lib/guest` is for
lexer/parser/AST/HM/match/test-runner — a logic-engine proof helper is a poor fit there.
If genuinely shared by ≥3 engines, a `lib/logic`-style substrate helper is the better
home than `lib/guest`. Still 2 consumers → stays Watching either way.
---
### W9 · Parent/child relationship tracking → the new `relations` subsystem (nascent)
- **New subsystem (pass 28):** `relations` (loops/relations, Phase 1 — `schema.sx`+`api.sx`,
rel facts + `relate`/`unrelate`/`children`/`parents`/`related`, 22 tests). Per CLAUDE.md
it's the canonical "cross-domain parent/child relationship tracking."
- **Why watch:** several subsystems already track parent/child *locally* — feed reply-to
threading (`thread`/`replies`), content nested block trees, events occurrence/RECURRENCE-ID
links. If `relations` becomes the shared home, those are candidate *delegators* (like
acl=authZ, persist=log). But it's **Phase 1, pre-Phase-2, moving target** — and each
local impl is currently domain-specific (different keys/semantics). Do NOT propose yet.
Re-check when relations is past Phase 2 AND ≥3 subsystems' relationship logic could
genuinely delegate to it. `artdag` also just spawned (nascent, 0 files) — tracking only.
(pass 32: `dream` + `maude` also spawned, nascent 0-files; `fed-prims` resumed.)
- **Update pass 29:** relations rocketed to **Phase 4** (one gate — past Phase 2 — now met),
but it's building ITSELF out (schema/federation), **not yet being consumed** by anyone.
The blocker is the other gate: 0 subsystems currently *delegate* their parent/child logic
to it (feed/content/events still track locally). Watch for the first real delegation.
(artdag also raced to Phase 6 — these ports advance fast; treat committed state as truth.)
### W8 · Durable externally-resumed orchestration on `lib/flow` (suspend→host-IO→resume)
- **The shared shape:** a durable `flow` that `request`s an external action (a suspend
point), the **host** performs the IO, then `flow/resume`s the flow with the outcome;
flow's deterministic replay means a completed step never re-runs on recovery.
- **Consumers (pass 24): 2 LIVE** (events delivery, commerce order saga).
- `events/lib/events/notify.sx` (**live**) — reminders/digests as durable flows;
suspend on delivery `dispatch`, resume with send outcome. At-least-once + idempotency key.
- `commerce` (**LIVE** as of pass 24 — "order lifecycle as a durable flow-on-sx flow,
21 tests, Phase 3 done") — order saga `(defflow ordf … (request 'reserve oid) … )`:
reserve→pay→fulfil as a flow, **payment stays suspended until the payment webhook calls
`flow/resume`**. Carries only the order-id; pure orchestration over `ledger.sx`.
- **Now 2 LIVE consumers** of the *same* pattern: long-running process, external resume
(delivery dispatch vs payment webhook). fed-sx/mod still roll their own outbox (watch
for convergence). Strengthens "lib/flow is the home"; still adoption, not extraction.
- **Disposition:** `lib/flow` IS the abstraction (events proves it, commerce adopts it) →
this is an **adoption** observation like W4, NOT an extraction. Home = `lib/flow`.
- **Flow-onboarding friction (light signal):** commerce's note logs real gotchas adopting
flow — `flow-make-env` returns a large likely-cyclic env (don't print it), env build is
slow (budget ~540s like flow's own suite). If ≥3 subsystems hit the same onboarding
gotchas, that's a signal to smooth `lib/flow`'s adopter API — flow's concern, flagged here.
- **Name-collision caveat:** `notify.sx` means two unrelated things — `feed/notify.sx` is
a *read-side digest* (group inbox by verb+object), NOT delivery. Do not pair them.
### W7 · Snapshot/projection-checkpoint reimplemented vs `persist/snapshot` (delegate)
- `persist/lib/persist/snapshot.sx` already provides a **generic** projection checkpoint:
store `{:value :seq}` in the kv facet under a namespaced key; the headline property is
**snapshot + tail == full replay** (pure, clock-free).
- `content/lib/content/snapshot.sx` **reimplements that same pattern on raw persist KV**
rather than delegating: `persist/kv-put b (content/-snap-key doc-id) {:doc … :seq seq}`
(:20), `persist/kv-has?`/`kv-get` (:27-28), and its own tail-replay (:53-59). It never
calls `persist/snapshot-*`. content's doc-materialisation *is* a projection fold over
its op stream — exactly what `persist/snapshot` checkpoints generically.
- **Disposition:** persist-adoption nudge (like W4): content could delegate to
`persist/snapshot` (its projection = "fold ops → doc"), dropping the duplicated
KV+replay code. Home already exists → NOT an extraction; owned by content × persist
loops. Only 1 reinventor today; watch whether commerce/events/identity also hand-roll a
snapshot on raw KV instead of using the facet (would strengthen the nudge). NB timeline:
unclear if `persist/snapshot` predated content's — flag, don't blame.
### W6 · Guarded lifecycle state machine (illegal transition = explicit error)
- Recurs as a **design principle**, NOT a shared structure (found pass 10):
- `mod/lib/mod/lifecycle.sx` — pure SX: immutable case `{:state :error :history …}`,
explicit transition table `mod/lc-transitions` (:31), illegal transition returns the
case unchanged with `:error` set. States open→triaged→decided→appealed→final.
- `identity/lib/identity/membership.sx` — an **Erlang `gen_server`** fragment (identity
runs on erlang-on-sx): a `receive` loop with `case find(...) of … {error, St}` guards.
States none→pending→active→lapsed→revoked.
- **Both share the guideline** ("invalid transitions are explicit errors, never silent
no-ops") but **implement it substrate-idiomatically** — SX transition-table over
immutable values vs an Erlang process loop with per-message case guards. Same W1/`api.sx`
trap: shared *idea*, divergent *structure*.
- **Disposition:** not an extraction target — the FSM mechanism is ~10 substrate-specific
lines; the value is in each domain's state graph, not the plumbing. At most a **design
guideline** ("model lifecycle as a guarded FSM with explicit-error transitions"). Watch
whether commerce-checkout / events-booking add their own — if so it confirms the
*guideline*, still not a lib. Do not propose extracting a shared state-machine lib.
## Rejected (considered, declined — do not re-propose)
- **"Continuous auto-implementing abstractor loop."** Rejected at design time: an
agent writing across `lib/<x>/**` breaks the worktree isolation that makes the
fleet safe, and is rewarded for manufacturing premature/wrong abstractions. The
radar is read-only by design. (This file is the alternative.)
- **Shared `api.sx` "public boundary" module (×6).** Rejected pass 4-5: every subsystem
has an `api.sx` (acl, feed, flow, mod, persist, search — a 100% filename match), but it
is a naming *convention for the public entry point*, not a shared structure. They
disagree on the most basic contract: acl/feed use **implicit module state**
(`acl/api.sx` "implicit current db", `feed/api.sx` "single mutable log") while
`persist/api.sx` threads an **explicit backend as every call's first arg**; flow's api
*builds a Scheme env*, search's api *concatenates a Haskell source string*, mod's is a
*lifecycle state-machine façade* (17 defs vs persist's 1). Same role, no common shape —
the W1 coincidental-resemblance trap. Do not re-propose on the filename.
- **Shared `wire.sx` "serialization" module (×2).** Rejected pass 15: content + mod both
have a `wire.sx`, but `content/wire.sx` uses the **generic SX serializer**
(`serialize`/`parse`, full-fidelity round-trip) while `mod/wire.sx` is a **bespoke
versioned pipe-delimited line** (subset of fields, `split` hand-built over slice/len
because mod's Prolog-loaded env strips string prims). Shared role (wire format),
divergent structure + substrate constraint → not a candidate; the SX serializer is
already the shared tool for SX-substrate subsystems, and mod can't use it. (Same family
as the `api.sx` rejection above.)
- **Dumping app-domain plumbing into `lib/guest`.** Rejected: `lib/guest` is for
language-implementation plumbing. App patterns route to acl/fed-sx/persist/
substrate/host instead (see the routing rule in the briefing).

View File

@@ -15,7 +15,7 @@ and federation extension. Reuses `lib/datalog/` evaluator and term model where p
## Status (rolling)
`bash lib/acl/conformance.sh`**0/0** (not yet started)
`bash lib/acl/conformance.sh`**145/145** (all four phases + hardening)
## Ground rules
@@ -57,46 +57,225 @@ lib/acl/facts.sx — builds Datalog query
## Phase 1 — Direct grants
- [ ] `lib/acl/schema.sx` — sorts: subject {user, group, role, service}, action,
- [x] `lib/acl/schema.sx` — sorts: subject {user, group, role, service}, action,
resource {page, post, thread, peer}
- [ ] `lib/acl/facts.sx``actor`, `resource`, `grant`, `deny` predicates as Datalog
- [x] `lib/acl/facts.sx``actor`, `resource`, `grant`, `deny` predicates as Datalog
EDB
- [ ] `lib/acl/engine.sx``(permit? subj act res db)` reduces to Datalog query
- [ ] `lib/acl/api.sx` — public `(acl/permit? ...)` taking implicit current db
- [ ] `lib/acl/tests/direct.sx` — 15+ cases: direct grant, missing grant, explicit deny
- [ ] `lib/acl/scoreboard.{json,md}` baseline
- [ ] `lib/acl/conformance.sh` runs the suite
- [x] `lib/acl/engine.sx``(permit? subj act res db)` reduces to Datalog query
- [x] `lib/acl/api.sx` — public `(acl/permit? ...)` taking implicit current db
- [x] `lib/acl/tests/direct.sx` — 15+ cases: direct grant, missing grant, explicit deny
- [x] `lib/acl/scoreboard.{json,md}` baseline
- [x] `lib/acl/conformance.sh` runs the suite
## Phase 2 — Inheritance
- [ ] `member_of(actor, group)` chain — group grants apply to members (transitive)
- [ ] `child_of(res, parent)` chain — parent grants apply to children (transitive)
- [ ] role expansion — role contains list of (action, resource) tuples
- [ ] deny-overrides — explicit deny wins over inherited allow
- [ ] `lib/acl/tests/inherit.sx` — 25+ cases: nested groups, deep resource trees,
- [x] `member_of(actor, group)` chain — group grants apply to members (transitive)
- [x] `child_of(res, parent)` chain — parent grants apply to children (transitive)
- [x] role expansion — role contains list of (action, resource) tuples
- [x] deny-overrides — explicit deny wins over inherited allow
- [x] `lib/acl/tests/inherit.sx` — 25+ cases: nested groups, deep resource trees,
conflict resolution, deny precedence
- [ ] document the deny-overrides choice in plan
- [x] document the deny-overrides choice in plan
### deny-overrides policy (the choice)
Encoded as stratified negation: `permit(S,A,R) :- eff_grant(S,A,R), not
eff_deny(S,A,R)`. Both `eff_grant` and `eff_deny` inherit through the *same*
`member_of` (group/role) and `child_of` (resource) chains. Consequences:
- An explicit deny on the exact (S,A,R) defeats any inherited allow.
- A **group-level** or **ancestor-resource** deny inherits down and defeats a
member's/descendant's grant — deny is authoritative across the closure, not
only at the leaf. This is the fail-safe reading: the most permissive
interpretation of "deny wins" would let a narrow grant escape a broad deny;
we chose the opposite.
- Deny is dimension-scoped: a deny on (S, edit, R) never blocks (S, read, R).
Stratifiable because neither `eff_grant` nor `eff_deny` depends on `permit`;
`permit` sits in a strictly higher stratum. Termination is guaranteed —
recursion is only over EDB `member_of`/`child_of` (no function symbols), so
cyclic membership/containment reaches a fixpoint rather than looping (tested).
## Phase 3 — Explanation + audit
- [ ] `(acl/explain subj act res)``{:allowed? T :proof <tree>}`
- [ ] proof tree extracts from Datalog's derivation
- [ ] `lib/acl/audit.sx` — append-only decision log (in-memory + serializer for disk)
- [ ] `(acl/audit-tail n)` for recent decisions
- [ ] `lib/acl/tests/explain.sx` — proof correctness, audit completeness
- [x] `(acl/explain subj act res)``{:allowed? T :proof <tree>}`
- [x] proof tree extracts from Datalog's derivation
- [x] `lib/acl/audit.sx` — append-only decision log (in-memory + serializer for disk)
- [x] `(acl/audit-tail n)` for recent decisions
- [x] `lib/acl/tests/explain.sx` — proof correctness, audit completeness
### proof reconstruction (the choice)
`lib/datalog/` records derived facts but not provenance, so `lib/acl/explain.sx`
reconstructs the proof by goal-directed search over the *saturated* db: for a
ground goal, find the first ACL rule (in `acl-rules` order) whose body holds,
take the first `dl-query` solution binding the rest, recurse on each body
literal; negated literals become verified `:neg-ok` leaves. The Datalog
derivation graph is a DAG (a fact may hold many ways) — we pick ONE **canonical
proof: first-rule, first-solution**, with EDB/direct rules ordered first so
proofs bottom out quickly. A depth cap (64) guards pathological cyclic data.
`acl-explain` returns `{:allowed? :proof :reason}`; on denial `:reason` carries
the blocking `eff_deny` proof (explicit or inherited) when one exists, else nil
(no grant). Audit log is append-only with monotonic seq numbers (no wall-clock,
for determinism); `acl-audit-decide!` is the logged path, `acl-permit?` stays
pure.
## Phase 4 — Federation
- [ ] peer trust facts — `peer(addr, kind)`, `trust(peer, level)`
- [ ] delegated grants — `delegate(peer, actor, action, resource)`
- [ ] cross-instance permit chain — query asks local + queries trusted peers via fed-sx
- [ ] revocation propagation — fact retraction across federation
- [ ] `lib/acl/tests/fed.sx` — federated grant chains (mock fed-sx transport in tests)
- [x] peer trust facts — `peer(addr, kind)`, `trust(peer, level)`
- [x] delegated grants — `delegate(peer, actor, action, resource)`
- [x] cross-instance permit chain — query asks local + queries trusted peers via fed-sx
- [x] revocation propagation — fact retraction across federation
- [x] `lib/acl/tests/fed.sx` — federated grant chains (mock fed-sx transport in tests)
### federation policy (the choice)
One engine rule carries federation:
`eff_grant(S,A,R) :- delegate(Peer,S,A,R), trust(Peer,L), level_covers(L,A)`.
- **Non-transitive trust.** A peer's `delegate` only grants if a *local*
`trust(Peer,L)` exists and that level `level_covers` the action. There is no
peer-to-peer trust propagation — trusting α never extends to peers α trusts.
- **Trust re-checked every query.** `trust`/`level_covers` are body literals
evaluated at decision time, never baked in at ingestion. Revoking trust or
narrowing a level takes effect on the next `acl-permit?`.
- **Deny still wins.** Federated grants are `eff_grant`, so local (and inherited)
deny overrides them exactly as for local grants.
- **Composes with inheritance.** A delegate to a group flows to members; a
delegate on a parent resource flows to children (federated `eff_grant` feeds
the same recursion).
- **Revocation = retraction.** `acl-revoke!` wraps `dl-retract!`; the next query
re-saturates. `acl-fed-assert!` wraps `dl-assert!` for newly-replicated facts.
- **Transport is fed-sx's job.** `lib/acl/federation.sx` mocks the pull as a
dict {peer-addr → delegate-facts}; `acl-fed-build-db` merges local policy +
pulled delegates.
## Progress log
(loop fills this in)
- **Phase 1 complete (24/24).** ACL is a thin layer over `lib/datalog/`:
- `schema.sx` — sorts (subject/resource kinds, well-known actions) + EDB
predicate arity table + `acl-fact-valid?` validator. Schema is data, since
Datalog is untyped.
- `facts.sx``acl-actor`/`acl-resource-fact`/`acl-grant`/`acl-deny`
constructors returning Datalog fact tuples.
- `engine.sx` — owns the ruleset `acl-phase1-rules` and reduces decisions to
`dl-query`. `acl-build-db` = `dl-program-data facts rules`; `acl-permit?` =
non-empty `(permit S A R)` query.
- `api.sx``acl/load!` rebuilds an implicit current db; `acl/permit?` queries
it. (Slash-symbols like `acl/permit?` parse fine as single tokens.)
- **deny-overrides** encoded as `permit(S,A,R) :- grant(S,A,R), not deny(S,A,R)`.
Stratifies cleanly because `deny` is EDB-only (no rule derives it). Verified:
grant+deny on same (S,A,R) → denied.
- Conformance: `conformance.conf` (datalog preloads + acl modules) + thin
`conformance.sh` wrapper over `lib/guest/conformance.sh`. Scoreboard
generated by the shared driver.
- **Shared-plumbing note (for eventual `lib/guest/rules/`):** the
`build-db = dl-program-data(facts, rules)` + `decide = non-empty ground query`
shape is exactly what mod-sx (Prolog moderation) will also need. The reusable
seam is engine.sx's two functions — facts→db and ground-query→bool — both
pure pass-throughs to the rule engine. Not extracting yet (wait for mod-sx as
second consumer per ground rules).
- **Phase 2 complete (54/54, +30 inherit).** Extended `acl-rules` with
`eff_grant`/`eff_deny` derived relations; `member_of` carries both group and
role membership, `child_of` carries resource trees, `role_grant` confers a
role's (action,resource) capabilities. Direct grants unchanged (base case of
`eff_grant`), Phase 1 suite still green. Constructors `acl-member-of`,
`acl-child-of`, `acl-role-grant` added; schema arity table extended. See the
deny-overrides policy section above. Verified cyclic membership terminates.
- **Shared-plumbing update:** the reusable seam is still just engine.sx's
`facts -> db` + `ground-query -> bool`. The inheritance *rules* are
ACL-specific (group/resource/role vocabulary); mod-sx will have its own. So
the `lib/guest/rules/` extraction stays at the build/decide level, not the
ruleset level.
- **Phase 3 complete (89/89, +35 explain).** Added `explain.sx` (proof
reconstruction, see policy section above), `audit.sx` (append-only log), and
extended `api.sx` with `acl/explain`/`acl/audit`/`acl/audit-tail`. No engine
changes — explanation reads the same saturated db the decision uses.
- **Substrate gotcha:** the host `=` compares symbols by interned identity,
which is *unstable* across `dl-query` saturation/substitution within a
single image — the same two structurally-equal symbol-lists compared `=`
true once and false moments later in the REPL. Conformance runs in a fresh
process per suite so it's deterministic there, but test assertions now use a
name-based `acl-et-eq?` (compare symbols via `symbol->string`), matching the
datalog suite's `dl-api-deep=?` convention. Worth flagging to the kernel
owners but out of acl scope.
- **Phase 4 complete (120/120, +31 fed).** Added `federation.sx` (mock
transport + `acl-fed-build-db`/`acl-revoke!`/`acl-fed-assert!`), one engine
rule (the trust-gated delegate rule), 4 fact constructors, 4 schema arities.
Federated proofs reconstruct for free — `explain.sx` iterates `acl-rules`, so
the delegate rule's EDB body (`delegate`/`trust`/`level_covers`) shows up as
proof leaves with no explain changes. **Roadmap done: all four phases green.**
- **Shared-plumbing final note (for `lib/guest/rules/`):** the durable
reusable seam across acl-sx and the coming mod-sx is exactly four
pass-throughs to the rule engine — `build-db(facts)`, `decide(ground-query)
-> bool`, `explain(goal) -> proof-tree`, and the revoke/assert mutators.
The *rulesets* and *vocabulary* are language-specific (ACL: grant/deny/
member_of/...; mod-sx: Prolog moderation predicates). When mod-sx lands,
extract those four functions (engine.sx + the generic half of explain.sx's
goal-directed reconstructor) into `lib/guest/rules/`, leaving each consumer
its own rules + fact constructors. Proof reconstruction is the non-obvious
reusable piece: it only needs the ruleset as data + a saturated db, both of
which any datalog-backed guest has.
- **dict-mode conformance is slow, not hung:** all suites load + run in one
process (~30-40s for 120 tests, no per-suite timeout). Do not kill early.
- **Tooling note:** sx-tree path-based edit tools (`sx_replace_node`,
`sx_read_subtree` with a path) ignored the path argument in this worktree
(always resolved to index 0 / `[0,..]`), in BOTH `(a b c)` and `(a,b,c)`
forms. `sx_write_file`, `sx_validate`, `sx_find_all`, `sx_summarise`,
`sx_eval` all work; used full-file rewrites instead of path edits throughout.
## Hardening (post-roadmap)
- **`lib/acl/tests/harden.sx` (+25).** Adversarial / cross-phase coverage beyond
the per-phase suites. **Prover-free by design** (see JIT blocker below): only
`acl-permit?` (compiled Datalog, safe at any depth) + pure data ops, never
`acl-explain`/`acl-prove-d`.
- Diamond hierarchies (resource and group): grant on one path + deny on
another → deny wins; both-grant → permit; deny does not leak to siblings.
- Chain inheritance (literal 4-link): top-group grant reaches leaf member and
intermediates; leaf-member deny overrides the top grant; deny on the leaf
doesn't block a higher level.
- Cycle termination: self-membership, self-child, and 2-node membership cycles
all reach a fixpoint and decide correctly.
- Federation conflicts: federated group-grant with a locally-denied member →
deny; multi-peer delegation (one trusted, or both trusted) → permit.
- Degenerate inputs: empty db permits nothing.
- Fact validation: `acl-validate-facts` surfaces wrong-arity + unknown
predicates; `acl-facts-valid?` on clean/empty sets.
- Audit save/restore: snapshot → clear → restore round-trips entries + seq;
seq continues without collision after restore; snapshot is an immutable copy.
- Proof reconstruction itself is covered by tests/explain.sx + tests/fed.sx
(both stay under the warm-process JIT depth threshold); the depth-cap safety
net is verified manually in a warm REPL image but excluded from conformance.
- **New API:** `acl-validate-facts`/`acl-facts-valid?` (schema.sx, opt-in — build
stays lenient); `acl-audit-snapshot`/`acl-audit-restore!`/`acl-audit-copy`
(audit.sx).
- **Substrate gotcha (recorded):** `append!` extends a list built with `(list)`
but **silently no-ops on a `map`/`rest`-derived list** in this runtime. Bit the
first cut of `acl-audit-restore!` (rebuilt the live log via `map`, so later
records didn't append). Fix: always rebuild mutable lists via `(list)` +
`append!` (`acl-audit-copy`). Worth flagging to kernel owners; out of acl scope.
## Blockers
(loop fills this in)
- **JIT loops on deep proof reconstruction (substrate, not acl).** Once the
kernel JIT-compiles the mutually-recursive prover (`acl-prove-d`/
`acl-prove-rules`/`acl-prove-build` in `explain.sx`) — which happens after a
process has run enough explains to cross the compile threshold — the compiled
version **loops indefinitely** on a `member_of`/`child_of` chain deeper than
~3. Symptoms: `acl-explain` over a 4+-deep chain returns instantly in a cold /
warm-REPL image but **hangs** in a long-lived process. The per-phase explain
and fed suites pass only because their proofs stay ≤3 deep; a 5th suite that
explained deeper chains hung the whole conformance run (no per-suite timeout
in dict mode). Matches `[[project_jit_bytecode_bug]]` (ACTIVE).
- *Impact beyond tests:* `acl-explain` is unsafe for deep hierarchies in a
warm production OCaml server. `acl-permit?` is unaffected (it reduces to
compiled Datalog, no SX-side recursion) — only the SX proof reconstructor is.
- *Workaround in acl:* harden suite is prover-free; explain is exercised only
at shallow depth. *Real fix is in the kernel JIT* (out of acl scope) — e.g.
the `_jit_compiling` guard / disabling JIT for the recursive prover, or
fixing the bytecode loop. Recommend the kernel owners reproduce with:
`acl-explain` over a 6-deep `member_of` chain after ~70 prior explains.
- *Minimal repro recorded.* Until fixed, callers needing explanations for
deep hierarchies should flatten or cap depth, or run explain in a cold
worker.

View File

@@ -1,117 +0,0 @@
# abstraction-radar loop agent (read-only scout)
Role: continuously scan **all** rose-ash subsystems for genuine abstraction /
deduplication opportunities and maintain a ranked, evidence-backed backlog at
`plans/abstractions.md`. You are a **scout, not an implementer** — you detect and
document; you never refactor across subsystems.
```
description: abstraction-radar (read-only scout)
subagent_type: general-purpose
run_in_background: true
isolation: worktree
```
## Prompt
You are the sole background agent on branch `loops/radar`, worktree
`/root/rose-ash-loops/radar`, forever. Self-paced. Your ONLY writes are to
`plans/abstractions.md` (and, rarely, refining this briefing). Push to
`origin/loops/radar` after each update. Never touch `main` or `architecture`.
## The one hard rule: you do NOT edit `lib/**` — ever
You read across every subsystem and write findings to `plans/abstractions.md`.
You do **not** implement abstractions, migrate code, or edit any `lib/<x>/**`
file in any worktree. Implementation is a separate, coordinated, human-triggered
step — proposing well is your whole job. An abstractor that writes across
subsystems would collide with the very isolation that keeps the other loops safe;
that is exactly why you are read-only.
## Dynamic discovery — re-enumerate every iteration, never hardcode
The set of subsystems grows as new loops are spawned. Each iteration, rebuild the
list from the filesystem + tmux so newly-added subsystems are automatically in
scope:
1. `ls -d /root/rose-ash-loops/*/` — every loop worktree. For a worktree named `X`,
its in-flight subsystem is `lib/X/` **inside that worktree**
(`/root/rose-ash-loops/X/lib/X/`) — that's the current, possibly-uncommitted
state. Read it there, not from your own worktree.
2. `ls -d /root/rose-ash/lib/*/` — subsystems merged into / dormant on the main repo
(e.g. `feed` once merged, the language substrates `apl`/`haskell`/`prolog`/…).
3. `tmux ls` — which subsystems are actively looping right now (affects whether a
candidate's consumers are "stable" — see the gate).
Treat the union as your scan surface. When a `commerce` or `identity` loop appears
later, step 1 picks it up with no change to you. Note in `abstractions.md` the
date and the subsystem set you scanned, so drift is visible.
## The AHA gate — before ANY candidate goes in the backlog as "proposed"
"Avoid Hasty Abstractions." A wrong shared abstraction is far costlier than the
duplication it replaces. A candidate may be listed as **proposed** only if ALL hold:
- **≥3 real consumers** (not 2 — three independent uses). Fewer → log it under
"Watching" with its consumer count, do not propose.
- **All consumers past Phase 2 and API-stable.** If a consumer's loop is mid-flight
and its interfaces are still moving (`tmux ls` shows it active + its plan has
unchecked early-phase boxes), the pattern is a moving target → "Watching."
- **Structurally identical, not superficially similar.** Show the shared shape with
file:line evidence from each consumer. Coincidental resemblance is the #1 trap.
- **It has a natural home.** And that home is usually **not** `lib/guest` — see the
routing rule below.
Anything failing a gate goes under **Watching** (with what's missing) or
**Rejected** (with why), never silently dropped — so it isn't re-proposed each pass.
## Routing rule — most patterns do NOT belong in lib/guest
`lib/guest` is for **language-implementation plumbing** (lexer/parser/AST/HM/match/
test-runner), and it has its own consumer-gated roadmap. App-subsystem patterns
almost always have a better home — route, don't dump:
| Pattern kind | Home (not lib/guest) |
|---|---|
| per-viewer visibility / permission filter | `acl-on-sx` (delegate to `permit?`) |
| federation scaffold (merge/ingest/backfill/trust) | `fed-sx` |
| durable store / event log / kv | `persist-on-sx` |
| collection math (group-by, dedupe, stable top-N) | the substrate (APL/Haskell/…) |
| HTTP/handler/middleware plumbing | `host-on-sx` |
| conformance/test harness | `lib/guest` (the one real exception — `test-runner.sx` + the shared driver live there) |
If a pattern's home is one of the subsystems, the recommended **action** is "adopt
/ delegate there," and the work belongs to that subsystem's own loop (in its
scope), not to a cross-cutting change.
## Each iteration
1. Re-discover the subsystem set (above). Record it + the date in `abstractions.md`.
2. Pick ONE thread: either deep-dive a "Watching" candidate to gather file:line
evidence and re-test its gates, or sweep for a new recurring shape across the
current set.
3. Update `plans/abstractions.md`: move items between Watching / Proposed /
In-progress (owned by a subsystem loop) / Done / Rejected, with evidence.
4. Keep it ranked by (consumers × effort-saved ÷ risk). Short, factual.
5. Commit (`radar: <one-line finding>`) and push to `origin/loops/radar`.
Do not invent work to look busy: if a pass finds nothing that clears the gate,
record "scanned N subsystems on <date>, no new candidates cleared the gate" and
stop until next iteration. Empty passes are a valid, honest result.
## Gotchas
- SX files: `sx-tree` MCP tools take `file:` not `path:`. But you mostly READ —
prefer `sx_find_across`, `sx_comp_usage`, `sx_comp_list`, `sx_summarise`, plus
`Grep`/`Glob`/`Bash` for cross-worktree scanning.
- `plans/abstractions.md` is a `.md` — edit it with normal Write/Edit, not sx-tree.
- Never run `sx_build`. You don't build anything; you read.
## Style
- Evidence over assertion: every claim cites file:line in ≥3 consumers.
- Honest empty passes. Rejected items stay rejected with a reason.
- One finding per commit. Update. Push. Next.
Go. Read `plans/abstractions.md` (seeded), re-discover the subsystem set, and
advance the highest-value thread.

View File

@@ -1,82 +0,0 @@
# commerce-on-sx: Catalog, cart, pricing & orders on miniKanren
> **DRAFT outline.** The revenue vertical. Depends on `persist-on-sx` (durable
> orders) and `flow-on-sx` (checkout as a durable flow). Don't start before
> persist-on-sx Phase 1 is green.
rose-ash's revenue engine — market (catalog), cart (checkout), orders (SumUp
payment, reconciliation) — has no SX subsystem. The hard part of commerce isn't
CRUD; it's **pricing**: discounts, bundles, tax, membership rates, promotions that
stack (or don't). These are relations, and a relational engine can run them in
multiple directions — forward ("what's the total?") and backward ("what promo code
yields this total?", "which line item triggered the discount?").
That's a miniKanren fit. Pricing/promotion rules are relational; cart and order
*lifecycle* (reserve → pay → fulfil → reconcile) is a durable `flow`; the order
ledger is a `persist` stream. Commerce is the first real **composition** subsystem.
End-state: a catalog model, a relational pricing/promotion engine, a cart with
deterministic totals, and an order lifecycle flow with payment-webhook
reconciliation — all auditable via the event log.
## Status (rolling)
`bash lib/commerce/conformance.sh`**0/0** (not yet started)
## Ground rules
- **Scope:** only `lib/commerce/**` and `plans/commerce-on-sx.md`. May **import**
from `lib/minikanren/`, and (once they exist) `lib/persist/` + `lib/flow/`. Do not
edit substrates.
- **Architecture:** prices/promotions are miniKanren relations over catalog facts;
a cart total is a *deterministic* query result (first solution under a fixed rule
order). Order lifecycle is a `flow` that suspends at the payment IO boundary.
Money is integer minor units — never floats.
- **Determinism:** promotion stacking must have explicit, tested precedence;
totals must be reproducible from the cart + catalog snapshot.
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
Catalog + cart Total / order
product(id,price,tags) {:subtotal :discounts :tax :total}
│ ▲
▼ │
lib/commerce/catalog.sx lib/commerce/price.sx
— product / variant / stock facts — miniKanren pricing relations
│ — promo stacking, membership rates
▼ ▲
lib/commerce/cart.sx lib/commerce/order.sx (flow + store)
— line items, quantities — reserve→pay→fulfil→reconcile
│ — SumUp webhook = flow resume
▼ │
lib/commerce/api.sx ── (commerce/add) (commerce/total) (commerce/checkout) ──┘
```
## Phase 1 — Catalog + cart + deterministic totals
- [ ] `catalog.sx` — product/variant/stock as facts
- [ ] `cart.sx` — line items, add/remove/qty
- [ ] `price.sx` — base pricing relation, subtotal; tax
- [ ] `api.sx` + tests + scoreboard + conformance.sh
## Phase 2 — Promotions (relational)
- [ ] promo rules: percentage, fixed, bundle, member rate
- [ ] explicit stacking precedence; "best price" backward query
- [ ] tests: stacking order, mutually-exclusive promos, member vs guest
## Phase 3 — Order lifecycle (flow + store)
- [ ] order flow: reserve stock → await payment → fulfil
- [ ] payment webhook resumes the suspended flow
- [ ] order ledger as a `persist` stream; idempotent reconciliation
## Phase 4 — Reconciliation + federation
- [ ] mismatch detection (paid≠ordered) as queries over the ledger
- [ ] cross-instance catalog (federated marketplace) — out-of-scope stub
- [ ] tests: webhook replay, partial refund, double-charge guard
## Progress log
(loop fills this in)
## Blockers
(loop fills this in)

View File

@@ -1,82 +0,0 @@
# content-on-sx: Documents, blocks & collaborative editing on Smalltalk
> **DRAFT outline.** The CMS vertical — blog, WYSIWYG editor, Ghost sync. Depends
> on `persist-on-sx` (document history as an event log). Ghost/CMS sync stays a thin
> external adapter (Python/FFI) until a native replacement exists.
rose-ash's `blog` domain is content management: a block-based WYSIWYG editor,
navigation, Ghost CMS sync. A document is a tree of live blocks; editing is a
stream of operations; collaboration needs conflict-free merge. That is an object
model — blocks are objects, edits are messages, and a document is the object graph
responding to them. Smalltalk's "everything is an object responding to messages"
maps directly to a block/WYSIWYG model, and a semilattice (CRDT) merge keeps
concurrent edits conflict-free.
End-state: a Smalltalk-on-SX document model (typed blocks, structural ops),
operation log + CRDT merge for collaborative editing, versioning/history via the
event store, and a render boundary to HTML/SX. External CMS (Ghost) sync is an
injected adapter, not core.
## Status (rolling)
`bash lib/content/conformance.sh`**0/0** (not yet started)
## Ground rules
- **Scope:** only `lib/content/**` and `plans/content-on-sx.md`. May **import**
from `lib/smalltalk/`, and (once it exists) `lib/persist/`. Do not edit substrates.
- **Architecture:** a document is an ordered tree of blocks (objects); an edit is a
message (`insert`/`update`/`move`/`delete`); concurrent edits merge via a
commutative (CRDT/semilattice) operation so order doesn't matter. History is the
`persist` event stream; any version is a replay.
- **Determinism:** merge must be commutative + idempotent (test: apply ops in any
order / twice → same document).
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
Edit op Rendered document
(insert block after id) ... HTML / SX tree
│ ▲
▼ │
lib/content/block.sx lib/content/render.sx
— typed blocks as objects — block tree → HTML/SX
— heading/text/image/embed — (reuses SX render boundary)
│ ▲
▼ │
lib/content/doc.sx lib/content/merge.sx
— ordered block tree — CRDT/semilattice op merge
— apply op, structural moves — concurrent-edit reconciliation
│ ▲
▼ │
lib/content/api.sx ── (content/edit) (content/render) (content/history) ──┐
│ │
├── op log + versions → persist │
└── Ghost/CMS sync → injected external adapter (thin, non-core) ──┘
```
## Phase 1 — Block document model
- [ ] `block.sx` — typed block objects
- [ ] `doc.sx` — ordered tree, apply edit op, structural moves
- [ ] `render.sx` — block tree → HTML/SX
- [ ] `api.sx` + tests + scoreboard + conformance.sh
## Phase 2 — Op log + versioning
- [ ] edit ops as `persist` events; replay to any version
- [ ] `(content/history doc)`, diff between versions
## Phase 3 — Collaborative merge (CRDT)
- [ ] commutative/idempotent op merge
- [ ] concurrent-edit tests (any order, double-apply → identical)
## Phase 4 — External sync + federation
- [ ] Ghost/CMS sync via injected adapter (import/export)
- [ ] federated documents (peer-authored blocks) — trust-gated stub
- [ ] tests: round-trip import/export, conflict on concurrent external edit
## Progress log
(loop fills this in)
## Blockers
(loop fills this in)

View File

@@ -1,81 +0,0 @@
# events-on-sx: Calendar, ticketing & notification delivery on Datalog
> **DRAFT outline.** The events vertical + the shared notification-delivery edge.
> Depends on `persist-on-sx` (bookings ledger) and `flow-on-sx` (reminders, retrying
> delivery). Pairs with `commerce-on-sx` for paid tickets.
rose-ash's `events` domain is calendar + ticketing: recurring events, availability,
capacity, bookings. Scheduling is constraint reasoning — "is this slot free given
recurrence, capacity, and the attendee's other bookings?" — which is rule
evaluation over facts. Datalog expresses availability, recurrence expansion, and
capacity as rules; a booking is a transaction; reminders and digests are durable
`flow`s. Notification *delivery* (email/push) — needed here and by `feed/notify`
is folded in as an injected transport, extractable later.
End-state: a Datalog-on-SX events layer with recurrence expansion, availability +
capacity rules, transactional booking, and a flow-driven notification dispatcher
(reminders, digests, retries) over an injected transport.
## Status (rolling)
`bash lib/events/conformance.sh`**0/0** (not yet started)
## Ground rules
- **Scope:** only `lib/events/**` and `plans/events-on-sx.md`. May **import** from
`lib/datalog/`, and (once they exist) `lib/persist/` + `lib/flow/`. Do not edit
substrates.
- **Architecture:** events/availability/capacity are Datalog facts + rules;
recurrence expands to occurrence facts within a window; a booking checks rules
then appends a `persist` event (idempotent, capacity-safe). Notifications are flows
that suspend on transport IO and retry on failure.
- **Determinism:** recurrence expansion + availability must be reproducible for a
fixed window + ruleset; capacity checks must be race-safe (no overbooking).
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
Event + booking Result
event(id,start,rrule,capacity) {:booked | :full | :conflict} + reminders
│ ▲
▼ │
lib/events/calendar.sx lib/events/availability.sx
— event facts, recurrence (RRULE) — free/busy + capacity rules (Datalog)
— expand occurrences in window │
│ ▲
▼ │
lib/events/booking.sx lib/events/notify.sx (flow)
— transactional, capacity-safe — reminders / digests, retry on fail
— bookings → persist ledger — injected transport (email/push)
│ │
▼ ▼
lib/events/api.sx ── (events/schedule) (events/book) (events/agenda) ──────┘
```
## Phase 1 — Calendar + recurrence
- [ ] `calendar.sx` — event facts, RRULE expansion in a window
- [ ] `availability.sx` — free/busy rules
- [ ] `api.sx` + tests + scoreboard + conformance.sh
## Phase 2 — Ticketing + booking
- [ ] capacity rules; transactional booking → `persist` (no overbooking)
- [ ] paid tickets compose with `commerce` order flow
- [ ] tests: capacity edge, double-book guard, conflict detection
## Phase 3 — Notification delivery (flow)
- [ ] `notify.sx` — reminder/digest flows over injected transport
- [ ] retry/backoff on transport failure (flow suspend/resume)
- [ ] tests: delivery success, retry path, idempotent re-send
- [ ] NOTE: shared with `feed/notify` — candidate for later extraction to a
`delivery-on-sx` once a second consumer is real
## Phase 4 — Federation
- [ ] cross-instance events (peer calendar) — trust-gated stub
- [ ] tests: federated agenda merge
## Progress log
(loop fills this in)
## Blockers
(loop fills this in)

View File

@@ -14,7 +14,7 @@ APL, ACL visibility filtering via `lib/acl/`, federation via fed-sx.
## Status (rolling)
`bash lib/feed/conformance.sh`**189/189** (Phases 14 + TF-IDF, notifications, home, smart-dedupe, trending, mute, pagination, threading)
`bash lib/feed/conformance.sh`**0/0** (not yet started)
## Ground rules
@@ -59,118 +59,47 @@ lib/feed/api.sx lib/feed/fed.sx
## Phase 1 — Stream model + basic ops
- [x] `lib/feed/normalize.sx` — activity record schema; coerce arbitrary inputs
- [x] `lib/feed/stream.sx` — APL vector representation; filter by predicate; sort by
- [ ] `lib/feed/normalize.sx` — activity record schema; coerce arbitrary inputs
- [ ] `lib/feed/stream.sx` — APL vector representation; filter by predicate; sort by
`:at`; take N (`↑`); reverse (`⌽`)
- [x] `lib/feed/api.sx``(feed/post activity)`, `(feed/all)`
- [x] `lib/feed/tests/basic.sx`30 cases: normalize defaults, filter, sort, take, api
- [x] `lib/feed/scoreboard.{json,md}`
- [x] `lib/feed/conformance.sh`
- [ ] `lib/feed/api.sx``(feed/post activity)`, `(feed/all)`
- [ ] `lib/feed/tests/basic.sx`15+ cases: post, query, filter, sort
- [ ] `lib/feed/scoreboard.{json,md}`
- [ ] `lib/feed/conformance.sh`
## Phase 2 — Fanout via outer product
- [x] follower graph: `followers user → vector of user ids` (`feed/follow-graph`,
`feed/followers`; graph = `{followee -> (followers)}` dict)
- [x] fanout: activities `∘.×` audience → matrix via `apl-outer feed/-mk-event`
- [x] flatten to inbox events vector (`feed/-flatten` rank-2 → rank-1)
- [x] dedupe — `feed/dedupe-inbox` by `(to, actor, verb, object)`; also
`feed/dedupe-activities` `(actor verb object)` and `feed/dedupe-collapse`
`(verb object)` for cross-actor likes
- [x] `lib/feed/tests/fanout.sx` — 29 cases: small graph, mutual follow, star
(high-fanout), empty graph, unfollowed actor, cross-post dedupe
- [ ] follower graph: `followers user → vector of user ids`
- [ ] fanout: activities `∘.×` followers → matrix `(activity, follower)` pairs
- [ ] flatten to inbox events vector
- [ ] dedupe — group by `(actor, verb, object)` collapse to one inbox event per
receiver
- [ ] `lib/feed/tests/fanout.sx` — 20+ cases: small graph, mutual follow, popular
actor (high-fanout), cross-post dedupe
## Phase 3 — Aggregation + ranking
- [x] group-by — `feed/group-by`/`feed/group-count` key-reduce; `feed/by-actor-day`
buckets `(actor, day)` via `feed/day` (string-joined keys)
- [x] velocity score — `feed/velocity` counts actor's activities in `(at-window, at]`
- [x] recency score — `feed/recency` half-life decay `0.5^(age/hl)`
- [x] composite rank — `feed/composite` weighted sum of `(weight scorer)` parts
- [x] top-N per timeline — `feed/top` = rank then take
- [x] `lib/feed/tests/rank.sx` — 24 cases: decay shape, velocity burst, stable
tie-break, top-N, composite
- [ ] group-by — `(actor, day) → count` via key-reduce
- [ ] velocity score — recent activity count over window
- [ ] recency score — decay by age
- [ ] composite rank — weighted sum of components
- [ ] top-N per timeline
- [ ] `lib/feed/tests/rank.sx` — 20+ cases: ranking stable on tie, decay shape,
per-user weighting
## Phase 4 — Visibility filter + federation
`lib/acl/` and fed-sx don't exist yet and are out of scope (import `lib/apl/`
only), so ACL/transport are injected: `permit?`, `remote?`, `send-fn`, `fetch-fn`
are function parameters. Real acl-sx / fed-sx wire in at the call site unchanged.
- [x] ACL filter — `feed/visible stream viewer permit?`; default `feed/permit-acl?`
reads `:visible-to` allowlist (+ author-sees-own); per-viewer, never cached
- [x] fed-sx outbound — `feed/federate`/`feed/deliver` fan out then partition
local vs remote inboxes; remote events handed to injected `send-fn`
- [x] fed-sx inbound — `feed/inbound` normalizes + `feed/ingest` dedupes peer
activities into the local stream
- [x] backfill on subscribe — `feed/backfill local fetch-fn peer-id`
- [x] `lib/feed/tests/integration.sx` — 22 cases incl. end-to-end
`feed/timeline` (federated → ACL for viewer → recency rank → top-N)
- [ ] ACL filter — each candidate activity passed through `(acl/permit? viewer :read
activity)`
- [ ] fed-sx outbound — local `feed/post` fans out to remote followers' inboxes
- [ ] fed-sx inbound — peer activities arrive at local inbox
- [ ] backfill on subscribe — request peer history, merge into local stream
- [ ] `lib/feed/tests/integration.sx` — federated timeline with ACL applied
## Progress log
- **Phase 1 done (30/30).** Stream = APL rank-1 array whose ravel holds activity
dicts. `normalize.sx` (record schema + accessors), `stream.sx` (filter via `/`
compress, sort via `⍋` grade-up [stable], take via `↑`, reverse via `⌽`,
by-actor/verb/object/since predicates), `api.sx` (mutable log: post/all/reset!/size).
Substrate: `apl-compress`, `apl-grade-up`, `apl-take`, `apl-reverse`, `make-array`.
Grade-up returns 1-based indices (⎕IO=1), is stable on ties → deterministic sort.
- **Phase 2 done (59/59 total).** `fanout.sx` (graph + `apl-outer` showcase),
`dedupe.sx` (per-key dedupe, first-wins stable). Key APL gotcha: `scalar?` is
true for ANY dict and `disclose` nils a non-array dict, so an apl-outer combiner
MUST `enclose` its event dict — apl-outer discloses it back intact. `apl-unique`
preserves first-occurrence order; dict `keys` order is NOT stable, so
`feed/audience` sorts (else recipient ordering flakes). `apl-compress` needs a
rank-1 array, so the (activity×follower) matrix is flattened to its ravel before
the edge-guard filter.
- **Phase 3 done (83/83 total).** `aggregate.sx` (group-by/count, day buckets) +
`rank.sx` (recency/velocity/engagement scorers, composite, top-N). `sort` is
single-arg ascending only — no comparator — so ranking uses a stable two-pass
`apl-grade-down` (by :at desc, then by score desc) for deterministic tie-breaks.
Dict keys must be strings, so composite group keys are string-joined ("actor#day").
- **Phase 4 done (105/105 total).** `acl.sx` (per-viewer `feed/visible`,
`feed/timeline` capstone) + `fed.sx` (merge/ingest/inbound/backfill/federate/
deliver). ACL/transport are dependency-injected (permit?/remote?/send-fn/fetch-fn)
since lib/acl + fed-sx don't exist. `feed/normalize` now MERGEs defaults over the
raw dict (was projecting to 5 keys) so extra metadata (:visible-to, peer fields)
survives — matches the "flexible bag" principle.
(loop fills this in)
## Roadmap is complete (all 4 phases). Possible follow-ups:
## Blockers
- Wire real acl-sx once `lib/acl/` exists (swap injected `permit?`).
- Wire real fed-sx transport (swap `send-fn`/`fetch-fn`).
- [x] TF-IDF over `:tags` for content ranking — `content.sx`: `feed/tag-df`,
`feed/tag-idf` (log N/df), `feed/tfidf-score`, `feed/by-relevance`; 15 tests.
Composes as a scorer with rank.sx. (120/120 total.)
- [x] Notification feed (verb-filtered, per-recipient) — `notify.sx`:
`feed/notifications`, `feed/notify-verbs`, `feed/notify-digest` (collapses
"X, Y liked Z" by (verb,object), sorted-deterministic); 8 tests. (128/128 total.)
- [x] **Capstone** `feed/home` — the whole pipeline as one line: fanout ∘ inbox ∘
dedupe ∘ ACL ∘ rank ∘ take (`home.sx`); 6 tests incl. per-viewer ACL + cross-post
dedupe. (134/134 total.)
- [x] Per-verb dedupe rules (briefing gotcha #3) — `feed/dedupe-smart` /
`feed/smart-key`: reactions (like/follow/boost/...) collapse cross-actor on
(verb,object); posts stay distinct per actor. `feed/collapse-verbs` is
rebindable policy; 9 tests. (143/143 total.)
- [x] Trending — `feed/trending` / `feed/trending-actors`: objects/actors ranked
by activity count in a recency window, count-desc with key-asc tiebreak
(`trending.sx`); 11 tests. (154/154 total.)
- [x] Mute/block — `feed/mute-actors` / `feed/mute-tags` / `feed/mute-objects` /
`feed/apply-prefs`: viewer-controlled per-request filtering (complements ACL's
author-controlled visibility) (`mute.sx`); 9 tests. (163/163 total.)
- [x] Pagination — `feed/page`/`feed/page-count` (offset) + `feed/before`/
`feed/after`/`feed/page-before`/`feed/next-cursor` (cursor by :at, stable under
inserts) (`page.sx`); 14 tests. (177/177 total.)
- [x] Threading — `feed/replies`/`feed/reply-count`/`feed/thread`/
`feed/thread-objects`/`feed/thread-size`: conversation closure over `:reply-to`
(transitive fixpoint), chronological (`thread.sx`); 12 tests. (189/189 total.)
(none)
## Notes for next iteration
- sx-tree MCP tools take `file:` NOT `path:` (CLAUDE.md is stale). Wrong key →
`Yojson Type_error("Expected string, got null")`. Looks like a broken binary, isn't.
- sx_server binary lives in main repo: `/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe`
(worktree has no `_build`). conformance.sh already points there with relative fallback.
- Phase 2 substrate verified available: `apl-outer` (∘.×), `apl-member` (∊),
`apl-unique`, `apl-iota` (1-based).
(loop fills this in)

View File

@@ -1,100 +0,0 @@
# host-on-sx: The SX web host — off Quart, onto the kernel (Dream-bound)
> **DRAFT outline.** The integration boundary that turns the subsystem libraries
> into running services, and the strangler path off Python/Quart. This is the
> dependency hub — it imports every subsystem. Decision recorded below: native
> server + SXTP **now**, `dream-on-sx` framework layer **next**, Python only at the
> external-integration edges.
The subsystems (`feed`, `search`, `acl`, `mod`, `flow`, `commerce`, `identity`,
`content`, `events`) are libraries. Something has to receive an HTTP request, route
it, call the right subsystem, and serialize the response. Today that's Python/Quart
— the one large non-SX component in the stack: separate runtime, deploy, and
failure mode. The goal is to move the web/host/domain layer onto the SX substrate
and retire Quart, **incrementally (strangler-fig), never big-bang.**
This is already underway: a native OCaml HTTP server is live in prod on
`sx.rose-ash.com` (~3ms cached, ~323 req/s, ~2MB RSS), `defhandler`/`defpage`
exist, and a partial **SXTP** protocol is specced. That is the unblocked near-term
host — no `ocaml-on-sx` dependency.
## Two layers, two timelines
1. **Now (unblocked): native server + SXTP adapter + SX handlers.** Route rose-ash
endpoints onto the SX host one at a time. Each migrated endpoint is an SX
handler dispatching to a subsystem; Quart proxies the rest until cut over.
2. **Next: `dream-on-sx` as the framework layer.** Dream gives Quart-grade
ergonomics — typed routing, middleware stacks, sessions, CSRF. It is gated on
`ocaml-on-sx` Phases 15 + minimal stdlib. **This plan is the concrete target
user that un-parks `dream-on-sx`** (see `plans/dream-on-sx.md`): "the subsystems
need an HTTP front door" is the real feature pulling Dream. Until then, do not
block migration on Dream — the native server is sufficient.
3. **Always: Python only at the edges.** External integrations — SumUp payments,
Ghost CMS, ActivityPub crypto, IPFS/Kubo — ride Python libraries today. They
stay as thin injected adapters (Python/FFI) behind subsystem interfaces until
native replacements exist. "Drop Quart" ≠ "drop every line of Python."
## Status (rolling)
`bash lib/host/conformance.sh`**0/0** (not yet started)
## Ground rules
- **Scope:** `lib/host/**` and `plans/host-on-sx.md`. May **import** every subsystem
+ the kernel's server/SXTP surface. Do **not** edit `spec/`, `hosts/`, `shared/`,
or subsystem internals — wire to their public APIs only. Host-primitive / server
changes belong in `hosts/` (out of scope) → Blockers.
- **Architecture:** a route maps (method, path) → handler; a handler is an SX fn
`request -> response` that calls subsystem APIs; middleware is composed handlers
(auth via `identity`, permission via `acl`, mute via subsystem prefs). SXTP is the
wire format between host and subsystem-as-service.
- **Migration discipline:** each endpoint moved must be behavior-equivalent to its
Quart original (golden-response test before flip). Keep a migration ledger.
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
HTTP request HTTP response
│ ▲
▼ │
native OCaml http server (prod) ──────► lib/host/router.sx
(hosts/ — out of scope) — (method,path) → handler
│ ▲
▼ │
lib/host/middleware.sx lib/host/handler.sx
— auth(identity) ∘ acl ∘ mute ∘ ... — request → subsystem call → response
│ ▲
▼ │
lib/host/sxtp.sx subsystem APIs (feed/search/commerce/…)
— wire format, host↔service — called via public interfaces
└── external edges: SumUp / Ghost / AP / IPFS → injected Python/FFI adapters
```
## Phase 1 — Router + handler + one real endpoint
- [ ] `router.sx` — route table, (method,path) match
- [ ] `handler.sx` — request/response model, subsystem dispatch
- [ ] migrate ONE read endpoint (e.g. a feed timeline) end-to-end, golden test
- [ ] `conformance.sh` + scoreboard
## Phase 2 — Middleware + SXTP
- [ ] `middleware.sx` — composable auth/acl/mute/error layers
- [ ] `sxtp.sx` — host↔subsystem wire format (align with existing spec)
- [ ] migrate a write endpoint (auth + permission + action)
## Phase 3 — Strangler migration ledger
- [ ] enumerate Quart endpoints; track migrated vs proxied
- [ ] golden-response harness vs the live Quart responses
- [ ] cut over a whole domain (smallest: `likes` or `relations`) as proof
## Phase 4 — Dream framework layer (gated)
- [ ] gate: `ocaml-on-sx` Phases 15 + minimal stdlib green
- [ ] adopt `dream-on-sx` routing/middleware/session ergonomics over the same handlers
- [ ] re-home external adapters as native where replacements land
## Progress log
(loop fills this in)
## Blockers
(loop fills this in)

View File

@@ -1,84 +0,0 @@
# identity-on-sx: OAuth2, sessions & membership on Erlang
> **DRAFT outline.** The identity core `acl-on-sx` assumes already exists. `acl`
> answers "may X do Y"; identity answers "who is X, and how did they prove it."
> Depends on `persist-on-sx` (grant/audit ledger). Pairs with `acl-on-sx`.
rose-ash's `account` domain is the OAuth2 authorization server every other app is
a client of: silent SSO, per-app first-party cookies, grant verification,
membership. Sessions and grants are **long-lived, concurrent, individually
addressable, and expire on their own** — that is the actor model. Erlang's
processes + mailboxes map cleanly: a session is a process, token issue/refresh/
revoke are messages, expiry is a process timeout, and SSO is one process answering
many apps.
End-state: an Erlang-on-SX layer with the OAuth2 authorization-code + silent
(`prompt=none`) flows as message protocols, a session/grant registry, token
lifecycle (issue/refresh/revoke/introspect), and membership state — all auditable
through the event log, all authorization questions delegated to `acl-on-sx`.
## Status (rolling)
`bash lib/identity/conformance.sh`**0/0** (not yet started)
## Ground rules
- **Scope:** only `lib/identity/**` and `plans/identity-on-sx.md`. May **import**
from `lib/erlang/`, and (once they exist) `lib/persist/` + `lib/acl/`. Do not edit
substrates.
- **Architecture:** a session/grant is a process holding its own state; the
registry routes messages by subject/client id. Tokens are opaque + introspected,
not self-validating (revocation must be real). Authorization decisions are NOT
made here — `identity` proves identity, `acl` decides permission.
- **Security:** revocation is immediate (kill the process / tombstone the grant);
no decision relies on a token that outlived its grant. Negative answers are
explicit, never "absence of a yes."
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
Auth request Token / session
(authorize client scope subject) {:access :refresh :expires :grant}
│ ▲
▼ │
lib/identity/oauth.sx lib/identity/token.sx
— authz-code + prompt=none flows — issue / refresh / revoke / introspect
— as Erlang message protocols — opaque tokens, grant-backed
│ ▲
▼ │
lib/identity/session.sx lib/identity/registry.sx
— session = process, expiry=timeout — route by subject/client; SSO fan-out
│ │
▼ ▼
lib/identity/api.sx ── (identity/login) (identity/grant?) (identity/revoke) ──┐
│ │
└──────── grant + audit events → persist ; permission? → acl ──────────┘
```
## Phase 1 — Sessions + tokens
- [ ] `session.sx` — session process, create/lookup/expire
- [ ] `token.sx` — issue/introspect/revoke (opaque, grant-backed)
- [ ] `registry.sx` — route by subject/client
- [ ] `api.sx` + tests + scoreboard + conformance.sh
## Phase 2 — OAuth2 flows
- [ ] authorization-code flow as a message protocol
- [ ] refresh + rotation; revocation cascades to issued tokens
- [ ] tests: full code exchange, refresh, revoke-then-use (must fail)
## Phase 3 — Silent SSO + membership
- [ ] `prompt=none` cross-app login (one session, many clients)
- [ ] membership state + per-app grant projection
- [ ] grant verification delegated cache (mirror Redis-cache pattern)
## Phase 4 — Audit + federation
- [ ] every issue/refresh/revoke is a `persist` event; `(identity/audit subject)`
- [ ] federated identity (peer-asserted subject) — advisory, trust-gated stub
- [ ] tests: audit completeness, cross-instance subject mapping
## Progress log
(loop fills this in)
## Blockers
(loop fills this in)

View File

@@ -1,119 +0,0 @@
# persist-on-sx: Durable state on the SX kernel
> **DRAFT outline.** Foundation subsystem — the durable substrate the other five
> currently fake with in-memory mutable lists. Build this first.
>
> **"persist" = persistence / data store, NOT the shop.** The shop/commerce vertical
> is `commerce-on-sx`.
rose-ash needs durable state: every subsystem (feed log, flow store, mod audit,
search index, acl grants, sessions) today hand-rolls an in-memory structure that
vanishes on restart. `persist-on-sx` is the one durable substrate they share. It
lives directly on the SX kernel's IO-suspension primitives (`perform`/`cek-resume`
— the third CEK phase) so a read/write `perform`s and the kernel persists at the
boundary. Concrete storage backends are injected.
## Does it cover ALL persistence? No — and on purpose.
Event-sourcing-everything is a known trap (replay cost, event schema evolution,
awkward ad-hoc queries, 5MB images in a log). So persist owns the **durable
source-of-truth substrate**, exposed as **two facets over one backend protocol**,
with two things explicitly delegated out:
| Shape | Owner | Notes |
|-------|-------|-------|
| **Event streams** (append-only, history matters) | persist — **log facet** | feed activities, mod audit, order ledger, flow state, content edits |
| **Current-state values** (KV / document, no history) | persist — **kv facet** | profiles, stock counts, config, session blobs; also where projections materialize |
| **Snapshots / read models** (derived, queryable) | persist — projections → kv/log | rebuildable from the log; persisted so you don't replay to answer a query |
| **Blobs / large objects** (images, media) | **delegated** → content-addressed store (artdag/IPFS already) | persist stores the *reference/CID*, never the bytes |
| **Cache** (ephemeral, evictable) | **out of scope** | not persistence — different lifecycle (Redis-shaped) |
| **Ad-hoc relational query** | the subsystem, over a projected read model | the log is bad at "all orders by X in March"; project into a queryable kv/SQL backend |
So: persist is the **single durable substrate** for state that's either a stream of
changes or a current value — but it does **not** force everything into an event
log, it does **not** hold blobs (only their content-addressed refs), and it does
**not** do caching. Those boundaries are the whole point of calling it a substrate
rather than "the database."
End-state: `log` (append/read streams) + `kv` (get/put/delete by key) facets, an
injectable backend protocol (mem → file → Postgres → IPFS-ref), pure projections
with incremental snapshots, optimistic concurrency, and a subscription hook so
read models (feeds, indices, audit logs) update incrementally.
## Status (rolling)
`bash lib/persist/conformance.sh`**0/0** (not yet started)
## Ground rules
- **Scope:** only `lib/persist/**` and `plans/persist-on-sx.md`. May **import** the
kernel's IO-suspension surface (`perform`, platform IO ops) — verify what's
exported first. Do not add host primitives; a missing durable IO op is a Blockers
entry (it belongs in `hosts/`, out of scope).
- **Architecture:** an event is `{:stream :seq :type :at :data}`; the log is an
ordered append-only vector; a projection is `(fold step seed events)`; a kv value
is `(get/put/delete key)`. Both facets sit on one injected backend
`{:append :read :kv-get :kv-put :snapshot-read :snapshot-write}`. The in-memory
backend is the test default; real backends wire in unchanged.
- **Determinism:** replay is pure — same log → same state, always. No clocks or
randomness inside projections; time lives on the event.
- **Blobs:** store the content-address/CID and metadata; never the bytes. The blob
backend is a separate injected dependency.
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
Command / write Read model / value
(append stream type data) (project stream step seed)
(kv-put key value) (kv-get key)
│ ▲
▼ │
lib/persist/event.sx lib/persist/project.sx
— {:stream :seq :type :at :data} — fold step seed; incremental from snapshot
│ ▲
▼ │
lib/persist/log.sx lib/persist/kv.sx lib/persist/snapshot.sx
— append/read — get/put/delete — checkpoint; replay = snapshot + tail
— optimistic seq — current-state
│ │ ▲
└──────────────────┴── (perform → backend) ───┘
lib/persist/backend.sx lib/persist/api.sx
— injected protocol — (persist/append) (persist/project)
— mem | file | pg | ipfs-ref — (persist/kv-get/put) (persist/subscribe)
└── blobs → content-addressed store (artdag/IPFS), by reference only
```
## Phase 1 — Log + kv + in-memory backend
- [ ] `event.sx` — event record, stream/seq helpers
- [ ] `backend.sx` — injectable protocol + in-memory impl (log + kv)
- [ ] `log.sx``append` (optimistic seq), `read`, `read-from`
- [ ] `kv.sx``get`/`put`/`delete` current-state
- [ ] `api.sx` + tests + scoreboard + conformance.sh
## Phase 2 — Projections + subscriptions
- [ ] `project.sx``(project stream step seed)`, incremental fold
- [ ] subscription hook — projection / kv read model re-runs on append
- [ ] concurrency conflict surfaced as a real result, not a crash
## Phase 3 — Snapshots + replay
- [ ] `snapshot.sx` — checkpoint a projection; replay = snapshot + tail
- [ ] compaction policy; replay-determinism tests
## Phase 4 — Durable backends via kernel IO
- [ ] file/log backend driven through `perform` (IO-suspension boundary)
- [ ] blob backend interface (store ref/CID; bytes live in artdag/IPFS)
- [ ] crash/restart replay test (mock IO platform)
- [ ] migration notes for swapping mem → durable under a live subsystem
## Consumers (post-foundation, not in scope here)
feed/-log, flow store, mod/audit, search index, acl grants, identity sessions all
become `persist` log or kv. Track each migration in that subsystem's plan.
## Progress log
(loop fills this in)
## Blockers
(loop fills this in)

View File

@@ -1,170 +0,0 @@
# Re-implementing rose-ash on SX — migration strategy
Status: **strategy proposal** (drafted by the `radar` loop, 2026-06-07). Not a
unilateral architecture decision — a starting point for the fleet to refine. Radar's
role here is detection: the `*-on-sx` subsystems have converged into a host-agnostic
re-implementation of rose-ash's domain logic, so this doc proposes *when* and *how* to
wire them to production.
---
## 1. Premise: we are ~70% into a re-implementation already
The fleet of `lib/<x>` SX subsystems is not a set of experiments — it is rose-ash's
domain logic, re-expressed substrate-by-substrate, deliberately **host-agnostic**:
| SX subsystem (`lib/`) | rose-ash production domain |
|---|---|
| content-on-sx (CRDT docs, versioning, `page.sx` HTML render) | **blog** |
| commerce-on-sx (catalog, pricing, cart, order + refund sagas) | **market + cart + orders** |
| events-on-sx (calendar, ticketing, booking) | **events** |
| feed-on-sx (activity streams, AP-shaped, threading) | **federation** |
| identity-on-sx (OAuth2, sessions, grants, membership) | **account** |
| acl-on-sx (permissions) | cross-cutting authZ |
| relations / likes | **relations / likes** (internal) |
| persist-on-sx (log / kv / snapshot facets) | per-service Postgres layer |
| flow-on-sx (durable sagas) | order/refund/delivery workflows |
| mod-on-sx, search-on-sx | new capabilities |
**The architectural enabler:** every core was built with *injected seams*`permit?`,
`send-fn`/`fetch-fn`, `transport`, `dispatch`, `backend`. That is ports-and-adapters
(hexagonal) on purpose. Evidence from the radar backlog (`plans/abstractions.md`):
W1 (7/7 federation modules inject the fed-sx transport), W4 (content/commerce/events run
live on `persist/log`), W8 (events+commerce run sagas on `lib/flow`). **The cores do not
depend on how they're hosted, persisted, or federated.**
**Corollary that makes the whole migration tractable:** because logic is separated from
rendering and storage, we can hold the **domain logic to parity** while **freely
redesigning the presentation** — the two are different layers with different rules.
---
## 2. The gating insight: the cores are *ahead of the host*
The domain logic is mature. What is *not* yet production-grade is the **host trio** — and
that is the real critical path:
- **host-on-sx** — HTTP / request-response / session host (briefing exists; the OCaml SX
HTTP server already serves `sx.rose-ash.com`).
- **host-persist** — durable storage adapter (real disk/pg/ipfs) under `persist`'s
facets (content-addressed blob blocker recently closed).
- **fed-sx** — the real ActivityPub transport every core injects (well into m2).
> **So "when do we start?" answers itself: start when the host trio is production-grade,
> not when the cores are done — they mostly already are.** Prioritise the host loops over
> further domain features.
---
## 3. The model: duplicate → cut over → diverge (per slice)
This is the "duplicate first, then change" approach, made precise. Each domain slice goes
through three phases independently:
**Phase A — Duplicate (hold logic to parity).** Stand the SX implementation of the slice
up *in parallel*, behind the existing edge, serving no users yet. Get its **domain/data
behaviour** to match Python (see §4 on how). Presentation can start as a rough port or an
early new design — it doesn't have to match.
**Phase B — Cut over (strangler flip).** Point the edge route for that slice at the SX
host. Python stays as instant rollback. The slice is now live on SX.
**Phase C — Diverge (change freely).** With the slice live and validated, evolve the
look/feel and functionality on the SX side. The validated domain logic underneath is
untouched, so UX/feature changes can't silently corrupt data.
You never rewrite the whole platform at once; you walk slices through A→B→C, oldest tree
strangled last.
---
## 4. The two techniques, and how "we'll change things" reshapes them
### Strangler edge
The edge (Caddy) is the front door every request hits. Add routing rules so **one route
at a time** goes to the SX host while everything else still goes to Python. Properties:
the site is never half-broken; any single route flips back to Python instantly; the old
app is strangled route-by-route. (Opposite of big-bang swap, which is how these die.)
### Shadow diff — split by layer
Run the new version on real traffic in the background, discard its output, and **log how
it differs** from Python. Flip the edge only when diffs are zero/intended.
But because we *intend* to change look/feel + functionality, parity is a tool we apply
**only where we want sameness**, not a straitjacket:
| Layer | Want parity? | Oracle |
|---|---|---|
| **Domain/data** (totals, tax, permissions, what's stored, who-sees-what) | **YES — silent difference = data corruption** | shadow-diff at the *core* boundary; deterministic cores → replay real request logs through the harness and diff |
| **Presentation/UX** (HTML, layout, look, feel, flows) | **NO — this is what we're changing** | manual QA + design review; this is the Phase-C divergence |
Practical shape: shadow-diff hits the **domain core's output** (the computed order, the
visible-activity set, the permission decision) — not the rendered HTML. The deterministic,
harness-replayable cores are the single biggest advantage we have here; it's the same
parity discipline that made the A1 conformance migration safe (one reference slice, hard
parity gate, revert on mismatch).
---
## 5. Readiness gates (start the production migration when ALL hold)
1. **Host trio production-grade** — host-on-sx (HTTP/session), host-persist (durable
adapter), fed-sx (AP transport) — each conformance-green.
2. **Data-migration story exists** — a way to get existing production Postgres state into
`persist` event streams (event-source the current state, or dual-write during overlap).
This is the honest long-pole; it is *not* domain logic and nobody has built it yet.
3. **One vertical slice proven end-to-end** at data-parity in production — the reference
migration, the way the conformance loop migrated one subsystem before the rest.
---
## 6. Sequencing
1. **Host trio first** (critical path — it's behind the cores).
2. **Build the strangler edge + shadow-diff harness** as first-class tooling: edge routing
rules + a dual-run logger that diffs *core outputs* (not HTML) and stores discrepancies.
3. **First slice = lowest risk × highest readiness × cleanest data oracle.**
Recommended: **the blog read path (content-on-sx)** or **the feed read path**
— read-heavy, no money, CRDT/versioning + `page.sx` HTML already exist, and the data
oracle is clean. *Avoid cart/orders/payments first* (transactional + SumUp webhooks =
highest blast radius).
4. **Persistence-first, federation-last.** Land host-persist + migrate per-domain event
stores before any cutover. Do fed-sx federation as a *coordinated* cut near the end —
W1 shows all 7 cores light up federation together once the shared transport ships.
5. **Walk the remaining slices A→B→C**, retiring Python routes as each cuts over.
---
## 7. The honest long tail (mostly host + adapters, not cores)
The cores are pure domain logic; the production *tail* is not in them yet and is most of
the remaining real effort:
- Auth: first-party cookies / Safari-ITP, CSRF, silent SSO, grant caching.
- Cross-cutting: rate limiting, observability/metrics, error pages, caching.
- Integrations: SumUp payment + webhooks, Ghost CMS sync.
- Presentation: the actual HTMX templates + CSS (this is also where the redesign happens).
- **Live data migration** — the single biggest non-core workstream.
---
## 8. Concrete next steps
1. Treat the **host trio** as the fleet's critical path; prioritise over more domain features.
2. Stand up the **strangler edge + core-level shadow-diff harness** as a tool.
3. Prove **one slice** (blog/content read path) end-to-end in production as the reference.
4. **Spec the Postgres → persist data migration** (the long-pole nobody has started).
5. Then walk slices through duplicate → cut over → diverge, redesigning UX in Phase C.
---
## 9. Why this is low-risk despite being a platform rewrite
- It's **wiring host-agnostic cores to a host**, not rewriting domain logic from scratch.
- The **strangler edge** means the site always works and any route reverts in seconds.
- **Deterministic cores** make data-parity *mechanically checkable* (replay + diff), so
correctness isn't a matter of faith.
- **Logic/presentation separation** lets us change look/feel + functionality (Phase C)
*without* re-risking the validated domain logic.
- It's the **same discipline that just shipped A1**: one reference migration, a hard
parity gate, honest exclusions, verify-before-merge.