Compare commits

..

28 Commits

Author SHA1 Message Date
5d62d08e1c search: did-you-mean spelling suggestion + 9 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 30s
suggest/suggestN rank indexed terms by edit distance to a (misspelled) query
term, alphabetical tiebreak. 234/234.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-07 00:46:22 +00:00
db2a5dc6ab search: boolean-filtered ranked search + 11 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 40s
searchRankTfIdf/searchRankBm25 parse a boolean query, filter docs via evalQuery,
then rank survivors by relevance over the query's leaf terms (queryTerms) — the
filter-then-rank pattern. 225/225.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 23:58:37 +00:00
cfa68c3db3 search: synonym / query expansion + 9 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 19s
A synonym map [(Term,[Term])] expands a query term to itself + synonyms
(expandTerm); synDocs unions and synRankTfIdf ranks the expanded set. 214/214.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 23:27:03 +00:00
cf4e613e43 search: proximity/NEAR search + 9 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 24s
nearDocs k t1 t2 returns docs where both terms occur within k positions
(unordered); candidates from the posting intersection, filtered on positional
postings. 205/205.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 23:01:42 +00:00
911a2f57c0 search: stemming (suffix stripping) + 18 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 16s
Deterministic English suffix stripping (stem), stemText/stemTokens, indexStemmed.
Worked around two haskell-on-sx string gotchas: take/drop over a String yield
char codes (rebuild via joinChars . map chr), and isSuffixOf's reverse trips ++
(manual suffix compare). 196/196.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 22:50:19 +00:00
7231cb651f search: highlight + snippet generation + 12 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 26s
highlight marks query-matching (normalized) tokens with [..]; snippet extracts a
context window around the first match. 178/178.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 22:08:00 +00:00
5945b51cfd search: fuzzy matching via edit distance + 18 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 41s
editDist as an O(m*n) row-based Levenshtein DP (naive recursion is exponential
and times out under load); fuzzyTerms/fuzzyDocs/fuzzyRankTfIdf expand a term to
indexed terms within a max edit distance. 166/166.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 21:47:56 +00:00
3ab8270a58 search: result pagination (offset/limit) + 12 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 26s
paginate windows a ranked list (take lim . drop off); pageTfIdf/pageBm25 and
resultCount. 148/148.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 20:55:25 +00:00
9d3b775b25 search: prefix/wildcard queries + 14 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 31s
prefixTerms matches indexed terms by prefix (allTerms + isPrefixOf); prefixDocs
unions their docs; prefixRankTfIdf ranks via the matched terms. 136/136.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 20:22:23 +00:00
77ab827b91 search: Phase 4 federation merge + ACL post-filter + 21 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 39s
fedIndex merges per-peer inverted indices (union posting lists per term) after
relabelling local DocIds to global gid = peer*1000 + local — dedupe by
(peer,doc-id) is automatic and positions survive, so ranking runs once over the
merge and interleaves peers by score. ACL is a post-rank filter over an injected
permit predicate (searchTfIdfAcl/topNTfIdfAcl/searchBm25Acl). Roadmap complete,
122/122.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 20:08:08 +00:00
a3f9d4f6c9 search: Phase 3 ranking TF-IDF + BM25 + top-N + 23 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 37s
rankTfIdf and rankBm25 (configurable k1/b) over the candidate set, float scores
with deterministic DocId tiebreak; topNTfIdf/topNBm25. df/idf derived from
posting-list length. Tests cover tf/idf behavior, a BM25-vs-TF-IDF flip from
length-norm + tf-saturation, the b-parameter effect, tiebreak stability. 101/101.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 19:56:50 +00:00
4c84decc01 search: Phase 2 query parser + 32 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 46s
Query tokenizer + recursive-descent parser: OR<AND<NOT precedence, implicit AND
on adjacency, quoted phrases, parens, case-insensitive keywords. parseQuery,
searchQuery, showQ. Worked around haskell-on-sx parser limits (ord-based
delimiters; multi-clause fns instead of []-pattern case alts). 78/78.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 19:43:10 +00:00
0f0da0319c search: Phase 2 query AST + boolean/phrase eval + 28 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 53s
Query ADT (Term|And|Or|Not|Phrase) and evalQuery over docid-sorted posting
lists: boolean ops as linear merges, Not over the allDocs universe, Phrase via
positional adjacency. Batched both test suites into one program eval each
(search-batch) so they finish under heavy CPU load. 46/46.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 18:47:42 +00:00
b8cf3eb1b8 search: Phase 1 tokenizer + inverted index + 18 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 53s
Tokenizer (lowercase, strip punctuation, positions) and a sorted assoc-list
inverted index [(Term,[(DocId,[Pos])])] with indexDoc/deleteDoc/lookupTerm/
docFreq/allTerms. Search lib is haskell-on-sx source assembled into search/src;
tests reuse hk-test counters via a search-eval helper. conformance.sh models
lib/haskell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 18:21:49 +00:00
e2de5a4675 briefings: add search-on-sx loop briefing
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m14s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:27:20 +00:00
1e4cf25015 Merge loops/feed into architecture: feed-on-sx activity feed engine on APL
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 42s
Activity feeds as APL array math on lib/apl/ — timelines, fanout, ranking,
visibility, federation. Roadmap (4 phases) + 8 extensions, 189/189 tests.

- Phase 1: stream model (normalize, filter/sort/take/reverse)
- Phase 2: fanout via outer product (∘.×), edge-guard, dedupe
- Phase 3: aggregation + ranking (recency/velocity/engagement, top-N)
- Phase 4: per-viewer ACL + federation (injected permit?/transport)
- Extensions: TF-IDF, notifications, home capstone, smart-dedupe,
  trending, mute, pagination, threading

Purely additive under lib/feed/**; no conflicts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:23:42 +00:00
9c4a5d1913 feed: conversation threading — :reply-to transitive closure (thread/replies/thread-size) + 12 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 42s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 17:00:10 +00:00
f91ac82434 feed: pagination — offset/limit + cursor-by-at (before/after/page-before/next-cursor) + 14 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 44s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:58:36 +00:00
5136249ae5 feed: viewer mute/block — mute actors/tags/objects + apply-prefs bag + 9 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 42s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:57:05 +00:00
6fc61147a8 feed: trending objects/actors by recent activity window, deterministic tiebreak + 11 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 40s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:55:55 +00:00
0122c41ecb feed: verb-aware smart dedupe — reactions collapse cross-actor, posts stay per-actor + 9 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 54s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:54:21 +00:00
58656b03e4 feed: feed/home capstone — fanout∘inbox∘dedupe∘ACL∘rank∘take as one line + 6 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 42s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:53:15 +00:00
b0feb7b01b feed: notification feed — per-recipient inbox, verb filter, (verb,object) digest + 8 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 54s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:51:53 +00:00
a979297959 feed: TF-IDF content ranking over :tags — tag-df/idf, tfidf-score, by-relevance + 15 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 43s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:50:36 +00:00
37226cf6eb feed: Phase 4 visibility + federation — per-viewer ACL, fanout partition, inbound/backfill/ingest, e2e feed/timeline + 22 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 36s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:48:27 +00:00
50a7f31a39 feed: Phase 3 aggregation + ranking — group-by, recency/velocity/engagement scorers, composite, top-N via stable grade-down + 24 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 42s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:44:04 +00:00
915f51b2b6 feed: Phase 2 fanout via outer product — activities ∘.× audience, flatten, edge-guard, dedupe + 29 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 21s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:40:34 +00:00
e7501bdf8f feed: Phase 1 stream model — normalize, APL-backed filter/sort/take/reverse, post/all api + 30 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 49s
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-06 16:31:36 +00:00
118 changed files with 4052 additions and 5446 deletions

38
lib/feed/acl.sx Normal file
View File

@@ -0,0 +1,38 @@
; feed/acl — per-viewer visibility filtering. The same candidate stream yields
; different timelines for different viewers, so ACL is applied per request and
; pre-ACL timelines are never cached.
;
; permit? is injected: (permit? viewer activity) -> bool. Wire a real acl-sx
; predicate here; feed/permit-acl? is a self-contained default that reads an
; optional :visible-to allowlist on the activity.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem?), lib/feed/rank.sx (feed/top).
; default permit: actor always sees own activity; absent/nil :visible-to is
; public; otherwise viewer must be in the allowlist.
(define
feed/permit-acl?
(fn
(viewer a)
(or
(equal? viewer (get a :actor))
(let
((allowed (get a :visible-to nil)))
(if (= allowed nil) true (feed/-elem? viewer allowed))))))
(define feed/permit-public? (fn (viewer a) true))
; filter a stream to what viewer may read
(define
feed/visible
(fn
(stream viewer permit?)
(feed/filter stream (fn (a) (permit? viewer a)))))
; the capstone: candidate stream -> ACL for viewer -> rank -> top-N
(define
feed/timeline
(fn
(stream viewer permit? score-fn n)
(feed/top (feed/visible stream viewer permit?) score-fn n)))

62
lib/feed/aggregate.sx Normal file
View File

@@ -0,0 +1,62 @@
; feed/aggregate — group-by / counting via key-reduce. Keys must be strings
; (dict keys), so composite keys (actor, day) are joined into one string.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx.
; group activities into a dict: key-string -> (list of activities), order-preserving
(define
feed/group-by
(fn
(stream key-fn)
(reduce
(fn
(g a)
(let
((k (key-fn a)))
(assoc g k (append (get g k (list)) (list a)))))
{}
(feed/items stream))))
; key-string -> count
(define
feed/group-count
(fn
(stream key-fn)
(reduce
(fn
(g a)
(let
((k (key-fn a)))
(assoc g k (+ (get g k 0) 1))))
{}
(feed/items stream))))
; --- composite keys ---------------------------------------------------------
(define feed/day (fn (at window) (floor (/ at window))))
; (actor, day-bucket) -> "actor#day"
(define
feed/actor-day-key
(fn
(window)
(fn
(a)
(string-append
(get a :actor)
"#"
(number->string (feed/day (get a :at) window))))))
(define
feed/by-actor-day
(fn (stream window) (feed/group-count stream (feed/actor-day-key window))))
; per-actor activity counts
(define
feed/actor-counts
(fn (stream) (feed/group-count stream feed/actor)))
; per-object activity counts (engagement)
(define
feed/object-counts
(fn (stream) (feed/group-count stream feed/object)))

24
lib/feed/api.sx Normal file
View File

@@ -0,0 +1,24 @@
; feed/api — ergonomic API over the stream layer for non-APL callers.
; A single mutable activity log; post appends, all returns it as a stream.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx (loaded by harness).
(define feed/-log (list))
; post — normalize then append. Returns the stored activity.
(define
feed/post
(fn
(raw)
(let
((a (feed/normalize raw)))
(begin (set! feed/-log (append feed/-log (list a))) a))))
; all — the whole log as a stream (insertion order)
(define feed/all (fn () (feed/stream feed/-log)))
; reset! — clear the log (test hygiene)
(define feed/reset! (fn () (begin (set! feed/-log (list)) nil)))
; size — number of posted activities
(define feed/size (fn () (len feed/-log)))

125
lib/feed/conformance.sh Executable file
View File

@@ -0,0 +1,125 @@
#!/usr/bin/env bash
# lib/feed/conformance.sh — run feed test suites, emit scoreboard.json + scoreboard.md.
set -uo pipefail
cd "$(git rev-parse --show-toplevel)"
SX_SERVER="${SX_SERVER:-/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe}"
if [ ! -x "$SX_SERVER" ]; then
SX_SERVER="hosts/ocaml/_build/default/bin/sx_server.exe"
fi
if [ ! -x "$SX_SERVER" ]; then
echo "ERROR: sx_server.exe not found." >&2
exit 1
fi
SUITES=(basic fanout rank integration content notify home dedupe trending mute page thread)
OUT_JSON="lib/feed/scoreboard.json"
OUT_MD="lib/feed/scoreboard.md"
run_suite() {
local suite=$1
local file="lib/feed/tests/${suite}.sx"
local TMP
TMP=$(mktemp)
cat > "$TMP" << EPOCHS
(epoch 1)
(load "spec/stdlib.sx")
(load "lib/r7rs.sx")
(load "lib/apl/runtime.sx")
(load "lib/feed/normalize.sx")
(load "lib/feed/stream.sx")
(load "lib/feed/api.sx")
(load "lib/feed/fanout.sx")
(load "lib/feed/dedupe.sx")
(load "lib/feed/aggregate.sx")
(load "lib/feed/rank.sx")
(load "lib/feed/acl.sx")
(load "lib/feed/fed.sx")
(load "lib/feed/content.sx")
(load "lib/feed/notify.sx")
(load "lib/feed/home.sx")
(load "lib/feed/trending.sx")
(load "lib/feed/mute.sx")
(load "lib/feed/page.sx")
(load "lib/feed/thread.sx")
(epoch 2)
(eval "(define feed-test-pass 0)")
(eval "(define feed-test-fail 0)")
(eval "(define feed-test (fn (name got expected) (if (= got expected) (set! feed-test-pass (+ feed-test-pass 1)) (set! feed-test-fail (+ feed-test-fail 1)))))")
(epoch 3)
(load "${file}")
(epoch 4)
(eval "(list feed-test-pass feed-test-fail)")
EPOCHS
local OUTPUT
OUTPUT=$(timeout 300 "$SX_SERVER" < "$TMP" 2>/dev/null)
rm -f "$TMP"
local LINE
LINE=$(echo "$OUTPUT" | awk '/^\(ok-len 4 / {getline; print; exit}')
if [ -z "$LINE" ]; then
LINE=$(echo "$OUTPUT" | grep -E '^\(ok 4 \([0-9]+ [0-9]+\)\)' | tail -1 \
| sed -E 's/^\(ok 4 //; s/\)$//')
fi
local P F
P=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\1/')
F=$(echo "$LINE" | sed -E 's/^\(([0-9]+) ([0-9]+)\).*/\2/')
P=${P:-0}
F=${F:-0}
echo "${P} ${F}"
}
declare -A SUITE_PASS
declare -A SUITE_FAIL
TOTAL_PASS=0
TOTAL_FAIL=0
echo "Running feed conformance suite..." >&2
for s in "${SUITES[@]}"; do
read -r p f < <(run_suite "$s")
SUITE_PASS[$s]=$p
SUITE_FAIL[$s]=$f
TOTAL_PASS=$((TOTAL_PASS + p))
TOTAL_FAIL=$((TOTAL_FAIL + f))
printf " %-12s %d/%d\n" "$s" "$p" "$((p+f))" >&2
done
# scoreboard.json
{
printf '{\n'
printf ' "suites": {\n'
first=1
for s in "${SUITES[@]}"; do
if [ $first -eq 0 ]; then printf ',\n'; fi
printf ' "%s": {"pass": %d, "fail": %d}' "$s" "${SUITE_PASS[$s]}" "${SUITE_FAIL[$s]}"
first=0
done
printf '\n },\n'
printf ' "total_pass": %d,\n' "$TOTAL_PASS"
printf ' "total_fail": %d,\n' "$TOTAL_FAIL"
printf ' "total": %d\n' "$((TOTAL_PASS + TOTAL_FAIL))"
printf '}\n'
} > "$OUT_JSON"
# scoreboard.md
{
printf '# feed Conformance Scoreboard\n\n'
printf '_Generated by `lib/feed/conformance.sh`_\n\n'
printf '| Suite | Pass | Fail | Total |\n'
printf '|-------|-----:|-----:|------:|\n'
for s in "${SUITES[@]}"; do
p=${SUITE_PASS[$s]}
f=${SUITE_FAIL[$s]}
printf '| %s | %d | %d | %d |\n' "$s" "$p" "$f" "$((p+f))"
done
printf '| **Total** | **%d** | **%d** | **%d** |\n' "$TOTAL_PASS" "$TOTAL_FAIL" "$((TOTAL_PASS + TOTAL_FAIL))"
} > "$OUT_MD"
echo "Wrote $OUT_JSON and $OUT_MD" >&2
echo "Total: $TOTAL_PASS pass, $TOTAL_FAIL fail" >&2
[ "$TOTAL_FAIL" -eq 0 ]

68
lib/feed/content.sx Normal file
View File

@@ -0,0 +1,68 @@
; feed/content — TF-IDF relevance over activity :tags. Rare tags carry more
; signal, so an activity matching an uncommon tag ranks above one matching a
; common tag. Composes with rank.sx: feed/tfidf-score is just another scorer.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-distinct), lib/feed/rank.sx (feed/rank).
; document frequency: tag -> number of activities whose :tags contain it
; (a tag repeated within one activity counts once toward df)
(define
feed/tag-df
(fn
(stream)
(reduce
(fn
(df a)
(reduce
(fn (d t) (assoc d t (+ (get d t 0) 1)))
df
(feed/-distinct (get a :tags))))
{}
(feed/items stream))))
; inverse document frequency: tag -> log(N / df)
(define
feed/tag-idf
(fn
(stream)
(let
((n (feed/count stream)) (df (feed/tag-df stream)))
(reduce
(fn (idf t) (assoc idf t (log (/ n (get df t)))))
{}
(keys df)))))
; term frequency within one activity: tag -> occurrence count
(define
feed/-tf
(fn
(a)
(reduce
(fn (tf t) (assoc tf t (+ (get tf t 0) 1)))
{}
(get a :tags))))
; relevance of an activity to a query (list of tags) given precomputed idf:
; sum over query tags of tf(tag in activity) * idf(tag in corpus)
(define
feed/tfidf-score
(fn
(idf query)
(fn
(a)
(let
((tf (feed/-tf a)))
(reduce
(fn
(acc t)
(+ acc (* (get tf t 0) (get idf t 0))))
0
query)))))
; rank a stream by relevance to query tags (idf computed over the stream itself)
(define
feed/by-relevance
(fn
(stream query)
(feed/rank stream (feed/tfidf-score (feed/tag-idf stream) query))))

76
lib/feed/dedupe.sx Normal file
View File

@@ -0,0 +1,76 @@
; feed/dedupe — collapse duplicate items, keeping first occurrence per key.
; Each verb may want its own key (see briefing): "alice posted X" keys on
; (actor verb object) — distinct per actor; "alice liked X / bob liked X"
; collapse on (verb object) so the cross-actor likes fold into one.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem? lives in fanout.sx).
; generic: dedupe a stream by key-fn, first occurrence wins (stable)
(define
feed/-dedup-by
(fn
(items key-fn)
(get
(reduce
(fn
(st x)
(let
((k (key-fn x)))
(if (feed/-elem? k (get st :seen)) st {:seen (append (get st :seen) (list k)) :out (append (get st :out) (list x))})))
{:seen (list) :out (list)}
items)
:out)))
(define
feed/dedupe
(fn
(stream key-fn)
(feed/stream (feed/-dedup-by (feed/items stream) key-fn))))
; --- keys -------------------------------------------------------------------
(define
feed/activity-key
(fn (a) (list (get a :actor) (get a :verb) (get a :object))))
; collapse cross-actor duplicates of the same verb+object (e.g. likes)
(define feed/collapse-key (fn (a) (list (get a :verb) (get a :object))))
; per-receiver inbox key — one inbox event per (receiver, actor, verb, object)
(define
feed/event-key
(fn
(ev)
(let
((a (get ev :activity)))
(list (get ev :to) (get a :actor) (get a :verb) (get a :object)))))
; verbs whose duplicates collapse across actors (reactions, not authorship).
; rebindable: callers can (set! feed/collapse-verbs ...) to tune the policy.
(define
feed/collapse-verbs
(list "like" "favourite" "follow" "boost" "repost"))
; per-verb key: collapse-verbs fold on (verb object); the rest key on
; (actor verb object).
(define
feed/smart-key
(fn
(a)
(if
(feed/-elem? (get a :verb) feed/collapse-verbs)
(feed/collapse-key a)
(feed/activity-key a))))
; --- ready-made dedupers ----------------------------------------------------
(define feed/dedupe-activities (fn (s) (feed/dedupe s feed/activity-key)))
(define feed/dedupe-collapse (fn (s) (feed/dedupe s feed/collapse-key)))
; verb-aware: reactions collapse cross-actor, posts stay distinct per actor
(define feed/dedupe-smart (fn (s) (feed/dedupe s feed/smart-key)))
; dedupe an inbox: at most one event per receiver per (actor verb object)
(define feed/dedupe-inbox (fn (inbox) (feed/dedupe inbox feed/event-key)))

114
lib/feed/fanout.sx Normal file
View File

@@ -0,0 +1,114 @@
; feed/fanout — THE SHOWCASE. Fan activities out to followers via the APL outer
; product (∘.×). activities ∘.× audience → an (activity × follower) matrix of
; inbox events; flatten to a vector; guard-keep only real follow edges.
;
; Requires: lib/apl/runtime.sx, lib/feed/normalize.sx, lib/feed/stream.sx.
;
; NOTE: apl-outer's combiner result is run through (if (scalar? r) (disclose r) r).
; A bare dict counts as a scalar (shape ()) and disclose nils it — so the combiner
; must (enclose ...) its event dict; apl-outer then discloses it back intact.
; --- graph: {followee -> (list of followers)} -------------------------------
(define feed/followers (fn (graph user) (get graph user (list))))
; build a graph from (follower followee) edges: "follower follows followee"
(define
feed/follow-graph
(fn
(edges)
(reduce
(fn
(g e)
(let
((follower (first e)) (followee (nth e 1)))
(assoc
g
followee
(append (feed/followers g followee) (list follower)))))
{}
edges)))
; --- helpers ----------------------------------------------------------------
; unwrap an apl-scalar (has :ravel) back to its value; pass activities through
(define
feed/-val
(fn
(x)
(if (and (= (type-of x) "dict") (has-key? x :ravel)) (disclose x) x)))
(define feed/-elem? (fn (x lst) (some (fn (y) (equal? x y)) lst)))
(define
feed/-distinct
(fn
(lst)
(if
(= (len lst) 0)
(list)
(get (apl-unique (make-array (list (len lst)) lst)) :ravel))))
; rank-2 matrix -> rank-1 stream of its ravel
(define feed/-flatten (fn (arr) (feed/stream (get arr :ravel))))
; distinct receivers across the whole graph, sorted for determinism
; (dict key order is unspecified, so sort to pin audience/recipient ordering)
(define
feed/audience
(fn
(graph)
(sort
(feed/-distinct
(reduce
(fn (acc k) (append acc (feed/followers graph k)))
(list)
(keys graph))))))
; --- the outer product ------------------------------------------------------
; one (activity, follower) inbox event, enclosed so apl-outer keeps the dict
(define feed/-mk-event (fn (a f) (enclose {:activity (feed/-val a) :to (feed/-val f)})))
; keep events where :to actually follows the activity's actor
(define
feed/-edge?
(fn
(graph)
(fn
(ev)
(feed/-elem?
(get ev :to)
(feed/followers graph (get (get ev :activity) :actor))))))
; fanout — activities ∘.× audience, flatten, guard-keep real edges
(define
feed/fanout
(fn
(stream graph)
(let
((matrix (apl-outer feed/-mk-event stream (feed/stream (feed/audience graph)))))
(feed/filter (feed/-flatten matrix) (feed/-edge? graph)))))
; --- inbox queries ----------------------------------------------------------
(define
feed/inbox-for
(fn
(inbox user)
(feed/filter inbox (fn (ev) (equal? (get ev :to) user)))))
(define
feed/recipients
(fn
(inbox)
(feed/-distinct (map (fn (ev) (get ev :to)) (feed/items inbox)))))
; the activities (unwrapped) destined for a user
(define
feed/inbox-activities
(fn
(inbox user)
(map
(fn (ev) (get ev :activity))
(feed/items (feed/inbox-for inbox user)))))

60
lib/feed/fed.sx Normal file
View File

@@ -0,0 +1,60 @@
; feed/fed — federation. Outbound: a local post fans out, then splits into local
; vs remote inboxes; remote events are handed to an injected send-fn. Inbound:
; peer activities merge into the local stream, deduped. Backfill: pull peer
; history via an injected fetch-fn and merge.
;
; remote? / send-fn / fetch-fn are injected so real fed-sx transport wires in here
; without feed depending on it.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx,
; lib/feed/dedupe.sx.
; --- merge / ingest ---------------------------------------------------------
(define
feed/merge
(fn (s1 s2) (feed/stream (append (feed/items s1) (feed/items s2)))))
; merge a peer stream into local, dropping (actor verb object) duplicates
(define
feed/ingest
(fn (local peer) (feed/dedupe-activities (feed/merge local peer))))
; --- inbound ----------------------------------------------------------------
; peer pushes raw activities to the local inbox; normalize + ingest
(define
feed/inbound
(fn
(local raw-activities)
(feed/ingest local (feed/stream (map feed/normalize raw-activities)))))
; backfill on subscribe: pull peer history via fetch-fn, normalize, ingest
(define
feed/backfill
(fn (local fetch-fn peer-id) (feed/inbound local (fetch-fn peer-id))))
; --- outbound ---------------------------------------------------------------
; split an inbox into local vs remote deliveries by viewer-id predicate
(define feed/partition-inbox (fn (inbox remote?) {:local (feed/filter inbox (fn (ev) (not (remote? (get ev :to))))) :remote (feed/filter inbox (fn (ev) (remote? (get ev :to))))}))
; fan a stream out over the graph, then partition by locality
(define
feed/federate
(fn
(stream graph remote?)
(feed/partition-inbox (feed/fanout stream graph) remote?)))
; deliver: hand each remote event to send-fn, return the local inbox to enqueue
(define
feed/deliver
(fn
(stream graph remote? send-fn)
(let
((parts (feed/federate stream graph remote?)))
(begin
(for-each
(fn (ev) (send-fn (get ev :to) (get ev :activity)))
(feed/items (get parts :remote)))
(get parts :local)))))

23
lib/feed/home.sx Normal file
View File

@@ -0,0 +1,23 @@
; feed/home — the capstone. A user's home timeline is the whole pipeline as one
; line: fan all activities out over the follow graph, take the events landing in
; the viewer's inbox, dedupe cross-posts, apply the viewer's ACL, rank, take N.
;
; Requires: fanout.sx, dedupe.sx, acl.sx (feed/timeline), rank.sx, stream.sx.
; the activities in a user's inbox, as a stream
(define
feed/inbox-stream
(fn (inbox user) (feed/stream (feed/inbox-activities inbox user))))
; fanout ∘ inbox ∘ dedupe ∘ ACL ∘ rank ∘ take
(define
feed/home
(fn
(stream graph viewer permit? score-fn n)
(feed/timeline
(feed/dedupe-activities
(feed/inbox-stream (feed/fanout stream graph) viewer))
viewer
permit?
score-fn
n)))

44
lib/feed/mute.sx Normal file
View File

@@ -0,0 +1,44 @@
; feed/mute — viewer-controlled filtering. ACL (acl.sx) is author-controlled
; visibility; mute is the reader's own preference: hide muted actors or tags.
; Like ACL it is per-viewer and applied per request, never cached.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem?).
; drop activities authored by a muted actor
(define
feed/mute-actors
(fn
(stream actors)
(feed/filter
stream
(fn (a) (not (feed/-elem? (get a :actor) actors))))))
; drop activities carrying any muted tag
(define
feed/mute-tags
(fn
(stream tags)
(feed/filter
stream
(fn (a) (not (some (fn (t) (feed/-elem? t tags)) (get a :tags)))))))
; drop activities about a muted object (thread mute)
(define
feed/mute-objects
(fn
(stream objects)
(feed/filter
stream
(fn (a) (not (feed/-elem? (get a :object) objects))))))
; apply a viewer preference bag: {:mute-actors (...) :mute-tags (...) :mute-objects (...)}
(define
feed/apply-prefs
(fn
(stream prefs)
(feed/mute-objects
(feed/mute-tags
(feed/mute-actors stream (get prefs :mute-actors (list)))
(get prefs :mute-tags (list)))
(get prefs :mute-objects (list)))))

31
lib/feed/normalize.sx Normal file
View File

@@ -0,0 +1,31 @@
; feed/normalize — coerce arbitrary input into the canonical activity record.
; An activity is a small dict {:actor :verb :object :at :tags}; a stream is an
; APL vector of such dicts (see stream.sx). Extra keys on the raw input survive
; (e.g. :visible-to for ACL, peer metadata for federation) — :tags is the
; flexible bag but the record is not closed.
(define feed/activity-keys (list :actor :verb :object :at :tags))
(define
feed/normalize
(fn
(raw)
(let
((d (if (= (type-of raw) "dict") raw {})))
(merge d {:actor (get d :actor "") :object (get d :object nil) :at (get d :at 0) :tags (let ((t (get d :tags (list)))) (if (list? t) t (list t))) :verb (get d :verb "post")}))))
(define
feed/activity
(fn (actor verb object at tags) (feed/normalize {:actor actor :object object :at at :tags tags :verb verb})))
(define feed/actor (fn (a) (get a :actor)))
(define feed/verb (fn (a) (get a :verb)))
(define feed/object (fn (a) (get a :object)))
(define feed/at (fn (a) (get a :at)))
(define feed/tags (fn (a) (get a :tags)))
(define
feed/activity?
(fn
(a)
(and (= (type-of a) "dict") (has-key? a :actor) (has-key? a :verb))))

45
lib/feed/notify.sx Normal file
View File

@@ -0,0 +1,45 @@
; feed/notify — a notification feed is a thin layer over a recipient's inbox:
; the events directed at a user, optionally verb-filtered, and a digest that
; collapses "alice, bob and 1 other liked X" by (verb, object).
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/inbox-for, feed/-elem?).
; all inbox events for a user (their raw notifications)
(define feed/notifications (fn (inbox user) (feed/inbox-for inbox user)))
; restrict to notification-worthy verbs (e.g. (list "like" "reply" "follow"))
(define
feed/notify-verbs
(fn
(inbox user verbs)
(feed/filter
(feed/inbox-for inbox user)
(fn (ev) (feed/-elem? (get (get ev :activity) :verb) verbs)))))
; group key "verb|object" — deterministic, sortable
(define
feed/-notify-key
(fn
(ev)
(let
((a (get ev :activity)))
(string-append (get a :verb) "|" (get a :object)))))
; digest: one entry per (verb, object) with the distinct actors and a count,
; ordered by key for determinism.
(define
feed/notify-digest
(fn
(inbox user)
(let
((events (feed/items (feed/inbox-for inbox user))))
(let
((groups (reduce (fn (g ev) (let ((a (get ev :activity)) (k (feed/-notify-key ev))) (let ((cur (get g k {:object (get a :object) :actors (list) :verb (get a :verb)}))) (assoc g k (assoc cur :actors (append (get cur :actors) (list (get a :actor)))))))) {} events)))
(map
(fn
(k)
(let
((grp (get groups k)))
(assoc grp :count (len (get grp :actors)))))
(sort (keys groups)))))))

50
lib/feed/page.sx Normal file
View File

@@ -0,0 +1,50 @@
; feed/page — pagination. Offset/limit for indexed access, and cursor-based
; (by :at) for recency feeds, which is stable under inserts: a cursor is the
; :at of the last item seen, and the next page is the newest items older than it.
;
; Requires: lib/feed/stream.sx (feed/recent, feed/take, feed/filter).
; --- offset / limit ---------------------------------------------------------
(define
feed/page
(fn
(stream offset limit)
(feed/stream (take (drop (feed/items stream) offset) limit))))
(define
feed/page-count
(fn (stream limit) (ceil (/ (feed/count stream) limit))))
; --- cursor (recency feeds) -------------------------------------------------
; activities strictly older than cursor (scroll down / load older)
(define
feed/before
(fn
(stream cursor)
(feed/filter stream (fn (a) (< (get a :at) cursor)))))
; activities strictly newer than cursor (load newer / "N new posts")
(define
feed/after
(fn
(stream cursor)
(feed/filter stream (fn (a) (> (get a :at) cursor)))))
; one page: the `limit` newest activities older than cursor, newest first
(define
feed/page-before
(fn
(stream cursor limit)
(feed/take (feed/recent (feed/before stream cursor)) limit)))
; cursor to fetch the next (older) page: :at of the last item of a page,
; or nil when the page is empty (end of feed)
(define
feed/next-cursor
(fn
(page)
(let
((items (feed/items page)))
(if (= (len items) 0) nil (get (last items) :at)))))

92
lib/feed/rank.sx Normal file
View File

@@ -0,0 +1,92 @@
; feed/rank — scoring + ranking. Scorers are (activity -> number). Ranking is a
; stable two-pass grade-down: first by :at descending (the tiebreak), then by
; score descending — so ties resolve by recency, then by input order. Fully
; deterministic on ties.
;
; Requires: lib/apl/runtime.sx, lib/feed/normalize.sx, lib/feed/stream.sx.
; --- scorers ----------------------------------------------------------------
; recency: half-life decay. score = 0.5 ^ (age / half-life). at==now -> 1.0.
(define
feed/recency
(fn
(now half-life)
(fn (a) (expt 0.5 (/ (- now (get a :at)) half-life)))))
; velocity: how many of this actor's activities fall in (at-window, at] —
; a burst of recent activity scores higher.
(define
feed/velocity
(fn
(stream window)
(fn
(a)
(len
(filter
(fn
(b)
(and
(equal? (get b :actor) (get a :actor))
(<= (get b :at) (get a :at))
(> (get b :at) (- (get a :at) window))))
(feed/items stream))))))
; engagement: how many activities in the stream touch this activity's :object
(define
feed/engagement
(fn
(stream)
(fn
(a)
(len
(filter
(fn (b) (equal? (get b :object) (get a :object)))
(feed/items stream))))))
; composite: weighted sum. parts = (list (list weight scorer) ...)
(define
feed/composite
(fn
(parts)
(fn
(a)
(reduce
(fn (acc p) (+ acc (* (first p) ((nth p 1) a))))
0
parts))))
; --- ranking ----------------------------------------------------------------
; stable reorder of items by key-fn, descending (grade-down is stable)
(define
feed/-desc-by
(fn
(items key-fn)
(let
((keys (make-array (list (len items)) (map key-fn items))))
(let
((order (get (apl-grade-down keys) :ravel)))
(map (fn (i) (nth items (- i 1))) order)))))
; rank by score descending; ties -> :at descending -> input order
(define
feed/rank
(fn
(stream score-fn)
(let
((by-at (feed/-desc-by (feed/items stream) feed/at)))
(feed/stream (feed/-desc-by by-at score-fn)))))
; attach a :score to each activity (for inspection / debugging)
(define
feed/with-scores
(fn
(stream score-fn)
(feed/stream
(map (fn (a) (assoc a :score (score-fn a))) (feed/items stream)))))
; top-N ranked timeline
(define
feed/top
(fn (stream score-fn n) (feed/take (feed/rank stream score-fn) n)))

19
lib/feed/scoreboard.json Normal file
View File

@@ -0,0 +1,19 @@
{
"suites": {
"basic": {"pass": 30, "fail": 0},
"fanout": {"pass": 29, "fail": 0},
"rank": {"pass": 24, "fail": 0},
"integration": {"pass": 22, "fail": 0},
"content": {"pass": 15, "fail": 0},
"notify": {"pass": 8, "fail": 0},
"home": {"pass": 6, "fail": 0},
"dedupe": {"pass": 9, "fail": 0},
"trending": {"pass": 11, "fail": 0},
"mute": {"pass": 9, "fail": 0},
"page": {"pass": 14, "fail": 0},
"thread": {"pass": 12, "fail": 0}
},
"total_pass": 189,
"total_fail": 0,
"total": 189
}

19
lib/feed/scoreboard.md Normal file
View File

@@ -0,0 +1,19 @@
# feed Conformance Scoreboard
_Generated by `lib/feed/conformance.sh`_
| Suite | Pass | Fail | Total |
|-------|-----:|-----:|------:|
| basic | 30 | 0 | 30 |
| fanout | 29 | 0 | 29 |
| rank | 24 | 0 | 24 |
| integration | 22 | 0 | 22 |
| content | 15 | 0 | 15 |
| notify | 8 | 0 | 8 |
| home | 6 | 0 | 6 |
| dedupe | 9 | 0 | 9 |
| trending | 11 | 0 | 11 |
| mute | 9 | 0 | 9 |
| page | 14 | 0 | 14 |
| thread | 12 | 0 | 12 |
| **Total** | **189** | **0** | **189** |

75
lib/feed/stream.sx Normal file
View File

@@ -0,0 +1,75 @@
; feed/stream — a stream is an APL vector (rank-1 array) whose ravel holds
; activity dicts. Operations lift APL primitives onto this shape: filter via
; compress (/), sort via grade (⍋), take via ↑, reverse via ⌽.
;
; Requires: lib/apl/runtime.sx, lib/feed/normalize.sx (loaded by harness).
(define feed/stream (fn (acts) (make-array (list (len acts)) acts)))
(define feed/items (fn (s) (get s :ravel)))
(define feed/count (fn (s) (len (get s :ravel))))
(define feed/empty (feed/stream (list)))
(define feed/empty? (fn (s) (= (feed/count s) 0)))
; filter — bool mask ∘ compress. pred : activity -> truthy
(define
feed/filter
(fn
(s pred)
(let
((items (get s :ravel)))
(let
((mask (make-array (list (len items)) (map (fn (a) (if (pred a) 1 0)) items))))
(apl-compress mask s)))))
; sort-by — ascending, stable on ties (grade-up is stable). key-fn : activity -> number
(define
feed/sort-by
(fn
(s key-fn)
(let
((items (get s :ravel)))
(let
((keys (make-array (list (len items)) (map key-fn items))))
(let
((order (get (apl-grade-up keys) :ravel)))
(feed/stream (map (fn (i) (nth items (- i 1))) order)))))))
(define feed/sort-by-at (fn (s) (feed/sort-by s feed/at)))
; newest-first: ascending sort then reverse (⌽)
(define feed/recent (fn (s) (apl-reverse (feed/sort-by-at s))))
; take N (↑), clamped to stream length so it never over-takes/pads
(define
feed/take
(fn
(s n)
(let
((c (feed/count s)))
(if (>= n c) s (apl-take (apl-scalar n) s)))))
(define feed/reverse (fn (s) (apl-reverse s)))
; common predicates
(define
feed/by-actor
(fn (s actor) (feed/filter s (fn (a) (equal? (get a :actor) actor)))))
(define
feed/by-verb
(fn (s verb) (feed/filter s (fn (a) (equal? (get a :verb) verb)))))
(define
feed/by-object
(fn
(s object)
(feed/filter s (fn (a) (equal? (get a :object) object)))))
; activities at or after timestamp t
(define
feed/since
(fn (s t) (feed/filter s (fn (a) (>= (get a :at) t)))))

118
lib/feed/tests/basic.sx Normal file
View File

@@ -0,0 +1,118 @@
; Phase 1 — normalize, stream ops, api. Uses the feed-test harness
; (feed-test name got expected) provided by conformance.sh.
; ---------- normalize ----------
(feed-test
"normalize default actor"
(feed/actor (feed/normalize {}))
"")
(feed-test
"normalize default verb"
(feed/verb (feed/normalize {}))
"post")
(feed-test
"normalize default at"
(feed/at (feed/normalize {}))
0)
(feed-test
"normalize default object"
(feed/object (feed/normalize {}))
nil)
(feed-test
"normalize default tags"
(feed/tags (feed/normalize {}))
(list))
(feed-test
"normalize keeps actor"
(feed/actor (feed/normalize {:actor "alice"}))
"alice")
(feed-test
"normalize keeps verb"
(feed/verb (feed/normalize {:verb "like"}))
"like")
(feed-test
"normalize scalar tag -> list"
(feed/tags (feed/normalize {:tags "x"}))
(list "x"))
(feed-test
"normalize list tags kept"
(feed/tags (feed/normalize {:tags (list "a" "b")}))
(list "a" "b"))
(feed-test
"activity constructor at"
(feed/at (feed/activity "a" "post" "o" 5 (list)))
5)
(feed-test
"activity? on activity"
(feed/activity? (feed/normalize {:actor "a"}))
true)
(feed-test "activity? on number" (feed/activity? 5) false)
(feed-test "activity? on bare dict" (feed/activity? {:foo 1}) false)
; ---------- stream ----------
(define
S
(feed/stream
(list
(feed/activity "alice" "post" "p1" 30 (list))
(feed/activity "bob" "like" "p1" 10 (list))
(feed/activity "alice" "post" "p2" 20 (list)))))
(feed-test "stream count" (feed/count S) 3)
(feed-test "stream items len" (len (feed/items S)) 3)
(feed-test
"sort-by-at actors asc"
(map feed/actor (feed/items (feed/sort-by-at S)))
(list "bob" "alice" "alice"))
(feed-test
"recent newest first"
(map feed/at (feed/items (feed/recent S)))
(list 30 20 10))
(feed-test
"take 2 of recent"
(feed/count (feed/take (feed/recent S) 2))
2)
(feed-test
"take clamps past end"
(feed/count (feed/take S 10))
3)
(feed-test
"by-actor alice count"
(feed/count (feed/by-actor S "alice"))
2)
(feed-test
"by-verb like actor"
(map feed/actor (feed/items (feed/by-verb S "like")))
(list "bob"))
(feed-test
"by-object p1 count"
(feed/count (feed/by-object S "p1"))
2)
(feed-test
"since 20 count"
(feed/count (feed/since S 20))
2)
(feed-test
"reverse ats"
(map feed/at (feed/items (feed/reverse S)))
(list 20 10 30))
(feed-test "empty? on empty" (feed/empty? feed/empty) true)
(feed-test
"empty? on filtered-out"
(feed/empty? (feed/by-actor S "zzz"))
true)
; ---------- api ----------
(feed/reset!)
(feed/post {:actor "x" :at 1 :verb "post"})
(feed/post {:actor "y" :at 2 :verb "like"})
(feed-test "api size after posts" (feed/size) 2)
(feed-test "api all count" (feed/count (feed/all)) 2)
(feed-test
"post returns normalized verb"
(feed/verb (feed/post {:actor "z"}))
"post")
(feed-test "api size after third post" (feed/size) 3)

85
lib/feed/tests/content.sx Normal file
View File

@@ -0,0 +1,85 @@
; Follow-up — TF-IDF content ranking over :tags. (feed-test name got expected)
(define
corpus
(feed/stream
(list
(feed/normalize {:actor "u" :object "o1" :at 10 :tags (list "cats" "funny")})
(feed/normalize {:actor "u" :object "o2" :at 20 :tags (list "cats" "news")})
(feed/normalize {:actor "u" :object "o3" :at 30 :tags (list "politics" "news")})
(feed/normalize {:actor "u" :object "o4" :at 40 :tags (list "cats")}))))
; ---------- document frequency ----------
(feed-test "df cats" (get (feed/tag-df corpus) "cats") 3)
(feed-test "df news" (get (feed/tag-df corpus) "news") 2)
(feed-test "df funny" (get (feed/tag-df corpus) "funny") 1)
(feed-test "df politics" (get (feed/tag-df corpus) "politics") 1)
(feed-test "df full" (feed/tag-df corpus) {:news 2 :funny 1 :politics 1 :cats 3})
; ---------- inverse document frequency ----------
(feed-test
"idf news = log(4/2)"
(get (feed/tag-idf corpus) "news")
(log 2))
(feed-test
"idf funny = log(4/1)"
(get (feed/tag-idf corpus) "funny")
(log 4))
(feed-test
"rarer tag has higher idf"
(>
(get (feed/tag-idf corpus) "funny")
(get (feed/tag-idf corpus) "cats"))
true)
; ---------- tf-idf scoring ----------
(define idf (feed/tag-idf corpus))
(feed-test
"score query funny on o1"
((feed/tfidf-score idf (list "funny")) (feed/normalize {:actor "u" :object "x" :tags (list "cats" "funny")}))
(log 4))
(feed-test
"score query funny on non-match"
((feed/tfidf-score idf (list "funny")) (feed/normalize {:actor "u" :object "x" :tags (list "cats")}))
0)
(feed-test
"unknown query tag scores 0"
((feed/tfidf-score idf (list "zzz")) (feed/normalize {:actor "u" :object "x" :tags (list "cats")}))
0)
; ---------- ranking by relevance ----------
; query news: o2,o3 match (score log2), o1,o4 don't (0); ties break by :at desc
(feed-test
"by-relevance news order"
(map
(fn (a) (get a :object))
(feed/items (feed/by-relevance corpus (list "news"))))
(list "o3" "o2" "o4" "o1"))
; query funny: only o1 matches -> ranks first
(feed-test
"by-relevance funny first"
(get
(nth (feed/items (feed/by-relevance corpus (list "funny"))) 0)
:object)
"o1")
; query (cats news): o2 carries both tags -> highest combined tf-idf
(feed-test
"by-relevance cats+news top"
(get
(nth
(feed/items (feed/by-relevance corpus (list "cats" "news")))
0)
:object)
"o2")
(feed-test
"by-relevance preserves count"
(feed/count (feed/by-relevance corpus (list "cats")))
4)

56
lib/feed/tests/dedupe.sx Normal file
View File

@@ -0,0 +1,56 @@
; Follow-up — verb-aware (smart) dedupe. (feed-test name got expected)
; reactions (like/follow) collapse cross-actor; posts stay distinct per actor
(define
M
(feed/stream
(list
(feed/activity "alice" "like" "X" 1 (list))
(feed/activity "bob" "like" "X" 2 (list))
(feed/activity "alice" "post" "P" 3 (list))
(feed/activity "bob" "post" "P" 4 (list))
(feed/activity "alice" "follow" "C" 5 (list))
(feed/activity "bob" "follow" "C" 6 (list))))) ; collapses
(feed-test
"smart dedupe total"
(feed/count (feed/dedupe-smart M))
4)
(feed-test
"smart keeps both posts"
(feed/count (feed/by-verb (feed/dedupe-smart M) "post"))
2)
(feed-test
"smart collapses likes to one"
(feed/count (feed/by-verb (feed/dedupe-smart M) "like"))
1)
(feed-test
"smart collapses follows to one"
(feed/count (feed/by-verb (feed/dedupe-smart M) "follow"))
1)
(feed-test
"collapsed like keeps first actor"
(map feed/actor (feed/items (feed/by-verb (feed/dedupe-smart M) "like")))
(list "alice"))
; contrast: plain activity dedupe keeps cross-actor likes distinct
(feed-test
"activity dedupe keeps both likes"
(feed/count (feed/by-verb (feed/dedupe-activities M) "like"))
2)
; contrast: blanket collapse folds the two posts (same verb+object) too
(feed-test
"collapse dedupe folds posts"
(feed/count (feed/by-verb (feed/dedupe-collapse M) "post"))
1)
; smart-key dispatch
(feed-test
"smart-key reaction -> (verb object)"
(feed/smart-key (feed/activity "alice" "like" "X" 0 (list)))
(list "like" "X"))
(feed-test
"smart-key post -> (actor verb object)"
(feed/smart-key (feed/activity "alice" "post" "P" 0 (list)))
(list "alice" "post" "P"))

187
lib/feed/tests/fanout.sx Normal file
View File

@@ -0,0 +1,187 @@
; Phase 2 — fanout via outer product + dedupe. (feed-test name got expected)
; ---------- graph ----------
; edges: (follower followee). bob,carol follow alice; carol,dave follow bob.
(define
G
(feed/follow-graph
(list
(list "bob" "alice")
(list "carol" "alice")
(list "carol" "bob")
(list "dave" "bob"))))
(feed-test "followers alice" (feed/followers G "alice") (list "bob" "carol"))
(feed-test "followers bob" (feed/followers G "bob") (list "carol" "dave"))
(feed-test "followers unknown" (feed/followers G "zzz") (list))
(feed-test "audience distinct" (feed/audience G) (list "bob" "carol" "dave"))
; ---------- fanout ----------
(define
S
(feed/stream
(list
(feed/activity "alice" "post" "p1" 10 (list))
(feed/activity "alice" "post" "p2" 20 (list))
(feed/activity "bob" "like" "p1" 30 (list)))))
(define IB (feed/fanout S G))
(feed-test "fanout total edges" (feed/count IB) 6)
(feed-test
"inbox bob count"
(feed/count (feed/inbox-for IB "bob"))
2)
(feed-test
"inbox carol count"
(feed/count (feed/inbox-for IB "carol"))
3)
(feed-test
"inbox dave count"
(feed/count (feed/inbox-for IB "dave"))
1)
(feed-test
"inbox alice (follows none)"
(feed/count (feed/inbox-for IB "alice"))
0)
(feed-test
"recipients order"
(feed/recipients IB)
(list "bob" "carol" "dave"))
(feed-test
"bob inbox objects"
(map (fn (a) (get a :object)) (feed/inbox-activities IB "bob"))
(list "p1" "p2"))
(feed-test
"dave inbox objects"
(map (fn (a) (get a :object)) (feed/inbox-activities IB "dave"))
(list "p1"))
(feed-test
"dave inbox verb"
(map (fn (a) (get a :verb)) (feed/inbox-activities IB "dave"))
(list "like"))
; empty graph → no audience → no edges
(feed-test
"empty graph fanout"
(feed/count (feed/fanout S {}))
0)
; actor nobody follows produces no edges
(define
Sghost
(feed/stream (list (feed/activity "ghost" "post" "g1" 5 (list)))))
(feed-test
"unfollowed actor fanout"
(feed/count (feed/fanout Sghost G))
0)
; ---------- high fanout (popular actor) ----------
(define
Gstar
(feed/follow-graph
(list
(list "u1" "star")
(list "u2" "star")
(list "u3" "star")
(list "u4" "star")
(list "u5" "star"))))
(define
Sstar
(feed/stream (list (feed/activity "star" "post" "s1" 1 (list)))))
(feed-test
"star fanout count"
(feed/count (feed/fanout Sstar Gstar))
5)
(feed-test "star audience size" (len (feed/audience Gstar)) 5)
; ---------- mutual follow ----------
(define Gmut (feed/follow-graph (list (list "a" "b") (list "b" "a"))))
(define
Smut
(feed/stream
(list
(feed/activity "a" "post" "pa" 1 (list))
(feed/activity "b" "post" "pb" 2 (list)))))
(define IBmut (feed/fanout Smut Gmut))
(feed-test "mutual total" (feed/count IBmut) 2)
(feed-test
"mutual a gets pb"
(map (fn (x) (get x :object)) (feed/inbox-activities IBmut "a"))
(list "pb"))
(feed-test
"mutual b gets pa"
(map (fn (x) (get x :object)) (feed/inbox-activities IBmut "b"))
(list "pa"))
; ---------- dedupe ----------
(define
Sdup2
(feed/stream
(list
(feed/activity "alice" "post" "p1" 1 (list))
(feed/activity "alice" "post" "p1" 9 (list))
(feed/activity "alice" "post" "p2" 2 (list)))))
(feed-test
"dedupe-activities collapses dup"
(feed/count (feed/dedupe-activities Sdup2))
2)
(feed-test
"dedupe-activities keeps distinct"
(map
(fn (a) (get a :object))
(feed/items (feed/dedupe-activities Sdup2)))
(list "p1" "p2"))
(define
Slikes
(feed/stream
(list
(feed/activity "alice" "like" "X" 1 (list))
(feed/activity "bob" "like" "X" 2 (list))
(feed/activity "carol" "like" "Y" 3 (list)))))
(feed-test
"collapse cross-actor likes"
(feed/count (feed/dedupe-collapse Slikes))
2)
(feed-test
"collapse keeps distinct objects"
(map
(fn (a) (get a :object))
(feed/items (feed/dedupe-collapse Slikes)))
(list "X" "Y"))
(feed-test
"activity-key shape"
(feed/activity-key (feed/activity "a" "post" "o" 0 (list)))
(list "a" "post" "o"))
(feed-test
"collapse-key shape"
(feed/collapse-key (feed/activity "a" "like" "o" 0 (list)))
(list "like" "o"))
; cross-post: alice posts p1 twice → bob's inbox has it twice → dedupe-inbox → once
(define
Scross
(feed/stream
(list
(feed/activity "alice" "post" "p1" 1 (list))
(feed/activity "alice" "post" "p1" 5 (list)))))
(define IBcross (feed/fanout Scross G))
(feed-test
"cross-post raw bob count"
(feed/count (feed/inbox-for IBcross "bob"))
2)
(feed-test
"cross-post deduped bob count"
(feed/count (feed/inbox-for (feed/dedupe-inbox IBcross) "bob"))
1)
(feed-test
"dedupe-inbox keeps distinct receivers"
(feed/count (feed/dedupe-inbox IBcross))
2)

73
lib/feed/tests/home.sx Normal file
View File

@@ -0,0 +1,73 @@
; Follow-up — feed/home capstone pipeline. (feed-test name got expected)
; alice follows star and bob (edges: follower followee)
(define
G
(feed/follow-graph (list (list "alice" "star") (list "alice" "bob"))))
; star posts s1 then s2; bob posts b1; star re-posts s1 (cross-post dup);
; zoe posts z1 (alice does NOT follow zoe)
(define
S
(feed/stream
(list
(feed/activity "star" "post" "s1" 10 (list))
(feed/activity "star" "post" "s2" 20 (list))
(feed/activity "bob" "post" "b1" 15 (list))
(feed/activity "star" "post" "s1" 5 (list))
(feed/activity "zoe" "post" "z1" 30 (list)))))
(define rec (feed/recency 100 10))
(feed-test
"home count (deduped, followed only)"
(feed/count (feed/home S G "alice" feed/permit-public? rec 10))
3)
(feed-test
"home order by recency"
(map
(fn (a) (get a :object))
(feed/items (feed/home S G "alice" feed/permit-public? rec 10)))
(list "s2" "b1" "s1"))
(feed-test
"home excludes unfollowed zoe"
(feed/-elem?
"z1"
(map
(fn (a) (get a :object))
(feed/items (feed/home S G "alice" feed/permit-public? rec 10))))
false)
(feed-test
"home top-2"
(map
(fn (a) (get a :object))
(feed/items (feed/home S G "alice" feed/permit-public? rec 2)))
(list "s2" "b1"))
(feed-test
"home dedupes cross-post (one s1)"
(len
(filter
(fn (o) (equal? o "s1"))
(map
(fn (a) (get a :object))
(feed/items
(feed/home S G "alice" feed/permit-public? rec 10)))))
1)
; ACL applied per-viewer in the home pipeline
(define
Sacl
(feed/stream
(list (feed/normalize {:actor "star" :object "pub" :at 20}) (feed/normalize {:actor "star" :object "sec" :visible-to (list "carol") :at 25}))))
(define Gacl (feed/follow-graph (list (list "alice" "star"))))
(feed-test
"home hides activity alice not permitted"
(map
(fn (a) (get a :object))
(feed/items (feed/home Sacl Gacl "alice" feed/permit-acl? rec 10)))
(list "pub"))

View File

@@ -0,0 +1,155 @@
; Phase 4 — visibility (ACL) + federation, and the end-to-end timeline.
; (feed-test name got expected)
; ---------- ACL visibility ----------
; pub: public. sec: bob, allows carol. dm: frank, allows dave.
(define
C
(feed/stream
(list
(feed/normalize {:actor "alice" :object "pub" :at 10})
(feed/normalize {:actor "bob" :object "sec" :visible-to (list "carol") :at 20})
(feed/normalize {:actor "frank" :object "dm" :visible-to (list "dave") :at 30}))))
(feed-test
"public visible to anyone"
(feed/count (feed/visible C "zoe" feed/permit-acl?))
1)
(feed-test
"carol sees allowlisted + public"
(feed/count (feed/visible C "carol" feed/permit-acl?))
2)
(feed-test
"dave sees dm + public"
(feed/count (feed/visible C "dave" feed/permit-acl?))
2)
(feed-test
"author always sees own private"
(feed/count (feed/visible C "frank" feed/permit-acl?))
2)
(feed-test
"permit-public? lets all through"
(feed/count (feed/visible C "zoe" feed/permit-public?))
3)
(feed-test
"visible objects for dave"
(map
(fn (a) (get a :object))
(feed/items (feed/visible C "dave" feed/permit-acl?)))
(list "pub" "dm"))
; per-viewer: same stream, different timelines
(feed-test
"zoe timeline differs from carol"
(not
(=
(feed/count (feed/visible C "zoe" feed/permit-acl?))
(feed/count (feed/visible C "carol" feed/permit-acl?))))
true)
; ---------- federation: merge / ingest ----------
(define
L
(feed/stream
(list
(feed/activity "alice" "post" "p1" 10 (list))
(feed/activity "alice" "post" "p2" 20 (list)))))
(define
P
(feed/stream
(list
(feed/activity "alice" "post" "p2" 20 (list))
(feed/activity "peer" "post" "p9" 25 (list)))))
(feed-test "merge concatenates" (feed/count (feed/merge L P)) 4)
(feed-test
"ingest dedupes overlap"
(feed/count (feed/ingest L P))
3)
(feed-test
"inbound normalizes + ingests"
(feed/count (feed/inbound L (list {:actor "peer" :object "p9" :at 25} {:actor "alice" :object "p1" :at 10})))
3)
; backfill via injected fetch-fn
(define peer-history (fn (peer-id) (list {:actor peer-id :object "h1" :at 1} {:actor peer-id :object "h2" :at 2})))
(feed-test
"backfill merges peer history"
(feed/count (feed/backfill L peer-history "remote"))
4)
(feed-test
"backfill objects present"
(map
(fn (a) (get a :object))
(feed/items
(feed/by-actor (feed/backfill L peer-history "remote") "remote")))
(list "h1" "h2"))
; ---------- federation: outbound partition ----------
; bob (local), alice@remote + carol@remote (remote) follow star
(define
Gf
(feed/follow-graph
(list
(list "bob" "star")
(list "alice@remote" "star")
(list "carol@remote" "star"))))
(define
Sf
(feed/stream (list (feed/activity "star" "post" "s1" 1 (list)))))
(define
remote?
(fn (id) (feed/-elem? id (list "alice@remote" "carol@remote"))))
(define parts (feed/federate Sf Gf remote?))
(feed-test "local deliveries" (feed/count (get parts :local)) 1)
(feed-test "remote deliveries" (feed/count (get parts :remote)) 2)
(feed-test
"local recipient is bob"
(feed/recipients (get parts :local))
(list "bob"))
; deliver: send-fn receives each remote event, local inbox returned
(define sent (list))
(define send-fn (fn (to act) (set! sent (append sent (list to)))))
(define local-inbox (feed/deliver Sf Gf remote? send-fn))
(feed-test "deliver returns local inbox" (feed/count local-inbox) 1)
(feed-test "deliver sent to both remotes" (len sent) 2)
(feed-test "deliver remote targets" sent (list "alice@remote" "carol@remote"))
; ---------- end-to-end: federated, ACL-filtered, ranked timeline ----------
(define
base
(feed/stream
(list
(feed/normalize {:actor "alice" :object "a1" :at 100})
(feed/normalize {:actor "bob" :object "b1" :visible-to (list "carol") :at 90})
(feed/normalize {:actor "eve" :object "e1" :visible-to (list "dave") :at 80}))))
(define federated (feed/inbound base (list {:actor "peer" :object "x1" :at 110})))
(define rec (feed/recency 120 10))
(define
carol-tl
(feed/timeline federated "carol" feed/permit-acl? rec 3))
; eve's :visible-to excludes carol -> filtered out; peer/alice public, bob allows carol
(feed-test "carol federated timeline count" (feed/count carol-tl) 3)
(feed-test
"carol timeline order (recency)"
(map (fn (a) (get a :object)) (feed/items carol-tl))
(list "x1" "a1" "b1"))
(feed-test
"eve dm excluded from carol"
(feed/-elem? "e1" (map (fn (a) (get a :object)) (feed/items carol-tl)))
false)
(feed-test
"dave sees eve dm not bob"
(map
(fn (a) (get a :object))
(feed/items
(feed/timeline federated "dave" feed/permit-acl? rec 5)))
(list "x1" "a1" "e1"))

68
lib/feed/tests/mute.sx Normal file
View File

@@ -0,0 +1,68 @@
; Follow-up — viewer mute/block filtering. (feed-test name got expected)
(define
S
(feed/stream
(list
(feed/normalize {:actor "alice" :object "P1" :at 1 :tags (list "news")})
(feed/normalize {:actor "bob" :object "P2" :at 2 :tags (list "spam")})
(feed/normalize {:actor "alice" :object "P3" :at 3 :tags (list "cats")})
(feed/normalize {:actor "carol" :object "P4" :at 4 :tags (list "news" "spam")}))))
; ---------- mute actors ----------
(feed-test
"mute bob drops his post"
(map
(fn (a) (get a :object))
(feed/items (feed/mute-actors S (list "bob"))))
(list "P1" "P3" "P4"))
(feed-test
"mute alice drops two"
(feed/count (feed/mute-actors S (list "alice")))
2)
(feed-test
"mute nobody keeps all"
(feed/count (feed/mute-actors S (list)))
4)
; ---------- mute tags ----------
(feed-test
"mute spam tag drops two"
(map
(fn (a) (get a :object))
(feed/items (feed/mute-tags S (list "spam"))))
(list "P1" "P3"))
(feed-test
"mute news+cats leaves spam-only"
(map
(fn (a) (get a :object))
(feed/items (feed/mute-tags S (list "news" "cats"))))
(list "P2"))
; ---------- mute objects ----------
(feed-test
"mute object P3 (thread mute)"
(feed/count (feed/mute-objects S (list "P3")))
3)
; ---------- combined prefs ----------
(feed-test
"apply-prefs actors + tags"
(map
(fn (a) (get a :object))
(feed/items (feed/apply-prefs S {:mute-actors (list "bob") :mute-tags (list "cats")})))
(list "P1" "P4"))
(feed-test
"apply-prefs empty keeps all"
(feed/count (feed/apply-prefs S {}))
4)
(feed-test
"apply-prefs all three filters"
(map
(fn (a) (get a :object))
(feed/items (feed/apply-prefs S {:mute-objects (list "P3") :mute-actors (list "carol") :mute-tags (list "spam")})))
(list "P1"))

69
lib/feed/tests/notify.sx Normal file
View File

@@ -0,0 +1,69 @@
; Follow-up — notification feed over an inbox. (feed-test name got expected)
; an inbox is a stream of {:to receiver :activity act} events
(define mk-ev (fn (to act) {:activity act :to to}))
(define
IB
(feed/stream
(list
(mk-ev "alice" (feed/activity "bob" "like" "P" 10 (list)))
(mk-ev "alice" (feed/activity "carol" "like" "P" 20 (list)))
(mk-ev "alice" (feed/activity "dave" "reply" "Q" 30 (list)))
(mk-ev "bob" (feed/activity "eve" "like" "R" 40 (list))))))
; ---------- raw notifications ----------
(feed-test
"alice notification count"
(feed/count (feed/notifications IB "alice"))
3)
(feed-test
"bob notification count"
(feed/count (feed/notifications IB "bob"))
1)
(feed-test
"zoe no notifications"
(feed/count (feed/notifications IB "zoe"))
0)
; ---------- verb filtering ----------
(feed-test
"alice likes only"
(feed/count (feed/notify-verbs IB "alice" (list "like")))
2)
(feed-test
"alice replies only"
(feed/count (feed/notify-verbs IB "alice" (list "reply")))
1)
(feed-test
"alice like+reply"
(feed/count (feed/notify-verbs IB "alice" (list "like" "reply")))
3)
(feed-test
"alice follow (none)"
(feed/count (feed/notify-verbs IB "alice" (list "follow")))
0)
; ---------- digest ----------
(define dig (feed/notify-digest IB "alice"))
(feed-test "digest group count" (len dig) 2)
(feed-test
"digest sorted by key (like|P before reply|Q)"
(map (fn (g) (get g :object)) dig)
(list "P" "Q"))
(feed-test
"like group actors"
(get (nth dig 0) :actors)
(list "bob" "carol"))
(feed-test "like group count" (get (nth dig 0) :count) 2)
(feed-test "like group verb" (get (nth dig 0) :verb) "like")
(feed-test "reply group count" (get (nth dig 1) :count) 1)
(feed-test
"reply group actors"
(get (nth dig 1) :actors)
(list "dave"))
(feed-test "empty digest for zoe" (feed/notify-digest IB "zoe") (list))

86
lib/feed/tests/page.sx Normal file
View File

@@ -0,0 +1,86 @@
; Follow-up — pagination (offset + cursor). (feed-test name got expected)
; ---------- offset / limit ----------
(define
O
(feed/stream
(list
(feed/activity "u" "post" "o1" 1 (list))
(feed/activity "u" "post" "o2" 2 (list))
(feed/activity "u" "post" "o3" 3 (list))
(feed/activity "u" "post" "o4" 4 (list))
(feed/activity "u" "post" "o5" 5 (list)))))
(feed-test
"page 1"
(map
(fn (a) (get a :object))
(feed/items (feed/page O 0 2)))
(list "o1" "o2"))
(feed-test
"page 2"
(map
(fn (a) (get a :object))
(feed/items (feed/page O 2 2)))
(list "o3" "o4"))
(feed-test
"page 3 (partial)"
(map
(fn (a) (get a :object))
(feed/items (feed/page O 4 2)))
(list "o5"))
(feed-test
"page past end empty"
(feed/count (feed/page O 10 2))
0)
(feed-test "page-count 5/2 = 3" (feed/page-count O 2) 3)
(feed-test "page-count 5/5 = 1" (feed/page-count O 5) 1)
; ---------- cursor (recency) ----------
(define
R
(feed/stream
(list
(feed/activity "u" "post" "a" 50 (list))
(feed/activity "u" "post" "b" 40 (list))
(feed/activity "u" "post" "c" 30 (list))
(feed/activity "u" "post" "d" 20 (list))
(feed/activity "u" "post" "e" 10 (list)))))
(define p1 (feed/page-before R 100 2))
(feed-test
"cursor page 1 newest first"
(map (fn (a) (get a :object)) (feed/items p1))
(list "a" "b"))
(feed-test "next cursor after page 1" (feed/next-cursor p1) 40)
(define p2 (feed/page-before R (feed/next-cursor p1) 2))
(feed-test
"cursor page 2"
(map (fn (a) (get a :object)) (feed/items p2))
(list "c" "d"))
(feed-test "next cursor after page 2" (feed/next-cursor p2) 20)
(define p3 (feed/page-before R (feed/next-cursor p2) 2))
(feed-test
"cursor page 3 (partial)"
(map (fn (a) (get a :object)) (feed/items p3))
(list "e"))
(feed-test
"empty page nil cursor"
(feed/next-cursor (feed/page-before R 5 2))
nil)
(feed-test
"after cursor loads newer"
(map
(fn (a) (get a :object))
(feed/items (feed/recent (feed/after R 30))))
(list "a" "b"))
(feed-test
"before cursor count"
(feed/count (feed/before R 30))
2)

160
lib/feed/tests/rank.sx Normal file
View File

@@ -0,0 +1,160 @@
; Phase 3 — aggregation + ranking. (feed-test name got expected)
; ---------- aggregation ----------
(define
A
(feed/stream
(list
(feed/activity "alice" "post" "p1" 5 (list))
(feed/activity "alice" "post" "p2" 15 (list))
(feed/activity "bob" "post" "p3" 25 (list))
(feed/activity "alice" "like" "p1" 35 (list)))))
(feed-test "actor-counts" (feed/actor-counts A) {:alice 3 :bob 1})
(feed-test "object-counts" (feed/object-counts A) {:p2 1 :p3 1 :p1 2})
(feed-test
"group-by actor alice len"
(len (get (feed/group-by A feed/actor) "alice"))
3)
(feed-test
"group-count empty"
(feed/group-count feed/empty feed/actor)
{})
; day bucketing
(define
D
(feed/stream
(list
(feed/activity "alice" "post" "p1" 5 (list))
(feed/activity "alice" "post" "p2" 8 (list))
(feed/activity "alice" "post" "p3" 12 (list)))))
(feed-test "feed/day floor" (feed/day 12 10) 1)
(feed-test "feed/day same bucket" (feed/day 8 10) 0)
(feed-test "by-actor-day" (feed/by-actor-day D 10) {:alice#0 2 :alice#1 1})
; ---------- recency ----------
(define rec (feed/recency 100 10))
(feed-test
"recency at=now -> 1"
(rec (feed/activity "x" "post" "o" 100 (list)))
1)
(feed-test
"recency age=hl -> .5"
(rec (feed/activity "x" "post" "o" 90 (list)))
0.5)
(feed-test
"recency age=2hl -> .25"
(rec (feed/activity "x" "post" "o" 80 (list)))
0.25)
; ---------- velocity ----------
(define vel (feed/velocity D 10))
(feed-test
"velocity burst (at=12)"
(vel (feed/activity "alice" "post" "z" 12 (list)))
3)
(feed-test
"velocity mid (at=8)"
(vel (feed/activity "alice" "post" "z" 8 (list)))
2)
(feed-test
"velocity first (at=5)"
(vel (feed/activity "alice" "post" "z" 5 (list)))
1)
(feed-test
"velocity other actor"
(vel (feed/activity "bob" "post" "z" 12 (list)))
0)
; ---------- engagement ----------
(define eng (feed/engagement A))
(feed-test
"engagement p1"
(eng (feed/activity "x" "post" "p1" 0 (list)))
2)
(feed-test
"engagement p2"
(eng (feed/activity "x" "post" "p2" 0 (list)))
1)
; ---------- composite ----------
(define
cmp1
(feed/composite (list (list 2 (fn (a) (get a :at))))))
(feed-test
"composite single part"
(cmp1 (feed/activity "x" "post" "o" 5 (list)))
10)
(define
cmp2
(feed/composite
(list
(list 2 (fn (a) (get a :at)))
(list 3 (fn (a) 1)))))
(feed-test
"composite two parts"
(cmp2 (feed/activity "x" "post" "o" 5 (list)))
13)
; ---------- ranking ----------
(define
R
(feed/stream
(list
(feed/activity "u" "post" "oC" 80 (list))
(feed/activity "u" "post" "oA" 100 (list))
(feed/activity "u" "post" "oB" 90 (list)))))
(feed-test
"rank by recency objects"
(map (fn (a) (get a :object)) (feed/items (feed/rank R rec)))
(list "oA" "oB" "oC"))
(feed-test
"top-2 by recency"
(map (fn (a) (get a :object)) (feed/items (feed/top R rec 2)))
(list "oA" "oB"))
(feed-test "top-2 count" (feed/count (feed/top R rec 2)) 2)
; constant score -> tiebreak by :at descending
(define
T
(feed/stream
(list
(feed/activity "u" "post" "f" 10 (list))
(feed/activity "u" "post" "g" 30 (list))
(feed/activity "u" "post" "h" 20 (list)))))
(feed-test
"tiebreak at-desc"
(map
(fn (a) (get a :object))
(feed/items (feed/rank T (fn (a) 0))))
(list "g" "h" "f"))
; equal score AND equal :at -> stable input order
(define
E
(feed/stream
(list
(feed/activity "u" "post" "first" 50 (list))
(feed/activity "u" "post" "second" 50 (list)))))
(feed-test
"stable equal-key input order"
(map
(fn (a) (get a :object))
(feed/items (feed/rank E (fn (a) 0))))
(list "first" "second"))
(feed-test
"with-scores attaches score"
(get (nth (feed/items (feed/with-scores R rec)) 1) :score)
1)
(feed-test "rank preserves count" (feed/count (feed/rank A rec)) 4)

49
lib/feed/tests/thread.sx Normal file
View File

@@ -0,0 +1,49 @@
; Follow-up — conversation threading via :reply-to closure. (feed-test name got expected)
(define
S
(feed/stream
(list
(feed/normalize {:actor "a" :object "root" :at 1})
(feed/normalize {:actor "b" :object "r1" :at 2 :verb "reply" :reply-to "root"})
(feed/normalize {:actor "c" :object "r2" :at 3 :verb "reply" :reply-to "root"})
(feed/normalize {:actor "d" :object "r3" :at 4 :verb "reply" :reply-to "r1"})
(feed/normalize {:actor "e" :object "x" :at 5}))))
; ---------- direct replies ----------
(feed-test "direct replies to root" (feed/reply-count S "root") 2)
(feed-test "direct replies to r1" (feed/reply-count S "r1") 1)
(feed-test "no replies to r3" (feed/reply-count S "r3") 0)
(feed-test
"replies objects to root"
(map (fn (a) (get a :object)) (feed/items (feed/replies S "root")))
(list "r1" "r2"))
; ---------- thread closure ----------
(feed-test
"thread objects root (transitive)"
(feed/thread-objects S "root")
(list "root" "r1" "r2" "r3"))
(feed-test
"thread root chronological"
(map (fn (a) (get a :object)) (feed/items (feed/thread S "root")))
(list "root" "r1" "r2" "r3"))
(feed-test "thread size root" (feed/thread-size S "root") 4)
(feed-test
"thread excludes unrelated x"
(feed/-elem?
"x"
(map (fn (a) (get a :object)) (feed/items (feed/thread S "root"))))
false)
; ---------- sub-thread ----------
(feed-test
"thread from r1 (sub-tree)"
(map (fn (a) (get a :object)) (feed/items (feed/thread S "r1")))
(list "r1" "r3"))
(feed-test "thread size r1" (feed/thread-size S "r1") 2)
(feed-test "leaf thread is itself" (feed/thread-size S "r3") 1)
(feed-test "unrelated thread is itself" (feed/thread-size S "x") 1)

View File

@@ -0,0 +1,82 @@
; Follow-up — trending objects/actors by recent activity. (feed-test name got expected)
; window (50,100]: X@60,X@70 (a), Y@80 (b), Z@90 (c); W@40 is too old
(define
S
(feed/stream
(list
(feed/activity "a" "post" "X" 60 (list))
(feed/activity "a" "post" "X" 70 (list))
(feed/activity "b" "post" "Y" 80 (list))
(feed/activity "c" "post" "Z" 90 (list))
(feed/activity "d" "post" "W" 40 (list)))))
; ---------- trending objects ----------
(feed-test
"trending count (3 in window)"
(len (feed/trending S 100 50 10))
3)
(feed-test
"trending top object"
(get
(nth (feed/trending S 100 50 10) 0)
:object)
"X")
(feed-test
"trending top count"
(get
(nth (feed/trending S 100 50 10) 0)
:count)
2)
(feed-test
"trending order (count desc, key asc tiebreak)"
(map
(fn (e) (get e :object))
(feed/trending S 100 50 10))
(list "X" "Y" "Z"))
(feed-test
"trending top-2"
(map
(fn (e) (get e :object))
(feed/trending S 100 50 2))
(list "X" "Y"))
(feed-test
"old object W excluded"
(feed/-elem?
"W"
(map
(fn (e) (get e :object))
(feed/trending S 100 50 10)))
false)
(feed-test
"narrow window keeps only newest"
(map
(fn (e) (get e :object))
(feed/trending S 100 15 10))
(list "Z"))
(feed-test
"empty window -> nothing"
(feed/trending S 100 5 10)
(list))
; ---------- trending actors ----------
(feed-test
"trending actor top"
(get
(nth (feed/trending-actors S 100 50 10) 0)
:actor)
"a")
(feed-test
"trending actor count"
(get
(nth (feed/trending-actors S 100 50 10) 0)
:count)
2)
(feed-test
"trending actors order"
(map
(fn (e) (get e :actor))
(feed/trending-actors S 100 50 10))
(list "a" "b" "c"))

59
lib/feed/thread.sx Normal file
View File

@@ -0,0 +1,59 @@
; feed/thread — conversation threading. A reply carries :reply-to <parent-object>
; (normalize preserves it). A thread is the transitive closure over :reply-to from
; a root object: root + replies + replies-to-replies, gathered chronologically.
;
; Requires: lib/feed/normalize.sx, lib/feed/stream.sx, lib/feed/fanout.sx
; (feed/-elem?, feed/-distinct).
; direct replies to an object
(define
feed/replies
(fn
(stream object)
(feed/filter stream (fn (a) (equal? (get a :reply-to) object)))))
(define
feed/reply-count
(fn (stream object) (feed/count (feed/replies stream object))))
; iterate f from x until the result stops growing (set-closure fixpoint)
(define
feed/-fixpoint
(fn
(f x)
(let
((nx (f x)))
(if (= (len nx) (len x)) x (feed/-fixpoint f nx)))))
; the set of object-ids in the thread rooted at `root`
(define
feed/thread-objects
(fn
(stream root)
(let
((all (feed/items stream)))
(feed/-fixpoint
(fn
(acc)
(feed/-distinct
(append
acc
(map
(fn (a) (get a :object))
(filter (fn (a) (feed/-elem? (get a :reply-to) acc)) all)))))
(list root)))))
; the full thread as a chronological stream (root + all descendants)
(define
feed/thread
(fn
(stream root)
(let
((objs (feed/thread-objects stream root)))
(feed/sort-by-at
(feed/filter stream (fn (a) (feed/-elem? (get a :object) objs)))))))
; how many activities are in the thread (root counts as 1)
(define
feed/thread-size
(fn (stream root) (feed/count (feed/thread stream root))))

42
lib/feed/trending.sx Normal file
View File

@@ -0,0 +1,42 @@
; feed/trending — what's hot right now: objects (or actors) ranked by activity
; count within a recency window. Deterministic: count descending, ties broken by
; key ascending (entries are pre-sorted by key, then stable grade-down by count).
;
; Requires: lib/feed/stream.sx, lib/feed/aggregate.sx (object/actor-counts),
; lib/feed/rank.sx (feed/-desc-by).
; activities within (now-window, now]
(define
feed/-recent
(fn
(stream now window)
(feed/filter
stream
(fn (a) (and (<= (get a :at) now) (> (get a :at) (- now window)))))))
; counts dict -> top-N entries {label key, :count n}, count desc, key asc
(define
feed/-top-counts
(fn
(counts label n)
(let
((entries (map (fn (k) (assoc {:count (get counts k)} label k)) (sort (keys counts)))))
(take (feed/-desc-by entries (fn (e) (get e :count))) n))))
; top-N trending objects in the window
(define
feed/trending
(fn
(stream now window n)
(feed/-top-counts
(feed/object-counts (feed/-recent stream now window))
:object n)))
; top-N most active actors in the window
(define
feed/trending-actors
(fn
(stream now window n)
(feed/-top-counts
(feed/actor-counts (feed/-recent stream now window))
:actor n)))

View File

@@ -1,40 +0,0 @@
;; lib/mod/activity.sx — export decisions as ActivityPub-shaped events.
;;
;; The rose-ash platform propagates cross-domain effects as ActivityPub-shaped
;; activities. A moderation decision maps to a moderation verb so the rest of the
;; platform (and federated peers) can act on it: remove→Delete, ban→Block,
;; hide/escalate→Flag, keep→no activity. The precise mod action is preserved in
;; :action so a consumer can disambiguate (e.g. hide vs escalate, both Flag).
(define
mod/action->verb
(fn
(action)
(cond
((= action "remove") "Delete")
((= action "ban") "Block")
((= action "hide") "Flag")
((= action "escalate") "Flag")
(true nil))))
(define
mod/decision->activity
(fn
(d actor)
(let
((verb (mod/action->verb (get d :action))))
(if (nil? verb) nil {:type verb :action (get d :action) :actor actor :summary (str "moderation/" (get d :action) " via " (get d :rule)) :object (get d :report-id) :rule (get d :rule)}))))
;; map a batch of decisions to activities, dropping the no-op keeps
(define
mod/decisions->activities
(fn
(decisions actor)
(reduce
(fn
(acc d)
(let
((a (mod/decision->activity d actor)))
(if (nil? a) acc (append acc (list a)))))
(list)
decisions)))

View File

@@ -1,163 +0,0 @@
;; lib/mod/api.sx — report registry + lifecycle façade + public entry points.
;;
;; mod/report files a report (assigning a sequential id) and opens a lifecycle
;; case for it; mod/add-evidence accumulates evidence; mod/decide runs the engine
;; and commits to the audit log. The lifecycle façade (mod/triage, mod/resolve,
;; mod/review, mod/appeal, mod/finalize) drives the per-report case through its
;; states, logging each committed decision to the audit trail.
(define mod/*reports* (list))
(define mod/*cases* (list))
(define mod/*counter* 0)
(define mod/*rules* mod/default-rules)
(define
mod/reset!
(fn
()
(begin
(set! mod/*reports* (list))
(set! mod/*cases* (list))
(set! mod/*counter* 0)
(mod/audit-reset!))))
(define
mod/report
(fn
(by about reason)
(begin
(set! mod/*counter* (+ mod/*counter* 1))
(let
((id (str "r" mod/*counter*)))
(let
((r (mod/mk-report id by about reason)))
(begin
(append! mod/*reports* r)
(append! mod/*cases* {:id id :case (mod/mk-case r)})
r))))))
(define
mod/get-report
(fn
(id)
(reduce
(fn (acc r) (if (= (mod/report-id r) id) r acc))
nil
mod/*reports*)))
(define
mod/add-evidence
(fn
(id kind val)
(let
((r (mod/get-report id)))
(if
(nil? r)
nil
(let
((updated (mod/attach-evidence r (mod/mk-evidence kind val))))
(begin
(set!
mod/*reports*
(map
(fn (x) (if (= (mod/report-id x) id) updated x))
mod/*reports*))
updated))))))
(define
mod/decide
(fn
(id)
(let
((r (mod/get-report id)))
(if
(nil? r)
nil
(let
((d (mod/decide-report r mod/*reports* mod/*rules*)))
(begin (mod/log-decision! d (mod/report-evidence r)) d))))))
;; ── lifecycle façade over the case registry ──
(define
mod/case-of
(fn
(id)
(reduce
(fn (acc rec) (if (= (get rec :id) id) (get rec :case) acc))
nil
mod/*cases*)))
(define
mod/case-store!
(fn
(id c)
(set!
mod/*cases*
(map
(fn (rec) (if (= (get rec :id) id) {:id id :case c} rec))
mod/*cases*))))
;; apply a lifecycle op to the stored case, persist it, and (when a decision was
;; committed cleanly) append it to the audit log; returns the updated case
(define
mod/case-apply!
(fn
(id op log?)
(let
((c (mod/case-of id)))
(if
(nil? c)
nil
(let
((c2 (op c)))
(begin
(mod/case-store! id c2)
(when
log?
(when
(nil? (mod/case-error c2))
(let
((d (mod/case-decision c2)))
(if
(nil? d)
nil
(mod/log-decision!
d
(mod/report-evidence (mod/case-report c2)))))))
c2))))))
(define
mod/triage
(fn
(id)
(mod/case-apply!
id
(fn (c) (mod/case-triage c mod/*reports* mod/*rules*))
false)))
(define
mod/resolve
(fn (id) (mod/case-apply! id (fn (c) (mod/case-resolve c)) true)))
(define
mod/review
(fn
(id kind val)
(mod/case-apply!
id
(fn (c) (mod/case-review c kind val mod/*reports* mod/*rules*))
true)))
(define
mod/appeal
(fn
(id kind val)
(mod/case-apply!
id
(fn (c) (mod/case-appeal c kind val mod/*reports* mod/*rules*))
true)))
(define
mod/finalize
(fn (id) (mod/case-apply! id (fn (c) (mod/case-finalize c)) false)))

View File

@@ -1,54 +0,0 @@
;; lib/mod/audit.sx — append-only decision log.
;;
;; Every decision the api commits is recorded as an immutable audit entry holding
;; the decision (action + matching rule), the proof tree (the derivation that
;; justified it), and a snapshot of the evidence in force at decision time. The
;; log is append-only: entries are never mutated or removed, only appended, each
;; with a monotonic sequence number. Retrieval is by report id (full history) or
;; by sequence.
(define mod/*audit-log* (list))
(define mod/*audit-seq* 0)
(define
mod/audit-reset!
(fn
()
(begin (set! mod/*audit-log* (list)) (set! mod/*audit-seq* 0))))
(define mod/mk-audit-entry (fn (seq decision evidence-snapshot) {:action (get decision :action) :evidence evidence-snapshot :proof (get decision :proof) :rule (get decision :rule) :report-id (get decision :report-id) :seq seq}))
(define
mod/log-decision!
(fn
(decision evidence-snapshot)
(begin
(set! mod/*audit-seq* (+ mod/*audit-seq* 1))
(let
((entry (mod/mk-audit-entry mod/*audit-seq* decision evidence-snapshot)))
(begin (append! mod/*audit-log* entry) entry)))))
;; entries for one report, in chronological (sequence) order
(define
mod/audit
(fn
(id)
(reduce
(fn
(acc e)
(if (= (get e :report-id) id) (append acc (list e)) acc))
(list)
mod/*audit-log*)))
(define mod/audit-all (fn () mod/*audit-log*))
(define mod/audit-count (fn () (len mod/*audit-log*)))
;; most recent decision logged for a report (nil if none)
(define
mod/audit-latest
(fn
(id)
(reduce
(fn (acc e) (if (= (get e :report-id) id) e acc))
nil
mod/*audit-log*)))

View File

@@ -1,55 +0,0 @@
;; lib/mod/batch.sx — batch triage + corpus analytics.
;;
;; Operational layer: decide a whole queue of reports at once, summarize the
;; outcomes by action, and measure which rules actually fire across a corpus.
;; mod/never-fired is the empirical complement to lint's static unreachable check
;; (Ext 5): lint finds rules that CAN'T fire by structure; never-fired finds rules
;; that DIDN'T fire on real data.
(define
mod/decide-batch
(fn
(reports rules)
(map (fn (r) (mod/decide-report r reports rules)) reports)))
(define
mod/count-action
(fn
(decisions action)
(reduce
(fn (acc d) (if (= (get d :action) action) (+ acc 1) acc))
0
decisions)))
(define mod/action-histogram (fn (decisions) {:keep (mod/count-action decisions "keep") :remove (mod/count-action decisions "remove") :escalate (mod/count-action decisions "escalate") :hide (mod/count-action decisions "hide") :ban (mod/count-action decisions "ban")}))
(define
mod/rule-fire-count
(fn
(decisions rule-name)
(reduce
(fn (acc d) (if (= (get d :rule) rule-name) (+ acc 1) acc))
0
decisions)))
(define
mod/rule-coverage
(fn
(reports rules)
(let
((decisions (mod/decide-batch reports rules)))
(map (fn (rule) {:rule (mod/rule-name rule) :fired (mod/rule-fire-count decisions (mod/rule-name rule))}) rules))))
(define
mod/never-fired
(fn
(reports rules)
(reduce
(fn
(acc c)
(if
(= (get c :fired) 0)
(append acc (list (get c :rule)))
acc))
(list)
(mod/rule-coverage reports rules))))

View File

@@ -1,60 +0,0 @@
# Mod conformance config — sourced by lib/guest/conformance.sh.
LANG_NAME=mod
MODE=dict
PRELOADS=(
lib/guest/pratt.sx
lib/prolog/tokenizer.sx
lib/prolog/parser.sx
lib/prolog/runtime.sx
lib/prolog/query.sx
lib/prolog/compiler.sx
lib/mod/schema.sx
lib/mod/policy.sx
lib/mod/defrule.sx
lib/mod/engine.sx
lib/mod/explain.sx
lib/mod/severity.sx
lib/mod/offenders.sx
lib/mod/quorum.sx
lib/mod/trace.sx
lib/mod/whatif.sx
lib/mod/batch.sx
lib/mod/temporal.sx
lib/mod/sla.sx
lib/mod/wire.sx
lib/mod/activity.sx
lib/mod/policies.sx
lib/mod/pipeline.sx
lib/mod/lifecycle.sx
lib/mod/audit.sx
lib/mod/api.sx
lib/mod/fed.sx
lib/mod/link.sx
lib/mod/lint.sx
)
SUITES=(
"decide:lib/mod/tests/decide.sx:(mod-decide-tests-run!)"
"audit:lib/mod/tests/audit.sx:(mod-audit-tests-run!)"
"escalation:lib/mod/tests/escalation.sx:(mod-escalation-tests-run!)"
"fed:lib/mod/tests/fed.sx:(mod-fed-tests-run!)"
"extensions:lib/mod/tests/extensions.sx:(mod-extensions-tests-run!)"
"link:lib/mod/tests/link.sx:(mod-link-tests-run!)"
"lint:lib/mod/tests/lint.sx:(mod-lint-tests-run!)"
"severity:lib/mod/tests/severity.sx:(mod-severity-tests-run!)"
"offenders:lib/mod/tests/offenders.sx:(mod-offenders-tests-run!)"
"quorum:lib/mod/tests/quorum.sx:(mod-quorum-tests-run!)"
"trace:lib/mod/tests/trace.sx:(mod-trace-tests-run!)"
"whatif:lib/mod/tests/whatif.sx:(mod-whatif-tests-run!)"
"batch:lib/mod/tests/batch.sx:(mod-batch-tests-run!)"
"temporal:lib/mod/tests/temporal.sx:(mod-temporal-tests-run!)"
"sla:lib/mod/tests/sla.sx:(mod-sla-tests-run!)"
"wire:lib/mod/tests/wire.sx:(mod-wire-tests-run!)"
"disjunction:lib/mod/tests/disjunction.sx:(mod-disjunction-tests-run!)"
"activity:lib/mod/tests/activity.sx:(mod-activity-tests-run!)"
"policies:lib/mod/tests/policies.sx:(mod-policies-tests-run!)"
"defrule:lib/mod/tests/defrule.sx:(mod-defrule-tests-run!)"
"pipeline:lib/mod/tests/pipeline.sx:(mod-pipeline-tests-run!)"
)

View File

@@ -1,16 +0,0 @@
;; lib/mod/defrule.sx — ergonomic rule / ruleset construction.
;;
;; The roadmap sketched a (defrule action :when conditions) surface. Conditions
;; already evaluate to plain data, so this needs no macro — variadic functions
;; suffice: mod/defrule collects its trailing condition forms via &rest (dropping
;; the explicit outer (list ...)), and mod/ruleset assembles rules the same way.
;;
;; (mod/ruleset
;; (mod/defrule "spam-hide" :hide (list :classification "spam"))
;; (mod/defrule "default-keep" :keep))
(define
mod/defrule
(fn (name action &rest conds) (mod/mk-rule name action conds)))
(define mod/ruleset (fn (&rest rules) rules))

View File

@@ -1,64 +0,0 @@
;; lib/mod/engine.sx — decide a report by querying the policy program.
;;
;; build-program assembles the report's facts plus the compiled policy clauses;
;; decide-report runs the Prolog query and returns a decision. A decision is a
;; proof, not a bare keyword: it carries the matching rule, the conditions it
;; required, the evidence that satisfied them, and a derivation — the proof tree.
;;
;; The proof tree is built constructively: for the matching rule, each body goal
;; is re-queried against the same DB with the report id bound, recording the goal
;; text, whether it was solved, and the bindings that satisfied it. That is a
;; genuine derivation drawn from the Prolog database, ready for the audit trail.
(define
mod/find-rule
(fn
(rules name)
(reduce
(fn
(acc r)
(if (nil? acc) (if (= (mod/rule-name r) name) r acc) acc))
nil
rules)))
(define
mod/build-program
(fn
(r count rules)
(str (mod/report-facts r count) "\n" (mod/rules->program rules))))
(define
mod/proof-goals
(fn
(db id conds)
(if
(empty? conds)
(list {:solved true :goal "true" :bindings {}})
(map
(fn
(c)
(let
((g (mod/cond->goal c id)))
(let ((sols (pl-query-all db g))) {:solved (if (empty? sols) false true) :goal g :bindings (if (empty? sols) {} (first sols))})))
conds))))
(define
mod/decide-report
(fn
(r reports rules)
(let
((count (mod/report-count (mod/report-about r) reports))
(kinds (mod/classify-keywords r))
(id (mod/report-id r)))
(let
((program (mod/build-program r count rules)))
(let
((db (pl-load program)))
(let
((sol (pl-query-one db (str "policy_action(" id ", Action, Rule)"))))
(if
(nil? sol)
{:action "keep" :proof {:goals (list) :evidence kinds :conditions (list) :rule "none" :count count} :report-id id :rule "none"}
(let
((rname (dict-get sol "Rule")))
(let ((rule (mod/find-rule rules rname))) {:action (mod/rule-action rule) :proof {:goals (mod/proof-goals db id (mod/rule-when rule)) :evidence kinds :conditions (mod/rule-when rule) :rule rname :count count} :report-id id :rule rname})))))))))

View File

@@ -1,55 +0,0 @@
;; lib/mod/explain.sx — human-readable proof explanation.
;;
;; Turns a decision (from mod/decide-report, or any audit entry) into a readable
;; multi-line "why": the action, the rule that fired, the evidence in play, and
;; the derivation goal-by-goal with [proved]/[unproved] marks and the unification
;; bindings that satisfied each goal. Pure SX over the Phase-2 proof tree.
(define
mod/explain-binds
(fn
(binds)
(mod/join-with
", "
(map (fn (k) (str k "=" (dict-get binds k))) (keys binds)))))
(define
mod/explain-goal
(fn
(g)
(let
((mark (if (get g :solved) " [proved] " " [unproved] "))
(binds (get g :bindings)))
(if
(empty? (keys binds))
(str mark (get g :goal))
(str mark (get g :goal) " {" (mod/explain-binds binds) "}")))))
(define
mod/explain-evidence
(fn
(evidence)
(if
(empty? evidence)
"Evidence: (none)"
(str "Evidence: " (mod/join-with ", " evidence)))))
(define
mod/explain
(fn
(decision)
(let
((id (get decision :report-id))
(action (get decision :action))
(rule (get decision :rule))
(proof (get decision :proof)))
(let
((goals (get proof :goals)) (evidence (get proof :evidence)))
(mod/join-with
"\n"
(append
(list
(str "Report " id ": " action " (rule: " rule ")")
(mod/explain-evidence evidence)
"Because:")
(map mod/explain-goal goals)))))))

View File

@@ -1,145 +0,0 @@
;; lib/mod/fed.sx — federation: cross-instance reports, decision sharing, trust,
;; revocation. fed-sx itself is mocked here (an in-memory outbox); the real wire
;; transport would replace mod/fed-send!.
;;
;; Trust is advisory by default (the hard rule): a peer's decision only binds
;; locally when (mod/trusted? peer :mod) holds. An untrusted peer's decision is
;; recorded as a suggestion in the advisory log and is NOT applied. Local
;; decisions propagate outward via the outbox. Revocation undoes a locally
;; applied action when its proof is invalidated, notifying the origin peer.
(define mod/*fed-trust* (list)) ;; {:peer :scope}
(define mod/*fed-outbox* (list)) ;; {:to :type :payload}
(define mod/*fed-advisory* (list)) ;; {:peer :decision} — received, not applied
(define mod/*fed-applied* (list)) ;; {:report-id :action :origin :revoked}
(define mod/*fed-origins* (list)) ;; {:id :origin}
(define
mod/fed-reset!
(fn
()
(begin
(set! mod/*fed-trust* (list))
(set! mod/*fed-outbox* (list))
(set! mod/*fed-advisory* (list))
(set! mod/*fed-applied* (list))
(set! mod/*fed-origins* (list)))))
;; ── trust model ──
(define
mod/trust-match?
(fn
(t peer scope)
(if (= (get t :peer) peer) (= (get t :scope) scope) false)))
(define
mod/grant-trust
(fn (peer scope) (begin (append! mod/*fed-trust* {:scope scope :peer peer}) true)))
(define
mod/revoke-trust
(fn
(peer scope)
(set!
mod/*fed-trust*
(reduce
(fn
(acc t)
(if (mod/trust-match? t peer scope) acc (append acc (list t))))
(list)
mod/*fed-trust*))))
(define
mod/trusted?
(fn
(peer scope)
(mod/any? (fn (t) (mod/trust-match? t peer scope)) mod/*fed-trust*)))
;; ── cross-instance reports ──
(define
mod/fed-receive-report
(fn
(peer by about reason)
(let
((r (mod/report by about reason)))
(begin (append! mod/*fed-origins* {:id (mod/report-id r) :origin peer}) r))))
(define
mod/report-origin
(fn
(id)
(reduce
(fn (acc o) (if (= (get o :id) id) (get o :origin) acc))
"local"
mod/*fed-origins*)))
;; ── decision sharing (mock fed-sx send) ──
(define
mod/fed-send!
(fn (to type payload) (begin (append! mod/*fed-outbox* {:type type :to to :payload payload}) true)))
(define mod/fed-outbox (fn () mod/*fed-outbox*))
(define
mod/fed-share-decision
(fn
(decision peers)
(reduce
(fn
(acc p)
(begin (mod/fed-send! p "decision" decision) (append acc (list p))))
(list)
peers)))
;; ── receiving a peer's decision (advisory unless trusted) ──
(define
mod/fed-applied-action
(fn
(report-id)
(reduce
(fn (acc a) (if (= (get a :report-id) report-id) a acc))
nil
mod/*fed-applied*)))
(define
mod/fed-receive-decision
(fn
(peer decision)
(if
(mod/trusted? peer :mod)
(begin (append! mod/*fed-applied* {:revoked false :action (get decision :action) :report-id (get decision :report-id) :origin peer}) {:advisory false :peer peer :applied true :decision decision})
(begin (append! mod/*fed-advisory* {:peer peer :decision decision}) {:advisory true :peer peer :applied false :decision decision}))))
;; ── revocation ──
(define
mod/fed-revoke!
(fn
(report-id reason)
(begin
(set!
mod/*fed-applied*
(map
(fn (a) (if (= (get a :report-id) report-id) {:revoked true :action (get a :action) :report-id (get a :report-id) :origin (get a :origin)} a))
mod/*fed-applied*))
(mod/fed-send! (mod/report-origin report-id) "revocation" {:report-id report-id :reason reason})
report-id)))
;; re-run the engine; if the action no longer holds, the prior decision's proof
;; is invalidated — revoke the applied moderation.
(define
mod/fed-revoke-if-invalidated
(fn
(report decision reports rules)
(let
((d2 (mod/decide-report report reports rules)))
(if
(= (get d2 :action) (get decision :action))
{:revoked false :decision d2}
(begin
(mod/fed-revoke! (get decision :report-id) "proof invalidated")
{:revoked true :decision d2})))))

View File

@@ -1,160 +0,0 @@
;; lib/mod/lifecycle.sx — report lifecycle state machine (pure SX over the engine).
;;
;; Lifecycle state is deliberately separate from policy: the Prolog rules answer
;; "what action?", this module answers "where in the process is this report?".
;;
;; :open ──triage──▶ :triaged ──resolve/review──▶ :decided ──appeal──▶ :appealed
;; │ │
;; └────finalize───▶ :final ◀┘
;;
;; A case is an immutable value {:report :state :decision :tier :error :history}.
;; Every transition returns a NEW case; illegal transitions return the case
;; unchanged with :error set. Tiers: triage runs the engine (auto-tier); a
;; terminal action (hide/remove/keep) resolves immediately, an :escalate action
;; flags the case for human review (human-tier) before it can be resolved.
(define mod/case* (fn (report state decision tier err history) {:history history :state state :report report :error err :tier tier :decision decision}))
(define
mod/mk-case
(fn (report) (mod/case* report "open" nil nil nil (list))))
(define mod/case-report (fn (c) (get c :report)))
(define mod/case-state (fn (c) (get c :state)))
(define mod/case-decision (fn (c) (get c :decision)))
(define mod/case-tier (fn (c) (get c :tier)))
(define mod/case-error (fn (c) (get c :error)))
(define mod/case-history (fn (c) (get c :history)))
;; ── transition table ──
(define mod/lc-transitions {:final (list) :appealed (list "final") :decided (list "appealed" "final") :open (list "triaged") :triaged (list "decided")})
(define mod/member? (fn (x lst) (mod/any? (fn (y) (= y x)) lst)))
(define
mod/lc-can-transition?
(fn
(from to)
(let
((outs (get mod/lc-transitions from)))
(if (nil? outs) false (mod/member? to outs)))))
;; ── core transition: validate, record history, or flag :error ──
(define
mod/case-goto
(fn
(c to note report decision tier)
(let
((from (mod/case-state c)))
(if
(mod/lc-can-transition? from to)
(mod/case*
report
to
decision
tier
nil
(append (mod/case-history c) (list {:note note :to to :from from})))
(mod/case*
(mod/case-report c)
from
(mod/case-decision c)
(mod/case-tier c)
(str "illegal transition: " from " -> " to)
(mod/case-history c))))))
(define
mod/case-error-set
(fn
(c msg)
(mod/case*
(mod/case-report c)
(mod/case-state c)
(mod/case-decision c)
(mod/case-tier c)
msg
(mod/case-history c))))
;; ── lifecycle operations ──
;; :open → :triaged — run the auto-tier first pass.
(define
mod/case-triage
(fn
(c reports rules)
(let
((d (mod/decide-report (mod/case-report c) reports rules)))
(let
((tier (if (= (get d :action) "escalate") "human" "auto")))
(mod/case-goto
c
"triaged"
"auto-tier first pass"
(mod/case-report c)
d
tier)))))
;; :triaged → :decided — auto-tier resolves; human-tier is blocked until review.
(define
mod/case-resolve
(fn
(c)
(if
(= (mod/case-tier c) "human")
(mod/case-error-set c "awaiting human review (escalated)")
(mod/case-goto
c
"decided"
"auto-tier resolved"
(mod/case-report c)
(mod/case-decision c)
(mod/case-tier c)))))
;; :triaged → :decided — human review: attach evidence, re-decide, resolve.
(define
mod/case-review
(fn
(c kind val reports rules)
(let
((nr (mod/attach-evidence (mod/case-report c) (mod/mk-evidence kind val))))
(let
((d (mod/decide-report nr reports rules)))
(mod/case-goto c "decided" (str "human review: " kind) nr d "human")))))
;; :decided → :appealed — appeal: attach evidence, re-decide (may override).
(define
mod/case-appeal
(fn
(c kind val reports rules)
(let
((nr (mod/attach-evidence (mod/case-report c) (mod/mk-evidence kind val))))
(let
((d (mod/decide-report nr reports rules)))
(mod/case-goto
c
"appealed"
(str "appeal: " kind)
nr
d
(mod/case-tier c))))))
;; :decided | :appealed → :final
(define
mod/case-finalize
(fn
(c)
(mod/case-goto
c
"final"
"finalized"
(mod/case-report c)
(mod/case-decision c)
(mod/case-tier c))))
(define
mod/case-action
(fn
(c)
(let ((d (mod/case-decision c))) (if (nil? d) nil (get d :action)))))

View File

@@ -1,92 +0,0 @@
;; lib/mod/link.sx — report linking + deduplication.
;;
;; Reports about the same subject form a cluster; identical reports (same
;; reporter + subject + reason) are duplicates. Linking is Prolog-backed: all
;; report facts are loaded and related ids are found by unification — the same
;; relational substrate the policy engine uses, here for retrieval rather than
;; decision. Dedup is pure SX over a normalized link key.
(define
mod/link-key
(fn
(r)
(str
(mod/report-by r)
"|"
(mod/report-about r)
"|"
(downcase (mod/report-reason r)))))
(define
mod/dedup-reports
(fn
(reports)
(reduce
(fn
(acc r)
(if
(mod/any? (fn (x) (= (mod/link-key x) (mod/link-key r))) acc)
acc
(append acc (list r))))
(list)
reports)))
(define
mod/duplicate-count
(fn (reports) (- (len reports) (len (mod/dedup-reports reports)))))
;; ── Prolog-backed relational retrieval ──
(define
mod/report-rel-facts
(fn
(reports)
(mod/join-with
"\n"
(map
(fn
(r)
(str
"report("
(mod/report-id r)
", "
(mod/pl-quote (mod/report-by r))
", "
(mod/pl-quote (mod/report-about r))
")."))
reports))))
(define
mod/related-ids
(fn
(subject reports)
(let
((db (pl-load (mod/report-rel-facts reports))))
(map
(fn (sol) (dict-get sol "Id"))
(pl-query-all db (str "report(Id, _, " (mod/pl-quote subject) ")"))))))
(define
mod/reporters-of
(fn
(subject reports)
(let
((db (pl-load (mod/report-rel-facts reports))))
(map
(fn (sol) (dict-get sol "By"))
(pl-query-all db (str "report(_, By, " (mod/pl-quote subject) ")"))))))
(define
mod/distinct
(fn
(items)
(reduce
(fn
(acc x)
(if (mod/any? (fn (y) (= y x)) acc) acc (append acc (list x))))
(list)
items)))
(define
mod/distinct-reporters-of
(fn (subject reports) (mod/distinct (mod/reporters-of subject reports))))

View File

@@ -1,69 +0,0 @@
;; lib/mod/lint.sx — static analysis of a policy rule set.
;;
;; Because precedence is "first matching clause wins" (pl-query-one), the rule
;; order has correctness consequences a moderator can get wrong: a rule placed
;; after an unconditional (empty :when) rule can never fire, and a rule set with
;; no unconditional rule may leave some reports undecided. lint-rules surfaces
;; these without running the engine.
(define mod/rule-unconditional? (fn (r) (empty? (mod/rule-when r))))
;; names of rules that follow the first unconditional rule — structurally dead,
;; since the unconditional rule always matches first
(define
mod/unreachable-rules
(fn
(rules)
(get
(reduce
(fn
(acc r)
(if
(get acc :hit)
{:dead (append (get acc :dead) (list (mod/rule-name r))) :hit true}
(if (mod/rule-unconditional? r) {:dead (get acc :dead) :hit true} acc)))
{:dead (list) :hit false}
rules)
:dead)))
(define
mod/has-catchall?
(fn (rules) (mod/any? mod/rule-unconditional? rules)))
(define
mod/count-eq
(fn
(x lst)
(reduce (fn (a y) (if (= y x) (+ a 1) a)) 0 lst)))
(define
mod/duplicate-rule-names
(fn
(rules)
(let
((names (map mod/rule-name rules)))
(mod/distinct
(reduce
(fn
(acc n)
(if
(< 1 (mod/count-eq n names))
(append acc (list n))
acc))
(list)
names)))))
(define mod/lint-rules (fn (rules) {:duplicate-names (mod/duplicate-rule-names rules) :has-catchall (mod/has-catchall? rules) :unreachable (mod/unreachable-rules rules)}))
;; a rule set is well-formed when nothing is dead, it has a catch-all, and rule
;; names are unique
(define
mod/rules-ok?
(fn
(rules)
(let
((l (mod/lint-rules rules)))
(if
(empty? (get l :unreachable))
(if (get l :has-catchall) (empty? (get l :duplicate-names)) false)
false))))

View File

@@ -1,59 +0,0 @@
;; lib/mod/offenders.sx — repeat-offender escalation (audit log as evidence).
;;
;; The append-only audit trail is itself a source of evidence: a subject already
;; sanctioned several times is a repeat offender. mod/decide-escalating decides a
;; report normally, then — if the action is a sanction and the subject has at
;; least k PRIOR sanctions in the audit log — upgrades it to :ban. This is the one
;; place a decision depends on history beyond the single report, and it reads that
;; history from the audit log rather than re-deriving it.
(define
mod/sanction?
(fn
(action)
(mod/any? (fn (a) (= a action)) (list "hide" "remove" "ban"))))
;; count of prior sanctioning decisions in the audit log about a subject
(define
mod/subject-sanctions
(fn
(subject)
(reduce
(fn
(acc e)
(let
((r (mod/get-report (get e :report-id))))
(if
(nil? r)
acc
(if
(if
(= (mod/report-about r) subject)
(mod/sanction? (get e :action))
false)
(+ acc 1)
acc))))
0
(mod/audit-all))))
(define
mod/repeat-offender?
(fn (subject k) (<= k (mod/subject-sanctions subject))))
(define
mod/decide-escalating
(fn
(id k)
(let
((r (mod/get-report id)))
(if
(nil? r)
nil
(let
((priors (mod/subject-sanctions (mod/report-about r))))
(let
((d (mod/decide id)))
(if
(if (mod/sanction? (get d :action)) (<= k priors) false)
{:action "ban" :proof {:goals (get (get d :proof) :goals) :prior-sanctions priors :evidence (get (get d :proof) :evidence) :conditions (list) :rule "repeat-offender-ban" :count (get (get d :proof) :count)} :report-id id :rule "repeat-offender-ban" :strategy "escalating"}
d)))))))

View File

@@ -1,18 +0,0 @@
;; lib/mod/pipeline.sx — end-to-end triage orchestration.
;;
;; A single entry point that runs a report through the subsystem and returns the
;; full artifact bundle: the decision (under the report's domain policy), a
;; human-readable explanation, an ActivityPub-shaped event for the bus, and the
;; wire line for federated peers. Composes policies (Ext 17), explain (Ext 3),
;; activity (Ext 16) and wire (Ext 14) — the modules are independent, this is just
;; the convenience that wires them together for the common "process a report" path.
(define
mod/triage-pipeline
(fn
(domain r reports actor)
(let ((d (mod/decide-in domain r reports))) {:activity (mod/decision->activity d actor) :action (get d :action) :wire (mod/decision->wire d) :rule (get d :rule) :decision d :explanation (mod/explain d)})))
(define mod/pipeline-action (fn (p) (get p :action)))
(define mod/pipeline-activity (fn (p) (get p :activity)))
(define mod/pipeline-wire (fn (p) (get p :wire)))

View File

@@ -1,40 +0,0 @@
;; lib/mod/policies.sx — per-domain policy registry.
;;
;; rose-ash spans domains (blog, market, events, federation, …) that want
;; different moderation — a marketplace listing and a blog comment are not held to
;; the same bar. This registry maps a domain to a rule set; mod/decide-in resolves
;; the right policy and decides. Unregistered domains fall back to the default
;; rules, so adding a domain never leaves it unmoderated.
(define mod/*policies* (list))
(define mod/policies-reset! (fn () (set! mod/*policies* (list))))
(define
mod/register-policy!
(fn (domain rules) (begin (append! mod/*policies* {:domain domain :rules rules}) true)))
(define
mod/policy-registered?
(fn
(domain)
(mod/any? (fn (p) (= (get p :domain) domain)) mod/*policies*)))
(define
mod/policy-for
(fn
(domain)
(reduce
(fn (acc p) (if (= (get p :domain) domain) (get p :rules) acc))
mod/default-rules
mod/*policies*)))
(define
mod/decide-in
(fn
(domain r reports)
(mod/decide-report r reports (mod/policy-for domain))))
(define
mod/registered-domains
(fn () (map (fn (p) (get p :domain)) mod/*policies*)))

View File

@@ -1,137 +0,0 @@
;; lib/mod/policy.sx — moderation rules → Prolog clauses.
;;
;; A rule is {:name :action :when}. :when is a list of condition forms; each
;; compiles to a Prolog goal. The conditions in a :when list are ANDed (joined by
;; ", "); :not negates and :any (a list of sub-conditions) disjoins — so the
;; condition language is a small boolean algebra over the leaf predicates.
;; Rule order is precedence: the engine queries with pl-query-one, so the first
;; clause that proves wins. The final default rule has an empty body (true) so
;; every report yields at least :keep — "no rule matched" is a real result, not a
;; query failure.
;;
;; cond->goal takes an id-term so the same condition can be compiled with the
;; head variable "Id" (for clause bodies) or a concrete report id (for proof-tree
;; goal-by-goal re-querying in the engine).
;;
;; Precedence (top wins): exoneration evidence (appeal override) > confirmed-abuse
;; evidence (human review) > spam/abuse classification > repeated-report count >
;; default keep.
(define mod/mk-rule (fn (name action conds) {:when conds :name name :action action}))
(define mod/rule-name (fn (r) (get r :name)))
(define mod/rule-action (fn (r) (get r :action)))
(define mod/rule-when (fn (r) (get r :when)))
(define
mod/default-rules
(list
(mod/mk-rule
"exonerated-keep"
:keep (list (list :evidence "exonerated")))
(mod/mk-rule
"reviewer-remove"
:remove (list (list :evidence "confirmed-abuse")))
(mod/mk-rule "spam-hide" :hide (list (list :classification "spam")))
(mod/mk-rule
"abuse-remove"
:remove (list (list :classification "abuse")))
(mod/mk-rule
"repeated-escalate"
:escalate (list (list :count-at-least 3)))
(mod/mk-rule "default-keep" :keep (list))))
;; ── condition → Prolog goal ──
;;
;; (:classification "spam") → classification(Id, spam)
;; (:evidence "kind") → evidence(Id, 'kind', _)
;; (:attr "verified") → attr(Id, verified)
;; (:not <cond>) → not(<cond>) (negation)
;; (:any (list c1 c2 ...)) → (g1 ; g2 ; ...) (disjunction)
;; (:count-at-least 3) → report(Id, B, S), report_count(S, N), N >= 3
;; (:score-at-least 5) → aggregate_all(sum(W), signal(Id, _, W), T), T >= 5
;; (:reporters-at-least 2) → report(Id, _, Sr), setof(Br, report(_, Br, Sr), Bsr),
;; length(Bsr, Nr), Nr >= 2 (quorum engine)
;; (:burst-at-least 3) → report(Id, _, Sb), burst_count(Sb, Nb), Nb >= 3
;; (temporal engine)
(define
mod/cond->goal
(fn
(c idterm)
(let
((tag (first c)))
(cond
((= tag :classification)
(str "classification(" idterm ", " (nth c 1) ")"))
((= tag :evidence)
(str
"evidence("
idterm
", "
(mod/pl-quote (nth c 1))
", _)"))
((= tag :attr) (str "attr(" idterm ", " (nth c 1) ")"))
((= tag :not)
(str "not(" (mod/cond->goal (nth c 1) idterm) ")"))
((= tag :any)
(str
"("
(mod/join-with
" ; "
(map
(fn (sub) (mod/cond->goal sub idterm))
(nth c 1)))
")"))
((= tag :count-at-least)
(str
"report("
idterm
", B, S), report_count(S, N), N >= "
(nth c 1)))
((= tag :score-at-least)
(str
"aggregate_all(sum(W), signal("
idterm
", _, W), T), T >= "
(nth c 1)))
((= tag :reporters-at-least)
(str
"report("
idterm
", _, Sr), setof(Br, report(_, Br, Sr), Bsr), "
"length(Bsr, Nr), Nr >= "
(nth c 1)))
((= tag :burst-at-least)
(str
"report("
idterm
", _, Sb), burst_count(Sb, Nb), Nb >= "
(nth c 1)))
(true "true")))))
(define
mod/conds->body
(fn
(conds idterm)
(if
(empty? conds)
"true"
(mod/join-with ", " (map (fn (c) (mod/cond->goal c idterm)) conds)))))
(define
mod/rule->clause
(fn
(r)
(str
"policy_action(Id, "
(mod/rule-action r)
", '"
(mod/rule-name r)
"') :- "
(mod/conds->body (mod/rule-when r) "Id")
".")))
(define
mod/rules->program
(fn (rules) (mod/join-with "\n" (map mod/rule->clause rules))))

View File

@@ -1,40 +0,0 @@
;; lib/mod/quorum.sx — quorum decisions over distinct reporters (anti-brigade).
;;
;; The base engine asserts only the decided report's report/3 fact, so it can't
;; reason about WHO reported a subject. The quorum engine additionally asserts
;; every report's report/3 fact (via link's rel-facts), letting a rule require N
;; *distinct* reporters with `setof`/`length` — so one user filing many reports
;; does not manufacture consensus. Same decision shape as the base engine, plus
;; :strategy "quorum".
(define
mod/build-quorum-program
(fn
(r count reports rules)
(str
(mod/report-rel-facts reports)
"\n"
(mod/report-facts r count)
"\n"
(mod/rules->program rules))))
(define
mod/decide-quorum
(fn
(r reports rules)
(let
((count (mod/report-count (mod/report-about r) reports))
(kinds (mod/classify-keywords r))
(id (mod/report-id r)))
(let
((program (mod/build-quorum-program r count reports rules)))
(let
((db (pl-load program)))
(let
((sol (pl-query-one db (str "policy_action(" id ", Action, Rule)"))))
(if
(nil? sol)
{:action "keep" :proof {:goals (list) :evidence kinds :conditions (list) :rule "none" :count count} :report-id id :rule "none" :strategy "quorum"}
(let
((rule (mod/find-rule rules (dict-get sol "Rule"))))
{:action (mod/rule-action rule) :proof {:goals (mod/proof-goals db id (mod/rule-when rule)) :evidence kinds :conditions (mod/rule-when rule) :rule (mod/rule-name rule) :count count} :report-id id :rule (mod/rule-name rule) :strategy "quorum"}))))))))

View File

@@ -1,259 +0,0 @@
;; lib/mod/schema.sx — report representation + Prolog fact generation.
;;
;; A report is a dict {:id :by :about :reason :evidence :attrs :signals :at}.
;; :evidence — accumulated {:kind :val} entries (human review, scanners)
;; :attrs — attribute names ("verified") for negation-as-failure conditions
;; :signals — weighted {:kind :weight} entries for aggregate scoring rules
;; :at — integer timestamp/tick (deterministic; supplied, not clock-read)
;; The engine derives keyword classifications from the reason text and projects
;; the report, its classifications, evidence, attributes, and signals into Prolog
;; facts that policy clauses match against.
(define mod/mk-report (fn (id by about reason) {:attrs (list) :id id :signals (list) :by by :evidence (list) :about about :at 0 :reason reason}))
(define mod/report-id (fn (r) (get r :id)))
(define mod/report-by (fn (r) (get r :by)))
(define mod/report-about (fn (r) (get r :about)))
(define mod/report-reason (fn (r) (get r :reason)))
(define
mod/report-evidence
(fn (r) (let ((e (get r :evidence))) (if (nil? e) (list) e))))
(define
mod/report-attrs
(fn (r) (let ((a (get r :attrs))) (if (nil? a) (list) a))))
(define
mod/report-signals
(fn (r) (let ((s (get r :signals))) (if (nil? s) (list) s))))
(define
mod/report-at
(fn (r) (let ((t (get r :at))) (if (nil? t) 0 t))))
(define mod/mk-evidence (fn (kind val) {:val val :kind kind}))
(define mod/evidence-kind (fn (e) (get e :kind)))
(define mod/evidence-val (fn (e) (get e :val)))
(define mod/mk-signal (fn (kind weight) {:kind kind :weight weight}))
(define mod/signal-kind (fn (s) (get s :kind)))
(define mod/signal-weight (fn (s) (get s :weight)))
(define mod/report* (fn (r evs attrs sigs at) {:attrs attrs :id (mod/report-id r) :signals sigs :by (mod/report-by r) :evidence evs :about (mod/report-about r) :at at :reason (mod/report-reason r)}))
(define
mod/with-evidence
(fn
(r evs)
(mod/report*
r
evs
(mod/report-attrs r)
(mod/report-signals r)
(mod/report-at r))))
(define
mod/with-attrs
(fn
(r attrs)
(mod/report*
r
(mod/report-evidence r)
attrs
(mod/report-signals r)
(mod/report-at r))))
(define
mod/with-signals
(fn
(r sigs)
(mod/report*
r
(mod/report-evidence r)
(mod/report-attrs r)
sigs
(mod/report-at r))))
(define
mod/with-at
(fn
(r at)
(mod/report*
r
(mod/report-evidence r)
(mod/report-attrs r)
(mod/report-signals r)
at)))
(define
mod/attach-evidence
(fn
(r e)
(mod/with-evidence r (append (mod/report-evidence r) (list e)))))
(define
mod/attach-attr
(fn (r a) (mod/with-attrs r (append (mod/report-attrs r) (list a)))))
(define
mod/attach-signal
(fn (r s) (mod/with-signals r (append (mod/report-signals r) (list s)))))
;; ── substring search (the prolog-loaded env lacks includes?; slice/len do work) ──
(define
mod/contains-at?
(fn
(hay needle hl nl pos)
(if
(< hl (+ pos nl))
false
(if
(= (slice hay pos (+ pos nl)) needle)
true
(mod/contains-at? hay needle hl nl (+ pos 1))))))
(define
mod/str-contains?
(fn
(hay needle)
(let
((hl (len hay)) (nl (len needle)))
(if
(= nl 0)
true
(mod/contains-at? hay needle hl nl 0)))))
;; ── evidence derivation (keyword classification) ──
(define
mod/spam-keywords
(list "spam" "buy now" "click here" "free money" "viagra" "limited offer"))
(define
mod/abuse-keywords
(list "abuse" "harassment" "threat" "slur" "hate speech"))
(define
mod/any?
(fn (pred coll) (reduce (fn (acc x) (if acc acc (pred x))) false coll)))
(define
mod/reason-matches?
(fn
(reason kws)
(let
((low (downcase reason)))
(mod/any? (fn (k) (mod/str-contains? low k)) kws))))
(define
mod/classify-keywords
(fn
(r)
(let
((reason (mod/report-reason r)) (kinds (list)))
(begin
(when
(mod/reason-matches? reason mod/spam-keywords)
(append! kinds "spam"))
(when
(mod/reason-matches? reason mod/abuse-keywords)
(append! kinds "abuse"))
kinds))))
(define
mod/report-count
(fn
(about reports)
(reduce
(fn
(acc r)
(if (= (mod/report-about r) about) (+ acc 1) acc))
0
reports)))
;; ── Prolog fact projection ──
(define
mod/join-with
(fn
(sep items)
(reduce (fn (acc x) (if (= acc "") x (str acc sep x))) "" items)))
(define mod/pl-quote (fn (s) (str "'" s "'")))
(define
mod/classification-facts
(fn
(id kinds)
(mod/join-with
"\n"
(map (fn (k) (str "classification(" id ", " k ").")) kinds))))
(define
mod/evidence-facts
(fn
(id evs)
(mod/join-with
"\n"
(map
(fn
(e)
(str
"evidence("
id
", "
(mod/pl-quote (mod/evidence-kind e))
", "
(mod/pl-quote (str (mod/evidence-val e)))
")."))
evs))))
(define
mod/attr-facts
(fn
(id attrs)
(mod/join-with "\n" (map (fn (a) (str "attr(" id ", " a ").")) attrs))))
(define
mod/signal-facts
(fn
(id sigs)
(mod/join-with
"\n"
(map
(fn
(s)
(str
"signal("
id
", "
(mod/pl-quote (mod/signal-kind s))
", "
(mod/signal-weight s)
")."))
sigs))))
(define
mod/report-facts
(fn
(r count)
(let
((id (mod/report-id r))
(by (mod/pl-quote (mod/report-by r)))
(about (mod/pl-quote (mod/report-about r))))
(let
((cls (mod/classification-facts id (mod/classify-keywords r)))
(evs (mod/evidence-facts id (mod/report-evidence r)))
(ats (mod/attr-facts id (mod/report-attrs r)))
(sgs (mod/signal-facts id (mod/report-signals r))))
(mod/join-with
"\n"
(list
(str "report(" id ", " by ", " about ").")
(str "report_count(" about ", " count ").")
cls
evs
ats
sgs))))))

View File

@@ -1,30 +0,0 @@
{
"lang": "mod",
"total_passed": 390,
"total_failed": 0,
"total": 390,
"suites": [
{"name":"decide","passed":31,"failed":0,"total":31},
{"name":"audit","passed":29,"failed":0,"total":29},
{"name":"escalation","passed":46,"failed":0,"total":46},
{"name":"fed","passed":26,"failed":0,"total":26},
{"name":"extensions","passed":32,"failed":0,"total":32},
{"name":"link","passed":12,"failed":0,"total":12},
{"name":"lint","passed":14,"failed":0,"total":14},
{"name":"severity","passed":14,"failed":0,"total":14},
{"name":"offenders","passed":19,"failed":0,"total":19},
{"name":"quorum","passed":9,"failed":0,"total":9},
{"name":"trace","passed":15,"failed":0,"total":15},
{"name":"whatif","passed":13,"failed":0,"total":13},
{"name":"batch","passed":17,"failed":0,"total":17},
{"name":"temporal","passed":15,"failed":0,"total":15},
{"name":"sla","passed":15,"failed":0,"total":15},
{"name":"wire","passed":16,"failed":0,"total":16},
{"name":"disjunction","passed":10,"failed":0,"total":10},
{"name":"activity","passed":17,"failed":0,"total":17},
{"name":"policies","passed":14,"failed":0,"total":14},
{"name":"defrule","passed":11,"failed":0,"total":11},
{"name":"pipeline","passed":15,"failed":0,"total":15}
],
"generated": "2026-06-06T19:40:03+00:00"
}

View File

@@ -1,27 +0,0 @@
# mod scoreboard
**390 / 390 passing** (0 failure(s)).
| Suite | Passed | Total | Status |
|-------|--------|-------|--------|
| decide | 31 | 31 | ok |
| audit | 29 | 29 | ok |
| escalation | 46 | 46 | ok |
| fed | 26 | 26 | ok |
| extensions | 32 | 32 | ok |
| link | 12 | 12 | ok |
| lint | 14 | 14 | ok |
| severity | 14 | 14 | ok |
| offenders | 19 | 19 | ok |
| quorum | 9 | 9 | ok |
| trace | 15 | 15 | ok |
| whatif | 13 | 13 | ok |
| batch | 17 | 17 | ok |
| temporal | 15 | 15 | ok |
| sla | 15 | 15 | ok |
| wire | 16 | 16 | ok |
| disjunction | 10 | 10 | ok |
| activity | 17 | 17 | ok |
| policies | 14 | 14 | ok |
| defrule | 11 | 11 | ok |
| pipeline | 15 | 15 | ok |

View File

@@ -1,60 +0,0 @@
;; lib/mod/severity.sx — "strictest-wins" decision strategy.
;;
;; The default engine resolves precedence by rule ORDER (first proven clause wins,
;; via pl-query-one). Some policies instead want the HARSHEST applicable sanction
;; regardless of order. mod/decide-strictest collects every rule that proves
;; (pl-query-all) and picks the highest-severity action. Same decision shape as
;; the engine, plus :strategy. Built over the engine's helpers; engine untouched.
(define
mod/action-severity
(fn
(action)
(cond
((= action "ban") 4)
((= action "remove") 3)
((= action "hide") 2)
((= action "escalate") 1)
(true 0))))
(define
mod/strictest-sol
(fn
(sols)
(reduce
(fn
(acc s)
(if
(nil? acc)
s
(if
(<
(mod/action-severity (dict-get acc "Action"))
(mod/action-severity (dict-get s "Action")))
s
acc)))
nil
sols)))
(define
mod/decide-strictest
(fn
(r reports rules)
(let
((count (mod/report-count (mod/report-about r) reports))
(kinds (mod/classify-keywords r))
(id (mod/report-id r)))
(let
((program (mod/build-program r count rules)))
(let
((db (pl-load program)))
(let
((sols (pl-query-all db (str "policy_action(" id ", Action, Rule)"))))
(let
((best (mod/strictest-sol sols)))
(if
(nil? best)
{:action "keep" :proof {:goals (list) :evidence kinds :conditions (list) :rule "none" :count count} :report-id id :rule "none" :strategy "strictest"}
(let
((rule (mod/find-rule rules (dict-get best "Rule"))))
{:action (mod/rule-action rule) :proof {:goals (mod/proof-goals db id (mod/rule-when rule)) :evidence kinds :conditions (mod/rule-when rule) :rule (mod/rule-name rule) :count count} :report-id id :rule (mod/rule-name rule) :strategy "strictest"})))))))))

View File

@@ -1,47 +0,0 @@
;; lib/mod/sla.sx — service-level sweep over pending lifecycle cases.
;;
;; Composes the Phase-3 lifecycle with the Ext-12 time dimension: a case left in a
;; pending state (open / triaged / appealed) past a deadline has breached SLA and
;; should resurface. A timed-case pairs a case with the tick it entered its
;; current state (the caller stamps this — the lifecycle stays timeless and pure).
;; Terminal states (decided / final) never breach.
(define mod/pending-states (list "open" "triaged" "appealed"))
(define mod/pending-state? (fn (s) (mod/member? s mod/pending-states)))
(define mod/mk-timed-case (fn (c entered-at) {:entered-at entered-at :case c}))
(define mod/tc-case (fn (tc) (get tc :case)))
(define mod/tc-entered-at (fn (tc) (get tc :entered-at)))
(define
mod/overdue?
(fn
(tc now deadline)
(if
(mod/pending-state? (mod/case-state (mod/tc-case tc)))
(< deadline (- now (mod/tc-entered-at tc)))
false)))
(define
mod/sla-sweep
(fn
(timed-cases now deadline)
(reduce
(fn
(acc tc)
(if
(mod/overdue? tc now deadline)
(append
acc
(list (mod/report-id (mod/case-report (mod/tc-case tc)))))
acc))
(list)
timed-cases)))
(define
mod/overdue-count
(fn
(timed-cases now deadline)
(len (mod/sla-sweep timed-cases now deadline))))
(define mod/age (fn (tc now) (- now (mod/tc-entered-at tc))))

View File

@@ -1,62 +0,0 @@
;; lib/mod/temporal.sx — burst detection over a time window.
;;
;; A plain report count can't tell a burst (N reports in minutes) from slow
;; accumulation (N reports over months). mod/decide-temporal takes a `now` tick
;; and a `window`, counts reports about the subject with :at within [now-window,
;; now], asserts it as burst_count/2, and lets a `(:burst-at-least K)` rule fire
;; only on a genuine burst. Time is supplied (deterministic), never clock-read.
(define
mod/window-count
(fn
(subject reports now window)
(reduce
(fn
(acc r)
(if
(if
(= (mod/report-about r) subject)
(<= (- now window) (mod/report-at r))
false)
(+ acc 1)
acc))
0
reports)))
(define
mod/build-temporal-program
(fn
(r count bcount rules)
(str
(mod/report-facts r count)
"\n"
"burst_count("
(mod/pl-quote (mod/report-about r))
", "
bcount
").\n"
(mod/rules->program rules))))
(define
mod/decide-temporal
(fn
(r reports rules now window)
(let
((about (mod/report-about r))
(id (mod/report-id r))
(kinds (mod/classify-keywords r)))
(let
((count (mod/report-count about reports))
(bcount (mod/window-count about reports now window)))
(let
((program (mod/build-temporal-program r count bcount rules)))
(let
((db (pl-load program)))
(let
((sol (pl-query-one db (str "policy_action(" id ", Action, Rule)"))))
(if
(nil? sol)
{:action "keep" :proof {:burst bcount :goals (list) :evidence kinds :conditions (list) :rule "none" :count count} :report-id id :rule "none" :strategy "temporal"}
(let
((rule (mod/find-rule rules (dict-get sol "Rule"))))
{:action (mod/rule-action rule) :proof {:burst bcount :goals (mod/proof-goals db id (mod/rule-when rule)) :evidence kinds :conditions (mod/rule-when rule) :rule (mod/rule-name rule) :count count} :report-id id :rule (mod/rule-name rule) :strategy "temporal"})))))))))

View File

@@ -1,95 +0,0 @@
;; lib/mod/tests/activity.sx — Ext 16: ActivityPub-shaped decision export.
(define mod-ap-count 0)
(define mod-ap-pass 0)
(define mod-ap-fail 0)
(define mod-ap-failures (list))
(define
mod-ap-test!
(fn
(name got expected)
(begin
(set! mod-ap-count (+ mod-ap-count 1))
(if
(= got expected)
(set! mod-ap-pass (+ mod-ap-pass 1))
(begin
(set! mod-ap-fail (+ mod-ap-fail 1))
(append!
mod-ap-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── action → AP verb ──
(mod-ap-test! "remove → Delete" (mod/action->verb "remove") "Delete")
(mod-ap-test! "ban → Block" (mod/action->verb "ban") "Block")
(mod-ap-test! "hide → Flag" (mod/action->verb "hide") "Flag")
(mod-ap-test! "escalate → Flag" (mod/action->verb "escalate") "Flag")
(mod-ap-test! "keep → nil (no activity)" (mod/action->verb "keep") nil)
;; ── single decision → activity ──
(define mod-ap-spam (mod/mk-report "r1" "a" "bob" "this is spam"))
(define
mod-ap-dec
(mod/decide-report mod-ap-spam (list mod-ap-spam) mod/default-rules))
(define mod-ap-act (mod/decision->activity mod-ap-dec "instance.example"))
(mod-ap-test! "activity type is Flag (hide)" (get mod-ap-act :type) "Flag")
(mod-ap-test! "activity object is report id" (get mod-ap-act :object) "r1")
(mod-ap-test!
"activity actor preserved"
(get mod-ap-act :actor)
"instance.example")
(mod-ap-test!
"activity preserves precise action"
(get mod-ap-act :action)
"hide")
(mod-ap-test! "activity carries rule" (get mod-ap-act :rule) "spam-hide")
(mod-ap-test!
"activity summary"
(get mod-ap-act :summary)
"moderation/hide via spam-hide")
;; ── keep produces no activity ──
(define mod-ap-clean (mod/mk-report "r2" "a" "b" "a fine post"))
(define
mod-ap-keep
(mod/decide-report mod-ap-clean (list mod-ap-clean) mod/default-rules))
(mod-ap-test!
"keep decision → nil activity"
(mod/decision->activity mod-ap-keep "x")
nil)
;; ── abuse → Delete ──
(define mod-ap-abuse (mod/mk-report "r3" "a" "b" "harassment here"))
(define
mod-ap-abuse-dec
(mod/decide-report mod-ap-abuse (list mod-ap-abuse) mod/default-rules))
(mod-ap-test!
"abuse decision → Delete activity"
(get (mod/decision->activity mod-ap-abuse-dec "x") :type)
"Delete")
;; ── batch export drops keeps ──
(define mod-ap-decisions (list mod-ap-dec mod-ap-keep mod-ap-abuse-dec))
(define mod-ap-acts (mod/decisions->activities mod-ap-decisions "inst"))
(mod-ap-test! "batch export drops the keep" (len mod-ap-acts) 2)
(mod-ap-test!
"batch export first is the Flag"
(get (first mod-ap-acts) :type)
"Flag")
(mod-ap-test!
"batch export second is the Delete"
(get (nth mod-ap-acts 1) :type)
"Delete")
(mod-ap-test!
"empty decisions → no activities"
(mod/decisions->activities (list) "inst")
(list))
(define mod-activity-tests-run! (fn () {:failures mod-ap-failures :total mod-ap-count :passed mod-ap-pass :failed mod-ap-fail}))

View File

@@ -1,187 +0,0 @@
;; lib/mod/tests/audit.sx — Phase 2: evidence accumulation + proof tree + audit.
(define mod-aud-count 0)
(define mod-aud-pass 0)
(define mod-aud-fail 0)
(define mod-aud-failures (list))
(define
mod-aud-test!
(fn
(name got expected)
(begin
(set! mod-aud-count (+ mod-aud-count 1))
(if
(= got expected)
(set! mod-aud-pass (+ mod-aud-pass 1))
(begin
(set! mod-aud-fail (+ mod-aud-fail 1))
(append!
mod-aud-failures
(str name "\n expected: " expected "\n got: " got)))))))
(define
mod-aud-decide1
(fn (r) (mod/decide-report r (list r) mod/default-rules)))
;; ── proof tree: keyword classification ──
(define
mod-aud-spam
(mod-aud-decide1 (mod/mk-report "r1" "alice" "bob" "this is spam")))
(define mod-aud-spam-goals (get (get mod-aud-spam :proof) :goals))
(mod-aud-test! "spam proof has one goal" (len mod-aud-spam-goals) 1)
(mod-aud-test!
"spam proof goal text"
(get (first mod-aud-spam-goals) :goal)
"classification(r1, spam)")
(mod-aud-test!
"spam proof goal solved"
(get (first mod-aud-spam-goals) :solved)
true)
;; ── proof tree: count rule with real bindings ──
(define mod-aud-rep-r (mod/mk-report "r3" "ann" "dave" "x"))
(define
mod-aud-rep
(mod/decide-report
mod-aud-rep-r
(list mod-aud-rep-r mod-aud-rep-r mod-aud-rep-r)
mod/default-rules))
(define mod-aud-rep-goals (get (get mod-aud-rep :proof) :goals))
(define mod-aud-rep-binds (get (first mod-aud-rep-goals) :bindings))
(mod-aud-test!
"count proof goal solved"
(get (first mod-aud-rep-goals) :solved)
true)
(mod-aud-test! "count proof binding N" (dict-get mod-aud-rep-binds "N") "3")
(mod-aud-test!
"count proof binding S (subject)"
(dict-get mod-aud-rep-binds "S")
"dave")
;; ── proof tree: default keep has a 'true' goal ──
(define
mod-aud-keep
(mod-aud-decide1 (mod/mk-report "rk" "a" "b" "a fine post")))
(define mod-aud-keep-goals (get (get mod-aud-keep :proof) :goals))
(mod-aud-test!
"keep proof goal text true"
(get (first mod-aud-keep-goals) :goal)
"true")
(mod-aud-test!
"keep proof goal solved"
(get (first mod-aud-keep-goals) :solved)
true)
;; ── evidence accumulation drives a rule ──
(define
mod-aud-rev-r
(mod/attach-evidence
(mod/mk-report "re" "a" "carol" "neutral")
(mod/mk-evidence "confirmed-abuse" "human")))
(define mod-aud-rev (mod-aud-decide1 mod-aud-rev-r))
(mod-aud-test!
"evidence has length 1"
(len (mod/report-evidence mod-aud-rev-r))
1)
(mod-aud-test!
"evidence reviewer-remove → remove"
(get mod-aud-rev :action)
"remove")
(mod-aud-test!
"evidence reviewer-remove rule"
(get mod-aud-rev :rule)
"reviewer-remove")
(mod-aud-test!
"evidence proof goal solved"
(get (first (get (get mod-aud-rev :proof) :goals)) :solved)
true)
(mod-aud-test!
"no evidence → not reviewer-remove"
(get (mod-aud-decide1 (mod/mk-report "rn" "a" "b" "neutral")) :rule)
"default-keep")
;; ── append-only audit log via the api ──
(mod/reset!)
(mod/report "alice" "bob" "this is spam")
(mod/report "carol" "eve" "fine post")
(define mod-aud-d1 (mod/decide "r1"))
(define mod-aud-d2 (mod/decide "r2"))
(mod-aud-test! "two decisions logged" (mod/audit-count) 2)
(mod-aud-test!
"first entry seq 1"
(get (first (mod/audit-all)) :seq)
1)
(mod-aud-test!
"audit r1 returns one entry"
(len (mod/audit "r1"))
1)
(mod-aud-test!
"audit r1 action matches decision"
(get (first (mod/audit "r1")) :action)
(get mod-aud-d1 :action))
(mod-aud-test!
"audit r1 rule matches decision"
(get (first (mod/audit "r1")) :rule)
"spam-hide")
(mod-aud-test!
"audit r1 entry carries proof goals"
(len (get (get (first (mod/audit "r1")) :proof) :goals))
1)
(mod-aud-test!
"audit r2 keep"
(get (first (mod/audit "r2")) :action)
"keep")
(mod-aud-test! "audit unknown report → empty" (mod/audit "r99") (list))
;; ── append-only: re-deciding appends, never mutates ──
(define mod-aud-d1b (mod/decide "r1"))
(mod-aud-test! "re-decide appends (count 3)" (mod/audit-count) 3)
(mod-aud-test!
"audit r1 now has 2 entries"
(len (mod/audit "r1"))
2)
(mod-aud-test!
"audit r1 seqs monotonic"
(get (nth (mod/audit "r1") 1) :seq)
3)
(mod-aud-test!
"audit-latest r1 is seq 3"
(get (mod/audit-latest "r1") :seq)
3)
(mod-aud-test!
"first r1 entry unchanged (still seq 1)"
(get (first (mod/audit "r1")) :seq)
1)
;; ── evidence snapshot captured at decision time ──
(mod/add-evidence "r2" "confirmed-abuse" "human")
(define mod-aud-d2b (mod/decide "r2"))
(mod-aud-test!
"post-evidence decision flips to remove"
(get mod-aud-d2b :action)
"remove")
(mod-aud-test!
"audit snapshot records evidence kind"
(mod/evidence-kind (first (get (mod/audit-latest "r2") :evidence)))
"confirmed-abuse")
(mod-aud-test!
"earlier r2 entry had empty evidence snapshot"
(len (get (first (mod/audit "r2")) :evidence))
0)
(define mod-audit-tests-run! (fn () {:failures mod-aud-failures :total mod-aud-count :passed mod-aud-pass :failed mod-aud-fail}))

View File

@@ -1,101 +0,0 @@
;; lib/mod/tests/batch.sx — Ext 11: batch triage + corpus analytics.
(define mod-b-count 0)
(define mod-b-pass 0)
(define mod-b-fail 0)
(define mod-b-failures (list))
(define
mod-b-test!
(fn
(name got expected)
(begin
(set! mod-b-count (+ mod-b-count 1))
(if
(= got expected)
(set! mod-b-pass (+ mod-b-pass 1))
(begin
(set! mod-b-fail (+ mod-b-fail 1))
(append!
mod-b-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; corpus: 2 spam, 1 abuse, 2 clean — distinct subjects so the count rule stays quiet
(define
mod-b-corpus
(list
(mod/mk-report "r1" "u" "s1" "this is spam")
(mod/mk-report "r2" "u" "s2" "buy now offer")
(mod/mk-report "r3" "u" "s3" "harassment here")
(mod/mk-report "r4" "u" "s4" "a fine post")
(mod/mk-report "r5" "u" "s5" "thanks for sharing")))
(define mod-b-decisions (mod/decide-batch mod-b-corpus mod/default-rules))
;; ── decide-batch ──
(mod-b-test! "one decision per report" (len mod-b-decisions) 5)
(mod-b-test!
"first decision is hide"
(get (first mod-b-decisions) :action)
"hide")
;; ── action histogram ──
(define mod-b-hist (mod/action-histogram mod-b-decisions))
(mod-b-test! "histogram hide count" (get mod-b-hist :hide) 2)
(mod-b-test! "histogram remove count" (get mod-b-hist :remove) 1)
(mod-b-test! "histogram keep count" (get mod-b-hist :keep) 2)
(mod-b-test! "histogram escalate count" (get mod-b-hist :escalate) 0)
(mod-b-test! "histogram ban count" (get mod-b-hist :ban) 0)
(mod-b-test!
"histogram totals match corpus"
(+
(+ (get mod-b-hist :hide) (get mod-b-hist :remove))
(+
(get mod-b-hist :keep)
(+ (get mod-b-hist :escalate) (get mod-b-hist :ban))))
5)
;; ── rule coverage (empirical) ──
(define mod-b-cov (mod/rule-coverage mod-b-corpus mod/default-rules))
(mod-b-test! "coverage has one row per rule" (len mod-b-cov) 6)
(mod-b-test!
"spam-hide fired twice"
(mod/rule-fire-count mod-b-decisions "spam-hide")
2)
(mod-b-test!
"abuse-remove fired once"
(mod/rule-fire-count mod-b-decisions "abuse-remove")
1)
(mod-b-test!
"default-keep fired twice"
(mod/rule-fire-count mod-b-decisions "default-keep")
2)
;; ── never-fired: rules not exercised by this corpus ──
(define mod-b-never (mod/never-fired mod-b-corpus mod/default-rules))
(mod-b-test!
"exonerated-keep never fired"
(mod/member? "exonerated-keep" mod-b-never)
true)
(mod-b-test!
"reviewer-remove never fired"
(mod/member? "reviewer-remove" mod-b-never)
true)
(mod-b-test!
"repeated-escalate never fired"
(mod/member? "repeated-escalate" mod-b-never)
true)
(mod-b-test!
"spam-hide DID fire (not in never-fired)"
(mod/member? "spam-hide" mod-b-never)
false)
(mod-b-test!
"three rules never fired on this corpus"
(len mod-b-never)
3)
(define mod-batch-tests-run! (fn () {:failures mod-b-failures :total mod-b-count :passed mod-b-pass :failed mod-b-fail}))

View File

@@ -1,215 +0,0 @@
;; lib/mod/tests/decide.sx — Phase 1: report representation + simple policy.
(define mod-dec-count 0)
(define mod-dec-pass 0)
(define mod-dec-fail 0)
(define mod-dec-failures (list))
(define
mod-dec-test!
(fn
(name got expected)
(begin
(set! mod-dec-count (+ mod-dec-count 1))
(if
(= got expected)
(set! mod-dec-pass (+ mod-dec-pass 1))
(begin
(set! mod-dec-fail (+ mod-dec-fail 1))
(append!
mod-dec-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; decide a single report (count over a 1-element registry)
(define
mod-dec-one
(fn
(reason)
(let
((r (mod/mk-report "r1" "alice" "bob" reason)))
(mod/decide-report r (list r) mod/default-rules))))
(define mod-dec-action (fn (reason) (get (mod-dec-one reason) :action)))
;; ── spam keyword → :hide ──
(mod-dec-test!
"spam keyword 'spam' → hide"
(mod-dec-action "this is spam")
"hide")
(mod-dec-test!
"spam keyword 'buy now' → hide"
(mod-dec-action "buy now while stocks last")
"hide")
(mod-dec-test!
"spam keyword case-insensitive 'CLICK HERE' → hide"
(mod-dec-action "CLICK HERE now")
"hide")
(mod-dec-test!
"spam keyword 'free money' → hide"
(mod-dec-action "win free money fast")
"hide")
;; ── abuse keyword → :remove ──
(mod-dec-test!
"abuse keyword 'harassment' → remove"
(mod-dec-action "ongoing harassment of users")
"remove")
(mod-dec-test!
"abuse keyword 'threat' → remove"
(mod-dec-action "this is a threat")
"remove")
(mod-dec-test!
"abuse keyword 'slur' → remove"
(mod-dec-action "contains a slur")
"remove")
;; ── no rule → :keep ──
(mod-dec-test!
"neutral reason → keep"
(mod-dec-action "I disagree with this post")
"keep")
(mod-dec-test! "empty reason → keep" (mod-dec-action "") "keep")
;; ── decision carries the matching rule (proof, not bare keyword) ──
(mod-dec-test!
"spam decision rule name"
(get (mod-dec-one "this is spam") :rule)
"spam-hide")
(mod-dec-test!
"keep decision rule name"
(get (mod-dec-one "fine post") :rule)
"default-keep")
(mod-dec-test!
"abuse decision rule name"
(get (mod-dec-one "harassment here") :rule)
"abuse-remove")
(mod-dec-test!
"spam proof :rule"
(get (get (mod-dec-one "spam!") :proof) :rule)
"spam-hide")
(mod-dec-test!
"spam proof :evidence"
(get (get (mod-dec-one "spam!") :proof) :evidence)
(list "spam"))
(mod-dec-test!
"spam proof :count"
(get (get (mod-dec-one "spam!") :proof) :count)
1)
;; ── classification (evidence derivation) ──
(mod-dec-test!
"classify spam"
(mod/classify-keywords (mod/mk-report "r1" "a" "b" "spam!"))
(list "spam"))
(mod-dec-test!
"classify abuse"
(mod/classify-keywords (mod/mk-report "r1" "a" "b" "abuse"))
(list "abuse"))
(mod-dec-test!
"classify neutral → empty"
(mod/classify-keywords (mod/mk-report "r1" "a" "b" "hello"))
(list))
(mod-dec-test!
"classify both spam+abuse"
(mod/classify-keywords (mod/mk-report "r1" "a" "b" "spam and abuse"))
(list "spam" "abuse"))
;; ── report-count + repeated → :escalate ──
(define
mod-dec-three
(list
(mod/mk-report "r1" "a" "bob" "x")
(mod/mk-report "r2" "c" "bob" "y")
(mod/mk-report "r3" "d" "bob" "z")))
(mod-dec-test!
"report-count counts subject"
(mod/report-count "bob" mod-dec-three)
3)
(mod-dec-test!
"3 reports about subject → escalate"
(get
(mod/decide-report (first mod-dec-three) mod-dec-three mod/default-rules)
:action)
"escalate")
(mod-dec-test!
"escalate rule name"
(get
(mod/decide-report (first mod-dec-three) mod-dec-three mod/default-rules)
:rule)
"repeated-escalate")
(define
mod-dec-two
(list
(mod/mk-report "r1" "a" "carol" "x")
(mod/mk-report "r2" "c" "carol" "y")))
(mod-dec-test!
"2 reports about subject → keep (below threshold)"
(get
(mod/decide-report (first mod-dec-two) mod-dec-two mod/default-rules)
:action)
"keep")
;; ── precedence: spam beats repeated ──
(define
mod-dec-spam-among-many
(list
(mod/mk-report "r1" "a" "dave" "buy now spam")
(mod/mk-report "r2" "c" "dave" "y")
(mod/mk-report "r3" "d" "dave" "z")))
(mod-dec-test!
"spam wins over repeated (precedence)"
(get
(mod/decide-report
(first mod-dec-spam-among-many)
mod-dec-spam-among-many
mod/default-rules)
:action)
"hide")
;; ── accessors ──
(mod-dec-test!
"report-about accessor"
(mod/report-about (mod/mk-report "r1" "a" "bob" "x"))
"bob")
(mod-dec-test!
"report-by accessor"
(mod/report-by (mod/mk-report "r1" "alice" "bob" "x"))
"alice")
;; ── api registry ──
(mod/reset!)
(define mod-dec-r1 (mod/report "alice" "bob" "this is spam"))
(define mod-dec-r2 (mod/report "carol" "eve" "fine post"))
(mod-dec-test!
"mod/report assigns sequential id r1"
(mod/report-id mod-dec-r1)
"r1")
(mod-dec-test!
"mod/report assigns sequential id r2"
(mod/report-id mod-dec-r2)
"r2")
(mod-dec-test!
"mod/decide via registry → hide"
(get (mod/decide "r1") :action)
"hide")
(mod-dec-test!
"mod/decide via registry → keep"
(get (mod/decide "r2") :action)
"keep")
(mod-dec-test! "mod/decide unknown id → nil" (mod/decide "r99") nil)
(define mod-decide-tests-run! (fn () {:failures mod-dec-failures :total mod-dec-count :passed mod-dec-pass :failed mod-dec-fail}))

View File

@@ -1,95 +0,0 @@
;; lib/mod/tests/defrule.sx — Ext 18: ergonomic defrule / ruleset.
(define mod-dr-count 0)
(define mod-dr-pass 0)
(define mod-dr-fail 0)
(define mod-dr-failures (list))
(define
mod-dr-test!
(fn
(name got expected)
(begin
(set! mod-dr-count (+ mod-dr-count 1))
(if
(= got expected)
(set! mod-dr-pass (+ mod-dr-pass 1))
(begin
(set! mod-dr-fail (+ mod-dr-fail 1))
(append!
mod-dr-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── defrule produces the same structure as mk-rule ──
(define
mod-dr-r
(mod/defrule "spam-hide" :hide (list :classification "spam")))
(mod-dr-test! "defrule name" (mod/rule-name mod-dr-r) "spam-hide")
(mod-dr-test! "defrule action" (mod/rule-action mod-dr-r) "hide")
(mod-dr-test!
"defrule when wraps the conditions"
(mod/rule-when mod-dr-r)
(list (list :classification "spam")))
(mod-dr-test!
"defrule equals mk-rule equivalent"
(mod/rule-when mod-dr-r)
(mod/rule-when
(mod/mk-rule "spam-hide" :hide (list (list :classification "spam")))))
;; ── multi-condition + no-condition ──
(define
mod-dr-multi
(mod/defrule
"strict"
:hide (list :classification "spam")
(list :not (list :attr "verified"))))
(mod-dr-test!
"defrule collects multiple conditions"
(len (mod/rule-when mod-dr-multi))
2)
(define mod-dr-catch (mod/defrule "default-keep" :keep))
(mod-dr-test!
"defrule with no conditions is unconditional"
(mod/rule-when mod-dr-catch)
(list))
;; ── ruleset assembles a list ──
(define
mod-dr-rules
(mod/ruleset
(mod/defrule "spam-hide" :hide (list :classification "spam"))
(mod/defrule "default-keep" :keep)))
(mod-dr-test! "ruleset length" (len mod-dr-rules) 2)
(mod-dr-test!
"ruleset first rule name"
(mod/rule-name (first mod-dr-rules))
"spam-hide")
;; ── engine works with defrule/ruleset-built policy ──
(define mod-dr-spam (mod/mk-report "r1" "a" "b" "this is spam"))
(define mod-dr-clean (mod/mk-report "r2" "a" "b" "a fine post"))
(mod-dr-test!
"defrule policy: spam → hide"
(get
(mod/decide-report mod-dr-spam (list mod-dr-spam) mod-dr-rules)
:action)
"hide")
(mod-dr-test!
"defrule policy: clean → keep"
(get
(mod/decide-report mod-dr-clean (list mod-dr-clean) mod-dr-rules)
:action)
"keep")
(mod-dr-test!
"defrule policy: spam names the rule"
(get (mod/decide-report mod-dr-spam (list mod-dr-spam) mod-dr-rules) :rule)
"spam-hide")
(define mod-defrule-tests-run! (fn () {:failures mod-dr-failures :total mod-dr-count :passed mod-dr-pass :failed mod-dr-fail}))

View File

@@ -1,145 +0,0 @@
;; lib/mod/tests/disjunction.sx — Ext 15: disjunctive (:any) conditions.
(define mod-or-count 0)
(define mod-or-pass 0)
(define mod-or-fail 0)
(define mod-or-failures (list))
(define
mod-or-test!
(fn
(name got expected)
(begin
(set! mod-or-count (+ mod-or-count 1))
(if
(= got expected)
(set! mod-or-pass (+ mod-or-pass 1))
(begin
(set! mod-or-fail (+ mod-or-fail 1))
(append!
mod-or-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; one rule, OR of two classifications → one action covers both
(define
mod-or-rules
(list
(mod/mk-rule
"spam-or-abuse-hide"
:hide (list
(list
:any (list (list :classification "spam") (list :classification "abuse")))))
(mod/mk-rule "default-keep" :keep (list))))
(define mod-or-spam (mod/mk-report "r1" "a" "b" "this is spam"))
(define mod-or-abuse (mod/mk-report "r2" "a" "b" "harassment here"))
(define mod-or-clean (mod/mk-report "r3" "a" "b" "a fine post"))
(mod-or-test!
"OR: spam branch → hide"
(get
(mod/decide-report mod-or-spam (list mod-or-spam) mod-or-rules)
:action)
"hide")
(mod-or-test!
"OR: abuse branch → hide"
(get
(mod/decide-report mod-or-abuse (list mod-or-abuse) mod-or-rules)
:action)
"hide")
(mod-or-test!
"OR: neither branch → keep"
(get
(mod/decide-report mod-or-clean (list mod-or-clean) mod-or-rules)
:action)
"keep")
;; ── goal text + proof ──
(mod-or-test!
"cond->goal :any joins with ;"
(mod/cond->goal
(list
:any (list (list :classification "spam") (list :classification "abuse")))
"Id")
"(classification(Id, spam) ; classification(Id, abuse))")
(define
mod-or-dec
(mod/decide-report mod-or-spam (list mod-or-spam) mod-or-rules))
(mod-or-test!
"OR proof goal solved"
(get (first (get (get mod-or-dec :proof) :goals)) :solved)
true)
(mod-or-test!
"OR proof goal text"
(get (first (get (get mod-or-dec :proof) :goals)) :goal)
"(classification(r1, spam) ; classification(r1, abuse))")
;; ── :any composes with :not (NOR-ish) and :attr ──
(define
mod-or-mixed-rules
(list
(mod/mk-rule
"spam-or-flagged-hide"
:hide (list
(list
:any (list (list :classification "spam") (list :attr "flagged")))))
(mod/mk-rule "default-keep" :keep (list))))
(define
mod-or-flagged
(mod/attach-attr (mod/mk-report "r4" "a" "b" "a fine post") "flagged"))
(mod-or-test!
"OR over classification|attr: flagged clean post → hide"
(get
(mod/decide-report
mod-or-flagged
(list mod-or-flagged)
mod-or-mixed-rules)
:action)
"hide")
(mod-or-test!
"cond->goal :any with :not branch"
(mod/cond->goal
(list
:any (list
(list :classification "spam")
(list :not (list :attr "verified"))))
"Id")
"(classification(Id, spam) ; not(attr(Id, verified)))")
;; AND still works alongside OR in the same :when list
(define
mod-or-and-rules
(list
(mod/mk-rule
"spam-and-not-verified"
:hide (list
(list
:any (list (list :classification "spam") (list :classification "abuse")))
(list :not (list :attr "verified"))))
(mod/mk-rule "default-keep" :keep (list))))
(define
mod-or-spam-verified
(mod/attach-attr (mod/mk-report "r5" "a" "b" "this is spam") "verified"))
(mod-or-test!
"AND of OR + NOT: verified spam → keep"
(get
(mod/decide-report
mod-or-spam-verified
(list mod-or-spam-verified)
mod-or-and-rules)
:action)
"keep")
(mod-or-test!
"AND of OR + NOT: unverified abuse → hide"
(get
(mod/decide-report mod-or-abuse (list mod-or-abuse) mod-or-and-rules)
:action)
"hide")
(define mod-disjunction-tests-run! (fn () {:failures mod-or-failures :total mod-or-count :passed mod-or-pass :failed mod-or-fail}))

View File

@@ -1,279 +0,0 @@
;; lib/mod/tests/escalation.sx — Phase 3: lifecycle state machine + escalation.
(define mod-esc-count 0)
(define mod-esc-pass 0)
(define mod-esc-fail 0)
(define mod-esc-failures (list))
(define
mod-esc-test!
(fn
(name got expected)
(begin
(set! mod-esc-count (+ mod-esc-count 1))
(if
(= got expected)
(set! mod-esc-pass (+ mod-esc-pass 1))
(begin
(set! mod-esc-fail (+ mod-esc-fail 1))
(append!
mod-esc-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── transition table guard ──
(mod-esc-test!
"open → triaged allowed"
(mod/lc-can-transition? "open" "triaged")
true)
(mod-esc-test!
"triaged → decided allowed"
(mod/lc-can-transition? "triaged" "decided")
true)
(mod-esc-test!
"decided → appealed allowed"
(mod/lc-can-transition? "decided" "appealed")
true)
(mod-esc-test!
"appealed → final allowed"
(mod/lc-can-transition? "appealed" "final")
true)
(mod-esc-test!
"open → decided rejected"
(mod/lc-can-transition? "open" "decided")
false)
(mod-esc-test!
"triaged → final rejected"
(mod/lc-can-transition? "triaged" "final")
false)
(mod-esc-test!
"final is terminal"
(mod/lc-can-transition? "final" "open")
false)
;; ── initial state ──
(define
mod-esc-c0
(mod/mk-case (mod/mk-report "r1" "alice" "bob" "this is spam")))
(mod-esc-test! "new case is open" (mod/case-state mod-esc-c0) "open")
(mod-esc-test! "new case has no decision" (mod/case-decision mod-esc-c0) nil)
;; ── auto-tier: spam triages + resolves to decided/hide ──
(define
mod-esc-spam-rep
(list (mod/mk-report "r1" "alice" "bob" "this is spam")))
(define
mod-esc-t1
(mod/case-triage mod-esc-c0 mod-esc-spam-rep mod/default-rules))
(mod-esc-test! "spam triaged" (mod/case-state mod-esc-t1) "triaged")
(mod-esc-test! "spam triage tier auto" (mod/case-tier mod-esc-t1) "auto")
(mod-esc-test! "spam triage action hide" (mod/case-action mod-esc-t1) "hide")
(define mod-esc-r1 (mod/case-resolve mod-esc-t1))
(mod-esc-test!
"auto resolve → decided"
(mod/case-state mod-esc-r1)
"decided")
(mod-esc-test!
"decision preserved through resolve"
(mod/case-action mod-esc-r1)
"hide")
;; ── illegal transition flags :error, leaves state ──
(define mod-esc-bad (mod/case-finalize mod-esc-c0))
(mod-esc-test!
"finalize from open is illegal"
(mod/case-state mod-esc-bad)
"open")
(mod-esc-test!
"illegal transition sets error"
(nil? (mod/case-error mod-esc-bad))
false)
;; ── human-tier: repeated report escalates, resolve blocked, review decides ──
(define mod-esc-rep-r (mod/mk-report "r3" "ann" "dave" "off-topic"))
(define mod-esc-rep-reports (list mod-esc-rep-r mod-esc-rep-r mod-esc-rep-r))
(define mod-esc-rep-c0 (mod/mk-case mod-esc-rep-r))
(define
mod-esc-rep-t
(mod/case-triage mod-esc-rep-c0 mod-esc-rep-reports mod/default-rules))
(mod-esc-test!
"repeated triage action escalate"
(mod/case-action mod-esc-rep-t)
"escalate")
(mod-esc-test!
"repeated triage tier human"
(mod/case-tier mod-esc-rep-t)
"human")
(mod-esc-test!
"repeated still triaged after triage"
(mod/case-state mod-esc-rep-t)
"triaged")
(define mod-esc-rep-block (mod/case-resolve mod-esc-rep-t))
(mod-esc-test!
"auto-resolve blocked on human tier (state unchanged)"
(mod/case-state mod-esc-rep-block)
"triaged")
(mod-esc-test!
"blocked resolve sets error"
(nil? (mod/case-error mod-esc-rep-block))
false)
(define
mod-esc-rep-rev
(mod/case-review
mod-esc-rep-t
"confirmed-abuse"
"human"
mod-esc-rep-reports
mod/default-rules))
(mod-esc-test!
"human review → decided"
(mod/case-state mod-esc-rep-rev)
"decided")
(mod-esc-test!
"human review action remove"
(mod/case-action mod-esc-rep-rev)
"remove")
(mod-esc-test!
"review attached evidence to report"
(len (mod/report-evidence (mod/case-report mod-esc-rep-rev)))
1)
(define mod-esc-rep-final (mod/case-finalize mod-esc-rep-rev))
(mod-esc-test!
"review case finalizes"
(mod/case-state mod-esc-rep-final)
"final")
;; ── appeal overrides a prior decision ──
(define
mod-esc-ap-c0
(mod/mk-case (mod/mk-report "r5" "u" "v" "buy now spam")))
(define mod-esc-ap-rep (list (mod/mk-report "r5" "u" "v" "buy now spam")))
(define
mod-esc-ap-t
(mod/case-triage mod-esc-ap-c0 mod-esc-ap-rep mod/default-rules))
(define mod-esc-ap-d (mod/case-resolve mod-esc-ap-t))
(mod-esc-test!
"appeal precondition decided/hide"
(mod/case-action mod-esc-ap-d)
"hide")
(define
mod-esc-ap-appealed
(mod/case-appeal
mod-esc-ap-d
"exonerated"
"moderator"
mod-esc-ap-rep
mod/default-rules))
(mod-esc-test!
"appeal → appealed state"
(mod/case-state mod-esc-ap-appealed)
"appealed")
(mod-esc-test!
"appeal overrides hide → keep"
(mod/case-action mod-esc-ap-appealed)
"keep")
(mod-esc-test!
"appeal recorded via exonerated-keep rule"
(get (mod/case-decision mod-esc-ap-appealed) :rule)
"exonerated-keep")
(define mod-esc-ap-final (mod/case-finalize mod-esc-ap-appealed))
(mod-esc-test! "appealed → final" (mod/case-state mod-esc-ap-final) "final")
;; ── history records the full traversal ──
(mod-esc-test!
"full lifecycle history length 4 (triage,resolve,appeal,finalize)"
(len (mod/case-history mod-esc-ap-final))
4)
(mod-esc-test!
"first history step open→triaged"
(get (first (mod/case-history mod-esc-ap-final)) :to)
"triaged")
(mod-esc-test!
"last history step → final"
(get (nth (mod/case-history mod-esc-ap-final) 3) :to)
"final")
;; ── api-level lifecycle façade ──
(mod/reset!)
(mod/report "alice" "bob" "this is spam")
(mod/report "carol" "dave" "off-topic")
(mod/report "carol" "dave" "off-topic")
(mod/report "carol" "dave" "off-topic")
(mod-esc-test!
"api: case opens at open"
(mod/case-state (mod/case-of "r1"))
"open")
(define mod-esc-api-t1 (mod/triage "r1"))
(mod-esc-test!
"api: triage spam → triaged"
(mod/case-state mod-esc-api-t1)
"triaged")
(mod-esc-test!
"api: triage spam action hide"
(mod/case-action mod-esc-api-t1)
"hide")
(define mod-esc-api-r1 (mod/resolve "r1"))
(mod-esc-test!
"api: resolve → decided"
(mod/case-state mod-esc-api-r1)
"decided")
(mod-esc-test!
"api: resolve logged decision"
(len (mod/audit "r1"))
1)
(define mod-esc-api-app (mod/appeal "r1" "exonerated" "mod"))
(mod-esc-test!
"api: appeal → appealed"
(mod/case-state mod-esc-api-app)
"appealed")
(mod-esc-test!
"api: appeal overrides → keep"
(mod/case-action mod-esc-api-app)
"keep")
(mod-esc-test!
"api: appeal logged second decision"
(len (mod/audit "r1"))
2)
(mod-esc-test!
"api: finalize → final"
(mod/case-state (mod/finalize "r1"))
"final")
;; r4 is the 3rd report about dave → escalates via the human tier
(define mod-esc-api-t4 (mod/triage "r4"))
(mod-esc-test!
"api: repeated triage escalates (human tier)"
(mod/case-tier mod-esc-api-t4)
"human")
(define mod-esc-api-blk (mod/resolve "r4"))
(mod-esc-test!
"api: escalated resolve blocked"
(mod/case-state mod-esc-api-blk)
"triaged")
(define mod-esc-api-rev (mod/review "r4" "confirmed-abuse" "human"))
(mod-esc-test!
"api: review → decided/remove"
(mod/case-action mod-esc-api-rev)
"remove")
(mod-esc-test! "api: unknown id → nil" (mod/triage "r99") nil)
(define mod-escalation-tests-run! (fn () {:failures mod-esc-failures :total mod-esc-count :passed mod-esc-pass :failed mod-esc-fail}))

View File

@@ -1,313 +0,0 @@
;; lib/mod/tests/extensions.sx — beyond-roadmap extensions.
;;
;; Ext 1: negation-as-failure conditions (:not / :attr) + report attributes.
;; "hide spam UNLESS the author is verified" (closed-world reasoning).
;; Ext 2: weighted/aggregate evidence scoring (:score-at-least) + report signals.
;; Many low-confidence signals accumulate past a threshold via Prolog
;; aggregate_all(sum(W), ...).
;; Ext 3: human-readable proof explanation (mod/explain) over the proof tree.
;; Demonstrated with custom rule sets so the default policy (and its conformance
;; tests) stays untouched.
(define mod-ext-count 0)
(define mod-ext-pass 0)
(define mod-ext-fail 0)
(define mod-ext-failures (list))
(define
mod-ext-test!
(fn
(name got expected)
(begin
(set! mod-ext-count (+ mod-ext-count 1))
(if
(= got expected)
(set! mod-ext-pass (+ mod-ext-pass 1))
(begin
(set! mod-ext-fail (+ mod-ext-fail 1))
(append!
mod-ext-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── Ext 1: report attributes ──
(define mod-ext-r0 (mod/mk-report "r1" "a" "b" "this is spam"))
(mod-ext-test!
"fresh report has no attrs"
(len (mod/report-attrs mod-ext-r0))
0)
(define mod-ext-rv (mod/attach-attr mod-ext-r0 "verified"))
(mod-ext-test!
"attach-attr adds one attr"
(len (mod/report-attrs mod-ext-rv))
1)
(mod-ext-test!
"attach-attr preserves evidence field"
(len
(mod/report-evidence
(mod/attach-evidence mod-ext-rv (mod/mk-evidence "x" "y"))))
1)
(mod-ext-test!
"attach-evidence preserves attrs"
(len
(mod/report-attrs
(mod/attach-evidence mod-ext-rv (mod/mk-evidence "x" "y"))))
1)
;; ── Ext 1: negation-as-failure: spam hidden unless author verified ──
(define
mod-ext-rules
(list
(mod/mk-rule
"spam-unverified-hide"
:hide (list
(list :classification "spam")
(list :not (list :attr "verified"))))
(mod/mk-rule "default-keep" :keep (list))))
(define mod-ext-spam-plain (mod/mk-report "p1" "a" "b" "this is spam"))
(define
mod-ext-spam-verified
(mod/attach-attr (mod/mk-report "p2" "a" "b" "this is spam") "verified"))
(define mod-ext-clean (mod/mk-report "p3" "a" "b" "a fine post"))
(mod-ext-test!
"unverified spam → hide"
(get
(mod/decide-report
mod-ext-spam-plain
(list mod-ext-spam-plain)
mod-ext-rules)
:action)
"hide")
(mod-ext-test!
"verified author spam → keep (negation blocks)"
(get
(mod/decide-report
mod-ext-spam-verified
(list mod-ext-spam-verified)
mod-ext-rules)
:action)
"keep")
(mod-ext-test!
"clean post → keep"
(get
(mod/decide-report mod-ext-clean (list mod-ext-clean) mod-ext-rules)
:action)
"keep")
;; ── Ext 1: negation appears in the goal text + proof ──
(define
mod-ext-dec
(mod/decide-report
mod-ext-spam-plain
(list mod-ext-spam-plain)
mod-ext-rules))
(define mod-ext-goals (get (get mod-ext-dec :proof) :goals))
(mod-ext-test!
"rule that matched is spam-unverified-hide"
(get mod-ext-dec :rule)
"spam-unverified-hide")
(mod-ext-test! "proof has two goals" (len mod-ext-goals) 2)
(mod-ext-test!
"negation goal text"
(get (nth mod-ext-goals 1) :goal)
"not(attr(p1, verified))")
(mod-ext-test!
"negation goal solved for unverified"
(get (nth mod-ext-goals 1) :solved)
true)
;; ── Ext 1: cond->goal compiles :attr and :not directly ──
(mod-ext-test!
"cond->goal :attr"
(mod/cond->goal (list :attr "verified") "Id")
"attr(Id, verified)")
(mod-ext-test!
"cond->goal :not wraps inner"
(mod/cond->goal (list :not (list :classification "spam")) "Id")
"not(classification(Id, spam))")
;; ── Ext 1: positive :attr condition (allowlist-style) ──
(define
mod-ext-allow-rules
(list
(mod/mk-rule "trusted-keep" :keep (list (list :attr "trusted")))
(mod/mk-rule "spam-hide" :hide (list (list :classification "spam")))
(mod/mk-rule "default-keep" :keep (list))))
(define
mod-ext-trusted-spam
(mod/attach-attr (mod/mk-report "t1" "a" "b" "this is spam") "trusted"))
(mod-ext-test!
"trusted attr exempts spam → keep"
(get
(mod/decide-report
mod-ext-trusted-spam
(list mod-ext-trusted-spam)
mod-ext-allow-rules)
:action)
"keep")
;; ── Ext 2: weighted signals + aggregate scoring ──
(define mod-ext-s0 (mod/mk-report "s1" "a" "b" "neutral"))
(mod-ext-test!
"fresh report has no signals"
(len (mod/report-signals mod-ext-s0))
0)
(define
mod-ext-s1
(mod/attach-signal mod-ext-s0 (mod/mk-signal "link" 2)))
(mod-ext-test!
"attach-signal adds one"
(len (mod/report-signals mod-ext-s1))
1)
(mod-ext-test!
"attach-signal preserves attrs"
(len
(mod/report-attrs
(mod/attach-signal mod-ext-rv (mod/mk-signal "x" 1))))
1)
(define
mod-ext-score-rules
(list
(mod/mk-rule
"high-score-hide"
:hide (list (list :score-at-least 5)))
(mod/mk-rule "default-keep" :keep (list))))
;; one weak signal (2) — below threshold
(define
mod-ext-weak
(mod/attach-signal
(mod/mk-report "w1" "a" "b" "neutral")
(mod/mk-signal "link" 2)))
(mod-ext-test!
"single weak signal → keep (below threshold)"
(get
(mod/decide-report mod-ext-weak (list mod-ext-weak) mod-ext-score-rules)
:action)
"keep")
;; three signals summing to 6 — over threshold
(define
mod-ext-strong0
(mod/attach-signal
(mod/mk-report "w2" "a" "b" "neutral")
(mod/mk-signal "link" 2)))
(define
mod-ext-strong1
(mod/attach-signal mod-ext-strong0 (mod/mk-signal "newaccount" 2)))
(define
mod-ext-strong
(mod/attach-signal mod-ext-strong1 (mod/mk-signal "burst" 2)))
(mod-ext-test!
"accumulated signals (2+2+2=6) → hide"
(get
(mod/decide-report
mod-ext-strong
(list mod-ext-strong)
mod-ext-score-rules)
:action)
"hide")
(mod-ext-test!
"scoring rule named in decision"
(get
(mod/decide-report
mod-ext-strong
(list mod-ext-strong)
mod-ext-score-rules)
:rule)
"high-score-hide")
;; exactly at threshold (5) fires
(define
mod-ext-exact0
(mod/attach-signal
(mod/mk-report "w3" "a" "b" "neutral")
(mod/mk-signal "link" 3)))
(define
mod-ext-exact
(mod/attach-signal mod-ext-exact0 (mod/mk-signal "burst" 2)))
(mod-ext-test!
"exactly at threshold (5) → hide"
(get
(mod/decide-report mod-ext-exact (list mod-ext-exact) mod-ext-score-rules)
:action)
"hide")
(mod-ext-test!
"cond->goal :score-at-least"
(mod/cond->goal (list :score-at-least 5) "Id")
"aggregate_all(sum(W), signal(Id, _, W), T), T >= 5")
;; ── Ext 3: human-readable proof explanation ──
(define mod-ext-spam-explain (mod/explain mod-ext-dec))
(mod-ext-test!
"explain mentions the report id"
(mod/str-contains? mod-ext-spam-explain "Report p1")
true)
(mod-ext-test!
"explain mentions the action"
(mod/str-contains? mod-ext-spam-explain "hide")
true)
(mod-ext-test!
"explain mentions the rule"
(mod/str-contains? mod-ext-spam-explain "spam-unverified-hide")
true)
(mod-ext-test!
"explain marks proved goals"
(mod/str-contains? mod-ext-spam-explain "[proved]")
true)
(mod-ext-test!
"explain renders the evidence line"
(mod/str-contains? mod-ext-spam-explain "Evidence: spam")
true)
;; count-rule explanation shows the unification bindings
(define mod-ext-rep-r (mod/mk-report "rc" "ann" "dave" "off-topic"))
(define
mod-ext-rep-d
(mod/decide-report
mod-ext-rep-r
(list mod-ext-rep-r mod-ext-rep-r mod-ext-rep-r)
mod/default-rules))
(define mod-ext-rep-explain (mod/explain mod-ext-rep-d))
(mod-ext-test!
"explain shows binding N=3"
(mod/str-contains? mod-ext-rep-explain "N=3")
true)
(mod-ext-test!
"explain shows subject binding"
(mod/str-contains? mod-ext-rep-explain "dave")
true)
;; explain-goal direct: unproved goal gets [unproved]
(mod-ext-test!
"explain-goal marks unproved"
(mod/str-contains? (mod/explain-goal {:solved false :goal "attr(x, foo)" :bindings {}}) "[unproved]")
true)
;; explain-binds renders key=value pairs
(mod-ext-test!
"explain-binds renders pair"
(mod/explain-binds {:N "3"})
"N=3")
;; no-evidence decision says (none)
(define
mod-ext-keep-d
(mod/decide-report mod-ext-clean (list mod-ext-clean) mod-ext-rules))
(mod-ext-test!
"explain (none) for empty evidence"
(mod/str-contains? (mod/explain mod-ext-keep-d) "Evidence: (none)")
true)
(define mod-extensions-tests-run! (fn () {:failures mod-ext-failures :total mod-ext-count :passed mod-ext-pass :failed mod-ext-fail}))

View File

@@ -1,154 +0,0 @@
;; lib/mod/tests/fed.sx — Phase 4: federation (mock fed-sx).
(define mod-fed-count 0)
(define mod-fed-pass 0)
(define mod-fed-fail 0)
(define mod-fed-failures (list))
(define
mod-fed-test!
(fn
(name got expected)
(begin
(set! mod-fed-count (+ mod-fed-count 1))
(if
(= got expected)
(set! mod-fed-pass (+ mod-fed-pass 1))
(begin
(set! mod-fed-fail (+ mod-fed-fail 1))
(append!
mod-fed-failures
(str name "\n expected: " expected "\n got: " got)))))))
(mod/reset!)
(mod/fed-reset!)
;; ── trust model (advisory by default) ──
(mod-fed-test! "trust initially false" (mod/trusted? "peerA" :mod) false)
(mod/grant-trust "peerA" :mod)
(mod-fed-test! "trust after grant" (mod/trusted? "peerA" :mod) true)
(mod-fed-test! "trust wrong scope" (mod/trusted? "peerA" :other) false)
(mod-fed-test! "trust other peer" (mod/trusted? "peerB" :mod) false)
(mod/revoke-trust "peerA" :mod)
(mod-fed-test! "trust after revoke" (mod/trusted? "peerA" :mod) false)
;; ── cross-instance reports ──
(define
mod-fed-fr
(mod/fed-receive-report "peerB" "alice" "bob" "this is spam"))
(mod-fed-test! "fed report assigned id r1" (mod/report-id mod-fed-fr) "r1")
(mod-fed-test! "fed report origin is peer" (mod/report-origin "r1") "peerB")
(define mod-fed-local (mod/report "carol" "dave" "fine post"))
(mod-fed-test!
"local report origin is local"
(mod/report-origin (mod/report-id mod-fed-local))
"local")
(mod-fed-test!
"engine decides fed report (spam → hide)"
(get
(mod/decide-report mod-fed-fr (list mod-fed-fr) mod/default-rules)
:action)
"hide")
;; ── decision sharing (outbox) ──
(define mod-fed-dec {:action "hide" :rule "spam-hide" :report-id "r1"})
(define
mod-fed-shared
(mod/fed-share-decision mod-fed-dec (list "peerB" "peerC")))
(mod-fed-test! "share returns notified peers" (len mod-fed-shared) 2)
(mod-fed-test! "outbox has two messages" (len (mod/fed-outbox)) 2)
(mod-fed-test!
"outbox message type decision"
(get (first (mod/fed-outbox)) :type)
"decision")
(mod-fed-test!
"outbox message addressed to peer"
(get (first (mod/fed-outbox)) :to)
"peerB")
;; ── receiving a peer decision: advisory unless trusted ──
(define mod-fed-untrusted (mod/fed-receive-decision "peerZ" {:action "remove" :rule "reviewer-remove" :report-id "rx"}))
(mod-fed-test!
"untrusted decision not applied"
(get mod-fed-untrusted :applied)
false)
(mod-fed-test!
"untrusted decision advisory"
(get mod-fed-untrusted :advisory)
true)
(mod-fed-test!
"untrusted decision absent from applied log"
(mod/fed-applied-action "rx")
nil)
(mod-fed-test!
"advisory log records suggestion"
(len mod/*fed-advisory*)
1)
(mod/grant-trust "peerT" :mod)
(define mod-fed-trusted (mod/fed-receive-decision "peerT" {:action "hide" :rule "spam-hide" :report-id "ry"}))
(mod-fed-test! "trusted decision applied" (get mod-fed-trusted :applied) true)
(mod-fed-test!
"trusted decision binds locally"
(get (mod/fed-applied-action "ry") :action)
"hide")
;; ── revocation ──
(mod-fed-test!
"applied action not yet revoked"
(get (mod/fed-applied-action "ry") :revoked)
false)
(mod/fed-revoke! "ry" "manual")
(mod-fed-test!
"revoke marks applied action revoked"
(get (mod/fed-applied-action "ry") :revoked)
true)
(mod-fed-test!
"revoke emits a revocation message"
(mod/any? (fn (m) (= (get m :type) "revocation")) (mod/fed-outbox))
true)
;; revoke-if-invalidated: proof still holds → no revocation
(define mod-fed-spam-r (mod/mk-report "rs" "a" "b" "this is spam"))
(define
mod-fed-spam-d
(mod/decide-report mod-fed-spam-r (list mod-fed-spam-r) mod/default-rules))
(mod-fed-test! "spam decision is hide" (get mod-fed-spam-d :action) "hide")
(define
mod-fed-rev-same
(mod/fed-revoke-if-invalidated
mod-fed-spam-r
mod-fed-spam-d
(list mod-fed-spam-r)
mod/default-rules))
(mod-fed-test!
"valid proof → not revoked"
(get mod-fed-rev-same :revoked)
false)
;; exoneration invalidates the proof → revocation
(define
mod-fed-exon-r
(mod/attach-evidence mod-fed-spam-r (mod/mk-evidence "exonerated" "mod")))
(define
mod-fed-rev-inv
(mod/fed-revoke-if-invalidated
mod-fed-exon-r
mod-fed-spam-d
(list mod-fed-exon-r)
mod/default-rules))
(mod-fed-test!
"invalidated proof → revoked"
(get mod-fed-rev-inv :revoked)
true)
(mod-fed-test!
"re-decision after exoneration is keep"
(get (get mod-fed-rev-inv :decision) :action)
"keep")
(define mod-fed-tests-run! (fn () {:failures mod-fed-failures :total mod-fed-count :passed mod-fed-pass :failed mod-fed-fail}))

View File

@@ -1,86 +0,0 @@
;; lib/mod/tests/link.sx — Ext 4: report linking + dedup.
(define mod-lnk-count 0)
(define mod-lnk-pass 0)
(define mod-lnk-fail 0)
(define mod-lnk-failures (list))
(define
mod-lnk-test!
(fn
(name got expected)
(begin
(set! mod-lnk-count (+ mod-lnk-count 1))
(if
(= got expected)
(set! mod-lnk-pass (+ mod-lnk-pass 1))
(begin
(set! mod-lnk-fail (+ mod-lnk-fail 1))
(append!
mod-lnk-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── link-key + dedup ──
(define mod-lnk-a (mod/mk-report "r1" "alice" "bob" "this is spam"))
(define mod-lnk-a2 (mod/mk-report "r2" "alice" "bob" "THIS IS SPAM"))
(define mod-lnk-b (mod/mk-report "r3" "carol" "bob" "abuse"))
(define mod-lnk-c (mod/mk-report "r4" "alice" "eve" "this is spam"))
(mod-lnk-test!
"identical reports share a link key (case-insensitive reason)"
(= (mod/link-key mod-lnk-a) (mod/link-key mod-lnk-a2))
true)
(mod-lnk-test!
"different reporter → different key"
(= (mod/link-key mod-lnk-a) (mod/link-key mod-lnk-b))
false)
(mod-lnk-test!
"different subject → different key"
(= (mod/link-key mod-lnk-a) (mod/link-key mod-lnk-c))
false)
(define mod-lnk-set (list mod-lnk-a mod-lnk-a2 mod-lnk-b mod-lnk-c))
(mod-lnk-test!
"dedup collapses identical reports"
(len (mod/dedup-reports mod-lnk-set))
3)
(mod-lnk-test!
"duplicate-count counts collapsed"
(mod/duplicate-count mod-lnk-set)
1)
(mod-lnk-test!
"dedup of all-distinct keeps all"
(len (mod/dedup-reports (list mod-lnk-a mod-lnk-b mod-lnk-c)))
3)
;; ── Prolog-backed relational linking ──
(mod-lnk-test!
"related-ids finds all reports about subject"
(len (mod/related-ids "bob" mod-lnk-set))
3)
(mod-lnk-test!
"related-ids returns the ids"
(mod/related-ids "eve" mod-lnk-set)
(list "r4"))
(mod-lnk-test!
"related-ids empty for unknown subject"
(mod/related-ids "nobody" mod-lnk-set)
(list))
;; reporters: bob reported by alice (x2) + carol → 3 raw, 2 distinct
(mod-lnk-test!
"reporters-of counts all reports"
(len (mod/reporters-of "bob" mod-lnk-set))
3)
(mod-lnk-test!
"distinct reporters-of dedups reporters"
(len (mod/distinct-reporters-of "bob" mod-lnk-set))
2)
(mod-lnk-test!
"distinct utility removes dups"
(mod/distinct (list "a" "b" "a" "c" "b"))
(list "a" "b" "c"))
(define mod-link-tests-run! (fn () {:failures mod-lnk-failures :total mod-lnk-count :passed mod-lnk-pass :failed mod-lnk-fail}))

View File

@@ -1,122 +0,0 @@
;; lib/mod/tests/lint.sx — Ext 5: policy rule-set static analysis.
(define mod-lint-count 0)
(define mod-lint-pass 0)
(define mod-lint-fail 0)
(define mod-lint-failures (list))
(define
mod-lint-test!
(fn
(name got expected)
(begin
(set! mod-lint-count (+ mod-lint-count 1))
(if
(= got expected)
(set! mod-lint-pass (+ mod-lint-pass 1))
(begin
(set! mod-lint-fail (+ mod-lint-fail 1))
(append!
mod-lint-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── the default rule set is well-formed ──
(mod-lint-test!
"default rules: no unreachable"
(mod/unreachable-rules mod/default-rules)
(list))
(mod-lint-test!
"default rules: has catch-all"
(mod/has-catchall? mod/default-rules)
true)
(mod-lint-test!
"default rules: no duplicate names"
(mod/duplicate-rule-names mod/default-rules)
(list))
(mod-lint-test!
"default rules: well-formed"
(mod/rules-ok? mod/default-rules)
true)
;; ── unreachable detection ──
(define
mod-lint-shadowed
(list
(mod/mk-rule "spam-hide" :hide (list (list :classification "spam")))
(mod/mk-rule "catch-all" :keep (list))
(mod/mk-rule
"abuse-remove"
:remove (list (list :classification "abuse")))
(mod/mk-rule
"repeated"
:escalate (list (list :count-at-least 3)))))
(mod-lint-test!
"rules after catch-all are unreachable"
(mod/unreachable-rules mod-lint-shadowed)
(list "abuse-remove" "repeated"))
(mod-lint-test!
"shadowed rule set is not ok"
(mod/rules-ok? mod-lint-shadowed)
false)
;; ── missing catch-all ──
(define
mod-lint-nocatch
(list
(mod/mk-rule "spam-hide" :hide (list (list :classification "spam")))
(mod/mk-rule
"abuse-remove"
:remove (list (list :classification "abuse")))))
(mod-lint-test!
"no catch-all detected"
(mod/has-catchall? mod-lint-nocatch)
false)
(mod-lint-test!
"no unreachable when no catch-all"
(mod/unreachable-rules mod-lint-nocatch)
(list))
(mod-lint-test!
"no-catch-all rule set is not ok"
(mod/rules-ok? mod-lint-nocatch)
false)
;; ── duplicate names ──
(define
mod-lint-dups
(list
(mod/mk-rule "x" :hide (list (list :classification "spam")))
(mod/mk-rule "x" :remove (list (list :classification "abuse")))
(mod/mk-rule "default" :keep (list))))
(mod-lint-test!
"duplicate names detected"
(mod/duplicate-rule-names mod-lint-dups)
(list "x"))
(mod-lint-test!
"duplicate-name rule set is not ok"
(mod/rules-ok? mod-lint-dups)
false)
;; ── helpers ──
(mod-lint-test!
"rule-unconditional? true for empty when"
(mod/rule-unconditional? (mod/mk-rule "d" :keep (list)))
true)
(mod-lint-test!
"rule-unconditional? false with conditions"
(mod/rule-unconditional?
(mod/mk-rule "s" :hide (list (list :classification "spam"))))
false)
(mod-lint-test!
"count-eq counts occurrences"
(mod/count-eq "a" (list "a" "b" "a"))
2)
(define mod-lint-tests-run! (fn () {:failures mod-lint-failures :total mod-lint-count :passed mod-lint-pass :failed mod-lint-fail}))

View File

@@ -1,115 +0,0 @@
;; lib/mod/tests/offenders.sx — Ext 7: repeat-offender escalation.
(define mod-off-count 0)
(define mod-off-pass 0)
(define mod-off-fail 0)
(define mod-off-failures (list))
(define
mod-off-test!
(fn
(name got expected)
(begin
(set! mod-off-count (+ mod-off-count 1))
(if
(= got expected)
(set! mod-off-pass (+ mod-off-pass 1))
(begin
(set! mod-off-fail (+ mod-off-fail 1))
(append!
mod-off-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── sanction? predicate ──
(mod-off-test! "hide is a sanction" (mod/sanction? "hide") true)
(mod-off-test! "remove is a sanction" (mod/sanction? "remove") true)
(mod-off-test! "ban is a sanction" (mod/sanction? "ban") true)
(mod-off-test! "keep is not a sanction" (mod/sanction? "keep") false)
(mod-off-test! "escalate is not a sanction" (mod/sanction? "escalate") false)
;; ── repeat-offender escalation over the audit log ──
(mod/reset!)
(mod/report "u1" "spammer" "this is spam")
(mod/report "u2" "spammer" "buy now offer")
(mod/report "u3" "spammer" "click here free money")
(mod/report "u4" "innocent" "fine post")
(mod-off-test!
"no sanctions before any decision"
(mod/subject-sanctions "spammer")
0)
(define mod-off-d1 (mod/decide-escalating "r1" 2))
(mod-off-test!
"first spam → hide (0 priors)"
(get mod-off-d1 :action)
"hide")
(mod-off-test!
"one sanction recorded"
(mod/subject-sanctions "spammer")
1)
(define mod-off-d2 (mod/decide-escalating "r2" 2))
(mod-off-test!
"second spam → hide (1 prior, below k=2)"
(get mod-off-d2 :action)
"hide")
(mod-off-test!
"two sanctions recorded"
(mod/subject-sanctions "spammer")
2)
(define mod-off-d3 (mod/decide-escalating "r3" 2))
(mod-off-test!
"third spam → ban (2 priors ≥ k)"
(get mod-off-d3 :action)
"ban")
(mod-off-test!
"ban decision names repeat-offender rule"
(get mod-off-d3 :rule)
"repeat-offender-ban")
(mod-off-test!
"ban proof records prior sanction count"
(get (get mod-off-d3 :proof) :prior-sanctions)
2)
;; ── different subjects accumulate independently ──
(define mod-off-d4 (mod/decide-escalating "r4" 2))
(mod-off-test!
"innocent keep → not escalated"
(get mod-off-d4 :action)
"keep")
(mod-off-test!
"innocent has no sanctions"
(mod/subject-sanctions "innocent")
0)
(mod-off-test!
"repeat-offender? true for spammer at k=2"
(mod/repeat-offender? "spammer" 2)
true)
(mod-off-test!
"repeat-offender? false for innocent at k=1"
(mod/repeat-offender? "innocent" 1)
false)
;; ── non-sanction decisions are never upgraded to ban ──
;; r5 is a clean post, but it is the 4th report about "spammer", so the
;; repeated-report rule escalates it. escalate is not a sanction, so it passes
;; through decide-escalating unchanged (never becomes :ban).
(mod/report "u5" "spammer" "a perfectly fine post")
(define mod-off-d5 (mod/decide-escalating "r5" 1))
(mod-off-test!
"non-sanction (escalate) decision is not upgraded to ban"
(get mod-off-d5 :action)
"escalate")
(mod-off-test!
"decide-escalating unknown id → nil"
(mod/decide-escalating "r99" 2)
nil)
(define mod-offenders-tests-run! (fn () {:failures mod-off-failures :total mod-off-count :passed mod-off-pass :failed mod-off-fail}))

View File

@@ -1,112 +0,0 @@
;; lib/mod/tests/pipeline.sx — Ext 19: end-to-end triage orchestration.
(define mod-pp-count 0)
(define mod-pp-pass 0)
(define mod-pp-fail 0)
(define mod-pp-failures (list))
(define
mod-pp-test!
(fn
(name got expected)
(begin
(set! mod-pp-count (+ mod-pp-count 1))
(if
(= got expected)
(set! mod-pp-pass (+ mod-pp-pass 1))
(begin
(set! mod-pp-fail (+ mod-pp-fail 1))
(append!
mod-pp-failures
(str name "\n expected: " expected "\n got: " got)))))))
(mod/policies-reset!)
(mod/register-policy!
"market"
(mod/ruleset
(mod/defrule "market-spam-remove" :remove (list :classification "spam"))
(mod/defrule "default-keep" :keep)))
;; ── spam in the market domain: full bundle ──
(define mod-pp-spam (mod/mk-report "r1" "u" "bob" "this is spam"))
(define
mod-pp
(mod/triage-pipeline "market" mod-pp-spam (list mod-pp-spam) "inst.example"))
(mod-pp-test!
"pipeline action (market policy → remove)"
(mod/pipeline-action mod-pp)
"remove")
(mod-pp-test! "pipeline rule" (get mod-pp :rule) "market-spam-remove")
(mod-pp-test!
"pipeline explanation mentions the action"
(mod/str-contains? (get mod-pp :explanation) "remove")
true)
(mod-pp-test!
"pipeline activity is Delete (remove)"
(get (mod/pipeline-activity mod-pp) :type)
"Delete")
(mod-pp-test!
"pipeline activity object is the report"
(get (mod/pipeline-activity mod-pp) :object)
"r1")
(mod-pp-test!
"pipeline wire round-trips to the same action"
(get (mod/wire->decision (mod/pipeline-wire mod-pp)) :action)
"remove")
;; ── same report, blog domain (default) → hide, Flag ──
(define
mod-pp-blog
(mod/triage-pipeline "blog" mod-pp-spam (list mod-pp-spam) "inst.example"))
(mod-pp-test!
"blog default policy → hide"
(mod/pipeline-action mod-pp-blog)
"hide")
(mod-pp-test!
"blog activity is Flag"
(get (mod/pipeline-activity mod-pp-blog) :type)
"Flag")
;; ── clean report: keep, no activity, explanation says (none) ──
(define mod-pp-clean (mod/mk-report "r2" "u" "eve" "a fine post"))
(define
mod-pp-k
(mod/triage-pipeline
"market"
mod-pp-clean
(list mod-pp-clean)
"inst.example"))
(mod-pp-test! "clean → keep" (mod/pipeline-action mod-pp-k) "keep")
(mod-pp-test! "keep → no activity" (mod/pipeline-activity mod-pp-k) nil)
(mod-pp-test!
"keep explanation says no evidence"
(mod/str-contains? (get mod-pp-k :explanation) "Evidence: (none)")
true)
(mod-pp-test!
"keep wire still round-trips"
(get (mod/wire->decision (mod/pipeline-wire mod-pp-k)) :rule)
"default-keep")
;; ── federated handoff: market decision crosses to a peer, trust-gated ──
(mod/fed-reset!)
(define mod-pp-peer-dec (mod/wire->decision (mod/pipeline-wire mod-pp)))
(mod-pp-test!
"untrusted peer: market decision is advisory"
(get (mod/fed-receive-decision "peerX" mod-pp-peer-dec) :applied)
false)
(mod/grant-trust "peerY" :mod)
(mod-pp-test!
"trusted peer: market decision applies"
(get (mod/fed-receive-decision "peerY" mod-pp-peer-dec) :applied)
true)
(mod-pp-test!
"applied action is remove"
(get (mod/fed-applied-action "r1") :action)
"remove")
(define mod-pipeline-tests-run! (fn () {:failures mod-pp-failures :total mod-pp-count :passed mod-pp-pass :failed mod-pp-fail}))

View File

@@ -1,112 +0,0 @@
;; lib/mod/tests/policies.sx — Ext 17: per-domain policy registry.
(define mod-pol-count 0)
(define mod-pol-pass 0)
(define mod-pol-fail 0)
(define mod-pol-failures (list))
(define
mod-pol-test!
(fn
(name got expected)
(begin
(set! mod-pol-count (+ mod-pol-count 1))
(if
(= got expected)
(set! mod-pol-pass (+ mod-pol-pass 1))
(begin
(set! mod-pol-fail (+ mod-pol-fail 1))
(append!
mod-pol-failures
(str name "\n expected: " expected "\n got: " got)))))))
(mod/policies-reset!)
;; market is strict: spam is removed outright, not just hidden
(define
mod-pol-market-rules
(list
(mod/mk-rule
"market-spam-remove"
:remove (list (list :classification "spam")))
(mod/mk-rule "default-keep" :keep (list))))
(mod-pol-test!
"unregistered domain falls back to default"
(mod/policy-registered? "market")
false)
(mod/register-policy! "market" mod-pol-market-rules)
(mod-pol-test!
"domain registered after register!"
(mod/policy-registered? "market")
true)
(define mod-pol-spam (mod/mk-report "r1" "a" "b" "this is spam"))
;; ── same report, different domain → different action ──
(mod-pol-test!
"market policy removes spam"
(get (mod/decide-in "market" mod-pol-spam (list mod-pol-spam)) :action)
"remove")
(mod-pol-test!
"market decision uses market rule"
(get (mod/decide-in "market" mod-pol-spam (list mod-pol-spam)) :rule)
"market-spam-remove")
(mod-pol-test!
"blog (unregistered) uses default → hide"
(get (mod/decide-in "blog" mod-pol-spam (list mod-pol-spam)) :action)
"hide")
(mod-pol-test!
"blog decision uses default rule"
(get (mod/decide-in "blog" mod-pol-spam (list mod-pol-spam)) :rule)
"spam-hide")
;; ── policy-for resolution ──
(mod-pol-test!
"policy-for market returns market rules"
(mod/policy-for "market")
mod-pol-market-rules)
(mod-pol-test!
"policy-for unknown returns default"
(mod/policy-for "events")
mod/default-rules)
(mod-pol-test!
"registered-domains lists market"
(mod/registered-domains)
(list "market"))
;; ── a second domain ──
(define
mod-pol-events-rules
(list (mod/mk-rule "events-keep-all" :keep (list))))
(mod/register-policy! "events" mod-pol-events-rules)
(mod-pol-test!
"events policy keeps everything (even spam)"
(get (mod/decide-in "events" mod-pol-spam (list mod-pol-spam)) :action)
"keep")
(mod-pol-test!
"two domains registered"
(len (mod/registered-domains))
2)
(mod-pol-test!
"market still removes after second registration"
(get (mod/decide-in "market" mod-pol-spam (list mod-pol-spam)) :action)
"remove")
;; ── clean report is keep everywhere ──
(define mod-pol-clean (mod/mk-report "r2" "a" "b" "a fine post"))
(mod-pol-test!
"clean report keep in market"
(get (mod/decide-in "market" mod-pol-clean (list mod-pol-clean)) :action)
"keep")
(mod-pol-test!
"clean report keep in blog"
(get (mod/decide-in "blog" mod-pol-clean (list mod-pol-clean)) :action)
"keep")
(define mod-policies-tests-run! (fn () {:failures mod-pol-failures :total mod-pol-count :passed mod-pol-pass :failed mod-pol-fail}))

View File

@@ -1,119 +0,0 @@
;; lib/mod/tests/quorum.sx — Ext 8: quorum over distinct reporters.
(define mod-q-count 0)
(define mod-q-pass 0)
(define mod-q-fail 0)
(define mod-q-failures (list))
(define
mod-q-test!
(fn
(name got expected)
(begin
(set! mod-q-count (+ mod-q-count 1))
(if
(= got expected)
(set! mod-q-pass (+ mod-q-pass 1))
(begin
(set! mod-q-fail (+ mod-q-fail 1))
(append!
mod-q-failures
(str name "\n expected: " expected "\n got: " got)))))))
(define
mod-q-rules
(list
(mod/mk-rule
"quorum-hide"
:hide (list (list :reporters-at-least 2)))
(mod/mk-rule "default-keep" :keep (list))))
;; ── two distinct reporters meet quorum ──
(define
mod-q-two
(list
(mod/mk-report "r1" "alice" "bob" "off-topic")
(mod/mk-report "r2" "carol" "bob" "off-topic")))
(mod-q-test!
"two distinct reporters → hide"
(get (mod/decide-quorum (first mod-q-two) mod-q-two mod-q-rules) :action)
"hide")
(mod-q-test!
"quorum decision names the rule"
(get (mod/decide-quorum (first mod-q-two) mod-q-two mod-q-rules) :rule)
"quorum-hide")
(mod-q-test!
"quorum decision tagged strategy"
(get (mod/decide-quorum (first mod-q-two) mod-q-two mod-q-rules) :strategy)
"quorum")
;; ── single reporter does not meet quorum ──
(define mod-q-one (list (mod/mk-report "r1" "alice" "bob" "off-topic")))
(mod-q-test!
"one reporter → keep (below quorum)"
(get (mod/decide-quorum (first mod-q-one) mod-q-one mod-q-rules) :action)
"keep")
;; ── anti-brigade: one user filing many reports does NOT meet quorum ──
(define
mod-q-brigade
(list
(mod/mk-report "r1" "alice" "bob" "off-topic")
(mod/mk-report "r2" "alice" "bob" "off-topic")
(mod/mk-report "r3" "alice" "bob" "off-topic")))
(mod-q-test!
"three reports, one reporter → keep (quorum counts distinct)"
(get
(mod/decide-quorum (first mod-q-brigade) mod-q-brigade mod-q-rules)
:action)
"keep")
;; contrast: the count rule WOULD fire on the same brigade (3 reports ≥ 3) —
;; quorum is strictly stronger against single-actor brigading
(mod-q-test!
"count rule fires on the brigade (distinct from quorum)"
(get
(mod/decide-report (first mod-q-brigade) mod-q-brigade mod/default-rules)
:action)
"escalate")
;; ── three distinct reporters ──
(define
mod-q-three
(list
(mod/mk-report "r1" "alice" "bob" "off-topic")
(mod/mk-report "r2" "carol" "bob" "off-topic")
(mod/mk-report "r3" "dave" "bob" "off-topic")))
(mod-q-test!
"three distinct reporters → hide"
(get
(mod/decide-quorum (first mod-q-three) mod-q-three mod-q-rules)
:action)
"hide")
(mod-q-test!
"quorum proof goal solved"
(get
(first
(get
(get
(mod/decide-quorum (first mod-q-three) mod-q-three mod-q-rules)
:proof)
:goals))
:solved)
true)
;; ── cond->goal compiles :reporters-at-least ──
(mod-q-test!
"cond->goal :reporters-at-least"
(mod/cond->goal (list :reporters-at-least 2) "Id")
"report(Id, _, Sr), setof(Br, report(_, Br, Sr), Bsr), length(Bsr, Nr), Nr >= 2")
(define mod-quorum-tests-run! (fn () {:failures mod-q-failures :total mod-q-count :passed mod-q-pass :failed mod-q-fail}))

View File

@@ -1,120 +0,0 @@
;; lib/mod/tests/severity.sx — Ext 6: strictest-wins decision strategy.
(define mod-sev-count 0)
(define mod-sev-pass 0)
(define mod-sev-fail 0)
(define mod-sev-failures (list))
(define
mod-sev-test!
(fn
(name got expected)
(begin
(set! mod-sev-count (+ mod-sev-count 1))
(if
(= got expected)
(set! mod-sev-pass (+ mod-sev-pass 1))
(begin
(set! mod-sev-fail (+ mod-sev-fail 1))
(append!
mod-sev-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── severity ranking ──
(mod-sev-test! "ban most severe" (mod/action-severity "ban") 4)
(mod-sev-test!
"remove > hide"
(< (mod/action-severity "hide") (mod/action-severity "remove"))
true)
(mod-sev-test! "keep least severe" (mod/action-severity "keep") 0)
(mod-sev-test!
"escalate above keep"
(< (mod/action-severity "keep") (mod/action-severity "escalate"))
true)
;; ── strictest agrees with default-rules on simple cases ──
(define mod-sev-spam (mod/mk-report "r1" "a" "b" "this is spam"))
(mod-sev-test!
"strictest spam → hide"
(get
(mod/decide-strictest mod-sev-spam (list mod-sev-spam) mod/default-rules)
:action)
"hide")
(define mod-sev-clean (mod/mk-report "r2" "a" "b" "a fine post"))
(mod-sev-test!
"strictest clean → keep"
(get
(mod/decide-strictest
mod-sev-clean
(list mod-sev-clean)
mod/default-rules)
:action)
"keep")
(mod-sev-test!
"decision tagged strategy strictest"
(get
(mod/decide-strictest mod-sev-spam (list mod-sev-spam) mod/default-rules)
:strategy)
"strictest")
;; ── strictest diverges from first-match when order ≠ severity ──
(define
mod-sev-rules
(list
(mod/mk-rule
"early-escalate"
:escalate (list (list :count-at-least 1)))
(mod/mk-rule "spam-remove" :remove (list (list :classification "spam")))
(mod/mk-rule "default-keep" :keep (list))))
(define mod-sev-r (mod/mk-report "r3" "a" "b" "this is spam"))
(mod-sev-test!
"first-match picks earliest rule (escalate)"
(get (mod/decide-report mod-sev-r (list mod-sev-r) mod-sev-rules) :action)
"escalate")
(mod-sev-test!
"strictest picks harshest action (remove)"
(get
(mod/decide-strictest mod-sev-r (list mod-sev-r) mod-sev-rules)
:action)
"remove")
(mod-sev-test!
"strictest names the harshest rule"
(get (mod/decide-strictest mod-sev-r (list mod-sev-r) mod-sev-rules) :rule)
"spam-remove")
(mod-sev-test!
"strictest carries proof goals"
(len
(get
(get
(mod/decide-strictest mod-sev-r (list mod-sev-r) mod-sev-rules)
:proof)
:goals))
1)
;; ── strictest among three matches (spam + repeated) ──
(define mod-sev-rep (mod/mk-report "r4" "a" "b" "buy now spam"))
(define mod-sev-reps (list mod-sev-rep mod-sev-rep mod-sev-rep))
(mod-sev-test!
"strictest among hide+escalate+keep → hide (default rules)"
(get
(mod/decide-strictest mod-sev-rep mod-sev-reps mod/default-rules)
:action)
"hide")
;; ── strictest-sol helper ──
(mod-sev-test!
"strictest-sol picks max severity"
(dict-get
(mod/strictest-sol (list {:Action "keep" :Rule "k"} {:Action "remove" :Rule "r"} {:Action "hide" :Rule "h"}))
"Action")
"remove")
(mod-sev-test! "strictest-sol nil for empty" (mod/strictest-sol (list)) nil)
(define mod-severity-tests-run! (fn () {:failures mod-sev-failures :total mod-sev-count :passed mod-sev-pass :failed mod-sev-fail}))

View File

@@ -1,108 +0,0 @@
;; lib/mod/tests/sla.sx — Ext 13: SLA sweep over pending lifecycle cases.
(define mod-sla-count 0)
(define mod-sla-pass 0)
(define mod-sla-fail 0)
(define mod-sla-failures (list))
(define
mod-sla-test!
(fn
(name got expected)
(begin
(set! mod-sla-count (+ mod-sla-count 1))
(if
(= got expected)
(set! mod-sla-pass (+ mod-sla-pass 1))
(begin
(set! mod-sla-fail (+ mod-sla-fail 1))
(append!
mod-sla-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── pending-state? ──
(mod-sla-test! "open is pending" (mod/pending-state? "open") true)
(mod-sla-test! "triaged is pending" (mod/pending-state? "triaged") true)
(mod-sla-test! "appealed is pending" (mod/pending-state? "appealed") true)
(mod-sla-test! "decided is not pending" (mod/pending-state? "decided") false)
(mod-sla-test! "final is not pending" (mod/pending-state? "final") false)
;; build cases in known states
(define mod-sla-spam (mod/mk-report "r1" "u" "bob" "this is spam"))
(define mod-sla-spam-reports (list mod-sla-spam))
(define
mod-sla-triaged
(mod/case-triage
(mod/mk-case mod-sla-spam)
mod-sla-spam-reports
mod/default-rules))
(define mod-sla-decided (mod/case-resolve mod-sla-triaged))
(define mod-sla-open (mod/mk-case (mod/mk-report "r2" "u" "eve" "hello")))
;; ── overdue? ──
(define mod-sla-tc-old (mod/mk-timed-case mod-sla-triaged 0))
(define mod-sla-tc-fresh (mod/mk-timed-case mod-sla-triaged 90))
(define mod-sla-tc-done (mod/mk-timed-case mod-sla-decided 0))
(mod-sla-test!
"old triaged case is overdue"
(mod/overdue? mod-sla-tc-old 100 50)
true)
(mod-sla-test!
"fresh triaged case not overdue"
(mod/overdue? mod-sla-tc-fresh 100 50)
false)
(mod-sla-test!
"decided case never overdue"
(mod/overdue? mod-sla-tc-done 100 50)
false)
(mod-sla-test!
"age computes elapsed ticks"
(mod/age mod-sla-tc-old 100)
100)
(mod-sla-test!
"boundary: exactly at deadline not overdue"
(mod/overdue?
(mod/mk-timed-case mod-sla-triaged 50)
100
50)
false)
(mod-sla-test!
"boundary: one past deadline overdue"
(mod/overdue?
(mod/mk-timed-case mod-sla-triaged 49)
100
50)
true)
;; ── sweep over a mixed queue ──
(define
mod-sla-queue
(list
(mod/mk-timed-case mod-sla-triaged 0)
(mod/mk-timed-case mod-sla-decided 0)
(mod/mk-timed-case mod-sla-open 90))) ;; r2, pending, age 10 → not
(mod-sla-test!
"sweep finds only the overdue pending case"
(mod/sla-sweep mod-sla-queue 100 50)
(list "r1"))
(mod-sla-test!
"overdue-count agrees"
(mod/overdue-count mod-sla-queue 100 50)
1)
;; tighten deadline so the young open case also breaches
(mod-sla-test!
"tighter deadline catches the open case too"
(mod/overdue-count mod-sla-queue 100 5)
2)
(mod-sla-test!
"empty queue → no breaches"
(mod/sla-sweep (list) 100 50)
(list))
(define mod-sla-tests-run! (fn () {:failures mod-sla-failures :total mod-sla-count :passed mod-sla-pass :failed mod-sla-fail}))

View File

@@ -1,156 +0,0 @@
;; lib/mod/tests/temporal.sx — Ext 12: burst detection over a time window.
(define mod-tm-count 0)
(define mod-tm-pass 0)
(define mod-tm-fail 0)
(define mod-tm-failures (list))
(define
mod-tm-test!
(fn
(name got expected)
(begin
(set! mod-tm-count (+ mod-tm-count 1))
(if
(= got expected)
(set! mod-tm-pass (+ mod-tm-pass 1))
(begin
(set! mod-tm-fail (+ mod-tm-fail 1))
(append!
mod-tm-failures
(str name "\n expected: " expected "\n got: " got)))))))
(define
mod-tm-at
(fn (id about t) (mod/with-at (mod/mk-report id "u" about "off-topic") t)))
(define
mod-tm-rules
(list
(mod/mk-rule "burst-hide" :hide (list (list :burst-at-least 3)))
(mod/mk-rule "default-keep" :keep (list))))
;; ── window-count helper ──
(define
mod-tm-burst
(list
(mod-tm-at "r1" "bob" 10)
(mod-tm-at "r2" "bob" 11)
(mod-tm-at "r3" "bob" 12)))
(define
mod-tm-slow
(list
(mod-tm-at "r1" "bob" 1)
(mod-tm-at "r2" "bob" 2)
(mod-tm-at "r3" "bob" 12)))
(mod-tm-test!
"window-count: all 3 within window"
(mod/window-count "bob" mod-tm-burst 12 5)
3)
(mod-tm-test!
"window-count: only 1 within window"
(mod/window-count "bob" mod-tm-slow 12 5)
1)
(mod-tm-test!
"window-count: subject filter"
(mod/window-count "eve" mod-tm-burst 12 5)
0)
;; ── burst fires; slow accumulation does not ──
(mod-tm-test!
"burst (3 in window) → hide"
(get
(mod/decide-temporal
(first mod-tm-burst)
mod-tm-burst
mod-tm-rules
12
5)
:action)
"hide")
(mod-tm-test!
"slow accumulation (1 in window) → keep"
(get
(mod/decide-temporal
(first mod-tm-slow)
mod-tm-slow
mod-tm-rules
12
5)
:action)
"keep")
;; ── contrast: the plain count rule fires on BOTH (3 total reports) ──
(mod-tm-test!
"count rule fires on slow case (distinct from burst)"
(get
(mod/decide-report (first mod-tm-slow) mod-tm-slow mod/default-rules)
:action)
"escalate")
;; ── decision shape ──
(define
mod-tm-d
(mod/decide-temporal
(first mod-tm-burst)
mod-tm-burst
mod-tm-rules
12
5))
(mod-tm-test! "burst decision rule" (get mod-tm-d :rule) "burst-hide")
(mod-tm-test!
"burst decision tagged strategy"
(get mod-tm-d :strategy)
"temporal")
(mod-tm-test!
"burst recorded in proof"
(get (get mod-tm-d :proof) :burst)
3)
(mod-tm-test!
"burst proof goal solved"
(get (first (get (get mod-tm-d :proof) :goals)) :solved)
true)
;; ── window boundary is inclusive ──
(define
mod-tm-edge
(list
(mod-tm-at "r1" "bob" 7)
(mod-tm-at "r2" "bob" 8)
(mod-tm-at "r3" "bob" 9)))
(mod-tm-test!
"window boundary inclusive (now-window = at)"
(mod/window-count "bob" mod-tm-edge 12 5)
3)
;; ── schema :at round-trips and survives evidence attach ──
(mod-tm-test!
"report-at reads timestamp"
(mod/report-at (mod-tm-at "r1" "bob" 42))
42)
(mod-tm-test!
"default report-at is 0"
(mod/report-at (mod/mk-report "r1" "a" "b" "x"))
0)
(mod-tm-test!
"attach-evidence preserves :at"
(mod/report-at
(mod/attach-evidence
(mod-tm-at "r1" "bob" 42)
(mod/mk-evidence "k" "v")))
42)
;; ── cond->goal :burst-at-least ──
(mod-tm-test!
"cond->goal :burst-at-least"
(mod/cond->goal (list :burst-at-least 3) "Id")
"report(Id, _, Sb), burst_count(Sb, Nb), Nb >= 3")
(define mod-temporal-tests-run! (fn () {:failures mod-tm-failures :total mod-tm-count :passed mod-tm-pass :failed mod-tm-fail}))

View File

@@ -1,116 +0,0 @@
;; lib/mod/tests/trace.sx — Ext 9: policy dry-run diagnostics.
(define mod-tr-count 0)
(define mod-tr-pass 0)
(define mod-tr-fail 0)
(define mod-tr-failures (list))
(define
mod-tr-test!
(fn
(name got expected)
(begin
(set! mod-tr-count (+ mod-tr-count 1))
(if
(= got expected)
(set! mod-tr-pass (+ mod-tr-pass 1))
(begin
(set! mod-tr-fail (+ mod-tr-fail 1))
(append!
mod-tr-failures
(str name "\n expected: " expected "\n got: " got)))))))
(define
mod-tr-find
(fn
(trace nm)
(reduce (fn (acc t) (if (= (get t :rule) nm) t acc)) nil trace)))
;; ── trace a spam report against the default rules ──
(define mod-tr-spam (mod/mk-report "r1" "alice" "bob" "this is spam"))
(define
mod-tr-t
(mod/trace-rules mod-tr-spam (list mod-tr-spam) mod/default-rules))
(mod-tr-test! "trace covers every rule" (len mod-tr-t) 6)
(mod-tr-test!
"spam-hide fires"
(get (mod-tr-find mod-tr-t "spam-hide") :proved)
true)
(mod-tr-test!
"default-keep always fires"
(get (mod-tr-find mod-tr-t "default-keep") :proved)
true)
(mod-tr-test!
"reviewer-remove does not fire (no evidence)"
(get (mod-tr-find mod-tr-t "reviewer-remove") :proved)
false)
(mod-tr-test!
"exonerated-keep does not fire"
(get (mod-tr-find mod-tr-t "exonerated-keep") :proved)
false)
(mod-tr-test!
"abuse-remove does not fire"
(get (mod-tr-find mod-tr-t "abuse-remove") :proved)
false)
;; ── winner matches the engine ──
(mod-tr-test!
"first-proved is spam-hide"
(get (mod/first-proved mod-tr-t) :rule)
"spam-hide")
(mod-tr-test!
"winner action matches decide-report"
(get (mod/first-proved mod-tr-t) :action)
(get
(mod/decide-report mod-tr-spam (list mod-tr-spam) mod/default-rules)
:action))
;; ── an unproved rule shows which goal failed ──
(define
mod-tr-rev-goals
(get (mod-tr-find mod-tr-t "reviewer-remove") :goals))
(mod-tr-test!
"reviewer-remove goal is unsolved"
(get (first mod-tr-rev-goals) :solved)
false)
(define mod-tr-spam-goals (get (mod-tr-find mod-tr-t "spam-hide") :goals))
(mod-tr-test!
"spam-hide goal is solved"
(get (first mod-tr-spam-goals) :solved)
true)
;; ── proved-rules list + rendering ──
(mod-tr-test!
"proved-rules lists fired rules in order"
(mod/proved-rules mod-tr-t)
(list "spam-hide" "default-keep"))
(mod-tr-test!
"trace-report marks a firing rule"
(mod/str-contains? (mod/trace-report mod-tr-t) "[fires] spam-hide")
true)
(mod-tr-test!
"trace-report marks a non-firing rule"
(mod/str-contains? (mod/trace-report mod-tr-t) "[ - ] reviewer-remove")
true)
;; ── clean report: only default-keep fires ──
(define mod-tr-clean (mod/mk-report "r2" "a" "b" "a fine post"))
(define
mod-tr-tc
(mod/trace-rules mod-tr-clean (list mod-tr-clean) mod/default-rules))
(mod-tr-test!
"clean report: only default-keep proves"
(mod/proved-rules mod-tr-tc)
(list "default-keep"))
(mod-tr-test!
"clean report winner is default-keep"
(get (mod/first-proved mod-tr-tc) :rule)
"default-keep")
(define mod-trace-tests-run! (fn () {:failures mod-tr-failures :total mod-tr-count :passed mod-tr-pass :failed mod-tr-fail}))

View File

@@ -1,117 +0,0 @@
;; lib/mod/tests/whatif.sx — Ext 10: policy what-if / impact analysis.
(define mod-wi-count 0)
(define mod-wi-pass 0)
(define mod-wi-fail 0)
(define mod-wi-failures (list))
(define
mod-wi-test!
(fn
(name got expected)
(begin
(set! mod-wi-count (+ mod-wi-count 1))
(if
(= got expected)
(set! mod-wi-pass (+ mod-wi-pass 1))
(begin
(set! mod-wi-fail (+ mod-wi-fail 1))
(append!
mod-wi-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; rules-b is the default policy with spam-hide removed: spam now falls through
;; to default-keep. A spam report flips hide → keep; everything else is unchanged.
(define mod-wi-rules-a mod/default-rules)
(define
mod-wi-rules-b
(list
(mod/mk-rule
"reviewer-remove"
:remove (list (list :evidence "confirmed-abuse")))
(mod/mk-rule
"abuse-remove"
:remove (list (list :classification "abuse")))
(mod/mk-rule
"repeated-escalate"
:escalate (list (list :count-at-least 3)))
(mod/mk-rule "default-keep" :keep (list))))
(define mod-wi-spam (mod/mk-report "r1" "a" "bob" "this is spam"))
(define mod-wi-abuse (mod/mk-report "r2" "a" "carol" "harassment here"))
(define mod-wi-clean (mod/mk-report "r3" "a" "dave" "a fine post"))
;; ── single-report diff ──
(define
mod-wi-d
(mod/decision-diff
mod-wi-spam
(list mod-wi-spam)
mod-wi-rules-a
mod-wi-rules-b))
(mod-wi-test! "spam before = hide" (get mod-wi-d :before) "hide")
(mod-wi-test! "spam after = keep" (get mod-wi-d :after) "keep")
(mod-wi-test! "spam decision flips" (get mod-wi-d :changed) true)
(mod-wi-test! "diff carries report id" (get mod-wi-d :report-id) "r1")
(define
mod-wi-da
(mod/decision-diff
mod-wi-abuse
(list mod-wi-abuse)
mod-wi-rules-a
mod-wi-rules-b))
(mod-wi-test! "abuse unchanged (remove both)" (get mod-wi-da :changed) false)
(mod-wi-test! "abuse stays remove" (get mod-wi-da :after) "remove")
(define
mod-wi-dc
(mod/decision-diff
mod-wi-clean
(list mod-wi-clean)
mod-wi-rules-a
mod-wi-rules-b))
(mod-wi-test! "clean unchanged (keep both)" (get mod-wi-dc :changed) false)
;; ── batch impact ──
(define mod-wi-batch (list mod-wi-spam mod-wi-abuse mod-wi-clean))
(define
mod-wi-impact
(mod/policy-impact mod-wi-batch mod-wi-rules-a mod-wi-rules-b))
(mod-wi-test!
"impact lists only changed reports"
(len mod-wi-impact)
1)
(mod-wi-test!
"impacted report is the spam one"
(get (first mod-wi-impact) :report-id)
"r1")
(mod-wi-test!
"impact-count agrees"
(mod/impact-count mod-wi-batch mod-wi-rules-a mod-wi-rules-b)
1)
;; ── identical rule sets → no impact ──
(mod-wi-test!
"same rules → zero impact"
(mod/impact-count mod-wi-batch mod-wi-rules-a mod-wi-rules-a)
0)
(mod-wi-test!
"same rules → empty report"
(mod/impact-report mod-wi-batch mod-wi-rules-a mod-wi-rules-a)
"No decisions change.")
;; ── rendering ──
(mod-wi-test!
"impact-report renders the flip"
(mod/str-contains?
(mod/impact-report mod-wi-batch mod-wi-rules-a mod-wi-rules-b)
"r1: hide → keep")
true)
(define mod-whatif-tests-run! (fn () {:failures mod-wi-failures :total mod-wi-count :passed mod-wi-pass :failed mod-wi-fail}))

View File

@@ -1,96 +0,0 @@
;; lib/mod/tests/wire.sx — Ext 14: decision wire format + federated transport.
(define mod-w-count 0)
(define mod-w-pass 0)
(define mod-w-fail 0)
(define mod-w-failures (list))
(define
mod-w-test!
(fn
(name got expected)
(begin
(set! mod-w-count (+ mod-w-count 1))
(if
(= got expected)
(set! mod-w-pass (+ mod-w-pass 1))
(begin
(set! mod-w-fail (+ mod-w-fail 1))
(append!
mod-w-failures
(str name "\n expected: " expected "\n got: " got)))))))
;; ── split-char ──
(mod-w-test! "split on pipe" (mod/split-char "a|b|c" "|") (list "a" "b" "c"))
(mod-w-test! "split single field" (mod/split-char "abc" "|") (list "abc"))
(mod-w-test!
"split four fields"
(len (mod/split-char "MOD1|r1|hide|spam-hide" "|"))
4)
;; ── serialize ──
(define
mod-w-dec
(mod/decide-report
(mod/mk-report "r1" "a" "bob" "this is spam")
(list (mod/mk-report "r1" "a" "bob" "this is spam"))
mod/default-rules))
(define mod-w-line (mod/decision->wire mod-w-dec))
(mod-w-test!
"wire is versioned + delimited"
mod-w-line
"MOD1|r1|hide|spam-hide")
(mod-w-test!
"wire-valid? accepts well-formed"
(mod/wire-valid? mod-w-line)
true)
(mod-w-test!
"wire-valid? rejects junk"
(mod/wire-valid? "not a wire line")
false)
(mod-w-test!
"wire-valid? rejects wrong version"
(mod/wire-valid? "MOD9|r1|hide|x")
false)
;; ── round-trip ──
(define mod-w-back (mod/wire->decision mod-w-line))
(mod-w-test! "round-trip report-id" (get mod-w-back :report-id) "r1")
(mod-w-test! "round-trip action" (get mod-w-back :action) "hide")
(mod-w-test! "round-trip rule" (get mod-w-back :rule) "spam-hide")
(mod-w-test! "round-trip tags :wire" (get mod-w-back :wire) true)
(mod-w-test! "malformed → nil" (mod/wire->decision "garbage") nil)
;; ── full federated transport: serialize → wire → deserialize → trust-gate ──
(mod/fed-reset!)
(define mod-w-peer-dec (mod/wire->decision mod-w-line))
;; untrusted peer: decision is advisory, not applied
(define mod-w-recv1 (mod/fed-receive-decision "peerX" mod-w-peer-dec))
(mod-w-test!
"wired decision from untrusted peer → advisory"
(get mod-w-recv1 :applied)
false)
(mod-w-test!
"untrusted wired decision not applied locally"
(mod/fed-applied-action "r1")
nil)
;; trusted peer: decision binds locally
(mod/grant-trust "peerY" :mod)
(define mod-w-recv2 (mod/fed-receive-decision "peerY" mod-w-peer-dec))
(mod-w-test!
"wired decision from trusted peer → applied"
(get mod-w-recv2 :applied)
true)
(mod-w-test!
"trusted wired decision binds locally"
(get (mod/fed-applied-action "r1") :action)
"hide")
(define mod-wire-tests-run! (fn () {:failures mod-w-failures :total mod-w-count :passed mod-w-pass :failed mod-w-fail}))

View File

@@ -1,56 +0,0 @@
;; lib/mod/trace.sx — policy dry-run diagnostics.
;;
;; decide-report returns the winning rule; a policy author debugging "why didn't
;; my rule fire?" needs the whole picture. mod/trace-rules evaluates a report
;; against every rule and reports each rule's proved/unproved status plus its
;; goal-by-goal derivation — so an unproved rule shows exactly which goal failed.
;; The winner is the first proved rule (same precedence as the engine).
(define
mod/trace-rules
(fn
(r reports rules)
(let
((count (mod/report-count (mod/report-about r) reports))
(id (mod/report-id r)))
(let
((db (pl-load (mod/build-program r count rules))))
(let
((proved-names (map (fn (s) (dict-get s "Rule")) (pl-query-all db (str "policy_action(" id ", _, Rule)")))))
(map
(fn (rule) (let ((nm (mod/rule-name rule))) {:proved (mod/member? nm proved-names) :goals (mod/proof-goals db id (mod/rule-when rule)) :action (mod/rule-action rule) :rule nm}))
rules))))))
(define
mod/first-proved
(fn
(trace)
(reduce
(fn (acc t) (if (nil? acc) (if (get t :proved) t acc) acc))
nil
trace)))
(define
mod/proved-rules
(fn
(trace)
(reduce
(fn
(acc t)
(if (get t :proved) (append acc (list (get t :rule))) acc))
(list)
trace)))
(define
mod/trace-row
(fn
(t)
(str
(if (get t :proved) "[fires] " "[ - ] ")
(get t :rule)
" → "
(get t :action))))
(define
mod/trace-report
(fn (trace) (mod/join-with "\n" (map mod/trace-row trace))))

View File

@@ -1,56 +0,0 @@
;; lib/mod/whatif.sx — policy what-if / impact analysis.
;;
;; Before shipping a policy change, a moderation team needs to know which past or
;; pending reports would decide differently. mod/decision-diff compares one
;; report's action under two rule sets; mod/policy-impact runs a whole batch and
;; returns only the reports whose decision flips. Pure SX over decide-report.
(define
mod/decision-diff
(fn
(r reports rules-a rules-b)
(let
((a (get (mod/decide-report r reports rules-a) :action))
(b (get (mod/decide-report r reports rules-b) :action)))
{:after b :changed (if (= a b) false true) :report-id (mod/report-id r) :before a})))
(define
mod/policy-impact
(fn
(reports rules-a rules-b)
(reduce
(fn
(acc r)
(let
((d (mod/decision-diff r reports rules-a rules-b)))
(if (get d :changed) (append acc (list d)) acc)))
(list)
reports)))
(define
mod/impact-count
(fn
(reports rules-a rules-b)
(len (mod/policy-impact reports rules-a rules-b))))
(define
mod/impact-report
(fn
(reports rules-a rules-b)
(let
((changed (mod/policy-impact reports rules-a rules-b)))
(if
(empty? changed)
"No decisions change."
(mod/join-with
"\n"
(map
(fn
(d)
(str
(get d :report-id)
": "
(get d :before)
" → "
(get d :after)))
changed))))))

View File

@@ -1,55 +0,0 @@
;; lib/mod/wire.sx — portable decision wire format for federation transport.
;;
;; fed.sx shares decisions as in-memory dicts and leaves mod/fed-send! as the
;; transport seam. This is the bytes that cross it: a versioned, pipe-delimited
;; line encoding the verdict a peer needs (report id, action, rule) — enough to
;; trust-gate and apply/advise, without shipping the whole proof tree. The
;; loaded env has no string split, so split is built over slice/len.
(define
mod/split-loop
(fn
(s ch n start pos acc)
(if
(= pos n)
(append acc (list (slice s start n)))
(if
(= (slice s pos (+ pos 1)) ch)
(mod/split-loop
s
ch
n
(+ pos 1)
(+ pos 1)
(append acc (list (slice s start pos))))
(mod/split-loop s ch n start (+ pos 1) acc)))))
(define
mod/split-char
(fn (s ch) (mod/split-loop s ch (len s) 0 0 (list))))
(define
mod/decision->wire
(fn
(d)
(str "MOD1|" (get d :report-id) "|" (get d :action) "|" (get d :rule))))
(define
mod/wire-valid?
(fn
(w)
(let
((parts (mod/split-char w "|")))
(if
(= (len parts) 4)
(= (nth parts 0) "MOD1")
false))))
(define
mod/wire->decision
(fn
(w)
(if
(mod/wire-valid? w)
(let ((parts (mod/split-char w "|"))) {:action (nth parts 2) :wire true :rule (nth parts 3) :report-id (nth parts 1)})
nil)))

44
lib/search/api.sx Normal file
View File

@@ -0,0 +1,44 @@
;; search public API — assembles the canonical Haskell source from all layers.
;; Tests and callers concatenate `search/src` with their own top-level bindings
;; (e.g. "result = lookupTerm \"cat\" idx\n") and evaluate via the haskell-on-sx
;; interpreter. Public Haskell entry points: indexDoc, lookupTerm, deleteDoc,
;; docFreq, allTerms, tokens, positioned, evalQuery, parseQuery, searchQuery,
;; rankTfIdf, rankBm25, topNTfIdf, topNBm25, fedIndex, aclFilter, searchTfIdfAcl,
;; topNTfIdfAcl, searchBm25Acl, prefixTerms, prefixDocs, prefixRankTfIdf,
;; paginate, pageTfIdf, pageBm25, resultCount, editDist, fuzzyTerms, fuzzyDocs,
;; fuzzyRankTfIdf, highlight, snippet, stem, stemText, stemTokens, indexStemmed,
;; nearDocs, expandTerm, synDocs, synRankTfIdf, queryTerms, searchRankTfIdf,
;; searchRankBm25, suggestN, suggest.
(define
search/src
(str
search/tokenize-src
"\n"
search/index-src
"\n"
search/query-src
"\n"
search/parse-src
"\n"
search/rank-src
"\n"
search/fed-src
"\n"
search/prefix-src
"\n"
search/page-src
"\n"
search/fuzzy-src
"\n"
search/highlight-src
"\n"
search/stem-src
"\n"
search/near-src
"\n"
search/syn-src
"\n"
search/rankq-src
"\n"
search/suggest-src))

View File

@@ -0,0 +1,55 @@
# search-on-sx conformance config — sourced by lib/guest/conformance.sh.
LANG_NAME=search
SCOREBOARD_DIR=lib/search
MODE=counters
COUNTERS_PASS=hk-test-pass
COUNTERS_FAIL=hk-test-fail
TIMEOUT_PER_SUITE=600
PRELOADS=(
lib/haskell/tokenizer.sx
lib/haskell/layout.sx
lib/haskell/parser.sx
lib/haskell/desugar.sx
lib/haskell/runtime.sx
lib/haskell/match.sx
lib/haskell/eval.sx
lib/haskell/map.sx
lib/haskell/set.sx
lib/haskell/testlib.sx
lib/search/tokenize.sx
lib/search/index.sx
lib/search/query.sx
lib/search/parse.sx
lib/search/rank.sx
lib/search/fed.sx
lib/search/prefix.sx
lib/search/page.sx
lib/search/fuzzy.sx
lib/search/highlight.sx
lib/search/stem.sx
lib/search/near.sx
lib/search/syn.sx
lib/search/rankq.sx
lib/search/suggest.sx
lib/search/api.sx
lib/search/testlib.sx
)
SUITES=(
"index:lib/search/tests/index.sx"
"boolean:lib/search/tests/boolean.sx"
"parse:lib/search/tests/parse.sx"
"rank:lib/search/tests/rank.sx"
"integration:lib/search/tests/integration.sx"
"prefix:lib/search/tests/prefix.sx"
"page:lib/search/tests/page.sx"
"fuzzy:lib/search/tests/fuzzy.sx"
"highlight:lib/search/tests/highlight.sx"
"stem:lib/search/tests/stem.sx"
"near:lib/search/tests/near.sx"
"syn:lib/search/tests/syn.sx"
"rankq:lib/search/tests/rankq.sx"
"suggest:lib/search/tests/suggest.sx"
)

View File

@@ -1,3 +1,3 @@
#!/usr/bin/env bash
# Thin wrapper — see lib/guest/conformance.sh and lib/mod/conformance.conf.
# Thin wrapper — see lib/guest/conformance.sh and lib/search/conformance.conf.
exec bash "$(dirname "$0")/../guest/conformance.sh" "$(dirname "$0")/conformance.conf" "$@"

16
lib/search/fed.sx Normal file
View File

@@ -0,0 +1,16 @@
;; search federation + ACL — Haskell source fragment. Depends on index + rank.
;; Federation merges per-peer INDICES (not ranked results): each peer's local
;; DocIds are relabelled to global ids `gid peer local = peer*1000 + local`
;; (dedupe by (peer,doc-id) is automatic via the bijection), then posting lists
;; are unioned per term. Ranking then runs once over the merged index, which is
;; rank-correct. ACL is a post-rank filter: an injected `permit :: DocId -> Bool`
;; predicate (viewer baked in by the caller) — never baked into the index.
;; fedIndex :: [(PeerId, Index)] -> Index
;; aclFilter :: (DocId -> Bool) -> [DocId] -> [DocId]
;; searchTfIdfAcl :: (DocId -> Bool) -> [Term] -> Index -> [DocId]
;; topNTfIdfAcl :: Int -> (DocId -> Bool) -> [Term] -> Index -> [DocId]
;; searchBm25Acl :: (DocId -> Bool) -> Float -> Float -> [Term] -> Index -> [DocId]
(define
search/fed-src
"gid peer local = peer * 1000 + local\nfedRelabelPosting peer p = (gid peer (fst p), snd p)\nfedRelabelEntry peer e = (fst e, map (fedRelabelPosting peer) (snd e))\nfedRelabelIndex peer idx = map (fedRelabelEntry peer) idx\nfedInsP p [] = [p]\nfedInsP p (q:qs) = if fst p < fst q then p : q : qs else if fst p == fst q then p : qs else q : fedInsP p qs\nfedMergePL a b = foldr fedInsP b a\nfedInsTerm t pl [] = [(t, pl)]\nfedInsTerm t pl (x:xs) = if t < fst x then (t, pl) : x : xs else if t == fst x then (fst x, fedMergePL pl (snd x)) : xs else x : fedInsTerm t pl xs\nfedMergeEntry idx e = fedInsTerm (fst e) (snd e) idx\nfedMergeTwo a b = foldl fedMergeEntry a b\nfedAddPeer acc pair = fedMergeTwo acc (fedRelabelIndex (fst pair) (snd pair))\nfedIndex pairs = foldl fedAddPeer emptyIndex pairs\naclFilter permit docs = filter permit docs\nsearchTfIdfAcl permit ts idx = aclFilter permit (rankTfIdf ts idx)\ntopNTfIdfAcl n permit ts idx = take n (aclFilter permit (rankTfIdf ts idx))\nsearchBm25Acl permit k1 b ts idx = aclFilter permit (rankBm25 k1 b ts idx)\n")

12
lib/search/fuzzy.sx Normal file
View File

@@ -0,0 +1,12 @@
;; search fuzzy matching — Haskell source fragment. Depends on index + rank.
;; Levenshtein edit distance (O(m*n) row-based DP — the naive recursive version is
;; exponential and far too slow under load) expands a query term to all indexed
;; terms within a max distance, then unions / ranks their docs.
;; editDist :: String -> String -> Int
;; fuzzyTerms :: Int -> String -> Index -> [Term] (sorted)
;; fuzzyDocs :: Int -> String -> Index -> [DocId] (sorted union)
;; fuzzyRankTfIdf :: Int -> String -> Index -> [DocId]
(define
search/fuzzy-src
"edMin3 a b c = min a (min b c)\nedCost x y = if x == y then 0 else 1\nedUpto i n = if i > n then [] else i : edUpto (i + 1) n\nedLast [x] = x\nedLast (x:xs) = edLast xs\nedNrow x [] prev left = []\nedNrow x (y:ys) prev left = let v = edMin3 (head (tail prev) + 1) (left + 1) (head prev + edCost x y) in v : edNrow x ys (tail prev) v\nedRow x ys prev = let f = head prev + 1 in f : edNrow x ys prev f\nedRows [] ys prev = prev\nedRows (x:xs) ys prev = edRows xs ys (edRow x ys prev)\neditDist xs ys = edLast (edRows xs ys (edUpto 0 (length ys)))\nqWithinDist maxd term t = editDist term t <= maxd\nfuzzyTerms maxd term idx = filter (qWithinDist maxd term) (allTerms idx)\nfuzzyDocs maxd term idx = foldl (candStep idx) [] (fuzzyTerms maxd term idx)\nfuzzyRankTfIdf maxd term idx = rankTfIdf (fuzzyTerms maxd term idx) idx\n")

10
lib/search/highlight.sx Normal file
View File

@@ -0,0 +1,10 @@
;; search highlight / snippet — Haskell source fragment. Depends on tokenize.
;; Operates on document text (not the index): marks query-matching tokens with
;; [..] and extracts a context window around the first match. Tokens are
;; normalized (lowercase, punctuation-stripped) by `tokens`, matching index side.
;; highlight :: [Term] -> String -> String
;; snippet :: Int -> [Term] -> String -> String (ctx tokens each side of 1st match)
(define
search/highlight-src
"hlMark terms t = if elem t terms then \"[\" ++ t ++ \"]\" else t\nhighlight terms text = unwords (map (hlMark terms) (tokens text))\nhlIdxFrom terms [] i = 0 - 1\nhlIdxFrom terms (t:ts) i = if elem t terms then i else hlIdxFrom terms ts (i + 1)\nhlIdx terms toks = hlIdxFrom terms toks 0\nhlMax0 x = if x < 0 then 0 else x\nsnipStart ctx i = if i < 0 then 0 else hlMax0 (i - ctx)\nsnipToks ctx terms toks = unwords (map (hlMark terms) (take (2 * ctx + 1) (drop (snipStart ctx (hlIdx terms toks)) toks)))\nsnippet ctx terms text = snipToks ctx terms (tokens text)\n")

15
lib/search/index.sx Normal file
View File

@@ -0,0 +1,15 @@
;; search inverted index — Haskell source fragment (depends on tokenize).
;; Index = [(Term, [(DocId, [Pos])])], sorted by Term; postings sorted by DocId.
;; Data.Map's public API lacks toList/keys/map/filter, so a sorted assoc-list
;; index is used — it is the conceptual `Map Term [(DocId,[Pos])]` and exposes
;; term iteration (allTerms) and df naturally for ranking.
;; emptyIndex :: Index
;; indexDoc :: DocId -> String -> Index -> Index (re-index replaces)
;; lookupTerm :: Term -> Index -> [(DocId, [Pos])]
;; deleteDoc :: DocId -> Index -> Index
;; docFreq :: Term -> Index -> Int
;; allTerms :: Index -> [Term]
(define
search/index-src
"emptyIndex = []\ngroupBump [] t p = [(t, [p])]\ngroupBump (g:gs) t p = if fst g == t then (t, snd g ++ [p]) : gs else g : groupBump gs t p\ngroupStep acc tp = groupBump acc (fst tp) (snd tp)\ngroupTok pairs = foldl groupStep [] pairs\ninsPosting d ps [] = [(d, ps)]\ninsPosting d ps (q:qs) = if d < fst q then (d, ps) : q : qs else if d == fst q then (d, ps) : qs else q : insPosting d ps qs\ninsTerm t d ps [] = [(t, [(d, ps)])]\ninsTerm t d ps (e:es) = if t < fst e then (t, [(d, ps)]) : e : es else if t == fst e then (fst e, insPosting d ps (snd e)) : es else e : insTerm t d ps es\nindexStep d ix tp = insTerm (fst tp) d (snd tp) ix\nindexDoc d text idx = foldl (indexStep d) idx (groupTok (positioned text))\nlookupTerm t idx = case lookup t idx of { Nothing -> []; Just pl -> pl }\ndocFreq t idx = length (lookupTerm t idx)\nallTerms idx = map fst idx\npostingKeep d q = fst q /= d\ndropTermDoc d e = (fst e, filter (postingKeep d) (snd e))\nplKeep e = not (null (snd e))\ndeleteDoc d idx = filter plKeep (map (dropTermDoc d) idx)\n")

8
lib/search/near.sx Normal file
View File

@@ -0,0 +1,8 @@
;; search proximity (NEAR) — Haskell source fragment. Depends on query (posIn,
;; docsWith, sortedInter). Finds docs where two terms occur within k positions of
;; each other (unordered), using the positional postings.
;; nearDocs :: Int -> Term -> Term -> Index -> [DocId] (sorted)
(define
search/near-src
"nrAbsDiff a b = if a > b then a - b else b - a\nnrCloseTo k x [] = False\nnrCloseTo k x (y:ys) = if nrAbsDiff x y <= k then True else nrCloseTo k x ys\nnrAnyClose k [] ys = False\nnrAnyClose k (x:xs) ys = if nrCloseTo k x ys then True else nrAnyClose k xs ys\nnearInDoc k t1 t2 d idx = nrAnyClose k (posIn t1 d idx) (posIn t2 d idx)\nnearHere k t1 t2 idx d = nearInDoc k t1 t2 d idx\nnearDocs k t1 t2 idx = filter (nearHere k t1 t2 idx) (sortedInter (docsWith t1 idx) (docsWith t2 idx))\n")

11
lib/search/page.sx Normal file
View File

@@ -0,0 +1,11 @@
;; search pagination — Haskell source fragment. Depends on rank.
;; Windows a ranked result list by offset/limit (offset >= length -> empty;
;; limit clamps to what remains).
;; paginate :: Int -> Int -> [DocId] -> [DocId] (offset, limit)
;; pageTfIdf :: Int -> Int -> [Term] -> Index -> [DocId]
;; pageBm25 :: Int -> Int -> Float -> Float -> [Term] -> Index -> [DocId]
;; resultCount :: [Term] -> Index -> Int
(define
search/page-src
"paginate off lim docs = take lim (drop off docs)\npageTfIdf off lim ts idx = paginate off lim (rankTfIdf ts idx)\npageBm25 off lim k1 b ts idx = paginate off lim (rankBm25 k1 b ts idx)\nresultCount ts idx = length (rankTfIdf ts idx)\n")

18
lib/search/parse.sx Normal file
View File

@@ -0,0 +1,18 @@
;; search query parser — Haskell source fragment. Depends on tokenize + query.
;; Grammar (precedence OR < AND < NOT):
;; expr = orExpr
;; orExpr = andExpr (OR andExpr)*
;; andExpr= notExpr ((AND | <implicit>) notExpr)* -- adjacency means AND
;; notExpr= NOT notExpr | atom
;; atom = '(' expr ')' | '"' word+ '"' | word
;; Keywords AND/OR/NOT are case-insensitive; bare words are normalized via tokens.
;; Gotchas: delimiters matched by ord (escaped char literals like '\"' break the
;; haskell-on-sx tokenizer); an [] *pattern* inside a `case` alt also breaks the
;; parser, so qNormTerm/qDropRP/showQ are written as multi-clause functions.
;; parseQuery :: String -> Query
;; searchQuery :: String -> Index -> [DocId]
;; showQ :: Query -> String -- canonical render for tests/debug
(define
search/parse-src
"data QTok = TAnd | TOr | TNot | TLP | TRP | TWord String | TPhrase [String]\nqIsSpace c = ord c == 32\nqIsLP c = ord c == 40\nqIsRP c = ord c == 41\nqIsQuote c = ord c == 34\nqDelim c = qIsSpace c || qIsLP c || qIsRP c || qIsQuote c\nqReadWord [] = ([], [])\nqReadWord (c:cs) = if qDelim c then ([], c:cs) else let (w, rest) = qReadWord cs in (c:w, rest)\nqReadPhrase [] = ([], [])\nqReadPhrase (c:cs) = if qIsQuote c then ([], cs) else let (w, rest) = qReadPhrase cs in (c:w, rest)\ntoUpperCh c = chr (toUpper (ord c))\nqUpper w = joinChars (map toUpperCh w)\nqFirstTok [] = \"\"\nqFirstTok (x:xs) = x\nqNormTerm w = qFirstTok (tokens w)\nqClassify w = if qUpper w == \"AND\" then TAnd else if qUpper w == \"OR\" then TOr else if qUpper w == \"NOT\" then TNot else TWord (qNormTerm w)\nqPhraseTok cs = let (p, rest) = qReadPhrase cs in TPhrase (tokens p) : qtokens rest\nqWordTok cs = let (w, rest) = qReadWord cs in qClassify w : qtokens rest\nqtokens [] = []\nqtokens (c:cs) = if qIsSpace c then qtokens cs else if qIsLP c then TLP : qtokens cs else if qIsRP c then TRP : qtokens cs else if qIsQuote c then qPhraseTok cs else qWordTok (c:cs)\nqDropRP (q, (TRP:rest)) = (q, rest)\nqDropRP (q, ts) = (q, ts)\nparseAtom [] = (Term \"\", [])\nparseAtom (TLP:ts) = qDropRP (parseExpr ts)\nparseAtom (TPhrase ps : ts) = (Phrase ps, ts)\nparseAtom (TWord w : ts) = (Term w, ts)\nparseAtom ts = (Term \"\", ts)\nqWrapNot (q, ts) = (Not q, ts)\nparseNot (TNot:ts) = qWrapNot (parseNot ts)\nparseNot ts = parseAtom ts\nqStartsAtom (TWord w : ts) = True\nqStartsAtom (TPhrase p : ts) = True\nqStartsAtom (TLP : ts) = True\nqStartsAtom (TNot : ts) = True\nqStartsAtom ts = False\nqAndStep left ts = let (r, rest) = parseNot ts in parseAndR (And left r) rest\nparseAndR left (TAnd:ts) = qAndStep left ts\nparseAndR left ts = if qStartsAtom ts then qAndStep left ts else (left, ts)\nparseAnd ts = let (l, rest) = parseNot ts in parseAndR l rest\nparseOrR left (TOr:ts) = let (r, rest) = parseAnd ts in parseOrR (Or left r) rest\nparseOrR left ts = (left, ts)\nparseExpr ts = let (l, rest) = parseAnd ts in parseOrR l rest\nparseQuery s = fst (parseExpr (qtokens s))\nsearchQuery s idx = evalQuery idx (parseQuery s)\njoinSp [] = \"\"\njoinSp [x] = x\njoinSp (x:xs) = x ++ \"-\" ++ joinSp xs\nshowQ (Term t) = \"T:\" ++ t\nshowQ (And a b) = \"(\" ++ showQ a ++ \" & \" ++ showQ b ++ \")\"\nshowQ (Or a b) = \"(\" ++ showQ a ++ \" | \" ++ showQ b ++ \")\"\nshowQ (Not a) = \"!\" ++ showQ a\nshowQ (Phrase ts) = \"P:\" ++ joinSp ts\n")

10
lib/search/prefix.sx Normal file
View File

@@ -0,0 +1,10 @@
;; search prefix / wildcard queries — Haskell source fragment. Depends on index +
;; rank (reuses candStep / rankTfIdf). A prefix matches every indexed term that
;; starts with it; the matching terms are unioned (OR) into a docid set.
;; prefixTerms :: String -> Index -> [Term] (sorted, from allTerms)
;; prefixDocs :: String -> Index -> [DocId] (sorted union)
;; prefixRankTfIdf :: String -> Index -> [DocId] (ranked by the matched terms)
(define
search/prefix-src
"prefixTerms pre idx = filter (isPrefixOf pre) (allTerms idx)\nprefixDocs pre idx = foldl (candStep idx) [] (prefixTerms pre idx)\nprefixRankTfIdf pre idx = rankTfIdf (prefixTerms pre idx) idx\n")

11
lib/search/query.sx Normal file
View File

@@ -0,0 +1,11 @@
;; search query AST + boolean/phrase evaluation — Haskell source fragment.
;; Depends on tokenize + index.
;; data Query = Term String | And Query Query | Or Query Query
;; | Not Query | Phrase [String]
;; evalQuery :: Index -> Query -> [DocId] (sorted, unique)
;; Boolean ops are linear merges over docid-sorted posting lists; Not uses
;; allDocs as the universe; Phrase checks positional adjacency.
(define
search/query-src
"data Query = Term String | And Query Query | Or Query Query | Not Query | Phrase [String]\ndocsWith t idx = map fst (lookupTerm t idx)\nsortedUnion [] ys = ys\nsortedUnion xs [] = xs\nsortedUnion (x:xs) (y:ys) = if x < y then x : sortedUnion xs (y:ys) else if x > y then y : sortedUnion (x:xs) ys else x : sortedUnion xs ys\nsortedInter [] ys = []\nsortedInter xs [] = []\nsortedInter (x:xs) (y:ys) = if x < y then sortedInter xs (y:ys) else if x > y then sortedInter (x:xs) ys else x : sortedInter xs ys\nsortedDiff [] ys = []\nsortedDiff xs [] = xs\nsortedDiff (x:xs) (y:ys) = if x < y then x : sortedDiff xs (y:ys) else if x > y then sortedDiff (x:xs) ys else sortedDiff xs ys\nmergeDocs acc e = sortedUnion acc (map fst (snd e))\nallDocs idx = foldl mergeDocs [] idx\nposIn t d idx = case lookup d (lookupTerm t idx) of { Nothing -> []; Just ps -> ps }\nelemSorted x [] = False\nelemSorted x (y:ys) = if x == y then True else if x < y then False else elemSorted x ys\nphraseAtAll [] d idx p i = True\nphraseAtAll (t:ts) d idx p i = if elemSorted (p + i) (posIn t d idx) then phraseAtAll ts d idx p (i + 1) else False\nphraseStartsAt ts d idx p = phraseAtAll ts d idx p 0\nphraseInDoc [] d idx = True\nphraseInDoc (t0:rest) d idx = any (phraseStartsAt (t0:rest) d idx) (posIn t0 d idx)\nphraseHere ts idx d = phraseInDoc ts d idx\ninterStep idx acc tt = sortedInter acc (docsWith tt idx)\nphraseCands [] idx = allDocs idx\nphraseCands (t:ts) idx = foldl (interStep idx) (docsWith t idx) ts\nphraseDocs ts idx = filter (phraseHere ts idx) (phraseCands ts idx)\nevalQuery idx q = case q of { Term t -> docsWith t idx ; And a b -> sortedInter (evalQuery idx a) (evalQuery idx b) ; Or a b -> sortedUnion (evalQuery idx a) (evalQuery idx b) ; Not a -> sortedDiff (allDocs idx) (evalQuery idx a) ; Phrase ts -> phraseDocs ts idx }\n")

14
lib/search/rank.sx Normal file
View File

@@ -0,0 +1,14 @@
;; search ranking — Haskell source fragment. Depends on tokenize + index + query.
;; Ranked retrieval over the candidate set (docs containing any query term).
;; Scores are floats; ties broken by DocId ascending (deterministic).
;; numDocs :: Index -> Int
;; docFreq :: Term -> Index -> Int (from index)
;; docLen :: DocId -> Index -> Int
;; rankTfIdf :: [Term] -> Index -> [DocId]
;; topNTfIdf :: Int -> [Term] -> Index -> [DocId]
;; rankBm25 :: Float -> Float -> [Term] -> Index -> [DocId] (k1, b)
;; topNBm25 :: Int -> Float -> Float -> [Term] -> Index -> [DocId]
(define
search/rank-src
"numDocs idx = length (allDocs idx)\ntfIn t d idx = length (posIn t d idx)\nqIdf n df = if df == 0 then 0 else log (n / df)\nidf t idx = qIdf (numDocs idx) (docFreq t idx)\ntermScoreTf idx d t = tfIn t d idx * idf t idx\ntfidfDoc ts idx d = sum (map (termScoreTf idx d) ts)\ncandStep idx acc t = sortedUnion acc (docsWith t idx)\ncandDocs ts idx = foldl (candStep idx) [] ts\ncmpScore p1 p2 = if fst p1 > fst p2 then LT else if fst p1 < fst p2 then GT else compare (snd p1) (snd p2)\nmkPair f ts idx d = (f ts idx d, d)\nrankWith f ts idx = map snd (sortBy cmpScore (map (mkPair f ts idx) (candDocs ts idx)))\nrankTfIdf ts idx = rankWith tfidfDoc ts idx\ntopNTfIdf n ts idx = take n (rankTfIdf ts idx)\ntfAt d idx t = tfIn t d idx\ndocLen d idx = sum (map (tfAt d idx) (allTerms idx))\nlenAt idx d = docLen d idx\navgDocLen idx = sum (map (lenAt idx) (allDocs idx)) / numDocs idx\nbm25idf t idx = log ((numDocs idx - docFreq t idx + 0.5) / (docFreq t idx + 0.5) + 1)\nbm25Term k1 b avgdl idx d t = bm25idf t idx * (tfIn t d idx * (k1 + 1)) / (tfIn t d idx + k1 * (1 - b + b * docLen d idx / avgdl))\nbm25Doc k1 b ts idx d = sum (map (bm25Term k1 b (avgDocLen idx) idx d) ts)\nrankBm25 k1 b ts idx = rankWith (bm25Doc k1 b) ts idx\ntopNBm25 n k1 b ts idx = take n (rankBm25 k1 b ts idx)\n")

11
lib/search/rankq.sx Normal file
View File

@@ -0,0 +1,11 @@
;; search boolean-filtered ranked search — Haskell source fragment.
;; Depends on parse (parseQuery/Query), query (evalQuery), rank (tfidfDoc/bm25Doc/
;; cmpScore). Filters by the boolean query, then ranks the surviving docs by
;; relevance over the query's leaf terms — the real-world filter-then-rank pattern.
;; queryTerms :: Query -> [Term]
;; searchRankTfIdf :: String -> Index -> [DocId]
;; searchRankBm25 :: Float -> Float -> String -> Index -> [DocId]
(define
search/rankq-src
"queryTerms (Term t) = [t]\nqueryTerms (And a b) = queryTerms a ++ queryTerms b\nqueryTerms (Or a b) = queryTerms a ++ queryTerms b\nqueryTerms (Not a) = queryTerms a\nqueryTerms (Phrase ts) = ts\nmkSubPair f terms idx d = (f terms idx d, d)\nrankSubsetWith f terms docs idx = map snd (sortBy cmpScore (map (mkSubPair f terms idx) docs))\nsearchRankTfIdf s idx = let q = parseQuery s in rankSubsetWith tfidfDoc (queryTerms q) (evalQuery idx q) idx\nsearchRankBm25 k1 b s idx = let q = parseQuery s in rankSubsetWith (bm25Doc k1 b) (queryTerms q) (evalQuery idx q) idx\n")

View File

@@ -0,0 +1,23 @@
{
"lang": "search",
"total_passed": 234,
"total_failed": 0,
"total": 234,
"suites": [
{"name":"index","passed":18,"failed":0,"total":18},
{"name":"boolean","passed":28,"failed":0,"total":28},
{"name":"parse","passed":32,"failed":0,"total":32},
{"name":"rank","passed":23,"failed":0,"total":23},
{"name":"integration","passed":21,"failed":0,"total":21},
{"name":"prefix","passed":14,"failed":0,"total":14},
{"name":"page","passed":12,"failed":0,"total":12},
{"name":"fuzzy","passed":18,"failed":0,"total":18},
{"name":"highlight","passed":12,"failed":0,"total":12},
{"name":"stem","passed":18,"failed":0,"total":18},
{"name":"near","passed":9,"failed":0,"total":9},
{"name":"syn","passed":9,"failed":0,"total":9},
{"name":"rankq","passed":11,"failed":0,"total":11},
{"name":"suggest","passed":9,"failed":0,"total":9}
],
"generated": "2026-06-07T00:44:05+00:00"
}

20
lib/search/scoreboard.md Normal file
View File

@@ -0,0 +1,20 @@
# search scoreboard
**234 / 234 passing** (0 failure(s)).
| Suite | Passed | Total | Status |
|-------|--------|-------|--------|
| index | 18 | 18 | ok |
| boolean | 28 | 28 | ok |
| parse | 32 | 32 | ok |
| rank | 23 | 23 | ok |
| integration | 21 | 21 | ok |
| prefix | 14 | 14 | ok |
| page | 12 | 12 | ok |
| fuzzy | 18 | 18 | ok |
| highlight | 12 | 12 | ok |
| stem | 18 | 18 | ok |
| near | 9 | 9 | ok |
| syn | 9 | 9 | ok |
| rankq | 11 | 11 | ok |
| suggest | 9 | 9 | ok |

15
lib/search/stem.sx Normal file
View File

@@ -0,0 +1,15 @@
;; search stemming — Haskell source fragment. Depends on tokenize + index.
;; Lightweight, deterministic English suffix stripping (recall-improving
;; normalizer). Rules are checked most-specific first; conservative length guards
;; avoid mangling short words. Not a full Porter stemmer.
;; Gotcha: take/drop over a String yield char CODES (ints), not char strings, so
;; rebuild strings with `stStr = joinChars . map chr`. (isSuffixOf's reverse also
;; trips `++` on the String representation, hence the manual stEnds.)
;; stem :: String -> String
;; stemText :: String -> String (tokenize + stem + rejoin)
;; stemTokens :: String -> [String]
;; indexStemmed:: DocId -> String -> Index -> Index (index the stemmed text)
(define
search/stem-src
"stStr cs = joinChars (map chr cs)\nstEnds suf w = let n = length w in let m = length suf in if m > n then False else stStr (drop (n - m) w) == suf\nstDropEnd k w = stStr (take (length w - k) w)\nstem w = if stEnds \"ies\" w && length w >= 5 then stDropEnd 3 w ++ \"y\" else if stEnds \"ss\" w then w else if stEnds \"es\" w && length w >= 5 then stDropEnd 2 w else if stEnds \"s\" w && length w >= 4 then stDropEnd 1 w else if stEnds \"ing\" w && length w >= 6 then stDropEnd 3 w else if stEnds \"ed\" w && length w >= 5 then stDropEnd 2 w else w\nstemTokens s = map stem (tokens s)\nstemText s = unwords (stemTokens s)\nindexStemmed d text idx = indexDoc d (stemText text) idx\n")

9
lib/search/suggest.sx Normal file
View File

@@ -0,0 +1,9 @@
;; search did-you-mean / spelling suggestion — Haskell source fragment.
;; Depends on fuzzy (editDist) + index (allTerms). Ranks indexed terms by edit
;; distance to a (possibly misspelled) query term; ties broken alphabetically.
;; suggestN :: Int -> String -> Index -> [Term]
;; suggest :: String -> Index -> Term ("" if the index has no terms)
(define
search/suggest-src
"sgMk term t = (editDist term t, t)\nsgPairs term idx = map (sgMk term) (allTerms idx)\nsgCmp p1 p2 = if fst p1 < fst p2 then LT else if fst p1 > fst p2 then GT else compare (snd p1) (snd p2)\nsuggestN n term idx = take n (map snd (sortBy sgCmp (sgPairs term idx)))\nsgHead [] = \"\"\nsgHead (x:xs) = x\nsuggest term idx = sgHead (suggestN 1 term idx)\n")

10
lib/search/syn.sx Normal file
View File

@@ -0,0 +1,10 @@
;; search synonym / query expansion — Haskell source fragment. Depends on index +
;; rank. A synonym map is an assoc list [(Term, [Term])]; a query term is expanded
;; to itself plus its synonyms, then the expanded set is unioned / ranked.
;; expandTerm :: [(Term,[Term])] -> Term -> [Term]
;; synDocs :: [(Term,[Term])] -> Term -> Index -> [DocId]
;; synRankTfIdf :: [(Term,[Term])] -> Term -> Index -> [DocId]
(define
search/syn-src
"synLookup synmap t = case lookup t synmap of { Nothing -> [] ; Just ss -> ss }\nexpandTerm synmap t = t : synLookup synmap t\nsynDocs synmap t idx = foldl (candStep idx) [] (expandTerm synmap t)\nsynRankTfIdf synmap t idx = rankTfIdf (expandTerm synmap t) idx\n")

50
lib/search/testlib.sx Normal file
View File

@@ -0,0 +1,50 @@
;; search test helpers — convert forced haskell values to plain SX and run
;; programs built on top of search/src. Reuses hk-test / counters from
;; lib/haskell/testlib.sx (preloaded by the conformance config).
;; Recursively turn a forced HK value into plain SX:
;; cons-list -> SX list, Tuple -> SX list, leaves unchanged.
(define
search-hk->sx
(fn
(v)
(cond
((and (list? v) (not (empty? v)) (= (first v) "[]")) (list))
((and (list? v) (not (empty? v)) (= (first v) ":"))
(cons
(search-hk->sx (nth v 1))
(search-hk->sx (nth v 2))))
((and (list? v) (not (empty? v)) (= (first v) "Tuple"))
(map search-hk->sx (rest v)))
(:else v))))
;; Evaluate `extra` (extra top-level Haskell bindings) on top of search/src
;; and return binding `name` as plain SX.
(define
search-eval
(fn
(extra name)
(search-hk->sx
(hk-deep-force
(get (hk-eval-program (hk-core (str search/src extra))) name)))))
(define
search-join
(fn
(sep xs)
(cond
((empty? xs) "")
((empty? (rest xs)) (first xs))
(:else (str (first xs) sep (search-join sep (rest xs)))))))
;; Batch many haskell expressions into ONE program evaluation (amortizes the
;; cost of parsing/binding search/src — important under heavy CPU load).
;; `setup` is extra top-level Haskell; `exprs` is a list of expression strings
;; whose results form a single haskell list. Returns the SX list of results.
(define
search-batch
(fn
(setup exprs)
(search-eval
(str setup "\nresult = [" (search-join ", " exprs) "]\n")
"result")))

123
lib/search/tests/boolean.sx Normal file
View File

@@ -0,0 +1,123 @@
;; Phase 2 — query AST + boolean/phrase evaluation (hand-built Query values).
;; Corpus:
;; doc 1 "the quick brown dog" -> the quick brown dog
;; doc 2 "a quick brown fox" -> a quick brown fox
;; doc 3 "the dog barks loudly" -> the dog barks loudly
;; All queries run in ONE program evaluation (search-batch) to stay fast.
(define
search-corpus
"idx = indexDoc 3 \"the dog barks loudly\" (indexDoc 2 \"a quick brown fox\" (indexDoc 1 \"the quick brown dog\" emptyIndex))\n")
(define
bool-cases
(list
(list
"term in two docs"
"evalQuery idx (Term \"quick\")"
(list 1 2))
(list
"term in two docs (the)"
"evalQuery idx (Term \"the\")"
(list 1 3))
(list "term in one doc" "evalQuery idx (Term \"fox\")" (list 2))
(list "term absent" "evalQuery idx (Term \"zzz\")" (list))
(list
"term case-sensitive at AST level"
"evalQuery idx (Term \"QUICK\")"
(list))
(list "term on empty index" "evalQuery emptyIndex (Term \"cat\")" (list))
(list
"and both terms"
"evalQuery idx (And (Term \"quick\") (Term \"brown\"))"
(list 1 2))
(list
"and overlap subset"
"evalQuery idx (And (Term \"the\") (Term \"dog\"))"
(list 1 3))
(list
"and disjoint is empty"
"evalQuery idx (And (Term \"the\") (Term \"fox\"))"
(list))
(list
"and right-nested"
"evalQuery idx (And (Term \"the\") (And (Term \"dog\") (Term \"barks\")))"
(list 3))
(list
"or two singletons"
"evalQuery idx (Or (Term \"fox\") (Term \"barks\"))"
(list 2 3))
(list
"or all docs"
"evalQuery idx (Or (Term \"quick\") (Term \"the\"))"
(list 1 2 3))
(list
"or with absent term"
"evalQuery idx (Or (Term \"fox\") (Term \"zzz\"))"
(list 2))
(list "not term" "evalQuery idx (Not (Term \"the\"))" (list 2))
(list "not term 2" "evalQuery idx (Not (Term \"quick\"))" (list 3))
(list
"and with not"
"evalQuery idx (And (Term \"quick\") (Not (Term \"the\")))"
(list 2))
(list
"double negation"
"evalQuery idx (Not (Not (Term \"fox\")))"
(list 2))
(list
"or of and with term"
"evalQuery idx (Or (And (Term \"the\") (Term \"dog\")) (Term \"fox\"))"
(list 1 2 3))
(list
"phrase adjacent both docs"
"evalQuery idx (Phrase [\"quick\", \"brown\"])"
(list 1 2))
(list
"phrase adjacent one doc"
"evalQuery idx (Phrase [\"brown\", \"dog\"])"
(list 1))
(list
"phrase the quick"
"evalQuery idx (Phrase [\"the\", \"quick\"])"
(list 1))
(list
"phrase dog barks"
"evalQuery idx (Phrase [\"dog\", \"barks\"])"
(list 3))
(list
"phrase non-adjacent empty"
"evalQuery idx (Phrase [\"quick\", \"dog\"])"
(list))
(list
"phrase order matters"
"evalQuery idx (Phrase [\"brown\", \"quick\"])"
(list))
(list
"phrase single term"
"evalQuery idx (Phrase [\"dog\"])"
(list 1 3))
(list
"phrase three terms"
"evalQuery idx (Phrase [\"the\", \"dog\", \"barks\"])"
(list 3))
(list
"and of phrase and term"
"evalQuery idx (And (Phrase [\"quick\", \"brown\"]) (Term \"dog\"))"
(list 1))
(list
"not of phrase"
"evalQuery idx (Not (Phrase [\"quick\", \"brown\"]))"
(list 3))))
(define
bool-results
(search-batch search-corpus (map (fn (c) (nth c 1)) bool-cases)))
(map-indexed
(fn
(i c)
(hk-test (nth c 0) (nth bool-results i) (nth c 2)))
bool-cases)
{:fail hk-test-fail :pass hk-test-pass :fails hk-test-fails}

74
lib/search/tests/fuzzy.sx Normal file
View File

@@ -0,0 +1,74 @@
;; Extension — fuzzy matching via Levenshtein edit distance.
;; Corpus: 1 "color flavor" 2 "colour kitten" 3 "colored"
;; allTerms: color colored colour flavor kitten
(define
fuzzy-setup
"idx = indexDoc 3 \"colored\" (indexDoc 2 \"colour kitten\" (indexDoc 1 \"color flavor\" emptyIndex))\n")
(define
fuzzy-cases
(list
(list
"editDist substitution"
"[editDist \"kitten\" \"sitten\"]"
(list 1))
(list "editDist equal" "[editDist \"abc\" \"abc\"]" (list 0))
(list "editDist deletion" "[editDist \"abc\" \"ab\"]" (list 1))
(list "editDist insertion" "[editDist \"ab\" \"abc\"]" (list 1))
(list "editDist from empty" "[editDist \"\" \"abc\"]" (list 3))
(list "editDist both empty" "[editDist \"\" \"\"]" (list 0))
(list
"editDist classic"
"[editDist \"kitten\" \"sitting\"]"
(list 3))
(list
"editDist color colour"
"[editDist \"color\" \"colour\"]"
(list 1))
(list
"editDist color colored"
"[editDist \"color\" \"colored\"]"
(list 2))
(list
"fuzzy terms dist 1"
"fuzzyTerms 1 \"color\" idx"
(list "color" "colour"))
(list
"fuzzy terms dist 2"
"fuzzyTerms 2 \"color\" idx"
(list "color" "colored" "colour"))
(list "fuzzy terms exact" "fuzzyTerms 0 \"color\" idx" (list "color"))
(list
"fuzzy terms other word"
"fuzzyTerms 1 \"flavour\" idx"
(list "flavor"))
(list
"fuzzy docs dist 1"
"fuzzyDocs 1 \"color\" idx"
(list 1 2))
(list
"fuzzy docs dist 2"
"fuzzyDocs 2 \"color\" idx"
(list 1 2 3))
(list "fuzzy docs none" "fuzzyDocs 1 \"zzzzz\" idx" (list))
(list
"fuzzy rank dist 1"
"fuzzyRankTfIdf 1 \"color\" idx"
(list 1 2))
(list
"fuzzy rank dist 2"
"fuzzyRankTfIdf 2 \"color\" idx"
(list 1 2 3))))
(define
fuzzy-results
(search-batch fuzzy-setup (map (fn (c) (nth c 1)) fuzzy-cases)))
(map-indexed
(fn
(i c)
(hk-test (nth c 0) (nth fuzzy-results i) (nth c 2)))
fuzzy-cases)
{:fail hk-test-fail :pass hk-test-pass :fails hk-test-fails}

Some files were not shown because too many files have changed in this diff Show More