giles
c3a0727645
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 19s
Plans for acl-on-sx (Datalog), flow-on-sx (Scheme), feed-on-sx (APL), mod-on-sx (Prolog), search-on-sx (Haskell). Each is a 4-phase queue sitting on its respective guest language, targeting rose-ash needs: access control, durable workflows, activity feeds, moderation, search. Federation extension in Phase 4 of each (plugs into fed-sx). Briefings for the three loops we're kicking off now: acl-loop, flow-loop, feed-loop. mod-sx and search-sx briefings will follow once the first three have surfaced any shared infrastructure worth extracting to lib/guest/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
106 lines
4.4 KiB
Markdown
106 lines
4.4 KiB
Markdown
# feed-on-sx: Activity Feeds on APL
|
||
|
||
Timelines, notifications, activity aggregation. The math is array math: filter, sort,
|
||
reduce, scan, outer product. APL is the densest possible expression of feed
|
||
composition — a fanout-and-rank pipeline reads as a single line.
|
||
|
||
rose-ash needs: per-user home timeline, notification feed, activity stream digestion,
|
||
backfill for new follows, deduplication across cross-posts. Every operation is an
|
||
array-shaped transformation.
|
||
|
||
End-state: an APL-flavored layer on `lib/apl/` with feed-specific combinators
|
||
(`fanout`, `dedupe`, `score`, `rank`), an SX adapter for callers who don't want raw
|
||
APL, ACL visibility filtering via `lib/acl/`, federation via fed-sx.
|
||
|
||
## Status (rolling)
|
||
|
||
`bash lib/feed/conformance.sh` → **0/0** (not yet started)
|
||
|
||
## Ground rules
|
||
|
||
- **Scope:** only touch `lib/feed/**` and `plans/feed-on-sx.md`. Do **not** edit
|
||
`spec/`, `hosts/`, `shared/`, `lib/apl/**`, or other `lib/<lang>/`. You may
|
||
**import** from `lib/apl/` (public API in `lib/apl/apl.sx`); do **not** modify APL.
|
||
- **Shared-file issues** go under "Blockers" with a minimal repro; do not fix here.
|
||
- **SX files:** use `sx-tree` MCP tools only.
|
||
- **Architecture:** an activity is a small dict (`{:actor :verb :object :at :tags}`); a
|
||
stream is an APL vector of such dicts. Operations are APL primitives lifted onto
|
||
this shape. SX adapter exposes ergonomic API to non-APL callers.
|
||
- **Unicode:** raw UTF-8 in `.sx` files. APL glyphs land directly.
|
||
- **Commits:** one feature per commit. Keep Progress log updated and tick boxes.
|
||
|
||
## Architecture sketch
|
||
|
||
```
|
||
Raw activities (any shape) Per-user view
|
||
│ ▲
|
||
▼ │
|
||
lib/feed/normalize.sx lib/feed/timeline.sx
|
||
— {:actor :verb :object — (timeline user)
|
||
:at :tags} record — applies filter ∘ rank ∘ take
|
||
│ ▲
|
||
▼ │
|
||
lib/feed/stream.sx lib/feed/rank.sx
|
||
— APL vector of activities — velocity, recency
|
||
— filter, sort, take — TF-IDF-ish over :tags
|
||
│ ▲
|
||
▼ │
|
||
lib/feed/fanout.sx lib/feed/dedupe.sx
|
||
— followers vector — group by :object
|
||
— activities ∘.× followers — collapse cross-posts
|
||
— flatten + dedupe
|
||
│
|
||
▼
|
||
lib/feed/api.sx lib/feed/fed.sx
|
||
— (feed/post activity) — inbox via fed-sx
|
||
— (feed/timeline user) — backfill on subscribe
|
||
— (feed/notify user)
|
||
```
|
||
|
||
## Phase 1 — Stream model + basic ops
|
||
|
||
- [ ] `lib/feed/normalize.sx` — activity record schema; coerce arbitrary inputs
|
||
- [ ] `lib/feed/stream.sx` — APL vector representation; filter by predicate; sort by
|
||
`:at`; take N (`↑`); reverse (`⌽`)
|
||
- [ ] `lib/feed/api.sx` — `(feed/post activity)`, `(feed/all)`
|
||
- [ ] `lib/feed/tests/basic.sx` — 15+ cases: post, query, filter, sort
|
||
- [ ] `lib/feed/scoreboard.{json,md}`
|
||
- [ ] `lib/feed/conformance.sh`
|
||
|
||
## Phase 2 — Fanout via outer product
|
||
|
||
- [ ] follower graph: `followers user → vector of user ids`
|
||
- [ ] fanout: activities `∘.×` followers → matrix `(activity, follower)` pairs
|
||
- [ ] flatten to inbox events vector
|
||
- [ ] dedupe — group by `(actor, verb, object)` collapse to one inbox event per
|
||
receiver
|
||
- [ ] `lib/feed/tests/fanout.sx` — 20+ cases: small graph, mutual follow, popular
|
||
actor (high-fanout), cross-post dedupe
|
||
|
||
## Phase 3 — Aggregation + ranking
|
||
|
||
- [ ] group-by — `(actor, day) → count` via key-reduce
|
||
- [ ] velocity score — recent activity count over window
|
||
- [ ] recency score — decay by age
|
||
- [ ] composite rank — weighted sum of components
|
||
- [ ] top-N per timeline
|
||
- [ ] `lib/feed/tests/rank.sx` — 20+ cases: ranking stable on tie, decay shape,
|
||
per-user weighting
|
||
|
||
## Phase 4 — Visibility filter + federation
|
||
|
||
- [ ] ACL filter — each candidate activity passed through `(acl/permit? viewer :read
|
||
activity)`
|
||
- [ ] fed-sx outbound — local `feed/post` fans out to remote followers' inboxes
|
||
- [ ] fed-sx inbound — peer activities arrive at local inbox
|
||
- [ ] backfill on subscribe — request peer history, merge into local stream
|
||
- [ ] `lib/feed/tests/integration.sx` — federated timeline with ACL applied
|
||
|
||
## Progress log
|
||
|
||
(loop fills this in)
|
||
|
||
## Blockers
|
||
|
||
(loop fills this in)
|