rose-ash/plans/feed-on-sx.md

# feed-on-sx: Activity Feeds on APL

Timelines, notifications, activity aggregation. The math is array math: filter, sort,
reduce, scan, outer product. APL is the densest possible expression of feed
composition — a fanout-and-rank pipeline reads as a single line.

rose-ash needs: per-user home timeline, notification feed, activity stream digestion,
backfill for new follows, deduplication across cross-posts. Every operation is an
array-shaped transformation.

End-state: an APL-flavored layer on `lib/apl/` with feed-specific combinators
(`fanout`, `dedupe`, `score`, `rank`), an SX adapter for callers who don't want raw
APL, ACL visibility filtering via `lib/acl/`, federation via fed-sx.

## Status (rolling)

`bash lib/feed/conformance.sh` → **30/30** (Phase 1 complete)

## Ground rules

- **Scope:** only touch `lib/feed/**` and `plans/feed-on-sx.md`. Do **not** edit
  `spec/`, `hosts/`, `shared/`, `lib/apl/**`, or other `lib/<lang>/`. You may
  **import** from `lib/apl/` (public API in `lib/apl/apl.sx`); do **not** modify APL.
- **Shared-file issues** go under "Blockers" with a minimal repro; do not fix here.
- **SX files:** use `sx-tree` MCP tools only.
- **Architecture:** an activity is a small dict (`{:actor :verb :object :at :tags}`); a
  stream is an APL vector of such dicts. Operations are APL primitives lifted onto
  this shape. SX adapter exposes ergonomic API to non-APL callers.
- **Unicode:** raw UTF-8 in `.sx` files. APL glyphs land directly.
- **Commits:** one feature per commit. Keep Progress log updated and tick boxes.

## Architecture sketch

```
Raw activities (any shape)              Per-user view
        │                                    ▲
        ▼                                    │
lib/feed/normalize.sx              lib/feed/timeline.sx
  — {:actor :verb :object             — (timeline user)
     :at :tags} record                — applies filter ∘ rank ∘ take
        │                                    ▲
        ▼                                    │
lib/feed/stream.sx                  lib/feed/rank.sx
  — APL vector of activities          — velocity, recency
  — filter, sort, take                — TF-IDF-ish over :tags
        │                                    ▲
        ▼                                    │
lib/feed/fanout.sx                  lib/feed/dedupe.sx
  — followers vector                  — group by :object
  — activities ∘.× followers          — collapse cross-posts
  — flatten + dedupe
        │
        ▼
lib/feed/api.sx           lib/feed/fed.sx
  — (feed/post activity)    — inbox via fed-sx
  — (feed/timeline user)    — backfill on subscribe
  — (feed/notify user)
```

## Phase 1 — Stream model + basic ops

- [x] `lib/feed/normalize.sx` — activity record schema; coerce arbitrary inputs
- [x] `lib/feed/stream.sx` — APL vector representation; filter by predicate; sort by
  `:at`; take N (`↑`); reverse (`⌽`)
- [x] `lib/feed/api.sx` — `(feed/post activity)`, `(feed/all)`
- [x] `lib/feed/tests/basic.sx` — 30 cases: normalize defaults, filter, sort, take, api
- [x] `lib/feed/scoreboard.{json,md}`
- [x] `lib/feed/conformance.sh`

## Phase 2 — Fanout via outer product

- [ ] follower graph: `followers user → vector of user ids`
- [ ] fanout: activities `∘.×` followers → matrix `(activity, follower)` pairs
- [ ] flatten to inbox events vector
- [ ] dedupe — group by `(actor, verb, object)` collapse to one inbox event per
  receiver
- [ ] `lib/feed/tests/fanout.sx` — 20+ cases: small graph, mutual follow, popular
  actor (high-fanout), cross-post dedupe

## Phase 3 — Aggregation + ranking

- [ ] group-by — `(actor, day) → count` via key-reduce
- [ ] velocity score — recent activity count over window
- [ ] recency score — decay by age
- [ ] composite rank — weighted sum of components
- [ ] top-N per timeline
- [ ] `lib/feed/tests/rank.sx` — 20+ cases: ranking stable on tie, decay shape,
  per-user weighting

## Phase 4 — Visibility filter + federation

- [ ] ACL filter — each candidate activity passed through `(acl/permit? viewer :read
  activity)`
- [ ] fed-sx outbound — local `feed/post` fans out to remote followers' inboxes
- [ ] fed-sx inbound — peer activities arrive at local inbox
- [ ] backfill on subscribe — request peer history, merge into local stream
- [ ] `lib/feed/tests/integration.sx` — federated timeline with ACL applied

## Progress log

- **Phase 1 done (30/30).** Stream = APL rank-1 array whose ravel holds activity
  dicts. `normalize.sx` (record schema + accessors), `stream.sx` (filter via `/`
  compress, sort via `⍋` grade-up [stable], take via `↑`, reverse via `⌽`,
  by-actor/verb/object/since predicates), `api.sx` (mutable log: post/all/reset!/size).
  Substrate: `apl-compress`, `apl-grade-up`, `apl-take`, `apl-reverse`, `make-array`.
  Grade-up returns 1-based indices (⎕IO=1), is stable on ties → deterministic sort.

## Blockers

(none)

## Notes for next iteration

- sx-tree MCP tools take `file:` NOT `path:` (CLAUDE.md is stale). Wrong key →
  `Yojson Type_error("Expected string, got null")`. Looks like a broken binary, isn't.
- sx_server binary lives in main repo: `/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe`
  (worktree has no `_build`). conformance.sh already points there with relative fallback.
- Phase 2 substrate verified available: `apl-outer` (∘.×), `apl-member` (∊),
  `apl-unique`, `apl-iota` (1-based).