rose-ash/plans/feed-on-sx.md

# feed-on-sx: Activity Feeds on APL

Timelines, notifications, activity aggregation. The math is array math: filter, sort,
reduce, scan, outer product. APL is the densest possible expression of feed
composition — a fanout-and-rank pipeline reads as a single line.

rose-ash needs: per-user home timeline, notification feed, activity stream digestion,
backfill for new follows, deduplication across cross-posts. Every operation is an
array-shaped transformation.

End-state: an APL-flavored layer on `lib/apl/` with feed-specific combinators
(`fanout`, `dedupe`, `score`, `rank`), an SX adapter for callers who don't want raw
APL, ACL visibility filtering via `lib/acl/`, federation via fed-sx.

## Status (rolling)

`bash lib/feed/conformance.sh` → **59/59** (Phases 1–2 complete)

## Ground rules

- **Scope:** only touch `lib/feed/**` and `plans/feed-on-sx.md`. Do **not** edit
  `spec/`, `hosts/`, `shared/`, `lib/apl/**`, or other `lib/<lang>/`. You may
  **import** from `lib/apl/` (public API in `lib/apl/apl.sx`); do **not** modify APL.
- **Shared-file issues** go under "Blockers" with a minimal repro; do not fix here.
- **SX files:** use `sx-tree` MCP tools only.
- **Architecture:** an activity is a small dict (`{:actor :verb :object :at :tags}`); a
  stream is an APL vector of such dicts. Operations are APL primitives lifted onto
  this shape. SX adapter exposes ergonomic API to non-APL callers.
- **Unicode:** raw UTF-8 in `.sx` files. APL glyphs land directly.
- **Commits:** one feature per commit. Keep Progress log updated and tick boxes.

## Architecture sketch

```
Raw activities (any shape)              Per-user view
        │                                    ▲
        ▼                                    │
lib/feed/normalize.sx              lib/feed/timeline.sx
  — {:actor :verb :object             — (timeline user)
     :at :tags} record                — applies filter ∘ rank ∘ take
        │                                    ▲
        ▼                                    │
lib/feed/stream.sx                  lib/feed/rank.sx
  — APL vector of activities          — velocity, recency
  — filter, sort, take                — TF-IDF-ish over :tags
        │                                    ▲
        ▼                                    │
lib/feed/fanout.sx                  lib/feed/dedupe.sx
  — followers vector                  — group by :object
  — activities ∘.× followers          — collapse cross-posts
  — flatten + dedupe
        │
        ▼
lib/feed/api.sx           lib/feed/fed.sx
  — (feed/post activity)    — inbox via fed-sx
  — (feed/timeline user)    — backfill on subscribe
  — (feed/notify user)
```

## Phase 1 — Stream model + basic ops

- [x] `lib/feed/normalize.sx` — activity record schema; coerce arbitrary inputs
- [x] `lib/feed/stream.sx` — APL vector representation; filter by predicate; sort by
  `:at`; take N (`↑`); reverse (`⌽`)
- [x] `lib/feed/api.sx` — `(feed/post activity)`, `(feed/all)`
- [x] `lib/feed/tests/basic.sx` — 30 cases: normalize defaults, filter, sort, take, api
- [x] `lib/feed/scoreboard.{json,md}`
- [x] `lib/feed/conformance.sh`

## Phase 2 — Fanout via outer product

- [x] follower graph: `followers user → vector of user ids` (`feed/follow-graph`,
  `feed/followers`; graph = `{followee -> (followers)}` dict)
- [x] fanout: activities `∘.×` audience → matrix via `apl-outer feed/-mk-event`
- [x] flatten to inbox events vector (`feed/-flatten` rank-2 → rank-1)
- [x] dedupe — `feed/dedupe-inbox` by `(to, actor, verb, object)`; also
  `feed/dedupe-activities` `(actor verb object)` and `feed/dedupe-collapse`
  `(verb object)` for cross-actor likes
- [x] `lib/feed/tests/fanout.sx` — 29 cases: small graph, mutual follow, star
  (high-fanout), empty graph, unfollowed actor, cross-post dedupe

## Phase 3 — Aggregation + ranking

- [ ] group-by — `(actor, day) → count` via key-reduce
- [ ] velocity score — recent activity count over window
- [ ] recency score — decay by age
- [ ] composite rank — weighted sum of components
- [ ] top-N per timeline
- [ ] `lib/feed/tests/rank.sx` — 20+ cases: ranking stable on tie, decay shape,
  per-user weighting

## Phase 4 — Visibility filter + federation

- [ ] ACL filter — each candidate activity passed through `(acl/permit? viewer :read
  activity)`
- [ ] fed-sx outbound — local `feed/post` fans out to remote followers' inboxes
- [ ] fed-sx inbound — peer activities arrive at local inbox
- [ ] backfill on subscribe — request peer history, merge into local stream
- [ ] `lib/feed/tests/integration.sx` — federated timeline with ACL applied

## Progress log

- **Phase 1 done (30/30).** Stream = APL rank-1 array whose ravel holds activity
  dicts. `normalize.sx` (record schema + accessors), `stream.sx` (filter via `/`
  compress, sort via `⍋` grade-up [stable], take via `↑`, reverse via `⌽`,
  by-actor/verb/object/since predicates), `api.sx` (mutable log: post/all/reset!/size).
  Substrate: `apl-compress`, `apl-grade-up`, `apl-take`, `apl-reverse`, `make-array`.
  Grade-up returns 1-based indices (⎕IO=1), is stable on ties → deterministic sort.
- **Phase 2 done (59/59 total).** `fanout.sx` (graph + `apl-outer` showcase),
  `dedupe.sx` (per-key dedupe, first-wins stable). Key APL gotcha: `scalar?` is
  true for ANY dict and `disclose` nils a non-array dict, so an apl-outer combiner
  MUST `enclose` its event dict — apl-outer discloses it back intact. `apl-unique`
  preserves first-occurrence order; dict `keys` order is NOT stable, so
  `feed/audience` sorts (else recipient ordering flakes). `apl-compress` needs a
  rank-1 array, so the (activity×follower) matrix is flattened to its ravel before
  the edge-guard filter.

(none)

## Notes for next iteration

- sx-tree MCP tools take `file:` NOT `path:` (CLAUDE.md is stale). Wrong key →
  `Yojson Type_error("Expected string, got null")`. Looks like a broken binary, isn't.
- sx_server binary lives in main repo: `/root/rose-ash/hosts/ocaml/_build/default/bin/sx_server.exe`
  (worktree has no `_build`). conformance.sh already points there with relative fallback.
- Phase 2 substrate verified available: `apl-outer` (∘.×), `apl-member` (∊),
  `apl-unique`, `apl-iota` (1-based).