rose-ash/plans/store-on-sx.md

# store-on-sx: Event Sourcing on the SX kernel

> **DRAFT outline.** Foundation subsystem — the durable substrate the other five
> currently fake with in-memory mutable lists. Build this first.

rose-ash needs durable state: every subsystem (feed log, flow store, mod audit,
search index, acl grants) today hand-rolls an in-memory list that vanishes on
restart. They all secretly want the same thing — an append-only event log with
pure projections over it. Event sourcing makes that one substrate: the log is the
source of truth, state is a fold, durability is an IO boundary.

This is **substrate-level**, not a guest language. It lives directly on the SX
kernel's IO-suspension primitives (`perform`/`cek-resume` — the third CEK phase)
so a projection can `perform` a read/append and the kernel persists at the
boundary. Storage backends (in-memory, file, later Postgres/IPFS) are injected.

End-state: an `append`/`fold`/`project`/`snapshot` API with an injectable backend,
optimistic-concurrency on streams, replay + snapshotting, and a subscription hook
so projections (feeds, indices, audit logs) update incrementally. The other
subsystems swap their mutable list for a `store/stream`.

## Status (rolling)

`bash lib/store/conformance.sh` → **0/0** (not yet started)

## Ground rules

- **Scope:** only `lib/store/**` and `plans/store-on-sx.md`. Do **not** edit
  `spec/`, `hosts/`, `shared/`, or `lib/<lang>/`. May **import** the kernel's
  IO-suspension surface (`perform`, the platform IO ops) — verify what's exported
  before relying on it. Do not add host primitives; if a needed durable IO op is
  missing, file it under Blockers (it belongs in `hosts/` / fed-prims, out of scope).
- **Architecture:** an event is `{:stream :seq :type :at :data}`. A log is an
  ordered, append-only vector of events. A projection is `(fold step seed events)`.
  Persistence is an injected backend `{:append :read :snapshot-read :snapshot-write}`;
  the in-memory backend is the test default, real backends wire in unchanged.
- **Determinism:** replay must be pure — same log → same state, always. No clocks
  or randomness inside projections; timestamps live on the event, not the fold.
- **Commits:** one feature per commit. Keep Progress log + tick boxes.

## Architecture sketch

```
Command                                 Read model
  (append stream type data)               (project stream step seed)
        │                                       ▲
        ▼                                       │
lib/store/event.sx                      lib/store/project.sx
  — {:stream :seq :type :at :data}        — fold step seed over a stream
  — stream ids, seq ordering              — incremental: resume from snapshot
        │                                       ▲
        ▼                                       │
lib/store/log.sx                        lib/store/snapshot.sx
  — append-only, optimistic concurrency   — periodic fold checkpoint
  — read range / read-from-seq            — replay = snapshot + tail
        │                                       ▲
        ▼  (perform → backend)                  │
lib/store/backend.sx                    lib/store/api.sx
  — injected {:append :read ...}          — (store/append ...) (store/project ...)
  — mem backend (tests) | file | pg       — (store/subscribe stream fn)
```

## Phase 1 — Log + in-memory backend
- [ ] `lib/store/event.sx` — event record, stream/seq helpers
- [ ] `lib/store/backend.sx` — injectable backend protocol + in-memory impl
- [ ] `lib/store/log.sx` — `append` (optimistic seq), `read`, `read-from`
- [ ] `lib/store/api.sx` — `(store/append ...)`, `(store/events stream)`
- [ ] `lib/store/tests/log.sx` + scoreboard + conformance.sh

## Phase 2 — Projections + subscriptions
- [ ] `lib/store/project.sx` — `(project stream step seed)`, incremental fold
- [ ] subscription hook — projection re-runs on append
- [ ] concurrency conflict surfaced as a real result, not a crash

## Phase 3 — Snapshots + replay
- [ ] `lib/store/snapshot.sx` — checkpoint a projection, replay = snapshot + tail
- [ ] compaction policy; replay determinism tests

## Phase 4 — Durable backend via kernel IO
- [ ] file/log backend driven through `perform` (IO-suspension boundary)
- [ ] crash/restart replay test (mock IO platform)
- [ ] migration notes for swapping mem → durable under a live subsystem

## Consumers (post-foundation, not in scope here)
feed/-log, flow store, mod/audit, search index, acl grant set all become
`store/stream`s. Track the migration in each subsystem's plan, not this one.

## Progress log
(loop fills this in)

## Blockers
(loop fills this in)