persist: compaction — drop snapshotted prefix, monotonic seq + 11 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m0s

Backend now tracks last-seq as a monotonic high-water mark (survives
truncation) and exposes :truncate-through. compaction.sx: persist/compact
checkpoints then drops events with seq <= snapshot seq; should-compact?/
maybe-compact give an explicit every-N policy. Determinism: post-compaction
replay value == uncompacted full replay. Phase 3 complete, 76/76.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-06 18:42:06 +00:00
parent b0874b1282
commit aff7d1e84f
8 changed files with 217 additions and 21 deletions

View File

@@ -42,7 +42,7 @@ read models (feeds, indices, audit logs) update incrementally.
## Status (rolling)
`bash lib/persist/conformance.sh`**54/54** (Phases 12 done)
`bash lib/persist/conformance.sh`**76/76** (Phases 13 done)
## Ground rules
@@ -100,7 +100,7 @@ lib/persist/backend.sx lib/persist/api.sx
## Phase 3 — Snapshots + replay
- [x] `snapshot.sx` — checkpoint a projection; replay = snapshot + tail
- [ ] compaction policy; replay-determinism tests
- [x] compaction policy; replay-determinism tests
## Phase 4 — Durable backends via kernel IO
- [ ] file/log backend driven through `perform` (IO-suspension boundary)
@@ -113,6 +113,15 @@ feed/-log, flow store, mod/audit, search index, acl grants, identity sessions al
become `persist` log or kv. Track each migration in that subsystem's plan.
## Progress log
- **Phase 3b (76/76) — Phase 3 complete.** Backend refactor: `last-seq` is now
a monotonic per-stream high-water mark (backend `seqs` dict), not physical
length, so a compacted log keeps assigning climbing seqs. Added backend
`:truncate-through` + `persist/truncate`. `compaction.sx``persist/compact`
checkpoints then drops events with seq <= snapshot seq; `should-compact?`/
`maybe-compact` give an explicit "compact every N tail events" policy. 11
tests: post-compaction replay value == uncompacted full replay (determinism),
seq continuity after truncation, idempotence. `persist/count` = physical
stored count (shrinks on compaction) vs `persist/last-seq` = logical.
- **Phase 3a (65/65).** `snapshot.sx` — a snapshot is a projection state
`{:value :seq}` stored in the kv facet under `snapshot/<name>`.
`persist/checkpoint` replays + saves; `persist/replay` = snapshot + tail.