persist: compaction — drop snapshotted prefix, monotonic seq + 11 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m0s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 1m0s
Backend now tracks last-seq as a monotonic high-water mark (survives truncation) and exposes :truncate-through. compaction.sx: persist/compact checkpoints then drops events with seq <= snapshot seq; should-compact?/ maybe-compact give an explicit every-N policy. Determinism: post-compaction replay value == uncompacted full replay. Phase 3 complete, 76/76. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -42,7 +42,7 @@ read models (feeds, indices, audit logs) update incrementally.
|
||||
|
||||
## Status (rolling)
|
||||
|
||||
`bash lib/persist/conformance.sh` → **54/54** (Phases 1–2 done)
|
||||
`bash lib/persist/conformance.sh` → **76/76** (Phases 1–3 done)
|
||||
|
||||
## Ground rules
|
||||
|
||||
@@ -100,7 +100,7 @@ lib/persist/backend.sx lib/persist/api.sx
|
||||
|
||||
## Phase 3 — Snapshots + replay
|
||||
- [x] `snapshot.sx` — checkpoint a projection; replay = snapshot + tail
|
||||
- [ ] compaction policy; replay-determinism tests
|
||||
- [x] compaction policy; replay-determinism tests
|
||||
|
||||
## Phase 4 — Durable backends via kernel IO
|
||||
- [ ] file/log backend driven through `perform` (IO-suspension boundary)
|
||||
@@ -113,6 +113,15 @@ feed/-log, flow store, mod/audit, search index, acl grants, identity sessions al
|
||||
become `persist` log or kv. Track each migration in that subsystem's plan.
|
||||
|
||||
## Progress log
|
||||
- **Phase 3b (76/76) — Phase 3 complete.** Backend refactor: `last-seq` is now
|
||||
a monotonic per-stream high-water mark (backend `seqs` dict), not physical
|
||||
length, so a compacted log keeps assigning climbing seqs. Added backend
|
||||
`:truncate-through` + `persist/truncate`. `compaction.sx` — `persist/compact`
|
||||
checkpoints then drops events with seq <= snapshot seq; `should-compact?`/
|
||||
`maybe-compact` give an explicit "compact every N tail events" policy. 11
|
||||
tests: post-compaction replay value == uncompacted full replay (determinism),
|
||||
seq continuity after truncation, idempotence. `persist/count` = physical
|
||||
stored count (shrinks on compaction) vs `persist/last-seq` = logical.
|
||||
- **Phase 3a (65/65).** `snapshot.sx` — a snapshot is a projection state
|
||||
`{:value :seq}` stored in the kv facet under `snapshot/<name>`.
|
||||
`persist/checkpoint` replays + saves; `persist/replay` = snapshot + tail.
|
||||
|
||||
Reference in New Issue
Block a user