persist: durable backend over the perform IO boundary + 15 tests
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Has been cancelled
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Has been cancelled
durable.sx: io-backend with an injectable transport — persist/durable-backend
performs each op as {:op "persist/..." :args (...)} (kernel suspends, host
resumes); persist/mock-durable services via persist/serve over an in-memory
disk. Identical request shapes mean the whole facet/projection/snapshot/
compaction stack runs unchanged on the durable backend. Crash/restart replay
recovers log+kv+snapshot. 91/91.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -42,7 +42,7 @@ read models (feeds, indices, audit logs) update incrementally.
|
||||
|
||||
## Status (rolling)
|
||||
|
||||
`bash lib/persist/conformance.sh` → **76/76** (Phases 1–3 done)
|
||||
`bash lib/persist/conformance.sh` → **91/91** (Phases 1–3 done, Phase 4 in progress)
|
||||
|
||||
## Ground rules
|
||||
|
||||
@@ -103,7 +103,7 @@ lib/persist/backend.sx lib/persist/api.sx
|
||||
- [x] compaction policy; replay-determinism tests
|
||||
|
||||
## Phase 4 — Durable backends via kernel IO
|
||||
- [ ] file/log backend driven through `perform` (IO-suspension boundary)
|
||||
- [x] file/log backend driven through `perform` (IO-suspension boundary)
|
||||
- [ ] blob backend interface (store ref/CID; bytes live in artdag/IPFS)
|
||||
- [ ] crash/restart replay test (mock IO platform)
|
||||
- [ ] migration notes for swapping mem → durable under a live subsystem
|
||||
@@ -113,6 +113,16 @@ feed/-log, flow store, mod/audit, search index, acl grants, identity sessions al
|
||||
become `persist` log or kv. Track each migration in that subsystem's plan.
|
||||
|
||||
## Progress log
|
||||
- **Phase 4a (91/91).** `durable.sx` — a backend whose every op crosses the
|
||||
kernel IO boundary via `(perform {:op "persist/..." :args (...)})`. The
|
||||
transport is injectable: `persist/durable-backend` uses the kernel's
|
||||
`perform` (suspends; host resumes); `persist/mock-durable` uses
|
||||
`persist/serve` over an in-memory disk. `persist/serve` is the reference host
|
||||
+ the mock-IO harness. Because the request shapes are identical, the ENTIRE
|
||||
facet stack (log/kv/project/snapshot/compaction) runs unchanged on
|
||||
mock-durable — verified. Crash/restart (drop backend, keep disk) recovers log
|
||||
+ kv + snapshot by replay; seq counter continues. 15 tests. See Blockers for
|
||||
why end-to-end perform suspension isn't exercised under sx_server.exe.
|
||||
- **Phase 3b (76/76) — Phase 3 complete.** Backend refactor: `last-seq` is now
|
||||
a monotonic per-stream high-water mark (backend `seqs` dict), not physical
|
||||
length, so a compacted log keeps assigning climbing seqs. Added backend
|
||||
@@ -152,6 +162,17 @@ become `persist` log or kv. Track each migration in that subsystem's plan.
|
||||
compared lists with list/nth.
|
||||
|
||||
## Blockers
|
||||
- **Phase 4 perform-suspension not exercised end-to-end under sx_server.exe (by
|
||||
design, not a bug).** The CEK suspension primitives (`cek-step-loop`,
|
||||
`cek-resume`, `cek-suspended?`, `cek-io-request`) and a settable SX-level IO
|
||||
hook are only bound by the `run_tests` OCaml binary (out of scope: hosts/, and
|
||||
sx_build is forbidden). Under `sx_server.exe`, an unhandled `perform` resolves
|
||||
through the OCaml io-request/io-response stdin bridge (production path) — not
|
||||
callable from the pure-eval conformance harness. Resolution: the durable
|
||||
backend's transport is injectable, so the production path is one line
|
||||
`(perform req)` (kernel-handled) and ALL durable logic is tested through the
|
||||
mock transport (`persist/serve` over an in-memory disk). The single untested
|
||||
line is the kernel primitive itself. No host primitive needed; nothing to fix.
|
||||
- **Not a blocker, a testing convention:** `map` returns an array-backed list
|
||||
that is NOT `equal?` to a `(list ...)` cons-literal (two `map` results do
|
||||
compare equal to each other). When asserting list-shaped results against a
|
||||
|
||||
Reference in New Issue
Block a user