plans: five rose-ash subsystem plans + three loop briefings
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 19s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 19s
Plans for acl-on-sx (Datalog), flow-on-sx (Scheme), feed-on-sx (APL), mod-on-sx (Prolog), search-on-sx (Haskell). Each is a 4-phase queue sitting on its respective guest language, targeting rose-ash needs: access control, durable workflows, activity feeds, moderation, search. Federation extension in Phase 4 of each (plugs into fed-sx). Briefings for the three loops we're kicking off now: acl-loop, flow-loop, feed-loop. mod-sx and search-sx briefings will follow once the first three have surfaced any shared infrastructure worth extracting to lib/guest/. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
108
plans/flow-on-sx.md
Normal file
108
plans/flow-on-sx.md
Normal file
@@ -0,0 +1,108 @@
|
||||
# flow-on-sx: Durable DAG Workflows on Scheme
|
||||
|
||||
rose-ash needs workflows that survive restarts: content pipelines (write → review →
|
||||
publish → federate), scheduled jobs (digest emails), multi-step user flows (signup,
|
||||
confirm, onboard). art-dag is the precedent — DAG-of-tasks with pause/resume at IO
|
||||
boundaries.
|
||||
|
||||
Scheme's `call/cc` + delimited continuations make pause/resume natural: a `suspend`
|
||||
captures the continuation, serializes it as part of the flow record, and `resume`
|
||||
re-enters at exactly that point. No state-machine bookkeeping by hand. R7RS-small is
|
||||
already at 2644/2644 (see kernel/architecture status).
|
||||
|
||||
End-state: a Scheme-on-SX layer over the existing scheme runtime, with combinators
|
||||
for sequence/parallel/branch/retry/timeout/suspend, persistent flow store, and a
|
||||
federation extension via fed-sx for remote-node execution.
|
||||
|
||||
## Status (rolling)
|
||||
|
||||
`bash lib/flow/conformance.sh` → **0/0** (not yet started)
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/flow/**` and `plans/flow-on-sx.md`. Do **not** edit
|
||||
`spec/`, `hosts/`, `shared/`, `lib/scheme/**`, or other `lib/<lang>/`. You may
|
||||
**import** from `lib/scheme/` (public API via `lib/scheme/scheme.sx`); do **not**
|
||||
modify Scheme.
|
||||
- **Shared-file issues** go under "Blockers" with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** flow combinators are Scheme macros + procedures. Runtime is a
|
||||
driver loop that walks the flow graph and invokes `call/cc` at `suspend` points.
|
||||
Persistence layer serializes the continuation + open file/socket placeholders are
|
||||
forbidden (continuations must be resumable across process restart).
|
||||
- **art-dag awareness:** read `plans/art-dag*` if it exists for design lineage; do not
|
||||
import code.
|
||||
- **Commits:** one feature per commit. Keep Progress log updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
(defflow publish
|
||||
(sequence
|
||||
(write-content)
|
||||
(parallel
|
||||
(review)
|
||||
(spell-check))
|
||||
(cond approved?
|
||||
(sequence (publish) (federate))
|
||||
(notify-author))))
|
||||
│
|
||||
▼
|
||||
lib/flow/spec.sx lib/flow/runtime.sx lib/flow/store.sx
|
||||
— defflow — driver loop — append-only flow log
|
||||
— sequence/parallel — node dispatch — checkpoint serialize
|
||||
— cond/retry/timeout — call/cc at suspend — restart loader
|
||||
— suspend/resume │ │
|
||||
▼ ▼
|
||||
lib/flow/api.sx lib/flow/remote.sx
|
||||
— (flow/start name args) — fed-sx adapter
|
||||
— (flow/resume id value) — node-on-peer execution
|
||||
— (flow/cancel id) — failure handling
|
||||
```
|
||||
|
||||
## Phase 1 — Declarative DAG + sequential execution
|
||||
|
||||
- [ ] `lib/flow/spec.sx` — `defflow` macro, `sequence` combinator
|
||||
- [ ] node = Scheme thunk; output threads to next node (data flow)
|
||||
- [ ] `parallel` combinator (sequential semantics for now — TRUE parallelism in Phase 3)
|
||||
- [ ] runtime executes a flow synchronously, returns final value
|
||||
- [ ] `lib/flow/api.sx` — `(flow/start name args)` entry point
|
||||
- [ ] `lib/flow/tests/basic.sx` — 15+ cases: linear sequence, nested sequences,
|
||||
data flow between nodes, parallel-with-join
|
||||
- [ ] `lib/flow/scoreboard.{json,md}`
|
||||
- [ ] `lib/flow/conformance.sh`
|
||||
|
||||
## Phase 2 — Control flow + error handling
|
||||
|
||||
- [ ] `cond` combinator — predicate selects branch
|
||||
- [ ] `retry n [backoff]` — re-runs node up to n times on exception
|
||||
- [ ] `timeout ms` — bounds node execution
|
||||
- [ ] `try-catch` — exception handler with reified error
|
||||
- [ ] error model — exceptions vs explicit `(fail :reason ...)` results
|
||||
- [ ] `lib/flow/tests/control.sx` — 25+ cases: each combinator + composition
|
||||
|
||||
## Phase 3 — Suspend / resume (the showcase)
|
||||
|
||||
- [ ] `(suspend reason)` — `call/cc` captures continuation, returns flow-id to caller
|
||||
- [ ] `lib/flow/store.sx` — serialize flow state (continuation + open vars)
|
||||
- [ ] `(flow/resume id value)` — load continuation, inject value, re-enter
|
||||
- [ ] `(flow/cancel id)` — explicit termination
|
||||
- [ ] crash recovery — on restart, scan store for paused flows, mark resumable
|
||||
- [ ] `lib/flow/tests/suspend.sx` — pause-resume scenarios, cancellation, "restart"
|
||||
scenarios (simulated by re-loading store)
|
||||
|
||||
## Phase 4 — Distributed nodes via fed-sx
|
||||
|
||||
- [ ] `(remote-node addr fn args)` — execute node on a federation peer
|
||||
- [ ] failure semantics — retry on different peer, fall through to local
|
||||
- [ ] persistence across instances — flow state replicates via fed-sx
|
||||
- [ ] handoff — flow started here can resume on a peer if the local instance is down
|
||||
- [ ] `lib/flow/tests/distributed.sx` — federated flow scenarios (mock fed-sx in tests)
|
||||
|
||||
## Progress log
|
||||
|
||||
(loop fills this in)
|
||||
|
||||
## Blockers
|
||||
|
||||
(loop fills this in)
|
||||
Reference in New Issue
Block a user