Files

Test, Build, and Deploy / test-build-deploy (push) Failing after 19s

Details

plans: five rose-ash subsystem plans + three loop briefings

Plans for acl-on-sx (Datalog), flow-on-sx (Scheme), feed-on-sx (APL),
mod-on-sx (Prolog), search-on-sx (Haskell). Each is a 4-phase queue
sitting on its respective guest language, targeting rose-ash needs:
access control, durable workflows, activity feeds, moderation, search.
Federation extension in Phase 4 of each (plugs into fed-sx).

Briefings for the three loops we're kicking off now: acl-loop,
flow-loop, feed-loop. mod-sx and search-sx briefings will follow
once the first three have surfaced any shared infrastructure
worth extracting to lib/guest/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

2026-06-06 15:55:39 +00:00

4.9 KiB

Raw Blame History

flow-on-sx: Durable DAG Workflows on Scheme

rose-ash needs workflows that survive restarts: content pipelines (write → review → publish → federate), scheduled jobs (digest emails), multi-step user flows (signup, confirm, onboard). art-dag is the precedent — DAG-of-tasks with pause/resume at IO boundaries.

Scheme's call/cc + delimited continuations make pause/resume natural: a suspend captures the continuation, serializes it as part of the flow record, and resume re-enters at exactly that point. No state-machine bookkeeping by hand. R7RS-small is already at 2644/2644 (see kernel/architecture status).

End-state: a Scheme-on-SX layer over the existing scheme runtime, with combinators for sequence/parallel/branch/retry/timeout/suspend, persistent flow store, and a federation extension via fed-sx for remote-node execution.

Status (rolling)

bash lib/flow/conformance.sh → 0/0 (not yet started)

Ground rules

Scope: only touch lib/flow/** and plans/flow-on-sx.md. Do not edit spec/, hosts/, shared/, lib/scheme/**, or other lib/<lang>/. You may import from lib/scheme/ (public API via lib/scheme/scheme.sx); do not modify Scheme.
Shared-file issues go under "Blockers" with a minimal repro; do not fix here.
SX files: use sx-tree MCP tools only.
Architecture: flow combinators are Scheme macros + procedures. Runtime is a driver loop that walks the flow graph and invokes call/cc at suspend points. Persistence layer serializes the continuation + open file/socket placeholders are forbidden (continuations must be resumable across process restart).
art-dag awareness: read plans/art-dag* if it exists for design lineage; do not import code.
Commits: one feature per commit. Keep Progress log updated and tick boxes.

Architecture sketch

(defflow publish
  (sequence
    (write-content)
    (parallel
      (review)
      (spell-check))
    (cond approved?
      (sequence (publish) (federate))
      (notify-author))))
        │
        ▼
lib/flow/spec.sx           lib/flow/runtime.sx          lib/flow/store.sx
  — defflow                  — driver loop                 — append-only flow log
  — sequence/parallel        — node dispatch               — checkpoint serialize
  — cond/retry/timeout       — call/cc at suspend          — restart loader
  — suspend/resume                  │                            │
                                    ▼                            ▼
                            lib/flow/api.sx              lib/flow/remote.sx
                              — (flow/start name args)     — fed-sx adapter
                              — (flow/resume id value)     — node-on-peer execution
                              — (flow/cancel id)           — failure handling

Phase 1 — Declarative DAG + sequential execution

lib/flow/spec.sx — defflow macro, sequence combinator
node = Scheme thunk; output threads to next node (data flow)
parallel combinator (sequential semantics for now — TRUE parallelism in Phase 3)
runtime executes a flow synchronously, returns final value
lib/flow/api.sx — (flow/start name args) entry point
lib/flow/tests/basic.sx — 15+ cases: linear sequence, nested sequences, data flow between nodes, parallel-with-join
lib/flow/scoreboard.{json,md}
lib/flow/conformance.sh

Phase 2 — Control flow + error handling

cond combinator — predicate selects branch
retry n [backoff] — re-runs node up to n times on exception
timeout ms — bounds node execution
try-catch — exception handler with reified error
error model — exceptions vs explicit (fail :reason ...) results
lib/flow/tests/control.sx — 25+ cases: each combinator + composition

Phase 3 — Suspend / resume (the showcase)

(suspend reason) — call/cc captures continuation, returns flow-id to caller
lib/flow/store.sx — serialize flow state (continuation + open vars)
(flow/resume id value) — load continuation, inject value, re-enter
(flow/cancel id) — explicit termination
crash recovery — on restart, scan store for paused flows, mark resumable
lib/flow/tests/suspend.sx — pause-resume scenarios, cancellation, "restart" scenarios (simulated by re-loading store)

Phase 4 — Distributed nodes via fed-sx

(remote-node addr fn args) — execute node on a federation peer
failure semantics — retry on different peer, fall through to local
persistence across instances — flow state replicates via fed-sx
handoff — flow started here can resume on a peer if the local instance is down
lib/flow/tests/distributed.sx — federated flow scenarios (mock fed-sx in tests)

Progress log

(loop fills this in)

Blockers