rose-ash/next/flow/README.md

flow-on-erlang — durable workflows in the fed-sx runtime

A native Erlang-on-SX port of the Scheme flow engine (lib/flow), so the fed-sx kernel can fan arriving activities out into durable, branching, multi-step business flows in its own runtime — no cross-guest FFI, no marshalling, no Scheme dependency. The seed of a real engine that can later supersede the Scheme one for substrate use.

Run the suite: bash next/flow/conformance.sh → engine conformance.

Model

A flow is an Erlang fun(Ctx) -> Ctx. Combinators (flow_spec) compose flows; user code stays value-level (the functions you hand to flow_node/branch/… take and return plain values). A flow that ignores its input is a thunk; composition is function composition.

F = flow_spec:sequence([
      flow_spec:flow_node(fun(Draft) -> Draft + 1 end),
      flow_spec:branch(fun(P) -> P >= 3 end,
                       flow_spec:flow_const(ok),
                       flow_spec:flow_const(rejected))]),
flow:run(F, 2)            %% => {flow_done, ok}

Durability — deterministic replay

Same semantics as the Scheme engine: a flow re-runs from the top on every resume; effects/non-determinism go through flow:suspend/1, whose resolved values are logged; an already-resolved suspend replays its logged value, and the first unresolved suspend short-circuits back to the driver. The persisted state is the replay log — plain [{Tag, Value}] data — so nothing live (no continuation, no process) is ever serialized; an instance survives restart by re-driving its named flow against its log.

flow_store:register_flow(publish, F),
{ok, Id, R} = flow_store:start(publish, Draft),  %% R = {flow_suspended, Tag} | {flow_done, V}
%% ... driver performs the effect for Tag, then:
flow_store:resume(Id, EffectResult)              %% re-drives; completes or suspends again

Why railway threading instead of call/cc + a global

The Scheme engine uses an escape-only call/cc plus a mutable global replay log. This Erlang-on-SX runtime can't do either, and has a third sharp edge:

• No re-enterable continuation — but suspend only needs to escape, which Erlang throw could do …
• … except a blocking receive / gen_server:call inside a try deadlocks the cooperative scheduler. So suspend must not consult the log via a registry process while inside a try.
• No process dictionary — so there is no ambient per-process slot to stash the replay log in.

The resolution: thread the replay log through a railway-style context and make suspend short-circuit (like a fail value) rather than throw. No ambient state, no throw, no gen_server in the hot path — purely functional, which sidesteps all three constraints. The driver (flow_store) is the only stateful part, and it calls the pure flow:drive/3 from inside handle_call, never wrapping a blocking receive.

A Ctx is {flow_cont, Value, Log} (running) or {flow_susp, Tag, Log} (short-circuited); every combinator passes a suspended context straight through.

Modules

Module	Role
flow.erl	pure replay driver: drive/3, run/2, suspend/1, the Ctx constructors/accessors
flow_spec.erl	combinator algebra: leaves, sequence/parallel/map_flow, flow_while/flow_until, branch, railway fail/recover/attempt, tap, try_catch/retry
flow_store.erl	durable gen_server: named-flow registry + instance table + start/resume/status

Consumed by

The fed-sx kernel's trigger fan-out (pipeline.erl + flow_dispatch) starts named flows from arriving activities; see plans/fed-sx-host-types.md and the triggers phases.

Not yet (later layers)

• Persisting instance logs to the kernel's durable on-disk log (the data shape is already restart-ready; only the backing is in-memory).
• parallel with multiple independent suspends resolving concurrently (current parallel is sequential under one shared log).
• Full parity with the Scheme engine's distributed/remote nodes.