fed-sx-types Phase 5: flow-on-erlang engine core (next/flow/)
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 49s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 49s
A native Erlang-on-SX durable workflow engine, so the fed-sx kernel can
fan activities out into business flows in its own runtime — no cross-
guest FFI to the Scheme lib/flow, no marshalling, no Scheme dependency.
The seed of a real engine (chosen over bridging Scheme flow) that can
later supersede it for substrate use.
- flow.erl — the deterministic-replay driver. Same durability model as
the Scheme engine (re-run from the top; effects go through suspend;
the replay log is plain [{Tag,Value}] data, restart-ready), but
adapted to three hard runtime constraints: no re-enterable
continuation, no process dictionary, and a blocking receive inside a
`try` deadlocks the cooperative scheduler. Resolution: thread the log
through a railway-style context and make suspend SHORT-CIRCUIT (like a
fail value) instead of throwing — purely functional, sidesteps all
three. Ctx = {flow_cont,V,Log} | {flow_susp,Tag,Log}.
- flow_spec.erl — combinator algebra mirrored from lib/flow/spec.sx:
leaves, sequence/parallel/map_flow, flow_while/flow_until, branch,
railway fail/recover/attempt, tap, try_catch/retry.
- flow_store.erl — durable gen_server: named-flow registry + instance
table + start/resume/status. Drives the pure flow from handle_call,
so no gen_server:call is ever inside the replay try-path.
Gate: next/flow/conformance.sh — 34/34. lib/erlang untouched (771/771).
See next/flow/README.md for the model + why railway threading.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
91
next/flow/README.md
Normal file
91
next/flow/README.md
Normal file
@@ -0,0 +1,91 @@
|
||||
# flow-on-erlang — durable workflows in the fed-sx runtime
|
||||
|
||||
A native Erlang-on-SX port of the Scheme flow engine (`lib/flow`), so
|
||||
the fed-sx kernel can fan arriving activities out into durable,
|
||||
branching, multi-step business flows **in its own runtime** — no
|
||||
cross-guest FFI, no marshalling, no Scheme dependency. The seed of a
|
||||
real engine that can later supersede the Scheme one for substrate use.
|
||||
|
||||
Run the suite: `bash next/flow/conformance.sh` → engine conformance.
|
||||
|
||||
## Model
|
||||
|
||||
A **flow** is an Erlang `fun(Ctx) -> Ctx`. Combinators (`flow_spec`)
|
||||
compose flows; user code stays value-level (the functions you hand to
|
||||
`flow_node`/`branch`/… take and return plain values). A flow that
|
||||
ignores its input is a thunk; composition *is* function composition.
|
||||
|
||||
```erlang
|
||||
F = flow_spec:sequence([
|
||||
flow_spec:flow_node(fun(Draft) -> Draft + 1 end),
|
||||
flow_spec:branch(fun(P) -> P >= 3 end,
|
||||
flow_spec:flow_const(ok),
|
||||
flow_spec:flow_const(rejected))]),
|
||||
flow:run(F, 2) %% => {flow_done, ok}
|
||||
```
|
||||
|
||||
## Durability — deterministic replay
|
||||
|
||||
Same semantics as the Scheme engine: a flow re-runs from the top on
|
||||
every resume; effects/non-determinism go through `flow:suspend/1`,
|
||||
whose resolved values are logged; an already-resolved suspend replays
|
||||
its logged value, and the first unresolved suspend short-circuits back
|
||||
to the driver. The persisted state is the **replay log** — plain
|
||||
`[{Tag, Value}]` data — so nothing live (no continuation, no process)
|
||||
is ever serialized; an instance survives restart by re-driving its
|
||||
named flow against its log.
|
||||
|
||||
```erlang
|
||||
flow_store:register_flow(publish, F),
|
||||
{ok, Id, R} = flow_store:start(publish, Draft), %% R = {flow_suspended, Tag} | {flow_done, V}
|
||||
%% ... driver performs the effect for Tag, then:
|
||||
flow_store:resume(Id, EffectResult) %% re-drives; completes or suspends again
|
||||
```
|
||||
|
||||
## Why railway threading instead of call/cc + a global
|
||||
|
||||
The Scheme engine uses an escape-only `call/cc` plus a mutable global
|
||||
replay log. This Erlang-on-SX runtime can't do either, and has a third
|
||||
sharp edge:
|
||||
|
||||
- **No re-enterable continuation** — but suspend only needs to *escape*,
|
||||
which Erlang `throw` could do …
|
||||
- **… except a blocking `receive` / `gen_server:call` inside a `try`
|
||||
deadlocks** the cooperative scheduler. So `suspend` must not consult
|
||||
the log via a registry process while inside a `try`.
|
||||
- **No process dictionary** — so there is no ambient per-process slot to
|
||||
stash the replay log in.
|
||||
|
||||
The resolution: thread the replay log through a railway-style **context**
|
||||
and make `suspend` *short-circuit* (like a `fail` value) rather than
|
||||
throw. No ambient state, no throw, no gen_server in the hot path —
|
||||
purely functional, which sidesteps all three constraints. The driver
|
||||
(`flow_store`) is the only stateful part, and it calls the pure
|
||||
`flow:drive/3` from inside `handle_call`, never wrapping a blocking
|
||||
receive.
|
||||
|
||||
A `Ctx` is `{flow_cont, Value, Log}` (running) or `{flow_susp, Tag,
|
||||
Log}` (short-circuited); every combinator passes a suspended context
|
||||
straight through.
|
||||
|
||||
## Modules
|
||||
|
||||
| Module | Role |
|
||||
|---|---|
|
||||
| `flow.erl` | pure replay driver: `drive/3`, `run/2`, `suspend/1`, the `Ctx` constructors/accessors |
|
||||
| `flow_spec.erl` | combinator algebra: leaves, `sequence`/`parallel`/`map_flow`, `flow_while`/`flow_until`, `branch`, railway `fail`/`recover`/`attempt`, `tap`, `try_catch`/`retry` |
|
||||
| `flow_store.erl` | durable gen_server: named-flow registry + instance table + `start`/`resume`/`status` |
|
||||
|
||||
## Consumed by
|
||||
|
||||
The fed-sx kernel's trigger fan-out (`pipeline.erl` + `flow_dispatch`)
|
||||
starts named flows from arriving activities; see
|
||||
`plans/fed-sx-host-types.md` and the triggers phases.
|
||||
|
||||
## Not yet (later layers)
|
||||
|
||||
- Persisting instance logs to the kernel's durable on-disk log (the
|
||||
data shape is already restart-ready; only the backing is in-memory).
|
||||
- `parallel` with multiple independent suspends resolving concurrently
|
||||
(current `parallel` is sequential under one shared log).
|
||||
- Full parity with the Scheme engine's distributed/remote nodes.
|
||||
Reference in New Issue
Block a user