166/166 across 11 suites, Phases 1-8. Combinators (sequence/parallel/branch/attempt/
map-flow/while/until + retry/timeout/try-catch/recover/tap/fail-model), durable
suspend/resume via deterministic replay (guest call/cc is escape-only), crash
recovery, fed-sx distribution (remote-node/failover/replication/handoff), operational
API + hygiene, and a host integration ABI + reference driver for art-dag / human-in-
the-loop. New lib/flow/** only; imports lib/scheme read-only.
Completes the host ABI from work-queue to driver loop: the host supplies only a
(kind payload) -> answer dispatch fn; flow-drive-host services one tick of pending
requests, flow-run-host ticks until quiescent (bounded). Tested via the art-dag
render -> human-review -> publish pipeline driven entirely by flow-run-host. The
art-dag integration is now: define dispatch, call flow-run-host. 166/166, 11 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The seam for hooking flow to art-dag and human-in-the-loop later. (request kind
payload) suspends with a typed (flow-request kind payload) envelope and returns the
host's resume value; await-human/await-render sugar. (flow-host-requests) is the
host work queue: (id kind payload) for every suspended flow awaiting a host effect;
request?/request-kind/request-payload parse a tag. Tests include the art-dag-shaped
driver loop (render -> human-review -> publish). Host owns IO+persistence; flow only
requests (replay-safe). 162/162 across 11 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replaces the bespoke 116-line conformance.sh with a conformance.conf + 1-line
exec shim, reusing lib/guest/conformance.sh. Surfaced + fixed a silent undercount:
the old awk extractor reported pipeline=40, but pipeline.sx has 152 assertions —
real total is 562/562, not 450/450. Driver reads counter globals directly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
User-facing docs for the flow engine: the node model, every combinator, the
suspend/resume durability contract (escape-only call/cc -> deterministic replay),
lifecycle/introspection/hygiene API, fed-sx distribution, and substrate notes.
Doc-only; 151/151 unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
flow/gc drops terminal (done/cancelled) records, keeps live suspended flows, returns
count removed; flow/forget id drops one terminal record and refuses live flows.
Bounds unbounded store growth (retention/GC). Bumped conformance sx_server timeout
to 540s for the 10-suite run under CPU contention. 151/151 across 10 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Realistic flows composing every phase: an order pipeline (validate via attempt ->
payment suspend -> branch -> ledger federation via remote-node) and an onboarding
flow, each run through the full lifecycle including a simulated crash (export/wipe/
import) and a peer handoff mid-flow, with flow/pending|status|result introspection.
142/142 across 9 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(attempt n1 n2 ...) threads like sequence but stops at the first node returning a
(fail ...) value, returning that failure. Makes the fail/recover error model
compose into validation/ETL pipelines (railway-oriented). 132/132 across 8 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(flow-while pred body max) / (flow-until pred body max) re-run body threading the
value while/until pred holds, capped at max steps for a deterministic bound (no
unbounded loops in pure SX). 122/122 across 7 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tap: side-effecting pass-through (returns input). recover: fail-VALUE counterpart
of try-catch (run node; on (fail r) run handler on r). map-flow: run a node over
each item of a list, join results sequentially. 116/116 across 7 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
flow/status id -> done|suspended|cancelled|unknown; flow/result id -> value or
error; flow/list -> (id status) per flow; flow/pending -> (id waiting-tag) for
suspended flows (operator view of what each awaits). Pure store introspection.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
flow-replicate-to copies the plain-data store export to a peer's replica slot;
flow-restore-from imports it. Handoff = replicate, local instance dies, peer
restores and resumes by id. The replay log survives the move, so all resolved
suspends carry over. Same durable-data mechanism as crash recovery, across
instances. All four phases complete: 93/93.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(remote-failover addrs fn local) tries fn on each peer in order, moves to the next
on any raised error, and runs the local node if every peer fails. Threads input,
composes in sequences.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(remote-node addr fn) runs a node on a federation peer. Transport is the fed-sx
boundary, mocked by a peer registry (flow-peer-register!); raises
flow-remote-unreachable / flow-remote-no-fn. Composes with sequence/suspend/retry.
Also fixes conformance.sh to load remote.sx before api.sx.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records are name-keyed (defflow registers names); flow-store-export nulls live
procs to plain data, flow-store-import! restores, flow-resumable-ids scans for
paused flows. Resume re-resolves the proc by name, so a flow survives a wiped
store (simulated restart). The whole durable model persists only plain data.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Guest Scheme call/cc is escape-only (re-entry hangs), so durable resume uses
deterministic replay: suspend escapes to the driver; resume re-runs the flow and
replays resolved suspends from a (tag value) log. No live continuation is ever
serialized — persisted state is plain data, survives restart. Adds flow/start
(now state-returning, backward compatible), flow/resume, flow/cancel, store.sx.
Harness reuses one env with a per-test reset (full env rebuild 66x was too slow).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(timeout budget node) bounds a node deterministically: nodes opt in via (tick),
budget ticks are allowed, the next raises flow-timeout. No scheduler/clock in pure
SX so the budget is a step count, not wall-clock. Budgets nest and are per-run.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(retry n node) re-runs up to n attempts on a raised exception; the last attempt's
exception propagates. Explicit (fail ...) values are NOT retried — they pass through.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
(try-catch node handler) runs node; on a raised exception calls (handler error)
with the reified error via Scheme guard, returns the handler value.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Explicit (fail reason) values flow downstream as data and are inspected with
failed?/fail-reason — distinct from raised exceptions (retry/try-catch territory).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2 control flow. (branch pred then else) selects then/else node by running
pred on the threaded input; named 'branch' since 'cond' is a Scheme special form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Flow combinators as a Scheme prelude loaded onto scheme-standard-env; a flow is a
Scheme procedure input->output, run inside the interpreter (sets up Phase 3 call/cc
suspend). flow/start entry point, conformance runner, scoreboard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Four small, contained substrate fixes that came out of the fed-sx-m1 milestone work — all scoped to
lib/erlang/, no other-language regressions:
c6f397c3 register binary_to_list/1 + list_to_binary/1 BIFs (+9 ffi tests, 738/738)
9fe5c904 $X char literals decode to char code in tokenizer (+12 eval tests, 750/750)
5098a8f0 atom_to_list/integer_to_list return Erlang charlists; list_to_* accept both (+9 eval, 759/759)
bcabed6b integer literals truncate to strict int (was float; broke integer->char)
Together these complete the byte-level term-codec primitive set:
binary_to_list / list_to_binary (iolist-aware; round-trips for free)
$X char literals decoding to int char codes
atom_to_list / integer_to_list returning standard Erlang charlists
integer literals coercing to strict int (not float)
Any Erlang-on-SX consumer that needs to construct/deconstruct byte sequences or work with charlists now
does so with standard Erlang semantics. Scoreboard: 759/759 (full Erlang suite).
Loop branch loops/erlang stays alive for future Erlang substrate work; this just lands the closed deliverables.
12 canonical Go programs running through the full pipeline (lex +
parse + types + eval + sched + stdlib): sieve-of-Eratosthenes via
boolean slice (modulo-free), linear search, slice reverse, fib(10),
sum-of-squares via generic Map+Reduce, word-freq counter, channel
pipeline (gen→sq→sum), worker pool, bubble sort, sentence-reverse,
Filter+len, Ackermann, defer+recover on div-by-zero.
Each test threads ONE self-contained Go program through go-eval-
program. The v0 limitations chiselled in earlier phases (float
division, sync spawn, type erasure, nil-as-unbound) are now
durable as commit-trail artifacts; e2e variants written to avoid
them where possible. HTTP-ish ping-pong + WaitGroup deferred
(real preemption + sync package needed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New :go-package NAME ENTRIES value type with field lookup via
extended go-eval-select. New :go-builtin-fn callable for closure-
based stdlib functions. lib/go/std/strings.sx ships 12 functions
(Contains, HasPrefix, HasSuffix, Index, Count, Repeat, Join,
ToUpper, ToLower, TrimSpace, Split, Replace) + lib/go/std/strconv.sx
ships Itoa/Atoi.
Pre-existing bug fixed: parser was emitting (:literal V) for both
`42` and `"42"`, relying on first-char heuristic in eval/types.
Now emits :literal-string for string/rune literals so Atoi("42")
correctly receives the string. 3 parse tests + 2 in-composite-key
tests updated to new shape.
Total 597/597. Stdlib 41/41 — +40 acceptance bar cleared. Sister
diary documents the 11 value-type kinds (struct/slice/map/chan/
fn/method/builtin/builtin-fn/package/panic/defer) all sharing the
"(:KIND PAYLOAD...)" shape, alongside AST nodes and sentinel signals
as the kit's three orthogonal first-class-tag axes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Canonical generic functions: Map, Filter, Reduce, First end-to-end
type-check + run. Plus 20+ typer-only shape tests covering Apply,
Compose, ToMap, Swap, Box, Triple, ToSlice, Take, Send, Fill, Eq,
Values, Pair, Inspect, etc. Index synth (slice/array/map →
element type) added to typer.
v0 limitations stamped in tests: SX `/` is float (no int mod
emulation), `var r []T` indistinguishable from unbound, single-name
constraints opaque (no type-set arithmetic).
Shape locked in: "the parser recognizes shapes, the validator
recognizes roles." Same AST + different role-validators = different
guest semantics. Diary documents this as the lemma the kit should
extract — three deliverables (binding-groups, control-flow sentinels,
index synthesis) now all instantiate it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gp-parse-type-params consumes the optional [NAMES CONSTRAINT, ...]
clause after a func name. AST stays backward-compatible: 5-slot
func-decl when no [...] is present, 6-slot when it is.
Typer binds each type-param name as (:ty-param NAME CONSTRAINT) so
body's (:ty-name "T") references resolve. Eval is type-erasing —
ignores type info, dispatches by name + arity.
10 new tests: parse (3), types (5), eval (2). Total 527/527.
Shape: the field binding-group from the canonical kit now feeds
6 consumers (struct fields, var-decls, const-decls, params,
receivers, type-params). Confirms it as a TRUE cross-deliverable
shape — sister-plan diary documents the 5 roles binding-groups
take and why the kit should expose ONE parser + pluggable validators.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wired panic through :go stmt (v0 sync surfaces back to spawner —
matches real Go's "crash whole program" end-effect) and through
go-eval-for (was swallowing panic at the loop boundary).
8 tests added: goroutine-panic-surfaces, goroutine-recover-via-
spawner-defer, multi-defer-LIFO-with-recover, defer-fires-on-panic-
path, panic(nil), panic-in-loop, defer-still-runs-in-panicking-fn,
args-eager-on-panic-path. 20 Phase-6 tests total; +20 acceptance
bar cleared (eval/ 80 → 100).
Shape: 4 control-flow sites now repeat the same sentinel dispatch
arm (return-value, break, continue, eval-error, go-panic). The
scheduler kit should bake in a single propagates? helper rather
than have each guest evaluator list every sentinel inline — diary
documents the cross-cutting abstraction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Panic/recover builtins + per-frame __go-panic-cell of shape
(STATE V). Body panic flips cell :none→:raised BEFORE defers drain
so recover() can find it. recover() walks env chain past shadowing
cells to the outermost :raised one — flips it :recovered, returns V.
Frame exit checks cell: :recovered → return clean; :raised →
propagate (:go-panic V).
6 tests: uncaught-from-program, panic-from-fn, defer-recover-swallow,
recover-captures-via-channel, propagation-through-no-defer-chain,
middle-frame-catches-deeper-panic.
Shape: panic cell is a frame-attached out-of-band channel that
survives function boundaries via env-chain walk. Same primitive
slots into the scheduler kit's termination-record + cleanup-with-
error-context hook. Maps cleanly to Erlang try/catch/after.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 6 first slice. New :defer stmt dispatch, go-eval-defer-stmt
captures (callee, eagerly-evaluated args) onto a frame-local
__go-defer-stack mutable list. go-eval-call installs the stack and
drains LIFO before returning; go-eval-program does the same for
the implicit main frame. New :quoted-value AST node lets defer
re-invoke calls with the frozen arg values.
6 eval tests: single defer, multi-LIFO, args-eager-at-defer-time,
fires-on-early-return, frame-local (no bleed to outer), defer-in-loop.
Shape: defer is a per-frame cleanup queue (LIFO on frame exit) that
the scheduler kit will reuse for panic-unwind + clean-exit + select-
case-rollback paths. Distinct from the scheduler's ready-queue —
diary updated to keep that distinction explicit.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Acceptance bar hit (40 runtime, 497 total). Tests: timer ready,
select-with-timeout, fan-in (3 producers), worker queue, pipeline,
fan-out-then-fan-in, select source-order, fallback case, default,
producer-consumer, two-stage pipeline, channel-counter, after+default,
tick-collector.
Shape chiselled: timer collapses "after duration" into
"channel ready immediately" — select needs only ready? from each
case. Real time is when the flip happens, not what the protocol is.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 cont. New go-eval-range-for handles the parser's :range-for
AST shape. Dispatches on the collection's runtime type:
:go-slice → bind index + element, iterate by position
:go-map → bind key + value, walk entries assoc list
:go-chan → bind value, drain until buffer empty (v0 limitation)
Each loop carries:
- go-range-extend: handles 0/1/2-name binding patterns uniformly
- go-range-body: evaluates body whether it's a :block or other shape
- per-collection loop helper: threads env, catches :break/:continue/
:return-value/:eval-error sentinels
**Subtle break fix:** loops were previously returning the *pre-loop*
env when break fired, clobbering all assignments made in prior
iterations. Now returns the current iteration's input env (which
carries forward successful iterations' state). Patched for the three
range variants and for the regular for-loop where the same pattern
applied. The shape:
(= r :break) env ;; was: (= r :break) original-env
Tests:
range: slice — sum of 1..5 = 15
range: slice — key only (index)
range: map — sum values
range: channel — collect all buffered
range: slice with break exits early
range: slice with continue skips an element
range: empty slice — body never runs
range: chan + goroutine producer
runtime 26/26, total 483/483.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 cont. Adds `select` statement evaluation:
go-select-try-case env COMM →
:not-ready / extended-env / :eval-error
go-select-pick env CASES DEFAULT-OR-NIL →
body-result / blocked-error
go-eval-select-stmt env STMT — public entry
Walks cases in declared order:
* :send case — always ready in v0 (unbounded buffer). Sends value
via go-chan-send! and returns env unchanged.
* :short-decl / :assign case — RHS expected to be unary <- on a
channel. Ready iff go-chan-len > 0; on success, recv-into-var
binds the new value in env.
* Bare recv (:app (:var "<-") [CHAN]) — ready iff len > 0; consumes
the value (discarded).
* :default — deferred until end of walk. Runs if no other case
ready. Absence + no ready case → (:eval-error :select-blocked-
no-default).
New `go-chan-len` accessor on the channel closure-bundle so the
select can peek without consuming.
Subtle bug fix: the :select stmt branch in go-eval-stmt was returning
the old env instead of the env returned by the case body. Assignments
inside select cases (`select { case <-ch: x = 1 ; default: x = 99 }`)
now stick.
Tests (6):
default fires when no case ready
recv case fires when ready
recv-into-var binds the value
send case always ready
picks first ready case (deterministic order in v0)
no default + nothing ready → blocked error
combined with goroutine fan-in
runtime 18/18, total 475/475.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 5 (goroutines + channels) opens.
lib/go/sched.sx is the **independent implementation** referenced by
plans/lib-guest-scheduler.md — the first-consumer cut whose realised
shape will inform the eventual sister kit.
Channel representation:
(list :go-chan SEND-FN RECV-FN CLOSED?-FN CLOSE!-FN)
Each closure shares a mutable `buf` (a list mutated via append! and
set!) and a `closed` flag. Channel identity is closure-instance —
two `make()` calls produce distinct values per Go spec § Channel types.
Primitive API in sched.sx:
go-make-chan / go-chan? / go-chan-send! / go-chan-recv! /
go-chan-closed? / go-chan-close!
Eval integration in eval.sx:
* `make` and `close` added as builtins. v0 `make()` takes no args
and returns an unbounded-buffer channel.
* `:send` stmt → go-chan-send! on the channel.
* Unary `<-` recv on channel values → go-chan-recv!. `:empty`
sentinel converted to nil (stand-in for blocking semantics).
* `:go expr` → synchronous eval (v0 limitation, see sched.sx
header).
**v0 concurrency model — synchronous goroutines.** SX doesn't expose
first-class continuations to guest code, so v0 runs `go f()`
immediately and depends on the spawned goroutine running to
completion before the main goroutine receives. This is the right
semantics for the simple producer/consumer patterns covered here.
True preemption with blocking send/recv is Phase 5b — requires either
a CEK-style trampolining eval rewrite or kit-level continuation
support. Logged in sched.sx header and in the sister-plan diary.
Runtime suite (12 tests):
* 6 direct API tests: identity, FIFO order, closed-flag
* 6 source-level: make + send + recv, go ping-pong, close,
multi-goroutine fan-in, worker-with-result
Sister-plan scheduler diary updated with the channel-as-closure-
bundle insight and the v0 synchronous-spawn caveat.
runtime 12/12, total 469/469.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
