rose-ash/plans/erlang-on-sx.md

Erlang-on-SX: actors on delimited continuations

The headline showcase for the SX runtime. Erlang is built around the one primitive — lightweight processes with mailboxes and selective receive — that delimited continuations implement natively. Most Erlang implementations ship a whole VM (BEAM) for this; on SX each process is a pair of continuations and the scheduler is ~50 lines of SX.

End-state goal: spawn a million processes, run the classic ring benchmark, plus a mini gen_server OTP subset and a test corpus of ~150 programs.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: Erlang/OTP 26 subset. No preprocessor, no parse transforms.
• Conformance: not BEAM-compat. "Looks like Erlang, runs like Erlang, not byte-compatible." We care about semantics, not BEAM bug-for-bug.
• Test corpus: custom — ring, ping-pong, fibonacci-server, bank-account-server, echo-server, plus ~100 hand-written tests for patterns/guards/BIFs. No ISO Common Test.
• Binaries: basic bytes-lists only; full binary pattern matching deferred.
• Hot code reload, distribution, NIFs: out of scope entirely.

Ground rules

• Scope: only touch lib/erlang/** and plans/erlang-on-sx.md. Don't edit spec/, hosts/, shared/, lib/js/**, lib/hyperscript/**, lib/lua/**, lib/prolog/**, lib/forth/**, lib/haskell/**, lib/stdlib.sx, or lib/ root. Erlang primitives go in lib/erlang/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

Erlang source
    │
    ▼
lib/erlang/tokenizer.sx  — atoms, vars, tuples, lists, binaries, operators
    │
    ▼
lib/erlang/parser.sx     — AST: modules, functions with clauses, patterns, guards
    │
    ▼
lib/erlang/transpile.sx  — AST → SX AST (entry: erlang-eval-ast)
    │
    ▼
lib/erlang/runtime.sx    — scheduler, processes, mailboxes, BIFs

Core mapping:

• Process = pair of delimited continuations (on-receive, on-resume) + mailbox list + pid + links
• Scheduler = round-robin list of runnable processes; cooperative yield on receive
• spawn = push a new process record, return its pid
• send = append to target mailbox; if target is blocked on receive, resume its continuation
• receive = selective — scan mailbox for first matching clause; if none, perform a suspend with the receive pattern; scheduler resumes when a matching message arrives
• Pattern matching = SX case on tagged values; vars bind on match
• Guards = side-effect-free predicate evaluated after unification
• Immutable data = native
• Links / monitors / exit signals = additional process-record fields, scheduler fires exit signals on death

Roadmap

Phase 1 — tokenizer + parser

• Tokenizer: atoms (bare + single-quoted), variables (Uppercase/_-prefixed), numbers (int, float, 16#HEX), strings "...", chars $c, punct ( ) { } [ ] , ; . : :: -> — 62/62 tests
• Parser: module declarations, -module/-export/-import attributes, function clauses with head patterns + guards + body — 52/52 tests
• Expressions: literals, vars, calls, tuples {...}, lists [...|...], if, case, receive, fun, try/catch, operators, precedence
• Binaries <<...>> — not yet parsed (deferred to Phase 6)
• Unit tests in lib/erlang/tests/parse.sx

Phase 2 — sequential eval + pattern matching + BIFs

• erlang-eval-ast: evaluate sequential expressions — 54/54 tests
• Pattern matching (atoms, numbers, vars, tuples, lists, [H|T], underscore, bound-var re-match) — 21 new eval tests; case ... of ... end wired
• Guards: is_integer, is_atom, is_list, is_tuple, comparisons, arithmetic — 20 new eval tests; local-call dispatch wired
• BIFs: length/1, hd/1, tl/1, element/2, tuple_size/1, atom_to_list/1, list_to_atom/1, lists:map/2, lists:foldl/3, lists:reverse/1, io:format/1-2 — 35 new eval tests; funs + closures wired
• 30+ tests in lib/erlang/tests/eval.sx — 130 tests green

Phase 3 — processes + mailboxes + receive (THE SHOWCASE)

• Scheduler in runtime.sx: runnable queue, pid counter, per-process state record — 39 runtime tests
• spawn/1, spawn/3, self/0 — 13 new eval tests; spawn/3 stubbed with "deferred to Phase 5" until modules land; is_pid/1 + pid equality also wired
• ! (send), receive ... end with selective pattern matching — 13 new eval tests; delimited continuations (shift/reset) power receive suspension; sync scheduler loop
• receive ... after Ms -> ... timeout clause (use SX timer primitive) — 9 new eval tests; synchronous-scheduler semantics: after 0 polls once; after Ms fires when runnable queue drains; after infinity = no timeout
• exit/1, basic process termination — 9 new eval tests; exit/2 (signal another) deferred to Phase 4 with links
• Classic programs in lib/erlang/tests/programs/:
  • ring.erl — N processes in a ring, pass a token around M times — 4 ring tests; suspension machinery rewritten from shift/reset to call/cc + raise/guard
  • ping_pong.erl — two processes exchanging messages
  • bank.erl — account server (deposit/withdraw/balance)
  • echo.erl — minimal server
  • fib_server.erl — compute fib on request
• lib/erlang/conformance.sh + runner, scoreboard.json + scoreboard.md
• Target: 5/5 classic programs + 1M-process ring benchmark runs

Phase 4 — links, monitors, exit signals

• link/1, unlink/1, monitor/2, demonitor/1
• Exit-signal propagation; trap_exit flag
• try/catch/of/end

Phase 5 — modules + OTP-lite

• -module(M). loading, M:F(...) calls across modules
• gen_server behaviour (the big OTP win)
• supervisor (simple one-for-one)
• Registered processes: register/2, whereis/1

Phase 6 — the rest

• List comprehensions [X*2 || X <- L]
• Binary pattern matching <<A:8, B:16>>
• ETS-lite (in-memory tables via SX dicts)
• More BIFs — target 200+ test corpus green

Progress log

Newest first.

• 2026-04-24 ring.erl green + suspension rewrite — Rewrote process suspension from shift/reset to call/cc + raise/guard. Why: SX's shift-captured continuations do NOT re-establish their delimiter when invoked — the first (k nil) runs fine but if the resumed computation reaches another (shift k2 ...) it raises "shift without enclosing reset". Ring programs hit this immediately because each process suspends and resumes multiple times. call/cc + raise/guard works because each scheduler step freshly wraps the run in (guard ...), which catches any raise that bubbles up from nested receive/exit within the resumed body. Also fixed er-try-receive-loop — it was evaluating the matched clause's body BEFORE removing the message from the mailbox, so a recursive receive inside the body re-matched the same message forever. Added lib/erlang/tests/programs/ring.sx with 4 tests (N=3 M=6, N=2 M=4, N=1 M=5 self-loop, N=3 M=9 hop-count via io-buffer). All process-communication eval tests still pass. Total suite 331/331.
• 2026-04-24 exit/1 + termination green — exit/1 BIF uses (shift k ...) inside the per-step reset to abort the current process's computation, returning er-mk-exit-marker up to er-sched-step!. Step handler records :exit-reason, clears :exit-result, marks dead. Normal fall-off-end still records reason normal. exit/2 errors with "deferred to Phase 4 (links)". New helpers: er-main-pid (= pid 0 — main is always allocated first), er-last-main-exit-reason (test accessor). 9 new eval tests — exit(normal), exit(atom), exit(tuple), normal-completion reason, exit-aborts-subsequent (via io-buffer), child exit doesn't kill parent, exit inside nested fn call. Total eval 174/174; suite 327/327.
• 2026-04-24 receive...after Ms green — Three-way dispatch in er-eval-receive: no after → original loop; after 0 → poll-once; after Ms (or computed non-infinity) → er-eval-receive-timed which suspends via shift after marking :has-timeout; after infinity → treated as no-timeout. er-sched-run-all! now recurses into er-sched-fire-one-timeout! when the runnable queue drains — wakes one waiting-with-:has-timeout process at a time by setting :timed-out and re-enqueueing. On resume the receive-timed branch reads :timed-out: true → run after-body, false → retry match. "Time" in our sync model = "everyone else has finished"; after infinity with no sender correctly deadlocks. 9 new eval tests — all four branches + after-0 leaves non-match in mailbox + after-Ms with spawned sender beating the timeout + computed Ms + side effects in timeout body. Total eval 165/165; suite 318/318.
• 2026-04-24 send + selective receive green — THE SHOWCASE — ! (send) in lib/erlang/transpile.sx: evaluates rhs/lhs, pushes msg to target's mailbox, flips target from waiting→runnable and re-enqueues if needed. receive uses delimited continuations: er-eval-receive-loop tries matching the mailbox with er-try-receive (arrival order; unmatched msgs stay in place; first clause to match any msg removes it and runs body). On no match, (shift k ...) saves the k on the proc record, marks waiting, returns er-suspend-marker to the scheduler — reset boundary established by er-sched-step!. Scheduler loop er-sched-run-all! pops runnable pids and calls either (reset ...) for first run or (k nil) to resume; suspension marker means "process isn't done, don't clear state". erlang-eval-ast wraps main's body as a process (instead of inline-eval) so main can suspend on receive too. Queue helpers added: er-q-nth, er-q-delete-at!. 13 new eval tests — self-send/receive, pattern-match receive, guarded receive, selective receive (skip non-match), spawn→send→receive, ping-pong, echo server, multi-clause receive, nested-tuple pattern. Total eval 156/156; suite 309/309. Deadlock detected if main never terminates.
• 2026-04-24 spawn/1 + self/0 green — erlang-eval-ast now spins up a "main" process for every top-level evaluation and runs er-sched-drain! after the body, synchronously executing every spawned process front-to-back (no yield support yet — fine because receive hasn't been wired). BIFs added in lib/erlang/runtime.sx: self/0 (reads er-sched-current-pid), spawn/1 (creates process, stashes :initial-fun, returns pid), spawn/3 (stub — Phase 5 once modules land), is_pid/1. Pids added to er-equal? (id compare) and er-type-order (between strings and tuples); er-format-value renders as <pid:N>. 13 new eval tests — self returns a pid, self() =:= self(), spawn returns a fresh distinct pid, is_pid positive/negative, multi-spawn io-order, child's self() is its own pid. Total eval 143/143; runtime 39/39; suite 296/296. Next: ! (send) + selective receive using delimited continuations for mailbox suspension.
• 2026-04-24 scheduler foundation green — lib/erlang/runtime.sx + lib/erlang/tests/runtime.sx. Amortised-O(1) FIFO queue (er-q-new, er-q-push!, er-q-pop!, er-q-peek, er-q-compact! at 128-entry head drift), tagged pids {:tag "pid" :id N} with er-pid?/er-pid-equal?, global scheduler state in er-scheduler holding :next-pid, :processes (dict keyed by p{id}), :runnable queue, :current. Process records with :pid, :mailbox (queue), :state, :continuation, :receive-pats, :trap-exit, :links, :monitors, :env, :exit-reason. 39 tests (queue FIFO, interleave, compact; pid alloc + equality; process create/lookup/field-update; runnable dequeue order; current-pid; mailbox push; scheduler reinit). Total erlang suite 283/283. Next: spawn/1, !, receive wired into the evaluator.
• 2026-04-24 core BIFs + funs green — Phase 2 complete. Added to lib/erlang/transpile.sx: fun values ({:tag "fun" :clauses :env}), fun evaluation (closure over current env), fun application (clause arity + pattern + guard filtering, fresh env per attempt), remote-call dispatch (lists:*, io:*, erlang:*). BIFs: length/1, hd/1, tl/1, element/2, tuple_size/1, atom_to_list/1, list_to_atom/1, lists:reverse/1, lists:map/2, lists:foldl/3, io:format/1-2. io:format writes to a capture buffer (er-io-buffer, er-io-flush!, er-io-buffer-content) and returns ok — supports ~n, ~p/~w/~s, ~~. 35 new eval tests. Total eval 130/130; erlang suite 244/244. Phase 2 complete — Phase 3 (processes, scheduler, receive) is next.
• 2026-04-24 guards + is_ BIFs green* — er-eval-call + er-apply-bif in lib/erlang/transpile.sx wire local function calls to a BIF dispatcher. Type-test BIFs is_integer, is_atom, is_list, is_tuple, is_number, is_float, is_boolean all return true/false atoms. Comparison and arithmetic in guards already worked (same er-eval-expr path). 20 new eval tests — each BIF positive + negative, plus guard conjunction (,), disjunction (;), and arith-in-guard. Total eval 95/95; erlang suite 209/209.
• 2026-04-24 pattern matching green — er-match! in lib/erlang/transpile.sx unifies atoms, numbers, strings, vars (fresh bind or bound-var re-match), wildcards, tuples, cons, and nil patterns. case ... of ... [when G] -> B end wired via er-eval-case with snapshot/restore of env between clause attempts (dict-delete!-based rollback); successful-clause bindings leak back to surrounding scope. 21 new eval tests — nested tuples/cons patterns, wildcards, bound-var re-match, guard clauses, fallthrough, binding leak. Total eval 75/75; erlang suite 189/189.
• 2026-04-24 eval (sequential) green — lib/erlang/transpile.sx (tree-walking interpreter) + lib/erlang/tests/eval.sx. 54/54 tests covering literals, arithmetic, comparison, logical (incl. short-circuit andalso/orelse), tuples, lists with ++, begin..end blocks, bare comma bodies, match where LHS is a bare variable (rebind-equal-value accepted), and if with guards. Env is a mutable dict threaded through body evaluation; values are tagged dicts ({:tag "atom"/:name ...}, {:tag "nil"}, {:tag "cons" :head :tail}, {:tag "tuple" :elements}). Numbers pass through as SX numbers. Gotcha: SX's parse-number coerces "1.0" → integer 1, so =:= can't distinguish 1 from 1.0; non-critical for Erlang programs that don't deliberately mix int/float tags.
• parser green — lib/erlang/parser.sx + parser-core.sx + parser-expr.sx + parser-module.sx. 52/52 in tests/parse.sx. Covers literals, tuples, lists (incl. [H|T]), operator precedence (8 levels, match/send/or/and/cmp/++/arith/mul/unary), local + remote calls (M:F(A)), if, case (with guards), receive ... after ... end, begin..end blocks, anonymous fun, try..of..catch..after..end with Class:Pattern catch clauses. Module-level: -module(M)., -export([...])., multi-clause functions with guards. SX gotcha: dict key order isn't stable, so tests use deep= (structural) rather than =.
• tokenizer green — lib/erlang/tokenizer.sx + lib/erlang/tests/tokenize.sx. Covers atoms (bare, quoted, node@host), variables, integers (incl. 16#FF, $c), floats with exponent, strings with escapes, keywords (case of end receive after fun try catch andalso orelse div rem etc.), punct (( ) { } [ ] , ; . : :: -> <- <= => << >> | ||), ops (+ - * / = == /= =:= =/= < > =< >= ++ -- ! ?), % line comments. 62/62 green.

Blockers

• (none yet)