rose-ash/plans/erlang-on-sx.md

Erlang-on-SX: actors on delimited continuations

The headline showcase for the SX runtime. Erlang is built around the one primitive — lightweight processes with mailboxes and selective receive — that delimited continuations implement natively. Most Erlang implementations ship a whole VM (BEAM) for this; on SX each process is a pair of continuations and the scheduler is ~50 lines of SX.

End-state goal: spawn a million processes, run the classic ring benchmark, plus a mini gen_server OTP subset and a test corpus of ~150 programs.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: Erlang/OTP 26 subset. No preprocessor, no parse transforms.
• Conformance: not BEAM-compat. "Looks like Erlang, runs like Erlang, not byte-compatible." We care about semantics, not BEAM bug-for-bug.
• Test corpus: custom — ring, ping-pong, fibonacci-server, bank-account-server, echo-server, plus ~100 hand-written tests for patterns/guards/BIFs. No ISO Common Test.
• Binaries: basic bytes-lists only; full binary pattern matching deferred.
• Hot code reload, distribution, NIFs: out of scope entirely.

Ground rules

• Scope: only touch lib/erlang/** and plans/erlang-on-sx.md. Don't edit spec/, hosts/, shared/, lib/js/**, lib/hyperscript/**, lib/lua/**, lib/prolog/**, lib/forth/**, lib/haskell/**, lib/stdlib.sx, or lib/ root. Erlang primitives go in lib/erlang/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

Erlang source
    │
    ▼
lib/erlang/tokenizer.sx  — atoms, vars, tuples, lists, binaries, operators
    │
    ▼
lib/erlang/parser.sx     — AST: modules, functions with clauses, patterns, guards
    │
    ▼
lib/erlang/transpile.sx  — AST → SX AST (entry: erlang-eval-ast)
    │
    ▼
lib/erlang/runtime.sx    — scheduler, processes, mailboxes, BIFs

Core mapping:

• Process = pair of delimited continuations (on-receive, on-resume) + mailbox list + pid + links
• Scheduler = round-robin list of runnable processes; cooperative yield on receive
• spawn = push a new process record, return its pid
• send = append to target mailbox; if target is blocked on receive, resume its continuation
• receive = selective — scan mailbox for first matching clause; if none, perform a suspend with the receive pattern; scheduler resumes when a matching message arrives
• Pattern matching = SX case on tagged values; vars bind on match
• Guards = side-effect-free predicate evaluated after unification
• Immutable data = native
• Links / monitors / exit signals = additional process-record fields, scheduler fires exit signals on death

Roadmap

Phase 1 — tokenizer + parser

• Tokenizer: atoms (bare + single-quoted), variables (Uppercase/_-prefixed), numbers (int, float, 16#HEX), strings "...", chars $c, punct ( ) { } [ ] , ; . : :: -> — 62/62 tests
• Parser: module declarations, -module/-export/-import attributes, function clauses with head patterns + guards + body — 52/52 tests
• Expressions: literals, vars, calls, tuples {...}, lists [...|...], if, case, receive, fun, try/catch, operators, precedence
• Binaries <<...>> — not yet parsed (deferred to Phase 6)
• Unit tests in lib/erlang/tests/parse.sx

Phase 2 — sequential eval + pattern matching + BIFs

• erlang-eval-ast: evaluate sequential expressions — 54/54 tests
• Pattern matching (atoms, numbers, vars, tuples, lists, [H|T], underscore, bound-var re-match) — 21 new eval tests; case ... of ... end wired
• Guards: is_integer, is_atom, is_list, is_tuple, comparisons, arithmetic — 20 new eval tests; local-call dispatch wired
• BIFs: length/1, hd/1, tl/1, element/2, tuple_size/1, atom_to_list/1, list_to_atom/1, lists:map/2, lists:foldl/3, lists:reverse/1, io:format/1-2 — 35 new eval tests; funs + closures wired
• 30+ tests in lib/erlang/tests/eval.sx — 130 tests green

Phase 3 — processes + mailboxes + receive (THE SHOWCASE)

• Scheduler in runtime.sx: runnable queue, pid counter, per-process state record — 39 runtime tests
• spawn/1, spawn/3, self/0 — 13 new eval tests; spawn/3 stubbed with "deferred to Phase 5" until modules land; is_pid/1 + pid equality also wired
• ! (send), receive ... end with selective pattern matching — 13 new eval tests; delimited continuations (shift/reset) power receive suspension; sync scheduler loop
• receive ... after Ms -> ... timeout clause (use SX timer primitive) — 9 new eval tests; synchronous-scheduler semantics: after 0 polls once; after Ms fires when runnable queue drains; after infinity = no timeout
• exit/1, basic process termination
• Classic programs in lib/erlang/tests/programs/:
  • ring.erl — N processes in a ring, pass a token around M times
  • ping_pong.erl — two processes exchanging messages
  • bank.erl — account server (deposit/withdraw/balance)
  • echo.erl — minimal server
  • fib_server.erl — compute fib on request
• lib/erlang/conformance.sh + runner, scoreboard.json + scoreboard.md
• Target: 5/5 classic programs + 1M-process ring benchmark runs

Phase 4 — links, monitors, exit signals

• link/1, unlink/1, monitor/2, demonitor/1
• Exit-signal propagation; trap_exit flag
• try/catch/of/end

Phase 5 — modules + OTP-lite

• -module(M). loading, M:F(...) calls across modules
• gen_server behaviour (the big OTP win)
• supervisor (simple one-for-one)
• Registered processes: register/2, whereis/1

Phase 6 — the rest

• List comprehensions [X*2 || X <- L]
• Binary pattern matching <<A:8, B:16>>
• ETS-lite (in-memory tables via SX dicts)
• More BIFs — target 200+ test corpus green

Progress log

Newest first.

• 2026-04-24 receive...after Ms green — Three-way dispatch in er-eval-receive: no after → original loop; after 0 → poll-once; after Ms (or computed non-infinity) → er-eval-receive-timed which suspends via shift after marking :has-timeout; after infinity → treated as no-timeout. er-sched-run-all! now recurses into er-sched-fire-one-timeout! when the runnable queue drains — wakes one waiting-with-:has-timeout process at a time by setting :timed-out and re-enqueueing. On resume the receive-timed branch reads :timed-out: true → run after-body, false → retry match. "Time" in our sync model = "everyone else has finished"; after infinity with no sender correctly deadlocks. 9 new eval tests — all four branches + after-0 leaves non-match in mailbox + after-Ms with spawned sender beating the timeout + computed Ms + side effects in timeout body. Total eval 165/165; suite 318/318.
• 2026-04-24 send + selective receive green — THE SHOWCASE — ! (send) in lib/erlang/transpile.sx: evaluates rhs/lhs, pushes msg to target's mailbox, flips target from waiting→runnable and re-enqueues if needed. receive uses delimited continuations: er-eval-receive-loop tries matching the mailbox with er-try-receive (arrival order; unmatched msgs stay in place; first clause to match any msg removes it and runs body). On no match, (shift k ...) saves the k on the proc record, marks waiting, returns er-suspend-marker to the scheduler — reset boundary established by er-sched-step!. Scheduler loop er-sched-run-all! pops runnable pids and calls either (reset ...) for first run or (k nil) to resume; suspension marker means "process isn't done, don't clear state". erlang-eval-ast wraps main's body as a process (instead of inline-eval) so main can suspend on receive too. Queue helpers added: er-q-nth, er-q-delete-at!. 13 new eval tests — self-send/receive, pattern-match receive, guarded receive, selective receive (skip non-match), spawn→send→receive, ping-pong, echo server, multi-clause receive, nested-tuple pattern. Total eval 156/156; suite 309/309. Deadlock detected if main never terminates.
• 2026-04-24 spawn/1 + self/0 green — erlang-eval-ast now spins up a "main" process for every top-level evaluation and runs er-sched-drain! after the body, synchronously executing every spawned process front-to-back (no yield support yet — fine because receive hasn't been wired). BIFs added in lib/erlang/runtime.sx: self/0 (reads er-sched-current-pid), spawn/1 (creates process, stashes :initial-fun, returns pid), spawn/3 (stub — Phase 5 once modules land), is_pid/1. Pids added to er-equal? (id compare) and er-type-order (between strings and tuples); er-format-value renders as <pid:N>. 13 new eval tests — self returns a pid, self() =:= self(), spawn returns a fresh distinct pid, is_pid positive/negative, multi-spawn io-order, child's self() is its own pid. Total eval 143/143; runtime 39/39; suite 296/296. Next: ! (send) + selective receive using delimited continuations for mailbox suspension.
• 2026-04-24 scheduler foundation green — lib/erlang/runtime.sx + lib/erlang/tests/runtime.sx. Amortised-O(1) FIFO queue (er-q-new, er-q-push!, er-q-pop!, er-q-peek, er-q-compact! at 128-entry head drift), tagged pids {:tag "pid" :id N} with er-pid?/er-pid-equal?, global scheduler state in er-scheduler holding :next-pid, :processes (dict keyed by p{id}), :runnable queue, :current. Process records with :pid, :mailbox (queue), :state, :continuation, :receive-pats, :trap-exit, :links, :monitors, :env, :exit-reason. 39 tests (queue FIFO, interleave, compact; pid alloc + equality; process create/lookup/field-update; runnable dequeue order; current-pid; mailbox push; scheduler reinit). Total erlang suite 283/283. Next: spawn/1, !, receive wired into the evaluator.
• 2026-04-24 core BIFs + funs green — Phase 2 complete. Added to lib/erlang/transpile.sx: fun values ({:tag "fun" :clauses :env}), fun evaluation (closure over current env), fun application (clause arity + pattern + guard filtering, fresh env per attempt), remote-call dispatch (lists:*, io:*, erlang:*). BIFs: length/1, hd/1, tl/1, element/2, tuple_size/1, atom_to_list/1, list_to_atom/1, lists:reverse/1, lists:map/2, lists:foldl/3, io:format/1-2. io:format writes to a capture buffer (er-io-buffer, er-io-flush!, er-io-buffer-content) and returns ok — supports ~n, ~p/~w/~s, ~~. 35 new eval tests. Total eval 130/130; erlang suite 244/244. Phase 2 complete — Phase 3 (processes, scheduler, receive) is next.
• 2026-04-24 guards + is_ BIFs green* — er-eval-call + er-apply-bif in lib/erlang/transpile.sx wire local function calls to a BIF dispatcher. Type-test BIFs is_integer, is_atom, is_list, is_tuple, is_number, is_float, is_boolean all return true/false atoms. Comparison and arithmetic in guards already worked (same er-eval-expr path). 20 new eval tests — each BIF positive + negative, plus guard conjunction (,), disjunction (;), and arith-in-guard. Total eval 95/95; erlang suite 209/209.
• 2026-04-24 pattern matching green — er-match! in lib/erlang/transpile.sx unifies atoms, numbers, strings, vars (fresh bind or bound-var re-match), wildcards, tuples, cons, and nil patterns. case ... of ... [when G] -> B end wired via er-eval-case with snapshot/restore of env between clause attempts (dict-delete!-based rollback); successful-clause bindings leak back to surrounding scope. 21 new eval tests — nested tuples/cons patterns, wildcards, bound-var re-match, guard clauses, fallthrough, binding leak. Total eval 75/75; erlang suite 189/189.
• 2026-04-24 eval (sequential) green — lib/erlang/transpile.sx (tree-walking interpreter) + lib/erlang/tests/eval.sx. 54/54 tests covering literals, arithmetic, comparison, logical (incl. short-circuit andalso/orelse), tuples, lists with ++, begin..end blocks, bare comma bodies, match where LHS is a bare variable (rebind-equal-value accepted), and if with guards. Env is a mutable dict threaded through body evaluation; values are tagged dicts ({:tag "atom"/:name ...}, {:tag "nil"}, {:tag "cons" :head :tail}, {:tag "tuple" :elements}). Numbers pass through as SX numbers. Gotcha: SX's parse-number coerces "1.0" → integer 1, so =:= can't distinguish 1 from 1.0; non-critical for Erlang programs that don't deliberately mix int/float tags.
• parser green — lib/erlang/parser.sx + parser-core.sx + parser-expr.sx + parser-module.sx. 52/52 in tests/parse.sx. Covers literals, tuples, lists (incl. [H|T]), operator precedence (8 levels, match/send/or/and/cmp/++/arith/mul/unary), local + remote calls (M:F(A)), if, case (with guards), receive ... after ... end, begin..end blocks, anonymous fun, try..of..catch..after..end with Class:Pattern catch clauses. Module-level: -module(M)., -export([...])., multi-clause functions with guards. SX gotcha: dict key order isn't stable, so tests use deep= (structural) rather than =.
• tokenizer green — lib/erlang/tokenizer.sx + lib/erlang/tests/tokenize.sx. Covers atoms (bare, quoted, node@host), variables, integers (incl. 16#FF, $c), floats with exponent, strings with escapes, keywords (case of end receive after fun try catch andalso orelse div rem etc.), punct (( ) { } [ ] , ; . : :: -> <- <= => << >> | ||), ops (+ - * / = == /= =:= =/= < > =< >= ++ -- ! ?), % line comments. 62/62 green.

Blockers

• (none yet)