diff --git a/plans/go-on-sx.md b/plans/go-on-sx.md index d6a93848..238412aa 100644 --- a/plans/go-on-sx.md +++ b/plans/go-on-sx.md @@ -1,24 +1,64 @@ -# Go-on-SX: Go on the CEK/VM +# Go-on-SX — Go as an SX guest language -Compile Go source to SX AST; the existing CEK evaluator runs it. The unique angle: Go's -goroutines and channels map cleanly onto SX's IO suspension machinery (`perform`/`cek-resume`) -— a goroutine is a `cek-step-loop` running in a cooperative scheduler, a channel send/receive -is a `perform` that suspends until the other end is ready. +Port Go to SX as the **first static-typed, bidirectional-checked guest** in +the rose-ash language family. Goal isn't a production Go compiler; it's to +prove the substrate from a paradigm angle the existing eleven guests don't +cover, and to chisel out the lib/guest kits that statically-typed guests N+1 +and N+2 will need. -End-state goal: **core Go programs running**, including goroutines, channels, defer/panic/recover, -interfaces, and structs. Not a full Go compiler — no generics, no CGo, no full stdlib — but -a faithful runtime for idiomatic Go concurrent programs. +Reference: +- `plans/lib-guest.md` — parent, chiselling discipline, two-language rule. +- `plans/lib-guest-scheduler.md` — sister kit; Go's scheduler pairs with + Erlang's. Extraction gated on this loop reaching Phase 5. +- `plans/lib-guest-static-types-bidirectional.md` — sister kit; Go's + checker pairs with a TBD second consumer. Extraction gated on this loop + reaching Phase 3. +- `plans/erlang-on-sx.md` — reference implementation for paradigm-port: + process model, BIF registry, hot reload, VM bytecode opcodes. -## Ground rules +**Branch:** `loops/go` (loop-style workstream once kicked off). SX files via +`sx-tree` MCP only. -- **Scope:** only touch `lib/go/**` and `plans/go-on-sx.md`. Do **not** edit `spec/`, - `hosts/`, `shared/`, or other `lib//`. -- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here. -- **SX files:** use `sx-tree` MCP tools only. -- **Architecture:** Go source → Go AST → SX AST. No standalone Go evaluator. -- **Concurrency model:** cooperative, not preemptive. Goroutines yield at channel ops and - `time.Sleep`. A round-robin scheduler in SX drives them. -- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes. +## Thesis — why Go + +Eleven guests already live in `lib/`: apl, common-lisp, datalog, erlang, +forth, haskell, hyperscript, js, kernel, lua, minikanren, ocaml, prolog, +ruby, scheme, smalltalk, tcl. Every one is either **dynamically typed** +(most) or **HM-inferred** (haskell, ocaml). None exercise: + +1. **Bidirectional static type checking** — annotation-driven, locally- + inferred, the dominant paradigm of modern statically-typed languages. +2. **Anonymous-channel concurrency** — Go's `chan` and `select`. Erlang has + addressed processes + mailboxes; Go has anonymous values + structural + pairing. Two different vocabularies for the same underlying scheduler + machinery. +3. **Structural interfaces** — `io.Reader` is "anything with this method + signature", not a declared subtype relationship. Different from Haskell + typeclasses (nominal), different from Lua duck typing (no declaration). + +These three together make Go an unusually high-value port for proving SX. +If SX can host Go cleanly, it can host the next decade of mainstream +statically-typed languages (Rust, TS, Swift, Kotlin, Scala 3, Hack) because +they share these three properties. + +Like Erlang-on-SX validated the actor model on the substrate, Go-on-SX +validates the goroutine model + bidirectional types. + +## Non-goals (deliberate) + +Out of scope. Reject feature requests for these without further consideration: + +- **`unsafe` package.** Memory mucking. Skip entirely. +- **CGo.** C interop. Out of scope at every level. +- **Full `reflect`.** Provide enough for `fmt.Println` to render values; + reject the rest. +- **Build tags, modules, vendoring.** Treat source as monolithic. One + package per file, no real import resolution. +- **Production performance.** Conformance tests pass; benchmarks don't. +- **Garbage collection tuning.** SX's GC is what you get. +- **Race detector, escape analysis, inlining.** Out of scope. +- **`os`, `net/http`, full stdlib.** Provide a deliberately small slice + (Phase 8 below). ## Architecture sketch @@ -26,113 +66,330 @@ a faithful runtime for idiomatic Go concurrent programs. Go source text │ ▼ -lib/go/tokenizer.sx — Go tokens: keywords, idents, string/rune/number literals, - │ operators, semicolon insertion rules +lib/go/lex.sx — tokens; ASI; literals; operators + │ (consumes lib/guest/core/lex.sx) ▼ -lib/go/parser.sx — Go AST: package, import, var, const, type, func, struct, - │ interface, goroutine, channel ops, defer, select, for range +lib/go/parse.sx — AST: package/import/var/const/type/func/struct/ + │ interface; expressions; statements + │ (consumes lib/guest/core/pratt.sx + ast.sx) ▼ -lib/go/transpile.sx — Go AST → SX AST - │ +lib/go/types.sx — bidirectional type checker. Synth + check judgments; + │ structural interface satisfaction; pluggable subtype + │ (INDEPENDENT — no lib/guest/static-types-bidirectional + │ yet; this loop builds the first consumer) ▼ -lib/go/runtime.sx — goroutine scheduler, channel primitives, defer stack, - │ panic/recover, interface dispatch, slice/map ops +lib/go/eval.sx — tree-walk evaluator on CEK. Variables as mutable cells; + │ slices = (length, capacity, backing-vector); maps = + │ SX dict; defer stack per frame. ▼ -CEK / VM +lib/go/sched.sx — goroutine scheduler + channels + select + │ (INDEPENDENT — no lib/guest/scheduler yet; this loop + │ builds the first consumer) + ▼ +lib/go/std/ — minimal stdlib slice (fmt, strings, strconv, sync, + time, errors) ``` -Key semantic mappings: -- `go fn()` → spawn new coroutine (SX coroutine primitive, Phase 4 of primitives) -- `ch <- v` (send) → `perform` that suspends until receiver ready; scheduler picks next goroutine -- `v := <-ch` (receive) → `perform` that suspends until sender ready -- `select { case ... }` → scheduler checks all channel readiness, picks first ready -- `defer fn()` → push onto a per-goroutine defer stack; run on return/panic -- `panic(v)` → `raise` the value; `recover()` catches it in deferred function -- `interface{}` → any SX value (duck typed) -- `struct { ... }` → SX hash table with field names as keys -- `slice` → SX vector with length + capacity metadata -- `map[K]V` → SX mutable hash table (Phase 10 of primitives) +Semantic mappings (operational): +- `go fn(args)` → `task-spawn` on the local scheduler. +- `ch <- v` → `task-block` with predicate "receiver waiting on ch". +- `v := <-ch` → `task-block` with predicate "sender waiting on ch". +- `select { case ... }` → `task-block` with predicate "any case ready". +- `defer fn()` → push thunk onto per-frame defer stack; runs LIFO on + return or panic. +- `panic(v)` → raise SX exception; deferred fns run while unwinding. +- `recover()` → CEK exception capture inside a deferred fn. +- `interface{T}` → type-check matches structurally against T's method + set; at runtime, the value carries its concrete-type metadata. +- `struct{...}` → SX dict + type tag; methods are functions in the type's + method table. +- `*T` (pointer) → mutable cell (Common Lisp port did the same). +- `[]T` (slice) → triple (length, capacity, backing-vector). +- `map[K]V` → SX dict; iteration order spec-undefined (v1 = sorted for + determinism — programs that depend on indeterminism fail loudly, which + is a feature not a bug). -## Roadmap +## Conformance scoreboard -### Phase 1 — tokenizer + parser -- [ ] Tokenizer: keywords (`package`, `import`, `func`, `var`, `const`, `type`, `struct`, - `interface`, `go`, `chan`, `select`, `defer`, `return`, `if`, `else`, `for`, `range`, - `switch`, `case`, `default`, `break`, `continue`, `goto`, `fallthrough`, `map`, - `make`, `new`, `nil`, `true`, `false`), automatic semicolon insertion, string literals - (interpreted + raw `` `...` ``), rune literals `'a'`, number literals (int, float, hex, - octal, binary, complex), operators, slices `[:]` -- [ ] Parser: package clause, imports, top-level `func`/`var`/`const`/`type`; function - bodies: short variable decl `:=`, assignments, `if`/`else`, `for`/`range`, `switch`, - `return`, struct literals, slice literals, map literals, composite literals, type - assertions `v.(T)`, method calls `v.Method(args)`, goroutine `go`, channel ops - `<-ch`, `ch <- v`, `defer`, `select` -- [ ] Tests in `lib/go/tests/parse.sx` +Following `lib/erlang/scoreboard.json` precedent. Add +`lib/go/scoreboard.json` on first iteration; populate as suites land. +Suites planned: -### Phase 2 — transpile: basic Go (no goroutines) -- [ ] `go-eval-ast` entry -- [ ] Arithmetic, string ops, comparison, boolean -- [ ] Variables, short decl, assignment, multiple assignment -- [ ] `if`/`else if`/`else` -- [ ] `for` (C-style), `for range` over slice/map/string -- [ ] Functions: named + anonymous, multiple return values (SX multiple values, Phase 8) -- [ ] Structs → SX hash tables; field access `.field`; struct literals `T{f: v}` -- [ ] Slices → SX vectors; `len`, `cap`, `append`, `copy`, slice expressions `s[a:b]` -- [ ] Maps → SX hash tables; `make(map[K]V)`, `m[k]`, `m[k] = v`, `delete(m, k)`, - comma-ok `v, ok := m[k]` -- [ ] Pointers — modelled as single-element mutable vectors; `&x` creates wrapper, `*p` dereferences -- [ ] `fmt.Println`/`fmt.Printf`/`fmt.Sprintf` → SX IO perform (print) -- [ ] 40+ eval tests in `lib/go/tests/eval.sx` +| Suite | Tests target | What it covers | +|---|---|---| +| `lex` | 50+ | Keywords, operators, literals, ASI | +| `parse` | 80+ | All statement & expression shapes | +| `types` | 90+ | Synth, check, interface satisfaction, generics | +| `eval` | 100+ | Tree-walk over typed AST | +| `runtime` | 60+ | Goroutines, channels, select, close | +| `stdlib` | 40+ | fmt, strings, strconv, sync, time, errors | +| `e2e` | 10+ | Complete representative programs | -### Phase 3 — defer / panic / recover -- [ ] Defer stack per function frame — SX list of thunks, run LIFO on return -- [ ] `defer` statement pushes thunk; transpiler wraps function body in try/finally equivalent -- [ ] `panic(v)` → `raise` with Go panic wrapper -- [ ] `recover()` → catches panic value inside a deferred function; returns nil otherwise -- [ ] Panic propagation across call stack until recovered or fatal -- [ ] Tests: defer ordering, panic/recover, panic in goroutine without recover +## Phasing — one feature per commit -### Phase 4 — goroutines + channels -- [ ] Coroutine-based goroutine type using SX coroutine primitive (Phase 4 of primitives) -- [ ] Round-robin scheduler in `lib/go/runtime.sx`: maintains run queue, steps each - goroutine one turn at a time, suspends at channel ops -- [ ] Unbuffered channels: `make(chan T)` → rendezvous point; send suspends until receive - and vice versa. Implemented as a pair of waiting queues + `cek-resume`. -- [ ] Buffered channels: `make(chan T, n)` → circular buffer; send only blocks when full, - receive only blocks when empty -- [ ] `close(ch)` — mark channel closed; receivers drain then get zero value + `false` -- [ ] `select` — scheduler inspects all cases, picks a ready one (random if multiple), - blocks if none ready until at least one becomes ready -- [ ] `go fn(args)` — spawns new goroutine on run queue -- [ ] `time.Sleep(d)` — yields current goroutine, re-queues after d milliseconds - (simulated with IO perform timer) -- [ ] Tests: ping-pong, fan-out, fan-in, select with default, range over channel +Loop-style. Each phase: implement → test → commit → tick `[ ]` → append +Progress-log line → push `origin/loops/go`. -### Phase 5 — interfaces -- [ ] Interface type → SX dict `{:type "T" :methods {...}}` dispatch table -- [ ] `interface{}` / `any` → any SX value (already implicit) -- [ ] Type assertion `v.(T)` → check `:type` field, panic if mismatch -- [ ] Type switch `switch v.(type) { case T: ... }` → dispatches on `:type` -- [ ] Method sets — structs implement interfaces implicitly if they have the right methods -- [ ] Value vs pointer receivers — pointer receiver gets the mutable vector wrapper -- [ ] Built-in interfaces: `error` (`Error() string`), `Stringer` (`String() string`) -- [ ] Tests: interface satisfaction, type assertion, type switch, error interface +### Phase 1 — Tokenizer (`lib/go/lex.sx`) ⬜ +- Consume `lib/guest/core/lex.sx`. Tag the chisel note `consumes-lex`. +- Keywords (25), operators + punctuation (47 distinct), identifiers, + literals (int / float / imaginary / rune / string with raw + interpreted + variants), comments. +- **Automatic semicolon insertion** — the one tricky bit. Newline becomes + `;` after identifier/literal/`)`/`]`/`}` per Go spec § Semicolons. Build + it into the tokenizer, not the parser. +- Tests: golden-token streams for every keyword/operator/literal kind + + ASI edge cases. +- **Acceptance:** lex/ suite at 50+ tests. -### Phase 6 — standard library subset -- [ ] `fmt` — `Println`, `Printf`, `Sprintf`, `Fprintf`, `Errorf`, `Stringer` dispatch -- [ ] `strings` — `Contains`, `HasPrefix`, `HasSuffix`, `Split`, `Join`, `TrimSpace`, - `ToUpper`, `ToLower`, `Replace`, `Index`, `Count`, `Repeat` -- [ ] `strconv` — `Itoa`, `Atoi`, `FormatFloat`, `ParseFloat`, `ParseInt`, `FormatInt` -- [ ] `math` — full surface via SX math primitives (Phase 15) -- [ ] `sort` — `sort.Slice`, `sort.Ints`, `sort.Strings` -- [ ] `errors` — `errors.New`, `errors.Is`, `errors.As` -- [ ] `sync` — `sync.Mutex` (cooperative — just a boolean flag + goroutine queue), - `sync.WaitGroup`, `sync.Once` -- [ ] `io` — `io.Reader`/`io.Writer` interfaces; `io.ReadAll`; `strings.NewReader` +### Phase 2 — Parser (`lib/go/parse.sx`) ⬜ +- Consume `lib/guest/core/pratt.sx` + `lib/guest/core/ast.sx`. Chisel notes + `consumes-pratt consumes-ast`. +- Grammar coverage: + - Declarations: `package`, `import`, `var`, `const`, `type`, `func` + - Types: basic, slice `[]T`, array `[N]T`, map `map[K]V`, chan `chan T`, + func `func(...)...`, struct, interface, pointer `*T` + - Expressions: literals, identifier, call, index `[]`, slice `[a:b]`, + type assertion `v.(T)`, operators + - Statements: `if`/`else`, `for` (C-style + range), `switch`, `select`, + `return`, `defer`, `go`, `break`/`continue`, assign, short-decl `:=`, + send `ch <- v`, recv `<-ch` +- Output: SX-shaped AST per `lib/guest/core/ast.sx` conventions. +- Tests: round-trip parse of hello world, fibonacci, FizzBuzz, goroutine + ping-pong, struct + method. +- **Acceptance:** parse/ suite at 80+ tests. -### Phase 7 — full conformance target -- [ ] Vendor a Go test suite or hand-build 100+ program tests in `lib/go/tests/programs/` -- [ ] Drive scoreboard +### Phase 3 — Bidirectional type checker, MVP (`lib/go/types.sx`) ⬜ +- **Independent implementation.** Do NOT use lib/guest/static-types- + bidirectional/ — that kit doesn't exist yet and depends on this work + for its design. See `plans/lib-guest-static-types-bidirectional.md`. +- Synth + check judgments. Context as a value (per-block scope). +- Coverage MVP: declared-type variables, function signatures (params + + returns), call type-checking, simple composite types (slice, map, chan + element), interface satisfaction (structural match against method sets), + short variable declaration `:=` (synth from RHS). +- **Untyped constants.** `42` has type `untyped int` until contextualised; + this is the canonical pitfall (see Gotchas below). +- Defer: generics (Phase 7), full conversion rules. +- Tests: positive (type-correct programs check) + negative (mismatched + types fail with informative errors carrying AST paths). +- **Acceptance:** types/ suite at 60+ tests. Chisel note `shapes-static- + types-bidirectional` — append a paragraph to the sister plan's design + diary describing what synth/check shape emerged. + +### Phase 4 — Tree-walk evaluator (`lib/go/eval.sx`) ⬜ +- AST-walking interpreter over CEK. Each Go statement maps to one step + function (precedent: `step-sf-if` etc. in spec/evaluator.sx). +- Variables: mutable cells. Pointer semantics: `&x` returns the cell, + `*p` dereferences. +- Slices: triple (length, capacity, backing-vector). `append` honours + capacity-grow per spec. +- Maps: SX dict + key-type metadata. +- Structs: SX dict + type tag. Methods looked up via type's method table. +- Functions: closures over enclosing scope; multiple return values. +- Channels: stub (Phase 5 wires them). +- Tests: arithmetic, control flow, recursion, closures, slices, maps, + structs, methods, pointer semantics, multiple-return. +- **Acceptance:** eval/ suite at 80+ tests. No concurrency yet. + +### Phase 5 — Goroutines + channels + select (`lib/go/sched.sx`) ⬜ +- **Independent implementation.** Do NOT use lib/guest/scheduler/ — that + kit doesn't exist yet and depends on this work for its design. See + `plans/lib-guest-scheduler.md`. +- `go expr` — spawn a goroutine; returns nothing. +- `chan T` — `make(chan T)` creates an unbuffered channel; `make(chan T,n)` + creates a buffered channel (Phase 5b — defer buffer to a sub-phase). +- `<-ch` — receive (blocks until sender ready). +- `ch <- v` — send (blocks until receiver ready for unbuffered, or buffer + has room for buffered). +- `select { case ... }` — non-deterministic multiplexing; `default` makes + it non-blocking. +- `close(ch)` — closes channel. Receive on closed → zero value + ok=false. +- Tests: ping-pong, fan-out/fan-in, work queue, select with default, + select with timeout (via a `time.After`-like stub), close semantics, + range over channel. +- **Acceptance:** runtime/ suite at 40+ tests. Chisel note `shapes- + scheduler` — append a paragraph to the sister plan's design diary + describing what task-spawn/block/wake/yield shape emerged. + +### Phase 5b — Buffered channels + select fairness ⬜ +- Buffered: send blocks only when buffer full; recv only when empty. +- `select` random case ordering (spec mandates pseudo-random; v1 uses a + fixed seed for determinism with a `runtime`-package knob to randomise). +- Tests: buffer-full blocking, buffer-empty blocking, select fairness + over many iterations. +- **Acceptance:** runtime/ +20 tests. + +### Phase 6 — `defer` + panic/recover ⬜ +- Defer stack per function frame; runs LIFO on return (normal or panic). +- `panic(v)` unwinds frames running deferreds; `recover()` inside a + deferred fn captures the panic value and stops unwinding. +- Goroutine panic propagation: a panicking goroutine that doesn't recover + crashes the whole program (honour Go spec, or document divergence). +- Tests: defer order (LIFO), defer + named-return mutation, panic/recover, + panic across goroutines, defer in a loop (push per iter, run on fn + return — common bug). +- **Acceptance:** eval/ +20 tests. + +### Phase 7 — Generics (Go 1.18+) ⬜ +- Type parameters with constraints (type sets: `interface{ int | float64 + }`, `comparable`, `any`). +- Type inference at call sites — basic; the full Go inference algorithm + is notoriously complex. Implement enough for common cases; document + limitations in a Blockers section below. +- Tests: generic function (`func Map[T, U any](xs []T, f func(T) U) []U`), + generic data structure (linked list), constrained type param. +- **Acceptance:** types/ +30 tests. + +### Phase 8 — Minimal stdlib (`lib/go/std/`) ⬜ +- Implement just what's needed for representative programs: + - `fmt` — `Println`, `Printf`, `Sprintf`, `Fprintf`, `Errorf`, + `Stringer` dispatch. Verbs: `%d %s %v %t %f %T %+v`. + - `strings` — `Contains`, `HasPrefix`, `HasSuffix`, `Split`, `Join`, + `TrimSpace`, `ToUpper`, `ToLower`, `Replace`, `Index`, `Count`, + `Repeat`, `NewReader`. + - `strconv` — `Itoa`, `Atoi`, `FormatFloat`, `ParseFloat`, `ParseInt`, + `FormatInt`. + - `errors` — `New`, `Is`, `As`, `Unwrap`. + - `sync` — `Mutex` (cooperative — flag + waiter queue), `WaitGroup`, + `Once`, `RWMutex`. + - `time` — `Now`, `Since`, `After` (channel-returning timer), `Sleep`, + `Duration`, `Time`. + - `io` — `Reader`/`Writer` interfaces; `ReadAll`; `Copy`. + - `sort` — `Slice`, `Ints`, `Strings`. +- Tests: round-trip Itoa/Atoi, fmt verb coverage, sync.WaitGroup with + goroutines, time.After in a select, sort.Slice with custom less fn. +- **Acceptance:** stdlib/ suite at 40+ tests. + +### Phase 9 — End-to-end programs ⬜ +- Complete programs from canonical sources (gopl.io, "concurrency + patterns" talk examples) running end-to-end: + - Concurrent prime sieve + - HTTP-ish ping-pong over stubbed transport + - Word frequency counter + - Pipeline (channel chain) + - Producer/consumer with sync.WaitGroup + - "Bounded parallelism" pattern (worker pool over a job channel) +- **Acceptance:** e2e/ suite at 10+ tests, all passing. + +### Phase 10 — lib/guest extraction enabler ⬜ +- Now that Go has lex+parse+types+eval+sched, sister plans are unblocked + on the Go side. This phase is **doc-only** in `loops/go`: + - Cross-reference `plans/lib-guest-scheduler.md` — mark its Phase 1 + (Go scheduler independent) as complete from Go's side. + - Cross-reference `plans/lib-guest-static-types-bidirectional.md` — + mark its Phase 1 as complete from Go's side. + - Update the chiselling diary in each sister plan with the actual + Go-side surface that emerged. +- **Acceptance:** sister plans cross-referenced + diaries updated. No + new Go code. + +### Phase 11 — VM bytecode opcodes (deferred, optional) ⬜ +- Following Erlang-on-SX Phase 10 precedent: identify hot paths in the + tree-walk evaluator, define Go-specific bytecode opcodes, compile hot + fns through them. Substantial work; only justified if Go programs + exercise enough volume that performance starts mattering. +- **Acceptance:** TBD on demand. + +## Ground rules (loop-style) + +- **Scope:** only `lib/go/**` and this plan. Do not touch `spec/`, + `hosts/`, `shared/`, `lib/guest/**` (read-only consumer at this phase), + or other `lib//`. +- **Consume `lib/guest/core/`** for lex/parse/ast/match/layout. Hand- + rolling defeats the chiselling goal. +- **Do NOT extract into `lib/guest/scheduler/` or `lib/guest/static- + types-bidirectional/` from this loop.** Those extractions are gated on + two consumers AND the discipline of writing each consumer + independently. Extraction is its own workstream after Go and the + second consumer both exist. +- **Substrate gaps** → Blockers entry with minimal repro. Don't fix the + substrate from this loop. Belongs to `sx-improvements.md`. +- **NEVER call `sx_build` without timeout awareness** — 600s watchdog. +- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after every edit. +- **Worktree:** branch `loops/go`, push `origin/loops/go`. Never `main`, + never `architecture`. +- **Commit granularity:** one feature per commit. Short factual messages: + `go: parse short-decl + 6 tests [consumes-pratt]`. Chisel note at end + in brackets. +- **Plan file:** update Progress log + tick boxes every commit. +- **If blocked** for two iterations on the same issue, add to Blockers + and move on. Phases 1-4 are sequential; Phases 5-8 are largely + independent once 4 lands. + +## Chisel discipline (per parent lib-guest plan) + +Every commit ends its message with a chisel note in brackets: + +- `[consumes-X]` — used `lib/guest/X` kit. +- `[shapes-scheduler]` / `[shapes-static-types-bidirectional]` — revealed + something about what the sister lib-guest kits should look like. Add a + paragraph to the relevant sister plan's design diary. +- `[proposes-Y]` — revealed a gap in another existing kit. Blockers entry + in the kit's plan. +- `[nothing]` — pure Go work that didn't touch substrate or lib/guest + story. Acceptable; if it shows up twice in a row, stop and reflect. + +## Go-specific gotchas + +- **ASI (automatic semicolon insertion).** Newline becomes `;` after + identifier/literal/`)`/`]`/`}`. Build into the tokenizer; the Go spec's + "Semicolons" section is unusually precise — follow it literally. +- **Untyped constants.** `42` has type `untyped int` until used in a + context that forces a type. The canonical example: `var x float64 = 42 + / 7` — must compute as `untyped int / untyped int = 6` then convert to + `float64 = 6.0`. Wrong: float-coercing eagerly gives 6.0 prematurely. + Wrong: integer-truncating after coercion gives `5.something`. Test it. +- **Methods vs functions.** `func (r Receiver) Method()` is a method + bound to a type; `func Function(r Receiver)` is just a function. + Methods on pointer-receivers vs value-receivers have asymmetric + satisfaction in interfaces — pointer-receiver methods are NOT in the + value's method set for interface satisfaction. +- **Interface satisfaction is structural and silent.** Type satisfies an + interface if its method set contains all the interface's methods. + Lazy check: at every point a value flows into an interface-typed slot. +- **Channels are first-class values.** Pass them, store them, send them + through other channels. Each channel has identity. +- **`select` with `default`** = non-blocking. Without `default`, blocks + until a case is ready. +- **`nil` is typed.** `var x *int` makes x a `(*int)(nil)`. Comparison + `x == nil` works on typed nil; but `var i interface{} = (*int)(nil); i + == nil` is `false` — i holds a typed-nil-of-type-`*int`, not untyped + nil. The classic Go footgun. Test it. +- **Goroutine panic propagation.** A panicking goroutine that doesn't + recover crashes the whole program. Implement faithfully or document + divergence. +- **`defer` in a loop.** Each iteration pushes; they all run on function + return. Common bug; tests should cover. +- **Iteration order of maps.** Spec: unspecified. v1 = sorted by SX- + canonical key order for determinism; document that programs depending + on iteration order are not Go-conformant. Add a `runtime`-package knob + to enable randomisation later. + +## Style + +- No comments in `.sx` unless non-obvious. Cite Go spec sections inline + for non-obvious decisions (Go's spec is rigorous; citations work). +- No new planning docs — update this plan inline. +- One feature per iteration. Commit. Log. Push. Next. + +## Open questions + +1. **Module/import model.** Go has packages and import paths. Probably + model "package" as one or more `.sx` files in a directory, no real + import resolution against a remote module graph. Decide in Phase 2. +2. **Goroutine identity.** Spec says goroutines have no identity; the + scheduler does internally. Expose to user code? No (not Go). Expose + for debugging? Yes via a `runtime`-package stub. +3. **Error handling: panic-as-exception vs explicit error returns.** Go + strongly prefers explicit errors. Stdlib stubs follow that: `strconv. + Atoi("x")` returns `(0, err)`, not panic. +4. **Memory model.** Go has a happens-before model for atomics + channel + ops. SX runtime is single-threaded under the scheduler — every channel + op is a synchronization point automatically. Don't model relaxed + memory; document the simplification. +5. **Iteration order of maps.** Already addressed in Gotchas; flagged + here as a known divergence from spec. ## Blockers @@ -140,6 +397,11 @@ _(none yet)_ ## Progress log -_Newest first._ +_Newest first. Append one dated entry per commit._ -_(awaiting phase 1)_ +- 2026-05-26 — Plan rewritten to integrate the lib/guest framework + (chiselling discipline, sister plans for scheduler + bidirectional + types, type-checker phase added, conformance scoreboard model adopted). + Original 2026-04-26 draft preserved in git history. Loop not yet + kicked off; Phase 1 (tokenizer) is the first iteration when this loop + spins up. diff --git a/plans/lib-guest-scheduler.md b/plans/lib-guest-scheduler.md new file mode 100644 index 00000000..51e4cbf6 --- /dev/null +++ b/plans/lib-guest-scheduler.md @@ -0,0 +1,235 @@ +# lib/guest/scheduler — extraction plan + +Two distinct concurrency models — Erlang's addressed processes + mailboxes, and +Go's anonymous channels + goroutines — sit on the same underlying machinery: +a fork/yield/block/resume scheduler over CEK io-suspended continuations. This +plan captures that machinery as `lib/guest/scheduler/` so language N+1 with a +new concurrency model costs ~200 lines of model-specific code instead of +re-inventing the scheduler. + +Reference: `plans/lib-guest.md` (parent — two-language rule, stratification), +`plans/erlang-on-sx.md` (first consumer, in production), Go-on-SX (second +consumer, see `plans/go-on-sx.md` once that lands). + +**Branch:** `architecture`. SX files via `sx-tree` MCP only. + +## Thesis + +The substrate already provides what a scheduler needs: CEK io-suspension +(`make-cek-suspended`, `cek-resume`) gives suspendable execution; first-class +environments give each unit of execution its own scope; the trampolined +evaluator means we never blow the host stack. What every guest with concurrency +*re-implements* on top of this is the **fork/yield/block/resume protocol** — +the bookkeeping that decides which suspended computation runs next. + +Two concrete consumers, two different concurrency vocabularies, sharing one +underlying scheduler, is the proof. If only Erlang lives on it, "scheduler kit" +is a euphemism for "Erlang scheduler with a Go skin." The two-language rule +is the gate. + +## Current state (2026-05-26) + +- **Erlang-on-SX** has the full pattern in production: 729/729 conformance, + spawn/send/receive, selective receive, monitor/link, hot reload. The + scheduler logic is currently coupled to Erlang-shaped concepts (PIDs, + mailboxes, links) — extraction-blocking but not extraction-defeating. +- **Go-on-SX** does not exist yet. `plans/go-on-sx.md` is the umbrella plan + (TBD); this scheduler plan is a sibling/dependency. +- **lib/guest/scheduler/** does not exist. The two-language rule blocks + extraction until Go-on-SX independently implements its scheduler. + +**Status: Phase 0 (Erlang shape capture).** No code change in this plan yet. + +## Why the two models actually share a kit + +The non-obvious claim is that Erlang processes and Go goroutines really do +share machinery beneath their different vocabularies. The mapping: + +| Concept | Erlang | Go | Common kit name | +|---|---|---|---| +| Unit of execution | process (PID-addressed) | goroutine (anonymous) | **task** | +| Spawn | `spawn(Fun)` → PID | `go expr` → nothing | `task-spawn` | +| Block target | mailbox match | channel send/recv | `task-block` | +| Wake condition | message arrives | counterpart ready | `task-resume` predicate | +| Yield | `receive` with no match | channel blocked | scheduler hands off | +| Termination | exit reason → linked tasks | panic / return | task lifecycle | +| Selection | selective `receive` | `select` statement | both = "wait for any of N predicates" | + +What the kit owns: +- The **task table** (token → suspended CEK continuation + status). +- The **runnable queue** + scheduling policy (round-robin v1; pluggable). +- The **block→resume protocol**: a blocked task registers a predicate; when + any task changes state, blocked tasks are re-polled; first whose predicate + fires becomes runnable. +- The **fairness/preemption budget** — gas per step before forced yield. + +What each language owns: +- The semantics layer on top: Erlang's PID→task map + mailbox per task + + selective-receive predicates; Go's channel value → blocked-task list per + channel + send/recv pairing + select multiplexing. +- The language-visible API (`spawn`/`!`/`receive` vs `go`/`<-`/`select`). + +This is exactly the lib/guest pattern: extract the dispatch skeleton, keep +the rules in the language layer. + +## API surface (proposed — design only, not yet implemented) + +``` +(make-scheduler &key gas-per-step ;; default 1000 + policy) ;; :round-robin | :fifo + -> scheduler-handle + +(task-spawn sched body-thunk) -> task-token + ;; body-thunk is a 0-arg fn whose body runs as the task. + ;; Returns immediately; task is enqueued runnable. + +(task-current sched) -> task-token + ;; Inside a task, the token of the running task. Useful for self-reference. + +(task-yield sched) -> nil + ;; Voluntary yield. Caller is re-enqueued at the tail of runnable. + +(task-block sched resume-predicate) -> any + ;; Caller suspends. Predicate is (fn () -> resume-value-or-#f). + ;; When predicate returns non-#f, caller resumes with that value. + ;; Predicate is polled on every scheduler tick when there's nothing + ;; obviously runnable. (Optimisation: language layer can wake explicitly — + ;; see task-wake.) + +(task-wake sched task) -> nil + ;; Hint to the scheduler: re-poll this task's resume-predicate now. + ;; Used by sender-side when a receiver might unblock. + +(task-status sched task) -> :runnable | :blocked | :finished | :crashed + +(task-result sched task) -> value | {:error reason} + ;; After :finished or :crashed. + +(scheduler-step sched) -> :ran | :idle | :all-done + ;; Run at most gas-per-step instructions of one task. Caller drives the + ;; loop. + +(scheduler-run sched) -> nil + ;; Run until :all-done. Equivalent to (until (= :all-done (scheduler-step + ;; sched))). +``` + +Notes on the design: +- `task-block` with a resume-predicate is the universal blocking primitive. + Erlang's `receive` is `(task-block sched (fn () (mailbox-match self pat)))`. + Go's `<-ch` is `(task-block sched (fn () (channel-recv-ready ch)))`. +- `task-wake` is the optimisation: instead of polling every blocked task + every step, the language layer wakes the specific task whose predicate + is now likely true. v1 can omit it; performance work later. +- `gas-per-step` gives fairness without true preemption. Tasks that don't + yield within their gas budget are force-yielded by the CEK loop. (CEK + io-suspension already does this for IO; gas budget extends to plain + instructions.) +- No priority/affinity in v1. Both Erlang and Go default to non-priority + scheduling; specialised cases (Erlang's high-priority processes) are + language-layer concerns. + +## Build order — phases + +This is a long-running plan paced against Go-on-SX. Phases are not loop-style +"one commit per phase" — they're milestone gates. + +### Phase 0 — Erlang shape capture (doc-only) ⬜ +- Read `lib/erlang/runtime.sx` scheduler code (currently coupled to Erlang + vocabulary). +- Write a 1-page summary of what's actually a scheduler and what's actually + Erlang. Identify the boundary. +- **Acceptance:** summary committed to this plan as a new section "Erlang + scheduler shape (captured 2026-MM-DD)". No code change. +- **Output:** clear-eyed mental model. Without this, we'll merge Erlang's + scheduler shape into the kit and pretend it generalises. + +### Phase 1 — Go scheduler independent implementation ⬜ +- During Go-on-SX, implement `lib/go/sched.sx` from scratch. Do NOT look at + Erlang's scheduler while doing this. (Or read it once, then close it.) +- Pass Go's channel + goroutine + select conformance tests. +- **Acceptance:** Go scheduler green, lib/go/scoreboard.json includes scheduler + tests, two-consumer rule now passable. +- **Output:** two independent, working implementations of the same idea. + +### Phase 2 — Diff and proposed kit ⬜ +- Side-by-side diff: Erlang's scheduler vs Go's scheduler. Where do they + agree? Where does each have language-specific bookkeeping? +- The diff is the kit. Things in *both* go in `lib/guest/scheduler/`; things + in only one stay in `lib/erlang/` or `lib/go/`. +- Draft `lib/guest/scheduler/api.sx` (signatures only, no body) reflecting the + proposed surface. +- **Acceptance:** API draft circulated for review; agreement that the surface + covers both consumers; no merge yet. + +### Phase 3 — Implement `lib/guest/scheduler/` ⬜ +- Implement the kit per the agreed API. New file(s) in `lib/guest/scheduler/`. +- The kit has its own tests in `lib/guest/scheduler/tests/` — agnostic of any + particular language vocabulary. +- **Acceptance:** kit tests pass. Erlang and Go conformance scoreboards + unchanged (the language implementations still use their own scheduler — + we haven't refactored yet). + +### Phase 4 — Refactor Erlang to use the kit ⬜ +- `lib/erlang/runtime.sx` scheduler logic deleted; replaced with calls into + `lib/guest/scheduler/`. Erlang's PID table, mailbox-per-PID, selective + receive stay in `lib/erlang/`. +- **No-regression gate:** Erlang conformance holds at current pass count + (currently 729/729). Hard requirement. +- **Acceptance:** Erlang scoreboard unchanged; `lib/erlang/runtime.sx` + meaningfully smaller (the scheduler code is gone). + +### Phase 5 — Refactor Go to use the kit ⬜ +- Same exercise for Go. `lib/go/sched.sx` shrinks to channel/goroutine + bookkeeping + delegation. +- **No-regression gate:** Go conformance scoreboard at its current pass + count. +- **Acceptance:** Go scoreboard unchanged; `lib/go/sched.sx` meaningfully + smaller. + +### Phase 6 — Documentation + design-diary close ⬜ +- Document `lib/guest/scheduler/` API in `lib/guest/README.md` (or wherever + the lib/guest API index lives). +- Capture the chiselling diary: what *almost* went in the kit but ended up + language-specific, and why. This is the load-bearing knowledge for the + third consumer when it arrives. +- **Acceptance:** API documented; diary section added to this plan. + +## Two-language rule — gating + +**The rule is hard.** No code in `lib/guest/scheduler/` lands until BOTH +Phase 1 (Go independent) AND Phase 0 (Erlang capture) are complete AND a +review confirms the two implementations actually share machinery in a way +the kit captures. + +If, during Phase 2 diff, we discover that the agreement is shallow (e.g., +both have a runnable queue but the policies are fundamentally incompatible), +the **right outcome is to NOT extract**. Add a "rejected extraction" note to +this plan documenting what we learned and close it. That outcome is fine — +it tells us the two concurrency models aren't actually sister, which is a +real result. + +## Open questions + +- **Preemption.** v1 is cooperative; gas-per-step gives fairness but not + hard preemption. Erlang BEAM does true preemption (reduction counting). + Go uses async preemption (signal-driven since 1.14). Neither extreme fits + cooperatively over CEK. Is gas-per-step + voluntary yield enough? Probably + for v1; revisit if a guest needs hard real-time. +- **Priority/affinity.** Both Erlang and Go can run without it. Defer. +- **Distribution.** Erlang nodes, Go's distributed channels — both are + language-specific layers on top of the local scheduler. Out of scope. +- **Cancellation.** Go has `context.Context`; Erlang has `exit/2`. Both + bottom out at "deliver an exception to a task." Worth modelling? Probably + as a kit primitive `(task-cancel sched task reason)` that delivers an + exception via CEK exception machinery, language layer wraps it. +- **Third consumer.** If/when JS-on-SX gets a proper async/await + Promise + scheduler, that'd be a great third consumer to validate the kit didn't + over-fit to Erlang+Go. + +## Progress log + +_Newest first. Append one dated entry per milestone landed._ + +- 2026-05-26 — Plan drafted. Phase 0 unstarted. Awaiting Go-on-SX to begin + Phase 1. diff --git a/plans/lib-guest-static-types-bidirectional.md b/plans/lib-guest-static-types-bidirectional.md new file mode 100644 index 00000000..e941cff9 --- /dev/null +++ b/plans/lib-guest-static-types-bidirectional.md @@ -0,0 +1,287 @@ +# lib/guest/static-types-bidirectional — design-diary plan + +Capture the dispatch skeleton of bidirectional type checking +(synthesis/checking judgments, context as a value, pluggable subtyping and +unification) as `lib/guest/static-types-bidirectional/`, so static-typed +guest languages that aren't Hindley-Milner-inferred cost ~300 lines of +language-specific rules instead of re-inventing the checker plumbing. + +Reference: `plans/lib-guest.md` (parent — two-language rule, stratification), +`lib/guest/hm.sx` (sister module — full Hindley-Milner for inference-heavy +languages like Haskell-on-SX), Go-on-SX (planned first consumer), TBD second +consumer. + +**Branch:** `architecture`. SX files via `sx-tree` MCP only. + +## Thesis + +`lib/guest/hm.sx` covers languages where the user writes few type annotations +and the checker infers the rest globally (Haskell-on-SX, an eventual ML port, +a typed-Scheme-with-Damas-Milner). But most modern statically-typed languages +in actual production — Go, Rust, Swift, TypeScript, Kotlin, Scala 3, Hack — +do **bidirectional checking instead**: declarations carry annotations, locals +are inferred from immediate context, return types thread inwards from call +sites. This isn't a weaker form of HM; it's a different design that scales +better to mutation, subtyping, ad-hoc polymorphism, and gradual typing — +none of which HM handles cleanly. + +If `lib/guest/` is going to credibly host the next decade of statically-typed +languages, it needs a bidirectional kit alongside `hm.sx`. They're sisters, +not rivals. + +**This plan is a design diary, not an implementation queue.** The two-language +rule blocks extraction until two consumers exist. Go-on-SX is the first; the +second is TBD. Until then, this plan documents what the API surface *should* +be based on a single consumer, openly acknowledging that the second consumer +will revise it. + +## Current state (2026-05-26) + +- `lib/guest/hm.sx` exists, used by Haskell-on-SX. 180 lines. The HM kit is + the sister extraction this plan complements. +- No bidirectional kit anywhere in `lib/guest/`. +- Go-on-SX does not exist yet. When it does, `lib/go/types.sx` will be the + first consumer. +- Second consumer is unidentified. Most likely candidates, in order: + 1. **TypeScript-on-SX** — purely structural, gradual typing, the most- + popular bidirectional language alive. Natural pair. + 2. **Rust-on-SX** — bidirectional with substantial extras (lifetimes, + traits, borrow checking). Heavyweight; lifetimes don't go in this kit. + 3. **Typed Racket subset** — if anyone ports it. Bidirectional + gradual. + 4. **Hack / Flow / Python-with-types** — same shape. + +**Status: Phase 0 (literature survey).** No code in this plan yet. + +## Why bidirectional, not HM (for the languages that need it) + +Five reasons HM doesn't fit these languages: + +1. **Subtyping.** HM unification requires equality of types; subtyping + requires a different judgment (`t ⊑ u`). Go's `interface{}` accepts any + concrete type that satisfies it — subtyping, not unification. +2. **Mutation.** HM's let-polymorphism interacts pathologically with + mutable references (the value restriction). Go, Rust, TS all have + first-class mutation and need rules that handle it directly. +3. **Annotations as ground truth.** Bidirectional treats declared types as + *given*, then propagates them. HM treats every type as a variable to be + solved. For languages where annotations are expected, bidirectional is + the natural shape. +4. **Generics with constraints.** Go's type parameters carry constraints + (`type T comparable`); Rust has trait bounds. HM has typeclasses but + they're orthogonal to its constraint solver. Bidirectional weaves + constraints into the checking rules naturally. +5. **Gradual typing.** TS's `any`, Hack's pessimistic mode, Python's + `Any` — gradual checking is built on bidirectional's "check or skip" + distinction. HM either checks or it doesn't. + +These languages collectively are the majority of new statically-typed code. +Hosting them on lib/guest at all requires the bidirectional shape. + +## API surface (proposed — design only, will revise with second consumer) + +``` +;; --- judgments --- + +(synth ctx expr) -> {:type T} | {:error msg} + ;; "expr synthesises type T in context ctx." + ;; Used at function calls (arg types known), let bindings, literals. + +(check ctx expr expected-type) -> :ok | {:error msg} + ;; "expr checks against expected-type in context ctx." + ;; Used in function bodies (return type known), arguments (param type known), + ;; assignments (LHS type known). + +;; --- context --- + +(make-ctx) -> ctx +(ctx-extend ctx name type) -> ctx ;; functional update +(ctx-lookup ctx name) -> type | nil + +;; --- pluggable rules --- + +(register-synth-rule! kit ast-tag synth-fn) -> nil + ;; ast-tag: a keyword identifying the AST node shape (eg. :call :let :lit-int) + ;; synth-fn: (ctx node) -> {:type T} | {:error msg} + +(register-check-rule! kit ast-tag check-fn) -> nil + ;; check-fn: (ctx node expected-type) -> :ok | {:error msg} + +(register-type-equiv! kit pred) -> nil + ;; pred: (t1 t2) -> bool. The "are these types compatible" predicate. + ;; For Go: structural-interface-match-or-equal. + ;; For TS: structural-equality-with-any-bidirectional-bottom. + ;; For Rust: nominal equality + trait obligations. + +(register-subtype! kit pred) -> nil + ;; pred: (sub super) -> bool. Optional; defaults to type-equiv. + ;; Go has no subtyping between concrete types but interface satisfaction + ;; is morally subtyping. TS has structural subtyping properly. + +(register-unify! kit unifier) -> nil + ;; Optional; for type-variable resolution (generics). + ;; unifier: (t1 t2 subst) -> {:subst s'} | {:error msg} + +;; --- driver --- + +(make-kit) -> kit +(check-program kit ctx program) -> {:ok ctx'} | {:error msg path-to-error} +``` + +Design notes: +- **The kit dispatches on AST tags**, which is what makes it pluggable. Each + language registers rules for its node types. There's no hardcoded set of + expression shapes in the kit. +- **Synth and check are mutually recursive.** Inside a synth-rule for `call`, + the rule synthesises the function's type, then `check`s each argument + against the corresponding parameter type. Inside a check-rule for `lambda`, + the rule pulls argument types from the expected function type and + `synth`s the body. This pingponging is the bidirectional core. +- **Pluggable type-equiv + subtype + unify** is the three-knob shape. Pierce + & Turner ("Local Type Inference") and Dunfield & Krishnaswami ("Sound and + Complete Bidirectional Typechecking") both factor it this way. +- **No type variables in the core API.** Generics handling is a kit + *extension*: when a language registers a `unify` predicate, the kit + threads a substitution through synth/check. Languages without generics + (early Go) leave it null. +- **Errors carry a path.** `{:error msg path}` where path is a list of AST + tags leading to the failure. Good error messages are why bidirectional is + practical; the kit must support them. + +## What's NOT in the kit (language-layer concerns) + +Per the chiselling discipline, the kit is the dispatch skeleton; rules stay +in the language. Specifically: + +- **The literal type table.** Go's `42` is `untyped int` until contextualised; + TS's `42` is the literal type `42`. Each language ships its own. +- **Specific subtyping rules.** Go's interface satisfaction is recursive + structural matching against method sets. TS's depends on object property + satisfaction. Each language ships its own predicate. +- **Generics constraint solving.** Go's type-set-based constraints, Rust's + trait bounds, TS's conditional types — each is non-trivial and language- + specific. The kit threads a substitution; the language defines what's in + it. +- **Effects, lifetimes, ownership.** Rust's borrow checker is not a type + checker in the bidirectional-kit sense — it's a separate dataflow pass. + Out of scope. +- **Gradual fallback.** TS's `any` lets unchecked code coexist with checked + code. The kit supports this via "check returns :ok on a sentinel any-type" + but the sentinel is registered by the language. + +## Build order — phases + +### Phase 0 — Literature survey + Go's type system specifics ⬜ +- Read: Pierce & Turner "Local Type Inference" (2000); Dunfield & Krishnaswami + "Sound and Complete Bidirectional Typechecking for Higher-Rank Polymorphism" + (2013, 2019 revision); the Go language spec § "Types" + "Expressions". +- Survey how Rust / TS / Kotlin / Scala 3 implement bidirectional in practice + (their compilers are open source). Note where they diverge. +- Output: a short summary section "Bidirectional design space (captured + 2026-MM-DD)" appended to this plan. Specifically: list every place + language implementations diverge, so we can predict which divergences will + show up between Go and the second consumer. +- **Acceptance:** survey committed to this plan. No code. + +### Phase 1 — Go independent implementation ⬜ +- During Go-on-SX, implement `lib/go/types.sx` from scratch. Do not write + with extraction in mind — write the simplest Go-specific bidirectional + checker. +- Hit Go's distinctive type-system features: untyped constants, interface + satisfaction (structural), generics (Go 1.18 type parameters with type-set + constraints — defer this if scope explodes). +- Pass Go's type-checker conformance tests. +- **Acceptance:** Go conformance scoreboard includes type-checker tests, all + passing. +- **Output:** one consumer. Two-language rule still not met; no extraction. + +### Phase 2 — Pick + start the second consumer ⬜ +- Decide between TS, Rust-subset, or typed-Scheme-subset. Recommendation: + **TypeScript** — most-different from Go (gradual, structural everywhere), + testing the kit's range maximally. Rust's lifetime/borrow machinery isn't + part of this kit, so a Rust port wouldn't actually exercise the kit very + hard. +- Implement just enough of the second language to type-check a non-trivial + function. Don't port the whole language; port the type checker. +- **Acceptance:** second consumer's type checker green on its small slice. + +### Phase 3 — Diff and proposed kit ⬜ +- Side-by-side: Go's checker vs the second consumer's checker. Where do they + agree (the kit). Where does each diverge (the language). +- Draft `lib/guest/static-types-bidirectional/api.sx` (signatures only). +- Compare against the API sketch in this plan. The API WILL change at this + step; that's the whole point of having two consumers. +- **Acceptance:** revised API committed to this plan; agreement that both + consumers can adopt it. + +### Phase 4 — Implement the kit ⬜ +- `lib/guest/static-types-bidirectional/` with the agreed API. Kit tests in + `lib/guest/static-types-bidirectional/tests/` — using a minimal "toy" + language (synth-rule for `:int`, check-rule for `:lambda`) to verify the + dispatch skeleton works. +- **Acceptance:** kit tests pass; both consumers' scoreboards still green + with their own implementations. + +### Phase 5 — Refactor both consumers to use the kit ⬜ +- Go: `lib/go/types.sx` becomes a thin layer over the kit — registers Go's + synth/check/equiv rules, calls `check-program`. Lifecycle code shrinks. +- Second consumer: same exercise. +- **No-regression gate:** both consumers' conformance scoreboards unchanged. +- **Acceptance:** both `lib//types.sx` files meaningfully smaller; kit + is doing real work. + +### Phase 6 — Documentation + chiselling diary ⬜ +- Document the API in lib/guest's README index. +- Diary section in this plan: what we considered putting in the kit but + ended up keeping language-specific, and why. +- **Acceptance:** documentation present; diary captured. + +## Two-language rule — gating + +Same as `lib-guest-scheduler.md`. The kit does not exist until both consumers +independently work AND we've reviewed the diff AND we believe the shared +skeleton is real. Rejected-extraction is a valid outcome. + +## Relationship to `lib/guest/hm.sx` + +Sister modules, not rivals. Some languages will use HM (full inference, +let-polymorphism); some will use bidirectional (annotation-driven, subtyping- +friendly). Some might use both — Scala-on-SX, hypothetically, has local-type- +inference in expressions and global-HM-style constraint solving in implicit +resolution. The kit boundaries are: + +- `hm.sx` — unification-based, whole-expression inference. Damas-Milner core. + Best for: ML family, Haskell, OCaml subset, Standard ML. +- `static-types-bidirectional/` — synth/check judgments, pluggable equiv + + subtype. Best for: Go, Rust, TS, Kotlin, Swift, Scala 3, Hack. + +A language can call into both: bidirectional for the surface, HM-style +unification inside generics resolution. That's actually how Scala 3 works. +The kits compose; design accordingly. + +## Open questions + +- **Variance.** Go has none; TS has covariant/contravariant/bivariant; Rust + has variance markers per type parameter. Does the kit need a variance + predicate as a fourth pluggable knob? Probably yes, but defer until the + second consumer forces the question. +- **Effect tracking.** Some bidirectional checkers (Koka, Eff, certain + capability-effect TS variants) track effects in types. Out of scope for + v1; the kit must not actively prevent it though. +- **Refinement types.** TS has narrowing (`typeof x === "string"` refines + `x` to `string`); Hack and Flow are similar. These layer above the kit + (the kit's `check` returns a refined context as part of `:ok`). Sketch + this in Phase 3 if TS is the second consumer. +- **Error recovery.** Real-world type checkers don't halt on first error; + they recover and continue to surface as many errors as possible. The kit + needs an error-accumulation mode. Design it in Phase 4. +- **Performance.** For toy languages, naive synth/check is fine. For Go- + sized programs, the checker has to be memoised on synthesised types of + subexpressions. Not a v1 concern; flag if it bites. + +## Progress log + +_Newest first. Append one dated entry per milestone landed._ + +- 2026-05-26 — Plan drafted as design diary. Phase 0 unstarted. Gated on + Go-on-SX (first consumer) and a TBD second consumer (recommendation: + TypeScript). No code yet — kit cannot exist before two consumers do.