rose-ash/plans/go-on-sx.md

Go-on-SX — Go as an SX guest language

Port Go to SX as the first static-typed, bidirectional-checked guest in the rose-ash language family. Goal isn't a production Go compiler; it's to prove the substrate from a paradigm angle the existing eleven guests don't cover, and to chisel out the lib/guest kits that statically-typed guests N+1 and N+2 will need.

Reference:

• plans/lib-guest.md — parent, chiselling discipline, two-language rule.
• plans/lib-guest-scheduler.md — sister kit; Go's scheduler pairs with Erlang's. Extraction gated on this loop reaching Phase 5.
• plans/lib-guest-static-types-bidirectional.md — sister kit; Go's checker pairs with a TBD second consumer. Extraction gated on this loop reaching Phase 3.
• plans/erlang-on-sx.md — reference implementation for paradigm-port: process model, BIF registry, hot reload, VM bytecode opcodes.

Branch: loops/go (loop-style workstream once kicked off). SX files via sx-tree MCP only.

Thesis — why Go

Eleven guests already live in lib/: apl, common-lisp, datalog, erlang, forth, haskell, hyperscript, js, kernel, lua, minikanren, ocaml, prolog, ruby, scheme, smalltalk, tcl. Every one is either dynamically typed (most) or HM-inferred (haskell, ocaml). None exercise:

1. Bidirectional static type checking — annotation-driven, locally- inferred, the dominant paradigm of modern statically-typed languages.
2. Anonymous-channel concurrency — Go's chan and select. Erlang has addressed processes + mailboxes; Go has anonymous values + structural pairing. Two different vocabularies for the same underlying scheduler machinery.
3. Structural interfaces — io.Reader is "anything with this method signature", not a declared subtype relationship. Different from Haskell typeclasses (nominal), different from Lua duck typing (no declaration).

These three together make Go an unusually high-value port for proving SX. If SX can host Go cleanly, it can host the next decade of mainstream statically-typed languages (Rust, TS, Swift, Kotlin, Scala 3, Hack) because they share these three properties.

Like Erlang-on-SX validated the actor model on the substrate, Go-on-SX validates the goroutine model + bidirectional types.

Non-goals (deliberate)

Out of scope. Reject feature requests for these without further consideration:

• unsafe package. Memory mucking. Skip entirely.
• CGo. C interop. Out of scope at every level.
• Full reflect. Provide enough for fmt.Println to render values; reject the rest.
• Build tags, modules, vendoring. Treat source as monolithic. One package per file, no real import resolution.
• Production performance. Conformance tests pass; benchmarks don't.
• Garbage collection tuning. SX's GC is what you get.
• Race detector, escape analysis, inlining. Out of scope.
• os, net/http, full stdlib. Provide a deliberately small slice (Phase 8 below).

Architecture sketch

Go source text
    │
    ▼
lib/go/lex.sx       — tokens; ASI; literals; operators
    │                 (consumes lib/guest/core/lex.sx)
    ▼
lib/go/parse.sx     — AST: package/import/var/const/type/func/struct/
    │                 interface; expressions; statements
    │                 (consumes lib/guest/core/pratt.sx + ast.sx)
    ▼
lib/go/types.sx     — bidirectional type checker. Synth + check judgments;
    │                 structural interface satisfaction; pluggable subtype
    │                 (INDEPENDENT — no lib/guest/static-types-bidirectional
    │                  yet; this loop builds the first consumer)
    ▼
lib/go/eval.sx      — tree-walk evaluator on CEK. Variables as mutable cells;
    │                 slices = (length, capacity, backing-vector); maps =
    │                 SX dict; defer stack per frame.
    ▼
lib/go/sched.sx     — goroutine scheduler + channels + select
    │                 (INDEPENDENT — no lib/guest/scheduler yet; this loop
    │                  builds the first consumer)
    ▼
lib/go/std/         — minimal stdlib slice (fmt, strings, strconv, sync,
                      time, errors)

Semantic mappings (operational):

• go fn(args) → task-spawn on the local scheduler.
• ch <- v → task-block with predicate "receiver waiting on ch".
• v := <-ch → task-block with predicate "sender waiting on ch".
• select { case ... } → task-block with predicate "any case ready".
• defer fn() → push thunk onto per-frame defer stack; runs LIFO on return or panic.
• panic(v) → raise SX exception; deferred fns run while unwinding.
• recover() → CEK exception capture inside a deferred fn.
• interface{T} → type-check matches structurally against T's method set; at runtime, the value carries its concrete-type metadata.
• struct{...} → SX dict + type tag; methods are functions in the type's method table.
• *T (pointer) → mutable cell (Common Lisp port did the same).
• []T (slice) → triple (length, capacity, backing-vector).
• map[K]V → SX dict; iteration order spec-undefined (v1 = sorted for determinism — programs that depend on indeterminism fail loudly, which is a feature not a bug).

Conformance scoreboard

Following lib/erlang/scoreboard.json precedent. Add lib/go/scoreboard.json on first iteration; populate as suites land. Suites planned:

Suite	Tests target	What it covers
lex	50+	Keywords, operators, literals, ASI
parse	80+	All statement & expression shapes
types	90+	Synth, check, interface satisfaction, generics
eval	100+	Tree-walk over typed AST
runtime	60+	Goroutines, channels, select, close
stdlib	40+	fmt, strings, strconv, sync, time, errors
e2e	10+	Complete representative programs

Phasing — one feature per commit

Loop-style. Each phase: implement → test → commit → tick [ ] → append Progress-log line → push origin/loops/go.

Phase 1 — Tokenizer (lib/go/lex.sx) ✅

• Scaffold + scoreboard + conformance runner (consumes lib/guest/lex.sx)
• Identifiers + 25 keywords
• Decimal integer literals
• Interpreted string literals "..." with \n \t \r \\ \" \' escapes
• Rune literals 'x' (single char + simple escapes)
• Line + block comments (block w/ newline triggers ASI)
• Common operator/punct set incl. := <- ++ -- == != <= >= && || ...
• Automatic semicolon insertion (Go spec § Semicolons) — newline, EOF, and block-comment-with-newline trigger ; after ident/int/string/rune/{break,continue,fallthrough,return}/{++,--,),],}}.
• Float / imaginary literals (decimal floats: 3.14 .5 1. 1e10 1.5e-3; imag: 2i 3.14i 1e2i; hex floats 0x1.fp0 deferred)
• Raw string literals `...` (multi-line, no escape processing, \r stripped per Go spec § String literals; same "string" type as interpreted strings)
• Hex/octal/binary integer literals (0x… 0o… 0b…) + underscores (legacy 0123 octal also accepted; consumes lex-hex-digit?)
• Full operator set audit (47 distinct per Go spec, plus ~ for generics type-sets). Exhaustive coverage tests in op-audit: block.
• Acceptance: lex/ suite at 50+ tests. Current: 129/129. Phase 1 done — hex floats deferred (rare). Move to Phase 2 next.

Phase 2 — Parser (lib/go/parse.sx) ⬜

• Parser scaffold + Go operator-precedence table (entry shape from lib/guest/pratt.sx) + primary expressions (int/float/imag/string/ rune/ident → ast-literal / ast-var via lib/guest/ast.sx).
• Binary operators (Pratt precedence climbing using pratt-op-lookup + Go precedence table). Operator app emitted as (ast-app (ast-var OP) [LHS RHS]); left-assoc raises right-min by 1.
• Unary operators (!x, -x, ^x, *p, &v, <-ch).
• Function calls f(a, b) and member access x.field.
• Index x[i] and slice x[a:b]/x[a:b:c].
• Type assertion v.(T).
• Type expressions: basic, slice []T, array [N]T, map map[K]V, chan chan T / chan<- T / <-chan T, func, struct, interface, pointer *T.
• Composite literals: T{...}, []T{...}, map[K]V{...}, struct{...}{...}.
• Declarations: package, import, var, const, type, func (including methods, parameter lists, return types).
• Statements: if/else, for (C-style + range), switch (expr + type), select, return, defer, go, break/continue, assign, short-decl :=, send ch <- v, recv <-ch.
• End-to-end: hello-world, fibonacci, FizzBuzz, goroutine ping-pong, struct + method.
• Acceptance: parse/ suite at 80+ tests. Current: 26/26.

Phase 3 — Bidirectional type checker, MVP (lib/go/types.sx) ⬜

• Independent implementation. Do NOT use lib/guest/static-types- bidirectional/ — that kit doesn't exist yet and depends on this work for its design. See plans/lib-guest-static-types-bidirectional.md.
• Synth + check judgments. Context as a value (per-block scope).
• Coverage MVP: declared-type variables, function signatures (params + returns), call type-checking, simple composite types (slice, map, chan element), interface satisfaction (structural match against method sets), short variable declaration := (synth from RHS).
• Untyped constants. 42 has type untyped int until contextualised; this is the canonical pitfall (see Gotchas below).
• Defer: generics (Phase 7), full conversion rules.
• Tests: positive (type-correct programs check) + negative (mismatched types fail with informative errors carrying AST paths).
• Acceptance: types/ suite at 60+ tests. Chisel note shapes-static- types-bidirectional — append a paragraph to the sister plan's design diary describing what synth/check shape emerged.

Phase 4 — Tree-walk evaluator (lib/go/eval.sx) ⬜

• AST-walking interpreter over CEK. Each Go statement maps to one step function (precedent: step-sf-if etc. in spec/evaluator.sx).
• Variables: mutable cells. Pointer semantics: &x returns the cell, *p dereferences.
• Slices: triple (length, capacity, backing-vector). append honours capacity-grow per spec.
• Maps: SX dict + key-type metadata.
• Structs: SX dict + type tag. Methods looked up via type's method table.
• Functions: closures over enclosing scope; multiple return values.
• Channels: stub (Phase 5 wires them).
• Tests: arithmetic, control flow, recursion, closures, slices, maps, structs, methods, pointer semantics, multiple-return.
• Acceptance: eval/ suite at 80+ tests. No concurrency yet.

Phase 5 — Goroutines + channels + select (lib/go/sched.sx) ⬜

• Independent implementation. Do NOT use lib/guest/scheduler/ — that kit doesn't exist yet and depends on this work for its design. See plans/lib-guest-scheduler.md.
• go expr — spawn a goroutine; returns nothing.
• chan T — make(chan T) creates an unbuffered channel; make(chan T,n) creates a buffered channel (Phase 5b — defer buffer to a sub-phase).
• <-ch — receive (blocks until sender ready).
• ch <- v — send (blocks until receiver ready for unbuffered, or buffer has room for buffered).
• select { case ... } — non-deterministic multiplexing; default makes it non-blocking.
• close(ch) — closes channel. Receive on closed → zero value + ok=false.
• Tests: ping-pong, fan-out/fan-in, work queue, select with default, select with timeout (via a time.After-like stub), close semantics, range over channel.
• Acceptance: runtime/ suite at 40+ tests. Chisel note shapes- scheduler — append a paragraph to the sister plan's design diary describing what task-spawn/block/wake/yield shape emerged.

Phase 5b — Buffered channels + select fairness ⬜

• Buffered: send blocks only when buffer full; recv only when empty.
• select random case ordering (spec mandates pseudo-random; v1 uses a fixed seed for determinism with a runtime-package knob to randomise).
• Tests: buffer-full blocking, buffer-empty blocking, select fairness over many iterations.
• Acceptance: runtime/ +20 tests.

Phase 6 — defer + panic/recover ⬜

• Defer stack per function frame; runs LIFO on return (normal or panic).
• panic(v) unwinds frames running deferreds; recover() inside a deferred fn captures the panic value and stops unwinding.
• Goroutine panic propagation: a panicking goroutine that doesn't recover crashes the whole program (honour Go spec, or document divergence).
• Tests: defer order (LIFO), defer + named-return mutation, panic/recover, panic across goroutines, defer in a loop (push per iter, run on fn return — common bug).
• Acceptance: eval/ +20 tests.

Phase 7 — Generics (Go 1.18+) ⬜

• Type parameters with constraints (type sets: interface{ int | float64 }, comparable, any).
• Type inference at call sites — basic; the full Go inference algorithm is notoriously complex. Implement enough for common cases; document limitations in a Blockers section below.
• Tests: generic function (func Map[T, U any](xs []T, f func(T) U) []U), generic data structure (linked list), constrained type param.
• Acceptance: types/ +30 tests.

Phase 8 — Minimal stdlib (lib/go/std/) ⬜

• Implement just what's needed for representative programs:
  • fmt — Println, Printf, Sprintf, Fprintf, Errorf, Stringer dispatch. Verbs: %d %s %v %t %f %T %+v.
  • strings — Contains, HasPrefix, HasSuffix, Split, Join, TrimSpace, ToUpper, ToLower, Replace, Index, Count, Repeat, NewReader.
  • strconv — Itoa, Atoi, FormatFloat, ParseFloat, ParseInt, FormatInt.
  • errors — New, Is, As, Unwrap.
  • sync — Mutex (cooperative — flag + waiter queue), WaitGroup, Once, RWMutex.
  • time — Now, Since, After (channel-returning timer), Sleep, Duration, Time.
  • io — Reader/Writer interfaces; ReadAll; Copy.
  • sort — Slice, Ints, Strings.
• Tests: round-trip Itoa/Atoi, fmt verb coverage, sync.WaitGroup with goroutines, time.After in a select, sort.Slice with custom less fn.
• Acceptance: stdlib/ suite at 40+ tests.

Phase 9 — End-to-end programs ⬜

• Complete programs from canonical sources (gopl.io, "concurrency patterns" talk examples) running end-to-end:
  • Concurrent prime sieve
  • HTTP-ish ping-pong over stubbed transport
  • Word frequency counter
  • Pipeline (channel chain)
  • Producer/consumer with sync.WaitGroup
  • "Bounded parallelism" pattern (worker pool over a job channel)
• Acceptance: e2e/ suite at 10+ tests, all passing.

Phase 10 — lib/guest extraction enabler ⬜

• Now that Go has lex+parse+types+eval+sched, sister plans are unblocked on the Go side. This phase is doc-only in loops/go:
  • Cross-reference plans/lib-guest-scheduler.md — mark its Phase 1 (Go scheduler independent) as complete from Go's side.
  • Cross-reference plans/lib-guest-static-types-bidirectional.md — mark its Phase 1 as complete from Go's side.
  • Update the chiselling diary in each sister plan with the actual Go-side surface that emerged.
• Acceptance: sister plans cross-referenced + diaries updated. No new Go code.

Phase 11 — VM bytecode opcodes (deferred, optional) ⬜

• Following Erlang-on-SX Phase 10 precedent: identify hot paths in the tree-walk evaluator, define Go-specific bytecode opcodes, compile hot fns through them. Substantial work; only justified if Go programs exercise enough volume that performance starts mattering.
• Acceptance: TBD on demand.

Ground rules (loop-style)

• Scope: only lib/go/** and this plan. Do not touch spec/, hosts/, shared/, lib/guest/** (read-only consumer at this phase), or other lib/<lang>/.
• Consume lib/guest/core/ for lex/parse/ast/match/layout. Hand- rolling defeats the chiselling goal.
• Do NOT extract into lib/guest/scheduler/ or lib/guest/static- types-bidirectional/ from this loop. Those extractions are gated on two consumers AND the discipline of writing each consumer independently. Extraction is its own workstream after Go and the second consumer both exist.
• Substrate gaps → Blockers entry with minimal repro. Don't fix the substrate from this loop. Belongs to sx-improvements.md.
• NEVER call sx_build without timeout awareness — 600s watchdog.
• SX files: sx-tree MCP tools ONLY. sx_validate after every edit.
• Worktree: branch loops/go, push origin/loops/go. Never main, never architecture.
• Commit granularity: one feature per commit. Short factual messages: go: parse short-decl + 6 tests [consumes-pratt]. Chisel note at end in brackets.
• Plan file: update Progress log + tick boxes every commit.
• If blocked for two iterations on the same issue, add to Blockers and move on. Phases 1-4 are sequential; Phases 5-8 are largely independent once 4 lands.

Chisel discipline (per parent lib-guest plan)

Every commit ends its message with a chisel note in brackets:

• [consumes-X] — used lib/guest/X kit.
• [shapes-scheduler] / [shapes-static-types-bidirectional] — revealed something about what the sister lib-guest kits should look like. Add a paragraph to the relevant sister plan's design diary.
• [proposes-Y] — revealed a gap in another existing kit. Blockers entry in the kit's plan.
• [nothing] — pure Go work that didn't touch substrate or lib/guest story. Acceptable; if it shows up twice in a row, stop and reflect.

Go-specific gotchas

• ASI (automatic semicolon insertion). Newline becomes ; after identifier/literal/)/]/}. Build into the tokenizer; the Go spec's "Semicolons" section is unusually precise — follow it literally.
• Untyped constants. 42 has type untyped int until used in a context that forces a type. The canonical example: var x float64 = 42 / 7 — must compute as untyped int / untyped int = 6 then convert to float64 = 6.0. Wrong: float-coercing eagerly gives 6.0 prematurely. Wrong: integer-truncating after coercion gives 5.something. Test it.
• Methods vs functions. func (r Receiver) Method() is a method bound to a type; func Function(r Receiver) is just a function. Methods on pointer-receivers vs value-receivers have asymmetric satisfaction in interfaces — pointer-receiver methods are NOT in the value's method set for interface satisfaction.
• Interface satisfaction is structural and silent. Type satisfies an interface if its method set contains all the interface's methods. Lazy check: at every point a value flows into an interface-typed slot.
• Channels are first-class values. Pass them, store them, send them through other channels. Each channel has identity.
• select with default = non-blocking. Without default, blocks until a case is ready.
• nil is typed. var x *int makes x a (*int)(nil). Comparison x == nil works on typed nil; but var i interface{} = (*int)(nil); i == nil is false — i holds a typed-nil-of-type-*int, not untyped nil. The classic Go footgun. Test it.
• Goroutine panic propagation. A panicking goroutine that doesn't recover crashes the whole program. Implement faithfully or document divergence.
• defer in a loop. Each iteration pushes; they all run on function return. Common bug; tests should cover.
• Iteration order of maps. Spec: unspecified. v1 = sorted by SX- canonical key order for determinism; document that programs depending on iteration order are not Go-conformant. Add a runtime-package knob to enable randomisation later.

Style

• No comments in .sx unless non-obvious. Cite Go spec sections inline for non-obvious decisions (Go's spec is rigorous; citations work).
• No new planning docs — update this plan inline.
• One feature per iteration. Commit. Log. Push. Next.

Open questions

1. Module/import model. Go has packages and import paths. Probably model "package" as one or more .sx files in a directory, no real import resolution against a remote module graph. Decide in Phase 2.
2. Goroutine identity. Spec says goroutines have no identity; the scheduler does internally. Expose to user code? No (not Go). Expose for debugging? Yes via a runtime-package stub.
3. Error handling: panic-as-exception vs explicit error returns. Go strongly prefers explicit errors. Stdlib stubs follow that: strconv. Atoi("x") returns (0, err), not panic.
4. Memory model. Go has a happens-before model for atomics + channel ops. SX runtime is single-threaded under the scheduler — every channel op is a synchronization point automatically. Don't model relaxed memory; document the simplification.
5. Iteration order of maps. Already addressed in Gotchas; flagged here as a known divergence from spec.

Blockers

Kit-gap proposals against lib/guest/lex.sx

Observed from building the Go tokenizer. Not blocking Phase 2; surfaced here for the substrate-maintainer / next statically-typed-guest loop:

1. No lex-oct-digit? / lex-bin-digit?. Go's prefixed integer forms 0o17 and 0b1010 need digit-class predicates that the kit doesn't provide. We rolled local gl-oct-digit? and gl-bin-digit?. Rust and Swift's lexers will need the same. Cheap to promote.

2. No table-driven longest-prefix matcher. Go has 47+ operator sequences with longest-match semantics. Our gl-match-op is a 25-clause cond ladder; Rust/Swift/TS will each need ~50+. A kit helper like (lex-match-longest TABLE SOURCE POS) that takes a sorted prefix table would collapse this. Worth proposing once a second statically-typed guest hits the same pattern.

Minimal repro: see lib/go/lex.sx#gl-oct-digit? and #gl-match-op.

Progress log

Newest first. Append one dated entry per commit.

• 2026-05-27 — Phase 2 cont.: binary operators via Pratt precedence climbing. gp-pratt-loop consumes pratt-op-lookup against go-precedence-table; left-assoc bumps right-min by 1, right-assoc keeps prec. Binary op nodes are (ast-app (ast-var OP) [LHS RHS]) — uses the canonical ast-app shape rather than inventing a Go-specific binary node. Covers: equal-prec left-to-right, * tighter than +, && tighter than ||, comparison tighter than &&, long chains. +9 tests, parse 26/26, total 155/155. [consumes-pratt].
• 2026-05-27 — Phase 2 first slice: lib/go/parse.sx parser scaffold. Defines go-precedence-table using lib/guest/pratt.sx entry shape (NAME PREC ASSOC) — five Go precedence levels, all left-associative per Go spec § Operator precedence. go-parse tokenises via go-tokenize, then gp-parse-primary reads one literal / identifier and emits a canonical AST node via lib/guest/ast.sx's ast-literal / ast-var. parse 17/17, lex still 129/129, total 146/146. [consumes-pratt consumes-ast].
• 2026-05-27 — Phase 1 complete. Operator-set audit: added missing ~ (Go 1.18+ generics type-set), exhaustive op coverage tests grouped by category. Two kit gaps observed and logged in Blockers: lex-oct-digit?/lex-bin-digit? predicates + lex-match-longest table-driven prefix matcher — both useful for future statically-typed guests. +6 tests, lex 129/129. [proposes-lex]. Phase 2 (parser) next.
• 2026-05-27 — Phase 1 cont.: raw string literals (backtick-delimited). Multi-line, no escape processing, \r stripped per Go spec § String literals. Same "string" token type as interpreted strings — parsers / type checkers don't need to distinguish. +9 tests, lex 123/123. [nothing] — pure Go work; raw strings don't touch the substrate or lib/guest story.
• 2026-05-27 — Phase 1 cont.: decimal float + imaginary literals. 3.14, .5, 1., 1e10, 1.5e-3, 2i, 3.14i. gl-finish-number! handles exponent + i suffix; gl-read-number! returns the type string (int/float/imag). ASI trigger list extended to float/imag. Greedy-grammar pin: 1.method lexes as float ident. Hex floats (0x1.fp0) deferred. +22 tests, lex 114/114. [consumes-lex].
• 2026-05-27 — Phase 1 cont.: prefixed integer literals (0x.., 0X.., 0b.., 0B.., 0o.., 0O.., legacy 0123) + underscore separators in any digit run. Dispatch in gl-read-number!; consumes lex-hex-digit? from the kit. +14 tests, lex 92/92. [consumes-lex].
• 2026-05-26 — Phase 1 first slice: lib/go/lex.sx tokenizer consuming lib/guest/lex.sx predicates. 25 keywords, ident/int/string/rune lits, line+block comments, common operators, automatic semicolon insertion per Go spec § Semicolons (newline / EOF / block-comment-with-newline triggers). Scoreboard + conformance.sh wired. 78/78 tests. [consumes-lex].
• 2026-05-26 — Plan rewritten to integrate the lib/guest framework (chiselling discipline, sister plans for scheduler + bidirectional types, type-checker phase added, conformance scoreboard model adopted). Original 2026-04-26 draft preserved in git history. Loop not yet kicked off; Phase 1 (tokenizer) is the first iteration when this loop spins up.