# Go-on-SX — Go as an SX guest language Port Go to SX as the **first static-typed, bidirectional-checked guest** in the rose-ash language family. Goal isn't a production Go compiler; it's to prove the substrate from a paradigm angle the existing eleven guests don't cover, and to chisel out the lib/guest kits that statically-typed guests N+1 and N+2 will need. Reference: - `plans/lib-guest.md` — parent, chiselling discipline, two-language rule. - `plans/lib-guest-scheduler.md` — sister kit; Go's scheduler pairs with Erlang's. Extraction gated on this loop reaching Phase 5. - `plans/lib-guest-static-types-bidirectional.md` — sister kit; Go's checker pairs with a TBD second consumer. Extraction gated on this loop reaching Phase 3. - `plans/erlang-on-sx.md` — reference implementation for paradigm-port: process model, BIF registry, hot reload, VM bytecode opcodes. **Branch:** `loops/go` (loop-style workstream once kicked off). SX files via `sx-tree` MCP only. ## Thesis — why Go Eleven guests already live in `lib/`: apl, common-lisp, datalog, erlang, forth, haskell, hyperscript, js, kernel, lua, minikanren, ocaml, prolog, ruby, scheme, smalltalk, tcl. Every one is either **dynamically typed** (most) or **HM-inferred** (haskell, ocaml). None exercise: 1. **Bidirectional static type checking** — annotation-driven, locally- inferred, the dominant paradigm of modern statically-typed languages. 2. **Anonymous-channel concurrency** — Go's `chan` and `select`. Erlang has addressed processes + mailboxes; Go has anonymous values + structural pairing. Two different vocabularies for the same underlying scheduler machinery. 3. **Structural interfaces** — `io.Reader` is "anything with this method signature", not a declared subtype relationship. Different from Haskell typeclasses (nominal), different from Lua duck typing (no declaration). These three together make Go an unusually high-value port for proving SX. If SX can host Go cleanly, it can host the next decade of mainstream statically-typed languages (Rust, TS, Swift, Kotlin, Scala 3, Hack) because they share these three properties. Like Erlang-on-SX validated the actor model on the substrate, Go-on-SX validates the goroutine model + bidirectional types. ## Non-goals (deliberate) Out of scope. Reject feature requests for these without further consideration: - **`unsafe` package.** Memory mucking. Skip entirely. - **CGo.** C interop. Out of scope at every level. - **Full `reflect`.** Provide enough for `fmt.Println` to render values; reject the rest. - **Build tags, modules, vendoring.** Treat source as monolithic. One package per file, no real import resolution. - **Production performance.** Conformance tests pass; benchmarks don't. - **Garbage collection tuning.** SX's GC is what you get. - **Race detector, escape analysis, inlining.** Out of scope. - **`os`, `net/http`, full stdlib.** Provide a deliberately small slice (Phase 8 below). ## Architecture sketch ``` Go source text │ ▼ lib/go/lex.sx — tokens; ASI; literals; operators │ (consumes lib/guest/core/lex.sx) ▼ lib/go/parse.sx — AST: package/import/var/const/type/func/struct/ │ interface; expressions; statements │ (consumes lib/guest/core/pratt.sx + ast.sx) ▼ lib/go/types.sx — bidirectional type checker. Synth + check judgments; │ structural interface satisfaction; pluggable subtype │ (INDEPENDENT — no lib/guest/static-types-bidirectional │ yet; this loop builds the first consumer) ▼ lib/go/eval.sx — tree-walk evaluator on CEK. Variables as mutable cells; │ slices = (length, capacity, backing-vector); maps = │ SX dict; defer stack per frame. ▼ lib/go/sched.sx — goroutine scheduler + channels + select │ (INDEPENDENT — no lib/guest/scheduler yet; this loop │ builds the first consumer) ▼ lib/go/std/ — minimal stdlib slice (fmt, strings, strconv, sync, time, errors) ``` Semantic mappings (operational): - `go fn(args)` → `task-spawn` on the local scheduler. - `ch <- v` → `task-block` with predicate "receiver waiting on ch". - `v := <-ch` → `task-block` with predicate "sender waiting on ch". - `select { case ... }` → `task-block` with predicate "any case ready". - `defer fn()` → push thunk onto per-frame defer stack; runs LIFO on return or panic. - `panic(v)` → raise SX exception; deferred fns run while unwinding. - `recover()` → CEK exception capture inside a deferred fn. - `interface{T}` → type-check matches structurally against T's method set; at runtime, the value carries its concrete-type metadata. - `struct{...}` → SX dict + type tag; methods are functions in the type's method table. - `*T` (pointer) → mutable cell (Common Lisp port did the same). - `[]T` (slice) → triple (length, capacity, backing-vector). - `map[K]V` → SX dict; iteration order spec-undefined (v1 = sorted for determinism — programs that depend on indeterminism fail loudly, which is a feature not a bug). ## Conformance scoreboard Following `lib/erlang/scoreboard.json` precedent. Add `lib/go/scoreboard.json` on first iteration; populate as suites land. Suites planned: | Suite | Tests target | What it covers | |---|---|---| | `lex` | 50+ | Keywords, operators, literals, ASI | | `parse` | 80+ | All statement & expression shapes | | `types` | 90+ | Synth, check, interface satisfaction, generics | | `eval` | 100+ | Tree-walk over typed AST | | `runtime` | 60+ | Goroutines, channels, select, close | | `stdlib` | 40+ | fmt, strings, strconv, sync, time, errors | | `e2e` | 10+ | Complete representative programs | ## Phasing — one feature per commit Loop-style. Each phase: implement → test → commit → tick `[ ]` → append Progress-log line → push `origin/loops/go`. ### Phase 1 — Tokenizer (`lib/go/lex.sx`) ✅ - [x] Scaffold + scoreboard + conformance runner (consumes lib/guest/lex.sx) - [x] Identifiers + 25 keywords - [x] Decimal integer literals - [x] Interpreted string literals `"..."` with `\n \t \r \\ \" \'` escapes - [x] Rune literals `'x'` (single char + simple escapes) - [x] Line + block comments (block w/ newline triggers ASI) - [x] Common operator/punct set incl. `:= <- ++ -- == != <= >= && || ...` - [x] **Automatic semicolon insertion** (Go spec § Semicolons) — newline, EOF, and block-comment-with-newline trigger `;` after ident/int/string/rune/{break,continue,fallthrough,return}/{++,--,),],}}. - [x] Float / imaginary literals (decimal floats: `3.14 .5 1. 1e10 1.5e-3`; imag: `2i 3.14i 1e2i`; hex floats `0x1.fp0` deferred) - [x] Raw string literals `` `...` `` (multi-line, no escape processing, `\r` stripped per Go spec § String literals; same `"string"` type as interpreted strings) - [x] Hex/octal/binary integer literals (0x… 0o… 0b…) + underscores (legacy 0123 octal also accepted; consumes lex-hex-digit?) - [x] Full operator set audit (47 distinct per Go spec, plus `~` for generics type-sets). Exhaustive coverage tests in `op-audit:` block. - **Acceptance:** lex/ suite at 50+ tests. Current: 129/129. **Phase 1 done** — hex floats deferred (rare). Move to Phase 2 next. ### Phase 2 — Parser (`lib/go/parse.sx`) ✅ - [x] Parser scaffold + Go operator-precedence table (entry shape from `lib/guest/pratt.sx`) + primary expressions (int/float/imag/string/ rune/ident → ast-literal / ast-var via `lib/guest/ast.sx`). - [x] Binary operators (Pratt precedence climbing using `pratt-op-lookup` + Go precedence table). Operator app emitted as `(ast-app (ast-var OP) [LHS RHS])`; left-assoc raises right-min by 1. - [x] Unary operators (`+x`, `-x`, `!x`, `^x`, `*p`, `&v`, `<-ch`). `gp-parse-unary` recursive, sits between `gp-parse-expr` and `gp-parse-primary`; right-associative chains (`!!x`). - [x] Function calls `f(a, b)` (canonical `ast-app`) and member access `x.field` (Go-specific `(list :select OBJ "field")` — the AST kit doesn't ship a selector node; this is a sister-plan-static-types data point about what the canonical AST is missing). - [x] Index `x[i]` and slice `x[a:b]`/`x[a:b:c]`. Go-specific `(list :index OBJ IDX)` and `(list :slice OBJ LOW HIGH MAX)` (LOW/HIGH/MAX may be nil) — kit lacks both. Permissive parser accepts `a[1::3]` (strict Go rejects, but type phase can enforce). - [x] Type assertion `v.(T)`. `(list :assert OBJ TYPE)`. Includes a minimal `gp-parse-type` (named / qualified `pkg.T` / pointer `*T` / `**T`); full type grammar still pending below. - [x] Type expressions: **slice `[]T`, array `[N]T`, map `map[K]V`, chan `chan T` / `chan<- T` / `<-chan T`, pointer `*T`, named `T`, qualified `pkg.T`, func `func(...) ...`, struct `struct{...}` with shared-type field rows (`x, y int`), interface `interface{...}` with methods + embedded interfaces (named and qualified)** all done — kit has no type primitives. Field tags, struct embeds, variadic, named func-params, Go 1.18 type sets, generics deferred. - [x] Composite literals: `T{...}`, `[]T{...}`, `[N]T{...}`, `map[K]V{...}`, `pkg.T{...}`, nested. Positional and keyed (`X: 1, Y: 2`) elements. AST `(list :composite TYPE-OR-EXPR ELEMS)`; elements are exprs or `(list :kv KEY VALUE)`. Note: in statement context (e.g. `if cond { ... }`) my parser would WRONGLY treat the body as a composite; statement parsing will need a "no- composite-here" mode flag — to be added when statements arrive. - [x] Declarations: `package`, `import`, `var`, `const`, `type`, `func` (with named-greedy params + method receivers + body skipped opaquely until statement parsing arrives). All five `:field` consumers now exist (struct fields, var, const, func params, method receivers) — strong signal that `ast-binding-group` belongs in the canonical AST kit. Grouped/parenthesized decls (`var (...)`, etc.) and variadic params deferred. Anonymous param-list disambiguation (`func(int, string)`) is a known parser-greedy limitation, flagged. - [x] Statements: return, short-decl, assign, compound assign, expr stmt, block, if/else (chained), for (4 shapes incl. range), break, continue, inc-dec, go, defer, send, **switch (tagged / tagless, multi-value cases, default), select (recv-into-var / send / bare-recv / default)** all done. Composite-literal `{` suppression active in control-flow conditions. Type-switch (`switch v := x.(type)`) deferred to a follow-up. - [x] End-to-end: hello-world, fibonacci, FizzBuzz, goroutine ping-pong, struct + method, interface, defer+select+range. `go-parse` extended to handle multi-form files: returns the single form for one-form input (backward compat) or `(list :file FORMS)` for multiple. Structural tests assert top-level decl-tag sequences via the `decl-tags` helper rather than full ASTs. - **Acceptance:** parse/ suite at 80+ tests. Current: **176/176**. **Phase 2 complete.** Type-switch is the one syntactic shape still deferred to a follow-up; it doesn't gate Phase 3. ### Phase 3 — Bidirectional type checker, MVP (`lib/go/types.sx`) ⬜ - **Independent implementation.** Do NOT use lib/guest/static-types- bidirectional/ — that kit doesn't exist yet and depends on this work for its design. See `plans/lib-guest-static-types-bidirectional.md`. - Synth + check judgments. Context as a value (per-block scope). - Coverage MVP: declared-type variables, function signatures (params + returns), call type-checking, simple composite types (slice, map, chan element), interface satisfaction (structural match against method sets), short variable declaration `:=` (synth from RHS). - **Untyped constants.** `42` has type `untyped int` until contextualised; this is the canonical pitfall (see Gotchas below). - Defer: generics (Phase 7), full conversion rules. - Tests: positive (type-correct programs check) + negative (mismatched types fail with informative errors carrying AST paths). - **Acceptance:** types/ suite at 60+ tests. Chisel note `shapes-static- types-bidirectional` — append a paragraph to the sister plan's design diary describing what synth/check shape emerged. ### Phase 4 — Tree-walk evaluator (`lib/go/eval.sx`) ⬜ - AST-walking interpreter over CEK. Each Go statement maps to one step function (precedent: `step-sf-if` etc. in spec/evaluator.sx). - Variables: mutable cells. Pointer semantics: `&x` returns the cell, `*p` dereferences. - Slices: triple (length, capacity, backing-vector). `append` honours capacity-grow per spec. - Maps: SX dict + key-type metadata. - Structs: SX dict + type tag. Methods looked up via type's method table. - Functions: closures over enclosing scope; multiple return values. - Channels: stub (Phase 5 wires them). - Tests: arithmetic, control flow, recursion, closures, slices, maps, structs, methods, pointer semantics, multiple-return. - **Acceptance:** eval/ suite at 80+ tests. No concurrency yet. ### Phase 5 — Goroutines + channels + select (`lib/go/sched.sx`) ⬜ - **Independent implementation.** Do NOT use lib/guest/scheduler/ — that kit doesn't exist yet and depends on this work for its design. See `plans/lib-guest-scheduler.md`. - `go expr` — spawn a goroutine; returns nothing. - `chan T` — `make(chan T)` creates an unbuffered channel; `make(chan T,n)` creates a buffered channel (Phase 5b — defer buffer to a sub-phase). - `<-ch` — receive (blocks until sender ready). - `ch <- v` — send (blocks until receiver ready for unbuffered, or buffer has room for buffered). - `select { case ... }` — non-deterministic multiplexing; `default` makes it non-blocking. - `close(ch)` — closes channel. Receive on closed → zero value + ok=false. - Tests: ping-pong, fan-out/fan-in, work queue, select with default, select with timeout (via a `time.After`-like stub), close semantics, range over channel. - **Acceptance:** runtime/ suite at 40+ tests. Chisel note `shapes- scheduler` — append a paragraph to the sister plan's design diary describing what task-spawn/block/wake/yield shape emerged. ### Phase 5b — Buffered channels + select fairness ⬜ - Buffered: send blocks only when buffer full; recv only when empty. - `select` random case ordering (spec mandates pseudo-random; v1 uses a fixed seed for determinism with a `runtime`-package knob to randomise). - Tests: buffer-full blocking, buffer-empty blocking, select fairness over many iterations. - **Acceptance:** runtime/ +20 tests. ### Phase 6 — `defer` + panic/recover ⬜ - Defer stack per function frame; runs LIFO on return (normal or panic). - `panic(v)` unwinds frames running deferreds; `recover()` inside a deferred fn captures the panic value and stops unwinding. - Goroutine panic propagation: a panicking goroutine that doesn't recover crashes the whole program (honour Go spec, or document divergence). - Tests: defer order (LIFO), defer + named-return mutation, panic/recover, panic across goroutines, defer in a loop (push per iter, run on fn return — common bug). - **Acceptance:** eval/ +20 tests. ### Phase 7 — Generics (Go 1.18+) ⬜ - Type parameters with constraints (type sets: `interface{ int | float64 }`, `comparable`, `any`). - Type inference at call sites — basic; the full Go inference algorithm is notoriously complex. Implement enough for common cases; document limitations in a Blockers section below. - Tests: generic function (`func Map[T, U any](xs []T, f func(T) U) []U`), generic data structure (linked list), constrained type param. - **Acceptance:** types/ +30 tests. ### Phase 8 — Minimal stdlib (`lib/go/std/`) ⬜ - Implement just what's needed for representative programs: - `fmt` — `Println`, `Printf`, `Sprintf`, `Fprintf`, `Errorf`, `Stringer` dispatch. Verbs: `%d %s %v %t %f %T %+v`. - `strings` — `Contains`, `HasPrefix`, `HasSuffix`, `Split`, `Join`, `TrimSpace`, `ToUpper`, `ToLower`, `Replace`, `Index`, `Count`, `Repeat`, `NewReader`. - `strconv` — `Itoa`, `Atoi`, `FormatFloat`, `ParseFloat`, `ParseInt`, `FormatInt`. - `errors` — `New`, `Is`, `As`, `Unwrap`. - `sync` — `Mutex` (cooperative — flag + waiter queue), `WaitGroup`, `Once`, `RWMutex`. - `time` — `Now`, `Since`, `After` (channel-returning timer), `Sleep`, `Duration`, `Time`. - `io` — `Reader`/`Writer` interfaces; `ReadAll`; `Copy`. - `sort` — `Slice`, `Ints`, `Strings`. - Tests: round-trip Itoa/Atoi, fmt verb coverage, sync.WaitGroup with goroutines, time.After in a select, sort.Slice with custom less fn. - **Acceptance:** stdlib/ suite at 40+ tests. ### Phase 9 — End-to-end programs ⬜ - Complete programs from canonical sources (gopl.io, "concurrency patterns" talk examples) running end-to-end: - Concurrent prime sieve - HTTP-ish ping-pong over stubbed transport - Word frequency counter - Pipeline (channel chain) - Producer/consumer with sync.WaitGroup - "Bounded parallelism" pattern (worker pool over a job channel) - **Acceptance:** e2e/ suite at 10+ tests, all passing. ### Phase 10 — lib/guest extraction enabler ⬜ - Now that Go has lex+parse+types+eval+sched, sister plans are unblocked on the Go side. This phase is **doc-only** in `loops/go`: - Cross-reference `plans/lib-guest-scheduler.md` — mark its Phase 1 (Go scheduler independent) as complete from Go's side. - Cross-reference `plans/lib-guest-static-types-bidirectional.md` — mark its Phase 1 as complete from Go's side. - Update the chiselling diary in each sister plan with the actual Go-side surface that emerged. - **Acceptance:** sister plans cross-referenced + diaries updated. No new Go code. ### Phase 11 — VM bytecode opcodes (deferred, optional) ⬜ - Following Erlang-on-SX Phase 10 precedent: identify hot paths in the tree-walk evaluator, define Go-specific bytecode opcodes, compile hot fns through them. Substantial work; only justified if Go programs exercise enough volume that performance starts mattering. - **Acceptance:** TBD on demand. ## Ground rules (loop-style) - **Scope:** only `lib/go/**` and this plan. Do not touch `spec/`, `hosts/`, `shared/`, `lib/guest/**` (read-only consumer at this phase), or other `lib//`. - **Consume `lib/guest/core/`** for lex/parse/ast/match/layout. Hand- rolling defeats the chiselling goal. - **Do NOT extract into `lib/guest/scheduler/` or `lib/guest/static- types-bidirectional/` from this loop.** Those extractions are gated on two consumers AND the discipline of writing each consumer independently. Extraction is its own workstream after Go and the second consumer both exist. - **Substrate gaps** → Blockers entry with minimal repro. Don't fix the substrate from this loop. Belongs to `sx-improvements.md`. - **NEVER call `sx_build` without timeout awareness** — 600s watchdog. - **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after every edit. - **Worktree:** branch `loops/go`, push `origin/loops/go`. Never `main`, never `architecture`. - **Commit granularity:** one feature per commit. Short factual messages: `go: parse short-decl + 6 tests [consumes-pratt]`. Chisel note at end in brackets. - **Plan file:** update Progress log + tick boxes every commit. - **If blocked** for two iterations on the same issue, add to Blockers and move on. Phases 1-4 are sequential; Phases 5-8 are largely independent once 4 lands. ## Chisel discipline (per parent lib-guest plan) Every commit ends its message with a chisel note in brackets: - `[consumes-X]` — used `lib/guest/X` kit. - `[shapes-scheduler]` / `[shapes-static-types-bidirectional]` — revealed something about what the sister lib-guest kits should look like. Add a paragraph to the relevant sister plan's design diary. - `[proposes-Y]` — revealed a gap in another existing kit. Blockers entry in the kit's plan. - `[nothing]` — pure Go work that didn't touch substrate or lib/guest story. Acceptable; if it shows up twice in a row, stop and reflect. ## Go-specific gotchas - **ASI (automatic semicolon insertion).** Newline becomes `;` after identifier/literal/`)`/`]`/`}`. Build into the tokenizer; the Go spec's "Semicolons" section is unusually precise — follow it literally. - **Untyped constants.** `42` has type `untyped int` until used in a context that forces a type. The canonical example: `var x float64 = 42 / 7` — must compute as `untyped int / untyped int = 6` then convert to `float64 = 6.0`. Wrong: float-coercing eagerly gives 6.0 prematurely. Wrong: integer-truncating after coercion gives `5.something`. Test it. - **Methods vs functions.** `func (r Receiver) Method()` is a method bound to a type; `func Function(r Receiver)` is just a function. Methods on pointer-receivers vs value-receivers have asymmetric satisfaction in interfaces — pointer-receiver methods are NOT in the value's method set for interface satisfaction. - **Interface satisfaction is structural and silent.** Type satisfies an interface if its method set contains all the interface's methods. Lazy check: at every point a value flows into an interface-typed slot. - **Channels are first-class values.** Pass them, store them, send them through other channels. Each channel has identity. - **`select` with `default`** = non-blocking. Without `default`, blocks until a case is ready. - **`nil` is typed.** `var x *int` makes x a `(*int)(nil)`. Comparison `x == nil` works on typed nil; but `var i interface{} = (*int)(nil); i == nil` is `false` — i holds a typed-nil-of-type-`*int`, not untyped nil. The classic Go footgun. Test it. - **Goroutine panic propagation.** A panicking goroutine that doesn't recover crashes the whole program. Implement faithfully or document divergence. - **`defer` in a loop.** Each iteration pushes; they all run on function return. Common bug; tests should cover. - **Iteration order of maps.** Spec: unspecified. v1 = sorted by SX- canonical key order for determinism; document that programs depending on iteration order are not Go-conformant. Add a `runtime`-package knob to enable randomisation later. ## Style - No comments in `.sx` unless non-obvious. Cite Go spec sections inline for non-obvious decisions (Go's spec is rigorous; citations work). - No new planning docs — update this plan inline. - One feature per iteration. Commit. Log. Push. Next. ## Open questions 1. **Module/import model.** Go has packages and import paths. Probably model "package" as one or more `.sx` files in a directory, no real import resolution against a remote module graph. Decide in Phase 2. 2. **Goroutine identity.** Spec says goroutines have no identity; the scheduler does internally. Expose to user code? No (not Go). Expose for debugging? Yes via a `runtime`-package stub. 3. **Error handling: panic-as-exception vs explicit error returns.** Go strongly prefers explicit errors. Stdlib stubs follow that: `strconv. Atoi("x")` returns `(0, err)`, not panic. 4. **Memory model.** Go has a happens-before model for atomics + channel ops. SX runtime is single-threaded under the scheduler — every channel op is a synchronization point automatically. Don't model relaxed memory; document the simplification. 5. **Iteration order of maps.** Already addressed in Gotchas; flagged here as a known divergence from spec. ## Blockers ### Kit-gap proposals against `lib/guest/ast.sx` Observed from building the Go parser: 1. **No selector / field-access node.** `obj.field` is a universal shape across nominally-typed languages — Go, Rust, Swift, TS, JS, Python, Ruby, Java, C#. The kit ships `ast-app` (function application) but not `ast-select`. We rolled `(list :select OBJ "field")` locally as a Go-specific tag. Worth promoting once a second consumer hits the same need (likely immediately — almost every guest needs it). 2. **No index / subscript node.** `x[i]` is universal across nearly every guest with arrays/maps. Rolled `(list :index OBJ IDX)` locally. 3. **No slice node.** Go's two- and three-index slice expressions are distinctive but the basic two-index `x[a:b]` shape covers Python, Rust, Swift, JS, Ruby slicing too. Rolled `(list :slice OBJ LOW HIGH MAX)` (LOW/HIGH/MAX may be nil for omitted indices). MAX-as-fourth-field is Go-specific; the canonical kit shape could ship as `(list :slice OBJ LOW HIGH)` for the common case and a separate `:slice3` or `:full-slice` for the Go variant. Minimal repro: see `lib/go/parse.sx#gp-parse-postfix` + `gp-parse-bracket`. 4. **No "named binding(s) of a type" node.** Building struct types surfaced a shape that recurs everywhere: ``` (list :field NAMES TYPE) ``` Same shape appears in: struct fields (`x, y int`), func parameters (`func(a, b int, c string)`), method receivers (`m(a, b int)`), variable declarations (`var x, y int`). Three Phase-2 sub-deliverables (struct fields, func decls, var decls) all want this shape. Promoting it once means Rust struct fields, Swift parameters, TS class fields, Java method signatures all get a free home. Candidate canonical name: `ast-binding-group` or `ast-named-of-type`. 5. **No type-expression primitives.** Every statically-typed guest needs to express types in source. Proposed canonical shapes: ``` (list :ty-name "T") — named type (list :ty-sel "pkg" "T") — qualified type (list :ty-ptr T) — pointer to T (list :ty-slice T) — slice / dynamic array of T (list :ty-array N T) — fixed array, N is an expr (list :ty-map K V) — map type (also Python dict, Rust HashMap) ``` The first six are universal: Rust, Swift, TS, Kotlin, Scala, Hack all need them. Go-specific extensions like `:ty-chan` (channel with direction) and `:ty-func` (parameter+return) should stay guest-specific until a second consumer wants them. Minimal repro: see `lib/go/parse.sx#gp-parse-type`. ### Kit-gap proposals against `lib/guest/lex.sx` Observed from building the Go tokenizer. Not blocking Phase 2; surfaced here for the substrate-maintainer / next statically-typed-guest loop: 1. **No `lex-oct-digit?` / `lex-bin-digit?`.** Go's prefixed integer forms `0o17` and `0b1010` need digit-class predicates that the kit doesn't provide. We rolled local `gl-oct-digit?` and `gl-bin-digit?`. Rust and Swift's lexers will need the same. Cheap to promote. 2. **No table-driven longest-prefix matcher.** Go has 47+ operator sequences with longest-match semantics. Our `gl-match-op` is a 25-clause `cond` ladder; Rust/Swift/TS will each need ~50+. A kit helper like `(lex-match-longest TABLE SOURCE POS)` that takes a sorted prefix table would collapse this. Worth proposing once a second statically-typed guest hits the same pattern. Minimal repro: see `lib/go/lex.sx#gl-oct-digit?` and `#gl-match-op`. ## Progress log _Newest first. Append one dated entry per commit._ - 2026-05-27 — **Phase 2 complete.** End-to-end multi-form file parsing. `go-parse` now returns single forms for backward compat (~169 tests unchanged) or `(list :file FORMS)` for multi-form input. Tests cover hello-world, fibonacci, FizzBuzz, goroutine ping-pong, struct+method, interface+method, and defer+select+range — each asserted via top- level `decl-tags`. Type-switch is the one syntactic shape still deferred. +7 tests, parse 176/176, total 305/305. Next: Phase 3 (bidirectional type checker). `[nothing]` — pure Go parser composition; the cross-language insights are already in the sister- plan diaries from earlier Phase 2 commits. - 2026-05-27 — Phase 2 cont.: `switch` and `select` statements. Tagged + tagless switch, multi-value cases, `default`, and select with recv-into-var / send / bare-recv / default cases. New `gp-parse-case-body` reads stmts until the next `case`/`default`/`}` without consuming the terminator. AST shapes: `(list :switch TAG CASES)`, `(list :case VALUES BODY)`, `(list :select CASES)`, `(list :select-case COMM-STMT BODY)`, `(list :default BODY)`. With this, **Phase 2 statement coverage is complete** — type-switch is the one remaining shape (deferred). +8 tests, parse 169/169, total 298/298. `[shapes-scheduler]` — sister-plan diary updated with the `:select-case` uniform shape insight (single kit primitive covers all four Go case kinds; default vs no-default determines blocking semantics; cross-references to Erlang's `receive ... after`). Sister-plan diary update follows. - 2026-05-27 — Phase 2 cont.: concurrency + iteration statements. `go EXPR`, `defer EXPR`, channel send `ch <- v`, and the four `for ... range` shapes (no-kv / k-only / k,v / assign-form). New `gp-for-find-range` pre-scans the for-header to dispatch between range and C-style/while forms cleanly. Send-stmt detection added to the LHS-list branch (after lhs, `<-` → send). +9 tests, parse 161/161, total 290/290. `[shapes-scheduler]` — Go's concurrency- primitive AST shapes (`:go`, `:defer`, `:send`, `:range-for`) all landed; sister-plan diary updated with the corresponding kit-API insights (uniform spawn-thunk shape, channel-recv ⇄ iteration polymorphism at the range-coll dispatch). Sister-plan diary update follows. - 2026-05-27 — Phase 2 cont.: control-flow statements. `if cond { } [else { }]` with chained else-if, `for { }` (infinite), `for cond { }` (while-like), `for init; cond; post { }` (C-style), `break`, `continue`, plus `x++` / `x--` inc-dec statements. **Closed the parser-mode caveat** flagged when composite literals landed: `gp-no-comp-lit` is a re-entrant counter that suppresses the postfix `{...}` → composite-lit interpretation inside control-flow condition positions, matching Go spec § Composite literals. `gp-parse-control- cond` wraps the increment/decrement so callers can't forget. +11 tests, parse 152/152, total 281/281. `[nothing]` — pure Go parser shape work; the bidirectional-checker-relevant shapes (cond/body) are already covered by the earlier `:field` insight. - 2026-05-27 — Phase 2 cont.: statements. First slice covers `return [exprs]`, short-decl `lhs := exprs`, assignment `lhs = exprs`, compound assignment (`+= -= *= /= %= &= |= ^= <<= >>= &^=`), bare expression statements, and nested blocks `{ ... }`. New `gp-parse-stmt` dispatches on the leading token; `gp-parse-block-body` replaces the func-decl `:body` sentinel with real `(:block STMTS)`. Existing func/method tests updated to the new body shape. **Progress guards** added to `gp-block-body-loop` and `gp-parse-composite-elems` — unsupported syntax (`if`, `for`, etc.) now advances one token instead of spinning. `gp-skip-block!` left as dead code; will be deleted once control-flow stmts land. +9 tests, parse 141/141, total 270/270. `[nothing]` — pure Go parser work; the cross-language statement shapes will become a chiselling target once a second statically-typed guest hits them. - 2026-05-27 — Phase 2 cont.: func declarations. `func f() {}`, `func add(x, y int) int { ... }`, multi-group params, multi-return, signature-only (no body), pointer-receiver and value-receiver methods, nested-brace body. New `gp-parse-decl-param-group` uses a named-greedy algorithm: collects consecutive `ident [, ident]*` then parses a type. `gp-skip-block!` brace-balances over the body opaquely; the AST stores `:body` as a sentinel pending statement parsing. With this, **all five `:field` binding-group consumers now exist** (struct fields, var, const, func params, method receivers) — strong cross- deliverable validation of the `ast-binding-group` kit proposal. Anonymous-param-list disambiguation (`func(int, string)`) is a known greedy-parser limitation flagged in plan. +8 tests, parse 132/132, total 261/261. `[shapes-static-types-bidirectional]` — the consistent use of `:field` across decls is what the sister kit's bidirectional checker will use to propagate types from declarations to bindings. Sister-plan diary update follows. - 2026-05-27 — Phase 2 cont.: declarations — `package N`, `import "p"`, `var name [TYPE] [= EXPRS]`, `const name [TYPE] [= EXPRS]`, `type NAME TYPE`. New `gp-parse-top` dispatcher routes the five decl keywords to `gp-parse-decl` while preserving expression parsing for everything else. `var` and `const` reuse the `:field` binding- group shape from Blockers — **first cross-deliverable use of the proposed kit shape**: struct fields, func params, and now var/const decls all share the same `(list :field NAMES TYPE)` envelope. `import` uses canonical `ast-import` directly. Grouped forms (`var (...)`) and `func` decls deferred. +10 tests, parse 124/124, total 253/253. `[consumes-ast]` — first concrete use of `ast-import` from the kit; also validates the `:field` shape across three contexts. - 2026-05-27 — Phase 2 cont.: composite literals. `T{}`, `T{1, 2}`, `T{X: 1, Y: 2}`, `[]T{...}`, `[N]T{...}`, `map[K]V{...}`, `pkg.T{...}`, nested composites. AST shape `(list :composite TYPE-OR-EXPR ELEMS)`; each element is an expression or `(list :kv KEY VALUE)`. Two parser entry points: type-prefixed (`gp-parse-primary` adds `[`/`map`/`struct` branches) and ident-prefixed (postfix loop adds `{` branch). **Known limitation flagged in plan:** when statement parsing arrives, the postfix `{` branch will misread `if cond { ... }` as composite literal — needs a "no-composite-here" parser-mode flag. +8 tests, parse 114/114, total 243/243. `[nothing]` — pure Go parser shape work. - 2026-05-27 — Phase 2 cont.: interface-type expressions. `interface {}`, `interface { Close() }`, `interface { String() string }`, `interface { Read([]byte) (int, error) }`, plus embedded interfaces (`Stringer`, `io.Reader`). AST shape: `(list :ty-interface ELEMS)` where each element is either `(list :method NAME PARAMS RESULTS)` or `(list :embed TYPE)`. Method params reuse `gp-parse-func-type-params` — same anonymous-only shape as func types. Go 1.18+ type sets (`~int | ~float64`) deferred to generics work. With this, all Phase-2 **type expressions** are complete. +8 tests, parse 106/106, total 235/235. `[nothing]` — pure Go parser; the field-binding-group kit-gap proposal from the previous commit covers the cross-language angle. - 2026-05-27 — Phase 2 cont.: struct-type expressions. `struct {}`, `struct { x int }`, `struct { x int; y string }`, `struct { x, y int }` (shared type), nested struct fields. `gp-parse-struct-fields` walks field rows tolerating ASI semis; each row is a name list + type. AST: `(list :ty-struct FIELDS)` with each field `(list :field NAMES TYPE)`. Embedded fields, tags, and methods deferred. +8 tests, parse 98/98, total 227/227. `[proposes-ast]` — the `:field` shape (NAMES + TYPE) recurs in func params, method receivers, var decls; flagged in Blockers as a unified `ast-binding-group` candidate for the kit. - 2026-05-27 — Phase 2 cont.: func-type expressions. `func()`, `func() int`, `func(int, string)`, `func(int) string`, `func() (int, error)`. AST shape `(list :ty-func PARAMS RESULTS)` where both are lists of type nodes. Results parsing reuses param parser for the multi-return `(T, T, ...)` case. Anonymous-only params for now — named params (`func(a int, b string)`) need a different shape and are required mainly for func DECLARATIONS not pure func-type expressions. Variadic deferred. Covers nested func-as-return and chan-of-func. +9 tests, parse 90/90, total 219/219. `[nothing]` — pure Go parser; type AST proposals already in Blockers. - 2026-05-27 — Phase 2 cont.: type expressions — slice `[]T`, array `[N]T`, map `map[K]V`, chan in all three directions (`chan T`, `chan<- T`, `<-chan T`). `gp-parse-type` now dispatches on `*`/`[`/`map`/`chan`/`<-`/ident; each branch recurses for nested types. Channel direction is `:both`/`:send`/`:recv`. AST stays Go-specific tagged lists — kit has no type primitives at all. Covers nested types end-to-end (slice-of-pointer, slice-of-slice, map-with-slice-value, chan-of-map, pointer-to-slice). **Parse acceptance bar (80+) crossed: +11 tests, parse 81/81, total 210/210.** Func / struct / interface types and generics still pending in Phase 2. `[proposes-ast]` — surfaces concrete type-node proposals (slice / array / map are universal across statically-typed guests; channel direction is Go-specific). Logged in Blockers. - 2026-05-27 — Phase 2 cont.: type assertion `v.(T)` postfix form. Postfix `.` branch now disambiguates between `.field` (selector) and `.(...)` (type assertion) by peeking at the next token. New `gp-parse-type` handles the minimum needed: named (`int`, `MyType`), qualified (`pkg.T`), pointer (`*T`, `**T`). AST shapes are Go-specific tagged lists — kit has no notion of types at all yet (this is a meta-gap: full bidirectional types arrive in Phase 3, but even the parser needs a type substrate). Covers chained, call-result, after-selector, and binary-precedence interactions. +9 tests, parse 70/70, total 199/199. `[nothing]`. - 2026-05-27 — Phase 2 cont.: index `x[i]` and slice `x[a:b]` / `x[a:b:c]` postfix forms. New `gp-parse-bracket` + `gp-parse-bracket-expr` branch off the same postfix loop as calls/selectors. AST: Go-specific `(list :index OBJ IDX)` and `(list :slice OBJ LOW HIGH MAX)` — LOW/HIGH/MAX may be nil for omitted indices. Two more kit gaps logged (no `:index`, no `:slice` in canonical AST). Permissive on `a[1::3]`. Covers: literal idx, var idx, expr idx, string idx, chained `a[0][1]`, mixed `a[0].field`, full slice with three indices. +12 tests, parse 61/61, total 190/190. `[proposes-ast]`. - 2026-05-27 — Phase 2 cont.: postfix forms — function calls `f(a, b)` via canonical `ast-app`, and member access `x.field` via Go-specific `(list :select OBJ "field")`. The AST kit has no selector node; logged in Blockers as `[proposes-ast]` — every nominally-typed guest will hit the same gap, worth promoting on the next consumer. Postfix loop sits between unary and primary so calls bind tighter than unary (`-f(x)` = `-(f(x))`). Covers nested calls, chained selectors, methods `obj.m(x)`, mixed precedence. +12 tests, parse 49/49, total 178/178. `[consumes-ast proposes-ast]`. - 2026-05-27 — Phase 2 cont.: unary prefix operators (`+`, `-`, `!`, `^`, `*`, `&`, `<-`). `gp-parse-unary` is recursive (`!!x`) and sits between `gp-parse-expr` and `gp-parse-primary` so unary always binds tighter than any binary. Symbols `+ - * & ^` are shared with binary; the positional split (expression-start vs mid-expression) disambiguates cleanly without lookback. Unary nodes are single-arg `ast-app`. +11 tests, parse 37/37, total 166/166. `[nothing]` — pure Go parser work. - 2026-05-27 — Phase 2 cont.: binary operators via Pratt precedence climbing. `gp-pratt-loop` consumes `pratt-op-lookup` against `go-precedence-table`; left-assoc bumps right-min by 1, right-assoc keeps prec. Binary op nodes are `(ast-app (ast-var OP) [LHS RHS])` — uses the canonical `ast-app` shape rather than inventing a Go-specific binary node. Covers: equal-prec left-to-right, `*` tighter than `+`, `&&` tighter than `||`, comparison tighter than `&&`, long chains. +9 tests, parse 26/26, total 155/155. `[consumes-pratt]`. - 2026-05-27 — Phase 2 first slice: `lib/go/parse.sx` parser scaffold. Defines `go-precedence-table` using `lib/guest/pratt.sx` entry shape `(NAME PREC ASSOC)` — five Go precedence levels, all left-associative per Go spec § Operator precedence. `go-parse` tokenises via `go-tokenize`, then `gp-parse-primary` reads one literal / identifier and emits a canonical AST node via `lib/guest/ast.sx`'s `ast-literal` / `ast-var`. parse 17/17, lex still 129/129, total 146/146. `[consumes-pratt consumes-ast]`. - 2026-05-27 — **Phase 1 complete.** Operator-set audit: added missing `~` (Go 1.18+ generics type-set), exhaustive op coverage tests grouped by category. Two kit gaps observed and logged in Blockers: `lex-oct-digit?`/`lex-bin-digit?` predicates + `lex-match-longest` table-driven prefix matcher — both useful for future statically-typed guests. +6 tests, lex 129/129. `[proposes-lex]`. Phase 2 (parser) next. - 2026-05-27 — Phase 1 cont.: raw string literals (backtick-delimited). Multi-line, no escape processing, `\r` stripped per Go spec § String literals. Same `"string"` token type as interpreted strings — parsers / type checkers don't need to distinguish. +9 tests, lex 123/123. `[nothing]` — pure Go work; raw strings don't touch the substrate or lib/guest story. - 2026-05-27 — Phase 1 cont.: decimal float + imaginary literals. `3.14`, `.5`, `1.`, `1e10`, `1.5e-3`, `2i`, `3.14i`. `gl-finish-number!` handles exponent + `i` suffix; `gl-read-number!` returns the type string (int/float/imag). ASI trigger list extended to float/imag. Greedy-grammar pin: `1.method` lexes as `float ident`. Hex floats (`0x1.fp0`) deferred. +22 tests, lex 114/114. `[consumes-lex]`. - 2026-05-27 — Phase 1 cont.: prefixed integer literals (`0x..`, `0X..`, `0b..`, `0B..`, `0o..`, `0O..`, legacy `0123`) + underscore separators in any digit run. Dispatch in `gl-read-number!`; consumes `lex-hex-digit?` from the kit. +14 tests, lex 92/92. `[consumes-lex]`. - 2026-05-26 — Phase 1 first slice: `lib/go/lex.sx` tokenizer consuming `lib/guest/lex.sx` predicates. 25 keywords, ident/int/string/rune lits, line+block comments, common operators, automatic semicolon insertion per Go spec § Semicolons (newline / EOF / block-comment-with-newline triggers). Scoreboard + conformance.sh wired. 78/78 tests. `[consumes-lex]`. - 2026-05-26 — Plan rewritten to integrate the lib/guest framework (chiselling discipline, sister plans for scheduler + bidirectional types, type-checker phase added, conformance scoreboard model adopted). Original 2026-04-26 draft preserved in git history. Loop not yet kicked off; Phase 1 (tokenizer) is the first iteration when this loop spins up.