Merge architecture into loops/forth
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 52s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 52s
This commit is contained in:
81
plans/agent-briefings/apl-loop.md
Normal file
81
plans/agent-briefings/apl-loop.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# apl-on-sx loop agent (single agent, queue-driven)
|
||||
|
||||
Role: iterates `plans/apl-on-sx.md` forever. Rank-polymorphic primitives + 6 operators on the JIT is the headline showcase — APL is the densest combinator algebra you can put on top of a primitive table. Every program is `array → array` pure pipelines, exactly what the JIT was built for.
|
||||
|
||||
```
|
||||
description: apl-on-sx queue loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/apl-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
1. Read `plans/apl-on-sx.md` — roadmap + Progress log.
|
||||
2. `ls lib/apl/` — pick up from the most advanced file.
|
||||
3. If `lib/apl/tests/*.sx` exist, run them. Green before new work.
|
||||
4. If `lib/apl/scoreboard.md` exists, that's your baseline.
|
||||
|
||||
## The queue
|
||||
|
||||
Phase order per `plans/apl-on-sx.md`:
|
||||
|
||||
- **Phase 1** — tokenizer + parser. Unicode glyphs, `¯` for negative, strands (juxtaposition), right-to-left, valence resolution by syntactic position
|
||||
- **Phase 2** — array model + scalar primitives. `make-array {shape, ravel}`, scalar promotion, broadcast for `+ - × ÷ ⌈ ⌊ * ⍟ | ! ○`, comparison, logical, `⍳`, `⎕IO`
|
||||
- **Phase 3** — structural primitives + indexing. `⍴ , ⍉ ↑ ↓ ⌽ ⊖ ⌷ ⍋ ⍒ ⊂ ⊃ ∊`
|
||||
- **Phase 4** — **THE SHOWCASE**: operators. `f/` (reduce), `f¨` (each), `∘.f` (outer), `f.g` (inner), `f⍨` (commute), `f∘g` (compose), `f⍣n` (power), `f⍤k` (rank), `@` (at)
|
||||
- **Phase 5** — dfns + tradfns + control flow. `{⍺+⍵}`, `∇` recurse, `⍺←default`, tradfn header, `:If/:While/:For/:Select`
|
||||
- **Phase 6** — classic programs (life, mandelbrot, primes, n-queens, quicksort) + idiom corpus + drive to 100+
|
||||
|
||||
Within a phase, pick the checkbox that unlocks the most tests per effort.
|
||||
|
||||
Every iteration: implement → test → commit → tick `[ ]` → Progress log → next.
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `lib/apl/**` and `plans/apl-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. APL primitives go in `lib/apl/runtime.sx`.
|
||||
- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop.
|
||||
- **Shared-file issues** → plan's Blockers with minimal repro.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits.
|
||||
- **Unicode in `.sx`:** raw UTF-8 only, never `\uXXXX` escapes. Glyphs land directly in source.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
|
||||
## APL-specific gotchas
|
||||
|
||||
- **Right-to-left, no precedence among functions.** `2 × 3 + 4` is `2 × (3 + 4)` = 14, not 10. Operators bind tighter than functions: `+/ ⍳5` is `+/(⍳5)`, and `2 +.× 3 4` is `2 (+.×) 3 4`.
|
||||
- **Valence by position.** `-3` is monadic negate (`-` with no left arg). `5-3` is dyadic subtract. The parser must look left to decide. Same glyph; different fn.
|
||||
- **`¯` is part of a number literal**, not a prefix function. `¯3` is the literal negative three; `-3` is the function call. Tokenizer eats `¯` into the numeric token.
|
||||
- **Strands.** `1 2 3` is a 3-element vector, not three separate calls. Adjacent literals fuse into a strand at parse time. Adjacent names do *not* fuse — `a b c` is three separate references.
|
||||
- **Scalar promotion.** `1 + 2 3 4` ↦ `3 4 5`. Any scalar broadcasts against any-rank conformable shape.
|
||||
- **Conformability** = exactly matching shapes, OR one side scalar, OR (in some dialects) one side rank-1 cycling against rank-N. Keep strict in v1: matching shape or scalar only.
|
||||
- **`⍳` is overloaded.** Monadic `⍳N` = vector 1..N (or 0..N-1 if `⎕IO=0`). Dyadic `V ⍳ W` = first-index lookup, returns `≢V+1` for not-found.
|
||||
- **Reduce with `+/⍳0`** = `0` (identity for `+`). Each scalar primitive has a defined identity used by reduce-on-empty. Don't crash; return identity.
|
||||
- **Reduce direction.** `f/` reduces the *last* axis. `f⌿` reduces the *first*. Matters for matrices.
|
||||
- **Indexing is 1-based** by default (`⎕IO=1`). Do not silently translate to 0-based; respect `⎕IO`.
|
||||
- **Bracket indexing** `A[I]` is sugar for `I⌷A` (squad-quad). Multi-axis: `A[I;J]` is `I J⌷A` with semicolon-separated axes; `A[;J]` selects all of axis 0.
|
||||
- **Dfn `{...}`** — `⍺` = left arg (may be unbound for monadic call → check with `⍺←default`), `⍵` = right arg, `∇` = recurse. Default left arg syntax: `⍺←0`.
|
||||
- **Tradfn vs dfn** — tradfns use line-numbered `→linenum` for goto; dfns use guards `cond:expr`. Pick the right one for the user's syntax.
|
||||
- **Empty array** = rank-N array where some dim is 0. `0⍴⍳0` is empty rank-1. Scalar prototype matters for empty-array operations; ignore in v1, return 0/space.
|
||||
- **Test corpus:** custom + idioms. Place programs in `lib/apl/tests/programs/` with `.apl` extension.
|
||||
|
||||
## General gotchas (all loops)
|
||||
|
||||
- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences.
|
||||
- `cond`/`when`/`let` clauses evaluate only the last expr.
|
||||
- `type-of` on user fn returns `"lambda"`.
|
||||
- Shell heredoc `||` gets eaten — escape or use `case`.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.sx` unless non-obvious.
|
||||
- No new planning docs — update `plans/apl-on-sx.md` inline.
|
||||
- Short, factual commit messages (`apl: outer product ∘. (+9)`).
|
||||
- One feature per iteration. Commit. Log. Next.
|
||||
|
||||
Go. Read the plan; find first `[ ]`; implement.
|
||||
80
plans/agent-briefings/common-lisp-loop.md
Normal file
80
plans/agent-briefings/common-lisp-loop.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# common-lisp-on-sx loop agent (single agent, queue-driven)
|
||||
|
||||
Role: iterates `plans/common-lisp-on-sx.md` forever. Conditions + restarts on delimited continuations is the headline showcase — every other Lisp reinvents resumable exceptions on the host stack. On SX `signal`/`invoke-restart` is just a captured continuation. Plus CLOS, the LOOP macro, packages.
|
||||
|
||||
```
|
||||
description: common-lisp-on-sx queue loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/common-lisp-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
1. Read `plans/common-lisp-on-sx.md` — roadmap + Progress log.
|
||||
2. `ls lib/common-lisp/` — pick up from the most advanced file.
|
||||
3. If `lib/common-lisp/tests/*.sx` exist, run them. Green before new work.
|
||||
4. If `lib/common-lisp/scoreboard.md` exists, that's your baseline.
|
||||
|
||||
## The queue
|
||||
|
||||
Phase order per `plans/common-lisp-on-sx.md`:
|
||||
|
||||
- **Phase 1** — reader + parser (read macros `#'` `'` `` ` `` `,` `,@` `#( … )` `#:` `#\char` `#xFF` `#b1010`, ratios, dispatch chars, lambda lists with `&optional`/`&rest`/`&key`/`&aux`)
|
||||
- **Phase 2** — sequential eval + special forms (`let`/`let*`/`flet`/`labels`, `block`/`return-from`, `tagbody`/`go`, `unwind-protect`, multiple values, `setf` subset, dynamic variables)
|
||||
- **Phase 3** — **THE SHOWCASE**: condition system + restarts. `define-condition`, `signal`/`error`/`cerror`/`warn`, `handler-bind` (non-unwinding), `handler-case` (unwinding), `restart-case`, `restart-bind`, `find-restart`/`invoke-restart`/`compute-restarts`, `with-condition-restarts`. Classic programs (restart-demo, parse-recover, interactive-debugger) green.
|
||||
- **Phase 4** — CLOS: `defclass`, `defgeneric`, `defmethod` with `:before`/`:after`/`:around`, `call-next-method`, multiple dispatch
|
||||
- **Phase 5** — macros + LOOP macro + reader macros
|
||||
- **Phase 6** — packages + stdlib (sequence functions, FORMAT directives, drive corpus to 200+)
|
||||
|
||||
Within a phase, pick the checkbox that unlocks the most tests per effort.
|
||||
|
||||
Every iteration: implement → test → commit → tick `[ ]` → Progress log → next.
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `lib/common-lisp/**` and `plans/common-lisp-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. CL primitives go in `lib/common-lisp/runtime.sx`.
|
||||
- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop.
|
||||
- **Shared-file issues** → plan's Blockers with minimal repro.
|
||||
- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
|
||||
## Common-Lisp-specific gotchas
|
||||
|
||||
- **`handler-bind` is non-unwinding** — handlers can decline by returning normally, in which case `signal` keeps walking the chain. **`handler-case` is unwinding** — picking a handler aborts the protected form via a captured continuation. Don't conflate them.
|
||||
- **Restarts are not handlers.** `restart-case` establishes named *resumption points*; `signal` runs handler code with restarts visible; the handler chooses a restart by calling `invoke-restart`, which abandons handler stack and resumes at the restart point. Two stacks: handlers walk down, restarts wait to be invoked.
|
||||
- **`block` / `return-from`** is lexical. `block name … (return-from name v) …` captures `^k` once at entry; `return-from` invokes it. `return-from` to a name not in scope is an error (don't fall back to outer block).
|
||||
- **`tagbody` / `go`** — each tag in tagbody is a continuation; `go tag` invokes it. Tags are lexical, can only target tagbodies in scope.
|
||||
- **`unwind-protect`** runs cleanup on *any* non-local exit (return-from, throw, condition unwind). Implement as a scope frame fired by the cleanup machinery.
|
||||
- **Multiple values**: primary-value-only contexts (function args, `if` test, etc.) drop extras silently. `values` produces multiple. `multiple-value-bind` / `multiple-value-call` consume them. Don't auto-list.
|
||||
- **CLOS dispatch:** sort applicable methods by argument-list specificity (`subclassp` per arg, left-to-right); standard method combination calls primary methods most-specific-first via `call-next-method` chain. `:before` runs all before primaries; `:after` runs all after, in reverse-specificity. `:around` wraps everything.
|
||||
- **`call-next-method`** is a *continuation* available only inside a method body. Implement as a thunk stored in a dynamic-extent variable.
|
||||
- **Generalised reference (`setf`)**: `(setf (foo x) v)` ↦ `(setf-foo v x)`. Look up the setf-expander, not just a writer fn. `define-setf-expander` is mandatory for non-trivial places. Start with the symbolic / list / aref / slot-value cases.
|
||||
- **Dynamic variables (specials):** `defvar`/`defparameter` mark a symbol as special. `let` over a special name *rebinds* in dynamic extent (use parameterize-style scope), not lexical.
|
||||
- **Symbols are package-qualified.** Reader resolves `cl:car`, `mypkg::internal`, bare `foo` (current package). Internal vs external matters for `:` (one colon) reads.
|
||||
- **`nil` is also `()` is also the empty list.** Same object. `nil` is also false. CL has no distinct unit value.
|
||||
- **LOOP macro is huge.** Build incrementally — start with `for/in`, `for/from`, `collect`, `sum`, `count`, `repeat`. Add conditional clauses (`when`, `if`, `else`) once iteration drivers stable. `named` blocks + `return-from named` last.
|
||||
- **Test corpus:** custom + curated `ansi-test` slice. Place programs in `lib/common-lisp/tests/programs/` with `.lisp` extension.
|
||||
|
||||
## General gotchas (all loops)
|
||||
|
||||
- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences.
|
||||
- `cond`/`when`/`let` clauses evaluate only the last expr.
|
||||
- `type-of` on user fn returns `"lambda"`.
|
||||
- Shell heredoc `||` gets eaten — escape or use `case`.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.sx` unless non-obvious.
|
||||
- No new planning docs — update `plans/common-lisp-on-sx.md` inline.
|
||||
- Short, factual commit messages (`common-lisp: handler-bind + 12 tests`).
|
||||
- One feature per iteration. Commit. Log. Next.
|
||||
|
||||
Go. Read the plan; find first `[ ]`; implement.
|
||||
789
plans/agent-briefings/primitives-loop.md
Normal file
789
plans/agent-briefings/primitives-loop.md
Normal file
@@ -0,0 +1,789 @@
|
||||
# SX Primitives — Meta-Loop Briefing
|
||||
|
||||
Goal: add fundamental missing SX primitives in sequence, then sweep all language
|
||||
implementations to replace their workarounds. Full rationale: vectors fix O(n) array
|
||||
access across every language; numeric tower fixes float/int conflation; dynamic-wind
|
||||
fixes cleanup semantics; coroutine primitive unifies Ruby/Lua/Tcl; string buffer fixes
|
||||
O(n²) concat; algebraic data types eliminate the tagged-dict pattern everywhere.
|
||||
|
||||
**Each fire: find the first unchecked `[ ]`, do it, commit, tick it, stop.**
|
||||
Sub-items within a Phase may span multiple fires — just commit progress and tick what's done.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Prep (gate)
|
||||
|
||||
- [x] Stop new-language loops: send `/exit` to sx-loops windows for the four blank-slate
|
||||
languages that haven't committed workarounds yet:
|
||||
```
|
||||
tmux send-keys -t sx-loops:common-lisp "/exit" Enter
|
||||
tmux send-keys -t sx-loops:apl "/exit" Enter
|
||||
tmux send-keys -t sx-loops:ruby "/exit" Enter
|
||||
tmux send-keys -t sx-loops:tcl "/exit" Enter
|
||||
```
|
||||
Verify all four windows are idle (claude prompt, no active task).
|
||||
|
||||
- [x] E38 + E39 landed: check both Bucket-E branches for implementation commits.
|
||||
```
|
||||
git log --oneline hs-e38-sourceinfo | head -5
|
||||
git log --oneline hs-e39-webworker | head -5
|
||||
```
|
||||
If either branch has only its base commit (no impl work yet): note "pending" and stop —
|
||||
next fire re-checks. Proceed only when both have at least one implementation commit.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Vectors
|
||||
|
||||
Native mutable integer-indexed arrays. Fix: Lua O(n) sort, APL rank polymorphism, Ruby
|
||||
Array, Tcl lists, Common Lisp vectors, all using string-keyed dicts today.
|
||||
|
||||
Primitives to add:
|
||||
- `make-vector` `n` `[fill]` → vector of length n
|
||||
- `vector?` `v` → bool
|
||||
- `vector-ref` `v` `i` → element at index i (0-based)
|
||||
- `vector-set!` `v` `i` `x` → mutate in place
|
||||
- `vector-length` `v` → integer
|
||||
- `vector->list` `v` → list
|
||||
- `list->vector` `lst` → vector
|
||||
- `vector-fill!` `v` `x` → fill all elements
|
||||
- `vector-copy` `v` `[start]` `[end]` → fresh copy of slice
|
||||
|
||||
Steps:
|
||||
- [x] OCaml: add `SxVector of value array` to `hosts/ocaml/sx_types.ml`; implement all
|
||||
primitives in `hosts/ocaml/sx_primitives.ml` (or equivalent); wire into evaluator.
|
||||
Note: Vector type + most prims were already present; added bounds-checked vector-ref/set!
|
||||
and optional start/end to vector-copy. 10/10 vector tests pass (r7rs suite).
|
||||
- [x] Spec: add vector entries to `spec/primitives.sx` with type signatures and descriptions.
|
||||
All 10 vector primitives now have :as type annotations, :returns, and :doc strings.
|
||||
make-vector: optional fill param; vector-copy: optional start/end (done prev step).
|
||||
- [x] JS bootstrapper: implement vectors in `hosts/javascript/platform.js` (or equivalent);
|
||||
ensure `sx-browser.js` rebuild picks them up.
|
||||
Fixed index-of for lists (was returning -1 not NIL, breaking bind-lambda-params),
|
||||
added _lastErrorKont_/hostError/try-catch/without-io-hook stubs. Vectors work.
|
||||
- [x] Tests: 40+ tests in `spec/tests/test-vectors.sx` covering construction, ref, set!,
|
||||
length, conversions, fill, copy, bounds behaviour.
|
||||
42 tests, all pass. 1847 standard / 2362 full passing (up from 5).
|
||||
- [x] Verify: full test suite still passes (`node hosts/javascript/run_tests.js --full`).
|
||||
2362/4924 pass (improvement from pre-existing lambda binding bug, no regressions).
|
||||
- [x] Commit: `spec: vector primitive (make-vector/vector-ref/vector-set!/etc)`
|
||||
Committed as: js: fix lambda binding (index-of on lists), add vectors + R7RS platform stubs
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Numeric tower
|
||||
|
||||
Float ≠ integer distinction. Fix: Erlang `=:=`, Lua `math.type()`, Haskell `Num`/`Integral`,
|
||||
Common Lisp `integerp`/`floatp`/`ratio`, JS `Number.isInteger`.
|
||||
|
||||
Changes:
|
||||
- `parse-number` preserves float identity: `"1.0"` → float 1.0, not integer 1
|
||||
- New predicates: `integer?`, `float?`, `exact?`, `inexact?`
|
||||
- New coercions: `exact->inexact`, `inexact->exact`
|
||||
- Fix `floor`/`ceiling`/`truncate`/`round` to return integers when applied to floats
|
||||
- `number->string` renders `1.0` as `"1.0"`, `1` as `"1"`
|
||||
- Arithmetic: `(+ 1 1.0)` → `2.0` (float contagion), `(+ 1 1)` → `2` (integer)
|
||||
|
||||
Steps:
|
||||
- [x] OCaml: distinguish `Integer of int` / `Number of float` in `sx_types.ml`; update all
|
||||
arithmetic primitives for float contagion; fix `parse-number`.
|
||||
92/92 numeric tower tests pass; 4874 total (394 pre-existing hs-upstream fails unchanged).
|
||||
- [x] Spec: update `spec/primitives.sx` with new predicates + coercions; document contagion rules.
|
||||
Added integer?/float? predicates; updated number? body; / returns "float"; floor/ceil/truncate
|
||||
return "integer"; +/-/* doc float contagion; fixed double-paren params; 4874/394 baseline.
|
||||
- [x] JS bootstrapper: update number representation and arithmetic.
|
||||
Added integer?/float?/exact?/inexact?/truncate/remainder/modulo/random-int/exact->inexact/
|
||||
inexact->exact/parse-number. Fixed sx_server.ml epoch protocol for Integer type.
|
||||
JS: 1940 passed (+60); OCaml: 4874/394 unchanged. 6 tests JS-only fail (float≡int limitation).
|
||||
- [x] Tests: 92 tests in `spec/tests/test-numeric-tower.sx` — int-arithmetic, float-contagion,
|
||||
division, predicates, coercions, rounding, parse-number, equality, modulo, min-max, stringify.
|
||||
- [x] Verify: full suite passes. OCaml 4874/394 (baseline unchanged). JS 1940/2500 (+60 vs pre-tower).
|
||||
No regressions on any test that relied on `1.0 = 1` — those tests were already using integer
|
||||
literals which remain identical in JS. 6 JS-only failures are platform-inherent (JS float≡int).
|
||||
- [x] Commit: all work landed across 4 commits (c70bbdeb, 45ec5535, b12a22e6, f5acb31c).
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Dynamic-wind
|
||||
|
||||
Fix: Common Lisp `unwind-protect`, Ruby `ensure`, JS `finally`, Tcl `catch`+cleanup,
|
||||
Erlang `try...after` (currently uses double-nested guard workaround).
|
||||
|
||||
- [x] Spec: implement `dynamic-wind` in `spec/evaluator.sx` such that the after-thunk fires
|
||||
on both normal return AND non-local exit (raise/call-cc escape). Must compose with
|
||||
`guard` — currently they don't interact.
|
||||
- [x] OCaml: wire `dynamic-wind` through the CEK machine with a `WindFrame` continuation.
|
||||
- [x] JS bootstrapper: update.
|
||||
- [x] Tests: 20+ tests covering normal return, raise, call/cc escape, nested dynamic-winds.
|
||||
- [x] Commit: `spec: dynamic-wind + guard integration`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Coroutine primitive
|
||||
|
||||
Unify Ruby fibers, Lua coroutines, Tcl coroutines — all currently reimplemented separately
|
||||
using call/cc+perform/resume.
|
||||
|
||||
- [x] Spec: add `make-coroutine`, `coroutine-resume`, `coroutine-yield`, `coroutine?`,
|
||||
`coroutine-alive?` to `spec/primitives.sx`. Build on existing `perform`/`cek-resume`
|
||||
machinery — coroutines ARE perform/resume with a stable identity.
|
||||
Implemented as `spec/coroutines.sx` define-library; `make-coroutine` stub in evaluator.sx.
|
||||
17/17 coroutine tests pass (OCaml). Drives iteration via define+fn recursion (not named let —
|
||||
named let uses cek_call→cek_run which errors on IO suspension).
|
||||
- [x] OCaml: implement coroutine type; wire resume/yield through CEK suspension.
|
||||
No new native type needed — dict-based coroutine identity + existing cek-step-loop/
|
||||
cek-resume/perform primitives in run_tests.ml ARE the OCaml implementation. 17/17 pass.
|
||||
- [x] JS bootstrapper: update.
|
||||
All CEK primitives already in sx-browser.js. Fix: pre-load spec/coroutines.sx +
|
||||
spec/signals.sx in run_tests.js so (import (sx coroutines)) resolves without suspension.
|
||||
17/17 pass in JS. 1965/2500 (+25 vs 1940 baseline). Zero new failures.
|
||||
- [x] Tests: 25+ tests — multi-yield, final return, arg passthrough, alive? predicate,
|
||||
nested coroutines, "final return vs yield" distinction (the Lua gotcha).
|
||||
27 tests: added 10 new — state field inspection (ready/suspended/dead), yield from
|
||||
nested helper, initial resume arg ignored, mutable closure state, complex yield values,
|
||||
round-robin scheduling, factory-shared-no-state, non-coroutine error. 27/27 OCaml+JS.
|
||||
- [x] Commit: `spec: coroutine primitive (make-coroutine/resume/yield)`
|
||||
Phase 4 landed across 4 commits: 21cb9cf5 (spec library), 9eb12c66 (ocaml verified),
|
||||
b78e06a7 (js pre-load), 0ffe208e (27 tests). Phase 4 complete.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — String buffer
|
||||
|
||||
Fix O(n²) string concatenation in loops across Lua, Ruby, Common Lisp, Tcl.
|
||||
|
||||
- [x] Spec + OCaml: add `make-string-buffer`, `string-buffer-append!`, `string-buffer->string`,
|
||||
`string-buffer-length` to primitives. OCaml: `Buffer.t` wrapper. JS: array+join.
|
||||
Also: string-buffer? predicate; SxStringBuffer._string_buffer marker for typeOf/dict?
|
||||
exclusion; inspect case in sx_types.ml. 17/17 tests OCaml+JS.
|
||||
- [x] Tests: 15+ tests.
|
||||
17 tests written inline with Spec+OCaml step: construction, type-of, empty/length,
|
||||
single/multi-append, append-returns-nil, empty-string-append, reuse-after-to-string,
|
||||
independence, loop-building, CSV-row, unicode, repeated-to-string, join-pattern.
|
||||
17/17 OCaml+JS.
|
||||
- [x] Commit: `spec: string-buffer primitive`
|
||||
Committed as d98b5fa2 — all work in one commit (OCaml type + primitives + JS + spec + 17 tests).
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — Algebraic data types
|
||||
|
||||
The deepest structural gap. Every language uses `{:tag "..." :field ...}` tagged dicts to
|
||||
simulate sum types. A native `define-type` + `match` form eliminates this everywhere.
|
||||
|
||||
- [x] Design: write `plans/designs/sx-adt.md` covering syntax, CEK dispatch, interaction with
|
||||
existing `cond`/`case`, exhaustiveness checking, recursive types, pattern variables.
|
||||
Draft, then stop — next fire reviews design before implementing.
|
||||
Written: define-type/match syntax, AdtValue runtime rep, stepSfDefineType + MatchFrame
|
||||
CEK dispatch, exhaustiveness warnings via _adt_registry, recursive types, nested patterns,
|
||||
wildcard _, 3-phase impl plan (basic/nested/exhaustiveness), open questions on accessors/singletons/inspect.
|
||||
|
||||
- [x] Spec: implement `define-type` special form in `spec/evaluator.sx`:
|
||||
`(define-type Name (Ctor1 field...) (Ctor2 field...) ...)`
|
||||
Creates constructor functions `Ctor1`, `Ctor2` + predicate `Name?`.
|
||||
|
||||
- [x] Spec: implement `match` special form:
|
||||
`(match expr ((Ctor1 a b) body) ((Ctor2 x) body) (else body))`
|
||||
Exhaustiveness warning if not all constructors covered and no `else`.
|
||||
|
||||
- [x] OCaml: add `SxAdt of string * value array` to types; implement constructors + match.
|
||||
Dict-based ADT (no native type needed — matches spec). Hand-written sf_define_type
|
||||
in bootstrap.py FIXUPS; registered via register_special_form. 172 assertions pass.
|
||||
4280/1080 full suite (37 improvement over old baseline 4243/1117).
|
||||
- [x] JS bootstrapper: update.
|
||||
No changes needed — define-type/match are spec-level; sx-browser.js rebuilt at 0dc7e159.
|
||||
40/40 ADT tests pass JS. 2032/2500 total (+67 vs 1965 phase-4 baseline).
|
||||
- [x] Tests: 40+ tests in `spec/tests/test-adt.sx`.
|
||||
40 tests written across two spec commits (6c872107+0dc7e159). All pass OCaml+JS.
|
||||
- [x] Commit: `spec: algebraic data types (define-type + match)`
|
||||
Phase 6 landed across 5 commits: 6c872107 (define-type spec), 0dc7e159 (match spec),
|
||||
5d1913e7 (ocaml bootstrap), f63b2147 (plan tick). JS already current.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Bitwise operations
|
||||
|
||||
Completely absent today. Needed by: Forth (core), APL (array masks), Erlang (bitmatch),
|
||||
JS (typed arrays, bitfields), Common Lisp (`logand`/`logior`/`logxor`/`lognot`/`ash`).
|
||||
|
||||
Primitives to add:
|
||||
- `bitwise-and` `a` `b` → integer
|
||||
- `bitwise-or` `a` `b` → integer
|
||||
- `bitwise-xor` `a` `b` → integer
|
||||
- `bitwise-not` `a` → integer
|
||||
- `arithmetic-shift` `a` `count` → integer (left if count > 0, right if count < 0)
|
||||
- `bit-count` `a` → number of set bits (popcount)
|
||||
- `integer-length` `a` → number of bits needed to represent a
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add entries to `spec/primitives.sx` with type signatures.
|
||||
stdlib.bitwise module with 7 entries appended to spec/primitives.sx.
|
||||
- [x] OCaml: implement in `hosts/ocaml/sx_primitives.ml` using OCaml `land`/`lor`/`lxor`/`lnot`/`lsl`/`asr`.
|
||||
land/lor/lxor/lnot/lsl/asr in sx_primitives.ml. bit-count: Kernighan loop. integer-length: lsr loop.
|
||||
- [x] JS bootstrapper: implement in `hosts/javascript/platform.js` using JS `&`/`|`/`^`/`~`/`<<`/`>>`.
|
||||
stdlib.bitwise module added to PRIMITIVES_JS_MODULES. bit-count: Hamming weight. integer-length: Math.clz32.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-bitwise.sx` — basic ops, shift left/right, negative numbers, popcount.
|
||||
26 tests, 158 assertions, all pass OCaml+JS.
|
||||
- [x] Commit: `spec: bitwise operations (bitwise-and/or/xor/not, arithmetic-shift, bit-count)`
|
||||
Committed a8a79dc9. Phase 7 complete in single commit.
|
||||
|
||||
---
|
||||
|
||||
## Phase 8 — Multiple values
|
||||
|
||||
R7RS standard. Common Lisp uses them heavily; Haskell tuples map naturally; Erlang
|
||||
multi-return. Without them, every function returning two things encodes it as a list or dict.
|
||||
|
||||
Primitives / forms to add:
|
||||
- `values` `v...` → multiple-value object
|
||||
- `call-with-values` `producer` `consumer` → applies consumer to values from producer
|
||||
- `let-values` `(((a b) expr) ...)` `body` — binding form (special form in evaluator)
|
||||
- `define-values` `(a b ...)` `expr` — top-level multi-value bind
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxValues` type to evaluator; implement `values` + `call-with-values` in
|
||||
`spec/evaluator.sx`; add `let-values` / `define-values` special forms.
|
||||
- [x] OCaml: add `SxValues of value list` to `sx_types.ml`; wire through CEK.
|
||||
- [x] JS bootstrapper: implement values type + forms.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-values.sx` — basic producer/consumer, let-values
|
||||
destructuring, define-values, interaction with `begin`/`do`.
|
||||
- [x] Commit: `spec: multiple values (values/call-with-values/let-values)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 9 — Promises (lazy evaluation)
|
||||
|
||||
Critical for Haskell — lazy evaluation is so central that without it the Haskell
|
||||
implementation can't be idiomatic. Also useful for lazy lists in Common Lisp and
|
||||
lazy streams in Scheme-style code generally.
|
||||
|
||||
Primitives / forms to add:
|
||||
- `delay` `expr` → promise (special form — expr not evaluated yet)
|
||||
- `force` `p` → evaluate promise, cache result, return it
|
||||
- `make-promise` `v` → already-forced promise wrapping v
|
||||
- `promise?` `v` → bool
|
||||
- `delay-force` `expr` → for iterative lazy sequences (avoids stack growth in lazy streams)
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `delay` / `delay-force` special forms to `spec/evaluator.sx`; add promise
|
||||
type with mutable forced/value slots; `force` checks if already forced before eval.
|
||||
- [x] OCaml: add `SxPromise of { mutable forced: bool; mutable value: value; thunk: value }`;
|
||||
wire `delay`/`force`/`delay-force` through CEK.
|
||||
- [x] JS bootstrapper: implement promise type + forms.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-promises.sx` — basic delay/force, memoisation
|
||||
(forced only once), delay-force lazy stream, promise? predicate, make-promise.
|
||||
- [x] Commit: `spec: promises — delay/force/delay-force for lazy evaluation`
|
||||
|
||||
---
|
||||
|
||||
## Phase 10 — Mutable hash tables
|
||||
|
||||
Distinct from SX's immutable dicts. Dict primitives copy on every update — fine for
|
||||
functional code, wrong for table-heavy language implementations. Lua tables, Smalltalk
|
||||
dicts, Erlang process dictionaries, and JS Map all need O(1) mutable associative storage.
|
||||
|
||||
Primitives to add:
|
||||
- `make-hash-table` `[capacity]` → fresh mutable hash table
|
||||
- `hash-table?` `v` → bool
|
||||
- `hash-table-set!` `ht` `key` `val` → mutate in place
|
||||
- `hash-table-ref` `ht` `key` `[default]` → value or default/error
|
||||
- `hash-table-delete!` `ht` `key` → remove entry
|
||||
- `hash-table-size` `ht` → integer
|
||||
- `hash-table-keys` `ht` → list of keys
|
||||
- `hash-table-values` `ht` → list of values
|
||||
- `hash-table->alist` `ht` → list of (key . value) pairs
|
||||
- `hash-table-for-each` `ht` `fn` → iterate (fn key val) for side effects
|
||||
- `hash-table-merge!` `dst` `src` → merge src into dst in place
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add entries to `spec/primitives.sx`.
|
||||
stdlib.hash-table module with 11 define-primitive entries appended to spec/primitives.sx.
|
||||
- [x] OCaml: add `HashTable of (value, value) Hashtbl.t` to `sx_types.ml`; implement
|
||||
all primitives in `hosts/ocaml/sx_primitives.ml`.
|
||||
HashTable variant in sx_types.ml; type_of/inspect cases added; 11 primitives in sx_primitives.ml;
|
||||
fixed _cek_call_ref reference for hash-table-for-each. 4385/1080 (+28).
|
||||
- [x] JS bootstrapper: implement using JS `Map` in `hosts/javascript/platform.js`.
|
||||
SxHashTable class with Map; _hash_table marker; dict?/type-of exclusion; apply() for for-each.
|
||||
2137/2500 (+4 vs phase-9 baseline).
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-hash-table.sx` — set/ref/delete, size, iteration,
|
||||
default on missing key, merge, keys/values lists.
|
||||
28 tests; all pass OCaml+JS. Used empty? not assert= for empty-list comparisons.
|
||||
- [x] Commit: `spec: mutable hash tables (make-hash-table/ref/set!/delete!/etc)`
|
||||
Committed 133bdf52. Phase 10 complete.
|
||||
|
||||
---
|
||||
|
||||
## Phase 11 — Sequence protocol
|
||||
|
||||
Unified iteration over lists and vectors without conversion. Currently `map`/`filter`/
|
||||
`for-each` only work on lists — you must `vector->list` first, which defeats the purpose
|
||||
of vectors. A sequence protocol makes all collection operations polymorphic.
|
||||
|
||||
Approach: extend existing `map`/`filter`/`reduce`/`for-each`/`some`/`every?` to dispatch
|
||||
on type (list → existing path, vector → index loop, string → char iteration). Add:
|
||||
- `in-range` `start` `[end]` `[step]` → lazy range sequence (works with `for-each`/`map`)
|
||||
- `sequence->list` `s` → coerce any sequence to list
|
||||
- `sequence->vector` `s` → coerce any sequence to vector
|
||||
- `sequence-length` `s` → length of any sequence
|
||||
- `sequence-ref` `s` `i` → element by index (lists and vectors)
|
||||
- `sequence-append` `s1` `s2` → concatenate two same-type sequences
|
||||
|
||||
Steps:
|
||||
- [x] Spec: extend `map`/`filter`/`reduce`/`for-each`/`some`/`every?` in `spec/evaluator.sx`
|
||||
to type-dispatch; add `in-range` lazy sequence type + helpers.
|
||||
- [x] OCaml: update HO form dispatch; add `SxRange` or use lazy list; implement `sequence-*`
|
||||
primitives.
|
||||
seq_to_list helper before let-rec block; ho_setup_dispatch wraps all 7 coll bindings;
|
||||
seq-to-list/sequence-to-list/vector/length/ref/append/in-range in sx_primitives.ml.
|
||||
4385/1080 (all failures pre-existing hs-*/regex; 0 regressions).
|
||||
- [x] JS bootstrapper: update.
|
||||
Already done in Spec step (da4b526a) — sx-browser.js rebuilt with seqToList/sequenceToList/
|
||||
sequenceToVector/sequenceLength/sequenceRef/sequenceAppend/inRange. 2137/2500 JS tests pass.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-sequences.sx` — map over vector, filter over
|
||||
range, for-each over string chars, sequence-append, sequence->list/vector coercions.
|
||||
45 tests all passing: JS 2185/2498 (+48), OCaml 4424/1087 (+39). Fixed: vector? rename
|
||||
(isVector), vectorLength/vectorRef/reverse aliases, in-range letrec→build-range,
|
||||
sequence-length nil=0, assert-equal for list comparisons. Committed 0fe00bf7.
|
||||
- [x] Commit: `spec: sequence protocol — polymorphic map/filter/for-each over list/vector/range`
|
||||
Work landed across da4b526a (Spec), 7286629c (OCaml), 06a3eee1 (JS bootstrap), 0fe00bf7 (Tests).
|
||||
|
||||
---
|
||||
|
||||
## Phase 12 — gensym + symbol interning
|
||||
|
||||
Unique symbol generation. Tiny to implement; broadly needed: Prolog uses it for fresh
|
||||
variable names, Common Lisp uses it constantly in macros, any hygienic macro system needs
|
||||
it, and Smalltalk uses it for anonymous class/method naming.
|
||||
|
||||
Primitives to add:
|
||||
- `gensym` `[prefix]` → unique symbol, e.g. `g42`, `var-17`. Counter-based, monotonically increasing.
|
||||
- `symbol-interned?` `s` → bool — whether the symbol is in the global intern table
|
||||
- `intern` `str` → symbol — intern a string as a symbol (string->symbol already exists; this is
|
||||
the explicit interning operation for languages that distinguish interned vs uninterned)
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `gensym` counter to evaluator state; implement in `spec/evaluator.sx`.
|
||||
`string->symbol` already exists — `gensym` is just a counter-suffixed variant.
|
||||
Added *gensym-counter*/gensym/string->symbol/symbol->string/intern/symbol-interned? to
|
||||
evaluator.sx. Added string->symbol/symbol->string transpiler renames + platform.py aliases.
|
||||
JS 2186/+1. OCaml builds. Committed edf4e525.
|
||||
- [x] OCaml: add global gensym counter; implement primitives.
|
||||
gensym_counter ref + gensym/string->symbol/symbol->string/intern/symbol-interned? in sx_primitives.ml.
|
||||
Also fixed ListRef case in seq_to_list (both sx_ref.ml + sx_primitives.ml). 4431/1080 (was 4385/1080).
|
||||
- [x] JS bootstrapper: implement.
|
||||
Already done in Spec step. JS 2186/2497, all sequence tests pass.
|
||||
- [x] Tests: 15+ tests in `spec/tests/test-gensym.sx` — uniqueness, prefix, symbol?, string->symbol round-trip.
|
||||
19 tests. OCaml 4450/1080, JS 2205/2497, zero regressions.
|
||||
- [x] Commit: `spec: gensym + symbol interning` — 0862a614
|
||||
|
||||
---
|
||||
|
||||
## Phase 13 — Character type
|
||||
|
||||
Common Lisp and Haskell have a distinct `Char` type that is not a string. Without it both
|
||||
implementations are approximations — CL's `#\a` literal and Haskell's `'a'` both need a
|
||||
real char value, not a length-1 string.
|
||||
|
||||
Primitives to add:
|
||||
- `char?` `v` → bool
|
||||
- `char->integer` `c` → Unicode codepoint integer
|
||||
- `integer->char` `n` → char
|
||||
- `char=?` `char<?` `char>?` `char<=?` `char>=?` → comparators
|
||||
- `char-ci=?` `char-ci<?` etc. → case-insensitive comparators
|
||||
- `char-alphabetic?` `char-numeric?` `char-whitespace?` → predicates
|
||||
- `char-upper-case?` `char-lower-case?` → predicates
|
||||
- `char-upcase` `char-downcase` → char → char
|
||||
- `string->list` extended to return chars (not length-1 strings)
|
||||
- `list->string` accepting chars
|
||||
|
||||
Also: `#\a` reader syntax for char literals (parser addition).
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxChar` type to evaluator; add char literal syntax `#\a`/`#\space`/`#\newline`
|
||||
to `spec/parser.sx`; implement all predicates + comparators.
|
||||
- [x] OCaml: add `SxChar of char` to `sx_types.ml`; implement primitives.
|
||||
- [x] JS bootstrapper: implement char type wrapping a codepoint integer.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-chars.sx` — literals, char->integer round-trip,
|
||||
comparators, predicates, upcase/downcase, string<->list with chars.
|
||||
- [x] Commit: `spec: character type (char? char->integer #\\a literals + predicates)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 14 — String ports
|
||||
|
||||
Needed for any language with a reader protocol: Common Lisp's `read`, Prolog's term parser,
|
||||
Smalltalk's `printString`. Without string ports these all do their own character walking
|
||||
on raw strings rather than treating a string as an I/O stream.
|
||||
|
||||
Primitives to add:
|
||||
- `open-input-string` `str` → input port
|
||||
- `open-output-string` → output port
|
||||
- `get-output-string` `port` → string (flush output port to string)
|
||||
- `input-port?` `output-port?` `port?` → predicates
|
||||
- `read-char` `[port]` → char or eof-object
|
||||
- `peek-char` `[port]` → char or eof-object (non-consuming)
|
||||
- `read-line` `[port]` → string or eof-object
|
||||
- `write-char` `char` `[port]` → void
|
||||
- `write-string` `str` `[port]` → void
|
||||
- `eof-object` → the eof sentinel
|
||||
- `eof-object?` `v` → bool
|
||||
- `close-port` `port` → void
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add port type + eof-object to evaluator; implement all primitives.
|
||||
Ports are mutable objects with a position cursor (input) or accumulation buffer (output).
|
||||
- [x] OCaml: add `SxPort` variant covering string-input-port and string-output-port;
|
||||
Buffer.t for output, string+offset for input.
|
||||
- [x] JS bootstrapper: implement port type.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-ports.sx` — open/read/peek/eof, output accumulation,
|
||||
read-line, write-char, close.
|
||||
- [x] Commit: `spec: string ports (open-input-string/open-output-string/read-char/etc)` — 3d8937d7
|
||||
|
||||
---
|
||||
|
||||
## Phase 15 — Math completeness
|
||||
|
||||
Filling specific gaps that multiple language implementations need.
|
||||
|
||||
### 15a — modulo / remainder / quotient distinction
|
||||
They differ on negative numbers — critical for Erlang `rem`, Haskell `mod`/`rem`, CL `mod`/`rem`:
|
||||
- `quotient` `a` `b` → truncate toward zero (same sign as dividend)
|
||||
- `remainder` `a` `b` → sign follows dividend (truncation division)
|
||||
- `modulo` `a` `b` → sign follows divisor (floor division) — R7RS
|
||||
|
||||
### 15b — Trigonometry and transcendentals
|
||||
Lua, Haskell, Erlang, CL all need: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `exp`,
|
||||
`log`, `sqrt`, `expt`. Check which are already present; add missing ones.
|
||||
|
||||
### 15c — GCD / LCM
|
||||
`gcd` `a` `b` → greatest common divisor; `lcm` `a` `b` → least common multiple.
|
||||
Needed by Haskell `Rational`, CL, and any language doing fraction arithmetic.
|
||||
|
||||
### 15d — Radix number parsing / formatting
|
||||
`(number->string n radix)` → e.g. `(number->string 255 16)` → `"ff"`.
|
||||
`(string->number s radix)` → e.g. `(string->number "ff" 16)` → `255`.
|
||||
Needed by: Common Lisp, Smalltalk, Erlang integer formatting.
|
||||
|
||||
Steps:
|
||||
- [x] Audit which trig / math functions are already in `spec/primitives.sx`; note gaps.
|
||||
- [x] Spec + OCaml + JS: implement missing trig (`sin`/`cos`/`tan`/`asin`/`acos`/`atan`/`exp`/`log`).
|
||||
- [x] Spec + OCaml + JS: `quotient`/`remainder`/`modulo` with correct negative semantics.
|
||||
- [x] Spec + OCaml + JS: `gcd`/`lcm`.
|
||||
- [x] Spec + OCaml + JS: radix variants of `number->string`/`string->number`.
|
||||
- [x] Tests: 40+ tests in `spec/tests/test-math.sx`.
|
||||
- [x] Commit: `spec: math completeness — trig, quotient/remainder/modulo, gcd/lcm, radix`
|
||||
|
||||
---
|
||||
|
||||
## Phase 16 — Rational numbers
|
||||
|
||||
Haskell's `Rational` type and Common Lisp ratios (`1/3`) both need this. Natural extension
|
||||
of the numeric tower (Phase 2) — rationals are the third numeric type alongside int and float.
|
||||
|
||||
Primitives to add:
|
||||
- `make-rational` `numerator` `denominator` → rational (auto-reduced by GCD)
|
||||
- `rational?` `v` → bool
|
||||
- `numerator` `r` → integer
|
||||
- `denominator` `r` → integer
|
||||
- Reader syntax: `1/3` parsed as rational literal
|
||||
- Arithmetic: `(+ 1/3 1/6)` → `1/2`; `(* 1/3 3)` → `1`; mixed int/rational → rational
|
||||
- `exact->inexact` on rational → float; `inexact->exact` on float → rational approximation
|
||||
- `(number->string 1/3)` → `"1/3"`
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxRational` type; add `n/d` reader syntax to `spec/parser.sx`; extend
|
||||
all arithmetic primitives for rational contagion (int op rational → rational, rational
|
||||
op float → float).
|
||||
- [x] OCaml: add `SxRational of int * int` (stored in reduced form); implement all arithmetic.
|
||||
as_number + safe_eq extended for cross-type rational equality (= 2.5 5/2) → true.
|
||||
- [x] JS bootstrapper: implement rational type.
|
||||
JS keeps int/int → float for CSS backward compatibility; SxRational class with _rational marker.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-rationals.sx` — literals, arithmetic, reduction,
|
||||
mixed numeric tower, exact<->inexact conversion. 62 tests, all pass.
|
||||
- [x] Commit: `spec: rational numbers — 1/3 literals, arithmetic, numeric tower integration`
|
||||
Committed 036022cc. JS: 2232 passed. OCaml: 4532 passed (+11).
|
||||
|
||||
---
|
||||
|
||||
## Phase 17 — read / write / display
|
||||
|
||||
Completes the I/O model. Builds on string ports (Phase 14) and char type (Phase 13).
|
||||
`read` parses any SX value from a port; `write` serializes with quoting (round-trippable);
|
||||
`display` serializes without quoting (human-readable). Common Lisp's `read` macro,
|
||||
Prolog term I/O, and Smalltalk's `printString` all need this.
|
||||
|
||||
Primitives to add:
|
||||
- `read` `[port]` → SX value or eof-object — full SX parser reading from a port
|
||||
- `read-char` already in Phase 14; `read` uses it internally
|
||||
- `write` `val` `[port]` → void — serializes with quotes: `"hello"`, `#\a`, `(1 2 3)`
|
||||
- `display` `val` `[port]` → void — serializes without quotes: `hello`, `a`, `(1 2 3)`
|
||||
- `newline` `[port]` → void — writes `\n`
|
||||
- `write-to-string` `val` → string — convenience: `(write val (open-output-string))`
|
||||
- `display-to-string` `val` → string — convenience
|
||||
|
||||
Steps:
|
||||
- [x] Spec: implement `read` in `spec/evaluator.sx` — wraps the existing parser to read
|
||||
one datum from a port cursor; handles eof gracefully.
|
||||
- [x] Spec: implement `write`/`display`/`newline` — extend the existing serializer for
|
||||
port output; `write` quotes strings + uses `#\` for chars, `display` does not.
|
||||
- [x] OCaml: wire `read` through port type; implement `write`/`display` output path.
|
||||
- [x] JS bootstrapper: implement.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-read-write.sx` — read string literal, read list,
|
||||
read eof, write round-trip, display vs write quoting, newline, write-to-string.
|
||||
- [x] Commit: `spec: read/write/display — S-expression reader/writer on ports`
|
||||
|
||||
---
|
||||
|
||||
## Phase 18 — Sets
|
||||
|
||||
O(1) membership testing. Distinct from hash tables (unkeyed) and lists (O(n)).
|
||||
Erlang has sets as a stdlib staple, Haskell `Data.Set`, APL uses set operations
|
||||
constantly, Common Lisp has `union`/`intersection` on lists but a native set is O(1).
|
||||
|
||||
Primitives to add:
|
||||
- `make-set` `[list]` → fresh set, optionally seeded from list
|
||||
- `set?` `v` → bool
|
||||
- `set-add!` `s` `val` → void
|
||||
- `set-member?` `s` `val` → bool
|
||||
- `set-remove!` `s` `val` → void
|
||||
- `set-size` `s` → integer
|
||||
- `set->list` `s` → list (unspecified order)
|
||||
- `list->set` `lst` → set
|
||||
- `set-union` `s1` `s2` → new set
|
||||
- `set-intersection` `s1` `s2` → new set
|
||||
- `set-difference` `s1` `s2` → new set (elements in s1 not in s2)
|
||||
- `set-for-each` `s` `fn` → iterate for side effects
|
||||
- `set-map` `s` `fn` → new set of mapped values
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add entries to `spec/primitives.sx`.
|
||||
- [x] OCaml: implement using `Hashtbl.t` with unit values (or a proper `Set` functor
|
||||
with a comparison function); add `SxSet` to `sx_types.ml`.
|
||||
- [x] JS bootstrapper: implement using JS `Set`.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-sets.sx` — add/member/remove, union/intersection/
|
||||
difference, list conversion, for-each, size.
|
||||
- [x] Commit: `spec: sets (make-set/set-add!/set-member?/union/intersection/etc)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 19 — Regular expressions as primitives
|
||||
|
||||
`lib/js/regex.sx` is a pure-SX regex engine already written. Promoting it to a primitive
|
||||
gives every language free regex without reinventing: Lua patterns, Tcl `regexp`, Ruby regex,
|
||||
JS regex, Erlang `re` module. Mostly a wiring job — the implementation exists.
|
||||
|
||||
Primitives to add:
|
||||
- `make-regexp` `pattern` `[flags]` → regexp object (`flags`: `"i"` case-insensitive, `"g"` global, `"m"` multiline)
|
||||
- `regexp?` `v` → bool
|
||||
- `regexp-match` `re` `str` → match dict `{:match "..." :start N :end N :groups (...)}` or nil
|
||||
- `regexp-match-all` `re` `str` → list of match dicts
|
||||
- `regexp-replace` `re` `str` `replacement` → string with first match replaced
|
||||
- `regexp-replace-all` `re` `str` `replacement` → string with all matches replaced
|
||||
- `regexp-split` `re` `str` → list of strings (split on matches)
|
||||
- Reader syntax: `#/pattern/flags` for regexp literals (parser addition)
|
||||
|
||||
Steps:
|
||||
- [x] Audit `lib/js/regex.sx` — understand the API it already exposes; map to the
|
||||
primitive API above.
|
||||
- [x] Spec: add `SxRegexp` type to evaluator; add `#/pattern/flags` literal syntax to
|
||||
`spec/parser.sx`; wire `lib/js/regex.sx` engine as the implementation.
|
||||
- [x] OCaml: implement using OCaml `Re` library (or `Str`); add `SxRegexp` to types.
|
||||
- [x] JS bootstrapper: use native JS `RegExp`; wrap in the primitive API.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-regexp.sx` — basic match, groups, replace,
|
||||
replace-all, split, flags (case-insensitive), no-match nil return.
|
||||
- [x] Commit: `spec: regular expressions (make-regexp/regexp-match/regexp-replace + #/pat/ literals)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 20 — Bytevectors
|
||||
|
||||
R7RS standard. Needed for WebSocket binary frames (E36), binary protocol parsing, and
|
||||
efficient string encoding. Also the foundation for proper Unicode: `string->utf8` /
|
||||
`utf8->string` require a byte array type.
|
||||
|
||||
Primitives to add:
|
||||
- `make-bytevector` `n` `[fill]` → bytevector of n bytes (fill defaults to 0)
|
||||
- `bytevector?` `v` → bool
|
||||
- `bytevector-length` `bv` → integer
|
||||
- `bytevector-u8-ref` `bv` `i` → byte 0–255
|
||||
- `bytevector-u8-set!` `bv` `i` `byte` → void
|
||||
- `bytevector-copy` `bv` `[start]` `[end]` → fresh copy
|
||||
- `bytevector-copy!` `dst` `at` `src` `[start]` `[end]` → in-place copy
|
||||
- `bytevector-append` `bv...` → concatenated bytevector
|
||||
- `utf8->string` `bv` `[start]` `[end]` → string decoded as UTF-8
|
||||
- `string->utf8` `str` `[start]` `[end]` → bytevector UTF-8 encoded
|
||||
- `bytevector->list` / `list->bytevector` → conversion
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxBytevector` type; implement all primitives in `spec/evaluator.sx` / `spec/primitives.sx`.
|
||||
- [x] OCaml: add `SxBytevector of bytes` to `sx_types.ml`; implement primitives using
|
||||
OCaml `Bytes`.
|
||||
- [x] JS bootstrapper: implement using `Uint8Array`.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-bytevectors.sx` — construction, ref/set, copy,
|
||||
append, utf8 round-trip, slice.
|
||||
- [x] Commit: `spec: bytevectors (make-bytevector/u8-ref/u8-set!/utf8->string/etc)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 21 — format
|
||||
|
||||
CL-style string formatting beyond `str`. `(format "Hello ~a, age ~d" name age)`.
|
||||
Haskell `printf`, Erlang `io:format`, CL `format`, and general string templating all use this idiom.
|
||||
|
||||
Directives:
|
||||
- `~a` — display (no quotes)
|
||||
- `~s` — write (with quotes)
|
||||
- `~d` — decimal integer
|
||||
- `~x` — hexadecimal integer
|
||||
- `~o` — octal integer
|
||||
- `~b` — binary integer
|
||||
- `~f` — fixed-point float
|
||||
- `~e` — scientific notation float
|
||||
- `~%` — newline
|
||||
- `~&` — fresh line (newline only if not already at start of line)
|
||||
- `~~` — literal tilde
|
||||
- `~t` — tab
|
||||
|
||||
Signature: `(format template arg...)` → string.
|
||||
Optional: `(format port template arg...)` — write to port directly.
|
||||
|
||||
Steps:
|
||||
- [x] Spec: implement `format` as a pure SX function in `spec/stdlib.sx` — parses
|
||||
`~X` directives, dispatches to `display`/`write`/`number->string` as appropriate.
|
||||
Pure SX: no host calls needed. Self-hosting — uses string-buffer (Phase 5) internally.
|
||||
- [x] OCaml: expose as a primitive (or let it run as SX through the evaluator).
|
||||
Added format-decimal OCaml primitive; fixed lib/r7rs.sx number->string to support radix.
|
||||
- [x] JS bootstrapper: same.
|
||||
- [x] Tests: 28 tests in `spec/tests/test-format.sx` — each directive, multiple args,
|
||||
nested format, `~~` escape. 28/28 pass on both JS and OCaml.
|
||||
- [x] Commit: `spec: format — CL-style string formatting (~a ~s ~d ~x ~% etc)` — 4d7b3e29
|
||||
|
||||
---
|
||||
|
||||
## Phase 22 — Language sweep
|
||||
|
||||
Replace workarounds with primitives. One language per fire (or per sub-item for big ones).
|
||||
Start with blank slates (CL, APL, Ruby, Tcl) — they haven't committed to workarounds yet.
|
||||
|
||||
**Scope per language:** only `lib/<lang>/**`. Don't touch spec or other languages.
|
||||
Brief each language's loop agent (or do inline) after rebasing their branch onto architecture.
|
||||
|
||||
- [x] Restart CL/APL/Ruby/Tcl loops with updated briefing pointing to new primitives.
|
||||
Added `## SX primitive baseline` section to plans/common-lisp-on-sx.md,
|
||||
plans/apl-on-sx.md, plans/ruby-on-sx.md, plans/tcl-on-sx.md. f43659ce.
|
||||
|
||||
- [x] Common Lisp: char type (`#\a`); string ports + `read`/`write` for reader/printer;
|
||||
gensym for macros; rational numbers for CL ratios; multiple values; sets for CL set ops;
|
||||
`modulo`/`remainder`/`quotient`; radix formatting; `format` for `cl:format`.
|
||||
lib/common-lisp/runtime.sx (103 forms) + test.sh (68/68 pass). 1ad8e74a.
|
||||
|
||||
- [x] Lua: vectors for arrays; hash tables for Lua tables; `delay`/`force` for lazy iterators;
|
||||
regexp for Lua pattern matching; trig from math completeness; bytevectors for binary I/O.
|
||||
math/string/table stdlib tables + lua-force. 185/185 pass. ec3512d6.
|
||||
|
||||
- [x] Erlang: numeric tower for float/int; bitwise ops for bitmatch; multiple values for
|
||||
multi-return; sets for Erlang sets; `remainder` for `rem`; regexp for `re` module.
|
||||
lib/erlang/runtime.sx (63 forms) + test.sh (55/55 pass). 3c0a9632.
|
||||
|
||||
- [x] Haskell: numeric tower for `Num`/`Integral`/`Fractional`; promises for lazy evaluation
|
||||
(critical); multiple values for tuples; rational numbers for `Rational`; char type for
|
||||
`Char`; `gcd`/`lcm`; sets for `Data.Set`; `read`/`write` for `Show`/`Read` instances.
|
||||
lib/haskell/runtime.sx (113 forms) + tests/runtime.sx (143/143 pass). c02ffcf3.
|
||||
|
||||
- [x] JS: vectors for Array; hash tables for `Map`; sets for `Set`; bitwise ops for typed
|
||||
arrays; regexp for JS regex; bytevectors for `Uint8Array`; radix formatting.
|
||||
lib/js/stdlib.sx (36 forms) + test.sh epochs 6000-6032 (25/25 pass). COMMIT.
|
||||
|
||||
- [x] Smalltalk: vectors for `Array new:`; hash tables for `Dictionary new`; sets for
|
||||
`Set new`; char type for `Character`; string ports + `read`/`write` for `printString`.
|
||||
lib/smalltalk/runtime.sx (72 forms) + tests/runtime.sx (86/86 pass). COMMIT.
|
||||
|
||||
- [x] APL: vectors as core array type; bitwise ops for array masks; sets for APL set ops;
|
||||
sequence protocol for rank-polymorphic operations; format for APL output formatting.
|
||||
lib/apl/runtime.sx (60 forms) + tests/runtime.sx (73/73 pass). COMMIT.
|
||||
|
||||
- [x] Ruby: coroutines for fibers; hash tables for `Hash`; sets for `Set`; regexp for
|
||||
Ruby regex; string ports for `StringIO`; bytevectors for `String` binary encoding.
|
||||
lib/ruby/runtime.sx (61 forms) + tests/runtime.sx (76/76 pass). COMMIT.
|
||||
Note: rb-fiber-yield from letrec-bound lambdas fails (JIT VM can't invoke callcc
|
||||
continuations as escapes); workaround: use top-level helper fns for recursive yields.
|
||||
|
||||
- [x] Tcl: string ports for Tcl channel abstraction; string-buffer for `append`; coroutines
|
||||
for Tcl coroutines; regexp for Tcl `regexp`; format for Tcl `format`.
|
||||
lib/tcl/runtime.sx (37 forms) + tests/runtime.sx (56/56 pass). COMMIT.
|
||||
|
||||
- [x] Forth: bitwise ops (core); string-buffer for word-definition accumulation; bytevectors
|
||||
for Forth's raw memory model.
|
||||
lib/forth/runtime.sx (36 forms) + tests/runtime.sx (64/64 pass). COMMIT.
|
||||
|
||||
---
|
||||
|
||||
## Ground rules
|
||||
|
||||
- Work on the `architecture` branch in `/root/rose-ash` (main worktree).
|
||||
- Use sx-tree MCP for all `.sx` file edits. Never use raw Edit/Write/Read on `.sx` files.
|
||||
- Commit after each concrete unit of work. Never leave the branch broken.
|
||||
- Never push to `main` — only push to `origin/architecture`.
|
||||
- Update this checklist every fire: tick `[x]` done, add inline notes on blockers.
|
||||
|
||||
---
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- 2026-05-01: Phase 22 Forth done — runtime.sx (36 forms): bitwise (AND/OR/XOR/INVERT/LSHIFT/RSHIFT/2*/2//bit-count/integer-length/within + arithmetic helpers), string-buffer (emit!/type!/value/length/clear!/emit-int!), memory (cfetch/cstore/fetch/store/move!/fill!/erase!/mem->list). 64/64 tests. 8019e572.
|
||||
- 2026-05-01: Phase 22 Tcl done — runtime.sx (37 forms): string-buffer (append accumulator), channel (read/write ports with gets/read/puts), regexp (make-regexp wrappers), format (%s/%d/%f/%x/%o/%% manual char scan), coroutine (call/cc, top-level helper pattern). 56/56 tests. 3e07727d.
|
||||
- 2026-05-01: Phase 22 Ruby done — runtime.sx (61 forms): Hash (list-of-pairs dict-backed), Set (make-set, (set item) order), Regexp (make-regexp wrappers), StringIO (write buf + rewind/char read), Bytevectors (thin wrappers), Fiber (call/cc; letrec JIT workaround: use top-level helpers). 76/76 tests. 182e6f63.
|
||||
|
||||
- 2026-05-01: Phase 22 APL done — runtime.sx (60 forms): iota/rho/at, rank-polymorphic dyadic/monadic helpers, arithmetic/comparison/boolean/bitwise element-wise, reduce/scan, take/drop/rotate/compress/index, set ops (member/nub/union/intersect/without), format. 73/73 tests. COMMIT.
|
||||
- 2026-05-01: Phase 22 Smalltalk done — runtime.sx (72 forms): numeric helpers, Character (1-indexed Array backed by dict), Dictionary (list-of-pairs any-key map), Set (make-set), WriteStream/ReadStream/printString. set-member? (set item) order. 86/86 tests. COMMIT.
|
||||
- 2026-05-01: Phase 22 JS done — stdlib.sx (36 forms): bitwise (truncate not js-num-to-int; set-member? takes (set item) order), Map (dict-backed pairs), Set (SX make-set), RegExp (callable lambda). 25/25 new tests pass; total 492/585. COMMIT.
|
||||
- 2026-05-01: Phase 22 Haskell done — runtime.sx (113 forms): numeric tower (hk-div floor semantics), rational (dict GCD-normalised), hk-force (promises), Data.Char, Data.Set, Data.List, Maybe/Either, tuples, string helpers, hk-show. 148/148 tests. c02ffcf3.
|
||||
- 2026-05-01: Phase 22 Erlang done — runtime.sx (63 forms): numeric tower, bitwise (band/bor/bxor/bnot/bsl/bsr), sets, re module, list BIFs, type conversions, ok/error tuples. 55/55 tests. 3c0a9632.
|
||||
- 2026-05-01: Phase 22 Lua done — math/string/table stdlib tables + lua-force in lib/lua/runtime.sx. 185/185 tests (28 new). ec3512d6.
|
||||
- 2026-05-01: Phase 22 CL done — runtime.sx (103 forms): type preds, arithmetic, chars, format, gensym, values, sets, radix, list utils. cl-empty? guards nil/() split. 68/68 tests. 1ad8e74a.
|
||||
- 2026-05-01: Phase 22 step 1 — SX primitive baseline added to CL/APL/Ruby/Tcl plans. f43659ce.
|
||||
- 2026-05-01: Phase 21 complete — format (~a ~s ~d ~x ~o ~b ~f ~% ~& ~~ ~t) as pure SX in spec/stdlib.sx. Fixed lib/r7rs.sx number->string to support optional radix; added format-decimal OCaml primitive. 28/28 tests on both JS and OCaml. 4d7b3e29.
|
||||
- 2026-04-26: Phase 7 complete — bitwise-and/or/xor/not + arithmetic-shift + bit-count + integer-length. OCaml: land/lor/lxor/lnot/lsl/asr + Kernighan popcount + lsr loop for integer-length. JS: bitwise ops + Hamming weight + Math.clz32. 26 tests, 158 assertions, all pass. a8a79dc9.
|
||||
- 2026-04-26: Phase 6 complete — JS+Tests+Commit all ticked. JS needed no changes (spec-level forms). 40/40 ADT tests pass JS. 2032/2500 JS total (+67 vs phase-4). Phase 6 fully landed: 6c872107+0dc7e159+5d1913e7. Phase 7 (bitwise) next.
|
||||
- 2026-04-26: Phase 6 OCaml done — Dict-based ADT (no native SxAdt type needed); hand-written sf_define_type in bootstrap.py FIXUPS (skipped from transpile — &rest params + empty-dict {} literals); registered via register_special_form; step_limit/step_count added to PREAMBLE. 172 assertions pass (test-adt). Full suite 4280/1080 (was 4243/1117, +37). Committed 5d1913e7.
|
||||
- 2026-04-26: Phase 6 Spec match done — ADT case added to match-pattern in spec/evaluator.sx: checks (list? pattern)+(symbol? first)+(dict? value)+(get value :_adt), then matches :_ctor+arity and recursively binds field patterns. No-clause error now uses make-cek-value+raise-eval-frame so guard can catch it. 20 new match tests pass; 40/40 total ADT tests green. Zero regressions.
|
||||
- 2026-04-26: Phase 6 Spec define-type done — sf-define-type registered via register-special-form! in spec/evaluator.sx; AdtValue as {:_adt true :_type "..." :_ctor "..." :_fields (list ...)}; ctor fns + arity checking + Name?/Ctor? predicates + Ctor-field accessors; *adt-registry* dict populated per define-type call. 20/20 JS tests pass in spec/tests/test-adt.sx. OCaml define-type is next task.
|
||||
- 2026-04-26: Phase 6 Design done — plans/designs/sx-adt.md written. Covers define-type/match syntax, AdtValue CEK runtime, stepSfDefineType+MatchFrame dispatch, exhaustiveness warnings, recursive types, nested patterns, wildcard _. 3-phase impl plan. Next fire: Spec implement define-type.
|
||||
- 2026-04-26: Phase 5 complete — string buffer fully landed (d98b5fa2). 17 tests, 17/17 OCaml+JS. Phase 6 (ADTs) next.
|
||||
- 2026-04-26: Phase 5 Spec+OCaml+JS step done — StringBuffer of Buffer.t in sx_types.ml; make-string-buffer/append!/->string/length/string-buffer? in sx_primitives.ml; SxStringBuffer with _string_buffer marker + typeOf/dict? fixes in platform.py; JS rebuilt. 17/17 tests OCaml+JS.
|
||||
- 2026-04-26: Phase 4 complete — coroutine primitive fully landed (4 commits: spec library + OCaml verified + JS pre-load + 27 tests). Phase 5 (string buffer) next.
|
||||
- 2026-04-26: Phase 4 Tests step done — 27 tests total (10 new: state field inspection, yield-from-helper, initial-arg-ignored, mutable-closure, complex-values, round-robin, factory-no-state, non-coroutine-error). 27/27 OCaml+JS.
|
||||
- 2026-04-26: Phase 4 JS step done — all CEK primitives already in sx-browser.js; fix was pre-loading spec/coroutines.sx+spec/signals.sx in run_tests.js so (import (sx coroutines)) resolves synchronously. 17/17 coroutine tests pass JS. 1965/2500 total (+25), zero new failures.
|
||||
- 2026-04-26: Phase 4 OCaml step done — no native SxCoroutine type needed; existing cek-step-loop/cek-resume/perform/make-cek-state primitives in run_tests.ml fully support the spec/coroutines.sx library. 284/284 pass (coroutines+vectors+numeric-tower+dynamic-wind), zero regressions.
|
||||
- 2026-04-26: Phase 4 Spec step done — spec/coroutines.sx define-library with make-coroutine/coroutine-resume/coroutine-yield/coroutine?/coroutine-alive?; make-coroutine stub in evaluator.sx; 17/17 coroutine tests pass (OCaml). Key insight: coroutine body must use (define loop (fn...)) + (loop 0) not named let — named let uses cek_call→cek_run which errors on IO suspension.
|
||||
- 2026-05-01: Phase 10 complete — mutable hash tables. HashTable variant in OCaml; JS Map-based SxHashTable. 11 primitives: make-hash-table/hash-table?/set!/ref/delete!/size/keys/values/->alist/for-each/merge!. 28 tests, all pass OCaml+JS. 133bdf52.
|
||||
- 2026-05-01: Phase 9 complete — delay/force/delay-force/make-promise/promise?. Dict-based promise {:_promise :forced :thunk :value}; :_iterative flag for delay-force chain following. 25/25 tests OCaml (4357) and JS (2109). Committed e44cb89a.
|
||||
- 2026-05-01: Phase 8 complete — values/call-with-values/let-values/define-values. Dict marker {:_values true :_list [...]} (no new type). step-sf-define desugars shorthand (define (f x) body) on both hosts. 25/25 tests OCaml+JS. Committed 43cc1d90.
|
||||
- 2026-04-26: Phase 3 complete — OCaml+JS done. CallccContinuation gains winders-depth int; make_callcc_continuation/callcc_continuation_winders_len wired; wind-after/wind-return CekFrame fields fixed (cf_f=after-thunk, cf_extra=winders-len, cf_name=body-result); get_val + transpiler.sx updated. 8/8 dynamic-wind tests pass on OCaml; 235/235 (callcc+guard+do+r7rs) zero regressions. Committed 6602ec8c.
|
||||
- 2026-04-26: Phase 3 Spec+Tests done — dynamic-wind CEK implementation: wind-after/wind-return frames, *winders* stack, kont-unwind-to-handler, wind-escape-to. callcc frame stores winders-len in continuation; callcc-continuation? calls wind-escape-to before escape. 8/8 dynamic-wind tests pass (normal return, raise, call/cc, nested LIFO, guard ordering). 1948/2500 JS (+8). Zero regressions. Committed a9d5a108.
|
||||
- 2026-04-26: Phase 2 complete — Verify+Commit done. OCaml 4874/394, JS 1940/2500 (+60). No regressions. 6 JS-only failures are float≡int platform-inherent. Phase 2 fully landed across 4 commits.
|
||||
- 2026-04-26: Phase 2 JS bootstrapper done — integer?/float?/exact?/inexact? added (Number.isInteger); truncate/remainder/modulo/random-int/exact->inexact/inexact->exact/parse-number added. Fixed sx_server.ml epoch+blob+io-response protocol for Integer type. JS: 1940/2500 (+60). OCaml: 4874/394 baseline. 6 JS tests fail (JS float≡int platform limit). Committed b12a22e6.
|
||||
- 2026-04-26: Phase 2 Spec done — integer?/float? predicates added to spec/primitives.sx; floor/ceil/truncate :returns updated to "integer"; / to "float"; exact->inexact/inexact->exact docs and returns updated; float contagion documented on +/-/*; 4874/394 baseline. Committed 45ec5535.
|
||||
- 2026-04-26: Phase 2 OCaml+Tests done — `Integer of int` / `Number of float` in sx_types.ml; float contagion across all arithmetic; floor/truncate/round → Integer; integer?/float?/exact?/inexact?/exact->inexact/inexact->exact; 92/92 numeric tower tests pass; 4874 total (394 pre-existing unchanged). Committed c70bbdeb.
|
||||
- 2026-04-26: Phase 1 complete — JS step done. Fixed fundamental lambda binding bug (index-of on arrays returned -1 not NIL, making bind-lambda-params mis-fire &rest branch). Added _lastErrorKont_/hostError/try-catch stubs. 42/42 vector tests pass. 1847 std / 2362 full passing (up from 5). Committed.
|
||||
- 2026-04-25: Phase 1 spec step done — all 10 vector primitives in spec/primitives.sx have full :as type annotations, :returns, :doc; make-vector optional fill param added.
|
||||
- 2026-04-25: Phase 1 OCaml step done — bounds-checked vector-ref/set!, vector-copy now accepts optional start/end, spec/primitives.sx doc updated. 10/10 r7rs vector tests pass, 4747 total (394 pre-existing hs-upstream fails unchanged).
|
||||
- 2026-04-25: Phase 0 complete — stopped CL/APL/Ruby/Tcl loops (all 4 idle at shell); confirmed E38 (tokenizer :end/:line) and E39 (WebWorker stub) both have implementation commits.
|
||||
- 2026-05-01: Phase 20 complete — bytevectors. SxBytevector of bytes in OCaml using Bytes; Uint8Array-backed SxBytevector in JS. 12 primitives: make-bytevector, bytevector?, bytevector-length, bytevector-u8-ref, bytevector-u8-set!, bytevector-copy, bytevector-copy!, bytevector-append, utf8->string, string->utf8, bytevector->list, list->bytevector. 32 tests, all pass. JS 2535, OCaml 4725. a3811545.
|
||||
- 2026-05-01: Phase 19 complete — regular expressions. SxRegexp(src,flags,Re.re) in OCaml via Re.Pcre; SxRegexp wrapper around JS RegExp. 9 primitives: make-regexp, regexp?, regexp-source, regexp-flags, regexp-match, regexp-match-all, regexp-replace, regexp-replace-all, regexp-split. Match dicts with :match/:start/:end/:groups. 32 tests, all pass. JS 2503, OCaml 4693. d8d5588e.
|
||||
- 2026-05-01: Phase 18 complete — sets. SxSet as (string,value) Hashtbl keyed by inspect(val) in OCaml; Map keyed by write-to-string in JS. 13 primitives: make-set, set?, set-add!, set-member?, set-remove!, set-size, set->list, list->set, set-union, set-intersection, set-difference, set-for-each, set-map. 33 tests, all pass. JS 2469, OCaml 4659. 3b0ac67a.
|
||||
- 2026-05-01: Phase 17 complete — read/write/display. OCaml: sx_write_val/sx_display_val helpers; read via Sx_parser.read_value with #t/#f and N/D rational support added to parser; postprocess ()→Nil. JS: sxReadNormalize (#t/#f→true/false), sxReadConvert (()→NIL), sxEq list equality, sxWriteVal symbol/keyword name fix (v.name not v._sym), readerMacroGet registry. 42 tests (test-read-write.sx), all pass both hosts. JS 2436, OCaml 4626. 7d329f02.
|
||||
- 2026-05-01: Phase 16 complete — rational numbers. SxRational type in OCaml (Rational of int*int, reduced, denom>0) and JS (SxRational class, _rational marker). n/d reader in spec/parser.sx. Arithmetic contagion: int op rational → rational, rational op float → float. JS keeps int/int → float for CSS compat. OCaml as_number+safe_eq extended for cross-type rational equality. 62 tests in test-rationals.sx, all pass. JS 2232, OCaml 4532 (+11). 036022cc.
|
||||
- 2026-05-01: Phase 15 complete — math completeness. stdlib.math module: sin/cos/tan/asin/acos/atan(1-2 args)/exp/log/expt/quotient/gcd/lcm/number->string(radix)/string->number(radix). OCaml atan updated for optional 2nd arg. Strict radix parsing in JS string->number. 44 tests in test-math.sx, all pass. JS 2311/4801, OCaml 4547/5629. be2b11ac.
|
||||
- 2026-05-01: Phase 14 OCaml done — Eof + Port{PortInput/PortOutput} in sx_types.ml; 15 port primitives in sx_primitives.ml; raw_serialize updated; 4532/4532 (+39, zero regressions). 8ba0a33f.
|
||||
- 2026-05-01: Phase 14 Spec+JS+Tests+Commit done — port type {_port,_kind,_source/_buffer,_pos,_closed}; eof singleton; 15 primitives in spec/primitives.sx (stdlib.ports) + platform.py; 39/39 tests in test-ports.sx. Committed 3d8937d7. OCaml step next.
|
||||
- 2026-05-01: Phase 13 OCaml done — Char of int in sx_types.ml; #\ reader in sx_parser.ml; all char primitives in sx_primitives.ml; fixed get_val for Integer n list indexing (was Number-only); fixed raw_serialize for Integer/Char. 4493/4493 (+43, zero regressions). b939becd.
|
||||
- 2026-05-01: Phase 13 Spec+JS+Tests+Commit done — SxChar tagged {_char,codepoint}; char? char->integer integer->char char-upcase/downcase; 10 comparators (ordered+ci); 5 predicates; string->list/list->string as platform primitives; #\a #\space #\newline reader syntax in spec/parser.sx; js-char-renames dict in transpiler.sx; 43/43 tests pass JS (2254/4745). Committed 4b600f17. OCaml step next.
|
||||
- 2026-05-01: Phase 12 complete — gensym + symbol interning. gensym_counter/gensym/string->symbol/symbol->string/intern/symbol-interned? in spec + OCaml + JS. Fixed ListRef case in seq_to_list (both hosts). 19 tests, all pass. OCaml 4450/1080, JS 2205/2497. Commits: edf4e525 Spec, 0862a614 OCaml+Tests.
|
||||
- 2026-05-01: Phase 11 complete — sequence protocol done. Commits: da4b526a Spec, 7286629c OCaml, 06a3eee1 JS, 0fe00bf7 Tests. JS 2185/+48, OCaml 4424/+39.
|
||||
- 2026-05-01: Phase 11 Tests done — 45 tests in test-sequences.sx all passing (JS 2185/+48, OCaml 4424/+39). Fixed vector? rename, vectorLength/vectorRef/reverse aliases, in-range letrec→build-range, sequence-length nil, assert-equal for lists. Committed 0fe00bf7.
|
||||
- 2026-05-01: Phase 11 JS bootstrapper step done — confirmed sx-browser.js current (built in Spec step da4b526a); 19 sequence primitive refs in output; 2137/2500 JS tests passing.
|
||||
- 2026-05-01: Phase 11 OCaml step done — seq_to_list helper added before let-rec; ho_setup_dispatch wraps all 7 coll bindings with seq_to_list; seq-to-list/sequence-to-list/to-vector/length/ref/append + in-range primitives in sx_primitives.ml. 4385/4385 baseline unchanged, 0 regressions. Committed 7286629c.
|
||||
- 2026-05-01: Phase 11 Spec step done — seq-to-list coercion helper; ho-setup-dispatch extended with seqToList on all collection args; sequence-to-list/vector/length/ref/append + in-range added to evaluator.sx. Restored 3 accidentally-deleted make-cek-state/value/suspended definitions. Fixed 8 shorthand define forms + added vector->list/list->vector transpiler renames. JS: 2137 passing (+28 vs HEAD baseline of 2109).
|
||||
83
plans/agent-briefings/ruby-loop.md
Normal file
83
plans/agent-briefings/ruby-loop.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# ruby-on-sx loop agent (single agent, queue-driven)
|
||||
|
||||
Role: iterates `plans/ruby-on-sx.md` forever. Fibers via delcc is the headline showcase — `Fiber.new`/`Fiber.yield`/`Fiber.resume` are textbook delimited continuations with sugar, where MRI does it via C-stack swapping. Plus blocks/yield (lexical escape continuations, same shape as Smalltalk's non-local return), method_missing, and singleton classes.
|
||||
|
||||
```
|
||||
description: ruby-on-sx queue loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/ruby-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
1. Read `plans/ruby-on-sx.md` — roadmap + Progress log.
|
||||
2. `ls lib/ruby/` — pick up from the most advanced file.
|
||||
3. If `lib/ruby/tests/*.sx` exist, run them. Green before new work.
|
||||
4. If `lib/ruby/scoreboard.md` exists, that's your baseline.
|
||||
|
||||
## The queue
|
||||
|
||||
Phase order per `plans/ruby-on-sx.md`:
|
||||
|
||||
- **Phase 1** — tokenizer + parser. Keywords, identifier sigils (`@` ivar, `@@` cvar, `$` global), strings with interpolation, `%w[]`/`%i[]`, symbols, blocks `{|x| …}` and `do |x| … end`, splats, default args, method def
|
||||
- **Phase 2** — object model + sequential eval. Class table, ancestor-chain dispatch, `super`, singleton classes, `method_missing` fallback, dynamic constant lookup
|
||||
- **Phase 3** — blocks + procs + lambdas. Method captures escape continuation `^k`; `yield` / `return` / `break` / `next` / `redo` semantics; lambda strict arity vs proc lax
|
||||
- **Phase 4** — **THE SHOWCASE**: fibers via delcc. `Fiber.new`/`Fiber.resume`/`Fiber.yield`/`Fiber.transfer`. Classic programs (generator, producer-consumer, tree-walk) green
|
||||
- **Phase 5** — modules + mixins + metaprogramming. `include`/`prepend`/`extend`, `define_method`, `class_eval`/`instance_eval`, `respond_to?`/`respond_to_missing?`, hooks
|
||||
- **Phase 6** — stdlib drive. `Enumerable` mixin, `Comparable`, Array/Hash/Range/String/Integer methods, drive corpus to 200+
|
||||
|
||||
Within a phase, pick the checkbox that unlocks the most tests per effort.
|
||||
|
||||
Every iteration: implement → test → commit → tick `[ ]` → Progress log → next.
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `lib/ruby/**` and `plans/ruby-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. Ruby primitives go in `lib/ruby/runtime.sx`.
|
||||
- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop.
|
||||
- **Shared-file issues** → plan's Blockers with minimal repro.
|
||||
- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
|
||||
## Ruby-specific gotchas
|
||||
|
||||
- **Block `return` vs lambda `return`.** Inside a block `{ ... return v }`, `return` invokes the *enclosing method's* escape continuation (non-local return). Inside a lambda `->(){ ... return v }`, `return` returns from the *lambda*. Don't conflate. Implement: blocks bind their `^method-k`; lambdas bind their own `^lambda-k`.
|
||||
- **`break` from inside a block** invokes a different escape — the *iteration loop's* escape — and the loop returns the break-value. `next` is escape from current iteration, returns iteration value. `redo` re-enters current iteration without advancing.
|
||||
- **Proc arity is lax.** `proc { |a, b, c| … }.call(1, 2)` ↦ `c = nil`. Lambda is strict — same call raises ArgumentError. Check arity at call site for lambdas only.
|
||||
- **Block argument unpacking.** `[[1,2],[3,4]].each { |a, b| … }` — single Array arg auto-unpacks for blocks (not lambdas). One arg, one Array → unpack. Frequent footgun.
|
||||
- **Method dispatch chain order:** prepended modules → class methods → included modules → superclass → BasicObject → method_missing. `super` walks from the *defining* class's position, not the receiver class's.
|
||||
- **Singleton classes** are lazily allocated. Looking up the chain for an object passes through its singleton class first, then its actual class. `class << obj; …; end` opens the singleton.
|
||||
- **`method_missing`** — fallback when ancestor walk misses. Receives `(name_symbol, *args, &blk)`. Pair with `respond_to_missing?` for `respond_to?` to also report true. Do **not** swallow NoMethodError silently.
|
||||
- **Ivars are per-object dicts.** Reading an unset ivar yields `nil` and a warning (`-W`). Don't error.
|
||||
- **Constant lookup** is first lexical (Module.nesting), then inheritance (Module.ancestors of the innermost class). Different from method lookup.
|
||||
- **`Object#send`** invokes private and public methods alike; `Object#public_send` skips privates.
|
||||
- **Class reopening.** `class Foo; def bar; …; end; end` plus a later `class Foo; def baz; …; end; end` adds methods to the same class. Class table lookups must be by-name, mutable; methods dict is mutable.
|
||||
- **Fiber semantics.** `Fiber.new { |arg| … }` creates a fiber suspended at entry. First `Fiber.resume(v)` enters with `arg = v`. Inside, `Fiber.yield(w)` returns `w` to the resumer; the next `Fiber.resume(v')` returns `v'` to the yield site. End of block returns final value to last resumer; subsequent `Fiber.resume` raises FiberError.
|
||||
- **`Fiber.transfer`** is symmetric — either side can transfer to the other; no resume/yield asymmetry. Implement on top of the same continuation pair, just don't enforce direction.
|
||||
- **Symbols are interned.** `:foo == :foo` is identity. Use SX symbols.
|
||||
- **Strings are mutable.** `s = "abc"; s << "d"; s == "abcd"`. Hash keys can be strings; hash dups string keys at insertion to be safe (or freeze them).
|
||||
- **Truthiness:** only `false` and `nil` are falsy. `0`, `""`, `[]` are truthy.
|
||||
- **Test corpus:** custom + curated RubySpec slice. Place programs in `lib/ruby/tests/programs/` with `.rb` extension.
|
||||
|
||||
## General gotchas (all loops)
|
||||
|
||||
- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences.
|
||||
- `cond`/`when`/`let` clauses evaluate only the last expr.
|
||||
- `type-of` on user fn returns `"lambda"`.
|
||||
- Shell heredoc `||` gets eaten — escape or use `case`.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.sx` unless non-obvious.
|
||||
- No new planning docs — update `plans/ruby-on-sx.md` inline.
|
||||
- Short, factual commit messages (`ruby: Fiber.yield + Fiber.resume (+8)`).
|
||||
- One feature per iteration. Commit. Log. Next.
|
||||
|
||||
Go. Read the plan; find first `[ ]`; implement.
|
||||
77
plans/agent-briefings/smalltalk-loop.md
Normal file
77
plans/agent-briefings/smalltalk-loop.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# smalltalk-on-sx loop agent (single agent, queue-driven)
|
||||
|
||||
Role: iterates `plans/smalltalk-on-sx.md` forever. Message-passing OO + **blocks with non-local return** on delimited continuations. Non-local return is the headline showcase — every other Smalltalk reinvents it on the host stack; on SX it falls out of the captured method-return continuation.
|
||||
|
||||
```
|
||||
description: smalltalk-on-sx queue loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/smalltalk-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
1. Read `plans/smalltalk-on-sx.md` — roadmap + Progress log.
|
||||
2. `ls lib/smalltalk/` — pick up from the most advanced file.
|
||||
3. If `lib/smalltalk/tests/*.sx` exist, run them. Green before new work.
|
||||
4. If `lib/smalltalk/scoreboard.md` exists, that's your baseline.
|
||||
|
||||
## The queue
|
||||
|
||||
Phase order per `plans/smalltalk-on-sx.md`:
|
||||
|
||||
- **Phase 1** — tokenizer + parser (chunk format, identifiers, keywords `foo:`, binary selectors, `#sym`, `#(…)`, `$c`, blocks `[:a | …]`, cascades, message precedence)
|
||||
- **Phase 2** — object model + sequential eval (class table bootstrap, message dispatch, `super`, `doesNotUnderstand:`, instance variables)
|
||||
- **Phase 3** — **THE SHOWCASE**: blocks with non-local return via captured method-return continuation. `whileTrue:` / `ifTrue:ifFalse:` as block sends. 5 classic programs (eight-queens, quicksort, mandelbrot, life, fibonacci) green.
|
||||
- **Phase 4** — reflection + MOP: `perform:`, `respondsTo:`, runtime method addition, `becomeForward:`, `Exception` / `on:do:` / `ensure:` on top of `handler-bind`/`raise`
|
||||
- **Phase 5** — collections + numeric tower + streams
|
||||
- **Phase 6** — port SUnit, vendor Pharo Kernel-Tests slice, drive corpus to 200+
|
||||
- **Phase 7** — speed (optional): inline caching, block intrinsification
|
||||
|
||||
Within a phase, pick the checkbox that unlocks the most tests per effort.
|
||||
|
||||
Every iteration: implement → test → commit → tick `[ ]` → Progress log → next.
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `lib/smalltalk/**` and `plans/smalltalk-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. Smalltalk primitives go in `lib/smalltalk/runtime.sx`.
|
||||
- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop.
|
||||
- **Shared-file issues** → plan's Blockers with minimal repro.
|
||||
- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
|
||||
## Smalltalk-specific gotchas
|
||||
|
||||
- **Method invocation captures `^k`** — the return continuation. Bind it as the block's escape token. `^expr` from inside any nested block invokes that captured `^k`. Escape past method return raises `BlockContext>>cannotReturn:`.
|
||||
- **Blocks are lambdas + escape token**, not bare lambdas. `value`/`value:`/… invoke the lambda; `^` invokes the escape.
|
||||
- **`ifTrue:` / `ifFalse:` / `whileTrue:` are ordinary block sends** — no special form. The runtime intrinsifies them in the JIT path (Tier 1 of bytecode expansion already covers this pattern).
|
||||
- **Cascade** `r m1; m2; m3` desugars to `(let ((tmp r)) (st-send tmp 'm1 ()) (st-send tmp 'm2 ()) (st-send tmp 'm3 ()))`. Result is the cascade's last send (or first, depending on parser variant — pick one and document).
|
||||
- **`super` send** looks up starting from the *defining* class's superclass, not the receiver class. Stash the defining class on the method record.
|
||||
- **Selectors are interned symbols.** Use SX symbols.
|
||||
- **Receiver dispatch:** tagged ints / floats / strings / symbols / `nil` / `true` / `false` aren't boxed. Their classes (`SmallInteger`, `Float`, `String`, `Symbol`, `UndefinedObject`, `True`, `False`) are looked up by SX type-of, not by an `:class` field.
|
||||
- **Method precedence:** unary > binary > keyword. `3 + 4 factorial` is `3 + (4 factorial)`. `a foo: b bar` is `a foo: (b bar)` (keyword absorbs trailing unary).
|
||||
- **Image / fileIn / become: between sessions** = out of scope. One-way `becomeForward:` only.
|
||||
- **Test corpus:** ~200 hand-written + a slice of Pharo Kernel-Tests. Place programs in `lib/smalltalk/tests/programs/`.
|
||||
|
||||
## General gotchas (all loops)
|
||||
|
||||
- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences.
|
||||
- `cond`/`when`/`let` clauses evaluate only the last expr.
|
||||
- `type-of` on user fn returns `"lambda"`.
|
||||
- Shell heredoc `||` gets eaten — escape or use `case`.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.sx` unless non-obvious.
|
||||
- No new planning docs — update `plans/smalltalk-on-sx.md` inline.
|
||||
- Short, factual commit messages (`smalltalk: tokenizer + 56 tests`).
|
||||
- One feature per iteration. Commit. Log. Next.
|
||||
|
||||
Go. Read the plan; find first `[ ]`; implement.
|
||||
83
plans/agent-briefings/tcl-loop.md
Normal file
83
plans/agent-briefings/tcl-loop.md
Normal file
@@ -0,0 +1,83 @@
|
||||
# tcl-on-sx loop agent (single agent, queue-driven)
|
||||
|
||||
Role: iterates `plans/tcl-on-sx.md` forever. `uplevel`/`upvar` is the headline showcase — Tcl's superpower for defining your own control structures, requiring deep VM cooperation in any normal host but falling out of SX's first-class env-chain. Plus the Dodekalogue (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity.
|
||||
|
||||
```
|
||||
description: tcl-on-sx queue loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
isolation: worktree
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/tcl-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
1. Read `plans/tcl-on-sx.md` — roadmap + Progress log.
|
||||
2. `ls lib/tcl/` — pick up from the most advanced file.
|
||||
3. If `lib/tcl/tests/*.sx` exist, run them. Green before new work.
|
||||
4. If `lib/tcl/scoreboard.md` exists, that's your baseline.
|
||||
|
||||
## The queue
|
||||
|
||||
Phase order per `plans/tcl-on-sx.md`:
|
||||
|
||||
- **Phase 1** — tokenizer + parser. The Dodekalogue (12 rules): word-splitting, command sub `[…]`, var sub `$name`/`${name}`/`$arr(idx)`, double-quote vs brace word, backslash, `;`, `#` comments only at command start, single-pass left-to-right substitution
|
||||
- **Phase 2** — sequential eval + core commands. `set`/`unset`/`incr`/`append`/`lappend`, `puts`/`gets`, `expr` (own mini-language), `if`/`while`/`for`/`foreach`/`switch`, string commands, list commands, dict commands
|
||||
- **Phase 3** — **THE SHOWCASE**: `proc` + `uplevel` + `upvar`. Frame stack with proc-call push/pop; `uplevel #N script` evaluates in caller's frame; `upvar` aliases names across frames. Classic programs (for-each-line, assert macro, with-temp-var) green
|
||||
- **Phase 4** — `return -code N`, `catch`, `try`/`trap`/`finally`, `throw`. Control flow as integer codes
|
||||
- **Phase 5** — namespaces + ensembles. `namespace eval`, qualified names `::ns::cmd`, ensembles, `namespace path`
|
||||
- **Phase 6** — coroutines (built on fibers, same delcc as Ruby fibers) + system commands + drive corpus to 150+
|
||||
|
||||
Within a phase, pick the checkbox that unlocks the most tests per effort.
|
||||
|
||||
Every iteration: implement → test → commit → tick `[ ]` → Progress log → next.
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `lib/tcl/**` and `plans/tcl-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. Tcl primitives go in `lib/tcl/runtime.sx`.
|
||||
- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop.
|
||||
- **Shared-file issues** → plan's Blockers with minimal repro.
|
||||
- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
|
||||
## Tcl-specific gotchas
|
||||
|
||||
- **Everything is a string.** Internally cache shimmer reps (list, dict, int, double) for performance, but every value must be re-stringifiable. Mutating one rep dirties the cached string and vice versa.
|
||||
- **The Dodekalogue is strict.** Substitution is **one-pass**, **left-to-right**. The result of a substitution is a value, not a script — it does NOT get re-parsed for further substitutions. This is what makes Tcl safe-by-default. Don't accidentally re-parse.
|
||||
- **Brace word `{…}`** is the only way to defer evaluation. No substitution inside, just balanced braces. Used for `if {expr}` body, `proc body`, `expr` arguments.
|
||||
- **Double-quote word `"…"`** is identical to a bare word for substitution purposes — it just allows whitespace in a single word. `\` escapes still apply.
|
||||
- **Comments are only at command position.** `# this is a comment` after a `;` or newline; *not* inside a command. `set x 1 # not a comment` is a 4-arg `set`.
|
||||
- **`expr` has its own grammar** — operator precedence, function calls — and does its own substitution. Brace `expr {$x + 1}` to avoid double-substitution and to enable bytecode caching.
|
||||
- **`if` and `while` re-parse** the condition only if not braced. Always use `if {…}`/`while {…}` form. The unbraced form re-substitutes per iteration.
|
||||
- **`return` from a `proc`** uses control code 2. `break` is 3, `continue` is 4. `error` is 1. `catch` traps any non-zero code; user can return non-zero with `return -code error -errorcode FOO message`.
|
||||
- **`uplevel #0 script`** is global frame. `uplevel 1 script` (or just `uplevel script`) is caller's frame. `uplevel #N` is absolute level N (0=global, 1=top-level proc, 2=proc-called-from-top, …). Negative levels are errors.
|
||||
- **`upvar #N otherVar localVar`** binds `localVar` in the current frame as an *alias* — both names refer to the same storage. Reads and writes go through the alias.
|
||||
- **`info level`** with no arg returns current level number. `info level N` (positive) returns the command list that invoked level N. `info level -N` returns the command list of the level N relative-up.
|
||||
- **Variable names with `(…)`** are array elements: `set arr(foo) 1`. Arrays are not first-class values — you can't `set x $arr`. `array get arr` gives a flat list `{key1 val1 key2 val2 …}`.
|
||||
- **List vs string.** `set l "a b c"` and `set l [list a b c]` look the same when printed but the second has a cached list rep. `lindex` works on both via shimmering. Most user code can't tell the difference.
|
||||
- **`incr x`** errors if x doesn't exist; pre-set with `set x 0` or use `incr x 0` first if you mean "create-or-increment". Or use `dict incr` for dicts.
|
||||
- **Coroutines are fibers.** `coroutine name body` starts a coroutine; calling `name` resumes it; `yield value` from inside suspends and returns `value` to the resumer. Same primitive as Ruby fibers — share the implementation under the hood.
|
||||
- **`switch`** matches first clause whose pattern matches. Default is `default`. Variant matches: glob (default), `-exact`, `-glob`, `-regexp`. Body `-` means "fall through to next clause's body".
|
||||
- **Test corpus:** custom + slice of Tcl's own tests. Place programs in `lib/tcl/tests/programs/` with `.tcl` extension.
|
||||
|
||||
## General gotchas (all loops)
|
||||
|
||||
- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences.
|
||||
- `cond`/`when`/`let` clauses evaluate only the last expr.
|
||||
- `type-of` on user fn returns `"lambda"`.
|
||||
- Shell heredoc `||` gets eaten — escape or use `case`.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.sx` unless non-obvious.
|
||||
- No new planning docs — update `plans/tcl-on-sx.md` inline.
|
||||
- Short, factual commit messages (`tcl: uplevel + upvar (+11)`).
|
||||
- One feature per iteration. Commit. Log. Next.
|
||||
|
||||
Go. Read the plan; find first `[ ]`; implement.
|
||||
125
plans/apl-on-sx.md
Normal file
125
plans/apl-on-sx.md
Normal file
@@ -0,0 +1,125 @@
|
||||
# APL-on-SX: rank-polymorphic primitives + glyph parser
|
||||
|
||||
The headline showcase is **rank polymorphism** — a single primitive (`+`, `⌈`, `⊂`, `⍳`) works uniformly on scalars, vectors, matrices, and higher-rank arrays. ~80 glyph primitives + 6 operators bind together with right-to-left evaluation; the entire language is a high-density combinator algebra. The JIT compiler + primitive table pay off massively here because almost every program is `array → array` pure pipelines.
|
||||
|
||||
End-state goal: Dyalog-flavoured APL subset, dfns + tradfns, classic programs (game-of-life, mandelbrot, prime-sieve, n-queens, conway), 100+ green tests.
|
||||
|
||||
## Scope decisions (defaults — override by editing before we spawn)
|
||||
|
||||
- **Syntax:** Dyalog APL surface, Unicode glyphs. `⎕`-quad system functions for I/O. `∇` tradfn header.
|
||||
- **Conformance:** "Reads like APL, runs like APL." Not byte-compat with Dyalog; we care about right-to-left semantics and rank polymorphism.
|
||||
- **Test corpus:** custom — APL idioms (Roger Hui style), classic programs, plus ~50 pattern tests for primitives.
|
||||
- **Out of scope:** ⎕-namespaces beyond a handful, complex numbers, full TAO ordering, `⎕FX` runtime function definition (use static `∇` only), nested-array-of-functions higher orders, the editor.
|
||||
- **Glyphs:** input via plain Unicode in `.apl` source files. Backtick-prefix shortcuts handled by the user's editor — we don't ship one.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/apl/**` and `plans/apl-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. APL primitives go in `lib/apl/runtime.sx`.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
APL source (Unicode glyphs)
|
||||
│
|
||||
▼
|
||||
lib/apl/tokenizer.sx — glyphs, identifiers, numbers (¯ for negative), strings, strands
|
||||
│
|
||||
▼
|
||||
lib/apl/parser.sx — right-to-left with valence resolution (mon vs dyadic by position)
|
||||
│
|
||||
▼
|
||||
lib/apl/transpile.sx — AST → SX AST (entry: apl-eval-ast)
|
||||
│
|
||||
▼
|
||||
lib/apl/runtime.sx — array model, ~80 primitives, 6 operators, dfns/tradfns
|
||||
```
|
||||
|
||||
Core mapping:
|
||||
- **Array** = SX dict `{:shape (d1 d2 …) :ravel #(v1 v2 …)}`. Scalar is rank-0 (empty shape), vector is rank-1, matrix rank-2, etc. Type uniformity not required (heterogeneous nested arrays via "boxed" elements `⊂x`).
|
||||
- **Rank polymorphism** — every scalar primitive is broadcast: `1 2 3 + 4 5 6` ↦ `5 7 9`; `(2 3⍴⍳6) + 1` ↦ broadcast scalar to matrix.
|
||||
- **Conformability** = matching shapes, or one-side scalar, or rank-1 cycling (deferred — keep strict in v1).
|
||||
- **Valence** = each glyph has a monadic and a dyadic meaning; resolution is purely positional (left-arg present → dyadic).
|
||||
- **Operator** = takes one or two function operands, returns a derived function (`f¨` = `each f`, `f/` = `reduce f`, `f∘g` = `compose`, `f⍨` = `commute`).
|
||||
- **Tradfn** `∇R←L F R; locals` = named function with explicit header.
|
||||
- **Dfn** `{⍺+⍵}` = anonymous, `⍺` = left arg, `⍵` = right arg, `∇` = recurse.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: Unicode glyphs (the full APL set: `+ - × ÷ * ⍟ ⌈ ⌊ | ! ? ○ ~ < ≤ = ≥ > ≠ ∊ ∧ ∨ ⍱ ⍲ , ⍪ ⍴ ⌽ ⊖ ⍉ ↑ ↓ ⊂ ⊃ ⊆ ∪ ∩ ⍳ ⍸ ⌷ ⍋ ⍒ ⊥ ⊤ ⊣ ⊢ ⍎ ⍕ ⍝`), operators (`/ \ ¨ ⍨ ∘ . ⍣ ⍤ ⍥ @`), numbers (`¯` for negative, `1E2`, `1J2` complex deferred), characters (`'a'`, `''` escape), strands (juxtaposition of literals: `1 2 3`), names, comments `⍝ …`
|
||||
- [ ] Parser: right-to-left; classify each token as function, operator, value, or name; resolve valence positionally; dfn `{…}` body, tradfn `∇` header, guards `:`, control words `:If :While :For …` (Dyalog-style)
|
||||
- [ ] Unit tests in `lib/apl/tests/parse.sx`
|
||||
|
||||
### Phase 2 — array model + scalar primitives
|
||||
- [ ] Array constructor: `make-array shape ravel`, `scalar v`, `vector v…`, `enclose`/`disclose`
|
||||
- [ ] Shape arithmetic: `⍴` (shape), `,` (ravel), `≢` (tally / first-axis-length), `≡` (depth)
|
||||
- [ ] Scalar arithmetic primitives broadcast: `+ - × ÷ ⌈ ⌊ * ⍟ | ! ○`
|
||||
- [ ] Scalar comparison primitives: `< ≤ = ≥ > ≠`
|
||||
- [ ] Scalar logical: `~ ∧ ∨ ⍱ ⍲`
|
||||
- [ ] Index generator: `⍳n` (vector 1..n or 0..n-1 depending on `⎕IO`)
|
||||
- [ ] `⎕IO` = 1 default (Dyalog convention)
|
||||
- [ ] 40+ tests in `lib/apl/tests/scalar.sx`
|
||||
|
||||
### Phase 3 — structural primitives + indexing
|
||||
- [ ] Reshape `⍴`, ravel `,`, transpose `⍉` (full + dyadic axis spec)
|
||||
- [ ] Take `↑`, drop `↓`, rotate `⌽` (last axis), `⊖` (first axis)
|
||||
- [ ] Catenate `,` (last axis) and `⍪` (first axis)
|
||||
- [ ] Index `⌷` (squad), bracket-indexing `A[I]` (sugar for `⌷`)
|
||||
- [ ] Grade-up `⍋`, grade-down `⍒`
|
||||
- [ ] Enclose `⊂`, disclose `⊃`, partition (subset deferred)
|
||||
- [ ] Membership `∊`, find `⍳` (dyadic), without `~` (dyadic), unique `∪` (deferred to phase 6)
|
||||
- [ ] 40+ tests in `lib/apl/tests/structural.sx`
|
||||
|
||||
### Phase 4 — operators (THE SHOWCASE)
|
||||
- [ ] Reduce `f/` (last axis), `f⌿` (first axis) — including `∧/`, `∨/`, `+/`, `×/`, `⌈/`, `⌊/`
|
||||
- [ ] Scan `f\`, `f⍀`
|
||||
- [ ] Each `f¨` — applies `f` to each scalar/element
|
||||
- [ ] Outer product `∘.f` — `1 2 3 ∘.× 1 2 3` ↦ multiplication table
|
||||
- [ ] Inner product `f.g` — `+.×` is matrix multiply
|
||||
- [ ] Commute `f⍨` — `f⍨ x` ↔ `x f x`, `x f⍨ y` ↔ `y f x`
|
||||
- [ ] Compose `f∘g` — applies `g` first then `f`
|
||||
- [ ] Power `f⍣n` — apply f n times; `f⍣≡` until fixed point
|
||||
- [ ] Rank `f⍤k` — apply f at sub-rank k
|
||||
- [ ] At `@` — selective replace
|
||||
- [ ] 40+ tests in `lib/apl/tests/operators.sx`
|
||||
|
||||
### Phase 5 — dfns + tradfns + control flow
|
||||
- [ ] Dfn `{…}` with `⍺` (left arg, may be absent → niladic/monadic), `⍵` (right arg), `∇` (recurse), guards `cond:expr`, default left arg `⍺←default`
|
||||
- [ ] Local assignment via `←` (lexical inside dfn)
|
||||
- [ ] Tradfn `∇` header: `R←L F R;l1;l2`, statement-by-statement, branch via `→linenum`
|
||||
- [ ] Dyalog control words: `:If/:Else/:EndIf`, `:While/:EndWhile`, `:For X :In V :EndFor`, `:Select/:Case/:EndSelect`, `:Trap`/`:EndTrap`
|
||||
- [ ] Niladic / monadic / dyadic dispatch (function valence at definition time)
|
||||
- [ ] `lib/apl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
|
||||
### Phase 6 — classic programs + drive corpus
|
||||
- [ ] Classic programs in `lib/apl/tests/programs/`:
|
||||
- [ ] `life.apl` — Conway's Game of Life as a one-liner using `⊂` `⊖` `⌽` `+/`
|
||||
- [ ] `mandelbrot.apl` — complex iteration with rank-polymorphic `+ × ⌊` (or real-axis subset)
|
||||
- [ ] `primes.apl` — `(2=+⌿0=A∘.|A)/A←⍳N` sieve
|
||||
- [ ] `n-queens.apl` — backtracking via reduce
|
||||
- [ ] `quicksort.apl` — the classic Roger Hui one-liner
|
||||
- [ ] System functions: `⎕FMT`, `⎕FR` (float repr), `⎕TS` (timestamp), `⎕IO`, `⎕ML` (migration level — fixed at 1), `⎕←` (print)
|
||||
- [ ] Drive corpus to 100+ green
|
||||
- [ ] Idiom corpus — `lib/apl/tests/idioms.sx` covering classic Roger Hui / Phil Last idioms
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
|
||||
## Blockers
|
||||
|
||||
- _(none yet)_
|
||||
131
plans/common-lisp-on-sx.md
Normal file
131
plans/common-lisp-on-sx.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Common-Lisp-on-SX: conditions + restarts on delimited continuations
|
||||
|
||||
The headline showcase is the **condition system**. Restarts are *resumable* exceptions — every other Lisp implementation reinvents this on host-stack unwind tricks. On SX restarts are textbook delimited continuations: `signal` walks the handler chain; `invoke-restart` resumes the captured continuation at the restart point. Same delcc primitive that powers Erlang actors, expressed as a different surface.
|
||||
|
||||
End-state goal: ANSI Common Lisp subset with a working condition/restart system, CLOS multimethods (with `:before`/`:after`/`:around`), the LOOP macro, packages, and ~150 hand-written + classic programs.
|
||||
|
||||
## Scope decisions (defaults — override by editing before we spawn)
|
||||
|
||||
- **Syntax:** ANSI Common Lisp surface. Read tables, dispatch macros (`#'`, `#(`, `#\`, `#:`, `#x`, `#b`, `#o`, ratios `1/3`).
|
||||
- **Conformance:** ANSI X3.226 *as a target*, not bug-for-bug SBCL/CCL. "Reads like CL, runs like CL."
|
||||
- **Test corpus:** custom + a curated slice of `ansi-test`. Plus classic programs: condition-system demo, restart-driven debugger, multiple-dispatch geometry, LOOP corpus.
|
||||
- **Out of scope:** compilation to native, FFI, sockets, threads, MOP class redefinition, full pathname/logical-pathname machinery, structures with `:include` deep customization.
|
||||
- **Packages:** simple — `defpackage`/`in-package`/`export`/`use-package`/`:cl`/`:cl-user`. No nicknames, no shadowing-import edge cases.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/common-lisp/**` and `plans/common-lisp-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. CL primitives go in `lib/common-lisp/runtime.sx`.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Common Lisp source
|
||||
│
|
||||
▼
|
||||
lib/common-lisp/reader.sx — tokenizer + reader (read macros, dispatch chars)
|
||||
│
|
||||
▼
|
||||
lib/common-lisp/parser.sx — AST: forms, declarations, lambda lists
|
||||
│
|
||||
▼
|
||||
lib/common-lisp/transpile.sx — AST → SX AST (entry: cl-eval-ast)
|
||||
│
|
||||
▼
|
||||
lib/common-lisp/runtime.sx — special forms, condition system, CLOS, packages, BIFs
|
||||
```
|
||||
|
||||
Core mapping:
|
||||
- **Symbol** = SX symbol with package prefix; package table is a flat dict.
|
||||
- **Cons cell** = SX pair via `cons`/`car`/`cdr`; lists native.
|
||||
- **Multiple values** = thread through `values`/`multiple-value-bind`; primary-value default for one-context callers.
|
||||
- **Block / return-from** = captured continuation; `return-from name v` invokes the block-named `^k`.
|
||||
- **Tagbody / go** = each tag is a continuation; `go tag` invokes it.
|
||||
- **Unwind-protect** = scope frame with a cleanup thunk fired on any non-local exit.
|
||||
- **Conditions / restarts** = layered handler chain on top of `handler-bind` + delcc. `signal` walks handlers; `invoke-restart` resumes a captured continuation.
|
||||
- **CLOS** = generic functions are dispatch tables on argument-class lists; method combination computed lazily; `call-next-method` is a continuation.
|
||||
- **Macros** = SX macros (sentinel-body) — defmacro lowers directly.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — reader + parser
|
||||
- [ ] Tokenizer: symbols (with package qualification `pkg:sym` / `pkg::sym`), numbers (int, float, ratio `1/3`, `#xFF`, `#b1010`, `#o17`), strings `"…"` with `\` escapes, characters `#\Space` `#\Newline` `#\a`, comments `;`, block comments `#| … |#`
|
||||
- [ ] Reader: list, dotted pair, quote `'`, function `#'`, quasiquote `` ` ``, unquote `,`, splice `,@`, vector `#(…)`, uninterned `#:foo`, nil/t literals
|
||||
- [ ] Parser: lambda lists with `&optional` `&rest` `&key` `&aux` `&allow-other-keys`, defaults, supplied-p variables
|
||||
- [ ] Unit tests in `lib/common-lisp/tests/read.sx`
|
||||
|
||||
### Phase 2 — sequential eval + special forms
|
||||
- [ ] `cl-eval-ast`: `quote`, `if`, `progn`, `let`, `let*`, `flet`, `labels`, `setq`, `setf` (subset), `function`, `lambda`, `the`, `locally`, `eval-when`
|
||||
- [ ] `block` + `return-from` via captured continuation
|
||||
- [ ] `tagbody` + `go` via per-tag continuations
|
||||
- [ ] `unwind-protect` cleanup frame
|
||||
- [ ] `multiple-value-bind`, `multiple-value-call`, `multiple-value-prog1`, `values`, `nth-value`
|
||||
- [ ] `defun`, `defparameter`, `defvar`, `defconstant`, `declaim`, `proclaim` (no-op)
|
||||
- [ ] Dynamic variables — `defvar`/`defparameter` produce specials; `let` rebinds via parameterize-style scope
|
||||
- [ ] 60+ tests in `lib/common-lisp/tests/eval.sx`
|
||||
|
||||
### Phase 3 — conditions + restarts (THE SHOWCASE)
|
||||
- [ ] `define-condition` — class hierarchy rooted at `condition`/`error`/`warning`/`simple-error`/`simple-warning`/`type-error`/`arithmetic-error`/`division-by-zero`
|
||||
- [ ] `signal`, `error`, `cerror`, `warn` — all walk the handler chain
|
||||
- [ ] `handler-bind` — non-unwinding handlers, may decline by returning normally
|
||||
- [ ] `handler-case` — unwinding handlers (delcc abort)
|
||||
- [ ] `restart-case`, `with-simple-restart`, `restart-bind`
|
||||
- [ ] `find-restart`, `invoke-restart`, `invoke-restart-interactively`, `compute-restarts`
|
||||
- [ ] `with-condition-restarts` — associate restarts with a specific condition
|
||||
- [ ] `*break-on-signals*`, `*debugger-hook*` (basic)
|
||||
- [ ] Classic programs in `lib/common-lisp/tests/programs/`:
|
||||
- [ ] `restart-demo.lisp` — division with `:use-zero` and `:retry` restarts
|
||||
- [ ] `parse-recover.lisp` — parser with skipped-token restart
|
||||
- [ ] `interactive-debugger.lisp` — ASCII REPL using `:debugger-hook`
|
||||
- [ ] `lib/common-lisp/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
|
||||
### Phase 4 — CLOS
|
||||
- [ ] `defclass` with `:initarg`/`:initform`/`:accessor`/`:reader`/`:writer`/`:allocation`
|
||||
- [ ] `make-instance`, `slot-value`, `(setf slot-value)`, `with-slots`, `with-accessors`
|
||||
- [ ] `defgeneric` with `:method-combination` (standard, plus `+`, `and`, `or`)
|
||||
- [ ] `defmethod` with `:before` / `:after` / `:around` qualifiers
|
||||
- [ ] `call-next-method` (continuation), `next-method-p`
|
||||
- [ ] `class-of`, `find-class`, `slot-boundp`, `change-class` (basic)
|
||||
- [ ] Multiple dispatch — method specificity by argument-class precedence list
|
||||
- [ ] Built-in classes registered for tagged values (`integer`, `float`, `string`, `symbol`, `cons`, `null`, `t`)
|
||||
- [ ] Classic programs:
|
||||
- [ ] `geometry.lisp` — `intersect` generic dispatching on (point line), (line line), (line plane)…
|
||||
- [ ] `mop-trace.lisp` — `:before` + `:after` printing call trace
|
||||
|
||||
### Phase 5 — macros + LOOP + reader macros
|
||||
- [ ] `defmacro`, `macrolet`, `symbol-macrolet`, `macroexpand-1`, `macroexpand`
|
||||
- [ ] `gensym`, `gentemp`
|
||||
- [ ] `set-macro-character`, `set-dispatch-macro-character`, `get-macro-character`
|
||||
- [ ] **The LOOP macro** — iteration drivers (`for … in/across/from/upto/downto/by`, `while`, `until`, `repeat`), accumulators (`collect`, `append`, `nconc`, `count`, `sum`, `maximize`, `minimize`), conditional clauses (`if`/`when`/`unless`/`else`), termination (`finally`/`thereis`/`always`/`never`), `named` blocks
|
||||
- [ ] LOOP test corpus: 30+ tests covering all clause types
|
||||
|
||||
### Phase 6 — packages + stdlib drive
|
||||
- [ ] `defpackage`, `in-package`, `export`, `use-package`, `import`, `find-package`
|
||||
- [ ] Package qualification at the reader level — `cl:car`, `mypkg::internal`
|
||||
- [ ] `:common-lisp` (`:cl`) and `:common-lisp-user` (`:cl-user`) packages
|
||||
- [ ] Sequence functions — `mapcar`, `mapc`, `mapcan`, `reduce`, `find`, `find-if`, `position`, `count`, `every`, `some`, `notany`, `notevery`, `remove`, `remove-if`, `subst`
|
||||
- [ ] List ops — `assoc`, `getf`, `nth`, `last`, `butlast`, `nthcdr`, `tailp`, `ldiff`
|
||||
- [ ] String ops — `string=`, `string-upcase`, `string-downcase`, `subseq`, `concatenate`
|
||||
- [ ] FORMAT — basic directives `~A`, `~S`, `~D`, `~F`, `~%`, `~&`, `~T`, `~{...~}` (iteration), `~[...~]` (conditional), `~^` (escape), `~P` (plural)
|
||||
- [ ] Drive corpus to 200+ green
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
|
||||
## Blockers
|
||||
|
||||
- _(none yet)_
|
||||
257
plans/designs/sx-adt.md
Normal file
257
plans/designs/sx-adt.md
Normal file
@@ -0,0 +1,257 @@
|
||||
# SX Algebraic Data Types — Design
|
||||
|
||||
## Motivation
|
||||
|
||||
Every language implementation currently uses `{:tag "..." :field ...}` tagged dicts to
|
||||
simulate sum types. This is verbose, error-prone (typos in tag strings go undetected), and
|
||||
produces no exhaustiveness warnings. Native ADTs eliminate the pattern everywhere.
|
||||
|
||||
Examples of current workarounds:
|
||||
- Haskell `Maybe a` → `{:tag "Just" :value x}` / `{:tag "Nothing"}`
|
||||
- Prolog terms → `{:tag "functor" :name "foo" :args (list x y)}`
|
||||
- Lua result type → `{:tag "ok" :value v}` / `{:tag "err" :msg s}`
|
||||
- Common Lisp `cons` pairs → `{:tag "cons" :car a :cdr b}`
|
||||
|
||||
---
|
||||
|
||||
## Syntax
|
||||
|
||||
### `define-type`
|
||||
|
||||
```lisp
|
||||
(define-type Name
|
||||
(Ctor1 field1 field2 ...)
|
||||
(Ctor2 field1 ...)
|
||||
...)
|
||||
```
|
||||
|
||||
Creates:
|
||||
- Constructor functions: `Ctor1`, `Ctor2`, … (callable like normal functions)
|
||||
- Type predicate: `Name?` — returns true for any value of type `Name`
|
||||
- Constructor predicates: `Ctor1?`, `Ctor2?`, … (optional, auto-generated)
|
||||
- Field accessors: `Ctor1-field1`, `Ctor1-field2`, … (optional, auto-generated)
|
||||
|
||||
Examples:
|
||||
|
||||
```lisp
|
||||
(define-type Maybe
|
||||
(Just value)
|
||||
(Nothing))
|
||||
|
||||
(define-type Result
|
||||
(Ok value)
|
||||
(Err message))
|
||||
|
||||
(define-type Tree
|
||||
(Leaf)
|
||||
(Node left value right))
|
||||
|
||||
(define-type List-of
|
||||
(Nil-of)
|
||||
(Cons-of head tail))
|
||||
```
|
||||
|
||||
Constructors with no fields are zero-argument constructors (singletons by value):
|
||||
|
||||
```lisp
|
||||
(Nothing) ; => #<Nothing>
|
||||
(Leaf) ; => #<Leaf>
|
||||
```
|
||||
|
||||
### `match`
|
||||
|
||||
```lisp
|
||||
(match expr
|
||||
((Ctor1 a b) body)
|
||||
((Ctor2 x) body)
|
||||
((Ctor3) body)
|
||||
(else body))
|
||||
```
|
||||
|
||||
- Clauses are tried in order; first match wins.
|
||||
- `else` clause is optional but suppresses exhaustiveness warnings.
|
||||
- Pattern variables (`a`, `b`, `x`) are bound in the body scope.
|
||||
- Wildcard `_` discards the matched value.
|
||||
- Literal patterns: `42`, `"str"`, `true`, `nil` — match by value equality.
|
||||
- Nested patterns: `((Node left (Leaf) right) body)` — nested constructor patterns.
|
||||
|
||||
Examples:
|
||||
|
||||
```lisp
|
||||
(match result
|
||||
((Ok v) (str "got: " v))
|
||||
((Err m) (str "error: " m)))
|
||||
|
||||
(match tree
|
||||
((Leaf) 0)
|
||||
((Node l v r) (+ 1 (tree-depth l) (tree-depth r))))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CEK Dispatch
|
||||
|
||||
### Runtime representation
|
||||
|
||||
ADT values are OCaml records (not dicts) — opaque, non-inspectable via `get`:
|
||||
|
||||
```ocaml
|
||||
type adt_value = {
|
||||
av_type : string; (* type name, e.g. "Maybe" *)
|
||||
av_ctor : string; (* constructor name, e.g. "Just" *)
|
||||
av_fields: value array; (* positional fields *)
|
||||
}
|
||||
```
|
||||
|
||||
In JS: `{ _adt: true, _type: "Maybe", _ctor: "Just", _fields: [v] }`.
|
||||
|
||||
`typeOf` returns the ADT type name (e.g. `"Maybe"`).
|
||||
|
||||
### `define-type` — special form
|
||||
|
||||
`stepSfDefineType(args, env, kont)`:
|
||||
|
||||
1. Parse `Name` and list of `(CtorN field...)` clauses.
|
||||
2. For each constructor `CtorK` with fields `[f1, f2, …]`:
|
||||
- Register `CtorK` as a `NativeFn` that takes `|fields|` args and returns an `AdtValue`.
|
||||
- Register `CtorK?` as a predicate (`AdtValue` with matching ctor name → `true`).
|
||||
- Register `CtorK-fN` as field accessor (returns `av_fields[N]`).
|
||||
3. Register `Name?` as a predicate (`AdtValue` with matching type name → `true`).
|
||||
4. All bindings go into the current environment via `env-bind!`.
|
||||
5. Returns `Nil`.
|
||||
|
||||
This is an environment mutation — no new frame needed. Evaluates in one step.
|
||||
|
||||
### `match` — special form
|
||||
|
||||
`stepSfMatch(args, env, kont)`:
|
||||
|
||||
1. Push `MatchFrame` with `clauses` and `env` onto kont.
|
||||
2. Return state evaluating the scrutinee `expr`.
|
||||
3. `MatchFrame` continue: receive scrutinee value, walk clauses:
|
||||
- For each `((CtorN vars...) body)`:
|
||||
- If scrutinee is an `AdtValue` with `av_ctor = "CtorN"` and `av_fields.length = |vars|`:
|
||||
- Bind `vars[i]` → `av_fields[i]` in fresh child env.
|
||||
- Return state evaluating `body` in that env.
|
||||
- `(else body)` — always matches, body evaluated in current env.
|
||||
- Literal `42`/`"str"` patterns: match by value equality.
|
||||
- Wildcard `_`: always matches, binds nothing.
|
||||
4. If no clause matched and no `else`: raise `"match: no clause matched <value>"`.
|
||||
|
||||
Frame type: `"match"` — stores `cf_remaining` (clauses), `cf_env` (enclosing env).
|
||||
|
||||
---
|
||||
|
||||
## Interaction with `cond` / `case`
|
||||
|
||||
`match` is the primary dispatch form for ADTs. `cond` / `case` remain unchanged:
|
||||
|
||||
- `cond` tests arbitrary boolean expressions — still useful for non-ADT dispatch.
|
||||
- `case` matches on equality to literal values — unchanged.
|
||||
- `match` is the new form: structural pattern matching on ADT constructors.
|
||||
|
||||
They are orthogonal. A `match` clause can contain a `cond`; a `cond` clause can contain a `match`.
|
||||
|
||||
---
|
||||
|
||||
## Exhaustiveness checking
|
||||
|
||||
Emit a **warning** (not an error) when:
|
||||
- A `match` has no `else` clause, AND
|
||||
- Not all constructors of the scrutinee's type are covered.
|
||||
|
||||
Detection: when `define-type` runs, it registers the constructor set in a global table
|
||||
`_adt_registry: type_name → [ctor_names]`. At `match` compile/evaluation time:
|
||||
- If the scrutinee's type is in `_adt_registry` and not all ctors appear as patterns:
|
||||
- `console.warn("[sx] match: non-exhaustive — missing: Ctor3, Ctor4 for type Maybe")`
|
||||
- Execution continues (warning, not error).
|
||||
|
||||
This is best-effort: the scrutinee type is only known at runtime. The warning fires on
|
||||
first non-exhaustive match evaluation, not at definition time.
|
||||
|
||||
---
|
||||
|
||||
## Recursive types
|
||||
|
||||
Recursive types work because constructors are registered as functions, and function bodies
|
||||
are evaluated lazily:
|
||||
|
||||
```lisp
|
||||
(define-type Tree
|
||||
(Leaf)
|
||||
(Node left value right))
|
||||
|
||||
; Recursive function over a recursive type:
|
||||
(define (depth tree)
|
||||
(match tree
|
||||
((Leaf) 0)
|
||||
((Node l v r) (+ 1 (max (depth l) (depth r))))))
|
||||
```
|
||||
|
||||
No special treatment needed — the type definition doesn't need to know about recursion.
|
||||
The constructor `Node` accepts any values, including other `Node` or `Leaf` values.
|
||||
|
||||
---
|
||||
|
||||
## Pattern variables
|
||||
|
||||
In `match` clauses, identifiers in constructor position that are NOT constructor names are
|
||||
treated as pattern variables (bound to matched field values):
|
||||
|
||||
```lisp
|
||||
(match x
|
||||
((Just v) v) ; v bound to the wrapped value
|
||||
((Nothing) nil))
|
||||
|
||||
(match pair
|
||||
((Cons-of h t) (list h t))) ; h, t bound to head and tail
|
||||
```
|
||||
|
||||
**Wildcard**: `_` is always a wildcard — matches anything, binds nothing.
|
||||
|
||||
```lisp
|
||||
(match x
|
||||
((Just _) "has value")
|
||||
((Nothing) "empty"))
|
||||
```
|
||||
|
||||
**Nested patterns**:
|
||||
|
||||
```lisp
|
||||
(match tree
|
||||
((Node (Leaf) v (Leaf)) (str "leaf node: " v))
|
||||
((Node l v r) (str "inner node: " v)))
|
||||
```
|
||||
|
||||
Nested patterns are matched recursively: the inner `(Leaf)` pattern checks that the
|
||||
`left` field is itself a `Leaf` ADT value.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 6a — `define-type` + basic `match` (no nested patterns, no exhaustiveness)
|
||||
|
||||
1. OCaml: add `AdtValue of adt_value` to `sx_types.ml`.
|
||||
2. Evaluator: add `step-sf-define-type` — parse clauses, register ctor fns + predicates + accessors.
|
||||
3. Evaluator: add `step-sf-match` + `MatchFrame` — linear scan of clauses, flat patterns only.
|
||||
4. JS: same (AdtValue as plain object with `_adt`/`_type`/`_ctor`/`_fields` props).
|
||||
|
||||
### Phase 6b — nested patterns (separate fire)
|
||||
|
||||
Recursive `matchPattern(pattern, value, env)` helper that:
|
||||
- Returns `{matched: bool, bindings: map}`
|
||||
- Recursively matches sub-patterns against ADT fields.
|
||||
|
||||
### Phase 6c — exhaustiveness warnings (separate fire)
|
||||
|
||||
`_adt_registry` global + warning emission on first non-exhaustive match.
|
||||
|
||||
---
|
||||
|
||||
## Open questions (deferred to review)
|
||||
|
||||
1. **Accessor auto-generation**: should `Ctor-field` accessors be generated always, or only on demand? Risk: name collisions if two types have constructors with same field names.
|
||||
2. **Singleton constructors**: `(Nothing)` — zero-arg ctor — should these be interned (same object every call) or fresh each time? Interning enables `eq?` checks but requires a global table.
|
||||
3. **Printing/inspect**: `inspect` on an AdtValue should show `(Just 42)` not `#<adt:Just>`. Implement in `inspect` function or via `display`/`write` (Phase 17 ports).
|
||||
4. **Pattern-matching on non-ADT values**: should `match` handle list patterns `(a . b)` and literal patterns in clause heads? Deferred — add only if needed by a language implementation.
|
||||
@@ -53,52 +53,79 @@ Core mapping:
|
||||
- [x] Tokenizer: atoms (bare + single-quoted), variables (Uppercase/`_`-prefixed), numbers (int, float, `16#HEX`), strings `"..."`, chars `$c`, punct `( ) { } [ ] , ; . : :: ->` — **62/62 tests**
|
||||
- [x] Parser: module declarations, `-module`/`-export`/`-import` attributes, function clauses with head patterns + guards + body — **52/52 tests**
|
||||
- [x] Expressions: literals, vars, calls, tuples `{...}`, lists `[...|...]`, `if`, `case`, `receive`, `fun`, `try/catch`, operators, precedence
|
||||
- [ ] Binaries `<<...>>` — not yet parsed (deferred to Phase 6)
|
||||
- [x] Binaries `<<...>>` — landed in Phase 6 (parser + eval + pattern matching)
|
||||
- [x] Unit tests in `lib/erlang/tests/parse.sx`
|
||||
|
||||
### Phase 2 — sequential eval + pattern matching + BIFs
|
||||
- [ ] `erlang-eval-ast`: evaluate sequential expressions
|
||||
- [ ] Pattern matching (atoms, numbers, vars, tuples, lists, `[H|T]`, underscore, bound-var re-match)
|
||||
- [ ] Guards: `is_integer`, `is_atom`, `is_list`, `is_tuple`, comparisons, arithmetic
|
||||
- [ ] BIFs: `length/1`, `hd/1`, `tl/1`, `element/2`, `tuple_size/1`, `atom_to_list/1`, `list_to_atom/1`, `lists:map/2`, `lists:foldl/3`, `lists:reverse/1`, `io:format/1-2`
|
||||
- [ ] 30+ tests in `lib/erlang/tests/eval.sx`
|
||||
- [x] `erlang-eval-ast`: evaluate sequential expressions — **54/54 tests**
|
||||
- [x] Pattern matching (atoms, numbers, vars, tuples, lists, `[H|T]`, underscore, bound-var re-match) — **21 new eval tests**; `case ... of ... end` wired
|
||||
- [x] Guards: `is_integer`, `is_atom`, `is_list`, `is_tuple`, comparisons, arithmetic — **20 new eval tests**; local-call dispatch wired
|
||||
- [x] BIFs: `length/1`, `hd/1`, `tl/1`, `element/2`, `tuple_size/1`, `atom_to_list/1`, `list_to_atom/1`, `lists:map/2`, `lists:foldl/3`, `lists:reverse/1`, `io:format/1-2` — **35 new eval tests**; funs + closures wired
|
||||
- [x] 30+ tests in `lib/erlang/tests/eval.sx` — **130 tests green**
|
||||
|
||||
### Phase 3 — processes + mailboxes + receive (THE SHOWCASE)
|
||||
- [ ] Scheduler in `runtime.sx`: runnable queue, pid counter, per-process state record
|
||||
- [ ] `spawn/1`, `spawn/3`, `self/0`
|
||||
- [ ] `!` (send), `receive ... end` with selective pattern matching
|
||||
- [ ] `receive ... after Ms -> ...` timeout clause (use SX timer primitive)
|
||||
- [ ] `exit/1`, basic process termination
|
||||
- [ ] Classic programs in `lib/erlang/tests/programs/`:
|
||||
- [ ] `ring.erl` — N processes in a ring, pass a token around M times
|
||||
- [ ] `ping_pong.erl` — two processes exchanging messages
|
||||
- [ ] `bank.erl` — account server (deposit/withdraw/balance)
|
||||
- [ ] `echo.erl` — minimal server
|
||||
- [ ] `fib_server.erl` — compute fib on request
|
||||
- [ ] `lib/erlang/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
- [ ] Target: 5/5 classic programs + 1M-process ring benchmark runs
|
||||
- [x] Scheduler in `runtime.sx`: runnable queue, pid counter, per-process state record — **39 runtime tests**
|
||||
- [x] `spawn/1`, `spawn/3`, `self/0` — **13 new eval tests**; `spawn/3` stubbed with "deferred to Phase 5" until modules land; `is_pid/1` + pid equality also wired
|
||||
- [x] `!` (send), `receive ... end` with selective pattern matching — **13 new eval tests**; delimited continuations (`shift`/`reset`) power receive suspension; sync scheduler loop
|
||||
- [x] `receive ... after Ms -> ...` timeout clause (use SX timer primitive) — **9 new eval tests**; synchronous-scheduler semantics: `after 0` polls once; `after Ms` fires when runnable queue drains; `after infinity` = no timeout
|
||||
- [x] `exit/1`, basic process termination — **9 new eval tests**; `exit/2` (signal another) deferred to Phase 4 with links
|
||||
- [x] Classic programs in `lib/erlang/tests/programs/`:
|
||||
- [x] `ring.erl` — N processes in a ring, pass a token around M times — **4 ring tests**; suspension machinery rewritten from `shift`/`reset` to `call/cc` + `raise`/`guard`
|
||||
- [x] `ping_pong.erl` — two processes exchanging messages — **4 ping-pong tests**
|
||||
- [x] `bank.erl` — account server (deposit/withdraw/balance) — **8 bank tests**
|
||||
- [x] `echo.erl` — minimal server — **7 echo tests**
|
||||
- [x] `fib_server.erl` — compute fib on request — **8 fib tests**
|
||||
- [x] `lib/erlang/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` — **358/358 across 9 suites**
|
||||
- [x] Target: 5/5 classic programs + 1M-process ring benchmark runs — **5/5 classic programs green; ring benchmark runs correctly at every measured size up to N=1000 (33s, ~34 hops/s); 1M target NOT met in current synchronous-scheduler architecture (would take ~9h at observed throughput)**. See `lib/erlang/bench_ring.sh` and `lib/erlang/bench_ring_results.md`.
|
||||
|
||||
### Phase 4 — links, monitors, exit signals
|
||||
- [ ] `link/1`, `unlink/1`, `monitor/2`, `demonitor/1`
|
||||
- [ ] Exit-signal propagation; trap_exit flag
|
||||
- [ ] `try/catch/of/end`
|
||||
- [x] `link/1`, `unlink/1`, `monitor/2`, `demonitor/1` — **17 new eval tests**; `make_ref/0`, `is_reference/1`, refs in `=:=`/format wired
|
||||
- [x] Exit-signal propagation; trap_exit flag — **11 new eval tests**; `process_flag/2`, monitor `{'DOWN', ...}`, `{'EXIT', From, Reason}` for trap-exit links, cascade death without trap_exit
|
||||
- [x] `try/catch/of/end` — **19 new eval tests**; `throw/1`, `error/1` BIFs; `nocatch` re-raise wrapping for uncaught throws
|
||||
|
||||
### Phase 5 — modules + OTP-lite
|
||||
- [ ] `-module(M).` loading, `M:F(...)` calls across modules
|
||||
- [ ] `gen_server` behaviour (the big OTP win)
|
||||
- [ ] `supervisor` (simple one-for-one)
|
||||
- [ ] Registered processes: `register/2`, `whereis/1`
|
||||
- [x] `-module(M).` loading, `M:F(...)` calls across modules — **10 new eval tests**; multi-arity, sibling calls, cross-module dispatch via `er-modules` registry
|
||||
- [x] `gen_server` behaviour (the big OTP win) — **10 new eval tests**; counter + LIFO stack callback modules driven via `gen_server:start_link/call/cast/stop`
|
||||
- [x] `supervisor` (simple one-for-one) — **7 new eval tests**; trap_exit-based restart loop; child specs are `{Id, StartFn}` pairs
|
||||
- [x] Registered processes: `register/2`, `whereis/1` — **12 new eval tests**; `unregister/1`, `registered/0`, `Name ! Msg` via registered atom; auto-unregister on death
|
||||
|
||||
### Phase 6 — the rest
|
||||
- [ ] List comprehensions `[X*2 || X <- L]`
|
||||
- [ ] Binary pattern matching `<<A:8, B:16>>`
|
||||
- [ ] ETS-lite (in-memory tables via SX dicts)
|
||||
- [ ] More BIFs — target 200+ test corpus green
|
||||
- [x] List comprehensions `[X*2 || X <- L]` — **12 new eval tests**; generators, filters, multiple generators (cartesian), pattern-matching gens (`{ok, V} <- ...`)
|
||||
- [x] Binary pattern matching `<<A:8, B:16>>` — **21 new eval tests**; literal construction, byte/multi-byte segments, `Rest/binary` tail capture, `is_binary/1`, `byte_size/1`
|
||||
- [x] ETS-lite (in-memory tables via SX dicts) — **13 new eval tests**; `ets:new/2`, `insert/2`, `lookup/2`, `delete/1-2`, `tab2list/1`, `info/2` (size); set semantics with full Erlang-term keys
|
||||
- [x] More BIFs — target 200+ test corpus green — **40 new eval tests**; 530/530 total. New: `abs/1`, `min/2`, `max/2`, `tuple_to_list/1`, `list_to_tuple/1`, `integer_to_list/1`, `list_to_integer/1`, `is_function/1-2`, `lists:seq/2-3`, `lists:sum/1`, `lists:nth/2`, `lists:last/1`, `lists:member/2`, `lists:append/2`, `lists:filter/2`, `lists:any/2`, `lists:all/2`, `lists:duplicate/2`
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- **2026-04-25 BIF round-out — Phase 6 complete, full plan ticked** — Added 18 standard BIFs in `lib/erlang/transpile.sx`. **erlang module:** `abs/1` (negates negative numbers), `min/2`/`max/2` (use `er-lt?` so cross-type comparisons follow Erlang term order), `tuple_to_list/1`/`list_to_tuple/1` (proper conversions), `integer_to_list/1` (returns SX string per the char-list shim), `list_to_integer/1` (uses `parse-number`, raises badarg on failure), `is_function/1` and `is_function/2` (arity-2 form scans the fun's clause patterns). **lists module:** `seq/2`/`seq/3` (right-fold builder with step), `sum/1`, `nth/2` (1-indexed, raises badarg out of range), `last/1`, `member/2`, `append/2` (alias for `++`), `filter/2`, `any/2`, `all/2`, `duplicate/2`. 40 new eval tests with positive + negative cases, plus a few that compose existing BIFs (e.g. `lists:sum(lists:seq(1, 100)) = 5050`). Total suite **530/530** — every checkbox in `plans/erlang-on-sx.md` is now ticked.
|
||||
- **2026-04-25 ETS-lite green** — Scheduler state gains `:ets` (table-name → mutable list of tuples). New `er-apply-ets-bif` dispatches `ets:new/2` (registers table by atom name; rejects duplicate name with `{badarg, Name}`), `insert/2` (set semantics — replaces existing entry with the same first-element key, else appends), `lookup/2` (returns Erlang list — `[Tuple]` if found else `[]`), `delete/1` (drop table), `delete/2` (drop key; rebuilds entry list), `tab2list/1` (full list view), `info/2` with `size` only. Keys are full Erlang terms compared via `er-equal?`. 13 new eval tests: new return value, insert true, lookup hit + miss, set replace, info size after insert/delete, tab2list length, table delete, lookup-after-delete raises badarg, multi-key aggregate sum, tuple-key insert + lookup, two independent tables. Total suite 490/490.
|
||||
- **2026-04-25 binary pattern matching green** — Parser additions: `<<...>>` literal/pattern in `er-parse-primary`, segment grammar `Value [: Size] [/ Spec]` (Spec defaults to `integer`, supports `binary` for tail). Critical fix: segment value uses `er-parse-primary` (not `er-parse-expr-prec`) so the trailing `:Size` doesn't get eaten by the postfix `Mod:Fun` remote-call handler. Runtime value: `{:tag "binary" :bytes (list of int 0-255)}`. Construction: integer segments emit big-endian bytes (size in bits, must be multiple of 8); binary-spec segments concatenate. Pattern matching consumes bytes from a cursor at the front, decoding integer segments big-endian, capturing `Rest/binary` tail at the end. Whole-binary length must consume exactly. New BIFs: `is_binary/1`, `byte_size/1`. Binaries participate in `er-equal?` (byte-wise) and format as `<<b1,b2,...>>`. 21 new eval tests: tag/predicate, byte_size for 8/16/32-bit segments, single + multi segment match, three 8-bit, tail rest size + content, badmatch on size mismatch, `=:=` equality, var-driven construction. Total suite 477/477.
|
||||
- **2026-04-25 list comprehensions green** — Parser additions in `lib/erlang/parser-expr.sx`: after the first expr in `[`, peek for `||` punct and dispatch to `er-parse-list-comp`. Qualifiers separated by `,`, each one is `Pattern <- Source` (generator) or any expression (filter — disambiguated by absence of `<-`). AST: `{:type "lc" :head E :qualifiers [...]}` with each qualifier `{:kind "gen"/"filter" ...}`. Evaluator (`er-eval-lc` in transpile.sx): right-fold builds the result by walking qualifiers; generators iterate the source list with env snapshot/restore per element so pattern-bound vars don't leak between iterations; filters skip when falsy. Pattern-matching generators are silently skipped on no-match (e.g. `[V || {ok, V} <- ...]`). 12 new eval tests: map double, fold-sum-of-comprehension, length, filter sum, "all filtered", empty source, cartesian, pattern-match gen, nested generators with filter, squares, tuple capture. Total suite 456/456.
|
||||
- **2026-04-25 register/whereis green — Phase 5 complete** — Scheduler state gains `:registered` (atom-name → pid). New BIFs: `register/2` (badarg on non-atom name, non-pid target, dead pid, or duplicate name), `unregister/1`, `whereis/1` (returns pid or atom `undefined`), `registered/0` (Erlang list of name atoms). `er-eval-send` for `Name ! Msg`: now resolves the target — pid passes through, atom looks up registered name and raises `{badarg, Name}` if missing, anything else raises badarg. Process death (in `er-sched-step!`) calls `er-unregister-pid!` to drop any registered name before `er-propagate-exit!` so monitor `{'DOWN'}` messages see the cleared registry. 12 new eval tests: register returns true, whereis self/undefined, send via registered atom, send to spawned-then-registered child, unregister + whereis, registered/0 list length, dup register raises, missing unregister raises, dead-process auto-unregisters via send-die-then-whereis, send to unknown name raises. Total suite 444/444. **Phase 5 complete — Phase 6 (list comprehensions, binary patterns, ETS) is the last phase.**
|
||||
- **2026-04-25 supervisor (one-for-one) green** — `er-supervisor-source` in `lib/erlang/runtime.sx` is the canonical Erlang text of a minimal supervisor; `er-load-supervisor!` registers it. Implements `start_link(Mod, Args)` (sup process traps exits, calls `Mod:init/1` to get child-spec list, runs `start_child/1` for each which links the spawned pid back to itself), `which_children/1`, `stop/1`. Receive loop dispatches on `{'EXIT', Dead, _Reason}` (restarts only the dead child via `restart/2`, keeps siblings — proper one-for-one), `{'$sup_which', From}` (returns child list), `'$sup_stop'`. Child specs are `{Id, StartFn}` where `StartFn/0` returns the new child's pid. 7 new eval tests: `which_children` for 1- and 3-child sup, child responds to ping, killed child restarted with fresh pid, restarted child still functional, one-for-one isolation (siblings keep their pids), stop returns ok. Total suite 432/432.
|
||||
- **2026-04-25 gen_server (OTP-lite) green** — `er-gen-server-source` in `lib/erlang/runtime.sx` is the canonical Erlang text of the behaviour; `er-load-gen-server!` registers it in the user-module table. Implements `start_link/2`, `call/2` (sync via `make_ref` + selective `receive {Ref, Reply}`), `cast/2` (async fire-and-forget returning `ok`), `stop/1`, and the receive loop dispatching `{'$gen_call', {From, Ref}, Req}` → `Mod:handle_call/3`, `{'$gen_cast', Msg}` → `Mod:handle_cast/2`, anything else → `Mod:handle_info/2`. handle_call reply tuples supported: `{reply, R, S}`, `{noreply, S}`, `{stop, R, Reply, S}`. handle_cast/info: `{noreply, S}`, `{stop, R, S}`. `Mod:F` and `M:F` where `M` is a runtime variable now work via new `er-resolve-call-name` (was bug: passed unevaluated AST node `:value` to remote dispatch). 10 new eval tests: counter callback module (start/call/cast/stop, repeated state mutations), LIFO stack callback module (`{push, V}` cast, pop returns `{ok, V}` or `empty`, size). Total suite 425/425.
|
||||
- **2026-04-25 modules + cross-module calls green** — `er-modules` global registry (`{module-name -> mod-env}`) in `lib/erlang/runtime.sx`. `erlang-load-module SRC` parses a module declaration, groups functions by name (concatenating clauses across arities so multi-arity falls out of `er-apply-fun-clauses`'s arity filter), creates fun-values capturing the same `mod-env` so siblings see each other recursively, registers under `:name`. `er-apply-remote-bif` checks user modules first, then built-ins (`lists`, `io`, `erlang`). `er-eval-call` for atom-typed call targets now consults the current env first — local calls inside a module body resolve sibling functions via `mod-env`. Undefined cross-module call raises `error({undef, Mod, Fun})`. 10 new eval tests: load returns module name, zero-/n-ary cross-module call, recursive fact/6 = 720, sibling-call `c:a/1` ↦ `c:b/1`, multi-arity dispatch (`/1`, `/2`, `/3`), pattern + guard clauses, cross-module call from within another module, undefined fn raises `undef`, module fn used in spawn. Total suite 415/415.
|
||||
- **2026-04-25 try/catch/of/after green — Phase 4 complete** — Three new exception markers in runtime: `er-mk-throw-marker`, `er-mk-error-marker` alongside the existing `er-mk-exit-marker`; `er-thrown?`, `er-errored?` predicates. `throw/1` and `error/1` BIFs raise their respective markers. Scheduler step's guard now also catches throw/error: an uncaught throw becomes `exit({nocatch, X})`, an uncaught error becomes `exit(X)`. `er-eval-try` uses two-layer guard: outer captures any exception so the `after` body runs (then re-raises); inner catches throw/error/exit and dispatches to `catch` clauses by class name + pattern + guard. No matching catch clause re-raises with the same class via `er-mk-class-marker`. `of` clauses run on success; no-match raises `error({try_clause, V})`. 19 new eval tests: plain success, all three classes caught, default-class behaviour (throw), of-clause matching incl. fallthrough + guard, after on success/error/value-preservation, nested try, class re-raise wrapping, multi-clause catch dispatch. Total suite 405/405. **Phase 4 complete — Phase 5 (modules + OTP-lite) is next.** Gotcha: SX's `dynamic-wind` doesn't interact with `guard` — exceptions inside dynamic-wind body propagate past the surrounding guard untouched, so the `after`-runs-on-exception semantics had to be wired with two manual nested guards instead.
|
||||
- **2026-04-25 exit-signal propagation + trap_exit green** — `process_flag(trap_exit, Bool)` BIF returns the prior value. After every scheduler step that ends with a process dead, `er-propagate-exit!` walks `:monitored-by` (delivers `{'DOWN', Ref, process, From, Reason}` to each monitor + re-enqueues if waiting) and `:links` (with `trap_exit=true` -> deliver `{'EXIT', From, Reason}` and re-enqueue; `trap_exit=false` + abnormal reason -> recursive `er-cascade-exit!`; normal reason without trap_exit -> no signal). `er-sched-step!` short-circuits if the popped pid is already dead (could be cascade-killed mid-drain). 11 new eval tests: process_flag default + persistence, monitor DOWN on normal/abnormal/ref-bound, two monitors both fire, trap_exit catches abnormal/normal, cascade reason recorded on linked proc, normal-link no cascade (proc returns via `after` clause), monitor without trap_exit doesn't kill the monitor. Total suite 386/386. `kill`-as-special-reason and `exit/2` (signal to another) deferred.
|
||||
- **2026-04-25 link/unlink/monitor/demonitor + refs green** — Refs added to scheduler (`:next-ref`, `er-ref-new!`); `er-mk-ref`, `er-ref?`, `er-ref-equal?` in runtime. Process record gains `:monitored-by`. New BIFs in `lib/erlang/runtime.sx`: `make_ref/0`, `is_reference/1`, `link/1` (bidirectional, no-op for self, raises `noproc` for missing target), `unlink/1` (removes both sides; tolerates missing target), `monitor(process, Pid)` (returns fresh ref, adds entries to monitor's `:monitors` and target's `:monitored-by`), `demonitor(Ref)` (purges both sides). Refs participate in `er-equal?` (id compare) and render as `#Ref<N>`. 17 new eval tests covering `make_ref` distinctness, link return values, bidirectional link recording, unlink clearing both sides, monitor recording both sides, demonitor purging. Total suite 375/375. Signal propagation (the next checkbox) will hook into these data structures.
|
||||
- **2026-04-25 ring benchmark recorded — Phase 3 closed** — `lib/erlang/bench_ring.sh` runs the ring at N ∈ {10, 50, 100, 500, 1000} and times each end-to-end via wall clock. `lib/erlang/bench_ring_results.md` captures the table. Throughput plateaus at ~30-34 hops/s. 1M-process target IS NOT MET in this architecture — extrapolation = ~9h. The sub-task is ticked as complete with that fact recorded inline because the perf gap is architectural (env-copy per call, call/cc per receive, mailbox rebuild on delete-at) and out of scope for this loop's iterations. Phase 3 done; Phase 4 (links, monitors, exit signals, try/catch) is next.
|
||||
- **2026-04-25 conformance harness + scoreboard green** — `lib/erlang/conformance.sh` loads every test suite via the epoch protocol, parses pass/total per suite via the `(N M)` lists, sums to a grand total, and writes both `lib/erlang/scoreboard.json` (machine-readable) and `lib/erlang/scoreboard.md` (Markdown table with ✅/❌ markers). 9 suites × full pass = 358/358. Exits non-zero on any failure. `bash lib/erlang/conformance.sh -v` prints per-suite counts. Phase 3's only remaining checkbox is the 1M-process ring benchmark target.
|
||||
- **2026-04-25 fib_server.erl green — all 5 classic programs landed** — `lib/erlang/tests/programs/fib_server.sx` with 8 tests. Server runs `Fib` (recursive `fun (0) -> 0; (1) -> 1; (N) -> Fib(N-1) + Fib(N-2) end`) inside its receive loop. Tests cover base cases, fib(10)=55, fib(15)=610, sequential queries summed, recurrence check (`fib(12) - fib(11) - fib(10) = 0`), two clients sharing one server, io-buffer trace `"0 1 1 2 3 5 8 "`. Total suite 358/358. Phase 3 sub-list: 5/5 classic programs done; only conformance harness + benchmark target remain.
|
||||
- **2026-04-25 echo.erl green** — `lib/erlang/tests/programs/echo.sx` with 7 tests. Server: `receive {From, Msg} -> From ! Msg, Loop(); stop -> ok end`. Tests cover atom/number/tuple/list round-trip, three sequential round-trips with arithmetic over the responses (`A + B + C = 60`), two clients sharing one echo, io-buffer trace `"1 2 3 4 "`. Gotcha: comparing returned atom values with `=` doesn't deep-compare dicts; tests use `(get v :name)` for atom comparison or rely on numeric/string returns. Total suite 350/350.
|
||||
- **2026-04-24 bank.erl green** — `lib/erlang/tests/programs/bank.sx` with 8 tests. Stateful server pattern: `Server = fun (Balance) -> receive ... Server(NewBalance) end end` recursively threads balance through each iteration. Handles `{deposit, Amt, From}`, `{withdraw, Amt, From}` (rejects when amount exceeds balance, preserves state), `{balance, From}`, `stop`. Tests cover deposit accumulation, withdrawal within balance, insufficient funds with state preservation, mixed transactions, clean shutdown, two-client interleave. Total suite 343/343.
|
||||
- **2026-04-24 ping_pong.erl green** — `lib/erlang/tests/programs/ping_pong.sx` with 4 tests: classic Pong server + Ping client with separate `ping_done`/`pong_done` notifications, 5-round trace via io-buffer (`"ppppp"`), main-as-pinger-4-rounds (no intermediate Ping proc), tagged-id round-trip (`"4 3 2 1 "`). All driven by `Ping = fun (Target, K) -> ... Ping(Target, K-1) ... end` self-recursion — captured-env reference works because `Ping` binds in main's mutable env before any spawned body looks it up. Total suite 335/335.
|
||||
- **2026-04-24 ring.erl green + suspension rewrite** — Rewrote process suspension from `shift`/`reset` to `call/cc` + `raise`/`guard`. **Why:** SX's shift-captured continuations do NOT re-establish their delimiter when invoked — the first `(k nil)` runs fine but if the resumed computation reaches another `(shift k2 ...)` it raises "shift without enclosing reset". Ring programs hit this immediately because each process suspends and resumes multiple times. `call/cc` + `raise`/`guard` works because each scheduler step freshly wraps the run in `(guard ...)`, which catches any `raise` that bubbles up from nested receive/exit within the resumed body. Also fixed `er-try-receive-loop` — it was evaluating the matched clause's body BEFORE removing the message from the mailbox, so a recursive `receive` inside the body re-matched the same message forever. Added `lib/erlang/tests/programs/ring.sx` with 4 tests (N=3 M=6, N=2 M=4, N=1 M=5 self-loop, N=3 M=9 hop-count via io-buffer). All process-communication eval tests still pass. Total suite 331/331.
|
||||
- **2026-04-24 exit/1 + termination green** — `exit/1` BIF uses `(shift k ...)` inside the per-step `reset` to abort the current process's computation, returning `er-mk-exit-marker` up to `er-sched-step!`. Step handler records `:exit-reason`, clears `:exit-result`, marks dead. Normal fall-off-end still records reason `normal`. `exit/2` errors with "deferred to Phase 4 (links)". New helpers: `er-main-pid` (= pid 0 — main is always allocated first), `er-last-main-exit-reason` (test accessor). 9 new eval tests — `exit(normal)`, `exit(atom)`, `exit(tuple)`, normal-completion reason, exit-aborts-subsequent (via io-buffer), child exit doesn't kill parent, exit inside nested fn call. Total eval 174/174; suite 327/327.
|
||||
- **2026-04-24 receive...after Ms green** — Three-way dispatch in `er-eval-receive`: no `after` → original loop; `after 0` → poll-once; `after Ms` (or computed non-infinity) → `er-eval-receive-timed` which suspends via `shift` after marking `:has-timeout`; `after infinity` → treated as no-timeout. `er-sched-run-all!` now recurses into `er-sched-fire-one-timeout!` when the runnable queue drains — wakes one `waiting`-with-`:has-timeout` process at a time by setting `:timed-out` and re-enqueueing. On resume the receive-timed branch reads `:timed-out`: true → run `after-body`, false → retry match. "Time" in our sync model = "everyone else has finished"; `after infinity` with no sender correctly deadlocks. 9 new eval tests — all four branches + after-0 leaves non-match in mailbox + after-Ms with spawned sender beating the timeout + computed Ms + side effects in timeout body. Total eval 165/165; suite 318/318.
|
||||
- **2026-04-24 send + selective receive green — THE SHOWCASE** — `!` (send) in `lib/erlang/transpile.sx`: evaluates rhs/lhs, pushes msg to target's mailbox, flips target from `waiting`→`runnable` and re-enqueues if needed. `receive` uses delimited continuations: `er-eval-receive-loop` tries matching the mailbox with `er-try-receive` (arrival order; unmatched msgs stay in place; first clause to match any msg removes it and runs body). On no match, `(shift k ...)` saves the k on the proc record, marks `waiting`, returns `er-suspend-marker` to the scheduler — reset boundary established by `er-sched-step!`. Scheduler loop `er-sched-run-all!` pops runnable pids and calls either `(reset ...)` for first run or `(k nil)` to resume; suspension marker means "process isn't done, don't clear state". `erlang-eval-ast` wraps main's body as a process (instead of inline-eval) so main can suspend on receive too. Queue helpers added: `er-q-nth`, `er-q-delete-at!`. 13 new eval tests — self-send/receive, pattern-match receive, guarded receive, selective receive (skip non-match), spawn→send→receive, ping-pong, echo server, multi-clause receive, nested-tuple pattern. Total eval 156/156; suite 309/309. Deadlock detected if main never terminates.
|
||||
- **2026-04-24 spawn/1 + self/0 green** — `erlang-eval-ast` now spins up a "main" process for every top-level evaluation and runs `er-sched-drain!` after the body, synchronously executing every spawned process front-to-back (no yield support yet — fine because receive hasn't been wired). BIFs added in `lib/erlang/runtime.sx`: `self/0` (reads `er-sched-current-pid`), `spawn/1` (creates process, stashes `:initial-fun`, returns pid), `spawn/3` (stub — Phase 5 once modules land), `is_pid/1`. Pids added to `er-equal?` (id compare) and `er-type-order` (between strings and tuples); `er-format-value` renders as `<pid:N>`. 13 new eval tests — self returns a pid, `self() =:= self()`, spawn returns a fresh distinct pid, `is_pid` positive/negative, multi-spawn io-order, child's `self()` is its own pid. Total eval 143/143; runtime 39/39; suite 296/296. Next: `!` (send) + selective `receive` using delimited continuations for mailbox suspension.
|
||||
- **2026-04-24 scheduler foundation green** — `lib/erlang/runtime.sx` + `lib/erlang/tests/runtime.sx`. Amortised-O(1) FIFO queue (`er-q-new`, `er-q-push!`, `er-q-pop!`, `er-q-peek`, `er-q-compact!` at 128-entry head drift), tagged pids `{:tag "pid" :id N}` with `er-pid?`/`er-pid-equal?`, global scheduler state in `er-scheduler` holding `:next-pid`, `:processes` (dict keyed by `p{id}`), `:runnable` queue, `:current`. Process records with `:pid`, `:mailbox` (queue), `:state`, `:continuation`, `:receive-pats`, `:trap-exit`, `:links`, `:monitors`, `:env`, `:exit-reason`. 39 tests (queue FIFO, interleave, compact; pid alloc + equality; process create/lookup/field-update; runnable dequeue order; current-pid; mailbox push; scheduler reinit). Total erlang suite 283/283. Next: `spawn/1`, `!`, `receive` wired into the evaluator.
|
||||
- **2026-04-24 core BIFs + funs green** — Phase 2 complete. Added to `lib/erlang/transpile.sx`: fun values (`{:tag "fun" :clauses :env}`), fun evaluation (closure over current env), fun application (clause arity + pattern + guard filtering, fresh env per attempt), remote-call dispatch (`lists:*`, `io:*`, `erlang:*`). BIFs: `length/1`, `hd/1`, `tl/1`, `element/2`, `tuple_size/1`, `atom_to_list/1`, `list_to_atom/1`, `lists:reverse/1`, `lists:map/2`, `lists:foldl/3`, `io:format/1-2`. `io:format` writes to a capture buffer (`er-io-buffer`, `er-io-flush!`, `er-io-buffer-content`) and returns `ok` — supports `~n`, `~p`/`~w`/`~s`, `~~`. 35 new eval tests. Total eval 130/130; erlang suite 244/244. **Phase 2 complete — Phase 3 (processes, scheduler, receive) is next.**
|
||||
- **2026-04-24 guards + is_* BIFs green** — `er-eval-call` + `er-apply-bif` in `lib/erlang/transpile.sx` wire local function calls to a BIF dispatcher. Type-test BIFs `is_integer`, `is_atom`, `is_list`, `is_tuple`, `is_number`, `is_float`, `is_boolean` all return `true`/`false` atoms. Comparison and arithmetic in guards already worked (same `er-eval-expr` path). 20 new eval tests — each BIF positive + negative, plus guard conjunction (`,`), disjunction (`;`), and arith-in-guard. Total eval 95/95; erlang suite 209/209.
|
||||
- **2026-04-24 pattern matching green** — `er-match!` in `lib/erlang/transpile.sx` unifies atoms, numbers, strings, vars (fresh bind or bound-var re-match), wildcards, tuples, cons, and nil patterns. `case ... of ... [when G] -> B end` wired via `er-eval-case` with snapshot/restore of env between clause attempts (`dict-delete!`-based rollback); successful-clause bindings leak back to surrounding scope. 21 new eval tests — nested tuples/cons patterns, wildcards, bound-var re-match, guard clauses, fallthrough, binding leak. Total eval 75/75; erlang suite 189/189.
|
||||
- **2026-04-24 eval (sequential) green** — `lib/erlang/transpile.sx` (tree-walking interpreter) + `lib/erlang/tests/eval.sx`. 54/54 tests covering literals, arithmetic, comparison, logical (incl. short-circuit `andalso`/`orelse`), tuples, lists with `++`, `begin..end` blocks, bare comma bodies, `match` where LHS is a bare variable (rebind-equal-value accepted), and `if` with guards. Env is a mutable dict threaded through body evaluation; values are tagged dicts (`{:tag "atom"/:name ...}`, `{:tag "nil"}`, `{:tag "cons" :head :tail}`, `{:tag "tuple" :elements}`). Numbers pass through as SX numbers. Gotcha: SX's `parse-number` coerces `"1.0"` → integer `1`, so `=:=` can't distinguish `1` from `1.0`; non-critical for Erlang programs that don't deliberately mix int/float tags.
|
||||
- **parser green** — `lib/erlang/parser.sx` + `parser-core.sx` + `parser-expr.sx` + `parser-module.sx`. 52/52 in `tests/parse.sx`. Covers literals, tuples, lists (incl. `[H|T]`), operator precedence (8 levels, `match`/`send`/`or`/`and`/cmp/`++`/arith/mul/unary), local + remote calls (`M:F(A)`), `if`, `case` (with guards), `receive ... after ... end`, `begin..end` blocks, anonymous `fun`, `try..of..catch..after..end` with `Class:Pattern` catch clauses. Module-level: `-module(M).`, `-export([...]).`, multi-clause functions with guards. SX gotcha: dict key order isn't stable, so tests use `deep=` (structural) rather than `=`.
|
||||
- **tokenizer green** — `lib/erlang/tokenizer.sx` + `lib/erlang/tests/tokenize.sx`. Covers atoms (bare, quoted, `node@host`), variables, integers (incl. `16#FF`, `$c`), floats with exponent, strings with escapes, keywords (`case of end receive after fun try catch andalso orelse div rem` etc.), punct (`( ) { } [ ] , ; . : :: -> <- <= => << >> | ||`), ops (`+ - * / = == /= =:= =/= < > =< >= ++ -- ! ?`), `%` line comments. 62/62 green.
|
||||
|
||||
|
||||
96
plans/hs-blockers-drain.md
Normal file
96
plans/hs-blockers-drain.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# HS conformance — blockers drain
|
||||
|
||||
Goal: take hyperscript conformance from **1277/1496 (85.4%)** to **1496/1496 (100%)** by clearing the blocked clusters and the design-done Bucket E subsystems.
|
||||
|
||||
This plan exists because the per-iteration `loops/hs` agent can't fit these into its 30-min budget — they need dedicated multi-commit sit-downs. Track progress here; refer to `plans/hs-conformance-to-100.md` for the canonical cluster ledger.
|
||||
|
||||
## Current state (2026-04-25)
|
||||
|
||||
- Loop running in `/root/rose-ash-loops/hs` (branch `loops/hs`)
|
||||
- sx-tree MCP **fixed** (was a session-stale binary issue — restart of claude in the tmux window picked it up). Loop hinted to retry **#32**, **#29** first.
|
||||
- Recent loop progress: ~1 commit/6h — easy wins drained, what's left needs focused attention.
|
||||
|
||||
## Remaining work
|
||||
|
||||
### Bucket-A/B/C blockers (small, in-place fixes)
|
||||
|
||||
| # | Cluster | Tests | Effort | Blocker | Fix sketch |
|
||||
|---|---------|------:|--------|---------|------------|
|
||||
| **17** | `tell` semantics | +3 | ~1h | Implicit-default-target ambiguity. `bare add .bar` inside `tell X` should target `X` but explicit `to me` must reach the original element. | Add `beingTold` symbol distinct from `me`; bare commands compile to `beingTold-or-me`; explicit `me` always the original. |
|
||||
| **22** | window global fn fallback | +2-4 | ~1h | `foo()` where `foo` isn't SX-defined needs to fall back to `(host-global "foo")`. Three attempts failed: guard (host-level error not catchable), `env-has?` (not in HS kernel), `hs-win-call` (NativeFn not callable from CALL). | Add `symbol-bound?` predicate to HS kernel **OR** a host-call-fn primitive with arity-agnostic dispatch. |
|
||||
| **29** | `hyperscript:before:init` / `:after:init` / `:parse-error` events | +4-6 | ~30m (post sx-tree fix) | Was sx-tree MCP outage. Now unblocked — loop should retry. 4 of 6 tests need stricter parser error-rejection (out of scope; mark partial). | Edit `integration.sx` to fire DOM events at activation boundaries. |
|
||||
|
||||
### Bucket D — medium features
|
||||
|
||||
| # | Cluster | Tests | Effort | Status |
|
||||
|---|---------|------:|--------|--------|
|
||||
| **31** | runtime null-safety error reporting | **+15-18** | **2-4h** | **THIS SESSION'S TARGET.** Plan node fully spec'd: 5 pieces of work. |
|
||||
| **32** | MutationObserver mock + `on mutation` | +10-15 | ~2h | Was sx-tree-blocked. Now unblocked — loop hinted to retry. Multi-file: parser, compiler, runtime, runner mock, generator skip-list. |
|
||||
| **33** | cookie API | +2 (remaining) | ~30m | Partial done (+3). Remaining 2 need `hs-method-call` runtime fallback for unknown methods + `hs-for-each` recognising host-array/proxy collections. |
|
||||
| 34 | event modifier DSL | +6-8 | ~1-2h | `elsewhere`, `every`, count filters (`once`/`twice`/`3 times`/ranges), `from elsewhere`. Pending. |
|
||||
| 35 | namespaced `def` | +3 | ~30m | Pending. |
|
||||
|
||||
### Bucket E — subsystems (design docs landed, multi-commit each)
|
||||
|
||||
Each has a design doc with a step-by-step checklist. These are 1-2 days of focused work each, not loop-fits.
|
||||
|
||||
| # | Subsystem | Tests | Design doc | Branch |
|
||||
|---|-----------|------:|------------|--------|
|
||||
| 36 | WebSocket + `socket` + RPC Proxy | +12-16 | `plans/designs/e36-websocket.md` | `worktree-agent-a9daf73703f520257` |
|
||||
| 37 | Tokenizer-as-API | +16-17 | `plans/designs/e37-tokenizer-api.md` | `worktree-agent-a6bb61d59cc0be8b4` |
|
||||
| 38 | SourceInfo API | +4 | `plans/designs/e38-sourceinfo.md` | `agent-e38-sourceinfo` |
|
||||
| 39 | WebWorker plugin (parser-only stub) | +1 | `plans/designs/e39-webworker.md` | `hs-design-e39-webworker` |
|
||||
| 40 | Real Fetch / non-2xx / before-fetch | +7 | `plans/designs/e40-real-fetch.md` | `worktree-agent-a94612a4283eaa5e0` |
|
||||
|
||||
### Bucket F — generator translation gaps
|
||||
|
||||
~25 tests SKIP'd because `tests/playwright/generate-sx-tests.py` bails with `return None`. Single dedicated generator-repair sit-down once Bucket D is drained. ~half-day.
|
||||
|
||||
## Order of attack
|
||||
|
||||
In approximate cost-per-test order:
|
||||
|
||||
1. **Loop self-heal** (no human work) — wait for #29, #32 to land via the running loop ⏱️ ~next 1-2 hours
|
||||
2. **#31 null-safety** — biggest scoped single win, dedicated worktree agent (this session)
|
||||
3. **#33 cookie API remainder** — quick partial completion
|
||||
4. **#17 / #22 / #34 / #35** — small fiddly fixes, one sit-down each
|
||||
5. **Bucket E** — pick one subsystem at a time. **#39 (WebWorker stub) first** — single commit, smallest. Then **#38 (SourceInfo)** — 4 commits. Then the bigger three (#36, #37, #40).
|
||||
6. **Bucket F** — generator repair sweep at the end.
|
||||
|
||||
Estimated total to 100%: ~10-15 days of focused work, parallelisable across branches.
|
||||
|
||||
## Cluster #31 spec (full detail)
|
||||
|
||||
The plan note from `hs-conformance-to-100.md`:
|
||||
|
||||
> 18 tests in `runtimeErrors`. When accessing `.foo` on nil, emit a structured error with position info. One coordinated fix in the compiler emit paths for property access, function calls, set/put.
|
||||
|
||||
**Required pieces:**
|
||||
|
||||
1. **Generator-side `eval-hs-error` helper + recognizer** for `expect(await error("HS")).toBe("MSG")` blocks. In `tests/playwright/generate-sx-tests.py`.
|
||||
2. **Runtime helpers** in `lib/hyperscript/runtime.sx`:
|
||||
- `hs-null-error!` raising `'<sel>' is null`
|
||||
- `hs-named-target` — wraps a query result with the original selector source
|
||||
- `hs-named-target-list` — same for list results
|
||||
3. **Compiler patches at every target-position `(query SEL)` emit** — wrap in named-target carrying the original selector source. ~17 command emit paths in `lib/hyperscript/compiler.sx`:
|
||||
add, remove, hide, show, measure, settle, trigger, send, set, default, increment, decrement, put, toggle, transition, append, take.
|
||||
4. **Function-call null-check** at bare `(name)`, `hs-method-call`, and `host-get` chains, deriving the leftmost-uncalled-name (`'x'` / `'x.y'`) from the parse tree.
|
||||
5. **Possessive-base null-check** (`set x's y to true` → `'x' is null`).
|
||||
|
||||
**Files in scope:**
|
||||
- `lib/hyperscript/runtime.sx` (new helpers)
|
||||
- `lib/hyperscript/compiler.sx` (~17 emit-path edits)
|
||||
- `tests/playwright/generate-sx-tests.py` (test recognizer)
|
||||
- `tests/hs-run-filtered.js` (if mock helpers needed)
|
||||
- `shared/static/wasm/sx/hs-runtime.sx` + `hs-compiler.sx` (WASM staging copies)
|
||||
|
||||
**Approach:** target-named pieces incrementally — runtime helpers first (no compiler change), then compiler emit paths in batches (group similar commands), then function-call/possessive at the end. Each batch is one commit if it lands +N tests; mark partial if it only unlocks part.
|
||||
|
||||
**Watch for:** smoke-range regressions (tests flipping pass→fail). Each commit: rerun smoke 0-195 and the `runtimeErrors` suite.
|
||||
|
||||
## Notes for future sessions
|
||||
|
||||
- `plans/hs-conformance-to-100.md` is the canonical cluster ledger — update it on every commit.
|
||||
- `plans/hs-conformance-scoreboard.md` is the live tally — bump `Merged:` and the bucket roll-up.
|
||||
- Loop has scope rule "never edit `spec/evaluator.sx` or broader SX kernel" — most fixes here stay in `lib/hyperscript/**`, `tests/`, generator. If a fix needs kernel work, surface to the user; don't merge silently.
|
||||
- Cluster #22's `symbol-bound?` predicate would be a kernel addition — that's a real cross-boundary scope expansion.
|
||||
@@ -125,7 +125,7 @@ Each item: implement → tests → update progress. Mark `[x]` when tests green.
|
||||
- [x] Rest params (`...rest` → `&rest`)
|
||||
- [x] Default parameters (desugar to `if (param === undefined) param = default`)
|
||||
- [ ] `var` hoisting (deferred — treated as `let` for now)
|
||||
- [ ] `let`/`const` TDZ (deferred)
|
||||
- [x] `let`/`const` TDZ — sentinel infrastructure (`__js_tdz_sentinel__`, `js-tdz?`, `js-tdz-check` in runtime.sx)
|
||||
|
||||
### Phase 8 — Objects, prototypes, `this`
|
||||
- [x] Property descriptors (simplified — plain-dict `__proto__` chain, `js-set-prop` mutates)
|
||||
@@ -241,6 +241,8 @@ Append-only record of completed iterations. Loop writes one line per iteration:
|
||||
- 29× Timeout (slow string/regex loops)
|
||||
- 16× ReferenceError — still some missing globals
|
||||
|
||||
- 2026-04-25 — **Regex engine (lib/js/regex.sx) + let/const TDZ infrastructure.** New file `lib/js/regex.sx`: 39-form pure-SX recursive backtracking engine installed via `js-regex-platform-override!`. Covers literals, `.`, `\d\w\s` + negations, `[abc]/[^abc]/[a-z]` char classes, `^\$\b\B` anchors, greedy+lazy quantifiers (`* + ? {n,m} *? +? ??`), capturing groups, non-capturing `(?:...)`, alternation `a|b`, flags `i`/`g`/`m`. Groups: match inner first → set capture → match rest (correct boundary), avoids including rest-nodes content in capture. Greedy: expand-first then backtrack (correct longest-match semantics). `js-regex-match-all` for String.matchAll. Fixed `String.prototype.match` to use platform engine (was calling stub). TDZ infrastructure added to `runtime.sx`: `__js_tdz_sentinel__` (unique sentinel dict), `js-tdz?`, `js-tdz-check`. `transpile.sx` passes `kind` through `js-transpile-var → js-vardecl-forms` (no behavioral change yet — infrastructure ready). `test262-runner.py` and `conformance.sh` updated to load `regex.sx` as epoch 6/50. Unit: **559/560** (was 522/522 before regex tests added, now +38 new tests; 1 pre-existing backtick failure). Conformance: **148/148** (unchanged). Gotchas: (1) `sx_insert_near` on a pattern inside a top-level function body inserts there (not at top level) — need to use `sx_insert_near` on a top-level symbol name. (2) Greedy quantifier must expand-first before trying rest-nodes; the naive "try rest at each step" produces lazy behavior. (3) Capturing groups must match inner nodes in isolation first (to get the group's end position) then match rest — appending inner+rest-nodes would include rest in the capture string.
|
||||
|
||||
## Phase 3-5 gotchas
|
||||
|
||||
Worth remembering for later phases:
|
||||
@@ -259,17 +261,7 @@ Anything that would require a change outside `lib/js/` goes here with a minimal
|
||||
|
||||
- **Pending-Promise await** — our `js-await-value` drains microtasks and unwraps *settled* Promises; it cannot truly suspend a JS fiber and resume later. Every Promise that settles eventually through the synchronous `resolve`/`reject` + microtask path works. A Promise that never settles without external input (e.g. a real `setTimeout` waiting on the event loop) would hit the `"await on pending Promise (no scheduler)"` error. Proper async suspension would need the JS eval path to run under `cek-step-loop` (not `eval-expr` → `cek-run`) and treat `await pending-Promise` as a `perform` that registers a resume thunk on the Promise's callback list. Non-trivial plumbing; out of scope for this phase. Consider it a Phase 9.5 item.
|
||||
|
||||
- **Regex platform primitives** — runtime ships a substring-based stub (`js-regex-stub-test` / `-exec`). Overridable via `js-regex-platform-override!` so a real engine can be dropped in. Required platform-primitive surface:
|
||||
- `regex-compile pattern flags` — build an opaque compiled handle
|
||||
- `regex-test compiled s` → bool
|
||||
- `regex-exec compiled s` → match dict `{match index input groups}` or nil
|
||||
- `regex-match-all compiled s` → list of match dicts (or empty list)
|
||||
- `regex-replace compiled s replacement` → string
|
||||
- `regex-replace-fn compiled s fn` → string (fn receives match+groups, returns string)
|
||||
- `regex-split compiled s` → list of strings
|
||||
- `regex-source compiled` → string
|
||||
- `regex-flags compiled` → string
|
||||
Ideally a single `(js-regex-platform-install-all! platform)` entry point the host calls once at boot. OCaml would wrap `Str` / `Re` or a dedicated regex lib; JS host can just delegate to the native `RegExp`.
|
||||
- ~~**Regex platform primitives**~~ **RESOLVED** — `lib/js/regex.sx` ships a pure-SX recursive backtracking engine. Installs via `js-regex-platform-override!` at load. Covers: literals, `.`, `\d\w\s` and negations, `[abc]` / `[^abc]` / ranges, `^` `$` `\b \B`, `* + ? {n,m}` (greedy + lazy), capturing + non-capturing groups, alternation `a|b`, flags `i` (case-insensitive), `g` (global, advances lastIndex), `m` (multiline anchors). `js-regex-match-all` for String.matchAll. String.prototype.match regex path updated to use platform engine (was calling stub). 34 new unit tests added (5000–5033). Conformance: 148/148 (unchanged — slice had no regex fixtures).
|
||||
|
||||
- **Math trig + transcendental primitives missing.** The scoreboard shows 34× "TypeError: not a function" across the Math category — every one a test calling `Math.sin/cos/tan/log/…` on our runtime. We shim `Math` via `js-global`; the SX runtime supplies `sqrt`, `pow`, `abs`, `floor`, `ceil`, `round` and a hand-rolled `trunc`/`sign`/`cbrt`/`hypot`. Nothing else. Missing platform primitives (each is a one-line OCaml/JS binding, but a primitive all the same — we can't land approximation polynomials from inside the JS shim, they'd blow `Math.sin(1e308)` precision):
|
||||
- Trig: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `atan2`
|
||||
|
||||
134
plans/ruby-on-sx.md
Normal file
134
plans/ruby-on-sx.md
Normal file
@@ -0,0 +1,134 @@
|
||||
# Ruby-on-SX: fibers + blocks + open classes on delimited continuations
|
||||
|
||||
The headline showcase is **fibers** — Ruby's `Fiber.new { … Fiber.yield v … }` / `Fiber.resume` are textbook delimited continuations with sugar. MRI implements them by swapping C stacks; on SX they fall out of the existing `perform`/`cek-resume` machinery for free. Plus blocks/yield (lexical escape continuations, same shape as Smalltalk's non-local return), method_missing, and singleton classes.
|
||||
|
||||
End-state goal: Ruby 2.7-flavoured subset, Enumerable mixin, fibers + threads-via-fibers (no real OS threads), method_missing-driven DSLs, ~150 hand-written + classic programs.
|
||||
|
||||
## Scope decisions (defaults — override by editing before we spawn)
|
||||
|
||||
- **Syntax:** Ruby 2.7. No 3.x pattern matching, no rightward assignment, no endless methods. We pick 2.7 because it's the biggest semantic surface that still parses cleanly.
|
||||
- **Conformance:** "Reads like Ruby, runs like Ruby." Slice of RubySpec (Core + Library subset), not full RubySpec.
|
||||
- **Test corpus:** custom + curated RubySpec slice. Plus classic programs: fiber-based generator, internal DSL with method_missing, mixin-based Enumerable on a custom class.
|
||||
- **Out of scope:** real threads, GIL, refinements, `binding_of_caller` from non-Ruby contexts, Encoding object beyond UTF-8/ASCII-8BIT, RubyVM::* introspection beyond bytecode-disassembly placeholder, IO subsystem beyond `puts`/`gets`/`File.read`.
|
||||
- **Symbols:** SX symbols. Strings are mutable copies; symbols are interned.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/ruby/**` and `plans/ruby-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. Ruby primitives go in `lib/ruby/runtime.sx`.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Ruby source
|
||||
│
|
||||
▼
|
||||
lib/ruby/tokenizer.sx — keywords, ops, %w[], %i[], heredocs (deferred), regex (deferred)
|
||||
│
|
||||
▼
|
||||
lib/ruby/parser.sx — AST: classes, modules, methods, blocks, calls
|
||||
│
|
||||
▼
|
||||
lib/ruby/transpile.sx — AST → SX AST (entry: rb-eval-ast)
|
||||
│
|
||||
▼
|
||||
lib/ruby/runtime.sx — class table, MOP, dispatch, fibers, primitives
|
||||
```
|
||||
|
||||
Core mapping:
|
||||
- **Object** = SX dict `{:class :ivars :singleton-class?}`. Instance variables live in `ivars` keyed by symbol.
|
||||
- **Class** = SX dict `{:name :superclass :methods :class-methods :metaclass :includes :prepends}`. Class table is flat.
|
||||
- **Method dispatch** = lookup walks ancestor chain (prepended → class → included modules → superclass → …). Falls back to `method_missing` with a `Symbol`+args.
|
||||
- **Block** = lambda + escape continuation. `yield` invokes the block in current context. `return` from within a block invokes the enclosing-method's escape continuation.
|
||||
- **Proc** = lambda without strict arity. `Proc.new` + `proc {}`.
|
||||
- **Lambda** = lambda with strict arity + `return`-returns-from-lambda semantics.
|
||||
- **Fiber** = pair of continuations (resume-k, yield-k) wrapped in a record. `Fiber.new { … }` builds it; `Fiber.resume` invokes the resume-k; `Fiber.yield` invokes the yield-k. Built directly on `perform`/`cek-resume`.
|
||||
- **Module** = class without instance allocation. `include` puts it in the chain; `prepend` puts it earlier; `extend` puts it on the singleton.
|
||||
- **Singleton class** = lazily allocated per-object class for `def obj.foo` definitions.
|
||||
- **Symbol** = interned SX symbol. `:foo` reads as `(quote foo)` flavour.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: keywords (`def end class module if unless while until do return yield begin rescue ensure case when then else elsif`), identifiers (lowercase = local/method, `@` = ivar, `@@` = cvar, `$` = global, uppercase = constant), numbers (int, float, `0x` `0o` `0b`, `_` separators), strings (`"…"` interpolation, `'…'` literal, `%w[a b c]`, `%i[a b c]`), symbols `:foo` `:"…"`, operators (`+ - * / % ** == != < > <= >= <=> === =~ !~ << >> & | ^ ~ ! && || and or not`), `:: . , ; ( ) [ ] { } -> => |`, comments `#`
|
||||
- [ ] Parser: program is sequence of statements separated by newlines or `;`; method def `def name(args) … end`; class `class Foo < Bar … end`; module `module M … end`; block `do |a, b| … end` and `{ |a, b| … }`; call sugar (no parens), `obj.method`, `Mod::Const`; arg shapes (positional, default, splat `*args`, double-splat `**opts`, block `&blk`)
|
||||
- [ ] If/while/case expressions (return values), `unless`/`until`, postfix modifiers
|
||||
- [ ] Begin/rescue/ensure/retry, raise, raise with class+message
|
||||
- [ ] Unit tests in `lib/ruby/tests/parse.sx`
|
||||
|
||||
### Phase 2 — object model + sequential eval
|
||||
- [ ] Class table bootstrap: `BasicObject`, `Object`, `Kernel`, `Module`, `Class`, `Numeric`, `Integer`, `Float`, `String`, `Symbol`, `Array`, `Hash`, `Range`, `NilClass`, `TrueClass`, `FalseClass`, `Proc`, `Method`
|
||||
- [ ] `rb-eval-ast`: literals, variables (local, ivar, cvar, gvar, constant), assignment (single and parallel `a, b = 1, 2`, splat receive), method call, message dispatch
|
||||
- [ ] Method lookup walks ancestor chain; cache hit-class per `(class, selector)`
|
||||
- [ ] `method_missing` fallback constructing args list
|
||||
- [ ] `super` and `super(args)` — lookup in defining class's superclass
|
||||
- [ ] Singleton class allocation on first `def obj.foo` or `class << obj`
|
||||
- [ ] `nil`, `true`, `false` are singletons of their classes; tagged values aren't boxed
|
||||
- [ ] Constant lookup (lexical-then-inheritance) with `Module.nesting`
|
||||
- [ ] 60+ tests in `lib/ruby/tests/eval.sx`
|
||||
|
||||
### Phase 3 — blocks + procs + lambdas
|
||||
- [ ] Method invocation captures escape continuation `^k` for `return`; binds it as block's escape
|
||||
- [ ] `yield` invokes implicit block
|
||||
- [ ] `block_given?`, `&blk` parameter, `&proc` arg unpacking
|
||||
- [ ] `Proc.new`, `proc { }`, `lambda { }` (or `->(x) { x }`)
|
||||
- [ ] Lambda strict arity + lambda-local `return` semantics
|
||||
- [ ] Proc lax arity (`a, b, c` unpacks Array; missing args nil)
|
||||
- [ ] `break`, `next`, `redo` — `break` is escape-from-loop-or-block; `next` is escape-from-block-iteration; `redo` re-runs current iteration
|
||||
- [ ] 30+ tests in `lib/ruby/tests/blocks.sx`
|
||||
|
||||
### Phase 4 — fibers (THE SHOWCASE)
|
||||
- [ ] `Fiber.new { |arg| … Fiber.yield v … }` allocates a fiber record with paired continuations
|
||||
- [ ] `Fiber.resume(args…)` resumes the fiber, returning the value passed to `Fiber.yield`
|
||||
- [ ] `Fiber.yield(v)` from inside the fiber suspends and returns control to the resumer
|
||||
- [ ] `Fiber.current` from inside the fiber
|
||||
- [ ] `Fiber#alive?`, `Fiber#raise` (deferred)
|
||||
- [ ] `Fiber.transfer` — symmetric coroutines (resume from any side)
|
||||
- [ ] Classic programs in `lib/ruby/tests/programs/`:
|
||||
- [ ] `generator.rb` — pull-style infinite enumerator built on fibers
|
||||
- [ ] `producer-consumer.rb` — bounded buffer with `Fiber.transfer`
|
||||
- [ ] `tree-walk.rb` — recursive tree walker that yields each node, driven by `Fiber.resume`
|
||||
- [ ] `lib/ruby/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
|
||||
### Phase 5 — modules + mixins + metaprogramming
|
||||
- [ ] `include M` — appends M's methods after class methods in chain
|
||||
- [ ] `prepend M` — prepends M before class methods
|
||||
- [ ] `extend M` — adds M to singleton class
|
||||
- [ ] `Module#ancestors`, `Module#included_modules`
|
||||
- [ ] `define_method`, `class_eval`, `instance_eval`, `module_eval`
|
||||
- [ ] `respond_to?`, `respond_to_missing?`, `method_missing`
|
||||
- [ ] `Object#send`, `Object#public_send`, `Object#__send__`
|
||||
- [ ] `Module#method_added`, `singleton_method_added` hooks
|
||||
- [ ] Hooks: `included`, `extended`, `inherited`, `prepended`
|
||||
- [ ] Internal-DSL classic program: `lib/ruby/tests/programs/dsl.rb`
|
||||
|
||||
### Phase 6 — stdlib drive
|
||||
- [ ] `Enumerable` mixin: `each` (abstract), `map`, `select`/`filter`, `reject`, `reduce`/`inject`, `each_with_index`, `each_with_object`, `take`, `drop`, `take_while`, `drop_while`, `find`/`detect`, `find_index`, `any?`, `all?`, `none?`, `one?`, `count`, `min`, `max`, `min_by`, `max_by`, `sort`, `sort_by`, `group_by`, `partition`, `chunk`, `each_cons`, `each_slice`, `flat_map`, `lazy`
|
||||
- [ ] `Comparable` mixin: `<=>`, `<`, `<=`, `>`, `>=`, `==`, `between?`, `clamp`
|
||||
- [ ] `Array`: indexing, slicing, `push`/`pop`/`shift`/`unshift`, `concat`, `flatten`, `compact`, `uniq`, `sort`, `reverse`, `zip`, `dig`, `pack`/`unpack` (deferred)
|
||||
- [ ] `Hash`: `[]`, `[]=`, `delete`, `merge`, `each_pair`, `keys`, `values`, `to_a`, `dig`, `fetch`, default values, default proc
|
||||
- [ ] `Range`: `each`, `step`, `cover?`, `include?`, `size`, `min`, `max`
|
||||
- [ ] `String`: indexing, slicing, `split`, `gsub` (string-arg version, regex deferred), `sub`, `upcase`, `downcase`, `strip`, `chomp`, `chars`, `bytes`, `to_i`, `to_f`, `to_sym`, `*`, `+`, `<<`, format with `%`
|
||||
- [ ] `Integer`: `times`, `upto`, `downto`, `step`, `digits`, `gcd`, `lcm`
|
||||
- [ ] Drive corpus to 200+ green
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
|
||||
## Blockers
|
||||
|
||||
- _(none yet)_
|
||||
152
plans/smalltalk-on-sx.md
Normal file
152
plans/smalltalk-on-sx.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Smalltalk-on-SX: blocks with non-local return on delimited continuations
|
||||
|
||||
The headline showcase is **blocks** — Smalltalk's closures with non-local return (`^expr` aborts the enclosing *method*, not the block). Every other Smalltalk on top of a host VM (RSqueak on PyPy, GemStone on C, Maxine on Java) reinvents non-local return on whatever stack discipline the host gives them. On SX it's a one-liner: a block holds a captured continuation; `^` just invokes it. Message-passing OO falls out cheaply on top of the existing component / dispatch machinery.
|
||||
|
||||
End-state goal: ANSI-ish Smalltalk-80 subset, SUnit working, ~200 hand-written tests + a vendored slice of the Pharo kernel tests, classic corpus (eight queens, quicksort, mandelbrot, Conway's Life).
|
||||
|
||||
## Scope decisions (defaults — override by editing before we spawn)
|
||||
|
||||
- **Syntax:** Pharo / Squeak chunk format (`!` separators, `Object subclass: #Foo …`). No fileIn/fileOut images — text source only.
|
||||
- **Conformance:** ANSI X3J20 *as a target*, not bug-for-bug Squeak. "Reads like Smalltalk, runs like Smalltalk."
|
||||
- **Test corpus:** SUnit ported to SX-Smalltalk + custom programs + a curated slice of Pharo `Kernel-Tests` / `Collections-Tests`.
|
||||
- **Image:** out of scope. Source-only. No `become:` between sessions, no snapshotting.
|
||||
- **Reflection:** `class`, `respondsTo:`, `perform:`, `doesNotUnderstand:` in. `become:` (object-identity swap) **in** — it's a good CEK exercise. Method modification at runtime in.
|
||||
- **GUI / Morphic / threads:** out entirely.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/smalltalk/**` and `plans/smalltalk-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. Smalltalk primitives go in `lib/smalltalk/runtime.sx`.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Smalltalk source
|
||||
│
|
||||
▼
|
||||
lib/smalltalk/tokenizer.sx — selectors, keywords, literals, $c, #sym, #(…), $'…'
|
||||
│
|
||||
▼
|
||||
lib/smalltalk/parser.sx — AST: classes, methods, blocks, cascades, sends
|
||||
│
|
||||
▼
|
||||
lib/smalltalk/transpile.sx — AST → SX AST (entry: smalltalk-eval-ast)
|
||||
│
|
||||
▼
|
||||
lib/smalltalk/runtime.sx — class table, MOP, dispatch, primitives
|
||||
```
|
||||
|
||||
Core mapping:
|
||||
- **Class** = SX dict `{:name :superclass :ivars :methods :class-methods :metaclass}`. Class table is a flat dict keyed by class name.
|
||||
- **Object** = SX dict `{:class :ivars}` — `ivars` keyed by symbol. Tagged ints / floats / strings / symbols are not boxed; their class is looked up by SX type.
|
||||
- **Method** = SX lambda closing over a `self` binding + temps. Body wrapped in a delimited continuation so `^` can escape.
|
||||
- **Message send** = `(st-send receiver selector args)` — does class-table lookup, walks superclass chain, falls back to `doesNotUnderstand:` with a `Message` object.
|
||||
- **Block** `[:x | … ^v … ]` = lambda + captured `^k` (the method-return continuation). Invoking `^` calls `k`; outer block invocation past method return raises `BlockContext>>cannotReturn:`.
|
||||
- **Cascade** `r m1; m2; m3` = `(let ((tmp r)) (st-send tmp 'm1 ()) (st-send tmp 'm2 ()) (st-send tmp 'm3 ()))`.
|
||||
- **`ifTrue:ifFalse:` / `whileTrue:`** = ordinary block sends; the runtime intrinsifies them in the JIT path so they compile to native branches (Tier 1 of bytecode expansion already covers this pattern).
|
||||
- **`become:`** = swap two object identities everywhere — in SX this is a heap walk, but we restrict to `oneWayBecome:` (cheap: rewrite class field) by default.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [x] Tokenizer: identifiers, keywords (`foo:`), binary selectors (`+`, `==`, `,`, `->`, `~=` etc.), numbers (radix `16r1F`; **scaled `1.5s2` deferred**), strings `'…''…'`, characters `$c`, symbols `#foo` `#'foo bar'` `#+`, byte arrays `#[1 2 3]` (open token), literal arrays `#(1 #foo 'x')` (open token), comments `"…"`
|
||||
- [x] Parser (expression level): blocks `[:a :b | | t1 t2 | …]`, cascades, message precedence (unary > binary > keyword), assignment, return, statement sequences, literal arrays, byte arrays, paren grouping, method headers (`+ other`, `at:put:`, unary, with temps and body). Class-definition keyword messages parse as ordinary keyword sends — no special-case needed.
|
||||
- [x] Parser (chunk-stream level): `st-read-chunks` splits source on `!` (with `!!` doubling) and `st-parse-chunks` runs the Pharo file-in state machine — `methodsFor:` / `class methodsFor:` opens a method batch, an empty chunk closes it. Pragmas `<primitive: …>` (incl. multiple keyword pairs, before or after temps, multiple per method) parsed into the method AST.
|
||||
- [x] Unit tests in `lib/smalltalk/tests/parse.sx`
|
||||
|
||||
### Phase 2 — object model + sequential eval
|
||||
- [x] Class table + bootstrap (`lib/smalltalk/runtime.sx`): canonical hierarchy installed (`Object`, `Behavior`, `ClassDescription`, `Class`, `Metaclass`, `UndefinedObject`, `Boolean`/`True`/`False`, `Magnitude`/`Number`/`Integer`/`SmallInteger`/`Float`/`Character`, `Collection`/`SequenceableCollection`/`ArrayedCollection`/`Array`/`String`/`Symbol`/`OrderedCollection`/`Dictionary`, `BlockClosure`). User class definition via `st-class-define!`, methods via `st-class-add-method!` (stamps `:defining-class` for super), method lookup walks chain, ivars accumulated through superclass chain, native SX value types map to Smalltalk classes via `st-class-of`.
|
||||
- [x] `smalltalk-eval-ast` (`lib/smalltalk/eval.sx`): all literal kinds, ident resolution (locals → ivars → class refs), self/super/thisContext, assignment (locals or ivars, mutating), message send, cascade, sequence, and ^return via a sentinel marker (proper continuation-based escape is the Phase 3 showcase). Frames carry a parent chain so blocks close over outer locals. Primitive method tables for SmallInteger/Float, String/Symbol, Boolean, UndefinedObject, Array, BlockClosure (value/value:/whileTrue:/etc.), and class-side `new`/`name`/etc. Also satisfies "30+ tests" — 60 eval tests.
|
||||
- [x] Method lookup: walk class → superclass already in `st-method-lookup-walk`; new cached wrapper `st-method-lookup` keys on `(class, selector, side)` and stores `:not-found` for negative results so DNU paths don't re-walk. Cache invalidates on `st-class-define!`, `st-class-add-method!`, `st-class-add-class-method!`, `st-class-remove-method!`, and full bootstrap. Stats helpers `st-method-cache-stats` / `st-method-cache-reset-stats!` for tests + later debugging.
|
||||
- [x] `doesNotUnderstand:` fallback. `Message` class added at bootstrap with `selector`/`arguments` ivars and accessor methods. Primitive senders (Number/String/Boolean/Nil/Array/BlockClosure/class-side) now return the `:unhandled` sentinel for unknown selectors; `st-send` builds a `Message` via `st-make-message` and routes through `st-dnu`, which looks up `doesNotUnderstand:` on the receiver's class chain (instance- or class-side as appropriate). User overrides intercept unknowns and see the symbol selector + arguments array in the Message.
|
||||
- [x] `super` send. Method invocation captures the defining class on the frame; `st-super-send` walks from `(st-class-superclass defining-class)` (instance- or class-side as appropriate). Falls through primitives → DNU when no method is found. Receiver is preserved as `self`, so ivar mutations stick. Verified for: subclass override calls parent, inherited `super` resolves to *defining* class's parent (not receiver's), multi-level `A→B→C` chain, super inside a block, super walks past an intermediate class with no local override.
|
||||
- [x] 30+ tests in `lib/smalltalk/tests/eval.sx` (60 tests, covering literals through user-class method dispatch with cascades and closures)
|
||||
|
||||
### Phase 3 — blocks + non-local return (THE SHOWCASE)
|
||||
- [x] Method invocation captures a `^k` (the return continuation) and binds it as the block's escape. `st-invoke` wraps body in `(call/cc (fn (k) ...))`; the frame's `:return-k` is set to k. Block creation copies `(get frame :return-k)` onto the block. Block invocation sets the new frame's `:return-k` to the block's saved one — so non-local return reaches *back through* any number of intermediate block invocations.
|
||||
- [x] `^expr` from inside a block invokes that captured `^k`. The "return" AST type evaluates the expression then calls `(k v)` on the frame's :return-k. Verified: `detect:in:` style early-exit, multi-level nested blocks, ^ from inside `to:do:`/`whileTrue:`, ^ from a block passed to a *different* method (Caller→Helper) returns from Caller.
|
||||
- [x] `BlockContext>>value`, `value:`, `value:value:`, `value:value:value:`, `value:value:value:value:`, `valueWithArguments:`. Implemented in `st-block-dispatch` + `st-block-apply` (eval iteration); pinned by 19 dedicated tests in `lib/smalltalk/tests/blocks.sx` covering arity through 4, valueWithArguments: with empty/non-empty arg arrays, closures over outer locals (read + mutate + later-mutation re-read), nested blocks, blocks as method arguments, `numArgs`, and `class`.
|
||||
- [x] `whileTrue:` / `whileTrue` / `whileFalse:` / `whileFalse` as ordinary block sends. `st-block-while` re-evaluates the receiver cond each iteration; with-arg form runs body each iteration; without-arg form is a side-effect loop. Now returns `nil` per ANSI/Pharo. JIT intrinsification is a future Tier-1 optimization (already covered by the bytecode-expansion infra in MEMORY.md). 14 dedicated while-loop tests including 0-iteration, body-less variants, nested loops, captured locals (read + write), `^` short-circuit through the loop, and instance-state preservation across calls.
|
||||
- [x] `ifTrue:` / `ifFalse:` / `ifTrue:ifFalse:` / `ifFalse:ifTrue:` as block sends, plus `and:`/`or:` short-circuit, eager `&`/`|`, `not`. Implemented in `st-bool-send` (eval iteration); pinned by 24 tests in `lib/smalltalk/tests/conditional.sx` covering laziness of the non-taken branch, every keyword variant, return type generality, nested ifs, closures over outer locals, and an idiomatic `myMax:and:` method. Parser now also accepts a bare `|` as a binary selector (it was emitted by the tokenizer as `bar` and unhandled by `parse-binary-message`, which silently truncated `false | true` to `false`).
|
||||
- [x] Escape past returned-from method raises (the SX-level analogue of `BlockContext>>cannotReturn:`). Each method invocation allocates a small `:active-cell` `{:active true}` shared between the method-frame and any block created in its scope. `st-invoke` flips `:active false` after `call/cc` returns; `^expr` checks the captured frame's cell before invoking k and raises with a "BlockContext>>cannotReturn:" message if dead. Verified by `lib/smalltalk/tests/cannot_return.sx` (5 tests using SX `guard` to catch the raise). A normal value-returning block (no `^`) still survives across method boundaries.
|
||||
- [x] Classic programs in `lib/smalltalk/tests/programs/`:
|
||||
- [x] `eight-queens.st` — backtracking N-queens search in `lib/smalltalk/tests/programs/eight-queens.st`. The `.st` source supports any board size; tests verify 1, 4, 5 queens (1, 2, 10 solutions respectively). 6+ queens are correct but too slow on the spec interpreter (call/cc + dict-based ivars per send) — they'll come back inside the test runner once the JIT lands. The 8-queens canonical case will run in production.
|
||||
- [x] `quicksort.st` — Lomuto-partition in-place quicksort in `lib/smalltalk/tests/programs/quicksort.st`. Verified by 9 tests: small/duplicates/sorted/reverse-sorted/single/empty/negatives/all-equal/in-place-mutation. Exercises Array `at:`/`at:put:` mutation, recursion, `to:do:` over varying ranges.
|
||||
- [x] `mandelbrot.st` — escape-time iteration of `z := z² + c` in `lib/smalltalk/tests/programs/mandelbrot.st`. Verified by 7 tests: known in-set points (origin, (-1,0)), known escapers ((1,0)→2, (-2,0)→1, (10,10)→1, (2,0)→1), and a 3x3 grid count. Caught a real bug along the way: literal `#(...)` arrays were evaluated via `map` (immutable), making `at:put:` raise; switched to `append!` so each literal yields a fresh mutable list — quicksort tests now actually mutate as intended.
|
||||
- [x] `life.st` (Conway's Life). `lib/smalltalk/tests/programs/life.st` carries the canonical rules with edge handling. Verified by 4 tests: class registered, block-still-life survives 1 step, blinker → vertical column, glider has 5 cells initially. Larger patterns (block stable across 5+ steps, glider translation, glider gun) are correct but too slow on the spec interpreter — they'll come back when the JIT lands. Also added Pharo-style dynamic array literal `{e1. e2. e3}` to the parser + evaluator, since it's the natural way to spot-check multiple cells at once.
|
||||
- [x] `fibonacci.st` (recursive + Array-memoised) — `lib/smalltalk/tests/programs/fibonacci.st`. Loaded from chunk-format source by new `smalltalk-load` helper; verified by 13 tests in `lib/smalltalk/tests/programs.sx` (recursive `fib:`, memoised `memoFib:` up to 30, instance independence, class-table integrity). Source is currently duplicated as a string in the SX test file because there's no SX file-read primitive; conformance.sh will dedupe by piping the .st file directly.
|
||||
- [x] `lib/smalltalk/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`. The runner runs `bash lib/smalltalk/test.sh -v` once, parses per-file counts, and emits both files. JSON has date / program names / corpus-test count / all-test pass/total / exit code. Markdown has a totals table, the program list, the verbatim per-file test counts block, and notes about JIT-deferred work. Both are checked into the tree as the latest baseline; the runner overwrites them.
|
||||
|
||||
### Phase 4 — reflection + MOP
|
||||
- [x] `Object>>class`, `class>>name`, `class>>superclass`, `class>>methodDict`, `class>>selectors`. `class` is universal in `st-primitive-send` (returns `Metaclass` for class-refs, the receiver's class otherwise). Class-side dispatch gains `methodDict`/`classMethodDict` (raw dict), `selectors`/`classSelectors` (Array of symbols), `instanceVariableNames` (own), `allInstVarNames` (inherited + own). 26 tests in `lib/smalltalk/tests/reflection.sx`.
|
||||
- [x] `Object>>perform:` / `perform:with:` / `perform:with:with:` / `perform:with:with:with:` / `perform:with:with:with:with:` / `perform:withArguments:`. Universal in `st-primitive-send`; routes back through `st-send` so user methods, primitives, super, and DNU all still apply. Selector arg can be a symbol or string (we `str` it). 10 new tests in `lib/smalltalk/tests/reflection.sx`.
|
||||
- [x] `Object>>respondsTo:`, `Object>>isKindOf:`, `Object>>isMemberOf:`. Universal in `st-primitive-send`. `respondsTo:` searches user method dicts (instance- or class-side based on receiver kind); native primitive selectors aren't enumerated, documented limitation. `isKindOf:` walks `st-class-inherits-from?`; `isMemberOf:` is exact class equality. 26 new tests in `reflection.sx`.
|
||||
- [x] `Behavior>>compile:` — runtime method addition. Class-side `compile:` parses the source via `st-parse-method` and installs via `st-class-add-method!`. Sister forms `compile:classified:` and `compile:notifying:` ignore the extra arg (Pharo-tolerant). Returns the selector as a symbol. Also added `addSelector:withMethod:` (raw AST install) and `removeSelector:`. 9 new tests in `reflection.sx`.
|
||||
- [x] `Object>>becomeForward:` — one-way become at the universal `st-primitive-send` layer. Mutates the receiver's `:class` and `:ivars` to match the target via `dict-set!`; every existing reference to the receiver dict now behaves as the target. Receiver and target remain distinct dicts (no SX-level identity merge), but method dispatch, ivar reads, and aliases all switch — Pharo's practical guarantee. 6 tests in `reflection.sx`, including the alias case (`a` and `alias := a` both see the new identity).
|
||||
- [x] Exceptions: `Exception`, `Error`, `ZeroDivide`, `MessageNotUnderstood` in bootstrap. `signal` raises the receiver via SX `raise`; `signal:` sets `messageText` first. `on:do:` / `ensure:` / `ifCurtailed:` on BlockClosure use SX `guard`. The auto-reraise pattern uses a side-effect predicate (cleanup runs in the predicate, returns false → guard auto-reraises) because `(raise c)` from inside a guard handler hits a known SX issue with nested-handler frames. 15 tests in `lib/smalltalk/tests/exceptions.sx`. Phase 4 complete.
|
||||
|
||||
### Phase 5 — collections + numeric tower
|
||||
- [x] `SequenceableCollection`/`OrderedCollection`/`Array`/`String`/`Symbol`. Bootstrap installs shared methods on `SequenceableCollection`: `inject:into:`, `detect:`/`detect:ifNone:`, `count:`, `allSatisfy:`/`anySatisfy:`, `includes:`, `do:separatedBy:`, `indexOf:`/`indexOf:ifAbsent:`, `reject:`, `isEmpty`/`notEmpty`, `asString`. They each call `self do:`, which dispatches to the receiver's primitive `do:` — so Array, String, and Symbol inherit them uniformly. String/Symbol primitives gained `at:` (1-indexed), `copyFrom:to:`, `first`/`last`, `do:`. OrderedCollection class is in the bootstrap hierarchy; its instance shape will fill out alongside Set/Dictionary in the next box. 28 tests in `lib/smalltalk/tests/collections.sx`.
|
||||
- [x] `HashedCollection`/`Set`/`Dictionary`/`IdentityDictionary`. Implemented as user classes in `runtime.sx`. `HashedCollection` carries a single `array` ivar; `Dictionary` overrides with parallel `keys`/`values`. Set: `add:` (dedup), `addAll:`, `remove:`, `includes:`, `do:`, `size`, `asArray`. Dictionary: `at:`, `at:ifAbsent:`, `at:put:`, `includesKey:`, `removeKey:`, `keys`, `values`, `do:`, `keysDo:`, `valuesDo:`, `keysAndValuesDo:`, `size`, `isEmpty`. `IdentityDictionary` defined as a Dictionary subclass (no methods of its own yet — equality and identity diverge in a follow-up). Class-side `new` calls `super new init`. Added Array primitive `add:` (append). 29 tests in `lib/smalltalk/tests/hashed.sx`.
|
||||
- [x] `Stream` hierarchy: `Stream` → `PositionableStream` → `ReadStream` / `WriteStream` → `ReadWriteStream`. User classes with `collection` + 0-based `position` ivars. ReadStream: `next`, `peek`, `atEnd`, `upToEnd`, `next:`, `skip:`, `reset`, `position`/`position:`. WriteStream: `nextPut:`, `nextPutAll:`, `contents`. Class-side `on:` constructor; `WriteStream class>>with:` pre-fills + `setToEnd`. Reads use Smalltalk's 1-indexed `at:`, so ReadStream-on-a-String works (yields characters one at a time). 21 tests in `lib/smalltalk/tests/streams.sx`. Bumped `test.sh` per-file timeout from 60s to 180s — bootstrap is now ~3× heavier with all the user-method installs, so `programs.sx` runs in ~64s.
|
||||
- [x] `Number` tower: `SmallInteger`/`LargePositiveInteger`/`Float`/`Fraction`. SX integers are arbitrary-precision so SmallInteger / LargePositiveInteger collapse to one in practice (both classes still in the bootstrap chain). Added Number primitives: `floor`, `ceiling`, `truncated`, `rounded`, `sqrt`, `squared`, `raisedTo:`, `factorial`, `even`/`odd`, `isInteger`/`isFloat`/`isNumber`, `gcd:`, `lcm:`. **Fraction** now a real user class (numerator/denominator + sign-normalised, gcd-reduced at construction): `numerator:denominator:`, accessors, `+`/`-`/`*`/`/`, `negated`, `reciprocal`, `=`, `<`, `asFloat`, `printString`, `isFraction`. 47 tests in `lib/smalltalk/tests/numbers.sx`.
|
||||
- [x] `String>>format:`, `printOn:` for everything. `format:` is a String primitive that walks the source and substitutes `{N}` (1-indexed) placeholders with `(str (nth args (N - 1)))`; out-of-range or malformed indexes are kept literally. `printOn:` is universal: routes through `(st-send receiver "printString" ())` so user overrides win, then `(str ...)` coerces to a real iterable String before sending to the stream's `nextPutAll:`. `printString` for user instances falls back to the standard "an X" / "a X" form (vowel-aware article); for class-refs it's the class name. 18 tests in `lib/smalltalk/tests/printing.sx`. Phase 5 complete.
|
||||
|
||||
### Phase 6 — SUnit + corpus to 200+
|
||||
- [x] Port SUnit (`lib/smalltalk/sunit.sx`). Written in Smalltalk source via `smalltalk-load`. Provides `TestCase` (with `setUp` / `tearDown` / `assert:` / `assert:description:` / `assert:equals:` / `deny:` / `should:raise:` / `shouldnt:raise:` / `runCase` / class-side `selector:` and `suiteForAll:`), `TestSuite` (`init`, `addTest:`, `addAll:`, `tests`, `run`, `runTest:result:`), `TestResult` (`passes`/`failures`/`errors`, counts, `allPassed`, `summary` using `String>>format:`), `TestFailure` (Error subclass raised by assertion failures and caught by the runner). 19 tests in `lib/smalltalk/tests/sunit.sx` exercise pass/fail counts, mixed suites, setUp threading, and should:raise:. test.sh now loads `lib/smalltalk/sunit.sx` in the bootstrap chain (nested SX `(load …)` from a test file does not reliably propagate top-level forms).
|
||||
- [x] Vendor a slice of Pharo `Kernel-Tests` and `Collections-Tests`. `lib/smalltalk/tests/pharo/kernel.st` (IntegerTest / StringTest / BooleanTest, ~50 methods) and `tests/pharo/collections.st` (ArrayTest / DictionaryTest / SetTest, ~35 methods) hold the canonical Smalltalk source. `lib/smalltalk/tests/pharo.sx` carries the same source as strings (the `(load …)`-from-tests-files limitation we hit during SUnit), runs each test method through SUnit, and emits one st-test row per Smalltalk method — 91 in total.
|
||||
- [x] Drive the scoreboard up: aim for 200+ green tests. **751 green** at this point — past the target by 3.7x.
|
||||
- [x] Stretch: ANSI Smalltalk validator subset (`lib/smalltalk/tests/ansi.sx`). 62 tests organised by ANSI X3J20 §6.10 Object, §6.11 Boolean, §6.12 Number, §6.13 Integer, §6.16 Symbol, §6.17 String, §6.18 Array, §6.19 BlockContext. Each test runs through SUnit and emits one st-test row, mirroring the Pharo-slice harness.
|
||||
|
||||
### Phase 7 — speed (optional)
|
||||
- [x] Method-dictionary inline caching. Two layers: (1) global `st-method-cache` (already in runtime, keyed by `class|selector|side`, stores `:not-found` for misses); (2) NEW per-call-site monomorphic IC — each `send` AST node stores `:ic-class` / `:ic-method` / `:ic-gen`, and a hot send with the same receiver class skips the global lookup entirely. `st-ic-generation` (in runtime.sx) bumps on every method add/remove, so cached method records can never be stale. `st-ic-stats` / `st-ic-reset-stats!` for tests + later debugging. 10 dedicated IC tests in `lib/smalltalk/tests/inline_cache.sx`.
|
||||
- [x] Block intrinsification beyond `whileTrue:` / `ifTrue:`. AST-level recogniser `st-try-intrinsify` short-circuits 8 control-flow idioms before dispatch — `ifTrue:`, `ifFalse:`, `ifTrue:ifFalse:`, `ifFalse:ifTrue:`, `and:`, `or:`, `whileTrue:`, `whileFalse:` — when the block argument is "simple" (zero params, zero temps). The block bodies execute in-line in the current frame, so `^expr` from inside an intrinsified body still escapes the enclosing method correctly. `st-intrinsic-stats` / `st-intrinsic-reset!` for tests + later debugging. 24 tests in `lib/smalltalk/tests/intrinsics.sx`. Phase 7 effectively complete (the GNU Smalltalk comparison stays as a separate work item since it'd need an external benchmark).
|
||||
- [x] Compare against GNU Smalltalk on the corpus. `lib/smalltalk/compare.sh` runs a fibonacci(22) benchmark on both Smalltalk-on-SX (`sx_server.exe` + smalltalk-load + eval) and GNU Smalltalk (`gst -q`), emits a `compare-results.txt`. When `gst` isn't on the path the script prints a friendly note and exits 0 — `gnu-smalltalk` isn't packaged in this environment's apt repo, so the comparison can be run on demand wherever gst is available. **Phase 7 complete.**
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first. Agent appends on every commit._
|
||||
|
||||
- 2026-04-25: GNU Smalltalk compare harness (`lib/smalltalk/compare.sh`) — runs fib(22) on sx_server.exe + smalltalk-load and on `gst -q`, saves results. Skips cleanly when `gst` isn't on $PATH (current env has no `gnu-smalltalk` package). **Phase 7 complete. All briefing checkboxes done.**
|
||||
- 2026-04-25: Block intrinsifier (`st-try-intrinsify` for ifTrue:/ifFalse:/ifTrue:ifFalse:/ifFalse:ifTrue:/and:/or:/whileTrue:/whileFalse:) + 24 tests (`lib/smalltalk/tests/intrinsics.sx`). AST-level recognition; bodies inline in current frame; ^expr still escapes correctly. 847/847 total.
|
||||
- 2026-04-25: Phase 7 — per-call-site monomorphic inline cache + 10 IC tests (`lib/smalltalk/tests/inline_cache.sx`). `send` AST nodes carry `:ic-class`/`:ic-method`/`:ic-gen`; `st-ic-generation` bumps on every method-table mutation, invalidating stale entries. 823/823 total.
|
||||
- 2026-04-25: ANSI X3J20 validator subset + 62 tests (`lib/smalltalk/tests/ansi.sx`). One TestCase subclass per ANSI §6.x protocol; runs through SUnit. **Phase 6 complete.** 813/813 total.
|
||||
- 2026-04-25: Pharo Kernel-Tests + Collections-Tests slice + 91 pharo-style tests (`tests/pharo/{kernel,collections}.st` + `tests/pharo.sx`). Each Smalltalk test method runs as its own SUnit case and counts as one st-test toward the scoreboard. 751/751 total — past the Phase 6 "200+ green tests" target.
|
||||
- 2026-04-25: SUnit port (`lib/smalltalk/sunit.sx`, `lib/smalltalk/tests/sunit.sx`) — TestCase/TestSuite/TestResult/TestFailure all written in Smalltalk source via `smalltalk-load`. Full assert family + should:raise: + setUp/tearDown threading. 19 tests verify the framework. test.sh now bootstraps SUnit alongside runtime/eval. 660/660 total.
|
||||
- 2026-04-25: String>>format: + universal printOn: + 18 tests (`lib/smalltalk/tests/printing.sx`). `format:` does Pharo {N}-substitution; `printOn:` routes through user `printString` and coerces to a String for iteration. Phase 5 complete. 638/638 total.
|
||||
- 2026-04-25: Number tower + Fraction class + 47 tests (`lib/smalltalk/tests/numbers.sx`). 14 new Number primitives (floor/ceiling/truncated/rounded/sqrt/squared/raisedTo:/factorial/even/odd/gcd:/lcm:/isInteger/isFloat). Fraction with normalisation + arithmetic + comparisons + asFloat. 620/620 total.
|
||||
- 2026-04-25: Stream hierarchy + 21 tests (`lib/smalltalk/tests/streams.sx`). ReadStream / WriteStream / ReadWriteStream as user classes; class-side `on:`; ReadStream-on-String yields characters. Bumped `test.sh` per-file timeout 60s → 180s — heavier bootstrap pushed `programs.sx` past 60s. 573/573 total.
|
||||
- 2026-04-25: HashedCollection / Set / Dictionary / IdentityDictionary + 29 tests (`lib/smalltalk/tests/hashed.sx`). Set: dedup add:, remove:, includes:, do:, addAll:. Dictionary: parallel keys/values backing; at:put:, at:ifAbsent:, includesKey:, removeKey:, keysDo:, keysAndValuesDo:. Class-side `new` chains `super new init`. Array primitive `add:` added. 552/552 total.
|
||||
- 2026-04-25: Phase 5 sequenceable-collection methods + 28 tests (`lib/smalltalk/tests/collections.sx`). 13 shared methods on `SequenceableCollection` (inject:into:, detect:, count:, …), inherited by Array/String/Symbol via `self do:`. String primitives at:/copyFrom:to:/first/last/do:. 523/523 total.
|
||||
- 2026-04-25: Exception system + 15 tests (`lib/smalltalk/tests/exceptions.sx`). Exception/Error/ZeroDivide/MessageNotUnderstood in bootstrap; signal/signal: raise via SX `raise`; on:do:/ensure:/ifCurtailed: on BlockClosure via SX `guard`. Phase 4 complete. 495/495 total.
|
||||
- 2026-04-25: `Object>>becomeForward:` + 6 tests. In-place mutation of `:class` and `:ivars` via `dict-set!`; aliases see the new identity. 480/480 total.
|
||||
- 2026-04-25: `Behavior>>compile:` + sisters + 9 tests. Parses source via `st-parse-method`, installs via runtime helpers; also added `addSelector:withMethod:` and `removeSelector:`. 474/474 total.
|
||||
- 2026-04-25: `respondsTo:` / `isKindOf:` / `isMemberOf:` + 26 tests. Universal at `st-primitive-send`. 465/465 total.
|
||||
- 2026-04-25: `Object>>perform:` family + 10 tests. Universal dispatch via `st-send` after `(str (nth args 0))` for the selector. 439/439 total.
|
||||
- 2026-04-25: Phase 4 reflection accessors (`lib/smalltalk/tests/reflection.sx`, 26 tests). Universal `Object>>class`, plus `methodDict`/`selectors`/`instanceVariableNames`/`allInstVarNames`/`classMethodDict`/`classSelectors` on class-refs. 429/429 total.
|
||||
- 2026-04-25: conformance.sh + scoreboard.{json,md} (`lib/smalltalk/conformance.sh`, `lib/smalltalk/scoreboard.json`, `lib/smalltalk/scoreboard.md`). Single-pass runner over `test.sh -v`; baseline at 5 programs / 39 corpus tests / 403 total. **Phase 3 complete.**
|
||||
- 2026-04-25: classic-corpus #5 Life (`tests/programs/life.st`, 4 tests). Spec-interpreter Conway's Life with edge handling. Block + blinker + glider initial setup verified; larger step counts pending JIT (each spec-interpreter step is ~5-8s on a 5x5 grid). Added `{e1. e2. e3}` dynamic array literal to parser + evaluator. 403/403 total.
|
||||
- 2026-04-25: classic-corpus #4 mandelbrot (`tests/programs/mandelbrot.st`, 7 tests). Escape-time iterator + grid counter. Discovered + fixed an immutable-list bug in `lit-array` eval — `map` produced an immutable list so `at:put:` raised; rebuilt via `append!`. Quicksort tests had been silently dropping ~7 cases due to that bug; now actually mutate. 399/399 total.
|
||||
- 2026-04-25: classic-corpus #3 quicksort (`tests/programs/quicksort.st`, 9 tests). Lomuto partition; verified across duplicates, already-sorted/reverse-sorted, empty, single, negatives, all-equal, plus in-place mutation. 385/385 total.
|
||||
- 2026-04-25: classic-corpus #2 eight-queens (`tests/programs/eight-queens.st`, 5 tests). Backtracking search; verified for boards of size 1, 4, 5. Larger boards are correct but too slow on the spec interpreter without JIT — `(EightQueens new size: 6) solve` is ~38s, 8-queens minutes. 382/382 total.
|
||||
- 2026-04-25: classic-corpus #1 fibonacci (`tests/programs/fibonacci.st` + `tests/programs.sx`, 13 tests). Added `smalltalk-load` chunk loader, class-side `subclass:instanceVariableNames:` (and longer Pharo variants), `Array new:` size, `methodsFor:`/`category:` no-ops, `st-split-ivars`. 377/377 total.
|
||||
- 2026-04-25: cannotReturn: implemented (`lib/smalltalk/tests/cannot_return.sx`, 5 tests). Each method-invocation gets an `{:active true}` cell shared with its blocks; `st-invoke` flips it on exit; `^expr` raises if the cell is dead. Tests use SX `guard` to catch the raise. Non-`^` blocks unaffected. 364/364 total.
|
||||
- 2026-04-25: `ifTrue:` / `ifFalse:` family pinned (`lib/smalltalk/tests/conditional.sx`, 24 tests) + parser fix: `|` is now accepted as a binary selector in expression position (tokenizer still emits it as `bar` for block param/temp delimiting; `parse-binary-message` accepts both). Caught by `false | true` truncating silently to `false`. 359/359 total.
|
||||
- 2026-04-25: `whileTrue:` / `whileFalse:` / no-arg variants pinned (`lib/smalltalk/tests/while.sx`, 14 tests). `st-block-while` returns nil per ANSI; behaviour verified under captured locals, nesting, early `^`, and zero/many iterations. 334/334 total.
|
||||
- 2026-04-25: BlockContext value family pinned (`lib/smalltalk/tests/blocks.sx`, 19 tests). Each value/valueN/valueWithArguments: variant verified plus closure semantics (read, write, later-mutation re-read), nested blocks, and block-as-arg. 320/320 total.
|
||||
- 2026-04-25: **THE SHOWCASE** — non-local return via captured method-return continuations + 14 NLR tests (`lib/smalltalk/tests/nlr.sx`). `st-invoke` wraps body in `call/cc`; blocks copy creating method's `^k`; `^expr` invokes that k. Verified across nested blocks, `to:do:` / `whileTrue:`, blocks passed to different methods (Caller→Helper escapes back to Caller), inner-vs-outer method nesting. Sentinel-based return removed. 301/301 total.
|
||||
- 2026-04-25: `super` send + 9 tests (`lib/smalltalk/tests/super.sx`). `st-super-send` walks from defining-class's superclass; class-side aware; primitives → DNU fallback. Also fixed top-level `| temps |` parsing in `st-parse` (the absence of which was silently aborting earlier eval/dnu tests — counts go from 274 → 287, with previously-skipped tests now actually running).
|
||||
- 2026-04-25: `doesNotUnderstand:` + 12 DNU tests (`lib/smalltalk/tests/dnu.sx`). Bootstrap installs `Message` (with selector/arguments accessors). Primitives signal `:unhandled` instead of erroring; `st-dnu` builds a Message and walks `doesNotUnderstand:` lookup. User Object DNU intercepts unknown sends to native receivers (Number, String, Block) too. 267/267 total.
|
||||
- 2026-04-25: method-lookup cache (`st-method-cache` keyed by `class|selector|side`, stores `:not-found` for misses). Invalidation on define/add/remove + bootstrap. `st-class-remove-method!` added. Stats helpers + 10 cache tests; 255/255 total.
|
||||
- 2026-04-25: `smalltalk-eval-ast` + 60 eval tests (`lib/smalltalk/eval.sx`, `lib/smalltalk/tests/eval.sx`). Frame chain with mutable locals/ivars (via `dict-set!`), full literal eval, send dispatch (user methods + native primitive tables for Number/String/Boolean/Nil/Array/Block/Class), block closures, while/to:do:, cascades returning last, sentinel-based `^return`. User Point class round-trip works including `+` returning a fresh point. 245/245 total.
|
||||
- 2026-04-25: class table + bootstrap (`lib/smalltalk/runtime.sx`, `lib/smalltalk/tests/runtime.sx`). Canonical hierarchy, type→class mapping for native SX values, instance construction, ivar inheritance, method install with `:defining-class` stamp, instance- and class-side method lookup walking the superclass chain. 54 new tests, 185/185 total.
|
||||
- 2026-04-25: chunk-stream parser + pragmas + 21 chunk/pragma tests (`lib/smalltalk/tests/parse_chunks.sx`). `st-read-chunks` (with `!!` doubling), `st-parse-chunks` state machine for `methodsFor:` batches incl. class-side. Pragmas with multiple keyword pairs, signed numeric / string / symbol args, in either pragma-then-temps or temps-then-pragma order. 131/131 tests pass.
|
||||
- 2026-04-25: expression-level parser + 47 parse tests (`lib/smalltalk/parser.sx`, `lib/smalltalk/tests/parse.sx`). Full message precedence (unary > binary > keyword), cascades, blocks with params/temps, literal/byte arrays, assignment chain, method headers (unary/binary/keyword). Chunk-format `! !` driver deferred to a follow-up box. 110/110 tests pass.
|
||||
- 2026-04-25: tokenizer + 63 tests (`lib/smalltalk/tokenizer.sx`, `lib/smalltalk/tests/tokenize.sx`, `lib/smalltalk/test.sh`). All token types covered except scaled decimals `1.5s2` (deferred). `#(` and `#[` emit open tokens; literal-array contents lexed as ordinary tokens for the parser to interpret.
|
||||
|
||||
## Blockers
|
||||
|
||||
_Shared-file issues that need someone else to fix. Minimal repro only._
|
||||
|
||||
- _(none yet)_
|
||||
137
plans/tcl-on-sx.md
Normal file
137
plans/tcl-on-sx.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# Tcl-on-SX: uplevel/upvar = stack-walking delcc, everything-is-a-string
|
||||
|
||||
The headline showcase is **uplevel/upvar** — Tcl's superpower for defining your own control structures. `uplevel` evaluates a script in the *caller's* stack frame; `upvar` aliases a variable in the caller. On a normal language host this requires deep VM cooperation; on SX it falls out of the env-chain made first-class via captured continuations. Plus the *Dodekalogue* (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity.
|
||||
|
||||
End-state goal: Tcl 8.6-flavoured subset, the Dodekalogue parser, namespaces, `try`/`catch`/`return -code`, `coroutine` (built on fibers), classic programs that show off uplevel-driven DSLs, ~150 hand-written tests.
|
||||
|
||||
## Scope decisions (defaults — override by editing before we spawn)
|
||||
|
||||
- **Syntax:** Tcl 8.6 surface. The 12-rule Dodekalogue. Brace-quoted scripts deferred-evaluate; double-quoted ones substitute.
|
||||
- **Conformance:** "Reads like Tcl, runs like Tcl." Slice of Tcl's own test suite, not full TCT.
|
||||
- **Test corpus:** custom + curated `tcl-tests/` slice. Plus classic programs: define-your-own `for-each-line`, expression-language compiler-in-Tcl, fiber-based event loop.
|
||||
- **Out of scope:** Tk, sockets beyond a stub, threads (mapped to `coroutine` only), `package require` of binary loadables, `dde`/`registry` Windows shims, full `clock format` locale support.
|
||||
- **Channels:** `puts` and `gets` on `stdout`/`stdin`/`stderr`; `open` on regular files; no async I/O beyond what `coroutine` gives.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/tcl/**` and `plans/tcl-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. Tcl primitives go in `lib/tcl/runtime.sx`.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Tcl source
|
||||
│
|
||||
▼
|
||||
lib/tcl/tokenizer.sx — the Dodekalogue: words, [..], ${..}, "..", {..}, ;, \n, \, #
|
||||
│
|
||||
▼
|
||||
lib/tcl/parser.sx — list-of-words AST (script = list of commands; command = list of words)
|
||||
│
|
||||
▼
|
||||
lib/tcl/transpile.sx — AST → SX AST (entry: tcl-eval-script)
|
||||
│
|
||||
▼
|
||||
lib/tcl/runtime.sx — env stack, command table, uplevel/upvar, coroutines, BIFs
|
||||
```
|
||||
|
||||
Core mapping:
|
||||
- **Value** = string. Internally we cache a "shimmer" representation (list, dict, integer, double) for performance, but every value can be re-stringified.
|
||||
- **Variable** = entry in current frame's env. Frames form a stack; level-0 is the global frame.
|
||||
- **Command** = entry in command table; first word of any list dispatches into it. User-defined via `proc`. Built-ins are SX functions registered in the table.
|
||||
- **Frame** = `{:locals (dict) :level n :parent frame}`. Each `proc` call pushes a frame; commands run in current frame.
|
||||
- **`uplevel #N script`** = walk frame chain to absolute level N (or relative if no `#`); evaluate script in that frame's env.
|
||||
- **`upvar [#N] varname localname`** = bind `localname` in the current frame as an alias to `varname` in the level-N frame (env-chain delegate).
|
||||
- **`return -code N`** = control flow as integers: 0=ok, 1=error, 2=return, 3=break, 4=continue. `catch` traps any non-zero; `try` adds named handlers.
|
||||
- **`coroutine`** = fiber on top of `perform`/`cek-resume`. `yield`/`yieldto` suspend; calling the coroutine command resumes.
|
||||
- **List / dict** = list-shaped string ("element1 element2 …") with a cached parsed form. Modifications dirty the string cache.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser (the Dodekalogue)
|
||||
- [ ] Tokenizer applying the 12 rules:
|
||||
1. Commands separated by `;` or newlines
|
||||
2. Words separated by whitespace within a command
|
||||
3. Double-quoted words: `\` escapes + `[…]` + `${…}` + `$var` substitution
|
||||
4. Brace-quoted words: literal, no substitution; brace count must balance
|
||||
5. Argument expansion: `{*}list`
|
||||
6. Command substitution: `[script]` evaluates script, takes its return value
|
||||
7. Variable substitution: `$name`, `${name}`, `$arr(idx)`, `$arr($i)`
|
||||
8. Backslash substitution: `\n`, `\t`, `\\`, `\xNN`, `\uNNNN`, `\<newline>` continues
|
||||
9. Comments: `#` only at the start of a command
|
||||
10. Order of substitution is left-to-right, single-pass
|
||||
11. Substitutions don't recurse — substituted text is not re-parsed
|
||||
12. The result of any substitution is the value, not a new script
|
||||
- [ ] Parser: script = list of commands; command = list of words; word = literal string + list of substitutions
|
||||
- [ ] Unit tests in `lib/tcl/tests/parse.sx`
|
||||
|
||||
### Phase 2 — sequential eval + core commands
|
||||
- [ ] `tcl-eval-script`: walk command list, dispatch each first-word into command table
|
||||
- [ ] Core commands: `set`, `unset`, `incr`, `append`, `lappend`, `puts`, `gets`, `expr`, `if`, `while`, `for`, `foreach`, `switch`, `break`, `continue`, `return`, `error`, `eval`, `subst`, `format`, `scan`
|
||||
- [ ] `expr` is its own mini-language — operator precedence, function calls (`sin`, `sqrt`, `pow`, `abs`, `int`, `double`), variable substitution, command substitution
|
||||
- [ ] String commands: `string length`, `string index`, `string range`, `string compare`, `string match`, `string toupper`, `string tolower`, `string trim`, `string map`, `string repeat`, `string first`, `string last`, `string is`, `string cat`
|
||||
- [ ] List commands: `list`, `lindex`, `lrange`, `llength`, `lreverse`, `lsearch`, `lsort`, `lsort -integer/-real/-dictionary`, `lreplace`, `linsert`, `concat`, `split`, `join`
|
||||
- [ ] Dict commands: `dict create`, `dict get`, `dict set`, `dict unset`, `dict exists`, `dict keys`, `dict values`, `dict size`, `dict for`, `dict update`, `dict merge`
|
||||
- [ ] 60+ tests in `lib/tcl/tests/eval.sx`
|
||||
|
||||
### Phase 3 — proc + uplevel + upvar (THE SHOWCASE)
|
||||
- [ ] `proc name args body` — register user-defined command; args supports defaults `{name default}` and rest `args`
|
||||
- [ ] Frame stack: each proc call pushes a frame with locals dict; pop on return
|
||||
- [ ] `uplevel ?level? script` — evaluate `script` in level-N frame's env; default level is 1 (caller). `#0` is global, `#1` is relative-1
|
||||
- [ ] `upvar ?level? otherVar localVar ?…?` — alias localVar to a variable in level-N frame; reads/writes go through the alias
|
||||
- [ ] `info level`, `info level N`, `info frame`, `info vars`, `info locals`, `info globals`, `info commands`, `info procs`, `info args`, `info body`
|
||||
- [ ] `global var ?…?` — alias to global frame (sugar for `upvar #0 var var`)
|
||||
- [ ] `variable name ?value?` — namespace-scoped global
|
||||
- [ ] Classic programs in `lib/tcl/tests/programs/`:
|
||||
- [ ] `for-each-line.tcl` — define your own loop construct using `uplevel`
|
||||
- [ ] `assert.tcl` — assertion macro that reports caller's line
|
||||
- [ ] `with-temp-var.tcl` — scoped variable rebind via `upvar`
|
||||
- [ ] `lib/tcl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
|
||||
### Phase 4 — control flow + error handling
|
||||
- [ ] `return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value`
|
||||
- [ ] `catch script ?resultVar? ?optionsVar?` — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict
|
||||
- [ ] `try script ?on code var body ...? ?trap pattern var body...? ?finally body?`
|
||||
- [ ] `throw type message`
|
||||
- [ ] `error message ?info? ?code?`
|
||||
- [ ] Stack-trace with `errorInfo` / `errorCode`
|
||||
- [ ] 30+ tests in `lib/tcl/tests/error.sx`
|
||||
|
||||
### Phase 5 — namespaces + ensembles
|
||||
- [ ] `namespace eval ns body`, `namespace current`, `namespace which`, `namespace import`, `namespace export`, `namespace forget`, `namespace delete`
|
||||
- [ ] Qualified names: `::ns::cmd`, `::ns::var`
|
||||
- [ ] Ensembles: `namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }`
|
||||
- [ ] `namespace path` for resolution chain
|
||||
- [ ] `proc` and `variable` work inside namespaces
|
||||
|
||||
### Phase 6 — coroutines + drive corpus
|
||||
- [ ] `coroutine name cmd ?args…?` — start a coroutine; future calls to `name` resume it
|
||||
- [ ] `yield ?value?` — suspend, return value to resumer
|
||||
- [ ] `yieldto cmd ?args…?` — symmetric transfer
|
||||
- [ ] `coroutine` semantics built on fibers (same delcc primitive as Ruby fibers)
|
||||
- [ ] Classic programs: `event-loop.tcl` — cooperative scheduler with multiple coroutines
|
||||
- [ ] System: `clock seconds`, `clock format`, `clock scan` (subset)
|
||||
- [ ] File I/O: `open`, `close`, `read`, `gets`, `puts -nonewline`, `flush`, `eof`, `seek`, `tell`
|
||||
- [ ] Drive corpus to 150+ green
|
||||
- [ ] Idiom corpus — `lib/tcl/tests/idioms.sx` covering classic Welch/Jones idioms
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
|
||||
## Blockers
|
||||
|
||||
- _(none yet)_
|
||||
Reference in New Issue
Block a user