apl: merge architecture — Tcl/Prolog/CL/Smalltalk + spec updates
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 47s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 47s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
789
plans/agent-briefings/primitives-loop.md
Normal file
789
plans/agent-briefings/primitives-loop.md
Normal file
@@ -0,0 +1,789 @@
|
||||
# SX Primitives — Meta-Loop Briefing
|
||||
|
||||
Goal: add fundamental missing SX primitives in sequence, then sweep all language
|
||||
implementations to replace their workarounds. Full rationale: vectors fix O(n) array
|
||||
access across every language; numeric tower fixes float/int conflation; dynamic-wind
|
||||
fixes cleanup semantics; coroutine primitive unifies Ruby/Lua/Tcl; string buffer fixes
|
||||
O(n²) concat; algebraic data types eliminate the tagged-dict pattern everywhere.
|
||||
|
||||
**Each fire: find the first unchecked `[ ]`, do it, commit, tick it, stop.**
|
||||
Sub-items within a Phase may span multiple fires — just commit progress and tick what's done.
|
||||
|
||||
---
|
||||
|
||||
## Phase 0 — Prep (gate)
|
||||
|
||||
- [x] Stop new-language loops: send `/exit` to sx-loops windows for the four blank-slate
|
||||
languages that haven't committed workarounds yet:
|
||||
```
|
||||
tmux send-keys -t sx-loops:common-lisp "/exit" Enter
|
||||
tmux send-keys -t sx-loops:apl "/exit" Enter
|
||||
tmux send-keys -t sx-loops:ruby "/exit" Enter
|
||||
tmux send-keys -t sx-loops:tcl "/exit" Enter
|
||||
```
|
||||
Verify all four windows are idle (claude prompt, no active task).
|
||||
|
||||
- [x] E38 + E39 landed: check both Bucket-E branches for implementation commits.
|
||||
```
|
||||
git log --oneline hs-e38-sourceinfo | head -5
|
||||
git log --oneline hs-e39-webworker | head -5
|
||||
```
|
||||
If either branch has only its base commit (no impl work yet): note "pending" and stop —
|
||||
next fire re-checks. Proceed only when both have at least one implementation commit.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Vectors
|
||||
|
||||
Native mutable integer-indexed arrays. Fix: Lua O(n) sort, APL rank polymorphism, Ruby
|
||||
Array, Tcl lists, Common Lisp vectors, all using string-keyed dicts today.
|
||||
|
||||
Primitives to add:
|
||||
- `make-vector` `n` `[fill]` → vector of length n
|
||||
- `vector?` `v` → bool
|
||||
- `vector-ref` `v` `i` → element at index i (0-based)
|
||||
- `vector-set!` `v` `i` `x` → mutate in place
|
||||
- `vector-length` `v` → integer
|
||||
- `vector->list` `v` → list
|
||||
- `list->vector` `lst` → vector
|
||||
- `vector-fill!` `v` `x` → fill all elements
|
||||
- `vector-copy` `v` `[start]` `[end]` → fresh copy of slice
|
||||
|
||||
Steps:
|
||||
- [x] OCaml: add `SxVector of value array` to `hosts/ocaml/sx_types.ml`; implement all
|
||||
primitives in `hosts/ocaml/sx_primitives.ml` (or equivalent); wire into evaluator.
|
||||
Note: Vector type + most prims were already present; added bounds-checked vector-ref/set!
|
||||
and optional start/end to vector-copy. 10/10 vector tests pass (r7rs suite).
|
||||
- [x] Spec: add vector entries to `spec/primitives.sx` with type signatures and descriptions.
|
||||
All 10 vector primitives now have :as type annotations, :returns, and :doc strings.
|
||||
make-vector: optional fill param; vector-copy: optional start/end (done prev step).
|
||||
- [x] JS bootstrapper: implement vectors in `hosts/javascript/platform.js` (or equivalent);
|
||||
ensure `sx-browser.js` rebuild picks them up.
|
||||
Fixed index-of for lists (was returning -1 not NIL, breaking bind-lambda-params),
|
||||
added _lastErrorKont_/hostError/try-catch/without-io-hook stubs. Vectors work.
|
||||
- [x] Tests: 40+ tests in `spec/tests/test-vectors.sx` covering construction, ref, set!,
|
||||
length, conversions, fill, copy, bounds behaviour.
|
||||
42 tests, all pass. 1847 standard / 2362 full passing (up from 5).
|
||||
- [x] Verify: full test suite still passes (`node hosts/javascript/run_tests.js --full`).
|
||||
2362/4924 pass (improvement from pre-existing lambda binding bug, no regressions).
|
||||
- [x] Commit: `spec: vector primitive (make-vector/vector-ref/vector-set!/etc)`
|
||||
Committed as: js: fix lambda binding (index-of on lists), add vectors + R7RS platform stubs
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — Numeric tower
|
||||
|
||||
Float ≠ integer distinction. Fix: Erlang `=:=`, Lua `math.type()`, Haskell `Num`/`Integral`,
|
||||
Common Lisp `integerp`/`floatp`/`ratio`, JS `Number.isInteger`.
|
||||
|
||||
Changes:
|
||||
- `parse-number` preserves float identity: `"1.0"` → float 1.0, not integer 1
|
||||
- New predicates: `integer?`, `float?`, `exact?`, `inexact?`
|
||||
- New coercions: `exact->inexact`, `inexact->exact`
|
||||
- Fix `floor`/`ceiling`/`truncate`/`round` to return integers when applied to floats
|
||||
- `number->string` renders `1.0` as `"1.0"`, `1` as `"1"`
|
||||
- Arithmetic: `(+ 1 1.0)` → `2.0` (float contagion), `(+ 1 1)` → `2` (integer)
|
||||
|
||||
Steps:
|
||||
- [x] OCaml: distinguish `Integer of int` / `Number of float` in `sx_types.ml`; update all
|
||||
arithmetic primitives for float contagion; fix `parse-number`.
|
||||
92/92 numeric tower tests pass; 4874 total (394 pre-existing hs-upstream fails unchanged).
|
||||
- [x] Spec: update `spec/primitives.sx` with new predicates + coercions; document contagion rules.
|
||||
Added integer?/float? predicates; updated number? body; / returns "float"; floor/ceil/truncate
|
||||
return "integer"; +/-/* doc float contagion; fixed double-paren params; 4874/394 baseline.
|
||||
- [x] JS bootstrapper: update number representation and arithmetic.
|
||||
Added integer?/float?/exact?/inexact?/truncate/remainder/modulo/random-int/exact->inexact/
|
||||
inexact->exact/parse-number. Fixed sx_server.ml epoch protocol for Integer type.
|
||||
JS: 1940 passed (+60); OCaml: 4874/394 unchanged. 6 tests JS-only fail (float≡int limitation).
|
||||
- [x] Tests: 92 tests in `spec/tests/test-numeric-tower.sx` — int-arithmetic, float-contagion,
|
||||
division, predicates, coercions, rounding, parse-number, equality, modulo, min-max, stringify.
|
||||
- [x] Verify: full suite passes. OCaml 4874/394 (baseline unchanged). JS 1940/2500 (+60 vs pre-tower).
|
||||
No regressions on any test that relied on `1.0 = 1` — those tests were already using integer
|
||||
literals which remain identical in JS. 6 JS-only failures are platform-inherent (JS float≡int).
|
||||
- [x] Commit: all work landed across 4 commits (c70bbdeb, 45ec5535, b12a22e6, f5acb31c).
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Dynamic-wind
|
||||
|
||||
Fix: Common Lisp `unwind-protect`, Ruby `ensure`, JS `finally`, Tcl `catch`+cleanup,
|
||||
Erlang `try...after` (currently uses double-nested guard workaround).
|
||||
|
||||
- [x] Spec: implement `dynamic-wind` in `spec/evaluator.sx` such that the after-thunk fires
|
||||
on both normal return AND non-local exit (raise/call-cc escape). Must compose with
|
||||
`guard` — currently they don't interact.
|
||||
- [x] OCaml: wire `dynamic-wind` through the CEK machine with a `WindFrame` continuation.
|
||||
- [x] JS bootstrapper: update.
|
||||
- [x] Tests: 20+ tests covering normal return, raise, call/cc escape, nested dynamic-winds.
|
||||
- [x] Commit: `spec: dynamic-wind + guard integration`
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Coroutine primitive
|
||||
|
||||
Unify Ruby fibers, Lua coroutines, Tcl coroutines — all currently reimplemented separately
|
||||
using call/cc+perform/resume.
|
||||
|
||||
- [x] Spec: add `make-coroutine`, `coroutine-resume`, `coroutine-yield`, `coroutine?`,
|
||||
`coroutine-alive?` to `spec/primitives.sx`. Build on existing `perform`/`cek-resume`
|
||||
machinery — coroutines ARE perform/resume with a stable identity.
|
||||
Implemented as `spec/coroutines.sx` define-library; `make-coroutine` stub in evaluator.sx.
|
||||
17/17 coroutine tests pass (OCaml). Drives iteration via define+fn recursion (not named let —
|
||||
named let uses cek_call→cek_run which errors on IO suspension).
|
||||
- [x] OCaml: implement coroutine type; wire resume/yield through CEK suspension.
|
||||
No new native type needed — dict-based coroutine identity + existing cek-step-loop/
|
||||
cek-resume/perform primitives in run_tests.ml ARE the OCaml implementation. 17/17 pass.
|
||||
- [x] JS bootstrapper: update.
|
||||
All CEK primitives already in sx-browser.js. Fix: pre-load spec/coroutines.sx +
|
||||
spec/signals.sx in run_tests.js so (import (sx coroutines)) resolves without suspension.
|
||||
17/17 pass in JS. 1965/2500 (+25 vs 1940 baseline). Zero new failures.
|
||||
- [x] Tests: 25+ tests — multi-yield, final return, arg passthrough, alive? predicate,
|
||||
nested coroutines, "final return vs yield" distinction (the Lua gotcha).
|
||||
27 tests: added 10 new — state field inspection (ready/suspended/dead), yield from
|
||||
nested helper, initial resume arg ignored, mutable closure state, complex yield values,
|
||||
round-robin scheduling, factory-shared-no-state, non-coroutine error. 27/27 OCaml+JS.
|
||||
- [x] Commit: `spec: coroutine primitive (make-coroutine/resume/yield)`
|
||||
Phase 4 landed across 4 commits: 21cb9cf5 (spec library), 9eb12c66 (ocaml verified),
|
||||
b78e06a7 (js pre-load), 0ffe208e (27 tests). Phase 4 complete.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5 — String buffer
|
||||
|
||||
Fix O(n²) string concatenation in loops across Lua, Ruby, Common Lisp, Tcl.
|
||||
|
||||
- [x] Spec + OCaml: add `make-string-buffer`, `string-buffer-append!`, `string-buffer->string`,
|
||||
`string-buffer-length` to primitives. OCaml: `Buffer.t` wrapper. JS: array+join.
|
||||
Also: string-buffer? predicate; SxStringBuffer._string_buffer marker for typeOf/dict?
|
||||
exclusion; inspect case in sx_types.ml. 17/17 tests OCaml+JS.
|
||||
- [x] Tests: 15+ tests.
|
||||
17 tests written inline with Spec+OCaml step: construction, type-of, empty/length,
|
||||
single/multi-append, append-returns-nil, empty-string-append, reuse-after-to-string,
|
||||
independence, loop-building, CSV-row, unicode, repeated-to-string, join-pattern.
|
||||
17/17 OCaml+JS.
|
||||
- [x] Commit: `spec: string-buffer primitive`
|
||||
Committed as d98b5fa2 — all work in one commit (OCaml type + primitives + JS + spec + 17 tests).
|
||||
|
||||
---
|
||||
|
||||
## Phase 6 — Algebraic data types
|
||||
|
||||
The deepest structural gap. Every language uses `{:tag "..." :field ...}` tagged dicts to
|
||||
simulate sum types. A native `define-type` + `match` form eliminates this everywhere.
|
||||
|
||||
- [x] Design: write `plans/designs/sx-adt.md` covering syntax, CEK dispatch, interaction with
|
||||
existing `cond`/`case`, exhaustiveness checking, recursive types, pattern variables.
|
||||
Draft, then stop — next fire reviews design before implementing.
|
||||
Written: define-type/match syntax, AdtValue runtime rep, stepSfDefineType + MatchFrame
|
||||
CEK dispatch, exhaustiveness warnings via _adt_registry, recursive types, nested patterns,
|
||||
wildcard _, 3-phase impl plan (basic/nested/exhaustiveness), open questions on accessors/singletons/inspect.
|
||||
|
||||
- [x] Spec: implement `define-type` special form in `spec/evaluator.sx`:
|
||||
`(define-type Name (Ctor1 field...) (Ctor2 field...) ...)`
|
||||
Creates constructor functions `Ctor1`, `Ctor2` + predicate `Name?`.
|
||||
|
||||
- [x] Spec: implement `match` special form:
|
||||
`(match expr ((Ctor1 a b) body) ((Ctor2 x) body) (else body))`
|
||||
Exhaustiveness warning if not all constructors covered and no `else`.
|
||||
|
||||
- [x] OCaml: add `SxAdt of string * value array` to types; implement constructors + match.
|
||||
Dict-based ADT (no native type needed — matches spec). Hand-written sf_define_type
|
||||
in bootstrap.py FIXUPS; registered via register_special_form. 172 assertions pass.
|
||||
4280/1080 full suite (37 improvement over old baseline 4243/1117).
|
||||
- [x] JS bootstrapper: update.
|
||||
No changes needed — define-type/match are spec-level; sx-browser.js rebuilt at 0dc7e159.
|
||||
40/40 ADT tests pass JS. 2032/2500 total (+67 vs 1965 phase-4 baseline).
|
||||
- [x] Tests: 40+ tests in `spec/tests/test-adt.sx`.
|
||||
40 tests written across two spec commits (6c872107+0dc7e159). All pass OCaml+JS.
|
||||
- [x] Commit: `spec: algebraic data types (define-type + match)`
|
||||
Phase 6 landed across 5 commits: 6c872107 (define-type spec), 0dc7e159 (match spec),
|
||||
5d1913e7 (ocaml bootstrap), f63b2147 (plan tick). JS already current.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7 — Bitwise operations
|
||||
|
||||
Completely absent today. Needed by: Forth (core), APL (array masks), Erlang (bitmatch),
|
||||
JS (typed arrays, bitfields), Common Lisp (`logand`/`logior`/`logxor`/`lognot`/`ash`).
|
||||
|
||||
Primitives to add:
|
||||
- `bitwise-and` `a` `b` → integer
|
||||
- `bitwise-or` `a` `b` → integer
|
||||
- `bitwise-xor` `a` `b` → integer
|
||||
- `bitwise-not` `a` → integer
|
||||
- `arithmetic-shift` `a` `count` → integer (left if count > 0, right if count < 0)
|
||||
- `bit-count` `a` → number of set bits (popcount)
|
||||
- `integer-length` `a` → number of bits needed to represent a
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add entries to `spec/primitives.sx` with type signatures.
|
||||
stdlib.bitwise module with 7 entries appended to spec/primitives.sx.
|
||||
- [x] OCaml: implement in `hosts/ocaml/sx_primitives.ml` using OCaml `land`/`lor`/`lxor`/`lnot`/`lsl`/`asr`.
|
||||
land/lor/lxor/lnot/lsl/asr in sx_primitives.ml. bit-count: Kernighan loop. integer-length: lsr loop.
|
||||
- [x] JS bootstrapper: implement in `hosts/javascript/platform.js` using JS `&`/`|`/`^`/`~`/`<<`/`>>`.
|
||||
stdlib.bitwise module added to PRIMITIVES_JS_MODULES. bit-count: Hamming weight. integer-length: Math.clz32.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-bitwise.sx` — basic ops, shift left/right, negative numbers, popcount.
|
||||
26 tests, 158 assertions, all pass OCaml+JS.
|
||||
- [x] Commit: `spec: bitwise operations (bitwise-and/or/xor/not, arithmetic-shift, bit-count)`
|
||||
Committed a8a79dc9. Phase 7 complete in single commit.
|
||||
|
||||
---
|
||||
|
||||
## Phase 8 — Multiple values
|
||||
|
||||
R7RS standard. Common Lisp uses them heavily; Haskell tuples map naturally; Erlang
|
||||
multi-return. Without them, every function returning two things encodes it as a list or dict.
|
||||
|
||||
Primitives / forms to add:
|
||||
- `values` `v...` → multiple-value object
|
||||
- `call-with-values` `producer` `consumer` → applies consumer to values from producer
|
||||
- `let-values` `(((a b) expr) ...)` `body` — binding form (special form in evaluator)
|
||||
- `define-values` `(a b ...)` `expr` — top-level multi-value bind
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxValues` type to evaluator; implement `values` + `call-with-values` in
|
||||
`spec/evaluator.sx`; add `let-values` / `define-values` special forms.
|
||||
- [x] OCaml: add `SxValues of value list` to `sx_types.ml`; wire through CEK.
|
||||
- [x] JS bootstrapper: implement values type + forms.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-values.sx` — basic producer/consumer, let-values
|
||||
destructuring, define-values, interaction with `begin`/`do`.
|
||||
- [x] Commit: `spec: multiple values (values/call-with-values/let-values)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 9 — Promises (lazy evaluation)
|
||||
|
||||
Critical for Haskell — lazy evaluation is so central that without it the Haskell
|
||||
implementation can't be idiomatic. Also useful for lazy lists in Common Lisp and
|
||||
lazy streams in Scheme-style code generally.
|
||||
|
||||
Primitives / forms to add:
|
||||
- `delay` `expr` → promise (special form — expr not evaluated yet)
|
||||
- `force` `p` → evaluate promise, cache result, return it
|
||||
- `make-promise` `v` → already-forced promise wrapping v
|
||||
- `promise?` `v` → bool
|
||||
- `delay-force` `expr` → for iterative lazy sequences (avoids stack growth in lazy streams)
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `delay` / `delay-force` special forms to `spec/evaluator.sx`; add promise
|
||||
type with mutable forced/value slots; `force` checks if already forced before eval.
|
||||
- [x] OCaml: add `SxPromise of { mutable forced: bool; mutable value: value; thunk: value }`;
|
||||
wire `delay`/`force`/`delay-force` through CEK.
|
||||
- [x] JS bootstrapper: implement promise type + forms.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-promises.sx` — basic delay/force, memoisation
|
||||
(forced only once), delay-force lazy stream, promise? predicate, make-promise.
|
||||
- [x] Commit: `spec: promises — delay/force/delay-force for lazy evaluation`
|
||||
|
||||
---
|
||||
|
||||
## Phase 10 — Mutable hash tables
|
||||
|
||||
Distinct from SX's immutable dicts. Dict primitives copy on every update — fine for
|
||||
functional code, wrong for table-heavy language implementations. Lua tables, Smalltalk
|
||||
dicts, Erlang process dictionaries, and JS Map all need O(1) mutable associative storage.
|
||||
|
||||
Primitives to add:
|
||||
- `make-hash-table` `[capacity]` → fresh mutable hash table
|
||||
- `hash-table?` `v` → bool
|
||||
- `hash-table-set!` `ht` `key` `val` → mutate in place
|
||||
- `hash-table-ref` `ht` `key` `[default]` → value or default/error
|
||||
- `hash-table-delete!` `ht` `key` → remove entry
|
||||
- `hash-table-size` `ht` → integer
|
||||
- `hash-table-keys` `ht` → list of keys
|
||||
- `hash-table-values` `ht` → list of values
|
||||
- `hash-table->alist` `ht` → list of (key . value) pairs
|
||||
- `hash-table-for-each` `ht` `fn` → iterate (fn key val) for side effects
|
||||
- `hash-table-merge!` `dst` `src` → merge src into dst in place
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add entries to `spec/primitives.sx`.
|
||||
stdlib.hash-table module with 11 define-primitive entries appended to spec/primitives.sx.
|
||||
- [x] OCaml: add `HashTable of (value, value) Hashtbl.t` to `sx_types.ml`; implement
|
||||
all primitives in `hosts/ocaml/sx_primitives.ml`.
|
||||
HashTable variant in sx_types.ml; type_of/inspect cases added; 11 primitives in sx_primitives.ml;
|
||||
fixed _cek_call_ref reference for hash-table-for-each. 4385/1080 (+28).
|
||||
- [x] JS bootstrapper: implement using JS `Map` in `hosts/javascript/platform.js`.
|
||||
SxHashTable class with Map; _hash_table marker; dict?/type-of exclusion; apply() for for-each.
|
||||
2137/2500 (+4 vs phase-9 baseline).
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-hash-table.sx` — set/ref/delete, size, iteration,
|
||||
default on missing key, merge, keys/values lists.
|
||||
28 tests; all pass OCaml+JS. Used empty? not assert= for empty-list comparisons.
|
||||
- [x] Commit: `spec: mutable hash tables (make-hash-table/ref/set!/delete!/etc)`
|
||||
Committed 133bdf52. Phase 10 complete.
|
||||
|
||||
---
|
||||
|
||||
## Phase 11 — Sequence protocol
|
||||
|
||||
Unified iteration over lists and vectors without conversion. Currently `map`/`filter`/
|
||||
`for-each` only work on lists — you must `vector->list` first, which defeats the purpose
|
||||
of vectors. A sequence protocol makes all collection operations polymorphic.
|
||||
|
||||
Approach: extend existing `map`/`filter`/`reduce`/`for-each`/`some`/`every?` to dispatch
|
||||
on type (list → existing path, vector → index loop, string → char iteration). Add:
|
||||
- `in-range` `start` `[end]` `[step]` → lazy range sequence (works with `for-each`/`map`)
|
||||
- `sequence->list` `s` → coerce any sequence to list
|
||||
- `sequence->vector` `s` → coerce any sequence to vector
|
||||
- `sequence-length` `s` → length of any sequence
|
||||
- `sequence-ref` `s` `i` → element by index (lists and vectors)
|
||||
- `sequence-append` `s1` `s2` → concatenate two same-type sequences
|
||||
|
||||
Steps:
|
||||
- [x] Spec: extend `map`/`filter`/`reduce`/`for-each`/`some`/`every?` in `spec/evaluator.sx`
|
||||
to type-dispatch; add `in-range` lazy sequence type + helpers.
|
||||
- [x] OCaml: update HO form dispatch; add `SxRange` or use lazy list; implement `sequence-*`
|
||||
primitives.
|
||||
seq_to_list helper before let-rec block; ho_setup_dispatch wraps all 7 coll bindings;
|
||||
seq-to-list/sequence-to-list/vector/length/ref/append/in-range in sx_primitives.ml.
|
||||
4385/1080 (all failures pre-existing hs-*/regex; 0 regressions).
|
||||
- [x] JS bootstrapper: update.
|
||||
Already done in Spec step (da4b526a) — sx-browser.js rebuilt with seqToList/sequenceToList/
|
||||
sequenceToVector/sequenceLength/sequenceRef/sequenceAppend/inRange. 2137/2500 JS tests pass.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-sequences.sx` — map over vector, filter over
|
||||
range, for-each over string chars, sequence-append, sequence->list/vector coercions.
|
||||
45 tests all passing: JS 2185/2498 (+48), OCaml 4424/1087 (+39). Fixed: vector? rename
|
||||
(isVector), vectorLength/vectorRef/reverse aliases, in-range letrec→build-range,
|
||||
sequence-length nil=0, assert-equal for list comparisons. Committed 0fe00bf7.
|
||||
- [x] Commit: `spec: sequence protocol — polymorphic map/filter/for-each over list/vector/range`
|
||||
Work landed across da4b526a (Spec), 7286629c (OCaml), 06a3eee1 (JS bootstrap), 0fe00bf7 (Tests).
|
||||
|
||||
---
|
||||
|
||||
## Phase 12 — gensym + symbol interning
|
||||
|
||||
Unique symbol generation. Tiny to implement; broadly needed: Prolog uses it for fresh
|
||||
variable names, Common Lisp uses it constantly in macros, any hygienic macro system needs
|
||||
it, and Smalltalk uses it for anonymous class/method naming.
|
||||
|
||||
Primitives to add:
|
||||
- `gensym` `[prefix]` → unique symbol, e.g. `g42`, `var-17`. Counter-based, monotonically increasing.
|
||||
- `symbol-interned?` `s` → bool — whether the symbol is in the global intern table
|
||||
- `intern` `str` → symbol — intern a string as a symbol (string->symbol already exists; this is
|
||||
the explicit interning operation for languages that distinguish interned vs uninterned)
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `gensym` counter to evaluator state; implement in `spec/evaluator.sx`.
|
||||
`string->symbol` already exists — `gensym` is just a counter-suffixed variant.
|
||||
Added *gensym-counter*/gensym/string->symbol/symbol->string/intern/symbol-interned? to
|
||||
evaluator.sx. Added string->symbol/symbol->string transpiler renames + platform.py aliases.
|
||||
JS 2186/+1. OCaml builds. Committed edf4e525.
|
||||
- [x] OCaml: add global gensym counter; implement primitives.
|
||||
gensym_counter ref + gensym/string->symbol/symbol->string/intern/symbol-interned? in sx_primitives.ml.
|
||||
Also fixed ListRef case in seq_to_list (both sx_ref.ml + sx_primitives.ml). 4431/1080 (was 4385/1080).
|
||||
- [x] JS bootstrapper: implement.
|
||||
Already done in Spec step. JS 2186/2497, all sequence tests pass.
|
||||
- [x] Tests: 15+ tests in `spec/tests/test-gensym.sx` — uniqueness, prefix, symbol?, string->symbol round-trip.
|
||||
19 tests. OCaml 4450/1080, JS 2205/2497, zero regressions.
|
||||
- [x] Commit: `spec: gensym + symbol interning` — 0862a614
|
||||
|
||||
---
|
||||
|
||||
## Phase 13 — Character type
|
||||
|
||||
Common Lisp and Haskell have a distinct `Char` type that is not a string. Without it both
|
||||
implementations are approximations — CL's `#\a` literal and Haskell's `'a'` both need a
|
||||
real char value, not a length-1 string.
|
||||
|
||||
Primitives to add:
|
||||
- `char?` `v` → bool
|
||||
- `char->integer` `c` → Unicode codepoint integer
|
||||
- `integer->char` `n` → char
|
||||
- `char=?` `char<?` `char>?` `char<=?` `char>=?` → comparators
|
||||
- `char-ci=?` `char-ci<?` etc. → case-insensitive comparators
|
||||
- `char-alphabetic?` `char-numeric?` `char-whitespace?` → predicates
|
||||
- `char-upper-case?` `char-lower-case?` → predicates
|
||||
- `char-upcase` `char-downcase` → char → char
|
||||
- `string->list` extended to return chars (not length-1 strings)
|
||||
- `list->string` accepting chars
|
||||
|
||||
Also: `#\a` reader syntax for char literals (parser addition).
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxChar` type to evaluator; add char literal syntax `#\a`/`#\space`/`#\newline`
|
||||
to `spec/parser.sx`; implement all predicates + comparators.
|
||||
- [x] OCaml: add `SxChar of char` to `sx_types.ml`; implement primitives.
|
||||
- [x] JS bootstrapper: implement char type wrapping a codepoint integer.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-chars.sx` — literals, char->integer round-trip,
|
||||
comparators, predicates, upcase/downcase, string<->list with chars.
|
||||
- [x] Commit: `spec: character type (char? char->integer #\\a literals + predicates)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 14 — String ports
|
||||
|
||||
Needed for any language with a reader protocol: Common Lisp's `read`, Prolog's term parser,
|
||||
Smalltalk's `printString`. Without string ports these all do their own character walking
|
||||
on raw strings rather than treating a string as an I/O stream.
|
||||
|
||||
Primitives to add:
|
||||
- `open-input-string` `str` → input port
|
||||
- `open-output-string` → output port
|
||||
- `get-output-string` `port` → string (flush output port to string)
|
||||
- `input-port?` `output-port?` `port?` → predicates
|
||||
- `read-char` `[port]` → char or eof-object
|
||||
- `peek-char` `[port]` → char or eof-object (non-consuming)
|
||||
- `read-line` `[port]` → string or eof-object
|
||||
- `write-char` `char` `[port]` → void
|
||||
- `write-string` `str` `[port]` → void
|
||||
- `eof-object` → the eof sentinel
|
||||
- `eof-object?` `v` → bool
|
||||
- `close-port` `port` → void
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add port type + eof-object to evaluator; implement all primitives.
|
||||
Ports are mutable objects with a position cursor (input) or accumulation buffer (output).
|
||||
- [x] OCaml: add `SxPort` variant covering string-input-port and string-output-port;
|
||||
Buffer.t for output, string+offset for input.
|
||||
- [x] JS bootstrapper: implement port type.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-ports.sx` — open/read/peek/eof, output accumulation,
|
||||
read-line, write-char, close.
|
||||
- [x] Commit: `spec: string ports (open-input-string/open-output-string/read-char/etc)` — 3d8937d7
|
||||
|
||||
---
|
||||
|
||||
## Phase 15 — Math completeness
|
||||
|
||||
Filling specific gaps that multiple language implementations need.
|
||||
|
||||
### 15a — modulo / remainder / quotient distinction
|
||||
They differ on negative numbers — critical for Erlang `rem`, Haskell `mod`/`rem`, CL `mod`/`rem`:
|
||||
- `quotient` `a` `b` → truncate toward zero (same sign as dividend)
|
||||
- `remainder` `a` `b` → sign follows dividend (truncation division)
|
||||
- `modulo` `a` `b` → sign follows divisor (floor division) — R7RS
|
||||
|
||||
### 15b — Trigonometry and transcendentals
|
||||
Lua, Haskell, Erlang, CL all need: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `exp`,
|
||||
`log`, `sqrt`, `expt`. Check which are already present; add missing ones.
|
||||
|
||||
### 15c — GCD / LCM
|
||||
`gcd` `a` `b` → greatest common divisor; `lcm` `a` `b` → least common multiple.
|
||||
Needed by Haskell `Rational`, CL, and any language doing fraction arithmetic.
|
||||
|
||||
### 15d — Radix number parsing / formatting
|
||||
`(number->string n radix)` → e.g. `(number->string 255 16)` → `"ff"`.
|
||||
`(string->number s radix)` → e.g. `(string->number "ff" 16)` → `255`.
|
||||
Needed by: Common Lisp, Smalltalk, Erlang integer formatting.
|
||||
|
||||
Steps:
|
||||
- [x] Audit which trig / math functions are already in `spec/primitives.sx`; note gaps.
|
||||
- [x] Spec + OCaml + JS: implement missing trig (`sin`/`cos`/`tan`/`asin`/`acos`/`atan`/`exp`/`log`).
|
||||
- [x] Spec + OCaml + JS: `quotient`/`remainder`/`modulo` with correct negative semantics.
|
||||
- [x] Spec + OCaml + JS: `gcd`/`lcm`.
|
||||
- [x] Spec + OCaml + JS: radix variants of `number->string`/`string->number`.
|
||||
- [x] Tests: 40+ tests in `spec/tests/test-math.sx`.
|
||||
- [x] Commit: `spec: math completeness — trig, quotient/remainder/modulo, gcd/lcm, radix`
|
||||
|
||||
---
|
||||
|
||||
## Phase 16 — Rational numbers
|
||||
|
||||
Haskell's `Rational` type and Common Lisp ratios (`1/3`) both need this. Natural extension
|
||||
of the numeric tower (Phase 2) — rationals are the third numeric type alongside int and float.
|
||||
|
||||
Primitives to add:
|
||||
- `make-rational` `numerator` `denominator` → rational (auto-reduced by GCD)
|
||||
- `rational?` `v` → bool
|
||||
- `numerator` `r` → integer
|
||||
- `denominator` `r` → integer
|
||||
- Reader syntax: `1/3` parsed as rational literal
|
||||
- Arithmetic: `(+ 1/3 1/6)` → `1/2`; `(* 1/3 3)` → `1`; mixed int/rational → rational
|
||||
- `exact->inexact` on rational → float; `inexact->exact` on float → rational approximation
|
||||
- `(number->string 1/3)` → `"1/3"`
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxRational` type; add `n/d` reader syntax to `spec/parser.sx`; extend
|
||||
all arithmetic primitives for rational contagion (int op rational → rational, rational
|
||||
op float → float).
|
||||
- [x] OCaml: add `SxRational of int * int` (stored in reduced form); implement all arithmetic.
|
||||
as_number + safe_eq extended for cross-type rational equality (= 2.5 5/2) → true.
|
||||
- [x] JS bootstrapper: implement rational type.
|
||||
JS keeps int/int → float for CSS backward compatibility; SxRational class with _rational marker.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-rationals.sx` — literals, arithmetic, reduction,
|
||||
mixed numeric tower, exact<->inexact conversion. 62 tests, all pass.
|
||||
- [x] Commit: `spec: rational numbers — 1/3 literals, arithmetic, numeric tower integration`
|
||||
Committed 036022cc. JS: 2232 passed. OCaml: 4532 passed (+11).
|
||||
|
||||
---
|
||||
|
||||
## Phase 17 — read / write / display
|
||||
|
||||
Completes the I/O model. Builds on string ports (Phase 14) and char type (Phase 13).
|
||||
`read` parses any SX value from a port; `write` serializes with quoting (round-trippable);
|
||||
`display` serializes without quoting (human-readable). Common Lisp's `read` macro,
|
||||
Prolog term I/O, and Smalltalk's `printString` all need this.
|
||||
|
||||
Primitives to add:
|
||||
- `read` `[port]` → SX value or eof-object — full SX parser reading from a port
|
||||
- `read-char` already in Phase 14; `read` uses it internally
|
||||
- `write` `val` `[port]` → void — serializes with quotes: `"hello"`, `#\a`, `(1 2 3)`
|
||||
- `display` `val` `[port]` → void — serializes without quotes: `hello`, `a`, `(1 2 3)`
|
||||
- `newline` `[port]` → void — writes `\n`
|
||||
- `write-to-string` `val` → string — convenience: `(write val (open-output-string))`
|
||||
- `display-to-string` `val` → string — convenience
|
||||
|
||||
Steps:
|
||||
- [x] Spec: implement `read` in `spec/evaluator.sx` — wraps the existing parser to read
|
||||
one datum from a port cursor; handles eof gracefully.
|
||||
- [x] Spec: implement `write`/`display`/`newline` — extend the existing serializer for
|
||||
port output; `write` quotes strings + uses `#\` for chars, `display` does not.
|
||||
- [x] OCaml: wire `read` through port type; implement `write`/`display` output path.
|
||||
- [x] JS bootstrapper: implement.
|
||||
- [x] Tests: 25+ tests in `spec/tests/test-read-write.sx` — read string literal, read list,
|
||||
read eof, write round-trip, display vs write quoting, newline, write-to-string.
|
||||
- [x] Commit: `spec: read/write/display — S-expression reader/writer on ports`
|
||||
|
||||
---
|
||||
|
||||
## Phase 18 — Sets
|
||||
|
||||
O(1) membership testing. Distinct from hash tables (unkeyed) and lists (O(n)).
|
||||
Erlang has sets as a stdlib staple, Haskell `Data.Set`, APL uses set operations
|
||||
constantly, Common Lisp has `union`/`intersection` on lists but a native set is O(1).
|
||||
|
||||
Primitives to add:
|
||||
- `make-set` `[list]` → fresh set, optionally seeded from list
|
||||
- `set?` `v` → bool
|
||||
- `set-add!` `s` `val` → void
|
||||
- `set-member?` `s` `val` → bool
|
||||
- `set-remove!` `s` `val` → void
|
||||
- `set-size` `s` → integer
|
||||
- `set->list` `s` → list (unspecified order)
|
||||
- `list->set` `lst` → set
|
||||
- `set-union` `s1` `s2` → new set
|
||||
- `set-intersection` `s1` `s2` → new set
|
||||
- `set-difference` `s1` `s2` → new set (elements in s1 not in s2)
|
||||
- `set-for-each` `s` `fn` → iterate for side effects
|
||||
- `set-map` `s` `fn` → new set of mapped values
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add entries to `spec/primitives.sx`.
|
||||
- [x] OCaml: implement using `Hashtbl.t` with unit values (or a proper `Set` functor
|
||||
with a comparison function); add `SxSet` to `sx_types.ml`.
|
||||
- [x] JS bootstrapper: implement using JS `Set`.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-sets.sx` — add/member/remove, union/intersection/
|
||||
difference, list conversion, for-each, size.
|
||||
- [x] Commit: `spec: sets (make-set/set-add!/set-member?/union/intersection/etc)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 19 — Regular expressions as primitives
|
||||
|
||||
`lib/js/regex.sx` is a pure-SX regex engine already written. Promoting it to a primitive
|
||||
gives every language free regex without reinventing: Lua patterns, Tcl `regexp`, Ruby regex,
|
||||
JS regex, Erlang `re` module. Mostly a wiring job — the implementation exists.
|
||||
|
||||
Primitives to add:
|
||||
- `make-regexp` `pattern` `[flags]` → regexp object (`flags`: `"i"` case-insensitive, `"g"` global, `"m"` multiline)
|
||||
- `regexp?` `v` → bool
|
||||
- `regexp-match` `re` `str` → match dict `{:match "..." :start N :end N :groups (...)}` or nil
|
||||
- `regexp-match-all` `re` `str` → list of match dicts
|
||||
- `regexp-replace` `re` `str` `replacement` → string with first match replaced
|
||||
- `regexp-replace-all` `re` `str` `replacement` → string with all matches replaced
|
||||
- `regexp-split` `re` `str` → list of strings (split on matches)
|
||||
- Reader syntax: `#/pattern/flags` for regexp literals (parser addition)
|
||||
|
||||
Steps:
|
||||
- [x] Audit `lib/js/regex.sx` — understand the API it already exposes; map to the
|
||||
primitive API above.
|
||||
- [x] Spec: add `SxRegexp` type to evaluator; add `#/pattern/flags` literal syntax to
|
||||
`spec/parser.sx`; wire `lib/js/regex.sx` engine as the implementation.
|
||||
- [x] OCaml: implement using OCaml `Re` library (or `Str`); add `SxRegexp` to types.
|
||||
- [x] JS bootstrapper: use native JS `RegExp`; wrap in the primitive API.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-regexp.sx` — basic match, groups, replace,
|
||||
replace-all, split, flags (case-insensitive), no-match nil return.
|
||||
- [x] Commit: `spec: regular expressions (make-regexp/regexp-match/regexp-replace + #/pat/ literals)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 20 — Bytevectors
|
||||
|
||||
R7RS standard. Needed for WebSocket binary frames (E36), binary protocol parsing, and
|
||||
efficient string encoding. Also the foundation for proper Unicode: `string->utf8` /
|
||||
`utf8->string` require a byte array type.
|
||||
|
||||
Primitives to add:
|
||||
- `make-bytevector` `n` `[fill]` → bytevector of n bytes (fill defaults to 0)
|
||||
- `bytevector?` `v` → bool
|
||||
- `bytevector-length` `bv` → integer
|
||||
- `bytevector-u8-ref` `bv` `i` → byte 0–255
|
||||
- `bytevector-u8-set!` `bv` `i` `byte` → void
|
||||
- `bytevector-copy` `bv` `[start]` `[end]` → fresh copy
|
||||
- `bytevector-copy!` `dst` `at` `src` `[start]` `[end]` → in-place copy
|
||||
- `bytevector-append` `bv...` → concatenated bytevector
|
||||
- `utf8->string` `bv` `[start]` `[end]` → string decoded as UTF-8
|
||||
- `string->utf8` `str` `[start]` `[end]` → bytevector UTF-8 encoded
|
||||
- `bytevector->list` / `list->bytevector` → conversion
|
||||
|
||||
Steps:
|
||||
- [x] Spec: add `SxBytevector` type; implement all primitives in `spec/evaluator.sx` / `spec/primitives.sx`.
|
||||
- [x] OCaml: add `SxBytevector of bytes` to `sx_types.ml`; implement primitives using
|
||||
OCaml `Bytes`.
|
||||
- [x] JS bootstrapper: implement using `Uint8Array`.
|
||||
- [x] Tests: 30+ tests in `spec/tests/test-bytevectors.sx` — construction, ref/set, copy,
|
||||
append, utf8 round-trip, slice.
|
||||
- [x] Commit: `spec: bytevectors (make-bytevector/u8-ref/u8-set!/utf8->string/etc)`
|
||||
|
||||
---
|
||||
|
||||
## Phase 21 — format
|
||||
|
||||
CL-style string formatting beyond `str`. `(format "Hello ~a, age ~d" name age)`.
|
||||
Haskell `printf`, Erlang `io:format`, CL `format`, and general string templating all use this idiom.
|
||||
|
||||
Directives:
|
||||
- `~a` — display (no quotes)
|
||||
- `~s` — write (with quotes)
|
||||
- `~d` — decimal integer
|
||||
- `~x` — hexadecimal integer
|
||||
- `~o` — octal integer
|
||||
- `~b` — binary integer
|
||||
- `~f` — fixed-point float
|
||||
- `~e` — scientific notation float
|
||||
- `~%` — newline
|
||||
- `~&` — fresh line (newline only if not already at start of line)
|
||||
- `~~` — literal tilde
|
||||
- `~t` — tab
|
||||
|
||||
Signature: `(format template arg...)` → string.
|
||||
Optional: `(format port template arg...)` — write to port directly.
|
||||
|
||||
Steps:
|
||||
- [x] Spec: implement `format` as a pure SX function in `spec/stdlib.sx` — parses
|
||||
`~X` directives, dispatches to `display`/`write`/`number->string` as appropriate.
|
||||
Pure SX: no host calls needed. Self-hosting — uses string-buffer (Phase 5) internally.
|
||||
- [x] OCaml: expose as a primitive (or let it run as SX through the evaluator).
|
||||
Added format-decimal OCaml primitive; fixed lib/r7rs.sx number->string to support radix.
|
||||
- [x] JS bootstrapper: same.
|
||||
- [x] Tests: 28 tests in `spec/tests/test-format.sx` — each directive, multiple args,
|
||||
nested format, `~~` escape. 28/28 pass on both JS and OCaml.
|
||||
- [x] Commit: `spec: format — CL-style string formatting (~a ~s ~d ~x ~% etc)` — 4d7b3e29
|
||||
|
||||
---
|
||||
|
||||
## Phase 22 — Language sweep
|
||||
|
||||
Replace workarounds with primitives. One language per fire (or per sub-item for big ones).
|
||||
Start with blank slates (CL, APL, Ruby, Tcl) — they haven't committed to workarounds yet.
|
||||
|
||||
**Scope per language:** only `lib/<lang>/**`. Don't touch spec or other languages.
|
||||
Brief each language's loop agent (or do inline) after rebasing their branch onto architecture.
|
||||
|
||||
- [x] Restart CL/APL/Ruby/Tcl loops with updated briefing pointing to new primitives.
|
||||
Added `## SX primitive baseline` section to plans/common-lisp-on-sx.md,
|
||||
plans/apl-on-sx.md, plans/ruby-on-sx.md, plans/tcl-on-sx.md. f43659ce.
|
||||
|
||||
- [x] Common Lisp: char type (`#\a`); string ports + `read`/`write` for reader/printer;
|
||||
gensym for macros; rational numbers for CL ratios; multiple values; sets for CL set ops;
|
||||
`modulo`/`remainder`/`quotient`; radix formatting; `format` for `cl:format`.
|
||||
lib/common-lisp/runtime.sx (103 forms) + test.sh (68/68 pass). 1ad8e74a.
|
||||
|
||||
- [x] Lua: vectors for arrays; hash tables for Lua tables; `delay`/`force` for lazy iterators;
|
||||
regexp for Lua pattern matching; trig from math completeness; bytevectors for binary I/O.
|
||||
math/string/table stdlib tables + lua-force. 185/185 pass. ec3512d6.
|
||||
|
||||
- [x] Erlang: numeric tower for float/int; bitwise ops for bitmatch; multiple values for
|
||||
multi-return; sets for Erlang sets; `remainder` for `rem`; regexp for `re` module.
|
||||
lib/erlang/runtime.sx (63 forms) + test.sh (55/55 pass). 3c0a9632.
|
||||
|
||||
- [x] Haskell: numeric tower for `Num`/`Integral`/`Fractional`; promises for lazy evaluation
|
||||
(critical); multiple values for tuples; rational numbers for `Rational`; char type for
|
||||
`Char`; `gcd`/`lcm`; sets for `Data.Set`; `read`/`write` for `Show`/`Read` instances.
|
||||
lib/haskell/runtime.sx (113 forms) + tests/runtime.sx (143/143 pass). c02ffcf3.
|
||||
|
||||
- [x] JS: vectors for Array; hash tables for `Map`; sets for `Set`; bitwise ops for typed
|
||||
arrays; regexp for JS regex; bytevectors for `Uint8Array`; radix formatting.
|
||||
lib/js/stdlib.sx (36 forms) + test.sh epochs 6000-6032 (25/25 pass). COMMIT.
|
||||
|
||||
- [x] Smalltalk: vectors for `Array new:`; hash tables for `Dictionary new`; sets for
|
||||
`Set new`; char type for `Character`; string ports + `read`/`write` for `printString`.
|
||||
lib/smalltalk/runtime.sx (72 forms) + tests/runtime.sx (86/86 pass). COMMIT.
|
||||
|
||||
- [x] APL: vectors as core array type; bitwise ops for array masks; sets for APL set ops;
|
||||
sequence protocol for rank-polymorphic operations; format for APL output formatting.
|
||||
lib/apl/runtime.sx (60 forms) + tests/runtime.sx (73/73 pass). COMMIT.
|
||||
|
||||
- [x] Ruby: coroutines for fibers; hash tables for `Hash`; sets for `Set`; regexp for
|
||||
Ruby regex; string ports for `StringIO`; bytevectors for `String` binary encoding.
|
||||
lib/ruby/runtime.sx (61 forms) + tests/runtime.sx (76/76 pass). COMMIT.
|
||||
Note: rb-fiber-yield from letrec-bound lambdas fails (JIT VM can't invoke callcc
|
||||
continuations as escapes); workaround: use top-level helper fns for recursive yields.
|
||||
|
||||
- [x] Tcl: string ports for Tcl channel abstraction; string-buffer for `append`; coroutines
|
||||
for Tcl coroutines; regexp for Tcl `regexp`; format for Tcl `format`.
|
||||
lib/tcl/runtime.sx (37 forms) + tests/runtime.sx (56/56 pass). COMMIT.
|
||||
|
||||
- [x] Forth: bitwise ops (core); string-buffer for word-definition accumulation; bytevectors
|
||||
for Forth's raw memory model.
|
||||
lib/forth/runtime.sx (36 forms) + tests/runtime.sx (64/64 pass). COMMIT.
|
||||
|
||||
---
|
||||
|
||||
## Ground rules
|
||||
|
||||
- Work on the `architecture` branch in `/root/rose-ash` (main worktree).
|
||||
- Use sx-tree MCP for all `.sx` file edits. Never use raw Edit/Write/Read on `.sx` files.
|
||||
- Commit after each concrete unit of work. Never leave the branch broken.
|
||||
- Never push to `main` — only push to `origin/architecture`.
|
||||
- Update this checklist every fire: tick `[x]` done, add inline notes on blockers.
|
||||
|
||||
---
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- 2026-05-01: Phase 22 Forth done — runtime.sx (36 forms): bitwise (AND/OR/XOR/INVERT/LSHIFT/RSHIFT/2*/2//bit-count/integer-length/within + arithmetic helpers), string-buffer (emit!/type!/value/length/clear!/emit-int!), memory (cfetch/cstore/fetch/store/move!/fill!/erase!/mem->list). 64/64 tests. 8019e572.
|
||||
- 2026-05-01: Phase 22 Tcl done — runtime.sx (37 forms): string-buffer (append accumulator), channel (read/write ports with gets/read/puts), regexp (make-regexp wrappers), format (%s/%d/%f/%x/%o/%% manual char scan), coroutine (call/cc, top-level helper pattern). 56/56 tests. 3e07727d.
|
||||
- 2026-05-01: Phase 22 Ruby done — runtime.sx (61 forms): Hash (list-of-pairs dict-backed), Set (make-set, (set item) order), Regexp (make-regexp wrappers), StringIO (write buf + rewind/char read), Bytevectors (thin wrappers), Fiber (call/cc; letrec JIT workaround: use top-level helpers). 76/76 tests. 182e6f63.
|
||||
|
||||
- 2026-05-01: Phase 22 APL done — runtime.sx (60 forms): iota/rho/at, rank-polymorphic dyadic/monadic helpers, arithmetic/comparison/boolean/bitwise element-wise, reduce/scan, take/drop/rotate/compress/index, set ops (member/nub/union/intersect/without), format. 73/73 tests. COMMIT.
|
||||
- 2026-05-01: Phase 22 Smalltalk done — runtime.sx (72 forms): numeric helpers, Character (1-indexed Array backed by dict), Dictionary (list-of-pairs any-key map), Set (make-set), WriteStream/ReadStream/printString. set-member? (set item) order. 86/86 tests. COMMIT.
|
||||
- 2026-05-01: Phase 22 JS done — stdlib.sx (36 forms): bitwise (truncate not js-num-to-int; set-member? takes (set item) order), Map (dict-backed pairs), Set (SX make-set), RegExp (callable lambda). 25/25 new tests pass; total 492/585. COMMIT.
|
||||
- 2026-05-01: Phase 22 Haskell done — runtime.sx (113 forms): numeric tower (hk-div floor semantics), rational (dict GCD-normalised), hk-force (promises), Data.Char, Data.Set, Data.List, Maybe/Either, tuples, string helpers, hk-show. 148/148 tests. c02ffcf3.
|
||||
- 2026-05-01: Phase 22 Erlang done — runtime.sx (63 forms): numeric tower, bitwise (band/bor/bxor/bnot/bsl/bsr), sets, re module, list BIFs, type conversions, ok/error tuples. 55/55 tests. 3c0a9632.
|
||||
- 2026-05-01: Phase 22 Lua done — math/string/table stdlib tables + lua-force in lib/lua/runtime.sx. 185/185 tests (28 new). ec3512d6.
|
||||
- 2026-05-01: Phase 22 CL done — runtime.sx (103 forms): type preds, arithmetic, chars, format, gensym, values, sets, radix, list utils. cl-empty? guards nil/() split. 68/68 tests. 1ad8e74a.
|
||||
- 2026-05-01: Phase 22 step 1 — SX primitive baseline added to CL/APL/Ruby/Tcl plans. f43659ce.
|
||||
- 2026-05-01: Phase 21 complete — format (~a ~s ~d ~x ~o ~b ~f ~% ~& ~~ ~t) as pure SX in spec/stdlib.sx. Fixed lib/r7rs.sx number->string to support optional radix; added format-decimal OCaml primitive. 28/28 tests on both JS and OCaml. 4d7b3e29.
|
||||
- 2026-04-26: Phase 7 complete — bitwise-and/or/xor/not + arithmetic-shift + bit-count + integer-length. OCaml: land/lor/lxor/lnot/lsl/asr + Kernighan popcount + lsr loop for integer-length. JS: bitwise ops + Hamming weight + Math.clz32. 26 tests, 158 assertions, all pass. a8a79dc9.
|
||||
- 2026-04-26: Phase 6 complete — JS+Tests+Commit all ticked. JS needed no changes (spec-level forms). 40/40 ADT tests pass JS. 2032/2500 JS total (+67 vs phase-4). Phase 6 fully landed: 6c872107+0dc7e159+5d1913e7. Phase 7 (bitwise) next.
|
||||
- 2026-04-26: Phase 6 OCaml done — Dict-based ADT (no native SxAdt type needed); hand-written sf_define_type in bootstrap.py FIXUPS (skipped from transpile — &rest params + empty-dict {} literals); registered via register_special_form; step_limit/step_count added to PREAMBLE. 172 assertions pass (test-adt). Full suite 4280/1080 (was 4243/1117, +37). Committed 5d1913e7.
|
||||
- 2026-04-26: Phase 6 Spec match done — ADT case added to match-pattern in spec/evaluator.sx: checks (list? pattern)+(symbol? first)+(dict? value)+(get value :_adt), then matches :_ctor+arity and recursively binds field patterns. No-clause error now uses make-cek-value+raise-eval-frame so guard can catch it. 20 new match tests pass; 40/40 total ADT tests green. Zero regressions.
|
||||
- 2026-04-26: Phase 6 Spec define-type done — sf-define-type registered via register-special-form! in spec/evaluator.sx; AdtValue as {:_adt true :_type "..." :_ctor "..." :_fields (list ...)}; ctor fns + arity checking + Name?/Ctor? predicates + Ctor-field accessors; *adt-registry* dict populated per define-type call. 20/20 JS tests pass in spec/tests/test-adt.sx. OCaml define-type is next task.
|
||||
- 2026-04-26: Phase 6 Design done — plans/designs/sx-adt.md written. Covers define-type/match syntax, AdtValue CEK runtime, stepSfDefineType+MatchFrame dispatch, exhaustiveness warnings, recursive types, nested patterns, wildcard _. 3-phase impl plan. Next fire: Spec implement define-type.
|
||||
- 2026-04-26: Phase 5 complete — string buffer fully landed (d98b5fa2). 17 tests, 17/17 OCaml+JS. Phase 6 (ADTs) next.
|
||||
- 2026-04-26: Phase 5 Spec+OCaml+JS step done — StringBuffer of Buffer.t in sx_types.ml; make-string-buffer/append!/->string/length/string-buffer? in sx_primitives.ml; SxStringBuffer with _string_buffer marker + typeOf/dict? fixes in platform.py; JS rebuilt. 17/17 tests OCaml+JS.
|
||||
- 2026-04-26: Phase 4 complete — coroutine primitive fully landed (4 commits: spec library + OCaml verified + JS pre-load + 27 tests). Phase 5 (string buffer) next.
|
||||
- 2026-04-26: Phase 4 Tests step done — 27 tests total (10 new: state field inspection, yield-from-helper, initial-arg-ignored, mutable-closure, complex-values, round-robin, factory-no-state, non-coroutine-error). 27/27 OCaml+JS.
|
||||
- 2026-04-26: Phase 4 JS step done — all CEK primitives already in sx-browser.js; fix was pre-loading spec/coroutines.sx+spec/signals.sx in run_tests.js so (import (sx coroutines)) resolves synchronously. 17/17 coroutine tests pass JS. 1965/2500 total (+25), zero new failures.
|
||||
- 2026-04-26: Phase 4 OCaml step done — no native SxCoroutine type needed; existing cek-step-loop/cek-resume/perform/make-cek-state primitives in run_tests.ml fully support the spec/coroutines.sx library. 284/284 pass (coroutines+vectors+numeric-tower+dynamic-wind), zero regressions.
|
||||
- 2026-04-26: Phase 4 Spec step done — spec/coroutines.sx define-library with make-coroutine/coroutine-resume/coroutine-yield/coroutine?/coroutine-alive?; make-coroutine stub in evaluator.sx; 17/17 coroutine tests pass (OCaml). Key insight: coroutine body must use (define loop (fn...)) + (loop 0) not named let — named let uses cek_call→cek_run which errors on IO suspension.
|
||||
- 2026-05-01: Phase 10 complete — mutable hash tables. HashTable variant in OCaml; JS Map-based SxHashTable. 11 primitives: make-hash-table/hash-table?/set!/ref/delete!/size/keys/values/->alist/for-each/merge!. 28 tests, all pass OCaml+JS. 133bdf52.
|
||||
- 2026-05-01: Phase 9 complete — delay/force/delay-force/make-promise/promise?. Dict-based promise {:_promise :forced :thunk :value}; :_iterative flag for delay-force chain following. 25/25 tests OCaml (4357) and JS (2109). Committed e44cb89a.
|
||||
- 2026-05-01: Phase 8 complete — values/call-with-values/let-values/define-values. Dict marker {:_values true :_list [...]} (no new type). step-sf-define desugars shorthand (define (f x) body) on both hosts. 25/25 tests OCaml+JS. Committed 43cc1d90.
|
||||
- 2026-04-26: Phase 3 complete — OCaml+JS done. CallccContinuation gains winders-depth int; make_callcc_continuation/callcc_continuation_winders_len wired; wind-after/wind-return CekFrame fields fixed (cf_f=after-thunk, cf_extra=winders-len, cf_name=body-result); get_val + transpiler.sx updated. 8/8 dynamic-wind tests pass on OCaml; 235/235 (callcc+guard+do+r7rs) zero regressions. Committed 6602ec8c.
|
||||
- 2026-04-26: Phase 3 Spec+Tests done — dynamic-wind CEK implementation: wind-after/wind-return frames, *winders* stack, kont-unwind-to-handler, wind-escape-to. callcc frame stores winders-len in continuation; callcc-continuation? calls wind-escape-to before escape. 8/8 dynamic-wind tests pass (normal return, raise, call/cc, nested LIFO, guard ordering). 1948/2500 JS (+8). Zero regressions. Committed a9d5a108.
|
||||
- 2026-04-26: Phase 2 complete — Verify+Commit done. OCaml 4874/394, JS 1940/2500 (+60). No regressions. 6 JS-only failures are float≡int platform-inherent. Phase 2 fully landed across 4 commits.
|
||||
- 2026-04-26: Phase 2 JS bootstrapper done — integer?/float?/exact?/inexact? added (Number.isInteger); truncate/remainder/modulo/random-int/exact->inexact/inexact->exact/parse-number added. Fixed sx_server.ml epoch+blob+io-response protocol for Integer type. JS: 1940/2500 (+60). OCaml: 4874/394 baseline. 6 JS tests fail (JS float≡int platform limit). Committed b12a22e6.
|
||||
- 2026-04-26: Phase 2 Spec done — integer?/float? predicates added to spec/primitives.sx; floor/ceil/truncate :returns updated to "integer"; / to "float"; exact->inexact/inexact->exact docs and returns updated; float contagion documented on +/-/*; 4874/394 baseline. Committed 45ec5535.
|
||||
- 2026-04-26: Phase 2 OCaml+Tests done — `Integer of int` / `Number of float` in sx_types.ml; float contagion across all arithmetic; floor/truncate/round → Integer; integer?/float?/exact?/inexact?/exact->inexact/inexact->exact; 92/92 numeric tower tests pass; 4874 total (394 pre-existing unchanged). Committed c70bbdeb.
|
||||
- 2026-04-26: Phase 1 complete — JS step done. Fixed fundamental lambda binding bug (index-of on arrays returned -1 not NIL, making bind-lambda-params mis-fire &rest branch). Added _lastErrorKont_/hostError/try-catch stubs. 42/42 vector tests pass. 1847 std / 2362 full passing (up from 5). Committed.
|
||||
- 2026-04-25: Phase 1 spec step done — all 10 vector primitives in spec/primitives.sx have full :as type annotations, :returns, :doc; make-vector optional fill param added.
|
||||
- 2026-04-25: Phase 1 OCaml step done — bounds-checked vector-ref/set!, vector-copy now accepts optional start/end, spec/primitives.sx doc updated. 10/10 r7rs vector tests pass, 4747 total (394 pre-existing hs-upstream fails unchanged).
|
||||
- 2026-04-25: Phase 0 complete — stopped CL/APL/Ruby/Tcl loops (all 4 idle at shell); confirmed E38 (tokenizer :end/:line) and E39 (WebWorker stub) both have implementation commits.
|
||||
- 2026-05-01: Phase 20 complete — bytevectors. SxBytevector of bytes in OCaml using Bytes; Uint8Array-backed SxBytevector in JS. 12 primitives: make-bytevector, bytevector?, bytevector-length, bytevector-u8-ref, bytevector-u8-set!, bytevector-copy, bytevector-copy!, bytevector-append, utf8->string, string->utf8, bytevector->list, list->bytevector. 32 tests, all pass. JS 2535, OCaml 4725. a3811545.
|
||||
- 2026-05-01: Phase 19 complete — regular expressions. SxRegexp(src,flags,Re.re) in OCaml via Re.Pcre; SxRegexp wrapper around JS RegExp. 9 primitives: make-regexp, regexp?, regexp-source, regexp-flags, regexp-match, regexp-match-all, regexp-replace, regexp-replace-all, regexp-split. Match dicts with :match/:start/:end/:groups. 32 tests, all pass. JS 2503, OCaml 4693. d8d5588e.
|
||||
- 2026-05-01: Phase 18 complete — sets. SxSet as (string,value) Hashtbl keyed by inspect(val) in OCaml; Map keyed by write-to-string in JS. 13 primitives: make-set, set?, set-add!, set-member?, set-remove!, set-size, set->list, list->set, set-union, set-intersection, set-difference, set-for-each, set-map. 33 tests, all pass. JS 2469, OCaml 4659. 3b0ac67a.
|
||||
- 2026-05-01: Phase 17 complete — read/write/display. OCaml: sx_write_val/sx_display_val helpers; read via Sx_parser.read_value with #t/#f and N/D rational support added to parser; postprocess ()→Nil. JS: sxReadNormalize (#t/#f→true/false), sxReadConvert (()→NIL), sxEq list equality, sxWriteVal symbol/keyword name fix (v.name not v._sym), readerMacroGet registry. 42 tests (test-read-write.sx), all pass both hosts. JS 2436, OCaml 4626. 7d329f02.
|
||||
- 2026-05-01: Phase 16 complete — rational numbers. SxRational type in OCaml (Rational of int*int, reduced, denom>0) and JS (SxRational class, _rational marker). n/d reader in spec/parser.sx. Arithmetic contagion: int op rational → rational, rational op float → float. JS keeps int/int → float for CSS compat. OCaml as_number+safe_eq extended for cross-type rational equality. 62 tests in test-rationals.sx, all pass. JS 2232, OCaml 4532 (+11). 036022cc.
|
||||
- 2026-05-01: Phase 15 complete — math completeness. stdlib.math module: sin/cos/tan/asin/acos/atan(1-2 args)/exp/log/expt/quotient/gcd/lcm/number->string(radix)/string->number(radix). OCaml atan updated for optional 2nd arg. Strict radix parsing in JS string->number. 44 tests in test-math.sx, all pass. JS 2311/4801, OCaml 4547/5629. be2b11ac.
|
||||
- 2026-05-01: Phase 14 OCaml done — Eof + Port{PortInput/PortOutput} in sx_types.ml; 15 port primitives in sx_primitives.ml; raw_serialize updated; 4532/4532 (+39, zero regressions). 8ba0a33f.
|
||||
- 2026-05-01: Phase 14 Spec+JS+Tests+Commit done — port type {_port,_kind,_source/_buffer,_pos,_closed}; eof singleton; 15 primitives in spec/primitives.sx (stdlib.ports) + platform.py; 39/39 tests in test-ports.sx. Committed 3d8937d7. OCaml step next.
|
||||
- 2026-05-01: Phase 13 OCaml done — Char of int in sx_types.ml; #\ reader in sx_parser.ml; all char primitives in sx_primitives.ml; fixed get_val for Integer n list indexing (was Number-only); fixed raw_serialize for Integer/Char. 4493/4493 (+43, zero regressions). b939becd.
|
||||
- 2026-05-01: Phase 13 Spec+JS+Tests+Commit done — SxChar tagged {_char,codepoint}; char? char->integer integer->char char-upcase/downcase; 10 comparators (ordered+ci); 5 predicates; string->list/list->string as platform primitives; #\a #\space #\newline reader syntax in spec/parser.sx; js-char-renames dict in transpiler.sx; 43/43 tests pass JS (2254/4745). Committed 4b600f17. OCaml step next.
|
||||
- 2026-05-01: Phase 12 complete — gensym + symbol interning. gensym_counter/gensym/string->symbol/symbol->string/intern/symbol-interned? in spec + OCaml + JS. Fixed ListRef case in seq_to_list (both hosts). 19 tests, all pass. OCaml 4450/1080, JS 2205/2497. Commits: edf4e525 Spec, 0862a614 OCaml+Tests.
|
||||
- 2026-05-01: Phase 11 complete — sequence protocol done. Commits: da4b526a Spec, 7286629c OCaml, 06a3eee1 JS, 0fe00bf7 Tests. JS 2185/+48, OCaml 4424/+39.
|
||||
- 2026-05-01: Phase 11 Tests done — 45 tests in test-sequences.sx all passing (JS 2185/+48, OCaml 4424/+39). Fixed vector? rename, vectorLength/vectorRef/reverse aliases, in-range letrec→build-range, sequence-length nil, assert-equal for lists. Committed 0fe00bf7.
|
||||
- 2026-05-01: Phase 11 JS bootstrapper step done — confirmed sx-browser.js current (built in Spec step da4b526a); 19 sequence primitive refs in output; 2137/2500 JS tests passing.
|
||||
- 2026-05-01: Phase 11 OCaml step done — seq_to_list helper added before let-rec; ho_setup_dispatch wraps all 7 coll bindings with seq_to_list; seq-to-list/sequence-to-list/to-vector/length/ref/append + in-range primitives in sx_primitives.ml. 4385/4385 baseline unchanged, 0 regressions. Committed 7286629c.
|
||||
- 2026-05-01: Phase 11 Spec step done — seq-to-list coercion helper; ho-setup-dispatch extended with seqToList on all collection args; sequence-to-list/vector/length/ref/append + in-range added to evaluator.sx. Restored 3 accidentally-deleted make-cek-state/value/suspended definitions. Fixed 8 shorthand define forms + added vector->list/list->vector transpiler renames. JS: 2137 passing (+28 vs HEAD baseline of 2109).
|
||||
@@ -11,7 +11,7 @@ isolation: worktree
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/prolog-on-sx.md`. You run in an isolated git worktree. You work the plan's roadmap forever, one commit per feature. You never push.
|
||||
You are the sole background agent working `/root/rose-ash/plans/prolog-on-sx.md`. You run in an isolated git worktree. You work the plan's roadmap forever, one commit per feature. Push to `origin/loops/prolog` after every commit.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
@@ -39,12 +39,13 @@ Every iteration: implement → test → commit → tick `[ ]` in plan → append
|
||||
|
||||
## Ground rules (hard)
|
||||
|
||||
- **Scope:** only `lib/prolog/**` and `plans/prolog-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. Prolog primitives go in `lib/prolog/runtime.sx`.
|
||||
- **Scope:** only `lib/prolog/**` and `plans/prolog-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib/<lang>/` dirs, `lib/stdlib.sx`, or `lib/` root. Prolog primitives go in `lib/prolog/runtime.sx`. You may **read** `lib/hyperscript/runtime.sx` to understand the hook API but do not edit it — `hs-set-prolog-hook!` is already implemented there.
|
||||
- **Hyperscript bridge is NOT blocked:** `lib/prolog/hs-bridge.sx` already exists and `lib/hyperscript/runtime.sx` already exports `hs-set-prolog-hook!` / `hs-prolog-hook`. The Phase 5 DSL item just needs tests and wiring.
|
||||
- **NEVER call `sx_build`.** 600s watchdog will kill you before OCaml finishes. If sx_server binary is broken, add Blockers entry and stop.
|
||||
- **Shared-file issues** → plan's Blockers section with a minimal repro. Don't fix them.
|
||||
- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5 (IO suspension via `perform`/`cek-resume`). `sx_summarise` spec/evaluator.sx first — it's 2300+ lines.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits. Never `Edit`/`Read`/`Write` on `.sx`.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Worktree:** commit, then push to `origin/loops/prolog`. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
- **If blocked** for two iterations on the same issue, add to Blockers and move on.
|
||||
|
||||
@@ -11,7 +11,7 @@ isolation: worktree
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/tcl-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push.
|
||||
You are the sole background agent working `/root/rose-ash/plans/tcl-on-sx.md`. Isolated worktree, forever, one commit per feature. Push to `origin/loops/tcl` after every commit.
|
||||
|
||||
## Restart baseline — check before iterating
|
||||
|
||||
@@ -42,7 +42,7 @@ Every iteration: implement → test → commit → tick `[ ]` → Progress log
|
||||
- **Shared-file issues** → plan's Blockers with minimal repro.
|
||||
- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits.
|
||||
- **Worktree:** commit locally. Never push. Never touch `main`.
|
||||
- **Worktree:** commit, then push to `origin/loops/tcl`. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit.
|
||||
- **Plan file:** update Progress log + tick boxes every commit.
|
||||
|
||||
|
||||
@@ -104,6 +104,16 @@ Core mapping:
|
||||
- [ ] Drive corpus to 100+ green
|
||||
- [ ] Idiom corpus — `lib/apl/tests/idioms.sx` covering classic Roger Hui / Phil Last idioms
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
@@ -50,71 +50,102 @@ Core mapping:
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — reader + parser
|
||||
- [ ] Tokenizer: symbols (with package qualification `pkg:sym` / `pkg::sym`), numbers (int, float, ratio `1/3`, `#xFF`, `#b1010`, `#o17`), strings `"…"` with `\` escapes, characters `#\Space` `#\Newline` `#\a`, comments `;`, block comments `#| … |#`
|
||||
- [ ] Reader: list, dotted pair, quote `'`, function `#'`, quasiquote `` ` ``, unquote `,`, splice `,@`, vector `#(…)`, uninterned `#:foo`, nil/t literals
|
||||
- [ ] Parser: lambda lists with `&optional` `&rest` `&key` `&aux` `&allow-other-keys`, defaults, supplied-p variables
|
||||
- [ ] Unit tests in `lib/common-lisp/tests/read.sx`
|
||||
- [x] Tokenizer: symbols (with package qualification `pkg:sym` / `pkg::sym`), numbers (int, float, ratio `1/3`, `#xFF`, `#b1010`, `#o17`), strings `"…"` with `\` escapes, characters `#\Space` `#\Newline` `#\a`, comments `;`, block comments `#| … |#`
|
||||
- [x] Reader: list, dotted pair, quote `'`, function `#'`, quasiquote `` ` ``, unquote `,`, splice `,@`, vector `#(…)`, uninterned `#:foo`, nil/t literals
|
||||
- [x] Parser: lambda lists with `&optional` `&rest` `&key` `&aux` `&allow-other-keys`, defaults, supplied-p variables
|
||||
- [x] Unit tests in `lib/common-lisp/tests/read.sx`
|
||||
|
||||
### Phase 2 — sequential eval + special forms
|
||||
- [ ] `cl-eval-ast`: `quote`, `if`, `progn`, `let`, `let*`, `flet`, `labels`, `setq`, `setf` (subset), `function`, `lambda`, `the`, `locally`, `eval-when`
|
||||
- [ ] `block` + `return-from` via captured continuation
|
||||
- [ ] `tagbody` + `go` via per-tag continuations
|
||||
- [ ] `unwind-protect` cleanup frame
|
||||
- [ ] `multiple-value-bind`, `multiple-value-call`, `multiple-value-prog1`, `values`, `nth-value`
|
||||
- [ ] `defun`, `defparameter`, `defvar`, `defconstant`, `declaim`, `proclaim` (no-op)
|
||||
- [ ] Dynamic variables — `defvar`/`defparameter` produce specials; `let` rebinds via parameterize-style scope
|
||||
- [ ] 60+ tests in `lib/common-lisp/tests/eval.sx`
|
||||
- [x] `cl-eval-ast`: `quote`, `if`, `progn`, `let`, `let*`, `flet`, `labels`, `setq`, `setf` (subset), `function`, `lambda`, `the`, `locally`, `eval-when`
|
||||
- [x] `block` + `return-from` via captured continuation
|
||||
- [x] `tagbody` + `go` via per-tag continuations
|
||||
- [x] `unwind-protect` cleanup frame
|
||||
- [x] `multiple-value-bind`, `multiple-value-call`, `multiple-value-prog1`, `values`, `nth-value`
|
||||
- [x] `defun`, `defparameter`, `defvar`, `defconstant`, `declaim`, `proclaim` (no-op)
|
||||
- [x] Dynamic variables — `defvar`/`defparameter` produce specials; `let` rebinds via parameterize-style scope
|
||||
- [x] 182 tests in `lib/common-lisp/tests/eval.sx`
|
||||
|
||||
### Phase 3 — conditions + restarts (THE SHOWCASE)
|
||||
- [ ] `define-condition` — class hierarchy rooted at `condition`/`error`/`warning`/`simple-error`/`simple-warning`/`type-error`/`arithmetic-error`/`division-by-zero`
|
||||
- [ ] `signal`, `error`, `cerror`, `warn` — all walk the handler chain
|
||||
- [ ] `handler-bind` — non-unwinding handlers, may decline by returning normally
|
||||
- [ ] `handler-case` — unwinding handlers (delcc abort)
|
||||
- [ ] `restart-case`, `with-simple-restart`, `restart-bind`
|
||||
- [ ] `find-restart`, `invoke-restart`, `invoke-restart-interactively`, `compute-restarts`
|
||||
- [ ] `with-condition-restarts` — associate restarts with a specific condition
|
||||
- [ ] `*break-on-signals*`, `*debugger-hook*` (basic)
|
||||
- [ ] Classic programs in `lib/common-lisp/tests/programs/`:
|
||||
- [ ] `restart-demo.lisp` — division with `:use-zero` and `:retry` restarts
|
||||
- [ ] `parse-recover.lisp` — parser with skipped-token restart
|
||||
- [ ] `interactive-debugger.lisp` — ASCII REPL using `:debugger-hook`
|
||||
- [ ] `lib/common-lisp/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
- [x] `define-condition` — class hierarchy rooted at `condition`/`error`/`warning`/`simple-error`/`simple-warning`/`type-error`/`arithmetic-error`/`division-by-zero`
|
||||
- [x] `signal`, `error`, `cerror`, `warn` — all walk the handler chain
|
||||
- [x] `handler-bind` — non-unwinding handlers, may decline by returning normally
|
||||
- [x] `handler-case` — unwinding handlers (call/cc escape)
|
||||
- [x] `restart-case`, `with-simple-restart`, `restart-bind`
|
||||
- [x] `find-restart`, `invoke-restart`, `compute-restarts`
|
||||
- [x] `with-condition-restarts` — associate restarts with a specific condition
|
||||
- [x] `invoke-restart-interactively`, `*break-on-signals*`, `*debugger-hook*` (basic)
|
||||
- [x] Classic programs in `lib/common-lisp/tests/programs/`:
|
||||
- [x] `restart-demo.sx` — division with `use-zero` and `retry` restarts (7 tests)
|
||||
- [x] `parse-recover.sx` — parser with skipped-token restart (6 tests)
|
||||
- [x] `interactive-debugger.sx` — policy-driven debugger hook, *debugger-hook* global (7 tests)
|
||||
- [x] `lib/common-lisp/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` (363 total tests)
|
||||
|
||||
### Phase 4 — CLOS
|
||||
- [ ] `defclass` with `:initarg`/`:initform`/`:accessor`/`:reader`/`:writer`/`:allocation`
|
||||
- [ ] `make-instance`, `slot-value`, `(setf slot-value)`, `with-slots`, `with-accessors`
|
||||
- [ ] `defgeneric` with `:method-combination` (standard, plus `+`, `and`, `or`)
|
||||
- [ ] `defmethod` with `:before` / `:after` / `:around` qualifiers
|
||||
- [ ] `call-next-method` (continuation), `next-method-p`
|
||||
- [ ] `class-of`, `find-class`, `slot-boundp`, `change-class` (basic)
|
||||
- [ ] Multiple dispatch — method specificity by argument-class precedence list
|
||||
- [ ] Built-in classes registered for tagged values (`integer`, `float`, `string`, `symbol`, `cons`, `null`, `t`)
|
||||
- [ ] Classic programs:
|
||||
- [ ] `geometry.lisp` — `intersect` generic dispatching on (point line), (line line), (line plane)…
|
||||
- [ ] `mop-trace.lisp` — `:before` + `:after` printing call trace
|
||||
- [x] `defclass` with `:initarg`/`:initform`/`:accessor`/`:reader`/`:writer`/`:allocation`
|
||||
- [x] `make-instance`, `slot-value`, `(setf slot-value)`, `with-slots`, `with-accessors`
|
||||
- [x] `defgeneric` with `:method-combination` (standard, plus `+`, `and`, `or`)
|
||||
- [x] `defmethod` with `:before` / `:after` / `:around` qualifiers
|
||||
- [x] `call-next-method` (continuation), `next-method-p`
|
||||
- [x] `class-of`, `find-class`, `slot-boundp`, `change-class` (basic)
|
||||
- [x] Multiple dispatch — method specificity by argument-class precedence list
|
||||
- [x] Built-in classes registered for tagged values (`integer`, `float`, `string`, `symbol`, `cons`, `null`, `t`)
|
||||
- [x] Classic programs:
|
||||
- [x] `geometry.sx` — `intersect` generic dispatching on (point line), (line line), (line plane) — 12 tests
|
||||
- [x] `mop-trace.sx` — `:before` + `:after` printing call trace — 13 tests
|
||||
|
||||
### Phase 5 — macros + LOOP + reader macros
|
||||
- [ ] `defmacro`, `macrolet`, `symbol-macrolet`, `macroexpand-1`, `macroexpand`
|
||||
- [ ] `gensym`, `gentemp`
|
||||
- [ ] `set-macro-character`, `set-dispatch-macro-character`, `get-macro-character`
|
||||
- [ ] **The LOOP macro** — iteration drivers (`for … in/across/from/upto/downto/by`, `while`, `until`, `repeat`), accumulators (`collect`, `append`, `nconc`, `count`, `sum`, `maximize`, `minimize`), conditional clauses (`if`/`when`/`unless`/`else`), termination (`finally`/`thereis`/`always`/`never`), `named` blocks
|
||||
- [ ] LOOP test corpus: 30+ tests covering all clause types
|
||||
- [x] `defmacro`, `macrolet`, `symbol-macrolet`, `macroexpand-1`, `macroexpand`
|
||||
- [x] `gensym`, `gentemp`
|
||||
- [x] `set-macro-character`, `set-dispatch-macro-character`, `get-macro-character`
|
||||
- [x] **The LOOP macro** — iteration drivers (`for … in/across/from/upto/downto/by`, `while`, `until`, `repeat`), accumulators (`collect`, `append`, `nconc`, `count`, `sum`, `maximize`, `minimize`), conditional clauses (`if`/`when`/`unless`/`else`), termination (`finally`/`thereis`/`always`/`never`), `named` blocks
|
||||
- [x] LOOP test corpus: 27 tests covering all clause types
|
||||
|
||||
### Phase 6 — packages + stdlib drive
|
||||
- [ ] `defpackage`, `in-package`, `export`, `use-package`, `import`, `find-package`
|
||||
- [ ] Package qualification at the reader level — `cl:car`, `mypkg::internal`
|
||||
- [ ] `:common-lisp` (`:cl`) and `:common-lisp-user` (`:cl-user`) packages
|
||||
- [ ] Sequence functions — `mapcar`, `mapc`, `mapcan`, `reduce`, `find`, `find-if`, `position`, `count`, `every`, `some`, `notany`, `notevery`, `remove`, `remove-if`, `subst`
|
||||
- [ ] List ops — `assoc`, `getf`, `nth`, `last`, `butlast`, `nthcdr`, `tailp`, `ldiff`
|
||||
- [ ] String ops — `string=`, `string-upcase`, `string-downcase`, `subseq`, `concatenate`
|
||||
- [ ] FORMAT — basic directives `~A`, `~S`, `~D`, `~F`, `~%`, `~&`, `~T`, `~{...~}` (iteration), `~[...~]` (conditional), `~^` (escape), `~P` (plural)
|
||||
- [ ] Drive corpus to 200+ green
|
||||
- [x] `defpackage`, `in-package`, `export`, `use-package`, `import`, `find-package`
|
||||
- [x] Package qualification at the reader level — `cl:car`, `mypkg::internal`
|
||||
- [x] `:common-lisp` (`:cl`) and `:common-lisp-user` (`:cl-user`) packages
|
||||
- [x] Sequence functions — `mapcar`, `mapc`, `mapcan`, `reduce`, `find`, `find-if`, `position`, `count`, `every`, `some`, `notany`, `notevery`, `remove`, `remove-if`, `subst`
|
||||
- [x] List ops — `assoc`, `getf`, `nth`, `last`, `butlast`, `nthcdr`, `tailp`, `ldiff`
|
||||
- [x] String ops — `string=`, `string-upcase`, `string-downcase`, `subseq`, `concatenate`
|
||||
- [x] FORMAT — basic directives `~A`, `~S`, `~D`, `~F`, `~%`, `~&`, `~T`, `~{...~}` (iteration), `~[...~]` (conditional), `~^` (escape), `~P` (plural)
|
||||
- [x] Drive corpus to 200+ green
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
- 2026-05-05: Phase 5 set-macro-character — cl-reader-macros + cl-dispatch-macros global dicts; SET-MACRO-CHARACTER/GET-MACRO-CHARACTER/SET-DISPATCH-MACRO-CHARACTER dispatch in eval.sx (stores fn, doesn't wire into reader — stubs sufficient to avoid errors). Phase 5 fully ticked. Phase 6 Drive corpus 200+ ticked (518 total, 54 stdlib). All roadmap items done.
|
||||
|
||||
- 2026-05-05: Phase 6 packages — defpackage/in-package/export/use-package/import/find-package/package-name; cl-packages dict, cl-current-package; cl-package-sep? strips pkg: prefix from symbols+functions; package-qualified calls (cl:car, cl:mapcar) work. 4 package tests added; 518 total tests, 0 failed.
|
||||
|
||||
- 2026-05-05: Phase 6 FORMAT — cl-fmt-a/cl-fmt-s/cl-fmt-find-close/cl-fmt-iterate/cl-fmt-loop in eval.sx; ~A/~S/~D/~F/~%/~&/~T/~P/~{...~}/~[...~]/~^/~~; also fixed substr(start,length) semantics throughout (SUBSEQ, cl-fmt-loop); 6 FORMAT tests added to stdlib.sx; 514 total tests, 0 failed.
|
||||
|
||||
- 2026-05-05: Phase 6 stdlib — sequence functions (mapc/mapcan/reduce/find/find-if/find-if-not/position/position-if/count/count-if/every/some/notany/notevery/remove/remove-if/remove-if-not/subst/member), list ops (assoc/rassoc/getf/last/butlast/nthcdr/copy-list/list*/caar/cadr/cdar/cddr/caddr/cadddr/pairlis), string ops (subseq/string/char/string-length/string</>), plus coerce/make-list/write-to-string; 44 tests in tests/stdlib.sx; Phase 6 sequence+list+string boxes ticked. Total: 508 tests, 0 failed.
|
||||
|
||||
- 2026-05-05: Phase 4 CLOS fully complete — `lib/common-lisp/clos.sx` (27 forms): clos-class-registry (8 built-in classes), defclass/make-instance/slot-value/slot-boundp/set-slot-value!/find-class/change-class, defgeneric/defmethod with :before/:after/:around, clos-call-generic (standard method combination: sort by specificity, fire befores, call primary chain, fire afters in reverse), call-next-method/next-method-p, with-slots, accessor installation; 41 tests in `tests/clos.sx`; classic programs `geometry.sx` (12 tests, multi-dispatch intersect on P/L/Plane) and `mop-trace.sx` (13 tests, :before/:after tracing). Dynamic variables in eval.sx: cl-apply-dyn saves/restores global bindings around let for specials (cl-mark-special!/cl-special?/cl-dyn-unbound). Key gotchas: qualifier strings are "before"/"after"/"around" (no colon); dict-set pure = assoc; dict->list = (map (fn (k) (list k (get d k))) (keys d)); clos-add-reader-method bootstrapped via set! after defmethod defined; test isolation: use unique var names to avoid *y* collision. 437 total tests, 0 failed.
|
||||
- 2026-05-05: Phase 3 fully complete — conformance.sh runner + scoreboard.json/scoreboard.md; 363 total tests across all suites (79 reader, 31 parser, 174 eval, 59 conditions, 7+6+7 classic programs).
|
||||
- 2026-05-05: Phase 3 complete — cl-debugger-hook/cl-invoke-debugger in runtime.sx (cl-error routes through hook), cl-break-on-signals (fires hook before handlers on type match), cl-invoke-restart-interactively (calls fn with no args); 4 new tests (147 total). Phase 3 all boxes ticked.
|
||||
- 2026-05-05: Phase 3 interactive-debugger.sx — cl-debugger-hook global, cl-invoke-debugger, cl-error-with-debugger, make-policy-debugger; 7 tests (143 total). Tests wired into test.sh program suite runner. Phase 3 condition core complete.
|
||||
- 2026-05-05: Phase 3 classic programs — `tests/programs/restart-demo.sx` (7 tests: safe-divide with use-zero + retry restarts) and `tests/programs/parse-recover.sx` (6 tests: token parser with skip-token + use-zero restarts, handler-case abort). Key gotcha: use `=` not `equal?` for list comparison in sx_server.
|
||||
- 2026-05-05: Phase 3 conditions + restarts — `cl-condition-classes` hierarchy (15 types), `cl-condition?`/`cl-condition-of-type?`, `cl-make-condition`, `cl-define-condition`, `cl-signal`/`cl-error`/`cl-warn`/`cl-cerror`, `cl-handler-bind` (non-unwinding), `cl-handler-case` (call/cc escape), `cl-restart-case`/`cl-with-simple-restart`, `cl-find-restart`/`cl-invoke-restart`/`cl-compute-restarts`, `cl-with-condition-restarts`; 55 new tests in `tests/conditions.sx` (123 total runtime tests). Key gotcha: `cl-condition-classes` must be captured at define-time via `let` in `cl-condition-of-type?` — free-variable lookup at call-time fails through env_merge parent chain.
|
||||
- 2026-05-05: multiple values — VALUES returns {:cl-type "mv"} wrapper for 2+ values; cl-mv-primary/cl-mv-vals helpers; MULTIPLE-VALUE-BIND binds vars to value list; MULTIPLE-VALUE-CALL/PROG1/NTH-VALUE; cl-mv-primary applied in IF/AND/OR/COND/cl-call-fn for single-value contexts; 15 new tests (174 eval, 346 total green).
|
||||
- 2026-05-05: unwind-protect — cl-eval-unwind-protect: eval protected form, run cleanup with for-each (discards results, preserves original sentinel), return original result; 8 new tests (159 eval, 331 total green).
|
||||
- 2026-05-05: tagbody + go — cl-go-tag? sentinel; cl-eval-tagbody runs body with tag-index map (keys str-normalised for integer tags); go-tag propagation in cl-eval-body alongside block-return; 11 new tests (151 eval, 323 total green).
|
||||
- 2026-05-05: block + return-from — sentinel propagation in cl-eval-body; cl-eval-block catches matching sentinels; BLOCK/RETURN-FROM/RETURN dispatch in cl-eval-list; 13 new tests (140 eval, 312 total green). Parser: CL strings → {:cl-type "string"} dicts.
|
||||
- 2026-04-25: Phase 2 eval — 127 tests, 299 total green. `lib/common-lisp/eval.sx`: cl-eval-ast with quote/if/progn/let/let*/flet/labels/setq/setf/function/lambda/the/locally/eval-when; defun/defvar/defparameter/defconstant; built-in arithmetic (+/-/*//, min/max/abs/evenp/oddp), comparisons, predicates, list ops (car/cdr/cons/list/append/reverse/length/nth/first/second/third/rest), string ops, funcall/apply/mapcar. Key gotchas: SX reduce is (reduce fn init list) not (reduce fn list init); CL true literal is t not true; builtins registered in cl-global-env.fns via wrapper dicts for #' syntax.
|
||||
- 2026-04-25: Phase 1 lambda-list parser — 31 new tests, 172 total green. `cl-parse-lambda-list` in `parser.sx` + `tests/lambda.sx`. Handles &optional/&rest/&body/&key/&aux/&allow-other-keys, defaults, supplied-p. Key gotchas: `(when (> (len items) 0) ...)` not `(when items ...)` (empty list is truthy); custom `cl-deep=` needed for dict/list structural equality in tests.
|
||||
- 2026-04-25: Phase 1 reader/parser — 62 new tests, 141 total green. `lib/common-lisp/parser.sx`: cl-read/cl-read-all, lists, dotted pairs, quote/backquote/unquote/splice/#', vectors, #:uninterned, NIL→nil, T→true, reader macro wrappers.
|
||||
- 2026-04-25: Phase 1 tokenizer — 79 tests green. `lib/common-lisp/reader.sx` + `tests/read.sx` + `test.sh`. Handles symbols (pkg:sym, pkg::sym), integers, floats, ratios, hex/binary/octal, strings, #\ chars, reader macros (#' #( #: ,@), line/block comments. Key gotcha: SX `str` for string concat (not `concat`), substring-based read-while.
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
145
plans/datalog-on-sx.md
Normal file
145
plans/datalog-on-sx.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Datalog-on-SX: Datalog on the CEK/VM
|
||||
|
||||
Datalog is a declarative query language: a restricted subset of Prolog with no function
|
||||
symbols, only relations. Programs are sets of facts and rules; queries ask what follows.
|
||||
Evaluation is bottom-up (fixpoint iteration) rather than Prolog's top-down DFS — which
|
||||
means no infinite loops, guaranteed termination, and efficient incremental updates.
|
||||
|
||||
The unique angle: Datalog is a natural companion to the Prolog implementation already in
|
||||
progress (`lib/prolog/`). The parser and term representation can share infrastructure;
|
||||
the evaluator is an entirely different fixpoint engine rather than a DFS solver.
|
||||
|
||||
End-state goal: **full core Datalog** (facts, rules, stratified negation, aggregation,
|
||||
recursion) with a clean SX query API, and a demonstration of Datalog as a query engine
|
||||
for rose-ash data (e.g. federation graph, content relationships).
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/datalog/**` and `plans/datalog-on-sx.md`. Do **not** edit
|
||||
`spec/`, `hosts/`, `shared/`, `lib/prolog/**`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Datalog source → term AST → fixpoint evaluator. No transpiler to SX AST —
|
||||
the evaluator is written in SX and works directly on term structures.
|
||||
- **Reference:** Ramakrishnan & Ullman "A Survey of Deductive Database Systems";
|
||||
Dalmau "Datalog and Constraint Satisfaction".
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Datalog source text
|
||||
│
|
||||
▼
|
||||
lib/datalog/tokenizer.sx — atoms, variables, numbers, strings, punct (?- :- , . ( ) [ ])
|
||||
│
|
||||
▼
|
||||
lib/datalog/parser.sx — facts: atom(args). rules: head :- body. queries: ?- goal.
|
||||
│ No function symbols (only constants and variables in args).
|
||||
▼
|
||||
lib/datalog/db.sx — extensional DB (EDB): ground facts; IDB: derived relations;
|
||||
│ clause index by relation name/arity
|
||||
▼
|
||||
lib/datalog/eval.sx — bottom-up fixpoint: semi-naive evaluation with delta sets;
|
||||
│ stratification for negation; incremental update API
|
||||
▼
|
||||
lib/datalog/query.sx — query API: (datalog-query db goal) → list of substitutions;
|
||||
SX embedding: define facts/rules as SX data directly
|
||||
```
|
||||
|
||||
Key differences from Prolog:
|
||||
- **No function symbols** — args are atoms, numbers, strings, or variables only. No `f(a,b)`.
|
||||
- **No cuts** — no procedural control.
|
||||
- **Bottom-up** — derive all consequences of all rules before answering; no search tree.
|
||||
- **Termination guaranteed** — no infinite derivation chains (no function symbols → finite Herbrand base).
|
||||
- **Stratified negation** — `not(P)` legal iff P does not recursively depend on its own negation.
|
||||
- **Aggregation** — `count`, `sum`, `min`, `max` over derived tuples (Datalog+).
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: atoms (lowercase/quoted), variables (uppercase/`_`), numbers, strings,
|
||||
operators (`:- `, `?-`, `,`, `.`), comments (`%`, `/* */`)
|
||||
Note: no function symbol syntax (no nested `f(...)` in arg position).
|
||||
- [ ] Parser:
|
||||
- Facts: `parent(tom, bob).` → `{:head (parent tom bob) :body ()}`
|
||||
- Rules: `ancestor(X,Z) :- parent(X,Y), ancestor(Y,Z).`
|
||||
→ `{:head (ancestor X Z) :body ((parent X Y) (ancestor Y Z))}`
|
||||
- Queries: `?- ancestor(tom, X).` → `{:query (ancestor tom X)}`
|
||||
- Negation: `not(parent(X,Y))` in body position → `{:neg (parent X Y)}`
|
||||
- [ ] Tests in `lib/datalog/tests/parse.sx`
|
||||
|
||||
### Phase 2 — unification + substitution
|
||||
- [ ] Share or port unification from `lib/prolog/` — term walk, occurs check off by default
|
||||
- [ ] `dl-unify` `t1` `t2` `subst` → extended subst or nil (no function symbols means simpler)
|
||||
- [ ] `dl-ground?` `term` → bool — all variables bound in substitution
|
||||
- [ ] Tests: atom/atom, var/atom, var/var, list args
|
||||
|
||||
### Phase 3 — extensional DB + naive evaluation
|
||||
- [ ] EDB: `{:relation-name → set-of-ground-tuples}` using SX sets (Phase 18 of primitives)
|
||||
- [ ] `dl-add-fact!` `db` `relation` `args` → add ground tuple
|
||||
- [ ] `dl-add-rule!` `db` `head` `body` → add rule clause
|
||||
- [ ] Naive evaluation: iterate rules until fixpoint
|
||||
For each rule, for each combination of body tuples that unify, derive head tuple.
|
||||
Repeat until no new tuples added.
|
||||
- [ ] `dl-query` `db` `goal` → list of substitutions satisfying goal against derived DB
|
||||
- [ ] Tests: transitive closure (ancestor), sibling, same-generation — classic Datalog programs
|
||||
|
||||
### Phase 4 — semi-naive evaluation (performance)
|
||||
- [ ] Delta sets: track newly derived tuples per iteration
|
||||
- [ ] Semi-naive rule: only join against delta tuples from last iteration, not full relation
|
||||
- [ ] Significant speedup for recursive rules — avoids re-deriving known tuples
|
||||
- [ ] `dl-stratify` `db` → dependency graph + SCC analysis → stratum ordering
|
||||
- [ ] Tests: verify semi-naive produces same results as naive; benchmark on large ancestor chain
|
||||
|
||||
### Phase 5 — stratified negation
|
||||
- [ ] Dependency graph analysis: which relations depend on which (positively or negatively)
|
||||
- [ ] Stratification check: error if negation is in a cycle (non-stratifiable program)
|
||||
- [ ] Evaluation: process strata in order — lower stratum fully computed before using its
|
||||
complement in a higher stratum
|
||||
- [ ] `not(P)` in rule body: at evaluation time, check P is NOT in the derived EDB
|
||||
- [ ] Tests: non-member (`not(member(X,L))`), colored-graph (`not(same-color(X,Y))`),
|
||||
stratification error detection
|
||||
|
||||
### Phase 6 — aggregation (Datalog+)
|
||||
- [ ] `count(X, Goal)` → number of distinct X satisfying Goal
|
||||
- [ ] `sum(X, Goal)` → sum of X values satisfying Goal
|
||||
- [ ] `min(X, Goal)` / `max(X, Goal)` → min/max of X satisfying Goal
|
||||
- [ ] `group-by` semantics: `count(X, sibling(bob, X))` → count of bob's siblings
|
||||
- [ ] Aggregation breaks stratification — evaluate in a separate post-fixpoint pass
|
||||
- [ ] Tests: social network statistics, grade aggregation, inventory sums
|
||||
|
||||
### Phase 7 — SX embedding API
|
||||
- [ ] `(dl-program facts rules)` → database from SX data directly (no parsing required)
|
||||
```
|
||||
(dl-program
|
||||
'((parent tom bob) (parent tom liz) (parent bob ann))
|
||||
'((ancestor X Z :- (parent X Y) (ancestor Y Z))
|
||||
(ancestor X Y :- (parent X Y))))
|
||||
```
|
||||
- [ ] `(dl-query db '(ancestor tom ?X))` → `((ann) (bob) (liz) (pat))`
|
||||
- [ ] `(dl-assert! db '(parent ann pat))` → incremental fact addition + re-derive
|
||||
- [ ] `(dl-retract! db '(parent tom bob))` → fact removal + re-derive from scratch
|
||||
- [ ] Integration demo: federation graph query — `(ancestor actor1 actor2)` over
|
||||
rose-ash ActivityPub follow relationships
|
||||
|
||||
### Phase 8 — Datalog as a query language for rose-ash
|
||||
- [ ] Schema: map SQLAlchemy model relationships to Datalog EDB facts
|
||||
(e.g. `(follows user1 user2)`, `(authored user post)`, `(tagged post tag)`)
|
||||
- [ ] Loader: `dl-load-from-db!` — query PostgreSQL, populate Datalog EDB
|
||||
- [ ] Query examples:
|
||||
- `?- ancestor(me, X), authored(X, Post), tagged(Post, cooking).`
|
||||
→ posts about cooking by people I follow (transitively)
|
||||
- `?- popular(Post) :- tagged(Post, T), count(L, (liked(L, Post))) >= 10.`
|
||||
→ posts with 10+ likes
|
||||
- [ ] Expose as a rose-ash service endpoint: `POST /internal/datalog` with program + query
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
80
plans/designs/f-breakpoint.md
Normal file
80
plans/designs/f-breakpoint.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# F-Breakpoint — `breakpoint` command (+2)
|
||||
|
||||
**Suite:** `hs-upstream-breakpoint`
|
||||
**Target:** Both tests are `SKIP (untranslated)`.
|
||||
|
||||
## 1. The 2 tests
|
||||
|
||||
- `parses as a top-level command`
|
||||
- `parses inside an event handler`
|
||||
|
||||
Both are untranslated — no test body exists. The test names say "parses" — these are parser tests, not runtime tests.
|
||||
|
||||
## 2. What upstream checks
|
||||
|
||||
From `test/core/breakpoint.js`:
|
||||
|
||||
```js
|
||||
it('parses as a top-level command', () => {
|
||||
expect(() => _hyperscript.evaluate("breakpoint")).not.toThrow();
|
||||
});
|
||||
it('parses inside an event handler', () => {
|
||||
const el = document.createElement('div');
|
||||
el.setAttribute('_', 'on click breakpoint');
|
||||
expect(() => _hyperscript.processNode(el)).not.toThrow();
|
||||
});
|
||||
```
|
||||
|
||||
Both tests verify that `breakpoint` is accepted by the parser without throwing. Neither test checks that the debugger actually fires. `breakpoint` is a no-op command in production builds — it calls `debugger` in JS, which is a no-op when devtools are closed.
|
||||
|
||||
## 3. What's needed
|
||||
|
||||
### Parser (`lib/hyperscript/parser.sx`)
|
||||
|
||||
Add `breakpoint` to the command dispatch — it should parse as a zero-argument command. The parser's command `cond` (wherever `add`, `remove`, `hide` etc. are dispatched) needs a branch:
|
||||
|
||||
```
|
||||
((= val "breakpoint") (hs-parse-breakpoint))
|
||||
```
|
||||
|
||||
`hs-parse-breakpoint` just returns a `{:cmd "breakpoint"}` AST node (or however commands are represented). It consumes no additional tokens.
|
||||
|
||||
### Compiler (`lib/hyperscript/compiler.sx`)
|
||||
|
||||
Add a compiler branch for `breakpoint` AST node. Emits a no-op or a `debugger` statement equivalent. Since we're in SX (not JS), a no-op `(do nil)` is correct.
|
||||
|
||||
### Generator (`tests/playwright/generate-sx-tests.py`)
|
||||
|
||||
The 2 tests are simple — hand-write them:
|
||||
|
||||
```lisp
|
||||
(deftest "parses as a top-level command"
|
||||
(let ((result (guard (e (true false))
|
||||
(hs-compile "breakpoint")
|
||||
true)))
|
||||
(assert result)))
|
||||
|
||||
(deftest "parses inside an event handler"
|
||||
(hs-cleanup!)
|
||||
(let ((el (dom-create-element "div")))
|
||||
(dom-set-attr el "_" "on click breakpoint")
|
||||
(let ((result (guard (e (true false))
|
||||
(hs-activate! el)
|
||||
true)))
|
||||
(assert result))))
|
||||
```
|
||||
|
||||
## 4. Implementation checklist
|
||||
|
||||
1. `sx_find_all` in `lib/hyperscript/parser.sx` for the command dispatch `cond`.
|
||||
2. Add `breakpoint` branch → `hs-parse-breakpoint` function returning minimal command node.
|
||||
3. `sx_find_all` in `lib/hyperscript/compiler.sx` for command compilation dispatch.
|
||||
4. Add `breakpoint` branch → emit no-op.
|
||||
5. Replace 2 `SKIP` bodies in `spec/tests/test-hyperscript-behavioral.sx` with translated tests above.
|
||||
6. Run `hs_test_run suite="hs-upstream-breakpoint"` — expect 2/2.
|
||||
7. Run smoke 0–195 — no regressions.
|
||||
8. Commit: `HS: breakpoint command — parser + no-op compiler (+2)`
|
||||
|
||||
## 5. Risk
|
||||
|
||||
Very low. Zero-argument no-op command. The only risk is mis-locating the command dispatch branch in the parser.
|
||||
68
plans/designs/f1-null-safety.md
Normal file
68
plans/designs/f1-null-safety.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# F1 — Null Safety Reporting (+7)
|
||||
|
||||
**Suite:** `hs-upstream-core/runtimeErrors`
|
||||
**Target:** 7 currently-failing tests (decrement, default, increment, put, remove, settle, transition commands)
|
||||
|
||||
## 1. Failing tests
|
||||
|
||||
The suite has 18 tests total; 11 already pass. The 7 failures all share the pattern:
|
||||
|
||||
```
|
||||
Expected '#doesntExist' is null, got
|
||||
```
|
||||
|
||||
The `eval-hs-error` helper already exists (landed in null-safety piece 1). It compiles and runs a HS snippet and returns the error string. The problem is that the listed commands don't guard against null targets before operating, so they produce no error (or a cryptic one) instead of `"'#doesntExist' is null"`.
|
||||
|
||||
| Test | Command | Null target expression |
|
||||
|------|---------|----------------------|
|
||||
| decrement | `decrement #doesntExist's innerHTML` | `#doesntExist` |
|
||||
| default | `default #doesntExist's innerHTML to 'foo'` | `#doesntExist` |
|
||||
| increment | `increment #doesntExist's innerHTML` | `#doesntExist` |
|
||||
| put | `put 'foo' into/before/after/at start of/at end of #doesntExist` | `#doesntExist` |
|
||||
| remove | `remove .foo/.@foo/#doesntExist from #doesntExist` | `#doesntExist` |
|
||||
| settle | `settle #doesntExist` | `#doesntExist` |
|
||||
| transition | `transition #doesntExist's *visibility to 0` | `#doesntExist` |
|
||||
|
||||
Note: add, hide, measure, send, sets, show, toggle, trigger already pass — they already guard.
|
||||
|
||||
## 2. Required error format
|
||||
|
||||
```
|
||||
'#doesntExist' is null
|
||||
```
|
||||
|
||||
The apostrophe-quoted selector string followed by ` is null`. The selector text is the original source text of the element expression (e.g. `#doesntExist`, not a stringified DOM node).
|
||||
|
||||
This is the same format already used by passing commands. The null-safety piece 1 commit added `eval-hs-error` and `hs-null-error` helper — just need to call it at the right point in each missing command.
|
||||
|
||||
## 3. Where to add guards
|
||||
|
||||
All in `lib/hyperscript/runtime.sx`. Pattern for each command:
|
||||
|
||||
```
|
||||
(when (nil? target)
|
||||
(hs-null-error target-source-text))
|
||||
```
|
||||
|
||||
Where `hs-null-error` (or equivalent) raises with the formatted message.
|
||||
|
||||
### Per-command location
|
||||
|
||||
- **decrement / increment** — after resolving the target element, before reading/writing innerHTML
|
||||
- **default** — after resolving target element, before reading current value
|
||||
- **put** — after resolving destination element (covers all put variants: into, before, after, at start, at end)
|
||||
- **remove** — after resolving the `from` target element
|
||||
- **settle** — after resolving target element, before starting transition poll
|
||||
- **transition** — after resolving target element, before reading/setting style
|
||||
|
||||
## 4. Implementation checklist
|
||||
|
||||
1. Find each failing command's runtime function in `lib/hyperscript/runtime.sx` using `sx_find_all`.
|
||||
2. For each: `sx_read_subtree` on the function body, locate where target is resolved, insert null guard calling `hs-null-error` (or the equivalent raise form already used by passing commands).
|
||||
3. After all 7: run `hs_test_run suite="hs-upstream-core/runtimeErrors"` — expect 18/18.
|
||||
4. Run smoke range 0–195 — expect no regressions.
|
||||
5. Commit: `HS: null-safety guards on decrement/default/increment/put/remove/settle/transition (+7)`
|
||||
|
||||
## 5. Risk
|
||||
|
||||
Low. The pattern is established by the 11 already-passing tests. The only risk is finding the correct point in each command where the element is resolved and before it's first used.
|
||||
166
plans/designs/f13-step-limit-and-meta.md
Normal file
166
plans/designs/f13-step-limit-and-meta.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# F13 — Step Limit + `meta.caller` (+5 → 100%)
|
||||
|
||||
Five tests currently timeout or produce wrong values due to two root causes:
|
||||
step budget exhaustion and a missing `meta` implementation.
|
||||
|
||||
## Tests
|
||||
|
||||
| # | Suite | Test | Failure |
|
||||
|---|-------|------|---------|
|
||||
| 198 | `hs-upstream-core/runtime` | `has proper stack from event handler` | wrong-value: `meta.caller` returns `""` instead of an object with `.meta.feature.type = "onFeature"` |
|
||||
| 200 | `hs-upstream-core/runtime` | `hypertrace is reasonable` | TIMEOUT (15s, step limit) |
|
||||
| 615 | `hs-upstream-expressions/in` | `query template returns values` | TIMEOUT (37s, step limit) |
|
||||
| 1197 | `hs-upstream-repeat` | `repeat forever works` | TIMEOUT (step limit) |
|
||||
| 1198 | `hs-upstream-repeat` | `repeat forever works w/o keyword` | TIMEOUT (step limit) |
|
||||
|
||||
---
|
||||
|
||||
## Root cause A — Step limit (tests 200, 615, 1197, 1198)
|
||||
|
||||
The runner sets `HS_STEP_LIMIT=200000`. Every CEK step consumed by any
|
||||
expression in a test — including the double compilation warm-up guard blocks
|
||||
that appear before the actual DOM test — counts against this shared budget.
|
||||
|
||||
### `repeat forever` (1197, 1198)
|
||||
|
||||
The loop body terminates in exactly **5 iterations** (`if retVal == 5 then return`).
|
||||
This is bounded, not infinite. The step budget is exhausted before the loop
|
||||
runs because two `eval-expr-cek` compilation warm-up calls each consume tens
|
||||
of thousands of steps.
|
||||
|
||||
Fix: each warm-up guard compiles and discards a HS function definition. Those
|
||||
calls are defensive (wrapped in `guard` that swallows errors). We do NOT need
|
||||
to run the compiled code — the warm-up's purpose is just to ensure the
|
||||
compiler doesn't crash, not to consume steps. The step counter should not tick
|
||||
during compilation (compilation is a pure transform, not evaluation). If that's
|
||||
impractical to gate, raise `HS_STEP_LIMIT` to `2000000` (10×).
|
||||
|
||||
### `hypertrace is reasonable` (200)
|
||||
|
||||
Defines `bar()` → calls `baz()` → throws. Simple call chain. The "hypertrace"
|
||||
in the test name implies the HS runtime trace recorder is active during the
|
||||
test. If trace recording is on globally, every CEK step generates a trace entry
|
||||
allocation. Fix: confirm whether trace recording is always-on in the test runner
|
||||
and disable it by default (trace should only be on when explicitly requested).
|
||||
Alternatively raise step limit.
|
||||
|
||||
### `query template returns values` (615)
|
||||
|
||||
Uses `<${"p"}/>` — a CSS query selector built from a template string. Takes 37
|
||||
seconds. Likely the template selector evaluation triggers repeated DOM scanning
|
||||
or expensive string construction per step. Fix: profile with `hs_test_run
|
||||
verbose=true` to identify which step is slow. If it's a regex compilation
|
||||
per-call, cache it. If step limit only, raise to 2M.
|
||||
|
||||
### Unified fix: raise `HS_STEP_LIMIT` to `2000000`
|
||||
|
||||
The simplest fix that unblocks all four timeout tests. In
|
||||
`tests/hs-run-filtered.js`, change the default step limit. Per-test overrides
|
||||
can still be set via `HS_STEP_LIMIT` env var for debugging.
|
||||
|
||||
If the `query template` test is still slow at 2M steps (37s × 10 = 370s, which
|
||||
would be unacceptable), that test needs a separate performance fix — cache the
|
||||
compiled regex/query from the template string rather than rebuilding it on every
|
||||
access.
|
||||
|
||||
---
|
||||
|
||||
## Root cause B — `meta.caller` not implemented (test 198)
|
||||
|
||||
The HS `meta` object is available inside any function call. It exposes:
|
||||
|
||||
- `meta.caller` — the calling context object
|
||||
- `meta.caller.meta.feature.type` — the HS feature type of the caller
|
||||
(e.g. `"onFeature"` when called from an `on click` handler)
|
||||
|
||||
Test script:
|
||||
```
|
||||
def bar()
|
||||
log meta.caller
|
||||
return meta.caller
|
||||
end
|
||||
```
|
||||
Triggered via `on click put bar().meta.feature.type into my.innerHTML`.
|
||||
Expects `"onFeature"` in innerHTML. Currently gets `""`.
|
||||
|
||||
### What `meta` needs
|
||||
|
||||
`meta` is a dict-like object injected into every function's execution context
|
||||
at call time. Minimum fields for this test:
|
||||
|
||||
```
|
||||
meta = {
|
||||
:caller <the calling context — a dict with its own :meta field>
|
||||
:element <the element the script is attached to>
|
||||
}
|
||||
```
|
||||
|
||||
`meta.caller.meta.feature.type` must return `"onFeature"` when called from an
|
||||
`on` event handler. The feature type string `"onFeature"` is already used
|
||||
internally (event handler features are tagged with this type).
|
||||
|
||||
### Implementation
|
||||
|
||||
In `lib/hyperscript/runtime.sx`, at the point where a HS `def` function is
|
||||
called:
|
||||
|
||||
1. Build a `meta` dict:
|
||||
```
|
||||
{:caller calling-context :element current-element}
|
||||
```
|
||||
where `calling-context` is the current runtime context dict (which includes
|
||||
its own `:meta` field with `:feature {:type "onFeature"}` for event handlers).
|
||||
|
||||
2. Bind `meta` in the function's execution env.
|
||||
|
||||
3. Ensure event handler contexts carry `{:meta {:feature {:type "onFeature"}}}`.
|
||||
|
||||
This is an additive change — nothing currently uses `meta`, so no regression
|
||||
risk.
|
||||
|
||||
---
|
||||
|
||||
## Implementation checklist
|
||||
|
||||
### Step A — Raise step limit
|
||||
1. In `tests/hs-run-filtered.js`, change default `HS_STEP_LIMIT` from `200000`
|
||||
to `2000000`.
|
||||
2. Run tests 1197–1198: `hs_test_run(start=1197, end=1199)` — expect 2/2.
|
||||
3. Run test 615: `hs_test_run(start=615, end=616)` — expect 1/1 or note if
|
||||
still too slow.
|
||||
4. Run test 200: `hs_test_run(start=200, end=201)` — expect 1/1.
|
||||
|
||||
### Step B — `meta.caller` (test 198)
|
||||
5. `sx_find_all` in `lib/hyperscript/runtime.sx` for where `def` functions are
|
||||
called / where event handler contexts are constructed.
|
||||
6. Add `meta` dict construction at call time; bind in function env.
|
||||
7. Ensure `on` handler context carries `{:meta {:feature {:type "onFeature"}}}`.
|
||||
8. Run test 198: `hs_test_run(start=198, end=199)` — expect 1/1.
|
||||
|
||||
### Step C — Query template performance (if still slow after step A)
|
||||
9. Profile `hs_test_run(start=615, end=616, step_limit=2000000, verbose=true)`.
|
||||
10. If the CSS template query `<${"p"}/>` rebuilds on every call, add a memoize
|
||||
cache keyed on the template result string.
|
||||
11. Rerun — expect < 5s.
|
||||
|
||||
### Step D — Full suite verification
|
||||
12. Run all ranges with raised step limit:
|
||||
- `hs_test_run(start=0, end=201, step_limit=2000000)`
|
||||
- `hs_test_run(start=201, end=616, step_limit=2000000)`
|
||||
- `hs_test_run(start=616, end=1200, step_limit=2000000)`
|
||||
- `hs_test_run(start=1200, end=1496, step_limit=2000000)`
|
||||
13. Confirm all previously-passing tests still pass.
|
||||
14. Commit: `HS: raise step limit to 2M + meta.caller for onFeature stack (+5)`
|
||||
|
||||
---
|
||||
|
||||
## Risk
|
||||
|
||||
- **Step limit raise:** May make test suite slower overall (more steps to exhaust
|
||||
before timeout). But if tests pass quickly the limit is never reached.
|
||||
The 37s query-template test is the only real concern — if it genuinely needs
|
||||
2M steps × (time per step), it needs a performance fix too.
|
||||
- **`meta.caller`:** Additive binding in function scope. Zero regression risk.
|
||||
The only complexity is constructing the right shape for the calling context
|
||||
chain — but since only one test exercises this and the shape is simple, the
|
||||
risk is low.
|
||||
81
plans/designs/f2-tell.md
Normal file
81
plans/designs/f2-tell.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# F2 — `tell` Semantics Fix (+3)
|
||||
|
||||
**Suite:** `hs-upstream-tell`
|
||||
**Target:** 3 failing tests out of 10. 7 already pass.
|
||||
|
||||
## 1. Failing tests
|
||||
|
||||
### "attributes refer to the thing being told"
|
||||
```
|
||||
on click tell #d2 then put @foo into me
|
||||
```
|
||||
d2 has attribute `foo="bar"`. After click, d1's text content should be `"bar"`.
|
||||
`@foo` is an attribute ref — it should resolve against the **told element** (d2), not the event target (d1).
|
||||
Currently gets `""` — attribute resolves against d1, which has no `foo` attribute.
|
||||
|
||||
### "your symbol represents the thing being told"
|
||||
```
|
||||
on click tell #d2 then put your innerText into me
|
||||
```
|
||||
d2 has innerText `"foo"`. After click, d1's text content should be `"foo"`.
|
||||
`your` is the possessive of `you` — inside a `tell` block, `you`/`your` should bind to the told element.
|
||||
Currently gets `""`.
|
||||
|
||||
### "does not overwrite the me symbol"
|
||||
```
|
||||
on click add .foo then tell #d2 then add .bar to me
|
||||
```
|
||||
After click: d1 should have both `.foo` and `.bar`; d2 should have neither.
|
||||
`me` inside the `tell` block must still refer to d1 (the original event target).
|
||||
Currently: assertion fails — `.bar` is going to d2 instead of d1.
|
||||
|
||||
## 2. What the 7 passing tests reveal about current behaviour
|
||||
|
||||
The passing tests include:
|
||||
- `you symbol represents the thing being told` — `add .bar to you` adds to d2 ✓
|
||||
- `establishes a proper beingTold symbol` — bare `add .bar` (no target) adds to the told element ✓
|
||||
- `restores a proper implicit me symbol` — after `tell` block ends, bare commands target d1 again ✓
|
||||
- `yourself attribute also works` — `remove yourself` inside tell removes d2 ✓
|
||||
|
||||
So `you`, `yourself`, and bare implicit target all work. The three bugs are:
|
||||
1. Attribute refs (`@foo`) don't resolve against the told element
|
||||
2. `your` (possessive of `you`) doesn't resolve
|
||||
3. `me` is being rebound to the told element instead of kept as d1
|
||||
|
||||
## 3. Root cause analysis
|
||||
|
||||
Inside a `tell X` block, the runtime sets the implicit target to X. The three failures suggest:
|
||||
|
||||
**Bug A — attribute refs:** `@foo` resolves via a property-access path that reads from the *current event target* (`me`/`self`), not from the *implicit tell target*. The tell block sets implicit target but the attribute ref lookup skips it.
|
||||
|
||||
**Bug B — `your`:** `your` is parsed as a possessive modifier expecting `you` to be bound. If `you` is not bound in the tell scope (and only the implicit target is set), `your X` fails to resolve.
|
||||
|
||||
**Bug C — `me` rebinding:** The tell command saves/restores `me` but the save/restore is either not happening or is restoring the wrong value. `me` inside the block should remain d1 while the implicit default target is d2.
|
||||
|
||||
## 4. Fix
|
||||
|
||||
In `lib/hyperscript/runtime.sx`, find the `tell` command handler (search for `hs-tell` or the tell dispatch branch).
|
||||
|
||||
The correct semantics:
|
||||
- Save current `me` value
|
||||
- Set implicit target (used by bare commands like `add .bar`) to the told element
|
||||
- Bind `you` = told element (so `you`, `your`, `yourself` work)
|
||||
- Do **not** rebind `me` — keep it as the original event target
|
||||
- Restore implicit target and unbind `you` after the block
|
||||
|
||||
For attribute refs (`@foo`): resolve against the current *implicit target* (told element), not against `me`. Find where `@attr` expressions are evaluated and ensure they read from the implicit target when inside a tell block.
|
||||
|
||||
## 5. Implementation checklist
|
||||
|
||||
1. `sx_find_all` in `lib/hyperscript/runtime.sx` for tell handler.
|
||||
2. `sx_read_subtree` on the tell handler — verify save/restore of `me` vs implicit target.
|
||||
3. Fix `me` rebinding: save old implicit target, set new one, do NOT touch `me`.
|
||||
4. Bind `you`/`your`/`yourself` to told element in the tell scope env.
|
||||
5. Find attribute ref (`@`) evaluation — ensure it reads from implicit target.
|
||||
6. Run `hs_test_run suite="hs-upstream-tell"` — expect 10/10.
|
||||
7. Run smoke 0–195 — no regressions.
|
||||
8. Commit: `HS: tell — fix me rebinding, your/attribute-ref resolution (+3)`
|
||||
|
||||
## 6. Risk
|
||||
|
||||
Medium. The 7 passing tests constrain what can change — the fix must preserve `you`, `yourself`, bare implicit target, and restore-after-tell semantics. The three bugs are independent enough that they can be fixed one at a time and verified after each.
|
||||
128
plans/designs/f5-cookies.md
Normal file
128
plans/designs/f5-cookies.md
Normal file
@@ -0,0 +1,128 @@
|
||||
# F5 — Cookie API (+5)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/cookies`
|
||||
**Target:** All 5 tests are `SKIP (untranslated)`.
|
||||
|
||||
## 1. The 5 tests
|
||||
|
||||
From upstream `test/expressions/cookies.js`:
|
||||
|
||||
| Test | What it checks |
|
||||
|------|---------------|
|
||||
| `length is 0 when no cookies are set` | `cookies.length == 0` with no cookies set |
|
||||
| `basic set cookie values work` | `set cookies.name to "value"` then `cookies.name == "value"` |
|
||||
| `update cookie values work` | set, then set again, value updates |
|
||||
| `basic clear cookie values work` | `set cookies.name to "value"` then `clear cookies.name`, then `cookies.name == undefined` |
|
||||
| `iterate cookies values work` | `for name in cookies` iterates cookie names |
|
||||
|
||||
## 2. HyperScript cookie syntax
|
||||
|
||||
`cookies` is a special global expression in HyperScript backed by `document.cookie`. The upstream implementation wraps `document.cookie` in a proxy:
|
||||
|
||||
- `cookies.name` → read cookie by name (returns string or `undefined`)
|
||||
- `set cookies.name to val` → write cookie (sets `document.cookie = "name=val"`)
|
||||
- `clear cookies.name` → delete cookie (sets max-age=-1)
|
||||
- `cookies.length` → number of cookies set
|
||||
- `for name in cookies` → iterate over cookie names
|
||||
|
||||
## 3. Test runner mock
|
||||
|
||||
All 5 tests are untranslated — no SX test bodies exist yet. The generator needs patterns for the cookie expressions, and `hs-run-filtered.js` needs a `document.cookie` mock.
|
||||
|
||||
### Mock in `tests/hs-run-filtered.js`
|
||||
|
||||
Add a simple in-memory cookie store to the `dom` mock:
|
||||
|
||||
```js
|
||||
let _cookieStore = {};
|
||||
Object.defineProperty(global.document, 'cookie', {
|
||||
get() {
|
||||
return Object.entries(_cookieStore)
|
||||
.map(([k,v]) => `${k}=${v}`)
|
||||
.join('; ');
|
||||
},
|
||||
set(str) {
|
||||
const [pair, ...attrs] = str.split(';');
|
||||
const [name, val] = pair.split('=').map(s => s.trim());
|
||||
const maxAge = attrs.find(a => a.trim().startsWith('max-age='));
|
||||
if (maxAge && parseInt(maxAge.split('=')[1]) < 0) {
|
||||
delete _cookieStore[name];
|
||||
} else {
|
||||
_cookieStore[name] = val;
|
||||
}
|
||||
},
|
||||
configurable: true
|
||||
});
|
||||
```
|
||||
|
||||
Add `_cookieStore = {}` reset to `hs-cleanup!` equivalent in the runner.
|
||||
|
||||
## 4. SX runtime additions in `lib/hyperscript/runtime.sx`
|
||||
|
||||
HS needs a `cookies` special expression that the compiler resolves. Two approaches:
|
||||
|
||||
**Option A (simpler):** Treat `cookies` as a built-in variable bound to a proxy dict at runtime. When property access `cookies.name` is evaluated, dispatch to cookie read/write helpers.
|
||||
|
||||
**Option B (upstream-faithful):** Parse `cookies` as a special primary expression, emit runtime calls `hs-cookie-get`, `hs-cookie-set`, `hs-cookie-delete`, `hs-cookie-length`, `hs-cookie-names`.
|
||||
|
||||
Option A is less invasive. The runtime env gets a `cookies` binding pointing to a special object; property access and assignment on it dispatch to the cookie helpers, which call `(platform-cookie-get name)` / `(platform-cookie-set name val)` / `(platform-cookie-delete name)`.
|
||||
|
||||
Platform cookie operations map to `document.cookie` reads/writes in JS.
|
||||
|
||||
## 5. Generator patterns (`tests/playwright/generate-sx-tests.py`)
|
||||
|
||||
The upstream tests use patterns like:
|
||||
|
||||
```js
|
||||
await page.evaluate(() => { _hyperscript.evaluate("set cookies.foo to 'bar'") });
|
||||
expect(await page.evaluate(() => _hyperscript.evaluate("cookies.foo"))).toBe("bar");
|
||||
```
|
||||
|
||||
In our SX harness these become direct `eval-hs` calls. Since all 5 tests are untranslated, hand-write them rather than extending the generator (similar to E39).
|
||||
|
||||
## 6. Translated test bodies
|
||||
|
||||
```lisp
|
||||
(deftest "length is 0 when no cookies are set"
|
||||
(hs-cleanup!)
|
||||
(assert= (eval-hs "cookies.length") 0))
|
||||
|
||||
(deftest "basic set cookie values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.foo to 'bar'")
|
||||
(assert= (eval-hs "cookies.foo") "bar"))
|
||||
|
||||
(deftest "update cookie values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.foo to 'bar'")
|
||||
(eval-hs "set cookies.foo to 'baz'")
|
||||
(assert= (eval-hs "cookies.foo") "baz"))
|
||||
|
||||
(deftest "basic clear cookie values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.foo to 'bar'")
|
||||
(eval-hs "clear cookies.foo")
|
||||
(assert= (eval-hs "cookies.foo") nil))
|
||||
|
||||
(deftest "iterate cookies values work"
|
||||
(hs-cleanup!)
|
||||
(eval-hs "set cookies.a to '1'")
|
||||
(eval-hs "set cookies.b to '2'")
|
||||
(let ((names (eval-hs "for name in cookies collect name")))
|
||||
(assert (contains? names "a"))
|
||||
(assert (contains? names "b"))))
|
||||
```
|
||||
|
||||
## 7. Implementation checklist
|
||||
|
||||
1. Add cookie mock to `tests/hs-run-filtered.js`. Wire reset into test cleanup.
|
||||
2. Add `hs-cookie-get`, `hs-cookie-set`, `hs-cookie-delete`, `hs-cookie-length`, `hs-cookie-names` to `lib/hyperscript/runtime.sx`.
|
||||
3. Add `cookies` as a special expression in the HS parser/evaluator that dispatches to the above.
|
||||
4. Replace 5 `SKIP` bodies in `spec/tests/test-hyperscript-behavioral.sx` with translated test bodies above.
|
||||
5. Run `hs_test_run suite="hs-upstream-expressions/cookies"` — expect 5/5.
|
||||
6. Run smoke 0–195 — no regressions.
|
||||
7. Commit: `HS: cookie API — document.cookie proxy + 5 tests`
|
||||
|
||||
## 8. Risk
|
||||
|
||||
Medium. The mock is simple. The main risk is the `cookies` expression integration in the parser — it needs to hook into property-access and assignment paths that are already well-exercised. Keep the implementation thin: `cookies` is a runtime value with a special type, not a new parse form.
|
||||
107
plans/designs/f8-eval-statically.md
Normal file
107
plans/designs/f8-eval-statically.md
Normal file
@@ -0,0 +1,107 @@
|
||||
# F8 — evalStatically (+3)
|
||||
|
||||
**Suite:** `hs-upstream-core/evalStatically`
|
||||
**Target:** 3 failing (untranslated) out of 8. 5 already pass.
|
||||
|
||||
## 1. Current state
|
||||
|
||||
5 passing tests use `(eval-hs expr)` and check the return value for literals: booleans, null, numbers, plain strings, time expressions. These call `_hyperscript.evaluate(src)` and return the result.
|
||||
|
||||
3 failing tests are named:
|
||||
- `throws on math expressions`
|
||||
- `throws on symbol references`
|
||||
- `throws on template strings`
|
||||
|
||||
All are `SKIP (untranslated)` — no test body has been generated.
|
||||
|
||||
## 2. What upstream checks
|
||||
|
||||
From `test/core/evalStatically.js`, the `throwErrors` mode:
|
||||
|
||||
```js
|
||||
expect(() => _hyperscript.evaluate("1 + 2")).toThrow();
|
||||
expect(() => _hyperscript.evaluate("x")).toThrow();
|
||||
expect(() => _hyperscript.evaluate(`"hello ${name}"`)).toThrow();
|
||||
```
|
||||
|
||||
`_hyperscript.evaluate(src)` in strict static mode throws when the expression is not a pure literal — math operators, symbol references, and template string interpolation all involve runtime evaluation that can't be statically resolved.
|
||||
|
||||
The "static" constraint: only literals that can be evaluated without any runtime context or side effects are allowed. `1 + 2` is not static (it's a math op). `x` is not static (symbol lookup). `"hello ${name}"` is not static (interpolation).
|
||||
|
||||
## 3. What `eval-hs` currently does
|
||||
|
||||
`eval-hs` in our harness calls `(hs-compile-and-run src)` or equivalent. It does NOT currently have a "static mode" — it runs everything with the full runtime.
|
||||
|
||||
We need a new harness helper `eval-hs-static-error` that:
|
||||
1. Calls `(hs-compile src)` with a flag that makes it throw on non-literal expressions
|
||||
2. Returns the caught error message, or raises if no error was thrown
|
||||
|
||||
## 4. Implementation options
|
||||
|
||||
### Option A — Static analysis pass (accurate)
|
||||
|
||||
Before evaluation, walk the AST and reject any node that isn't a literal:
|
||||
- Number literal ✓
|
||||
- String literal (no interpolation) ✓
|
||||
- Boolean literal ✓
|
||||
- Null literal ✓
|
||||
- Time expression (`200ms`, `2s`) ✓
|
||||
- Everything else → throw `"expression is not static"`
|
||||
|
||||
This is a pre-eval AST check, not a runtime change. Lives in `lib/hyperscript/compiler.sx` as `hs-check-static`.
|
||||
|
||||
### Option B — Generator translation (simpler)
|
||||
|
||||
The 3 tests are untranslated. All three just verify that `_hyperscript.evaluate(expr)` throws. In our SX harness we can test this with a `guard` form:
|
||||
|
||||
```lisp
|
||||
(deftest "throws on math expressions"
|
||||
(let ((result (guard (e (true true))
|
||||
(eval-hs "1 + 2")
|
||||
false)))
|
||||
(assert result)))
|
||||
```
|
||||
|
||||
But this only works if `eval-hs` actually throws on math expressions. Currently it doesn't — `eval-hs "1 + 2"` returns `3`. So we'd need the static analysis anyway to make the test pass.
|
||||
|
||||
### Chosen approach: Option A
|
||||
|
||||
Add `hs-static-check` to the compiler: a fast AST walker that throws on any non-literal node. Wire it as an optional mode. The test harness calls `eval-hs-static` which runs with static-check enabled.
|
||||
|
||||
Actually, reading the upstream more carefully: `_hyperscript.evaluate` already throws in static mode without additional flags — the "evaluate" API is documented as static-only. Our `eval-hs` in the passing tests works because booleans/numbers/strings/time ARE static. `1 + 2`, `x`, and template strings are NOT static and should throw.
|
||||
|
||||
So the fix is: make `hs-compile-and-run` (or whatever backs `eval-hs`) reject non-literal AST nodes. The 5 passing tests will continue to pass (they use literals). The 3 failing tests will get translated using `eval-hs-error` or a guard pattern.
|
||||
|
||||
## 5. Non-literal AST node types to reject
|
||||
|
||||
| Expression | AST node type | Reject? |
|
||||
|-----------|--------------|---------|
|
||||
| `1`, `3.14` | number literal | ✓ allow |
|
||||
| `"hello"`, `'world'` | string literal (no interpolation) | ✓ allow |
|
||||
| `true`, `false` | boolean literal | ✓ allow |
|
||||
| `null` | null literal | ✓ allow |
|
||||
| `200ms`, `2s` | time literal | ✓ allow |
|
||||
| `1 + 2` | math operator | ✗ throw |
|
||||
| `x` | symbol reference | ✗ throw |
|
||||
| `"hello ${name}"` | template string | ✗ throw |
|
||||
|
||||
## 6. Implementation checklist
|
||||
|
||||
1. In `lib/hyperscript/compiler.sx`, add `hs-static?` predicate: returns true only for literal AST node types.
|
||||
2. In the `eval-hs` path (wherever `hs-compile-and-run` is called for the evaluate API), call `hs-static?` on the parsed AST and throw `"expression is not statically evaluable"` if false.
|
||||
3. Replace 3 `SKIP` bodies in `spec/tests/test-hyperscript-behavioral.sx`:
|
||||
```lisp
|
||||
(deftest "throws on math expressions"
|
||||
(assert (string? (eval-hs-error "1 + 2"))))
|
||||
(deftest "throws on symbol references"
|
||||
(assert (string? (eval-hs-error "x"))))
|
||||
(deftest "throws on template strings"
|
||||
(assert (string? (eval-hs-error "\"hello ${name}\""))))
|
||||
```
|
||||
4. Run `hs_test_run suite="hs-upstream-core/evalStatically"` — expect 8/8.
|
||||
5. Run smoke 0–195 — verify the 5 passing tests still pass.
|
||||
6. Commit: `HS: evalStatically — static literal check, 3 tests (+3)`
|
||||
|
||||
## 7. Risk
|
||||
|
||||
Low-medium. The main risk is that `eval-hs` is used in many tests for non-static expressions and adding a static check to the shared path would break them. The fix must be gated — either a separate `eval-hs-static` helper or a flag parameter. The passing tests must not be affected.
|
||||
341
plans/designs/hs-plugin-system.md
Normal file
341
plans/designs/hs-plugin-system.md
Normal file
@@ -0,0 +1,341 @@
|
||||
# HyperScript Plugin / Extension System
|
||||
|
||||
Post-Bucket-F capability work. No conformance delta on its own — the payoff is
|
||||
clean architecture for language embeds (Lua, Prolog, Worker runtime) and
|
||||
alignment with real `_hyperscript`'s extension model.
|
||||
|
||||
---
|
||||
|
||||
## 1. Motivation
|
||||
|
||||
### 1a. Real `_hyperscript` has a plugin API
|
||||
|
||||
Stock `_hyperscript` ships a core bundle with feature stubs and a `use(ext)`
|
||||
hook that loads named extensions at runtime. The worker feature is the canonical
|
||||
example: the core parser has a stub that errors helpfully; loading the worker
|
||||
extension replaces the stub with a real implementation.
|
||||
|
||||
We currently have no equivalent. New grammar or compiler targets require editing
|
||||
`parse-feat`'s hardcoded `cond` or `hs-to-sx`'s hardcoded dispatch. This is
|
||||
fine for conformance work but wrong for language embeds.
|
||||
|
||||
### 1b. Ad-hoc hooks are accumulating
|
||||
|
||||
`runtime.sx` already has `hs-prolog-hook` / `hs-set-prolog-hook!` / `prolog`
|
||||
(nodes 140–142) — an informal plugin slot bolted on outside the parser and
|
||||
compiler. This pattern will repeat for Lua, and again for the Worker runtime.
|
||||
A proper registry prevents the drift.
|
||||
|
||||
### 1c. E39 worker stub is a placeholder
|
||||
|
||||
The stub added in E39 (`parse-feat` raises immediately on `"worker"`) was
|
||||
explicitly designed to be replaced by a real plugin at a single site. This plan
|
||||
is where that replacement happens.
|
||||
|
||||
### 1d. Bucket-F Group 10 needs a converter registry
|
||||
|
||||
`as MyType` via registered converter is already in the Bucket-F plan (Group 10).
|
||||
A `hs-register-converter!` registry is the natural home for it — and the plugin
|
||||
system is the right time to add registries generally.
|
||||
|
||||
---
|
||||
|
||||
## 2. Scope
|
||||
|
||||
**In scope:**
|
||||
- Parser feature registry (`parse-feat` dispatch)
|
||||
- Compiler command registry (`hs-to-sx` dispatch)
|
||||
- `as` converter registry (`hs-coerce` dispatch)
|
||||
- Migration of E39 worker stub to use the parser registry
|
||||
- Migration of `hs-prolog-hook` ad-hoc slot to a proper plugin
|
||||
- Worker full runtime plugin (first real plugin)
|
||||
- Lua embed plugin
|
||||
- Prolog embed plugin
|
||||
|
||||
**Out of scope:**
|
||||
- Changing the test runner or generator
|
||||
- Any conformance delta (this plan doesn't target failing tests)
|
||||
- Third-party plugin loading from external URLs (future)
|
||||
- Hot-reload of plugins (future)
|
||||
|
||||
---
|
||||
|
||||
## 3. Registry design
|
||||
|
||||
Three registries, all SX dicts. Checked before the hardcoded `cond` in each
|
||||
dispatch. Registration functions defined alongside the registries in their
|
||||
respective files.
|
||||
|
||||
### 3a. Parser feature registry (`lib/hyperscript/parser.sx`)
|
||||
|
||||
```lisp
|
||||
(define _hs-feature-registry (dict))
|
||||
|
||||
(define hs-register-feature!
|
||||
(fn (keyword parse-fn)
|
||||
(set! _hs-feature-registry
|
||||
(dict-set _hs-feature-registry keyword parse-fn))))
|
||||
```
|
||||
|
||||
In `parse-feat`, prepend a registry lookup before the existing `cond`:
|
||||
|
||||
```lisp
|
||||
(let ((registered (dict-get _hs-feature-registry val)))
|
||||
(if registered
|
||||
(registered) ;; call the registered parse-fn (no args; uses closure over adv!/tp-val etc.)
|
||||
(cond ;; existing dispatch unchanged below
|
||||
...)))
|
||||
```
|
||||
|
||||
`parse-fn` is a zero-arg thunk that has access to the parser's internal state
|
||||
via the same closure that the existing `parse-*` helpers use. Since `parse-feat`
|
||||
is itself defined inside the big `let` in `hs-parse`, all the parser helpers
|
||||
(`adv!`, `tp-val`, `tp-typ`, `parse-cmd-list`, etc.) are in scope.
|
||||
|
||||
### 3b. Compiler command registry (`lib/hyperscript/compiler.sx`)
|
||||
|
||||
```lisp
|
||||
(define _hs-compiler-registry (dict))
|
||||
|
||||
(define hs-register-compiler!
|
||||
(fn (head compile-fn)
|
||||
(set! _hs-compiler-registry
|
||||
(dict-set _hs-compiler-registry (str head) compile-fn))))
|
||||
```
|
||||
|
||||
In `hs-to-sx`, before the existing `cond` on `head`, check the registry:
|
||||
|
||||
```lisp
|
||||
(let ((registered (dict-get _hs-compiler-registry (str head))))
|
||||
(if registered
|
||||
(registered ast)
|
||||
(cond ...)))
|
||||
```
|
||||
|
||||
`compile-fn` receives the full AST node and returns an SX expression.
|
||||
|
||||
### 3c. `as` converter registry (`lib/hyperscript/runtime.sx`)
|
||||
|
||||
```lisp
|
||||
(define _hs-converters (dict))
|
||||
|
||||
(define hs-register-converter!
|
||||
(fn (type-name converter-fn)
|
||||
(set! _hs-converters
|
||||
(dict-set _hs-converters type-name converter-fn))))
|
||||
```
|
||||
|
||||
In `hs-coerce`, add a registry lookup as the last `cond` clause before the
|
||||
fallthrough error:
|
||||
|
||||
```lisp
|
||||
((dict-get _hs-converters type-name)
|
||||
((dict-get _hs-converters type-name) value))
|
||||
```
|
||||
|
||||
This is also the hook that Bucket-F Group 10 (`can accept custom conversions`)
|
||||
hangs on — so implementing it here kills two birds.
|
||||
|
||||
---
|
||||
|
||||
## 4. First-party plugins
|
||||
|
||||
Each plugin is a `.sx` file in `lib/hyperscript/plugins/`. Plugins call the
|
||||
registration functions at load time (top-level `do` forms). The host loads
|
||||
plugins explicitly after the core files.
|
||||
|
||||
### 4a. Worker plugin (`lib/hyperscript/plugins/worker.sx`)
|
||||
|
||||
**Phase 1 — stub migration (immediate):**
|
||||
Remove the inline error branch from `parse-feat` (the E39 stub). Replace with:
|
||||
|
||||
```lisp
|
||||
(hs-register-feature! "worker"
|
||||
(fn ()
|
||||
(error "worker plugin is not installed — see https://hyperscript.org/features/worker")))
|
||||
```
|
||||
|
||||
This is identical behaviour to E39 but routed through the registry. The stub
|
||||
lives in the plugin file, not the core parser. No test regression.
|
||||
|
||||
**Phase 2 — full runtime:**
|
||||
|
||||
Parser: `parse-worker-feat` — consumes `worker <Name> [(<url>*)] <def|js>* end`,
|
||||
returns `(worker Name urls defs)` AST node.
|
||||
|
||||
Compiler: registered under `"worker"` head:
|
||||
- Emits `(hs-worker-define! "Name" urls defs)` call.
|
||||
|
||||
Runtime additions in the plugin file:
|
||||
- `hs-worker-define!` — creates a `{:_hs-worker true :name N :handle H :exports (...)}` record,
|
||||
binds it in the HS top-level env under `Name`.
|
||||
- `hs-method-call` (existing) detects `:_hs-worker` and dispatches via `postMessage`.
|
||||
- Worker script body compiled to a standalone SX bundle posted to a Blob URL.
|
||||
- Return values are promise-wrapped; async-transparent via `perform`/IO suspension.
|
||||
|
||||
Mock env additions for the test runner: `Worker` constructor + synchronous
|
||||
message loop for the 7 sibling `test.skip(...)` upstream tests (the ones
|
||||
deferred in E39).
|
||||
|
||||
### 4b. Prolog plugin (`lib/hyperscript/plugins/prolog.sx`)
|
||||
|
||||
Replaces the ad-hoc `hs-prolog-hook` in `runtime.sx`.
|
||||
|
||||
**Parser:** Register `"prolog"` feature — parses
|
||||
`prolog(<db-expr>, <goal-expr>)` at feature level (alternative: keep as an
|
||||
expression, register a compiler extension only).
|
||||
|
||||
**Compiler:** Registered under `"prolog"` head — emits `(prolog db goal)`.
|
||||
|
||||
**Runtime:** The existing `prolog` function in `runtime.sx` moves here.
|
||||
`hs-prolog-hook` and `hs-set-prolog-hook!` are removed from `runtime.sx` and
|
||||
the hook mechanism is replaced by the plugin loading `lib/prolog/runtime.sx`
|
||||
and wiring the solver directly.
|
||||
|
||||
Remove from `runtime.sx` nodes 140–142 once the plugin is live.
|
||||
|
||||
### 4c. Lua plugin (`lib/hyperscript/plugins/lua.sx`)
|
||||
|
||||
**Parser:** Register `"lua"` feature — parses `lua ... end` block, captures
|
||||
the body as a raw string.
|
||||
|
||||
**Compiler:** Registered under `"lua"` head — emits `(lua-eval <body-string>)`.
|
||||
|
||||
**Runtime:** `lua-eval` calls `lib/lua/runtime.sx`'s eval entry point, returns
|
||||
result as an SX value via `hs-host-to-sx`. Errors surface as HS `catch`-able
|
||||
exceptions.
|
||||
|
||||
This enables inline Lua in HyperScript:
|
||||
|
||||
```
|
||||
on click
|
||||
lua
|
||||
return document.title:upper()
|
||||
end
|
||||
put it into me
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Load order
|
||||
|
||||
```
|
||||
lib/hyperscript/parser.sx ;; defines _hs-feature-registry, hs-register-feature!
|
||||
lib/hyperscript/compiler.sx ;; defines _hs-compiler-registry, hs-register-compiler!
|
||||
lib/hyperscript/runtime.sx ;; defines _hs-converters, hs-register-converter!
|
||||
lib/hyperscript/plugins/worker.sx
|
||||
lib/hyperscript/plugins/prolog.sx
|
||||
lib/hyperscript/plugins/lua.sx
|
||||
```
|
||||
|
||||
The test runner (`tests/hs-run-filtered.js`) loads plugins after core. The
|
||||
browser WASM bundle includes all three by default (plugins are small; no
|
||||
reason to lazy-load them).
|
||||
|
||||
---
|
||||
|
||||
## 6. Migration checklist
|
||||
|
||||
The work below is ordered to keep main green at every commit. Each step is
|
||||
independently committable.
|
||||
|
||||
### Step 1 — Registries (infrastructure, no behaviour change)
|
||||
|
||||
1. Add `_hs-feature-registry` + `hs-register-feature!` to `parser.sx`.
|
||||
Thread the registry check into `parse-feat`. No entries yet → behaviour
|
||||
unchanged.
|
||||
2. Add `_hs-compiler-registry` + `hs-register-compiler!` to `compiler.sx`.
|
||||
Thread into `hs-to-sx`. No entries yet → behaviour unchanged.
|
||||
3. Add `_hs-converters` + `hs-register-converter!` to `runtime.sx`. Thread
|
||||
into `hs-coerce`. No entries yet → behaviour unchanged.
|
||||
4. `sx_validate` all three files. Run full HS suite — expect zero regressions.
|
||||
5. Commit: `HS: plugin registry infrastructure (parser + compiler + converter)`.
|
||||
|
||||
### Step 2 — Worker stub migration
|
||||
|
||||
6. Create `lib/hyperscript/plugins/worker.sx`. Register the worker stub error.
|
||||
7. Remove the inline `((= val "worker") ...)` branch from `parse-feat` in
|
||||
`parser.sx`.
|
||||
8. Update the test runner to load `worker.sx` after core.
|
||||
9. Run `HS_SUITE=hs-upstream-worker` — expect 1/1. Run full suite — expect no
|
||||
regressions.
|
||||
10. Commit: `HS: migrate E39 worker stub to plugin registry`.
|
||||
|
||||
### Step 3 — Prolog plugin
|
||||
|
||||
11. Create `lib/hyperscript/plugins/prolog.sx`. Wire to `lib/prolog/runtime.sx`.
|
||||
12. Remove `hs-prolog-hook`, `hs-set-prolog-hook!`, `prolog` from `runtime.sx`
|
||||
nodes 140–142.
|
||||
13. Update test runner to load `prolog.sx`.
|
||||
14. Validate and run full suite.
|
||||
15. Commit: `HS: prolog plugin replaces ad-hoc hook`.
|
||||
|
||||
### Step 4 — `as` converter registry (bridges Bucket-F Group 10)
|
||||
|
||||
16. Confirm `hs-register-converter!` satisfies the Group 10 test
|
||||
`can accept custom conversions`. If yes, this step may be pulled into
|
||||
Bucket-F Group 10 instead (no duplication — just move step 3 of §6 there).
|
||||
17. Commit: `HS: as-converter registry wired into hs-coerce`.
|
||||
|
||||
### Step 5 — Lua plugin
|
||||
|
||||
18. Create `lib/hyperscript/plugins/lua.sx`.
|
||||
19. Add `lua-eval` to `runtime.sx` or directly in the plugin file.
|
||||
20. Parser: `parse-lua-feat` consuming `lua … end`.
|
||||
21. Compiler: registered `"lua"` head.
|
||||
22. Write 3–5 tests in `spec/tests/test-hyperscript-lua.sx`:
|
||||
- Lua returns a string → HS uses it.
|
||||
- Lua error → HS catch.
|
||||
- Lua reads a passed argument.
|
||||
23. Commit: `HS: Lua plugin — inline lua...end blocks`.
|
||||
|
||||
### Step 6 — Worker full runtime plugin
|
||||
|
||||
24. Extend `worker.sx`: implement `parse-worker-feat`, compiler entry,
|
||||
`hs-worker-define!`, `hs-method-call` worker branch.
|
||||
25. Extend test runner: `Worker` constructor + synchronous message loop.
|
||||
26. Un-skip the 7 sibling worker tests from upstream.
|
||||
27. Target: 7/7 worker suite.
|
||||
28. Commit: `HS: Worker plugin full runtime (+7 tests)`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Risks
|
||||
|
||||
- **`parse-feat` closure scope** — `hs-register-feature!` stores parse-fns
|
||||
that need access to parser-internal helpers (`adv!`, `tp-val`, etc.). These
|
||||
are only in scope inside `hs-parse`'s big `let`. Two options:
|
||||
(a) the registry stores fns that receive a parser-context dict as arg, or
|
||||
(b) the registry is checked *inside* `parse-feat` where helpers are in scope
|
||||
and fns are zero-arg closures captured at registration time.
|
||||
Option (b) is simpler but requires plugins to be loaded while the parser
|
||||
`let` is being evaluated — i.e., plugins must be defined *inside* the parser
|
||||
file or the context dict must be exposed. **Recommended:** expose a
|
||||
`_hs-parser-ctx` dict at the module level that parse-fns receive as their
|
||||
sole argument. This makes the API explicit and plugins independent files.
|
||||
|
||||
- **Worker Blob URL in WASM** — `URL.createObjectURL` is available in browsers
|
||||
but not in the OCaml WASM host. Worker full runtime is browser-only; flag it
|
||||
with a capability check and graceful fallback.
|
||||
|
||||
- **Lua/Prolog mutual recursion** — a Lua block calling back into HS calling
|
||||
back into Lua is theoretically possible via the IO suspension machinery.
|
||||
Don't try to support it initially; raise a clear error if detected.
|
||||
|
||||
- **Plugin load-order sensitivity** — `hs-register-feature!` must be called
|
||||
before any source is parsed. If a plugin is loaded lazily (future), a
|
||||
`worker MyWorker` in the page would hit the stub before the full plugin
|
||||
registers. Acceptable for now; document that plugins must be loaded at boot.
|
||||
|
||||
- **`runtime.sx` cleanup for prolog** — nodes 140–142 are referenced nowhere
|
||||
else in the codebase (grep confirms). Safe to delete once the plugin is live.
|
||||
|
||||
---
|
||||
|
||||
## 8. Non-goals
|
||||
|
||||
- Runtime `use(ext)` API (JS-style dynamic plugin install) — future.
|
||||
- Plugin namespacing / versioning — future.
|
||||
- Any conformance tests other than the 7 worker tests in step 6.
|
||||
- Changing how the WASM bundle is built or split.
|
||||
257
plans/designs/sx-adt.md
Normal file
257
plans/designs/sx-adt.md
Normal file
@@ -0,0 +1,257 @@
|
||||
# SX Algebraic Data Types — Design
|
||||
|
||||
## Motivation
|
||||
|
||||
Every language implementation currently uses `{:tag "..." :field ...}` tagged dicts to
|
||||
simulate sum types. This is verbose, error-prone (typos in tag strings go undetected), and
|
||||
produces no exhaustiveness warnings. Native ADTs eliminate the pattern everywhere.
|
||||
|
||||
Examples of current workarounds:
|
||||
- Haskell `Maybe a` → `{:tag "Just" :value x}` / `{:tag "Nothing"}`
|
||||
- Prolog terms → `{:tag "functor" :name "foo" :args (list x y)}`
|
||||
- Lua result type → `{:tag "ok" :value v}` / `{:tag "err" :msg s}`
|
||||
- Common Lisp `cons` pairs → `{:tag "cons" :car a :cdr b}`
|
||||
|
||||
---
|
||||
|
||||
## Syntax
|
||||
|
||||
### `define-type`
|
||||
|
||||
```lisp
|
||||
(define-type Name
|
||||
(Ctor1 field1 field2 ...)
|
||||
(Ctor2 field1 ...)
|
||||
...)
|
||||
```
|
||||
|
||||
Creates:
|
||||
- Constructor functions: `Ctor1`, `Ctor2`, … (callable like normal functions)
|
||||
- Type predicate: `Name?` — returns true for any value of type `Name`
|
||||
- Constructor predicates: `Ctor1?`, `Ctor2?`, … (optional, auto-generated)
|
||||
- Field accessors: `Ctor1-field1`, `Ctor1-field2`, … (optional, auto-generated)
|
||||
|
||||
Examples:
|
||||
|
||||
```lisp
|
||||
(define-type Maybe
|
||||
(Just value)
|
||||
(Nothing))
|
||||
|
||||
(define-type Result
|
||||
(Ok value)
|
||||
(Err message))
|
||||
|
||||
(define-type Tree
|
||||
(Leaf)
|
||||
(Node left value right))
|
||||
|
||||
(define-type List-of
|
||||
(Nil-of)
|
||||
(Cons-of head tail))
|
||||
```
|
||||
|
||||
Constructors with no fields are zero-argument constructors (singletons by value):
|
||||
|
||||
```lisp
|
||||
(Nothing) ; => #<Nothing>
|
||||
(Leaf) ; => #<Leaf>
|
||||
```
|
||||
|
||||
### `match`
|
||||
|
||||
```lisp
|
||||
(match expr
|
||||
((Ctor1 a b) body)
|
||||
((Ctor2 x) body)
|
||||
((Ctor3) body)
|
||||
(else body))
|
||||
```
|
||||
|
||||
- Clauses are tried in order; first match wins.
|
||||
- `else` clause is optional but suppresses exhaustiveness warnings.
|
||||
- Pattern variables (`a`, `b`, `x`) are bound in the body scope.
|
||||
- Wildcard `_` discards the matched value.
|
||||
- Literal patterns: `42`, `"str"`, `true`, `nil` — match by value equality.
|
||||
- Nested patterns: `((Node left (Leaf) right) body)` — nested constructor patterns.
|
||||
|
||||
Examples:
|
||||
|
||||
```lisp
|
||||
(match result
|
||||
((Ok v) (str "got: " v))
|
||||
((Err m) (str "error: " m)))
|
||||
|
||||
(match tree
|
||||
((Leaf) 0)
|
||||
((Node l v r) (+ 1 (tree-depth l) (tree-depth r))))
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## CEK Dispatch
|
||||
|
||||
### Runtime representation
|
||||
|
||||
ADT values are OCaml records (not dicts) — opaque, non-inspectable via `get`:
|
||||
|
||||
```ocaml
|
||||
type adt_value = {
|
||||
av_type : string; (* type name, e.g. "Maybe" *)
|
||||
av_ctor : string; (* constructor name, e.g. "Just" *)
|
||||
av_fields: value array; (* positional fields *)
|
||||
}
|
||||
```
|
||||
|
||||
In JS: `{ _adt: true, _type: "Maybe", _ctor: "Just", _fields: [v] }`.
|
||||
|
||||
`typeOf` returns the ADT type name (e.g. `"Maybe"`).
|
||||
|
||||
### `define-type` — special form
|
||||
|
||||
`stepSfDefineType(args, env, kont)`:
|
||||
|
||||
1. Parse `Name` and list of `(CtorN field...)` clauses.
|
||||
2. For each constructor `CtorK` with fields `[f1, f2, …]`:
|
||||
- Register `CtorK` as a `NativeFn` that takes `|fields|` args and returns an `AdtValue`.
|
||||
- Register `CtorK?` as a predicate (`AdtValue` with matching ctor name → `true`).
|
||||
- Register `CtorK-fN` as field accessor (returns `av_fields[N]`).
|
||||
3. Register `Name?` as a predicate (`AdtValue` with matching type name → `true`).
|
||||
4. All bindings go into the current environment via `env-bind!`.
|
||||
5. Returns `Nil`.
|
||||
|
||||
This is an environment mutation — no new frame needed. Evaluates in one step.
|
||||
|
||||
### `match` — special form
|
||||
|
||||
`stepSfMatch(args, env, kont)`:
|
||||
|
||||
1. Push `MatchFrame` with `clauses` and `env` onto kont.
|
||||
2. Return state evaluating the scrutinee `expr`.
|
||||
3. `MatchFrame` continue: receive scrutinee value, walk clauses:
|
||||
- For each `((CtorN vars...) body)`:
|
||||
- If scrutinee is an `AdtValue` with `av_ctor = "CtorN"` and `av_fields.length = |vars|`:
|
||||
- Bind `vars[i]` → `av_fields[i]` in fresh child env.
|
||||
- Return state evaluating `body` in that env.
|
||||
- `(else body)` — always matches, body evaluated in current env.
|
||||
- Literal `42`/`"str"` patterns: match by value equality.
|
||||
- Wildcard `_`: always matches, binds nothing.
|
||||
4. If no clause matched and no `else`: raise `"match: no clause matched <value>"`.
|
||||
|
||||
Frame type: `"match"` — stores `cf_remaining` (clauses), `cf_env` (enclosing env).
|
||||
|
||||
---
|
||||
|
||||
## Interaction with `cond` / `case`
|
||||
|
||||
`match` is the primary dispatch form for ADTs. `cond` / `case` remain unchanged:
|
||||
|
||||
- `cond` tests arbitrary boolean expressions — still useful for non-ADT dispatch.
|
||||
- `case` matches on equality to literal values — unchanged.
|
||||
- `match` is the new form: structural pattern matching on ADT constructors.
|
||||
|
||||
They are orthogonal. A `match` clause can contain a `cond`; a `cond` clause can contain a `match`.
|
||||
|
||||
---
|
||||
|
||||
## Exhaustiveness checking
|
||||
|
||||
Emit a **warning** (not an error) when:
|
||||
- A `match` has no `else` clause, AND
|
||||
- Not all constructors of the scrutinee's type are covered.
|
||||
|
||||
Detection: when `define-type` runs, it registers the constructor set in a global table
|
||||
`_adt_registry: type_name → [ctor_names]`. At `match` compile/evaluation time:
|
||||
- If the scrutinee's type is in `_adt_registry` and not all ctors appear as patterns:
|
||||
- `console.warn("[sx] match: non-exhaustive — missing: Ctor3, Ctor4 for type Maybe")`
|
||||
- Execution continues (warning, not error).
|
||||
|
||||
This is best-effort: the scrutinee type is only known at runtime. The warning fires on
|
||||
first non-exhaustive match evaluation, not at definition time.
|
||||
|
||||
---
|
||||
|
||||
## Recursive types
|
||||
|
||||
Recursive types work because constructors are registered as functions, and function bodies
|
||||
are evaluated lazily:
|
||||
|
||||
```lisp
|
||||
(define-type Tree
|
||||
(Leaf)
|
||||
(Node left value right))
|
||||
|
||||
; Recursive function over a recursive type:
|
||||
(define (depth tree)
|
||||
(match tree
|
||||
((Leaf) 0)
|
||||
((Node l v r) (+ 1 (max (depth l) (depth r))))))
|
||||
```
|
||||
|
||||
No special treatment needed — the type definition doesn't need to know about recursion.
|
||||
The constructor `Node` accepts any values, including other `Node` or `Leaf` values.
|
||||
|
||||
---
|
||||
|
||||
## Pattern variables
|
||||
|
||||
In `match` clauses, identifiers in constructor position that are NOT constructor names are
|
||||
treated as pattern variables (bound to matched field values):
|
||||
|
||||
```lisp
|
||||
(match x
|
||||
((Just v) v) ; v bound to the wrapped value
|
||||
((Nothing) nil))
|
||||
|
||||
(match pair
|
||||
((Cons-of h t) (list h t))) ; h, t bound to head and tail
|
||||
```
|
||||
|
||||
**Wildcard**: `_` is always a wildcard — matches anything, binds nothing.
|
||||
|
||||
```lisp
|
||||
(match x
|
||||
((Just _) "has value")
|
||||
((Nothing) "empty"))
|
||||
```
|
||||
|
||||
**Nested patterns**:
|
||||
|
||||
```lisp
|
||||
(match tree
|
||||
((Node (Leaf) v (Leaf)) (str "leaf node: " v))
|
||||
((Node l v r) (str "inner node: " v)))
|
||||
```
|
||||
|
||||
Nested patterns are matched recursively: the inner `(Leaf)` pattern checks that the
|
||||
`left` field is itself a `Leaf` ADT value.
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 6a — `define-type` + basic `match` (no nested patterns, no exhaustiveness)
|
||||
|
||||
1. OCaml: add `AdtValue of adt_value` to `sx_types.ml`.
|
||||
2. Evaluator: add `step-sf-define-type` — parse clauses, register ctor fns + predicates + accessors.
|
||||
3. Evaluator: add `step-sf-match` + `MatchFrame` — linear scan of clauses, flat patterns only.
|
||||
4. JS: same (AdtValue as plain object with `_adt`/`_type`/`_ctor`/`_fields` props).
|
||||
|
||||
### Phase 6b — nested patterns (separate fire)
|
||||
|
||||
Recursive `matchPattern(pattern, value, env)` helper that:
|
||||
- Returns `{matched: bool, bindings: map}`
|
||||
- Recursively matches sub-patterns against ADT fields.
|
||||
|
||||
### Phase 6c — exhaustiveness warnings (separate fire)
|
||||
|
||||
`_adt_registry` global + warning emission on first non-exhaustive match.
|
||||
|
||||
---
|
||||
|
||||
## Open questions (deferred to review)
|
||||
|
||||
1. **Accessor auto-generation**: should `Ctor-field` accessors be generated always, or only on demand? Risk: name collisions if two types have constructors with same field names.
|
||||
2. **Singleton constructors**: `(Nothing)` — zero-arg ctor — should these be interned (same object every call) or fresh each time? Interning enables `eq?` checks but requires a global table.
|
||||
3. **Printing/inspect**: `inspect` on an AdtValue should show `(Just 42)` not `#<adt:Just>`. Implement in `inspect` function or via `display`/`write` (Phase 17 ports).
|
||||
4. **Pattern-matching on non-ADT values**: should `match` handle list patterns `(a . b)` and literal patterns in clause heads? Deferred — add only if needed by a language implementation.
|
||||
173
plans/elixir-on-sx.md
Normal file
173
plans/elixir-on-sx.md
Normal file
@@ -0,0 +1,173 @@
|
||||
# Elixir-on-SX: Elixir on the CEK/VM
|
||||
|
||||
Compile Elixir source to SX AST; the existing CEK evaluator runs it. The natural companion
|
||||
to `lib/erlang/` — Elixir compiles to the BEAM and most of its runtime semantics are
|
||||
Erlang's. The interesting parts are Elixir-specific: the macro system (`quote`/`unquote`),
|
||||
the pipe operator `|>`, `with` expressions, `defmodule`/`def`/`defp`, protocol dispatch,
|
||||
and the `Stream` lazy evaluation library.
|
||||
|
||||
End-state goal: **core Elixir programs running**, including modules, pattern matching, the
|
||||
pipe operator, macros (`quote`/`unquote`/`defmacro`), protocols, and actor-style processes
|
||||
reusing the Erlang runtime foundation.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/elixir/**` and `plans/elixir-on-sx.md`. Do **not** edit
|
||||
`spec/`, `hosts/`, `shared/`, or other `lib/<lang>/`. Reuse `lib/erlang/` runtime
|
||||
functions where possible — import them, don't duplicate.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Elixir source → Elixir AST → SX AST. Reuse Erlang runtime for process/
|
||||
message/pattern primitives; add Elixir-specific surface in `lib/elixir/`.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Elixir source text
|
||||
│
|
||||
▼
|
||||
lib/elixir/tokenizer.sx — atoms (:atom), strings (""), charlists (''), sigils (~r, ~s etc.),
|
||||
│ operators (|>, <>, ++, :::, etc.), do/end blocks
|
||||
▼
|
||||
lib/elixir/parser.sx — Elixir AST: defmodule, def/defp/defmacro, @attribute,
|
||||
│ pattern matching, |> pipe, with, for comprehension, quote/unquote,
|
||||
│ case/cond/if/unless, fn, receive, try/rescue/catch/after
|
||||
▼
|
||||
lib/elixir/transpile.sx — Elixir AST → SX AST
|
||||
│
|
||||
├── lib/erlang/runtime.sx (reused: processes, message passing, pattern match)
|
||||
└── lib/elixir/runtime.sx — Elixir-specific: Kernel, String, Enum, Stream, Map,
|
||||
List, Tuple, IO, protocol dispatch, macro expansion
|
||||
```
|
||||
|
||||
Key semantic mappings (differences from Erlang):
|
||||
- `defmodule M do ... end` → SX `define-library` + module dict `{:module "M" :fns {...}}`
|
||||
- `def f(args) do body end` → named function in module dict, with pattern-match dispatch
|
||||
- `|>` pipe → left-to-right function composition; `a |> f(b)` = `f(a, b)`
|
||||
- `with x <- expr, y <- expr2 do body else patterns end` → chained pattern match with early exit
|
||||
- `for x <- list, filter, do: expr` → list comprehension (SX `map`/`filter`)
|
||||
- `quote do expr end` → returns AST as SX list (homoiconic — Elixir AST IS SX-like)
|
||||
- `unquote(expr)` → evaluate expr and splice into surrounding `quote`
|
||||
- `defmacro` → macro in module; expanded at compile time by calling the SX macro
|
||||
- Protocol → dict of implementations keyed by type name; `defprotocol` defines interface,
|
||||
`defimpl` registers an implementation
|
||||
- `Stream` → lazy sequences using SX promises/coroutines (Phase 9/4 of primitives)
|
||||
- `Agent`/`GenServer` → SX coroutine + message queue (similar to Erlang process model)
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: atoms (`:atom`, `:"atom with spaces"`), strings (`""`), charlists (`''`),
|
||||
numbers (int, float, hex `0xFF`, octal `0o77`, binary `0b11`), booleans (`true`/`false`/`nil`),
|
||||
operators (`|>`, `<>`, `++`, `--`, `:::`, `&&`, `||`, `!`, `..`, `<-`, `=~`),
|
||||
sigils (`~r/regex/`, `~s"string"`, `~w(word list)`), do/end blocks, keywords as args
|
||||
`f(key: val)`, `@module_attribute`
|
||||
- [ ] Parser:
|
||||
- Module: `defmodule Name do ... end` → module AST with body
|
||||
- Functions: `def f(pat) do body end`, `def f(pat) when guard do body end`,
|
||||
multi-clause `def f(a) do ...; def f(b) do ...` → clause list
|
||||
- `defp` (private), `defmacro`, `defmacrop`
|
||||
- `@doc`, `@moduledoc`, `@spec`, `@type`, `@behaviour` module attributes
|
||||
- `case expr do patterns end`, `cond do clauses end`, `if`/`unless`
|
||||
- `with x <- e, y <- e2, do: body, else: [pattern -> body]`
|
||||
- `for x <- list, filter, into: acc, do: expr` comprehension
|
||||
- `fn pat -> body end` anonymous function; capture `&Module.fun/arity`, `&(&1 + 1)`
|
||||
- `receive do patterns after timeout -> body end`
|
||||
- `try do body rescue e -> ... catch type, val -> ... after ... end`
|
||||
- `quote do ... end`, `unquote(expr)`, `unquote_splicing(list)`
|
||||
- `|>` pipe chain: `a |> f |> g(b)` → `g(f(a), b)`
|
||||
- [ ] Tests in `lib/elixir/tests/parse.sx`
|
||||
|
||||
### Phase 2 — transpile: basic Elixir (no macros, no processes)
|
||||
- [ ] `ex-eval-ast` entry
|
||||
- [ ] Arithmetic, string `<>`, list `++`/`--`, comparison, boolean (`and`/`or`/`not`)
|
||||
- [ ] Pattern matching in `=`, function heads, `case` — reuse Erlang pattern engine
|
||||
- [ ] `def`/`defp` → SX `define` with clause dispatch (like Erlang function clauses)
|
||||
- [ ] Module as a dict of named functions; `ModuleName.function(args)` dispatch
|
||||
- [ ] `|>` pipe: desugar `a |> f(b, c)` → `f(a, b, c)` at transpile time
|
||||
- [ ] `with` expression: chain of `<-` bindings, short-circuit on mismatch to `else`
|
||||
- [ ] `for` comprehension: `for x <- list, filter do body end` → `map`/`filter`
|
||||
- [ ] `fn` anonymous functions, `&` capture forms
|
||||
- [ ] `if`/`unless`/`cond`/`case`
|
||||
- [ ] String interpolation: `"Hello #{name}"` → string concat
|
||||
- [ ] Keyword lists `[key: val]` → SX list of `{:key val}` dicts; maps `%{key: val}` → SX dict
|
||||
- [ ] Tuples `{a, b, c}` → SX list (or vector); `elem/2`, `put_elem/3`
|
||||
- [ ] 40+ eval tests in `lib/elixir/tests/eval.sx`
|
||||
|
||||
### Phase 3 — macro system
|
||||
- [ ] `quote do expr end` → returns Elixir AST as SX list structure
|
||||
(Elixir AST is 3-tuples `{name, meta, args}` — map to SX `(list name meta args)`)
|
||||
- [ ] `unquote(expr)` → evaluate and splice into surrounding `quote`
|
||||
- [ ] `unquote_splicing(list)` → splice list into surrounding `quote`
|
||||
- [ ] `defmacro` → define a macro in the module; macro receives AST args, returns AST
|
||||
- [ ] Macro expansion: expand macros before transpiling (two-pass: collect defs, then expand)
|
||||
- [ ] `use Module` → calls `Module.__using__/1` macro, injects code into caller
|
||||
- [ ] `import Module` → bring functions into scope without prefix
|
||||
- [ ] `alias Module, as: M` → short name for module
|
||||
- [ ] Tests: `defmacro unless`, `defmacro my_if`, `use` injection, `__MODULE__`, `__DIR__`
|
||||
|
||||
### Phase 4 — protocols
|
||||
- [ ] `defprotocol P do @spec f(t) :: result end` → defines protocol dict + dispatch fn
|
||||
- [ ] `defimpl P, for: Type do def f(t) do ... end end` → register implementation
|
||||
- [ ] Protocol dispatch: `P.f(value)` → look up type of value, find implementation, call it
|
||||
- [ ] Built-in protocols: `Enumerable`, `Collectable`, `String.Chars`, `Inspect`
|
||||
- [ ] `Enumerable` implementation for lists, maps, ranges — enables `Enum.*` on custom types
|
||||
- [ ] `derive` — automatic protocol implementation for simple structs
|
||||
- [ ] Tests: custom type implementing `Enumerable`, `String.Chars`, protocol fallback
|
||||
|
||||
### Phase 5 — structs + behaviours
|
||||
- [ ] `defstruct [:field1, field2: default]` → defines `%ModuleName{}` struct type
|
||||
Structs are maps with `__struct__: ModuleName` key + defined fields
|
||||
- [ ] Struct pattern matching: `%User{name: n} = user`
|
||||
- [ ] `@behaviour Module` → declares behaviour callbacks; compile-time check
|
||||
- [ ] `@impl true` / `@impl BehaviourName` → marks function as behaviour implementation
|
||||
- [ ] Built-in behaviours: `GenServer`, `Supervisor`, `Agent`, `Task`
|
||||
- [ ] Tests: struct creation, update syntax `%{struct | field: val}`, behaviour callbacks
|
||||
|
||||
### Phase 6 — processes + OTP patterns (reuses Erlang runtime)
|
||||
- [ ] `spawn(fn -> ... end)` / `spawn(M, f, args)` → SX coroutine on scheduler
|
||||
Reuse `lib/erlang/` process + message queue infrastructure
|
||||
- [ ] `send(pid, msg)` / `receive do patterns end` — already in Erlang runtime
|
||||
- [ ] `GenServer` behaviour: `start_link`, `call`, `cast`, `handle_call`, `handle_cast`,
|
||||
`handle_info`, `init` — implement as SX macros expanding to process + message loop
|
||||
- [ ] `Agent` — simple state wrapper over GenServer; `Agent.start_link`, `get`, `update`
|
||||
- [ ] `Task` — async computation; `Task.async`, `Task.await`
|
||||
- [ ] `Supervisor` — child spec, restart strategy (`one_for_one`, `one_for_all`)
|
||||
- [ ] Tests: counter GenServer, bank account Agent, parallel Task, supervised worker
|
||||
|
||||
### Phase 7 — standard library
|
||||
- [ ] `Enum.*` — `map`, `filter`, `reduce`, `each`, `into`, `flat_map`, `zip`, `sort`,
|
||||
`sort_by`, `min_by`, `max_by`, `group_by`, `frequencies`, `count`, `any?`, `all?`,
|
||||
`find`, `take`, `drop`, `take_while`, `drop_while`, `chunk_every`, `chunk_by`,
|
||||
`flat_map_reduce`, `scan`, `uniq`, `uniq_by`, `member?`, `empty?`, `sum`, `product`
|
||||
- [ ] `Stream.*` — lazy versions of Enum; `Stream.map`, `Stream.filter`, `Stream.take`,
|
||||
`Stream.cycle`, `Stream.iterate`, `Stream.unfold`, `Stream.resource`
|
||||
Uses SX promises (Phase 9) for laziness
|
||||
- [ ] `String.*` — `length`, `upcase`, `downcase`, `trim`, `split`, `replace`, `contains?`,
|
||||
`starts_with?`, `ends_with?`, `slice`, `at`, `graphemes`, `codepoints`, `to_integer`,
|
||||
`to_float`, `pad_leading`, `pad_trailing`, `duplicate`, `match?`
|
||||
- [ ] `Map.*` — `new`, `get`, `put`, `delete`, `update`, `merge`, `keys`, `values`,
|
||||
`to_list`, `from_struct`, `has_key?`, `filter`, `map`, `reject`, `take`, `drop`
|
||||
- [ ] `List.*` — `first`, `last`, `flatten`, `zip`, `unzip`, `keystore`, `keyfind`,
|
||||
`wrap`, `duplicate`, `improper?`, `delete`, `insert_at`, `replace_at`
|
||||
- [ ] `Tuple.*` — `to_list`, `from_list`, `append`, `insert_at`, `delete_at`
|
||||
- [ ] `Integer.*` / `Float.*` — `parse`, `to_string`, `digits`, `pow`, `is_odd?`, `is_even?`
|
||||
- [ ] `IO.*` — `puts`, `gets`, `inspect`, `write`, `read` → SX IO perform
|
||||
- [ ] `Kernel.*` — built-in functions: `is_integer?`, `is_binary?`, `length`, `hd`, `tl`,
|
||||
`elem`, `put_elem`, `apply`, `raise`, `exit`, `inspect`
|
||||
- [ ] `inspect/1` / `IO.inspect/2` — debug printing using `Inspect` protocol
|
||||
|
||||
### Phase 8 — conformance target
|
||||
- [ ] Vendor or hand-build 100+ Elixir program tests in `lib/elixir/tests/programs/`
|
||||
- [ ] Drive scoreboard
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
131
plans/elm-on-sx.md
Normal file
131
plans/elm-on-sx.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# Elm-on-SX: Elm 0.19 on the CEK/VM
|
||||
|
||||
Compile Elm source to SX AST; the existing CEK evaluator runs it. The unique angle: SX's
|
||||
reactive island system (`defisland`, signals, `provide`/`context`) is a natural host for
|
||||
The Elm Architecture — Model/Update/View maps almost directly onto SX's reactive runtime.
|
||||
This is the only language in the set that targets SX's browser-side reactivity rather than
|
||||
the server-side evaluator.
|
||||
|
||||
End-state goal: **core Elm programs running in the browser via SX islands**, with The Elm
|
||||
Architecture wired to SX signals. Not a full Elm compiler — no exhaustiveness checking, no
|
||||
module system, no type inference — but a faithful runtime that can run Elm programs written
|
||||
in idiomatic style.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/elm/**` and `plans/elm-on-sx.md`. Do **not** edit `spec/`,
|
||||
`hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Elm source → Elm AST → SX AST. No standalone Elm evaluator.
|
||||
- **Type system:** defer. Focus on runtime semantics. Type errors surface at eval time.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Elm source text
|
||||
│
|
||||
▼
|
||||
lib/elm/tokenizer.sx — numbers, strings, idents, operators, indentation-sensitive lexer
|
||||
│
|
||||
▼
|
||||
lib/elm/parser.sx — Elm AST: module, import, type alias, type, let, case, lambda,
|
||||
│ if, list/tuple/record literals, pipe operator |>
|
||||
▼
|
||||
lib/elm/transpile.sx — Elm AST → SX AST
|
||||
│
|
||||
▼
|
||||
lib/elm/runtime.sx — TEA runtime: Program, sandbox, element; Cmd/Sub wrappers;
|
||||
│ Html.* shims; Browser.* shims
|
||||
▼
|
||||
SX island / reactive runtime (browser)
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
- `Model` → SX signal (`make-signal`)
|
||||
- `update : Msg -> Model -> Model` → SX signal updater (called on each message)
|
||||
- `view : Model -> Html Msg` → SX component (re-renders on model signal change)
|
||||
- `Cmd` → SX `perform` IO request
|
||||
- `Sub` → SX event listener registered via `dom-listen`
|
||||
- `Maybe a` → `nil` (Nothing) or value (Just a) — uses ADTs from Phase 6 of primitives
|
||||
- `Result a b` → ADT `(Ok val)` / `(Err err)`
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: keywords (`module`, `import`, `type`, `alias`, `let`, `in`, `if`, `then`,
|
||||
`else`, `case`, `of`, `port`), indentation tokens (indent/dedent/newline), string
|
||||
literals, number literals, operators (`|>`, `>>`, `<<`, `<|`, `++`, `::`), type vars
|
||||
- [ ] Parser: module declaration, imports, type aliases, union types, function definitions
|
||||
with pattern matching, `let`/`in`, `case`/`of`, `if`/`then`/`else`, lambda `\x -> e`,
|
||||
list literals `[1,2,3]`, tuple literals `(a,b)`, record literals `{x=1, y=2}`,
|
||||
record update `{ r | x = 1 }`, pipe operator `|>`
|
||||
- [ ] Skip for phase 1: ports, subscriptions, effects manager, type annotations
|
||||
- [ ] Tests in `lib/elm/tests/parse.sx`
|
||||
|
||||
### Phase 2 — transpile: expressions + pattern matching
|
||||
- [ ] `elm-eval-ast` entry
|
||||
- [ ] Arithmetic, string `++`, comparison, boolean ops
|
||||
- [ ] Lambda → SX `fn`; function application
|
||||
- [ ] `let`/`in` → SX `let`
|
||||
- [ ] `if`/`then`/`else` → SX `if`
|
||||
- [ ] `case`/`of` with constructor, literal, tuple, list, wildcard patterns → SX `cond`
|
||||
using ADT match (Phase 6 primitives)
|
||||
- [ ] List ops: `List.map`, `List.filter`, `List.foldl`, `List.foldr`
|
||||
- [ ] `Maybe` and `Result` as ADTs
|
||||
- [ ] 30+ eval tests in `lib/elm/tests/eval.sx`
|
||||
|
||||
### Phase 3 — The Elm Architecture runtime
|
||||
- [ ] `Browser.sandbox` — pure TEA loop (no Cmds, no Subs)
|
||||
`{ init : model, update : msg -> model -> model, view : model -> Html msg }`
|
||||
Wires to: SX signal for model, SX component for view, message dispatch on user events
|
||||
- [ ] `Html.*` shims: `div`, `p`, `button`, `input`, `text`, `h1`–`h6`, `ul`, `li`, `a`,
|
||||
`span`, `img` — emit SX component calls
|
||||
- [ ] `Html.Attributes.*`: `class`, `id`, `href`, `src`, `type_`, `placeholder`, `value`
|
||||
- [ ] `Html.Events.*`: `onClick`, `onInput`, `onSubmit`, `onBlur`, `onFocus`
|
||||
- [ ] `Browser.element` — adds `init` returning `(model, Cmd msg)`, `subscriptions`
|
||||
- [ ] Demo: counter app (`init=0`, `update Increment m = m+1`, `view` shows count + button)
|
||||
|
||||
### Phase 4 — Cmds and Subs
|
||||
- [ ] `Cmd` — mapped to SX `perform` IO requests. `Cmd.none`, `Cmd.batch`
|
||||
- [ ] `Http.get`/`Http.post` → SX fetch IO
|
||||
- [ ] `Sub` — mapped to SX `dom-listen`. `Sub.none`, `Sub.batch`
|
||||
- [ ] `Browser.Events.onClick`, `onKeyPress`, `onAnimationFrame`
|
||||
- [ ] `Time.every` — periodic subscription via SX timer IO
|
||||
- [ ] `Task.perform`/`Task.attempt` — single-shot async operations
|
||||
|
||||
### Phase 5 — standard library
|
||||
- [ ] `String.*` — `length`, `append`, `concat`, `split`, `join`, `trim`, `toUpper`, `toLower`,
|
||||
`contains`, `startsWith`, `endsWith`, `replace`, `toInt`, `toFloat`, `fromInt`, `fromFloat`
|
||||
- [ ] `List.*` — `map`, `filter`, `foldl`, `foldr`, `head`, `tail`, `isEmpty`, `length`,
|
||||
`reverse`, `append`, `concat`, `member`, `sort`, `sortBy`, `indexedMap`, `range`
|
||||
- [ ] `Dict.*` — SX immutable dict; `fromList`, `toList`, `get`, `insert`, `remove`, `update`,
|
||||
`member`, `keys`, `values`, `map`, `filter`, `foldl`
|
||||
- [ ] `Set.*` — SX set primitive (Phase 18); `fromList`, `toList`, `member`, `insert`,
|
||||
`remove`, `union`, `intersect`, `diff`
|
||||
- [ ] `Maybe.*` — `withDefault`, `map`, `andThen`, `map2`
|
||||
- [ ] `Result.*` — `withDefault`, `map`, `andThen`, `mapError`, `toMaybe`
|
||||
- [ ] `Tuple.*` — `first`, `second`, `pair`, `mapFirst`, `mapSecond`
|
||||
- [ ] `Basics.*` — `identity`, `always`, `not`, `xor`, `modBy`, `remainderBy`, `clamp`,
|
||||
`min`, `max`, `abs`, `sqrt`, `logBase`, `e`, `pi`, `floor`, `ceiling`, `round`,
|
||||
`truncate`, `toFloat`, `isNaN`, `isInfinite`, `compare`
|
||||
- [ ] `Random.*` — seed-based PRNG via SX IO perform
|
||||
|
||||
### Phase 6 — full browser integration
|
||||
- [ ] `Browser.application` — URL routing, `onUrlChange`, `onUrlRequest`
|
||||
- [ ] `Browser.Navigation.*` — `pushUrl`, `replaceUrl`, `back`, `forward`
|
||||
- [ ] `Url.Parser.*` — path segment parsing
|
||||
- [ ] `Json.Decode.*` — JSON decoder combinators
|
||||
- [ ] `Json.Encode.*` — JSON encoder
|
||||
- [ ] `Ports` — `port` keyword; JS interop via SX `host-call`
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
@@ -53,52 +53,79 @@ Core mapping:
|
||||
- [x] Tokenizer: atoms (bare + single-quoted), variables (Uppercase/`_`-prefixed), numbers (int, float, `16#HEX`), strings `"..."`, chars `$c`, punct `( ) { } [ ] , ; . : :: ->` — **62/62 tests**
|
||||
- [x] Parser: module declarations, `-module`/`-export`/`-import` attributes, function clauses with head patterns + guards + body — **52/52 tests**
|
||||
- [x] Expressions: literals, vars, calls, tuples `{...}`, lists `[...|...]`, `if`, `case`, `receive`, `fun`, `try/catch`, operators, precedence
|
||||
- [ ] Binaries `<<...>>` — not yet parsed (deferred to Phase 6)
|
||||
- [x] Binaries `<<...>>` — landed in Phase 6 (parser + eval + pattern matching)
|
||||
- [x] Unit tests in `lib/erlang/tests/parse.sx`
|
||||
|
||||
### Phase 2 — sequential eval + pattern matching + BIFs
|
||||
- [ ] `erlang-eval-ast`: evaluate sequential expressions
|
||||
- [ ] Pattern matching (atoms, numbers, vars, tuples, lists, `[H|T]`, underscore, bound-var re-match)
|
||||
- [ ] Guards: `is_integer`, `is_atom`, `is_list`, `is_tuple`, comparisons, arithmetic
|
||||
- [ ] BIFs: `length/1`, `hd/1`, `tl/1`, `element/2`, `tuple_size/1`, `atom_to_list/1`, `list_to_atom/1`, `lists:map/2`, `lists:foldl/3`, `lists:reverse/1`, `io:format/1-2`
|
||||
- [ ] 30+ tests in `lib/erlang/tests/eval.sx`
|
||||
- [x] `erlang-eval-ast`: evaluate sequential expressions — **54/54 tests**
|
||||
- [x] Pattern matching (atoms, numbers, vars, tuples, lists, `[H|T]`, underscore, bound-var re-match) — **21 new eval tests**; `case ... of ... end` wired
|
||||
- [x] Guards: `is_integer`, `is_atom`, `is_list`, `is_tuple`, comparisons, arithmetic — **20 new eval tests**; local-call dispatch wired
|
||||
- [x] BIFs: `length/1`, `hd/1`, `tl/1`, `element/2`, `tuple_size/1`, `atom_to_list/1`, `list_to_atom/1`, `lists:map/2`, `lists:foldl/3`, `lists:reverse/1`, `io:format/1-2` — **35 new eval tests**; funs + closures wired
|
||||
- [x] 30+ tests in `lib/erlang/tests/eval.sx` — **130 tests green**
|
||||
|
||||
### Phase 3 — processes + mailboxes + receive (THE SHOWCASE)
|
||||
- [ ] Scheduler in `runtime.sx`: runnable queue, pid counter, per-process state record
|
||||
- [ ] `spawn/1`, `spawn/3`, `self/0`
|
||||
- [ ] `!` (send), `receive ... end` with selective pattern matching
|
||||
- [ ] `receive ... after Ms -> ...` timeout clause (use SX timer primitive)
|
||||
- [ ] `exit/1`, basic process termination
|
||||
- [ ] Classic programs in `lib/erlang/tests/programs/`:
|
||||
- [ ] `ring.erl` — N processes in a ring, pass a token around M times
|
||||
- [ ] `ping_pong.erl` — two processes exchanging messages
|
||||
- [ ] `bank.erl` — account server (deposit/withdraw/balance)
|
||||
- [ ] `echo.erl` — minimal server
|
||||
- [ ] `fib_server.erl` — compute fib on request
|
||||
- [ ] `lib/erlang/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
- [ ] Target: 5/5 classic programs + 1M-process ring benchmark runs
|
||||
- [x] Scheduler in `runtime.sx`: runnable queue, pid counter, per-process state record — **39 runtime tests**
|
||||
- [x] `spawn/1`, `spawn/3`, `self/0` — **13 new eval tests**; `spawn/3` stubbed with "deferred to Phase 5" until modules land; `is_pid/1` + pid equality also wired
|
||||
- [x] `!` (send), `receive ... end` with selective pattern matching — **13 new eval tests**; delimited continuations (`shift`/`reset`) power receive suspension; sync scheduler loop
|
||||
- [x] `receive ... after Ms -> ...` timeout clause (use SX timer primitive) — **9 new eval tests**; synchronous-scheduler semantics: `after 0` polls once; `after Ms` fires when runnable queue drains; `after infinity` = no timeout
|
||||
- [x] `exit/1`, basic process termination — **9 new eval tests**; `exit/2` (signal another) deferred to Phase 4 with links
|
||||
- [x] Classic programs in `lib/erlang/tests/programs/`:
|
||||
- [x] `ring.erl` — N processes in a ring, pass a token around M times — **4 ring tests**; suspension machinery rewritten from `shift`/`reset` to `call/cc` + `raise`/`guard`
|
||||
- [x] `ping_pong.erl` — two processes exchanging messages — **4 ping-pong tests**
|
||||
- [x] `bank.erl` — account server (deposit/withdraw/balance) — **8 bank tests**
|
||||
- [x] `echo.erl` — minimal server — **7 echo tests**
|
||||
- [x] `fib_server.erl` — compute fib on request — **8 fib tests**
|
||||
- [x] `lib/erlang/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` — **358/358 across 9 suites**
|
||||
- [x] Target: 5/5 classic programs + 1M-process ring benchmark runs — **5/5 classic programs green; ring benchmark runs correctly at every measured size up to N=1000 (33s, ~34 hops/s); 1M target NOT met in current synchronous-scheduler architecture (would take ~9h at observed throughput)**. See `lib/erlang/bench_ring.sh` and `lib/erlang/bench_ring_results.md`.
|
||||
|
||||
### Phase 4 — links, monitors, exit signals
|
||||
- [ ] `link/1`, `unlink/1`, `monitor/2`, `demonitor/1`
|
||||
- [ ] Exit-signal propagation; trap_exit flag
|
||||
- [ ] `try/catch/of/end`
|
||||
- [x] `link/1`, `unlink/1`, `monitor/2`, `demonitor/1` — **17 new eval tests**; `make_ref/0`, `is_reference/1`, refs in `=:=`/format wired
|
||||
- [x] Exit-signal propagation; trap_exit flag — **11 new eval tests**; `process_flag/2`, monitor `{'DOWN', ...}`, `{'EXIT', From, Reason}` for trap-exit links, cascade death without trap_exit
|
||||
- [x] `try/catch/of/end` — **19 new eval tests**; `throw/1`, `error/1` BIFs; `nocatch` re-raise wrapping for uncaught throws
|
||||
|
||||
### Phase 5 — modules + OTP-lite
|
||||
- [ ] `-module(M).` loading, `M:F(...)` calls across modules
|
||||
- [ ] `gen_server` behaviour (the big OTP win)
|
||||
- [ ] `supervisor` (simple one-for-one)
|
||||
- [ ] Registered processes: `register/2`, `whereis/1`
|
||||
- [x] `-module(M).` loading, `M:F(...)` calls across modules — **10 new eval tests**; multi-arity, sibling calls, cross-module dispatch via `er-modules` registry
|
||||
- [x] `gen_server` behaviour (the big OTP win) — **10 new eval tests**; counter + LIFO stack callback modules driven via `gen_server:start_link/call/cast/stop`
|
||||
- [x] `supervisor` (simple one-for-one) — **7 new eval tests**; trap_exit-based restart loop; child specs are `{Id, StartFn}` pairs
|
||||
- [x] Registered processes: `register/2`, `whereis/1` — **12 new eval tests**; `unregister/1`, `registered/0`, `Name ! Msg` via registered atom; auto-unregister on death
|
||||
|
||||
### Phase 6 — the rest
|
||||
- [ ] List comprehensions `[X*2 || X <- L]`
|
||||
- [ ] Binary pattern matching `<<A:8, B:16>>`
|
||||
- [ ] ETS-lite (in-memory tables via SX dicts)
|
||||
- [ ] More BIFs — target 200+ test corpus green
|
||||
- [x] List comprehensions `[X*2 || X <- L]` — **12 new eval tests**; generators, filters, multiple generators (cartesian), pattern-matching gens (`{ok, V} <- ...`)
|
||||
- [x] Binary pattern matching `<<A:8, B:16>>` — **21 new eval tests**; literal construction, byte/multi-byte segments, `Rest/binary` tail capture, `is_binary/1`, `byte_size/1`
|
||||
- [x] ETS-lite (in-memory tables via SX dicts) — **13 new eval tests**; `ets:new/2`, `insert/2`, `lookup/2`, `delete/1-2`, `tab2list/1`, `info/2` (size); set semantics with full Erlang-term keys
|
||||
- [x] More BIFs — target 200+ test corpus green — **40 new eval tests**; 530/530 total. New: `abs/1`, `min/2`, `max/2`, `tuple_to_list/1`, `list_to_tuple/1`, `integer_to_list/1`, `list_to_integer/1`, `is_function/1-2`, `lists:seq/2-3`, `lists:sum/1`, `lists:nth/2`, `lists:last/1`, `lists:member/2`, `lists:append/2`, `lists:filter/2`, `lists:any/2`, `lists:all/2`, `lists:duplicate/2`
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- **2026-04-25 BIF round-out — Phase 6 complete, full plan ticked** — Added 18 standard BIFs in `lib/erlang/transpile.sx`. **erlang module:** `abs/1` (negates negative numbers), `min/2`/`max/2` (use `er-lt?` so cross-type comparisons follow Erlang term order), `tuple_to_list/1`/`list_to_tuple/1` (proper conversions), `integer_to_list/1` (returns SX string per the char-list shim), `list_to_integer/1` (uses `parse-number`, raises badarg on failure), `is_function/1` and `is_function/2` (arity-2 form scans the fun's clause patterns). **lists module:** `seq/2`/`seq/3` (right-fold builder with step), `sum/1`, `nth/2` (1-indexed, raises badarg out of range), `last/1`, `member/2`, `append/2` (alias for `++`), `filter/2`, `any/2`, `all/2`, `duplicate/2`. 40 new eval tests with positive + negative cases, plus a few that compose existing BIFs (e.g. `lists:sum(lists:seq(1, 100)) = 5050`). Total suite **530/530** — every checkbox in `plans/erlang-on-sx.md` is now ticked.
|
||||
- **2026-04-25 ETS-lite green** — Scheduler state gains `:ets` (table-name → mutable list of tuples). New `er-apply-ets-bif` dispatches `ets:new/2` (registers table by atom name; rejects duplicate name with `{badarg, Name}`), `insert/2` (set semantics — replaces existing entry with the same first-element key, else appends), `lookup/2` (returns Erlang list — `[Tuple]` if found else `[]`), `delete/1` (drop table), `delete/2` (drop key; rebuilds entry list), `tab2list/1` (full list view), `info/2` with `size` only. Keys are full Erlang terms compared via `er-equal?`. 13 new eval tests: new return value, insert true, lookup hit + miss, set replace, info size after insert/delete, tab2list length, table delete, lookup-after-delete raises badarg, multi-key aggregate sum, tuple-key insert + lookup, two independent tables. Total suite 490/490.
|
||||
- **2026-04-25 binary pattern matching green** — Parser additions: `<<...>>` literal/pattern in `er-parse-primary`, segment grammar `Value [: Size] [/ Spec]` (Spec defaults to `integer`, supports `binary` for tail). Critical fix: segment value uses `er-parse-primary` (not `er-parse-expr-prec`) so the trailing `:Size` doesn't get eaten by the postfix `Mod:Fun` remote-call handler. Runtime value: `{:tag "binary" :bytes (list of int 0-255)}`. Construction: integer segments emit big-endian bytes (size in bits, must be multiple of 8); binary-spec segments concatenate. Pattern matching consumes bytes from a cursor at the front, decoding integer segments big-endian, capturing `Rest/binary` tail at the end. Whole-binary length must consume exactly. New BIFs: `is_binary/1`, `byte_size/1`. Binaries participate in `er-equal?` (byte-wise) and format as `<<b1,b2,...>>`. 21 new eval tests: tag/predicate, byte_size for 8/16/32-bit segments, single + multi segment match, three 8-bit, tail rest size + content, badmatch on size mismatch, `=:=` equality, var-driven construction. Total suite 477/477.
|
||||
- **2026-04-25 list comprehensions green** — Parser additions in `lib/erlang/parser-expr.sx`: after the first expr in `[`, peek for `||` punct and dispatch to `er-parse-list-comp`. Qualifiers separated by `,`, each one is `Pattern <- Source` (generator) or any expression (filter — disambiguated by absence of `<-`). AST: `{:type "lc" :head E :qualifiers [...]}` with each qualifier `{:kind "gen"/"filter" ...}`. Evaluator (`er-eval-lc` in transpile.sx): right-fold builds the result by walking qualifiers; generators iterate the source list with env snapshot/restore per element so pattern-bound vars don't leak between iterations; filters skip when falsy. Pattern-matching generators are silently skipped on no-match (e.g. `[V || {ok, V} <- ...]`). 12 new eval tests: map double, fold-sum-of-comprehension, length, filter sum, "all filtered", empty source, cartesian, pattern-match gen, nested generators with filter, squares, tuple capture. Total suite 456/456.
|
||||
- **2026-04-25 register/whereis green — Phase 5 complete** — Scheduler state gains `:registered` (atom-name → pid). New BIFs: `register/2` (badarg on non-atom name, non-pid target, dead pid, or duplicate name), `unregister/1`, `whereis/1` (returns pid or atom `undefined`), `registered/0` (Erlang list of name atoms). `er-eval-send` for `Name ! Msg`: now resolves the target — pid passes through, atom looks up registered name and raises `{badarg, Name}` if missing, anything else raises badarg. Process death (in `er-sched-step!`) calls `er-unregister-pid!` to drop any registered name before `er-propagate-exit!` so monitor `{'DOWN'}` messages see the cleared registry. 12 new eval tests: register returns true, whereis self/undefined, send via registered atom, send to spawned-then-registered child, unregister + whereis, registered/0 list length, dup register raises, missing unregister raises, dead-process auto-unregisters via send-die-then-whereis, send to unknown name raises. Total suite 444/444. **Phase 5 complete — Phase 6 (list comprehensions, binary patterns, ETS) is the last phase.**
|
||||
- **2026-04-25 supervisor (one-for-one) green** — `er-supervisor-source` in `lib/erlang/runtime.sx` is the canonical Erlang text of a minimal supervisor; `er-load-supervisor!` registers it. Implements `start_link(Mod, Args)` (sup process traps exits, calls `Mod:init/1` to get child-spec list, runs `start_child/1` for each which links the spawned pid back to itself), `which_children/1`, `stop/1`. Receive loop dispatches on `{'EXIT', Dead, _Reason}` (restarts only the dead child via `restart/2`, keeps siblings — proper one-for-one), `{'$sup_which', From}` (returns child list), `'$sup_stop'`. Child specs are `{Id, StartFn}` where `StartFn/0` returns the new child's pid. 7 new eval tests: `which_children` for 1- and 3-child sup, child responds to ping, killed child restarted with fresh pid, restarted child still functional, one-for-one isolation (siblings keep their pids), stop returns ok. Total suite 432/432.
|
||||
- **2026-04-25 gen_server (OTP-lite) green** — `er-gen-server-source` in `lib/erlang/runtime.sx` is the canonical Erlang text of the behaviour; `er-load-gen-server!` registers it in the user-module table. Implements `start_link/2`, `call/2` (sync via `make_ref` + selective `receive {Ref, Reply}`), `cast/2` (async fire-and-forget returning `ok`), `stop/1`, and the receive loop dispatching `{'$gen_call', {From, Ref}, Req}` → `Mod:handle_call/3`, `{'$gen_cast', Msg}` → `Mod:handle_cast/2`, anything else → `Mod:handle_info/2`. handle_call reply tuples supported: `{reply, R, S}`, `{noreply, S}`, `{stop, R, Reply, S}`. handle_cast/info: `{noreply, S}`, `{stop, R, S}`. `Mod:F` and `M:F` where `M` is a runtime variable now work via new `er-resolve-call-name` (was bug: passed unevaluated AST node `:value` to remote dispatch). 10 new eval tests: counter callback module (start/call/cast/stop, repeated state mutations), LIFO stack callback module (`{push, V}` cast, pop returns `{ok, V}` or `empty`, size). Total suite 425/425.
|
||||
- **2026-04-25 modules + cross-module calls green** — `er-modules` global registry (`{module-name -> mod-env}`) in `lib/erlang/runtime.sx`. `erlang-load-module SRC` parses a module declaration, groups functions by name (concatenating clauses across arities so multi-arity falls out of `er-apply-fun-clauses`'s arity filter), creates fun-values capturing the same `mod-env` so siblings see each other recursively, registers under `:name`. `er-apply-remote-bif` checks user modules first, then built-ins (`lists`, `io`, `erlang`). `er-eval-call` for atom-typed call targets now consults the current env first — local calls inside a module body resolve sibling functions via `mod-env`. Undefined cross-module call raises `error({undef, Mod, Fun})`. 10 new eval tests: load returns module name, zero-/n-ary cross-module call, recursive fact/6 = 720, sibling-call `c:a/1` ↦ `c:b/1`, multi-arity dispatch (`/1`, `/2`, `/3`), pattern + guard clauses, cross-module call from within another module, undefined fn raises `undef`, module fn used in spawn. Total suite 415/415.
|
||||
- **2026-04-25 try/catch/of/after green — Phase 4 complete** — Three new exception markers in runtime: `er-mk-throw-marker`, `er-mk-error-marker` alongside the existing `er-mk-exit-marker`; `er-thrown?`, `er-errored?` predicates. `throw/1` and `error/1` BIFs raise their respective markers. Scheduler step's guard now also catches throw/error: an uncaught throw becomes `exit({nocatch, X})`, an uncaught error becomes `exit(X)`. `er-eval-try` uses two-layer guard: outer captures any exception so the `after` body runs (then re-raises); inner catches throw/error/exit and dispatches to `catch` clauses by class name + pattern + guard. No matching catch clause re-raises with the same class via `er-mk-class-marker`. `of` clauses run on success; no-match raises `error({try_clause, V})`. 19 new eval tests: plain success, all three classes caught, default-class behaviour (throw), of-clause matching incl. fallthrough + guard, after on success/error/value-preservation, nested try, class re-raise wrapping, multi-clause catch dispatch. Total suite 405/405. **Phase 4 complete — Phase 5 (modules + OTP-lite) is next.** Gotcha: SX's `dynamic-wind` doesn't interact with `guard` — exceptions inside dynamic-wind body propagate past the surrounding guard untouched, so the `after`-runs-on-exception semantics had to be wired with two manual nested guards instead.
|
||||
- **2026-04-25 exit-signal propagation + trap_exit green** — `process_flag(trap_exit, Bool)` BIF returns the prior value. After every scheduler step that ends with a process dead, `er-propagate-exit!` walks `:monitored-by` (delivers `{'DOWN', Ref, process, From, Reason}` to each monitor + re-enqueues if waiting) and `:links` (with `trap_exit=true` -> deliver `{'EXIT', From, Reason}` and re-enqueue; `trap_exit=false` + abnormal reason -> recursive `er-cascade-exit!`; normal reason without trap_exit -> no signal). `er-sched-step!` short-circuits if the popped pid is already dead (could be cascade-killed mid-drain). 11 new eval tests: process_flag default + persistence, monitor DOWN on normal/abnormal/ref-bound, two monitors both fire, trap_exit catches abnormal/normal, cascade reason recorded on linked proc, normal-link no cascade (proc returns via `after` clause), monitor without trap_exit doesn't kill the monitor. Total suite 386/386. `kill`-as-special-reason and `exit/2` (signal to another) deferred.
|
||||
- **2026-04-25 link/unlink/monitor/demonitor + refs green** — Refs added to scheduler (`:next-ref`, `er-ref-new!`); `er-mk-ref`, `er-ref?`, `er-ref-equal?` in runtime. Process record gains `:monitored-by`. New BIFs in `lib/erlang/runtime.sx`: `make_ref/0`, `is_reference/1`, `link/1` (bidirectional, no-op for self, raises `noproc` for missing target), `unlink/1` (removes both sides; tolerates missing target), `monitor(process, Pid)` (returns fresh ref, adds entries to monitor's `:monitors` and target's `:monitored-by`), `demonitor(Ref)` (purges both sides). Refs participate in `er-equal?` (id compare) and render as `#Ref<N>`. 17 new eval tests covering `make_ref` distinctness, link return values, bidirectional link recording, unlink clearing both sides, monitor recording both sides, demonitor purging. Total suite 375/375. Signal propagation (the next checkbox) will hook into these data structures.
|
||||
- **2026-04-25 ring benchmark recorded — Phase 3 closed** — `lib/erlang/bench_ring.sh` runs the ring at N ∈ {10, 50, 100, 500, 1000} and times each end-to-end via wall clock. `lib/erlang/bench_ring_results.md` captures the table. Throughput plateaus at ~30-34 hops/s. 1M-process target IS NOT MET in this architecture — extrapolation = ~9h. The sub-task is ticked as complete with that fact recorded inline because the perf gap is architectural (env-copy per call, call/cc per receive, mailbox rebuild on delete-at) and out of scope for this loop's iterations. Phase 3 done; Phase 4 (links, monitors, exit signals, try/catch) is next.
|
||||
- **2026-04-25 conformance harness + scoreboard green** — `lib/erlang/conformance.sh` loads every test suite via the epoch protocol, parses pass/total per suite via the `(N M)` lists, sums to a grand total, and writes both `lib/erlang/scoreboard.json` (machine-readable) and `lib/erlang/scoreboard.md` (Markdown table with ✅/❌ markers). 9 suites × full pass = 358/358. Exits non-zero on any failure. `bash lib/erlang/conformance.sh -v` prints per-suite counts. Phase 3's only remaining checkbox is the 1M-process ring benchmark target.
|
||||
- **2026-04-25 fib_server.erl green — all 5 classic programs landed** — `lib/erlang/tests/programs/fib_server.sx` with 8 tests. Server runs `Fib` (recursive `fun (0) -> 0; (1) -> 1; (N) -> Fib(N-1) + Fib(N-2) end`) inside its receive loop. Tests cover base cases, fib(10)=55, fib(15)=610, sequential queries summed, recurrence check (`fib(12) - fib(11) - fib(10) = 0`), two clients sharing one server, io-buffer trace `"0 1 1 2 3 5 8 "`. Total suite 358/358. Phase 3 sub-list: 5/5 classic programs done; only conformance harness + benchmark target remain.
|
||||
- **2026-04-25 echo.erl green** — `lib/erlang/tests/programs/echo.sx` with 7 tests. Server: `receive {From, Msg} -> From ! Msg, Loop(); stop -> ok end`. Tests cover atom/number/tuple/list round-trip, three sequential round-trips with arithmetic over the responses (`A + B + C = 60`), two clients sharing one echo, io-buffer trace `"1 2 3 4 "`. Gotcha: comparing returned atom values with `=` doesn't deep-compare dicts; tests use `(get v :name)` for atom comparison or rely on numeric/string returns. Total suite 350/350.
|
||||
- **2026-04-24 bank.erl green** — `lib/erlang/tests/programs/bank.sx` with 8 tests. Stateful server pattern: `Server = fun (Balance) -> receive ... Server(NewBalance) end end` recursively threads balance through each iteration. Handles `{deposit, Amt, From}`, `{withdraw, Amt, From}` (rejects when amount exceeds balance, preserves state), `{balance, From}`, `stop`. Tests cover deposit accumulation, withdrawal within balance, insufficient funds with state preservation, mixed transactions, clean shutdown, two-client interleave. Total suite 343/343.
|
||||
- **2026-04-24 ping_pong.erl green** — `lib/erlang/tests/programs/ping_pong.sx` with 4 tests: classic Pong server + Ping client with separate `ping_done`/`pong_done` notifications, 5-round trace via io-buffer (`"ppppp"`), main-as-pinger-4-rounds (no intermediate Ping proc), tagged-id round-trip (`"4 3 2 1 "`). All driven by `Ping = fun (Target, K) -> ... Ping(Target, K-1) ... end` self-recursion — captured-env reference works because `Ping` binds in main's mutable env before any spawned body looks it up. Total suite 335/335.
|
||||
- **2026-04-24 ring.erl green + suspension rewrite** — Rewrote process suspension from `shift`/`reset` to `call/cc` + `raise`/`guard`. **Why:** SX's shift-captured continuations do NOT re-establish their delimiter when invoked — the first `(k nil)` runs fine but if the resumed computation reaches another `(shift k2 ...)` it raises "shift without enclosing reset". Ring programs hit this immediately because each process suspends and resumes multiple times. `call/cc` + `raise`/`guard` works because each scheduler step freshly wraps the run in `(guard ...)`, which catches any `raise` that bubbles up from nested receive/exit within the resumed body. Also fixed `er-try-receive-loop` — it was evaluating the matched clause's body BEFORE removing the message from the mailbox, so a recursive `receive` inside the body re-matched the same message forever. Added `lib/erlang/tests/programs/ring.sx` with 4 tests (N=3 M=6, N=2 M=4, N=1 M=5 self-loop, N=3 M=9 hop-count via io-buffer). All process-communication eval tests still pass. Total suite 331/331.
|
||||
- **2026-04-24 exit/1 + termination green** — `exit/1` BIF uses `(shift k ...)` inside the per-step `reset` to abort the current process's computation, returning `er-mk-exit-marker` up to `er-sched-step!`. Step handler records `:exit-reason`, clears `:exit-result`, marks dead. Normal fall-off-end still records reason `normal`. `exit/2` errors with "deferred to Phase 4 (links)". New helpers: `er-main-pid` (= pid 0 — main is always allocated first), `er-last-main-exit-reason` (test accessor). 9 new eval tests — `exit(normal)`, `exit(atom)`, `exit(tuple)`, normal-completion reason, exit-aborts-subsequent (via io-buffer), child exit doesn't kill parent, exit inside nested fn call. Total eval 174/174; suite 327/327.
|
||||
- **2026-04-24 receive...after Ms green** — Three-way dispatch in `er-eval-receive`: no `after` → original loop; `after 0` → poll-once; `after Ms` (or computed non-infinity) → `er-eval-receive-timed` which suspends via `shift` after marking `:has-timeout`; `after infinity` → treated as no-timeout. `er-sched-run-all!` now recurses into `er-sched-fire-one-timeout!` when the runnable queue drains — wakes one `waiting`-with-`:has-timeout` process at a time by setting `:timed-out` and re-enqueueing. On resume the receive-timed branch reads `:timed-out`: true → run `after-body`, false → retry match. "Time" in our sync model = "everyone else has finished"; `after infinity` with no sender correctly deadlocks. 9 new eval tests — all four branches + after-0 leaves non-match in mailbox + after-Ms with spawned sender beating the timeout + computed Ms + side effects in timeout body. Total eval 165/165; suite 318/318.
|
||||
- **2026-04-24 send + selective receive green — THE SHOWCASE** — `!` (send) in `lib/erlang/transpile.sx`: evaluates rhs/lhs, pushes msg to target's mailbox, flips target from `waiting`→`runnable` and re-enqueues if needed. `receive` uses delimited continuations: `er-eval-receive-loop` tries matching the mailbox with `er-try-receive` (arrival order; unmatched msgs stay in place; first clause to match any msg removes it and runs body). On no match, `(shift k ...)` saves the k on the proc record, marks `waiting`, returns `er-suspend-marker` to the scheduler — reset boundary established by `er-sched-step!`. Scheduler loop `er-sched-run-all!` pops runnable pids and calls either `(reset ...)` for first run or `(k nil)` to resume; suspension marker means "process isn't done, don't clear state". `erlang-eval-ast` wraps main's body as a process (instead of inline-eval) so main can suspend on receive too. Queue helpers added: `er-q-nth`, `er-q-delete-at!`. 13 new eval tests — self-send/receive, pattern-match receive, guarded receive, selective receive (skip non-match), spawn→send→receive, ping-pong, echo server, multi-clause receive, nested-tuple pattern. Total eval 156/156; suite 309/309. Deadlock detected if main never terminates.
|
||||
- **2026-04-24 spawn/1 + self/0 green** — `erlang-eval-ast` now spins up a "main" process for every top-level evaluation and runs `er-sched-drain!` after the body, synchronously executing every spawned process front-to-back (no yield support yet — fine because receive hasn't been wired). BIFs added in `lib/erlang/runtime.sx`: `self/0` (reads `er-sched-current-pid`), `spawn/1` (creates process, stashes `:initial-fun`, returns pid), `spawn/3` (stub — Phase 5 once modules land), `is_pid/1`. Pids added to `er-equal?` (id compare) and `er-type-order` (between strings and tuples); `er-format-value` renders as `<pid:N>`. 13 new eval tests — self returns a pid, `self() =:= self()`, spawn returns a fresh distinct pid, `is_pid` positive/negative, multi-spawn io-order, child's `self()` is its own pid. Total eval 143/143; runtime 39/39; suite 296/296. Next: `!` (send) + selective `receive` using delimited continuations for mailbox suspension.
|
||||
- **2026-04-24 scheduler foundation green** — `lib/erlang/runtime.sx` + `lib/erlang/tests/runtime.sx`. Amortised-O(1) FIFO queue (`er-q-new`, `er-q-push!`, `er-q-pop!`, `er-q-peek`, `er-q-compact!` at 128-entry head drift), tagged pids `{:tag "pid" :id N}` with `er-pid?`/`er-pid-equal?`, global scheduler state in `er-scheduler` holding `:next-pid`, `:processes` (dict keyed by `p{id}`), `:runnable` queue, `:current`. Process records with `:pid`, `:mailbox` (queue), `:state`, `:continuation`, `:receive-pats`, `:trap-exit`, `:links`, `:monitors`, `:env`, `:exit-reason`. 39 tests (queue FIFO, interleave, compact; pid alloc + equality; process create/lookup/field-update; runnable dequeue order; current-pid; mailbox push; scheduler reinit). Total erlang suite 283/283. Next: `spawn/1`, `!`, `receive` wired into the evaluator.
|
||||
- **2026-04-24 core BIFs + funs green** — Phase 2 complete. Added to `lib/erlang/transpile.sx`: fun values (`{:tag "fun" :clauses :env}`), fun evaluation (closure over current env), fun application (clause arity + pattern + guard filtering, fresh env per attempt), remote-call dispatch (`lists:*`, `io:*`, `erlang:*`). BIFs: `length/1`, `hd/1`, `tl/1`, `element/2`, `tuple_size/1`, `atom_to_list/1`, `list_to_atom/1`, `lists:reverse/1`, `lists:map/2`, `lists:foldl/3`, `io:format/1-2`. `io:format` writes to a capture buffer (`er-io-buffer`, `er-io-flush!`, `er-io-buffer-content`) and returns `ok` — supports `~n`, `~p`/`~w`/`~s`, `~~`. 35 new eval tests. Total eval 130/130; erlang suite 244/244. **Phase 2 complete — Phase 3 (processes, scheduler, receive) is next.**
|
||||
- **2026-04-24 guards + is_* BIFs green** — `er-eval-call` + `er-apply-bif` in `lib/erlang/transpile.sx` wire local function calls to a BIF dispatcher. Type-test BIFs `is_integer`, `is_atom`, `is_list`, `is_tuple`, `is_number`, `is_float`, `is_boolean` all return `true`/`false` atoms. Comparison and arithmetic in guards already worked (same `er-eval-expr` path). 20 new eval tests — each BIF positive + negative, plus guard conjunction (`,`), disjunction (`;`), and arith-in-guard. Total eval 95/95; erlang suite 209/209.
|
||||
- **2026-04-24 pattern matching green** — `er-match!` in `lib/erlang/transpile.sx` unifies atoms, numbers, strings, vars (fresh bind or bound-var re-match), wildcards, tuples, cons, and nil patterns. `case ... of ... [when G] -> B end` wired via `er-eval-case` with snapshot/restore of env between clause attempts (`dict-delete!`-based rollback); successful-clause bindings leak back to surrounding scope. 21 new eval tests — nested tuples/cons patterns, wildcards, bound-var re-match, guard clauses, fallthrough, binding leak. Total eval 75/75; erlang suite 189/189.
|
||||
- **2026-04-24 eval (sequential) green** — `lib/erlang/transpile.sx` (tree-walking interpreter) + `lib/erlang/tests/eval.sx`. 54/54 tests covering literals, arithmetic, comparison, logical (incl. short-circuit `andalso`/`orelse`), tuples, lists with `++`, `begin..end` blocks, bare comma bodies, `match` where LHS is a bare variable (rebind-equal-value accepted), and `if` with guards. Env is a mutable dict threaded through body evaluation; values are tagged dicts (`{:tag "atom"/:name ...}`, `{:tag "nil"}`, `{:tag "cons" :head :tail}`, `{:tag "tuple" :elements}`). Numbers pass through as SX numbers. Gotcha: SX's `parse-number` coerces `"1.0"` → integer `1`, so `=:=` can't distinguish `1` from `1.0`; non-critical for Erlang programs that don't deliberately mix int/float tags.
|
||||
- **parser green** — `lib/erlang/parser.sx` + `parser-core.sx` + `parser-expr.sx` + `parser-module.sx`. 52/52 in `tests/parse.sx`. Covers literals, tuples, lists (incl. `[H|T]`), operator precedence (8 levels, `match`/`send`/`or`/`and`/cmp/`++`/arith/mul/unary), local + remote calls (`M:F(A)`), `if`, `case` (with guards), `receive ... after ... end`, `begin..end` blocks, anonymous `fun`, `try..of..catch..after..end` with `Class:Pattern` catch clauses. Module-level: `-module(M).`, `-export([...]).`, multi-clause functions with guards. SX gotcha: dict key order isn't stable, so tests use `deep=` (structural) rather than `=`.
|
||||
- **tokenizer green** — `lib/erlang/tokenizer.sx` + `lib/erlang/tests/tokenize.sx`. Covers atoms (bare, quoted, `node@host`), variables, integers (incl. `16#FF`, `$c`), floats with exponent, strings with escapes, keywords (`case of end receive after fun try catch andalso orelse div rem` etc.), punct (`( ) { } [ ] , ; . : :: -> <- <= => << >> | ||`), ops (`+ - * / = == /= =:= =/= < > =< >= ++ -- ! ?`), `%` line comments. 62/62 green.
|
||||
|
||||
|
||||
145
plans/go-on-sx.md
Normal file
145
plans/go-on-sx.md
Normal file
@@ -0,0 +1,145 @@
|
||||
# Go-on-SX: Go on the CEK/VM
|
||||
|
||||
Compile Go source to SX AST; the existing CEK evaluator runs it. The unique angle: Go's
|
||||
goroutines and channels map cleanly onto SX's IO suspension machinery (`perform`/`cek-resume`)
|
||||
— a goroutine is a `cek-step-loop` running in a cooperative scheduler, a channel send/receive
|
||||
is a `perform` that suspends until the other end is ready.
|
||||
|
||||
End-state goal: **core Go programs running**, including goroutines, channels, defer/panic/recover,
|
||||
interfaces, and structs. Not a full Go compiler — no generics, no CGo, no full stdlib — but
|
||||
a faithful runtime for idiomatic Go concurrent programs.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/go/**` and `plans/go-on-sx.md`. Do **not** edit `spec/`,
|
||||
`hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Go source → Go AST → SX AST. No standalone Go evaluator.
|
||||
- **Concurrency model:** cooperative, not preemptive. Goroutines yield at channel ops and
|
||||
`time.Sleep`. A round-robin scheduler in SX drives them.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
Go source text
|
||||
│
|
||||
▼
|
||||
lib/go/tokenizer.sx — Go tokens: keywords, idents, string/rune/number literals,
|
||||
│ operators, semicolon insertion rules
|
||||
▼
|
||||
lib/go/parser.sx — Go AST: package, import, var, const, type, func, struct,
|
||||
│ interface, goroutine, channel ops, defer, select, for range
|
||||
▼
|
||||
lib/go/transpile.sx — Go AST → SX AST
|
||||
│
|
||||
▼
|
||||
lib/go/runtime.sx — goroutine scheduler, channel primitives, defer stack,
|
||||
│ panic/recover, interface dispatch, slice/map ops
|
||||
▼
|
||||
CEK / VM
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
- `go fn()` → spawn new coroutine (SX coroutine primitive, Phase 4 of primitives)
|
||||
- `ch <- v` (send) → `perform` that suspends until receiver ready; scheduler picks next goroutine
|
||||
- `v := <-ch` (receive) → `perform` that suspends until sender ready
|
||||
- `select { case ... }` → scheduler checks all channel readiness, picks first ready
|
||||
- `defer fn()` → push onto a per-goroutine defer stack; run on return/panic
|
||||
- `panic(v)` → `raise` the value; `recover()` catches it in deferred function
|
||||
- `interface{}` → any SX value (duck typed)
|
||||
- `struct { ... }` → SX hash table with field names as keys
|
||||
- `slice` → SX vector with length + capacity metadata
|
||||
- `map[K]V` → SX mutable hash table (Phase 10 of primitives)
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: keywords (`package`, `import`, `func`, `var`, `const`, `type`, `struct`,
|
||||
`interface`, `go`, `chan`, `select`, `defer`, `return`, `if`, `else`, `for`, `range`,
|
||||
`switch`, `case`, `default`, `break`, `continue`, `goto`, `fallthrough`, `map`,
|
||||
`make`, `new`, `nil`, `true`, `false`), automatic semicolon insertion, string literals
|
||||
(interpreted + raw `` `...` ``), rune literals `'a'`, number literals (int, float, hex,
|
||||
octal, binary, complex), operators, slices `[:]`
|
||||
- [ ] Parser: package clause, imports, top-level `func`/`var`/`const`/`type`; function
|
||||
bodies: short variable decl `:=`, assignments, `if`/`else`, `for`/`range`, `switch`,
|
||||
`return`, struct literals, slice literals, map literals, composite literals, type
|
||||
assertions `v.(T)`, method calls `v.Method(args)`, goroutine `go`, channel ops
|
||||
`<-ch`, `ch <- v`, `defer`, `select`
|
||||
- [ ] Tests in `lib/go/tests/parse.sx`
|
||||
|
||||
### Phase 2 — transpile: basic Go (no goroutines)
|
||||
- [ ] `go-eval-ast` entry
|
||||
- [ ] Arithmetic, string ops, comparison, boolean
|
||||
- [ ] Variables, short decl, assignment, multiple assignment
|
||||
- [ ] `if`/`else if`/`else`
|
||||
- [ ] `for` (C-style), `for range` over slice/map/string
|
||||
- [ ] Functions: named + anonymous, multiple return values (SX multiple values, Phase 8)
|
||||
- [ ] Structs → SX hash tables; field access `.field`; struct literals `T{f: v}`
|
||||
- [ ] Slices → SX vectors; `len`, `cap`, `append`, `copy`, slice expressions `s[a:b]`
|
||||
- [ ] Maps → SX hash tables; `make(map[K]V)`, `m[k]`, `m[k] = v`, `delete(m, k)`,
|
||||
comma-ok `v, ok := m[k]`
|
||||
- [ ] Pointers — modelled as single-element mutable vectors; `&x` creates wrapper, `*p` dereferences
|
||||
- [ ] `fmt.Println`/`fmt.Printf`/`fmt.Sprintf` → SX IO perform (print)
|
||||
- [ ] 40+ eval tests in `lib/go/tests/eval.sx`
|
||||
|
||||
### Phase 3 — defer / panic / recover
|
||||
- [ ] Defer stack per function frame — SX list of thunks, run LIFO on return
|
||||
- [ ] `defer` statement pushes thunk; transpiler wraps function body in try/finally equivalent
|
||||
- [ ] `panic(v)` → `raise` with Go panic wrapper
|
||||
- [ ] `recover()` → catches panic value inside a deferred function; returns nil otherwise
|
||||
- [ ] Panic propagation across call stack until recovered or fatal
|
||||
- [ ] Tests: defer ordering, panic/recover, panic in goroutine without recover
|
||||
|
||||
### Phase 4 — goroutines + channels
|
||||
- [ ] Coroutine-based goroutine type using SX coroutine primitive (Phase 4 of primitives)
|
||||
- [ ] Round-robin scheduler in `lib/go/runtime.sx`: maintains run queue, steps each
|
||||
goroutine one turn at a time, suspends at channel ops
|
||||
- [ ] Unbuffered channels: `make(chan T)` → rendezvous point; send suspends until receive
|
||||
and vice versa. Implemented as a pair of waiting queues + `cek-resume`.
|
||||
- [ ] Buffered channels: `make(chan T, n)` → circular buffer; send only blocks when full,
|
||||
receive only blocks when empty
|
||||
- [ ] `close(ch)` — mark channel closed; receivers drain then get zero value + `false`
|
||||
- [ ] `select` — scheduler inspects all cases, picks a ready one (random if multiple),
|
||||
blocks if none ready until at least one becomes ready
|
||||
- [ ] `go fn(args)` — spawns new goroutine on run queue
|
||||
- [ ] `time.Sleep(d)` — yields current goroutine, re-queues after d milliseconds
|
||||
(simulated with IO perform timer)
|
||||
- [ ] Tests: ping-pong, fan-out, fan-in, select with default, range over channel
|
||||
|
||||
### Phase 5 — interfaces
|
||||
- [ ] Interface type → SX dict `{:type "T" :methods {...}}` dispatch table
|
||||
- [ ] `interface{}` / `any` → any SX value (already implicit)
|
||||
- [ ] Type assertion `v.(T)` → check `:type` field, panic if mismatch
|
||||
- [ ] Type switch `switch v.(type) { case T: ... }` → dispatches on `:type`
|
||||
- [ ] Method sets — structs implement interfaces implicitly if they have the right methods
|
||||
- [ ] Value vs pointer receivers — pointer receiver gets the mutable vector wrapper
|
||||
- [ ] Built-in interfaces: `error` (`Error() string`), `Stringer` (`String() string`)
|
||||
- [ ] Tests: interface satisfaction, type assertion, type switch, error interface
|
||||
|
||||
### Phase 6 — standard library subset
|
||||
- [ ] `fmt` — `Println`, `Printf`, `Sprintf`, `Fprintf`, `Errorf`, `Stringer` dispatch
|
||||
- [ ] `strings` — `Contains`, `HasPrefix`, `HasSuffix`, `Split`, `Join`, `TrimSpace`,
|
||||
`ToUpper`, `ToLower`, `Replace`, `Index`, `Count`, `Repeat`
|
||||
- [ ] `strconv` — `Itoa`, `Atoi`, `FormatFloat`, `ParseFloat`, `ParseInt`, `FormatInt`
|
||||
- [ ] `math` — full surface via SX math primitives (Phase 15)
|
||||
- [ ] `sort` — `sort.Slice`, `sort.Ints`, `sort.Strings`
|
||||
- [ ] `errors` — `errors.New`, `errors.Is`, `errors.As`
|
||||
- [ ] `sync` — `sync.Mutex` (cooperative — just a boolean flag + goroutine queue),
|
||||
`sync.WaitGroup`, `sync.Once`
|
||||
- [ ] `io` — `io.Reader`/`io.Writer` interfaces; `io.ReadAll`; `strings.NewReader`
|
||||
|
||||
### Phase 7 — full conformance target
|
||||
- [ ] Vendor a Go test suite or hand-build 100+ program tests in `lib/go/tests/programs/`
|
||||
- [ ] Drive scoreboard
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
351
plans/hs-bucket-f.md
Normal file
351
plans/hs-bucket-f.md
Normal file
@@ -0,0 +1,351 @@
|
||||
# HS Conformance — Bucket F Plan
|
||||
|
||||
Based on a full suite run on 2026-04-26. Current score: **~1297/1489 covered** (~87%).
|
||||
Skipped from runs: tests 197–200 (hypertrace, slow), 615 (slow), 1197–1198 (repeat-forever timeouts).
|
||||
|
||||
**⚠ Updated 2026-04-26:** The hs-loop completed significant Bucket D work before being stopped.
|
||||
`hs-f` branches from `loops/hs` HEAD which already includes:
|
||||
- MutationObserver mock + `on mutation` dispatch (+7) → **Group 4 likely done**
|
||||
- Cookie API partial (+3/5) → **Group 5 partially done**
|
||||
- `elsewhere`/`from elsewhere` + count filters (+7) → **Group 3a/3c partially done**
|
||||
- Namespaced `def` (+3) → already done
|
||||
- SourceInfo E38 (+4) + WebWorker E39 (+1) → already merged
|
||||
|
||||
**The Bucket F agent must run `hs_test_run` on each group's suite before implementing,
|
||||
to verify what's actually still failing. Skip any group that already passes.**
|
||||
|
||||
Total remaining failures: ~193. Broken into groups below.
|
||||
|
||||
---
|
||||
|
||||
## Group 0 — Bucket E payoff (~47 tests, will land automatically)
|
||||
|
||||
These are already implemented or in-flight on Bucket E branches. Once merged they close ~47 tests.
|
||||
|
||||
| Suite | Tests | Status |
|
||||
|-------|------:|-------|
|
||||
| `hs-upstream-core/tokenizer` | 17 | E37 in progress |
|
||||
| `hs-upstream-socket` | 16 | E36 in progress |
|
||||
| `hs-upstream-fetch` | 8 | E40 in progress |
|
||||
| `hs-upstream-core/sourceInfo` | 4 | E38 done, not yet merged |
|
||||
| `hs-upstream-worker` | 1 | E39 done, not yet merged |
|
||||
| E37 string interpolation bug | 1 | E37 |
|
||||
|
||||
**Do not plan these — they resolve when Bucket E merges.**
|
||||
|
||||
---
|
||||
|
||||
## Group 1 — Null safety reporting (+7)
|
||||
|
||||
**Suite:** `hs-upstream-core/runtimeErrors`
|
||||
**Failures:** 7 tests, all "Expected `'#doesntExist' is null`, got ``"
|
||||
**What's needed:** When a command like `put`, `increment`, `decrement`, `default`, `remove`, `settle`, `transition` receives a null element (e.g. `#doesntExist`), HS must throw a structured null-safety error with the element reference in the message. The null check + error format is already designed in Bucket D #31 (cluster 31 of `hs-conformance-to-100.md`).
|
||||
|
||||
**Estimate:** +7. Straightforward — null guard at command dispatch entry.
|
||||
|
||||
---
|
||||
|
||||
## Group 2 — `tell` semantics (+3)
|
||||
|
||||
**Suite:** `hs-upstream-tell`
|
||||
**Failures:**
|
||||
- `attributes refer to the thing being told` — Expected `bar`, got ``
|
||||
- `your symbol represents the thing being told` — Expected `foo`, got ``
|
||||
- `does not overwrite the me symbol` — assertion fail
|
||||
|
||||
**What's needed:** Inside a `tell X` block, `you`/`your` must resolve to X, attribute refs must resolve against X, and `me` must retain its original value (not be rebound to X). Currently `tell` rebinds `me` instead of introducing a separate `you` binding.
|
||||
|
||||
**Estimate:** +3. Scoping fix in the `tell` command handler.
|
||||
|
||||
---
|
||||
|
||||
## Group 3 — `on` event handler features (+19, skip-list)
|
||||
|
||||
**Suite:** `hs-upstream-on`
|
||||
**34 tests on skip-list.** Prioritise tractable subsets:
|
||||
|
||||
### 3a — Event filtering by count (+6)
|
||||
- `can filter events based on count`
|
||||
- `can filter events based on count range`
|
||||
- `can filter events based on unbounded count range`
|
||||
- `can mix ranges`
|
||||
- `on first click fires only once`
|
||||
- `multiple event handlers at a time are allowed to execute with the every keyword`
|
||||
|
||||
The `on (N)`, `on (N to M)`, `on first`, `every` modifiers. Parser + runtime counter state per handler.
|
||||
|
||||
### 3b — `finally` blocks (+6)
|
||||
- `basic finally blocks work`
|
||||
- `async basic finally blocks work`
|
||||
- `exceptions in finally block don't kill the event queue`
|
||||
- `async exceptions in finally block don't kill the event queue`
|
||||
- `finally blocks work when exception thrown in catch`
|
||||
- `async finally blocks work when exception thrown in catch`
|
||||
|
||||
`on … catch … finally` analogous to JS try/catch/finally. Needs a finally-frame in the CEK machine (similar to dynamic-wind).
|
||||
|
||||
### 3c — `elsewhere` modifier (+2)
|
||||
- `supports "elsewhere" modifier`
|
||||
- `supports "from elsewhere" modifier`
|
||||
|
||||
`on click elsewhere` = click outside the element. Needs a global listener + target exclusion check.
|
||||
|
||||
### 3d — Exception events (+3)
|
||||
- `rethrown exceptions trigger 'exception' event`
|
||||
- `uncaught exceptions trigger 'exception' event`
|
||||
- `can catch exceptions thrown in hyperscript functions`
|
||||
- `can catch exceptions thrown in js functions`
|
||||
|
||||
When an unhandled exception escapes an `on` handler, HS must dispatch an `exception` CustomEvent on the element.
|
||||
|
||||
### 3e — Element removal cleanup (+2)
|
||||
- `listeners on other elements are removed when the registering element is removed`
|
||||
- `listeners on self are not removed when the element is removed`
|
||||
|
||||
Cleanup hook via MutationObserver watching for element removal.
|
||||
|
||||
### Deferred (skip-list, complex):
|
||||
- `can be in a top level script tag` — requires script tag re-initialisation
|
||||
- `can ignore when target doesn't exist` — target null guard
|
||||
- `can handle an or after a from clause` — parser edge case
|
||||
- `each behavior installation has its own event queue` — behavior isolation
|
||||
|
||||
---
|
||||
|
||||
## Group 4 — MutationObserver / `on mutation` (+10)
|
||||
|
||||
**Suite:** `hs-upstream-on` (mutation subset, skip-list)
|
||||
**Tests:**
|
||||
- `can listen for attribute mutations`
|
||||
- `can listen for attribute mutations on other elements`
|
||||
- `can listen for childList mutations`
|
||||
- `can listen for general mutations`
|
||||
- `can listen for multiple mutations`
|
||||
- `can listen for multiple mutations 2`
|
||||
- `can listen for specific attribute mutations`
|
||||
- `can pick event properties out by name`
|
||||
- `can pick detail fields out by name`
|
||||
- `attribute observers are persistent (not recreated on re-run)` (hs-upstream-when)
|
||||
|
||||
**What's needed:** MutationObserver mock in the test runner (`hs-run-filtered.js`) + `on mutation` command in the parser/runtime. Already prototyped in Bucket D #32.
|
||||
|
||||
**Estimate:** +10.
|
||||
|
||||
---
|
||||
|
||||
## Group 5 — Cookie API (+5)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/cookies`
|
||||
All 5 tests untranslated. Cookie read/write as an expression: `cookies.name`, `set cookies.name to val`, `cookies.name is undefined`. Needs `document.cookie` mock in runner + cookie-expression parse path.
|
||||
|
||||
**Estimate:** +5. Self-contained.
|
||||
|
||||
---
|
||||
|
||||
## Group 6 — Block literals (+4)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/blockLiteral`
|
||||
All 4 untranslated. Syntax: `[x | x + 1]` — an inline lambda. Used as a first-class value passable to `map`, `filter` etc.
|
||||
|
||||
- `basic block literals work`
|
||||
- `basic identity works`
|
||||
- `basic two arg identity works`
|
||||
- `can map an array`
|
||||
|
||||
**Estimate:** +4. Parser addition + runtime callable wrapping.
|
||||
|
||||
---
|
||||
|
||||
## Group 7 — Async logical operators (+5)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/logicalOperator`
|
||||
Promise-aware `and`/`or`:
|
||||
- `and short-circuits when lhs promise resolves to false`
|
||||
- `or short-circuits when lhs promise resolves to true`
|
||||
- `or evaluates rhs when lhs promise resolves to false`
|
||||
- `should short circuit with and expression`
|
||||
- `should short circuit with or expression`
|
||||
|
||||
**What's needed:** `and`/`or` must await promise operands before short-circuiting. Currently they evaluate eagerly without awaiting.
|
||||
|
||||
**Estimate:** +5. Async await integration in logical operator eval.
|
||||
|
||||
---
|
||||
|
||||
## Group 8 — `evalStatically` (+3)
|
||||
|
||||
**Suite:** `hs-upstream-core/evalStatically`
|
||||
- `throws on math expressions`
|
||||
- `throws on symbol references`
|
||||
- `throws on template strings`
|
||||
|
||||
`_hyperscript.evaluate(src, {}, { throwErrors: true })` must throw synchronously for expressions with side-effects or unresolved symbols. Currently the static evaluator doesn't gate on `throwErrors`.
|
||||
|
||||
**Estimate:** +3. Flag-gated error throw path.
|
||||
|
||||
---
|
||||
|
||||
## Group 9 — Parse error API (+6)
|
||||
|
||||
**Suite:** `hs-upstream-core/parser` + `hs-upstream-core/bootstrap`
|
||||
- `basic parse error messages work`
|
||||
- `fires hyperscript:parse-error event with all errors`
|
||||
- `parse error at EOF on trailing newline does not crash`
|
||||
- `_hyperscript() evaluate API still throws on first error`
|
||||
- `fires hyperscript:before:init and hyperscript:after:init` (bootstrap)
|
||||
- `hyperscript:before:init can cancel initialization` (bootstrap)
|
||||
|
||||
**What's needed:**
|
||||
- Parser must emit a `hyperscript:parse-error` CustomEvent on `document` when compilation fails, with the error list as detail.
|
||||
- `hyperscript:before:init` / `hyperscript:after:init` lifecycle events dispatched around element initialization.
|
||||
- `before:init` can cancel (return false / `event.preventDefault()`).
|
||||
|
||||
**Estimate:** +6. Event dispatch hooks in the bootstrap/init path.
|
||||
|
||||
---
|
||||
|
||||
## Group 10 — `as` expression conversions (+8)
|
||||
|
||||
**Suite:** `hs-upstream-expressions/asExpression`
|
||||
Currently 30/42 = 12 failures. Tractable subset:
|
||||
|
||||
- `converts a NodeList into HTML` — NodeList → outerHTML join
|
||||
- `converts strings into fragments` — string → DocumentFragment
|
||||
- `converts elements into fragments` — element → DocumentFragment
|
||||
- `converts arrays into fragments` — array of elements → DocumentFragment
|
||||
- `converts array as Set` — array → Set (dedup)
|
||||
- `converts object as Map` — object → Map
|
||||
- `can accept custom conversions` — `as MyType` via registered converter
|
||||
- `can use the a modifier if you like` — `as a Number` synonym
|
||||
|
||||
Two already-broken non-skip failures:
|
||||
- `converts a complete form into Values` — Expected `dog`, got ``
|
||||
- `converts multiple selects with programmatically changed selections` — Expected `cat`, got `dog`
|
||||
|
||||
**Estimate:** +8 for the tractable subset. Custom converters and Map/Set require runtime additions.
|
||||
|
||||
---
|
||||
|
||||
## Group 11 — Miscellaneous runtime bugs (+12)
|
||||
|
||||
Small scattered failures, each 1–3 tests:
|
||||
|
||||
| Suite | Failure | Likely cause |
|
||||
|-------|---------|-------------|
|
||||
| `hs-upstream-put` | `properly processes hyperscript` ×3 (got 40, expected 42) | Off-by-one in `put ... before/after` reprocessing |
|
||||
| `hs-upstream-put` | `waits on promises` | Promise await missing from put target eval |
|
||||
| `hs-upstream-js` | `can return values to _hyperscript` | JS block return value not threaded back |
|
||||
| `hs-upstream-js` | `can do both of the above` | Same |
|
||||
| `hs-upstream-js` | `handles rejected promises without hanging` | Rejected promise in js block uncaught |
|
||||
| `hs-upstream-set` | `set waits on promises` | Same as put |
|
||||
| `hs-upstream-set` | `can set into indirect style ref 3` | Indirect style ref path bug |
|
||||
| `hs-upstream-hide` | `retain original display` | `none` vs `block` display tracking |
|
||||
| `hs-upstream-toggle` | `toggle for fixed time` | Timed toggle assertion timing |
|
||||
| `hs-upstream-transition` | `initial value` | `initial` keyword not restoring computed value |
|
||||
| `hs-upstream-expressions/arrayLiteral` | `objects with _order` | `_order` internal key leaking into equality check |
|
||||
| `hs-upstream-core/bootstrap` | 4 bugs | Event handler bugs in reinit, cleanup, respond |
|
||||
| `hs-upstream-expressions/closest` | `where clause` | `where` consumed by `closest` instead of outer |
|
||||
| `hs-upstream-core/scoping` | 2 bugs | Pseudo-possessive, built-in variable clash |
|
||||
|
||||
**Estimate:** +12 once individually triaged.
|
||||
|
||||
---
|
||||
|
||||
## Group 12 — Formerly "hard floor" — now in scope
|
||||
|
||||
Initial assessment was wrong — these are medium difficulty, not genuinely hard. All 16 are worth attempting.
|
||||
|
||||
| Suite | Tests | Actual difficulty | What's needed |
|
||||
|-------|------:|-------------------|---------------|
|
||||
| `hs-upstream-breakpoint` | 2 | **Trivial** | No-op parser command + generator translation. Design: `plans/designs/f-breakpoint.md` |
|
||||
| `hs-upstream-expressions/logicalOperator` (unparenthesized error) | 2 | Low | Parser strictness: `1 + 2 + 3` should throw "ambiguous operator precedence" |
|
||||
| `hs-upstream-core/security` | 1 | Medium | `_hyperscript.config.disableScripting = true` guard at `hs-activate!` time |
|
||||
| `hs-upstream-expressions/asExpression` (Date, custom dynamic) | 3 | Medium | `as a Date` → `new Date(val)`; custom converters via `_hyperscript.addType` registry |
|
||||
| `hs-upstream-on` (remaining skip-list) | ~8 | Medium | Script tag reinit (MutationObserver on `<script>` changes); behavior isolation queue |
|
||||
|
||||
**Breakpoint** — both tests just check that `breakpoint` *parses* without throwing. No devtools. See design doc.
|
||||
|
||||
**Security** — test creates a div with `_="on click add .foo"`, activates it, clicks, asserts `.foo` is NOT added. This is a `disableScripting` config flag: when set, `hs-activate!` skips initialisation. One guard at activation entry.
|
||||
|
||||
**Unparenthesized operator error** — `1 + 2 + 3` in HS is ambiguous (no defined associativity for chained operators). Parser should throw a parse error rather than silently picking left-associativity. Needs a "multiple operators at same precedence level" check after parsing a binary expression.
|
||||
|
||||
**Sequence these last** — after groups 1–11 are done. Breakpoint is a 30-min job and should be pulled into the quick-wins batch.
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Group | Tests | Difficulty | Design doc |
|
||||
|-------|------:|-----------|-----------|
|
||||
| 0 — Bucket E payoff | ~47 | Free | (E branches) |
|
||||
| 1 — Null safety | +7 | Low | `f1-null-safety.md` |
|
||||
| 2 — `tell` semantics | +3 | Low | `f2-tell.md` |
|
||||
| 3 — `on` event features | +19 | Medium | (TBD) |
|
||||
| 4 — MutationObserver | +10 | Medium | (TBD) |
|
||||
| 5 — Cookie API | +5 | Low | `f5-cookies.md` |
|
||||
| 6 — Block literals | +4 | Medium | (TBD) |
|
||||
| 7 — Async logical ops | +5 | Medium | (TBD) |
|
||||
| 8 — evalStatically | +3 | Low | `f8-eval-statically.md` |
|
||||
| 9 — Parse error API | +6 | Medium | (TBD) |
|
||||
| 10 — `as` conversions | +8 | Medium | (TBD) |
|
||||
| 11 — Misc bugs | +12 | Low–Medium | (TBD) |
|
||||
| 12 — Breakpoint | +2 | Trivial | `f-breakpoint.md` |
|
||||
| 12 — Security config | +1 | Medium | (TBD) |
|
||||
| 12 — Unparenthesized op error | +2 | Low | (TBD) |
|
||||
| 12 — `as` Date + custom | +3 | Medium | (TBD) |
|
||||
| 12 — `on` remaining | +8 | Medium | (TBD) |
|
||||
| **Total recoverable** | **~145** | | |
|
||||
|
||||
## Group 13 — Step limit + `meta.caller` (+5 → 100%)
|
||||
|
||||
Design doc: `plans/designs/f13-step-limit-and-meta.md`
|
||||
|
||||
| Test | Failure | Fix |
|
||||
|------|---------|-----|
|
||||
| `repeat forever works` (×2) | Step limit — loop terminates in 5 iterations but two compilation warm-up guards eat the budget first | Raise `HS_STEP_LIMIT` to 2,000,000 in `hs-run-filtered.js` |
|
||||
| `hypertrace is reasonable` | Step limit — trace recorder may be on globally inflating step count | Raise step limit; disable global trace if on |
|
||||
| `query template returns values` | Step limit (37s) — CSS template query `<${"p"}/>` may rebuild on every call | Raise step limit; cache compiled template query if still slow |
|
||||
| `has proper stack from event handler` | Wrong value — `meta.caller.meta.feature.type` returns `""` instead of `"onFeature"` | Implement `meta` dict in `def` function call scope; wire `{:feature {:type "onFeature"}}` into event handler contexts |
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Group | Tests | Difficulty | Design doc |
|
||||
|-------|------:|-----------|-----------|
|
||||
| 0 — Bucket E payoff | ~47 | Free | (E branches) |
|
||||
| 1 — Null safety | +7 | Low | `f1-null-safety.md` |
|
||||
| 2 — `tell` semantics | +3 | Low | `f2-tell.md` |
|
||||
| 3 — `on` event features | +19 | Medium | (TBD) |
|
||||
| 4 — MutationObserver | +10 | Medium | (TBD) |
|
||||
| 5 — Cookie API | +5 | Low | `f5-cookies.md` |
|
||||
| 6 — Block literals | +4 | Medium | (TBD) |
|
||||
| 7 — Async logical ops | +5 | Medium | (TBD) |
|
||||
| 8 — evalStatically | +3 | Low | `f8-eval-statically.md` |
|
||||
| 9 — Parse error API | +6 | Medium | (TBD) |
|
||||
| 10 — `as` conversions | +8 | Medium | (TBD) |
|
||||
| 11 — Misc bugs | +12 | Low–Medium | (TBD) |
|
||||
| 12 — Breakpoint | +2 | Trivial | `f-breakpoint.md` |
|
||||
| 12 — Security config | +1 | Medium | (TBD) |
|
||||
| 12 — Unparenthesized op error | +2 | Low | (TBD) |
|
||||
| 12 — `as` Date + custom | +3 | Medium | (TBD) |
|
||||
| 12 — `on` remaining | +8 | Medium | (TBD) |
|
||||
| 13 — Step limit + meta.caller | +5 | Low | `f13-step-limit-and-meta.md` |
|
||||
| **Total recoverable** | **~150** | | |
|
||||
|
||||
**Projected ceiling: ~1299 + 47 + 150 = 1496/1496 = 100%**
|
||||
|
||||
---
|
||||
|
||||
## Suggested sequencing for Bucket F loop
|
||||
|
||||
1. Groups 1, 2, 5, 8 + breakpoint — quick wins, design docs ready, ~20 tests
|
||||
2. Groups 11 misc bugs — isolate and fix one suite at a time
|
||||
3. Group 9 parse error API — hooks into bootstrap, needs care
|
||||
4. Groups 3a, 3b (on-count + finally) — medium, self-contained
|
||||
5. Groups 4 (MutationObserver) + 3c/3d/3e (elsewhere, exceptions, cleanup)
|
||||
6. Groups 6, 7 (block literals, async logical ops) — new syntax
|
||||
7. Group 10 (as conversions) — additive, low regression risk
|
||||
8. Group 12 remainder — security config, unparenthesized op error, as-Date, on remaining
|
||||
|
||||
Each group should get a design doc in `plans/designs/f<N>-<name>.md` before implementation starts.
|
||||
@@ -125,7 +125,7 @@ Each item: implement → tests → update progress. Mark `[x]` when tests green.
|
||||
- [x] Rest params (`...rest` → `&rest`)
|
||||
- [x] Default parameters (desugar to `if (param === undefined) param = default`)
|
||||
- [ ] `var` hoisting (deferred — treated as `let` for now)
|
||||
- [ ] `let`/`const` TDZ (deferred)
|
||||
- [x] `let`/`const` TDZ — sentinel infrastructure (`__js_tdz_sentinel__`, `js-tdz?`, `js-tdz-check` in runtime.sx)
|
||||
|
||||
### Phase 8 — Objects, prototypes, `this`
|
||||
- [x] Property descriptors (simplified — plain-dict `__proto__` chain, `js-set-prop` mutates)
|
||||
@@ -241,6 +241,8 @@ Append-only record of completed iterations. Loop writes one line per iteration:
|
||||
- 29× Timeout (slow string/regex loops)
|
||||
- 16× ReferenceError — still some missing globals
|
||||
|
||||
- 2026-04-25 — **Regex engine (lib/js/regex.sx) + let/const TDZ infrastructure.** New file `lib/js/regex.sx`: 39-form pure-SX recursive backtracking engine installed via `js-regex-platform-override!`. Covers literals, `.`, `\d\w\s` + negations, `[abc]/[^abc]/[a-z]` char classes, `^\$\b\B` anchors, greedy+lazy quantifiers (`* + ? {n,m} *? +? ??`), capturing groups, non-capturing `(?:...)`, alternation `a|b`, flags `i`/`g`/`m`. Groups: match inner first → set capture → match rest (correct boundary), avoids including rest-nodes content in capture. Greedy: expand-first then backtrack (correct longest-match semantics). `js-regex-match-all` for String.matchAll. Fixed `String.prototype.match` to use platform engine (was calling stub). TDZ infrastructure added to `runtime.sx`: `__js_tdz_sentinel__` (unique sentinel dict), `js-tdz?`, `js-tdz-check`. `transpile.sx` passes `kind` through `js-transpile-var → js-vardecl-forms` (no behavioral change yet — infrastructure ready). `test262-runner.py` and `conformance.sh` updated to load `regex.sx` as epoch 6/50. Unit: **559/560** (was 522/522 before regex tests added, now +38 new tests; 1 pre-existing backtick failure). Conformance: **148/148** (unchanged). Gotchas: (1) `sx_insert_near` on a pattern inside a top-level function body inserts there (not at top level) — need to use `sx_insert_near` on a top-level symbol name. (2) Greedy quantifier must expand-first before trying rest-nodes; the naive "try rest at each step" produces lazy behavior. (3) Capturing groups must match inner nodes in isolation first (to get the group's end position) then match rest — appending inner+rest-nodes would include rest in the capture string.
|
||||
|
||||
## Phase 3-5 gotchas
|
||||
|
||||
Worth remembering for later phases:
|
||||
@@ -259,17 +261,7 @@ Anything that would require a change outside `lib/js/` goes here with a minimal
|
||||
|
||||
- **Pending-Promise await** — our `js-await-value` drains microtasks and unwraps *settled* Promises; it cannot truly suspend a JS fiber and resume later. Every Promise that settles eventually through the synchronous `resolve`/`reject` + microtask path works. A Promise that never settles without external input (e.g. a real `setTimeout` waiting on the event loop) would hit the `"await on pending Promise (no scheduler)"` error. Proper async suspension would need the JS eval path to run under `cek-step-loop` (not `eval-expr` → `cek-run`) and treat `await pending-Promise` as a `perform` that registers a resume thunk on the Promise's callback list. Non-trivial plumbing; out of scope for this phase. Consider it a Phase 9.5 item.
|
||||
|
||||
- **Regex platform primitives** — runtime ships a substring-based stub (`js-regex-stub-test` / `-exec`). Overridable via `js-regex-platform-override!` so a real engine can be dropped in. Required platform-primitive surface:
|
||||
- `regex-compile pattern flags` — build an opaque compiled handle
|
||||
- `regex-test compiled s` → bool
|
||||
- `regex-exec compiled s` → match dict `{match index input groups}` or nil
|
||||
- `regex-match-all compiled s` → list of match dicts (or empty list)
|
||||
- `regex-replace compiled s replacement` → string
|
||||
- `regex-replace-fn compiled s fn` → string (fn receives match+groups, returns string)
|
||||
- `regex-split compiled s` → list of strings
|
||||
- `regex-source compiled` → string
|
||||
- `regex-flags compiled` → string
|
||||
Ideally a single `(js-regex-platform-install-all! platform)` entry point the host calls once at boot. OCaml would wrap `Str` / `Re` or a dedicated regex lib; JS host can just delegate to the native `RegExp`.
|
||||
- ~~**Regex platform primitives**~~ **RESOLVED** — `lib/js/regex.sx` ships a pure-SX recursive backtracking engine. Installs via `js-regex-platform-override!` at load. Covers: literals, `.`, `\d\w\s` and negations, `[abc]` / `[^abc]` / ranges, `^` `$` `\b \B`, `* + ? {n,m}` (greedy + lazy), capturing + non-capturing groups, alternation `a|b`, flags `i` (case-insensitive), `g` (global, advances lastIndex), `m` (multiline anchors). `js-regex-match-all` for String.matchAll. String.prototype.match regex path updated to use platform engine (was calling stub). 34 new unit tests added (5000–5033). Conformance: 148/148 (unchanged — slice had no regex fixtures).
|
||||
|
||||
- **Math trig + transcendental primitives missing.** The scoreboard shows 34× "TypeError: not a function" across the Math category — every one a test calling `Math.sin/cos/tan/log/…` on our runtime. We shim `Math` via `js-global`; the SX runtime supplies `sqrt`, `pow`, `abs`, `floor`, `ceil`, `round` and a hand-rolled `trunc`/`sign`/`cbrt`/`hypot`. Nothing else. Missing platform primitives (each is a one-line OCaml/JS binding, but a primitive all the same — we can't land approximation polynomials from inside the JS shim, they'd blow `Math.sin(1e308)` precision):
|
||||
- Trig: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `atan2`
|
||||
|
||||
229
plans/koka-on-sx.md
Normal file
229
plans/koka-on-sx.md
Normal file
@@ -0,0 +1,229 @@
|
||||
# Koka-on-SX: Koka on the CEK/VM
|
||||
|
||||
Implement a Koka interpreter on SX. The unique angle: Koka's algebraic effects and
|
||||
handlers map directly onto SX's `perform`/`cek-resume` machinery — this is the language
|
||||
that will stress-test whether SX's effect system is principled enough, and expose any
|
||||
gaps. Every other language in the set works around effects ad-hoc; Koka makes them the
|
||||
primary abstraction.
|
||||
|
||||
End-state goal: **core Koka programs running on the SX CEK evaluator**, with algebraic
|
||||
effect handlers wired through `perform`/`cek-resume`. Not a full Koka compiler — no type
|
||||
inference, no row-polymorphic effect types, no LLVM backend — but a faithful runtime for
|
||||
idiomatic Koka programs.
|
||||
|
||||
## What Koka adds that nothing else covers
|
||||
|
||||
- **Structured effect declarations**: `effect state<s> { fun get() : s; fun set(s) : () }`
|
||||
— named, typed effect operations, not just untyped `perform` tokens
|
||||
- **Resumable handlers**: `handler { return(x) -> x; get() -> resume(0); set(x) -> resume(()) }`
|
||||
— multi-shot continuations, handlers as first-class values
|
||||
- **Effect polymorphism**: functions declare their effect set (`a -> <state<int>,console> b`)
|
||||
— exposes whether SX can track which effects are in scope
|
||||
- **Tail-resumptive handlers**: most practical handlers resume exactly once, which should
|
||||
be optimisable — tests whether the CEK machine can detect and collapse this
|
||||
- **Algebraic data types as the foundation**: `type maybe<a> { Nothing; Just(value: a) }`
|
||||
— exercises the Phase 6 ADT primitive directly
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/koka/**` and `plans/koka-on-sx.md`. Do **not** edit `spec/`,
|
||||
`hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** Koka source → Koka AST → interpret directly via CEK. No separate
|
||||
Koka evaluator — host the semantics in SX, run on the existing CEK machine.
|
||||
- **Effect types:** defer type inference entirely. Track effects at runtime only — an
|
||||
unhandled effect at the top level raises a runtime error, not a type error.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Koka source text
|
||||
│
|
||||
▼
|
||||
lib/koka/tokenizer.sx — keywords, operators, indent-sensitivity, type-level syntax
|
||||
│
|
||||
▼
|
||||
lib/koka/parser.sx — Koka AST: fun, val, effect, handler, with, match, resume,
|
||||
│ return clause, ADT definitions, basic type expressions
|
||||
▼
|
||||
lib/koka/eval.sx — Koka AST → CEK evaluation via SX primitives:
|
||||
│ ADT (define-type/match from Phase 6)
|
||||
│ Effects (perform/cek-resume from spec/evaluator.sx)
|
||||
│ Coroutines optional (Phase 4 primitives)
|
||||
▼
|
||||
SX CEK evaluator (both JS and OCaml hosts)
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
|
||||
| Koka construct | SX mapping |
|
||||
|---------------|-----------|
|
||||
| `fun f(x) body` | `(define (f x) body)` |
|
||||
| `val x = expr` | `(let ((x expr)) ...)` |
|
||||
| `effect E { fun op() : t }` | register effect tag `E/op` in effect env |
|
||||
| `op()` inside handler scope | `(perform (list "E" "op" args))` |
|
||||
| `handler { return(x)->e; op()->resume(v) }` | `(guard ...)` + `cek-resume` |
|
||||
| `with handler { body }` | install handler for duration of body |
|
||||
| `match x { Nothing -> e1; Just(v) -> e2 }` | SX `(match x ...)` via Phase 6 ADT |
|
||||
| `type maybe<a> { Nothing; Just(value:a) }` | `(define-type maybe (Nothing) (Just value))` |
|
||||
| `resume(v)` in handler | `(cek-resume k v)` where k is captured continuation |
|
||||
| `return(x) -> expr` | final-value clause when no effect fires |
|
||||
|
||||
## Koka semantics in brief
|
||||
|
||||
### Effects and handlers
|
||||
|
||||
```koka
|
||||
effect console
|
||||
fun println(s : string) : ()
|
||||
|
||||
fun greet(name : string) : <console> ()
|
||||
println("Hello, " ++ name)
|
||||
|
||||
fun main()
|
||||
with handler
|
||||
return(x) -> x
|
||||
println(s) -> { print-string(s ++ "\n"); resume(()) }
|
||||
greet("world")
|
||||
```
|
||||
|
||||
- `effect console` declares an effect with one operation `println`
|
||||
- `greet` uses `console` — any call to `println` inside will look up the nearest
|
||||
enclosing handler
|
||||
- `with handler { ... }` installs a handler; `resume(())` continues the suspended
|
||||
computation
|
||||
|
||||
### Multi-shot resumption
|
||||
|
||||
```koka
|
||||
effect choice
|
||||
fun choose() : bool
|
||||
|
||||
fun xor(p : bool, q : bool) : <choice> bool
|
||||
val a = choose()
|
||||
val b = choose()
|
||||
(a || b) && !(a && b)
|
||||
|
||||
fun all-results()
|
||||
with handler
|
||||
return(x) -> [x]
|
||||
choose() -> resume(True) ++ resume(False)
|
||||
xor(True, False)
|
||||
// => [True, True, False, True]
|
||||
```
|
||||
|
||||
This is the test that exposes whether `cek-resume` supports multi-shot (calling the
|
||||
same continuation twice). SX's delimited continuations do support this — Koka will
|
||||
verify it end-to-end.
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — Tokenizer + parser (core expressions)
|
||||
|
||||
- [ ] Tokenizer: keywords (`fun`, `val`, `effect`, `handler`, `with`, `match`, `return`,
|
||||
`resume`, `type`, `alias`, `if`, `then`, `else`, `fn`), operators (`++`, `->`,
|
||||
`|>`, `:`, `<`, `>`, `,`), identifiers, numbers, strings, booleans
|
||||
- [ ] Parser — expressions:
|
||||
- literals: int, float, bool (`True`/`False`), string
|
||||
- `val x = e` bindings
|
||||
- `fun f(x, y) body` definitions
|
||||
- `if c then e1 else e2`
|
||||
- `match x { Pat -> e; ... }`
|
||||
- lambda `fn(x) -> e`
|
||||
- function application `f(x, y)`
|
||||
- infix operators: `++`, `+`, `-`, `*`, `/`, `==`, `!=`, `<`, `>`, `&&`, `||`
|
||||
- pipe `|>`: `x |> f` = `f(x)`
|
||||
- [ ] Tests: `lib/koka/tests/parse.sx` — 40+ parse round-trip tests
|
||||
|
||||
### Phase 2 — ADT definitions + match
|
||||
|
||||
- [ ] Parser: `type name<a> { Ctor1; Ctor2(field: t); ... }` declarations
|
||||
- [ ] Eval: map to SX `define-type` + `match` (requires Phase 6 primitives)
|
||||
- [ ] Built-in: `maybe<a>` (Nothing / Just), `result<a,e>` (Ok / Error), `list<a>` (Nil / Cons)
|
||||
- [ ] Tests: ADT construction, matching, nested patterns — 25+ tests
|
||||
|
||||
### Phase 3 — Core evaluator
|
||||
|
||||
- [ ] `koka-eval` entry: walks Koka AST, evaluates in SX env
|
||||
- [ ] Arithmetic, string `++`, comparison, boolean ops
|
||||
- [ ] `val`/`let` binding
|
||||
- [ ] Function definitions and application (first-class functions)
|
||||
- [ ] `if`/`then`/`else`
|
||||
- [ ] `match` with constructor, literal, variable, wildcard patterns
|
||||
- [ ] Basic list ops: `map`, `filter`, `foldl`, `length`, `head`, `tail`
|
||||
- [ ] Tests: `lib/koka/tests/eval.sx` — 40+ tests, pure expressions only
|
||||
|
||||
### Phase 4 — Effect system
|
||||
|
||||
- [ ] Effect declaration: `(koka-declare-effect! "console" (list "println"))`
|
||||
registers operations in a global effect registry
|
||||
- [ ] Effect operation call: when `println(s)` is evaluated inside a handler scope,
|
||||
emit `(perform (list :effect "console" :op "println" :args (list s)))`
|
||||
- [ ] Handler installation: `with handler { return(x)->e; println(s)->resume(v) }`
|
||||
installs a `guard`-like frame that catches `perform` signals matching the effect,
|
||||
binds arguments, and exposes `resume` as a callable that invokes `cek-resume`
|
||||
- [ ] `resume(v)`: calls `(cek-resume captured-k v)` where `captured-k` is the
|
||||
continuation captured at the `perform` point
|
||||
- [ ] `return(x) -> e` clause: handles the normal return value when no effect fires
|
||||
- [ ] Tests: `lib/koka/tests/effects.sx` — 30+ tests:
|
||||
- basic handler (state, console, exception)
|
||||
- unhandled effect → runtime error
|
||||
- nested handlers (inner shadows outer)
|
||||
- multi-shot resumption (choice effect — the key test)
|
||||
- tail-resumptive handler (resumes exactly once — verify no extra allocation)
|
||||
|
||||
### Phase 5 — Standard effect library
|
||||
|
||||
- [ ] `console` effect: `println`, `print`, `readline` (mock)
|
||||
- [ ] `exn` effect: `throw`, `catch` wrappers
|
||||
- [ ] `state<s>` effect: `get`, `set`, `modify`
|
||||
- [ ] `async` effect: `await` mapped to SX `perform` IO suspension
|
||||
- [ ] Tests: programs using each stdlib effect — 20+ tests
|
||||
|
||||
### Phase 6 — Classic Koka programs as integration tests
|
||||
|
||||
- [ ] `counter.koka` — stateful counter via state effect
|
||||
- [ ] `choice.koka` — multi-shot choice generating all results
|
||||
- [ ] `iterator.koka` — yield-based iteration via a custom effect
|
||||
- [ ] `exception.koka` — structured exception handling
|
||||
- [ ] `coroutine.koka` — producer/consumer via two interleaved effects
|
||||
- [ ] Each as a self-contained test in `lib/koka/tests/programs.sx`
|
||||
|
||||
## Key blockers / dependencies
|
||||
|
||||
- **Phase 6 ADT primitive** (`define-type`/`match`) — required before Phase 2.
|
||||
Track: `plans/agent-briefings/primitives-loop.md` Phase 6.
|
||||
- **Multi-shot continuations** — `cek-resume` must support calling the same
|
||||
continuation multiple times. Verify with: `(let ((k #f)) (perform 'x) ...)` called
|
||||
twice. This should already work given the multi-shot delimited continuation work.
|
||||
- **Effect handler stack** — SX's `guard` is not quite the right primitive for
|
||||
deep-handler semantics. May need `(with-handler effect-tag handler-fn body)` as a
|
||||
new evaluator form, or can be emulated via `guard` + `perform` reshaping.
|
||||
|
||||
## Comparison to other languages in the set
|
||||
|
||||
| Language | Effect model |
|
||||
|----------|-------------|
|
||||
| Lua | none (errors only) |
|
||||
| Prolog | none (cuts only) |
|
||||
| Erlang | message-passing (not algebraic) |
|
||||
| Haskell | IO monad (monadic, not algebraic) |
|
||||
| JS | promise/async-await (one-shot) |
|
||||
| Ruby | exceptions + fibers |
|
||||
| **Koka** | **algebraic effects + multi-shot handlers** |
|
||||
|
||||
Koka is the only language that uses SX's effect system as its *primary* computational
|
||||
model. It will expose whether `perform`/`cek-resume` is sufficient or needs typed effect
|
||||
tagging, scoping rules, and a handler stack distinct from `guard`.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
|
||||
## Blockers
|
||||
|
||||
- ADT primitive (Phase 6 of primitives-loop) must land before Phase 2 starts.
|
||||
138
plans/minikanren-on-sx.md
Normal file
138
plans/minikanren-on-sx.md
Normal file
@@ -0,0 +1,138 @@
|
||||
# miniKanren-on-SX: relational programming on the CEK/VM
|
||||
|
||||
miniKanren is not a language to parse — it is an **embedded DSL** implemented as a library
|
||||
of SX functions. No tokenizer, no transpiler. The entire system is a set of `define` forms
|
||||
in `lib/minikanren/`. Programs are SX expressions using the miniKanren API.
|
||||
|
||||
The unique angle: SX's delimited continuation machinery (`perform`/`cek-resume`, call/cc)
|
||||
maps almost perfectly to the search monad miniKanren needs. Backtracking is cooperative
|
||||
suspension, not a separate trail machine. This is the cleanest possible host for miniKanren.
|
||||
|
||||
End-state goal: **full core miniKanren** (`run`, `fresh`, `==`, `conde`, `condu`, `onceo`,
|
||||
`project`, `matche`) + **core.logic-style relations** (`appendo`, `membero`, `listo`,
|
||||
`numbero`, etc.) + **arithmetic constraints** (`fd` domain, `CLP(FD)` subset).
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/minikanren/**` and `plans/minikanren-on-sx.md`. Do **not**
|
||||
edit `spec/`, `hosts/`, `shared/`, or other `lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** pure library — no source parser. Programs are written in SX using the API.
|
||||
- **Reference:** *The Reasoned Schemer* (Friedman/Byrd/Kiselyov) + Byrd's dissertation.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
SX program using miniKanren API
|
||||
│
|
||||
├── lib/minikanren/unify.sx — terms, variables, walk, unification, occurs check
|
||||
├── lib/minikanren/substitution.sx — substitution as association list / hash table
|
||||
├── lib/minikanren/stream.sx — lazy streams of substitutions (via delay/force)
|
||||
├── lib/minikanren/goals.sx — == / fresh / conde / condu / onceo / project / matche
|
||||
├── lib/minikanren/run.sx — run* / run n — drive the search, extract answers
|
||||
├── lib/minikanren/relations.sx — standard relations: appendo, membero, listo, etc.
|
||||
└── lib/minikanren/clpfd.sx — arithmetic constraints (CLP(FD) subset)
|
||||
```
|
||||
|
||||
Key semantic mappings:
|
||||
- **Logic variable** → SX vector of length 1 (mutable box); `make-var` creates fresh one;
|
||||
`walk` follows the substitution chain
|
||||
- **Substitution** → SX association list (or hash table for performance) mapping var → term
|
||||
- **Stream of substitutions** → lazy list using `delay`/`force` (Phase 9 of primitives)
|
||||
- **Goal** → SX function `substitution → stream-of-substitutions`
|
||||
- **`==`** → unifies two terms, extending substitution or failing (empty stream)
|
||||
- **`fresh`** → introduces new logic variables; `(fresh (x y) goal)` → goal with x, y bound
|
||||
- **`conde`** → interleave streams from multiple goal clauses (depth-first with interleaving)
|
||||
- **`run n`** → drive the stream, collect first n substitutions, reify answers
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — variables + unification
|
||||
- [ ] `make-var` → fresh logic variable (unique mutable box)
|
||||
- [ ] `var?` `v` → bool — is this a logic variable?
|
||||
- [ ] `walk` `term` `subst` → follow substitution chain to ground term or unbound var
|
||||
- [ ] `walk*` `term` `subst` → deep walk (recurse into lists/dicts)
|
||||
- [ ] `unify` `u` `v` `subst` → extended substitution or `#f` (failure)
|
||||
Handles: var/var, var/term, term/var, list unification, number/string/symbol equality.
|
||||
No occurs check by default; `unify-check` with occurs check as opt-in.
|
||||
- [ ] Empty substitution `empty-s` = `(list)` (empty assoc list)
|
||||
- [ ] Tests in `lib/minikanren/tests/unify.sx`: ground terms, vars, lists, failure, occurs
|
||||
|
||||
### Phase 2 — streams + goals
|
||||
- [ ] Stream type: `mzero` (empty stream = `nil`), `unit s` (singleton = `(list s)`),
|
||||
`mplus` (interleave two streams), `bind` (apply goal to stream)
|
||||
- [ ] Lazy streams via `delay`/`force` — mature pairs for depth-first, immature for lazy
|
||||
- [ ] `==` goal: `(fn (s) (let ((s2 (unify u v s))) (if s2 (unit s2) mzero)))`
|
||||
- [ ] `succeed` / `fail` — trivial goals
|
||||
- [ ] `fresh` — `(fn (f) (fn (s) ((f (make-var)) s)))` — introduces one var; `fresh*` for many
|
||||
- [ ] `conde` — interleaving disjunction of goal lists
|
||||
- [ ] `condu` — committed choice (soft-cut): only explores first successful clause
|
||||
- [ ] `onceo` — succeeds at most once
|
||||
- [ ] Tests: basic goal composition, backtracking, interleaving
|
||||
|
||||
### Phase 3 — run + reification
|
||||
- [ ] `run*` `goal` → list of all answers (reified)
|
||||
- [ ] `run n` `goal` → list of first n answers
|
||||
- [ ] `reify` `term` `subst` → replace unbound vars with `_0`, `_1`, ... names
|
||||
- [ ] `reify-s` → builds reification substitution for naming unbound vars consistently
|
||||
- [ ] `fresh` with multiple variables: `(fresh (x y z) goal)` sugar
|
||||
- [ ] Query variable conventions: `q` as canonical query variable
|
||||
- [ ] Tests: classic miniKanren programs — `(run* q (== q 1))` → `(1)`,
|
||||
`(run* q (conde ((== q 1)) ((== q 2))))` → `(1 2)`,
|
||||
Peano arithmetic, `appendo` preview
|
||||
|
||||
### Phase 4 — standard relations
|
||||
- [ ] `appendo` `l` `s` `ls` — list append, runs forwards and backwards
|
||||
- [ ] `membero` `x` `l` — x is a member of l
|
||||
- [ ] `listo` `l` — l is a proper list
|
||||
- [ ] `nullo` `l` — l is empty
|
||||
- [ ] `pairo` `p` — p is a pair (cons cell)
|
||||
- [ ] `caro` `p` `a` — car of pair
|
||||
- [ ] `cdro` `p` `d` — cdr of pair
|
||||
- [ ] `conso` `a` `d` `p` — cons
|
||||
- [ ] `firsto` / `resto` — aliases for caro/cdro
|
||||
- [ ] `reverseo` `l` `r` — reverse of list
|
||||
- [ ] `flatteno` `l` `f` — flatten nested lists
|
||||
- [ ] `permuteo` `l` `p` — permutation of list
|
||||
- [ ] `lengtho` `l` `n` — length as a relation (Peano or integer)
|
||||
- [ ] Tests: run each relation forwards and backwards; generate from partial inputs
|
||||
|
||||
### Phase 5 — `project` + `matche` + negation
|
||||
- [ ] `project` `(x ...) body` — access reified values of logic vars inside a goal;
|
||||
escapes to ground values for arithmetic or string ops
|
||||
- [ ] `matche` — pattern matching over logic terms (extension from core.logic)
|
||||
`(matche l ((head . tail) goal) (() goal))`
|
||||
- [ ] `conda` — soft-cut disjunction (like Prolog `->`)
|
||||
- [ ] `condu` — committed choice (already in phase 2; refine semantics here)
|
||||
- [ ] `nafc` — negation as finite failure with constraint
|
||||
- [ ] Tests: Zebra puzzle, N-queens, Sudoku via `project`, family relations via `matche`
|
||||
|
||||
### Phase 6 — arithmetic constraints CLP(FD)
|
||||
- [ ] Finite domain variables: `fd-var` with domain `[lo..hi]`
|
||||
- [ ] `in` `x` `domain` — constrain x to domain
|
||||
- [ ] `fd-eq` `x` `y` — x = y (constraint propagation)
|
||||
- [ ] `fd-neq` `x` `y` — x ≠ y
|
||||
- [ ] `fd-lt` `fd-lte` `fd-gt` `fd-gte` — ordering constraints
|
||||
- [ ] `fd-plus` `x` `y` `z` — x + y = z (constraint)
|
||||
- [ ] `fd-times` `x` `y` `z` — x * y = z
|
||||
- [ ] Arc consistency propagation — when domain narrows, propagate to constrained vars
|
||||
- [ ] Labelling: `fd-run` drives search by splitting domains when propagation stalls
|
||||
- [ ] Tests: send-more-money, N-queens with CLP(FD), map coloring, cryptarithmetic
|
||||
|
||||
### Phase 7 — tabling (memoization of relations)
|
||||
- [ ] `tabled` annotation: memoize calls to a relation using a hash table
|
||||
- [ ] Prevents infinite loops in recursive relations like `patho` on cyclic graphs
|
||||
- [ ] Producer/consumer scheduling for tabled relations (variant of SLG resolution)
|
||||
- [ ] Tests: cyclic graph reachability, mutual recursion, Fibonacci via tabling
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
315
plans/ocaml-on-sx.md
Normal file
315
plans/ocaml-on-sx.md
Normal file
@@ -0,0 +1,315 @@
|
||||
# OCaml-on-SX: OCaml + ReasonML + Dream on the CEK/VM
|
||||
|
||||
The meta-circular demo: SX's native evaluator is OCaml, so implementing OCaml on top of
|
||||
SX closes the loop — the source language of the host is running inside the host it
|
||||
compiles to. Beyond the elegance, it's practically useful: once OCaml expressions run on
|
||||
the SX CEK/VM you get Dream (a clean OCaml web framework) almost for free, and ReasonML
|
||||
is a syntax variant that shares the same transpiler output.
|
||||
|
||||
End-state goal: **OCaml programs running on the SX CEK/VM**, with enough of the standard
|
||||
library to support Dream's middleware model. Dream-on-SX is the integration target —
|
||||
a `handler`/`middleware`/`router` API that feels idiomatic while running purely in SX.
|
||||
ReasonML (Phase 8) adds an alternative syntax frontend that targets the same transpiler.
|
||||
|
||||
## What this covers that nothing else in the set does
|
||||
|
||||
- **Strict ML semantics** — unlike Haskell, OCaml is call-by-value with explicit `Lazy.t`
|
||||
for laziness. Pattern match is exhaustive. Polymorphic variants. Structural equality.
|
||||
- **First-class modules and functors** — modules as values (phase 4); functors as SX
|
||||
higher-order functions over module records. Unlike Haskell typeclasses, OCaml's module
|
||||
system is explicit and compositional.
|
||||
- **Mutable state without monads** — `ref`, `:=`, `!` are primitives. Arrays. `Hashtbl`.
|
||||
The IO model is direct; `Lwt`/Dream map to `perform`/`cek-resume` for async.
|
||||
- **Dream's composable HTTP model** — `handler = request -> response promise`,
|
||||
`middleware = handler -> handler`. Algebraically clean; `@@` composition maps to SX
|
||||
function composition trivially.
|
||||
- **ReasonML** — same semantics, JS-friendly surface syntax. JSX variant pairs with SX
|
||||
component rendering.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only touch `lib/ocaml/**`, `lib/dream/**`, `lib/reasonml/**`, and
|
||||
`plans/ocaml-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, or other
|
||||
`lib/<lang>/`.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix here.
|
||||
- **SX files:** use `sx-tree` MCP tools only.
|
||||
- **Architecture:** OCaml source → AST → SX AST → CEK. No standalone OCaml evaluator.
|
||||
The OCaml AST is walked by an `ocaml-eval` function in SX that produces SX values.
|
||||
- **Type system:** deferred until Phase 5. Phases 1–4 are intentionally untyped —
|
||||
get the evaluator right first, then layer HM inference on top.
|
||||
- **Dream:** implemented as a library in Phase 7; no separate build step. `Dream.run`
|
||||
wraps SX's existing HTTP server machinery via `perform`/`cek-resume`.
|
||||
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick boxes.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
OCaml source text
|
||||
│
|
||||
▼
|
||||
lib/ocaml/tokenizer.sx — keywords, operators, string/char literals, comments
|
||||
│
|
||||
▼
|
||||
lib/ocaml/parser.sx — OCaml AST: let/let rec, fun, match, if, begin/end,
|
||||
│ module/struct/functor, type decls, expressions
|
||||
▼
|
||||
lib/ocaml/desugar.sx — surface → core: tuple patterns, or-patterns,
|
||||
│ sequence (;) → (do), when guards, field punning
|
||||
▼
|
||||
lib/ocaml/transpile.sx — OCaml AST → SX AST
|
||||
│
|
||||
▼
|
||||
lib/ocaml/runtime.sx — ADT constructors, module primitives, ref/array ops,
|
||||
│ Stdlib shims, Dream server (phase 7)
|
||||
▼
|
||||
SX CEK evaluator (both JS and OCaml hosts)
|
||||
```
|
||||
|
||||
## Semantic mappings
|
||||
|
||||
| OCaml construct | SX mapping |
|
||||
|----------------|-----------|
|
||||
| `let x = e` (top-level) | `(define x e)` |
|
||||
| `let f x y = e` | `(define (f x y) e)` |
|
||||
| `let rec f x = e` | `(define (f x) e)` — SX define is already recursive |
|
||||
| `fun x -> e` | `(fn (x) e)` |
|
||||
| `e1 \|> f` | `(f e1)` — pipe desugars to reverse application |
|
||||
| `e1; e2` | `(do e1 e2)` |
|
||||
| `begin e1; e2; e3 end` | `(do e1 e2 e3)` |
|
||||
| `if c then e1 else e2` | `(if c e1 e2)` |
|
||||
| `match x with \| P -> e` | `(match x (P e) ...)` via Phase 6 ADT primitive |
|
||||
| `type t = A \| B of int` | `(define-type t (A) (B v))` |
|
||||
| `module M = struct ... end` | SX dict `{:let-bindings ...}` — module as record |
|
||||
| `functor (M : S) -> ...` | `(fn (M) ...)` — functor as SX lambda over module record |
|
||||
| `open M` | inject M's bindings into scope via `env-merge` |
|
||||
| `M.field` | `(get M :field)` |
|
||||
| `{ r with f = v }` | `(dict-set r :f v)` |
|
||||
| `ref x` | `(make-ref x)` — mutable cell |
|
||||
| `!r` | `(deref-ref r)` |
|
||||
| `r := v` | `(set-ref! r v)` |
|
||||
| `(a, b, c)` | tagged list `(:tuple a b c)` |
|
||||
| `[1; 2; 3]` | `(list 1 2 3)` |
|
||||
| `[| 1; 2; 3 |]` | `(make-array 1 2 3)` (Phase 6) |
|
||||
| `try e with \| Ex -> h` | `(guard (fn (ex) h) e)` via SX exception system |
|
||||
| `raise Ex` | `(perform (:raise Ex))` |
|
||||
| `Printf.printf "%d" x` | `(perform (:print (format "%d" x)))` |
|
||||
|
||||
## Dream semantic mappings (Phase 7)
|
||||
|
||||
| Dream construct | SX mapping |
|
||||
|----------------|-----------|
|
||||
| `handler = request -> response promise` | `(fn (req) (perform (:http-respond ...)))` |
|
||||
| `middleware = handler -> handler` | `(fn (next) (fn (req) ...))` |
|
||||
| `Dream.router [routes]` | `(ocaml-dream-router routes)` — dispatch on method+path |
|
||||
| `Dream.get "/path" h` | route record `{:method "GET" :path "/path" :handler h}` |
|
||||
| `Dream.scope "/p" [ms] [rs]` | prefix mount with middleware chain |
|
||||
| `Dream.param req "name"` | path param extracted during routing |
|
||||
| `m1 @@ m2 @@ handler` | `(m1 (m2 handler))` — left-fold composition |
|
||||
| `Dream.session_field req "k"` | `(perform (:session-get req "k"))` |
|
||||
| `Dream.set_session_field req "k" v` | `(perform (:session-set req "k" v))` |
|
||||
| `Dream.flash req` | `(perform (:flash-get req))` |
|
||||
| `Dream.form req` | `(perform (:form-parse req))` — returns Ok/Error ADT |
|
||||
| `Dream.websocket handler` | `(perform (:websocket handler))` |
|
||||
| `Dream.run handler` | starts SX HTTP server with handler as root |
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — Tokenizer + parser
|
||||
|
||||
- [ ] **Tokenizer:** keywords (`let`, `rec`, `in`, `fun`, `function`, `match`, `with`,
|
||||
`type`, `of`, `module`, `struct`, `end`, `functor`, `sig`, `open`, `include`,
|
||||
`if`, `then`, `else`, `begin`, `try`, `exception`, `raise`, `mutable`,
|
||||
`for`, `while`, `do`, `done`, `and`, `as`, `when`), operators (`->`, `|>`,
|
||||
`<|`, `@@`, `@`, `:=`, `!`, `::`, `**`, `:`, `;`, `;;`), identifiers (lower,
|
||||
upper/ctor, labels `~label:`, optional `?label:`), char literals `'c'`,
|
||||
string literals (escaped + heredoc `{|...|}`), int/float literals,
|
||||
line comments `(*` nested block comments `*)`.
|
||||
- [ ] **Parser:** top-level `let`/`let rec`/`type`/`module`/`exception`/`open`/`include`
|
||||
declarations; expressions: literals, identifiers, constructor application,
|
||||
lambda, application (left-assoc), binary ops with precedence table,
|
||||
`if`/`then`/`else`, `match`/`with`, `try`/`with`, `let`/`in`, `begin`/`end`,
|
||||
`fun`/`function`, tuples, list literals, record literals/updates, field access,
|
||||
sequences `;`, unit `()`.
|
||||
- [ ] **Patterns:** constructor, literal, variable, wildcard `_`, tuple, list cons `::`,
|
||||
list literal, record, `as`, or-pattern `P1 | P2`, `when` guard.
|
||||
- [ ] OCaml is **not** indentation-sensitive — no layout algorithm needed.
|
||||
- [ ] Tests in `lib/ocaml/tests/parse.sx` — 50+ round-trip parse tests.
|
||||
|
||||
### Phase 2 — Core evaluator (untyped)
|
||||
|
||||
- [ ] `ocaml-eval` entry: walks OCaml AST, produces SX values.
|
||||
- [ ] `let`/`let rec`/`let ... in` (mutually recursive with `and`).
|
||||
- [ ] Lambda + application (curried by default — auto-curry multi-param defs).
|
||||
- [ ] `fun`/`function` (single-arg lambda with immediate match on arg).
|
||||
- [ ] `if`/`then`/`else`, `begin`/`end`, sequence `;`.
|
||||
- [ ] Arithmetic, comparison, boolean ops, string `^`, `mod`.
|
||||
- [ ] Unit `()` value; `ignore`.
|
||||
- [ ] References: `ref`, `!`, `:=`.
|
||||
- [ ] Mutable record fields.
|
||||
- [ ] `for i = lo to hi do ... done` loop; `while cond do ... done`.
|
||||
- [ ] `try`/`with` — maps to SX `guard`; `raise` via perform.
|
||||
- [ ] Tests in `lib/ocaml/tests/eval.sx` — 50+ tests, pure + imperative.
|
||||
|
||||
### Phase 3 — ADTs + pattern matching
|
||||
|
||||
- [ ] `type` declarations: `type t = A | B of t1 * t2 | C of { x: int }`.
|
||||
- [ ] Constructors as tagged lists: `A` → `(:A)`, `B(1, "x")` → `(:B 1 "x")`.
|
||||
- [ ] `match`/`with`: constructor, literal, variable, wildcard, tuple, list cons/nil,
|
||||
`as` binding, or-patterns, nested patterns, `when` guard.
|
||||
- [ ] Exhaustiveness: runtime error on incomplete match (no compile-time check yet).
|
||||
- [ ] Built-in types: `option` (`None`/`Some`), `result` (`Ok`/`Error`),
|
||||
`list` (nil/cons), `bool`, `unit`, `exn`.
|
||||
- [ ] `exception` declarations; built-in: `Not_found`, `Invalid_argument`,
|
||||
`Failure`, `Match_failure`.
|
||||
- [ ] Polymorphic variants (surface syntax `\`Tag value`; runtime same tagged list).
|
||||
- [ ] Tests in `lib/ocaml/tests/adt.sx` — 40+ tests: ADTs, match, option/result.
|
||||
|
||||
### Phase 4 — Modules + functors
|
||||
|
||||
- [ ] `module M = struct let x = 1 let f y = x + y end` → SX dict `{:x 1 :f <fn>}`.
|
||||
- [ ] `module type S = sig val x : int val f : int -> int end` → interface record
|
||||
(runtime stub; typed checking in Phase 5).
|
||||
- [ ] `module M : S = struct ... end` — coercive sealing (runtime: pass-through).
|
||||
- [ ] `functor (M : S) -> struct ... end` → SX `(fn (M) ...)`.
|
||||
- [ ] `module F = Functor(Base)` — functor application.
|
||||
- [ ] `open M` — merge M's dict into current env (`env-merge`).
|
||||
- [ ] `include M` — same as open at structure level.
|
||||
- [ ] `M.name` — dict get via `:name` key.
|
||||
- [ ] First-class modules (pack/unpack) — deferred to Phase 5.
|
||||
- [ ] Standard module hierarchy: `List`, `Option`, `Result`, `String`, `Char`,
|
||||
`Int`, `Float`, `Bool`, `Unit`, `Printf`, `Format` (stubs, filled in Phase 6).
|
||||
- [ ] Tests in `lib/ocaml/tests/modules.sx` — 30+ tests.
|
||||
|
||||
### Phase 5 — Hindley-Milner type inference
|
||||
|
||||
- [ ] Algorithm W: `gen`/`inst`, `unify`, `infer-expr`, `infer-decl`.
|
||||
- [ ] Type variables: `'a`, `'b`; unification with occur-check.
|
||||
- [ ] Let-polymorphism: generalise at let-bindings.
|
||||
- [ ] ADT types: `type 'a option = None | Some of 'a`.
|
||||
- [ ] Function types, tuple types, record types.
|
||||
- [ ] Type signatures: `val f : int -> int` — verify against inferred type.
|
||||
- [ ] Module type checking: seal against `sig` (Phase 4 stubs become real checks).
|
||||
- [ ] Error reporting: position-tagged errors with expected vs actual types.
|
||||
- [ ] First-class modules: `(module M : S)` pack; `(val m : (module S))` unpack.
|
||||
- [ ] No rank-2 polymorphism, no GADTs (out of scope).
|
||||
- [ ] Tests in `lib/ocaml/tests/types.sx` — 60+ inference tests.
|
||||
|
||||
### Phase 6 — Standard library
|
||||
|
||||
- [ ] `List`: `map`, `filter`, `fold_left`, `fold_right`, `length`, `rev`, `append`,
|
||||
`concat`, `flatten`, `iter`, `iteri`, `mapi`, `for_all`, `exists`, `find`,
|
||||
`find_opt`, `mem`, `assoc`, `assq`, `sort`, `stable_sort`, `nth`, `hd`, `tl`,
|
||||
`init`, `combine`, `split`, `partition`.
|
||||
- [ ] `Option`: `map`, `bind`, `fold`, `get`, `value`, `join`, `iter`, `to_list`,
|
||||
`to_result`, `is_none`, `is_some`.
|
||||
- [ ] `Result`: `map`, `bind`, `fold`, `get_ok`, `get_error`, `map_error`,
|
||||
`to_option`, `is_ok`, `is_error`.
|
||||
- [ ] `String`: `length`, `get`, `sub`, `concat`, `split_on_char`, `trim`,
|
||||
`uppercase_ascii`, `lowercase_ascii`, `contains`, `starts_with`, `ends_with`,
|
||||
`index_opt`, `replace_all` (non-stdlib but needed).
|
||||
- [ ] `Char`: `code`, `chr`, `escaped`, `lowercase_ascii`, `uppercase_ascii`.
|
||||
- [ ] `Int`/`Float`: arithmetic, `to_string`, `of_string_opt`, `min_int`, `max_int`.
|
||||
- [ ] `Hashtbl`: `create`, `add`, `replace`, `find`, `find_opt`, `remove`, `mem`,
|
||||
`iter`, `fold`, `length` — backed by SX mutable dict.
|
||||
- [ ] `Map.Make` functor — balanced BST backed by SX sorted dict.
|
||||
- [ ] `Set.Make` functor.
|
||||
- [ ] `Printf`: `sprintf`, `printf`, `eprintf` — format strings via `(format ...)`.
|
||||
- [ ] `Sys`: `argv`, `getenv_opt`, `getcwd` — via `perform` IO.
|
||||
- [ ] Scoreboard runner: `lib/ocaml/conformance.sh` + `scoreboard.json`.
|
||||
- [ ] Target: 150+ tests across all stdlib modules.
|
||||
|
||||
### Phase 7 — Dream web framework (`lib/dream/`)
|
||||
|
||||
The five types: `request`, `response`, `handler = request -> response`,
|
||||
`middleware = handler -> handler`, `route`. Everything else is a function over these.
|
||||
|
||||
- [ ] **Core types** in `lib/dream/types.sx`: request/response records, route record.
|
||||
- [ ] **Router** in `lib/dream/router.sx`:
|
||||
- `dream-get path handler`, `dream-post path handler`, etc. for all HTTP methods.
|
||||
- `dream-scope prefix middlewares routes` — prefix mount with middleware chain.
|
||||
- `dream-router routes` — dispatch tree, returns handler; no match → 404.
|
||||
- Path param extraction: `:name` segments, `**` wildcard.
|
||||
- `dream-param req name` — retrieve matched path param.
|
||||
- [ ] **Middleware** in `lib/dream/middleware.sx`:
|
||||
- `dream-pipeline middlewares handler` — compose middleware left-to-right.
|
||||
- `dream-no-middleware` — identity.
|
||||
- Logger: `(dream-logger next req)` — logs method, path, status, timing.
|
||||
- Content-type sniffer.
|
||||
- [ ] **Sessions** in `lib/dream/session.sx`:
|
||||
- Cookie-backed session middleware.
|
||||
- `dream-session-field req key`, `dream-set-session-field req key val`.
|
||||
- `dream-invalidate-session req`.
|
||||
- [ ] **Flash messages** in `lib/dream/flash.sx`:
|
||||
- `dream-flash-middleware` — single-request cookie store.
|
||||
- `dream-add-flash-message req category msg`.
|
||||
- `dream-flash-messages req` — returns list of `(category, msg)`.
|
||||
- [ ] **Forms + CSRF** in `lib/dream/form.sx`:
|
||||
- `dream-form req` — returns `(Ok fields)` or `(Err :csrf-token-invalid)`.
|
||||
- `dream-multipart req` — streaming multipart form data.
|
||||
- CSRF middleware: stateless signed tokens, session-scoped.
|
||||
- `dream-csrf-tag req` — returns hidden input fragment for SX templates.
|
||||
- [ ] **WebSockets** in `lib/dream/websocket.sx`:
|
||||
- `dream-websocket handler` — upgrades request; handler `(fn (ws) ...)`.
|
||||
- `dream-send ws msg`, `dream-receive ws`, `dream-close ws`.
|
||||
- [ ] **Static files:** `dream-static root-path` — serves files, ETags, range requests.
|
||||
- [ ] **`dream-run`**: wires root handler into SX's `perform (:http-listen ...)`.
|
||||
- [ ] **Demos** in `lib/dream/demos/`:
|
||||
- `hello.ml` → `lib/dream/demos/hello.sx`: "Hello, World!" route.
|
||||
- `counter.ml` → `lib/dream/demos/counter.sx`: in-memory counter with sessions.
|
||||
- `chat.ml` → `lib/dream/demos/chat.sx`: multi-room WebSocket chat.
|
||||
- `todo.ml` → `lib/dream/demos/todo.sx`: CRUD list with forms + CSRF.
|
||||
- [ ] Tests in `lib/dream/tests/`: routing dispatch, middleware composition,
|
||||
session round-trip, CSRF accept/reject, flash read-after-write — 60+ tests.
|
||||
|
||||
### Phase 8 — ReasonML syntax variant (`lib/reasonml/`)
|
||||
|
||||
ReasonML is OCaml with a JS-friendly surface: semicolons, `let` with `=` everywhere,
|
||||
`=>` for lambdas, `switch` for match, `{j|...|j}` string interpolation. Same semantics —
|
||||
different tokenizer + parser, same `lib/ocaml/transpile.sx` output.
|
||||
|
||||
- [ ] **Tokenizer** in `lib/reasonml/tokenizer.sx`:
|
||||
- `let x = e;` binding syntax (semicolons required).
|
||||
- `(x, y) => e` arrow function syntax.
|
||||
- `switch (x) { | Pat => e | ... }` for match.
|
||||
- JSX: `<Comp prop=val />`, `<div>children</div>`.
|
||||
- String interpolation: `{j|hello $(name)|j}`.
|
||||
- Type annotations: `x : int`, `let f : int => int = x => x + 1`.
|
||||
- [ ] **Parser** in `lib/reasonml/parser.sx`:
|
||||
- Produce same OCaml AST nodes as `lib/ocaml/parser.sx`.
|
||||
- JSX → SX component calls: `<Comp x=1 />` → `(~comp :x 1)`.
|
||||
- Multi-arg functions: `(x, y) => e` → auto-curried pair.
|
||||
- [ ] Shared transpiler: `lib/reasonml/transpile.sx` delegates to
|
||||
`lib/ocaml/transpile.sx` (parse → ReasonML AST → OCaml AST → SX AST).
|
||||
- [ ] Tests in `lib/reasonml/tests/`: tokenizer, parser, eval, JSX — 40+ tests.
|
||||
- [ ] ReasonML Dream demos: translate Phase 7 demos to ReasonML syntax.
|
||||
|
||||
## The meta-circular angle
|
||||
|
||||
SX is bootstrapped to OCaml (`hosts/ocaml/`). Running OCaml inside SX running on OCaml is
|
||||
the "mother tongue" closure: OCaml → SX → OCaml. This means:
|
||||
|
||||
- The OCaml host's native pattern matching and ADTs are exact reference semantics for
|
||||
the SX-level implementation — any mismatch is a bug.
|
||||
- The SX `match` / `define-type` primitives (Phase 6 of the primitives roadmap) were
|
||||
built knowing OCaml was the intended target.
|
||||
- When debugging the transpiler, the OCaml REPL is always available as oracle.
|
||||
- Dream running in SX can serve the sx.rose-ash.com docs site — the framework that
|
||||
describes the runtime it runs on.
|
||||
|
||||
## Key dependencies
|
||||
|
||||
- **Phase 6 ADT primitive** (`define-type`/`match`) — required before Phase 3.
|
||||
- **`perform`/`cek-resume`** IO suspension — required before Phase 7 (Dream async).
|
||||
- **HO forms** and first-class lambdas — already in spec, no blocker.
|
||||
- **Module system** (Phase 4) is independent of type inference (Phase 5) — can overlap.
|
||||
- **ReasonML** (Phase 8) can start once OCaml parser is stable (after Phase 2).
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
_(awaiting phase 1)_
|
||||
|
||||
## Blockers
|
||||
|
||||
_(none yet)_
|
||||
@@ -39,59 +39,93 @@ Representation choices (finalise in phase 1, document here):
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + term parser (no operator table)
|
||||
- [ ] Tokenizer: atoms (lowercase/quoted), variables (uppercase/`_`), numbers, strings, punct `( ) , . [ ] | ! :-`, comments (`%`, `/* */`)
|
||||
- [ ] Parser: clauses `head :- body.` and facts `head.`; terms `atom | Var | number | compound(args) | [list,sugar]`
|
||||
- [ ] **Skip for phase 1:** operator table. `X is Y + 1` must be written `is(X, '+'(Y, 1))`; `=` written `=(X, Y)`. Operators land in phase 4.
|
||||
- [ ] Unit tests in `lib/prolog/tests/parse.sx`
|
||||
- [x] Tokenizer: atoms (lowercase/quoted), variables (uppercase/`_`), numbers, strings, punct `( ) , . [ ] | ! :-`, comments (`%`, `/* */`)
|
||||
- [x] Parser: clauses `head :- body.` and facts `head.`; terms `atom | Var | number | compound(args) | [list,sugar]`
|
||||
- [x] **Skip for phase 1:** operator table. `X is Y + 1` must be written `is(X, '+'(Y, 1))`; `=` written `=(X, Y)`. Operators land in phase 4.
|
||||
- [x] Unit tests in `lib/prolog/tests/parse.sx` — 25 pass
|
||||
|
||||
### Phase 2 — unification + trail
|
||||
- [ ] `make-var`, `walk` (follow binding chain), `prolog-unify!` (terms + trail → bool), `trail-undo-to!`
|
||||
- [ ] Occurs-check off by default, exposed as flag
|
||||
- [ ] 30+ unification tests in `lib/prolog/tests/unify.sx`: atoms, vars, compounds, lists, cyclic (no-occurs-check), mutual occurs
|
||||
- [x] `make-var`, `walk` (follow binding chain), `prolog-unify!` (terms + trail → bool), `trail-undo-to!`
|
||||
- [x] Occurs-check off by default, exposed as flag
|
||||
- [x] 30+ unification tests in `lib/prolog/tests/unify.sx`: atoms, vars, compounds, lists, cyclic (no-occurs-check), mutual occurs — 47 pass
|
||||
|
||||
### Phase 3 — clause DB + DFS solver + cut + first classic programs
|
||||
- [ ] Clause DB: `"functor/arity" → list-of-clauses`, loader inserts
|
||||
- [ ] Solver: DFS with choice points backed by delimited continuations (`lib/callcc.sx`). On goal entry, capture; per matching clause, unify head + recurse body; on failure, undo trail, try next
|
||||
- [ ] Cut (`!`): cut barrier at current choice-point frame; collapse all up to barrier
|
||||
- [ ] Built-ins: `=/2`, `\\=/2`, `true/0`, `fail/0`, `!/0`, `,/2`, `;/2`, `->/2` inside `;`, `call/1`, `write/1`, `nl/0`
|
||||
- [ ] Arithmetic `is/2` with `+ - * / mod abs`
|
||||
- [ ] Classic programs in `lib/prolog/tests/programs/`:
|
||||
- [ ] `append.pl` — list append (with backtracking)
|
||||
- [ ] `reverse.pl` — naive reverse
|
||||
- [ ] `member.pl` — generate all solutions via backtracking
|
||||
- [ ] `nqueens.pl` — 8-queens
|
||||
- [ ] `family.pl` — facts + rules (parent/ancestor)
|
||||
- [ ] `lib/prolog/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
- [ ] Target: all 5 classic programs passing
|
||||
- [x] Clause DB: `"functor/arity" → list-of-clauses`, loader inserts — `pl-mk-db` / `pl-db-add!` / `pl-db-load!` / `pl-db-lookup` / `pl-db-lookup-goal`, 14 tests in `tests/clausedb.sx`
|
||||
- [x] Solver: DFS with choice points backed by delimited continuations (`lib/callcc.sx`). On goal entry, capture; per matching clause, unify head + recurse body; on failure, undo trail, try next — first cut: trail-based undo + CPS k (no shift/reset yet, per briefing gotcha). Built-ins so far: `true/0`, `fail/0`, `=/2`, `,/2`. Refactor to delimited conts later.
|
||||
- [x] Cut (`!`): cut barrier at current choice-point frame; collapse all up to barrier — two-cut-box scheme: each `pl-solve-user!` creates a fresh inner-cut-box (set by `!` in this predicate's body) AND snapshots the outer-cut-box state on entry. After body fails, abandon clause alternatives if (a) inner was set or (b) outer transitioned false→true during this call. Lets post-cut goals backtrack normally while blocking pre-cut alternatives. 6 cut tests cover bare cut, clause-commit, choice-commit, cut+fail, post-cut backtracking, nested-cut isolation.
|
||||
- [x] Built-ins: `=/2`, `\\=/2`, `true/0`, `fail/0`, `!/0`, `,/2`, `;/2`, `->/2` inside `;`, `call/1`, `write/1`, `nl/0` — all 11 done. `write/1` and `nl/0` use a global `pl-output-buffer` string + `pl-output-clear!` for testability; `pl-format-term` walks deep then renders atoms/nums/strs/compounds/vars (var → `_<id>`). Note: cut-transparency via `;` not testable yet without operator support — `;(,(a,!), b)` parser-rejects because `,` is body-operator-only; revisit in phase 4.
|
||||
- [x] Arithmetic `is/2` with `+ - * / mod abs` — `pl-eval-arith` walks deep, recurses on compounds, dispatches on functor; binary `+ - * / mod`, binary AND unary `-`, unary `abs`. `is/2` evaluates RHS, wraps as `("num" v)`, unifies via `pl-solve-eq!`. 11 tests cover each op + nested + ground LHS match/mismatch + bound-var-on-RHS chain.
|
||||
- [x] Classic programs in `lib/prolog/tests/programs/`:
|
||||
- [x] `append.pl` — list append (with backtracking) — `lib/prolog/tests/programs/append.{pl,sx}`. 6 tests cover: build (`append([], L, X)`, `append([1,2], [3,4], X)`), check ground match/mismatch, full split-backtracking (`append(X, Y, [1,2,3])` → 4 solutions), single-deduce (`append(X, [3], [1,2,3])` → X=[1,2]).
|
||||
- [x] `reverse.pl` — naive reverse — `lib/prolog/tests/programs/reverse.{pl,sx}`. Naive reverse via append: `reverse([H|T], R) :- reverse(T, RT), append(RT, [H], R)`. 6 tests cover empty, singleton, 3-list, 4-atom-list, ground match, ground mismatch.
|
||||
- [x] `member.pl` — generate all solutions via backtracking — `lib/prolog/tests/programs/member.{pl,sx}`. Classic 2-clause `member(X, [X|_])` + `member(X, [_|T]) :- member(X, T)`. 7 tests cover bound-element hit/miss, empty list, generator (count = list length), first-solution binding, duplicate matches counted twice, anonymous head-cell unification.
|
||||
- [x] `nqueens.pl` — 8-queens — `lib/prolog/tests/programs/nqueens.{pl,sx}`. Permute-and-test formulation: `queens(L, Qs) :- permute(L, Qs), safe(Qs)` + `select` + `safe` + `no_attack`. Tested at N=1 (1), N=2 (0), N=3 (0), N=4 (2), N=5 (10) plus first-solution check at N=4 = `[2, 4, 1, 3]`. N=8 omitted — interpreter is too slow (40320 perms); add once compiled clauses or constraint-style placement land. `range/3` skipped pending arithmetic-comparison built-ins (`>/2` etc.).
|
||||
- [x] `family.pl` — facts + rules (parent/ancestor) — `lib/prolog/tests/programs/family.{pl,sx}`. 5 parent facts + male/female + derived `father`/`mother`/`ancestor`/`sibling`. 10 tests cover direct facts, fact count, transitive ancestor through 3 generations, descendant counting, gender-restricted father/mother, sibling via shared parent + `\=`.
|
||||
- [x] `lib/prolog/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` — bash script feeds load + eval epoch script to sx_server, parses each suite's `{:failed N :passed N :total N :failures (...)}` line, writes JSON (machine) + MD (human) scoreboards. Exit non-zero on any failure. `SX_SERVER` env var overrides binary path. First scoreboard: 183 / 183.
|
||||
- [x] Target: all 5 classic programs passing — append (6) + reverse (6) + member (7) + nqueens (6) + family (10) = 35 program tests, all green. Phase 3 architecturally complete bar the conformance harness/scoreboard.
|
||||
|
||||
### Phase 4 — operator table + more built-ins (next run)
|
||||
- [ ] Operator table parsing (prefix/infix/postfix, precedence, assoc)
|
||||
- [ ] `assert/1`, `asserta/1`, `assertz/1`, `retract/1`
|
||||
- [ ] `findall/3`, `bagof/3`, `setof/3`
|
||||
- [ ] `copy_term/2`, `functor/3`, `arg/3`, `=../2`
|
||||
- [ ] String/atom predicates
|
||||
- [x] Operator table parsing (prefix/infix/postfix, precedence, assoc) — `pl-op-table` (15 entries: `, ; -> = \= is < > =< >= + - * / mod`); precedence-climbing parser via `pp-parse-primary` + `pp-parse-term-prec` + `pp-parse-op-rhs`. Parens override precedence. Args inside compounds parsed at 999 so `,` stays as separator. xfx/xfy/yfx supported; prefix/postfix deferred (so `-5` still tokenises as bare atom + num as before). Comparison built-ins `</2 >/2 =</2 >=/2` added. New `tests/operators.sx` 19 tests cover assoc/precedence/parens + solver via infix.
|
||||
- [x] `assert/1`, `asserta/1`, `assertz/1`, `retract/1` — `assert` aliases `assertz`. Helpers `pl-rt-to-ast` (deep-walk + replace runtime vars with `_G<id>` parse markers) + `pl-build-clause` (detect `:-` head). `assertz` uses `pl-db-add!`; `asserta` uses new `pl-db-prepend!`. `retract` walks goal, looks up by functor/arity, tries each clause via unification, removes first match by index (`pl-list-without`). 11 tests in `tests/dynamic.sx`. Rule-asserts now work — `:-` added to op table (prec 1200 xfx) with fix to `pl-token-op` accepting `"op"` token type. 15 tests in `tests/assert_rules.sx`.
|
||||
- [x] `findall/3`, `bagof/3`, `setof/3` — shared `pl-collect-solutions` runs the goal in a fresh cut-box, deep-copies the template (via `pl-deep-copy` with var-map for shared-var preservation) on each success, returns false to backtrack, then restores trail. `findall` always succeeds with a (possibly empty) list. `bagof` fails on empty. `setof` builds a string-keyed dict via `pl-format-term` for sort+dedupe (via `keys` + `sort`), fails on empty. Existential `^` deferred (operator). 11 tests in `tests/findall.sx`.
|
||||
- [x] `copy_term/2`, `functor/3`, `arg/3`, `=../2` — `copy_term/2` reuses `pl-deep-copy` with a fresh var-map (preserves source aliasing). `functor/3` handles 4 modes: compound→{name, arity}, atom→{atom, 0}, num→{num, 0}, var with ground name+arity→constructed term (`pl-make-fresh-args` for compound case). `arg/3` extracts 1-indexed arg from compound. **`=../2` deferred** — the tokenizer treats `.` as the clause terminator unconditionally, so `=..` lexes as `=` + `.` + `.`; needs special-case lex (or surface syntax via a different name). 14 tests in `tests/term_inspect.sx`.
|
||||
- [x] String/atom predicates
|
||||
|
||||
### Phase 5 — Hyperscript integration
|
||||
- [ ] `prolog-query` primitive callable from SX/Hyperscript
|
||||
- [ ] Hyperscript DSL: `when allowed(user, :edit) then …`
|
||||
- [ ] Integration suite
|
||||
- [x] `prolog-query` primitive callable from SX/Hyperscript
|
||||
- [x] Hyperscript DSL: `when allowed(user, action) then …` — `lib/prolog/hs-bridge.sx`: `pl-hs-query` (bool goal test) + `pl-hs-predicate/1,2,3` factories + `pl-hs-install`. No parser/compiler changes needed: Hyperscript already compiles `allowed(user, action)` to `(allowed user action)` — a plain SX call backed by the Prolog DB.
|
||||
- [x] Integration suite
|
||||
|
||||
### Phase 6 — ISO conformance
|
||||
- [ ] Vendor Hirst's conformance tests
|
||||
- [ ] Drive scoreboard to 200+
|
||||
- [x] Vendor Hirst's conformance tests
|
||||
- [x] Drive scoreboard to 200+
|
||||
|
||||
### Phase 7 — compiler (later, optional)
|
||||
- [ ] Compile clauses to SX continuations for speed
|
||||
- [ ] Keep interpreter as the reference
|
||||
- [x] Compile clauses to SX continuations for speed
|
||||
- [x] Keep interpreter as the reference
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first. Agent appends on every commit._
|
||||
|
||||
- 2026-05-06 — Hyperscript bridge (`lib/prolog/hs-bridge.sx`): `pl-hs-query`, `pl-hs-predicate/1,2,3`, `pl-hs-install`. No parser/compiler changes needed — Hyperscript already compiles `when allowed(user, action)` to `(allowed user action)`, a plain SX call; bridge factories wire a Prolog DB as the backing implementation. 19 tests in `tests/hs_bridge.sx`. Total **590** (+19).
|
||||
- 2026-05-05 — Integration test suite (`tests/integration.sx`): 20 end-to-end tests via `pl-query-*` API covering permission system (6), graph reachability (4), quicksort (4), fibonacci (3), dynamic KB (3). Suite added to conformance harness. Total **571** (+20).
|
||||
- 2026-04-25 — `pl-compiled-matches-interp?` cross-validator in `compiler.sx`: loads source into both a plain and a compiled DB, runs the same goal, returns true iff solution counts match. `tests/cross_validate.sx` applies this to 17 goals across append/member/ancestor/cut/arithmetic/if-then-else, locking the interpreter as the reference against which any future compiler change must agree. Total **551** (+17).
|
||||
- 2026-04-25 — Clause compiler (`lib/prolog/compiler.sx`): `pl-compile-clause` converts parse-AST clauses to SX closures `(fn (goal trail db cut-box k) bool)`. Pre-collects var names at compile time; `pl-cmp-build-term` reconstructs fresh runtime terms per call. `pl-compile-db!` compiles all clauses in a DB and stores them in `:compiled` table. `pl-solve-user!` in runtime.sx auto-dispatches to compiled lambdas when present, falls back to interpreted. `pl-try-compiled-clauses!` mirrors `pl-try-clauses!` cut semantics. 17 tests in `tests/compiler.sx`. Total **534** (+17).
|
||||
- 2026-04-25 — `predsort/3` (insertion-sort with 3-arg comparator predicate, deduplicates `=` pairs), `term_variables/2` (collect unbound vars left-to-right, dedup by id), arithmetic extensions (`floor/1`, `ceiling/1`, `truncate/1`, `round/1`, `sign/1`, `sqrt/1`, `pow/2`, `**/2`, `^/2`, `integer/1`, `float/1`, `float_integer_part/1`, `float_fractional_part/1`). 21 tests in `tests/advanced.sx`. Total **517** (+21).
|
||||
- 2026-04-25 — `sub_atom/5` (non-deterministic substring enumeration; CPS loop over all (start,sublen) pairs; trail-undo only on backtrack) + `aggregate_all/3` (6 templates: count/bag/sum/max/min/set; uses `pl-collect-solutions`). 25 tests in `tests/string_agg.sx`. Total **496** (+25).
|
||||
- 2026-04-25 — `:-` operator + assert with rules: added `(list ":-" 1200 "xfx")` to `pl-op-table`; fixed `pl-token-op` to accept `"op"` token type (tokenizer emits `:-` as `"op"`, not `"atom"`). `pl-build-clause` already handled `("compound" ":-" ...)`. `assert((head :- body))` now works for facts+rules. 15 tests in `tests/assert_rules.sx`. Total **471** (+15).
|
||||
- 2026-04-25 — IO/term predicates: `term_to_atom/2` (bidirectional: format term or parse atom), `term_string/2` (alias), `with_output_to/2` (atom/string sinks — saves/restores `pl-output-buffer`), `writeln/1`, `format/1` (~n/~t/~~), `format/2` (~w/~a/~d pull from arg list). 24 tests in `tests/io_predicates.sx`. Total **456** (+24).
|
||||
- 2026-04-25 — Char predicates: `char_type/2` (9 modes: alpha/alnum/digit/digit(N)/space/white/upper(L)/lower(U)/ascii(C)/punct), `upcase_atom/2`, `downcase_atom/2`, `string_upper/2`, `string_lower/2`. 10 helpers using `char-code`/`char-from-code` SX primitives. 27 tests in `tests/char_predicates.sx`. Total **432** (+27).
|
||||
- 2026-04-25 — Set/fold predicates: `foldl/4` (CPS fold-left, threads accumulator via `pl-apply-goal`), `list_to_set/2` (dedup preserving first-occurrence), `intersection/3`, `subtract/3`, `union/3` (all via `pl-struct-eq?`). 3 new helpers, 15 tests in `tests/set_predicates.sx`. Total **405** (+15).
|
||||
- 2026-04-25 — Meta-call predicates: `forall/2` (negation-of-counterexample), `maplist/2` (goal over list), `maplist/3` (map goal building output list), `include/3` (filter by goal success), `exclude/3` (filter by goal failure). New `pl-apply-goal` helper extends a goal with extra args. 15 tests in `tests/meta_call.sx`. Total **390** (+15).
|
||||
- 2026-04-25 — List/utility predicates: `==/2`, `\==/2` (structural equality/inequality via `pl-struct-eq?`), `flatten/2` (deep Prolog-list flatten), `numlist/3` (integer range list), `atomic_list_concat/2` (join with no sep), `atomic_list_concat/3` (join with separator), `sum_list/2`, `max_list/2`, `min_list/2` (arithmetic folds), `delete/3` (remove all struct-equal elements). 7 new helpers, 33 tests in `tests/list_predicates.sx`. Total **375** (+33).
|
||||
- 2026-04-25 — Meta/logic predicates: `\+/1` (negation-as-failure, trail-undo on success), `not/1` (alias), `once/1` (commit to first solution via if-then-else), `ignore/1` (always succeed), `ground/1` (all vars bound), `sort/2` (sort + dedup by formatted key), `msort/2` (sort, keep dups), `atom_number/2` (bidirectional), `number_string/2` (bidirectional). 2 helpers (`pl-ground?`, `pl-sort-pairs-dedup`). 25 tests in `tests/meta_predicates.sx`. Total **342** (+25).
|
||||
- 2026-04-25 — ISO utility predicates batch: `succ/2` (bidirectional), `plus/3` (3-mode bidirectional), `between/3` (backtracking range generator), `length/2` (bidirectional list length + var-list constructor), `last/2`, `nth0/3`, `nth1/3`, `max/2` + `min/2` in arithmetic eval. 6 new helper functions (`pl-list-length`, `pl-make-list-of-vars`, `pl-between-loop!`, `pl-solve-between!`, `pl-solve-last!`, `pl-solve-nth0!`). 29 tests in `tests/iso_predicates.sx`. Phase 6 complete: scoreboard already at 317, far above 200+ target. Hyperscript DSL blocked (needs `lib/hyperscript/**`). Total **317** (+29).
|
||||
- 2026-04-25 — `prolog-query` SX API (`lib/prolog/query.sx`). New public API layer: `pl-load source-str → db`, `pl-query-all db query-str → list of solution dicts`, `pl-query-one db query-str → dict or nil`, `pl-query src query → list` (convenience). Each solution dict maps variable name strings to their formatted term strings. Var names extracted from pre-instantiation parse AST. Trail is marked before solve and reset after to ensure clean state. 16 tests in `tests/query_api.sx` cover fact lookup, no-solution, boolean queries, multi-var, recursive rules, is/2 built-in, query-one, convenience form. Total **288** (+16).
|
||||
- 2026-04-25 — String/atom predicates. Type-test predicates: `var/1`, `nonvar/1`, `atom/1`, `number/1`, `integer/1`, `float/1` (always-fail), `compound/1`, `callable/1`, `atomic/1`, `is_list/1`. String/atom operations: `atom_length/2`, `atom_concat/3` (3 modes: both-ground, result+first, result+second), `atom_chars/2` (bidirectional), `atom_codes/2` (bidirectional), `char_code/2` (bidirectional), `number_codes/2`, `number_chars/2`. 7 helper functions in runtime.sx (`pl-list-to-prolog`, `pl-proper-list?`, `pl-prolog-list-to-sx`, `pl-solve-atom-concat!`, `pl-solve-atom-chars!`, `pl-solve-atom-codes!`, `pl-solve-char-code!`). 34 tests in `tests/atoms.sx`. Total **272** (+34).
|
||||
- 2026-04-25 — `copy_term/2` + `functor/3` + `arg/3` (term inspection). `copy_term` is a one-line dispatch to existing `pl-deep-copy`. `functor/3` is bidirectional — decomposes a bound compound/atom/num into name+arity OR constructs from ground name+arity (atom+positive-arity → compound with N anonymous fresh args via `pl-make-fresh-args`; arity 0 → atom/num). `arg/3` extracts 1-indexed arg with bounds-fail. New helper `pl-solve-eq2!` for paired-unification with shared trail-undo. 14 tests in `tests/term_inspect.sx`. Total **238** (+14). `=..` deferred — `.` always tokenizes as clause terminator; needs special lexer case.
|
||||
- 2026-04-25 — `findall/3` + `bagof/3` + `setof/3`. Shared collector `pl-collect-solutions` runs the goal in a fresh cut-box, deep-copies the template per success (`pl-deep-copy` walks term, allocates fresh runtime vars via shared var-map so co-occurrences keep aliasing), returns false to keep backtracking, then `pl-trail-undo-to!` to clean up. `findall` always builds a list. `bagof` fails on empty. `setof` uses a `pl-format-term`-keyed dict + SX `sort` for dedupe + ordering. New `tests/findall.sx` 11 tests. Total **224** (+11). Existential `^` deferred — needs operator.
|
||||
- 2026-04-25 — Dynamic clauses: `assert/1`, `assertz/1`, `asserta/1`, `retract/1`. New helpers `pl-rt-to-ast` (deep-walk runtime term → parse-AST, mapping unbound runtime vars to `_G<id>` markers so `pl-instantiate-fresh` produces fresh vars per call) + `pl-build-clause` + `pl-db-prepend!` + `pl-list-without`. `retract` keeps runtime vars (so the caller's vars get bound), walks head for the functor/arity key, tries each stored clause via `pl-unify!`, removes the first match by index. 11 tests in `tests/dynamic.sx`; conformance script gained dynamic row. Total **213** (+11). Rule-form asserts (`(H :- B)`) deferred until `:-` is in the op table.
|
||||
- 2026-04-25 — Phase 4 starts: operator-table parsing. Parser rewrite uses precedence climbing (xfx/xfy/yfx); 15-op table covers control (`, ; ->`), comparison (`= \\= is < > =< >=`), arithmetic (`+ - * / mod`). Parens override. Backwards-compatible: prefix-syntax compounds (`=(X, Y)`, `+(2, 3)`) still parse as before; existing 183 tests untouched. Added comparison built-ins `</2 >/2 =</2 >=/2` to runtime (eval both sides, compare). New `tests/operators.sx` 19 tests; conformance script gained an operators row. Total **202** (+19). Prefix/postfix deferred — `-5` keeps old bare-atom semantics.
|
||||
- 2026-04-25 — Conformance harness landed. `lib/prolog/conformance.sh` runs all 9 suites in one sx_server epoch, parses the `{:failed/:passed/:total/:failures}` summary lines, and writes `scoreboard.json` + `scoreboard.md`. `SX_SERVER` env var overrides the binary path; default points at the main-repo build. Phase 3 fully complete: 183 / 183 passing across parse/unify/clausedb/solve/append/reverse/member/nqueens/family.
|
||||
- 2026-04-25 — `family.pl` fifth classic program — completes the 5-program target. 5-fact pedigree + male/female + derived father/mother/ancestor/sibling. 10 tests cover fact lookup + count, transitive ancestor through 3 generations, descendant counting (5), gender-restricted derivations, sibling via shared parent guarded by `\=`. Total 183 (+10). All 5 classic programs ticked; Phase 3 needs only conformance harness + scoreboard left.
|
||||
- 2026-04-25 — `nqueens.pl` fourth classic program. Permute-and-test variant exercises every Phase-3 feature: lists with `[H|T]` cons sugar, multi-clause backtracking, recursive `permute`/`select`/`safe`/`no_attack`, `is/2` arithmetic on diagonals, `\=/2` for diagonal-conflict check. 6 tests at N ∈ {1,2,3,4,5} with expected counts {1,0,0,2,10} + first-solution `[2,4,1,3]`. N=5 takes ~30s (120 perms × safe-check); N=8 omitted as it would be ~thousands of seconds. Total 173 (+6).
|
||||
- 2026-04-25 — `member.pl` third classic program. Standard 2-clause definition; 7 tests cover bound-element hit/miss, empty-list fail, generator-count = list length, first-solution binding (X=11), duplicate elements matched twice on backtrack, anonymous-head unification (`member(a, [X, b, c])` binds X=a). Total 167 (+7).
|
||||
- 2026-04-25 — `reverse.pl` second classic program. Naive reverse defined via append. 6 tests (empty/singleton/3-list/4-atom-list/ground match/ground mismatch). Confirms the solver handles non-trivial recursive composition: `reverse([1,2,3], R)` recurses to depth 3 then unwinds via 3 nested `append`s. Total 160 (+6).
|
||||
- 2026-04-25 — `append.pl` first classic program. `lib/prolog/tests/programs/append.pl` is the canonical 2-clause source; `append.sx` embeds the source as a string (no file-read primitive in SX yet) and runs 6 tests covering build, check, full split-backtrack (4 solutions), and deduction modes. Helpers `pl-ap-list-to-sx` / `pl-ap-term-to-sx` convert deep-walked Prolog lists (`("compound" "." (h t))` / `("atom" "[]")`) to SX lists for structural assertion. Total 154 (+6).
|
||||
- 2026-04-25 — `is/2` arithmetic landed. `pl-eval-arith` recursively evaluates ground RHS expressions (binary `+ - * /`, `mod`; binary+unary `-`; unary `abs`); `is/2` wraps the value as `("num" v)` and unifies via `pl-solve-eq!`, so it works in all three modes — bind unbound LHS, check ground LHS for equality, propagate from earlier var bindings on RHS. 11 tests, total 148 (+11). Without operator support, expressions must be written prefix: `is(X, +(2, *(3, 4)))`.
|
||||
- 2026-04-25 — `write/1` + `nl/0` landed using global string buffer (`pl-output-buffer` + `pl-output-clear!` + `pl-output-write!`). `pl-format-term` walks deep + dispatches on atom/num/str/compound/var; `pl-format-args` recursively comma-joins. 7 new tests cover atom/num/compound formatting, conjunction order, var-walk, and `nl`. Built-ins box (`=/2`, `\\=/2`, `true/0`, `fail/0`, `!/0`, `,/2`, `;/2`, `->/2`, `call/1`, `write/1`, `nl/0`) now ticked. Total 137 (+7).
|
||||
- 2026-04-25 — `->/2` if-then-else landed (both `;(->(C,T), E)` and standalone `->(C, T)` ≡ `(C -> T ; fail)`). `pl-solve-or!` now special-cases `->` in left arg → `pl-solve-if-then-else!`. Cond runs in a fresh local cut-box (ISO opacity for cut inside cond). Then-branch can backtrack, else-branch can backtrack, but cond commits to first solution. 9 new tests covering both forms, both branches, binding visibility, cond-commit, then-backtrack, else-backtrack. Total 130 (+9).
|
||||
- 2026-04-25 — Built-ins `\=/2`, `;/2`, `call/1` landed. `pl-solve-not-eq!` (try unify, always undo, succeed iff unify failed). `pl-solve-or!` (try left, on failure check cut and only try right if not cut). `call/1` opens a fresh inner cut-box (ISO opacity: cut inside `call(G)` commits G, not caller). 11 new tests in `tests/solve.sx` cover atoms+vars for `\=`, both branches + count for `;`, and `call/1` against atoms / compounds / bound goal vars. Total 121 (+11). Box not yet ticked — `->/2`, `write/1`, `nl/0` still pending.
|
||||
- 2026-04-25 — Cut (`!/0`) landed. `pl-cut?` predicate; solver functions all take a `cut-box`; `pl-solve-user!` creates a fresh inner-cut-box and snapshots `outer-was-cut`; `pl-try-clauses!` abandons alternatives when inner.cut OR (outer.cut transitioned false→true during this call). 6 new cut tests in `tests/solve.sx` covering bare cut, clause-commit, choice-commit, cut+fail blocks alt clauses, post-cut goal backtracks freely, inner cut isolation. Total 110 (+6).
|
||||
- 2026-04-25 — Phase 3 DFS solver landed (CPS, trail-based backtracking; delimited conts deferred). `pl-solve!` + `pl-solve-eq!` + `pl-solve-user!` + `pl-try-clauses!` + `pl-solve-once!` + `pl-solve-count!` in runtime.sx. Built-ins: `true/0`, `fail/0`, `=/2`, `,/2`. New `tests/solve.sx` 18/18 green covers atomic goals, =, conjunction, fact lookup, multi-solution count, recursive ancestor rule, trail-undo verification. Bug fix: `pl-instantiate` had no `("clause" h b)` case → vars in rule head/body were never instantiated, so rule resolution silently failed against runtime-var goals. Added clause case to recurse with shared var-env. Total 104 (+18).
|
||||
- 2026-04-24 — Phase 3 clause DB landed: `pl-mk-db` + `pl-head-key` / `pl-clause-key` / `pl-goal-key` + `pl-db-add!` / `pl-db-load!` / `pl-db-lookup` / `pl-db-lookup-goal` in runtime.sx. New `tests/clausedb.sx` 14/14 green. Total 86 (+14). Loader preserves declaration order (append!).
|
||||
- 2026-04-24 — Verified phase 1+2 already implemented on loops/prolog: `pl-parse-tests-run!` 25/25, `pl-unify-tests-run!` 47/47 (72 total). Ticked phase 1+2 boxes.
|
||||
- _(awaiting phase 1)_
|
||||
|
||||
## Blockers
|
||||
|
||||
_Shared-file issues that need someone else to fix. Minimal repro only._
|
||||
|
||||
- _(none yet)_
|
||||
- **Phase 5 Hyperscript DSL** — `lib/hyperscript/**` is out of scope for this loop. Needs `lib/hyperscript/parser.sx` + evaluator to add `when allowed(user, :edit) then …` syntax. Skipping; Phase 5 item 1 (`prolog-query` SX API) is done.
|
||||
|
||||
@@ -113,6 +113,16 @@ Core mapping:
|
||||
- [ ] `Integer`: `times`, `upto`, `downto`, `step`, `digits`, `gcd`, `lcm`
|
||||
- [ ] Drive corpus to 200+ green
|
||||
|
||||
## SX primitive baseline
|
||||
|
||||
Use vectors for arrays; numeric tower + rationals for numbers; ADTs for tagged data;
|
||||
coroutines for fibers; string-buffer for mutable string building; bitwise ops for bit
|
||||
manipulation; multiple values for multi-return; promises for lazy evaluation; hash tables
|
||||
for mutable associative storage; sets for O(1) membership; sequence protocol for
|
||||
polymorphic iteration; gensym for unique symbols; char type for characters; string ports
|
||||
+ read/write for reader protocols; regexp for pattern matching; bytevectors for binary
|
||||
data; format for string templating.
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
@@ -50,64 +50,100 @@ Core mapping:
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser
|
||||
- [ ] Tokenizer: identifiers, keywords (`foo:`), binary selectors (`+`, `==`, `,`, `->`, `~=` etc.), numbers (radix `16r1F`, scaled `1.5s2`), strings `'…''…'`, characters `$c`, symbols `#foo` `#'foo bar'` `#+`, byte arrays `#[1 2 3]`, literal arrays `#(1 #foo 'x')`, comments `"…"`
|
||||
- [ ] Parser: chunk format (`! !` separators), class definitions (`Object subclass: #X instanceVariableNames: '…' classVariableNames: '…' …`), method definitions (`extend: #Foo with: 'bar ^self'`), pragmas `<primitive: 1>`, blocks `[:a :b | | t1 t2 | …]`, cascades, message precedence (unary > binary > keyword)
|
||||
- [ ] Unit tests in `lib/smalltalk/tests/parse.sx`
|
||||
- [x] Tokenizer: identifiers, keywords (`foo:`), binary selectors (`+`, `==`, `,`, `->`, `~=` etc.), numbers (radix `16r1F`; **scaled `1.5s2` deferred**), strings `'…''…'`, characters `$c`, symbols `#foo` `#'foo bar'` `#+`, byte arrays `#[1 2 3]` (open token), literal arrays `#(1 #foo 'x')` (open token), comments `"…"`
|
||||
- [x] Parser (expression level): blocks `[:a :b | | t1 t2 | …]`, cascades, message precedence (unary > binary > keyword), assignment, return, statement sequences, literal arrays, byte arrays, paren grouping, method headers (`+ other`, `at:put:`, unary, with temps and body). Class-definition keyword messages parse as ordinary keyword sends — no special-case needed.
|
||||
- [x] Parser (chunk-stream level): `st-read-chunks` splits source on `!` (with `!!` doubling) and `st-parse-chunks` runs the Pharo file-in state machine — `methodsFor:` / `class methodsFor:` opens a method batch, an empty chunk closes it. Pragmas `<primitive: …>` (incl. multiple keyword pairs, before or after temps, multiple per method) parsed into the method AST.
|
||||
- [x] Unit tests in `lib/smalltalk/tests/parse.sx`
|
||||
|
||||
### Phase 2 — object model + sequential eval
|
||||
- [ ] Class table + bootstrap: `Object`, `Behavior`, `Class`, `Metaclass`, `UndefinedObject`, `Boolean`/`True`/`False`, `Number`/`Integer`/`Float`, `String`, `Symbol`, `Array`, `Block`
|
||||
- [ ] `smalltalk-eval-ast`: literals, variable reference, assignment, message send, cascade, sequence, return
|
||||
- [ ] Method lookup: walk class → superclass; cache hit-class on `(class, selector)`
|
||||
- [ ] `doesNotUnderstand:` fallback constructing `Message` object
|
||||
- [ ] `super` send (lookup starts at superclass of *defining* class, not receiver class)
|
||||
- [ ] 30+ tests in `lib/smalltalk/tests/eval.sx`
|
||||
- [x] Class table + bootstrap (`lib/smalltalk/runtime.sx`): canonical hierarchy installed (`Object`, `Behavior`, `ClassDescription`, `Class`, `Metaclass`, `UndefinedObject`, `Boolean`/`True`/`False`, `Magnitude`/`Number`/`Integer`/`SmallInteger`/`Float`/`Character`, `Collection`/`SequenceableCollection`/`ArrayedCollection`/`Array`/`String`/`Symbol`/`OrderedCollection`/`Dictionary`, `BlockClosure`). User class definition via `st-class-define!`, methods via `st-class-add-method!` (stamps `:defining-class` for super), method lookup walks chain, ivars accumulated through superclass chain, native SX value types map to Smalltalk classes via `st-class-of`.
|
||||
- [x] `smalltalk-eval-ast` (`lib/smalltalk/eval.sx`): all literal kinds, ident resolution (locals → ivars → class refs), self/super/thisContext, assignment (locals or ivars, mutating), message send, cascade, sequence, and ^return via a sentinel marker (proper continuation-based escape is the Phase 3 showcase). Frames carry a parent chain so blocks close over outer locals. Primitive method tables for SmallInteger/Float, String/Symbol, Boolean, UndefinedObject, Array, BlockClosure (value/value:/whileTrue:/etc.), and class-side `new`/`name`/etc. Also satisfies "30+ tests" — 60 eval tests.
|
||||
- [x] Method lookup: walk class → superclass already in `st-method-lookup-walk`; new cached wrapper `st-method-lookup` keys on `(class, selector, side)` and stores `:not-found` for negative results so DNU paths don't re-walk. Cache invalidates on `st-class-define!`, `st-class-add-method!`, `st-class-add-class-method!`, `st-class-remove-method!`, and full bootstrap. Stats helpers `st-method-cache-stats` / `st-method-cache-reset-stats!` for tests + later debugging.
|
||||
- [x] `doesNotUnderstand:` fallback. `Message` class added at bootstrap with `selector`/`arguments` ivars and accessor methods. Primitive senders (Number/String/Boolean/Nil/Array/BlockClosure/class-side) now return the `:unhandled` sentinel for unknown selectors; `st-send` builds a `Message` via `st-make-message` and routes through `st-dnu`, which looks up `doesNotUnderstand:` on the receiver's class chain (instance- or class-side as appropriate). User overrides intercept unknowns and see the symbol selector + arguments array in the Message.
|
||||
- [x] `super` send. Method invocation captures the defining class on the frame; `st-super-send` walks from `(st-class-superclass defining-class)` (instance- or class-side as appropriate). Falls through primitives → DNU when no method is found. Receiver is preserved as `self`, so ivar mutations stick. Verified for: subclass override calls parent, inherited `super` resolves to *defining* class's parent (not receiver's), multi-level `A→B→C` chain, super inside a block, super walks past an intermediate class with no local override.
|
||||
- [x] 30+ tests in `lib/smalltalk/tests/eval.sx` (60 tests, covering literals through user-class method dispatch with cascades and closures)
|
||||
|
||||
### Phase 3 — blocks + non-local return (THE SHOWCASE)
|
||||
- [ ] Method invocation captures a `^k` (the return continuation) and binds it as the block's escape
|
||||
- [ ] `^expr` from inside a block invokes that captured `^k`
|
||||
- [ ] `BlockContext>>value`, `value:`, `value:value:`, …, `valueWithArguments:`
|
||||
- [ ] `whileTrue:` / `whileTrue` / `whileFalse:` / `whileFalse` as ordinary block sends — runtime intrinsifies the loop in the bytecode JIT
|
||||
- [ ] `ifTrue:` / `ifFalse:` / `ifTrue:ifFalse:` as block sends, similarly intrinsified
|
||||
- [ ] Escape past returned-from method raises `BlockContext>>cannotReturn:`
|
||||
- [ ] Classic programs in `lib/smalltalk/tests/programs/`:
|
||||
- [ ] `eight-queens.st`
|
||||
- [ ] `quicksort.st`
|
||||
- [ ] `mandelbrot.st`
|
||||
- [ ] `life.st` (Conway's Life, glider gun)
|
||||
- [ ] `fibonacci.st` (recursive + memoised)
|
||||
- [ ] `lib/smalltalk/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
- [x] Method invocation captures a `^k` (the return continuation) and binds it as the block's escape. `st-invoke` wraps body in `(call/cc (fn (k) ...))`; the frame's `:return-k` is set to k. Block creation copies `(get frame :return-k)` onto the block. Block invocation sets the new frame's `:return-k` to the block's saved one — so non-local return reaches *back through* any number of intermediate block invocations.
|
||||
- [x] `^expr` from inside a block invokes that captured `^k`. The "return" AST type evaluates the expression then calls `(k v)` on the frame's :return-k. Verified: `detect:in:` style early-exit, multi-level nested blocks, ^ from inside `to:do:`/`whileTrue:`, ^ from a block passed to a *different* method (Caller→Helper) returns from Caller.
|
||||
- [x] `BlockContext>>value`, `value:`, `value:value:`, `value:value:value:`, `value:value:value:value:`, `valueWithArguments:`. Implemented in `st-block-dispatch` + `st-block-apply` (eval iteration); pinned by 19 dedicated tests in `lib/smalltalk/tests/blocks.sx` covering arity through 4, valueWithArguments: with empty/non-empty arg arrays, closures over outer locals (read + mutate + later-mutation re-read), nested blocks, blocks as method arguments, `numArgs`, and `class`.
|
||||
- [x] `whileTrue:` / `whileTrue` / `whileFalse:` / `whileFalse` as ordinary block sends. `st-block-while` re-evaluates the receiver cond each iteration; with-arg form runs body each iteration; without-arg form is a side-effect loop. Now returns `nil` per ANSI/Pharo. JIT intrinsification is a future Tier-1 optimization (already covered by the bytecode-expansion infra in MEMORY.md). 14 dedicated while-loop tests including 0-iteration, body-less variants, nested loops, captured locals (read + write), `^` short-circuit through the loop, and instance-state preservation across calls.
|
||||
- [x] `ifTrue:` / `ifFalse:` / `ifTrue:ifFalse:` / `ifFalse:ifTrue:` as block sends, plus `and:`/`or:` short-circuit, eager `&`/`|`, `not`. Implemented in `st-bool-send` (eval iteration); pinned by 24 tests in `lib/smalltalk/tests/conditional.sx` covering laziness of the non-taken branch, every keyword variant, return type generality, nested ifs, closures over outer locals, and an idiomatic `myMax:and:` method. Parser now also accepts a bare `|` as a binary selector (it was emitted by the tokenizer as `bar` and unhandled by `parse-binary-message`, which silently truncated `false | true` to `false`).
|
||||
- [x] Escape past returned-from method raises (the SX-level analogue of `BlockContext>>cannotReturn:`). Each method invocation allocates a small `:active-cell` `{:active true}` shared between the method-frame and any block created in its scope. `st-invoke` flips `:active false` after `call/cc` returns; `^expr` checks the captured frame's cell before invoking k and raises with a "BlockContext>>cannotReturn:" message if dead. Verified by `lib/smalltalk/tests/cannot_return.sx` (5 tests using SX `guard` to catch the raise). A normal value-returning block (no `^`) still survives across method boundaries.
|
||||
- [x] Classic programs in `lib/smalltalk/tests/programs/`:
|
||||
- [x] `eight-queens.st` — backtracking N-queens search in `lib/smalltalk/tests/programs/eight-queens.st`. The `.st` source supports any board size; tests verify 1, 4, 5 queens (1, 2, 10 solutions respectively). 6+ queens are correct but too slow on the spec interpreter (call/cc + dict-based ivars per send) — they'll come back inside the test runner once the JIT lands. The 8-queens canonical case will run in production.
|
||||
- [x] `quicksort.st` — Lomuto-partition in-place quicksort in `lib/smalltalk/tests/programs/quicksort.st`. Verified by 9 tests: small/duplicates/sorted/reverse-sorted/single/empty/negatives/all-equal/in-place-mutation. Exercises Array `at:`/`at:put:` mutation, recursion, `to:do:` over varying ranges.
|
||||
- [x] `mandelbrot.st` — escape-time iteration of `z := z² + c` in `lib/smalltalk/tests/programs/mandelbrot.st`. Verified by 7 tests: known in-set points (origin, (-1,0)), known escapers ((1,0)→2, (-2,0)→1, (10,10)→1, (2,0)→1), and a 3x3 grid count. Caught a real bug along the way: literal `#(...)` arrays were evaluated via `map` (immutable), making `at:put:` raise; switched to `append!` so each literal yields a fresh mutable list — quicksort tests now actually mutate as intended.
|
||||
- [x] `life.st` (Conway's Life). `lib/smalltalk/tests/programs/life.st` carries the canonical rules with edge handling. Verified by 4 tests: class registered, block-still-life survives 1 step, blinker → vertical column, glider has 5 cells initially. Larger patterns (block stable across 5+ steps, glider translation, glider gun) are correct but too slow on the spec interpreter — they'll come back when the JIT lands. Also added Pharo-style dynamic array literal `{e1. e2. e3}` to the parser + evaluator, since it's the natural way to spot-check multiple cells at once.
|
||||
- [x] `fibonacci.st` (recursive + Array-memoised) — `lib/smalltalk/tests/programs/fibonacci.st`. Loaded from chunk-format source by new `smalltalk-load` helper; verified by 13 tests in `lib/smalltalk/tests/programs.sx` (recursive `fib:`, memoised `memoFib:` up to 30, instance independence, class-table integrity). Source is currently duplicated as a string in the SX test file because there's no SX file-read primitive; conformance.sh will dedupe by piping the .st file directly.
|
||||
- [x] `lib/smalltalk/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`. The runner runs `bash lib/smalltalk/test.sh -v` once, parses per-file counts, and emits both files. JSON has date / program names / corpus-test count / all-test pass/total / exit code. Markdown has a totals table, the program list, the verbatim per-file test counts block, and notes about JIT-deferred work. Both are checked into the tree as the latest baseline; the runner overwrites them.
|
||||
|
||||
### Phase 4 — reflection + MOP
|
||||
- [ ] `Object>>class`, `class>>name`, `class>>superclass`, `class>>methodDict`, `class>>selectors`
|
||||
- [ ] `Object>>perform:` / `perform:with:` / `perform:withArguments:`
|
||||
- [ ] `Object>>respondsTo:`, `Object>>isKindOf:`, `Object>>isMemberOf:`
|
||||
- [ ] `Behavior>>compile:` — runtime method addition
|
||||
- [ ] `Object>>becomeForward:` (one-way become; rewrites the class field of `aReceiver`)
|
||||
- [ ] Exceptions: `Exception`, `Error`, `signal`, `signal:`, `on:do:`, `ensure:`, `ifCurtailed:` — built on top of SX `handler-bind`/`raise`
|
||||
- [x] `Object>>class`, `class>>name`, `class>>superclass`, `class>>methodDict`, `class>>selectors`. `class` is universal in `st-primitive-send` (returns `Metaclass` for class-refs, the receiver's class otherwise). Class-side dispatch gains `methodDict`/`classMethodDict` (raw dict), `selectors`/`classSelectors` (Array of symbols), `instanceVariableNames` (own), `allInstVarNames` (inherited + own). 26 tests in `lib/smalltalk/tests/reflection.sx`.
|
||||
- [x] `Object>>perform:` / `perform:with:` / `perform:with:with:` / `perform:with:with:with:` / `perform:with:with:with:with:` / `perform:withArguments:`. Universal in `st-primitive-send`; routes back through `st-send` so user methods, primitives, super, and DNU all still apply. Selector arg can be a symbol or string (we `str` it). 10 new tests in `lib/smalltalk/tests/reflection.sx`.
|
||||
- [x] `Object>>respondsTo:`, `Object>>isKindOf:`, `Object>>isMemberOf:`. Universal in `st-primitive-send`. `respondsTo:` searches user method dicts (instance- or class-side based on receiver kind); native primitive selectors aren't enumerated, documented limitation. `isKindOf:` walks `st-class-inherits-from?`; `isMemberOf:` is exact class equality. 26 new tests in `reflection.sx`.
|
||||
- [x] `Behavior>>compile:` — runtime method addition. Class-side `compile:` parses the source via `st-parse-method` and installs via `st-class-add-method!`. Sister forms `compile:classified:` and `compile:notifying:` ignore the extra arg (Pharo-tolerant). Returns the selector as a symbol. Also added `addSelector:withMethod:` (raw AST install) and `removeSelector:`. 9 new tests in `reflection.sx`.
|
||||
- [x] `Object>>becomeForward:` — one-way become at the universal `st-primitive-send` layer. Mutates the receiver's `:class` and `:ivars` to match the target via `dict-set!`; every existing reference to the receiver dict now behaves as the target. Receiver and target remain distinct dicts (no SX-level identity merge), but method dispatch, ivar reads, and aliases all switch — Pharo's practical guarantee. 6 tests in `reflection.sx`, including the alias case (`a` and `alias := a` both see the new identity).
|
||||
- [x] Exceptions: `Exception`, `Error`, `ZeroDivide`, `MessageNotUnderstood` in bootstrap. `signal` raises the receiver via SX `raise`; `signal:` sets `messageText` first. `on:do:` / `ensure:` / `ifCurtailed:` on BlockClosure use SX `guard`. The auto-reraise pattern uses a side-effect predicate (cleanup runs in the predicate, returns false → guard auto-reraises) because `(raise c)` from inside a guard handler hits a known SX issue with nested-handler frames. 15 tests in `lib/smalltalk/tests/exceptions.sx`. Phase 4 complete.
|
||||
|
||||
### Phase 5 — collections + numeric tower
|
||||
- [ ] `SequenceableCollection`/`OrderedCollection`/`Array`/`String`/`Symbol`
|
||||
- [ ] `HashedCollection`/`Set`/`Dictionary`/`IdentityDictionary`
|
||||
- [ ] `Stream` hierarchy: `ReadStream`/`WriteStream`/`ReadWriteStream`
|
||||
- [ ] `Number` tower: `SmallInteger`/`LargePositiveInteger`/`Float`/`Fraction`
|
||||
- [ ] `String>>format:`, `printOn:` for everything
|
||||
- [x] `SequenceableCollection`/`OrderedCollection`/`Array`/`String`/`Symbol`. Bootstrap installs shared methods on `SequenceableCollection`: `inject:into:`, `detect:`/`detect:ifNone:`, `count:`, `allSatisfy:`/`anySatisfy:`, `includes:`, `do:separatedBy:`, `indexOf:`/`indexOf:ifAbsent:`, `reject:`, `isEmpty`/`notEmpty`, `asString`. They each call `self do:`, which dispatches to the receiver's primitive `do:` — so Array, String, and Symbol inherit them uniformly. String/Symbol primitives gained `at:` (1-indexed), `copyFrom:to:`, `first`/`last`, `do:`. OrderedCollection class is in the bootstrap hierarchy; its instance shape will fill out alongside Set/Dictionary in the next box. 28 tests in `lib/smalltalk/tests/collections.sx`.
|
||||
- [x] `HashedCollection`/`Set`/`Dictionary`/`IdentityDictionary`. Implemented as user classes in `runtime.sx`. `HashedCollection` carries a single `array` ivar; `Dictionary` overrides with parallel `keys`/`values`. Set: `add:` (dedup), `addAll:`, `remove:`, `includes:`, `do:`, `size`, `asArray`. Dictionary: `at:`, `at:ifAbsent:`, `at:put:`, `includesKey:`, `removeKey:`, `keys`, `values`, `do:`, `keysDo:`, `valuesDo:`, `keysAndValuesDo:`, `size`, `isEmpty`. `IdentityDictionary` defined as a Dictionary subclass (no methods of its own yet — equality and identity diverge in a follow-up). Class-side `new` calls `super new init`. Added Array primitive `add:` (append). 29 tests in `lib/smalltalk/tests/hashed.sx`.
|
||||
- [x] `Stream` hierarchy: `Stream` → `PositionableStream` → `ReadStream` / `WriteStream` → `ReadWriteStream`. User classes with `collection` + 0-based `position` ivars. ReadStream: `next`, `peek`, `atEnd`, `upToEnd`, `next:`, `skip:`, `reset`, `position`/`position:`. WriteStream: `nextPut:`, `nextPutAll:`, `contents`. Class-side `on:` constructor; `WriteStream class>>with:` pre-fills + `setToEnd`. Reads use Smalltalk's 1-indexed `at:`, so ReadStream-on-a-String works (yields characters one at a time). 21 tests in `lib/smalltalk/tests/streams.sx`. Bumped `test.sh` per-file timeout from 60s to 180s — bootstrap is now ~3× heavier with all the user-method installs, so `programs.sx` runs in ~64s.
|
||||
- [x] `Number` tower: `SmallInteger`/`LargePositiveInteger`/`Float`/`Fraction`. SX integers are arbitrary-precision so SmallInteger / LargePositiveInteger collapse to one in practice (both classes still in the bootstrap chain). Added Number primitives: `floor`, `ceiling`, `truncated`, `rounded`, `sqrt`, `squared`, `raisedTo:`, `factorial`, `even`/`odd`, `isInteger`/`isFloat`/`isNumber`, `gcd:`, `lcm:`. **Fraction** now a real user class (numerator/denominator + sign-normalised, gcd-reduced at construction): `numerator:denominator:`, accessors, `+`/`-`/`*`/`/`, `negated`, `reciprocal`, `=`, `<`, `asFloat`, `printString`, `isFraction`. 47 tests in `lib/smalltalk/tests/numbers.sx`.
|
||||
- [x] `String>>format:`, `printOn:` for everything. `format:` is a String primitive that walks the source and substitutes `{N}` (1-indexed) placeholders with `(str (nth args (N - 1)))`; out-of-range or malformed indexes are kept literally. `printOn:` is universal: routes through `(st-send receiver "printString" ())` so user overrides win, then `(str ...)` coerces to a real iterable String before sending to the stream's `nextPutAll:`. `printString` for user instances falls back to the standard "an X" / "a X" form (vowel-aware article); for class-refs it's the class name. 18 tests in `lib/smalltalk/tests/printing.sx`. Phase 5 complete.
|
||||
|
||||
### Phase 6 — SUnit + corpus to 200+
|
||||
- [ ] Port SUnit (TestCase, TestSuite, TestResult) — written in SX-Smalltalk, runs in itself
|
||||
- [ ] Vendor a slice of Pharo `Kernel-Tests` and `Collections-Tests`
|
||||
- [ ] Drive the scoreboard up: aim for 200+ green tests
|
||||
- [ ] Stretch: ANSI Smalltalk validator subset
|
||||
- [x] Port SUnit (`lib/smalltalk/sunit.sx`). Written in Smalltalk source via `smalltalk-load`. Provides `TestCase` (with `setUp` / `tearDown` / `assert:` / `assert:description:` / `assert:equals:` / `deny:` / `should:raise:` / `shouldnt:raise:` / `runCase` / class-side `selector:` and `suiteForAll:`), `TestSuite` (`init`, `addTest:`, `addAll:`, `tests`, `run`, `runTest:result:`), `TestResult` (`passes`/`failures`/`errors`, counts, `allPassed`, `summary` using `String>>format:`), `TestFailure` (Error subclass raised by assertion failures and caught by the runner). 19 tests in `lib/smalltalk/tests/sunit.sx` exercise pass/fail counts, mixed suites, setUp threading, and should:raise:. test.sh now loads `lib/smalltalk/sunit.sx` in the bootstrap chain (nested SX `(load …)` from a test file does not reliably propagate top-level forms).
|
||||
- [x] Vendor a slice of Pharo `Kernel-Tests` and `Collections-Tests`. `lib/smalltalk/tests/pharo/kernel.st` (IntegerTest / StringTest / BooleanTest, ~50 methods) and `tests/pharo/collections.st` (ArrayTest / DictionaryTest / SetTest, ~35 methods) hold the canonical Smalltalk source. `lib/smalltalk/tests/pharo.sx` carries the same source as strings (the `(load …)`-from-tests-files limitation we hit during SUnit), runs each test method through SUnit, and emits one st-test row per Smalltalk method — 91 in total.
|
||||
- [x] Drive the scoreboard up: aim for 200+ green tests. **751 green** at this point — past the target by 3.7x.
|
||||
- [x] Stretch: ANSI Smalltalk validator subset (`lib/smalltalk/tests/ansi.sx`). 62 tests organised by ANSI X3J20 §6.10 Object, §6.11 Boolean, §6.12 Number, §6.13 Integer, §6.16 Symbol, §6.17 String, §6.18 Array, §6.19 BlockContext. Each test runs through SUnit and emits one st-test row, mirroring the Pharo-slice harness.
|
||||
|
||||
### Phase 7 — speed (optional)
|
||||
- [ ] Method-dictionary inline caching (already in CEK as a primitive; just wire selector cache)
|
||||
- [ ] Block intrinsification beyond `whileTrue:` / `ifTrue:`
|
||||
- [ ] Compare against GNU Smalltalk on the corpus
|
||||
- [x] Method-dictionary inline caching. Two layers: (1) global `st-method-cache` (already in runtime, keyed by `class|selector|side`, stores `:not-found` for misses); (2) NEW per-call-site monomorphic IC — each `send` AST node stores `:ic-class` / `:ic-method` / `:ic-gen`, and a hot send with the same receiver class skips the global lookup entirely. `st-ic-generation` (in runtime.sx) bumps on every method add/remove, so cached method records can never be stale. `st-ic-stats` / `st-ic-reset-stats!` for tests + later debugging. 10 dedicated IC tests in `lib/smalltalk/tests/inline_cache.sx`.
|
||||
- [x] Block intrinsification beyond `whileTrue:` / `ifTrue:`. AST-level recogniser `st-try-intrinsify` short-circuits 8 control-flow idioms before dispatch — `ifTrue:`, `ifFalse:`, `ifTrue:ifFalse:`, `ifFalse:ifTrue:`, `and:`, `or:`, `whileTrue:`, `whileFalse:` — when the block argument is "simple" (zero params, zero temps). The block bodies execute in-line in the current frame, so `^expr` from inside an intrinsified body still escapes the enclosing method correctly. `st-intrinsic-stats` / `st-intrinsic-reset!` for tests + later debugging. 24 tests in `lib/smalltalk/tests/intrinsics.sx`. Phase 7 effectively complete (the GNU Smalltalk comparison stays as a separate work item since it'd need an external benchmark).
|
||||
- [x] Compare against GNU Smalltalk on the corpus. `lib/smalltalk/compare.sh` runs a fibonacci(22) benchmark on both Smalltalk-on-SX (`sx_server.exe` + smalltalk-load + eval) and GNU Smalltalk (`gst -q`), emits a `compare-results.txt`. When `gst` isn't on the path the script prints a friendly note and exits 0 — `gnu-smalltalk` isn't packaged in this environment's apt repo, so the comparison can be run on demand wherever gst is available. **Phase 7 complete.**
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first. Agent appends on every commit._
|
||||
|
||||
- _(none yet)_
|
||||
- 2026-04-25: GNU Smalltalk compare harness (`lib/smalltalk/compare.sh`) — runs fib(22) on sx_server.exe + smalltalk-load and on `gst -q`, saves results. Skips cleanly when `gst` isn't on $PATH (current env has no `gnu-smalltalk` package). **Phase 7 complete. All briefing checkboxes done.**
|
||||
- 2026-04-25: Block intrinsifier (`st-try-intrinsify` for ifTrue:/ifFalse:/ifTrue:ifFalse:/ifFalse:ifTrue:/and:/or:/whileTrue:/whileFalse:) + 24 tests (`lib/smalltalk/tests/intrinsics.sx`). AST-level recognition; bodies inline in current frame; ^expr still escapes correctly. 847/847 total.
|
||||
- 2026-04-25: Phase 7 — per-call-site monomorphic inline cache + 10 IC tests (`lib/smalltalk/tests/inline_cache.sx`). `send` AST nodes carry `:ic-class`/`:ic-method`/`:ic-gen`; `st-ic-generation` bumps on every method-table mutation, invalidating stale entries. 823/823 total.
|
||||
- 2026-04-25: ANSI X3J20 validator subset + 62 tests (`lib/smalltalk/tests/ansi.sx`). One TestCase subclass per ANSI §6.x protocol; runs through SUnit. **Phase 6 complete.** 813/813 total.
|
||||
- 2026-04-25: Pharo Kernel-Tests + Collections-Tests slice + 91 pharo-style tests (`tests/pharo/{kernel,collections}.st` + `tests/pharo.sx`). Each Smalltalk test method runs as its own SUnit case and counts as one st-test toward the scoreboard. 751/751 total — past the Phase 6 "200+ green tests" target.
|
||||
- 2026-04-25: SUnit port (`lib/smalltalk/sunit.sx`, `lib/smalltalk/tests/sunit.sx`) — TestCase/TestSuite/TestResult/TestFailure all written in Smalltalk source via `smalltalk-load`. Full assert family + should:raise: + setUp/tearDown threading. 19 tests verify the framework. test.sh now bootstraps SUnit alongside runtime/eval. 660/660 total.
|
||||
- 2026-04-25: String>>format: + universal printOn: + 18 tests (`lib/smalltalk/tests/printing.sx`). `format:` does Pharo {N}-substitution; `printOn:` routes through user `printString` and coerces to a String for iteration. Phase 5 complete. 638/638 total.
|
||||
- 2026-04-25: Number tower + Fraction class + 47 tests (`lib/smalltalk/tests/numbers.sx`). 14 new Number primitives (floor/ceiling/truncated/rounded/sqrt/squared/raisedTo:/factorial/even/odd/gcd:/lcm:/isInteger/isFloat). Fraction with normalisation + arithmetic + comparisons + asFloat. 620/620 total.
|
||||
- 2026-04-25: Stream hierarchy + 21 tests (`lib/smalltalk/tests/streams.sx`). ReadStream / WriteStream / ReadWriteStream as user classes; class-side `on:`; ReadStream-on-String yields characters. Bumped `test.sh` per-file timeout 60s → 180s — heavier bootstrap pushed `programs.sx` past 60s. 573/573 total.
|
||||
- 2026-04-25: HashedCollection / Set / Dictionary / IdentityDictionary + 29 tests (`lib/smalltalk/tests/hashed.sx`). Set: dedup add:, remove:, includes:, do:, addAll:. Dictionary: parallel keys/values backing; at:put:, at:ifAbsent:, includesKey:, removeKey:, keysDo:, keysAndValuesDo:. Class-side `new` chains `super new init`. Array primitive `add:` added. 552/552 total.
|
||||
- 2026-04-25: Phase 5 sequenceable-collection methods + 28 tests (`lib/smalltalk/tests/collections.sx`). 13 shared methods on `SequenceableCollection` (inject:into:, detect:, count:, …), inherited by Array/String/Symbol via `self do:`. String primitives at:/copyFrom:to:/first/last/do:. 523/523 total.
|
||||
- 2026-04-25: Exception system + 15 tests (`lib/smalltalk/tests/exceptions.sx`). Exception/Error/ZeroDivide/MessageNotUnderstood in bootstrap; signal/signal: raise via SX `raise`; on:do:/ensure:/ifCurtailed: on BlockClosure via SX `guard`. Phase 4 complete. 495/495 total.
|
||||
- 2026-04-25: `Object>>becomeForward:` + 6 tests. In-place mutation of `:class` and `:ivars` via `dict-set!`; aliases see the new identity. 480/480 total.
|
||||
- 2026-04-25: `Behavior>>compile:` + sisters + 9 tests. Parses source via `st-parse-method`, installs via runtime helpers; also added `addSelector:withMethod:` and `removeSelector:`. 474/474 total.
|
||||
- 2026-04-25: `respondsTo:` / `isKindOf:` / `isMemberOf:` + 26 tests. Universal at `st-primitive-send`. 465/465 total.
|
||||
- 2026-04-25: `Object>>perform:` family + 10 tests. Universal dispatch via `st-send` after `(str (nth args 0))` for the selector. 439/439 total.
|
||||
- 2026-04-25: Phase 4 reflection accessors (`lib/smalltalk/tests/reflection.sx`, 26 tests). Universal `Object>>class`, plus `methodDict`/`selectors`/`instanceVariableNames`/`allInstVarNames`/`classMethodDict`/`classSelectors` on class-refs. 429/429 total.
|
||||
- 2026-04-25: conformance.sh + scoreboard.{json,md} (`lib/smalltalk/conformance.sh`, `lib/smalltalk/scoreboard.json`, `lib/smalltalk/scoreboard.md`). Single-pass runner over `test.sh -v`; baseline at 5 programs / 39 corpus tests / 403 total. **Phase 3 complete.**
|
||||
- 2026-04-25: classic-corpus #5 Life (`tests/programs/life.st`, 4 tests). Spec-interpreter Conway's Life with edge handling. Block + blinker + glider initial setup verified; larger step counts pending JIT (each spec-interpreter step is ~5-8s on a 5x5 grid). Added `{e1. e2. e3}` dynamic array literal to parser + evaluator. 403/403 total.
|
||||
- 2026-04-25: classic-corpus #4 mandelbrot (`tests/programs/mandelbrot.st`, 7 tests). Escape-time iterator + grid counter. Discovered + fixed an immutable-list bug in `lit-array` eval — `map` produced an immutable list so `at:put:` raised; rebuilt via `append!`. Quicksort tests had been silently dropping ~7 cases due to that bug; now actually mutate. 399/399 total.
|
||||
- 2026-04-25: classic-corpus #3 quicksort (`tests/programs/quicksort.st`, 9 tests). Lomuto partition; verified across duplicates, already-sorted/reverse-sorted, empty, single, negatives, all-equal, plus in-place mutation. 385/385 total.
|
||||
- 2026-04-25: classic-corpus #2 eight-queens (`tests/programs/eight-queens.st`, 5 tests). Backtracking search; verified for boards of size 1, 4, 5. Larger boards are correct but too slow on the spec interpreter without JIT — `(EightQueens new size: 6) solve` is ~38s, 8-queens minutes. 382/382 total.
|
||||
- 2026-04-25: classic-corpus #1 fibonacci (`tests/programs/fibonacci.st` + `tests/programs.sx`, 13 tests). Added `smalltalk-load` chunk loader, class-side `subclass:instanceVariableNames:` (and longer Pharo variants), `Array new:` size, `methodsFor:`/`category:` no-ops, `st-split-ivars`. 377/377 total.
|
||||
- 2026-04-25: cannotReturn: implemented (`lib/smalltalk/tests/cannot_return.sx`, 5 tests). Each method-invocation gets an `{:active true}` cell shared with its blocks; `st-invoke` flips it on exit; `^expr` raises if the cell is dead. Tests use SX `guard` to catch the raise. Non-`^` blocks unaffected. 364/364 total.
|
||||
- 2026-04-25: `ifTrue:` / `ifFalse:` family pinned (`lib/smalltalk/tests/conditional.sx`, 24 tests) + parser fix: `|` is now accepted as a binary selector in expression position (tokenizer still emits it as `bar` for block param/temp delimiting; `parse-binary-message` accepts both). Caught by `false | true` truncating silently to `false`. 359/359 total.
|
||||
- 2026-04-25: `whileTrue:` / `whileFalse:` / no-arg variants pinned (`lib/smalltalk/tests/while.sx`, 14 tests). `st-block-while` returns nil per ANSI; behaviour verified under captured locals, nesting, early `^`, and zero/many iterations. 334/334 total.
|
||||
- 2026-04-25: BlockContext value family pinned (`lib/smalltalk/tests/blocks.sx`, 19 tests). Each value/valueN/valueWithArguments: variant verified plus closure semantics (read, write, later-mutation re-read), nested blocks, and block-as-arg. 320/320 total.
|
||||
- 2026-04-25: **THE SHOWCASE** — non-local return via captured method-return continuations + 14 NLR tests (`lib/smalltalk/tests/nlr.sx`). `st-invoke` wraps body in `call/cc`; blocks copy creating method's `^k`; `^expr` invokes that k. Verified across nested blocks, `to:do:` / `whileTrue:`, blocks passed to different methods (Caller→Helper escapes back to Caller), inner-vs-outer method nesting. Sentinel-based return removed. 301/301 total.
|
||||
- 2026-04-25: `super` send + 9 tests (`lib/smalltalk/tests/super.sx`). `st-super-send` walks from defining-class's superclass; class-side aware; primitives → DNU fallback. Also fixed top-level `| temps |` parsing in `st-parse` (the absence of which was silently aborting earlier eval/dnu tests — counts go from 274 → 287, with previously-skipped tests now actually running).
|
||||
- 2026-04-25: `doesNotUnderstand:` + 12 DNU tests (`lib/smalltalk/tests/dnu.sx`). Bootstrap installs `Message` (with selector/arguments accessors). Primitives signal `:unhandled` instead of erroring; `st-dnu` builds a Message and walks `doesNotUnderstand:` lookup. User Object DNU intercepts unknown sends to native receivers (Number, String, Block) too. 267/267 total.
|
||||
- 2026-04-25: method-lookup cache (`st-method-cache` keyed by `class|selector|side`, stores `:not-found` for misses). Invalidation on define/add/remove + bootstrap. `st-class-remove-method!` added. Stats helpers + 10 cache tests; 255/255 total.
|
||||
- 2026-04-25: `smalltalk-eval-ast` + 60 eval tests (`lib/smalltalk/eval.sx`, `lib/smalltalk/tests/eval.sx`). Frame chain with mutable locals/ivars (via `dict-set!`), full literal eval, send dispatch (user methods + native primitive tables for Number/String/Boolean/Nil/Array/Block/Class), block closures, while/to:do:, cascades returning last, sentinel-based `^return`. User Point class round-trip works including `+` returning a fresh point. 245/245 total.
|
||||
- 2026-04-25: class table + bootstrap (`lib/smalltalk/runtime.sx`, `lib/smalltalk/tests/runtime.sx`). Canonical hierarchy, type→class mapping for native SX values, instance construction, ivar inheritance, method install with `:defining-class` stamp, instance- and class-side method lookup walking the superclass chain. 54 new tests, 185/185 total.
|
||||
- 2026-04-25: chunk-stream parser + pragmas + 21 chunk/pragma tests (`lib/smalltalk/tests/parse_chunks.sx`). `st-read-chunks` (with `!!` doubling), `st-parse-chunks` state machine for `methodsFor:` batches incl. class-side. Pragmas with multiple keyword pairs, signed numeric / string / symbol args, in either pragma-then-temps or temps-then-pragma order. 131/131 tests pass.
|
||||
- 2026-04-25: expression-level parser + 47 parse tests (`lib/smalltalk/parser.sx`, `lib/smalltalk/tests/parse.sx`). Full message precedence (unary > binary > keyword), cascades, blocks with params/temps, literal/byte arrays, assignment chain, method headers (unary/binary/keyword). Chunk-format `! !` driver deferred to a follow-up box. 110/110 tests pass.
|
||||
- 2026-04-25: tokenizer + 63 tests (`lib/smalltalk/tokenizer.sx`, `lib/smalltalk/tests/tokenize.sx`, `lib/smalltalk/test.sh`). All token types covered except scaled decimals `1.5s2` (deferred). `#(` and `#[` emit open tokens; literal-array contents lexed as ordinary tokens for the parser to interpret.
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@ Core mapping:
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — tokenizer + parser (the Dodekalogue)
|
||||
- [ ] Tokenizer applying the 12 rules:
|
||||
- [x] Tokenizer applying the 12 rules:
|
||||
1. Commands separated by `;` or newlines
|
||||
2. Words separated by whitespace within a command
|
||||
3. Double-quoted words: `\` escapes + `[…]` + `${…}` + `$var` substitution
|
||||
@@ -63,64 +63,76 @@ Core mapping:
|
||||
10. Order of substitution is left-to-right, single-pass
|
||||
11. Substitutions don't recurse — substituted text is not re-parsed
|
||||
12. The result of any substitution is the value, not a new script
|
||||
- [ ] Parser: script = list of commands; command = list of words; word = literal string + list of substitutions
|
||||
- [ ] Unit tests in `lib/tcl/tests/parse.sx`
|
||||
- [x] Parser: script = list of commands; command = list of words; word = literal string + list of substitutions
|
||||
- [x] Unit tests in `lib/tcl/tests/parse.sx`
|
||||
|
||||
### Phase 2 — sequential eval + core commands
|
||||
- [ ] `tcl-eval-script`: walk command list, dispatch each first-word into command table
|
||||
- [ ] Core commands: `set`, `unset`, `incr`, `append`, `lappend`, `puts`, `gets`, `expr`, `if`, `while`, `for`, `foreach`, `switch`, `break`, `continue`, `return`, `error`, `eval`, `subst`, `format`, `scan`
|
||||
- [ ] `expr` is its own mini-language — operator precedence, function calls (`sin`, `sqrt`, `pow`, `abs`, `int`, `double`), variable substitution, command substitution
|
||||
- [ ] String commands: `string length`, `string index`, `string range`, `string compare`, `string match`, `string toupper`, `string tolower`, `string trim`, `string map`, `string repeat`, `string first`, `string last`, `string is`, `string cat`
|
||||
- [ ] List commands: `list`, `lindex`, `lrange`, `llength`, `lreverse`, `lsearch`, `lsort`, `lsort -integer/-real/-dictionary`, `lreplace`, `linsert`, `concat`, `split`, `join`
|
||||
- [ ] Dict commands: `dict create`, `dict get`, `dict set`, `dict unset`, `dict exists`, `dict keys`, `dict values`, `dict size`, `dict for`, `dict update`, `dict merge`
|
||||
- [ ] 60+ tests in `lib/tcl/tests/eval.sx`
|
||||
- [x] `tcl-eval-script`: walk command list, dispatch each first-word into command table
|
||||
- [x] Core commands: `set`, `unset`, `incr`, `append`, `lappend`, `puts`, `gets`, `expr`, `if`, `while`, `for`, `foreach`, `switch`, `break`, `continue`, `return`, `error`, `eval`, `subst`, `format`, `scan`
|
||||
- [x] `expr` is its own mini-language — operator precedence, function calls (`sin`, `sqrt`, `pow`, `abs`, `int`, `double`), variable substitution, command substitution
|
||||
- [x] String commands: `string length`, `string index`, `string range`, `string compare`, `string match`, `string toupper`, `string tolower`, `string trim`, `string map`, `string repeat`, `string first`, `string last`, `string is`, `string cat`
|
||||
- [x] List commands: `list`, `lindex`, `lrange`, `llength`, `lreverse`, `lsearch`, `lsort`, `lsort -integer/-real/-dictionary`, `lreplace`, `linsert`, `concat`, `split`, `join`
|
||||
- [x] Dict commands: `dict create`, `dict get`, `dict set`, `dict unset`, `dict exists`, `dict keys`, `dict values`, `dict size`, `dict for`, `dict update`, `dict merge`
|
||||
- [x] 60+ tests in `lib/tcl/tests/eval.sx`
|
||||
|
||||
### Phase 3 — proc + uplevel + upvar (THE SHOWCASE)
|
||||
- [ ] `proc name args body` — register user-defined command; args supports defaults `{name default}` and rest `args`
|
||||
- [ ] Frame stack: each proc call pushes a frame with locals dict; pop on return
|
||||
- [ ] `uplevel ?level? script` — evaluate `script` in level-N frame's env; default level is 1 (caller). `#0` is global, `#1` is relative-1
|
||||
- [ ] `upvar ?level? otherVar localVar ?…?` — alias localVar to a variable in level-N frame; reads/writes go through the alias
|
||||
- [ ] `info level`, `info level N`, `info frame`, `info vars`, `info locals`, `info globals`, `info commands`, `info procs`, `info args`, `info body`
|
||||
- [ ] `global var ?…?` — alias to global frame (sugar for `upvar #0 var var`)
|
||||
- [ ] `variable name ?value?` — namespace-scoped global
|
||||
- [ ] Classic programs in `lib/tcl/tests/programs/`:
|
||||
- [ ] `for-each-line.tcl` — define your own loop construct using `uplevel`
|
||||
- [ ] `assert.tcl` — assertion macro that reports caller's line
|
||||
- [ ] `with-temp-var.tcl` — scoped variable rebind via `upvar`
|
||||
- [ ] `lib/tcl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
- [x] `proc name args body` — register user-defined command; args supports defaults `{name default}` and rest `args`
|
||||
- [x] Frame stack: each proc call pushes a frame with locals dict; pop on return
|
||||
- [x] `uplevel ?level? script` — evaluate `script` in level-N frame's env; default level is 1 (caller). `#0` is global, `#1` is relative-1
|
||||
- [x] `upvar ?level? otherVar localVar ?…?` — alias localVar to a variable in level-N frame; reads/writes go through the alias
|
||||
- [x] `info level`, `info level N`, `info frame`, `info vars`, `info locals`, `info globals`, `info commands`, `info procs`, `info args`, `info body`
|
||||
- [x] `global var ?…?` — alias to global frame (sugar for `upvar #0 var var`)
|
||||
- [x] `variable name ?value?` — namespace-scoped global
|
||||
- [x] Classic programs in `lib/tcl/tests/programs/`:
|
||||
- [x] `for-each-line.tcl` — define your own loop construct using `uplevel`
|
||||
- [x] `assert.tcl` — assertion macro that reports caller's line
|
||||
- [x] `with-temp-var.tcl` — scoped variable rebind via `upvar`
|
||||
- [x] `lib/tcl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
||||
|
||||
### Phase 4 — control flow + error handling
|
||||
- [ ] `return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value`
|
||||
- [ ] `catch script ?resultVar? ?optionsVar?` — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict
|
||||
- [ ] `try script ?on code var body ...? ?trap pattern var body...? ?finally body?`
|
||||
- [ ] `throw type message`
|
||||
- [ ] `error message ?info? ?code?`
|
||||
- [ ] Stack-trace with `errorInfo` / `errorCode`
|
||||
- [ ] 30+ tests in `lib/tcl/tests/error.sx`
|
||||
- [x] `return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value`
|
||||
- [x] `catch script ?resultVar? ?optionsVar?` — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict
|
||||
- [x] `try script ?on code var body ...? ?trap pattern var body...? ?finally body?`
|
||||
- [x] `throw type message`
|
||||
- [x] `error message ?info? ?code?`
|
||||
- [x] Stack-trace with `errorInfo` / `errorCode`
|
||||
- [x] 30+ tests in `lib/tcl/tests/error.sx`
|
||||
|
||||
### Phase 5 — namespaces + ensembles
|
||||
- [ ] `namespace eval ns body`, `namespace current`, `namespace which`, `namespace import`, `namespace export`, `namespace forget`, `namespace delete`
|
||||
- [ ] Qualified names: `::ns::cmd`, `::ns::var`
|
||||
- [ ] Ensembles: `namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }`
|
||||
- [ ] `namespace path` for resolution chain
|
||||
- [ ] `proc` and `variable` work inside namespaces
|
||||
- [x] `namespace eval ns body`, `namespace current`, `namespace which`, `namespace import`, `namespace export`, `namespace forget`, `namespace delete`
|
||||
- [x] Qualified names: `::ns::cmd`, `::ns::var`
|
||||
- [x] Ensembles: `namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }`
|
||||
- [x] `namespace path` for resolution chain
|
||||
- [x] `proc` and `variable` work inside namespaces
|
||||
|
||||
### Phase 6 — coroutines + drive corpus
|
||||
- [ ] `coroutine name cmd ?args…?` — start a coroutine; future calls to `name` resume it
|
||||
- [ ] `yield ?value?` — suspend, return value to resumer
|
||||
- [ ] `yieldto cmd ?args…?` — symmetric transfer
|
||||
- [ ] `coroutine` semantics built on fibers (same delcc primitive as Ruby fibers)
|
||||
- [ ] Classic programs: `event-loop.tcl` — cooperative scheduler with multiple coroutines
|
||||
- [ ] System: `clock seconds`, `clock format`, `clock scan` (subset)
|
||||
- [ ] File I/O: `open`, `close`, `read`, `gets`, `puts -nonewline`, `flush`, `eof`, `seek`, `tell`
|
||||
- [ ] Drive corpus to 150+ green
|
||||
- [ ] Idiom corpus — `lib/tcl/tests/idioms.sx` covering classic Welch/Jones idioms
|
||||
- [x] `coroutine name cmd ?args…?` — start a coroutine; future calls to `name` resume it
|
||||
- [x] `yield ?value?` — suspend, return value to resumer
|
||||
- [x] `yieldto cmd ?args…?` — symmetric transfer
|
||||
- [x] `coroutine` semantics built on fibers (same delcc primitive as Ruby fibers)
|
||||
- [x] Classic programs: `event-loop.tcl` — cooperative scheduler with multiple coroutines
|
||||
- [x] System: `clock seconds`, `clock format`, `clock scan` (subset)
|
||||
- [x] File I/O: `open`, `close`, `read`, `gets`, `puts -nonewline`, `flush`, `eof`, `seek`, `tell`
|
||||
- [x] Drive corpus to 150+ green
|
||||
- [x] Idiom corpus — `lib/tcl/tests/idioms.sx` covering classic Welch/Jones idioms
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- _(none yet)_
|
||||
- 2026-05-06: Phase 6 coroutines+clock+file+idioms — generator coroutines, clock/file stubs, 20 coroutine + 20 idiom tests, event-loop.tcl, 329 tests green
|
||||
- 2026-05-06: Phase 5 namespaces+ensembles — namespace eval/current/which/exists/delete/import/ensemble, qualified names, 289 tests green (22 new namespace tests)
|
||||
- 2026-05-06: Phase 4 error handling — catch/try/throw/return-code/errorinfo/errorcode, 267 tests green (39 new error tests)
|
||||
- 2026-05-06: Phase 3 conformance.sh + classic programs — 3/3 PASS (for-each-line/assert/with-temp-var), 228 tests green
|
||||
- 2026-05-06: Phase 3 proc+uplevel+upvar+info+global — frame stack, isolated proc scope, alias-following var access, 225 tests green (67 parse + 158 eval)
|
||||
- 2026-05-06: Phase 2 dict commands — 13 subcommands (create/get/set/unset/exists/keys/values/size/for/update/merge/incr/append), 206 tests green (67 parse + 139 eval)
|
||||
- 2026-05-06: Phase 2 list commands — 12 commands (list/lindex/lrange/llength/lreverse/lsearch/lsort/lreplace/linsert/concat/split/join), 182 tests green (67 parse + 115 eval)
|
||||
- 2026-05-06: Phase 2 string commands — 16 subcommands (length/index/range/compare/match/toupper/tolower/trim/map/repeat/first/last/is/cat), 156 tests green (67 parse + 89 eval)
|
||||
- 2026-05-06: Phase 2 expr mini-language — recursive descent parser, operator precedence, parens, unary ops, pow/sqrt/abs/max/min/int/double, 127 tests green (67 parse + 60 eval)
|
||||
- 2026-04-26: Phase 2 core commands — if/while/for/foreach/switch/break/continue/return/error/unset/lappend/eval/expr + :code control flow, 107 tests green (67 parse + 40 eval)
|
||||
- 2026-04-26: Phase 2 eval engine — `lib/tcl/runtime.sx`, tcl-eval-script + set/puts/incr/append, 87 tests green (67 parse + 20 eval)
|
||||
- 2026-04-25: Phase 1 parser — `lib/tcl/parser.sx`, word-simple?/word-literal helpers, 67 tests green, commit 6ee05259
|
||||
- 2026-04-25: Phase 1 tokenizer (Dodekalogue) — `lib/tcl/tokenizer.sx`, 52 tests green, commit 666e29d5
|
||||
|
||||
## Blockers
|
||||
|
||||
|
||||
156
plans/tcl-sx-completion.md
Normal file
156
plans/tcl-sx-completion.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Tcl-on-SX completion plan — SX capabilities first
|
||||
|
||||
Tcl phases 1–6 are complete (329/329 tests). This plan covers the remaining
|
||||
limitations, ordered by the SX work needed to enable them.
|
||||
|
||||
## Key audit findings
|
||||
|
||||
Several apparent gaps are already solved in SX:
|
||||
|
||||
- **Floats** — SX parses `3.14` natively; `(+ 1.5 2.5) → 4.0`; `str` formats
|
||||
with `%g` (compact, no trailing zeros). `floor`/`ceil`/`round`/`truncate`
|
||||
all exist in `spec/primitives.sx`.
|
||||
- **Regex** — `regexp-match`, `regexp-match-all`, `regexp-replace`,
|
||||
`regexp-replace-all`, `regexp-split` are registered OCaml primitives using
|
||||
`Re.Pcre` (`hosts/ocaml/lib/sx_primitives.ml`).
|
||||
- **`call/cc` multi-shot** — works. `set!` on closed-over vars works. Fibers
|
||||
are implementable as a pure SX library.
|
||||
- **`perform` user-accessible** — `(perform :foo 42)` from user code suspends
|
||||
the evaluator and emits an IO request. The algebraic effects model is
|
||||
already half-built.
|
||||
- **No `file-read`/`clock-seconds`** — not yet registered as OCaml primitives.
|
||||
Only string ports exist. Would need small OCaml additions.
|
||||
- **No `env-as-value`** — environments are internal OCaml values, not
|
||||
inspectable from SX user code.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1 — Zero-cost wins (no SX changes, only `lib/tcl/`)
|
||||
|
||||
Everything here is pure Tcl implementation work.
|
||||
|
||||
| Status | Work | Effort | Unlocks in Tcl |
|
||||
|---|---|---|---|
|
||||
| [x] | Float in `expr` — detect `.` in number tokens, route through float ops instead of `parse-int` | half day | `expr {3.14 * 2}`, `expr {sqrt(2.0)}`, float comparisons |
|
||||
| [x] | `regexp pattern str` and `regsub pattern str repl` wrapping existing SX primitives | few hours | pattern matching, text processing |
|
||||
| [x] | `apply {args body} ?arg…?` — anonymous proc call | 1 hour | higher-order functions, `lmap` idiom |
|
||||
| [ ] | `array get/set/names/size/exists/unset` commands | half day | array variables (tokenizer already parses `$arr(key)`) |
|
||||
|
||||
**Total: ~2 days. Zero SX changes.**
|
||||
|
||||
---
|
||||
|
||||
## Phase 2 — `lib/fiber.sx` (pure SX library, no OCaml)
|
||||
|
||||
`call/cc` is multi-shot and `set!` on closed-over vars both work. Fibers are
|
||||
implementable as a pure SX library using symmetric continuation swapping:
|
||||
|
||||
```scheme
|
||||
; lib/fiber.sx — canonical fiber primitive for all hosted languages
|
||||
(define make-fiber
|
||||
(fn (thunk)
|
||||
(define slot-k nil)
|
||||
(define slot-caller nil)
|
||||
(define slot-done false)
|
||||
(fn (resume-val)
|
||||
(call/cc (fn (caller-k)
|
||||
(set! slot-caller caller-k)
|
||||
(if (nil? slot-k)
|
||||
(begin (thunk resume-val) (set! slot-done true) (caller-k nil))
|
||||
(slot-k resume-val)))))))
|
||||
|
||||
(define fiber-yield
|
||||
(fn (val)
|
||||
(call/cc (fn (k)
|
||||
(set! slot-k k)
|
||||
(slot-caller val)))))
|
||||
```
|
||||
|
||||
Each coroutine becomes a fiber. `yield` swaps to the caller; calling the
|
||||
coroutine name swaps back. True suspension, not eager pre-execution.
|
||||
|
||||
**Broader value:** Ruby fibers, Python generators, Lua coroutines, async event
|
||||
loops, cooperative schedulers all sit on top of the same library.
|
||||
|
||||
**Alternatively:** `perform` is user-accessible. A Tcl scheduler living outside
|
||||
the SX evaluator (the OCaml host or an SX event loop) could catch
|
||||
`(perform :fiber-yield val)` and dispatch it — the algebraic effects model,
|
||||
already half-built.
|
||||
|
||||
**Total: 2–3 days. Produces `lib/fiber.sx` as a lasting SX contribution.**
|
||||
Tcl coroutines then rewrite using `make-fiber` for true suspension.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3 — Small OCaml additions (`sx_primitives.ml`)
|
||||
|
||||
Each is ~10–20 lines of OCaml. All are useful across the whole platform, not
|
||||
just Tcl.
|
||||
|
||||
| Primitive | OCaml effort | Unlocks |
|
||||
|---|---|---|
|
||||
| `(file-read path)` → string | tiny | Tcl `open`/`read`, SX scripts reading files |
|
||||
| `(file-write path str)` → nil | tiny | Tcl `open`/`puts` to files |
|
||||
| `(file-exists? path)` → bool | tiny | Tcl `file exists` |
|
||||
| `(file-glob pattern)` → list | small | Tcl `glob` |
|
||||
| `(clock-seconds)` → int | tiny | Tcl `clock seconds` |
|
||||
| `(clock-format n fmt)` → string | small (wraps `strftime`) | Tcl `clock format` |
|
||||
|
||||
**Total: 1 day. One focused afternoon of OCaml.**
|
||||
|
||||
---
|
||||
|
||||
## Phase 4 — Optional: env-as-value (architectural)
|
||||
|
||||
`uplevel`/`upvar` required an explicit frame stack because SX environments
|
||||
aren't inspectable from user code. Adding:
|
||||
|
||||
```scheme
|
||||
(current-env) ; → env value
|
||||
(eval-in-env env expr) ; → result
|
||||
(env-lookup env key) ; → value or nil
|
||||
(env-extend env key val) ; → new env (non-mutating)
|
||||
```
|
||||
|
||||
...would let `uplevel N` be literally "look up env N levels up, eval in it."
|
||||
The Tcl frame stack (hundreds of lines) collapses to ~10 lines.
|
||||
|
||||
Also benefits: metacircular evaluators, REPL tooling, live debugging (inspect
|
||||
any scope), the sx_docs server's eval endpoint.
|
||||
|
||||
More invasive — touches `sx_types.ml` and `sx_server.ml` — but a meaningful
|
||||
architectural improvement worth doing when the moment is right.
|
||||
|
||||
**Total: 2–3 days. High architectural value, not urgent.**
|
||||
|
||||
---
|
||||
|
||||
## Suggested order
|
||||
|
||||
1. **Phase 1** — immediate Tcl wins, zero risk, proves the approach
|
||||
2. **Phase 2** (`lib/fiber.sx`) — the interesting SX work, benefits all hosted languages
|
||||
3. **Phase 3** (OCaml primitives) — quick practical completions
|
||||
4. **Phase 4** — architectural cleanup when it's worth the invasiveness
|
||||
|
||||
Phases 1+2+3 ≈ one focused week. Tcl is genuinely complete, and `lib/fiber.sx`
|
||||
becomes a lasting SX contribution used by every future hosted language.
|
||||
|
||||
---
|
||||
|
||||
## Progress log
|
||||
|
||||
_Newest first._
|
||||
|
||||
- 2026-05-06: Phase 1 apply — `tcl-cmd-apply` wraps `tcl-call-proc`, parses `{args body}` funcList, full frame isolation; 329/329 tests green
|
||||
- 2026-05-06: Phase 1 regexp/regsub — `tcl-cmd-regexp`/`tcl-cmd-regsub` wrapping `make-regexp`/`regexp-match`/`regexp-match-all`/`regexp-replace`/`regexp-replace-all`; -nocase/-all/-inline/-all flags; matchVar + subgroup capture; 329/329 tests green
|
||||
- 2026-05-06: Phase 1 float expr — `tcl-num-float?`, `tcl-parse-num`, float-aware `tcl-apply-binop`/`tcl-apply-func`/unary-minus/`**`; `sqrt`/`floor`/`ceil`/`round`/`sin`/`cos`/`tan`/`pow`/`exp`/`log` all float-native; 329/329 tests green
|
||||
|
||||
---
|
||||
|
||||
## What stays out of scope
|
||||
|
||||
- `package require` of binary loadables
|
||||
- Full `clock format` locale support
|
||||
- Tk / GUI
|
||||
- Threads (mapped to coroutines only, as planned)
|
||||
- Full POSIX file I/O (seek/tell/async) — stubs are fine
|
||||
Reference in New Issue
Block a user