diff --git a/plans/agent-briefings/apl-loop.md b/plans/agent-briefings/apl-loop.md new file mode 100644 index 00000000..c84c5c2a --- /dev/null +++ b/plans/agent-briefings/apl-loop.md @@ -0,0 +1,81 @@ +# apl-on-sx loop agent (single agent, queue-driven) + +Role: iterates `plans/apl-on-sx.md` forever. Rank-polymorphic primitives + 6 operators on the JIT is the headline showcase — APL is the densest combinator algebra you can put on top of a primitive table. Every program is `array → array` pure pipelines, exactly what the JIT was built for. + +``` +description: apl-on-sx queue loop +subagent_type: general-purpose +run_in_background: true +isolation: worktree +``` + +## Prompt + +You are the sole background agent working `/root/rose-ash/plans/apl-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push. + +## Restart baseline — check before iterating + +1. Read `plans/apl-on-sx.md` — roadmap + Progress log. +2. `ls lib/apl/` — pick up from the most advanced file. +3. If `lib/apl/tests/*.sx` exist, run them. Green before new work. +4. If `lib/apl/scoreboard.md` exists, that's your baseline. + +## The queue + +Phase order per `plans/apl-on-sx.md`: + +- **Phase 1** — tokenizer + parser. Unicode glyphs, `¯` for negative, strands (juxtaposition), right-to-left, valence resolution by syntactic position +- **Phase 2** — array model + scalar primitives. `make-array {shape, ravel}`, scalar promotion, broadcast for `+ - × ÷ ⌈ ⌊ * ⍟ | ! ○`, comparison, logical, `⍳`, `⎕IO` +- **Phase 3** — structural primitives + indexing. `⍴ , ⍉ ↑ ↓ ⌽ ⊖ ⌷ ⍋ ⍒ ⊂ ⊃ ∊` +- **Phase 4** — **THE SHOWCASE**: operators. `f/` (reduce), `f¨` (each), `∘.f` (outer), `f.g` (inner), `f⍨` (commute), `f∘g` (compose), `f⍣n` (power), `f⍤k` (rank), `@` (at) +- **Phase 5** — dfns + tradfns + control flow. `{⍺+⍵}`, `∇` recurse, `⍺←default`, tradfn header, `:If/:While/:For/:Select` +- **Phase 6** — classic programs (life, mandelbrot, primes, n-queens, quicksort) + idiom corpus + drive to 100+ + +Within a phase, pick the checkbox that unlocks the most tests per effort. + +Every iteration: implement → test → commit → tick `[ ]` → Progress log → next. + +## Ground rules (hard) + +- **Scope:** only `lib/apl/**` and `plans/apl-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib//` dirs, `lib/stdlib.sx`, or `lib/` root. APL primitives go in `lib/apl/runtime.sx`. +- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop. +- **Shared-file issues** → plan's Blockers with minimal repro. +- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits. +- **Unicode in `.sx`:** raw UTF-8 only, never `\uXXXX` escapes. Glyphs land directly in source. +- **Worktree:** commit locally. Never push. Never touch `main`. +- **Commit granularity:** one feature per commit. +- **Plan file:** update Progress log + tick boxes every commit. + +## APL-specific gotchas + +- **Right-to-left, no precedence among functions.** `2 × 3 + 4` is `2 × (3 + 4)` = 14, not 10. Operators bind tighter than functions: `+/ ⍳5` is `+/(⍳5)`, and `2 +.× 3 4` is `2 (+.×) 3 4`. +- **Valence by position.** `-3` is monadic negate (`-` with no left arg). `5-3` is dyadic subtract. The parser must look left to decide. Same glyph; different fn. +- **`¯` is part of a number literal**, not a prefix function. `¯3` is the literal negative three; `-3` is the function call. Tokenizer eats `¯` into the numeric token. +- **Strands.** `1 2 3` is a 3-element vector, not three separate calls. Adjacent literals fuse into a strand at parse time. Adjacent names do *not* fuse — `a b c` is three separate references. +- **Scalar promotion.** `1 + 2 3 4` ↦ `3 4 5`. Any scalar broadcasts against any-rank conformable shape. +- **Conformability** = exactly matching shapes, OR one side scalar, OR (in some dialects) one side rank-1 cycling against rank-N. Keep strict in v1: matching shape or scalar only. +- **`⍳` is overloaded.** Monadic `⍳N` = vector 1..N (or 0..N-1 if `⎕IO=0`). Dyadic `V ⍳ W` = first-index lookup, returns `≢V+1` for not-found. +- **Reduce with `+/⍳0`** = `0` (identity for `+`). Each scalar primitive has a defined identity used by reduce-on-empty. Don't crash; return identity. +- **Reduce direction.** `f/` reduces the *last* axis. `f⌿` reduces the *first*. Matters for matrices. +- **Indexing is 1-based** by default (`⎕IO=1`). Do not silently translate to 0-based; respect `⎕IO`. +- **Bracket indexing** `A[I]` is sugar for `I⌷A` (squad-quad). Multi-axis: `A[I;J]` is `I J⌷A` with semicolon-separated axes; `A[;J]` selects all of axis 0. +- **Dfn `{...}`** — `⍺` = left arg (may be unbound for monadic call → check with `⍺←default`), `⍵` = right arg, `∇` = recurse. Default left arg syntax: `⍺←0`. +- **Tradfn vs dfn** — tradfns use line-numbered `→linenum` for goto; dfns use guards `cond:expr`. Pick the right one for the user's syntax. +- **Empty array** = rank-N array where some dim is 0. `0⍴⍳0` is empty rank-1. Scalar prototype matters for empty-array operations; ignore in v1, return 0/space. +- **Test corpus:** custom + idioms. Place programs in `lib/apl/tests/programs/` with `.apl` extension. + +## General gotchas (all loops) + +- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences. +- `cond`/`when`/`let` clauses evaluate only the last expr. +- `type-of` on user fn returns `"lambda"`. +- Shell heredoc `||` gets eaten — escape or use `case`. + +## Style + +- No comments in `.sx` unless non-obvious. +- No new planning docs — update `plans/apl-on-sx.md` inline. +- Short, factual commit messages (`apl: outer product ∘. (+9)`). +- One feature per iteration. Commit. Log. Next. + +Go. Read the plan; find first `[ ]`; implement. diff --git a/plans/agent-briefings/common-lisp-loop.md b/plans/agent-briefings/common-lisp-loop.md new file mode 100644 index 00000000..b82192d0 --- /dev/null +++ b/plans/agent-briefings/common-lisp-loop.md @@ -0,0 +1,80 @@ +# common-lisp-on-sx loop agent (single agent, queue-driven) + +Role: iterates `plans/common-lisp-on-sx.md` forever. Conditions + restarts on delimited continuations is the headline showcase — every other Lisp reinvents resumable exceptions on the host stack. On SX `signal`/`invoke-restart` is just a captured continuation. Plus CLOS, the LOOP macro, packages. + +``` +description: common-lisp-on-sx queue loop +subagent_type: general-purpose +run_in_background: true +isolation: worktree +``` + +## Prompt + +You are the sole background agent working `/root/rose-ash/plans/common-lisp-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push. + +## Restart baseline — check before iterating + +1. Read `plans/common-lisp-on-sx.md` — roadmap + Progress log. +2. `ls lib/common-lisp/` — pick up from the most advanced file. +3. If `lib/common-lisp/tests/*.sx` exist, run them. Green before new work. +4. If `lib/common-lisp/scoreboard.md` exists, that's your baseline. + +## The queue + +Phase order per `plans/common-lisp-on-sx.md`: + +- **Phase 1** — reader + parser (read macros `#'` `'` `` ` `` `,` `,@` `#( … )` `#:` `#\char` `#xFF` `#b1010`, ratios, dispatch chars, lambda lists with `&optional`/`&rest`/`&key`/`&aux`) +- **Phase 2** — sequential eval + special forms (`let`/`let*`/`flet`/`labels`, `block`/`return-from`, `tagbody`/`go`, `unwind-protect`, multiple values, `setf` subset, dynamic variables) +- **Phase 3** — **THE SHOWCASE**: condition system + restarts. `define-condition`, `signal`/`error`/`cerror`/`warn`, `handler-bind` (non-unwinding), `handler-case` (unwinding), `restart-case`, `restart-bind`, `find-restart`/`invoke-restart`/`compute-restarts`, `with-condition-restarts`. Classic programs (restart-demo, parse-recover, interactive-debugger) green. +- **Phase 4** — CLOS: `defclass`, `defgeneric`, `defmethod` with `:before`/`:after`/`:around`, `call-next-method`, multiple dispatch +- **Phase 5** — macros + LOOP macro + reader macros +- **Phase 6** — packages + stdlib (sequence functions, FORMAT directives, drive corpus to 200+) + +Within a phase, pick the checkbox that unlocks the most tests per effort. + +Every iteration: implement → test → commit → tick `[ ]` → Progress log → next. + +## Ground rules (hard) + +- **Scope:** only `lib/common-lisp/**` and `plans/common-lisp-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib//` dirs, `lib/stdlib.sx`, or `lib/` root. CL primitives go in `lib/common-lisp/runtime.sx`. +- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop. +- **Shared-file issues** → plan's Blockers with minimal repro. +- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines. +- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits. +- **Worktree:** commit locally. Never push. Never touch `main`. +- **Commit granularity:** one feature per commit. +- **Plan file:** update Progress log + tick boxes every commit. + +## Common-Lisp-specific gotchas + +- **`handler-bind` is non-unwinding** — handlers can decline by returning normally, in which case `signal` keeps walking the chain. **`handler-case` is unwinding** — picking a handler aborts the protected form via a captured continuation. Don't conflate them. +- **Restarts are not handlers.** `restart-case` establishes named *resumption points*; `signal` runs handler code with restarts visible; the handler chooses a restart by calling `invoke-restart`, which abandons handler stack and resumes at the restart point. Two stacks: handlers walk down, restarts wait to be invoked. +- **`block` / `return-from`** is lexical. `block name … (return-from name v) …` captures `^k` once at entry; `return-from` invokes it. `return-from` to a name not in scope is an error (don't fall back to outer block). +- **`tagbody` / `go`** — each tag in tagbody is a continuation; `go tag` invokes it. Tags are lexical, can only target tagbodies in scope. +- **`unwind-protect`** runs cleanup on *any* non-local exit (return-from, throw, condition unwind). Implement as a scope frame fired by the cleanup machinery. +- **Multiple values**: primary-value-only contexts (function args, `if` test, etc.) drop extras silently. `values` produces multiple. `multiple-value-bind` / `multiple-value-call` consume them. Don't auto-list. +- **CLOS dispatch:** sort applicable methods by argument-list specificity (`subclassp` per arg, left-to-right); standard method combination calls primary methods most-specific-first via `call-next-method` chain. `:before` runs all before primaries; `:after` runs all after, in reverse-specificity. `:around` wraps everything. +- **`call-next-method`** is a *continuation* available only inside a method body. Implement as a thunk stored in a dynamic-extent variable. +- **Generalised reference (`setf`)**: `(setf (foo x) v)` ↦ `(setf-foo v x)`. Look up the setf-expander, not just a writer fn. `define-setf-expander` is mandatory for non-trivial places. Start with the symbolic / list / aref / slot-value cases. +- **Dynamic variables (specials):** `defvar`/`defparameter` mark a symbol as special. `let` over a special name *rebinds* in dynamic extent (use parameterize-style scope), not lexical. +- **Symbols are package-qualified.** Reader resolves `cl:car`, `mypkg::internal`, bare `foo` (current package). Internal vs external matters for `:` (one colon) reads. +- **`nil` is also `()` is also the empty list.** Same object. `nil` is also false. CL has no distinct unit value. +- **LOOP macro is huge.** Build incrementally — start with `for/in`, `for/from`, `collect`, `sum`, `count`, `repeat`. Add conditional clauses (`when`, `if`, `else`) once iteration drivers stable. `named` blocks + `return-from named` last. +- **Test corpus:** custom + curated `ansi-test` slice. Place programs in `lib/common-lisp/tests/programs/` with `.lisp` extension. + +## General gotchas (all loops) + +- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences. +- `cond`/`when`/`let` clauses evaluate only the last expr. +- `type-of` on user fn returns `"lambda"`. +- Shell heredoc `||` gets eaten — escape or use `case`. + +## Style + +- No comments in `.sx` unless non-obvious. +- No new planning docs — update `plans/common-lisp-on-sx.md` inline. +- Short, factual commit messages (`common-lisp: handler-bind + 12 tests`). +- One feature per iteration. Commit. Log. Next. + +Go. Read the plan; find first `[ ]`; implement. diff --git a/plans/agent-briefings/ruby-loop.md b/plans/agent-briefings/ruby-loop.md new file mode 100644 index 00000000..9a745a8b --- /dev/null +++ b/plans/agent-briefings/ruby-loop.md @@ -0,0 +1,83 @@ +# ruby-on-sx loop agent (single agent, queue-driven) + +Role: iterates `plans/ruby-on-sx.md` forever. Fibers via delcc is the headline showcase — `Fiber.new`/`Fiber.yield`/`Fiber.resume` are textbook delimited continuations with sugar, where MRI does it via C-stack swapping. Plus blocks/yield (lexical escape continuations, same shape as Smalltalk's non-local return), method_missing, and singleton classes. + +``` +description: ruby-on-sx queue loop +subagent_type: general-purpose +run_in_background: true +isolation: worktree +``` + +## Prompt + +You are the sole background agent working `/root/rose-ash/plans/ruby-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push. + +## Restart baseline — check before iterating + +1. Read `plans/ruby-on-sx.md` — roadmap + Progress log. +2. `ls lib/ruby/` — pick up from the most advanced file. +3. If `lib/ruby/tests/*.sx` exist, run them. Green before new work. +4. If `lib/ruby/scoreboard.md` exists, that's your baseline. + +## The queue + +Phase order per `plans/ruby-on-sx.md`: + +- **Phase 1** — tokenizer + parser. Keywords, identifier sigils (`@` ivar, `@@` cvar, `$` global), strings with interpolation, `%w[]`/`%i[]`, symbols, blocks `{|x| …}` and `do |x| … end`, splats, default args, method def +- **Phase 2** — object model + sequential eval. Class table, ancestor-chain dispatch, `super`, singleton classes, `method_missing` fallback, dynamic constant lookup +- **Phase 3** — blocks + procs + lambdas. Method captures escape continuation `^k`; `yield` / `return` / `break` / `next` / `redo` semantics; lambda strict arity vs proc lax +- **Phase 4** — **THE SHOWCASE**: fibers via delcc. `Fiber.new`/`Fiber.resume`/`Fiber.yield`/`Fiber.transfer`. Classic programs (generator, producer-consumer, tree-walk) green +- **Phase 5** — modules + mixins + metaprogramming. `include`/`prepend`/`extend`, `define_method`, `class_eval`/`instance_eval`, `respond_to?`/`respond_to_missing?`, hooks +- **Phase 6** — stdlib drive. `Enumerable` mixin, `Comparable`, Array/Hash/Range/String/Integer methods, drive corpus to 200+ + +Within a phase, pick the checkbox that unlocks the most tests per effort. + +Every iteration: implement → test → commit → tick `[ ]` → Progress log → next. + +## Ground rules (hard) + +- **Scope:** only `lib/ruby/**` and `plans/ruby-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib//` dirs, `lib/stdlib.sx`, or `lib/` root. Ruby primitives go in `lib/ruby/runtime.sx`. +- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop. +- **Shared-file issues** → plan's Blockers with minimal repro. +- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines. +- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits. +- **Worktree:** commit locally. Never push. Never touch `main`. +- **Commit granularity:** one feature per commit. +- **Plan file:** update Progress log + tick boxes every commit. + +## Ruby-specific gotchas + +- **Block `return` vs lambda `return`.** Inside a block `{ ... return v }`, `return` invokes the *enclosing method's* escape continuation (non-local return). Inside a lambda `->(){ ... return v }`, `return` returns from the *lambda*. Don't conflate. Implement: blocks bind their `^method-k`; lambdas bind their own `^lambda-k`. +- **`break` from inside a block** invokes a different escape — the *iteration loop's* escape — and the loop returns the break-value. `next` is escape from current iteration, returns iteration value. `redo` re-enters current iteration without advancing. +- **Proc arity is lax.** `proc { |a, b, c| … }.call(1, 2)` ↦ `c = nil`. Lambda is strict — same call raises ArgumentError. Check arity at call site for lambdas only. +- **Block argument unpacking.** `[[1,2],[3,4]].each { |a, b| … }` — single Array arg auto-unpacks for blocks (not lambdas). One arg, one Array → unpack. Frequent footgun. +- **Method dispatch chain order:** prepended modules → class methods → included modules → superclass → BasicObject → method_missing. `super` walks from the *defining* class's position, not the receiver class's. +- **Singleton classes** are lazily allocated. Looking up the chain for an object passes through its singleton class first, then its actual class. `class << obj; …; end` opens the singleton. +- **`method_missing`** — fallback when ancestor walk misses. Receives `(name_symbol, *args, &blk)`. Pair with `respond_to_missing?` for `respond_to?` to also report true. Do **not** swallow NoMethodError silently. +- **Ivars are per-object dicts.** Reading an unset ivar yields `nil` and a warning (`-W`). Don't error. +- **Constant lookup** is first lexical (Module.nesting), then inheritance (Module.ancestors of the innermost class). Different from method lookup. +- **`Object#send`** invokes private and public methods alike; `Object#public_send` skips privates. +- **Class reopening.** `class Foo; def bar; …; end; end` plus a later `class Foo; def baz; …; end; end` adds methods to the same class. Class table lookups must be by-name, mutable; methods dict is mutable. +- **Fiber semantics.** `Fiber.new { |arg| … }` creates a fiber suspended at entry. First `Fiber.resume(v)` enters with `arg = v`. Inside, `Fiber.yield(w)` returns `w` to the resumer; the next `Fiber.resume(v')` returns `v'` to the yield site. End of block returns final value to last resumer; subsequent `Fiber.resume` raises FiberError. +- **`Fiber.transfer`** is symmetric — either side can transfer to the other; no resume/yield asymmetry. Implement on top of the same continuation pair, just don't enforce direction. +- **Symbols are interned.** `:foo == :foo` is identity. Use SX symbols. +- **Strings are mutable.** `s = "abc"; s << "d"; s == "abcd"`. Hash keys can be strings; hash dups string keys at insertion to be safe (or freeze them). +- **Truthiness:** only `false` and `nil` are falsy. `0`, `""`, `[]` are truthy. +- **Test corpus:** custom + curated RubySpec slice. Place programs in `lib/ruby/tests/programs/` with `.rb` extension. + +## General gotchas (all loops) + +- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences. +- `cond`/`when`/`let` clauses evaluate only the last expr. +- `type-of` on user fn returns `"lambda"`. +- Shell heredoc `||` gets eaten — escape or use `case`. + +## Style + +- No comments in `.sx` unless non-obvious. +- No new planning docs — update `plans/ruby-on-sx.md` inline. +- Short, factual commit messages (`ruby: Fiber.yield + Fiber.resume (+8)`). +- One feature per iteration. Commit. Log. Next. + +Go. Read the plan; find first `[ ]`; implement. diff --git a/plans/agent-briefings/tcl-loop.md b/plans/agent-briefings/tcl-loop.md new file mode 100644 index 00000000..449fe757 --- /dev/null +++ b/plans/agent-briefings/tcl-loop.md @@ -0,0 +1,83 @@ +# tcl-on-sx loop agent (single agent, queue-driven) + +Role: iterates `plans/tcl-on-sx.md` forever. `uplevel`/`upvar` is the headline showcase — Tcl's superpower for defining your own control structures, requiring deep VM cooperation in any normal host but falling out of SX's first-class env-chain. Plus the Dodekalogue (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity. + +``` +description: tcl-on-sx queue loop +subagent_type: general-purpose +run_in_background: true +isolation: worktree +``` + +## Prompt + +You are the sole background agent working `/root/rose-ash/plans/tcl-on-sx.md`. Isolated worktree, forever, one commit per feature. Never push. + +## Restart baseline — check before iterating + +1. Read `plans/tcl-on-sx.md` — roadmap + Progress log. +2. `ls lib/tcl/` — pick up from the most advanced file. +3. If `lib/tcl/tests/*.sx` exist, run them. Green before new work. +4. If `lib/tcl/scoreboard.md` exists, that's your baseline. + +## The queue + +Phase order per `plans/tcl-on-sx.md`: + +- **Phase 1** — tokenizer + parser. The Dodekalogue (12 rules): word-splitting, command sub `[…]`, var sub `$name`/`${name}`/`$arr(idx)`, double-quote vs brace word, backslash, `;`, `#` comments only at command start, single-pass left-to-right substitution +- **Phase 2** — sequential eval + core commands. `set`/`unset`/`incr`/`append`/`lappend`, `puts`/`gets`, `expr` (own mini-language), `if`/`while`/`for`/`foreach`/`switch`, string commands, list commands, dict commands +- **Phase 3** — **THE SHOWCASE**: `proc` + `uplevel` + `upvar`. Frame stack with proc-call push/pop; `uplevel #N script` evaluates in caller's frame; `upvar` aliases names across frames. Classic programs (for-each-line, assert macro, with-temp-var) green +- **Phase 4** — `return -code N`, `catch`, `try`/`trap`/`finally`, `throw`. Control flow as integer codes +- **Phase 5** — namespaces + ensembles. `namespace eval`, qualified names `::ns::cmd`, ensembles, `namespace path` +- **Phase 6** — coroutines (built on fibers, same delcc as Ruby fibers) + system commands + drive corpus to 150+ + +Within a phase, pick the checkbox that unlocks the most tests per effort. + +Every iteration: implement → test → commit → tick `[ ]` → Progress log → next. + +## Ground rules (hard) + +- **Scope:** only `lib/tcl/**` and `plans/tcl-on-sx.md`. Do **not** edit `spec/`, `hosts/`, `shared/`, other `lib//` dirs, `lib/stdlib.sx`, or `lib/` root. Tcl primitives go in `lib/tcl/runtime.sx`. +- **NEVER call `sx_build`.** 600s watchdog. If sx_server binary broken → Blockers entry, stop. +- **Shared-file issues** → plan's Blockers with minimal repro. +- **Delimited continuations** are in `lib/callcc.sx` + `spec/evaluator.sx` Step 5. `sx_summarise` spec/evaluator.sx first — 2300+ lines. +- **SX files:** `sx-tree` MCP tools ONLY. `sx_validate` after edits. +- **Worktree:** commit locally. Never push. Never touch `main`. +- **Commit granularity:** one feature per commit. +- **Plan file:** update Progress log + tick boxes every commit. + +## Tcl-specific gotchas + +- **Everything is a string.** Internally cache shimmer reps (list, dict, int, double) for performance, but every value must be re-stringifiable. Mutating one rep dirties the cached string and vice versa. +- **The Dodekalogue is strict.** Substitution is **one-pass**, **left-to-right**. The result of a substitution is a value, not a script — it does NOT get re-parsed for further substitutions. This is what makes Tcl safe-by-default. Don't accidentally re-parse. +- **Brace word `{…}`** is the only way to defer evaluation. No substitution inside, just balanced braces. Used for `if {expr}` body, `proc body`, `expr` arguments. +- **Double-quote word `"…"`** is identical to a bare word for substitution purposes — it just allows whitespace in a single word. `\` escapes still apply. +- **Comments are only at command position.** `# this is a comment` after a `;` or newline; *not* inside a command. `set x 1 # not a comment` is a 4-arg `set`. +- **`expr` has its own grammar** — operator precedence, function calls — and does its own substitution. Brace `expr {$x + 1}` to avoid double-substitution and to enable bytecode caching. +- **`if` and `while` re-parse** the condition only if not braced. Always use `if {…}`/`while {…}` form. The unbraced form re-substitutes per iteration. +- **`return` from a `proc`** uses control code 2. `break` is 3, `continue` is 4. `error` is 1. `catch` traps any non-zero code; user can return non-zero with `return -code error -errorcode FOO message`. +- **`uplevel #0 script`** is global frame. `uplevel 1 script` (or just `uplevel script`) is caller's frame. `uplevel #N` is absolute level N (0=global, 1=top-level proc, 2=proc-called-from-top, …). Negative levels are errors. +- **`upvar #N otherVar localVar`** binds `localVar` in the current frame as an *alias* — both names refer to the same storage. Reads and writes go through the alias. +- **`info level`** with no arg returns current level number. `info level N` (positive) returns the command list that invoked level N. `info level -N` returns the command list of the level N relative-up. +- **Variable names with `(…)`** are array elements: `set arr(foo) 1`. Arrays are not first-class values — you can't `set x $arr`. `array get arr` gives a flat list `{key1 val1 key2 val2 …}`. +- **List vs string.** `set l "a b c"` and `set l [list a b c]` look the same when printed but the second has a cached list rep. `lindex` works on both via shimmering. Most user code can't tell the difference. +- **`incr x`** errors if x doesn't exist; pre-set with `set x 0` or use `incr x 0` first if you mean "create-or-increment". Or use `dict incr` for dicts. +- **Coroutines are fibers.** `coroutine name body` starts a coroutine; calling `name` resumes it; `yield value` from inside suspends and returns `value` to the resumer. Same primitive as Ruby fibers — share the implementation under the hood. +- **`switch`** matches first clause whose pattern matches. Default is `default`. Variant matches: glob (default), `-exact`, `-glob`, `-regexp`. Body `-` means "fall through to next clause's body". +- **Test corpus:** custom + slice of Tcl's own tests. Place programs in `lib/tcl/tests/programs/` with `.tcl` extension. + +## General gotchas (all loops) + +- SX `do` = R7RS iteration. Use `begin` for multi-expr sequences. +- `cond`/`when`/`let` clauses evaluate only the last expr. +- `type-of` on user fn returns `"lambda"`. +- Shell heredoc `||` gets eaten — escape or use `case`. + +## Style + +- No comments in `.sx` unless non-obvious. +- No new planning docs — update `plans/tcl-on-sx.md` inline. +- Short, factual commit messages (`tcl: uplevel + upvar (+11)`). +- One feature per iteration. Commit. Log. Next. + +Go. Read the plan; find first `[ ]`; implement. diff --git a/plans/apl-on-sx.md b/plans/apl-on-sx.md new file mode 100644 index 00000000..d22cdd92 --- /dev/null +++ b/plans/apl-on-sx.md @@ -0,0 +1,115 @@ +# APL-on-SX: rank-polymorphic primitives + glyph parser + +The headline showcase is **rank polymorphism** — a single primitive (`+`, `⌈`, `⊂`, `⍳`) works uniformly on scalars, vectors, matrices, and higher-rank arrays. ~80 glyph primitives + 6 operators bind together with right-to-left evaluation; the entire language is a high-density combinator algebra. The JIT compiler + primitive table pay off massively here because almost every program is `array → array` pure pipelines. + +End-state goal: Dyalog-flavoured APL subset, dfns + tradfns, classic programs (game-of-life, mandelbrot, prime-sieve, n-queens, conway), 100+ green tests. + +## Scope decisions (defaults — override by editing before we spawn) + +- **Syntax:** Dyalog APL surface, Unicode glyphs. `⎕`-quad system functions for I/O. `∇` tradfn header. +- **Conformance:** "Reads like APL, runs like APL." Not byte-compat with Dyalog; we care about right-to-left semantics and rank polymorphism. +- **Test corpus:** custom — APL idioms (Roger Hui style), classic programs, plus ~50 pattern tests for primitives. +- **Out of scope:** ⎕-namespaces beyond a handful, complex numbers, full TAO ordering, `⎕FX` runtime function definition (use static `∇` only), nested-array-of-functions higher orders, the editor. +- **Glyphs:** input via plain Unicode in `.apl` source files. Backtick-prefix shortcuts handled by the user's editor — we don't ship one. + +## Ground rules + +- **Scope:** only touch `lib/apl/**` and `plans/apl-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib//**`. APL primitives go in `lib/apl/runtime.sx`. +- **SX files:** use `sx-tree` MCP tools only. +- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes. + +## Architecture sketch + +``` +APL source (Unicode glyphs) + │ + ▼ +lib/apl/tokenizer.sx — glyphs, identifiers, numbers (¯ for negative), strings, strands + │ + ▼ +lib/apl/parser.sx — right-to-left with valence resolution (mon vs dyadic by position) + │ + ▼ +lib/apl/transpile.sx — AST → SX AST (entry: apl-eval-ast) + │ + ▼ +lib/apl/runtime.sx — array model, ~80 primitives, 6 operators, dfns/tradfns +``` + +Core mapping: +- **Array** = SX dict `{:shape (d1 d2 …) :ravel #(v1 v2 …)}`. Scalar is rank-0 (empty shape), vector is rank-1, matrix rank-2, etc. Type uniformity not required (heterogeneous nested arrays via "boxed" elements `⊂x`). +- **Rank polymorphism** — every scalar primitive is broadcast: `1 2 3 + 4 5 6` ↦ `5 7 9`; `(2 3⍴⍳6) + 1` ↦ broadcast scalar to matrix. +- **Conformability** = matching shapes, or one-side scalar, or rank-1 cycling (deferred — keep strict in v1). +- **Valence** = each glyph has a monadic and a dyadic meaning; resolution is purely positional (left-arg present → dyadic). +- **Operator** = takes one or two function operands, returns a derived function (`f¨` = `each f`, `f/` = `reduce f`, `f∘g` = `compose`, `f⍨` = `commute`). +- **Tradfn** `∇R←L F R; locals` = named function with explicit header. +- **Dfn** `{⍺+⍵}` = anonymous, `⍺` = left arg, `⍵` = right arg, `∇` = recurse. + +## Roadmap + +### Phase 1 — tokenizer + parser +- [ ] Tokenizer: Unicode glyphs (the full APL set: `+ - × ÷ * ⍟ ⌈ ⌊ | ! ? ○ ~ < ≤ = ≥ > ≠ ∊ ∧ ∨ ⍱ ⍲ , ⍪ ⍴ ⌽ ⊖ ⍉ ↑ ↓ ⊂ ⊃ ⊆ ∪ ∩ ⍳ ⍸ ⌷ ⍋ ⍒ ⊥ ⊤ ⊣ ⊢ ⍎ ⍕ ⍝`), operators (`/ \ ¨ ⍨ ∘ . ⍣ ⍤ ⍥ @`), numbers (`¯` for negative, `1E2`, `1J2` complex deferred), characters (`'a'`, `''` escape), strands (juxtaposition of literals: `1 2 3`), names, comments `⍝ …` +- [ ] Parser: right-to-left; classify each token as function, operator, value, or name; resolve valence positionally; dfn `{…}` body, tradfn `∇` header, guards `:`, control words `:If :While :For …` (Dyalog-style) +- [ ] Unit tests in `lib/apl/tests/parse.sx` + +### Phase 2 — array model + scalar primitives +- [ ] Array constructor: `make-array shape ravel`, `scalar v`, `vector v…`, `enclose`/`disclose` +- [ ] Shape arithmetic: `⍴` (shape), `,` (ravel), `≢` (tally / first-axis-length), `≡` (depth) +- [ ] Scalar arithmetic primitives broadcast: `+ - × ÷ ⌈ ⌊ * ⍟ | ! ○` +- [ ] Scalar comparison primitives: `< ≤ = ≥ > ≠` +- [ ] Scalar logical: `~ ∧ ∨ ⍱ ⍲` +- [ ] Index generator: `⍳n` (vector 1..n or 0..n-1 depending on `⎕IO`) +- [ ] `⎕IO` = 1 default (Dyalog convention) +- [ ] 40+ tests in `lib/apl/tests/scalar.sx` + +### Phase 3 — structural primitives + indexing +- [ ] Reshape `⍴`, ravel `,`, transpose `⍉` (full + dyadic axis spec) +- [ ] Take `↑`, drop `↓`, rotate `⌽` (last axis), `⊖` (first axis) +- [ ] Catenate `,` (last axis) and `⍪` (first axis) +- [ ] Index `⌷` (squad), bracket-indexing `A[I]` (sugar for `⌷`) +- [ ] Grade-up `⍋`, grade-down `⍒` +- [ ] Enclose `⊂`, disclose `⊃`, partition (subset deferred) +- [ ] Membership `∊`, find `⍳` (dyadic), without `~` (dyadic), unique `∪` (deferred to phase 6) +- [ ] 40+ tests in `lib/apl/tests/structural.sx` + +### Phase 4 — operators (THE SHOWCASE) +- [ ] Reduce `f/` (last axis), `f⌿` (first axis) — including `∧/`, `∨/`, `+/`, `×/`, `⌈/`, `⌊/` +- [ ] Scan `f\`, `f⍀` +- [ ] Each `f¨` — applies `f` to each scalar/element +- [ ] Outer product `∘.f` — `1 2 3 ∘.× 1 2 3` ↦ multiplication table +- [ ] Inner product `f.g` — `+.×` is matrix multiply +- [ ] Commute `f⍨` — `f⍨ x` ↔ `x f x`, `x f⍨ y` ↔ `y f x` +- [ ] Compose `f∘g` — applies `g` first then `f` +- [ ] Power `f⍣n` — apply f n times; `f⍣≡` until fixed point +- [ ] Rank `f⍤k` — apply f at sub-rank k +- [ ] At `@` — selective replace +- [ ] 40+ tests in `lib/apl/tests/operators.sx` + +### Phase 5 — dfns + tradfns + control flow +- [ ] Dfn `{…}` with `⍺` (left arg, may be absent → niladic/monadic), `⍵` (right arg), `∇` (recurse), guards `cond:expr`, default left arg `⍺←default` +- [ ] Local assignment via `←` (lexical inside dfn) +- [ ] Tradfn `∇` header: `R←L F R;l1;l2`, statement-by-statement, branch via `→linenum` +- [ ] Dyalog control words: `:If/:Else/:EndIf`, `:While/:EndWhile`, `:For X :In V :EndFor`, `:Select/:Case/:EndSelect`, `:Trap`/`:EndTrap` +- [ ] Niladic / monadic / dyadic dispatch (function valence at definition time) +- [ ] `lib/apl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` + +### Phase 6 — classic programs + drive corpus +- [ ] Classic programs in `lib/apl/tests/programs/`: + - [ ] `life.apl` — Conway's Game of Life as a one-liner using `⊂` `⊖` `⌽` `+/` + - [ ] `mandelbrot.apl` — complex iteration with rank-polymorphic `+ × ⌊` (or real-axis subset) + - [ ] `primes.apl` — `(2=+⌿0=A∘.|A)/A←⍳N` sieve + - [ ] `n-queens.apl` — backtracking via reduce + - [ ] `quicksort.apl` — the classic Roger Hui one-liner +- [ ] System functions: `⎕FMT`, `⎕FR` (float repr), `⎕TS` (timestamp), `⎕IO`, `⎕ML` (migration level — fixed at 1), `⎕←` (print) +- [ ] Drive corpus to 100+ green +- [ ] Idiom corpus — `lib/apl/tests/idioms.sx` covering classic Roger Hui / Phil Last idioms + +## Progress log + +_Newest first._ + +- _(none yet)_ + +## Blockers + +- _(none yet)_ diff --git a/plans/common-lisp-on-sx.md b/plans/common-lisp-on-sx.md new file mode 100644 index 00000000..3b59215d --- /dev/null +++ b/plans/common-lisp-on-sx.md @@ -0,0 +1,121 @@ +# Common-Lisp-on-SX: conditions + restarts on delimited continuations + +The headline showcase is the **condition system**. Restarts are *resumable* exceptions — every other Lisp implementation reinvents this on host-stack unwind tricks. On SX restarts are textbook delimited continuations: `signal` walks the handler chain; `invoke-restart` resumes the captured continuation at the restart point. Same delcc primitive that powers Erlang actors, expressed as a different surface. + +End-state goal: ANSI Common Lisp subset with a working condition/restart system, CLOS multimethods (with `:before`/`:after`/`:around`), the LOOP macro, packages, and ~150 hand-written + classic programs. + +## Scope decisions (defaults — override by editing before we spawn) + +- **Syntax:** ANSI Common Lisp surface. Read tables, dispatch macros (`#'`, `#(`, `#\`, `#:`, `#x`, `#b`, `#o`, ratios `1/3`). +- **Conformance:** ANSI X3.226 *as a target*, not bug-for-bug SBCL/CCL. "Reads like CL, runs like CL." +- **Test corpus:** custom + a curated slice of `ansi-test`. Plus classic programs: condition-system demo, restart-driven debugger, multiple-dispatch geometry, LOOP corpus. +- **Out of scope:** compilation to native, FFI, sockets, threads, MOP class redefinition, full pathname/logical-pathname machinery, structures with `:include` deep customization. +- **Packages:** simple — `defpackage`/`in-package`/`export`/`use-package`/`:cl`/`:cl-user`. No nicknames, no shadowing-import edge cases. + +## Ground rules + +- **Scope:** only touch `lib/common-lisp/**` and `plans/common-lisp-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib//**`. CL primitives go in `lib/common-lisp/runtime.sx`. +- **SX files:** use `sx-tree` MCP tools only. +- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes. + +## Architecture sketch + +``` +Common Lisp source + │ + ▼ +lib/common-lisp/reader.sx — tokenizer + reader (read macros, dispatch chars) + │ + ▼ +lib/common-lisp/parser.sx — AST: forms, declarations, lambda lists + │ + ▼ +lib/common-lisp/transpile.sx — AST → SX AST (entry: cl-eval-ast) + │ + ▼ +lib/common-lisp/runtime.sx — special forms, condition system, CLOS, packages, BIFs +``` + +Core mapping: +- **Symbol** = SX symbol with package prefix; package table is a flat dict. +- **Cons cell** = SX pair via `cons`/`car`/`cdr`; lists native. +- **Multiple values** = thread through `values`/`multiple-value-bind`; primary-value default for one-context callers. +- **Block / return-from** = captured continuation; `return-from name v` invokes the block-named `^k`. +- **Tagbody / go** = each tag is a continuation; `go tag` invokes it. +- **Unwind-protect** = scope frame with a cleanup thunk fired on any non-local exit. +- **Conditions / restarts** = layered handler chain on top of `handler-bind` + delcc. `signal` walks handlers; `invoke-restart` resumes a captured continuation. +- **CLOS** = generic functions are dispatch tables on argument-class lists; method combination computed lazily; `call-next-method` is a continuation. +- **Macros** = SX macros (sentinel-body) — defmacro lowers directly. + +## Roadmap + +### Phase 1 — reader + parser +- [ ] Tokenizer: symbols (with package qualification `pkg:sym` / `pkg::sym`), numbers (int, float, ratio `1/3`, `#xFF`, `#b1010`, `#o17`), strings `"…"` with `\` escapes, characters `#\Space` `#\Newline` `#\a`, comments `;`, block comments `#| … |#` +- [ ] Reader: list, dotted pair, quote `'`, function `#'`, quasiquote `` ` ``, unquote `,`, splice `,@`, vector `#(…)`, uninterned `#:foo`, nil/t literals +- [ ] Parser: lambda lists with `&optional` `&rest` `&key` `&aux` `&allow-other-keys`, defaults, supplied-p variables +- [ ] Unit tests in `lib/common-lisp/tests/read.sx` + +### Phase 2 — sequential eval + special forms +- [ ] `cl-eval-ast`: `quote`, `if`, `progn`, `let`, `let*`, `flet`, `labels`, `setq`, `setf` (subset), `function`, `lambda`, `the`, `locally`, `eval-when` +- [ ] `block` + `return-from` via captured continuation +- [ ] `tagbody` + `go` via per-tag continuations +- [ ] `unwind-protect` cleanup frame +- [ ] `multiple-value-bind`, `multiple-value-call`, `multiple-value-prog1`, `values`, `nth-value` +- [ ] `defun`, `defparameter`, `defvar`, `defconstant`, `declaim`, `proclaim` (no-op) +- [ ] Dynamic variables — `defvar`/`defparameter` produce specials; `let` rebinds via parameterize-style scope +- [ ] 60+ tests in `lib/common-lisp/tests/eval.sx` + +### Phase 3 — conditions + restarts (THE SHOWCASE) +- [ ] `define-condition` — class hierarchy rooted at `condition`/`error`/`warning`/`simple-error`/`simple-warning`/`type-error`/`arithmetic-error`/`division-by-zero` +- [ ] `signal`, `error`, `cerror`, `warn` — all walk the handler chain +- [ ] `handler-bind` — non-unwinding handlers, may decline by returning normally +- [ ] `handler-case` — unwinding handlers (delcc abort) +- [ ] `restart-case`, `with-simple-restart`, `restart-bind` +- [ ] `find-restart`, `invoke-restart`, `invoke-restart-interactively`, `compute-restarts` +- [ ] `with-condition-restarts` — associate restarts with a specific condition +- [ ] `*break-on-signals*`, `*debugger-hook*` (basic) +- [ ] Classic programs in `lib/common-lisp/tests/programs/`: + - [ ] `restart-demo.lisp` — division with `:use-zero` and `:retry` restarts + - [ ] `parse-recover.lisp` — parser with skipped-token restart + - [ ] `interactive-debugger.lisp` — ASCII REPL using `:debugger-hook` +- [ ] `lib/common-lisp/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` + +### Phase 4 — CLOS +- [ ] `defclass` with `:initarg`/`:initform`/`:accessor`/`:reader`/`:writer`/`:allocation` +- [ ] `make-instance`, `slot-value`, `(setf slot-value)`, `with-slots`, `with-accessors` +- [ ] `defgeneric` with `:method-combination` (standard, plus `+`, `and`, `or`) +- [ ] `defmethod` with `:before` / `:after` / `:around` qualifiers +- [ ] `call-next-method` (continuation), `next-method-p` +- [ ] `class-of`, `find-class`, `slot-boundp`, `change-class` (basic) +- [ ] Multiple dispatch — method specificity by argument-class precedence list +- [ ] Built-in classes registered for tagged values (`integer`, `float`, `string`, `symbol`, `cons`, `null`, `t`) +- [ ] Classic programs: + - [ ] `geometry.lisp` — `intersect` generic dispatching on (point line), (line line), (line plane)… + - [ ] `mop-trace.lisp` — `:before` + `:after` printing call trace + +### Phase 5 — macros + LOOP + reader macros +- [ ] `defmacro`, `macrolet`, `symbol-macrolet`, `macroexpand-1`, `macroexpand` +- [ ] `gensym`, `gentemp` +- [ ] `set-macro-character`, `set-dispatch-macro-character`, `get-macro-character` +- [ ] **The LOOP macro** — iteration drivers (`for … in/across/from/upto/downto/by`, `while`, `until`, `repeat`), accumulators (`collect`, `append`, `nconc`, `count`, `sum`, `maximize`, `minimize`), conditional clauses (`if`/`when`/`unless`/`else`), termination (`finally`/`thereis`/`always`/`never`), `named` blocks +- [ ] LOOP test corpus: 30+ tests covering all clause types + +### Phase 6 — packages + stdlib drive +- [ ] `defpackage`, `in-package`, `export`, `use-package`, `import`, `find-package` +- [ ] Package qualification at the reader level — `cl:car`, `mypkg::internal` +- [ ] `:common-lisp` (`:cl`) and `:common-lisp-user` (`:cl-user`) packages +- [ ] Sequence functions — `mapcar`, `mapc`, `mapcan`, `reduce`, `find`, `find-if`, `position`, `count`, `every`, `some`, `notany`, `notevery`, `remove`, `remove-if`, `subst` +- [ ] List ops — `assoc`, `getf`, `nth`, `last`, `butlast`, `nthcdr`, `tailp`, `ldiff` +- [ ] String ops — `string=`, `string-upcase`, `string-downcase`, `subseq`, `concatenate` +- [ ] FORMAT — basic directives `~A`, `~S`, `~D`, `~F`, `~%`, `~&`, `~T`, `~{...~}` (iteration), `~[...~]` (conditional), `~^` (escape), `~P` (plural) +- [ ] Drive corpus to 200+ green + +## Progress log + +_Newest first._ + +- _(none yet)_ + +## Blockers + +- _(none yet)_ diff --git a/plans/ruby-on-sx.md b/plans/ruby-on-sx.md new file mode 100644 index 00000000..c10a4035 --- /dev/null +++ b/plans/ruby-on-sx.md @@ -0,0 +1,124 @@ +# Ruby-on-SX: fibers + blocks + open classes on delimited continuations + +The headline showcase is **fibers** — Ruby's `Fiber.new { … Fiber.yield v … }` / `Fiber.resume` are textbook delimited continuations with sugar. MRI implements them by swapping C stacks; on SX they fall out of the existing `perform`/`cek-resume` machinery for free. Plus blocks/yield (lexical escape continuations, same shape as Smalltalk's non-local return), method_missing, and singleton classes. + +End-state goal: Ruby 2.7-flavoured subset, Enumerable mixin, fibers + threads-via-fibers (no real OS threads), method_missing-driven DSLs, ~150 hand-written + classic programs. + +## Scope decisions (defaults — override by editing before we spawn) + +- **Syntax:** Ruby 2.7. No 3.x pattern matching, no rightward assignment, no endless methods. We pick 2.7 because it's the biggest semantic surface that still parses cleanly. +- **Conformance:** "Reads like Ruby, runs like Ruby." Slice of RubySpec (Core + Library subset), not full RubySpec. +- **Test corpus:** custom + curated RubySpec slice. Plus classic programs: fiber-based generator, internal DSL with method_missing, mixin-based Enumerable on a custom class. +- **Out of scope:** real threads, GIL, refinements, `binding_of_caller` from non-Ruby contexts, Encoding object beyond UTF-8/ASCII-8BIT, RubyVM::* introspection beyond bytecode-disassembly placeholder, IO subsystem beyond `puts`/`gets`/`File.read`. +- **Symbols:** SX symbols. Strings are mutable copies; symbols are interned. + +## Ground rules + +- **Scope:** only touch `lib/ruby/**` and `plans/ruby-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib//**`. Ruby primitives go in `lib/ruby/runtime.sx`. +- **SX files:** use `sx-tree` MCP tools only. +- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes. + +## Architecture sketch + +``` +Ruby source + │ + ▼ +lib/ruby/tokenizer.sx — keywords, ops, %w[], %i[], heredocs (deferred), regex (deferred) + │ + ▼ +lib/ruby/parser.sx — AST: classes, modules, methods, blocks, calls + │ + ▼ +lib/ruby/transpile.sx — AST → SX AST (entry: rb-eval-ast) + │ + ▼ +lib/ruby/runtime.sx — class table, MOP, dispatch, fibers, primitives +``` + +Core mapping: +- **Object** = SX dict `{:class :ivars :singleton-class?}`. Instance variables live in `ivars` keyed by symbol. +- **Class** = SX dict `{:name :superclass :methods :class-methods :metaclass :includes :prepends}`. Class table is flat. +- **Method dispatch** = lookup walks ancestor chain (prepended → class → included modules → superclass → …). Falls back to `method_missing` with a `Symbol`+args. +- **Block** = lambda + escape continuation. `yield` invokes the block in current context. `return` from within a block invokes the enclosing-method's escape continuation. +- **Proc** = lambda without strict arity. `Proc.new` + `proc {}`. +- **Lambda** = lambda with strict arity + `return`-returns-from-lambda semantics. +- **Fiber** = pair of continuations (resume-k, yield-k) wrapped in a record. `Fiber.new { … }` builds it; `Fiber.resume` invokes the resume-k; `Fiber.yield` invokes the yield-k. Built directly on `perform`/`cek-resume`. +- **Module** = class without instance allocation. `include` puts it in the chain; `prepend` puts it earlier; `extend` puts it on the singleton. +- **Singleton class** = lazily allocated per-object class for `def obj.foo` definitions. +- **Symbol** = interned SX symbol. `:foo` reads as `(quote foo)` flavour. + +## Roadmap + +### Phase 1 — tokenizer + parser +- [ ] Tokenizer: keywords (`def end class module if unless while until do return yield begin rescue ensure case when then else elsif`), identifiers (lowercase = local/method, `@` = ivar, `@@` = cvar, `$` = global, uppercase = constant), numbers (int, float, `0x` `0o` `0b`, `_` separators), strings (`"…"` interpolation, `'…'` literal, `%w[a b c]`, `%i[a b c]`), symbols `:foo` `:"…"`, operators (`+ - * / % ** == != < > <= >= <=> === =~ !~ << >> & | ^ ~ ! && || and or not`), `:: . , ; ( ) [ ] { } -> => |`, comments `#` +- [ ] Parser: program is sequence of statements separated by newlines or `;`; method def `def name(args) … end`; class `class Foo < Bar … end`; module `module M … end`; block `do |a, b| … end` and `{ |a, b| … }`; call sugar (no parens), `obj.method`, `Mod::Const`; arg shapes (positional, default, splat `*args`, double-splat `**opts`, block `&blk`) +- [ ] If/while/case expressions (return values), `unless`/`until`, postfix modifiers +- [ ] Begin/rescue/ensure/retry, raise, raise with class+message +- [ ] Unit tests in `lib/ruby/tests/parse.sx` + +### Phase 2 — object model + sequential eval +- [ ] Class table bootstrap: `BasicObject`, `Object`, `Kernel`, `Module`, `Class`, `Numeric`, `Integer`, `Float`, `String`, `Symbol`, `Array`, `Hash`, `Range`, `NilClass`, `TrueClass`, `FalseClass`, `Proc`, `Method` +- [ ] `rb-eval-ast`: literals, variables (local, ivar, cvar, gvar, constant), assignment (single and parallel `a, b = 1, 2`, splat receive), method call, message dispatch +- [ ] Method lookup walks ancestor chain; cache hit-class per `(class, selector)` +- [ ] `method_missing` fallback constructing args list +- [ ] `super` and `super(args)` — lookup in defining class's superclass +- [ ] Singleton class allocation on first `def obj.foo` or `class << obj` +- [ ] `nil`, `true`, `false` are singletons of their classes; tagged values aren't boxed +- [ ] Constant lookup (lexical-then-inheritance) with `Module.nesting` +- [ ] 60+ tests in `lib/ruby/tests/eval.sx` + +### Phase 3 — blocks + procs + lambdas +- [ ] Method invocation captures escape continuation `^k` for `return`; binds it as block's escape +- [ ] `yield` invokes implicit block +- [ ] `block_given?`, `&blk` parameter, `&proc` arg unpacking +- [ ] `Proc.new`, `proc { }`, `lambda { }` (or `->(x) { x }`) +- [ ] Lambda strict arity + lambda-local `return` semantics +- [ ] Proc lax arity (`a, b, c` unpacks Array; missing args nil) +- [ ] `break`, `next`, `redo` — `break` is escape-from-loop-or-block; `next` is escape-from-block-iteration; `redo` re-runs current iteration +- [ ] 30+ tests in `lib/ruby/tests/blocks.sx` + +### Phase 4 — fibers (THE SHOWCASE) +- [ ] `Fiber.new { |arg| … Fiber.yield v … }` allocates a fiber record with paired continuations +- [ ] `Fiber.resume(args…)` resumes the fiber, returning the value passed to `Fiber.yield` +- [ ] `Fiber.yield(v)` from inside the fiber suspends and returns control to the resumer +- [ ] `Fiber.current` from inside the fiber +- [ ] `Fiber#alive?`, `Fiber#raise` (deferred) +- [ ] `Fiber.transfer` — symmetric coroutines (resume from any side) +- [ ] Classic programs in `lib/ruby/tests/programs/`: + - [ ] `generator.rb` — pull-style infinite enumerator built on fibers + - [ ] `producer-consumer.rb` — bounded buffer with `Fiber.transfer` + - [ ] `tree-walk.rb` — recursive tree walker that yields each node, driven by `Fiber.resume` +- [ ] `lib/ruby/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` + +### Phase 5 — modules + mixins + metaprogramming +- [ ] `include M` — appends M's methods after class methods in chain +- [ ] `prepend M` — prepends M before class methods +- [ ] `extend M` — adds M to singleton class +- [ ] `Module#ancestors`, `Module#included_modules` +- [ ] `define_method`, `class_eval`, `instance_eval`, `module_eval` +- [ ] `respond_to?`, `respond_to_missing?`, `method_missing` +- [ ] `Object#send`, `Object#public_send`, `Object#__send__` +- [ ] `Module#method_added`, `singleton_method_added` hooks +- [ ] Hooks: `included`, `extended`, `inherited`, `prepended` +- [ ] Internal-DSL classic program: `lib/ruby/tests/programs/dsl.rb` + +### Phase 6 — stdlib drive +- [ ] `Enumerable` mixin: `each` (abstract), `map`, `select`/`filter`, `reject`, `reduce`/`inject`, `each_with_index`, `each_with_object`, `take`, `drop`, `take_while`, `drop_while`, `find`/`detect`, `find_index`, `any?`, `all?`, `none?`, `one?`, `count`, `min`, `max`, `min_by`, `max_by`, `sort`, `sort_by`, `group_by`, `partition`, `chunk`, `each_cons`, `each_slice`, `flat_map`, `lazy` +- [ ] `Comparable` mixin: `<=>`, `<`, `<=`, `>`, `>=`, `==`, `between?`, `clamp` +- [ ] `Array`: indexing, slicing, `push`/`pop`/`shift`/`unshift`, `concat`, `flatten`, `compact`, `uniq`, `sort`, `reverse`, `zip`, `dig`, `pack`/`unpack` (deferred) +- [ ] `Hash`: `[]`, `[]=`, `delete`, `merge`, `each_pair`, `keys`, `values`, `to_a`, `dig`, `fetch`, default values, default proc +- [ ] `Range`: `each`, `step`, `cover?`, `include?`, `size`, `min`, `max` +- [ ] `String`: indexing, slicing, `split`, `gsub` (string-arg version, regex deferred), `sub`, `upcase`, `downcase`, `strip`, `chomp`, `chars`, `bytes`, `to_i`, `to_f`, `to_sym`, `*`, `+`, `<<`, format with `%` +- [ ] `Integer`: `times`, `upto`, `downto`, `step`, `digits`, `gcd`, `lcm` +- [ ] Drive corpus to 200+ green + +## Progress log + +_Newest first._ + +- _(none yet)_ + +## Blockers + +- _(none yet)_ diff --git a/plans/tcl-on-sx.md b/plans/tcl-on-sx.md new file mode 100644 index 00000000..ab472686 --- /dev/null +++ b/plans/tcl-on-sx.md @@ -0,0 +1,127 @@ +# Tcl-on-SX: uplevel/upvar = stack-walking delcc, everything-is-a-string + +The headline showcase is **uplevel/upvar** — Tcl's superpower for defining your own control structures. `uplevel` evaluates a script in the *caller's* stack frame; `upvar` aliases a variable in the caller. On a normal language host this requires deep VM cooperation; on SX it falls out of the env-chain made first-class via captured continuations. Plus the *Dodekalogue* (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity. + +End-state goal: Tcl 8.6-flavoured subset, the Dodekalogue parser, namespaces, `try`/`catch`/`return -code`, `coroutine` (built on fibers), classic programs that show off uplevel-driven DSLs, ~150 hand-written tests. + +## Scope decisions (defaults — override by editing before we spawn) + +- **Syntax:** Tcl 8.6 surface. The 12-rule Dodekalogue. Brace-quoted scripts deferred-evaluate; double-quoted ones substitute. +- **Conformance:** "Reads like Tcl, runs like Tcl." Slice of Tcl's own test suite, not full TCT. +- **Test corpus:** custom + curated `tcl-tests/` slice. Plus classic programs: define-your-own `for-each-line`, expression-language compiler-in-Tcl, fiber-based event loop. +- **Out of scope:** Tk, sockets beyond a stub, threads (mapped to `coroutine` only), `package require` of binary loadables, `dde`/`registry` Windows shims, full `clock format` locale support. +- **Channels:** `puts` and `gets` on `stdout`/`stdin`/`stderr`; `open` on regular files; no async I/O beyond what `coroutine` gives. + +## Ground rules + +- **Scope:** only touch `lib/tcl/**` and `plans/tcl-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib//**`. Tcl primitives go in `lib/tcl/runtime.sx`. +- **SX files:** use `sx-tree` MCP tools only. +- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes. + +## Architecture sketch + +``` +Tcl source + │ + ▼ +lib/tcl/tokenizer.sx — the Dodekalogue: words, [..], ${..}, "..", {..}, ;, \n, \, # + │ + ▼ +lib/tcl/parser.sx — list-of-words AST (script = list of commands; command = list of words) + │ + ▼ +lib/tcl/transpile.sx — AST → SX AST (entry: tcl-eval-script) + │ + ▼ +lib/tcl/runtime.sx — env stack, command table, uplevel/upvar, coroutines, BIFs +``` + +Core mapping: +- **Value** = string. Internally we cache a "shimmer" representation (list, dict, integer, double) for performance, but every value can be re-stringified. +- **Variable** = entry in current frame's env. Frames form a stack; level-0 is the global frame. +- **Command** = entry in command table; first word of any list dispatches into it. User-defined via `proc`. Built-ins are SX functions registered in the table. +- **Frame** = `{:locals (dict) :level n :parent frame}`. Each `proc` call pushes a frame; commands run in current frame. +- **`uplevel #N script`** = walk frame chain to absolute level N (or relative if no `#`); evaluate script in that frame's env. +- **`upvar [#N] varname localname`** = bind `localname` in the current frame as an alias to `varname` in the level-N frame (env-chain delegate). +- **`return -code N`** = control flow as integers: 0=ok, 1=error, 2=return, 3=break, 4=continue. `catch` traps any non-zero; `try` adds named handlers. +- **`coroutine`** = fiber on top of `perform`/`cek-resume`. `yield`/`yieldto` suspend; calling the coroutine command resumes. +- **List / dict** = list-shaped string ("element1 element2 …") with a cached parsed form. Modifications dirty the string cache. + +## Roadmap + +### Phase 1 — tokenizer + parser (the Dodekalogue) +- [ ] Tokenizer applying the 12 rules: + 1. Commands separated by `;` or newlines + 2. Words separated by whitespace within a command + 3. Double-quoted words: `\` escapes + `[…]` + `${…}` + `$var` substitution + 4. Brace-quoted words: literal, no substitution; brace count must balance + 5. Argument expansion: `{*}list` + 6. Command substitution: `[script]` evaluates script, takes its return value + 7. Variable substitution: `$name`, `${name}`, `$arr(idx)`, `$arr($i)` + 8. Backslash substitution: `\n`, `\t`, `\\`, `\xNN`, `\uNNNN`, `\` continues + 9. Comments: `#` only at the start of a command + 10. Order of substitution is left-to-right, single-pass + 11. Substitutions don't recurse — substituted text is not re-parsed + 12. The result of any substitution is the value, not a new script +- [ ] Parser: script = list of commands; command = list of words; word = literal string + list of substitutions +- [ ] Unit tests in `lib/tcl/tests/parse.sx` + +### Phase 2 — sequential eval + core commands +- [ ] `tcl-eval-script`: walk command list, dispatch each first-word into command table +- [ ] Core commands: `set`, `unset`, `incr`, `append`, `lappend`, `puts`, `gets`, `expr`, `if`, `while`, `for`, `foreach`, `switch`, `break`, `continue`, `return`, `error`, `eval`, `subst`, `format`, `scan` +- [ ] `expr` is its own mini-language — operator precedence, function calls (`sin`, `sqrt`, `pow`, `abs`, `int`, `double`), variable substitution, command substitution +- [ ] String commands: `string length`, `string index`, `string range`, `string compare`, `string match`, `string toupper`, `string tolower`, `string trim`, `string map`, `string repeat`, `string first`, `string last`, `string is`, `string cat` +- [ ] List commands: `list`, `lindex`, `lrange`, `llength`, `lreverse`, `lsearch`, `lsort`, `lsort -integer/-real/-dictionary`, `lreplace`, `linsert`, `concat`, `split`, `join` +- [ ] Dict commands: `dict create`, `dict get`, `dict set`, `dict unset`, `dict exists`, `dict keys`, `dict values`, `dict size`, `dict for`, `dict update`, `dict merge` +- [ ] 60+ tests in `lib/tcl/tests/eval.sx` + +### Phase 3 — proc + uplevel + upvar (THE SHOWCASE) +- [ ] `proc name args body` — register user-defined command; args supports defaults `{name default}` and rest `args` +- [ ] Frame stack: each proc call pushes a frame with locals dict; pop on return +- [ ] `uplevel ?level? script` — evaluate `script` in level-N frame's env; default level is 1 (caller). `#0` is global, `#1` is relative-1 +- [ ] `upvar ?level? otherVar localVar ?…?` — alias localVar to a variable in level-N frame; reads/writes go through the alias +- [ ] `info level`, `info level N`, `info frame`, `info vars`, `info locals`, `info globals`, `info commands`, `info procs`, `info args`, `info body` +- [ ] `global var ?…?` — alias to global frame (sugar for `upvar #0 var var`) +- [ ] `variable name ?value?` — namespace-scoped global +- [ ] Classic programs in `lib/tcl/tests/programs/`: + - [ ] `for-each-line.tcl` — define your own loop construct using `uplevel` + - [ ] `assert.tcl` — assertion macro that reports caller's line + - [ ] `with-temp-var.tcl` — scoped variable rebind via `upvar` +- [ ] `lib/tcl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md` + +### Phase 4 — control flow + error handling +- [ ] `return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value` +- [ ] `catch script ?resultVar? ?optionsVar?` — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict +- [ ] `try script ?on code var body ...? ?trap pattern var body...? ?finally body?` +- [ ] `throw type message` +- [ ] `error message ?info? ?code?` +- [ ] Stack-trace with `errorInfo` / `errorCode` +- [ ] 30+ tests in `lib/tcl/tests/error.sx` + +### Phase 5 — namespaces + ensembles +- [ ] `namespace eval ns body`, `namespace current`, `namespace which`, `namespace import`, `namespace export`, `namespace forget`, `namespace delete` +- [ ] Qualified names: `::ns::cmd`, `::ns::var` +- [ ] Ensembles: `namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }` +- [ ] `namespace path` for resolution chain +- [ ] `proc` and `variable` work inside namespaces + +### Phase 6 — coroutines + drive corpus +- [ ] `coroutine name cmd ?args…?` — start a coroutine; future calls to `name` resume it +- [ ] `yield ?value?` — suspend, return value to resumer +- [ ] `yieldto cmd ?args…?` — symmetric transfer +- [ ] `coroutine` semantics built on fibers (same delcc primitive as Ruby fibers) +- [ ] Classic programs: `event-loop.tcl` — cooperative scheduler with multiple coroutines +- [ ] System: `clock seconds`, `clock format`, `clock scan` (subset) +- [ ] File I/O: `open`, `close`, `read`, `gets`, `puts -nonewline`, `flush`, `eof`, `seek`, `tell` +- [ ] Drive corpus to 150+ green +- [ ] Idiom corpus — `lib/tcl/tests/idioms.sx` covering classic Welch/Jones idioms + +## Progress log + +_Newest first._ + +- _(none yet)_ + +## Blockers + +- _(none yet)_ diff --git a/scripts/sx-loops-down.sh b/scripts/sx-loops-down.sh index fca473ec..29b777c3 100755 --- a/scripts/sx-loops-down.sh +++ b/scripts/sx-loops-down.sh @@ -30,7 +30,7 @@ fi if [ "$CLEAN" = "1" ]; then cd "$(dirname "$0")/.." - for lang in lua prolog forth erlang haskell js hs smalltalk; do + for lang in lua prolog forth erlang haskell js hs smalltalk common-lisp apl ruby tcl; do wt="$WORKTREE_BASE/$lang" if [ -d "$wt" ]; then git worktree remove --force "$wt" 2>/dev/null || rm -rf "$wt" @@ -39,5 +39,5 @@ if [ "$CLEAN" = "1" ]; then done git worktree prune echo "Worktree branches (loops/) are preserved. Delete manually if desired:" - echo " git branch -D loops/lua loops/prolog loops/forth loops/erlang loops/haskell loops/js loops/hs loops/smalltalk" + echo " git branch -D loops/lua loops/prolog loops/forth loops/erlang loops/haskell loops/js loops/hs loops/smalltalk loops/common-lisp loops/apl loops/ruby loops/tcl" fi diff --git a/scripts/sx-loops-up.sh b/scripts/sx-loops-up.sh index 3a93a3b1..bf75a60c 100755 --- a/scripts/sx-loops-up.sh +++ b/scripts/sx-loops-up.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# Spawn 8 claude sessions in tmux, one per language loop. +# Spawn 12 claude sessions in tmux, one per language loop. # Each runs in its own git worktree rooted at /root/rose-ash-loops/, # on branch loops/. No two loops share a working tree, so there's # zero risk of file collisions between languages. @@ -9,7 +9,7 @@ # # After the script prints done: # tmux a -t sx-loops -# Ctrl-B + to switch (0=lua ... 7=smalltalk) +# Ctrl-B + to switch (0=lua ... 11=tcl) # Ctrl-B + d to detach (loops keep running, SSH-safe) # # Stop: ./scripts/sx-loops-down.sh @@ -39,8 +39,12 @@ declare -A BRIEFING=( [js]=loop.md [hs]=hs-loop.md [smalltalk]=smalltalk-loop.md + [common-lisp]=common-lisp-loop.md + [apl]=apl-loop.md + [ruby]=ruby-loop.md + [tcl]=tcl-loop.md ) -ORDER=(lua prolog forth erlang haskell js hs smalltalk) +ORDER=(lua prolog forth erlang haskell js hs smalltalk common-lisp apl ruby tcl) mkdir -p "$WORKTREE_BASE" @@ -61,13 +65,13 @@ for lang in "${ORDER[@]}"; do fi done -# Create tmux session with 7 windows, each cwd in its worktree +# Create tmux session with one window per language, each cwd in its worktree tmux new-session -d -s "$SESSION" -n "${ORDER[0]}" -c "$WORKTREE_BASE/${ORDER[0]}" for lang in "${ORDER[@]:1}"; do tmux new-window -t "$SESSION" -n "$lang" -c "$WORKTREE_BASE/$lang" done -echo "Starting 8 claude sessions..." +echo "Starting ${#ORDER[@]} claude sessions..." for lang in "${ORDER[@]}"; do tmux send-keys -t "$SESSION:$lang" "claude" C-m done @@ -90,10 +94,10 @@ for lang in "${ORDER[@]}"; do done echo "" -echo "Done. 8 loops started in tmux session '$SESSION', each in its own worktree." +echo "Done. ${#ORDER[@]} loops started in tmux session '$SESSION', each in its own worktree." echo "" echo " Attach: tmux a -t $SESSION" -echo " Switch: Ctrl-B <0..7> (0=lua 1=prolog 2=forth 3=erlang 4=haskell 5=js 6=hs 7=smalltalk)" +echo " Switch: Ctrl-B <0..11> (0=lua 1=prolog 2=forth 3=erlang 4=haskell 5=js 6=hs 7=smalltalk 8=common-lisp 9=apl 10=ruby 11=tcl)" echo " List: Ctrl-B w" echo " Detach: Ctrl-B d" echo " Stop: ./scripts/sx-loops-down.sh"