giles
fb72c4ab9c
Plans + briefings for four new language loops, each with a delcc/JIT showcase that the runtime already supports natively: - common-lisp — conditions + restarts on delimited continuations - apl — rank-polymorphic primitives + 6 operators on the JIT - ruby — fibers as delcc, blocks/yield as escape continuations - tcl — uplevel/upvar via first-class env chain, the Dodekalogue Launcher scripts now spawn 12 windows (was 8).
128 lines
8.4 KiB
Markdown
128 lines
8.4 KiB
Markdown
# Tcl-on-SX: uplevel/upvar = stack-walking delcc, everything-is-a-string
|
|
|
|
The headline showcase is **uplevel/upvar** — Tcl's superpower for defining your own control structures. `uplevel` evaluates a script in the *caller's* stack frame; `upvar` aliases a variable in the caller. On a normal language host this requires deep VM cooperation; on SX it falls out of the env-chain made first-class via captured continuations. Plus the *Dodekalogue* (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity.
|
|
|
|
End-state goal: Tcl 8.6-flavoured subset, the Dodekalogue parser, namespaces, `try`/`catch`/`return -code`, `coroutine` (built on fibers), classic programs that show off uplevel-driven DSLs, ~150 hand-written tests.
|
|
|
|
## Scope decisions (defaults — override by editing before we spawn)
|
|
|
|
- **Syntax:** Tcl 8.6 surface. The 12-rule Dodekalogue. Brace-quoted scripts deferred-evaluate; double-quoted ones substitute.
|
|
- **Conformance:** "Reads like Tcl, runs like Tcl." Slice of Tcl's own test suite, not full TCT.
|
|
- **Test corpus:** custom + curated `tcl-tests/` slice. Plus classic programs: define-your-own `for-each-line`, expression-language compiler-in-Tcl, fiber-based event loop.
|
|
- **Out of scope:** Tk, sockets beyond a stub, threads (mapped to `coroutine` only), `package require` of binary loadables, `dde`/`registry` Windows shims, full `clock format` locale support.
|
|
- **Channels:** `puts` and `gets` on `stdout`/`stdin`/`stderr`; `open` on regular files; no async I/O beyond what `coroutine` gives.
|
|
|
|
## Ground rules
|
|
|
|
- **Scope:** only touch `lib/tcl/**` and `plans/tcl-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. Tcl primitives go in `lib/tcl/runtime.sx`.
|
|
- **SX files:** use `sx-tree` MCP tools only.
|
|
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.
|
|
|
|
## Architecture sketch
|
|
|
|
```
|
|
Tcl source
|
|
│
|
|
▼
|
|
lib/tcl/tokenizer.sx — the Dodekalogue: words, [..], ${..}, "..", {..}, ;, \n, \, #
|
|
│
|
|
▼
|
|
lib/tcl/parser.sx — list-of-words AST (script = list of commands; command = list of words)
|
|
│
|
|
▼
|
|
lib/tcl/transpile.sx — AST → SX AST (entry: tcl-eval-script)
|
|
│
|
|
▼
|
|
lib/tcl/runtime.sx — env stack, command table, uplevel/upvar, coroutines, BIFs
|
|
```
|
|
|
|
Core mapping:
|
|
- **Value** = string. Internally we cache a "shimmer" representation (list, dict, integer, double) for performance, but every value can be re-stringified.
|
|
- **Variable** = entry in current frame's env. Frames form a stack; level-0 is the global frame.
|
|
- **Command** = entry in command table; first word of any list dispatches into it. User-defined via `proc`. Built-ins are SX functions registered in the table.
|
|
- **Frame** = `{:locals (dict) :level n :parent frame}`. Each `proc` call pushes a frame; commands run in current frame.
|
|
- **`uplevel #N script`** = walk frame chain to absolute level N (or relative if no `#`); evaluate script in that frame's env.
|
|
- **`upvar [#N] varname localname`** = bind `localname` in the current frame as an alias to `varname` in the level-N frame (env-chain delegate).
|
|
- **`return -code N`** = control flow as integers: 0=ok, 1=error, 2=return, 3=break, 4=continue. `catch` traps any non-zero; `try` adds named handlers.
|
|
- **`coroutine`** = fiber on top of `perform`/`cek-resume`. `yield`/`yieldto` suspend; calling the coroutine command resumes.
|
|
- **List / dict** = list-shaped string ("element1 element2 …") with a cached parsed form. Modifications dirty the string cache.
|
|
|
|
## Roadmap
|
|
|
|
### Phase 1 — tokenizer + parser (the Dodekalogue)
|
|
- [ ] Tokenizer applying the 12 rules:
|
|
1. Commands separated by `;` or newlines
|
|
2. Words separated by whitespace within a command
|
|
3. Double-quoted words: `\` escapes + `[…]` + `${…}` + `$var` substitution
|
|
4. Brace-quoted words: literal, no substitution; brace count must balance
|
|
5. Argument expansion: `{*}list`
|
|
6. Command substitution: `[script]` evaluates script, takes its return value
|
|
7. Variable substitution: `$name`, `${name}`, `$arr(idx)`, `$arr($i)`
|
|
8. Backslash substitution: `\n`, `\t`, `\\`, `\xNN`, `\uNNNN`, `\<newline>` continues
|
|
9. Comments: `#` only at the start of a command
|
|
10. Order of substitution is left-to-right, single-pass
|
|
11. Substitutions don't recurse — substituted text is not re-parsed
|
|
12. The result of any substitution is the value, not a new script
|
|
- [ ] Parser: script = list of commands; command = list of words; word = literal string + list of substitutions
|
|
- [ ] Unit tests in `lib/tcl/tests/parse.sx`
|
|
|
|
### Phase 2 — sequential eval + core commands
|
|
- [ ] `tcl-eval-script`: walk command list, dispatch each first-word into command table
|
|
- [ ] Core commands: `set`, `unset`, `incr`, `append`, `lappend`, `puts`, `gets`, `expr`, `if`, `while`, `for`, `foreach`, `switch`, `break`, `continue`, `return`, `error`, `eval`, `subst`, `format`, `scan`
|
|
- [ ] `expr` is its own mini-language — operator precedence, function calls (`sin`, `sqrt`, `pow`, `abs`, `int`, `double`), variable substitution, command substitution
|
|
- [ ] String commands: `string length`, `string index`, `string range`, `string compare`, `string match`, `string toupper`, `string tolower`, `string trim`, `string map`, `string repeat`, `string first`, `string last`, `string is`, `string cat`
|
|
- [ ] List commands: `list`, `lindex`, `lrange`, `llength`, `lreverse`, `lsearch`, `lsort`, `lsort -integer/-real/-dictionary`, `lreplace`, `linsert`, `concat`, `split`, `join`
|
|
- [ ] Dict commands: `dict create`, `dict get`, `dict set`, `dict unset`, `dict exists`, `dict keys`, `dict values`, `dict size`, `dict for`, `dict update`, `dict merge`
|
|
- [ ] 60+ tests in `lib/tcl/tests/eval.sx`
|
|
|
|
### Phase 3 — proc + uplevel + upvar (THE SHOWCASE)
|
|
- [ ] `proc name args body` — register user-defined command; args supports defaults `{name default}` and rest `args`
|
|
- [ ] Frame stack: each proc call pushes a frame with locals dict; pop on return
|
|
- [ ] `uplevel ?level? script` — evaluate `script` in level-N frame's env; default level is 1 (caller). `#0` is global, `#1` is relative-1
|
|
- [ ] `upvar ?level? otherVar localVar ?…?` — alias localVar to a variable in level-N frame; reads/writes go through the alias
|
|
- [ ] `info level`, `info level N`, `info frame`, `info vars`, `info locals`, `info globals`, `info commands`, `info procs`, `info args`, `info body`
|
|
- [ ] `global var ?…?` — alias to global frame (sugar for `upvar #0 var var`)
|
|
- [ ] `variable name ?value?` — namespace-scoped global
|
|
- [ ] Classic programs in `lib/tcl/tests/programs/`:
|
|
- [ ] `for-each-line.tcl` — define your own loop construct using `uplevel`
|
|
- [ ] `assert.tcl` — assertion macro that reports caller's line
|
|
- [ ] `with-temp-var.tcl` — scoped variable rebind via `upvar`
|
|
- [ ] `lib/tcl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`
|
|
|
|
### Phase 4 — control flow + error handling
|
|
- [ ] `return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value`
|
|
- [ ] `catch script ?resultVar? ?optionsVar?` — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict
|
|
- [ ] `try script ?on code var body ...? ?trap pattern var body...? ?finally body?`
|
|
- [ ] `throw type message`
|
|
- [ ] `error message ?info? ?code?`
|
|
- [ ] Stack-trace with `errorInfo` / `errorCode`
|
|
- [ ] 30+ tests in `lib/tcl/tests/error.sx`
|
|
|
|
### Phase 5 — namespaces + ensembles
|
|
- [ ] `namespace eval ns body`, `namespace current`, `namespace which`, `namespace import`, `namespace export`, `namespace forget`, `namespace delete`
|
|
- [ ] Qualified names: `::ns::cmd`, `::ns::var`
|
|
- [ ] Ensembles: `namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }`
|
|
- [ ] `namespace path` for resolution chain
|
|
- [ ] `proc` and `variable` work inside namespaces
|
|
|
|
### Phase 6 — coroutines + drive corpus
|
|
- [ ] `coroutine name cmd ?args…?` — start a coroutine; future calls to `name` resume it
|
|
- [ ] `yield ?value?` — suspend, return value to resumer
|
|
- [ ] `yieldto cmd ?args…?` — symmetric transfer
|
|
- [ ] `coroutine` semantics built on fibers (same delcc primitive as Ruby fibers)
|
|
- [ ] Classic programs: `event-loop.tcl` — cooperative scheduler with multiple coroutines
|
|
- [ ] System: `clock seconds`, `clock format`, `clock scan` (subset)
|
|
- [ ] File I/O: `open`, `close`, `read`, `gets`, `puts -nonewline`, `flush`, `eof`, `seek`, `tell`
|
|
- [ ] Drive corpus to 150+ green
|
|
- [ ] Idiom corpus — `lib/tcl/tests/idioms.sx` covering classic Welch/Jones idioms
|
|
|
|
## Progress log
|
|
|
|
_Newest first._
|
|
|
|
- _(none yet)_
|
|
|
|
## Blockers
|
|
|
|
- _(none yet)_
|