# Tcl-on-SX: uplevel/upvar = stack-walking delcc, everything-is-a-string

The headline showcase is **uplevel/upvar** — Tcl's superpower for defining your own control structures. `uplevel` evaluates a script in the *caller's* stack frame; `upvar` aliases a variable in the caller. On a normal language host this requires deep VM cooperation; on SX it falls out of the env-chain made first-class via captured continuations. Plus the *Dodekalogue* (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity.

End-state goal: Tcl 8.6-flavoured subset, the Dodekalogue parser, namespaces, `try`/`catch`/`return -code`, `coroutine` (built on fibers), classic programs that show off uplevel-driven DSLs, ~150 hand-written tests.

## Scope decisions (defaults — override by editing before we spawn)

- **Syntax:** Tcl 8.6 surface. The 12-rule Dodekalogue. Brace-quoted scripts deferred-evaluate; double-quoted ones substitute.
- **Conformance:** "Reads like Tcl, runs like Tcl." Slice of Tcl's own test suite, not full TCT.
- **Test corpus:** custom + curated `tcl-tests/` slice. Plus classic programs: define-your-own `for-each-line`, expression-language compiler-in-Tcl, fiber-based event loop.
- **Out of scope:** Tk, sockets beyond a stub, threads (mapped to `coroutine` only), `package require` of binary loadables, `dde`/`registry` Windows shims, full `clock format` locale support.
- **Channels:** `puts` and `gets` on `stdout`/`stdin`/`stderr`; `open` on regular files; no async I/O beyond what `coroutine` gives.

## Ground rules

- **Scope:** only touch `lib/tcl/**` and `plans/tcl-on-sx.md`. Don't edit `spec/`, `hosts/`, `shared/`, or any other `lib/<lang>/**`. Tcl primitives go in `lib/tcl/runtime.sx`.
- **SX files:** use `sx-tree` MCP tools only.
- **Commits:** one feature per commit. Keep `## Progress log` updated and tick roadmap boxes.

## Architecture sketch

```
Tcl source
    │
    ▼
lib/tcl/tokenizer.sx   — the Dodekalogue: words, [..], ${..}, "..", {..}, ;, \n, \, #
    │
    ▼
lib/tcl/parser.sx      — list-of-words AST (script = list of commands; command = list of words)
    │
    ▼
lib/tcl/transpile.sx   — AST → SX AST (entry: tcl-eval-script)
    │
    ▼
lib/tcl/runtime.sx     — env stack, command table, uplevel/upvar, coroutines, BIFs
```

Core mapping:
- **Value** = string. Internally we cache a "shimmer" representation (list, dict, integer, double) for performance, but every value can be re-stringified.
- **Variable** = entry in current frame's env. Frames form a stack; level-0 is the global frame.
- **Command** = entry in command table; first word of any list dispatches into it. User-defined via `proc`. Built-ins are SX functions registered in the table.
- **Frame** = `{:locals (dict) :level n :parent frame}`. Each `proc` call pushes a frame; commands run in current frame.
- **`uplevel #N script`** = walk frame chain to absolute level N (or relative if no `#`); evaluate script in that frame's env.
- **`upvar [#N] varname localname`** = bind `localname` in the current frame as an alias to `varname` in the level-N frame (env-chain delegate).
- **`return -code N`** = control flow as integers: 0=ok, 1=error, 2=return, 3=break, 4=continue. `catch` traps any non-zero; `try` adds named handlers.
- **`coroutine`** = fiber on top of `perform`/`cek-resume`. `yield`/`yieldto` suspend; calling the coroutine command resumes.
- **List / dict** = list-shaped string ("element1 element2 …") with a cached parsed form. Modifications dirty the string cache.

## Roadmap

### Phase 1 — tokenizer + parser (the Dodekalogue)
- [x] Tokenizer applying the 12 rules:
  1. Commands separated by `;` or newlines
  2. Words separated by whitespace within a command
  3. Double-quoted words: `\` escapes + `[…]` + `${…}` + `$var` substitution
  4. Brace-quoted words: literal, no substitution; brace count must balance
  5. Argument expansion: `{*}list`
  6. Command substitution: `[script]` evaluates script, takes its return value
  7. Variable substitution: `$name`, `${name}`, `$arr(idx)`, `$arr($i)`
  8. Backslash substitution: `\n`, `\t`, `\\`, `\xNN`, `\uNNNN`, `\<newline>` continues
  9. Comments: `#` only at the start of a command
  10. Order of substitution is left-to-right, single-pass
  11. Substitutions don't recurse — substituted text is not re-parsed
  12. The result of any substitution is the value, not a new script
- [x] Parser: script = list of commands; command = list of words; word = literal string + list of substitutions
- [x] Unit tests in `lib/tcl/tests/parse.sx`

### Phase 2 — sequential eval + core commands
- [x] `tcl-eval-script`: walk command list, dispatch each first-word into command table
- [x] Core commands: `set`, `unset`, `incr`, `append`, `lappend`, `puts`, `gets`, `expr`, `if`, `while`, `for`, `foreach`, `switch`, `break`, `continue`, `return`, `error`, `eval`, `subst`, `format`, `scan`
- [x] `expr` is its own mini-language — operator precedence, function calls (`sin`, `sqrt`, `pow`, `abs`, `int`, `double`), variable substitution, command substitution
- [x] String commands: `string length`, `string index`, `string range`, `string compare`, `string match`, `string toupper`, `string tolower`, `string trim`, `string map`, `string repeat`, `string first`, `string last`, `string is`, `string cat`
- [x] List commands: `list`, `lindex`, `lrange`, `llength`, `lreverse`, `lsearch`, `lsort`, `lsort -integer/-real/-dictionary`, `lreplace`, `linsert`, `concat`, `split`, `join`
- [x] Dict commands: `dict create`, `dict get`, `dict set`, `dict unset`, `dict exists`, `dict keys`, `dict values`, `dict size`, `dict for`, `dict update`, `dict merge`
- [x] 60+ tests in `lib/tcl/tests/eval.sx`

### Phase 3 — proc + uplevel + upvar (THE SHOWCASE)
- [x] `proc name args body` — register user-defined command; args supports defaults `{name default}` and rest `args`
- [x] Frame stack: each proc call pushes a frame with locals dict; pop on return
- [x] `uplevel ?level? script` — evaluate `script` in level-N frame's env; default level is 1 (caller). `#0` is global, `#1` is relative-1
- [x] `upvar ?level? otherVar localVar ?…?` — alias localVar to a variable in level-N frame; reads/writes go through the alias
- [x] `info level`, `info level N`, `info frame`, `info vars`, `info locals`, `info globals`, `info commands`, `info procs`, `info args`, `info body`
- [x] `global var ?…?` — alias to global frame (sugar for `upvar #0 var var`)
- [x] `variable name ?value?` — namespace-scoped global
- [x] Classic programs in `lib/tcl/tests/programs/`:
  - [x] `for-each-line.tcl` — define your own loop construct using `uplevel`
  - [x] `assert.tcl` — assertion macro that reports caller's line
  - [x] `with-temp-var.tcl` — scoped variable rebind via `upvar`
- [x] `lib/tcl/conformance.sh` + runner, `scoreboard.json` + `scoreboard.md`

### Phase 4 — control flow + error handling
- [x] `return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value`
- [x] `catch script ?resultVar? ?optionsVar?` — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict
- [x] `try script ?on code var body ...? ?trap pattern var body...? ?finally body?`
- [x] `throw type message`
- [x] `error message ?info? ?code?`
- [x] Stack-trace with `errorInfo` / `errorCode`
- [x] 30+ tests in `lib/tcl/tests/error.sx`

### Phase 5 — namespaces + ensembles
- [x] `namespace eval ns body`, `namespace current`, `namespace which`, `namespace import`, `namespace export`, `namespace forget`, `namespace delete`
- [x] Qualified names: `::ns::cmd`, `::ns::var`
- [x] Ensembles: `namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }`
- [x] `namespace path` for resolution chain
- [x] `proc` and `variable` work inside namespaces

### Phase 6 — coroutines + drive corpus
- [x] `coroutine name cmd ?args…?` — start a coroutine; future calls to `name` resume it
- [x] `yield ?value?` — suspend, return value to resumer
- [x] `yieldto cmd ?args…?` — symmetric transfer
- [x] `coroutine` semantics built on fibers (same delcc primitive as Ruby fibers)
- [x] Classic programs: `event-loop.tcl` — cooperative scheduler with multiple coroutines
- [x] System: `clock seconds`, `clock format`, `clock scan` (subset)
- [x] File I/O: `open`, `close`, `read`, `gets`, `puts -nonewline`, `flush`, `eof`, `seek`, `tell`
- [x] Drive corpus to 150+ green
- [x] Idiom corpus — `lib/tcl/tests/idioms.sx` covering classic Welch/Jones idioms

## Progress log

_Newest first._

- 2026-05-06: Phase 6 coroutines+clock+file+idioms — generator coroutines, clock/file stubs, 20 coroutine + 20 idiom tests, event-loop.tcl, 329 tests green
- 2026-05-06: Phase 5 namespaces+ensembles — namespace eval/current/which/exists/delete/import/ensemble, qualified names, 289 tests green (22 new namespace tests)
- 2026-05-06: Phase 4 error handling — catch/try/throw/return-code/errorinfo/errorcode, 267 tests green (39 new error tests)
- 2026-05-06: Phase 3 conformance.sh + classic programs — 3/3 PASS (for-each-line/assert/with-temp-var), 228 tests green
- 2026-05-06: Phase 3 proc+uplevel+upvar+info+global — frame stack, isolated proc scope, alias-following var access, 225 tests green (67 parse + 158 eval)
- 2026-05-06: Phase 2 dict commands — 13 subcommands (create/get/set/unset/exists/keys/values/size/for/update/merge/incr/append), 206 tests green (67 parse + 139 eval)
- 2026-05-06: Phase 2 list commands — 12 commands (list/lindex/lrange/llength/lreverse/lsearch/lsort/lreplace/linsert/concat/split/join), 182 tests green (67 parse + 115 eval)
- 2026-05-06: Phase 2 string commands — 16 subcommands (length/index/range/compare/match/toupper/tolower/trim/map/repeat/first/last/is/cat), 156 tests green (67 parse + 89 eval)
- 2026-05-06: Phase 2 expr mini-language — recursive descent parser, operator precedence, parens, unary ops, pow/sqrt/abs/max/min/int/double, 127 tests green (67 parse + 60 eval)
- 2026-04-26: Phase 2 core commands — if/while/for/foreach/switch/break/continue/return/error/unset/lappend/eval/expr + :code control flow, 107 tests green (67 parse + 40 eval)
- 2026-04-26: Phase 2 eval engine — `lib/tcl/runtime.sx`, tcl-eval-script + set/puts/incr/append, 87 tests green (67 parse + 20 eval)
- 2026-04-25: Phase 1 parser — `lib/tcl/parser.sx`, word-simple?/word-literal helpers, 67 tests green, commit 6ee05259
- 2026-04-25: Phase 1 tokenizer (Dodekalogue) — `lib/tcl/tokenizer.sx`, 52 tests green, commit 666e29d5

## Blockers

- _(none yet)_
