rose-ash/plans/tcl-on-sx.md

Tcl-on-SX: uplevel/upvar = stack-walking delcc, everything-is-a-string

The headline showcase is uplevel/upvar — Tcl's superpower for defining your own control structures. uplevel evaluates a script in the caller's stack frame; upvar aliases a variable in the caller. On a normal language host this requires deep VM cooperation; on SX it falls out of the env-chain made first-class via captured continuations. Plus the Dodekalogue (12 rules), command-substitution everywhere, and "everything is a string" homoiconicity.

End-state goal: Tcl 8.6-flavoured subset, the Dodekalogue parser, namespaces, try/catch/return -code, coroutine (built on fibers), classic programs that show off uplevel-driven DSLs, ~150 hand-written tests.

Scope decisions (defaults — override by editing before we spawn)

• Syntax: Tcl 8.6 surface. The 12-rule Dodekalogue. Brace-quoted scripts deferred-evaluate; double-quoted ones substitute.
• Conformance: "Reads like Tcl, runs like Tcl." Slice of Tcl's own test suite, not full TCT.
• Test corpus: custom + curated tcl-tests/ slice. Plus classic programs: define-your-own for-each-line, expression-language compiler-in-Tcl, fiber-based event loop.
• Out of scope: Tk, sockets beyond a stub, threads (mapped to coroutine only), package require of binary loadables, dde/registry Windows shims, full clock format locale support.
• Channels: puts and gets on stdout/stdin/stderr; open on regular files; no async I/O beyond what coroutine gives.

Ground rules

• Scope: only touch lib/tcl/** and plans/tcl-on-sx.md. Don't edit spec/, hosts/, shared/, or any other lib/<lang>/**. Tcl primitives go in lib/tcl/runtime.sx.
• SX files: use sx-tree MCP tools only.
• Commits: one feature per commit. Keep ## Progress log updated and tick roadmap boxes.

Architecture sketch

Tcl source
    │
    ▼
lib/tcl/tokenizer.sx   — the Dodekalogue: words, [..], ${..}, "..", {..}, ;, \n, \, #
    │
    ▼
lib/tcl/parser.sx      — list-of-words AST (script = list of commands; command = list of words)
    │
    ▼
lib/tcl/transpile.sx   — AST → SX AST (entry: tcl-eval-script)
    │
    ▼
lib/tcl/runtime.sx     — env stack, command table, uplevel/upvar, coroutines, BIFs

Core mapping:

• Value = string. Internally we cache a "shimmer" representation (list, dict, integer, double) for performance, but every value can be re-stringified.
• Variable = entry in current frame's env. Frames form a stack; level-0 is the global frame.
• Command = entry in command table; first word of any list dispatches into it. User-defined via proc. Built-ins are SX functions registered in the table.
• Frame = {:locals (dict) :level n :parent frame}. Each proc call pushes a frame; commands run in current frame.
• uplevel #N script = walk frame chain to absolute level N (or relative if no #); evaluate script in that frame's env.
• upvar [#N] varname localname = bind localname in the current frame as an alias to varname in the level-N frame (env-chain delegate).
• return -code N = control flow as integers: 0=ok, 1=error, 2=return, 3=break, 4=continue. catch traps any non-zero; try adds named handlers.
• coroutine = fiber on top of perform/cek-resume. yield/yieldto suspend; calling the coroutine command resumes.
• List / dict = list-shaped string ("element1 element2 …") with a cached parsed form. Modifications dirty the string cache.

Roadmap

Phase 1 — tokenizer + parser (the Dodekalogue)

• Tokenizer applying the 12 rules:
  1. Commands separated by ; or newlines
  2. Words separated by whitespace within a command
  3. Double-quoted words: \ escapes + […] + ${…} + $var substitution
  4. Brace-quoted words: literal, no substitution; brace count must balance
  5. Argument expansion: {*}list
  6. Command substitution: [script] evaluates script, takes its return value
  7. Variable substitution: $name, ${name}, $arr(idx), $arr($i)
  8. Backslash substitution: \n, \t, \\, \xNN, \uNNNN, \<newline> continues
  9. Comments: # only at the start of a command
  10. Order of substitution is left-to-right, single-pass
  11. Substitutions don't recurse — substituted text is not re-parsed
  12. The result of any substitution is the value, not a new script
• Parser: script = list of commands; command = list of words; word = literal string + list of substitutions
• Unit tests in lib/tcl/tests/parse.sx

Phase 2 — sequential eval + core commands

• tcl-eval-script: walk command list, dispatch each first-word into command table
• Core commands: set, unset, incr, append, lappend, puts, gets, expr, if, while, for, foreach, switch, break, continue, return, error, eval, subst, format, scan
• expr is its own mini-language — operator precedence, function calls (sin, sqrt, pow, abs, int, double), variable substitution, command substitution
• String commands: string length, string index, string range, string compare, string match, string toupper, string tolower, string trim, string map, string repeat, string first, string last, string is, string cat
• List commands: list, lindex, lrange, llength, lreverse, lsearch, lsort, lsort -integer/-real/-dictionary, lreplace, linsert, concat, split, join
• Dict commands: dict create, dict get, dict set, dict unset, dict exists, dict keys, dict values, dict size, dict for, dict update, dict merge
• 60+ tests in lib/tcl/tests/eval.sx

Phase 3 — proc + uplevel + upvar (THE SHOWCASE)

• proc name args body — register user-defined command; args supports defaults {name default} and rest args
• Frame stack: each proc call pushes a frame with locals dict; pop on return
• uplevel ?level? script — evaluate script in level-N frame's env; default level is 1 (caller). #0 is global, #1 is relative-1
• upvar ?level? otherVar localVar ?…? — alias localVar to a variable in level-N frame; reads/writes go through the alias
• info level, info level N, info frame, info vars, info locals, info globals, info commands, info procs, info args, info body
• global var ?…? — alias to global frame (sugar for upvar #0 var var)
• variable name ?value? — namespace-scoped global
• Classic programs in lib/tcl/tests/programs/:
  • for-each-line.tcl — define your own loop construct using uplevel
  • assert.tcl — assertion macro that reports caller's line
  • with-temp-var.tcl — scoped variable rebind via upvar
• lib/tcl/conformance.sh + runner, scoreboard.json + scoreboard.md

Phase 4 — control flow + error handling

• return -code (ok|error|return|break|continue|N) -errorinfo … -errorcode … -level N value
• catch script ?resultVar? ?optionsVar? — runs script, returns code; sets resultVar to return value/message; optionsVar to the dict
• try script ?on code var body ...? ?trap pattern var body...? ?finally body?
• throw type message
• error message ?info? ?code?
• Stack-trace with errorInfo / errorCode
• 30+ tests in lib/tcl/tests/error.sx

Phase 5 — namespaces + ensembles

• namespace eval ns body, namespace current, namespace which, namespace import, namespace export, namespace forget, namespace delete
• Qualified names: ::ns::cmd, ::ns::var
• Ensembles: namespace ensemble create -map { sub1 cmd1 sub2 cmd2 }
• namespace path for resolution chain
• proc and variable work inside namespaces

Phase 6 — coroutines + drive corpus

• coroutine name cmd ?args…? — start a coroutine; future calls to name resume it
• yield ?value? — suspend, return value to resumer
• yieldto cmd ?args…? — symmetric transfer
• coroutine semantics built on fibers (same delcc primitive as Ruby fibers)
• Classic programs: event-loop.tcl — cooperative scheduler with multiple coroutines
• System: clock seconds, clock format, clock scan (subset)
• File I/O: open, close, read, gets, puts -nonewline, flush, eof, seek, tell
• Drive corpus to 150+ green
• Idiom corpus — lib/tcl/tests/idioms.sx covering classic Welch/Jones idioms

Progress log

Newest first.

• (none yet)

Blockers

• (none yet)