giles
40f0e73386
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 44s
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
169 lines
7.3 KiB
Markdown
169 lines
7.3 KiB
Markdown
# Tcl-on-SX completion plan — SX capabilities first
|
||
|
||
Tcl phases 1–6 are complete (329/329 tests). This plan covers the remaining
|
||
limitations, ordered by the SX work needed to enable them.
|
||
|
||
## Key audit findings
|
||
|
||
Several apparent gaps are already solved in SX:
|
||
|
||
- **Floats** — SX parses `3.14` natively; `(+ 1.5 2.5) → 4.0`; `str` formats
|
||
with `%g` (compact, no trailing zeros). `floor`/`ceil`/`round`/`truncate`
|
||
all exist in `spec/primitives.sx`.
|
||
- **Regex** — `regexp-match`, `regexp-match-all`, `regexp-replace`,
|
||
`regexp-replace-all`, `regexp-split` are registered OCaml primitives using
|
||
`Re.Pcre` (`hosts/ocaml/lib/sx_primitives.ml`).
|
||
- **`call/cc` multi-shot** — works. `set!` on closed-over vars works. Fibers
|
||
are implementable as a pure SX library.
|
||
- **`perform` user-accessible** — `(perform :foo 42)` from user code suspends
|
||
the evaluator and emits an IO request. The algebraic effects model is
|
||
already half-built.
|
||
- **No `file-read`/`clock-seconds`** — not yet registered as OCaml primitives.
|
||
Only string ports exist. Would need small OCaml additions.
|
||
- **No `env-as-value`** — environments are internal OCaml values, not
|
||
inspectable from SX user code.
|
||
|
||
---
|
||
|
||
## Phase 1 — Zero-cost wins (no SX changes, only `lib/tcl/`)
|
||
|
||
Everything here is pure Tcl implementation work.
|
||
|
||
| Status | Work | Effort | Unlocks in Tcl |
|
||
|---|---|---|---|
|
||
| [x] | Float in `expr` — detect `.` in number tokens, route through float ops instead of `parse-int` | half day | `expr {3.14 * 2}`, `expr {sqrt(2.0)}`, float comparisons |
|
||
| [x] | `regexp pattern str` and `regsub pattern str repl` wrapping existing SX primitives | few hours | pattern matching, text processing |
|
||
| [x] | `apply {args body} ?arg…?` — anonymous proc call | 1 hour | higher-order functions, `lmap` idiom |
|
||
| [x] | `array get/set/names/size/exists/unset` commands | half day | array variables (tokenizer already parses `$arr(key)`) |
|
||
|
||
**Total: ~2 days. Zero SX changes.**
|
||
|
||
---
|
||
|
||
## Phase 2 — `lib/fiber.sx` (pure SX library, no OCaml)
|
||
|
||
| Status | Work |
|
||
|---|---|
|
||
| [x] | Create `lib/fiber.sx` — `make-fiber` / `fiber-resume` / `fiber-done?` |
|
||
| [x] | Rewrite `tcl-cmd-coroutine` to use `make-fiber` (true suspension) |
|
||
|
||
`call/cc` is multi-shot and `set!` on closed-over vars both work. Fibers are
|
||
implementable as a pure SX library using symmetric continuation swapping:
|
||
|
||
```scheme
|
||
; lib/fiber.sx — canonical fiber primitive for all hosted languages
|
||
(define make-fiber
|
||
(fn (thunk)
|
||
(define slot-k nil)
|
||
(define slot-caller nil)
|
||
(define slot-done false)
|
||
(fn (resume-val)
|
||
(call/cc (fn (caller-k)
|
||
(set! slot-caller caller-k)
|
||
(if (nil? slot-k)
|
||
(begin (thunk resume-val) (set! slot-done true) (caller-k nil))
|
||
(slot-k resume-val)))))))
|
||
|
||
(define fiber-yield
|
||
(fn (val)
|
||
(call/cc (fn (k)
|
||
(set! slot-k k)
|
||
(slot-caller val)))))
|
||
```
|
||
|
||
Each coroutine becomes a fiber. `yield` swaps to the caller; calling the
|
||
coroutine name swaps back. True suspension, not eager pre-execution.
|
||
|
||
**Broader value:** Ruby fibers, Python generators, Lua coroutines, async event
|
||
loops, cooperative schedulers all sit on top of the same library.
|
||
|
||
**Alternatively:** `perform` is user-accessible. A Tcl scheduler living outside
|
||
the SX evaluator (the OCaml host or an SX event loop) could catch
|
||
`(perform :fiber-yield val)` and dispatch it — the algebraic effects model,
|
||
already half-built.
|
||
|
||
**Total: 2–3 days. Produces `lib/fiber.sx` as a lasting SX contribution.**
|
||
Tcl coroutines then rewrite using `make-fiber` for true suspension.
|
||
|
||
---
|
||
|
||
## Phase 3 — Small OCaml additions (`sx_primitives.ml`)
|
||
|
||
Each is ~10–20 lines of OCaml. All are useful across the whole platform, not
|
||
just Tcl.
|
||
|
||
| Status | Primitive | OCaml effort | Unlocks |
|
||
|---|---|---|---|
|
||
| [x] | `(file-read path)` → string | tiny | Tcl `open`/`read`, SX scripts reading files |
|
||
| [x] | `(file-write path str)` → nil | tiny | Tcl `open`/`puts` to files |
|
||
| [x] | `(file-exists? path)` → bool | tiny | Tcl `file exists` |
|
||
| [x] | `(file-glob pattern)` → list | small | Tcl `glob` |
|
||
| [x] | `(clock-seconds)` → int | tiny | Tcl `clock seconds` |
|
||
| [x] | `(clock-format n fmt)` → string | small (wraps `strftime`) | Tcl `clock format` |
|
||
|
||
**Total: 1 day. One focused afternoon of OCaml.**
|
||
|
||
---
|
||
|
||
## Phase 4 — env-as-value (architectural) ✓
|
||
|
||
|
||
|
||
`uplevel`/`upvar` required an explicit frame stack because SX environments
|
||
aren't inspectable from user code. Adding:
|
||
|
||
```scheme
|
||
(current-env) ; → env value
|
||
(eval-in-env env expr) ; → result
|
||
(env-lookup env key) ; → value or nil
|
||
(env-extend env key val) ; → new env (non-mutating)
|
||
```
|
||
|
||
...would let `uplevel N` be literally "look up env N levels up, eval in it."
|
||
The Tcl frame stack (hundreds of lines) collapses to ~10 lines.
|
||
|
||
Also benefits: metacircular evaluators, REPL tooling, live debugging (inspect
|
||
any scope), the sx_docs server's eval endpoint.
|
||
|
||
More invasive — touches `sx_types.ml` and `sx_server.ml` — but a meaningful
|
||
architectural improvement worth doing when the moment is right.
|
||
|
||
**Total: 2–3 days. High architectural value, not urgent.**
|
||
|
||
---
|
||
|
||
## Suggested order
|
||
|
||
1. **Phase 1** — immediate Tcl wins, zero risk, proves the approach
|
||
2. **Phase 2** (`lib/fiber.sx`) — the interesting SX work, benefits all hosted languages
|
||
3. **Phase 3** (OCaml primitives) — quick practical completions
|
||
4. **Phase 4** — architectural cleanup when it's worth the invasiveness
|
||
|
||
Phases 1+2+3 ≈ one focused week. Tcl is genuinely complete, and `lib/fiber.sx`
|
||
becomes a lasting SX contribution used by every future hosted language.
|
||
|
||
---
|
||
|
||
## Progress log
|
||
|
||
_Newest first._
|
||
|
||
- 2026-05-06: Phase 4 env-as-value — current-env (special form via Sx_ref.register_special_form), eval-in-env (primitive in setup_evaluator_bridge), env-lookup + env-extend (in setup_env_operations); 5 idiom tests; 342/342 green
|
||
- 2026-05-06: Phase 3 OCaml primitives — file-read/write/append/exists?/glob + clock-seconds/milliseconds/format in sx_primitives.ml + unix dep; tcl-cmd-clock/file wired up; 337/337 green
|
||
- 2026-05-06: Phase 2 coroutine rewrite — `tcl-cmd-coroutine` now creates a `make-fiber`; `tcl-cmd-yield` calls `:coro-yield-fn` (threaded through interp); true suspension; 337/337 green
|
||
- 2026-05-06: Phase 2 fiber.sx — `make-fiber`/`fiber-resume`/`fiber-done?` using call/cc + set!; bidirectional value passing; generator and echo tests pass
|
||
- 2026-05-06: Phase 1 array — `tcl-cmd-array` get/set/names/size/exists/unset; frame-local key scanning with prefix `arrname(`; 337/337 tests green
|
||
- 2026-05-06: Phase 1 apply — `tcl-cmd-apply` wraps `tcl-call-proc`, parses `{args body}` funcList, full frame isolation; 329/329 tests green
|
||
- 2026-05-06: Phase 1 regexp/regsub — `tcl-cmd-regexp`/`tcl-cmd-regsub` wrapping `make-regexp`/`regexp-match`/`regexp-match-all`/`regexp-replace`/`regexp-replace-all`; -nocase/-all/-inline/-all flags; matchVar + subgroup capture; 329/329 tests green
|
||
- 2026-05-06: Phase 1 float expr — `tcl-num-float?`, `tcl-parse-num`, float-aware `tcl-apply-binop`/`tcl-apply-func`/unary-minus/`**`; `sqrt`/`floor`/`ceil`/`round`/`sin`/`cos`/`tan`/`pow`/`exp`/`log` all float-native; 329/329 tests green
|
||
|
||
---
|
||
|
||
## What stays out of scope
|
||
|
||
- `package require` of binary loadables
|
||
- Full `clock format` locale support
|
||
- Tk / GUI
|
||
- Threads (mapped to coroutines only, as planned)
|
||
- Full POSIX file I/O (seek/tell/async) — stubs are fine
|