eval.sx adds the `guard` syntactic operator with R7RS-compliant
clause dispatch: var binds to raised value in a fresh child env;
clauses tried in order; `else` is catch-all; no match re-raises.
Implementation uses a "catch-once-then-handle-outside" pattern to
avoid the handler self-raise loop:
outcome = host-guard {body} ;; tag raise vs success
if outcome was raise:
try clauses → either result or sentinel
if sentinel: re-raise OUTSIDE the host-guard scope
runtime.sx binds R7RS exception primitives:
- raise V
- error MSG IRRITANT... → {:scm-error MSG :irritants LIST}
- error-object?, error-object-message, error-object-irritants
- with-exception-handler HANDLER THUNK
(same outcome-sentinel pattern — handler's own raises propagate
outward instead of re-entering)
12 tests cover: catch on raise, predicate dispatch, else catch-all,
no-error pass-through, first-clause-wins, re-raise-on-no-match,
error-object construction and accessors.
232 total Scheme tests now (62 + 23 + 49 + 78 + 20).
scheme-standard-env binds:
- call/cc — primary
- call-with-current-continuation — alias
Implementation wraps SX's host call/cc, presenting the captured
continuation k as a Scheme procedure that accepts a single value
(or a list of values for multi-arg invocation). Single-shot
escape semantics: when k is invoked, control jumps out of the
surrounding call/cc form. Multi-shot re-entry isn't safely
testable without delimited-continuation infrastructure (the
captured continuation re-enters indefinitely if invoked after
the call/cc returns) — deferred to a follow-up commit if needed.
Tests cover:
- No-escape return value
- Escape past arithmetic frames
- Detect/early-exit idiom over for-each
- Procedure? on the captured k
220 total Scheme tests now (62 + 23 + 49 + 78 + 8).
lib/scheme/runtime.sx — full R7RS-base surface:
- Arithmetic: variadic +/-/*//, abs, min, max, modulo, quotient,
remainder. Predicates zero?/positive?/negative?.
- Comparison: chained =/</>/<=/>=.
- Type predicates: number?/boolean?/symbol?/string?/char?/vector?/
null?/pair?/procedure?/not.
- List: cons/car/cdr/list/length/reverse/append.
- Higher-order: map/filter/fold-left/fold-right/for-each/apply.
These re-enter scheme-apply to invoke user-supplied procs.
- String: string-length/string=?/string-append/substring.
- Char: char=?.
- Vector: vector/vector-length/vector-ref/vector->list/list->vector/
make-vector.
- Equality: eqv?/equal?/eq? (all = under the hood for now).
Built via small adapters: scm-unary, scm-binary, scm-fold (variadic
left-fold with identity + one-arity special), scm-chain (n-ary
chained comparison).
**Bugfix in eval.sx set! handler.** The :else branch had two
expressions `(dict-set! ...) val` — SX cond branches don't run
multiple expressions, they return nil silently (or evaluate only
the first, depending on shape). Wrapped in (begin ...) to force
sequential execution. This fix also unblocks 4 set!-dependent
tests in lib/scheme/tests/syntax.sx that were silently raising
during load (and thus not counted) — syntax test count jumps
from 45 → 49.
Classic programs verified:
- factorial 10 → 3628800
- fib 10 → 55
- recursive list reverse → working
- sum of squares via fold-left + map → 55
212 total Scheme tests: parse 62 + eval 23 + syntax 49 + runtime 78.
All green.
The env-as-value section in runtime tests demonstrates
scheme-standard-env IS a refl-env? — kit primitives operate on it
directly, confirming the third-consumer adoption with zero adapter.
Adds the rest of the standard syntactic operators, all built on the
existing eval/closure infrastructure from Phase 3:
- let — parallel bindings in fresh child env; values evaluated in
outer env (RHS sees pre-let bindings only). Multi-body via
scheme-eval-body.
- let* — sequential bindings, each in a nested child env; later
bindings see earlier ones.
- cond — clauses walked in order; first truthy test wins. `else`
symbol is the catch-all. Test-only clauses (no body) return the
test value. Scheme truthiness: only #f is false.
- when / unless — single-test conditional execution, multi-body
body via scheme-eval-body.
- and / or — short-circuit boolean. Empty `(and)` = true,
`(or)` = false. Both return the actual value at the point
of short-circuit (not coerced to bool), matching R7RS.
130 total Scheme tests (62 parse + 23 eval + 45 syntax). The
Scheme port is now self-hosting enough to write any non-stdlib
program — factorial, list operations via primitives, closures
with mutable state, all working.
Next phase: standard env (runtime.sx) with variadic +/-, list
ops as Scheme-visible applicatives.
eval.sx grows: five new syntactic operators wired via the table-
driven dispatch from Phase 2. lambda creates closures
{:scm-tag :closure :params :rest :body :env} that capture the
static env; scheme-apply-closure binds formals + rest-arg, evaluates
multi-expression body in (extend static-env), returns last value.
Supports lambda formals shapes:
() → no args
(a b c) → fixed arity
args → bare symbol; binds all call-args as a list
Dotted-pair tail (a b . rest) deferred until parser supports it.
define has both flavours:
(define name expr) — direct binding
(define (name . formals) body...) — lambda sugar
set! walks the env chain via refl-env-find-frame, mutates at the
binding's source frame (no shadowing). Raises on unbound name.
24 new tests in lib/scheme/tests/syntax.sx, including:
- Factorial 5 → 120 and 10 → 3628800 (recursion + closures)
- make-counter via closed-over set! state
- Curried (((curry+ 1) 2) 3) → 6
- (lambda args args) rest-arg binding
- Multi-body lambdas with internal define
109 total Scheme tests (62 parse + 23 eval + 24 syntax).
lib/scheme/eval.sx — R7RS evaluator skeleton:
- Self-evaluating: numbers, booleans, characters, vectors, strings
- Symbol lookup: refl-env-lookup
- Lists: syntactic-operator table dispatch, else applicative call
- Table-driven syntactic ops (Phase 2 wires `quote` only; full set
in Phase 3)
- Apply: callable host fn or scheme closure (closure stub for Phase 3)
scheme-make-env / scheme-env-bind! / etc. are THIN ALIASES for the
refl-env-* primitives from lib/guest/reflective/env.sx. No adapter
cfg needed — Scheme's lexical-scope semantics ARE the canonical
wire shape. This is the THIRD CONSUMER for env.sx after Kernel and
Tcl + Smalltalk's variant adapters; the first to use it without
any bridging code. Validates the kit handles canonical-shape
adoption with zero ceremony.
23 tests in lib/scheme/tests/eval.sx cover literals, symbol
lookup with parent-chain shadowing, quote (special form + sugar),
primitive application with nested calls, and an env-as-value
section explicitly demonstrating the kit primitives work on
Scheme envs.
85 total Scheme tests (62 parse + 23 eval).
chisel: consumes-env (third consumer for lib/guest/reflective/env.sx).
lib/guest/reflective/env.sx — added refl-env-find-frame-with (returns
the scope where NAME is bound, or nil). Needed by consumers like
Smalltalk that mutate variables at the source frame rather than
shadowing at the current one. Also added refl-env-find-frame for
the canonical shape.
lib/smalltalk/eval.sx — new st-frame-cfg adapter for the kit.
st-lookup-local now delegates parent-walk to refl-env-find-frame-with
while preserving its Smalltalk-flavoured {:found :value :frame}
return shape (which is used to mutate at the binding's source
frame, not the current one).
lib/smalltalk/test.sh + compare.sh — load lib/guest/reflective/env.sx
before lib/smalltalk/eval.sx.
Three genuinely different wire shapes now share the parent-walk:
- Kernel: {:refl-tag :env :bindings :parent} mutable bindings
- Tcl: {:level :locals :parent} functional update
- Smalltalk: {:self :method-class :locals :parent mutable bindings,
:return-k :active-cell} rich metadata
All three consumers' full test suites unchanged: Smalltalk 847/847,
Kernel 322/322, Tcl 427/427. The cfg adapter pattern (modelled after
lib/guest/match.sx) cleanly handles all three.
Phase 2 of the lib-guest-reflective extraction.
lib/tcl/runtime.sx — frame-lookup and frame-set-top now delegate to
refl-env-lookup-or-nil-with and refl-env-bind!-with via a new
tcl-frame-cfg adapter. Tcl keeps its existing {:level :locals :parent}
frame shape unchanged; the cfg bridges it to the kit's generic
algorithms. Functional update semantics preserved (cfg's :bind!
returns the new frame via assoc).
lib/tcl/test.sh + conformance.sh — load lib/guest/reflective/env.sx
before lib/tcl/runtime.sx.
Both consumers' full test suites unchanged:
- Tcl: 427/427 (parse 67, eval 169, error 39, namespace 22, coro 20,
idiom 110)
- Kernel: 322/322 across 7 suites
The extraction is now real: two consumers, two genuinely different
wire shapes (mutable canonical vs functional frame), sharing the
parent-walk algorithm via cfg adapter — same pattern as
lib/guest/match.sx.
Phase 1 of the lib-guest-reflective extraction plan.
lib/guest/reflective/env.sx — canonical wire shape
{:refl-tag :env :bindings DICT :parent ENV-OR-NIL} with mutable
defaults (dict-set!), plus *-with adapter-cfg variants for consumers
with their own shape (modelled after lib/guest/match.sx). 13 forms,
~5 KB.
lib/kernel/eval.sx — env block collapses from ~30 lines to 6 thin
wrappers (kernel-env? = refl-env?, etc.). No semantic change; envs
now carry :refl-tag :env instead of :knl-tag :env. All 322 Kernel
tests pass unchanged across 7 suites (parse 62, eval 36, vau 38,
standard 127, encap 19, hygiene 26, metacircular 14).
Next: Phase 2 — Tcl adapter cfg in lib/tcl/runtime.sx using
refl-env-lookup-with against the existing :level/:locals/:parent
frame shape.
Five type predicates (number?, string?, list?, boolean?, symbol?).
New tests/metacircular.sx: m-eval defined in Kernel walks expressions
itself, recursing on applicative-call args and delegating to host
eval only for operatives and symbol lookup. 14 demo tests.
The demo surfaced a real bug: map/filter/reduce called kernel-combine
on applicative head-vals directly, which re-evaluates already-
evaluated element values; nested-list elements crashed. Fix: extracted
knl-apply-op (unwrap-applicative-or-pass-through) and use it in all
three combinators before kernel-combine. Mirrors apply's approach.
Added knl-apply-op as a proposed entry in the reflective combiner.sx
API. 322 tests total.
(apply F (list V1 V2 V3)) ≡ (F V1 V2 V3). Unwrap applicative first to
skip auto-eval (args are values), then kernel-combine with the
underlying operative. Universal pattern in reflective Lisps —
sketched into the combiner.sx API. 296 tests total.
Added kernel-make-primitive-applicative-with-env in eval.sx — IMPL
receives (args dyn-env), needed by combinators that re-enter the
evaluator. map/filter/reduce in runtime.sx use it to call user-supplied
combiners on each element with the caller's dynamic env preserved.
Sketched the env-blind vs env-aware applicative split as a new entry
in the proposed combiner.sx reflective API. 289 tests total.
Standard Kernel control flow. $cond walks clauses in order with `else`
catch-all; clauses past the first match are NOT evaluated. $when/$unless
are simple guards. 12 tests, 242 total.
kernel-quasiquote-operative walks the template via mutually-recursive
knl-quasi-walk ↔ knl-quasi-walk-list. $unquote forms eval in dyn-env;
$unquote-splicing splices list-valued results. No depth tracking
(nested quasiquotes flatten). 8 new tests, 230 total. Sketched the
universal reflective quoting kit API for the eventual Phase 7 extraction.
:body slot holds a LIST of forms now (was single expression). New
knl-eval-body in eval.sx evaluates each form in sequence, returning
the last. $vau and $lambda accept (formals env-param body...) /
(formals body...). No $sequence dependency. 223 tests total.
Parser now reads 'expr, \`expr, ,expr, ,@expr as the four standard
shorthands. Quote uses existing $quote operative; quasiquote /
unquote / unquote-splicing recognised but not yet expanded at runtime
(left for first consumer to drive). 218 tests total across six suites.
Hygiene-by-default was already present: user operatives close over
static-env and bind formals + body $define!s in (extend STATIC-ENV),
caller's env untouched. $let evaluates values in caller env, binds
in fresh child env, runs body there. $define-in! explicitly targets
an env. Full scope-set / frame-stamp hygiene is research-grade
and documented as deferred future work in the reflective API notes.
kernel-eval/kernel-combine dispatch on tagged values: operatives see
un-evaluated args + dynamic env; applicatives evaluate args then recurse.
No hardcoded special forms — $if/$quote tested as ordinary operatives
built on the fly. Pure-SX env representation
{:knl-tag :env :bindings DICT :parent P}, surfaced as a candidate
lib/guest/reflective/env.sx API since SX make-env is HTTP-mode only.
Tcl tokenizer treats $::g-name as $::g + literal -name, so the var
lookup fails. Renamed test vars to ::gname / ::nval (no hyphens).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the watchdog-bump approach with an automated check. The next 5× (or
worse) substrate regression will trip the alarm at build time instead of
hiding behind a deadline bump and only being noticed weeks later.
Components:
* lib/perf-smoke.sx — four micro-benchmarks chosen for distinct substrate
failure modes: function-call dispatch (fib), env construction (let-chain),
HO-form dispatch + lambda creation (map-sq), TCO + primitive dispatch
(tail-loop). Warm-up pass populates JIT cache before the timed pass so we
measure the steady state.
* scripts/perf-smoke.sh — pipes lib/perf-smoke.sx to sx_server.exe, parses
per-bench wall-time, asserts each is within FACTOR× of the recorded
reference (default 5×). `--update` rewrites the reference in-place.
* scripts/sx-build-all.sh — perf-smoke wired in as a post-step after JS
tests. Hard fail if any benchmark regressed beyond budget.
Reference numbers: minimum across 6 back-to-back runs on this dev machine
under typical concurrent-loop contention (load ~9, 2 vCPU, 7.6 GiB RAM,
OCaml 5.2.0, architecture @ 92f6f187). Documented in
plans/jit-perf-regression.md including how to update them.
The 5× factor is chosen so contention noise (~1–2× variance) doesn't trigger
false alarms but a real ≥5× substrate regression — the kind that motivated
this whole investigation — fails the build immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflict in lib/tcl/test.sh: architecture had bumped `timeout 2400 → 7200`,
this branch had restored it to `timeout 300` based on the Phase 1
quiet-machine measurement (376/376 in 57.8s wall, 16.3s user). Resolved by
keeping `timeout 300` — the 7200s bump was preemptive against contention,
not against an actual substrate regression. Phase 1 confirms the original
180s deadline is comfortable; 300s gives 5× headroom for moderate noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 of the jit-perf-regression plan reproduced and quantified the alleged
30× substrate slowdown across 5 guests (tcl, lua, erlang, prolog, haskell). On
a quiet machine all five suites pass cleanly:
tcl test.sh 57.8s wall, 16.3s user, 376/376 ✓
lua test.sh 27.3s wall, 4.2s user, 185/185 ✓
erlang conformance 3m25s wall, 36.8s user, 530/530 ✓ (needs ≥600s budget)
prolog conformance 3m54s wall, 1m08s user, 590/590 ✓
haskell conformance 6m59s wall, 2m37s user, 156/156 ✓
Per-test user-time at architecture HEAD vs pre-substrate-merge baseline
(83dbb595) is essentially flat (tcl 0.83×, lua 1.4×, prolog 0.82×). The
symptoms reported in the plan (test timeouts, OOMs, 30-min hangs) were heavy
CPU contention from concurrent loops + one undersized internal `timeout 120`
in erlang's conformance script. There is no substrate regression to bisect.
Changes:
* lib/tcl/test.sh: `timeout 2400` → `timeout 300`. The original 180s deadline
is comfortable on a quiet machine (3.1× headroom); 300s gives some safety
margin for moderate contention without masking real regressions.
* lib/erlang/conformance.sh: `timeout 120` → `timeout 600`. The 120s budget
was actually too tight for the full 9-suite chain even before this work.
* lib/erlang/scoreboard.{json,md}: 0/0 → 530/530 — populated by a successful
conformance run with the new deadline. The previous 0/0 was a stale
artefact of the run timing out before parsing any markers.
* plans/jit-perf-regression.md: full Phase 1 progress log including
per-guest perf table, quiet-machine re-measurement, and conclusion.
Phases 2–4 (bisect, diagnose, fix) skipped — there is no substrate regression
to find. Phase 6 (perf-regression alarm) still planned to catch the next
quadratic blow-up early instead of via watchdog bumps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vwait used frame-lookup which doesn't honor `::` global routing. So
`vwait ::done` after `set ::done fired` (where set routes to root frame)
never saw the var change in the local frame, looping forever.
Added tcl-vwait-lookup helper that mirrors tcl-var-get's `::` routing
but returns nil instead of erroring on missing vars.
Was the deadlock that hung the full test suite past test 32.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous version put (define _throttle-ms ...) (define _debounce-ms ...)
(define _strip-throttle-debounce ...) inside emit-on's body, redefining
them on every call to emit-on. The kernel JIT-compiled the helper fn
fresh each invocation, doubling compile time across the suite and
pushing many tests over their wall-clock deadline (35 cumulative-only
timeouts in the latest batched run, up from 0).
Move the three definitions to module-level. Use (set! _throttle-ms nil)
(set! _debounce-ms nil) at the top of emit-on to reset state for each
call. JIT compilation of _strip-throttle-debounce now happens once.
Verified: hs-upstream-expressions/dom-scope went from 18/20 (with two
state-related timeouts) back to 20/20, suite wall-time 232s → 75s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lib/hyperscript/tokenizer.sx — added cursor + follow-set wrapper over
the existing flat-list tokenize output:
hs-stream src → {:tokens :pos :follows :last-match :last-ws}
hs-stream-current s → next non-WS token (skips WS, captures :last-ws)
hs-stream-match s value → consume if value matches & not in follow set
hs-stream-match-type s ...types → consume if upstream type name matches
hs-stream-match-any s ...names → consume if value matches any name
hs-stream-match-any-op s ...ops → consume if op token & value matches
hs-stream-peek s value n → look n non-WS tokens ahead, no consume
hs-stream-consume-until s marker → collect tokens until marker
hs-stream-consume-until-ws s → collect until next whitespace
hs-stream-push-follow! / pop-follow!
hs-stream-push-follows! / pop-follows! n
hs-stream-clear-follows! → saved / restore-follows! saved
hs-stream-last-match / last-ws
hs-stream-type-map maps our lowercase type names to upstream's
("ident" → "IDENTIFIER", "number" → "NUMBER", etc.) so type-based
matching works against upstream test expectations.
13 tokenizer-stream tests now pass; 30/30 in hs-upstream-core/tokenizer.
Skips remaining: 5 (down from 18).
- 2 template-component scope tests
- 1 async event dispatch (until event keyword works)
- left for later: needs more architectural work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
parser.sx parse-toggle-cmd: when seeing 'toggle .foo for', peek the
following two tokens. If they are '<ident> in', it is a for-in loop
and toggle does NOT consume 'for' as a duration clause. Restores the
trailing for-in to the command list.
parser.sx parse-on (handler modifiers): recognize 'throttled at <ms>'
and 'debounced at <ms>' as handler modifiers. Captured as :throttle /
:debounce kwargs in the on-form parts list.
compiler.sx emit-on: pre-extract :throttle / :debounce from parts via
new _strip-throttle-debounce helper before scan-on, then wrap the built
handler with (hs-throttle! handler ms) or (hs-debounce! handler ms).
runtime.sx: hs-throttle! — closure with __hs-last-fire timestamp,
fires immediately and drops events arriving within ms of the last fire.
hs-debounce! — closure with __hs-timer, clears any pending timer and
schedules a new setTimeout(handler, ms) so only the last burst event
fires.
Both formerly-architectural skips now pass:
- "toggle does not consume a following for-in loop"
- "throttled at <time> drops events within the window"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Brings the architecture branch (559 commits ahead — R7RS step 4-6, JIT
expansion, host_error wrapping, bytecode compiler, etc.) into the
loops/haskell line of work. Conflict in lib/haskell/conformance.sh:
architecture replaced the inline driver with a thin wrapper delegating
to lib/guest/conformance.sh + a config file. Resolved by taking the
wrapper and extending lib/haskell/conformance.conf with all programs
added under loops/haskell (caesar, runlength-str, showadt, showio,
partial, statistics, newton, wordfreq, mapgraph, uniquewords, setops,
shapes, person, config, counter, accumulate, safediv, trycatch) plus
adding map.sx and set.sx to PRELOADS.
plans/haskell-completeness.md gains three new follow-up phases:
- Phase 17 — parser polish (`(x :: Int)` annotations, mid-file imports)
- Phase 18 — one ambitious conformance program (lambda-calc / Dijkstra /
JSON parser candidate list)
- Phase 19 — conformance speed (batch all suites in one sx_server
process to compress the 25-min run to single-digit minutes)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hk-bind-exceptions! in eval.sx registers throwIO, throw, evaluate, catch,
try, handle, displayException. SomeException constructor pre-registered
in runtime.sx (arity 1, type SomeException).
throwIO and the existing error primitive both raise via SX `raise` with a
uniform "hk-error: msg" string. catch/try/handle parse it back into a
SomeException via hk-exception-of, which strips nested
'Unhandled exception: "..."' host wraps (CEK's host_error formatter) and
the "hk-error: " prefix.
catch and handle evaluate the handler outside the guard scope (build an
"ok"/"exn" outcome tag inside guard, then dispatch outside) so that a
re-throw from the handler propagates past this catch — matching Haskell
semantics rather than infinite-looping in the same guard.
14 unit tests in tests/exceptions.sx (catch success, catch error, try
Right/Left, handle, throwIO + catch/try, evaluate, nested catch, do-bind
through catch, branch on try result, IORef-mutating handler).
Conformance: safediv.hs (8/8) and trycatch.hs (8/8). Scoreboard now
285/285 tests, 36/36 programs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trivial wrapper: apl-run-file = apl-run ∘ file-read, where
file-read is built-in to OCaml SX.
Tests verify primes.apl, life.apl, quicksort.apl all parse
end-to-end (their last form is a :dfn AST). Source-then-call
test confirms the loaded file's defined fn is callable, even
when the algorithm itself can't fully execute (primes' inline
⍵ rebinding still missing — :glyph-token, not :name-token).
Parser: :name clause now detects 'name ← rhs' patterns inside
expressions. When seen, consumes the remaining tokens as RHS,
parses recursively, and emits a (:assign-expr name parsed-rhs)
value segment.
Eval-ast :dyad and :monad: when the right operand is an
:assign-expr node, capture the binding into env before
evaluating the left operand. This realises the primes idiom:
apl-run "(2 = +⌿ 0 = a ∘.| a) / a ← ⍳ 30"
→ 2 3 5 7 11 13 17 19 23 29
Also: top-level x←5 now evaluates to scalar 5 (apl-eval-ast
:assign just unwraps to its RHS value).
Caveat: ⍵-rebinding (the original primes.apl uses
'⍵←⍳⍵') is a :glyph-token; only :name-tokens are handled.
A regular variable name (like 'a') works.
