lib/guest/reflective/env.sx — added refl-env-find-frame-with (returns
the scope where NAME is bound, or nil). Needed by consumers like
Smalltalk that mutate variables at the source frame rather than
shadowing at the current one. Also added refl-env-find-frame for
the canonical shape.
lib/smalltalk/eval.sx — new st-frame-cfg adapter for the kit.
st-lookup-local now delegates parent-walk to refl-env-find-frame-with
while preserving its Smalltalk-flavoured {:found :value :frame}
return shape (which is used to mutate at the binding's source
frame, not the current one).
lib/smalltalk/test.sh + compare.sh — load lib/guest/reflective/env.sx
before lib/smalltalk/eval.sx.
Three genuinely different wire shapes now share the parent-walk:
- Kernel: {:refl-tag :env :bindings :parent} mutable bindings
- Tcl: {:level :locals :parent} functional update
- Smalltalk: {:self :method-class :locals :parent mutable bindings,
:return-k :active-cell} rich metadata
All three consumers' full test suites unchanged: Smalltalk 847/847,
Kernel 322/322, Tcl 427/427. The cfg adapter pattern (modelled after
lib/guest/match.sx) cleanly handles all three.
Phase 2 of the lib-guest-reflective extraction.
lib/tcl/runtime.sx — frame-lookup and frame-set-top now delegate to
refl-env-lookup-or-nil-with and refl-env-bind!-with via a new
tcl-frame-cfg adapter. Tcl keeps its existing {:level :locals :parent}
frame shape unchanged; the cfg bridges it to the kit's generic
algorithms. Functional update semantics preserved (cfg's :bind!
returns the new frame via assoc).
lib/tcl/test.sh + conformance.sh — load lib/guest/reflective/env.sx
before lib/tcl/runtime.sx.
Both consumers' full test suites unchanged:
- Tcl: 427/427 (parse 67, eval 169, error 39, namespace 22, coro 20,
idiom 110)
- Kernel: 322/322 across 7 suites
The extraction is now real: two consumers, two genuinely different
wire shapes (mutable canonical vs functional frame), sharing the
parent-walk algorithm via cfg adapter — same pattern as
lib/guest/match.sx.
Phase 1 of the lib-guest-reflective extraction plan.
lib/guest/reflective/env.sx — canonical wire shape
{:refl-tag :env :bindings DICT :parent ENV-OR-NIL} with mutable
defaults (dict-set!), plus *-with adapter-cfg variants for consumers
with their own shape (modelled after lib/guest/match.sx). 13 forms,
~5 KB.
lib/kernel/eval.sx — env block collapses from ~30 lines to 6 thin
wrappers (kernel-env? = refl-env?, etc.). No semantic change; envs
now carry :refl-tag :env instead of :knl-tag :env. All 322 Kernel
tests pass unchanged across 7 suites (parse 62, eval 36, vau 38,
standard 127, encap 19, hygiene 26, metacircular 14).
Next: Phase 2 — Tcl adapter cfg in lib/tcl/runtime.sx using
refl-env-lookup-with against the existing :level/:locals/:parent
frame shape.
Five type predicates (number?, string?, list?, boolean?, symbol?).
New tests/metacircular.sx: m-eval defined in Kernel walks expressions
itself, recursing on applicative-call args and delegating to host
eval only for operatives and symbol lookup. 14 demo tests.
The demo surfaced a real bug: map/filter/reduce called kernel-combine
on applicative head-vals directly, which re-evaluates already-
evaluated element values; nested-list elements crashed. Fix: extracted
knl-apply-op (unwrap-applicative-or-pass-through) and use it in all
three combinators before kernel-combine. Mirrors apply's approach.
Added knl-apply-op as a proposed entry in the reflective combiner.sx
API. 322 tests total.
(apply F (list V1 V2 V3)) ≡ (F V1 V2 V3). Unwrap applicative first to
skip auto-eval (args are values), then kernel-combine with the
underlying operative. Universal pattern in reflective Lisps —
sketched into the combiner.sx API. 296 tests total.
Added kernel-make-primitive-applicative-with-env in eval.sx — IMPL
receives (args dyn-env), needed by combinators that re-enter the
evaluator. map/filter/reduce in runtime.sx use it to call user-supplied
combiners on each element with the caller's dynamic env preserved.
Sketched the env-blind vs env-aware applicative split as a new entry
in the proposed combiner.sx reflective API. 289 tests total.
Standard Kernel control flow. $cond walks clauses in order with `else`
catch-all; clauses past the first match are NOT evaluated. $when/$unless
are simple guards. 12 tests, 242 total.
kernel-quasiquote-operative walks the template via mutually-recursive
knl-quasi-walk ↔ knl-quasi-walk-list. $unquote forms eval in dyn-env;
$unquote-splicing splices list-valued results. No depth tracking
(nested quasiquotes flatten). 8 new tests, 230 total. Sketched the
universal reflective quoting kit API for the eventual Phase 7 extraction.
:body slot holds a LIST of forms now (was single expression). New
knl-eval-body in eval.sx evaluates each form in sequence, returning
the last. $vau and $lambda accept (formals env-param body...) /
(formals body...). No $sequence dependency. 223 tests total.
Parser now reads 'expr, \`expr, ,expr, ,@expr as the four standard
shorthands. Quote uses existing $quote operative; quasiquote /
unquote / unquote-splicing recognised but not yet expanded at runtime
(left for first consumer to drive). 218 tests total across six suites.
Hygiene-by-default was already present: user operatives close over
static-env and bind formals + body $define!s in (extend STATIC-ENV),
caller's env untouched. $let evaluates values in caller env, binds
in fresh child env, runs body there. $define-in! explicitly targets
an env. Full scope-set / frame-stamp hygiene is research-grade
and documented as deferred future work in the reflective API notes.
kernel-eval/kernel-combine dispatch on tagged values: operatives see
un-evaluated args + dynamic env; applicatives evaluate args then recurse.
No hardcoded special forms — $if/$quote tested as ordinary operatives
built on the fly. Pure-SX env representation
{:knl-tag :env :bindings DICT :parent P}, surfaced as a candidate
lib/guest/reflective/env.sx API since SX make-env is HTTP-mode only.
Tcl tokenizer treats $::g-name as $::g + literal -name, so the var
lookup fails. Renamed test vars to ::gname / ::nval (no hyphens).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the watchdog-bump approach with an automated check. The next 5× (or
worse) substrate regression will trip the alarm at build time instead of
hiding behind a deadline bump and only being noticed weeks later.
Components:
* lib/perf-smoke.sx — four micro-benchmarks chosen for distinct substrate
failure modes: function-call dispatch (fib), env construction (let-chain),
HO-form dispatch + lambda creation (map-sq), TCO + primitive dispatch
(tail-loop). Warm-up pass populates JIT cache before the timed pass so we
measure the steady state.
* scripts/perf-smoke.sh — pipes lib/perf-smoke.sx to sx_server.exe, parses
per-bench wall-time, asserts each is within FACTOR× of the recorded
reference (default 5×). `--update` rewrites the reference in-place.
* scripts/sx-build-all.sh — perf-smoke wired in as a post-step after JS
tests. Hard fail if any benchmark regressed beyond budget.
Reference numbers: minimum across 6 back-to-back runs on this dev machine
under typical concurrent-loop contention (load ~9, 2 vCPU, 7.6 GiB RAM,
OCaml 5.2.0, architecture @ 92f6f187). Documented in
plans/jit-perf-regression.md including how to update them.
The 5× factor is chosen so contention noise (~1–2× variance) doesn't trigger
false alarms but a real ≥5× substrate regression — the kind that motivated
this whole investigation — fails the build immediately.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Conflict in lib/tcl/test.sh: architecture had bumped `timeout 2400 → 7200`,
this branch had restored it to `timeout 300` based on the Phase 1
quiet-machine measurement (376/376 in 57.8s wall, 16.3s user). Resolved by
keeping `timeout 300` — the 7200s bump was preemptive against contention,
not against an actual substrate regression. Phase 1 confirms the original
180s deadline is comfortable; 300s gives 5× headroom for moderate noise.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 1 of the jit-perf-regression plan reproduced and quantified the alleged
30× substrate slowdown across 5 guests (tcl, lua, erlang, prolog, haskell). On
a quiet machine all five suites pass cleanly:
tcl test.sh 57.8s wall, 16.3s user, 376/376 ✓
lua test.sh 27.3s wall, 4.2s user, 185/185 ✓
erlang conformance 3m25s wall, 36.8s user, 530/530 ✓ (needs ≥600s budget)
prolog conformance 3m54s wall, 1m08s user, 590/590 ✓
haskell conformance 6m59s wall, 2m37s user, 156/156 ✓
Per-test user-time at architecture HEAD vs pre-substrate-merge baseline
(83dbb595) is essentially flat (tcl 0.83×, lua 1.4×, prolog 0.82×). The
symptoms reported in the plan (test timeouts, OOMs, 30-min hangs) were heavy
CPU contention from concurrent loops + one undersized internal `timeout 120`
in erlang's conformance script. There is no substrate regression to bisect.
Changes:
* lib/tcl/test.sh: `timeout 2400` → `timeout 300`. The original 180s deadline
is comfortable on a quiet machine (3.1× headroom); 300s gives some safety
margin for moderate contention without masking real regressions.
* lib/erlang/conformance.sh: `timeout 120` → `timeout 600`. The 120s budget
was actually too tight for the full 9-suite chain even before this work.
* lib/erlang/scoreboard.{json,md}: 0/0 → 530/530 — populated by a successful
conformance run with the new deadline. The previous 0/0 was a stale
artefact of the run timing out before parsing any markers.
* plans/jit-perf-regression.md: full Phase 1 progress log including
per-guest perf table, quiet-machine re-measurement, and conclusion.
Phases 2–4 (bisect, diagnose, fix) skipped — there is no substrate regression
to find. Phase 6 (perf-regression alarm) still planned to catch the next
quadratic blow-up early instead of via watchdog bumps.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
vwait used frame-lookup which doesn't honor `::` global routing. So
`vwait ::done` after `set ::done fired` (where set routes to root frame)
never saw the var change in the local frame, looping forever.
Added tcl-vwait-lookup helper that mirrors tcl-var-get's `::` routing
but returns nil instead of erroring on missing vars.
Was the deadlock that hung the full test suite past test 32.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous version put (define _throttle-ms ...) (define _debounce-ms ...)
(define _strip-throttle-debounce ...) inside emit-on's body, redefining
them on every call to emit-on. The kernel JIT-compiled the helper fn
fresh each invocation, doubling compile time across the suite and
pushing many tests over their wall-clock deadline (35 cumulative-only
timeouts in the latest batched run, up from 0).
Move the three definitions to module-level. Use (set! _throttle-ms nil)
(set! _debounce-ms nil) at the top of emit-on to reset state for each
call. JIT compilation of _strip-throttle-debounce now happens once.
Verified: hs-upstream-expressions/dom-scope went from 18/20 (with two
state-related timeouts) back to 20/20, suite wall-time 232s → 75s.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
lib/hyperscript/tokenizer.sx — added cursor + follow-set wrapper over
the existing flat-list tokenize output:
hs-stream src → {:tokens :pos :follows :last-match :last-ws}
hs-stream-current s → next non-WS token (skips WS, captures :last-ws)
hs-stream-match s value → consume if value matches & not in follow set
hs-stream-match-type s ...types → consume if upstream type name matches
hs-stream-match-any s ...names → consume if value matches any name
hs-stream-match-any-op s ...ops → consume if op token & value matches
hs-stream-peek s value n → look n non-WS tokens ahead, no consume
hs-stream-consume-until s marker → collect tokens until marker
hs-stream-consume-until-ws s → collect until next whitespace
hs-stream-push-follow! / pop-follow!
hs-stream-push-follows! / pop-follows! n
hs-stream-clear-follows! → saved / restore-follows! saved
hs-stream-last-match / last-ws
hs-stream-type-map maps our lowercase type names to upstream's
("ident" → "IDENTIFIER", "number" → "NUMBER", etc.) so type-based
matching works against upstream test expectations.
13 tokenizer-stream tests now pass; 30/30 in hs-upstream-core/tokenizer.
Skips remaining: 5 (down from 18).
- 2 template-component scope tests
- 1 async event dispatch (until event keyword works)
- left for later: needs more architectural work
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
parser.sx parse-toggle-cmd: when seeing 'toggle .foo for', peek the
following two tokens. If they are '<ident> in', it is a for-in loop
and toggle does NOT consume 'for' as a duration clause. Restores the
trailing for-in to the command list.
parser.sx parse-on (handler modifiers): recognize 'throttled at <ms>'
and 'debounced at <ms>' as handler modifiers. Captured as :throttle /
:debounce kwargs in the on-form parts list.
compiler.sx emit-on: pre-extract :throttle / :debounce from parts via
new _strip-throttle-debounce helper before scan-on, then wrap the built
handler with (hs-throttle! handler ms) or (hs-debounce! handler ms).
runtime.sx: hs-throttle! — closure with __hs-last-fire timestamp,
fires immediately and drops events arriving within ms of the last fire.
hs-debounce! — closure with __hs-timer, clears any pending timer and
schedules a new setTimeout(handler, ms) so only the last burst event
fires.
Both formerly-architectural skips now pass:
- "toggle does not consume a following for-in loop"
- "throttled at <time> drops events within the window"
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Brings the architecture branch (559 commits ahead — R7RS step 4-6, JIT
expansion, host_error wrapping, bytecode compiler, etc.) into the
loops/haskell line of work. Conflict in lib/haskell/conformance.sh:
architecture replaced the inline driver with a thin wrapper delegating
to lib/guest/conformance.sh + a config file. Resolved by taking the
wrapper and extending lib/haskell/conformance.conf with all programs
added under loops/haskell (caesar, runlength-str, showadt, showio,
partial, statistics, newton, wordfreq, mapgraph, uniquewords, setops,
shapes, person, config, counter, accumulate, safediv, trycatch) plus
adding map.sx and set.sx to PRELOADS.
plans/haskell-completeness.md gains three new follow-up phases:
- Phase 17 — parser polish (`(x :: Int)` annotations, mid-file imports)
- Phase 18 — one ambitious conformance program (lambda-calc / Dijkstra /
JSON parser candidate list)
- Phase 19 — conformance speed (batch all suites in one sx_server
process to compress the 25-min run to single-digit minutes)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
hk-bind-exceptions! in eval.sx registers throwIO, throw, evaluate, catch,
try, handle, displayException. SomeException constructor pre-registered
in runtime.sx (arity 1, type SomeException).
throwIO and the existing error primitive both raise via SX `raise` with a
uniform "hk-error: msg" string. catch/try/handle parse it back into a
SomeException via hk-exception-of, which strips nested
'Unhandled exception: "..."' host wraps (CEK's host_error formatter) and
the "hk-error: " prefix.
catch and handle evaluate the handler outside the guard scope (build an
"ok"/"exn" outcome tag inside guard, then dispatch outside) so that a
re-throw from the handler propagates past this catch — matching Haskell
semantics rather than infinite-looping in the same guard.
14 unit tests in tests/exceptions.sx (catch success, catch error, try
Right/Left, handle, throwIO + catch/try, evaluate, nested catch, do-bind
through catch, branch on try result, IORef-mutating handler).
Conformance: safediv.hs (8/8) and trycatch.hs (8/8). Scoreboard now
285/285 tests, 36/36 programs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trivial wrapper: apl-run-file = apl-run ∘ file-read, where
file-read is built-in to OCaml SX.
Tests verify primes.apl, life.apl, quicksort.apl all parse
end-to-end (their last form is a :dfn AST). Source-then-call
test confirms the loaded file's defined fn is callable, even
when the algorithm itself can't fully execute (primes' inline
⍵ rebinding still missing — :glyph-token, not :name-token).
Parser: :name clause now detects 'name ← rhs' patterns inside
expressions. When seen, consumes the remaining tokens as RHS,
parses recursively, and emits a (:assign-expr name parsed-rhs)
value segment.
Eval-ast :dyad and :monad: when the right operand is an
:assign-expr node, capture the binding into env before
evaluating the left operand. This realises the primes idiom:
apl-run "(2 = +⌿ 0 = a ∘.| a) / a ← ⍳ 30"
→ 2 3 5 7 11 13 17 19 23 29
Also: top-level x←5 now evaluates to scalar 5 (apl-eval-ast
:assign just unwraps to its RHS value).
Caveat: ⍵-rebinding (the original primes.apl uses
'⍵←⍳⍵') is a :glyph-token; only :name-tokens are handled.
A regular variable name (like 'a') works.
Haskell strings are [Char]. Calling reverse / head / length on a SX raw
string transparently produces a cons-list of char codes (via hk-str-head /
hk-str-tail in runtime.sx), but (==) then compared the original raw string
against the char-code cons-list and always returned False — so
"racecar" == reverse "racecar" was False.
Added hk-try-charlist-to-string and hk-normalize-for-eq in eval.sx; routed
== and /= through hk-normalize-for-eq so a string compares equal to any
cons-list whose elements are valid Unicode code points spelling the same
characters, and "[]" ↔ "".
palindrome.hs lifts from 9/12 → 12/12; conformance 33/34 → 34/34 programs,
266/269 → 269/269 tests.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships the algebra for HM-style type inference, riding on
lib/guest/match.sx (terms + unify) and ast.sx (canonical AST):
• Type constructors: hm-tv, hm-arrow, hm-con, hm-int, hm-bool, hm-string
• Schemes: hm-scheme / hm-monotype + accessors
• Free type-vars: hm-ftv, hm-ftv-scheme, hm-ftv-env
• Substitution: hm-apply, hm-apply-scheme, hm-apply-env, hm-compose
• Generalize / Instantiate (with shared fresh-tv counter)
• hm-fresh-tv (counter is a (list N) the caller threads)
• hm-infer-literal (the only fully-closed inference rule)
24 self-tests in lib/guest/tests/hm.sx covering every function above.
The lambda / app / let inference rules — the substitution-threading
core of Algorithm W — intentionally live in HOST CODE rather than the
kit, because each host's AST shape and substitution-threading idiom
differ subtly enough that forcing one shared assembly here proved
brittle in practice (an earlier inline-assembled hm-infer faulted with
"Not callable: nil" only when defined in the kit, despite working when
inline-eval'd or in a separate file — a load/closure interaction not
worth chasing inside this step's budget). The host gets the algebra
plus a spec; assembly stays close to the AST it reasons over.
PARTIAL — algebra + literal rule shipped; full Algorithm W deferred
to host consumers (haskell/infer.sx, lib/ocaml/types.sx when
OCaml-on-SX Phase 5 lands per the brief's sequencing note). Haskell
infer.sx untouched; haskell scoreboard still 156/156 baseline.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
apl-permutations was doing (append acc <new-perms>) which is
O(|acc|) and acc grows ~N! big — total cost O(N!²).
Swapped to (append <new-perms> acc) — append is O(|first|)
so cost is O((n+1)·N!_prev) per layer, total O(N!). q(7)
went from 32s to 12s; q(8)=92 now finishes well within the
300s timeout, so the queens(8) test is restored.
497/497. Phase 8 complete.
