JIT-vs-CEK test parity: both now pass 3938/534 (identical failures).
Three fixes in sx_vm.ml + run_tests.ml:
1. OP_CALL_PRIM: fallback to Sx_primitives.get_primitive when vm.globals
misses. Primitives registered after JIT setup (host-global, host-get,
etc. bound inside run_spec_tests) become resolvable at call time.
2. jit_compile_lambda: early-exit for anonymous lambdas, nested lambdas
(closure has parent — recreated per outer call), and a known-broken
name list: parser combinators, hyperscript parse/compile orchestrators,
test helpers, compile-timeout functions, and hs loop runtime (which
uses guard/raise for break/continue). Lives inside jit_compile_lambda
so both the CEK _jit_try_call_fn hook and VM OP_CALL Lambda path
honor the skip list.
3. run_tests.ml _jit_try_call_fn: catch TIMEOUT during jit_compile_lambda.
Sentinel is set before compile, so subsequent calls skip JIT; this
ensures the first call of a suite also falls back to CEK cleanly when
compile exceeds the 5s test budget.
Also includes run_tests.ml 'reset' form helpers refactor (form-element
reset command) that was pending in the working tree.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When an error occurs during resumed VM execution (after perform/hs-wait),
resume_vm now checks the VM's handler_stack. If a handler exists (from a
compiled guard form's OP_PUSH_HANDLER), it unwinds frames and jumps to
the catch block — exactly like OP_RAISE. This enables try/catch across
async perform/resume boundaries.
The guard form compiles to OP_PUSH_HANDLER which lives on the vm struct
and survives across setTimeout-based async resume. Previously, errors
during resume escaped to the JS console as unhandled exceptions.
Also restored guard in the test runner (was cek-try which doesn't survive
async) and restored error-throwing assertions in run-action.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The guard form (call/cc + handler-bind expansion) doesn't survive async
IO suspension — the CEK continuation from guard's call/cc captures frames
that become invalid after the VM resumes from hs-wait. Replacing guard
with cek-try (which compiles to VM-native OP_PUSH_HANDLER/OP_POP_HANDLER)
avoids the CEK boundary crossing.
The test runner now executes: suspends on hs-wait, resumes, runs test
actions, and test assertions fire correctly. The "Not callable: nil"
error is eliminated. Remaining: test assertion errors from iframe content
not loading fast enough (timing issue, not a framework bug).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Not callable: nil error happens on a stub VM (frames=[], sp=0) during
cek_resume with 12 CEK kont frames. The error is from a reactive signal
subscriber (reset! current ...) that triggers during run vm after resume.
The subscriber callback goes through CEK via cek_call_or_suspend and the
CEK continuation tries to call nil.
This is a reactive subscriber notification issue, not a perform/resume
frame management issue. The VM frames are correctly restored — the error
happens during a synchronous reset! call within the resumed VM execution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: when perform fires inside a VM closure chain (call_closure_reuse),
the caller frames are saved to reuse_stack on the ACTIVE VM. But the
_cek_io_suspend_hook and _cek_eval_lambda_ref create a NEW stub VM for the
VmSuspended exception. On resume, resume_vm runs on the STUB VM which has
an empty reuse_stack — the caller frames are orphaned on the original VM.
Fix: transfer reuse_stack from _active_vm to the stub VM before raising
VmSuspended. This ensures resume_vm -> restore_reuse can find and restore
the caller's frames after async resume via _driveAsync/setTimeout.
Also restore step_limit/step_count refs dropped by bootstrap.py regeneration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _cek_io_suspend_hook creates a stub VM to carry the suspended CEK
state. Previously used empty globals, which caused "Not callable: nil"
when the CEK resume needed platform functions. Now uses _default_vm_globals
(set to _vm_globals by sx_browser.ml) so all platform functions and
definitions are available during resume.
Remaining issue: still getting "resume: Not callable: nil" — the CEK
continuation env may not include letrec bindings from the island body.
The suspension point is inside reload-frame → hs-wait, and the resume
needs to call wait-boot (a letrec binding).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: cek_run_iterative (used by eval_expr/trampoline) raised
"IO suspension in non-IO context" when the CEK hit a perform. This
blocked IO suspension from propagating through nested eval_expr calls
(event handler → trampoline → eval_expr → for-each callback → hs-wait).
Fix: added _cek_io_suspend_hook (Sx_types) that converts CEK suspension
to VmSuspended, set by sx_vm.ml at init. cek_run_iterative now calls the
hook instead of erroring. The VmSuspended propagates to the value_to_js
wrapper which has _driveAsync handling.
+42 test passes (3924→3966), zero regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
VM frame merging bug: call_closure_reuse now saves caller continuations
on a reuse_stack instead of merging frames. resume_vm restores them in
innermost-first order. Fixes frame count corruption when nested closures
suspend via OP_PERFORM. Zero test regressions (3924/3924).
Island hydration: hydrate-island now looks up components from (global-env)
instead of render-env, triggering the symbol resolve hook. Added JS-level
preload-island-defs that scans DOM for data-sx-island and loads definitions
from the content-addressed manifest BEFORE hydration — avoids K.load
reentrancy when the resolve hook fires inside env_get.
loadDefinitionByHash: fixed isMultiDefine check — defcomp/defisland bodies
containing nested (define ...) forms no longer suppress name insertion.
Added K.load return value checking for silent error string returns.
sx_browser.ml: resolve hook falls back to global_env.bindings when
_vm_globals miss (sync gap). Snapshot reuse_stack alongside pending_cek.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sx_vm.ml: VM timeout now compares vm_insn_count > step_limit instead of
unconditionally throwing after 65536 instructions when limit > 0
- sx_browser.ml: Expose setStepLimit/resetStepCount APIs on SxKernel;
callFn now returns {__sx_error, message} on Eval_error instead of null
- compiler.sx: emit-set handles array-index targets (host-set! instead of
nth) and 'of' property chains (dom-set-prop with chain navigation)
- hs-run-fast.js: New Node.js test runner with step-limit timeouts,
SX-level guard for error detection, insertAdjacentHTML mock,
range selection (HS_START/HS_END), wall-clock timeout in driveAsync
- hs-debug-test.js: Single-test debugger with DOM state inspection
- hs-verify.js: Assertion verification (proves pass/fail detection works)
Test results: 415/831 (50%), up from 408/831 (49%) baseline.
Fixes: set my style["color"], set X of Y, put at end of (insertAdjacentHTML).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The infinite loops in the HS parser are in transpiled native OCaml code,
not in the VM or CEK step loop. Neither step counters (in cek_step_loop,
cek_step, trampoline) nor VM instruction checks caught them because
the loops are in direct OCaml recursion.
Fix: SIGALRM handler raises Eval_error to break out of native loops.
Also sets step_limit flag to catch VM loops. Combined approach handles
both native OCaml recursion (alarm+raise) and VM bytecode (step check).
The alarm+raise can become unreliable after ~13 timeouts in a single
process, but handles the common case well. Reverts the fork-based
approach which lost inter-test state.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- orchestration.sx: add hs-boot-subtree! call to process-elements
- integration.sx: remove load-library! calls (browser loads via manifest)
- sx_vm.ml: add __resolve-symbol hook to OP_GLOBAL_GET for lazy loading
- compile-modules.js: add HS modules as lazy_deps in manifest
HS compilation works in browser (tokenize→parse→compile verified).
Activation pipeline partially working — hs-activate! needs debugging
(dom-get-data/dom-set-data interaction with WASM host-get on functions).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When cek_call_or_suspend runs a CEK machine for a non-bytecoded Lambda
(e.g. a thunk), _active_vm still pointed to the caller's VM. VmClosure
calls inside the CEK (e.g. hs-wait) would merge their frames with the
caller's VM via call_closure_reuse, causing the VM to skip the CEK's
remaining continuation on resume — producing wrong DOM mutation order
(+active, +active, -active instead of +active, -active, +active).
Fix: swap _active_vm with an empty isolation VM before running the CEK,
restore after. This keeps VmClosure calls on their own frame stack while
preserving js_of_ocaml exception identity (Some path, not None).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: nested cek_call_or_suspend calls on the same VM (from
synchronous callbacks like dom-listen firing handler immediately)
overwrote pending_cek before the first resume ran.
Fix: _vm_suspension_to_dict snapshots pending_cek at capture time
and restores it in the resume closure before calling resume_vm.
This ensures each suspension's CEK state is preserved regardless
of nested overwrite.
test_bytecode_repeat.js: 4/4 pass (was 3/4).
Source: 6 suspensions ✓ Bytecode: 6 suspensions ✓
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause identified: nested cek_call_or_suspend calls on same VM
overwrite pending_cek. First call suspends (thunk's hs-wait), second
call from synchronous dom-listen callback overwrites before resume.
sandbox host-callback: removed _driveAsync call to prevent duplicate
resume chains. Still 3/6 in Node.js test — issue is in OCaml call
stack nesting, not JS async.
Next: prevent pending_cek overwrite in nested CEK→VM→CEK→VM chains.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lambda calls in sx_call now go through the CEK machine instead of
returning a Thunk for the tree-walker trampoline. This lets perform/
IO suspension work everywhere — including hyperscript wait/bounce.
Key changes:
- sx_runtime: Lambda case calls _cek_eval_lambda_ref (forward ref)
- sx_vm: initializes ref with cek_step_loop + stub VM for suspension
- sx_apply_cek: VmSuspended → __vm_suspended marker dict (not exception)
- continue_with_call callable path: handles __vm_suspended with
vm-resume-frame, matching the existing JIT Lambda pattern
- sx_render: let VmSuspended propagate through try_catch
- Remove invalid io-contract test (perform now suspends, not errors)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- jit_compile_lambda: merge closure bindings into effective_globals so
GLOBAL_GET resolves variables from let/define blocks (emit-on, etc.)
- code_from_value: scan bytecode for max LOCAL_GET/SET slot to compute
vc_locals (fixes LOCAL_GET overflow in large functions like hs-parse)
3127/3127 no-JIT, 3116/3127 JIT (11 hyperscript on-event: specific
bytecode correctness issue in recursive parser — wrong branch taken
strips on/event-name from result).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- jit_compile_lambda: call compile directly via VM when it has bytecode
(100-400x faster JIT compilation, server pre-warm 1.6s vs hung)
- code_from_value: scan bytecode for highest LOCAL_GET/SET slot to
compute vc_locals correctly (fixes hyperscript LOCAL_GET overflow)
- code_from_value: accept both compiler keys (bytecode) and SX VM
keys (vc-bytecode) for interop
- jit_compile_lambda: skip &key/:as params (compiler can't emit them)
- Test runner: seed VM globals with primitives + env bindings,
native vm-execute-module with suspension fallback to SX version,
_jit_refresh_globals syncs globals after module loading,
VmSuspended + "VM undefined" caught and sentineled
3127/3127 without JIT, 3116/3127 with JIT (11 hyperscript on-event
parsing — specific closure/scope issue, not infrastructure).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Break up the 1735-line handle_tool match into 45 individual handler functions
with hashtable-based dispatch. Add mtime-based file parse caching (AST + CST),
consolidated run_command helper replacing 9 bare open_process_in patterns,
require_file/require_dir input validation, and pagination (limit/offset) for
sx_find_across, sx_comp_list, sx_comp_usage. Also includes pending VM changes:
rest-arity support, hyperscript parser, compiler/transpiler updates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three interacting JIT bugs caused infinite loops and server hangs:
1. _jit_compiling cascade: the re-entrancy flag was local to each
binary's hook. When vm_call triggered JIT compilation internally,
compiler functions got JIT-compiled during compilation, creating
infinite cascades. Fix: shared _jit_compiling flag in sx_vm.ml,
set in jit_compile_lambda itself.
2. call_closure always created new VMs: every HO primitive callback
(for-each, map, filter) allocated a fresh VM. With 43K+ calls
during compilation, this was the direct cause of hangs. Fix:
call_closure_reuse reuses the active VM by isolating frames and
running re-entrantly. VmSuspended is handled by merging frames
for proper IO resumption.
3. vm_call for compiled Lambdas: OP_CALL dispatching to a Lambda
with cached bytecode created a new VM instead of pushing a frame
on the current one. Fix: push_closure_frame directly.
Additional MCP server fixes:
- Hot-reload: auto-execv when binary on disk is newer (no restart needed)
- Robust JSON: to_int_safe/to_int_or handle null, string, int params
- sx_summarise depth now optional (default 2)
- Per-request error handling (malformed JSON doesn't crash server)
- sx_test uses pre-built binary (skips dune rebuild overhead)
- Timed module loading for startup diagnostics
sx_server.ml fixes:
- Uses shared _jit_compiling flag
- Marks lambdas as jit_failed_sentinel on compile failure (no retry spam)
- call_closure_reuse with VmSuspended frame merging for IO support
Compiled compiler bytecode bug: deeply nested cond/case/let forms
(e.g. tw-resolve-style) cause the compiled compiler to loop.
Workaround: _jit_compiling guard prevents compiled function execution
during compilation. Compilation uses CEK (slower but correct).
Test suite: 3127/3127 passed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The previous commit fixed lib/vm.sx (SX spec) but the server uses
sx_vm.ml (hand-maintained native OCaml) and sx_vm_ref.ml (transpiled).
Both had the same globals-first lookup bug. Now all three implementations
check closure env before vm.globals, matching vm-global-set.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tier 1 — Component keyword dispatch on VM:
- Components/islands JIT-compile bodies via jit_compile_comp
- parse_keyword_args matches keyword names against component params
- Added i_compiled field to island type for JIT cache
- Component calls no longer fall back to CEK
Tier 2 — OP_SWAP (opcode 7):
- New stack swap operation for future HO loop compilation
- HO forms already efficient via NativeFn + VmClosure callbacks
Tier 3 — Exception handler stack:
- OP_PUSH_HANDLER (35), OP_POP_HANDLER (36), OP_RAISE (37)
- VM gains handler_stack with frame depth tracking
- Compiler handles guard and raise as bytecode
- Functions with exception handling no longer cause JIT failure
Tier 4 — Scope forms as bytecode:
- Compiler handles provide, context, peek, scope, provide!,
bind, emit!, emitted via CALL_PRIM sequences
- Functions using reactive scope no longer trigger JIT failure
4 new opcodes (SWAP, PUSH_HANDLER, POP_HANDLER, RAISE) → 37 total.
2776/2776 tests pass, zero regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bytecode compiler now emits OP_PERFORM for (import ...) and compiles
(define-library ...) bodies. The VM stores the import request in
globals["__io_request"] and stops the run loop — no exceptions needed.
vm-execute-module returns a suspension dict, vm-resume-module continues.
Browser: sx_browser.ml detects suspension dicts from execute_module and
returns JS {suspended, op, request, resume} objects. The sx-platform.js
while loop handles cascading suspensions via handleImportSuspension.
13 modules load via .sxbc bytecode in 226ms (manifest-driven), both
islands hydrate, all handlers wired. 2650/2650 tests pass including
6 new vm-import-suspension tests.
Also: consolidated sx-platform-2.js → sx-platform.js, fixed
vm-execute-module missing code-from-value call, fixed bootstrap.py
protocol registry transpiler issues.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Seed all primitives into vm_globals as NativeFn values at init.
CALL_PRIM now looks up vm.globals only (not the separate primitives
table). This means OP_DEFINE and registerNative naturally override
primitives — browser.sx's (define set-cookie ...) now takes effect.
The primitives Hashtbl remains for the compiler's primitive? predicate
but has no runtime dispatch role.
Tests: 2435 pass / 64 fail (pre-existing), vs 1718/771 baseline.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: sx_primitives.ml registered "effect" as a native no-op (for SSR).
The bytecode compiler's (primitive? "effect") returned true, so it emitted
OP_CALL_PRIM instead of OP_GLOBAL_GET + OP_CALL. The VM's CALL_PRIM handler
found the native Nil-returning stub and never called the real effect function
from core-signals.sx.
Fix: Remove effect and register-in-scope from the primitives table. The server
overrides them via env_bind in sx_server.ml (after compilation), which doesn't
affect primitive? checks.
Also: VM CALL_PRIM now falls back to cek_call for non-NativeFn values (safety
net for any other functions that get misclassified).
15/15 source mode, 15/15 bytecode mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT compiler:
- Fix jit_compile_lambda: resolve `compile` via symbol lookup in env
instead of embedding VmClosure in AST (CEK dispatches differently)
- Register eval-defcomp/eval-defisland/eval-defmacro runtime helpers
in browser kernel for bytecoded defcomp forms
- Disable broken .sxbc.json path (missing arity in nested code blocks),
use .sxbc text format only
- Mark JIT-failed closures as sentinel to stop retrying
CSSX in browser:
- Add cssx.sx symlink + cssx.sxbc to browser web stack
- Add flush-cssx! to orchestration.sx post-swap for SPA nav
- Add cssx.sx to compile-modules.js and mcp_tree.ml bytecode lists
SPA navigation:
- Fix double-fetch: check e.defaultPrevented in click delegation
(bind-event already handled the click)
- Fix layout destruction: change nav links from outerHTML to innerHTML
swap (outerHTML destroyed #main-panel when response lacked it)
- Guard JS popstate handler when SX engine is booted
- Rename sx-platform.js → sx-platform-2.js to bust immutable cache
Playwright tests:
- Add trackErrors() helper to all test specs
- Add SPA DOM comparison test (SPA nav vs fresh load)
- Add single-fetch + no-duplicate-elements test
- Improve MCP tool output: show failure details and error messages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: _env_bind_hook mirrored ALL env_bind calls (including
lambda parameter bindings) to the shared VM globals table. Factory
functions like make-page-fn that return closures capturing different
values for the same param names (default-name, prefix, suffix) would
have the last call's values overwrite all previous closures' captured
state in globals. OP_GLOBAL_GET reads globals first, so all closures
returned the last factory call's values.
Fix: only sync root-env bindings (parent=None) to VM globals. Lambda
parameter bindings stay in their local env, found via vm_closure_env
fallback in OP_GLOBAL_GET.
Also in this commit:
- OP_CLOSURE propagates parent vm_closure_env to child closures
- Remove JIT globals injection (closure vars found via env chain)
- sx_server.ml: SX-Request header → returns text/sx (aser only)
- sx_server.ml: diagnostic endpoint GET /sx/_debug/{env,eval,route}
- sx_server.ml: page helper stubs for deep page rendering
- sx_server.ml: skip client-libs/ dir (browser-only definitions)
- adapter-html.sx: unknown components → HTML comment (not error)
- sx-platform.js: .sxbc fallback loader for bytecode modules
- Delete sx_http.ml (standalone HTTP server, unused)
- Delete stale .sxbc.json files (arity=0 bug, replaced by .sxbc)
- 7 new closure isolation tests in test-closure-isolation.sx
- mcp_tree.ml: emit arity + upvalue-count in .sxbc.json output
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Browser kernel:
- Add `parse` native fn (matches server: unwrap single, list for multiple)
- Restore env==global_env guard on _env_bind_hook (let bindings must not
leak to _vm_globals — caused JIT CSSX "Not callable: nil" errors)
- Add _env_bind_hook call in env_set_id so set! mutations sync to VM globals
- Fire _vm_global_set_hook from OP_DEFINE so VM defines sync back to CEK env
CEK evaluator:
- Replace recursive cek_run with iterative while loop using sx_truthy
(previous attempt used strict Bool true matching, broke in wasm_of_ocaml)
- Remove dead cek_run_iterative function
Web modules:
- Remove find-matching-route and parse-route-pattern stubs from
boot-helpers.sx that shadowed real implementations from router.sx
- Sync boot-helpers.sx to dist/static dirs for bytecode compilation
Platform (sx-platform.js):
- Set data-sx-ready attribute after boot completes (was only in boot-init
which sx-platform.js doesn't call — it steps through boot manually)
- Add document-level click delegation for a[sx-get] links as workaround
for bytecoded bind-event not attaching per-element listeners (VM closure
issue under investigation — bind-event runs but dom-add-listener calls
don't result in addEventListener)
Tests:
- New test_kernel.js: 24 tests covering env sync, parse, route matching,
host FFI/preventDefault, deep recursion
- New navigation test: "sx-get link fetches SX not HTML and preserves layout"
(currently catches layout breakage after SPA swap — known issue)
Known remaining issues:
- JIT CSSX failures: closure-captured variables resolve to nil in VM bytecode
- SPA content swap via execute-request breaks page layout
- Bytecoded bind-event doesn't attach per-element addEventListener (root
cause unknown — when listen-target guard appears to block despite element
being valid)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The optimization to call the compiler through VM directly (instead
of CEK) when it's JIT-compiled was producing incorrect bytecode for
all subsequently compiled functions, causing "Expected number, got
symbol" errors across render-to-html, parse-loop, etc.
Revert to always using CEK for compilation. The compiler runs via
CEK which is slower but produces correct bytecode. JIT-compiled
USER functions still run at VM speed.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removing injection broke GLOBAL_GET for all JIT-compiled functions,
not just mutable closures. Top-level functions like render-to-html
need their referenced bindings in the VM globals table.
Restore the original injection (only injects values not already in
globals). Mutable closure vars (parser's pos etc.) still get stale
snapshots and fall back to CEK — that's the known limitation to fix
with cell-based boxing later.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: jit_compile_lambda copied closure variable VALUES into the
VM globals table. GLOBAL_GET found these stale snapshots instead of
falling through to vm_closure_env which has live bindings. When set!
mutated a variable (like parser's pos), the JIT code read the old
snapshot value.
Fix: don't inject closure bindings into globals at all. GLOBAL_GET
already has a fallback path that walks vm_closure_env — this sees
live env bindings that are updated by set!. No new opcodes needed.
This should fix JIT for parse-loop, skip-ws, read-expr and other
closure functions that use mutable variables.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jit_compile_lambda now uses the live globals table directly instead
of Hashtbl.copy. Closure bindings that aren't already in globals are
injected into the live table. This ensures GLOBAL_GET always sees
the latest define values.
Previously: Hashtbl.copy created a stale snapshot. Functions defined
after the copy (like cssx-process-token from cssx.sx) resolved to nil
in JIT-compiled closures.
JIT error test now passes — 0 CSSX errors during navigation.
7/8 navigation tests pass. Remaining: back button content update.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT runtime errors now log once per function name via _jit_warned
hashtable, then stay quiet for that function. No more silent swallowing
(which hid real errors) or per-call flooding (which spammed thousands
of lines and blocked the server).
VM-level fallbacks (inside JIT-compiled code calling other JIT code)
are silent — dedup happens at the hook level.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT runtime errors now silently fall back to CEK without disabling the
compiled bytecode or logging. This prevents render-to-html from being
permanently disabled when one page has an unresolved symbol (e.g. the
homepage stepper's <home).
Load spec/signals.sx and web/engine.sx for reactive primitives.
Skip test files and non-rendering directories (tests/, plans/, essays/).
Performance with JIT active (warm, single process, 2MB RSS):
- Aser (component expansion): 21-87ms — faster than Quart baseline
- SSR + shell: variable due to homepage <home fallback issue
- Geography total: ~500ms when JIT stays active
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reverse hook syncs VM GLOBAL_SET mutations back to global_env so CEK reads
see JIT-written values. Isomorphic nav: store primitives, event-bridge,
client? predicate. Browser JS and bytecode rebuilt.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
List.find returns the element that matched, but SX some should return
the callback's truthy return value. This caused get-verb-info to return
"get" (the verb string) instead of the {method, url} dict.
Also added _active_vm tracking to VM for future HO primitive optimization,
and reverted get-verb-info to use some (no longer needs for-each workaround).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sx-get links were doing full page refreshes because click handlers
never attached. Root causes: VM frame management bug, missing primitives,
CEK/VM type dispatch mismatch, and silent error swallowing.
Fixes:
- VM frame exhaustion: frames <- [] now properly pops to rest_frames
- length primitive: add alias for len in OCaml primitives
- call_sx_fn: use sx_call directly instead of eval_expr (CEK checks
for type "lambda" but VmClosure reports "function")
- Boot error surfacing: Sx.init() now has try/catch + failure summary
- Callback error surfacing: catch-all handler for non-Eval_error exceptions
- Silent JIT failures: log before CEK fallback instead of swallowing
- vm→env sync: loadModule now calls sync_vm_to_env()
- sx_build_bytecode MCP tool added for bytecode compilation
Tests: 50 new tests across test-vm.sx and test-vm-primitives.sx covering
nested VM calls, frame integrity, CEK bridge, primitive availability,
cross-module symbol resolution, and callback dispatch.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the VM called a JIT-compiled lambda, it passed vm.globals
(the caller's global env) instead of cl.vm_env_ref (the closure's
captured env that was merged at compile time). Closure-captured
variables like code-tokens from island let/letrec scopes were
invisible at runtime, causing "Undefined symbol" errors that
cascaded to disable render-to-html globally.
Fix: call_closure uses cl.vm_env_ref at both call sites (cached
bytecode and fresh compilation).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three bugs broke island SSR rendering of the home stepper widget:
1. Inline VM opcodes (OP_ADD..OP_DEC) broke JIT-compiled functions.
The compiler emitted single-byte opcodes for first/rest/len/= etc.
that produced wrong results in complex recursive code (sx-parse
returned nil, split-tag produced 1 step instead of 16). Reverted
compiler to use CALL_PRIM for all primitives. VM opcode handlers
kept for future use.
2. Named let (let loop ((x init)) body) had no compiler support —
silently produced broken bytecode. Added desugaring to letrec.
3. URL-encoded cookie values not decoded server-side. Client set-cookie
uses encodeURIComponent but Werkzeug doesn't decode cookie values.
Added unquote() in bridge cookie injection.
Also: call-lambda used eval_expr which copies Dict values (signals),
breaking mutations through aser lambda calls. Switched to cek_call.
Also: stepper preview now includes ~cssx/tw spreads for SSR styling.
Tests: 1317 JS, 1114 OCaml, 26 integration (2 pre-existing failures)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
16 new opcodes (160-175) bypass the CALL_PRIM hashtable lookup for
the most frequently called primitives:
Arithmetic: OP_ADD, OP_SUB, OP_MUL, OP_DIV, OP_INC, OP_DEC, OP_NEG
Comparison: OP_EQ, OP_LT, OP_GT, OP_NOT
Collection: OP_LEN, OP_FIRST, OP_REST, OP_NTH, OP_CONS
The compiler (compiler.sx) recognizes these names at compile time and
emits the inline opcode instead of CALL_PRIM. The opcode is self-
contained — no constant pool index, no argc byte. Each primitive is
a single byte in the bytecode stream.
Implementation in all three VMs:
- OCaml (sx_vm.ml): direct pattern match, no allocation
- SX spec (vm.sx): delegates to existing primitives
- JS (transpiled): same as SX spec
66 new tests in spec/tests/vm-inline.sx covering arithmetic, comparison,
collection ops, composition, and edge cases.
Tests: 1314 JS (full), 1114 OCaml, 32 Playwright
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Route all rendering through OCaml bridge — render_to_html no longer uses
Python async_eval. Fix register_components to parse &key params and &rest
children from defcomp forms. Remove all dead sx_ref.py imports.
Epoch protocol (prevents pipe desync):
- Every command prefixed with (epoch N), all responses tagged with epoch
- Both sides discard stale-epoch messages — desync structurally impossible
- OCaml main loop discards stale io-responses between commands
Consolidate scope primitives into sx_scope.ml:
- Single source of truth for scope-push!/pop!/peek, collect!/collected,
emit!/emitted, context, and 12 other scope operations
- Removes duplicate registrations from sx_server.ml (including bugs where
scope-emit! and clear-collected! were registered twice with different impls)
- Bind scope prims into env so JIT VM finds them via OP_GLOBAL_GET
JIT VM fixes:
- Trampoline thunks before passing args to CALL_PRIM
- as_list resolves thunks via _sx_trampoline_fn
- len handles all value types (Bool, Number, RawHTML, SxExpr, Spread, etc.)
Other fixes:
- ~cssx/tw signature: (tokens) → (&key tokens) to match callers
- Minimal Python evaluator in html.py for sync sx() Jinja function
- Python scope primitive stubs (thread-local) for non-OCaml paths
- Reader macro resolution via OcamlSync instead of sx_ref.py
Tests: 1114 OCaml, 1078 JS, 35 Python regression, 6/6 Playwright SSR
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT allowlist (sx_server.ml):
- Replace try-every-lambda strategy with StringSet allowlist. Only
functions in the list get JIT compiled (compiler, parser, pure transforms).
Render functions that need dynamic scope skip JIT entirely — no retry
overhead, no silent fallbacks.
- Add (jit-allow name) command for dynamic expansion from Python bridge.
- JIT failures log once with "[jit] DISABLED fn — reason" then go silent.
Standalone --test mode (sx_server.ml):
- New --test flag loads full env (spec + adapters + compiler + signals),
supports --eval and --load flags. Quick kernel testing without Docker.
Example: dune exec bin/sx_server.exe -- --test --eval '(len HTML_TAGS)'
Integration tests (integration_tests.ml):
- New binary exercising the full rendering pipeline: loads spec + adapters
into a server-like env, renders HTML via both native and SX adapter paths.
- 26 tests: HTML tags, special forms (when/if/let), letrec with side
effects, component rendering, eval-expr with HTML tag functions.
- Would have caught the "Undefined symbol: div/lake/init" issues from
the previous commit immediately without Docker.
VM cleanup (sx_vm.ml):
- Remove temporary debug logging (insn counter, call_closure counter,
VmClosure depth tracking) added during debugging.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Core VM changes:
- Add VmClosure value variant — inner closures created by OP_CLOSURE are
first-class VM values, not NativeFn wrappers around call_closure
- Convert `run` from recursive to while-loop — zero OCaml stack growth,
true TCO for VmClosure tail calls
- vm_call handles VmClosure by pushing frame on current VM (no new VM
allocation per call)
- Forward ref _vm_call_closure_ref for cross-boundary calls (CEK/primitives)
Compiler (spec/compiler.sx):
- Define hoisting in compile-begin: pre-allocate local slots for all
define forms before compiling any values. Fixes forward references
between inner functions (e.g. read-expr referencing skip-ws in sx-parse)
- scope-define-local made idempotent (skip if slot already exists)
Server (sx_server.ml):
- JIT fail-once sentinel: mark l_compiled as failed after first VM runtime
error. Eliminates thousands of retry attempts per page render.
- HTML tag bindings: register all HTML tags as pass-through NativeFns so
eval-expr can handle (div ...) etc. in island component bodies.
- Log VM FAIL errors with function name before disabling JIT.
SSR fixes:
- adapter-html.sx letrec handler: evaluate bindings in proper letrec scope
(pre-bind nil, then evaluate), render body with render-to-html instead of
eval-expr. Fixes island SSR for components using letrec.
- Add `init` primitive to OCaml kernel (all-but-last of list).
- VmClosure handling in sx_runtime.ml sx_call dispatch.
Tests: 971/971 OCaml (+19 new), 0 failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
vm_closure now stores the original closure env (vm_closure_env).
GLOBAL_GET walks the closure env chain when the variable isn't in
vm.globals. GLOBAL_SET writes to the correct env in the chain.
This enables JIT compilation of all named functions regardless of
closure depth. No more closure skip check needed.
Pre-compile time back to ~7s (was 37s with closure skip).
Note: sx-parse sibling list parsing still has issues — the root
cause is in how the JIT-compiled letrec + OP_CLOSURE interacts
with the upvalue cell mechanism. Investigation ongoing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
