Clicking a blog link now fragment-swaps #content with URL push + working back
button, no full reload — the SX-htmx engine driving the same OCaml kernel the
server runs. Six bugs in the source-load + boost path, found by bisecting in
chromium, all fixed:
1. Import double-apply (sx_server.ml x2, sx_browser.ml): the import suspension
handlers computed `key = library_name_key lib_spec` then called
`library_loaded_p key` — but library_loaded_p applies library_name_key
itself, so it ran sx_to_list on a string and crashed ("Expected list, got
string"). Only unloaded libs suspend, so it only bit lazy imports. Pass the
spec, not the key.
2. Unloaded-import crash (spec/evaluator.sx + sx_ref.ml library_exports): an
import of a not-yet-loaded library returned nil exports, and bind-import-set
did (keys nil) -> crash. Return an empty dict so the import is a graceful
no-op (lazy symbol resolution covers real usage).
3. value_to_js missing Integer (sx_browser.ml): integers passed to host methods
were mishandled, so dom-query-all's (host-call node-list "item" i) ignored i
and returned node 0 for every index — every element aliased the first, so
only one link ever boosted. Add the Integer -> JS number case.
4. browser-same-origin? rejected relative URLs (browser.sx x2): it only did
(starts-with? url origin), so "/alpha/" was treated as cross-origin and
should-boost-link? refused every relative link. Accept scheme-less,
non-protocol-relative URLs.
5. dom-query-in undefined (orchestration.sx x2): the swap path called a function
that exists nowhere; it's just dom-query with a container arg.
6. Lazy-deps never loaded under source fallback (sx-platform.js): lazy symbol
resolution only fires on the VM GLOBAL_GET path, but source-loaded swap
callbacks run on the CEK and raise instead of lazy-loading, so the post-swap
hs-boot-subtree!/htmx-boot-subtree! were undefined and aborted URL push.
Preload the manifest's lazy-deps.
Verified: native host conformance 271/271; lib/host/playwright/spa-check 4/4
(boot, boost, fragment swap + URL push, back button) in real chromium against an
ephemeral durable host server.
Two related kernel bugs were causing the HS conformance test
"arrays containing objects work" to fail with the misleading message
"Expected ({:a 1} {:b 2}) but got ({:a 1} {:b 2})".
1. sx_primitives.ml safe_eq: Dict/Dict only returned true for DOM-wrapped
dicts (those carrying __host_handle); all other dict pairs returned
false unconditionally. Plain dict literals can never have been =
to each other. Add the structural-equality fallback: when neither
side has a host handle, compare lengths and walk keys.
2. sx_browser.ml deep_equal (the kernel binding for equal?): had a
Number/Number branch but no Integer/Integer or cross-Integer/Number
branches, so since the numeric tower change Integer 1 vs Integer 1
was falling through to the catch-all and returning false. Mirror the
cases from run_tests.ml deep_equal which already had them.
Verified via direct kernel probe:
(= {:a 1} {:a 1}) => true (was false)
(= {:a 1 :b 2} {:b 2 :a 1}) => true (was false)
(equal? 1 1) => true (was false)
(equal? {:a 1} {:a 1}) => true (was false)
(equal? (list {:a 1}) (list {:a 1})) => true (was false)
HS suite arrayLiteral: 7/8 → 8/8.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The shared/static/wasm/sx_browser.bc.js artifact now reflects the OCaml
kernel with JIT Phase 1 (tiered compilation), Phase 2 (LRU eviction),
and Phase 3 (manual reset) — same source as previously committed,
just the rebuilt binary so test/dev consumers pick it up without
needing a local sx_build.
tests/hs-run-batched.js: TOTAL default 1496 → 1514. The conformance
suite grew by 18 tests since the constant was last set; without this
the batched runner stops short of the final 14 tests.
Verified via batched run (75-test batches, parallelism=2):
1436 / 1439 reported pass (3 failures, all in suites where the
underlying parser/dict-equality gap is independent of WASM).
Batch 150-225 didn't complete inside 15 min — 75 reactivity /
regressions / runtime tests at 5-11s each blow past the wall; a
per-batch deadline raise is the right knob, not a kernel change.
Per-test timing (new vs old WASM, slice 170-195) is comparable
(60s vs 78s on new/threshold=4 — Phase 1+2 is NOT a perf regression
on HS code; the slow tests are slow on both kernels because the
underlying CEK path doesn't get JIT-compiled either way — HS emits
anonymous lambdas that bypass the named-only JIT gate).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- _NO_STEP_LIMIT set exempts hypertrace tests from the 200k step cap
- globalThis.__hs_deadline exposed so cek_step_loop wall-clock check
(every 10k steps) can terminate runaway async loops without needing
to go through host-call or _driveAsync
- meta + _hs-on-caller added to hs-runtime.sx (both lib and bundled):
on-event handlers now set meta.caller to an object with
meta.feature.type = "onFeature" before calling the handler
Tests 196 (async hypertrace), 198 (meta.caller), 199, 200 now pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously these primitives threw Eval_error if either arg was non-string.
Now they return false, preventing crashes when DOM attributes return nil
values during element processing (e.g. htmx-boot-subtree! iterating
elements with undefined attribute names).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When an error occurs during resumed VM execution (after perform/hs-wait),
resume_vm now checks the VM's handler_stack. If a handler exists (from a
compiled guard form's OP_PUSH_HANDLER), it unwinds frames and jumps to
the catch block — exactly like OP_RAISE. This enables try/catch across
async perform/resume boundaries.
The guard form compiles to OP_PUSH_HANDLER which lives on the vm struct
and survives across setTimeout-based async resume. Previously, errors
during resume escaped to the JS console as unhandled exceptions.
Also restored guard in the test runner (was cek-try which doesn't survive
async) and restored error-throwing assertions in run-action.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The guard form (call/cc + handler-bind expansion) doesn't survive async
IO suspension — the CEK continuation from guard's call/cc captures frames
that become invalid after the VM resumes from hs-wait. Replacing guard
with cek-try (which compiles to VM-native OP_PUSH_HANDLER/OP_POP_HANDLER)
avoids the CEK boundary crossing.
The test runner now executes: suspends on hs-wait, resumes, runs test
actions, and test assertions fire correctly. The "Not callable: nil"
error is eliminated. Remaining: test assertion errors from iframe content
not loading fast enough (timing issue, not a framework bug).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The Not callable: nil error happens on a stub VM (frames=[], sp=0) during
cek_resume with 12 CEK kont frames. The error is from a reactive signal
subscriber (reset! current ...) that triggers during run vm after resume.
The subscriber callback goes through CEK via cek_call_or_suspend and the
CEK continuation tries to call nil.
This is a reactive subscriber notification issue, not a perform/resume
frame management issue. The VM frames are correctly restored — the error
happens during a synchronous reset! call within the resumed VM execution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Shows pending_cek, reuse_stack count, and frames count in the error.
Also transfers reuse_stack from _active_vm at VmSuspended catch sites.
Finding: the Not callable: nil happens during cek_resume (pending_cek=false,
kont=12 frames). The CEK continuation tries to call a letrec function that
is nil because letrec bindings are in VM local SLOTS, not in the CEK env.
The VM→CEK boundary crossing during suspension loses the local slot values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
bootstrap.py regenerated cek_run as a simple "raise if suspended" without
the _cek_io_resolver and _cek_io_suspend_hook checks. Also lost the
CekPerformRequest catch in cek_step_loop and step_limit checks.
This was the direct cause of "IO suspension in non-IO context" when island
click handlers called perform (via hs-wait). The CEK had no way to propagate
the suspension to the VM/JS boundary.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: when perform fires inside a VM closure chain (call_closure_reuse),
the caller frames are saved to reuse_stack on the ACTIVE VM. But the
_cek_io_suspend_hook and _cek_eval_lambda_ref create a NEW stub VM for the
VmSuspended exception. On resume, resume_vm runs on the STUB VM which has
an empty reuse_stack — the caller frames are orphaned on the original VM.
Fix: transfer reuse_stack from _active_vm to the stub VM before raising
VmSuspended. This ensures resume_vm -> restore_reuse can find and restore
the caller's frames after async resume via _driveAsync/setTimeout.
Also restore step_limit/step_count refs dropped by bootstrap.py regeneration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: sx_browser.ml registered all HTML tags (a, b, i, p, s, u, g, etc.)
as custom special forms. The evaluator's step_eval_list checked custom special
forms BEFORE checking local env bindings. So (let ((a (fn () 42))) (a))
matched the HTML tag <a> instead of calling the local function a.
Fix: skip custom special forms AND render-check when the symbol is bound in
the local env. Added (not (env-has? env name)) guard to both checks in
step-eval-list (spec/evaluator.sx and transpiled sx_ref.ml).
This was the root cause of "[sx] resume: Not callable: nil" — after hs-wait
resumed, calling letrec-bound functions like wait-boot (which is not an HTML
tag) worked, but any function whose name collided with an HTML tag failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
callable? in boot-helpers.sx checked for "native-fn" but type-of returns
"function" for NativeFn — broke make-spread and all native fn dispatch
in aser. Restore 20 behavioral tests replaced with NOT IMPLEMENTED stubs
by the test regeneration commit. Add host-* platform primitive stubs to
sx_server.ml so boot-helpers.sx loads without errors server-side.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _cek_io_suspend_hook creates a stub VM to carry the suspended CEK
state. Previously used empty globals, which caused "Not callable: nil"
when the CEK resume needed platform functions. Now uses _default_vm_globals
(set to _vm_globals by sx_browser.ml) so all platform functions and
definitions are available during resume.
Remaining issue: still getting "resume: Not callable: nil" — the CEK
continuation env may not include letrec bindings from the island body.
The suspension point is inside reload-frame → hs-wait, and the resume
needs to call wait-boot (a letrec binding).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _cek_io_suspend_hook was only added to cek_run_iterative (line 986)
but the actual code path went through cek_run (line 624). Added the hook
check to both functions.
This fixes the "IO suspension in non-IO context" error that blocked
hs-wait/perform from propagating through event handler → trampoline →
eval_expr call chains. IO suspension now converts to VmSuspended via the
hook, which the value_to_js wrapper catches and drives with _driveAsync.
+42 OCaml test passes (3924→3966). IO suspension verified working in
browser WASM: dom-on click handler → hs-wait → perform → suspend →
_driveAsync → setTimeout → resume.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recompiled WASM kernel (wasm_of_ocaml) to include the VM reuse_stack
fix from sx_vm.ml. Recompiled boot.sxbc with the clear-and-replace
hydration (replaceChildren + nil hydrating scope + dom-append).
sx-platform.js deployed with island preload, isMultiDefine fix, and
K.load error checking.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sx_vm.ml: VM timeout now compares vm_insn_count > step_limit instead of
unconditionally throwing after 65536 instructions when limit > 0
- sx_browser.ml: Expose setStepLimit/resetStepCount APIs on SxKernel;
callFn now returns {__sx_error, message} on Eval_error instead of null
- compiler.sx: emit-set handles array-index targets (host-set! instead of
nth) and 'of' property chains (dom-set-prop with chain navigation)
- hs-run-fast.js: New Node.js test runner with step-limit timeouts,
SX-level guard for error detection, insertAdjacentHTML mock,
range selection (HS_START/HS_END), wall-clock timeout in driveAsync
- hs-debug-test.js: Single-test debugger with DOM state inspection
- hs-verify.js: Assertion verification (proves pass/fail detection works)
Test results: 415/831 (50%), up from 408/831 (49%) baseline.
Fixes: set my style["color"], set X of Y, put at end of (insertAdjacentHTML).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: sx_insert_near placed break-lines-greedy, pretext-position-line,
pretext-layout-lines OUTSIDE the define-library begin block. The bytecode
compiler only compiles forms inside begin as STORE_GLOBAL — forms outside
are invisible to the browser VM.
Fix: moved all function definitions inside (begin ...) of (define-library).
Bytecode now includes all 17 functions (11K compiled, was 9K).
Browser load-sxbc: simplified VmSuspended handling — just catch and
continue, since STORE_GLOBAL ops already ran before the import OP_PERFORM.
sync_vm_to_env copies them to global_env.
Island now calls break-lines and pretext-layout-lines from bytecode-compiled
library — runs on VM, not CEK interpreter.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- text-layout.sx added to WASM bytecode pipeline (9K compiled)
- Fix multi-list map calls (map-indexed + nth instead of map fn list1 list2)
- pretext-layout-lines and pretext-position-line moved to library exports
- Browser load-sxbc: handle VmSuspended for import, copy library exports
to global_env after module load (define-library export fix)
- compile-modules.js: text-layout in SOURCE_MAP, FILES, and entry deps
- Island uses library functions (break-lines, pretext-layout-lines)
instead of inlining — runs on bytecode VM when exports resolve
Known issue: define-library exports don't propagate to browser global env
yet. The load-sxbc import suspension handler resumes correctly but
bind_import_set doesn't fire. Needs deeper investigation into how the
WASM kernel's define-library registers exports vs how other libraries
(adapter-html, tw) make their exports available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compile-let now handles dict destructuring patterns:
(let {:key1 var1 :key2 var2} source body). This unblocked core-signals.sx
(deref uses dict destructuring) which was the sole bytecode skip.
Rewrote stripLibraryWrapper from line-based to paren-aware extraction.
The old regex missed (define-library on its own line (no trailing space),
silently passing the full wrapper to the compiler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The value_to_js resume handler was calling _driveAsync on re-suspension,
but the JS driveAsync caller also processes the returned suspension.
This caused the second wait in each iteration to fire immediately (0ms)
instead of respecting the delay.
Fix: resume handler just returns the suspension object, lets the JS
driveAsync handle scheduling via setTimeout.
Verified: repeat 3 times add/wait 300ms/remove/wait 300ms produces
6 transitions at correct 300ms intervals (1504ms total).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The resume callback in the value_to_js VmSuspended handler now catches
VmSuspended recursively, building a new suspension object and calling
_driveAsync for each iteration. Fixes repeat N times ... wait ... end
which produces N sequential suspensions.
Bounce works on repeated clicks. 4/4 regression tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sx-platform.js: add _driveAsync to platform (was sandbox-only) for
driving wait/fetch IO suspension chains in live site
- sx-platform.js: host-callback wrapper calls _driveAsync on callFn result
- sx_browser.ml: value_to_js callable wrapper catches VmSuspended, builds
suspension object, and calls _driveAsync directly
Toggle and count clicks work fully. Bounce adds class but wait/remove
requires IO suspension in CEK context (eval-expr-cek doesn't support
perform — needs VM-path evaluation in hs-handler).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sx_browser.ml: restore VmSuspended handler in api_call_fn with
make_js_callFn_suspension for IO suspension chains (wait, fetch)
- runtime.sx: delete host-get stub that shadowed platform native —
hs-toggle-class! now uses real FFI host-get for classList access
All three live demo examples work:
Toggle Color — classList.toggle on click
Bounce — add .animate-bounce, wait 1s suspend, remove
Count Clicks — increment @data-count, put into innerHTML
4/4 bytecode regression tests pass (was 0/4 without VmSuspended).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- orchestration.sx: add hs-boot-subtree! call to process-elements
- integration.sx: remove load-library! calls (browser loads via manifest)
- sx_vm.ml: add __resolve-symbol hook to OP_GLOBAL_GET for lazy loading
- compile-modules.js: add HS modules as lazy_deps in manifest
HS compilation works in browser (tokenize→parse→compile verified).
Activation pipeline partially working — hs-activate! needs debugging
(dom-get-data/dom-set-data interaction with WASM host-get on functions).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reset to last known-good state (908f4f80) where links, stepper, and
islands all work, then recovered all hyperscript implementation,
conformance tests, behavioral tests, Playwright specs, site sandbox,
IO-aware server loading, and upstream test suite from f271c88a.
Excludes runtime changes (VM resolve hook, VmSuspended browser handler,
sx_ref.ml guard recovery) that need careful re-integration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When cek_call_or_suspend runs a CEK machine for a non-bytecoded Lambda
(e.g. a thunk), _active_vm still pointed to the caller's VM. VmClosure
calls inside the CEK (e.g. hs-wait) would merge their frames with the
caller's VM via call_closure_reuse, causing the VM to skip the CEK's
remaining continuation on resume — producing wrong DOM mutation order
(+active, +active, -active instead of +active, -active, +active).
Fix: swap _active_vm with an empty isolation VM before running the CEK,
restore after. This keeps VmClosure calls on their own frame stack while
preserving js_of_ocaml exception identity (Some path, not None).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All .sxbc recompiled with fixed sx_vm.ml. 32/32 WASM tests, 4/4
bytecode regression tests. hs-repeat-times correctly does 6 io-sleep
suspensions in bytecode mode.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lambda calls in sx_call now go through the CEK machine instead of
returning a Thunk for the tree-walker trampoline. This lets perform/
IO suspension work everywhere — including hyperscript wait/bounce.
Key changes:
- sx_runtime: Lambda case calls _cek_eval_lambda_ref (forward ref)
- sx_vm: initializes ref with cek_step_loop + stub VM for suspension
- sx_apply_cek: VmSuspended → __vm_suspended marker dict (not exception)
- continue_with_call callable path: handles __vm_suspended with
vm-resume-frame, matching the existing JIT Lambda pattern
- sx_render: let VmSuspended propagate through try_catch
- Remove invalid io-contract test (perform now suspends, not errors)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Break up the 1735-line handle_tool match into 45 individual handler functions
with hashtable-based dispatch. Add mtime-based file parse caching (AST + CST),
consolidated run_command helper replacing 9 bare open_process_in patterns,
require_file/require_dir input validation, and pagination (limit/offset) for
sx_find_across, sx_comp_list, sx_comp_usage. Also includes pending VM changes:
rest-arity support, hyperscript parser, compiler/transpiler updates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The /sx/ prefix mismatch: defpage declares paths like /language/docs/<slug>
but browser URLs are /sx/(language.(doc.slug)). find-matching-route used
starts-with? "/(", missing the /sx/ prefix entirely.
Fix: find-matching-route now uses (index-of path "/(") to detect the SX
URL portion regardless of prefix. Works for /sx/, /myapp/, any prefix.
No hardcoded paths.
Also fixed deps-satisfied?: nil deps (unknown) now returns false instead
of true, preventing client-side eval of pages with unresolved components.
Correctly falls back to server fetch.
Verified with Playwright: clicking "Getting Started" on the docs page now
shows "sx:route deps miss for docs-page" → "sx:route server fetch" instead
of the old "sx:route no match (51 routes)".
2 new router tests for prefix stripping. 2914/2914 total, zero failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Provide subscribers stored in global *provide-subscribers* dict (keyed
by name) instead of on provide frames. Fixes subscriber loss when
frames are reconstructed, and enables cross-cek_run notification.
Batch integration: batch-begin!/batch-end! primitives manage
*provide-batch-depth*. fire-provide-subscribers defers to queue when
depth > 0, batch-end! flushes deduped. signals.sx batch calls both.
context now prefers scope-peek over frame value — scope stack is the
source of truth since provide! always updates it (even in nested
cek_run where provide frames aren't on the kont).
2754/2768 OCaml (14 pre-existing). 32/32 WASM.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: context called inside lambdas (e.g. swap!) went through
nested cek_run with empty kont, so provide frames weren't found and
never tracked to *bind-tracking*.
Three changes in evaluator.sx:
- step-sf-context: track context names (not frames) to *bind-tracking*
— names work across cek_run boundaries via scope-peek fallback
- bind continue: resolve tracked names to frames via kont-find-provide
on rest-k before registering subscribers
- subscriber: use empty kont instead of kont-extract-provides — old
approach created provide frames whose continue handlers called
scope-pop!, corrupting the scope stack
2752/2768 OCaml tests pass (all 7 bind subscriber tests fixed).
32/32 WASM native tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Bytecode compiler now emits OP_PERFORM for (import ...) and compiles
(define-library ...) bodies. The VM stores the import request in
globals["__io_request"] and stops the run loop — no exceptions needed.
vm-execute-module returns a suspension dict, vm-resume-module continues.
Browser: sx_browser.ml detects suspension dicts from execute_module and
returns JS {suspended, op, request, resume} objects. The sx-platform.js
while loop handles cascading suspensions via handleImportSuspension.
13 modules load via .sxbc bytecode in 226ms (manifest-driven), both
islands hydrate, all handlers wired. 2650/2650 tests pass including
6 new vm-import-suspension tests.
Also: consolidated sx-platform-2.js → sx-platform.js, fixed
vm-execute-module missing code-from-value call, fixed bootstrap.py
protocol registry transpiler issues.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The transpiled VM (sx_vm_ref.ml, from lib/vm.sx) is now the ACTIVE
bytecode execution engine. sx_server.ml and sx_browser.ml call
Sx_vm_ref.execute_module instead of Sx_vm.execute_module.
Results:
- OCaml tests: 2644 passed, 0 failed
- WASM tests: 32 passed, 0 failed
- Browser: zero errors, zero warnings, islands hydrate
- Server: pages render, JIT compiles, all routes work
The VM logic now lives in ONE place: lib/vm.sx (SX).
OCaml gets it via transpilation (bootstrap_vm.py).
JS/browser gets it via bytecode compilation (compile-modules.js).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All 26 browser modules recompiled with define-library/import forms.
Compilation works without vm-compile-adapter (JIT pre-compilation
hangs with library wrappers in some JIT paths — skipped for now,
CEK compilation is ~34s total).
Key fixes:
- eval command: import-aware loop that handles define-library/import
locally without touching the Python bridge pipe (avoids deadlock)
- compile-modules.js: skip vm-compile-adapter, bump timeout
2621/2621 OCaml tests passing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
