Registering map/for-each/reduce as PRIMITIVES caused the compiler to
emit CALL_PRIM for them everywhere. The OCaml VM's call-primitive
can't invoke VM closures, causing "Undefined symbol: resource" crashes.
Revert vm.sx to original CALL_PRIM handler. Remove map/for-each/reduce
from JS PRIMITIVES so compiler emits OP_CALL instead (handled by
vm-call which dispatches correctly).
3 JS VM tests remain failing (VM closure interop) but production is stable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each demo island now has a live rendering and its highlighted SX source
code via (component-source), matching the geography/cek.sx pattern.
14 subsections covering all 15 demo islands.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- sx_browser.ml: use cek_call instead of sx_call in call-lambda to
avoid eval_expr deep-copying Dict values (breaks signal mutation)
- sx-browser.js: rebuilt with latest transpiler changes
- reactive-islands/index.sx: pretty-printed (no semantic changes)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Handle for-each/map/reduce in VM's CALL_PRIM handler instead of JS
primitives. VM closures are called via vm-call-closure (fresh VM,
shared upvalue cells). Native callables dispatch directly.
Named let doesn't work inside vm-step context — use explicit define
+ recursion for reduce loop.
JS tests: 1585/1585 (0 failures)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
dom-id returns "" for elements without an id attribute, and "" is
truthy in SX. This caused morph-children to build spurious entries
in old-by-id keyed by "", then match unrelated children via the
empty key — advancing oi past id-keyed children like #main-content
and skipping them in cleanup.
Three changes in morph-children (engine.sx):
- old-by-id reduce: normalize empty dom-id to nil so id-less
elements are excluded from the lookup dict
- match-id binding: normalize empty dom-id to nil so new children
without ids don't spuriously match old children
- Case 2 skip condition: use (not (empty? ...)) instead of bare
(dom-id old-child) to avoid treating "" as a real id
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add pretty-print, read-file, env-list-typed primitives to OCaml kernel.
Convert Python reference data (attrs, headers, events, primitives) to SX
data files. Implement page helpers (component-source, handler-source,
read-spec-file, reference-data, etc.) as pure SX functions.
The helper dispatcher in HTTP mode looks up named functions in the env
and calls them directly, replacing the Python IO bridge path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
signal-add-sub! used (append! subscribers f) which returns a new list
for immutable List but discards the result — after signal-remove-sub!
replaces the subscribers list via dict-set!, re-adding subscribers
silently fails. Counter island only worked once (0→1 then stuck).
Fix: use (dict-set! s "subscribers" (append ...)) to explicitly update
the dict field, matching signal-remove-sub!'s pattern.
Build pipeline fixes:
- sx-build-all.sh now bundles spec→dist and recompiles .sxbc bytecode
- compile-modules.js syncs .sx source files alongside .sxbc to wasm/sx/
- Per-file cache busting: wasm, platform JS, and sxbc each get own hash
- bundle.sh adds cssx.sx to dist
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three bugs in the DOM morph algorithm (web/engine.sx):
1. Empty-id keying: dom-id returns "" (not nil) for elements without an
id attribute, and "" is truthy in SX. Every id-less element was stored
under key "" in old-by-id, causing all new children to match the same
old element via the keyed branch — collapsing all children into one.
Fix: guard with (and id (not (empty? id))) in map building and matching.
2. Cleanup bug: the oi-cursor cleanup (range oi len) removed keyed elements
that were matched and moved from positions >= oi, and failed to remove
unmatched elements at positions < oi. Fix: track consumed indices in a
dict and remove all unconsumed elements regardless of position.
3. Island attr sync: morph-node delegated to morph-island-children without
first syncing the island element's own attributes (e.g. data-sx-state).
Fix: call sync-attrs before morph-island-children.
Also: pass explicit `true` to all dom-clone calls (deep clone parameter).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT compiler:
- Fix jit_compile_lambda: resolve `compile` via symbol lookup in env
instead of embedding VmClosure in AST (CEK dispatches differently)
- Register eval-defcomp/eval-defisland/eval-defmacro runtime helpers
in browser kernel for bytecoded defcomp forms
- Disable broken .sxbc.json path (missing arity in nested code blocks),
use .sxbc text format only
- Mark JIT-failed closures as sentinel to stop retrying
CSSX in browser:
- Add cssx.sx symlink + cssx.sxbc to browser web stack
- Add flush-cssx! to orchestration.sx post-swap for SPA nav
- Add cssx.sx to compile-modules.js and mcp_tree.ml bytecode lists
SPA navigation:
- Fix double-fetch: check e.defaultPrevented in click delegation
(bind-event already handled the click)
- Fix layout destruction: change nav links from outerHTML to innerHTML
swap (outerHTML destroyed #main-panel when response lacked it)
- Guard JS popstate handler when SX engine is booted
- Rename sx-platform.js → sx-platform-2.js to bust immutable cache
Playwright tests:
- Add trackErrors() helper to all test specs
- Add SPA DOM comparison test (SPA nav vs fresh load)
- Add single-fetch + no-duplicate-elements test
- Improve MCP tool output: show failure details and error messages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- AJAX requests (SX-Request: true) now render on main thread instead
of queueing behind slow full-page renders in worker pool
- Remove pushState from click handler — handle-history does it after
swap succeeds, preventing double-push that triggered popstate handler
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: _env_bind_hook mirrored ALL env_bind calls (including
lambda parameter bindings) to the shared VM globals table. Factory
functions like make-page-fn that return closures capturing different
values for the same param names (default-name, prefix, suffix) would
have the last call's values overwrite all previous closures' captured
state in globals. OP_GLOBAL_GET reads globals first, so all closures
returned the last factory call's values.
Fix: only sync root-env bindings (parent=None) to VM globals. Lambda
parameter bindings stay in their local env, found via vm_closure_env
fallback in OP_GLOBAL_GET.
Also in this commit:
- OP_CLOSURE propagates parent vm_closure_env to child closures
- Remove JIT globals injection (closure vars found via env chain)
- sx_server.ml: SX-Request header → returns text/sx (aser only)
- sx_server.ml: diagnostic endpoint GET /sx/_debug/{env,eval,route}
- sx_server.ml: page helper stubs for deep page rendering
- sx_server.ml: skip client-libs/ dir (browser-only definitions)
- adapter-html.sx: unknown components → HTML comment (not error)
- sx-platform.js: .sxbc fallback loader for bytecode modules
- Delete sx_http.ml (standalone HTTP server, unused)
- Delete stale .sxbc.json files (arity=0 bug, replaced by .sxbc)
- 7 new closure isolation tests in test-closure-isolation.sx
- mcp_tree.ml: emit arity + upvalue-count in .sxbc.json output
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sx_compiler.compile/compile_module now emit arity (local slot count) in
the bytecode dict. MCP sx_build_bytecode serializes arity into .sxbc.json.
sx-platform.js passes arity through to K.loadModule(). Without this, the
VM allocated only 16 local slots per module frame.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Browser kernel:
- Add `parse` native fn (matches server: unwrap single, list for multiple)
- Restore env==global_env guard on _env_bind_hook (let bindings must not
leak to _vm_globals — caused JIT CSSX "Not callable: nil" errors)
- Add _env_bind_hook call in env_set_id so set! mutations sync to VM globals
- Fire _vm_global_set_hook from OP_DEFINE so VM defines sync back to CEK env
CEK evaluator:
- Replace recursive cek_run with iterative while loop using sx_truthy
(previous attempt used strict Bool true matching, broke in wasm_of_ocaml)
- Remove dead cek_run_iterative function
Web modules:
- Remove find-matching-route and parse-route-pattern stubs from
boot-helpers.sx that shadowed real implementations from router.sx
- Sync boot-helpers.sx to dist/static dirs for bytecode compilation
Platform (sx-platform.js):
- Set data-sx-ready attribute after boot completes (was only in boot-init
which sx-platform.js doesn't call — it steps through boot manually)
- Add document-level click delegation for a[sx-get] links as workaround
for bytecoded bind-event not attaching per-element listeners (VM closure
issue under investigation — bind-event runs but dom-add-listener calls
don't result in addEventListener)
Tests:
- New test_kernel.js: 24 tests covering env sync, parse, route matching,
host FFI/preventDefault, deep recursion
- New navigation test: "sx-get link fetches SX not HTML and preserves layout"
(currently catches layout breakage after SPA swap — known issue)
Known remaining issues:
- JIT CSSX failures: closure-captured variables resolve to nil in VM bytecode
- SPA content swap via execute-request breaks page layout
- Bytecoded bind-event doesn't attach per-element addEventListener (root
cause unknown — when listen-target guard appears to block despite element
being valid)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The deployed sx_browser.bc.wasm.js was actually the js_of_ocaml output
(pure JS), not the wasm_of_ocaml loader. Nothing synced the correct
build output from _build/ to shared/static/wasm/.
- sx_build target=ocaml now auto-syncs WASM kernel + JS fallback + assets
- sx-build-all.sh syncs after dune build
- Correct 68KB WASM loader replaces 3.6MB JS imposter
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
boot-init now sets data-sx-ready on <html> and dispatches an sx:ready
CustomEvent after all islands are hydrated. Playwright tests use this
instead of networkidle + hard-coded sleeps (50+ seconds eliminated).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Iterative cek_run broke page-script parsing in browser. Reverted to
recursive — bytecode compilation overflow handled by native Sx_compiler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace Node.js compile-modules.js with direct Sx_compiler.compile_module
calls in mcp_tree.ml. No subprocess, no JIT warm-up, no Node.js.
23 files compile in 1.9 seconds.
Also includes rebuilt WASM kernel (iterative cek_run) and all 23
bytecode modules recompiled with native compiler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite compile-modules.js to use the native OCaml sx_server binary
instead of the js_of_ocaml kernel in Node.js. Compiles 23 modules in
23s (was 3+ minutes). Uses batch epoch protocol with latin1 encoding
to preserve byte positions for multi-byte UTF-8 content.
- Add compile-blob server command: parse source natively, compile via
SX compile-module, return bytecode dict
- Fix orchestration.sxbc.json and boot.sxbc.json — never compiled
successfully with the old JS kernel, now work with native compiler
- Auto-copy compiled bytecode to shared/static/wasm/sx/ for serving
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The CEK evaluator's cek_run was recursive (calls itself via cek_step).
Native OCaml handles deep recursion but wasm_of_ocaml compiles to JS
which has ~10K frame stack limit. Complex expressions (bytecode compiler)
exceeded this, causing "Maximum call stack size exceeded" in all WASM
bytecode compilation.
Replace recursive cek_run with iterative while loop — same semantics,
zero stack growth. Fixes sx_build_bytecode and browser-side evaluation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove prism.js, sweetalert2, body.js, sx-browser.js from shell —
only WASM kernel (sx_browser.bc.wasm.js + sx-platform.js) loads
- Restore request-handler.sx integration: SX handles routing + AJAX
detection, OCaml does aser → SSR → shell render pipeline
- AJAX fragment support: SX-Request header returns content fragment
(~14KB) instead of full page (~858KB), cached with "ajax:" prefix
- Fix language/applications/etc page functions to return empty fragment
instead of nil (was causing 404s)
- Shared JIT VM globals: env_bind hook mirrors ALL bindings to a single
shared globals table — eliminates stale-snapshot class of JIT bugs
- Add native `parse` function for components that need SX parsing
- Clean up unused shell params (sx-js-hash, body-js-hash, head-scripts,
body-scripts, use-wasm) from shell.sx, helpers.py, and server.ml
14/32 Playwright tests pass (navigation, SSR, isomorphic, geography).
Remaining failures are client-side (WASM bytecode 404s block hydration).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite geography-demos, demo-interactions, and reactive-nav specs
to use 300-500ms waits instead of 2-3s sleeps. Each test still does
a fresh page.goto() for isolation (no knock-on failures) but benefits
from server response cache for near-instant page loads.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore page pre-warming at HTTP server startup (was skipped) and
increase render workers from 2→4. Tighten Playwright timeouts and
run 3 workers in parallel for faster test runs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previous sed edits corrupted the file. Restored from 5c8b05a and
applied only the def-store change cleanly via python (no sed/sx-tools).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All OCaml files restored to the last known working state (5c8b05a).
All SX changes preserved and verified working with native server.
Docker compose updated to run sx_server.exe --http directly.
859KB homepage renders, 7/9 pages cached, ~30s startup (JIT fallback).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The sx-handle-request function (which routes URLs to page rendering)
was removed from request-handler.sx. Without it, the server's
http_render_page calls sx-handle-request which doesn't exist,
returning nil for every request → 500 errors.
Restored from the last known working version (394c86b).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The optimization to call the compiler through VM directly (instead
of CEK) when it's JIT-compiled was producing incorrect bytecode for
all subsequently compiled functions, causing "Expected number, got
symbol" errors across render-to-html, parse-loop, etc.
Revert to always using CEK for compilation. The compiler runs via
CEK which is slower but produces correct bytecode. JIT-compiled
USER functions still run at VM speed.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The iterative cek_run with Atomic step counting and debug prints
added overhead and complexity. The debug print called value_to_str
twice per million steps which could be very slow on large expressions.
Restore the original recursive cek_run from before the iterative
conversion. Remove the step limit mechanism (was causing render
timeouts). The recursive version is simpler and proven.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Removing injection broke GLOBAL_GET for all JIT-compiled functions,
not just mutable closures. Top-level functions like render-to-html
need their referenced bindings in the VM globals table.
Restore the original injection (only injects values not already in
globals). Mutable closure vars (parser's pos etc.) still get stale
snapshots and fall back to CEK — that's the known limitation to fix
with cell-based boxing later.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The SX pretty-printer was reformatting compiler.sx on every sx-tree
edit, subtly breaking JIT bytecode output. Restored the exact original
file from f3f70cc and added compile-match + dispatch entry using sed
(no sx-tree tools that trigger pretty-printing).
Also: stop injecting stale closure snapshots into VM globals — let
GLOBAL_GET fall through to vm_closure_env for live bindings.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: jit_compile_lambda copied closure variable VALUES into the
VM globals table. GLOBAL_GET found these stale snapshots instead of
falling through to vm_closure_env which has live bindings. When set!
mutated a variable (like parser's pos), the JIT code read the old
snapshot value.
Fix: don't inject closure bindings into globals at all. GLOBAL_GET
already has a fallback path that walks vm_closure_env — this sees
live env bindings that are updated by set!. No new opcodes needed.
This should fix JIT for parse-loop, skip-ws, read-expr and other
closure functions that use mutable variables.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The SX pretty-printer reformatted the entire compiler — different
indentation, removed comments, changed dict literals. This broke
JIT compilation for render-to-html and other functions that were
previously working.
Restore the exact original formatting from before the match commits
and add compile-match as a clean insertion (no reformatting). The
compiler's own dispatch stays as cond (safe with JIT).
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Atomic.fetch_and_add on every CEK step added unnecessary overhead.
The step counter is per-invocation (not shared across threads), so
a plain int ref with incr is sufficient and faster.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prints phase + control expression every 1,048,576 steps so infinite
loops become visible in logs. Only fires when step count is high
enough to indicate a real hang, not normal execution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Setting jit_failed_sentinel on new instances of warned function names
caused hanging — the mutation interfered with CEK execution. Now just
return None without touching l_compiled. The _jit_warned hash lookup
is a cheap check per call with no side effects.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New closure instances of functions like parse-loop get l_compiled=None
even though a previous instance already failed. Now check _jit_warned
by name and immediately sentinel-mark new instances of known failures.
The first failure is always logged. Subsequent instances are silently
sentinel-marked — no recompilation, no log spam, no suppressed errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Revert the _jit_warned check that silenced repeated JIT failures.
These errors indicate real JIT limitations (mutable closures) that
should remain visible until properly fixed. The sentinel on the
instance still prevents the same lambda from retrying.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The compiler's own match dispatch caused cascading JIT failures —
when compile-list is pre-compiled, the match bytecode is embedded
in it, and any subtle issue propagates to ALL subsequent compilations.
compile-list reverts to cond (battle-tested with JIT). compile-match
remains available for user code that uses match expressions.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a JIT-compiled function failed on first call, the function name
was added to _jit_warned but this was never checked before recompiling.
Closures like parse-loop create new lambda instances on each call,
each with l_compiled=None, triggering fresh compilation + failure
in an infinite loop.
Fix: check _jit_warned before attempting compilation, and mark the
lambda with jit_failed_sentinel on first-call failure so the same
instance also stops retrying.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
compile-match-clauses was a separate define that the JIT VM couldn't
find at runtime ("VM undefined: compile-match-clauses"). Inline it
as a letrec-bound local function inside compile-match so it's
captured in the closure and visible to JIT-compiled code.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The JIT compiler now handles the match special form, emitting the
same DUP/compare/JUMP_IF_FALSE bytecode pattern as case. Supports
literal patterns (string, number, boolean, nil), _ wildcard, symbol
binding, and quoted symbol patterns.
This fixes the infinite JIT retry loop where compile-list (which
used match for dispatch) couldn't be JIT-compiled, causing
parse-loop to endlessly fall back to CEK evaluation.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The match special form in compile-list caused JIT-compiled functions
to fail with "Expected number, got symbol" errors, creating an
infinite retry loop in parse-loop. The JIT compiler can't compile
match expressions, so compile-list fell back to CEK on every call.
Revert to the original cond-based dispatch. The evaluator.sx match
conversions are fine (not used by JIT), only the compiler was affected.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The home stepper's step-idx signal was not persisting across SX
navigation because set-cookie/freeze-to-sx wasn't working in the
WASM kernel. Replace with def-store which uses a global registry
that survives island re-hydration.
Also fix sx_http.exe build: add sx_http back to dune, inline scope
primitives (Sx_scope module was removed), add declarative form
stubs and render stubs, fix /sx/ home route mapping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
