Add pretty-print, read-file, env-list-typed primitives to OCaml kernel.
Convert Python reference data (attrs, headers, events, primitives) to SX
data files. Implement page helpers (component-source, handler-source,
read-spec-file, reference-data, etc.) as pure SX functions.
The helper dispatcher in HTTP mode looks up named functions in the env
and calls them directly, replacing the Python IO bridge path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
signal-add-sub! used (append! subscribers f) which returns a new list
for immutable List but discards the result — after signal-remove-sub!
replaces the subscribers list via dict-set!, re-adding subscribers
silently fails. Counter island only worked once (0→1 then stuck).
Fix: use (dict-set! s "subscribers" (append ...)) to explicitly update
the dict field, matching signal-remove-sub!'s pattern.
Build pipeline fixes:
- sx-build-all.sh now bundles spec→dist and recompiles .sxbc bytecode
- compile-modules.js syncs .sx source files alongside .sxbc to wasm/sx/
- Per-file cache busting: wasm, platform JS, and sxbc each get own hash
- bundle.sh adds cssx.sx to dist
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT compiler:
- Fix jit_compile_lambda: resolve `compile` via symbol lookup in env
instead of embedding VmClosure in AST (CEK dispatches differently)
- Register eval-defcomp/eval-defisland/eval-defmacro runtime helpers
in browser kernel for bytecoded defcomp forms
- Disable broken .sxbc.json path (missing arity in nested code blocks),
use .sxbc text format only
- Mark JIT-failed closures as sentinel to stop retrying
CSSX in browser:
- Add cssx.sx symlink + cssx.sxbc to browser web stack
- Add flush-cssx! to orchestration.sx post-swap for SPA nav
- Add cssx.sx to compile-modules.js and mcp_tree.ml bytecode lists
SPA navigation:
- Fix double-fetch: check e.defaultPrevented in click delegation
(bind-event already handled the click)
- Fix layout destruction: change nav links from outerHTML to innerHTML
swap (outerHTML destroyed #main-panel when response lacked it)
- Guard JS popstate handler when SX engine is booted
- Rename sx-platform.js → sx-platform-2.js to bust immutable cache
Playwright tests:
- Add trackErrors() helper to all test specs
- Add SPA DOM comparison test (SPA nav vs fresh load)
- Add single-fetch + no-duplicate-elements test
- Improve MCP tool output: show failure details and error messages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- AJAX requests (SX-Request: true) now render on main thread instead
of queueing behind slow full-page renders in worker pool
- Remove pushState from click handler — handle-history does it after
swap succeeds, preventing double-push that triggered popstate handler
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: _env_bind_hook mirrored ALL env_bind calls (including
lambda parameter bindings) to the shared VM globals table. Factory
functions like make-page-fn that return closures capturing different
values for the same param names (default-name, prefix, suffix) would
have the last call's values overwrite all previous closures' captured
state in globals. OP_GLOBAL_GET reads globals first, so all closures
returned the last factory call's values.
Fix: only sync root-env bindings (parent=None) to VM globals. Lambda
parameter bindings stay in their local env, found via vm_closure_env
fallback in OP_GLOBAL_GET.
Also in this commit:
- OP_CLOSURE propagates parent vm_closure_env to child closures
- Remove JIT globals injection (closure vars found via env chain)
- sx_server.ml: SX-Request header → returns text/sx (aser only)
- sx_server.ml: diagnostic endpoint GET /sx/_debug/{env,eval,route}
- sx_server.ml: page helper stubs for deep page rendering
- sx_server.ml: skip client-libs/ dir (browser-only definitions)
- adapter-html.sx: unknown components → HTML comment (not error)
- sx-platform.js: .sxbc fallback loader for bytecode modules
- Delete sx_http.ml (standalone HTTP server, unused)
- Delete stale .sxbc.json files (arity=0 bug, replaced by .sxbc)
- 7 new closure isolation tests in test-closure-isolation.sx
- mcp_tree.ml: emit arity + upvalue-count in .sxbc.json output
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sx_compiler.compile/compile_module now emit arity (local slot count) in
the bytecode dict. MCP sx_build_bytecode serializes arity into .sxbc.json.
sx-platform.js passes arity through to K.loadModule(). Without this, the
VM allocated only 16 local slots per module frame.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The deployed sx_browser.bc.wasm.js was actually the js_of_ocaml output
(pure JS), not the wasm_of_ocaml loader. Nothing synced the correct
build output from _build/ to shared/static/wasm/.
- sx_build target=ocaml now auto-syncs WASM kernel + JS fallback + assets
- sx-build-all.sh syncs after dune build
- Correct 68KB WASM loader replaces 3.6MB JS imposter
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace Node.js compile-modules.js with direct Sx_compiler.compile_module
calls in mcp_tree.ml. No subprocess, no JIT warm-up, no Node.js.
23 files compile in 1.9 seconds.
Also includes rebuilt WASM kernel (iterative cek_run) and all 23
bytecode modules recompiled with native compiler.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite compile-modules.js to use the native OCaml sx_server binary
instead of the js_of_ocaml kernel in Node.js. Compiles 23 modules in
23s (was 3+ minutes). Uses batch epoch protocol with latin1 encoding
to preserve byte positions for multi-byte UTF-8 content.
- Add compile-blob server command: parse source natively, compile via
SX compile-module, return bytecode dict
- Fix orchestration.sxbc.json and boot.sxbc.json — never compiled
successfully with the old JS kernel, now work with native compiler
- Auto-copy compiled bytecode to shared/static/wasm/sx/ for serving
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Remove prism.js, sweetalert2, body.js, sx-browser.js from shell —
only WASM kernel (sx_browser.bc.wasm.js + sx-platform.js) loads
- Restore request-handler.sx integration: SX handles routing + AJAX
detection, OCaml does aser → SSR → shell render pipeline
- AJAX fragment support: SX-Request header returns content fragment
(~14KB) instead of full page (~858KB), cached with "ajax:" prefix
- Fix language/applications/etc page functions to return empty fragment
instead of nil (was causing 404s)
- Shared JIT VM globals: env_bind hook mirrors ALL bindings to a single
shared globals table — eliminates stale-snapshot class of JIT bugs
- Add native `parse` function for components that need SX parsing
- Clean up unused shell params (sx-js-hash, body-js-hash, head-scripts,
body-scripts, use-wasm) from shell.sx, helpers.py, and server.ml
14/32 Playwright tests pass (navigation, SSR, isomorphic, geography).
Remaining failures are client-side (WASM bytecode 404s block hydration).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Restore page pre-warming at HTTP server startup (was skipped) and
increase render workers from 2→4. Tighten Playwright timeouts and
run 3 workers in parallel for faster test runs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All OCaml files restored to the last known working state (5c8b05a).
All SX changes preserved and verified working with native server.
Docker compose updated to run sx_server.exe --http directly.
859KB homepage renders, 7/9 pages cached, ~30s startup (JIT fallback).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The iterative cek_run with Atomic step counting and debug prints
added overhead and complexity. The debug print called value_to_str
twice per million steps which could be very slow on large expressions.
Restore the original recursive cek_run from before the iterative
conversion. Remove the step limit mechanism (was causing render
timeouts). The recursive version is simpler and proven.
1166 passed, 0 failed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Setting jit_failed_sentinel on new instances of warned function names
caused hanging — the mutation interfered with CEK execution. Now just
return None without touching l_compiled. The _jit_warned hash lookup
is a cheap check per call with no side effects.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New closure instances of functions like parse-loop get l_compiled=None
even though a previous instance already failed. Now check _jit_warned
by name and immediately sentinel-mark new instances of known failures.
The first failure is always logged. Subsequent instances are silently
sentinel-marked — no recompilation, no log spam, no suppressed errors.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Revert the _jit_warned check that silenced repeated JIT failures.
These errors indicate real JIT limitations (mutable closures) that
should remain visible until properly fixed. The sentinel on the
instance still prevents the same lambda from retrying.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a JIT-compiled function failed on first call, the function name
was added to _jit_warned but this was never checked before recompiling.
Closures like parse-loop create new lambda instances on each call,
each with l_compiled=None, triggering fresh compilation + failure
in an infinite loop.
Fix: check _jit_warned before attempting compilation, and mark the
lambda with jit_failed_sentinel on first-call failure so the same
instance also stops retrying.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The home stepper's step-idx signal was not persisting across SX
navigation because set-cookie/freeze-to-sx wasn't working in the
WASM kernel. Replace with def-store which uses a global registry
that survives island re-hydration.
Also fix sx_http.exe build: add sx_http back to dune, inline scope
primitives (Sx_scope module was removed), add declarative form
stubs and render stubs, fix /sx/ home route mapping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Island markers rendered during SX navigation responses had no
data-sx-state attribute, so hydration found empty kwargs and path
was nil in the copyright display. Now adapter-dom.sx serializes
keyword args into data-sx-state on island markers, matching what
adapter-html.sx does for SSR.
Also fix post-swap to use parent element for outerHTML swaps in
SX responses (was using detached old target). Add SX source file
hashes to wasm_hash for proper browser cache busting — changing
any .sx file now busts the cache. Remove stale .sxbc bytecode
cache files.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Convert large cond chains doing string equality dispatch to use the
match special form: step-eval-list (42 arms), step-continue (31 arms),
compile-list (30 arms), ho-setup-dispatch (7 arms), value-matches-type?
(10 arms). Also fix test-canonical.sx to use defsuite/deftest format
and load canonical.sx in both test runners.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New web/request-handler.sx: configurable SX function (sx-handle-request)
that receives path + headers + env and returns rendered HTML.
The handler decides full page vs AJAX fragment.
OCaml server: http_render_page now just calls the SX handler.
All routing, layout selection, AJAX detection moved to SX.
Header parsing added. is_sx_request removed from OCaml.
Configurable via SX_REQUEST_HANDLER env var (default: sx-handle-request).
WIP: handler has parse errors on some URL formats. Needs debugging.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server always returns full page. Client sx-select handles extraction.
No application logic in OCaml server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AJAX fragment: extract #main-panel with matching close tag (depth
tracking) instead of taking everything to end of file. Prevents
shell closing tags from breaking the DOM swap.
Back button test: verifies content actually changes — checks for
"Geography" and "Rendering Pipeline" after going back, not just
that body has >100 chars. Tests forward nav content change too.
7/7 navigation tests pass including back button content verification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace blocking domain pool with non-blocking architecture:
- Main accept loop handles ALL connections immediately
- Cached responses: served in microseconds from main loop (no queuing)
- Static files: served immediately from main loop
- Cache misses: queued to render worker pool (domain workers)
- Socket timeouts (5s recv, 10s send) prevent connection hangs
- TCP backlog increased to 1024
No more connection resets under load. 22/26 Playwright tests pass
(4 failures from stale worktree test copies, 0 from main tree).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AJAX navigation: detect SX-Request/HX-Request headers and return just
the #main-panel fragment instead of the full page shell. Fixes layout
break where header and content appeared side-by-side after navigation.
New navigation test suite (tests/playwright/navigation.spec.js):
- layout stays vertical after clicking nav link
- content updates after navigation
- no raw SX component calls visible after navigation
- header island survives navigation
- full page width is used (no side-by-side split)
All 5 tests pass. 14 total Playwright tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move client library sources (cssx.sx) before defcomp/defisland
definitions in component-defs so defines are evaluated first.
Identified root cause of CSSX "Not callable: nil" errors:
JIT compiler captures free variable values at compile time instead of
looking them up at runtime from vm_globals. When ~cssx/tw's JIT code
calls cssx-process-token, it uses the compile-time snapshot (nil)
instead of the runtime value (lambda). The function IS in global_env
(type-of returns "lambda") but the JIT bytecode doesn't see it.
Fix needed: JIT compiler should emit GLOBAL_GET instructions for free
variables that reference vm_globals at runtime, not capture at compile.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch SSR + shell from native Sx_render to SX adapter (render-to-html
from adapter-html.sx) via CEK evaluator. Handles reactive primitives
(signals, deref, computed) for island bodies. Native renderer as fallback.
Header island SSRs correctly: (<sx>), tagline, copyright, path.
Stepper renders body but ~cssx/tw shows as raw SX (client-affinity
component not expanded server-side, client not expanding either).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Island SSR reverted to placeholder approach (full SSR crashed on
reactive code). Placeholders include SX call expression for client
hydration. CSS hides placeholder text until WASM renders.
Remaining: need SX adapter (adapter-html.sx) for island SSR instead
of native Sx_render — the SX adapter handles signals/deref/computed
via the CEK machine. Native renderer doesn't support reactive primitives.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Include cssx.sx source in component-defs for client CSSX runtime.
Island placeholders now contain SX call expression for client hydration.
escape_sx_string only escapes </script, not all </ sequences.
serialize_value: no (list) wrapper, matching Python serialize() exactly.
Homepage: header renders (<sx>, tagline, copyright), stepper shows
raw SX (cssx/tw not expanding client-side). Geography fully rendered
including island hydration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
serialize_value: all lists emit (items...) not (list items...), matching
Python serialize() exactly. Empty lists emit (). This fixes let bindings,
fn params, and data structures for client-side parsing.
Component-defs now include named lambdas, macros, dicts, and other named
values from the env — client needs CSSX functions (cssx-process-token,
cssx-colour-props, cssx-spacing-props etc.) for island hydration.
Fixes: cssx-process-token, cssx-colour-props undefined errors.
Geography page: fully rendered with header island hydration working.
Homepage: nav renders, no error banners, stepper silent.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
serialize_value: lists with symbol head emit (fn args...) not (list fn args).
This lets the client SX parser recognise special forms like let, when, cond
inside component-defs. Data lists (starting with non-symbols) keep the
(list ...) wrapper.
Fixes "Undefined symbol: let" WASM error that broke all island hydration.
Header island now reaches the JIT evaluator (fails on for-each VM error,
not parsing). Geography, SXTP, CEK pages fully rendered.
Removed debug logging (ssr-debug, debug dump, debug endpoint).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Homepage fix: path_expr "home" was evaluated as a symbol lookup (returning
the Lambda) instead of a function call. Now wraps bare symbols in a list:
home → (home) → calls the page function → returns component call.
Slug routing: auto_quote converts unknown symbols to strings before eval.
(etc (plan sx-host)) → (etc (plan "sx-host")) — resolves nested slug URLs.
Resilient SSR: render_to_buf catches Eval_error per-element and continues
rendering. Partial SSR output preserved even when some elements fail.
WASM kernel rebuilt (define shorthand + island placeholder changes).
Remaining: WASM kernel "Undefined symbol: let" — pre-existing bug in
client-side component-defs parsing. (list let ...) triggers symbol lookup
instead of special form recognition. Affects island hydration on all pages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
escape_sx_string now escapes </ as <\\/ inside SX string literals,
matching Python's serialize() behavior. This prevents the HTML parser
from matching </script> inside component-defs strings while keeping
the SX valid for the client parser.
Component-defs back to inline <script data-components> (reverts
external endpoint approach). init-sx triggers client render when
sx-root is empty.
Geography page: fully rendered with header, nav, content, styling.
Header island hydration warning (Undefined symbol: let) is a
pre-existing WASM kernel issue, not related to the HTTP server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Islands now always emit <span data-sx-island="name"> placeholder
instead of attempting SSR. Island bodies contain client-only symbols
(signals, DOM refs) that cascade errors during native render.
Component-defs moved to /static/sx-components.sx endpoint instead of
inline <script> — avoids </script> escaping issues that broke the
client-side SX parser.
Remaining: client WASM kernel not loading components from external
endpoint (expects inline script tag). Need to configure client boot
to fetch from /static/sx-components.sx.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Escape </script sequences in component-defs (replace </ with <\/ before s/S)
- Build pages-sx registry from defpage definitions (51 routes for client router)
- SSR fallback: emit minimal layout HTML when full SSR fails
- Redirect / → /sx/
- Load sxc/ components for ~docs/page
- Block .wasm.assets/ build artifacts but allow .wasm runtime files
Geography page renders correctly with full styling and content.
Homepage still blank — client boot doesn't render from page-sx on
initial load (only on navigation). Needs homepage SSR to succeed,
which requires fixing the stepper island's <home symbol.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OCaml HTTP server now serves /static/* files with MIME types, immutable
cache headers, and in-memory caching. Computes MD5 hashes for JS/WASM
files and injects them as ?v= cache-busting query params.
Inline CSS: reads tw.css + basics.css at startup, injects into
<style id="sx-css"> tag. Pages now render with full Tailwind styling.
Shell statics now include real file hashes:
sx-js-hash, body-js-hash, wasm-hash — 12-char MD5 hex
Docker compose: mounts shared/static as /app/static for the container.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Load sx/sxc/ directory for core layout components like ~docs/page,
~docs/section, ~docs/code. Auto-detect Docker (/app/sxc) vs dev
(/project/sx/sxc) paths with SX_SXC_DIR env override.
Fixes pages that use ~docs/page — all content pages now render
correctly with full component expansion.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
docker-compose.dev-sx-native.yml overrides entrypoint to run OCaml
HTTP server inside Docker container (Caddy on externalnet can reach it).
Auto-detect sx component directory: checks /app/sx (Docker) first,
falls back to /project/sx/sx (dev). SX_COMPONENTS_DIR env override.
Live on sx.rose-ash.com via Caddy → Docker → OCaml HTTP server.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Render 5 key pages during startup to trigger JIT compilation of page
functions, aser, render-to-html, and all component paths. Steady-state
performance after warmup:
Homepage: 212-280ms (aser=136-173ms, ssr=44-90ms)
Geography: 282-354ms (aser=215-273ms, ssr=44-61ms)
Reactive: 356-522ms (aser=306-390ms, ssr=38-109ms)
SSR phase is fast (streaming renderer). Aser is the bottleneck —
response caching eliminates it for static pages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New render_to_buf / render_to_html_streaming in sx_render.ml — writes
HTML directly to a Buffer.t instead of building intermediate strings.
Eliminates hundreds of string concatenations per page render.
Full parallel renderer: render_to_buf, render_element_buf,
render_component_buf, render_cond_buf, render_let_buf, render_map_buf,
render_for_each_buf — all buffer-native.
HTTP server SSR + shell now use streaming renderer.
Performance (warm, 2 worker domains, 2MB RSS):
Homepage: 138-195ms TTFB (Quart: 202ms) — faster
Geography: 218-286ms TTFB (Quart: 144ms)
Throughput: 6.85 req/s at c=5 (Quart: 6.8 req/s) — matched
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
JIT runtime errors now log once per function name via _jit_warned
hashtable, then stay quiet for that function. No more silent swallowing
(which hid real errors) or per-call flooding (which spammed thousands
of lines and blocked the server).
VM-level fallbacks (inside JIT-compiled code calling other JIT code)
are silent — dedup happens at the hook level.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
