Added vm-compile command: iterates env, compiles lambdas to bytecode,
replaces with NativeFn VM wrappers (with CEK fallback on error).
Tested: 3/109 compile, reduces CEK steps 23%.
Disabled auto-compile in production — the compiler doesn't handle
closures with upvalues yet, and compiled functions that reference
dynamic env vars crash. Infrastructure stays for when compiler
handles all SX features.
Also: added set-nth! and mutable-list primitives (needed by
compiler.sx for bytecode patching). Fixed compiler.sx to use
mutable lists on OCaml (ListRef for append!/set-nth! mutation).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After loading .sx files, (vm-compile) iterates all named lambdas,
compiles each body to bytecode, replaces with NativeFn VM wrapper.
Results: 3/109 functions compiled (compiler needs more features).
CEK steps: 49911 → 38083 (23% fewer) for home page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Eliminated double-aser for HTMX requests: build OOB wrapper AST
(~shared:layout/oob-sx :content wrapped_ast) and aser_slot in ONE
pass — same pattern as the full-page path. Halves aser_slot calls.
Added kernel-side timing to stderr:
[aser-slot] eval=3.6s io_flush=0.0s batched=3 result=22235 chars
Results show batch IO works (io_flush=0.0s for 3 highlight calls)
and the bottleneck is pure CEK evaluation time, not IO.
Performance after single-pass fix:
Home: 0.7s eval (was 2.2s total)
Reactive: 3.6s eval (was 6.8s total)
Language: 1.1s eval (was 18.9s total — double-aser eliminated)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OCaml kernel (sx_server.ml):
- Batch IO mode for aser-slot: batchable helpers (highlight,
component-source) return placeholders during evaluation instead
of blocking on stdin. After aser completes, all batched requests
are flushed to Python at once.
- Python processes them concurrently with asyncio.gather.
- Placeholders (using «IO:N» markers) are replaced with actual
values in the result string.
- Non-batchable IO (query, action, ctx, request-arg) still uses
blocking mode — their results drive control flow.
Python bridge (ocaml_bridge.py):
- _read_until_ok handles batched protocol: collects io-request
lines with numeric IDs, processes on (io-done N) with gather.
- IO result cache for pure helpers — eliminates redundant calls.
- _handle_io_request strips batch ID from request format.
Component caching (jinja_bridge.py):
- Hash computed from FULL component env (all names + bodies),
not per-page subset. Stable across all pages — browser caches
once, no re-download on navigation between pages.
- invalidate_component_hash() called on hot-reload.
Tests: 15/15 OCaml helper tests pass (2 new batch IO tests).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: OcamlBridge._send() used write() without drain().
asyncio.StreamWriter buffers writes — without drain(), multiple
commands accumulate and flush as a batch. The kernel processes
them sequentially, sending responses, but Python only reads one
response per command → pipe desync → "unexpected response" errors.
Fix: _send() is now async, calls drain() after every write.
All 14 callers updated to await.
Playwright tests rewritten:
- test_home_has_header: verifies #logo-opacity visible (was only
checking for "sx" text — never caught missing header)
- test_home_has_nav_children: Geography link must be visible
- test_home_has_main_panel: #main-panel must have child elements
- TestDirectPageLoad: fresh browser.new_context() per test to
avoid stale component hash in localStorage
- _setup_error_capture + _check_no_fatal_errors helpers
_render_to_sx uses aser_slot (not aser) — layout wrappers contain
re-parsed content that needs full expansion capability.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three fixes completing the aser_slot migration:
1. Single-pass full-page rendering: eval_sx_url builds layout+content
AST and aser_slots it in ONE call — avoids double-aser where
re-parsed content hits "Undefined symbol: title/deref" errors.
2. Pipe desync fix: _inject_helpers_locked runs INSIDE the aser_slot
lock acquisition (not as a separate lock). Prevents interleaved
commands from other coroutines between injection and aser-slot.
3. _render_to_sx uses aser_slot (not aser): layout wrappers like
oob_page_sx contain re-parsed content from earlier aser_slot
calls. Regular aser fails on symbols that were bound during
the earlier expansion. aser_slot handles them correctly.
HTMX path: aser_slot the content, then oob_page_sx wraps it.
Full page path: build (~shared:layout/app-body :content wrapped_ast),
aser_slot in one pass, pass directly to sx_page.
New Playwright tests: test_navigate_geography_to_reactive,
test_direct_load_reactive_page.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three bugs in aser-expand-component (adapter-sx.sx):
- Keyword values were eval'd (eval-expr can't handle <>, HTML tags);
now asered, matching the aser's rendering capabilities
- Missing default nil binding for unset &key params (caused
"Undefined symbol" errors for optional params like header-rows)
- aserCall string-quoted keyword values that were already serialized
SX — now inlines values starting with "(" directly
Server-affinity annotations for layout/nav shells:
- ~shared:layout/app-body, ~shared:layout/oob-sx — page structure
- ~layouts/nav-sibling-row, ~layouts/nav-children — server-side data
- ~layouts/doc already had :affinity :server
- ~cssx/flush marked :affinity :client (browser-only state)
Navigation fix: restore oob_page_sx wrapper for HTMX responses
so #main-panel section exists for sx-select/sx-swap targeting.
OCaml bridge: lazy page helper injection into kernel via IO proxy
(define name (fn (...) (helper "name" ...))) — enables aser_slot
to evaluate highlight/component-source etc. via coroutine bridge.
Playwright tests: added pageerror listener to test_no_console_errors,
new test_navigate_from_home_to_geography for HTMX nav regression.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ocaml_bridge: 10MB readline buffer for large spec explorer responses
- nav-data: evaluator.sx filename (was eval.sx, actual spec file is evaluator.sx)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Major architectural change: page function dispatch and handler execution
now go through the OCaml kernel instead of the Python bootstrapped evaluator.
OCaml integration:
- Page dispatch: bridge.eval() evaluates SX URL expressions (geography, marshes, etc.)
- Handler aser: bridge.aser() serializes handler responses as SX wire format
- _ensure_components loads all .sx files into OCaml kernel (spec, web adapter, handlers)
- defhandler/defpage registered as no-op special forms so handler files load
- helper IO primitive dispatches to Python page helpers + IO handlers
- ok-raw response format for SX wire format (no double-escaping)
- Natural list serialization in eval (no (list ...) wrapper)
- Clean pipe: _read_until_ok always sends io-response on error
SX adapter (aser):
- scope-emit!/scope-peek aliases to avoid CEK special form conflict
- aser-fragment/aser-call: strings starting with "(" pass through unserialized
- Registered cond-scheme?, is-else-clause?, primitive?, get-primitive in kernel
- random-int, parse-int as kernel primitives; json-encode, into via IO bridge
Handler migration:
- All IO calls converted to (helper "name" args...) pattern
- request-arg, request-form, state-get, state-set!, now, component-source etc.
- Fixed bare (effect ...) in island bodies leaking disposer functions as text
- Fixed lower-case → lower, ~search-results → ~examples/search-results
Reactive islands:
- sx-hydrate-islands called after client-side navigation swap
- force-dispose-islands-in for outerHTML swaps (clears hydration markers)
- clear-processed! platform primitive for re-hydration
Content restructuring:
- Design, event bridge, named stores, phase 2 consolidated into reactive overview
- Marshes split into overview + 5 example sub-pages
- Nav links use sx-get/sx-target for client-side navigation
Playwright test suite (sx/tests/test_demos.py):
- 83 tests covering hypermedia demos, reactive islands, marshes, spec explorer
- Server-side rendering, handler interactions, island hydration, navigation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
