Commit Graph

401 Commits

Author SHA1 Message Date
a823e59376 Fix root cause: skip JIT for closure lambdas in BOTH hook and vm_call
The closure check was only in vm_call (sx_vm.ml) but inner functions
like read-list-loop were also compiled through the JIT hook in
sx_server.ml. The hook compiled them with closure merging, producing
incorrect bytecode (read-list-loop mishandled closing parens).

Added the same closure check to the JIT hook: skip lambdas with
non-empty closures. Now sx-parse works correctly:
  (a (b) (c)) → 3 siblings, not (a (b (c)))

Pre-compiled count increased from 17 to 33 — more top-level
functions compiled (inner ones correctly skipped to CEK).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 19:24:14 +00:00
96f50b9dfa Add sibling sublist parser tests + reset JIT sx-parse
Tests: parse "(a (b) (c))" must produce 3 siblings, not nested.
Catches JIT compilation bug where closing parens cause sibling
lists to become children.

Reset sx-parse to CEK on the OCaml kernel — the JIT-compiled
version of sx-parse's complex letrec produces wrong bytecode.
CEK interpretation works correctly (tests pass on all platforms).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 19:17:05 +00:00
2727a2ed8c Skip JIT for lambdas with closure bindings
The closure merging in jit_compile_lambda (copying globals + injecting
closure bindings into vm_env_ref) produces incorrect variable resolution
for inner functions. Symptoms: sx-parse's read-list-loop mishandles
closing parens (siblings become children), parser produces wrong ASTs.

Fix: vm_call skips JIT compilation for lambdas with non-empty closures.
These run on CEK which handles closures correctly. Top-level defines
(empty closure) are still JIT-compiled.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 18:45:40 +00:00
c4224925f9 CSSX flush appends to persistent head stylesheet
~cssx/flush now appends rules to <style id="sx-css"> in <head>
instead of creating ephemeral inline <style> tags that get morphed
away during SPA navigation. Rules accumulate across navigations.

Future: reference-count rules and remove when no elements use them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 18:19:04 +00:00
fe84b57bed Move CSSX rules to <head>, skip client-affinity components in SSR
- Shell inlines CSSX flush logic in <head> (collected/clear-collected!)
  so island CSS rules survive #main-panel morphs during SPA navigation
- OCaml render_to_html skips :affinity :client components during
  Phase 1b SSR (prevents ~cssx/flush rendering inside body-html)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 18:14:43 +00:00
9ff913c312 Fix root cause: parse-int in primitives table handles 2-arg form
The CSSX colour resolution failure was NOT a JIT compiler bug.
CALL_PRIM looks up primitives table (not env), and parse-int in
the primitives table only handled 1-arg calls. The 2-arg form
(parse-int "699" nil) returned Nil, causing cssx-resolve's colour
branch to fail its and-condition.

Fix: update Sx_primitives.register "parse-int" with same 2-arg
handling as the env binding. Remove the vm-reset-fn workaround.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 17:02:14 +00:00
b1de591e9e Fix CSSX colour rules: reset cssx-resolve JIT to force CEK
cssx-resolve has a complex cond with nested and conditions that the
JIT compiler miscompiles — the colour branch is skipped even when
all conditions are true. Reset to jit_failed_sentinel after loading
so it runs on CEK (which evaluates correctly).

Added vm-reset-fn kernel command for targeted JIT bypass.

All CSSX colour tokens now generate rules: text-violet-699,
text-stone-500, bg-stone-50, etc.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 16:49:49 +00:00
f4610e1799 Fix thunk handling for island SSR + effect no-op on server
- trampoline resolves Thunk values (sf-letrec returns them for TCO)
- render-to-html handles "thunk" type by unwrapping expr+env
- effect overridden to no-op after loading signals.sx (prevents
  reactive loops during SSR — effects are DOM side-effects)
- Added thunk?/thunk-expr/thunk-env primitives
- Added DOM API stubs for SSR (dom-query, schedule-idle, etc.)

Header island renders fully with styling. Stepper island still
fails SSR (letrec + complex body hits "Undefined symbol: div"
in eval path — render mode not active during CEK letrec eval).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 15:31:58 +00:00
f3c0cbd8e2 CSSX rules from island SSR: flush collected rules via ~cssx/flush in shell
Added (~cssx/flush) to shell after sx-root div — picks up CSS rules
generated during island SSR via (collect! "cssx" ...). Registered
clear-collected! primitive for the flush component.

Standard CSSX classes now styled server-side. Custom colour shades
(e.g. text-violet-699) still need investigation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 14:53:31 +00:00
6e1d28d1d7 Load freeze.sx + browser API stubs for complete island SSR
All islands now render server-side:
- freeze.sx loaded into kernel (freeze-scope for home/stepper)
- Browser-only APIs stubbed (local-storage-get/set, dom-listen,
  dom-dispatch, dom-set-data, dom-get-data, promise-then)
  → return nil on server, client hydrates with real behavior

Zero island failures. Both layouts/header and home/stepper render
with hydration markers, CSS classes, and initial signal values.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 14:44:51 +00:00
92bfef6406 Island SSR: defislands render to HTML server-side with hydration markers
Islands now render their initial state as HTML on the server, like
React SSR. The client hydrates with reactive behavior on boot.

Root causes fixed:
- is_signal/signal_value now recognize Dict-based signals (from
  signals.sx) in addition to native Signal values
- Register "context" as a primitive so the CEK deref frame handler
  can read scope stacks for reactive tracking
- Load adapter-html.sx into kernel for SX-level render-to-html
  (islands use this instead of the OCaml render module)
- Component accessors (params, body, has-children?, affinity) handle
  Island values with ? suffix aliases
- Add platform primitives: make-raw-html, raw-html-content,
  empty-dict?, for-each-indexed, cek-call

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 14:33:04 +00:00
894321db18 Isomorphic SSR: server renders HTML body, client takes over with SX
Server now renders page content as HTML inside <div id="sx-root">,
visible immediately before JavaScript loads. The SX source is still
included in a <script data-mount="#sx-root"> tag for client hydration.

SSR pipeline: after aser produces the SX wire format, parse and
render-to-html it (~17ms for a 22KB page). Islands with reactive
state gracefully fall back to empty — client hydrates them.

Supporting changes:
- Load signals.sx into OCaml kernel (reactive primitives for island SSR)
- Add cek-call and context to kernel env (needed by signals/deref)
- Island-aware component accessors in sx_types.ml
- render-to-html handles Island values (renders as component with fallback)
- Fix 431 (Request Header Fields Too Large): replace SX-Components
  header (full component name list) with SX-Components-Hash (12 chars)
- CORS allow SX-Components-Hash header

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 14:01:41 +00:00
318c818728 Lazy JIT compilation: lambdas compile to bytecode on first call
Replace AOT adapter compilation with lazy JIT — each named lambda is
compiled to VM bytecode on first call, cached in l_compiled field for
subsequent calls. Compilation failures fall back to CEK gracefully.

VM types (vm_code, vm_upvalue_cell, vm_closure) moved to sx_types.ml
mutual recursion block. Lambda and Component records gain mutable
l_compiled/c_compiled cache fields. jit_compile_lambda in sx_vm.ml
wraps body as (fn (params) body), invokes spec/compiler.sx via CEK,
extracts inner closure from OP_CLOSURE constant.

JIT hooks in both paths:
- vm_call: Lambda calls from compiled VM code
- continue_with_call: Lambda calls from CEK step loop (injected by
  bootstrap.py post-processing)

Pre-mark sentinel prevents re-entrancy (compile function itself was
hanging when JIT'd mid-compilation). VM execution errors caught and
fall back to CEK with sentinel marking.

Also: add kbd/samp/var to HTML_TAGS, rebuild sx-browser.js, add page
URL to sx-page-full-py timing log.

Performance: first page 28s (JIT compiles 17 functions), subsequent
pages 0.31s home / 0.71s wittgenstein (was 2.3s). All 1945 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:18:44 +00:00
bb34b4948b OCaml raw! in HTML renderer + SX_USE_OCAML env promotion + golden tests
- sx_render.ml: add raw! handler to HTML renderer (inject pre-rendered
  content without HTML escaping)
- docker-compose.yml: move SX_USE_OCAML/SX_OCAML_BIN to shared env
  (available to all services, not just sx_docs)
- hosts/ocaml/Dockerfile: OCaml kernel build stage
- shared/sx/tests/: golden test data + generator for OCaml render tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 22:21:04 +00:00
df461beec2 SxExpr aser wire format fix + Playwright test infrastructure + blob protocol
Aser serialization: aser-call/fragment now return SxExpr instead of String.
serialize/inspect passes SxExpr through unquoted, preventing the double-
escaping (\" → \\\" ) that broke client-side parsing when aser wire format
was output via raw! into <script> tags. Added make-sx-expr + sx-expr-source
primitives to OCaml and JS hosts.

Binary blob protocol: eval, aser, aser-slot, and sx-page-full now send SX
source as length-prefixed blobs instead of escaped strings. Eliminates pipe
desync from concurrent requests and removes all string-escape round-trips
between Python and OCaml.

Bridge safety: re-entrancy guard (_in_io_handler) raises immediately if an
IO handler tries to call the bridge, preventing silent deadlocks.

Fetch error logging: orchestration.sx error callback now logs method + URL
via log-warn. Platform catches (fetchAndRestore, fetchPreload, bindBoostForm)
also log errors instead of silently swallowing them.

Transpiler fixes: makeEnv, scopePeek, scopeEmit, makeSxExpr added as
platform function definitions + transpiler mappings — were referenced in
transpiled code but never defined as JS functions.

Playwright test infrastructure:
- nav() captures JS errors and fails fast with the actual error message
- Checks for [object Object] rendering artifacts
- New tests: delete-row interaction, full page refresh, back button,
  direct load with fresh context, code block content verification
- Default base URL changed to localhost:8013 (standalone dev server)
- docker-compose.dev-sx.yml: port 8013 exposed for local testing
- test-sx-build.sh: build + unit tests + Playwright smoke tests

Geography content: index page component written (sx/sx/geography/index.sx)
describing OCaml evaluator, wire formats, rendering pipeline, and topic
links. Wiring blocked by aser-expand-component children passing issue.

Tests: 1080/1080 JS, 952/952 OCaml, 66/66 Playwright

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 22:17:43 +00:00
6d73edf297 Length-prefixed binary framing for OCaml↔Python pipe
Replace newline-delimited text protocol with length-prefixed blobs
for all response data (send_ok_string, send_ok_raw). The OCaml side
sends (ok-len N)\n followed by exactly N raw bytes + \n. Python reads
the length, then readexactly(N).

This eliminates all pipe desync issues:
- No escaping needed for any content (HTML, SX with newlines, quotes)
- No size limits (1MB+ responses work cleanly)
- No multi-line response splitting
- No double-escaping bugs

The old (ok "...") and (ok-raw ...) formats are still parsed as
fallbacks for backward compatibility.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:48:52 +00:00
373a4f0134 Fix pipe desync: send_ok_raw escapes newlines, expand-components? in env
- send_ok_raw: when SX wire format contains newlines (string literals),
  fall back to (ok "...escaped...") instead of (ok-raw ...) to keep
  the pipe single-line. Prevents multi-line responses from desyncing
  subsequent requests.
- expand-components? flag set in kernel env (not just VM adapter globals)
  so aser-list's env-has? check finds it during component expansion.
- SX_STANDALONE: restore no_oauth but generate CSRF via session cookie
  so mutation handlers (DELETE etc.) still work without account service.
- Shell statics injection: only inject small values (hashes, URLs) as
  kernel vars. Large blobs (CSS, component_defs) use placeholder tokens.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:32:03 +00:00
ae0e87fbf8 VM aser-slot → sx-page-full: single-call page render, 0.55s warm
Compiler fixes:
- Upvalue re-lookup returns own position (uv-index), not parent slot
- Spec: cek-call uses (make-env) not (dict) — OCaml Dict≠Env
- Bootstrap post-processes transpiler Dict→Env for cek_call

VM runtime fixes:
- compile_adapter evaluates constant defines (SPECIAL_FORM_NAMES etc.)
  via execute_module instead of wrapping as NativeFn closures
- Native primitives: map-indexed, some, every?
- Nil-safe HO forms: map/filter/for-each/some/every? accept nil as empty
- expand-components? set in kernel env (not just VM globals)
- unwrap_env diagnostic: reports actual type received

sx-page-full command:
- Single OCaml call: aser-slot body + render-to-html shell
- Eliminates two pipe round-trips (was: aser-slot→Python→shell render)
- Shell statics (component_defs, CSS, pages_sx) cached in Python,
  injected into kernel once, referenced by symbol in per-request command
- Large blobs use placeholder tokens — Python splices post-render,
  pipe transfers ~51KB instead of 2MB

Performance (warm):
- Server total: 0.55s (was ~2s)
- aser-slot VM: 0.3s, shell render: 0.01s, pipe: 0.06s
- kwargs computation: 0.000s (cached)

SX_STANDALONE mode for sx_docs dev (skips fragment fetches).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:06:04 +00:00
231bfbecb5 VM aser-slot routing: isolated globals, inner code extraction, debug
aser-slot now routes through the VM when adapter is compiled:
- compile_adapter: compiles each define body, extracts inner code
  from OP_CLOSURE wrapper, stores as NativeFn in separate globals
- vm_adapter_globals: isolated from kernel env (no cross-contamination)
- aser-slot checks vm_adapter_globals, calls VM aser directly

Status: 2/12 adapter functions compile and run on VM. 6 fail during
OCaml-side compilation with "index out of bounds" — likely from
set-nth! silent failure on ListRef during bytecode jump patching.

Debug output shows outer code structure is correct (4 bytes, 1 const).
Inner code_from_value conversion needs fixing for nested closures.

Also: vm-compile-adapter command inside _ensure_components lock
(fixes pipe desync from concurrent requests).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 22:18:21 +00:00
df256b5607 VM adapter: compile works, env isolation needed
adapter-sx.sx compiles to 25 code objects (4044 bytes bytecode).
vm-load-module loads it. But replacing Lambda values in env.bindings
with NativeFn wrappers breaks the CEK machine for non-aser functions.

Root cause: shared env.bindings between CEK and VM. The CEK needs
Lambda values (for closure merging). The VM needs NativeFn wrappers.
Both can't coexist in the same env.

Fix needed: VM adapter gets its own globals table (with compiled
closures). The aser-slot command routes directly to the VM with
its own globals, not through the CEK with shared env.

Disabled vm-load-module. Pages render correctly via CEK.

Also: OP_CALL_PRIM now logs primitive name + argc in error messages
for easier debugging.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 21:36:38 +00:00
0ce23521b7 Aser adapter compiles + loads as VM module — first VM execution
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 21:18:34 +00:00
f12bbae6c9 40/40 VM tests pass, auto-compile disabled until full aser compilation
All VM tests green: closures with shared mutable upvalues, map/filter/
for-each via CALL_PRIM, recursive functions, nested closures.

Auto-compile disabled: replacing individual Lambdas with NativeFn VM
wrappers changes how the CEK dispatches calls, causing scope errors
when mixed CEK+VM execution hits aser-expand-component. The fix is
compiling the ENTIRE aser render path to run on the VM — no mixing.

The VM infrastructure is complete and tested. Next step: compile
adapter-sx.sx as a whole module, run the aser on the VM.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 20:57:59 +00:00
c8c4b322a9 All 40 VM tests pass: map/filter/for-each + mutable closures fixed
Two fixes:

1. HO forms (map/filter/for-each/reduce): registered as Python
   primitives so compiler emits OP_CALL_PRIM (direct dispatch to
   OCaml primitive) instead of OP_CALL (which routed through CEK
   HO special forms and failed on NativeFn closure args).

2. Mutable closures: locals captured by closures now share an
   upvalue_cell. OP_LOCAL_GET/SET check frame.local_cells first —
   if the slot has a shared cell, read/write through it. OP_CLOSURE
   creates or reuses cells for is_local=1 captures. Both parent
   and closure see the same mutations.

   Frame type extended with local_cells hashtable for captured slots.

40/40 tests pass:
  - 12 compiler output tests
  - 18 VM execution tests (arithmetic, control flow, closures,
    nested let, higher-order, cond, string ops)
  - 10 auto-compile pattern tests (recursive, map, filter,
    for-each, mutable closures, multiple closures, type dispatch)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 20:47:40 +00:00
e7da397f8e VM upvalues + HO primitives + 40 tests (36 pass, 4 fail)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 20:41:23 +00:00
1bb40415a8 VM upvalue support: closures capture variables from enclosing scopes
Compiler (compiler.sx):
- Function scopes marked is-function=true; let scopes share parent frame
- scope-resolve only creates upvalue captures at function boundaries
- Let scope locals use parent's slot numbering (same frame)
- OP_CLOSURE emits upvalue descriptors: (is_local, index) per capture

VM (sx_vm.ml):
- upvalue_cell type: shared mutable reference to captured value
- OP_UPVALUE_GET/SET: read/write from closure's upvalue array
- OP_CLOSURE: reads upvalue descriptors, creates cells from
  enclosing frame's locals (is_local=1) or upvalues (is_local=0)
- vm_closure carries live env_ref (not snapshot)
- vm_call falls back to CEK for Lambda/Component/Island values

Verified: (let ((x 10)) (let ((add-x (fn (y) (+ x y)))) (add-x 5)))
  Compiles to: CONST 10, LOC_SET #0, CLOSURE [UV_GET#0 LOC_GET#0 CPRIM+ RET]
  with upvalue descriptor: is_local=1 index=0
  VM executes → 15 ✓

Auto-compile: 6/117 functions compile (up from 3). Disabled until
compiler handles all features — fallback can't reconstruct closure
scope for variables like nav-state bound in caller's let*.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 20:13:17 +00:00
a62b7c8a5e Disable auto-compile until CEK fallback is robust
The vm-compile replaces Lambda values with NativeFn wrappers.
When the VM can't execute (missing env vars, unsupported ops),
it falls back to cek_call. But cek_call needs proper Env values
that the snapshot doesn't provide.

Fix needed: VM closures must capture the LIVE env (not snapshot),
or the CEK fallback must construct a proper Env from the globals.
Disabling until this is resolved.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 19:37:23 +00:00
5ca2ee92bc VM auto-compile infrastructure + disable until compiler is complete
Added vm-compile command: iterates env, compiles lambdas to bytecode,
replaces with NativeFn VM wrappers (with CEK fallback on error).
Tested: 3/109 compile, reduces CEK steps 23%.

Disabled auto-compile in production — the compiler doesn't handle
closures with upvalues yet, and compiled functions that reference
dynamic env vars crash. Infrastructure stays for when compiler
handles all SX features.

Also: added set-nth! and mutable-list primitives (needed by
compiler.sx for bytecode patching). Fixed compiler.sx to use
mutable lists on OCaml (ListRef for append!/set-nth! mutation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 19:30:54 +00:00
e14fc9b0e1 Auto-compile: lambdas → bytecode VM at load time
After loading .sx files, (vm-compile) iterates all named lambdas,
compiles each body to bytecode, replaces with NativeFn VM wrapper.

Results: 3/109 functions compiled (compiler needs more features).
CEK steps: 49911 → 38083 (23% fewer) for home page.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 19:07:42 +00:00
8707f21ca2 Single-pass aser_slot for HTMX path + kernel eval timing + stable hash
Eliminated double-aser for HTMX requests: build OOB wrapper AST
(~shared:layout/oob-sx :content wrapped_ast) and aser_slot in ONE
pass — same pattern as the full-page path. Halves aser_slot calls.

Added kernel-side timing to stderr:
  [aser-slot] eval=3.6s io_flush=0.0s batched=3 result=22235 chars

Results show batch IO works (io_flush=0.0s for 3 highlight calls)
and the bottleneck is pure CEK evaluation time, not IO.

Performance after single-pass fix:
  Home: 0.7s eval (was 2.2s total)
  Reactive: 3.6s eval (was 6.8s total)
  Language: 1.1s eval (was 18.9s total — double-aser eliminated)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 17:03:47 +00:00
96e7bbbac1 Non-blocking batch IO for OCaml kernel + stable component hash
OCaml kernel (sx_server.ml):
- Batch IO mode for aser-slot: batchable helpers (highlight,
  component-source) return placeholders during evaluation instead
  of blocking on stdin. After aser completes, all batched requests
  are flushed to Python at once.
- Python processes them concurrently with asyncio.gather.
- Placeholders (using «IO:N» markers) are replaced with actual
  values in the result string.
- Non-batchable IO (query, action, ctx, request-arg) still uses
  blocking mode — their results drive control flow.

Python bridge (ocaml_bridge.py):
- _read_until_ok handles batched protocol: collects io-request
  lines with numeric IDs, processes on (io-done N) with gather.
- IO result cache for pure helpers — eliminates redundant calls.
- _handle_io_request strips batch ID from request format.

Component caching (jinja_bridge.py):
- Hash computed from FULL component env (all names + bodies),
  not per-page subset. Stable across all pages — browser caches
  once, no re-download on navigation between pages.
- invalidate_component_hash() called on hot-reload.

Tests: 15/15 OCaml helper tests pass (2 new batch IO tests).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:53:01 +00:00
d3b3b4b720 Fix pipe desync: async drain on _send, robust Playwright tests
Root cause: OcamlBridge._send() used write() without drain().
asyncio.StreamWriter buffers writes — without drain(), multiple
commands accumulate and flush as a batch. The kernel processes
them sequentially, sending responses, but Python only reads one
response per command → pipe desync → "unexpected response" errors.

Fix: _send() is now async, calls drain() after every write.
All 14 callers updated to await.

Playwright tests rewritten:
- test_home_has_header: verifies #logo-opacity visible (was only
  checking for "sx" text — never caught missing header)
- test_home_has_nav_children: Geography link must be visible
- test_home_has_main_panel: #main-panel must have child elements
- TestDirectPageLoad: fresh browser.new_context() per test to
  avoid stale component hash in localStorage
- _setup_error_capture + _check_no_fatal_errors helpers

_render_to_sx uses aser_slot (not aser) — layout wrappers contain
re-parsed content that needs full expansion capability.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:11:55 +00:00
f819fda587 aser_slot migration: single-pass expansion, pipe desync fix, _render_to_sx
Three fixes completing the aser_slot migration:

1. Single-pass full-page rendering: eval_sx_url builds layout+content
   AST and aser_slots it in ONE call — avoids double-aser where
   re-parsed content hits "Undefined symbol: title/deref" errors.

2. Pipe desync fix: _inject_helpers_locked runs INSIDE the aser_slot
   lock acquisition (not as a separate lock). Prevents interleaved
   commands from other coroutines between injection and aser-slot.

3. _render_to_sx uses aser_slot (not aser): layout wrappers like
   oob_page_sx contain re-parsed content from earlier aser_slot
   calls. Regular aser fails on symbols that were bound during
   the earlier expansion. aser_slot handles them correctly.

HTMX path: aser_slot the content, then oob_page_sx wraps it.
Full page path: build (~shared:layout/app-body :content wrapped_ast),
aser_slot in one pass, pass directly to sx_page.

New Playwright tests: test_navigate_geography_to_reactive,
test_direct_load_reactive_page.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 14:56:55 +00:00
d06de87bca Island guard in aser expansion + page helper IO tests (13 tests)
Fix: islands (defisland) pass component? check but must NEVER be
expanded server-side — they use browser-only reactive primitives
(signal, deref, computed). Added (not (island? comp)) guard in
adapter-sx.sx aser component dispatch.

New test file: shared/sx/tests/test_ocaml_helpers.py
- TestHelperInjection: 5 tests — helper IO proxy, 2-arg calls,
  aser/aser_slot with helpers, undefined helper error
- TestHelperIOPerformance: 2 tests — 20 sequential IO round-trips
  complete in <5s, aser_slot with 5 helpers in <3s
- TestAserSlotClientAffinity: 6 tests — island exclusion, client
  affinity exclusion, server affinity expansion, auto affinity
  behavior in aser vs aser_slot

eval_sx_url stays on bridge.aser() (server-affinity only) for now.
Switching to aser_slot requires fixing the double-aser issue in
_render_to_sx where content gets re-parsed and re-asered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 12:48:34 +00:00
109ca7c70b Fix aser server-affinity expansion: keyword values, OOB wrapper, page helpers
Three bugs in aser-expand-component (adapter-sx.sx):
- Keyword values were eval'd (eval-expr can't handle <>, HTML tags);
  now asered, matching the aser's rendering capabilities
- Missing default nil binding for unset &key params (caused
  "Undefined symbol" errors for optional params like header-rows)
- aserCall string-quoted keyword values that were already serialized
  SX — now inlines values starting with "(" directly

Server-affinity annotations for layout/nav shells:
- ~shared:layout/app-body, ~shared:layout/oob-sx — page structure
- ~layouts/nav-sibling-row, ~layouts/nav-children — server-side data
- ~layouts/doc already had :affinity :server
- ~cssx/flush marked :affinity :client (browser-only state)

Navigation fix: restore oob_page_sx wrapper for HTMX responses
so #main-panel section exists for sx-select/sx-swap targeting.

OCaml bridge: lazy page helper injection into kernel via IO proxy
(define name (fn (...) (helper "name" ...))) — enables aser_slot
to evaluate highlight/component-source etc. via coroutine bridge.

Playwright tests: added pageerror listener to test_no_console_errors,
new test_navigate_from_home_to_geography for HTMX nav regression.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 12:06:24 +00:00
e61dc4974b Fix readline buffer limit (10MB) and evaluator spec filename
- ocaml_bridge: 10MB readline buffer for large spec explorer responses
- nav-data: evaluator.sx filename (was eval.sx, actual spec file is evaluator.sx)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 19:37:59 +00:00
71c2003a60 OCaml evaluator for page dispatch + handler aser, 83/83 Playwright tests
Major architectural change: page function dispatch and handler execution
now go through the OCaml kernel instead of the Python bootstrapped evaluator.

OCaml integration:
- Page dispatch: bridge.eval() evaluates SX URL expressions (geography, marshes, etc.)
- Handler aser: bridge.aser() serializes handler responses as SX wire format
- _ensure_components loads all .sx files into OCaml kernel (spec, web adapter, handlers)
- defhandler/defpage registered as no-op special forms so handler files load
- helper IO primitive dispatches to Python page helpers + IO handlers
- ok-raw response format for SX wire format (no double-escaping)
- Natural list serialization in eval (no (list ...) wrapper)
- Clean pipe: _read_until_ok always sends io-response on error

SX adapter (aser):
- scope-emit!/scope-peek aliases to avoid CEK special form conflict
- aser-fragment/aser-call: strings starting with "(" pass through unserialized
- Registered cond-scheme?, is-else-clause?, primitive?, get-primitive in kernel
- random-int, parse-int as kernel primitives; json-encode, into via IO bridge

Handler migration:
- All IO calls converted to (helper "name" args...) pattern
- request-arg, request-form, state-get, state-set!, now, component-source etc.
- Fixed bare (effect ...) in island bodies leaking disposer functions as text
- Fixed lower-case → lower, ~search-results → ~examples/search-results

Reactive islands:
- sx-hydrate-islands called after client-side navigation swap
- force-dispose-islands-in for outerHTML swaps (clears hydration markers)
- clear-processed! platform primitive for re-hydration

Content restructuring:
- Design, event bridge, named stores, phase 2 consolidated into reactive overview
- Marshes split into overview + 5 example sub-pages
- Nav links use sx-get/sx-target for client-side navigation

Playwright test suite (sx/tests/test_demos.py):
- 83 tests covering hypermedia demos, reactive islands, marshes, spec explorer
- Server-side rendering, handler interactions, island hydration, navigation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 17:22:51 +00:00
313f7d6be1 OCaml bootstrapper Phase 2: HTML renderer, SX server, Python bridge
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 23:28:48 +00:00
3a268e7277 Data-first HO forms, fix plan pages, aser error handling (1080/1080)
Some checks failed
Build and Deploy / build-and-deploy (push) Has been cancelled
Evaluator: data-first higher-order forms — ho-swap-args auto-detects
(map coll fn) vs (map fn coll), both work. Threading + HO: (-> data
(map fn)) dispatches through CEK HO machinery via quoted-value splice.
17 new tests in test-cek-advanced.sx.

Fix plan pages: add mother-language, isolated-evaluator, rust-wasm-host
to page-functions.sx plan() — were in defpage but missing from URL router.

Aser error handling: pages.py now catches EvalError separately, renders
visible error banner instead of silently sending empty content. All
except blocks include traceback in logs.

Scope primitives: register collect!/collected/clear-collected!/emitted/
emit!/context in shared/sx/primitives.py so hand-written _aser can
resolve them (fixes ~cssx/flush expansion failure).

New test file: shared/sx/tests/test_aser_errors.py — 19 pytest tests
for error propagation through all aser control flow forms.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 18:05:00 +00:00
b4df216fae Phase 2: Remove dead tree-walk code from eval.sx
eval.sx: 1272 → 846 lines (-33%). sx-browser.js: 392KB → 377KB.

Deleted (superseded by CEK step handlers in cek.sx):
- eval-list: tree-walk dispatch table
- eval-call: tree-walk function dispatch
- sf-if, sf-when, sf-cond (3 variants), sf-case (2 variants)
- sf-and, sf-or, sf-let, sf-begin, sf-quote, sf-quasiquote
- sf-thread-first, sf-set!, sf-define
- ho-map, ho-filter, ho-reduce, ho-some, ho-every, ho-for-each,
  ho-map-indexed, call-fn

Kept (still called by CEK as delegates):
- sf-lambda, sf-defcomp, sf-defisland, sf-defmacro, sf-defstyle,
  sf-deftype, sf-defeffect, sf-letrec, sf-named-let
- sf-scope, sf-provide, sf-dynamic-wind
- expand-macro, qq-expand, cond-scheme?
- call-lambda, call-component, parse-keyword-args
- Strict mode, type helpers

eval-expr is now a stub overridden by CEK fixup.
All tests unchanged: JS 747/747, Full 864/870, Python 679/679.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 13:28:09 +00:00
293af75821 Phase 1: CEK is now the sole evaluator on JavaScript
- Override evalExpr/trampoline in CEK_FIXUPS_JS to route through
  cekRun (matching what Python already does)
- Always include frames+cek in JS builds (not just when DOM present)
- Remove CONTINUATIONS_JS extension (CEK handles shift/reset natively)
- Remove Continuation constructor guard (always define it)
- Add strict-mode type checking to CEK call path via head-name
  propagation through ArgFrame

Standard build: 746/747 passing (1 dotimes macro edge case)
Full build: 858/870 passing (6 continuation edge cases, 5 deftype
issues, 1 dotimes — all pre-existing CEK behavioral differences)

The tree-walk eval-expr, eval-list, eval-call, and all sf-*/ho-*
forms in eval.sx are now dead code — never reached at runtime.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 12:49:08 +00:00
7cde140c7e Phase 5-7: Clean up duplicates, verify end-to-end
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 1m50s
Remove all duplicate .sx files from shared/sx/ref/ — originals now in
spec/, web/, hosts/. Only generated runtime (sx_ref.py), async shim,
and theorem prover tools remain in shared/sx/ref/.

Final structure:
  spec/          10 .sx files (core language)
  spec/tests/     8 .sx files (core tests)
  web/           10 .sx files (web framework)
  web/tests/      7 .sx files (web tests)
  hosts/python/   bootstrapper + platform + 5 test runners
  hosts/javascript/ bootstrapper + CLI + platform
  shared/sx/ref/  generated runtime only

All 89 tests pass. Both bootstrappers build fully.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 02:29:37 +00:00
72eaefac13 Phase 4: Move web framework files to web/ and web/tests/
signals.sx, engine.sx, orchestration.sx, boot.sx, router.sx, deps.sx,
forms.sx, page-helpers.sx, adapters, boundary files → web/
Web tests → web/tests/
Test runners updated with _SPEC_TESTS and _WEB_TESTS paths.
All 89 tests pass (20 signal + 43 CEK + 26 CEK reactive).
Both bootstrappers build fully (5993 Python lines, 387KB JS).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 02:26:18 +00:00
7036621be8 Phase 3: Move host files to hosts/python/ and hosts/javascript/
Python: bootstrap.py, platform.py, transpiler.sx, boundary_parser.py, tests/
JavaScript: bootstrap.py, cli.py, platform.py, transpiler.sx
Both bootstrappers verified — build from new locations, output to shared/.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 02:18:56 +00:00
05f7b10864 Phase 2: Move core spec files to spec/ and spec/tests/
git mv eval.sx, parser.sx, primitives.sx, render.sx, cek.sx, frames.sx,
continuations.sx, callcc.sx, types.sx, special-forms.sx → spec/
Tests → spec/tests/
Both bootstrappers verified — find files via spec/ → web/ → shared/sx/ref/

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 02:12:13 +00:00
1a3d7b3d77 Separate core spec from web framework
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 4m49s
Three-layer architecture:
  spec/  — Core language (19 files): evaluator, parser, primitives,
           CEK machine, types, continuations. Host-independent.
  web/   — Web framework (20 files): signals, adapters, engine,
           orchestration, boot, router, CSSX. Built on core spec.
  sx/    — Application (sx-docs website). Built on web framework.

Split boundary.sx into boundary-core.sx (type-of, make-env, identical?)
and boundary-web.sx (IO primitives, signals, spreads, page helpers).

Bootstrappers search spec/ → web/ → shared/sx/ref/ for .sx files.
Original files remain in shared/sx/ref/ as fallback during transition.
All 63 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 01:42:19 +00:00
e2940e1c5f Add Content Addressing page under CEK Machine
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 15m20s
Dedicated page documenting and demonstrating content-addressed
computation. How it works, why it matters, the path to IPFS.

Live demo: counter + name widget with CID generation, history,
and restore-from-CID input.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 00:27:14 +00:00
f7debec7c6 Content-addressed computation: freeze → hash → CID → thaw
Some checks failed
Build and Deploy / build-and-deploy (push) Has been cancelled
Hash frozen SX to a content identifier (djb2 → hex). Same state
always produces the same CID. Store by CID, retrieve by CID.

- content-hash: djb2 hash of SX text → hex string
- content-put/get: in-memory content store
- freeze-to-cid: freeze scope → store → return CID
- thaw-from-cid: look up CID → thaw signals
- char-code-at / to-hex primitives for both platforms
- Live demo: counter + name widget, content-address button,
  CID display, restore from CID input, CID history

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 00:17:29 +00:00
488fc53fda Persist home stepper state to localStorage via freeze scope
Some checks failed
Build and Deploy / build-and-deploy (push) Has been cancelled
- freeze-scope "home-stepper" captures step-idx signal
- Each step/back saves to localStorage via freeze-to-sx
- On mount, restores from localStorage via thaw-from-sx
- Invalid state resets to default (step 9)
- Clear preview lake before replay to prevent duplicates
- Register local-storage-get/set/remove as primitives
- Arrows 3x bigger

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-15 00:04:32 +00:00
cb4f4b85e5 Named freeze scopes for serializable reactive state
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 3m20s
Replace raw CEK state serialization with named freeze scopes.
A freeze scope collects signals registered within it. On freeze,
signal values are serialized to SX. On thaw, values are restored.

- freeze-scope: scoped effect delimiter for signal collection
- freeze-signal: register a signal with a name in the current scope
- cek-freeze-scope / cek-thaw-scope: freeze/thaw by scope name
- freeze-to-sx / thaw-from-sx: full SX text round-trip
- cek-freeze-all / cek-thaw-all: batch operations

Also: register boolean?, symbol?, keyword? predicates in both
Python and JS platforms with proper var aliases.

Demo: counter + name input with Freeze/Thaw buttons.
Frozen SX: {:name "demo" :signals {:count 5 :name "world"}}

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 23:21:38 +00:00
a759f4da3b Add Freeze/Thaw page under CEK Machine with live demo
Some checks failed
Build and Deploy / build-and-deploy (push) Has been cancelled
Documents and demonstrates serializable CEK state. Type an expression,
step to any point, click Freeze to see the frozen SX. Click Thaw to
resume from the frozen state and get the result.

- New page at /sx/(geography.(cek.freeze))
- Nav entry under CEK Machine
- Interactive island demo with step/run/freeze/thaw buttons
- Documentation: the idea, freeze format, thaw/resume, what it enables

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-14 22:31:34 +00:00