Commit Graph

414 Commits

Author SHA1 Message Date
c0b001d3c2 Fix VM reuse_stack lost across stub VM boundary on IO suspension
Root cause: when perform fires inside a VM closure chain (call_closure_reuse),
the caller frames are saved to reuse_stack on the ACTIVE VM. But the
_cek_io_suspend_hook and _cek_eval_lambda_ref create a NEW stub VM for the
VmSuspended exception. On resume, resume_vm runs on the STUB VM which has
an empty reuse_stack — the caller frames are orphaned on the original VM.

Fix: transfer reuse_stack from _active_vm to the stub VM before raising
VmSuspended. This ensures resume_vm -> restore_reuse can find and restore
the caller's frames after async resume via _driveAsync/setTimeout.

Also restore step_limit/step_count refs dropped by bootstrap.py regeneration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 13:31:55 +00:00
bceccccedb Sync sx_ref.ml with bootstrap.py output
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 13:16:28 +00:00
0e152721cc Remove cek_resume debug tracing, rebuild WASM
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 13:08:43 +00:00
c641b445f8 Fix: local bindings now shadow HTML tag special forms in browser evaluator
Root cause: sx_browser.ml registered all HTML tags (a, b, i, p, s, u, g, etc.)
as custom special forms. The evaluator's step_eval_list checked custom special
forms BEFORE checking local env bindings. So (let ((a (fn () 42))) (a))
matched the HTML tag <a> instead of calling the local function a.

Fix: skip custom special forms AND render-check when the symbol is bound in
the local env. Added (not (env-has? env name)) guard to both checks in
step-eval-list (spec/evaluator.sx and transpiled sx_ref.ml).

This was the root cause of "[sx] resume: Not callable: nil" — after hs-wait
resumed, calling letrec-bound functions like wait-boot (which is not an HTML
tag) worked, but any function whose name collided with an HTML tag failed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 12:53:09 +00:00
0f9bb68ba2 MCP tree server: add failure logging to /tmp/mcp-tree.log
Logs timestamps, tool calls, errors, slow calls, stack overflow, OOM.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 08:26:54 +00:00
15e593b725 Restore sx_server.ml, add host-* stubs for boot-helpers.sx
The previous commit accidentally lost ~1100 lines from sx_server.ml
due to a git stash conflict resolution that silently deleted the
hash-index, manifest generation, and /sx/h/ route handler code.
Restored from 97818c6d. Only change: added host-* platform primitive
stubs (host-get, host-set!, host-call, etc.) needed because the
callable? fix in boot-helpers.sx now properly loads code paths that
reference these browser-only functions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 07:25:19 +00:00
8c85e892c2 Fix callable? type mismatch, restore 20 HS test regressions, add host-* server stubs
callable? in boot-helpers.sx checked for "native-fn" but type-of returns
"function" for NativeFn — broke make-spread and all native fn dispatch
in aser. Restore 20 behavioral tests replaced with NOT IMPLEMENTED stubs
by the test regeneration commit. Add host-* platform primitive stubs to
sx_server.ml so boot-helpers.sx loads without errors server-side.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 22:15:55 +00:00
76f7e3b68a HS: return/guard, repeat while/until, if-then fix, script extraction
Parser: if-then consumes 'then' keyword before parsing then-body.
Compiler: return→raise, def→guard, repeat while/until dispatch.
Runtime: hs-repeat-while, hs-repeat-until.
Test gen: script block extraction for def functions.
repeat suite: 10→13/30.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 21:33:55 +00:00
ca9196a693 Stub VM uses real globals for CEK resume after IO suspension
The _cek_io_suspend_hook creates a stub VM to carry the suspended CEK
state. Previously used empty globals, which caused "Not callable: nil"
when the CEK resume needed platform functions. Now uses _default_vm_globals
(set to _vm_globals by sx_browser.ml) so all platform functions and
definitions are available during resume.

Remaining issue: still getting "resume: Not callable: nil" — the CEK
continuation env may not include letrec bindings from the island body.
The suspension point is inside reload-frame → hs-wait, and the resume
needs to call wait-boot (a letrec binding).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 17:26:38 +00:00
d981e5f620 Remove debug logging from sx_browser.ml and sx-platform.js
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 17:08:01 +00:00
1bce1b701b Fix IO suspension in both cek_run and cek_run_iterative
The _cek_io_suspend_hook was only added to cek_run_iterative (line 986)
but the actual code path went through cek_run (line 624). Added the hook
check to both functions.

This fixes the "IO suspension in non-IO context" error that blocked
hs-wait/perform from propagating through event handler → trampoline →
eval_expr call chains. IO suspension now converts to VmSuspended via the
hook, which the value_to_js wrapper catches and drives with _driveAsync.

+42 OCaml test passes (3924→3966). IO suspension verified working in
browser WASM: dom-on click handler → hs-wait → perform → suspend →
_driveAsync → setTimeout → resume.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 16:43:07 +00:00
b86d0b7e15 IO suspension: _cek_io_suspend_hook propagates perform through eval_expr
Root cause: cek_run_iterative (used by eval_expr/trampoline) raised
"IO suspension in non-IO context" when the CEK hit a perform. This
blocked IO suspension from propagating through nested eval_expr calls
(event handler → trampoline → eval_expr → for-each callback → hs-wait).

Fix: added _cek_io_suspend_hook (Sx_types) that converts CEK suspension
to VmSuspended, set by sx_vm.ml at init. cek_run_iterative now calls the
hook instead of erroring. The VmSuspended propagates to the value_to_js
wrapper which has _driveAsync handling.

+42 test passes (3924→3966), zero regressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 16:34:56 +00:00
133edd4c5e WIP: IO suspension diagnosis — call-lambda CALL_PRIM converts VmSuspended→Eval_error
Root cause found: when the click handler calls run-all → for-each → callback → hs-wait → perform,
the perform raises VmSuspended. But the call path goes through sx_apply_cek
(from the call-lambda CALL_PRIM) which converts VmSuspended → CekPerformRequest.
The inner CEK context has no IO handler, so it raises "IO suspension in non-IO context"
instead of propagating the suspension to the outer context.

Fix needed: either (a) make sx_apply_cek NOT convert VmSuspended when in a context
that supports IO suspension, or (b) ensure the inner CEK from call-lambda propagates
perform as a suspension state rather than erroring.

Debug logging still present in sx_browser.ml (js_to_value traces).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 16:14:20 +00:00
fec3194464 Island body: letrec instead of define (fixes render-to-dom), host-object JS fns
runner.sx: Converted define forms inside island body to letrec. Multiple
define forms in a let body cause render-to-dom to fall back to eval-expr
for the whole body, which evaluates (div ...) as a list instead of
rendering it to DOM. letrec keeps the last body expression (div) as the
render target.

sx_browser.ml: js_to_value now stores plain JS functions as host objects
(Dict with __host_handle) instead of wrapping as NativeFn. This preserves
the original JS function identity through the SX→JS round-trip, keeping
_driveAsync wrappers from host-callback intact when passed to
addEventListener via host-call.

Remaining: IO suspension in click handler is caught as "IO suspension in
non-IO context" instead of being driven by _driveAsync. The host-callback
wrapper creates the right JS function, but the event dispatch path doesn't
go through K.callFn.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 15:45:06 +00:00
c9634ba649 VM: fix nested IO suspension frame corruption, island hydration preload
VM frame merging bug: call_closure_reuse now saves caller continuations
on a reuse_stack instead of merging frames. resume_vm restores them in
innermost-first order. Fixes frame count corruption when nested closures
suspend via OP_PERFORM. Zero test regressions (3924/3924).

Island hydration: hydrate-island now looks up components from (global-env)
instead of render-env, triggering the symbol resolve hook. Added JS-level
preload-island-defs that scans DOM for data-sx-island and loads definitions
from the content-addressed manifest BEFORE hydration — avoids K.load
reentrancy when the resolve hook fires inside env_get.

loadDefinitionByHash: fixed isMultiDefine check — defcomp/defisland bodies
containing nested (define ...) forms no longer suppress name insertion.
Added K.load return value checking for silent error string returns.

sx_browser.ml: resolve hook falls back to global_env.bindings when
_vm_globals miss (sync gap). Snapshot reuse_stack alongside pending_cek.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 13:23:35 +00:00
4aa49e42e8 htmx demos working: activation, fetch, swap, OOB filtering, test runner page
- htmx-boot-subtree! wired into process-elements for auto-activation
- Fixed cond compilation bug in hx-verb-info (Clojure-style flat cond)
- Platform io-fetch upgraded: method/body/headers support, full response dict
- Replaced perform IO ops with browser primitives (set-timeout, browser-confirm, etc)
- SX→HTML rendering in hx-do-swap with OOB section filtering
- hx-collect-params: collects input name/value for all methods
- Handler naming: ex-{slug} convention, removed perform IO dependencies
- Test runner page at (test.(applications.(htmx))) with iframe-based runner
- Header "test" link on every page linking to test URL
- Page file restructure: 285 files moved to URL-matching paths (a/b/c/index.sx)
- page-functions.sx: ~100 component name references updated
- _test added to skip_dirs, test- file prefix convention for test files

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 11:56:15 +00:00
b1666a5fe2 HS tests: VM step limit fix, callFn error propagation, compiler emit-set fixes
- sx_vm.ml: VM timeout now compares vm_insn_count > step_limit instead of
  unconditionally throwing after 65536 instructions when limit > 0
- sx_browser.ml: Expose setStepLimit/resetStepCount APIs on SxKernel;
  callFn now returns {__sx_error, message} on Eval_error instead of null
- compiler.sx: emit-set handles array-index targets (host-set! instead of
  nth) and 'of' property chains (dom-set-prop with chain navigation)
- hs-run-fast.js: New Node.js test runner with step-limit timeouts,
  SX-level guard for error detection, insertAdjacentHTML mock,
  range selection (HS_START/HS_END), wall-clock timeout in driveAsync
- hs-debug-test.js: Single-test debugger with DOM state inspection
- hs-verify.js: Assertion verification (proves pass/fail detection works)

Test results: 415/831 (50%), up from 408/831 (49%) baseline.
Fixes: set my style["color"], set X of Y, put at end of (insertAdjacentHTML).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 19:27:03 +00:00
b81c26c45b HS tests: sync textContent when innerHTML is set
When innerHTML is set on a mock element, textContent now updates to
match (with HTML tags stripped). Many HS tests do `put "foo" into me`
(which sets innerHTML) then check textContent. Previously textContent
stayed empty because only innerHTML was updated.

Also fixes innerHTML="" to fully detach children from parent.

393 → 408/831 HS tests (+15).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 12:58:23 +00:00
23e8379622 HS tests: innerHTML/textContent clears children in mock DOM
Setting innerHTML="" on a mock element now detaches and removes all
children, matching browser behavior. Previously hs-cleanup! (which
sets body.innerHTML="") left stale children attached, causing
querySelector to find elements from prior tests.

Also clears children when textContent is set (browser behavior).

375 → 393/831 HS tests (+18).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 12:47:34 +00:00
a6eb125dcc HS tests: stringify DOM properties (innerHTML, textContent, value)
In a real browser, innerHTML/textContent/value are always strings.
The mock was storing raw SX values (Number, Bool, Nil), causing type
mismatches like "Expected 1, got 1" where the value was correct but
Number 1.0 != String "1".

Now coerces to string on host-set! for innerHTML, textContent, value,
outerHTML, innerText. Fixes 10 increment tests that were doing
`put value into me` with numeric results.

367 → 375/831 HS tests (+8 net, +10 new passes, -2 regressions).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 12:38:33 +00:00
e3eb46d0dc HS tests: SIGALRM + raise timeout for native OCaml loops
The infinite loops in the HS parser are in transpiled native OCaml code,
not in the VM or CEK step loop. Neither step counters (in cek_step_loop,
cek_step, trampoline) nor VM instruction checks caught them because
the loops are in direct OCaml recursion.

Fix: SIGALRM handler raises Eval_error to break out of native loops.
Also sets step_limit flag to catch VM loops. Combined approach handles
both native OCaml recursion (alarm+raise) and VM bytecode (step check).

The alarm+raise can become unreliable after ~13 timeouts in a single
process, but handles the common case well. Reverts the fork-based
approach which lost inter-test state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:57:33 +00:00
3d7fffe4eb HS tests: host-get method truthiness + fork-based test timeout
Two critical fixes for the mock DOM test runner:

1. host-get returns truthy for DOM method names on mock elements.
   dom.sx guards like `(and el (host-get el "setAttribute"))` were
   silently skipping setAttribute/getAttribute calls because the mock
   dict had no "setAttribute" key. Now returns Bool true for known
   DOM method names, fixing hs-activate! → dom-set-attr → dom-get-attr
   chain. Also adds firstElementChild, nextElementSibling, etc. as
   computed properties.

2. Fork-based per-test timeout (5 seconds). The HS parser has infinite
   loops on certain syntax ([@attr], complex put targets). Signal-based
   alarm doesn't work reliably in OCaml 5. Fork + waitpid + select
   gives hard OS-level timeout protection.

Also adds step_limit/step_count to sx_ref.ml trampoline (currently
unused but available for future CEK-level timeout).

Result: 525/963 total, up from 498. Many more add/remove/toggle/set
tests now pass because hs-activate! actually wires up event handlers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:04:03 +00:00
1d83ccba3c Content-addressed on-demand loading: Merkle DAG for all browser assets
Replace the monolithic 500KB <script data-components> block with a 25KB
JSON manifest mapping names to content hashes. Every definition —
components, islands, macros, client libraries, bytecode modules, and
WASM binaries — is now content-addressed and loaded on demand.

Server (sx_server.ml):
- build_hash_index: Merkle DAG over all definitions — topological sort,
  hash leaves first, component refs become @h:{hash} in instantiated form
- /sx/h/{hash} endpoint: serves definitions with Cache-Control: immutable
- Per-page manifest in <script data-sx-manifest> with defs + modules + boot
- Client library .sx files hashed as whole units (tw.sx, tw-layout.sx, etc.)
- .sxbc modules and WASM kernel hashed individually

Browser (sx-platform.js):
- Content-addressed boot: inline script loads kernel + platform by hash
- loadDefinitionByHash: recursive dep resolution with @h: rewriting
- resolveHash: 3-tier cache (memory → localStorage → fetch /sx/h/{hash})
- __resolve-symbol extended for manifest-based component + library loading
- Cache API wrapper intercepts .wasm fetches for offline caching
- Eager pre-loading of plain symbol deps for CEK evaluator compatibility

Shell template (shell.sx):
- Monolithic <script data-components> removed
- data-sx-manifest script with full hash manifest
- Inline bootstrap replaces <script src="...?v="> with CID-based loading

Second visit loads zero bytes from network. Changed content gets a new
hash — only that item refetched (Merkle propagation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 10:14:39 +00:00
2cba359fdf HS behavioral tests: mock DOM + eval-hs in OCaml test runner
Add mock DOM layer to run_tests.ml so hyperscript behavioral tests
(spec/tests/test-hyperscript-behavioral.sx) can run in the OCaml test
runner without a browser. Previously these tests required Playwright
which crashed after 10 minutes from WASM page reboots.

Mock DOM implementation:
- host-global, host-get, host-set!, host-call, host-new, host-callback,
  host-typeof, host-await — OCaml primitives operating on SX Dict elements
- Mock elements with classList, style, attributes, event dispatch + bubbling
- querySelector/querySelectorAll with #id, .class, tag, [attr] selectors
- Load web/lib/dom.sx and web/lib/browser.sx for dom-* wrappers
- eval-hs function for expression-only tests (comparisonOperator, etc.)

Result: 367/831 HS tests pass in ~30 seconds (was: Playwright crash).
14 suites at 100%: live, component, liveTemplate, scroll, call, go,
focus, log, reactive-properties, resize, measure, attributeRef,
objectLiteral, queryRef.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 10:03:26 +00:00
7ec42386fb Fix Pretext island: library functions inside define-library begin block
Root cause: sx_insert_near placed break-lines-greedy, pretext-position-line,
pretext-layout-lines OUTSIDE the define-library begin block. The bytecode
compiler only compiles forms inside begin as STORE_GLOBAL — forms outside
are invisible to the browser VM.

Fix: moved all function definitions inside (begin ...) of (define-library).
Bytecode now includes all 17 functions (11K compiled, was 9K).

Browser load-sxbc: simplified VmSuspended handling — just catch and
continue, since STORE_GLOBAL ops already ran before the import OP_PERFORM.
sync_vm_to_env copies them to global_env.

Island now calls break-lines and pretext-layout-lines from bytecode-compiled
library — runs on VM, not CEK interpreter.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 17:53:50 +00:00
699dd5ad69 Step 17b: bytecode-compiled text-layout, WASM library import fix
- text-layout.sx added to WASM bytecode pipeline (9K compiled)
- Fix multi-list map calls (map-indexed + nth instead of map fn list1 list2)
- pretext-layout-lines and pretext-position-line moved to library exports
- Browser load-sxbc: handle VmSuspended for import, copy library exports
  to global_env after module load (define-library export fix)
- compile-modules.js: text-layout in SOURCE_MAP, FILES, and entry deps
- Island uses library functions (break-lines, pretext-layout-lines)
  instead of inlining — runs on bytecode VM when exports resolve

Known issue: define-library exports don't propagate to browser global env
yet. The load-sxbc import suspension handler resumes correctly but
bind_import_set doesn't fire. Needs deeper investigation into how the
WASM kernel's define-library registers exports vs how other libraries
(adapter-html, tw) make their exports available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 16:37:04 +00:00
1eadefd0c1 Step 17b: Pretext — DOM-free text layout with otfm font measurement
Pure SX text layout library with one IO boundary (text-measure perform).
Knuth-Plass optimal line breaking, Liang's hyphenation, position calculation.

Library (lib/text-layout.sx):
- break-lines: Knuth-Plass DP over word widths
- break-lines-greedy: simple word-wrap for comparison
- hyphenate-word: Liang's trie algorithm
- position-line/position-lines: running x/y sums
- measure-text: single perform (text-measure IO)

Server font measurement (otfm):
- Reads OpenType cmap + hmtx tables from .ttf files
- DejaVu Serif/Sans bundled in shared/static/fonts/
- _cek_io_resolver hook: perform works inside aser/eval_expr
- JIT VM suspension inline resolution for IO in compiled code

~font component (shared/sx/templates/font.sx):
- Works like ~tw: emits @font-face CSS via cssx scope
- Sets font-family on parent via spread
- Deduplicates font declarations

Infrastructure fixes:
- stdin load command: per-expression error handling (was aborting on first error)
- cek_run IO hook: _cek_io_resolver in sx_types.ml
- JIT VmSuspended: inline IO resolution when resolver installed
- ListRef handling in IO resolver (perform creates ListRef, not List)

Demo page at /sx/(applications.(pretext)):
- Hero: justified paragraph with otfm-measured proportional widths
- Greedy vs Knuth-Plass side-by-side comparison
- Badness scoring visualization
- Hyphenation syllable decomposition

25 new tests (spec/tests/test-text-layout.sx), 3201/3201 passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 15:13:00 +00:00
f60d22e86e Hyperscript: focus command, diagnostic test output, blur keyword
Parser/compiler/runtime for focus command. Tokenizer: focus, blur,
precedes, follows, ignoring, case keywords. Test spec: per-test
failure output for diagnosis.

374/831 (45%)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 12:38:05 +00:00
1783f4805a Fix streaming resolveSuspense: use callFn instead of eval string interpolation
The previous K.eval() approach double-escaped backslashes in SX source
strings, breaking the \/ → / unescaping that the server serializer adds
for HTML safety. Using K.callFn() passes strings directly as arguments,
bypassing the escaping problem entirely.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 12:35:37 +00:00
7d798be14f Hyperscript: precedes/follows comparisons, tokenizer keywords
Parser: precedes/follows comparison operators in parse-cmp.
Tokenizer: precedes, follows, ignoring, case keywords.
Runtime: precedes?, follows? string comparison functions.

372/831 (45%)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 12:20:13 +00:00
854ed9c027 Hyperscript conformance: 372→373 — hide/show strategy, generator toEqual
Parser: hide/show handle `with opacity/visibility/display` strategy,
target detection for then-less chaining (add/remove/set/put as boundary).
Generator: inline run().toEqual([...]) pattern for eval-only tests.
Compiler: hide/show emit correct CSS property per strategy.

373/831 (45%)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 11:42:28 +00:00
56855eee7f Fix streaming resolve: color dict keys + defer resolveSuspense until after boot
stream-colors dict had green/blue keys but data used emerald/violet — all three
slots now render with correct Tailwind color classes. Platform: resolveSuspense
must not exist on Sx until boot completes, otherwise bootstrap __sxResolve calls
it before web stack loads and resolves silently fail. Moved to post-boot setup
so all pre-boot resolves queue in __sxPending and drain correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 09:58:18 +00:00
6e27442d57 Step 17: streaming render — hyperscript enhancements, WASM builds, live server tests
Streaming chunked transfer with shell-first suspense and resolve scripts.
Hyperscript parser/compiler/runtime expanded for conformance. WASM static
assets added to OCaml host. Playwright streaming and page-level test suites.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-12 08:41:38 +00:00
7aefe4da8f Fix streaming: resolve scripts inside </body>, live server tests
The shell HTML included closing </body></html> tags. Resolve script
chunks arrived AFTER the document end — browser ignored them
(ERR_INCOMPLETE_CHUNKED_ENCODING). Now strips </body></html> from
shell, sends resolve scripts inside the body, closes document last.

Added live server Playwright tests that hit the actual streaming
endpoint and verify suspense slots resolve with content.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 16:40:49 +00:00
d4c0be52b1 Fix ListRef handling in streaming data — list from SX is ListRef in OCaml
The streaming render matched `List items` but SX's `(list ...)` produces
`ListRef` (mutable list) in the OCaml runtime. Data items were rejected
with "returned list, expected dict or list" — 0 resolve chunks sent.

Fixed both streaming render and AJAX paths to handle ListRef.
Added sandbox test for streaming-demo-data return type validation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 16:34:50 +00:00
c850737c60 Async IO in streaming render — staggered resolve with io-sleep
Server (sx_server.ml):
- eval_with_io: CEK evaluator with IO suspension handling (io-sleep, import)
- io-sleep platform primitive: raises CekPerformRequest, resolved by eval_with_io
- Streaming render uses eval_with_io for data + content evaluation
- Data items with "delay" field sleep before resolving (async streaming)
- Removed hardcoded streaming-demo-data — application logic belongs in .sx

Application (streaming-demo.sx):
- streaming-demo-data defined in SX: 3 items with 1s/3s/5s delays
- Each item has delay, stream-id, and display data fields
- Shell renders instantly, slots fill progressively as IO completes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 16:19:21 +00:00
eaf5af4cd8 Step 17: streaming render — chunked transfer, shell-first suspense, resolve scripts
Server (sx_server.ml):
- Chunked HTTP transport (Transfer-Encoding: chunked)
- Streaming page detection via scan_defpages (:stream true)
- Shell-first render: outer layout + shell AST → aser → SSR → flush
- Data resolution: evaluate :data, render :content per slot, flush __sxResolve scripts
- AJAX streaming: synchronous eval + OOB swaps for SPA navigation
- SX URL → flat path conversion for defpage matching
- Error boundaries per resolve section
- streaming-demo-data helper for the demo page

Client (sx-platform.js):
- Sx.resolveSuspense: finds [data-suspense] element, parses SX, renders to DOM
- Fallback define for resolve-suspense when boot.sx imports fail in WASM
- __sxPending drain on boot (queued resolves from before sx.js loads)
- __sxResolve direct dispatch after boot

Tests (streaming.spec.js):
- 5 sandbox tests using real WASM kernel
- Suspense placeholder rendering, __sxResolve replacement, independent slot resolution
- Full layout with gutters, end-to-end resolve with streaming-demo/chunk components

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 16:12:28 +00:00
ef8f8b7c03 Compiler: dict destructuring in let, paren-aware library stripping — 31/31 sxbc
compile-let now handles dict destructuring patterns:
(let {:key1 var1 :key2 var2} source body). This unblocked core-signals.sx
(deref uses dict destructuring) which was the sole bytecode skip.

Rewrote stripLibraryWrapper from line-based to paren-aware extraction.
The old regex missed (define-library on its own line (no trailing space),
silently passing the full wrapper to the compiler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 08:52:36 +00:00
bca0d8e4e5 Step 15: bytecode + CEK state serialization — 16 tests
bytecode-serialize/deserialize: sxbc v2 format wrapping compiled code
dicts. cek-serialize/deserialize: cek-state v1 format wrapping suspended
CEK state (phase, request, env, kont). Both use SX s-expression
round-trip via inspect/parse. lib/serialize.sx has pure SX versions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 08:19:16 +00:00
99c5c44cc1 Step 14: source locations — pos-to-loc, error-loc, sx-parse-loc — 15 tests
Pure SX layer: pos-to-loc (offset→line/col), error-loc (parse result→loc),
format-parse-error (human-readable error with source context line).
OCaml platform: cst_to_ast_loc (CST spans→loc dicts), sx-parse-loc
primitive (parse with locations), source-loc accessor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 08:03:45 +00:00
79ba9c2d40 Fix stepper SSR/hydration flash: server reads cookie, cache bypass
Three changes to eliminate the stepper flash:

1. home-stepper.sx: server path reads cookie via (get-cookie) for
   step-idx initial value. Client path reads document.cookie via
   def-store. Both default to 0 when no cookie exists.

2. sx_server.ml: bypass response cache when sx-home-stepper cookie
   is present. Render on main thread (not worker) so get-cookie
   sees the parsed request cookies.

3. site-full.spec.js: flash detection test sets cookie=7 via
   Playwright context, checks SSR HTML matches hydrated state.

Test: "No flash: SSR=7 hydrated=7 (cookie=7)" — passes.
Tested on fresh stack=site server subprocess.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 07:28:47 +00:00
0bed9e3664 Fix repeat timing: don't double-drive IO suspensions
The value_to_js resume handler was calling _driveAsync on re-suspension,
but the JS driveAsync caller also processes the returned suspension.
This caused the second wait in each iteration to fire immediately (0ms)
instead of respecting the delay.

Fix: resume handler just returns the suspension object, lets the JS
driveAsync handle scheduling via setTimeout.

Verified: repeat 3 times add/wait 300ms/remove/wait 300ms produces
6 transitions at correct 300ms intervals (1504ms total).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 21:47:48 +00:00
9982cd5926 Fix chained IO suspensions in value_to_js callback wrapper
The resume callback in the value_to_js VmSuspended handler now catches
VmSuspended recursively, building a new suspension object and calling
_driveAsync for each iteration. Fixes repeat N times ... wait ... end
which produces N sequential suspensions.

Bounce works on repeated clicks. 4/4 regression tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 21:37:52 +00:00
cf10e9a2d6 Fix: load HS modules as bytecode, not source — restores IO suspension chain
Reverts the source-loading workaround. Bytecode modules go through the
VM which handles IO suspension (perform/wait/fetch) correctly. The
endModuleLoad sync copies VM globals to CEK env, so eval-expr-cek in
hs-handler can find hs-on/hs-toggle-class!/etc.

All three HS examples fully working on live site:
  Toggle Color — toggle classes on click
  Bounce — add class, wait 1s (IO suspend+resume), remove class
  Count Clicks — increment counter, update innerHTML

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 21:28:58 +00:00
0365ecb2b9 IO suspension driver: _driveAsync in platform, VmSuspended in value_to_js
- sx-platform.js: add _driveAsync to platform (was sandbox-only) for
  driving wait/fetch IO suspension chains in live site
- sx-platform.js: host-callback wrapper calls _driveAsync on callFn result
- sx_browser.ml: value_to_js callable wrapper catches VmSuspended, builds
  suspension object, and calls _driveAsync directly

Toggle and count clicks work fully. Bounce adds class but wait/remove
requires IO suspension in CEK context (eval-expr-cek doesn't support
perform — needs VM-path evaluation in hs-handler).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 21:22:25 +00:00
de9ab4ca07 Hyperscript examples working: toggle, bounce, count clicks
- sx_browser.ml: restore VmSuspended handler in api_call_fn with
  make_js_callFn_suspension for IO suspension chains (wait, fetch)
- runtime.sx: delete host-get stub that shadowed platform native —
  hs-toggle-class! now uses real FFI host-get for classList access

All three live demo examples work:
  Toggle Color — classList.toggle on click
  Bounce — add .animate-bounce, wait 1s suspend, remove
  Count Clicks — increment @data-count, put into innerHTML

4/4 bytecode regression tests pass (was 0/4 without VmSuspended).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 21:04:45 +00:00
c6df054957 Fix HS browser activation: host-get function sentinel, runtime symbol shadow, lazy dep chain
Three bugs fixed:
1. host-get in sx-platform.js: return true for function-valued properties
   so dom-get-attr/dom-set-attr guards pass (functions can't cross WASM boundary)
2. hs-runtime.sx: renamed host-get→hs-host-get and dom-query→hs-dom-query to
   stop shadowing platform natives when loaded as .sx source
3. compile-modules.js: HS dependency chain (integration→runtime→compiler→parser→tokenizer)
   so lazy loading pulls in all deps. Non-library modules load as .sx source
   for CEK env visibility.

Result: 8/8 elements activate, hs-on attaches listeners. Click handler needs
IO suspension support (VmSuspended in sx_browser.ml) to fire — next step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 20:50:10 +00:00
7f273dc7c2 Wire hyperscript activation into browser boot pipeline
- orchestration.sx: add hs-boot-subtree! call to process-elements
- integration.sx: remove load-library! calls (browser loads via manifest)
- sx_vm.ml: add __resolve-symbol hook to OP_GLOBAL_GET for lazy loading
- compile-modules.js: add HS modules as lazy_deps in manifest

HS compilation works in browser (tokenize→parse→compile verified).
Activation pipeline partially working — hs-activate! needs debugging
(dom-get-data/dom-set-data interaction with WASM host-get on functions).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 19:59:04 +00:00
7492ceac4e Restore hyperscript work on stable site base (908f4f80)
Reset to last known-good state (908f4f80) where links, stepper, and
islands all work, then recovered all hyperscript implementation,
conformance tests, behavioral tests, Playwright specs, site sandbox,
IO-aware server loading, and upstream test suite from f271c88a.

Excludes runtime changes (VM resolve hook, VmSuspended browser handler,
sx_ref.ml guard recovery) that need careful re-integration.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-09 19:29:56 +00:00
908f4f80d4 Fix bytecode resume mutation order: isolate VM frames in cek_call_or_suspend
When cek_call_or_suspend runs a CEK machine for a non-bytecoded Lambda
(e.g. a thunk), _active_vm still pointed to the caller's VM. VmClosure
calls inside the CEK (e.g. hs-wait) would merge their frames with the
caller's VM via call_closure_reuse, causing the VM to skip the CEK's
remaining continuation on resume — producing wrong DOM mutation order
(+active, +active, -active instead of +active, -active, +active).

Fix: swap _active_vm with an empty isolation VM before running the CEK,
restore after. This keeps VmClosure calls on their own frame stack while
preserving js_of_ocaml exception identity (Some path, not None).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-08 22:55:26 +00:00