Committed replacement for the review's ephemeral 130-probe corpus:
spec/tests/differential-probes.txt (49 probes: F-1 int/float display, K18
overflow, F-3 apply + dict order, S-4 float printing, strings,
collections, special forms, error normalization) evaluated on the native
server (epoch protocol printer) and the SHIPPED WASM kernel
(eval_wasm_probes.js via guest sx-serialize), diffed by
scripts/test-differential.sh with a KNOWN_DIVERGENT heal-detecting ledger.
Result: 46/49 agree. All 3 divergences share one root cause, verified
live: bare sx_server's `apply` does not spread its argument list —
(apply + (list 1 2 3)) errors "Expected number, got list", (apply str l)
returns the serialized list; the WASM kernel spreads correctly and the
test runner masks the bug with its own apply binding (F-7 class).
Finding refinement: F-1's float-display divergence (0.3 vs
0.30000000000000004) is a K.eval JS-boundary artifact — guest-serialized
output agrees across hosts; the battery therefore compares guest
serialization.
This completes the W14 checklist: 7 pin suites, 6 gate scripts/runners,
2 harness capabilities, C9 label cleanup, adapter-dom render coverage.
Test-only: no semantics edits, no push.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
conformance.md F-2: no runner fed spec/tests through the shipped
sx_browser.bc.wasm.js — the F-1/F-3 native/WASM divergences existed
undetected because of exactly this gap.
Add hosts/ocaml/browser/run_wasm_corpus.js: boots the shipped kernel
headless in Node (stub block + module preload mirroring
test_wasm_native.js, the blessed boot path), registers the test-framework
hooks, runs ONE test file per process and emits a parseable CORPUS-RESULT
line — process isolation means a hanging file is killed by the driver's
per-file timeout without ending the sweep.
Add scripts/test-wasm-corpus.sh: sweeps spec/tests, applies a SKIP /
KNOWN_FAIL ledger (green-flip on a KNOWN_FAIL fails the run so the ledger
cannot rot), gates on everything else.
Empirical baseline (2026-07-04): 83 files, 80 fully green, 5192 passes,
zero test failures on the shipped kernel — including test-gate-pins
(29/29). KNOWN_FAIL: test-hash-table/test-r7rs/test-sets hit an opaque
jsoo load-error mid-file (22/87/30 tests pass first). Full sweep ~13 min;
sx-build-all.sh wiring deferred to the D3 gate-definition decision.
Test-only: no semantics edits, no push.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Server (sx_server.ml):
- HTTP mode: JIT hook now opt-in via SX_SERVING_JIT, matching epoch mode
(was unconditional — live serving-JIT miscompiles J1/J2/J3 de-risked)
- command channel: malformed/non-ASCII line returns an error response
instead of killing the shared process (C1/C1b)
- response cache: soft error pages no longer cached (S4);
http_render_page returns (html, is_error)
Kernel spec + regen:
- crit-2: signal-return frame stored the saved kont under :f but the reader
looked up "saved-kont" — handler value became the whole program's result
and the covering test passed vacuously. Fixed; raise-continuable now also
resumes at the raise site (rest-k, not unwound-k), mirroring signal-condition
- quasiquote: R7RS longhand unquote-splicing aliased to splice-unquote
(used to serialize literally — silent zero-splice)
- guard: re-raise sentinel gensym'd per execution (was forgeable by any
(list '__guard-reraise__ x) value)
- do: IIFE-head form no longer misparses as a Scheme do-loop
- render: area/base/embed/param/track added to HTML_TAGS (were void-only
and rendered as Undefined symbol)
- REGEN REPAIR: checked-in sx_ref.ml carried hand-written additions that
every regeneration silently lost (let-values/define-values/delay/
delay-force registrations, AdtValue define-type) plus 5 regen blockers
(arrow-name mangling, 3-arg get, &rest defines, HO-position helper refs,
transpiler prim-table gaps). Moved into bootstrap.py FIXUPS/skips and the
transpiler prim table — regen is now reproducible, compiles, and tests
at baseline (CI Dockerfile.test steps 3-4 could not previously have
produced a compiling kernel)
Primitives:
- contains?: dict key-check arm per its spec doc
- expt: promotes to float on int63 overflow ((expt 2 100) returned 0)
- mcp_tree parity with sx_primitives: get (Integer indices + 3-arg default),
split (literal substring, was char-class — the historical gotcha lived
here), empty? on ""/{}, contains?, equal?, keyword-name, char-code
(Integer), parse-number (Integer-aware)
Python/docs:
- shared/sx/boundary.py: dead validation now logs a one-time WARNING instead
of silently no-oping (full revival gated: tier-1 declarations deleted and
SX_BOUNDARY_STRICT=1 is live in production compose)
- CLAUDE.md: canonical reference now points at spec/*.sx; island authoring
rules corrected (let IS sequential, bodies ARE implicit begin)
Verification: full suite 5762 passed / 274 failed — fail set byte-identical
to the pre-change baseline (273 in-progress hs-* + pre-existing r7rs radix
shadow). All repros verified fixed on both the native binary and the rebuilt
WASM browser kernel. Review findings: /tmp/sx-review/*.md
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
web/console-render.sx: render-to-console walks a live DOM element tree through the
engine's own dom-* accessors and prints it as terminal text — the results <ul>
becomes a bulleted list, the filter <input> a text field, the load-more sentinel a
"…" line, an .sx-error element a flagged line. It's the console platform's draw
step: the browser PAINTS the engine's tree, the harness ASSERTS it, this PRINTS it
— one tree, three bindings, the proof the engine is a general runtime not a browser
library.
Wired into the picker's SX engine tests (web/tests/test-relate-picker.sx): the load
and error tests now ALSO assert their console rendering — the same tree the engine
built drives both the DOM assertion and the terminal output, so Phase 1's suite is
the console renderer's regression suite for free. Plus a relate-picker:console suite
for the field/bullet/sentinel/error shapes. 7/7 green, no web-suite regressions.
(Class membership reads the live classList via dom-has-class?, not the static class
attribute — the engine adds .sx-error through classList.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port the rest of the relate-picker's interactive behaviours from Playwright into
the SX harness, driving the real engine against the mock DOM:
- load: the form's "load" trigger populates the results on first render
- filter: a debounced "input" re-fetches and narrows the candidates
- paging: revealing the load-more sentinel pages in the next page (outerHTML
swap replaces the sentinel)
- error-retry: a dropped fetch marks .sx-error, and the next request clears it
Models two browser natives the OCaml runner lacks: observe-intersection (a
recording stub the test fires to simulate the sentinel scrolling into view) and
the synchronous-timer retry (stripped in the error test — backoff timing is a
test-engine.sx concern; here we assert the visible state).
Mock-DOM completeness (run_tests.ml): firstChild/lastChild on elements, so
children-to-fragment can drain a parsed fragment into an innerHTML/outerHTML swap
target. (Also repairs one pre-existing web test that needed firstChild.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port the relate-picker's relate-delete behaviour from Playwright into an SX
harness test that drives the real engine (web/engine.sx + web/orchestration.sx)
against the OCaml runner's in-memory mock DOM. Builds the candidate row, runs
process-elements to bind the form's submit, mocks fetch-request to return the
host's empty 200, fires submit, and asserts the row is deleted in place — the
full fetch→swap→DOM-mutation loop in pure SX.
Mock-DOM completeness (run_tests.ml): NodeList.item(i) so dom-query-all can
iterate querySelectorAll results, and a DOMParser mock so the empty-body
sx-swap=delete path (handle-html-response → parseFromString) works as in a
browser.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The declarative picker markup is now a reusable SX component
(lib/host/sx/relate-picker.sx, defcomp ~relate-picker &key slug kind) instead of
inline markup in the editor. It is a CONTENT-ADDRESSED, CLIENT-EXPANDED component:
- Server: on a full page load render-page expands ~relate-picker server-side
(SEO / no-JS), exactly as before.
- Client: on a boosted SPA nav the edit body serialises to the compact
(~relate-picker :slug … :kind …), and the CLIENT expands it. The component
module is compiled to a content-addressed .sxbc, served immutably from
/sx/h/{hash}, and listed in the page's data-sx-manifest "boot" array so the
client eager-loads it after the web stack — registering its defcomp before any
boosted fragment references it.
Wiring:
- lib/host/sx/relate-picker.sx — the component.
- lib/host/blog.sx — editor emits (~relate-picker :slug s :kind k); the inline
form markup is gone.
- lib/host/static.sx — host/static-manifest-json emits boot:["relate-picker.sxbc"]
(the previously-empty boot array, now used as designed).
- hosts/ocaml/browser/sx-platform.js — loadWebStack eager-loads the page manifest's
boot[] modules (content-addressed) after the web stack.
- bundle.sh + compile-modules.js — copy/compile the component to .sxbc.
- serve.sh + conformance.sh — load the component module server-side.
This gives the host an app-component system: app defcomps shipped to the client by
hash, the same machinery as the kernel modules — the picker is the first, and it's
the model for publishing components externally.
Tests: conformance 272/272 (server expansion); relate-picker.spec.js 6/6 incl. the
boosted-nav populate (proves client-side component load + expansion) and the
error/retry case. WASM stack rebuilt (relate-picker.sxbc @ 6818110a).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three composing pieces that make the blog SPA correct and resilient.
Content-addressed module cache (lib/host/static.sx, serve.sh, blog.sx shell,
conformance.sh): index each web-stack .sxbc by the content hash in its head,
serve GET /sx/h/{hash} immutable text/sx, and emit <script data-sx-manifest>
{file->hash} so the WASM client loads modules content-addressed (localStorage +
immutable) instead of path + max-age. serve.sh builds the index at boot;
conformance.sh now loads static.sx before blog.sx (the shell calls
host/static-manifest-json).
Declarative relate picker (lib/host/blog.sx, lib/dream/form.sx): replace the
inline /relate-picker.js blob — which never ran on swapped-in content, so the
candidate list was empty after a boosted nav to /<slug>/edit — with a declarative
SX-htmx form: sx-get relate-options on "load" + debounced "input", innerHTML-swap
the results ul; infinite scroll via a server-emitted "load more" sentinel
(sx-trigger revealed, sx-swap outerHTML) that pages the rest, q preserved via a
new symmetric dr/url-encode. The engine re-binds these triggers on swapped
content, so the picker populates on full load AND boosted SPA nav. Candidate
relate forms get :sx-disable (plain POST->303->reload, their original behavior;
the engine would otherwise boost them and swap the redirect unreliably).
sx-retry "exponential:1000:30000" on the form+sentinel retries a dropped/offline
fetch forever (the cap bounds the interval, not the attempts).
SIGPIPE hardening (hosts/ocaml/bin/sx_server.ml): the native http-listen server
had no SIGPIPE handler, so a client aborting an in-flight fetch (the engine
cancels superseded requests on a debounced filter/fast nav) closed the socket
mid-write and killed the whole process (exit 141). Ignore SIGPIPE so the failed
write becomes a catchable Sys_error the per-connection handler already swallows.
Tests: host conformance 272/272; relate-picker.spec.js 5/5 incl. a boosted-nav
populate regression; spa-check 4/4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every .sxbc shipped with `:bytecode (nil nil ...)` and `:arity nil`, so the WASM
kernel's vm.sx hit "VM: unknown opcode 0" on every module and fell back to .sx
source (slower, noisy console). Root cause: `raw_serialize` in the `compile-blob`
command (sx_server.ml) handles `Number` but not `Integer`, and bytecode opcodes +
arity/upvalue-count are `Integer`s — so they fell through to the `_ -> "nil"`
catch-all and serialized as nil. Same class of bug as the value_to_js Integer gap
(689dae7d). It went unnoticed because source-fallback masks it. Add the Integer
case and regenerate: the web stack now loads entirely from bytecode (0 unknown-
opcode warnings, 0 source fallbacks), boost + SPA unchanged. compiler.sx in the
bundle was also stale — re-synced to the canonical lib/compiler.sx.
Verified: native host conformance 271/271; chromium boots with 0 unknown-opcode
warnings + 0 source-fallback loads; spa-check still passes (boost 6/6, fragment
swap). Prereq for content-addressing the assets (caching real bytecode, not nil).
Clicking a blog link now fragment-swaps #content with URL push + working back
button, no full reload — the SX-htmx engine driving the same OCaml kernel the
server runs. Six bugs in the source-load + boost path, found by bisecting in
chromium, all fixed:
1. Import double-apply (sx_server.ml x2, sx_browser.ml): the import suspension
handlers computed `key = library_name_key lib_spec` then called
`library_loaded_p key` — but library_loaded_p applies library_name_key
itself, so it ran sx_to_list on a string and crashed ("Expected list, got
string"). Only unloaded libs suspend, so it only bit lazy imports. Pass the
spec, not the key.
2. Unloaded-import crash (spec/evaluator.sx + sx_ref.ml library_exports): an
import of a not-yet-loaded library returned nil exports, and bind-import-set
did (keys nil) -> crash. Return an empty dict so the import is a graceful
no-op (lazy symbol resolution covers real usage).
3. value_to_js missing Integer (sx_browser.ml): integers passed to host methods
were mishandled, so dom-query-all's (host-call node-list "item" i) ignored i
and returned node 0 for every index — every element aliased the first, so
only one link ever boosted. Add the Integer -> JS number case.
4. browser-same-origin? rejected relative URLs (browser.sx x2): it only did
(starts-with? url origin), so "/alpha/" was treated as cross-origin and
should-boost-link? refused every relative link. Accept scheme-less,
non-protocol-relative URLs.
5. dom-query-in undefined (orchestration.sx x2): the swap path called a function
that exists nowhere; it's just dom-query with a container arg.
6. Lazy-deps never loaded under source fallback (sx-platform.js): lazy symbol
resolution only fires on the VM GLOBAL_GET path, but source-loaded swap
callbacks run on the CEK and raise instead of lazy-loading, so the post-swap
hs-boot-subtree!/htmx-boot-subtree! were undefined and aborted URL push.
Preload the manifest's lazy-deps.
Verified: native host conformance 271/271; lib/host/playwright/spa-check 4/4
(boot, boost, fragment swap + URL push, back button) in real chromium against an
ephemeral durable host server.
The kernel's sha2/cbor/cid/ed25519 modules were labelled 'WASM-safe' but assumed
63-bit native int. On the web targets — js_of_ocaml (32-bit int) and
wasm_of_ocaml (31-bit int) — they truncated, producing wrong digests/CIDs and a
Char.chr crash at kernel INIT (ed25519 precomputes sqrtm1 + base_point at module
load, driving the base-2^26 bignum). This is why a freshly-built browser kernel
crashed on boot while the stale committed artifact (older toolchain) still ran.
Fixes (all verified bit-identical to the 63-bit native build, conformance 271/271):
- sx_sha2: SHA-256 round words via Int32 (were native int + land 0xFFFFFFFF,
which is a no-op on 31-bit and overflows the constants); both SHA-256/512
length-encoding via Int64 shifts (native "lsr 32" is shift-mod-32 on js, which
leaked the length byte into a higher word). NIST vectors pass native/js/wasm.
- sx_cbor: write_head width selection + byte emission via Int64 (the 0x100000000
literal truncated to 0 on js, sending small ints to the 8-byte branch; and
"v lsr (8*i)" with i>=4 was shift-mod-32).
- sx_cid: base32_lower keeps acc bounded to the unconsumed low bits (it grew 8
bits/byte and overflowed). cid_from_sx now matches native<->js exactly.
- sx_ed25519: bignum mul accumulates in Int64 (26x26=52-bit products overflow);
div_small running remainder in Int64 (rem<<26 ~= 2^34). This was the boot gate
— the browser kernel now boots (SxKernel live, crypto-sha256 correct on js).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The OP_DIV/numeric-tower work on this branch made the OCaml `/` primitive
return an exact Rational for (/ int int) (e.g. (/ 5 2)=5/2), diverging from
the canonical spec ("/ always returns inexact float"), the test-rationals.sx
header ("in the JS host, (/ int int) returns float — backward-compatible"),
and the JS host itself. That leaked rationals into arithmetic results and
rendered CSS (tw-opacity emitted `opacity:1/20` instead of `0.05`).
Decision (with the user): keep exact rationals as an explicit opt-in
(literals 1/3, make-rational) but bring `/` back into spec/host parity —
the isomorphic SSR↔hydration invariant requires both hosts to agree, and
JS has no native rational type.
- sx_primitives.ml `/`: (/ int int) → integer when exactly divisible, else
inexact float; a Rational operand still yields an exact rational (matches
test-numeric-tower: (/ 6 2)=3, (/ 1 4)=0.25, (/ 5 2)=2.5, (/ 1/2 2)=1/4).
- sx_primitives.ml number? / exact?: recognise the Rational type (real bugs —
test-rationals asserts (number? 1/3) and (exact? 1/3); inexact?/float?
already returned false for Rational, correct).
- sx_vm.ml OP_DIV: comment updated (it delegates to the now-float `/`).
- test-rationals.sx: fix typo in "rational * float = float" — used int 2,
should be 2.0 (1/2 * 2 = 1 exact, not a float; name + siblings use floats).
OCaml conformance 4834→4863 (+29 fixed, zero new failures); rationals,
numeric-tower, arithmetic, tw-opacity suites all 0 failures. Remaining run_tests
failures are the pre-existing environmental hyperscript (host-call-fn) set.
JS host already handles number?/exact? on rationals and float `/`; its
remaining float?/contagion failures are a separate pre-existing limitation
(JS has no distinct float type), out of scope here.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The serving-JIT perform-in-HO-callback miscompile (map/rest/drop wrong
CALL_PRIM args → blank pages, empty picker) is now fully fixed, so the host
runs 100% serving JIT with NO jit-exclude.
sx-vm-extensions 81177d0e resolves a suspended HO-callback's IO inline
(instead of unwinding the native map/filter loop and corrupting the stack),
but ONLY when a synchronous resolver is installed (!_cek_io_resolver = Some).
The host serves via the http-listen primitive, whose handler drove durable IO
through cek_run_with_io with the resolver = None — so it hit the unwinding
path the fix doesn't cover. (The vm-ext repro installed a resolver, so it
never exercised the host's real no-resolver path.)
Fix: extract cek_run_with_io's IO resolution into resolve_io_request, and have
http-listen install _cek_io_resolver := Some (fun req _ -> resolve_io_request
req) — byte-identical resolution, so the inline path resolves durable reads
exactly as the CEK loop would.
Verified: host conformance 271/271; ephemeral durable server at 100% JIT (no
exclude) zero fallbacks + real content + related shown + picker 12 candidates;
live blog.rose-ash.com home/post/tags 200 with related posts, zero error-log
lines; relate-picker Playwright 4/4 (infinite-scroll + filter + relate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fixes the silent miscompile under SX_SERVING_JIT=1 (http-listen + cek_run_with_io):
a perform inside a native HO-primitive callback (map/filter/reduce/for-each)
unwound the native loop, corrupting the stack so the next CALL_PRIM read garbage
args (map/rest/drop). (A) call_closure_reuse resolves callback IO inline in
serving mode so the loop survives; (A') resume_vm restores _active_vm; (B)
register_jit_hook resolve_loop falls back to CEK on resume error (no 500).
Repro 9/9 (hosts/ocaml/bin/repro_jit_resume.ml); conformance unchanged 4834/1110.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause (found via bin/repro_jit_resume.ml, 9 surgical cases): when a
`perform` (durable kv read) fires inside a native HO-primitive callback
(map/filter/reduce/for-each/some/every?), the VmSuspended unwound through
the primitive's native OCaml loop (List.map etc.), destroying the loop's
iteration state. The remaining elements were dropped and the stack left
misaligned, so the NEXT CALL_PRIM (map/rest/drop) read wrong args —
"map: expected (fn list)", "rest: 1 list arg", "drop: list and number".
Only triggers in the http-listen + cek_run_with_io serving path (epoch
eval has no synchronous resolver, so conformance was 271/271).
(A) lib/sx_vm.ml call_closure_reuse: when a callback suspends AND a
synchronous IO resolver is installed (serving mode), resolve the
callback's IO inline and run it to completion right there, returning its
value to the native loop — so the loop is never unwound. Scoped to the
resolver-set path; the CEK-driven path (flow/reactive/async tests) keeps
its existing reuse_stack behaviour, so nothing else changes. reuse_stack
is isolated across the nested resume.
(A') lib/sx_vm.ml resume_vm: re-assert _active_vm := Some vm for the
duration of the resumed run (mirrors call_closure). call_closure restored
_active_vm to the caller when VmSuspended unwound, so HO callbacks during
a resume could land on the wrong VM. Latent-bug fix.
(B) bin/sx_server.ml register_jit_hook: the resolve_loop runs inside the
VmSuspended handler, so a non-VmSuspended exception from resume_vm escaped
to the http handler (→ 500). Catch it and fall back to CEK for THIS call
(mark jit_failed, return None → interpreter re-runs it). Self-heals on the
first hit, not a retry. Defense-in-depth; with (A) it shouldn't trigger.
Verification: repro 9/9 (incl. host shape: map[cb→interpreted-helper
perform]→drop = (7 8); reduce; nested map). Standard + --full OCaml
conformance unchanged at 4834/1110 (baseline identical — the 1110 are
pre-existing environmental: host-call-fn/browser-platform symbols,
rational display, tw/regex). Host loop to re-verify 271/271 serving and
drop its (jit-exclude! "host/*" "dream-*" "dr/*") band-aid.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 6 common-lisp opt-in-JIT failures were all condition-system continuation
escape: cl-restart-case/cl-handler-case/cl-handler-bind wrap their body in
call/cc (restarts + non-local handler exit). When an SX function that drives
the condition system (the parse-recover / interactive-debugger fixtures, e.g.
parse-numbers, make-policy-debugger) is JIT-compiled, the call/cc form runs in
a NESTED cek-run where invoking the captured continuation
runs-to-completion-and-returns instead of escaping — so a restart fails to
abort and the body falls through. Observed as result accumulation
(got (1 3 0 3) vs (1 3)) and no-abort (restart returns the 999 sentinel).
These callers are arbitrary user/fixture code, not a fixed namespace, so they
can't be prefix-excluded. New data-driven mechanism:
- jit-exclude-callers-of! registers call/cc-establishing form names in
Sx_types.jit_excluded_caller_names.
- jit_compile_lambda skips any function whose constant pool (recursively,
incl. nested closures) references a registered name — code_refs_escaping_caller.
Guarded by Hashtbl.length > 0 so it's a no-op for every guest that doesn't
register (zero effect outside CL).
- lib/common-lisp/runtime.sx registers the establish side (cl-restart-case,
cl-handler-case, cl-handler-bind) and the invoke side (cl-invoke-restart,
cl-invoke-debugger, cl-signal, cl-error-with-debugger).
Result: CL conformance under SX_SERVING_JIT=1 = 487/0, EXACTLY matching the CEK
baseline (was 484/6 with a +3 double-execution over-count). parse-recover
3/4 -> 6/0, interactive-debugger 7/2 -> 7/0.
Note: the geometry/mop-trace suites report 0/0 on BOTH CEK and JIT — they error
"Undefined symbol: refl-class-chain-depth-with" (the CLOS suites don't preload
lib/guest/reflective/class-chain.sx). Pre-existing conformance-harness gap, not
a JIT issue; left as-is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A post whose sx_content is malformed SX (e.g. "<h1 ...)" — a typo'd paren)
made GET /<slug>/ return 502, surfaced as a Cloudflare error page. Root
cause: the kernel `parse` raises a native Parse_error that an SX (guard ...)
cannot catch (guard only traps SX conditions), so host/blog-render's guard
around (parse sx) was ineffective; the exception escaped to the http-listen
loop, which swallowed it and wrote NO response — a dropped connection that
Caddy/Cloudflare relay as 502.
- kernel: add `parse-safe` — like parse but returns nil on malformed input
(value-returning, so untrusted text can be handled without a host exception).
- kernel: http-listen now synthesises a 500 response on ANY handler exception
instead of dropping the connection, so the origin stays responsive (no more
proxy 502 / branded error page) and the error is logged. This is also the
only place a native exception can be trapped, since SX guard can't.
- blog: host/blog-render uses (parse-safe sx) — malformed bodies render the
existing "(unparseable content)" placeholder; the per-block render guard
already covers unknown components (~kg-*), so /mddddd/ recovers too.
Verified live: /try-thus/ and /mddddd/ now 200 with placeholders; working
posts, home, and login unaffected. 193/193 conformance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unblock the guarded blog write routes for browsers: a login form sets a
signed session cookie that the same routes accept (alongside Bearer), so
publishing works end-to-end on blog.rose-ash.com without Quart.
- kernel: http-listen emit serialises a response :set-cookies LIST as one
Set-Cookie header each (a headers dict can't hold more than one). Purely
additive — responses without :set-cookies are unchanged.
- server.sx: host/-dream->native forwards :set-cookies to the native resp.
- lib/host/session.sx: durable, signed sessions on the persist KV
(session/create|exists|get|set|clear), wired via dream-sessions-signed.
- lib/host/auth.sx: GET/POST /login + POST /logout; host/require-user accepts
a session principal OR a Bearer token.
- router.sx: host/make-app wraps the whole app in the session middleware and
auto-mounts /login + /logout — the front door always has sessions.
- blog.sx: write routes use host/require-user; serve.sh flips POST /new from
the experimental UNGUARDED route to the guarded write routes, with admin
creds + signing secret + ACL grant from the container env.
- session conformance suite (12): login->cookie->guarded write 201; no
cookie/forged/logged-out -> 401; Bearer fallback still works.
Verified live on blog.rose-ash.com: 401 unauthenticated, 303 login, 303
publish, anonymous read renders, post persists across container recreate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The host combined-binary integration test exposed a new JIT-unsafe class:
Dream's error middleware (host/wrap-errors -> dream-catch-with) failed to catch
a thrown error under JIT — it escaped as "Unhandled exception" and truncated the
host middleware suite (7/9 vs 9/9 on CEK).
Root cause: the VM's OP_PUSH_HANDLER (the compiled form of `guard`) only
intercepts a VM-level RAISE (opcode 37); it does NOT catch the OCaml Eval_error
that the `error` primitive throws from a CALL/CALL_PRIM in a callee frame. So a
JIT-compiled `guard` silently fails to catch. dream-catch-with is curried
((fn (on-error) (fn (next) (fn (req) (guard ...))))), so the guard lives in a
NESTED closure — JIT-compiling the outer function mints that inner guard as a
VmClosure with the broken VM handler.
Fix (central, not per-callsite): scan a JIT candidate's bytecode RECURSIVELY —
including nested closure code in the constant pool — for OP_PUSH_HANDLER, and
skip JIT for any handler-installing function. It then runs on the CEK, whose
guard catches correctly. Covers dream-catch-with, host wrap-errors/blog-render,
and every other guard / handler-bind user automatically.
Verified: minimal direct guard and curried cross-frame guard both return the
caught value under JIT (were "Unhandled exception"); the host run's "kaboom"
escapes went 2 -> 0. (Remaining host blog/page failures are "Undefined symbol:
render-page" — the host's native render fn, absent from the standalone
sx_server.exe; identical on CEK, i.e. an environment artifact, not a JIT
regression. The combined host binary has render-page.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enabling the epoch serving-mode JIT globally regressed continuation-based guest
interpreters (the epoch mode is the shared command channel every loop's
conformance runner uses). Two-part fix:
1. SAFE DEFAULT GATE. register_jit_hook in the persistent server branch is now
opt-in via SX_SERVING_JIT=1 (default OFF). Default behaviour is unchanged
(no JIT in epoch serving) → zero regression for sibling loops. The
content/Smalltalk page server opts in.
2. GENERAL FIXES + per-guest interpret-only declarations:
- callable? (sx_server/run_tests/integration_tests/mcp_tree) now accepts
VmClosure. A JIT-compiled higher-order function returns its inner closure
as a VmClosure; callable? previously rejected it, so scheme-apply's
(callable? proc) guard failed with "not a procedure: <vm:anon>".
- jit-exclude! gains a trailing-"*" namespace-prefix form
(Sx_types.jit_excluded_prefixes), the robust way to mark a whole guest
interpreter interpret-only (a name-list misses functions in extra files —
it left erlang's vm/dispatcher JIT'd and 13 tests short).
- Per-guest exclusions in each guest's runtime.sx:
scheme "scheme-*" "scm-*" erlang "er-*" "erlang-*"
prolog "pl-*" common-lisp "cl-*" "clos-*"
js "js-*" haskell "hk-*"
Verified under opt-in JIT (== CEK, no hang): smalltalk 847/847, scheme/flow
166/166, erlang 530/530, prolog 590/590, apl 152/152, js 147/148. Residual
(documented, protected by the default gate): common-lisp 6 fails in advanced
suites (parser-recovery/debugger/CLOS/MOP). lua (0/16) and tcl (3/4) fail
identically on CEK — pre-existing, not JIT. run_tests --jit/no-jit unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
register_jit_hook is now installed in the persistent (epoch) serving-mode
branch of sx_server.ml, not just --http/cli/site. Smalltalk-on-SX conformance
under JIT is 847/847 — identical to the no-JIT baseline; Datalog 356/356.
run_tests --jit/no-jit are byte-identical before/after (no regression).
Five distinct root causes fixed (not one "miscompile"):
1. Serving mode never loaded lib/compiler.sx, so JIT used the native
Sx_compiler.compile stub (arity-0 bytecode, params as GLOBAL_GET →
"VM undefined: <param>"). Server-mode branch now loads compiler.sx
before registering the hook, matching http/cli/site.
2. compile-cond / compile-case-clauses / compile-guard-clauses only treated
keyword :else and true as the catch-all, not the bare symbol `else` that
the CEK's is-else-clause? accepts → GLOBAL_GET "else". (lib/compiler.sx)
3. OP_DIV produced a float for non-divisible Integer/Integer (1/2 → 0.5)
instead of the exact Rational the "/" primitive returns. Now delegates to
the primitive, matching CEK. (sx_vm.ml)
4. OP_EQ / _fast_eq lacked Rational/ListRef cases that the "=" primitive's
safe_eq has → (= 1/2 1/2) false under JIT. OP_EQ now delegates non-scalars
to the "=" primitive; _fast_eq gained rational + ListRef. (sx_vm.ml,
sx_runtime.ml)
5. Continuation-based control flow (Smalltalk ^expr non-local return, block
escape, exceptions via call/cc) can't run in the stack VM. New data-driven
exclusion set Sx_types.jit_excluded + `jit-exclude!` primitive, consulted in
jit_compile_lambda (covers both the CEK hook and vm_call's tiered path).
lib/smalltalk/eval.sx self-declares its continuation dispatch core
interpret-only; pure helpers still JIT. The SUnit suite-runner test helper
pharo-test-class miscompiles mid-loop and is excluded in tests/tokenize.sx.
Also adds SX_JIT_DENY / SX_JIT_ONLY env-var bisection filters to the serving
hook. Known residual documented in plans/jit-bytecode-correctness.md: the hook
re-runs a failed VM execution via CEK (correct result, possible duplicate side
effects); adopting run_tests' propagate-don't-rerun semantics is deferred to
avoid changing shared VM/CEK behavior under this loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
KERNEL: add a render-page primitive (sx_server.ml, persistent mode) that renders
an UNEVALUATED SX expression with the server env via sx_render_to_html.
render-to-html expands defcomp components and collects keyword attrs itself; SX
handlers can't reach the server env, so the prim supplies it. Fixes the attr
mangling — bare render-to-html on an EVALUATED component tree turns (form :id ..)
into <form>idpost-new-form..; rendering the unevaluated expr keeps :id an attr.
HOST: lib/host/page.sx — host/page (expr -> HTML response) + host/page-route
(mount on a GET path). New page suite (8 tests) proves a generic attributed +
nested component renders correctly through a host route; verified ~editor/form
renders right too. This is the component-render step of the generic
interactive-SX-page capability; shell + static assets + hydration (5.2-5.4) next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KERNEL (sx_server.ml): route http-listen handlers through cek_run_with_io
instead of bare Sx_runtime.sx_call, so handlers resolve per-request IO
(durable persist reads/writes) via the same IO-driving runner the REPL uses.
Verified: per-request read+write, 10 concurrent writes (15 on disk, no
corruption), handler errors don't crash the server, http contract 6/6.
BLOG: fully dynamic — host/blog-post reads the post from the durable store
(content/head) AND renders (content/html) per request, no in-memory view, no
cached output. Possible because of the IO fix. Honest ~2s due to interpreted
Smalltalk render.
Render speed is NOT solved here: the JIT (precompiler) isn't installed in the
serving mode and currently miscompiles the Smalltalk evaluator's nested ASTs
(enabling it breaks ~60% of tests). Fixing the JIT is a separate, high-payoff
effort. Documented in the plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote lib/host into the docker stack behind blog.rose-ash.com (reusing a
down Quart subdomain). New compose service sx_host runs lib/host/serve.sh on
externalnet; Caddy reverse-proxies blog.rose-ash.com -> sx-dev-sx_host-1:8000.
hosts/ fix: http-listen bound inet_addr_loopback only, unreachable from other
containers. Add SX_HTTP_HOST env (default loopback for tests/local; stack sets
0.0.0.0) in sx_server.ml. serve.sh made container-friendly (SX_PROJECT_DIR).
Verified live through Cloudflare->Caddy: /health, /feed, relations reads serve
real JSON; / 404 (no root route yet). rose-ash.com untouched. Conformance
145/145 green with the rebuilt binary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
blob/put|get|has? backed by <root>/blobs/<cid>, CIDv1 (raw codec,
sha2-256 via Sx_cid/Sx_sha2). put idempotent; persist stores only the
{:cid :size :mime} ref. persist_durable_test.sh extended (8/8): blob
round-trip + content-address idempotency + bytes/ref surviving real
restart. Mock blob suite 14/0 on worktree binary. Durable-storage
Blocker now CLOSED.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sx_persist_store services every persist/* IO op against on-disk storage
(append-only log + separate monotonic .seq high-water + per-key kv files,
SX-serialized). Wired into the (eval) suspension loop, cek_run_with_io
bridge, and in-process _cek_io_resolver. Data-loss repro now (3 3 3).
New persist_durable_test.sh: durable + monotonic-seq + streams + kv +
real process restart all green (5/5).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
NATIVE-ONLY http-request primitive (bin/sx_server.ml). HTTP/1.1 over
Unix sockets + gethostbyname; inline http:// URL parsing (full
url-parse deferred to Phase K); Connection: close + Host +
Content-Length headers auto-supplied; reads response via
Content-Length or read-to-EOF; chunked transfer-encoding rejected.
Test bin/test_http_client.sh spins a Phase-H echo server and drives
a second sx_server: GET+query, POST+body, 404, custom request
header reflected, non-http scheme rejected, integer status — 6/6.
WASM boot green (prim not in lib); Erlang conformance 530/530.
