The host combined-binary integration test exposed a new JIT-unsafe class:
Dream's error middleware (host/wrap-errors -> dream-catch-with) failed to catch
a thrown error under JIT — it escaped as "Unhandled exception" and truncated the
host middleware suite (7/9 vs 9/9 on CEK).
Root cause: the VM's OP_PUSH_HANDLER (the compiled form of `guard`) only
intercepts a VM-level RAISE (opcode 37); it does NOT catch the OCaml Eval_error
that the `error` primitive throws from a CALL/CALL_PRIM in a callee frame. So a
JIT-compiled `guard` silently fails to catch. dream-catch-with is curried
((fn (on-error) (fn (next) (fn (req) (guard ...))))), so the guard lives in a
NESTED closure — JIT-compiling the outer function mints that inner guard as a
VmClosure with the broken VM handler.
Fix (central, not per-callsite): scan a JIT candidate's bytecode RECURSIVELY —
including nested closure code in the constant pool — for OP_PUSH_HANDLER, and
skip JIT for any handler-installing function. It then runs on the CEK, whose
guard catches correctly. Covers dream-catch-with, host wrap-errors/blog-render,
and every other guard / handler-bind user automatically.
Verified: minimal direct guard and curried cross-frame guard both return the
caught value under JIT (were "Unhandled exception"); the host run's "kaboom"
escapes went 2 -> 0. (Remaining host blog/page failures are "Undefined symbol:
render-page" — the host's native render fn, absent from the standalone
sx_server.exe; identical on CEK, i.e. an environment artifact, not a JIT
regression. The combined host binary has render-page.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enabling the epoch serving-mode JIT globally regressed continuation-based guest
interpreters (the epoch mode is the shared command channel every loop's
conformance runner uses). Two-part fix:
1. SAFE DEFAULT GATE. register_jit_hook in the persistent server branch is now
opt-in via SX_SERVING_JIT=1 (default OFF). Default behaviour is unchanged
(no JIT in epoch serving) → zero regression for sibling loops. The
content/Smalltalk page server opts in.
2. GENERAL FIXES + per-guest interpret-only declarations:
- callable? (sx_server/run_tests/integration_tests/mcp_tree) now accepts
VmClosure. A JIT-compiled higher-order function returns its inner closure
as a VmClosure; callable? previously rejected it, so scheme-apply's
(callable? proc) guard failed with "not a procedure: <vm:anon>".
- jit-exclude! gains a trailing-"*" namespace-prefix form
(Sx_types.jit_excluded_prefixes), the robust way to mark a whole guest
interpreter interpret-only (a name-list misses functions in extra files —
it left erlang's vm/dispatcher JIT'd and 13 tests short).
- Per-guest exclusions in each guest's runtime.sx:
scheme "scheme-*" "scm-*" erlang "er-*" "erlang-*"
prolog "pl-*" common-lisp "cl-*" "clos-*"
js "js-*" haskell "hk-*"
Verified under opt-in JIT (== CEK, no hang): smalltalk 847/847, scheme/flow
166/166, erlang 530/530, prolog 590/590, apl 152/152, js 147/148. Residual
(documented, protected by the default gate): common-lisp 6 fails in advanced
suites (parser-recovery/debugger/CLOS/MOP). lua (0/16) and tcl (3/4) fail
identically on CEK — pre-existing, not JIT. run_tests --jit/no-jit unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
register_jit_hook is now installed in the persistent (epoch) serving-mode
branch of sx_server.ml, not just --http/cli/site. Smalltalk-on-SX conformance
under JIT is 847/847 — identical to the no-JIT baseline; Datalog 356/356.
run_tests --jit/no-jit are byte-identical before/after (no regression).
Five distinct root causes fixed (not one "miscompile"):
1. Serving mode never loaded lib/compiler.sx, so JIT used the native
Sx_compiler.compile stub (arity-0 bytecode, params as GLOBAL_GET →
"VM undefined: <param>"). Server-mode branch now loads compiler.sx
before registering the hook, matching http/cli/site.
2. compile-cond / compile-case-clauses / compile-guard-clauses only treated
keyword :else and true as the catch-all, not the bare symbol `else` that
the CEK's is-else-clause? accepts → GLOBAL_GET "else". (lib/compiler.sx)
3. OP_DIV produced a float for non-divisible Integer/Integer (1/2 → 0.5)
instead of the exact Rational the "/" primitive returns. Now delegates to
the primitive, matching CEK. (sx_vm.ml)
4. OP_EQ / _fast_eq lacked Rational/ListRef cases that the "=" primitive's
safe_eq has → (= 1/2 1/2) false under JIT. OP_EQ now delegates non-scalars
to the "=" primitive; _fast_eq gained rational + ListRef. (sx_vm.ml,
sx_runtime.ml)
5. Continuation-based control flow (Smalltalk ^expr non-local return, block
escape, exceptions via call/cc) can't run in the stack VM. New data-driven
exclusion set Sx_types.jit_excluded + `jit-exclude!` primitive, consulted in
jit_compile_lambda (covers both the CEK hook and vm_call's tiered path).
lib/smalltalk/eval.sx self-declares its continuation dispatch core
interpret-only; pure helpers still JIT. The SUnit suite-runner test helper
pharo-test-class miscompiles mid-loop and is excluded in tests/tokenize.sx.
Also adds SX_JIT_DENY / SX_JIT_ONLY env-var bisection filters to the serving
hook. Known residual documented in plans/jit-bytecode-correctness.md: the hook
re-runs a failed VM execution via CEK (correct result, possible duplicate side
effects); adopting run_tests' propagate-don't-rerun semantics is deferred to
avoid changing shared VM/CEK behavior under this loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A tz event now exports DTSTART;TZID=<name>:<local> (EXDATE/RDATE likewise;
UNTIL stays UTC per RFC), and the VCALENDAR emits a VTIMEZONE per distinct zone
with DAYLIGHT/STANDARD sub-components generated from the zone's transition rules
(offsets + FREQ=YEARLY;BYMONTH;BYDAY) — London/Paris blocks match real-world
definitions. Clients recur at fixed wall-clock time, DST-correct (prior caveat
gone). Importer tolerates ;TZID= params. 376/376 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug: tz events store wall-clock LOCAL times but export stamped them with a Z
(UTC) suffix, so a London 18:00 event falsely read as 18:00 UTC. ev-ical-conv
now converts a tz event's DTSTART/UNTIL/EXDATE/RDATE local->UTC before
formatting (London summer 18:00 -> 170000Z; Paris -> 160000Z); non-tz events
unchanged. Caveat: UTC RRULE drifts from wall-clock-stable tz recurrence across
a DST boundary (VTIMEZONE deferred). 366/366 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ical.sx parses VEVENT/VCALENDAR text back into events (ev/ical-lines->event,
ev/parse-vcalendar): DTSTART/DURATION/RRULE (ordinal BYDAY, BYMONTHDAY, UNTIL/
COUNT/INTERVAL) + EXDATE/RDATE. Round-trip is occurrence-exact — export->import
expands to the identical occurrence set. Completes bidirectional interop.
360/360 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ev/book-series! / ev/cancel-series! apply a booking/cancel to every occurrence
of one event in a window (RSVP the whole weekly class), returning per-
occurrence (occ-key status) results; capacity still enforced per occurrence
(some :booked, some :full), idempotent re-book (:already). ev/series-count,
ev/series-booked. 341/341 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Surgical add of the two radar-authored planning docs onto architecture (both new
files, no conflict). Migration strategy: duplicate->cutover->diverge, strangler edge
+ layer-split shadow-diff, host-trio critical path. abstractions.md is the evidence
base the strategy cites (A1 done, W1/W4/W8 substrate-adoption findings).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Doc-only: records that the http-listen 'handler-mutex deadlock' is not a
mutex bug but an Erlang-scheduler-context issue (handler runs on a native
Thread.create outside any er-sched step, so gen_server:call->receive can
never complete). Pattern A inapplicable; correct fix is Pattern B in
er-bif-http-listen (lib/erlang, m2 scope). Full diagnosis + patch sketch in
plans/fed-sx-host-primitives.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Facade read-by-id was top-level only while content/edit's update/delete are
tree-wide — could not read back a nested block content/edit just modified.
Added generic ct-find-id (doc.sx) + doc-find-deep/doc-has-deep?; content/find
+ has? now descend into sections. content/find-top/has-top? keep top-level
lookup. Audit: remaining doc-find/ct-index-of callers are positional
insert/move (top-level by design). +6 api tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Investigated the http-listen "handler-mutex deadlock" per
plans/agent-briefings/fed-prims-mutex-fix.md. Reproduced deterministically
(single kernel-route request returns empty reply while a non-kernel route
returns 200; also reproduced with a 3-line minimal echo gen_server).
Root cause is in the Erlang substrate, not the OCaml mutex: native
http-listen runs each handler on a fresh Thread.create outside any Erlang
scheduler step, so gen_server:call -> receive (which raises er-suspend-marker
expecting an enclosing er-sched-step-alive! guard + er-sched-run-all! pump)
can never complete.
Pattern A is inapplicable: the failure reproduces on a single request with
zero contention, so it is not a mutex-contention deadlock; the mutex is in
fact required and must stay. Sx_runtime.sx_call is fully synchronous and no
OCaml symbol reaches the SX-level scheduler, so there is no OCaml-only fix.
The correct fix is Pattern B done entirely in er-bif-http-listen
(lib/erlang/runtime.sx) — spawn the handler as an er-process and
er-sched-run-all! to completion — which is m2 / loops/erlang scope.
Doc-only: full diagnosis + concrete patch sketch added to the Blockers and
Progress log of plans/fed-sx-host-primitives.md. No bin/sx_server.ml change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ical.sx serializes events to VEVENT/VCALENDAR text for import by standard
clients: UTC basic-format stamps, DURATION (PT#H#M), full RRULE
(FREQ/INTERVAL/COUNT/UNTIL/BYDAY incl. monthly ordinals 2TU/-1FR/BYMONTHDAY)
plus EXDATE/RDATE. Line-oriented (ev/event->ical-lines / ev/events->ical-lines)
with ev/ical-render joining CRLF for the wire format. 332/332 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
oauth.sx routes the PKCE check through pkce_ok: an S256 challenge carried as
{s256, Hash} compares crypto:hash(sha256, Verifier) =:= Hash; a bare
challenge stays plain (§4.1), so both methods coexist with no change to
existing flows (the bare path is the old =:= behaviour). Raw sha256 digests
are compared (base64url is wire encoding, omitted). New tests/pkce.sx (6,
incl. S256 through PAR). Verified pkce 6/6; substrate fix is in the
preceding commit. 239 total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
er-eval-binary-segment evaluated a string-valued segment (the parser
represents <<"abc">> as one integer segment whose value is the whole string
"abc") by calling er-emit-int! on the string, emitting a single bogus 0
byte. So every <<"...">> literal became {:tag "binary" :bytes (0)} — which
made binary =:= read as "always equal" and crypto:hash input-independent.
Fix: the integer branch now expands a string value to one byte per
character (Erlang semantics: <<"abc">> ≡ <<97,98,99>>). Verified:
byte_size(<<"abc">>)=3, <<"a">> =:= <<"b">> is false, crypto:hash distinct
per input.
(User-authorized cross-scope fix from the identity loop; loops/erlang
should adopt this as the owner of lib/erlang.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ev/book-checked! prevents an attendee double-booking themselves across
different events by consulting their persist-derived availability for the
occurrence window (:time-conflict on overlap; same-occurrence re-book stays
idempotent).
ev/book-checked! prevents an attendee double-booking themselves across
different events: consults their persist-derived availability (ev/free-p?) for
the occurrence window, returns :time-conflict on overlap else the normal
ev/book-occ! result. Re-booking the same occurrence stays idempotent
(:already); other actors unaffected. ev/would-time-conflict? predicate.
311/311 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tests/integration.sx — one narrative across every module: catalog -> stock
check -> quote (promo+stack+tax) -> attribution -> order flow -> payment
envelope -> settle -> recon -> refund flow -> ledger mismatch, asserting the
seams tie together with consistent numbers. Proves the three-substrate
composition (minikanren pricing + flow lifecycle + persist ledger) end to end.
Total 297/297 across 18 suites.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
