The lone js opt-in-JIT residual was async/await_in_loop, which failed to PARSE
under JIT ("Unexpected token: op '<'" on `i < 5`) while passing on CEK. The js
exclusion was "js-*", but the recursive-descent parser is the jp-* namespace
(75 functions in lib/js/parser.sx) — only the lexer/transpile/runtime are js-*.
So the parser was left JIT-eligible and a jp-* function miscompiled this
construct (the long-standing parser-miscompile class).
Fix: extend the js exclusion to "js-* jp-*" so the parser is interpret-only too,
matching how every other guest's front-end is handled. js conformance under
SX_SERVING_JIT=1 is now 148/148, == CEK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 6 common-lisp opt-in-JIT failures were all condition-system continuation
escape: cl-restart-case/cl-handler-case/cl-handler-bind wrap their body in
call/cc (restarts + non-local handler exit). When an SX function that drives
the condition system (the parse-recover / interactive-debugger fixtures, e.g.
parse-numbers, make-policy-debugger) is JIT-compiled, the call/cc form runs in
a NESTED cek-run where invoking the captured continuation
runs-to-completion-and-returns instead of escaping — so a restart fails to
abort and the body falls through. Observed as result accumulation
(got (1 3 0 3) vs (1 3)) and no-abort (restart returns the 999 sentinel).
These callers are arbitrary user/fixture code, not a fixed namespace, so they
can't be prefix-excluded. New data-driven mechanism:
- jit-exclude-callers-of! registers call/cc-establishing form names in
Sx_types.jit_excluded_caller_names.
- jit_compile_lambda skips any function whose constant pool (recursively,
incl. nested closures) references a registered name — code_refs_escaping_caller.
Guarded by Hashtbl.length > 0 so it's a no-op for every guest that doesn't
register (zero effect outside CL).
- lib/common-lisp/runtime.sx registers the establish side (cl-restart-case,
cl-handler-case, cl-handler-bind) and the invoke side (cl-invoke-restart,
cl-invoke-debugger, cl-signal, cl-error-with-debugger).
Result: CL conformance under SX_SERVING_JIT=1 = 487/0, EXACTLY matching the CEK
baseline (was 484/6 with a +3 double-execution over-count). parse-recover
3/4 -> 6/0, interactive-debugger 7/2 -> 7/0.
Note: the geometry/mop-trace suites report 0/0 on BOTH CEK and JIT — they error
"Undefined symbol: refl-class-chain-depth-with" (the CLOS suites don't preload
lib/guest/reflective/class-chain.sx). Pre-existing conformance-harness gap, not
a JIT issue; left as-is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Enabling the epoch serving-mode JIT globally regressed continuation-based guest
interpreters (the epoch mode is the shared command channel every loop's
conformance runner uses). Two-part fix:
1. SAFE DEFAULT GATE. register_jit_hook in the persistent server branch is now
opt-in via SX_SERVING_JIT=1 (default OFF). Default behaviour is unchanged
(no JIT in epoch serving) → zero regression for sibling loops. The
content/Smalltalk page server opts in.
2. GENERAL FIXES + per-guest interpret-only declarations:
- callable? (sx_server/run_tests/integration_tests/mcp_tree) now accepts
VmClosure. A JIT-compiled higher-order function returns its inner closure
as a VmClosure; callable? previously rejected it, so scheme-apply's
(callable? proc) guard failed with "not a procedure: <vm:anon>".
- jit-exclude! gains a trailing-"*" namespace-prefix form
(Sx_types.jit_excluded_prefixes), the robust way to mark a whole guest
interpreter interpret-only (a name-list misses functions in extra files —
it left erlang's vm/dispatcher JIT'd and 13 tests short).
- Per-guest exclusions in each guest's runtime.sx:
scheme "scheme-*" "scm-*" erlang "er-*" "erlang-*"
prolog "pl-*" common-lisp "cl-*" "clos-*"
js "js-*" haskell "hk-*"
Verified under opt-in JIT (== CEK, no hang): smalltalk 847/847, scheme/flow
166/166, erlang 530/530, prolog 590/590, apl 152/152, js 147/148. Residual
(documented, protected by the default gate): common-lisp 6 fails in advanced
suites (parser-recovery/debugger/CLOS/MOP). lua (0/16) and tcl (3/4) fail
identically on CEK — pre-existing, not JIT. run_tests --jit/no-jit unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
register_jit_hook is now installed in the persistent (epoch) serving-mode
branch of sx_server.ml, not just --http/cli/site. Smalltalk-on-SX conformance
under JIT is 847/847 — identical to the no-JIT baseline; Datalog 356/356.
run_tests --jit/no-jit are byte-identical before/after (no regression).
Five distinct root causes fixed (not one "miscompile"):
1. Serving mode never loaded lib/compiler.sx, so JIT used the native
Sx_compiler.compile stub (arity-0 bytecode, params as GLOBAL_GET →
"VM undefined: <param>"). Server-mode branch now loads compiler.sx
before registering the hook, matching http/cli/site.
2. compile-cond / compile-case-clauses / compile-guard-clauses only treated
keyword :else and true as the catch-all, not the bare symbol `else` that
the CEK's is-else-clause? accepts → GLOBAL_GET "else". (lib/compiler.sx)
3. OP_DIV produced a float for non-divisible Integer/Integer (1/2 → 0.5)
instead of the exact Rational the "/" primitive returns. Now delegates to
the primitive, matching CEK. (sx_vm.ml)
4. OP_EQ / _fast_eq lacked Rational/ListRef cases that the "=" primitive's
safe_eq has → (= 1/2 1/2) false under JIT. OP_EQ now delegates non-scalars
to the "=" primitive; _fast_eq gained rational + ListRef. (sx_vm.ml,
sx_runtime.ml)
5. Continuation-based control flow (Smalltalk ^expr non-local return, block
escape, exceptions via call/cc) can't run in the stack VM. New data-driven
exclusion set Sx_types.jit_excluded + `jit-exclude!` primitive, consulted in
jit_compile_lambda (covers both the CEK hook and vm_call's tiered path).
lib/smalltalk/eval.sx self-declares its continuation dispatch core
interpret-only; pure helpers still JIT. The SUnit suite-runner test helper
pharo-test-class miscompiles mid-loop and is excluded in tests/tokenize.sx.
Also adds SX_JIT_DENY / SX_JIT_ONLY env-var bisection filters to the serving
hook. Known residual documented in plans/jit-bytecode-correctness.md: the hook
re-runs a failed VM execution via CEK (correct result, possible duplicate side
effects); adopting run_tests' propagate-don't-rerun semantics is deferred to
avoid changing shared VM/CEK behavior under this loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Substrate fix: two-line change to lib/erlang/runtime.sx that lets
http-listen handler routes call gen_server:call without deadlocking.
1. er-sched-step-alive!: pass :pending-args (when set) to the
initial-fun call instead of always passing an empty list.
Default behavior (no field) stays (list) — drop-in safe.
2. er-bif-http-listen sx-handler: instead of er-apply-fun handler
inline (which blows up on receive's er-suspend-marker because
the connection thread has no scheduler step on its stack),
create a real er-process with :initial-fun = handler and
:pending-args = (list req-pl), then er-sched-run-all! to drain.
Any receive (e.g. gen_server:call) suspends + resumes inside
the SX scheduler frame the process owns. Read :exit-result
for the response proplist; marshal back to SX dict.
Investigation arc (see plans/fed-sx-milestone-2.md Blockers #4 +
Progress log):
- loops/fed-prims bf8d0bf2 diagnosed it as Erlang-substrate, not
OCaml mutex (Pattern A wrong, Pattern B right but sketchy).
- First Pattern B attempt failed: tried er-spawn-fun on a raw SX
lambda, hit (er-fun? fv) gate. Connection-thread bisect
pinpointed the exact line.
- Real fix: use the existing er-fun (user's handler) directly,
but feed it via :pending-args so step-alive's hardcoded
(list) doesn't drop the request arg.
Acceptance:
- new next/tests/smoke_kernel_route.sh: 6/6 over real HTTP
(welcome /, /actors/alice, /actors/alice/outbox with
gen_server-backed tip, /actors/alice/inbox, unknown-actor,
via http_server:start(P, [{kernel, nx_kernel}])).
- next/tests/http_server_tcp.sh: 5/5 (bumped wait_bound from
30s to 180s — cold boot is slow under sibling-loop CPU load
and the per-handler scheduler ramp adds a small margin).
- Erlang conformance: 761/761.
Step 12's two-instance smoke test is now unblocked — its full
Follow / Accept / Note flow can layer on top of this kernel-route
surface. m2 plan updated.
Pre-existing httpc_request.sh flakiness ("Undefined symbol:
http-request" on the live-call epochs) reproduces WITHOUT this
change — see git stash A/B in the investigation. Unrelated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A tz event now exports DTSTART;TZID=<name>:<local> (EXDATE/RDATE likewise;
UNTIL stays UTC per RFC), and the VCALENDAR emits a VTIMEZONE per distinct zone
with DAYLIGHT/STANDARD sub-components generated from the zone's transition rules
(offsets + FREQ=YEARLY;BYMONTH;BYDAY) — London/Paris blocks match real-world
definitions. Clients recur at fixed wall-clock time, DST-correct (prior caveat
gone). Importer tolerates ;TZID= params. 376/376 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bug: tz events store wall-clock LOCAL times but export stamped them with a Z
(UTC) suffix, so a London 18:00 event falsely read as 18:00 UTC. ev-ical-conv
now converts a tz event's DTSTART/UNTIL/EXDATE/RDATE local->UTC before
formatting (London summer 18:00 -> 170000Z; Paris -> 160000Z); non-tz events
unchanged. Caveat: UTC RRULE drifts from wall-clock-stable tz recurrence across
a DST boundary (VTIMEZONE deferred). 366/366 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ical.sx parses VEVENT/VCALENDAR text back into events (ev/ical-lines->event,
ev/parse-vcalendar): DTSTART/DURATION/RRULE (ordinal BYDAY, BYMONTHDAY, UNTIL/
COUNT/INTERVAL) + EXDATE/RDATE. Round-trip is occurrence-exact — export->import
expands to the identical occurrence set. Completes bidirectional interop.
360/360 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ev/book-series! / ev/cancel-series! apply a booking/cancel to every occurrence
of one event in a window (RSVP the whole weekly class), returning per-
occurrence (occ-key status) results; capacity still enforced per occurrence
(some :booked, some :full), idempotent re-book (:already). ev/series-count,
ev/series-booked. 341/341 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Facade read-by-id was top-level only while content/edit's update/delete are
tree-wide — could not read back a nested block content/edit just modified.
Added generic ct-find-id (doc.sx) + doc-find-deep/doc-has-deep?; content/find
+ has? now descend into sections. content/find-top/has-top? keep top-level
lookup. Audit: remaining doc-find/ct-index-of callers are positional
insert/move (top-level by design). +6 api tests.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ical.sx serializes events to VEVENT/VCALENDAR text for import by standard
clients: UTC basic-format stamps, DURATION (PT#H#M), full RRULE
(FREQ/INTERVAL/COUNT/UNTIL/BYDAY incl. monthly ordinals 2TU/-1FR/BYMONTHDAY)
plus EXDATE/RDATE. Line-oriented (ev/event->ical-lines / ev/events->ical-lines)
with ev/ical-render joining CRLF for the wire format. 332/332 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
oauth.sx routes the PKCE check through pkce_ok: an S256 challenge carried as
{s256, Hash} compares crypto:hash(sha256, Verifier) =:= Hash; a bare
challenge stays plain (§4.1), so both methods coexist with no change to
existing flows (the bare path is the old =:= behaviour). Raw sha256 digests
are compared (base64url is wire encoding, omitted). New tests/pkce.sx (6,
incl. S256 through PAR). Verified pkce 6/6; substrate fix is in the
preceding commit. 239 total.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
er-eval-binary-segment evaluated a string-valued segment (the parser
represents <<"abc">> as one integer segment whose value is the whole string
"abc") by calling er-emit-int! on the string, emitting a single bogus 0
byte. So every <<"...">> literal became {:tag "binary" :bytes (0)} — which
made binary =:= read as "always equal" and crypto:hash input-independent.
Fix: the integer branch now expands a string value to one byte per
character (Erlang semantics: <<"abc">> ≡ <<97,98,99>>). Verified:
byte_size(<<"abc">>)=3, <<"a">> =:= <<"b">> is false, crypto:hash distinct
per input.
(User-authorized cross-scope fix from the identity loop; loops/erlang
should adopt this as the owner of lib/erlang.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ev/book-checked! prevents an attendee double-booking themselves across
different events by consulting their persist-derived availability for the
occurrence window (:time-conflict on overlap; same-occurrence re-book stays
idempotent).
ev/book-checked! prevents an attendee double-booking themselves across
different events: consults their persist-derived availability (ev/free-p?) for
the occurrence window, returns :time-conflict on overlap else the normal
ev/book-occ! result. Re-booking the same occurrence stays idempotent
(:already); other actors unaffected. ev/would-time-conflict? predicate.
311/311 green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The er-bif-http-listen BIF body in lib/erlang/runtime.sx referenced
er-http-resp-to-sx / er-http-req-of-sx — helpers deleted by 78eae9ef
("fed-sx-m1: 8b-bridge cleanup") because the BIF body never picked
them up. Listener bound but every request handler crashed on first
call to the undefined helpers; curl got 000 / empty body.
Rewrote the sx-handler bridge to thread through the live marshallers
that the cleanup commit's message claimed were already in use:
Inbound: SX Dict {:method :path :query :headers :body}
-> er-request-dict-to-proplist
-> Erlang request proplist matching http_server:route/2 shape
(binaries for path/method/body, dict-like proplist for headers)
Outbound: Erlang [{status, N}, {headers, [{Bin, Bin}, ...]}, {body, Bin}]
-> er-proplist-to-dict
-> SX Dict matching what native http-listen serialises
(er-to-sx-deep auto-converts binary values to strings and
flattens the 2-tuple headers cons to a nested SX dict)
This is technically substrate work in lib/erlang/runtime.sx but
stays within the m2 briefing's allowed exception scope — the http
BIF wrappers (Step 8a / 8e / now 12-prep) are the explicit substrate
carve-outs. Unblocks Step 12's REAL two-instance smoke test rather
than an in-process loopback variant.
Test: next/tests/http_server_tcp.sh 5/5
- GET / -> 200
- GET /.well-known/sx-capabilities -> 200 (body contains "kernel:")
- GET /no-such-path -> 404
- POST /activity (no bearer) -> 401
- POST /activity (bad bearer) -> 401
No-regression gates green: Erlang conformance 761/761,
httpc_request 10/10, dispatch_http 10/10, http_listen_bif 5/5,
discovery_fetch 11/11, http_multi_actor 44/44, http_marshal 10/10.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
