vm-ext: gate serving-JIT behind SX_SERVING_JIT + fix continuation-guest regressions

Enabling the epoch serving-mode JIT globally regressed continuation-based guest
interpreters (the epoch mode is the shared command channel every loop's
conformance runner uses). Two-part fix:

1. SAFE DEFAULT GATE. register_jit_hook in the persistent server branch is now
   opt-in via SX_SERVING_JIT=1 (default OFF). Default behaviour is unchanged
   (no JIT in epoch serving) → zero regression for sibling loops. The
   content/Smalltalk page server opts in.

2. GENERAL FIXES + per-guest interpret-only declarations:
   - callable? (sx_server/run_tests/integration_tests/mcp_tree) now accepts
     VmClosure. A JIT-compiled higher-order function returns its inner closure
     as a VmClosure; callable? previously rejected it, so scheme-apply's
     (callable? proc) guard failed with "not a procedure: <vm:anon>".
   - jit-exclude! gains a trailing-"*" namespace-prefix form
     (Sx_types.jit_excluded_prefixes), the robust way to mark a whole guest
     interpreter interpret-only (a name-list misses functions in extra files —
     it left erlang's vm/dispatcher JIT'd and 13 tests short).
   - Per-guest exclusions in each guest's runtime.sx:
       scheme  "scheme-*" "scm-*"   erlang "er-*" "erlang-*"
       prolog  "pl-*"               common-lisp "cl-*" "clos-*"
       js      "js-*"               haskell "hk-*"

Verified under opt-in JIT (== CEK, no hang): smalltalk 847/847, scheme/flow
166/166, erlang 530/530, prolog 590/590, apl 152/152, js 147/148. Residual
(documented, protected by the default gate): common-lisp 6 fails in advanced
suites (parser-recovery/debugger/CLOS/MOP). lua (0/16) and tcl (3/4) fail
identically on CEK — pre-existing, not JIT. run_tests --jit/no-jit unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-19 22:22:40 +00:00
parent 952ff2289c
commit bf298684fd
14 changed files with 163 additions and 36 deletions

View File

@@ -121,3 +121,50 @@ Five distinct root causes were found and fixed (not one "miscompile"):
- A debug aid was added to the serving hook: `SX_JIT_DENY=name,...` /
`SX_JIT_ONLY=name,...` env vars to bisect which named lambda the VM
mishandles (hook-path only).
---
## Guest-loop regression sweep + safe-default gate (2026-06-19, follow-up)
Host-loop verification found that enabling serving-mode JIT **globally**
regresses continuation-based guest interpreters (the epoch serving mode is the
shared command channel for every loop's conformance runner). Failure modes:
- **VmClosure not callable** — a JIT'd higher-order function returns its inner
closure as a `VmClosure`; the native `callable?` predicate didn't list
`VmClosure`, so `scheme-apply`'s `(callable? proc)` guard rejected it
("scheme-eval: not a procedure: <vm:anon>"). FIXED generally: `callable?`
(all 4 bindings) now accepts `VmClosure`.
- **Continuation escape** — Scheme `call/cc`, Erlang receive, CL conditions,
JS exceptions: a JIT'd frame can't transfer control through a CEK
continuation.
- **Non-terminating miscompile (HANG)** — Erlang/Prolog/Haskell recursive
evaluators miscompiled into an infinite loop (worse than an error: can't
fall back).
### Mechanism
- `jit-exclude!` now accepts a trailing `*` wildcard → namespace-prefix
exclusion (`Sx_types.jit_excluded_prefixes`, checked in
`jit_compile_lambda` for both JIT entry points). One declaration per guest,
robust vs name-lists (which missed e.g. the erlang `vm/dispatcher`).
### Per-guest exclusions added (in each guest's runtime, loaded with it)
| Guest | Declaration | Status under opt-in JIT |
|-------|-------------|--------------------------|
| smalltalk | name-list (dispatch core) + `pharo-test-class` | 847/847 == CEK |
| scheme | `(jit-exclude! "scheme-*" "scm-*")` | flow 166/166 == CEK |
| erlang | `(jit-exclude! "er-*" "erlang-*")` | 530/530 == CEK, no hang |
| prolog | `(jit-exclude! "pl-*")` | 590/590 == CEK |
| common-lisp | `(jit-exclude! "cl-*" "clos-*")` | residual: 6 fail (advanced suites) |
| js | `(jit-exclude! "js-*")` | (verifying) |
| haskell | `(jit-exclude! "hk-*")` | (verifying) |
Not JIT-related (fail identically on CEK and JIT, pre-existing): lua 0/16,
tcl 3/4. apl/datalog/forth/ocaml: clean under JIT as-is (no continuations).
### Safe-default gate
Serving-mode JIT is now **opt-in via `SX_SERVING_JIT=1` (default OFF)** in
`sx_server.ml`. Default behavior is unchanged (no JIT in epoch serving) ⇒
**zero regression** for every sibling loop's conformance. The content/Smalltalk
page server opts in. This bounds risk: guests are validated and excluded
incrementally; until then the default protects them. Common-Lisp's advanced
suites still need investigation before CL is opt-in-clean.