rose-ash/plans/HANDOFF-enable-serving-jit.md

Hand-off: enable serving-mode JIT for ~3–4× request CPU

From the sx-vm-extensions loop (2026-06-28). The serving-mode JIT is merged to architecture and is the host's real perf win — it just needs switching on. No further engine work is required from your side.

TL;DR

Run the host server on the merged architecture binary with SX_SERVING_JIT=1 in its environment. Expected: ~3–4× lower per-request CPU (measured ~9 ms → ~2.7 ms on the /feed pipeline). Already verified correct: full host conformance is 181/181 under SX_SERVING_JIT=1.

What changed (already merged to architecture)

The bytecode JIT now works in the persistent/epoch serving mode, opt-in via the SX_SERVING_JIT env var (default OFF). Default-off means zero change until you opt in — nothing regressed for any loop. Merge commit on architecture: 089ed88f (rebuild the shared binary from architecture to pick it up).

The JIT is safe for the host's request pipeline because:

• The pipeline (dream router + feed/relations/blog handlers + JSON + render-to-html) is pure SX with no call/cc; the only continuation-style code is guard (Dream's dream-catch-with / wrap-errors), which the JIT auto-detects and runs interpreted (recursive PUSH_HANDLER scan). So error handling stays correct; everything else JITs.
• Proven end-to-end: combined host+JIT binary, full conformance under SX_SERVING_JIT=1 = 181/181, all 10 suites green (handler 14, middleware 9, sxtp 39, router 6, feed 14, relations 22, blog 27, page 8, server 13, ledger 29).

How to enable

1. Rebuild the shared binary from architecture (it carries the merge): cd hosts/ocaml && dune build bin/sx_server.exe
2. Launch the host server process with SX_SERVING_JIT=1 set in its environment (whatever wrapper/serve path you use — lib/host/serve.sx / the http-listen entry). Default-off means you must set it explicitly.
3. One-time cost: JIT compiles hot functions on first call (~+1 s at startup / first requests). Amortized immediately for a long-lived server.

Measurements (this is the evidence)

In-process, full request pipeline (host/native-handler (host/make-app …) → /feed, 2000 requests, in-memory persist backend):

	per-request CPU	total 2000 reqs
CEK (default, no JIT)	~9 ms	~15–20 s
JIT (SX_SERVING_JIT=1)	~2.7 ms	~5–6 s

JIT is also markedly less variable run-to-run. The cost is the pipeline (routing + feed normalize/stream + handler + JSON), not rendering — render-to-html alone is only ~50 µs/render and is already fast.

What was ruled out (don't chase these)

The original kickoff framed the slowness as "interpreted Smalltalk (content/html) in ~2 s". The host does not load lib/smalltalk or lib/content — that was a different subsystem. We measured and confirmed:

• The host's render path is render-to-html (SX markup → HTML), already fast.
• The proposed big engine projects — VM continuation-escape and a compile-to-closures Smalltalk interpreter — would not help the host (wrong subsystem) and are not needed. (Scoping kept in the vm-extensions loop under plans/vm-continuation-escape.md / plans/smalltalk-dispatch-perf.md if a Smalltalk-backed workload ever needs them.)

Caveat — this is CPU only

The ~3–4× is the in-process CPU path (which JIT controls). It does not touch network/IO latency. If your production TTFB is dominated by a non-in-memory persist backend, cross-service fetches, TLS/connection setup, or the known homepage SSR-stepper issue, profile those separately — JIT won't move them. To find your real split, break a live TTFB into: request parse → route → handler (+ persist read) → render → serialize → network. The in-memory measurement above says the code path is ~2.7 ms under JIT; anything beyond that in production is infrastructure, not the SX engine.

One known residual (not host-affecting, for awareness)

The serving hook re-runs a JIT'd function on the CEK if it fails mid-execution (correct result, but could duplicate side effects for an impure function that fails mid-run). The host conformance is clean (181/181), so nothing triggers it on your paths today. The clean general fix (propagate-don't-rerun) is deferred in the vm-extensions loop.

Correction (host loop, 2026-06-28)

The premise above ("~2s interpreted-Smalltalk render") is STALE: the blog moved off content-on-sx Smalltalk to render-to-html long ago (render-page ~2ms). The actual post-page unresponsiveness was NOT CPU/render — it was the DURABLE READ COUNT: host/blog--relation-blocks did ~7 kv-keys performs per page (each host/blog-out/in re-scanned the KV). Collapsing to one shared kv-keys read fixed it (~1s -> ~0.02s; commit 0a2f1a61). So serving-JIT was NOT the fix here.

Serving-JIT may still be a worthwhile general speedup (the ~3-4× CPU claim, and the Datalog instances-of on /tags is CPU-bound), but it requires running the host on the merged architecture binary — this worktree's binary has no SX_SERVING_JIT gate. Treat it as an optional future win, not the perf blocker.