The 100%-JIT experiment surfaced something worse than the 500s: the kernel
miscompile also returns WRONG RESULTS with no error — blank pages (render map
yields empty) and an empty relate picker (drop in relate-options yields []).
Conformance (CEK) passes these, so the code is correct; the JIT silently
produces garbage. Silent corruption is worse than a crash, so the request path
runs on CEK again (IO-bound — no perf loss). Datalog/relations JIT stays on
(/tags 0.16s). Restoring it brought back content + the 17-candidate picker.
Go 100% JIT again once sx-vm-extensions fixes the OP_PERFORM-resume bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per dev intent: don't mask JIT errors. Remove (jit-exclude! "host/*" "dream-*"
"dr/*") so ALL request-path SX runs under JIT. Host handlers miscompile on first
call in the http-listen path (map/rest arg bug → 500, self-heals on retry); that
surfacing is the point — it exercises the JIT against real durable-IO traffic and
gives the sx-vm-extensions loop the full miscompile list to fix (kernel bugs A/B
in plans/HANDOFF-jit-miscompile.md). Datalog JIT win stays (/tags fast).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Datalog/relations saturation is CPU-bound and JITs cleanly: host conformance
271/271 under JIT, 5.4x faster (1m43s -> 19s, same binary); live /tags 2.5s ->
0.76s. loops/host now carries the merged sx-vm-extensions kernel (the JIT engine
+ gate), built into the binary the container bind-mounts.
- docker-compose: SX_SERVING_JIT=1 (default-OFF gate; opt-in here).
- serve.sh: when JIT is on, (jit-exclude! "host/*" "dream-*" "dr/*"). The host app
+ Dream framework MISCOMPILE on first call in the http-listen + cek_run_with_io
path (map/rest emit wrong CALL_PRIM args -> 500; the JIT->CEK fallback marks the
fn failed but does NOT recover the failed call). They're IO-bound, so CEK is no
slower — but the miscompile is a real kernel-JIT bug to fix upstream (see
plans/HANDOFF-jit-miscompile.md), after which this exclude can be dropped.
Verified live: cold pages 200 (no first-hit 500), relate picker lists candidates,
relate round-trip works, /tags fast, datalog still JITs (78 dl-* compiles).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 6 — gradual schema validation made real:
- host/blog-type-schemas now carries a declarative schema (a list of
{:block :msg} required-element rules); "article" requires an h1.
- host/blog--all-tags / --schema-issues / host/blog-type-issues walk the parsed
content and report each missing required block; host/blog-type-valid? = no
issues. A type with no schema imposes nothing (gradual).
- seed an "article" type-post (article subtype-of type). edit-submit now lists
the specific schema issues on a 400 ("an article needs a heading"), so a post
that is-a article must satisfy it on save.
Post-page performance (the unresponsiveness): a post page was ~1s even with no
relations and no load — NOT CPU (render-page ~2ms, in-memory handler ~5ms) but
the DURABLE read path: host/blog--relation-blocks called host/blog-out/in, each
re-scanning the whole KV (host/blog-slugs + an all-edges scan), so a page did ~7
kv-keys performs deep in the call stack. Each durable perform routes through
cek_run_with_io and is costly there. Fixes:
- host/blog-out/in read DIRECT edges from the durable edge store (string scan),
not lib/relations (whose queries re-saturate the Datalog ruleset, ~seconds).
- host/blog--relation-blocks reads the KV key list ONCE and derives both the post
set and the edges in memory (host/blog--edges-for / --recs-slugs), one kv-keys
plus a host/blog-get per linked post. Post pages: ~1s -> ~0.02s (46x); live
11-135s -> ~0.15s. lib/relations stays for TRANSITIVE queries only.
- conformance timeout 300 -> 600s: the relations-heavy blog suite is CPU-bound
under shared-box contention and was tripping a false truncation at 300.
271/271 (blog 100). Verified live: post pages fast, Tags/Related/Tagged-with-this
render, schema rejection works.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the hard-coded related/tagged blocks with iteration over the registry,
so adding a kind renders automatically — no handler edit.
- host/blog--relation-blocks: iterates host/blog-rel-kinds; each kind contributes
its outgoing block (label) and, if it has an inverse, its incoming block
(inverse-label, e.g. tagged -> "Tagged with this", is-a -> "Instances"). Empty
blocks dropped; one kv-keys read up front, relation lookups in-memory.
host/blog--relations-or-hint adds the logged-in "add some" hint when empty.
- host/blog--relation-editors: one editor per registry kind on the edit page
(Related / Types / Subtype of / Tags), replacing the hard-coded two.
- GET /tags: index of every tag (a post that is-a tag), each linking its own page.
- dropped host/blog--related-block / --kind-block / --tagged-with-block (folded
into host/blog--edges-block + the registry iteration).
- GOTCHA (4th time): host/blog-tags-index called host/blog-get INSIDE the item
quasiquote -> VmSuspended/500 live (conformance in-memory store can't see it);
pre-fetch records before the quasiquote.
5 tests (relations-section hint, registry render of Related+Tags, inverse block
for a tag, /tags lists + 200). 265/265; Playwright 4/4. Verified live: /tags,
post pages show registry blocks, tag page shows Types + Tagged-with-this, edit
page has a picker per kind.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A tag is just a post that is-a tag; tagging is a "tagged" edge to it. End to end:
mark a post a tag, tag posts with it, see a post's tags and a tag's members.
- helpers: host/blog-is-tag? (= is-a? slug "tag"), host/blog-tags (out tagged),
host/blog-tagged-with (in tagged), host/blog-instances-of (a type's members,
O(#subtypes) not O(#posts) — the efficient candidate source).
- picker generalised to be KIND-AWARE and MULTI-INSTANCE: relate-options takes
&kind=, candidates come from the kind's registry :candidates (all/tags/types);
/relate-picker.js wires every .relate-picker box by data-kind (a Related picker
and a Tags picker now coexist on the edit page).
- render: post page gains a "Tags" block; a tag post additionally lists "Tagged
with this" (its members). edit page: a Related editor + a Tags editor + an
"is this post a tag" toggle (reuses /relate kind=is-a — no new route).
- GOTCHA (again): host/blog--relation-editor read host/blog-out INSIDE its
quasiquote -> VmSuspended/500 under http-listen + durable edges; moved the read
to a let before the quasiquote (conformance can't see it — in-memory store;
the ephemeral Playwright run caught it).
6 conformance tests (is-tag?, instances-of, tag+tagged-with, tagged picker offers
only tags, related picker still all, is-a-tag toggle) -> 261/261. Playwright
multi-picker 4/4. Verified live: ocaml made a tag, welcome tagged ocaml, Tags
block + Tagged-with-this both render.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The spine: types ARE posts, and typing is transitive the right way. is-a
(instance-of) does NOT chain on its own, but subsumption does — an instance of a
subtype is an instance of the supertype.
- registry gains "subtype-of" (directed, transitive). host/blog-types-of(slug) =
declared is-a targets PLUS every subtype-of-ancestor of each (composed host-side
over relations/descendants — no new Datalog rules). host/blog-is-a?(slug,type)
is transitive through subtype-of.
- host/blog-seed-types! seeds the root type-posts "type" and "tag" (real posts
that document themselves) with tag subtype-of type, so anything is-a tag is
transitively a type. Idempotent; wired into serve.sh.
- gradual-validation seam: host/blog-type-schemas (empty) + host/blog-schema-of +
host/blog-type-valid? (vacuously true with no schemas) wired into edit-submit
alongside the parse check — enforcement is a one-line add later, not a retrofit.
6 tests: types-of = declared + all subtype-of supertypes; is-a? transitive
through subtype-of; is-a alone does NOT chain; instance of tag is transitively a
type; type-valid vacuous with no schemas. 255/255.
Verified live: /type/ + /tag/ render as posts, tag subtype-of type survived a
recreate (durable), ocaml is-a tag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The lone js opt-in-JIT residual was async/await_in_loop, which failed to PARSE
under JIT ("Unexpected token: op '<'" on `i < 5`) while passing on CEK. The js
exclusion was "js-*", but the recursive-descent parser is the jp-* namespace
(75 functions in lib/js/parser.sx) — only the lexer/transpile/runtime are js-*.
So the parser was left JIT-eligible and a jp-* function miscompiled this
construct (the long-standing parser-miscompile class).
Fix: extend the js exclusion to "js-* jp-*" so the parser is interpret-only too,
matching how every other guest's front-end is handled. js conformance under
SX_SERVING_JIT=1 is now 148/148, == CEK.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sessions were in-memory, so a restart logged everyone out (same class as the
relation wipe). Move them to the durable store, but LAZILY so anonymous/crawler
traffic doesn't spam it: session/create mints a sid with no row; the row appears
on the first session/set (a login). A per-boot epoch (one durable write at
startup, host/session-init!) keeps sids unique across restarts without a write
per request.
- lib/host/session.sx: lazy backend (create = no row, set = create row,
exists = row written) + epoch/in-memory-counter sid generation.
- serve.sh: point the session store at the durable backend + host/session-init!.
- blog.sx: host/current-principal is now a durable read, so host/auth-footer
(home + post footers) had to move OUT of the quasiquote into let bindings —
a perform during page-tree build raises VmSuspended (the whole site 500'd for
a beat). Principal computed once per page.
- 2 session tests: create writes no row, set creates the row.
249/249. Verified live: site renders (anon + authed), login + footer survive a
container force-recreate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 6 common-lisp opt-in-JIT failures were all condition-system continuation
escape: cl-restart-case/cl-handler-case/cl-handler-bind wrap their body in
call/cc (restarts + non-local handler exit). When an SX function that drives
the condition system (the parse-recover / interactive-debugger fixtures, e.g.
parse-numbers, make-policy-debugger) is JIT-compiled, the call/cc form runs in
a NESTED cek-run where invoking the captured continuation
runs-to-completion-and-returns instead of escaping — so a restart fails to
abort and the body falls through. Observed as result accumulation
(got (1 3 0 3) vs (1 3)) and no-abort (restart returns the 999 sentinel).
These callers are arbitrary user/fixture code, not a fixed namespace, so they
can't be prefix-excluded. New data-driven mechanism:
- jit-exclude-callers-of! registers call/cc-establishing form names in
Sx_types.jit_excluded_caller_names.
- jit_compile_lambda skips any function whose constant pool (recursively,
incl. nested closures) references a registered name — code_refs_escaping_caller.
Guarded by Hashtbl.length > 0 so it's a no-op for every guest that doesn't
register (zero effect outside CL).
- lib/common-lisp/runtime.sx registers the establish side (cl-restart-case,
cl-handler-case, cl-handler-bind) and the invoke side (cl-invoke-restart,
cl-invoke-debugger, cl-signal, cl-error-with-debugger).
Result: CL conformance under SX_SERVING_JIT=1 = 487/0, EXACTLY matching the CEK
baseline (was 484/6 with a +3 double-execution over-count). parse-recover
3/4 -> 6/0, interactive-debugger 7/2 -> 7/0.
Note: the geometry/mop-trace suites report 0/0 on BOTH CEK and JIT — they error
"Undefined symbol: refl-class-chain-depth-with" (the CLOS suites don't preload
lib/guest/reflective/class-chain.sx). Pre-existing conformance-harness gap, not
a JIT issue; left as-is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
lib/relations holds the graph in memory only (a Datalog cache), so related/tags/
types were wiped on every restart while the posts (durable KV) survived — fatal
for a model where tags and types ARE relations. Make the host the durable source
of truth.
- every physical edge is also a KV row "edge:<src>|<kind>|<dst>" in the blog
store (host/blog--add-edge!/--del-edge! wrap relations/relate+unrelate with
kv-put/kv-delete). '|' is safe: slugs are [a-z0-9-], kinds are registry names.
- host/blog-load-edges! rebuilds the in-memory graph from edge:* keys; serve.sh
calls it on boot right after pointing the store at the durable backend.
- lib/relations stays an in-memory cache; the durable KV is the source of truth
(same shape as the blog pointing at the durable backend).
3 tests: KV row written on relate, replay rebuilds the graph after an in-memory
wipe (restart sim), unrelate deletes the row. 247/247.
Verified live: related welcome<->hello, force-recreated the container (wipes the
in-memory graph), the relation + its rendered block survived the restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Plan: plans/typed-posts-and-relations.md. "Typing is just relating to a type",
types are posts. Phase 1 lifts the hard-coded kind:"related" into a parameter,
driven by one registry — the spine the later phases (type resolution, tags,
picker) build on. Zero user-visible change.
- host/blog-rel-kinds registry: {kind,label,symmetric,candidates[,inverse-label]}
for related (symmetric) / is-a / tagged (directed). One place knows each kind's
direction, label, and candidate set.
- host/blog-relate!/unrelate! take a kind; symmetric kinds write both directions,
directed kinds write one. host/blog-out/in read children/parents per kind;
host/blog-related = out(slug,"related") (back-compat).
- relate/unrelate routes carry a `kind` form field (default "related"), validated
against the registry. delete drops edges across ALL kinds + both directions.
6 tests: symmetric reads both sides, directed writes one (inverse via host/blog-in),
unrelate is kind-scoped, unknown kind rejected, default kind = related. 244/244;
Playwright picker 4/4 (related path unchanged).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Wire a browser check for the picker, run it against an ephemeral host server,
and fix the two real bugs it surfaced.
- lib/host/playwright/relate-picker.spec.js — drives login-redirect-return,
JS candidate load + infinite scroll, debounced filter, and click-to-relate
(asserting the relation shows on the post page).
- lib/host/playwright/run-picker-check.sh — spins up an ephemeral host server
(this worktree's binary + lib, temp persist), seeds a host post + 25
candidates, runs the spec in the main worktree's Playwright/chromium, tears
everything down. No live-site dependency, no live-data pollution. 4/4 pass.
Bugs the check caught:
1. Query params weren't %-decoded — dream's form parser decodes but its query
parser doesn't, so a filter "Item 13" arrived as "Item%2013" and matched
nothing. Fix: decode q with dream's own dr/url-decode in host/blog-relate-
options. (+ conformance test for a spaced filter.)
2. A filter typed while a load was in flight got dropped (busy guard returned
with no trailing fetch). Fix: a `pending` flag re-runs the load when the
in-flight one finishes, coalescing to the latest query.
239/239 conformance; JS node --check clean. Verified live: spaced filter
returns matches; served JS carries the pending-reload fix.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make relating discoverable and pleasant: a hint on posts with no relations, and
a real candidate picker on the edit page.
- post page: when a post has no relations AND the viewer is logged in, show a
subtle "No related posts yet — add some" hint linking to the edit page;
anonymous viewers still see nothing.
- GET /<slug>/relate-options?q=&offset= — SX endpoint returning one page of
candidate rows (HTML <li> fragment): every post except itself and ones already
related, narrowed by q (case-insensitive title/slug substring), title-sorted,
paginated by host/blog--picker-limit. Public read; the relate POST stays
guarded.
- GET /relate-picker.js — small vanilla glue (debounced live filter +
scroll-to-load-more) served from a route. The host serves static HTML (no SX
island hydration), so the interactive layer is a cached script, not an island;
data-slug on the input carries the post to it.
- edit page: the plain "slug to relate" box becomes a filter input + scrollable
results list (#relate-filter/#relate-results) populated by the script; each row
is a one-click relate form.
8 tests: endpoint lists/excludes-self/filters-by-q/excludes-already-related, JS
route content-type + glue, hint shown logged-in / hidden anonymous. 238/238.
Verified live: hint (logged-in only), candidate rows, q=filter, JS route
(node --check OK), edit picker UI with data-slug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compose two already-migrated domains: a post is a relations-graph node
"blog:<slug>", and a "related" link is a symmetric pair of edges
(lib/relations). The post page shows a "Related posts" block; the edit page
gets an editor to add (by slug) and remove relations.
- host/blog-relate!/unrelate!/related: symmetric edges under kind "related";
related slugs = blog children, existence-filtered against ONE kv-keys read.
- post page: "Related posts" links block; edit page: related editor (remove
buttons + add-by-slug box).
- POST /:slug/relate, /:slug/unrelate — guarded browser routes (redirect to
login like the other write routes); relate validates the other post exists.
- delete cleans up a post's related edges (no dangling links).
IO ORDERING (the live 500 that conformance missed): host/blog--related-block/
-editor do durable reads (perform). Performing inside the quasiquote, via
unquote, while the page tree renders raised Sx_vm.VmSuspended under http-listen;
the in-memory conformance store never performs, so it passed. Fix mirrors
host/blog-home: do the reads in the handler's let bindings BEFORE the
quasiquote, and check related-existence against a single host/blog-slugs read
rather than a perform per candidate inside filter.
9 relate tests (guard, symmetry, render, no-op on missing, unrelate both ways,
delete cleanup). Verified live: relate -> Related block both ways; unrelate
clears it; posts without relations and the whole site stay 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Login had no visible entry point — you could only reach it by hitting a guard.
Add an auth footer the pages splice in: "log in" when logged out, "signed in
as <user> · log out" when logged in.
- host/auth-footer: SX fragment reading the session principal; guards a
session-less request so it's safe to call anywhere.
- GET /logout added alongside POST so the footer link is a plain <a> (logout
is low-harm; GET is acceptable). Clears the session, redirects home.
- home and post pages splice (host/auth-footer req) into their footer.
Tests: home + post footers show a login link when anonymous; GET /logout ->
303. 221/221. Verified live: anonymous shows "log in"; logged in shows
"signed in as admin · log out"; /logout reverts it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Clicking "edit" while logged out returned a raw JSON 401
{"ok":false,"error":"unauthorized"} — a dead end in the browser. HTML routes
now redirect to a usable login page and return you afterwards.
- host/require-login: browser-shaped guard. Same session-or-bearer check as
host/require-user, but on failure REDIRECTS to /login?next=<path> instead of
JSON 401. (host/require-user stays for JSON/API routes.)
- host/-principal-of: shared session-then-bearer resolution.
- login honours ?next=: GET /login renders a hidden next field; POST /login
redirects there on success and re-renders the form (with next) on failure.
- host/-safe-next: only same-site absolute paths are honoured — //evil.com and
http://… fall back to "/", closing the open-redirect.
- blog: host/blog--protect-html (require-login) guards the browser routes —
POST /new, GET/POST /:slug/edit; the JSON /posts routes keep host/require-user.
Do we need login? Yes — it's the write/edit auth boundary; without it anyone
could edit or delete posts. The bug was the dead-end 401, not the gate. Now
logged-out edit -> login -> back to edit is a clean flow.
Tests: blog no-auth write routes assert 303 + Location /login(+next); session
suite gains next round-trip + open-redirect-guard cases. 218/218.
Verified live: /welcome/edit logged out -> 303 /login?next=/welcome/edit;
login -> 303 back to /welcome/edit -> 200.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Posts ARE SX source, so expose it: a public raw-source view and a guarded
in-browser source editor.
- GET /<slug>/source — raw sx_content as text/plain (public; a published
post's source isn't secret).
- GET /<slug>/edit — edit form pre-filled with the post's title, raw source
(in a textarea, render-to-html-escaped so it shows verbatim), and status
(current value pre-selected). Guarded (editor only). Slug is preserved.
- POST /<slug>/edit — save the edited source; same write-time validation as
create (unparseable body -> 400, post left intact); 303 back to the post.
- post page gains "view source · edit · all posts" footer links.
Routing: /:slug/source + /:slug/edit are two-segment patterns; the router
consumes :param as exactly one segment and requires a full match, so /:slug
does not shadow them (asserted). 14 new blog tests cover view (200/text-plain/
raw body/404/no-shadow) and edit (401 unauth GET+POST, 200 form, source shown,
303 save, persisted, slug preserved, 400 malformed, 404 missing).
Verified live on blog.rose-ash.com: view source, guarded edit form, save
round-trip (rendered post + source both reflect the edit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Complete the malformed-post defence: instead of only degrading on read,
refuse to store a post whose body won't parse, so bad content never enters
the durable store in the first place.
- host/blog-content-ok?: empty body is allowed, otherwise it must parse
(parse-safe non-nil).
- POST /new (form): missing title OR unparseable body -> 400 HTML page.
- POST /posts (JSON): unparseable sx_content -> 400 "invalid sx_content".
- PUT /posts/:slug (JSON): unparseable sx_content -> 400, existing post left
intact.
- 6 new blog tests: each write path rejects "<h1 broken)" with 400 and does
not store / does not mutate.
Verified live: malformed publish -> 400 + slug 404 (not stored); valid
publish unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A post whose sx_content is malformed SX (e.g. "<h1 ...)" — a typo'd paren)
made GET /<slug>/ return 502, surfaced as a Cloudflare error page. Root
cause: the kernel `parse` raises a native Parse_error that an SX (guard ...)
cannot catch (guard only traps SX conditions), so host/blog-render's guard
around (parse sx) was ineffective; the exception escaped to the http-listen
loop, which swallowed it and wrote NO response — a dropped connection that
Caddy/Cloudflare relay as 502.
- kernel: add `parse-safe` — like parse but returns nil on malformed input
(value-returning, so untrusted text can be handled without a host exception).
- kernel: http-listen now synthesises a 500 response on ANY handler exception
instead of dropping the connection, so the origin stays responsive (no more
proxy 502 / branded error page) and the error is logged. This is also the
only place a native exception can be trapped, since SX guard can't.
- blog: host/blog-render uses (parse-safe sx) — malformed bodies render the
existing "(unparseable content)" placeholder; the per-block render guard
already covers unknown components (~kg-*), so /mddddd/ recovers too.
Verified live: /try-thus/ and /mddddd/ now 200 with placeholders; working
posts, home, and login unaffected. 193/193 conformance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unblock the guarded blog write routes for browsers: a login form sets a
signed session cookie that the same routes accept (alongside Bearer), so
publishing works end-to-end on blog.rose-ash.com without Quart.
- kernel: http-listen emit serialises a response :set-cookies LIST as one
Set-Cookie header each (a headers dict can't hold more than one). Purely
additive — responses without :set-cookies are unchanged.
- server.sx: host/-dream->native forwards :set-cookies to the native resp.
- lib/host/session.sx: durable, signed sessions on the persist KV
(session/create|exists|get|set|clear), wired via dream-sessions-signed.
- lib/host/auth.sx: GET/POST /login + POST /logout; host/require-user accepts
a session principal OR a Bearer token.
- router.sx: host/make-app wraps the whole app in the session middleware and
auto-mounts /login + /logout — the front door always has sessions.
- blog.sx: write routes use host/require-user; serve.sh flips POST /new from
the experimental UNGUARDED route to the guarded write routes, with admin
creds + signing secret + ACL grant from the container env.
- session conformance suite (12): login->cookie->guarded write 201; no
cookie/forged/logged-out -> 401; Bearer fallback still works.
Verified live on blog.rose-ash.com: 401 unauthenticated, 303 login, 303
publish, anonymous read renders, post persists across container recreate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Enabling the epoch serving-mode JIT globally regressed continuation-based guest
interpreters (the epoch mode is the shared command channel every loop's
conformance runner uses). Two-part fix:
1. SAFE DEFAULT GATE. register_jit_hook in the persistent server branch is now
opt-in via SX_SERVING_JIT=1 (default OFF). Default behaviour is unchanged
(no JIT in epoch serving) → zero regression for sibling loops. The
content/Smalltalk page server opts in.
2. GENERAL FIXES + per-guest interpret-only declarations:
- callable? (sx_server/run_tests/integration_tests/mcp_tree) now accepts
VmClosure. A JIT-compiled higher-order function returns its inner closure
as a VmClosure; callable? previously rejected it, so scheme-apply's
(callable? proc) guard failed with "not a procedure: <vm:anon>".
- jit-exclude! gains a trailing-"*" namespace-prefix form
(Sx_types.jit_excluded_prefixes), the robust way to mark a whole guest
interpreter interpret-only (a name-list misses functions in extra files —
it left erlang's vm/dispatcher JIT'd and 13 tests short).
- Per-guest exclusions in each guest's runtime.sx:
scheme "scheme-*" "scm-*" erlang "er-*" "erlang-*"
prolog "pl-*" common-lisp "cl-*" "clos-*"
js "js-*" haskell "hk-*"
Verified under opt-in JIT (== CEK, no hang): smalltalk 847/847, scheme/flow
166/166, erlang 530/530, prolog 590/590, apl 152/152, js 147/148. Residual
(documented, protected by the default gate): common-lisp 6 fails in advanced
suites (parser-recovery/debugger/CLOS/MOP). lua (0/16) and tcl (3/4) fail
identically on CEK — pre-existing, not JIT. run_tests --jit/no-jit unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The handler runs the dynamic logic in the full evaluator and builds a static SX
element tree via quasiquote; render-page (5.1) renders it. No aser pipeline
needed for server-rendered pages. host/blog--page is now an (html (head..)(body..))
tree; home builds the posts <ul> via map+quasiquote; the post body is rendered
per-block then injected with (raw! ...); /new is an SX form tree. Only the
doctype prefix remains as a string (render-to-html doesn't emit it). 181/181.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull out the debt that revived the legacy editor: removed kg-compat.sx (uncommitted
bare->namespaced kg-card aliases), the ./blog container mount, the legacy
sx-editor.js + hardcoded asset URLs + ~editor/sx-editor-styles reuse at /new, and
the blog/sx preloads. /new is now a clean minimal form.
Finding that reshapes Phase 5: render-page (5.1) renders STATIC component trees
but is NOT the full evaluator — a component with a data loop ((map fn items) over
(unquote data)) errors 'Not callable: nil'. So clean dynamic component pages + a
native island editor need the aser SSR pipeline (5.2), not just render-page.
Posts still render via per-block guarded render-page; unsupported editor cards
(~kg-md) show placeholders by design (no alias shim). All endpoints 200, boot clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
register_jit_hook is now installed in the persistent (epoch) serving-mode
branch of sx_server.ml, not just --http/cli/site. Smalltalk-on-SX conformance
under JIT is 847/847 — identical to the no-JIT baseline; Datalog 356/356.
run_tests --jit/no-jit are byte-identical before/after (no regression).
Five distinct root causes fixed (not one "miscompile"):
1. Serving mode never loaded lib/compiler.sx, so JIT used the native
Sx_compiler.compile stub (arity-0 bytecode, params as GLOBAL_GET →
"VM undefined: <param>"). Server-mode branch now loads compiler.sx
before registering the hook, matching http/cli/site.
2. compile-cond / compile-case-clauses / compile-guard-clauses only treated
keyword :else and true as the catch-all, not the bare symbol `else` that
the CEK's is-else-clause? accepts → GLOBAL_GET "else". (lib/compiler.sx)
3. OP_DIV produced a float for non-divisible Integer/Integer (1/2 → 0.5)
instead of the exact Rational the "/" primitive returns. Now delegates to
the primitive, matching CEK. (sx_vm.ml)
4. OP_EQ / _fast_eq lacked Rational/ListRef cases that the "=" primitive's
safe_eq has → (= 1/2 1/2) false under JIT. OP_EQ now delegates non-scalars
to the "=" primitive; _fast_eq gained rational + ListRef. (sx_vm.ml,
sx_runtime.ml)
5. Continuation-based control flow (Smalltalk ^expr non-local return, block
escape, exceptions via call/cc) can't run in the stack VM. New data-driven
exclusion set Sx_types.jit_excluded + `jit-exclude!` primitive, consulted in
jit_compile_lambda (covers both the CEK hook and vm_call's tiered path).
lib/smalltalk/eval.sx self-declares its continuation dispatch core
interpret-only; pure helpers still JIT. The SUnit suite-runner test helper
pharo-test-class miscompiles mid-loop and is excluded in tests/tokenize.sx.
Also adds SX_JIT_DENY / SX_JIT_ONLY env-var bisection filters to the serving
hook. Known residual documented in plans/jit-bytecode-correctness.md: the hook
re-runs a failed VM execution via CEK (correct result, possible duplicate side
effects); adopting run_tests' propagate-don't-rerun semantics is deferred to
avoid changing shared VM/CEK behavior under this loop.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A post created with the editor stored sx_content containing components the host
can't resolve: the legacy editor emits bare ~kg-md while the cards are
~kg_cards/kg-md (drift — not papered over with aliases). render-to-html threw on
the undefined symbol and host/blog-render had no error handling -> handler crash
-> 502 on a REAL post (/mddddd/).
Fix: render each block of the (<> ...) fragment under its own guard via
render-page (env-supplied). Real prose (p/h1/ul/...) renders; an unsupported or
malformed block degrades to a <div class=blk-unsupported> placeholder; a bad
block never crashes the handler. Verified live: /mddddd/ + all junk posts now
200 (text shown, cards placeheld). Full kg-card rendering = follow-on (resolve
the name drift / native editor).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The editor was unstyled: editor.css is .koenig-lexical-scoped (the OTHER editor);
the sx-editor's .sx-* styles live in the ~editor/sx-editor-styles component
(inline <style> in blog/sx/editor.sx). Inline them into /new by rendering that
component with the 5.1 render-page primitive (dogfooding the capability live), +
FontAwesome for the +/slash-menu icons. 79 .sx- rules now inlined.
Also: the sx_host container only mounted spec+lib, so web/adapter-html.sx (and
now blog/sx/{layouts,editor}.sx) silently failed to load at boot -> render-page
errored -> /new 502. Mount ./web + ./blog (ro) so they load. (Transitional reuse
of the legacy blog editor component + its styles; retire via the asset-manifest +
native SX-island editor.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the plain textarea at GET /new with the real Ghost/Koenig-style block
editor (shared/static/scripts/sx-editor.js): a #sx-editor mount point + hidden
sx_content field + title + status; on submit getSx() fills sx_content and POSTs
to /new (the proven ingest). Assets (sx-browser.js, sx-editor.js, editor.css)
referenced from the docs static host (sx.rose-ash.com/static/scripts) — STOPGAP
hardcoded URLs pending an asset-manifest (Phase 5.2) and a native SX-island
editor. SxEditor.mount({}) is safe (all opts guarded); getSx() needs no SX
runtime. Wiring + assets + mount-safety validated; browser mount needs visual
check (no Playwright in this worktree).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KERNEL: add a render-page primitive (sx_server.ml, persistent mode) that renders
an UNEVALUATED SX expression with the server env via sx_render_to_html.
render-to-html expands defcomp components and collects keyword attrs itself; SX
handlers can't reach the server env, so the prim supplies it. Fixes the attr
mangling — bare render-to-html on an EVALUATED component tree turns (form :id ..)
into <form>idpost-new-form..; rendering the unevaluated expr keeps :id an attr.
HOST: lib/host/page.sx — host/page (expr -> HTML response) + host/page-route
(mount on a GET path). New page suite (8 tests) proves a generic attributed +
nested component renders correctly through a host route; verified ~editor/form
renders right too. This is the component-render step of the generic
interactive-SX-page capability; shell + static assets + hydration (5.2-5.4) next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
host/blog-open-create-routes mounts POST /new with error-trapping but NO auth
(create-only; no PUT/DELETE), so the SX editor can publish to the host
end-to-end on the experimental subdomain. VALIDATED LIVE: editor-style
form-urlencoded POST -> 303 -> post renders at /<slug>/ and lists on /.
Deliberate short-lived public write hole (create-only, obscure subdomain).
MUST be gated before real use: Caddy basicauth on /new, or session auth.
Swap host/blog-open-create-routes -> host/blog-write-routes <resolver> to gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pivot blog to the SX editor's content model. The editor (blog/sx/editor.sx)
emits sx_content = SX element markup, NOT content-on-sx CtDoc blocks. So a post
is now a {slug,title,sx_content,status} record in the durable persist KV, and a
post page is render-to-html(parse sx_content) — server-side, static, no client
runtime needed to view.
Endpoints: GET / (HTML index), /<slug>/ (rendered post), /posts (JSON list),
/new (create form); POST /new (form-urlencoded editor ingest, slug from title,
303 redirect), POST /posts (JSON create), PUT/DELETE /posts/<slug>. Writes
behind auth+ACL (edit/blog). Dropped the content-on-sx/Smalltalk preload chain;
added spec/render + web/adapter-html (render-to-html) + lib/dream/form.
BONUS: render-to-html is ~0ms (vs the 2s content-on-sx Smalltalk asHTML) — it
doesn't hit the JIT-miscompiled path, so blog rendering is no longer slow.
Live: blog.rose-ash.com/ lists posts, /welcome/ renders instantly. Reads live;
the form-ingest write path needs an auth decision before going live (browser
forms can't send bearer; needs session or a Caddy basicauth gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GET / renders an HTML index listing every post (title linking to /<slug>/),
built from host/blog-list; empty -> 'No posts yet'. GET /posts stays the JSON
API. Live: blog.rose-ash.com/ lists the welcome post linking to /welcome/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CRUD on the durable content store, per-request IO:
GET /posts list (public) -> [{slug,title}]
GET /<slug>/ read (public) -> HTML / 404
POST /posts create (auth+ACL edit/blog) -> 201/400/409
PUT /posts/<slug> update title+body -> 200/400/404
DELETE /posts/<slug> delete (truncate) -> 200/404
Writes behind the auth+ACL pipeline; create=insert ops, update=op-updates,
delete=stream truncate. 16 new CRUD tests (full lifecycle + 401/403/409/404).
GOTCHA fixed: is a reserved CEK special form — a (let ((guard ...)))
helper was shadowed by it ((guard h) ran the guard special form -> 'first:
expected list'). Renamed to host/blog--protect; namespace-prefix all helpers.
HARDENING: conformance.sh now FAILS LOUD on load/eval errors. A test file that
errors mid-load silently truncates its suite and reports a false green (this hid
the CRUD failure as 'blog 13 passed, 0 failed'). The runner greps for error
markers and aborts. Documented the SX gotcha set + prevention ladder in the plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KERNEL (sx_server.ml): route http-listen handlers through cek_run_with_io
instead of bare Sx_runtime.sx_call, so handlers resolve per-request IO
(durable persist reads/writes) via the same IO-driving runner the REPL uses.
Verified: per-request read+write, 10 concurrent writes (15 on disk, no
corruption), handler errors don't crash the server, http contract 6/6.
BLOG: fully dynamic — host/blog-post reads the post from the durable store
(content/head) AND renders (content/html) per request, no in-memory view, no
cached output. Possible because of the IO fix. Honest ~2s due to interpreted
Smalltalk render.
Render speed is NOT solved here: the JIT (precompiler) isn't installed in the
serving mode and currently miscompiles the Smalltalk evaluator's nested ASTs
(enabling it breaks ~60% of tests). Fixing the JIT is a separate, high-payoff
effort. Documented in the plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Blog posts now live in the durable SX store (persist/durable-backend, on-disk
under $SX_PERSIST_DIR — already built: sx_persist_store.ml + lib/persist/
durable.sx). Publishing appends insert ops to the slug's content stream; posts
survive restarts (verified: seq/log stable across container restart, re-seed
idempotent).
Read path: http-listen handlers can't drive per-request perform/IO (sx_call
doesn't resolve the CEK IO suspension the way the main loop does), so posts are
materialised from the store into an in-memory view at boot (host/blog-load-all!
+ host/blog-seed!) and request handlers read the view — perform-free. Store is
source of truth; view is a boot-rebuilt cache.
Deploy: docker-compose.dev-sx-host.yml mounts /root/sx-host-persist (chowned to
appuser 10001) at /data/persist; SX_PERSIST_DIR set. blog.rose-ash.com/welcome/
live. Per-request-IO kernel fix tracked in the plan as the next task.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/blog.sx serves blog posts as HTML at GET /<slug>/ (the original
strangler target, Quart blog post_detail). A post is a content-on-sx CtDoc
rendered via content/html; anonymous + world-visible. In-memory slug->doc
registry now (host/blog-lookup swappable for a persist-backed content stream
later, handler/route unchanged). :slug catch-all mounted LAST so /feed,
/health, /internal/* take precedence. Needs the Smalltalk+persist+content
preload chain + (st-bootstrap-classes!)+(content/bootstrap!) — blog.sx
self-bootstraps at load. serve.sh loads the chain + seeds a welcome post.
Ledger gains the migrated blog post-detail (off-Quart 50% -> 53%).
LIVE: blog.rose-ash.com/welcome/ renders real HTML through Cloudflare->Caddy;
/feed still JSON (precedence verified), unknown slug 404.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote lib/host into the docker stack behind blog.rose-ash.com (reusing a
down Quart subdomain). New compose service sx_host runs lib/host/serve.sh on
externalnet; Caddy reverse-proxies blog.rose-ash.com -> sx-dev-sx_host-1:8000.
hosts/ fix: http-listen bound inet_addr_loopback only, unreachable from other
containers. Add SX_HTTP_HOST env (default loopback for tests/local; stack sets
0.0.0.0) in sx_server.ml. serve.sh made container-friendly (SX_PROJECT_DIR).
Verified live through Cloudflare->Caddy: /health, /feed, relations reads serve
real JSON; / 404 (no root route yet). rose-ash.com untouched. Conformance
145/145 green with the rebuilt binary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/server.sx adapts the native http-listen contract (string-keyed
{method,path,query,headers,body} -> {:status :headers :body}) to the Dream
host app: native->dream reassembles path+query into a target dream-request
parses; dream->native is near-identity (dream-response is already
{:body :headers :status}). host/serve = http-listen over host/native-handler
. host/make-app. lib/host/serve.sh boots the full module set and serves in the
foreground (container-entry shaped). Verified live on a host port: health/feed/
feed?actor=/relations reads serve real JSON, unknown->404. server suite (13)
covers the bridge as pure functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate the container relations write actions onto lib/relations: POST
/internal/actions/attach-child + /detach-child dispatch to relations/relate
and relations/unrelate over the same "type:id" node model, behind the
auth+ACL pipeline (wrap-errors . require-auth . require-permission), mirroring
POST /feed. Closed-loop test: attach -> visible via get-children -> detach ->
gone; 401/403/400 guards. Ledger now models the full relations surface (7
endpoints): container reads+writes migrated, typed relate/unrelate/can-relate
proxied (registry+cardinality validation not in lib/relations). Off-Quart
coverage 45% -> 50%.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate the two internal relations read queries onto lib/relations: GET
/internal/data/get-children + /get-parents dispatch to relations/children
and relations/parents. Bridge the Quart (type,id) node key to a graph atom
symbol "type:id" with relation-type as the edge kind; optional child/parent
-type params filter by "type:" prefix. Golden tests pin each endpoint to
subsystem-call + envelope. Ledger entries flipped to :migrated (off-Quart
coverage 27% -> 45%).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/sxtp.sx implements the host<->subsystem wire format per
applications/sxtp/spec.sx:
- message algebra (request/response/condition/event + status helpers
ok/created/not-found/forbidden/invalid/fail) as string-keyed dicts;
verb/status/type stored as symbols (ride the wire bare)
- codec: sxtp/serialize (dict -> text/sx list form, deterministic top-level
field order, nested messages emitted in their own list form, no :msg leak)
and sxtp/parse (text/sx -> dict via a deep keyword-token->string normaliser)
- Dream bridge: sxtp/from-dream (HTTP req -> SXTP req, method->verb,
query->params) and sxtp/to-dream (SXTP resp -> HTTP resp, status->code,
body serialised to text/sx)
- 39-test suite covering algebra, serialise/parse round-trip, mappings, bridge
Runtime notes: serialize renders string-keyed dicts as {:k v} and symbols
bare; parsed keyword tokens are a distinct type (not = to string literals) so
parse normalises; unquote-splicing is unreliable so the emitter is str-based.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Composable handler->handler layers over Dream's primitives, with auth and
permission POLICY injected so the layer is policy-free and testable:
- middleware.sx: host/wrap-errors (JSON 500 via dream-catch-with),
host/require-auth (bearer->principal via dream-bearer-token, JSON 401,
injected token resolver), host/require-permission (lib/acl acl/permit? gate,
JSON 403, injected resource extractor), host/pipeline (first = outermost)
- feed.sx: POST /feed via host/feed-write-routes — auth ∘ ACL(post,feed) ∘
wrap-errors over host/feed-create (parse JSON body -> feed/post -> 201;
non-object -> 400). Created activity reads back via GET /feed.
- middleware suite (9) + feed write tests (6 new); conformance preloads now
include the Datalog engine + ACL subsystem + Dream auth/error.
ACL works with string atoms (no symbol coercion). Mute/prefs layer and sxtp.sx
deferred to the next tick.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First migrated endpoint onto the SX host. lib/host is a thin wiring layer:
a host handler is a Dream handler (request->response) that calls a subsystem
public API and serialises via a shared JSON envelope.
- handler.sx: host/ok, host/ok-status, host/error, host/json-status (Dream's
dream-json is 200-only), host/query-int
- router.sx: host/make-app assembles per-domain route groups + /health probe
into one dream-router (reuses dr/flatten-routes)
- feed.sx: GET /feed reads feed/all + stream combinators, recent-first, with
?actor= filter and ?limit= cap
- 3 test suites incl. a golden test (body == subsystem recent stream + envelope)
- conformance.sh mirrors lib/dream's runner
Builds on dream-on-sx (merged, gate green 480/480) rather than a throwaway
native request model; collapses most of plan Phase 4 into Phase 1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
