The full fed-sx production layer, live-verified across A (blog.rose-ash.com) and B (sx_host_b).
ACTOR MODEL + FOLLOWER GRAPH: activities carry a real :actor (SX_ACTOR); delivery targets FOLLOWERS,
not a static peer list. A peer subscribes by POSTing {verb:follow, actor, base} to /inbox
(host/blog--add-follower!); B follows A at boot (SX_FOLLOW) so A delivers to B. host/blog--{actor,
self-base, followers, follow!, delivery-bases} + durable followers store.
BACKGROUND DELIVERY TIMER: serve.sh's detached _fed_delivery_loop hits GET /fed-tick every 15s
(over /dev/tcp) → re-follow (idempotent, recovers a target that was down at boot) + flush the durable
outbox. Federation is eventually-consistent, not best-effort-at-emit.
SIGNATURE VERIFICATION: every federated POST is signed (host/blog--fed-sign = dr/sess-sig shared-secret
MAC over the body, SX_FED_SECRET); /inbox rejects a bad/missing signature with 403 (empty secret =
open). Applies to both follows and activity delivery.
PUBLIC DOMAIN: B joins externalnet so Caddy CAN reverse_proxy a subdomain to it — the DNS + Caddy
route itself is external ops config (no local Caddyfile).
LIVE PROOF: B follows A (followers:1); publish on A → SIGNED delivery to follower B → B verifies +
fires validate+notify; a forged POST (bad x-fed-sig) → 403; B down → publish queues → the background
timer auto-delivers the backlog when B returns (no manual flush). blog 218/218, full conformance green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Step 3 — federation, live-verified with TWO real host instances.
- host/ta.sx: host/ta--post/make-http-wire/federate (POST a serialized activity to a peer's /inbox
over real HTTP). host/blog.sx: POST /inbox (host/blog-inbox → receive! → process locally, does NOT
re-federate — no loops).
- DURABLE OUTBOX (fed-sx reliability, after the user asked 'if B is down does it still work?'):
emit! processes locally (always succeeds), QUEUES per-peer to a persisted outbox, delivers
best-effort. A peer being DOWN no longer fails the publish — delivery is GUARDED (SX guard catches
the http-request connection error), failed items stay queued and retry on next emit / on boot /
manual /flows?flush=1. /flows shows the outbox depth.
- serve.sh: SX_PEERS → peers; boot load+flush of the outbox. docker-compose: a 2nd host sx_host_b
(peer B, own store, no peers).
LIVE PROOF: (1) a peer POSTs create/article to blog.rose-ash.com/inbox → A fires validate+notify.
(2) publish on A → federates to B → B fires ITS behaviors on A's activity (B's /flows + /activities).
(3) RESILIENCE: publish with B DOWN → A returns 303 (was 500) + queues; start B + flush → B receives
the backlog + fires. blog 218/218 (+TA receive test), full host conformance green.
A = blog.rose-ash.com (public/Caddy); B = sx_host_b (internal docker DNS only, no public domain).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Steps 1+2 of RA-live/TA-live, live-verified end-to-end on blog.rose-ash.com.
(1) DEPLOY: docker-compose.dev-sx-host.yml gains an sx_kernel service running next/kernel/serve.sh
(the durable-execution kernel), SX_HTTP_HOST=0.0.0.0 so the host container reaches it at
http://sx_kernel:8930.
(2) HOST AS CLIENT: lib/host/ra.sx gains a KERNEL runner — host/ra--make-kernel-runner drives the
kernel over HTTP (http-request, native primitive; returns {status headers body}). It advertises
{effect,branch,each,suspend}, so select-runner routes a durable DAG to it. host/blog.sx: the DAG
registry + runner fleet are now mutable (register-dag!/add-runner!); emit! records SUSPENSIONS in a
durable pending log; /flows shows suspended instances with a resume link (?resume=<id>) driving
host/ra--kernel-resume. serve.sh wires it: set kernel-base, add the kernel runner, register the
durable 'blog-digest' DAG, declare a DURABLE behavior on article (create→publish SYNC, update→
blog-digest DURABLE), add a 'category' field.
LIVE PROOF: editing a published newsletter article → Update → routes to the kernel runner → POST
/flow/start/newsletter → kernel SUSPENDS (instance 5, shown pending on /flows) → /flows?resume=5 →
host re-drives the kernel → DONE → digest-sent effect + pending cleared. Durable suspend/resume
across separate HTTP requests, on a deployed persistent kernel. urgent edits complete immediately
(digest). http-request works in the serving context. blog 217/217, full conformance green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/ta.sx — a seam transport {:emit :deliver} over a DIRECTIONAL wire (out = outbox→followers,
in = inbox←follows). The transport is the SERIALIZATION boundary: activities cross the wire as
SX-source strings (host/ta--serialize/deserialize map the keyword-keyed activity ↔ a flat
string-keyed wire form of the P2 activity fields). host/ta--make-transport(out-wire, in-wire) +
host/ta--make-mem-wire (an in-memory directional queue for tests).
Proven (ta 5/5): content + relation activities round-trip through the wire; the FEDERATION LOOP —
instance A emits an activity → the wire carries it → instance B's behavior/pump delivers + processes
it → B's engine fires ITS behavior on A's activity; DIRECTIONAL (B re-emits to its own outbox, not
back into the inbox — no loop). 'Everything works over fed-sx', proven at the seam.
TA-live (deferred, same shape as RA-live): swap the mem-wire for the real next/ delivery wire —
needs a PERSISTENT next/ kernel (gen_servers don't survive across erlang-eval-ast calls) + the ACTOR
MODEL (peer_actors/follower_graph decide who the out-wire delivers to) + pushing /activities onto it.
Full host conformance green (+ta 5).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generalizes emission beyond publish to the full event source. TWO ActivityPub-faithful classes:
- CONTENT (host/blog--content-activity): Create on first publish, Update on a subsequent published
edit. object-type is DERIVED from the post's is-a (host/blog--post-type), not hardcoded 'article'.
- RELATION (host/blog--relation-activity): Add/Remove, carrying :relation + :target (the edge).
host/blog--emit! runs any activity through behavior/process (logged + matched). emit-content-change!
(create/update) wired into form-submit + edit-submit; emit-relation! (add/remove) wired into
relate-submit + unrelate-submit.
DEBT #1 FIXED — per-EVENT :id (not the bare CID): content = create:/update:+cid; relation =
add:/remove:+src:kind:dst (EDGE-based, because a relation change doesn't shift the CID, so a
CID-based id would false-dedup different edges on one object).
The activity log is now the DURABLE EVENT SOURCE (string-keyed records under 'activitylog',
boot-loaded), surfaced at /activities — what TA will push to peers.
LIVE PROOF (blog.rose-ash.com): publish → /activities 'create article <cid>'; relate → 'add article
p2-events — add welcome related'; unrelate → 'remove …'. blog 217/217 (+4 P2, reframed P0.3 fire
tests for Update semantics), full host conformance 614/614.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generalizes the hardcoded publish trigger into declared, capability-routed behavior.
- Types carry :behavior — flat string-keyed bindings {"verb" "type" "dag"} on the type-post
(persist-safe, like :type-relations). The "article" type declares on-create → the "publish" DAG.
- host/blog--load-behaviors! gathers ALL posts' declarations into a registry at boot (serve.sh); the
trigger match (host/blog--triggers :match = host/blog--match-behaviors) consults it. Hardcoded
create+article trigger removed.
- Runner DERIVED (DEBT #2 fixed): match resolves :dag via host/blog--dag-registry and picks the
runner via host/flow--select-runner over host/blog--runner-fleet ([exec-runner]; RA joins at
RA-live). Each binding carries its :runner; behavior/-run-binding now uses the binding's runner
(else the engine default) — so the capability model drives the LIVE engine.
- The type-def view shows each behavior + its derived runner (host/blog--behavior-lines).
LIVE PROOF: /article shows 'on create → publish DAG · needs {effect, branch} · runner: synchronous
(exec-fold)'; publishing on blog.rose-ash.com fired /flows validate+notify via the DECLARED path.
blog 213/213 (+3 P1), full host conformance 610/610. FINDING: load-behaviors! scans all posts, not
is-type?-filtered (article failed is-type? on the durable store though it passed in-memory).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/ra.sx — a PURE-SX seam runner (advertises {effect,branch,each,suspend}) with an INJECTED
erl-eval (real = er-to-sx-deep ∘ erlang-eval-ast; mock in unit tests), so it loads in the plain host
(Erlang refs resolve lazily inside lambdas) and is unit-testable without the Erlang runtime.
host/ra--{atom,bin,erl-src,start-expr,resume-expr,parse,make-runner,resume,real-eval}: marshals our
canonical activity → Erlang source (CID as <<"…">> binary, atoms single-quoted), starts a named
next/ flow via flow_store, parses (ok Id (flow_done V))→{:status done :effects V :flow-id} /
(ok Id (flow_suspended T))→{:status suspended :resume {:id :tag}}.
DUAL-RUNNER ROUTING (flows.sx): host/flow--required-caps now handles a {:erl-flow :needs} DAG
(declared caps, since a foreign flow can't be introspected); host/flow--select-runner picks the
cheapest runner whose capabilities cover the DAG's needs. The capability model is now REAL with two
runners — an {effect,branch} composition lands on exec-runner; a {suspend} DAG routes to RA.
Verified: ra 9/9 (mock erl-eval) + plans/ra-integration.sh 4/4 (the REAL module driving live
flow_store: urgent→done, newsletter→suspended with a resume handle, digest_sent effect-as-data).
Full host conformance 607/607; next/tests/triggers_e2e.sh 10/10 baseline intact.
FINDING → RA-LIVE deferred: gen_servers don't persist across separate erlang-eval-ast calls (flow
README), so true cross-call suspend/resume needs a PERSISTENT next/ kernel process. The runner +
marshalling + suspend/resume mechanics are proven; RA-live is process lifecycle + wiring, documented.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The driver now persists each effect record to the blog store (string-keyed to dodge the keyword/
persist top-level split), and host/blog-load-flowlog! rebuilds the in-memory log on boot (wired into
serve.sh after load-edges!). So /flows survives a restart — closing the P0.3 gap.
LIVE PROOF: published a post on blog.rose-ash.com → /flows showed validate+notify → RESTARTED the
container (in-memory log lost) → /flows STILL showed them, reloaded from the durable store.
Round-trip also covered by a conformance test (persist → clear → reload → identical). blog 208/208,
full host conformance 599/599. Note: whole-list rewrite per effect — fine at P0 volume, cap/rotate later.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The hypothesis test. FINDING: a synchronous business flow expresses NATURALLY as an EXECUTE-FOLD
composition (host/execute.sx: seq/effect/alt — the category branch IS 'alt'), NOT an artdag
DATAFLOW DAG (which has no control flow). So 'business logic = art-dag' holds at the ABSTRACTION
(both content-addressed op-DAGs) and is REFINED at the vocabulary: the synchronous control-flow
runner is the execute-fold (caps {effect,branch,each}); artdag is the dataflow sibling. Two
instances of one thing, run very differently — exactly the framing.
lib/host/flows.sx: capability typing (host/flow--node-cap/required-caps derive a DAG's capability
set from its node vocabulary; effect→effect, alt→branch, each→each, wait→suspend), the execute-fold
seam runner (advertises {effect,branch,each}), and host/flow--bind (required ⊆ advertised → derive
the runner, else fail-fast). host/blog--publish-dag (the publish workflow) + publish-ctx.
Verified: publish-DAG required-caps = {effect,branch} → binds to the sync runner; runs →
newsletter→[validate,digest] / urgent→[validate,notify] / other→[validate,skip]; a node →
{suspend} → binds FAIL-FAST against the exec-runner (would need the Erlang runner, RA). Runner is
DERIVED, not chosen. flows 7/7, blog 203/203, full host conformance 591/591.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/behavior.sx — the substrate-independent seam every runner/transport/registry/driver
plugs into. An engine bundles four dict-of-functions adapters (trigger-registry, runner,
transport, driver); behavior/process folds an ACTIVITY through the pipeline: emit → match
triggers → run each behavior DAG → dispatch each effect-as-data → recurse on new activities
(loop closure, depth-guarded at 8). Every stage injected, so the same DAG + engine run over the
synchronous op-table runner / Erlang durable / celery-sx / fed-sx transport unchanged.
Reference tests (mock adapters) prove the contract: publish→trigger→runner→effect flows; a
non-matching activity fires nothing (log complete, execution precise); an effect that emits a new
activity re-triggers (loop closes); an unbounded loop is depth-guarded (terminates). Wired into
conformance.sh + serve.sh MODULES. behavior 4/4; full host conformance 575/575.
Next: P0 supplies the REAL adapters (publish activity ← host/blog--publish-activity, local-SX
trigger, sync op-table runner over a publish-DAG, host driver) — same engine.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dashboard gains a per-route latency bar chart (nested p50/p95/p99 bars, tail
visible) + status-colored waterfall with ms duration labels + a real 3s
auto-refresh (replacing the non-functional data-on-load SSE attr). serve.sh
self-warms the serving JIT over /dev/tcp so the first visitor after a restart
gets ~78ms not the one-time ~2.5s compile. otel suite 125/125.
The otel dashboard route (GET /otel) is single-segment, so blog-routes' /:slug catch-all
shadowed it (404 'no post: otel'); only /otel/stream (two segments) survived. Move otel/routes
ahead of the blog routes. Live-only wiring fix (route order); no test change.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds lib/host/otel.sx to serve.sh MODULES and otel/routes to the host/serve
group list so GET /otel (+ /otel/stream) serve on the live host once merged.
Build-time wiring only; no container touched.
lib/host/htmlsx.sx — a pure-SX HTML → SX converter (char-level tokenizer + stack parser):
host/html->sx turns a post's HTML into an (article …) tree that host/blog--decompose! consumes
— img / p / figure+figcaption / iframe / headings / blockquote / lists, inline strong/em/a kept
nested (decompose flattens to text), entities decoded to UTF-8, comments+doctype skipped. This
replaces the one-off external Python converter used for the nt-live-encore import.
import-post! now accepts a raw "html" field (converted via html->sx, serialized to sx_content,
decomposed) alongside "sx_content" — so importing real Ghost HTML is first-class. Wired
htmlsx.sx into conformance.sh + serve.sh module lists (loads in conformance AND live).
New htmlsx suite 8/8 (text/entities/void/nested/figure/iframe/comments + an html→sx→decompose→
typed-cards round-trip); blog 197/197 (+ import-from-html test).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
host/blog-seed-landing-demo! (+ host/blog--seed-card! fixed-slug helper): a Landing TYPE with
TWO composition fields — :body (heading/text/image + cond/each) and :aside (text/callout, no
controls) — plus a populated landing-demo instance, wired into serve.sh (survives wipes),
idempotent (fixed card slugs, set-comp! overwrites). /landing-demo/ renders both fields; its
edit page shows two independent block editors (#comp-body, #comp-aside); /landing/ reads the
two-field definition. Demonstrates layer 2 end to end on the live site.
blog 196/196 (+ tests: idempotent 2-field seed, both fields render).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
host/blog-seed-nt-live-encore! imports the real post (its HTML-derived sx_content embedded)
via host/blog-import-post!, decomposing it into the :body composition of typed cards; wired
into serve.sh next to the demo seeds. Verified: after a full store wipe + reboot it reseeds
(HTTP 200, 4 images, 3 video embeds, tagged nt-live/films). Idempotent. blog 175/175.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two concrete demonstrations of the composition architecture:
THIRD DOMAIN (proves step 8's "a new domain is just a dict + leaf, no new control flow").
host/comp-deps folds a composition to the object ids it TRANSCLUDES — the static contains
DAG of a body. It reuses host/comp-fold's seq/alt/each dispatch verbatim; only the leaf
(collect `(ref ID)`) + accumulator (concat) are new. Useful in its own right (what a
(seq (ref c0) (each … (ref …))) body pulls in; context-specific — alt picks the taken
branch). compose suite 20/20.
LIVE EXECUTE-FOLD DEMO (makes step 7 tangible, parallel to /compose-demo for render).
/workflow-demo runs ONE composition object's :body through host/exec-run — the SAME structure
the render-fold would turn into HTML, folded by execute into a plan of effects (validate →
branch on status → notify each recipient). host/blog-seed-workflow-demo! + host/blog-workflow-
demo + route + serve.sh seed. Shows the behaviour model IS an execute-fold over a composition
object — the same object the block editor authors. blog suite 165/165.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The keystone validation of the universal-algebra thesis. lib/host/execute.sx is a SECOND
interpreter over the SAME seq/alt/each composition algebra as the render-fold — but a
different fold: leaves are EFFECTS, seq = steps in order, alt+when = branch, each =
for-each, and the accumulator is an effect log instead of an HTML string. It REUSES
compose.sx's shared machinery verbatim — host/comp--pred? (when), host/comp--field
(field/value), host/comp--source (each source) — so the predicate set, context-environment,
and iteration source are domain-agnostic; only the leaf semantics + accumulator are new.
KEYSTONE (tested): ONE (alt (when (has "auth") …) …) skeleton + ONE context folds two ways
— render picks the branch → "<b>in</b>", execute picks the SAME branch → {:verb "enter"}.
A publish workflow (validate → branch-on-status → notify-each) runs as one execute-fold over
a composition object. So the behaviour model (Slice 9) is "an execute-fold over a composition
object", not a separate system — the way the recursive tree proved recursion, this proves the
algebra is domain-agnostic. host/exec-run; 13/13 (new execute suite); wired into conformance
+ serve. Full host conformance 371/373 in 42s (warm); the 2 fails are the pre-existing
relate-picker pair.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A record may carry a :body (a composition node); host/blog-post renders it via the
render-fold (host/comp-render) against a context built from the principal (auth), else the
legacy sx_content path. compose.sx loaded into the host (serve.sh + conformance.sh module
lists). host/blog-body-of / host/blog--set-body!.
Seeded /compose-demo: ONE composition object that shows seq + alt(when auth) + row(par) +
each, and renders DIFFERENTLY by context. Verified live-path (ephemeral SX_SERVING_JIT=1):
anon -> login-prompt (else) + columns + event list; authed -> member block (when auth),
login-prompt gone. The object is the program; the render is the execution -- now live.
Focused eval confirms the in-process render matches the test (ANON<span>..> vs MEMBER<..>).
Tests added; full blog suite still box-contended.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Imported Ghost posts' sx_content holds (~kg_cards/kg-*) (from the lexical_to_sx converter);
the host's render-page resolves components, but the kg-cards weren't loaded so they
degraded to '(unsupported block)' placeholders. Copied blog/sx/kg_cards.sx ->
lib/host/sx/kg-cards.sx (host self-contained, not coupled to the legacy blog/ Quart dir)
+ added the one host-local dep ~rich-text (was only a test fixture) + registered it in
serve.sh + conformance.sh module lists. Verified: the real 'Free DVD Box Sets!' post now
renders <figure class=kg-card kg-image-card> for all images, zero placeholders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
is-a/subtype-of/tagged/related are now POSTS (each is-a a new `relation` root),
owning their metadata in a :rel slot {:symmetric :label :inverse-label}. The static
host/blog-rel-kinds registry is gone: kind-spec/rel-kinds/kind-symmetric? read the
relation-posts (via an in-memory cache), and the relation list derives from
host/blog-in "relation" "is-a".
Perform-budget fixes (a durable read inside the http-listen render VM raises
VmSuspended; too many per request 500s the page):
- relation metadata is loaded into a cache at boot (host/blog-load-rel-kinds!,
like load-edges!), so kind-spec is pure on render paths;
- the initial edit page renders its pickers EMPTY (the load trigger fills each) —
only the relate/unrelate FRAGMENT server-renders candidates (with-cands flag).
Previously every edit page render did candidate-get × 4 pickers and 500'd.
host conformance 287/287 (+4 slice-2: kind-spec reads :rel, kind-symmetric? off the
post, unknown kind has no spec, rel-kinds derived from the graph). run-picker-check
3/3 (edit page boots, relate/unrelate flow works, no client errors).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The declarative picker markup is now a reusable SX component
(lib/host/sx/relate-picker.sx, defcomp ~relate-picker &key slug kind) instead of
inline markup in the editor. It is a CONTENT-ADDRESSED, CLIENT-EXPANDED component:
- Server: on a full page load render-page expands ~relate-picker server-side
(SEO / no-JS), exactly as before.
- Client: on a boosted SPA nav the edit body serialises to the compact
(~relate-picker :slug … :kind …), and the CLIENT expands it. The component
module is compiled to a content-addressed .sxbc, served immutably from
/sx/h/{hash}, and listed in the page's data-sx-manifest "boot" array so the
client eager-loads it after the web stack — registering its defcomp before any
boosted fragment references it.
Wiring:
- lib/host/sx/relate-picker.sx — the component.
- lib/host/blog.sx — editor emits (~relate-picker :slug s :kind k); the inline
form markup is gone.
- lib/host/static.sx — host/static-manifest-json emits boot:["relate-picker.sxbc"]
(the previously-empty boot array, now used as designed).
- hosts/ocaml/browser/sx-platform.js — loadWebStack eager-loads the page manifest's
boot[] modules (content-addressed) after the web stack.
- bundle.sh + compile-modules.js — copy/compile the component to .sxbc.
- serve.sh + conformance.sh — load the component module server-side.
This gives the host an app-component system: app defcomps shipped to the client by
hash, the same machinery as the kernel modules — the picker is the first, and it's
the model for publishing components externally.
Tests: conformance 272/272 (server expansion); relate-picker.spec.js 6/6 incl. the
boosted-nav populate (proves client-side component load + expansion) and the
error/retry case. WASM stack rebuilt (relate-picker.sxbc @ 6818110a).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three composing pieces that make the blog SPA correct and resilient.
Content-addressed module cache (lib/host/static.sx, serve.sh, blog.sx shell,
conformance.sh): index each web-stack .sxbc by the content hash in its head,
serve GET /sx/h/{hash} immutable text/sx, and emit <script data-sx-manifest>
{file->hash} so the WASM client loads modules content-addressed (localStorage +
immutable) instead of path + max-age. serve.sh builds the index at boot;
conformance.sh now loads static.sx before blog.sx (the shell calls
host/static-manifest-json).
Declarative relate picker (lib/host/blog.sx, lib/dream/form.sx): replace the
inline /relate-picker.js blob — which never ran on swapped-in content, so the
candidate list was empty after a boosted nav to /<slug>/edit — with a declarative
SX-htmx form: sx-get relate-options on "load" + debounced "input", innerHTML-swap
the results ul; infinite scroll via a server-emitted "load more" sentinel
(sx-trigger revealed, sx-swap outerHTML) that pages the rest, q preserved via a
new symmetric dr/url-encode. The engine re-binds these triggers on swapped
content, so the picker populates on full load AND boosted SPA nav. Candidate
relate forms get :sx-disable (plain POST->303->reload, their original behavior;
the engine would otherwise boost them and swap the redirect unreliably).
sx-retry "exponential:1000:30000" on the form+sentinel retries a dropped/offline
fetch forever (the cap bounds the interval, not the attempts).
SIGPIPE hardening (hosts/ocaml/bin/sx_server.ml): the native http-listen server
had no SIGPIPE handler, so a client aborting an in-flight fetch (the engine
cancels superseded requests on a debounced filter/fast nav) closed the socket
mid-write and killed the whole process (exit 141). Ignore SIGPIPE so the failed
write becomes a catchable Sys_error the per-connection handler already swallows.
Tests: host conformance 272/272; relate-picker.spec.js 5/5 incl. a boosted-nav
populate regression; spa-check 4/4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Turn the blog into a SPA using the SX-htmx engine (web/engine.sx) booting the
WASM OCaml kernel (same evaluator as the server) in-browser, with sx-boost
fragment-swapping every link into #content.
Server side DONE + verified:
- lib/host/static.sx: GET /static/** serves shared/static via the file-read
primitive (ctype by ext, traversal-guarded, 404 on missing). Wired into
serve.sh (module + route group). Tested: kernel JS + .wasm binary-exact.
- host/blog--page is now the SPA shell: full page = WASM boot scripts +
sx-boost=#content wrapper + #content; on SX-Request:true returns ONLY the
inner content fragment for the engine to swap. All 13 handlers thread req.
- docker-compose mounts ./shared/static.
- lib/host/playwright/spa-check.{spec.js,run-spa-check.sh}: boot/boost/swap/back.
Client side: the WASM kernel BOOTS (SxKernel object, data-sx-ready=true, web
stack loads). BLOCKER: the bundled .sxbc throw 'VM: unknown opcode 0' vs this
worktree's kernel -> .sx source fallback -> boot.sx source fails 'Expected
list, got string' -> process-boosted never binds links (boosted 0/N). Fix =
rebuild a consistent WASM bundle (recompile .sxbc against the kernel via
scripts/sx-build-all.sh); the browser wasm target isn't built here yet. See
plans/host-spa.md. Live NOT redeployed (stays on pre-SPA process).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The serving-JIT perform-in-HO-callback miscompile (map/rest/drop wrong
CALL_PRIM args → blank pages, empty picker) is now fully fixed, so the host
runs 100% serving JIT with NO jit-exclude.
sx-vm-extensions 81177d0e resolves a suspended HO-callback's IO inline
(instead of unwinding the native map/filter loop and corrupting the stack),
but ONLY when a synchronous resolver is installed (!_cek_io_resolver = Some).
The host serves via the http-listen primitive, whose handler drove durable IO
through cek_run_with_io with the resolver = None — so it hit the unwinding
path the fix doesn't cover. (The vm-ext repro installed a resolver, so it
never exercised the host's real no-resolver path.)
Fix: extract cek_run_with_io's IO resolution into resolve_io_request, and have
http-listen install _cek_io_resolver := Some (fun req _ -> resolve_io_request
req) — byte-identical resolution, so the inline path resolves durable reads
exactly as the CEK loop would.
Verified: host conformance 271/271; ephemeral durable server at 100% JIT (no
exclude) zero fallbacks + real content + related shown + picker 12 candidates;
live blog.rose-ash.com home/post/tags 200 with related posts, zero error-log
lines; relate-picker Playwright 4/4 (infinite-scroll + filter + relate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 100%-JIT experiment surfaced something worse than the 500s: the kernel
miscompile also returns WRONG RESULTS with no error — blank pages (render map
yields empty) and an empty relate picker (drop in relate-options yields []).
Conformance (CEK) passes these, so the code is correct; the JIT silently
produces garbage. Silent corruption is worse than a crash, so the request path
runs on CEK again (IO-bound — no perf loss). Datalog/relations JIT stays on
(/tags 0.16s). Restoring it brought back content + the 17-candidate picker.
Go 100% JIT again once sx-vm-extensions fixes the OP_PERFORM-resume bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per dev intent: don't mask JIT errors. Remove (jit-exclude! "host/*" "dream-*"
"dr/*") so ALL request-path SX runs under JIT. Host handlers miscompile on first
call in the http-listen path (map/rest arg bug → 500, self-heals on retry); that
surfacing is the point — it exercises the JIT against real durable-IO traffic and
gives the sx-vm-extensions loop the full miscompile list to fix (kernel bugs A/B
in plans/HANDOFF-jit-miscompile.md). Datalog JIT win stays (/tags fast).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Datalog/relations saturation is CPU-bound and JITs cleanly: host conformance
271/271 under JIT, 5.4x faster (1m43s -> 19s, same binary); live /tags 2.5s ->
0.76s. loops/host now carries the merged sx-vm-extensions kernel (the JIT engine
+ gate), built into the binary the container bind-mounts.
- docker-compose: SX_SERVING_JIT=1 (default-OFF gate; opt-in here).
- serve.sh: when JIT is on, (jit-exclude! "host/*" "dream-*" "dr/*"). The host app
+ Dream framework MISCOMPILE on first call in the http-listen + cek_run_with_io
path (map/rest emit wrong CALL_PRIM args -> 500; the JIT->CEK fallback marks the
fn failed but does NOT recover the failed call). They're IO-bound, so CEK is no
slower — but the miscompile is a real kernel-JIT bug to fix upstream (see
plans/HANDOFF-jit-miscompile.md), after which this exclude can be dropped.
Verified live: cold pages 200 (no first-hit 500), relate picker lists candidates,
relate round-trip works, /tags fast, datalog still JITs (78 dl-* compiles).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The spine: types ARE posts, and typing is transitive the right way. is-a
(instance-of) does NOT chain on its own, but subsumption does — an instance of a
subtype is an instance of the supertype.
- registry gains "subtype-of" (directed, transitive). host/blog-types-of(slug) =
declared is-a targets PLUS every subtype-of-ancestor of each (composed host-side
over relations/descendants — no new Datalog rules). host/blog-is-a?(slug,type)
is transitive through subtype-of.
- host/blog-seed-types! seeds the root type-posts "type" and "tag" (real posts
that document themselves) with tag subtype-of type, so anything is-a tag is
transitively a type. Idempotent; wired into serve.sh.
- gradual-validation seam: host/blog-type-schemas (empty) + host/blog-schema-of +
host/blog-type-valid? (vacuously true with no schemas) wired into edit-submit
alongside the parse check — enforcement is a one-line add later, not a retrofit.
6 tests: types-of = declared + all subtype-of supertypes; is-a? transitive
through subtype-of; is-a alone does NOT chain; instance of tag is transitively a
type; type-valid vacuous with no schemas. 255/255.
Verified live: /type/ + /tag/ render as posts, tag subtype-of type survived a
recreate (durable), ocaml is-a tag.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sessions were in-memory, so a restart logged everyone out (same class as the
relation wipe). Move them to the durable store, but LAZILY so anonymous/crawler
traffic doesn't spam it: session/create mints a sid with no row; the row appears
on the first session/set (a login). A per-boot epoch (one durable write at
startup, host/session-init!) keeps sids unique across restarts without a write
per request.
- lib/host/session.sx: lazy backend (create = no row, set = create row,
exists = row written) + epoch/in-memory-counter sid generation.
- serve.sh: point the session store at the durable backend + host/session-init!.
- blog.sx: host/current-principal is now a durable read, so host/auth-footer
(home + post footers) had to move OUT of the quasiquote into let bindings —
a perform during page-tree build raises VmSuspended (the whole site 500'd for
a beat). Principal computed once per page.
- 2 session tests: create writes no row, set creates the row.
249/249. Verified live: site renders (anon + authed), login + footer survive a
container force-recreate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/relations holds the graph in memory only (a Datalog cache), so related/tags/
types were wiped on every restart while the posts (durable KV) survived — fatal
for a model where tags and types ARE relations. Make the host the durable source
of truth.
- every physical edge is also a KV row "edge:<src>|<kind>|<dst>" in the blog
store (host/blog--add-edge!/--del-edge! wrap relations/relate+unrelate with
kv-put/kv-delete). '|' is safe: slugs are [a-z0-9-], kinds are registry names.
- host/blog-load-edges! rebuilds the in-memory graph from edge:* keys; serve.sh
calls it on boot right after pointing the store at the durable backend.
- lib/relations stays an in-memory cache; the durable KV is the source of truth
(same shape as the blog pointing at the durable backend).
3 tests: KV row written on relate, replay rebuilds the graph after an in-memory
wipe (restart sim), unrelate deletes the row. 247/247.
Verified live: related welcome<->hello, force-recreated the container (wipes the
in-memory graph), the relation + its rendered block survived the restart.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unblock the guarded blog write routes for browsers: a login form sets a
signed session cookie that the same routes accept (alongside Bearer), so
publishing works end-to-end on blog.rose-ash.com without Quart.
- kernel: http-listen emit serialises a response :set-cookies LIST as one
Set-Cookie header each (a headers dict can't hold more than one). Purely
additive — responses without :set-cookies are unchanged.
- server.sx: host/-dream->native forwards :set-cookies to the native resp.
- lib/host/session.sx: durable, signed sessions on the persist KV
(session/create|exists|get|set|clear), wired via dream-sessions-signed.
- lib/host/auth.sx: GET/POST /login + POST /logout; host/require-user accepts
a session principal OR a Bearer token.
- router.sx: host/make-app wraps the whole app in the session middleware and
auto-mounts /login + /logout — the front door always has sessions.
- blog.sx: write routes use host/require-user; serve.sh flips POST /new from
the experimental UNGUARDED route to the guarded write routes, with admin
creds + signing secret + ACL grant from the container env.
- session conformance suite (12): login->cookie->guarded write 201; no
cookie/forged/logged-out -> 401; Bearer fallback still works.
Verified live on blog.rose-ash.com: 401 unauthenticated, 303 login, 303
publish, anonymous read renders, post persists across container recreate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull out the debt that revived the legacy editor: removed kg-compat.sx (uncommitted
bare->namespaced kg-card aliases), the ./blog container mount, the legacy
sx-editor.js + hardcoded asset URLs + ~editor/sx-editor-styles reuse at /new, and
the blog/sx preloads. /new is now a clean minimal form.
Finding that reshapes Phase 5: render-page (5.1) renders STATIC component trees
but is NOT the full evaluator — a component with a data loop ((map fn items) over
(unquote data)) errors 'Not callable: nil'. So clean dynamic component pages + a
native island editor need the aser SSR pipeline (5.2), not just render-page.
Posts still render via per-block guarded render-page; unsupported editor cards
(~kg-md) show placeholders by design (no alias shim). All endpoints 200, boot clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The editor was unstyled: editor.css is .koenig-lexical-scoped (the OTHER editor);
the sx-editor's .sx-* styles live in the ~editor/sx-editor-styles component
(inline <style> in blog/sx/editor.sx). Inline them into /new by rendering that
component with the 5.1 render-page primitive (dogfooding the capability live), +
FontAwesome for the +/slash-menu icons. 79 .sx- rules now inlined.
Also: the sx_host container only mounted spec+lib, so web/adapter-html.sx (and
now blog/sx/{layouts,editor}.sx) silently failed to load at boot -> render-page
errored -> /new 502. Mount ./web + ./blog (ro) so they load. (Transitional reuse
of the legacy blog editor component + its styles; retire via the asset-manifest +
native SX-island editor.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
host/blog-open-create-routes mounts POST /new with error-trapping but NO auth
(create-only; no PUT/DELETE), so the SX editor can publish to the host
end-to-end on the experimental subdomain. VALIDATED LIVE: editor-style
form-urlencoded POST -> 303 -> post renders at /<slug>/ and lists on /.
Deliberate short-lived public write hole (create-only, obscure subdomain).
MUST be gated before real use: Caddy basicauth on /new, or session auth.
Swap host/blog-open-create-routes -> host/blog-write-routes <resolver> to gate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pivot blog to the SX editor's content model. The editor (blog/sx/editor.sx)
emits sx_content = SX element markup, NOT content-on-sx CtDoc blocks. So a post
is now a {slug,title,sx_content,status} record in the durable persist KV, and a
post page is render-to-html(parse sx_content) — server-side, static, no client
runtime needed to view.
Endpoints: GET / (HTML index), /<slug>/ (rendered post), /posts (JSON list),
/new (create form); POST /new (form-urlencoded editor ingest, slug from title,
303 redirect), POST /posts (JSON create), PUT/DELETE /posts/<slug>. Writes
behind auth+ACL (edit/blog). Dropped the content-on-sx/Smalltalk preload chain;
added spec/render + web/adapter-html (render-to-html) + lib/dream/form.
BONUS: render-to-html is ~0ms (vs the 2s content-on-sx Smalltalk asHTML) — it
doesn't hit the JIT-miscompiled path, so blog rendering is no longer slow.
Live: blog.rose-ash.com/ lists posts, /welcome/ renders instantly. Reads live;
the form-ingest write path needs an auth decision before going live (browser
forms can't send bearer; needs session or a Caddy basicauth gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KERNEL (sx_server.ml): route http-listen handlers through cek_run_with_io
instead of bare Sx_runtime.sx_call, so handlers resolve per-request IO
(durable persist reads/writes) via the same IO-driving runner the REPL uses.
Verified: per-request read+write, 10 concurrent writes (15 on disk, no
corruption), handler errors don't crash the server, http contract 6/6.
BLOG: fully dynamic — host/blog-post reads the post from the durable store
(content/head) AND renders (content/html) per request, no in-memory view, no
cached output. Possible because of the IO fix. Honest ~2s due to interpreted
Smalltalk render.
Render speed is NOT solved here: the JIT (precompiler) isn't installed in the
serving mode and currently miscompiles the Smalltalk evaluator's nested ASTs
(enabling it breaks ~60% of tests). Fixing the JIT is a separate, high-payoff
effort. Documented in the plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Blog posts now live in the durable SX store (persist/durable-backend, on-disk
under $SX_PERSIST_DIR — already built: sx_persist_store.ml + lib/persist/
durable.sx). Publishing appends insert ops to the slug's content stream; posts
survive restarts (verified: seq/log stable across container restart, re-seed
idempotent).
Read path: http-listen handlers can't drive per-request perform/IO (sx_call
doesn't resolve the CEK IO suspension the way the main loop does), so posts are
materialised from the store into an in-memory view at boot (host/blog-load-all!
+ host/blog-seed!) and request handlers read the view — perform-free. Store is
source of truth; view is a boot-rebuilt cache.
Deploy: docker-compose.dev-sx-host.yml mounts /root/sx-host-persist (chowned to
appuser 10001) at /data/persist; SX_PERSIST_DIR set. blog.rose-ash.com/welcome/
live. Per-request-IO kernel fix tracked in the plan as the next task.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/blog.sx serves blog posts as HTML at GET /<slug>/ (the original
strangler target, Quart blog post_detail). A post is a content-on-sx CtDoc
rendered via content/html; anonymous + world-visible. In-memory slug->doc
registry now (host/blog-lookup swappable for a persist-backed content stream
later, handler/route unchanged). :slug catch-all mounted LAST so /feed,
/health, /internal/* take precedence. Needs the Smalltalk+persist+content
preload chain + (st-bootstrap-classes!)+(content/bootstrap!) — blog.sx
self-bootstraps at load. serve.sh loads the chain + seeds a welcome post.
Ledger gains the migrated blog post-detail (off-Quart 50% -> 53%).
LIVE: blog.rose-ash.com/welcome/ renders real HTML through Cloudflare->Caddy;
/feed still JSON (precedence verified), unknown slug 404.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Promote lib/host into the docker stack behind blog.rose-ash.com (reusing a
down Quart subdomain). New compose service sx_host runs lib/host/serve.sh on
externalnet; Caddy reverse-proxies blog.rose-ash.com -> sx-dev-sx_host-1:8000.
hosts/ fix: http-listen bound inet_addr_loopback only, unreachable from other
containers. Add SX_HTTP_HOST env (default loopback for tests/local; stack sets
0.0.0.0) in sx_server.ml. serve.sh made container-friendly (SX_PROJECT_DIR).
Verified live through Cloudflare->Caddy: /health, /feed, relations reads serve
real JSON; / 404 (no root route yet). rose-ash.com untouched. Conformance
145/145 green with the rebuilt binary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/server.sx adapts the native http-listen contract (string-keyed
{method,path,query,headers,body} -> {:status :headers :body}) to the Dream
host app: native->dream reassembles path+query into a target dream-request
parses; dream->native is near-identity (dream-response is already
{:body :headers :status}). host/serve = http-listen over host/native-handler
. host/make-app. lib/host/serve.sh boots the full module set and serves in the
foreground (container-entry shaped). Verified live on a host port: health/feed/
feed?actor=/relations reads serve real JSON, unknown->404. server suite (13)
covers the bridge as pure functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
