A record may carry a :body (a composition node); host/blog-post renders it via the
render-fold (host/comp-render) against a context built from the principal (auth), else the
legacy sx_content path. compose.sx loaded into the host (serve.sh + conformance.sh module
lists). host/blog-body-of / host/blog--set-body!.
Seeded /compose-demo: ONE composition object that shows seq + alt(when auth) + row(par) +
each, and renders DIFFERENTLY by context. Verified live-path (ephemeral SX_SERVING_JIT=1):
anon -> login-prompt (else) + columns + event list; authed -> member block (when auth),
login-prompt gone. The object is the program; the render is the execution -- now live.
Focused eval confirms the in-process render matches the test (ANON<span>..> vs MEMBER<..>).
Tests added; full blog suite still box-contended.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Imported Ghost posts' sx_content holds (~kg_cards/kg-*) (from the lexical_to_sx converter);
the host's render-page resolves components, but the kg-cards weren't loaded so they
degraded to '(unsupported block)' placeholders. Copied blog/sx/kg_cards.sx ->
lib/host/sx/kg-cards.sx (host self-contained, not coupled to the legacy blog/ Quart dir)
+ added the one host-local dep ~rich-text (was only a test fixture) + registered it in
serve.sh + conformance.sh module lists. Verified: the real 'Free DVD Box Sets!' post now
renders <figure class=kg-card kg-image-card> for all images, zero placeholders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- live-check.sh + run-picker-check.sh now set SX_SERVING_JIT=1 to MATCH THE CONTAINER:
that env gates the http-listen IO resolver, so without it perform-heavy paths (the is-a/
tags picker's reach-down BFS) falsely raise VmSuspended -> 500 in the harness while the
live site is fine (confirmed live is-a picker = 200). Harness must mirror what the
container runs.
- conformance.sh: 600s -> 1200s cap (overridable via SX_CONF_TIMEOUT). A sibling loop at
load ~6 pushed the Datalog-heavy blog suite past 600s -> false 'no suite results parsed'.
- plan: types can specify SPECIALISED EDITORS — a type's :editor slot = a content-addressed
editor component (WYSIWYG, map picker) shipped to the client like ~relate-picker. Generic
form is the default, not the ceiling; spectrum = generic -> per-field widget -> :editor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- conformance.sh [suite] runs ONE suite (filters the SUITES array so result-parser
indices stay aligned; all MODULES still load). 'conformance.sh sxtp' = 0.3s vs ~8min.
- lib/host/live-check.sh: non-browser live smoke — boot ephemeral host, login, seed a
post (exercises form-ingest write), print status|content-type|body-head per path,
assert reads are text/sx + no JSON leak + no 5xx. The counterpart to run-picker-check.sh.
- plans/NOTE-render-diff-for-vm-ext.md: defer host_render_diff (JIT-vs-interpreter
regression oracle) to the sx-vm-extensions loop — it's their fix's oracle, not a host
feature; building it from loops/host would fork JIT-engine understanding.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The declarative picker markup is now a reusable SX component
(lib/host/sx/relate-picker.sx, defcomp ~relate-picker &key slug kind) instead of
inline markup in the editor. It is a CONTENT-ADDRESSED, CLIENT-EXPANDED component:
- Server: on a full page load render-page expands ~relate-picker server-side
(SEO / no-JS), exactly as before.
- Client: on a boosted SPA nav the edit body serialises to the compact
(~relate-picker :slug … :kind …), and the CLIENT expands it. The component
module is compiled to a content-addressed .sxbc, served immutably from
/sx/h/{hash}, and listed in the page's data-sx-manifest "boot" array so the
client eager-loads it after the web stack — registering its defcomp before any
boosted fragment references it.
Wiring:
- lib/host/sx/relate-picker.sx — the component.
- lib/host/blog.sx — editor emits (~relate-picker :slug s :kind k); the inline
form markup is gone.
- lib/host/static.sx — host/static-manifest-json emits boot:["relate-picker.sxbc"]
(the previously-empty boot array, now used as designed).
- hosts/ocaml/browser/sx-platform.js — loadWebStack eager-loads the page manifest's
boot[] modules (content-addressed) after the web stack.
- bundle.sh + compile-modules.js — copy/compile the component to .sxbc.
- serve.sh + conformance.sh — load the component module server-side.
This gives the host an app-component system: app defcomps shipped to the client by
hash, the same machinery as the kernel modules — the picker is the first, and it's
the model for publishing components externally.
Tests: conformance 272/272 (server expansion); relate-picker.spec.js 6/6 incl. the
boosted-nav populate (proves client-side component load + expansion) and the
error/retry case. WASM stack rebuilt (relate-picker.sxbc @ 6818110a).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three composing pieces that make the blog SPA correct and resilient.
Content-addressed module cache (lib/host/static.sx, serve.sh, blog.sx shell,
conformance.sh): index each web-stack .sxbc by the content hash in its head,
serve GET /sx/h/{hash} immutable text/sx, and emit <script data-sx-manifest>
{file->hash} so the WASM client loads modules content-addressed (localStorage +
immutable) instead of path + max-age. serve.sh builds the index at boot;
conformance.sh now loads static.sx before blog.sx (the shell calls
host/static-manifest-json).
Declarative relate picker (lib/host/blog.sx, lib/dream/form.sx): replace the
inline /relate-picker.js blob — which never ran on swapped-in content, so the
candidate list was empty after a boosted nav to /<slug>/edit — with a declarative
SX-htmx form: sx-get relate-options on "load" + debounced "input", innerHTML-swap
the results ul; infinite scroll via a server-emitted "load more" sentinel
(sx-trigger revealed, sx-swap outerHTML) that pages the rest, q preserved via a
new symmetric dr/url-encode. The engine re-binds these triggers on swapped
content, so the picker populates on full load AND boosted SPA nav. Candidate
relate forms get :sx-disable (plain POST->303->reload, their original behavior;
the engine would otherwise boost them and swap the redirect unreliably).
sx-retry "exponential:1000:30000" on the form+sentinel retries a dropped/offline
fetch forever (the cap bounds the interval, not the attempts).
SIGPIPE hardening (hosts/ocaml/bin/sx_server.ml): the native http-listen server
had no SIGPIPE handler, so a client aborting an in-flight fetch (the engine
cancels superseded requests on a debounced filter/fast nav) closed the socket
mid-write and killed the whole process (exit 141). Ignore SIGPIPE so the failed
write becomes a catchable Sys_error the per-connection handler already swallows.
Tests: host conformance 272/272; relate-picker.spec.js 5/5 incl. a boosted-nav
populate regression; spa-check 4/4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 6 — gradual schema validation made real:
- host/blog-type-schemas now carries a declarative schema (a list of
{:block :msg} required-element rules); "article" requires an h1.
- host/blog--all-tags / --schema-issues / host/blog-type-issues walk the parsed
content and report each missing required block; host/blog-type-valid? = no
issues. A type with no schema imposes nothing (gradual).
- seed an "article" type-post (article subtype-of type). edit-submit now lists
the specific schema issues on a 400 ("an article needs a heading"), so a post
that is-a article must satisfy it on save.
Post-page performance (the unresponsiveness): a post page was ~1s even with no
relations and no load — NOT CPU (render-page ~2ms, in-memory handler ~5ms) but
the DURABLE read path: host/blog--relation-blocks called host/blog-out/in, each
re-scanning the whole KV (host/blog-slugs + an all-edges scan), so a page did ~7
kv-keys performs deep in the call stack. Each durable perform routes through
cek_run_with_io and is costly there. Fixes:
- host/blog-out/in read DIRECT edges from the durable edge store (string scan),
not lib/relations (whose queries re-saturate the Datalog ruleset, ~seconds).
- host/blog--relation-blocks reads the KV key list ONCE and derives both the post
set and the edges in memory (host/blog--edges-for / --recs-slugs), one kv-keys
plus a host/blog-get per linked post. Post pages: ~1s -> ~0.02s (46x); live
11-135s -> ~0.15s. lib/relations stays for TRANSITIVE queries only.
- conformance timeout 300 -> 600s: the relations-heavy blog suite is CPU-bound
under shared-box contention and was tripping a false truncation at 300.
271/271 (blog 100). Verified live: post pages fast, Tags/Related/Tagged-with-this
render, schema rejection works.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Unblock the guarded blog write routes for browsers: a login form sets a
signed session cookie that the same routes accept (alongside Bearer), so
publishing works end-to-end on blog.rose-ash.com without Quart.
- kernel: http-listen emit serialises a response :set-cookies LIST as one
Set-Cookie header each (a headers dict can't hold more than one). Purely
additive — responses without :set-cookies are unchanged.
- server.sx: host/-dream->native forwards :set-cookies to the native resp.
- lib/host/session.sx: durable, signed sessions on the persist KV
(session/create|exists|get|set|clear), wired via dream-sessions-signed.
- lib/host/auth.sx: GET/POST /login + POST /logout; host/require-user accepts
a session principal OR a Bearer token.
- router.sx: host/make-app wraps the whole app in the session middleware and
auto-mounts /login + /logout — the front door always has sessions.
- blog.sx: write routes use host/require-user; serve.sh flips POST /new from
the experimental UNGUARDED route to the guarded write routes, with admin
creds + signing secret + ACL grant from the container env.
- session conformance suite (12): login->cookie->guarded write 201; no
cookie/forged/logged-out -> 401; Bearer fallback still works.
Verified live on blog.rose-ash.com: 401 unauthenticated, 303 login, 303
publish, anonymous read renders, post persists across container recreate.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pull out the debt that revived the legacy editor: removed kg-compat.sx (uncommitted
bare->namespaced kg-card aliases), the ./blog container mount, the legacy
sx-editor.js + hardcoded asset URLs + ~editor/sx-editor-styles reuse at /new, and
the blog/sx preloads. /new is now a clean minimal form.
Finding that reshapes Phase 5: render-page (5.1) renders STATIC component trees
but is NOT the full evaluator — a component with a data loop ((map fn items) over
(unquote data)) errors 'Not callable: nil'. So clean dynamic component pages + a
native island editor need the aser SSR pipeline (5.2), not just render-page.
Posts still render via per-block guarded render-page; unsupported editor cards
(~kg-md) show placeholders by design (no alias shim). All endpoints 200, boot clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The editor was unstyled: editor.css is .koenig-lexical-scoped (the OTHER editor);
the sx-editor's .sx-* styles live in the ~editor/sx-editor-styles component
(inline <style> in blog/sx/editor.sx). Inline them into /new by rendering that
component with the 5.1 render-page primitive (dogfooding the capability live), +
FontAwesome for the +/slash-menu icons. 79 .sx- rules now inlined.
Also: the sx_host container only mounted spec+lib, so web/adapter-html.sx (and
now blog/sx/{layouts,editor}.sx) silently failed to load at boot -> render-page
errored -> /new 502. Mount ./web + ./blog (ro) so they load. (Transitional reuse
of the legacy blog editor component + its styles; retire via the asset-manifest +
native SX-island editor.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KERNEL: add a render-page primitive (sx_server.ml, persistent mode) that renders
an UNEVALUATED SX expression with the server env via sx_render_to_html.
render-to-html expands defcomp components and collects keyword attrs itself; SX
handlers can't reach the server env, so the prim supplies it. Fixes the attr
mangling — bare render-to-html on an EVALUATED component tree turns (form :id ..)
into <form>idpost-new-form..; rendering the unevaluated expr keeps :id an attr.
HOST: lib/host/page.sx — host/page (expr -> HTML response) + host/page-route
(mount on a GET path). New page suite (8 tests) proves a generic attributed +
nested component renders correctly through a host route; verified ~editor/form
renders right too. This is the component-render step of the generic
interactive-SX-page capability; shell + static assets + hydration (5.2-5.4) next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pivot blog to the SX editor's content model. The editor (blog/sx/editor.sx)
emits sx_content = SX element markup, NOT content-on-sx CtDoc blocks. So a post
is now a {slug,title,sx_content,status} record in the durable persist KV, and a
post page is render-to-html(parse sx_content) — server-side, static, no client
runtime needed to view.
Endpoints: GET / (HTML index), /<slug>/ (rendered post), /posts (JSON list),
/new (create form); POST /new (form-urlencoded editor ingest, slug from title,
303 redirect), POST /posts (JSON create), PUT/DELETE /posts/<slug>. Writes
behind auth+ACL (edit/blog). Dropped the content-on-sx/Smalltalk preload chain;
added spec/render + web/adapter-html (render-to-html) + lib/dream/form.
BONUS: render-to-html is ~0ms (vs the 2s content-on-sx Smalltalk asHTML) — it
doesn't hit the JIT-miscompiled path, so blog rendering is no longer slow.
Live: blog.rose-ash.com/ lists posts, /welcome/ renders instantly. Reads live;
the form-ingest write path needs an auth decision before going live (browser
forms can't send bearer; needs session or a Caddy basicauth gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
CRUD on the durable content store, per-request IO:
GET /posts list (public) -> [{slug,title}]
GET /<slug>/ read (public) -> HTML / 404
POST /posts create (auth+ACL edit/blog) -> 201/400/409
PUT /posts/<slug> update title+body -> 200/400/404
DELETE /posts/<slug> delete (truncate) -> 200/404
Writes behind the auth+ACL pipeline; create=insert ops, update=op-updates,
delete=stream truncate. 16 new CRUD tests (full lifecycle + 401/403/409/404).
GOTCHA fixed: is a reserved CEK special form — a (let ((guard ...)))
helper was shadowed by it ((guard h) ran the guard special form -> 'first:
expected list'). Renamed to host/blog--protect; namespace-prefix all helpers.
HARDENING: conformance.sh now FAILS LOUD on load/eval errors. A test file that
errors mid-load silently truncates its suite and reports a false green (this hid
the CRUD failure as 'blog 13 passed, 0 failed'). The runner greps for error
markers and aborts. Documented the SX gotcha set + prevention ladder in the plan.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Blog posts now live in the durable SX store (persist/durable-backend, on-disk
under $SX_PERSIST_DIR — already built: sx_persist_store.ml + lib/persist/
durable.sx). Publishing appends insert ops to the slug's content stream; posts
survive restarts (verified: seq/log stable across container restart, re-seed
idempotent).
Read path: http-listen handlers can't drive per-request perform/IO (sx_call
doesn't resolve the CEK IO suspension the way the main loop does), so posts are
materialised from the store into an in-memory view at boot (host/blog-load-all!
+ host/blog-seed!) and request handlers read the view — perform-free. Store is
source of truth; view is a boot-rebuilt cache.
Deploy: docker-compose.dev-sx-host.yml mounts /root/sx-host-persist (chowned to
appuser 10001) at /data/persist; SX_PERSIST_DIR set. blog.rose-ash.com/welcome/
live. Per-request-IO kernel fix tracked in the plan as the next task.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/blog.sx serves blog posts as HTML at GET /<slug>/ (the original
strangler target, Quart blog post_detail). A post is a content-on-sx CtDoc
rendered via content/html; anonymous + world-visible. In-memory slug->doc
registry now (host/blog-lookup swappable for a persist-backed content stream
later, handler/route unchanged). :slug catch-all mounted LAST so /feed,
/health, /internal/* take precedence. Needs the Smalltalk+persist+content
preload chain + (st-bootstrap-classes!)+(content/bootstrap!) — blog.sx
self-bootstraps at load. serve.sh loads the chain + seeds a welcome post.
Ledger gains the migrated blog post-detail (off-Quart 50% -> 53%).
LIVE: blog.rose-ash.com/welcome/ renders real HTML through Cloudflare->Caddy;
/feed still JSON (precedence verified), unknown slug 404.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/server.sx adapts the native http-listen contract (string-keyed
{method,path,query,headers,body} -> {:status :headers :body}) to the Dream
host app: native->dream reassembles path+query into a target dream-request
parses; dream->native is near-identity (dream-response is already
{:body :headers :status}). host/serve = http-listen over host/native-handler
. host/make-app. lib/host/serve.sh boots the full module set and serves in the
foreground (container-entry shaped). Verified live on a host port: health/feed/
feed?actor=/relations reads serve real JSON, unknown->404. server suite (13)
covers the bridge as pure functions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate the two internal relations read queries onto lib/relations: GET
/internal/data/get-children + /get-parents dispatch to relations/children
and relations/parents. Bridge the Quart (type,id) node key to a graph atom
symbol "type:id" with relation-type as the edge kind; optional child/parent
-type params filter by "type:" prefix. Golden tests pin each endpoint to
subsystem-call + envelope. Ledger entries flipped to :migrated (off-Quart
coverage 27% -> 45%).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/sxtp.sx implements the host<->subsystem wire format per
applications/sxtp/spec.sx:
- message algebra (request/response/condition/event + status helpers
ok/created/not-found/forbidden/invalid/fail) as string-keyed dicts;
verb/status/type stored as symbols (ride the wire bare)
- codec: sxtp/serialize (dict -> text/sx list form, deterministic top-level
field order, nested messages emitted in their own list form, no :msg leak)
and sxtp/parse (text/sx -> dict via a deep keyword-token->string normaliser)
- Dream bridge: sxtp/from-dream (HTTP req -> SXTP req, method->verb,
query->params) and sxtp/to-dream (SXTP resp -> HTTP resp, status->code,
body serialised to text/sx)
- 39-test suite covering algebra, serialise/parse round-trip, mappings, bridge
Runtime notes: serialize renders string-keyed dicts as {:k v} and symbols
bare; parsed keyword tokens are a distinct type (not = to string literals) so
parse normalises; unquote-splicing is unreliable so the emitter is str-based.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Composable handler->handler layers over Dream's primitives, with auth and
permission POLICY injected so the layer is policy-free and testable:
- middleware.sx: host/wrap-errors (JSON 500 via dream-catch-with),
host/require-auth (bearer->principal via dream-bearer-token, JSON 401,
injected token resolver), host/require-permission (lib/acl acl/permit? gate,
JSON 403, injected resource extractor), host/pipeline (first = outermost)
- feed.sx: POST /feed via host/feed-write-routes — auth ∘ ACL(post,feed) ∘
wrap-errors over host/feed-create (parse JSON body -> feed/post -> 201;
non-object -> 400). Created activity reads back via GET /feed.
- middleware suite (9) + feed write tests (6 new); conformance preloads now
include the Datalog engine + ACL subsystem + Dream auth/error.
ACL works with string atoms (no symbol coercion). Mute/prefs layer and sxtp.sx
deferred to the next tick.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First migrated endpoint onto the SX host. lib/host is a thin wiring layer:
a host handler is a Dream handler (request->response) that calls a subsystem
public API and serialises via a shared JSON envelope.
- handler.sx: host/ok, host/ok-status, host/error, host/json-status (Dream's
dream-json is 200-only), host/query-int
- router.sx: host/make-app assembles per-domain route groups + /health probe
into one dream-router (reuses dr/flatten-routes)
- feed.sx: GET /feed reads feed/all + stream combinators, recent-first, with
?actor= filter and ?limit= cap
- 3 test suites incl. a golden test (body == subsystem recent stream + envelope)
- conformance.sh mirrors lib/dream's runner
Builds on dream-on-sx (merged, gate green 480/480) rather than a throwaway
native request model; collapses most of plan Phase 4 into Phase 1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
