host/blog-seed-nt-live-encore! now embeds the RAW Ghost HTML (from rose-ash.com/rss) and
imports via the "html" field, so host/html->sx converts it at boot — no more pre-converted
sx_content from the external Python script. Verified: the converter produces the identical 11
cards (card-image/text ×4 pairs + 3 card-embed), handling the real post's kg-card comments,
srcset, and nested figcaption markup. blog 197/197.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/host/htmlsx.sx — a pure-SX HTML → SX converter (char-level tokenizer + stack parser):
host/html->sx turns a post's HTML into an (article …) tree that host/blog--decompose! consumes
— img / p / figure+figcaption / iframe / headings / blockquote / lists, inline strong/em/a kept
nested (decompose flattens to text), entities decoded to UTF-8, comments+doctype skipped. This
replaces the one-off external Python converter used for the nt-live-encore import.
import-post! now accepts a raw "html" field (converted via html->sx, serialized to sx_content,
decomposed) alongside "sx_content" — so importing real Ghost HTML is first-class. Wired
htmlsx.sx into conformance.sh + serve.sh module lists (loads in conformance AND live).
New htmlsx suite 8/8 (text/entities/void/nested/figure/iframe/comments + an html→sx→decompose→
typed-cards round-trip); blog 197/197 (+ import-from-html test).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
host/blog-seed-landing-demo! (+ host/blog--seed-card! fixed-slug helper): a Landing TYPE with
TWO composition fields — :body (heading/text/image + cond/each) and :aside (text/callout, no
controls) — plus a populated landing-demo instance, wired into serve.sh (survives wipes),
idempotent (fixed card slugs, set-comp! overwrites). /landing-demo/ renders both fields; its
edit page shows two independent block editors (#comp-body, #comp-aside); /landing/ reads the
two-field definition. Demonstrates layer 2 end to end on the live site.
blog 196/196 (+ tests: idempotent 2-field seed, both fields render).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every type post reads as schema + extension. Added host/blog--type-population (host/blog--take
helper): a type's page shows its instances (posts is-a it, first 24 + count) and its subtypes
(is-a / subtype-of inverses), next to the read-only type definition. Injected in host/blog-post
when host/blog--is-type?. So /article/ shows what an article IS *and* which posts are articles;
/card/ shows its subtypes; every card type / tag / type reads its own definition (all are
is-type?).
blog 194/194 (+ tests: population lists instances + count, a parent type lists subtypes, GET
/article/ shows Population).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A type post's public page (/article/) now shows a read-only Type-definition panel: its fields,
each Composition field's block grammar ("may contain: heading, text, image, …; control blocks:
cond, each"), and the relations its instances may use — so anyone can read what a type IS, not
just admins on the edit page. host/blog--type-def-view (the read form of host/blog--type-def-
editor's data); injected in host/blog-post after the body when host/blog--is-type?.
blog 191/191, full conformance 420/420 (+ tests: the view renders fields/grammar/relations;
GET /article/ shows it, an instance's page doesn't).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
related / is-a / subtype-of / tagged are part of an object's composition (external — NOT in the
CID), and the TYPE declares which relation kinds its instances may use (:type-relations; absent
-> all kinds, so metamodel types keep full freedom). host/blog--{all-rel-kinds, type-relations,
set-type-relations!, allowed-relations, relation-allowed?}. The relation editors filter to the
permitted kinds; relate-submit ENFORCES it. article declares (related is-a tagged) — an article
instance can't be subtyped. The type-def editor (Part C) gains a relation CHECKLIST + POST
/<type>/relations, so the type's inline block-grammar AND external relations are edited in one
place: "it's just more composition."
blog 189/189 (+ Part B tests: allowed-relations excludes subtype-of for article, editors filter,
relate rejects a forbidden kind, checklist renders, POST /relations sets it). Full conformance
deferred — the sibling OTel loop is contending on the shared warm-conf dir; Part B touches only
blog.sx, so the other 7 suites are unaffected. Verifying live instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
"It's just more composition": a type post's edit page now shows a Type-definition editor —
each field as name:type, and each Composition field with a GRAMMAR CHECKLIST (a checkbox per
card kind = permitted, + conditional/repeater toggles). Editing it changes what the type's
instances may contain. host/blog--{is-type?, set-field-grammar!, own-field, checkbox,
grammar-form, type-def-editor}; POST /<type>/grammar reads the checklist (uniquely-named
blk-<ct> / allow-<ctrl> boxes, since form fields are single-value) → set-field-grammar!.
Shown only when host/blog--is-type? (declares fields, or subtype-of type) — a type's page has
it, an instance's doesn't.
blog 184/184, full conformance 413/413 (+ Part C tests: is-type?, set-field-grammar!, the
checklist renders, POST /grammar sets it, appears on a type page not an instance's).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The type now GOVERNS the composition, not just declares the slot. A Composition field carries
its grammar: {:name "body" :type "Composition" :blocks (…card types…) :allow ("cond" "each")}.
:blocks absent -> any card subtype (back-compat); :allow absent -> both control blocks.
- host/blog--{field-decl, allowed-blocks, allows-control?, block-allowed?, comp-violations}.
- The editor PALETTE is the grammar: one <option> per allowed card type (spliced as direct
<select> children), and the conditional/repeater add-forms appear only if :allow permits.
- block-add-submit ENFORCES it (was a coarse "any card subtype" check) — the type governs writes.
- comp-violations flags a composition holding a forbidden block (the save/import gate).
- article declares its :body grammar (all 7 card kinds + cond/each).
blog 179/179, full conformance 408/408 (+ grammar tests: allowed-blocks/allows-control?,
palette shows only permitted kinds, add rejects a forbidden card, violations flags one).
Part B (relations as type-governed composition) + Part C (edit the type definition) next.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
host/blog-seed-nt-live-encore! imports the real post (its HTML-derived sx_content embedded)
via host/blog-import-post!, decomposing it into the :body composition of typed cards; wired
into serve.sh next to the demo seeds. Verified: after a full store wipe + reboot it reseeds
(HTTP 200, 4 images, 3 video embeds, tagged nt-live/films). Idempotent. blog 175/175.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prep for importing a real blog post into the :body composition:
- article now DECLARES {:name "body" :type "Composition"} (layer 2 — the type defines that an
article's body is a composition). The edit FORM + submit read scalar-fields only, so the
Composition field never gets a stray text input (or gets nil'd on save).
- decompose handles real-post block kinds: <figure> → card-image WITH its <figcaption> as the
caption (host/blog--find-child digs out the inner <img>); <iframe>/<embed>/<video> →
card-embed with src as :url. card-embed's template now renders an actual <iframe> (videos
play) instead of the url as text.
blog 175/175, full host conformance 404/404 (+ test: figure→card-image(caption) & iframe→
card-embed via import). Next: wipe content (reseed types+demos), import nt-live-encore.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
:body was hardwired; now a TYPE declares which of its fields are compositions
({:name "body" :type "Composition"}), and an object may carry several (:body, :aside, :body-1).
The edit page renders ONE block editor per declared field (host/blog--block-editors →
host/blog--composition-fields → the type's Composition fields, default ["body"]); each editor
is independent, targets #comp-<field>, and its cards get field-qualified slugs
(<container>__<field>__<name>). Every block op takes a `field` (threaded via a hidden "field"
input, so routes are unchanged); the response re-renders just that field's editor.
STORAGE: compositions moved into a STRING-KEYED sub-dict :comps (like :field-values) —
string keys round-trip through persist cleanly, whereas a mix of a keyword :body and a string
"body" top-level key does NOT survive serialization as one key (it splits the data). body-of/
set-body! delegate to comp-of/set-comp! with "body" + a legacy top-level :body read fallback,
so existing bodies still render (the demos reseed into :comps on boot).
blog 174/174, full host conformance 403/403 (+ tests: a Landing type with two Composition
fields → two independent #comp-body/#comp-aside editors; block-add! to a named field; default
[body]). Editor still renders any node kind (no "unknown block"); #block-editor wrapper kept
so the Playwright selectors hold.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The block editor assumed cards-as-objects leaves (ref/alt-with-refs/each-with-ref), so a
hand-authored composition (the compose-demo: text/row/alt-with-text/each-with-inline) fell
through to "(unknown block)" for every text/row node. Now every node kind gets a labelled row
+ preview + move/remove controls: card (✎ chip), text (its content), layout (row/grid + item
count), field, group, and a graceful "other". Conditionals/repeaters display each branch via
host/blog--node-display (a ref → ✎ chip, else the inline text/summary) instead of assuming a
ref. host/blog--node-kind extended (text/layout/field/group); +node-display/+branch-display.
TEST-FIRST: a mixed body (text + alt-with-text + row + each-with-inline) asserts the editor
has NO "unknown block" and labels text/layout/for-each. RED before, GREEN after. blog 171/171.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The block editor now edits the object's ONE root composition (:body) as three block kinds —
CARD (a ref leaf, the "and"/content), CONDITIONAL (alt+when, the "or": render the first
branch whose live-context condition holds), and REPEATER (each: render a template per graph
query). The render-fold already interprets seq/alt/when/each/ref, so authored compositions
render for free; this adds the editing model + UI.
ADDRESSING (per the design discussion — refs are IPNS-like, not frozen CIDs): refs are
RELATIVE-STORED + RESOLVE-IN-CONTEXT. A :body stores (ref "body__b0") (field-relative); the
render context carries the CONTAINER (the object being rendered) and the resolver combines
them -> the card's storage slug <container>__<field>__<name>. So a body is portable (doesn't
pin the container's name), and editing a card updates everything that refs it for free (no
cascade). A cross-domain ref is absolute with an authority ("market:…"); the resolver
dispatches on the prefix (local today, fetch_data/AP later). A compat shim resolves an older
absolute ref directly. (Snapshot-to-absolute-CID stays a future on-demand op; the CID —
hash(record incl :body) — is the immutable layer over this naming layer.)
MODEL: host/blog--{card-slug,resolve-ref,slug->ref,new-card!,node-kind,node-refs,node-pred,
node-each-type,cond->pred,pred->ckey}; block-add!/add-cond!/add-each!; index-addressed
block-move-idx!/remove-idx!/set-cond! (alt/each aren't single refs). UI: host/blog--block-row
renders by kind (card / "if <cond> → … else → …" / "for each <type> → …") with a condition
<select> + ✎ links to each card's own /<cslug>/edit (external object, CID-neutral). Routes:
POST /:slug/blocks/{add, add-cond, add-each, :idx/{move,remove,cond}}.
Types-define-structure is the next layer (a type declares its composition field(s) + block
grammar). Full host conformance 399/399 (blog 170, incl. 5 new and/or/each tests: add-cond/
add-each/set-cond, a conditional rendering the context-chosen branch, the 3-form editor).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The add-block dropdown wrapped its <option>s in a <span> — (select :name "ctype" (span
(option…)…)) — to splice a dynamic list. A <select> only renders <option>/<optgroup> direct
children, so the dropdown was empty. A full-page load hid it (the browser's HTML parser hoists
mis-nested options out of the select), but on a BOOSTED nav the DOM is built programmatically
(no parser error-recovery), so the span stayed and the dropdown was empty. The card types are
a fixed set — inline the options directly as <select> children.
TEST-FIRST: 4th boost-nav.spec.js case (LOGGED IN: boosted nav to edit → assert
select[name=ctype] > option count is 5, incl card-heading). RED before (0 direct-child
options — span-wrapped), GREEN after. All 4 boost-nav tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reported: logged in, go to an edit page, then press Home — nothing happens.
Root cause (browser-DOM trace): a boosted nav home→post morphs a home footer <a> into the
post's "edit" link (morph reuses nodes positionally). morph-node's sync-attrs then STRIPS any
attribute the old node has but the SERVER node lacks — which removes the boost's
client-injected sx-swap="innerHTML" (the server never sends it). With sx-swap gone the swap
defaults to outerHTML, so clicking edit REPLACES #content (the <div id=content>) with the edit
fragment's <div> (no id) — DOM trace: "sx-boost children [NAV, DIV#content]" → "[NAV, DIV]".
#content is destroyed, so every later boosted nav (Home) fetches but has no swap target
("post-swap: root=nil") → nothing updates.
Fix: sync-attrs no longer removes the boost's injected navigation attributes (sx-target /
sx-swap / sx-push-url / sx-get / sx-select) when the new (server) node lacks them — they're
identical across all boosted links, so a reused node keeps sx-swap="innerHTML" and the swap
morphs #content's children instead of replacing #content. Recompiled the web stack. Pairs
with a511b21d (fresh href) + 88f8b427 (SX-Redirect) — three facets of the morph-node-reuse
problem (stale href, lost swap attr, guarded-redirect clobber).
TEST-FIRST: added a 3rd boost-nav.spec.js case (LOGGED IN: home→post→edit, assert #content
survives + Home works). Reproduced RED via DOM traces (#content count 0), GREEN after — on the
ephemeral server AND live. No regressions: picker 3/3 + block-editor 1/1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reported: go to an edit page, then press Home — nothing happens; navigation stops updating.
Root cause (found via a browser trace): a guarded route (host/require-login) answered a
BOOSTED (SX-Request) request with a 303 to /login. The browser's fetch follows the redirect
but DROPS the SX-Request header on the way, so /login returned the full HTML shell (<!doctype
html>…), not a text/sx fragment. Morphing that whole document into #content DESTROYS the
#content swap target (diagnostic: "#content count: 0"), so every later boosted nav fetches
but has nowhere to swap ("post-swap: root=nil") — the persistent nav Home appears to do
nothing.
Fix (host-side, no engine change — the engine already supports SX-Redirect): for a boosted
request require-login now returns 200 + an `SX-Redirect: /login?next=…` header. The engine
does a FULL navigation (browser-navigate) to a real /login page — #content is never
clobbered. Non-boosted requests still get a plain 303. Also added a "← Home" link to the
login shell (it's a standalone page with no persistent nav, so a logged-out user who followed
a guarded link was otherwise stranded — the literal "press Home" case).
TEST-FIRST: added a second boost-nav.spec.js case (home → post → click edit → assert clean
full-nav to /login, NOT a clobbered SPA, and Home works from there). Confirmed RED before
(Home did nothing on the clobbered page), GREEN after — verified on the ephemeral server AND
live. No regressions: picker 3/3 + block-editor 1/1 (login flow intact).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reported: on blog.rose-ash.com, home --boosted nav--> a post --click "edit"--> lands on
/tags (a HOME footer link), not /<slug>/edit; subsequent navs stop updating.
Root cause: an innerHTML boost swap uses morph-children, which REUSES DOM nodes in place
(matched positionally when links have no id). The home footer's <a href="/tags"> element is
re-purposed as the post's <a href="/compose-demo/edit"> — its href attribute is rewritten,
but bind-client-route-click had captured the OLD href in its click closure, and the element's
is-processed? mark survived the morph (so boost-descendants skipped re-binding it). Clicking
the reused "edit" link fired the stale /tags closure.
Fix: bind-client-route-click now reads the href FRESH from the element at click time
(dom-get-attr link "href", falling back to the captured value) instead of trusting the
closure. A reused node then always follows its CURRENT href — robust to morph reuse without
needing to clear marks or remove listeners. Recompiled the web stack (.sxbc + manifest).
TEST-FIRST: lib/host/playwright/{boost-nav.spec.js, run-boost-nav-check.sh} reproduces the
exact flow (home -> boosted nav -> click edit -> assert URL is /compose-demo/edit, NOT /tags)
against an ephemeral server. Confirmed RED before the fix (landed on /tags), GREEN after. No
regressions: relate-picker 3/3 (incl. boosted-nav populate) + block-editor 1/1 still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The block-editor move/remove controls used :sx-disable "true" (the OLD relate-picker pattern
= plain POST → 303 → full reload). Switched to :sx-post + :sx-target #block-editor + :sx-swap
outerHTML (the current pattern): the click is a text/sx form round-trip through the WASM
engine, the handler returns the re-rendered #block-editor, and it swaps IN PLACE — no reload.
Added lib/host/playwright/{block-editor.spec.js, run-block-check.sh} (the run-picker-check
harness pattern: ephemeral host server + one editable post + the main worktree's chromium).
Verifies the irreducibly-browser behaviour the SX conformance can't see: adding, reordering
(↑), and removing blocks re-render #block-editor live, and the controls RE-BIND on the
content each swap brings in. PASSES (1/1, 16s). blog conformance still 165/165.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two concrete demonstrations of the composition architecture:
THIRD DOMAIN (proves step 8's "a new domain is just a dict + leaf, no new control flow").
host/comp-deps folds a composition to the object ids it TRANSCLUDES — the static contains
DAG of a body. It reuses host/comp-fold's seq/alt/each dispatch verbatim; only the leaf
(collect `(ref ID)`) + accumulator (concat) are new. Useful in its own right (what a
(seq (ref c0) (each … (ref …))) body pulls in; context-specific — alt picks the taken
branch). compose suite 20/20.
LIVE EXECUTE-FOLD DEMO (makes step 7 tangible, parallel to /compose-demo for render).
/workflow-demo runs ONE composition object's :body through host/exec-run — the SAME structure
the render-fold would turn into HTML, folded by execute into a plan of effects (validate →
branch on status → notify each recipient). host/blog-seed-workflow-demo! + host/blog-workflow-
demo + route + serve.sh seed. Shows the behaviour model IS an execute-fold over a composition
object — the same object the block editor authors. blog suite 165/165.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both tests pre-dated the metamodel growth (types/cards/relations are now posts), so the
`related` candidate pool — which by design offers EVERY post (a relation with no declaration
is unrestricted; plans/relations-as-posts.md) — grew past one 20-item page, and the tests
asserted single-page behaviour:
- "omits the load-more sentinel on a short last page" assumed alpha-post's pool < 20;
- "offers all posts" checked P Doc (pdoc, itself a type-def) was on page 1.
Both now test the actual behaviour without depending on absolute counts: the sentinel test
pages past the end (offset=100000 → empty page → no sentinel), and the unrestricted-pool test
filters (?q=doc → finds the pdoc type-def regardless of pagination — confirming `related` is
unrestricted, unlike `tagged`). Behaviour unchanged; the design ("related offers all") stands.
blog suite now 164/164.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`eval <expr>` evals an SX expression against the warm image and reports round-trip time —
the profiling primitive that isolated relations/relate at 6s/call (super-linear). `reload
<files>` hot-reloads specific modules into the warm image. GOTCHAS baked in: the epoch
protocol rejects bare exprs ("Unknown command") so eval wraps in (eval "<src>") with quote/
backslash escaping; an (eval …) acks as (ok-len N C) with the result on its own line (NOT
(ok N R), which is the LOAD ack), errors as (error N …).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The roadmap's capstone: now that two folds exist (render, execute), extract the machinery
they share. host/comp-fold (compose.sx) is the reusable core — the seq/alt/each combinator
dispatch + the `when` predicate set (host/comp--pred?) + the context-environment + the `each`
source (host/comp--source) + recursion + the depth guard, ALL in one place. A domain plugs in
via a small dict {:empty :combine :leaf :overflow}; only its leaves and how results combine
differ:
render = {:empty "" :combine str …} leaf -> markup (+ row/grid layout combinators)
execute = {:empty (list) :combine concat …} leaf -> effect
host/comp-render and host/exec-run are now one-liners over host/comp-fold with their domain.
execute.sx shed its own seq/alt/each dispatch — it's just a dict + a leaf. A THIRD domain
(eval/reduce/extent over the same algebra) is now only a new dict + leaf, no new control flow.
Both folds went through the core with ZERO behaviour change: new tests/compose.sx exercises
the core + render domain directly (17/17 — leaves, seq, row, alt+when (has/eq/not), each
(items/query/empty), tmpl recursion over a (children) tree + depth guard, ref transclude, one
object two contexts); execute 13/13; blog 162/164 (2 pre-existing relate-picker fails). Full
host conformance 388/390. Wired tests/compose.sx into conformance.
plans/composition-objects.md roadmap steps 1-8 COMPLETE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The keystone validation of the universal-algebra thesis. lib/host/execute.sx is a SECOND
interpreter over the SAME seq/alt/each composition algebra as the render-fold — but a
different fold: leaves are EFFECTS, seq = steps in order, alt+when = branch, each =
for-each, and the accumulator is an effect log instead of an HTML string. It REUSES
compose.sx's shared machinery verbatim — host/comp--pred? (when), host/comp--field
(field/value), host/comp--source (each source) — so the predicate set, context-environment,
and iteration source are domain-agnostic; only the leaf semantics + accumulator are new.
KEYSTONE (tested): ONE (alt (when (has "auth") …) …) skeleton + ONE context folds two ways
— render picks the branch → "<b>in</b>", execute picks the SAME branch → {:verb "enter"}.
A publish workflow (validate → branch-on-status → notify-each) runs as one execute-fold over
a composition object. So the behaviour model (Slice 9) is "an execute-fold over a composition
object", not a separate system — the way the recursive tree proved recursion, this proves the
algebra is domain-agnostic. host/exec-run; 13/13 (new execute suite); wired into conformance
+ serve. Full host conformance 371/373 in 42s (warm); the 2 fails are the pre-existing
relate-picker pair.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The post body is now editable as a composition. Model ops over the :body ref-seq (and the
ordered `contains` edges): host/blog-block-add! (create a card object is-a a card-type +
fields, contains edge, append a ref), -remove! (drop ref + edge), -move! (swap adjacent).
host/blog--block-editor renders a row per block — type + a content preview + ↑/↓/remove
controls + a "fields" link — plus an add-block form, injected into the edit page. Routes
POST /:slug/blocks/{add, :cslug/remove, :cslug/move} (guarded; SX-htmx sx-post + outerHTML
swap of #block-editor, redirect fallback for no-JS).
Cards-as-objects pays off: per-block FIELD editing is free — a card IS an object, so its
fields are edited via its own /<cslug>/edit page; the block editor only owns structure.
Guard fix: a card type is a SUBTYPE-OF card (not is-a), so the add validates ctype against
the down-closure of "card", not host/blog-is-a?. Verified via the warm server (162/164; the
2 fails are the pre-existing relate-picker pair). Deferred: Playwright live-swap check;
alt/each block insertion (the core editor handles the seq of refs).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A REAL production perf bug, surfaced while profiling slow conformance. host/blog--add-edge!
mirrored every edge into lib/relations via relations/relate, which RE-SATURATES the whole
CEK-interpreted Datalog ruleset on every single write — super-linear in the fact base
(profiled: 1.1s → 3.5s → 6.1s per edge as the graph grows 10→20→30 facts; O(graph) per
write, O(edges²) to build). This hit the LIVE SITE on every content op: importing a Ghost
post (decompose! = ~4 edges/block), tagging, relating, is-a, the metamodel editor — all
getting slower as the site grows.
Since typing now reads direct KV edges (host/blog--subtype-closure et al.), NOTHING in the
blog domain reads lib/relations anymore — the mirror was pure, very expensive dead weight.
So edges are now KV-only: add/del-edge! just kv-put/kv-delete (~20ms FLAT, O(1)); reads
already walk the edge:* rows directly. host/blog-load-edges! (which replayed every edge into
lib/relations on boot — O(edges²)) is now a no-op. conj/disj operands were already KV-only,
proving the whole graph can be. host/relations.sx (the relations DOMAIN service, its own
type:id nodes) is separate and untouched.
Result: blog-relate! 6.1s→20ms/call (and now FLAT, not growing); full blog suite ~23min→19s;
all 11 host suites 353/355 in 36s (the 2 fails are the pre-existing relate-picker pair). Live
writes drop from seconds to ~20ms. Pairs with the typing-reads-from-KV fix (prev commit).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
STEP 5 (cards-as-objects). The importer no longer carries a Ghost body as one opaque
sx_content string: host/blog--decompose! splits an (article …) into one stored card OBJECT
per top-level block (is-a the mapped card-type + its field-values), links each by an ordered
`contains` edge, and sets the post :body = (seq (ref c0) (ref c1) …). Card types now carry a
render :template, so the new `ref` combinator (compose.sx) transcludes each card via the
SAME typed-block path articles use. /import wired to decompose; the home index filtered to
published so the "block"-status card objects stay hidden. Added the `val` leaf (raw field
value, no <span>) for attribute interpolation in templates (href/src). The post page renders
the transcluded cards — verified end-to-end (conformance 157/159; the 2 fails are the
pre-existing relate-picker pagination pair, unrelated).
PERF (the conformance-speed fix). host/blog typing — types-of / instances-of / type-defs —
computed the subtype closure via lib/relations descendants/ancestors, and EVERY such call
re-saturates the whole CEK-interpreted Datalog ruleset (~seconds each). Typing is the hottest
path (is-a?/types-of/instances-of run per post, per picker, per render), so this dominated
both the blog suite and live page latency. Now the closure is a host-side BFS over the DIRECT
subtype-of edges (the edge:* KV rows, via host/blog--subtype-closure) — one snapshot per
closure, O(edges), cycle-safe, Datalog-free. Same transitive set (KV == relations for direct
edges, host/blog-relate! writes both), so exact, not approximate. Drops Datalog out of the
typing hot path entirely — speeds conformance AND the live site (/tags etc.).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
conformance.sh cold-loads all ~57 modules every run (a multi-minute tax, worst under box
contention). warm-conf.sh keeps a long-lived sx_server with the 44 heavy dependency modules
(datalog/acl/relations/persist/dream) loaded ONCE, and per run reloads only the 16 lib/host/*
modules + the suite's test file — the things you actually edit — then evals the runner.
Reads the MODULES + SUITES arrays straight from conformance.sh (no duplication/drift). Safe
across runs: each test file re-opens a fresh persist store, and (since blog typing now reads
direct KV edges, not lib/relations) the warm Datalog DB no longer feeds blog results, so
stale facts can't pollute a re-run. Usage: warm-conf.sh start | run [suite] | stop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from loops/sx-vm-extensions 2d24c0cf + dcc5d9fa (file hunks only), on top of
be071d56 (compile-let/letrec residue fixes).
1. compiler.sx: desugar map/filter/reduce/for-each/some/every? (literal-fn arg0) to
resumable named-let bytecode loops instead of CALL_PRIM into a native OCaml loop.
The general fix for the serving-JIT "perform-in-HO-callback drops all-but-first"
miscompile — the bytecode loop suspends/resumes within the VM and survives, so the
call_closure_reuse inline-resolve band-aid (and boot-loader jit-exclude! recipes)
are no longer needed. Data-first/symbol-fn forms fall back to CALL_PRIM unchanged.
Proven zero-regression: full run_tests --jit failure SETS byte-identical with/without.
2. lib/hyperscript/runtime.sx: (jit-exclude! "hs-*") — hyperscript was the only guest
missing its jit-exclude! decl; its recursive-descent tokenizer/parser combinators hit
the parser-combinator JIT bug. Runs on CEK (correct); hyperscript compiles to SX at
author time so no serve-time cost.
Together these take run_tests --jit to 4862/1082 = EXACT parity with the CEK baseline
(zero deterministic JIT-specific failures, verified by failure-set diff).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The render context is now the live EXECUTION environment: host/blog--comp-ctx reads device
(mobile/desktop from User-Agent) and locale (from Accept-Language) PURELY from the request
headers — no perform — alongside auth + the graph-query resolver. So the SAME composition
object renders responsively/personalised: `(alt (when (eq "device" "mobile") …) …)` is a
responsive layout, `(when (eq "locale" "fr") …)` a localised variant. The object (its
when-variants) is the definition; the context picks which path renders.
host/blog--device-of / host/blog--locale-of; comp-ctx now (principal req) — post handler
passes req; /compose-demo gains a device-variant block. Reactive/live values plug into the
same context later with no new combinators (the plan's "make the context live" axis).
Verified via focused harness eval (mobile+fr vs desktop+en contexts render M/D variants;
no-req ctx omits device). Tests added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An object's `each` source can now be a GRAPH QUERY: `(query is-a TYPE)` resolves to
whatever is-a TYPE *right now* — the list isn't baked into the body, it's the live graph.
The object's `each` IS the query; the render is the run over current data (the unifying
property, now over real data).
compose.sx stays self-contained: the `query` source delegates to a resolver bound in the
render context under "query" — it asks the context for data, never reaching into the graph
itself. The host supplies graph access via host/blog--comp-query (`(query is-a TYPE)` ->
host/blog-instances-of -> full records) injected by host/blog--comp-ctx (auth + resolver);
the post handler renders :body against that context.
Added a `val` leaf — the raw field value with no markup wrapper, for use inside attributes
(href/src). `field` stays span-wrapped for display; `(val :slug)` makes a real link in the
each template. /compose-demo's each is now a live (query is-a compose-item) over two seeded
instances instead of a baked literal list.
Verified end-to-end via a focused harness eval over the full relations+persist+blog stack
(query iterates real instances; clean href via val; empty query -> empty, not an error).
Blog suite 151/153 — the 2 fails ("relate-options load-more sentinel", "related picker
offers all posts") are PRE-EXISTING (clean HEAD is 149/151 with the identical 2 fails, a
relate-picker pagination-boundary issue) and unrelated to composition; my 2 new tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both compile-let (regular + dict-destructure bindings) and compile-letrec
(named-let / letrec slot-init + value-assign) emitted `<value>; LOCAL_SET slot`
with NO POP. Slots are pre-allocated and LOCAL_SET peeks (doesn't pop), so each
binding value was left as stack residue. In TAIL position this was masked
(OP_TAIL_CALL resets sp to frame.base); in NON-TAIL position the residue leaked
to the enclosing frame, so the caller popped the wrong value ("not callable: 7",
"rest: 1 list arg", "first: expected list, got true") under the serving JIT.
Fix: emit POP (op 5) after each binding's LOCAL_SET. Broad correctness win for
any let/letrec in non-tail position under JIT (and the precondition for landing
the HO-loop desugar, which builds on named-let recur).
Ported from loops/sx-vm-extensions 3126d728 + 41d46f1b (compiler.sx hunks only;
spike commit and worktree .mcp.json/lock noise excluded). --jit conformance in
the source branch: 4842/1102 -> 4847/1097 (+5 fixed, zero regressions).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A record may carry a :body (a composition node); host/blog-post renders it via the
render-fold (host/comp-render) against a context built from the principal (auth), else the
legacy sx_content path. compose.sx loaded into the host (serve.sh + conformance.sh module
lists). host/blog-body-of / host/blog--set-body!.
Seeded /compose-demo: ONE composition object that shows seq + alt(when auth) + row(par) +
each, and renders DIFFERENTLY by context. Verified live-path (ephemeral SX_SERVING_JIT=1):
anon -> login-prompt (else) + columns + event list; authed -> member block (when auth),
login-prompt gone. The object is the program; the render is the execution -- now live.
Focused eval confirms the in-process render matches the test (ANON<span>..> vs MEMBER<..>).
Tests added; full blog suite still box-contended.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cards-as-OBJECTS model (plans/composition-objects.md): an object's :body is a tiny UI
language over content-addressed object refs; the render-fold is its interpreter. Four
combinators — seq (sequence) / row,grid (layout/par) / alt+when (conditional/or) / each
(iteration/loop) — plus field/text/card leaves, ref (transclude), and tmpl (recursion).
The two fundamentals designed IN: (1) recursion via self-referential named templates
(tmpl) + each over (children) + a depth guard — renders trees (verified: a nested type
hierarchy -> [Types[Article][Card[Image][Callout]]]); (2) the context is an extensible
ENVIRONMENT — reads it, extends it (:item, :depth) — so behaviour (Slice 9)
and reactivity (signals) plug in via the context with no new combinators.
and/or/choice fall out of one axis ( on forks) x the container strategy (render-all
vs render-first), so Alt isn't a new node — it's 'first'. The unifying property, proven:
the object's CID is its DEFINITION (query/template/every when-variant); render is the
EXECUTION (which items/branch/context). One object renders two ways by context (anon ->
'Please log in', authed -> 'Members area'). Render-fold and the Slice-9 behaviour interpreter
are the same shape — interpreters over content-addressed objects.
lib/host/compose.sx is self-contained (no blog deps); verified via sx_eval (every combinator
+ a recursive tree + a full composed doc across two contexts). Roadmap: wire :body into
host/blog-render, each-source=graph-query, live context, Lexical->card-objects import, block
editor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Imported Ghost posts' sx_content holds (~kg_cards/kg-*) (from the lexical_to_sx converter);
the host's render-page resolves components, but the kg-cards weren't loaded so they
degraded to '(unsupported block)' placeholders. Copied blog/sx/kg_cards.sx ->
lib/host/sx/kg-cards.sx (host self-contained, not coupled to the legacy blog/ Quart dir)
+ added the one host-local dep ~rich-text (was only a test fixture) + registered it in
serve.sh + conformance.sh module lists. Verified: the real 'Free DVD Box Sets!' post now
renders <figure class=kg-card kg-image-card> for all images, zero placeholders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds two new top-level SXTP message types alongside
request/response/condition/event, modelled on Datastar's
datastar-patch-elements and datastar-patch-signals SSE events:
(patch :target "#x" :mode outer :body (~card)) - DOM fragment
morph. Subsumes HTMX swap modes. Mode is outer (default) |
inner | replace | prepend | append | before | after | remove.
(signals :values {:n 3} :only-if-missing false) - reactive
state patch. nil value removes the signal. only-if-missing
skips existing signals (lazy init).
A server response stream can mix both freely; clients dispatch
by head symbol, ordering preserved. Cleaner than HTMX's
swap-mode-per-trigger because the patch shape is decoupled from
the triggering element/attribute.
Spec at applications/sxtp/spec.sx (patch-fields, signals-fields,
patch-modes, example-patch-stream). Constructors / predicates /
accessors / serialise / parse in lib/host/sxtp.sx. 25 new tests
in lib/host/tests/sxtp.sx (predicates, mode normalisation, fixed
field order, remove-without-body, signals round-trip). Host
conformance 129/129 (was 104/104).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The genesis-import seam for the loops/radar migration (NOTE-blog-types-for-radar.md):
an old Ghost post lands not as bare sx_content but as a TYPED Article.
- host/blog-import-post!(ghost-dict): put! the {slug,title,sx_content,status} record +
is-a article + Ghost columns -> article :field-values (custom_excerpt->subtitle,
feature_image->hero) + tags -> tag-posts with tagged edges. Idempotent. The Ghost body
is already sx_content ((~kg_cards/kg-*) from the Python lexical_to_sx migration), so we
carry it as-is. host/blog-import-all! for batches.
- POST /import (guarded): body = a text/sx LIST of Ghost column dicts (radar's Postgres
reader serialises rows to this); imports each typed; -> {:ok true :data {:imported N
:slugs (...)}}. Runs in the serving handler (IO resolver installed) so the per-post/
per-tag loops are JIT-safe.
Verified live-path end-to-end (ephemeral SX_SERVING_JIT=1): POST a fixture Ghost post ->
imported 1; the post's edit form is pre-filled (subtitle='An imported standfirst',
hero=the feature image), its page renders the subtitle standfirst via the article template
+ the body, and its tags (News/SX) land in the graph. Tests added; full blog suite still
blocked by box contention.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/blogimport — data-migration tooling (plans/migration/data-migration.md): lexical
-> content blocks, genesis import into content:<id> op-log, at-rest shadow-diff verify,
and the Q-M4 internal-data-query live source (injected fetch-fn). Additive (new dir);
composes content-on-sx + persist + dream-json. drafts/ holds the blog-side query to add.
For loops/host to consume when ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the loops/erlang stdlib hardening into the canonical files so
every erlang consumer (fed-sx, identity, ...) gets them — not just a
separate conformance-only file.
lists: sort/1,2 usort/1 keyfind/keymember/keydelete/keyreplace/keystore/
keytake/keysort foldr partition takewhile dropwhile splitwith flatten
max min zip zipwith unzip sublist/2,3 nthtail split droplast flatmap
filtermap mapfoldl search.
proplists: get_value/2,3 get_all_values is_defined lookup delete.
Impls appended to transpile.sx; registrations added directly inside
er-register-builtin-bifs! (so they survive the registry resets that
tests/runtime.sx performs — no wrapper needed when folded in). Full
term order via self-contained er-ext-lt? (the shared er-lt? does not
deep-compare tuples/lists). New lists_ext suite wired into
conformance.conf (dict mode). Conformance 771 -> 874/874.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seed the kg-card / content-on-sx block kinds as types: a 'card' root (subtype-of type) +
card-heading/text/image/quote/code/embed/callout as subtypes, each with its own fields
(host/blog--seed-card-type!). They appear in /meta (Types 11) and define (a) the editor's
future card palette and (b) the radar migrator's target vocabulary. Instances-as-blocks vs
instances-as-posts is a later decision — this is the vocabulary.
plans/NOTE-blog-types-for-radar.md: the TYPE CONTRACT for the loops/radar migration — a
blog post -> is-a article + typed field-values; body Ghost/Koenig cards -> these card-types.
Two paths mapped onto radar's duplicate->cutover->diverge (type-at-import vs type-in-diverge),
plus the open cards-as-blocks-vs-posts question for them to inform from the Ghost corpus.
Verified live-path (/meta Types 11, card-types with fields) + focused eval (type-defs has
card-image; fields src/alt/caption, heading level/text). Full blog conformance still blocked
by box contention; test added for a quiet re-run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
source.sx refactored to a single published-posts batch query returning full rows
(incl. lexical) — the existing post-by-id/slug DTO lacks lexical (sx_content/html
only), so the canonical lexical->blocks path needs a dedicated migration provider.
backfill-ids! now filters client-side (no extra query).
drafts/published-posts.sx + drafts/README.md: paste-ready blog-app change (defquery +
SqlBlogService.list_published_posts returning rows incl. raw lexical). README updated.
source 21/21; total 76/76.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Define a relation through the UI (metamodel editor surface 1, completing it):
POST /meta/new-relation creates a relation-post (is-a relation, :rel metadata) and
registers it via a runtime concat onto host/blog-rel-kinds — safe because the serving
handler has the IO resolver installed. /meta gains a '+ Relation' form (name, label,
symmetric). Verified: define 'Blocks' (symmetric) -> Relations(5), its editor renders on
edit pages, kind-spec + symmetric correct; auth-guarded.
SESSION-SCOPED: the relation-post + edges persist durably, but the rel-kinds registry
entry is lost on restart because load-rel-kinds! must stay UNROLLED — it runs at BOOT
where it is JIT-compiled but the IO resolver is NOT yet installed, so a dynamic loader
(map/reduce over instances-of 'relation' with a durable read per item) silently returns []
(verified: dynamic -> /meta Relations(0)). The serving-JIT HO-callback-perform fix only
engages with the resolver = serve time. Flagged to sx-vm-extensions (NOTE-render-diff-for-
vm-ext.md); they ACKed + are tracking the boot-resolver fix. Reverted the dynamic loader,
kept the unroll with a comment explaining why.
VERIFICATION NOTE: the full blog suite could not complete — the box is under extreme
contention from sibling loops (load 14, multiple full conformance + erlang/vm-ext rebuilds)
and the Datalog-heavy 140-test suite times out even at a 1800s cap. Verified instead two
ways: (1) live-path HTTP (real route + auth + editor render, ephemeral SX_SERVING_JIT=1),
(2) a focused in-process eval of the create-relation core (exists/is-a/kind-spec/symmetric/
registry-len = true,true,true,true,5). Prior full run was 140/140; changes since are purely
additive (handler + form + route + 3 tests). Re-run the blog suite when the box is quiet.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
source.sx: live-source adapter resolving Q-M4 (internal-data query, not direct PG).
Injected fetch-fn transport port (hexagonal seam); parse-row maps a blog post-row to
the importer post dict and parses the :lexical JSON string via dream-json-parse.
End-to-end drivers: backfill! (enumerate->fetch->import) and sync-verify
(enumerate->fetch->verify), + backfill-ids! explicit-id fallback.
Tests mock the transport against the documented response contract incl. a real lexical
JSON string. README flags the one blog-side gap (add a published-posts enumeration
query) + production fetch_data wiring (lives in lib/host). source 20/20; total 75/75.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the 'types define the UI' loop and adds the editor's create half.
8c (render template): a type declares a :template — a parameterised SX tree (stored as
source) with (field "name") placeholders that resolve to the instance's field-values at
render. host/blog-template-of / --set-template! / --instantiate (pure tree-walk) /
--typed-block (per the post's types, parse+instantiate, pre-fetched in the handler).
host/blog-post renders it above the body. Article seeded a subtitle standfirst template.
So ONE field definition now drives BOTH the edit form AND the rendered page.
create-type (metamodel editor surface 1): POST /meta/new-type creates a published post
subtype-of "type" -> appears in host/blog-type-defs / the /meta Types list, ready to be
given fields/schema/template. Guarded (unauthed -> login, not created). /meta gains a
'+ Type' form. You can now DEFINE A TYPE THROUGH THE UI.
Verified live-path: typed post's subtitle renders on its page; create 'Recipe' via the
form -> Types(4). Blog suite 140/140.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- live-check.sh + run-picker-check.sh now set SX_SERVING_JIT=1 to MATCH THE CONTAINER:
that env gates the http-listen IO resolver, so without it perform-heavy paths (the is-a/
tags picker's reach-down BFS) falsely raise VmSuspended -> 500 in the harness while the
live site is fine (confirmed live is-a picker = 200). Harness must mirror what the
container runs.
- conformance.sh: 600s -> 1200s cap (overridable via SX_CONF_TIMEOUT). A sibling loop at
load ~6 pushed the Datalog-heavy blog suite past 600s -> false 'no suite results parsed'.
- plan: types can specify SPECIALISED EDITORS — a type's :editor slot = a content-addressed
editor component (WYSIWYG, map picker) shipped to the client like ~relate-picker. Generic
form is the default, not the ceiling; spectrum = generic -> per-field widget -> :editor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The keystone: a type declares :fields [{name, value-type, widget}], an instance carries
:field-values, and the SAME edit form is generated from the type definitions — no per-type
code. 'The editor maps onto the types.'
8a (field model): host/blog-value-types (String/Text/URL/Int/Date/Bool -> default widget),
host/blog--widget-for (explicit > value-type default > text), host/blog-fields-of +
--set-fields! (on the type-post, like schema), --fields-summary. Article seeded with
subtitle:String + hero:URL. /meta gains a Fields column. host/blog-type-defs (the subtype-of
hierarchy = type DEFINITIONS, vs instances-of = is-a instances).
8b (instance form): host/blog-field-values-of + --set-field-values!; host/blog--fields-for-post
(union of the post's transitive types' fields, deduped); host/blog--field-inputs (one labelled
input per field, widget per value-type, pre-filled). edit-form injects the Fields section
(durable reads pre-fetched); edit-submit reads field-* inputs via host/field and stores them.
Verified live-path (ephemeral, SX_SERVING_JIT=1): relate is-a article -> field inputs appear
-> save -> values persist. Blog suite 132/132.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'see the system you've defined' page: every type-post (with its schema's required
blocks) and every relation-post (with its signature), each linking to the post that
defines it. The surface the metamodel editor hangs off (North Star UI surface 1 of 3).
- host/blog-type-defs: the type DEFINITIONS = the subtype-of hierarchy rooted at 'type'
(type + transitive subtypes). NOT host/blog-instances-of 'type' (that's the is-a
INSTANCES — typed content, not the definitions, which are linked by subtype-of).
- host/blog-meta-index (GET /meta, mounted before /:slug): pure read, all durable reads
pre-fetched into let bindings before the quasiquote (perform-in-tree = VmSuspend);
relations from the boot-populated host/blog-rel-kinds VALUE. Types + relations tables.
- Home footer links to /meta + /tags.
Verified live (ephemeral): Types (3: Type/Tag/Article, Article shows required block h1),
Relations (4: related symmetric, is-a/subtype-of/tagged directed). Blog suite 122/122.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- conformance.sh [suite] runs ONE suite (filters the SUITES array so result-parser
indices stay aligned; all MODULES still load). 'conformance.sh sxtp' = 0.3s vs ~8min.
- lib/host/live-check.sh: non-browser live smoke — boot ephemeral host, login, seed a
post (exercises form-ingest write), print status|content-type|body-head per path,
assert reads are text/sx + no JSON leak + no 5xx. The counterpart to run-picker-check.sh.
- plans/NOTE-render-diff-for-vm-ext.md: defer host_render_diff (JIT-vs-interpreter
regression oracle) to the sx-vm-extensions loop — it's their fix's oracle, not a host
feature; building it from loops/host would fork JIT-engine understanding.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
