conformance.sh cold-loads all ~57 modules every run (a multi-minute tax, worst under box
contention). warm-conf.sh keeps a long-lived sx_server with the 44 heavy dependency modules
(datalog/acl/relations/persist/dream) loaded ONCE, and per run reloads only the 16 lib/host/*
modules + the suite's test file — the things you actually edit — then evals the runner.
Reads the MODULES + SUITES arrays straight from conformance.sh (no duplication/drift). Safe
across runs: each test file re-opens a fresh persist store, and (since blog typing now reads
direct KV edges, not lib/relations) the warm Datalog DB no longer feeds blog results, so
stale facts can't pollute a re-run. Usage: warm-conf.sh start | run [suite] | stop.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ported from loops/sx-vm-extensions 2d24c0cf + dcc5d9fa (file hunks only), on top of
be071d56 (compile-let/letrec residue fixes).
1. compiler.sx: desugar map/filter/reduce/for-each/some/every? (literal-fn arg0) to
resumable named-let bytecode loops instead of CALL_PRIM into a native OCaml loop.
The general fix for the serving-JIT "perform-in-HO-callback drops all-but-first"
miscompile — the bytecode loop suspends/resumes within the VM and survives, so the
call_closure_reuse inline-resolve band-aid (and boot-loader jit-exclude! recipes)
are no longer needed. Data-first/symbol-fn forms fall back to CALL_PRIM unchanged.
Proven zero-regression: full run_tests --jit failure SETS byte-identical with/without.
2. lib/hyperscript/runtime.sx: (jit-exclude! "hs-*") — hyperscript was the only guest
missing its jit-exclude! decl; its recursive-descent tokenizer/parser combinators hit
the parser-combinator JIT bug. Runs on CEK (correct); hyperscript compiles to SX at
author time so no serve-time cost.
Together these take run_tests --jit to 4862/1082 = EXACT parity with the CEK baseline
(zero deterministic JIT-specific failures, verified by failure-set diff).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The render context is now the live EXECUTION environment: host/blog--comp-ctx reads device
(mobile/desktop from User-Agent) and locale (from Accept-Language) PURELY from the request
headers — no perform — alongside auth + the graph-query resolver. So the SAME composition
object renders responsively/personalised: `(alt (when (eq "device" "mobile") …) …)` is a
responsive layout, `(when (eq "locale" "fr") …)` a localised variant. The object (its
when-variants) is the definition; the context picks which path renders.
host/blog--device-of / host/blog--locale-of; comp-ctx now (principal req) — post handler
passes req; /compose-demo gains a device-variant block. Reactive/live values plug into the
same context later with no new combinators (the plan's "make the context live" axis).
Verified via focused harness eval (mobile+fr vs desktop+en contexts render M/D variants;
no-req ctx omits device). Tests added.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An object's `each` source can now be a GRAPH QUERY: `(query is-a TYPE)` resolves to
whatever is-a TYPE *right now* — the list isn't baked into the body, it's the live graph.
The object's `each` IS the query; the render is the run over current data (the unifying
property, now over real data).
compose.sx stays self-contained: the `query` source delegates to a resolver bound in the
render context under "query" — it asks the context for data, never reaching into the graph
itself. The host supplies graph access via host/blog--comp-query (`(query is-a TYPE)` ->
host/blog-instances-of -> full records) injected by host/blog--comp-ctx (auth + resolver);
the post handler renders :body against that context.
Added a `val` leaf — the raw field value with no markup wrapper, for use inside attributes
(href/src). `field` stays span-wrapped for display; `(val :slug)` makes a real link in the
each template. /compose-demo's each is now a live (query is-a compose-item) over two seeded
instances instead of a baked literal list.
Verified end-to-end via a focused harness eval over the full relations+persist+blog stack
(query iterates real instances; clean href via val; empty query -> empty, not an error).
Blog suite 151/153 — the 2 fails ("relate-options load-more sentinel", "related picker
offers all posts") are PRE-EXISTING (clean HEAD is 149/151 with the identical 2 fails, a
relate-picker pagination-boundary issue) and unrelated to composition; my 2 new tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The motivating end-to-end demonstration (fed-sx-triggers-loop.md Phase
4): one trigger arriving in the pipeline drives a multi-step business
flow with a branch, a timer suspension, an injected effect, and a
follow-up activity emit — all in the kernel's own runtime.
- flow.erl: flow:wait/1 — a timer-style suspend that PRESERVES the value
on resume (vs flow:suspend/1, which returns the logged result), so a
"wait until morning" step lets the env flow through to later steps.
- next/flow/flows/blog_publish_digest.erl: the flow. Branches on the
article :category (newsletter -> wait-until-morning -> send + emit;
urgent -> send + emit now; else -> skip), fetches followers (injected),
builds a digest email per follower, and emits a DigestSent activity
OBJECT. Effect-as-data: a flow can't call kernel gen_servers from
inside the drive (a blocking call there deadlocks the scheduler), so
it returns the emails + DigestSent object for a driver to dispatch and
append — which can then trigger downstream flows, closing the loop.
Test: triggers_e2e.sh (10) — urgent completes in one cycle with 3 emails
+ a DigestSent object; newsletter suspends on the morning timer, then
resumes to the same on "advancing the clock"; draft takes the else
branch (no emails); a non-Article note is rejected by the guard; a
duplicate activity fires once. flow:wait covered in next/flow (36/36).
plans/fed-sx-design.md §13.10 documents the trigger fan-out as a
kernel convention. lib/erlang 771/771.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The post-append fan-out that fires durable flows from arriving
activities (fed-sx-triggers-loop.md Phases 2+3), native into next/flow
— no cross-guest FFI.
- pipeline.erl: apply_triggers/3 runs AFTER the kernel append (rejected
activities never reach it). It looks the activity's type up in the
trigger registry, drops specs whose guard/actor-scope fails or whose
{activity_cid, trigger_cid} pair already fired (federation can deliver
the same activity twice — dedup is keyed on that pair, read from the
actor's :triggers_fired), and dispatches the rest. Returns the audit
triples for the kernel to fold into :triggers_fired + its projection.
Must not be called inside a `try` (it does gen_server:calls, which
deadlock the scheduler inside a try); running post-append in its own
step satisfies that.
- flow_dispatch.erl: bridges a matched trigger to flow_store:start, with
the activity bound into the flow's input env. guard_passes/3 gates on
actor-scope + guard. Failures (unknown flow, crashing first step) come
back as {error, _}, never raised — one flow can't take down the rest.
- flow_store.erl: drive wrapped in try (the drive is pure, so the try is
safe) so a flow whose step raises yields {error, {flow_crashed, _}}
instead of crashing the store.
Tests: flow_dispatch.sh (12), pipeline_triggers.sh (10). lib/erlang
771/771, next/flow 34/34.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The trigger declaration layer (fed-sx-triggers-loop.md Phase 1): bind an
activity-type to a durable flow so an arriving activity can fan out into
a business flow.
- next/genesis/activity-types/define_trigger.sx — the DefineTrigger verb
(DefineActivity form, nested-get schema). :object carries
:activity-type, :flow-name, optional :guard / :actor-scope.
- next/kernel/trigger_registry.erl — pure core + registered gen_server,
mirroring peer_actors/peer_types. Keyed by activity-type, multiple
specs per type fire independently. Spec = {TriggerCid, FlowName,
Guard, ActorScope}. Hydrates on start from a fold over DefineTrigger
activities (restart-safe, same content-addressing as define_registry).
Manifest activity-types 7->8 (total bundle 38->39); the four bootstrap
count suites + genesis_parse bumped, and bootstrap_load's internal
timeout raised (the larger bundle's double cid:to_string was truncating).
Tests: define_trigger.sh (6), trigger_registry.sh (17). lib/erlang
771/771 + next/flow 34/34 untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both compile-let (regular + dict-destructure bindings) and compile-letrec
(named-let / letrec slot-init + value-assign) emitted `<value>; LOCAL_SET slot`
with NO POP. Slots are pre-allocated and LOCAL_SET peeks (doesn't pop), so each
binding value was left as stack residue. In TAIL position this was masked
(OP_TAIL_CALL resets sp to frame.base); in NON-TAIL position the residue leaked
to the enclosing frame, so the caller popped the wrong value ("not callable: 7",
"rest: 1 list arg", "first: expected list, got true") under the serving JIT.
Fix: emit POP (op 5) after each binding's LOCAL_SET. Broad correctness win for
any let/letrec in non-tail position under JIT (and the precondition for landing
the HO-loop desugar, which builds on named-let recur).
Ported from loops/sx-vm-extensions 3126d728 + 41d46f1b (compiler.sx hunks only;
spike commit and worktree .mcp.json/lock noise excluded). --jit conformance in
the source branch: 4842/1102 -> 4847/1097 (+5 fixed, zero regressions).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A native Erlang-on-SX durable workflow engine, so the fed-sx kernel can
fan activities out into business flows in its own runtime — no cross-
guest FFI to the Scheme lib/flow, no marshalling, no Scheme dependency.
The seed of a real engine (chosen over bridging Scheme flow) that can
later supersede it for substrate use.
- flow.erl — the deterministic-replay driver. Same durability model as
the Scheme engine (re-run from the top; effects go through suspend;
the replay log is plain [{Tag,Value}] data, restart-ready), but
adapted to three hard runtime constraints: no re-enterable
continuation, no process dictionary, and a blocking receive inside a
`try` deadlocks the cooperative scheduler. Resolution: thread the log
through a railway-style context and make suspend SHORT-CIRCUIT (like a
fail value) instead of throwing — purely functional, sidesteps all
three. Ctx = {flow_cont,V,Log} | {flow_susp,Tag,Log}.
- flow_spec.erl — combinator algebra mirrored from lib/flow/spec.sx:
leaves, sequence/parallel/map_flow, flow_while/flow_until, branch,
railway fail/recover/attempt, tap, try_catch/retry.
- flow_store.erl — durable gen_server: named-flow registry + instance
table + start/resume/status. Drives the pure flow from handle_call,
so no gen_server:call is ever inside the replay try-path.
Gate: next/flow/conformance.sh — 34/34. lib/erlang untouched (771/771).
See next/flow/README.md for the model + why railway threading.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A record may carry a :body (a composition node); host/blog-post renders it via the
render-fold (host/comp-render) against a context built from the principal (auth), else the
legacy sx_content path. compose.sx loaded into the host (serve.sh + conformance.sh module
lists). host/blog-body-of / host/blog--set-body!.
Seeded /compose-demo: ONE composition object that shows seq + alt(when auth) + row(par) +
each, and renders DIFFERENTLY by context. Verified live-path (ephemeral SX_SERVING_JIT=1):
anon -> login-prompt (else) + columns + event list; authed -> member block (when auth),
login-prompt gone. The object is the program; the render is the execution -- now live.
Focused eval confirms the in-process render matches the test (ANON<span>..> vs MEMBER<..>).
Tests added; full blog suite still box-contended.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The composition DAG is not a content mechanism; render is fold #1. The same structure
(content-addressed objects + ordered labelled forks + seq/par/alt/each) is interpreted by a
different fold per domain: content=render, behaviour=execute (flow-on-sx), query=eval
(Datalog), pipeline=reduce (artdag, literally a content-addressed composition DAG),
types=extent (and/or = intersection/union). "Relations just a fork" generalises: relation
kind + fold = domain. The X-on-sx loops already ARE these folds — the composition DAG is the
fleet convergence point. Payoff: build composition once, reuse per domain via interpreters;
the block editor + metamodel UI generalise to every fold (author a workflow like a document).
System collapses to four ideas: content-addressed objects + composition algebra + per-domain
folds + decidable-core predicates. Roadmap +2: prove universality with a second (execute)
fold over the same seq/alt/each; then factor out the shared compose core.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cards-as-OBJECTS model (plans/composition-objects.md): an object's :body is a tiny UI
language over content-addressed object refs; the render-fold is its interpreter. Four
combinators — seq (sequence) / row,grid (layout/par) / alt+when (conditional/or) / each
(iteration/loop) — plus field/text/card leaves, ref (transclude), and tmpl (recursion).
The two fundamentals designed IN: (1) recursion via self-referential named templates
(tmpl) + each over (children) + a depth guard — renders trees (verified: a nested type
hierarchy -> [Types[Article][Card[Image][Callout]]]); (2) the context is an extensible
ENVIRONMENT — reads it, extends it (:item, :depth) — so behaviour (Slice 9)
and reactivity (signals) plug in via the context with no new combinators.
and/or/choice fall out of one axis ( on forks) x the container strategy (render-all
vs render-first), so Alt isn't a new node — it's 'first'. The unifying property, proven:
the object's CID is its DEFINITION (query/template/every when-variant); render is the
EXECUTION (which items/branch/context). One object renders two ways by context (anon ->
'Please log in', authed -> 'Members area'). Render-fold and the Slice-9 behaviour interpreter
are the same shape — interpreters over content-addressed objects.
lib/host/compose.sx is self-contained (no blog deps); verified via sx_eval (every combinator
+ a recursive tree + a full composed doc across two contexts). Roadmap: wire :body into
host/blog-render, each-source=graph-query, live context, Lexical->card-objects import, block
editor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Companion to plans/fed-sx-host-types.md. Build sheet for the deferred
lib/host adapter slice (fed_sx_outbox / fed_sx_inbox): projects the
host's existing type-post metamodel (blog.sx: :cid, :schema, subtype-of
graph) onto the fed-sx DefineType/SubtypeOf verbs, ingests peers' types
into peer_types, validates inbound typed objects via
pipeline:apply_object_schema/2, and serves GET /types/<cid>.
Surfaces the two gating decisions for loops/host: the SX-host <->
Erlang-on-SX runtime boundary (recommends an HTTP boundary to dodge the
er-scheduler gen_server:call deadlock) and the type-CID identity choice.
Scope is the inverse of this loop: lib/host/** only, no next/ edits.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ran build-all.sh with wasm_of_ocaml 6.3.2: output .wasm units came out byte-identical to
the Jun-29 backup (same hashes, diff -rq clean), so 6.3.2 still emits legacy 'try'. A plain
rebuild is a dead end; the fix needs a newer wasm_of_ocaml (or flag) that emits try_table.
No harm done — deployed artifacts unchanged, live SPA intact. apt wabt/wasm2wat can't read
these wasm-GC binaries (0x5e); need wasm-tools or a real-browser check.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
pipeline:apply_object_schema/2 (+ stage_object_schema/1 factory) — the
object-schema stage between activity-type validation and the kernel
append (plans/fed-sx-host-types.md step 4). When an inbound activity's
:object declares a refinement type ({type, TypeName}), resolve it
(Cfg type_index: TypeName -> TypeCid; then peer_types:lookup_or_fetch/2,
a local hit or a wire fetch) and apply the record's refinement schema
to the object's :field_values, rejecting on schema-fail with
{error, {validation_failed, object_schema}}.
The schema is either a 1-arity Erlang predicate (substrate stand-in,
locally stored) or a term_codec-safe {required, [Field,...]} constraint
(so a wire-fetched record validates too). Default
strict_object_schema = false: an unresolvable type is let through (the
skip is where a validation_skipped log belongs); strict rejects.
Objects with no declared type, and names absent from the local index,
are skipped (open-world).
Test: next/tests/object_schema.sh (15) — local hit, wire fetch, fetch
failure strict/non-strict, no peer_types, untyped object, undeclared
name, fun + data schema forms, no-schema record, stage composition.
No regression: pipeline_signature, pipeline_driver green. Plan doc
steps 1-4 marked done.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire format for serving + fetching type docs (plans/fed-sx-host-types.md
step 3).
http_server.erl:
- new type_doc Accept format + content type
(application/vnd.fed-sx.type-doc), distinct from actor-doc.
- GET /types/<cid> -> the cached TypeRecord term_codec-encoded, 404 if
not in the peer_types cache. Reads peer_types via a Cfg
{peer_types, peer_types} guard (hardcoded registered atom, mirroring
the actor-doc route's kernel guard).
discovery_type_fetch.erl — sibling of discovery_fetch. make_fetch_fn
produces the fun/2 peer_types:lookup_or_fetch calls: GET
<base>/types/<cid> with the type-doc Accept header, returning the RAW
bytes (peer_types owns the term_codec decode, so the wire format lives
in one place — the route encodes, the cache decodes). Cfg carries
type_url / type_url_fn for TypeCid -> base URL resolution.
Tests: next/tests/peer_types_route.sh (13, in-process route dispatch),
next/tests/discovery_type_fetch.sh (9, closure vs a python type-doc
stub, end-to-end through peer_types:lookup_or_fetch).
No regression: http_accept, http_actors, http_get_format,
discovery_fetch all still green. Conformance 771/771.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Real browser console on blog.rose-ash.com shows the WASM kernel (Jun-29 artifact, built
with an older wasm_of_ocaml) emits the legacy 'try' exception instruction (deprecated; use
try_table) + loadManifest does a sync XHR. Not breaking yet (SPA boots; the day's symptom
was a stale cached loader, cleared by hard refresh) but will break when browsers drop 'try'.
Fix = rebuild the kernel with the current 6.3.2 toolchain (may emit try_table) + verify in a
real browser + make loadManifest async. hosts/ocaml/browser toolchain; schedule when the box
is quiet with a rollback path, don't rush.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
next/kernel/peer_types.erl — a mirror of peer_actors keyed by type CID
(plans/fed-sx-host-types.md step 2). State [{TypeCidBytes, TypeRecord}],
where TypeRecord is the parsed DefineType :object payload. Refinement
schemas are immutable per CID, so cache entries never go stale.
Pure API: new/0, lookup/2, store/3, evict/2, types/1, lookup_or_fetch/3.
gen_server API (registered `peer_types`): put/2, lookup/1, state_for/1,
known_types/0, lookup_or_fetch/2, start_link/0,1.
lookup_or_fetch pulls a Cfg-supplied
type_fetch_fn :: fun ((TypeCid, Cfg) -> {ok, Bytes} | {error, _})
on a miss, decodes the wire bytes via term_codec into the TypeRecord,
and caches it. No fn -> {error, no_fetch_fn}; fetch error / bad bytes
don't poison the cache (caller can retry). Keeping transport in the
closure (Phase 3 discovery_type_fetch) keeps the cache testable.
Test: next/tests/peer_types.sh (18) — pure + gen_server surface, fetch
miss/hit, no-fn, error-no-poison, undecodable-bytes, prepopulate.
Conformance 771/771.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two new DefineActivity-form genesis activity-types for host-type
federation (plans/fed-sx-host-types.md step 1):
- next/genesis/activity-types/define_type.sx — DefineType verb; schema
accepts an :object with a string :name and optional list :fields.
- next/genesis/activity-types/subtype_of.sx — SubtypeOf verb; schema
accepts an :object carrying string :child-type-cid + :parent-type-cid.
Schema bodies use nested `get` (not keyword-threading) so they are
directly evaluatable — keywords are not callable getters in the kernel.
Both registered in manifest.sx (activity-types now 7); the four bootstrap
suites' bundle counts bumped (5->7, total 36->38).
Tests: next/tests/define_type.sh (7), subtype_of.sh (6) — parse shape,
schema accept/reject, term_codec envelope round-trip.
Also load follower_graph + delivery in bootstrap_start.sh: its check-26
publish path exercises outbox:compute_delivery_set/3 (follower_graph:new
+ delivery:delivery_set), which an m2 substrate change had left unloaded
in that suite — a pre-existing red unrelated to the count bump.
Conformance 771/771; all touched next/tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Imported Ghost posts' sx_content holds (~kg_cards/kg-*) (from the lexical_to_sx converter);
the host's render-page resolves components, but the kg-cards weren't loaded so they
degraded to '(unsupported block)' placeholders. Copied blog/sx/kg_cards.sx ->
lib/host/sx/kg-cards.sx (host self-contained, not coupled to the legacy blog/ Quart dir)
+ added the one host-local dep ~rich-text (was only a test fixture) + registered it in
serve.sh + conformance.sh module lists. Verified: the real 'Free DVD Box Sets!' post now
renders <figure class=kg-card kg-image-card> for all images, zero placeholders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds two new top-level SXTP message types alongside
request/response/condition/event, modelled on Datastar's
datastar-patch-elements and datastar-patch-signals SSE events:
(patch :target "#x" :mode outer :body (~card)) - DOM fragment
morph. Subsumes HTMX swap modes. Mode is outer (default) |
inner | replace | prepend | append | before | after | remove.
(signals :values {:n 3} :only-if-missing false) - reactive
state patch. nil value removes the signal. only-if-missing
skips existing signals (lazy init).
A server response stream can mix both freely; clients dispatch
by head symbol, ordering preserved. Cleaner than HTMX's
swap-mode-per-trigger because the patch shape is decoupled from
the triggering element/attribute.
Spec at applications/sxtp/spec.sx (patch-fields, signals-fields,
patch-modes, example-patch-stream). Constructors / predicates /
accessors / serialise / parse in lib/host/sxtp.sx. 25 new tests
in lib/host/tests/sxtp.sx (predicates, mode normalisation, fixed
field order, remove-without-body, signals round-trip). Host
conformance 129/129 (was 104/104).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The genesis-import seam for the loops/radar migration (NOTE-blog-types-for-radar.md):
an old Ghost post lands not as bare sx_content but as a TYPED Article.
- host/blog-import-post!(ghost-dict): put! the {slug,title,sx_content,status} record +
is-a article + Ghost columns -> article :field-values (custom_excerpt->subtitle,
feature_image->hero) + tags -> tag-posts with tagged edges. Idempotent. The Ghost body
is already sx_content ((~kg_cards/kg-*) from the Python lexical_to_sx migration), so we
carry it as-is. host/blog-import-all! for batches.
- POST /import (guarded): body = a text/sx LIST of Ghost column dicts (radar's Postgres
reader serialises rows to this); imports each typed; -> {:ok true :data {:imported N
:slugs (...)}}. Runs in the serving handler (IO resolver installed) so the per-post/
per-tag loops are JIT-safe.
Verified live-path end-to-end (ephemeral SX_SERVING_JIT=1): POST a fixture Ghost post ->
imported 1; the post's edit form is pre-filled (subtitle='An imported standfirst',
hero=the feature image), its page renders the subtitle standfirst via the article template
+ the body, and its tags (News/SX) land in the graph. Tests added; full blog suite still
blocked by box contention.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Staged cross-loop hand-off (not started here): when the cards-as-types work lands, swap
host/blog-lookup's in-memory registry for content/head over content:<id> streams
populated by lib/blogimport (merged to local architecture a746b6ab, 76/76). Adds a
Phase 4 checklist item + plans/blogimport-pickup.md with concrete steps (merge
architecture, apply blog-side published-posts draft, inject fetch_data as fetch-fn,
backfill, swap lookup, sync-verify parity gate).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
lib/blogimport — data-migration tooling (plans/migration/data-migration.md): lexical
-> content blocks, genesis import into content:<id> op-log, at-rest shadow-diff verify,
and the Q-M4 internal-data-query live source (injected fetch-fn). Additive (new dir);
composes content-on-sx + persist + dream-json. drafts/ holds the blog-side query to add.
For loops/host to consume when ready.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Brings the loops/erlang stdlib hardening into the canonical files so
every erlang consumer (fed-sx, identity, ...) gets them — not just a
separate conformance-only file.
lists: sort/1,2 usort/1 keyfind/keymember/keydelete/keyreplace/keystore/
keytake/keysort foldr partition takewhile dropwhile splitwith flatten
max min zip zipwith unzip sublist/2,3 nthtail split droplast flatmap
filtermap mapfoldl search.
proplists: get_value/2,3 get_all_values is_defined lookup delete.
Impls appended to transpile.sx; registrations added directly inside
er-register-builtin-bifs! (so they survive the registry resets that
tests/runtime.sx performs — no wrapper needed when folded in). Full
term order via self-contained er-ext-lt? (the shared er-lt? does not
deep-compare tuples/lists). New lists_ext suite wired into
conformance.conf (dict mode). Conformance 771 -> 874/874.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Seed the kg-card / content-on-sx block kinds as types: a 'card' root (subtype-of type) +
card-heading/text/image/quote/code/embed/callout as subtypes, each with its own fields
(host/blog--seed-card-type!). They appear in /meta (Types 11) and define (a) the editor's
future card palette and (b) the radar migrator's target vocabulary. Instances-as-blocks vs
instances-as-posts is a later decision — this is the vocabulary.
plans/NOTE-blog-types-for-radar.md: the TYPE CONTRACT for the loops/radar migration — a
blog post -> is-a article + typed field-values; body Ghost/Koenig cards -> these card-types.
Two paths mapped onto radar's duplicate->cutover->diverge (type-at-import vs type-in-diverge),
plus the open cards-as-blocks-vs-posts question for them to inform from the Ghost corpus.
Verified live-path (/meta Types 11, card-types with fields) + focused eval (type-defs has
card-image; fields src/alt/caption, heading level/text). Full blog conformance still blocked
by box contention; test added for a quiet re-run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
source.sx refactored to a single published-posts batch query returning full rows
(incl. lexical) — the existing post-by-id/slug DTO lacks lexical (sx_content/html
only), so the canonical lexical->blocks path needs a dedicated migration provider.
backfill-ids! now filters client-side (no extra query).
drafts/published-posts.sx + drafts/README.md: paste-ready blog-app change (defquery +
SqlBlogService.list_published_posts returning rows incl. raw lexical). README updated.
source 21/21; total 76/76.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes m2's last open box. The delivery_worker now wires its retry
loop on erlang:send_after / cancel_timer self-casts: failing flush
arms the per-Cid backoff timer; handle_info({retry, Cid}) redrives
one Cid through deliver_one_pure; success clears state, failure
schedules next slot or dead-letters on attempt 6.
m2 carries three cherry-picks of the send_after substrate work
(originally landed on loops/erlang via 3709460d/98b0104c/b10e55f0).
Those same commits are already on this architecture via the earlier
loops/erlang merge (154681a4); merging m2's duplicates is a
mechanical conflict-resolve to whichever copy git picks first.
Highlights since the previous m2 merge (2bafb4f7):
- 8b-timer wiring + 5 new tests in delivery_retry_timer.sh
- :timers state field tracks live refs; cancel_timer_for before
re-arming so stale timers don't keep the scheduler alive
- state_srv/1 + timer_ref_for/2 for test introspection
- merge-prep note documenting the duplicate-fix rebase strategy
m2 is now feature-complete. Conformance gate 771/771.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
# Conflicts:
# lib/erlang/conformance.sh
# lib/erlang/scoreboard.md
Wires the delivery_worker's retry loop on top of the
erlang:send_after / cancel_timer primitives just landed on
loops/erlang (3709460d, 98b0104c, 779e53b2 — cherry-picked here
since origin/architecture hasn't caught up yet).
Surface:
- new :timers [{Cid, Ref}] state field tracks live timer refs
- handle_call(flush): drain (existing semantics) + arm_retry_timer
per retried Cid (computes backoff slot from the now-bumped attempt
count, sets next_retry_at, send_after self-cast). Reply shape
unchanged.
- handle_info({retry, Cid}, S): redrives that one Cid through
deliver_one_pure. Success → record_success_pure + clear pending.
Failure → schedule_retry_for (which bumps attempts, dead-letters on
slot 6, or arms next slot).
- cancel_timer_for/2 before arming a new timer so stale timers don't
keep the scheduler's run loop alive after the work is done.
- state_srv/1 + timer_ref_for/2 for test introspection.
5/5 in new delivery_retry_timer.sh; existing delivery_worker.sh
17/17 and delivery_retry.sh 11/11 still green. Conformance gate
771/771 (was 761/761; the +10 is the cherry-picked send_after
suite).
Closes Blockers #3. m2 is now feature-complete.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Define a relation through the UI (metamodel editor surface 1, completing it):
POST /meta/new-relation creates a relation-post (is-a relation, :rel metadata) and
registers it via a runtime concat onto host/blog-rel-kinds — safe because the serving
handler has the IO resolver installed. /meta gains a '+ Relation' form (name, label,
symmetric). Verified: define 'Blocks' (symmetric) -> Relations(5), its editor renders on
edit pages, kind-spec + symmetric correct; auth-guarded.
SESSION-SCOPED: the relation-post + edges persist durably, but the rel-kinds registry
entry is lost on restart because load-rel-kinds! must stay UNROLLED — it runs at BOOT
where it is JIT-compiled but the IO resolver is NOT yet installed, so a dynamic loader
(map/reduce over instances-of 'relation' with a durable read per item) silently returns []
(verified: dynamic -> /meta Relations(0)). The serving-JIT HO-callback-perform fix only
engages with the resolver = serve time. Flagged to sx-vm-extensions (NOTE-render-diff-for-
vm-ext.md); they ACKed + are tracking the boot-resolver fix. Reverted the dynamic loader,
kept the unroll with a comment explaining why.
VERIFICATION NOTE: the full blog suite could not complete — the box is under extreme
contention from sibling loops (load 14, multiple full conformance + erlang/vm-ext rebuilds)
and the Datalog-heavy 140-test suite times out even at a 1800s cap. Verified instead two
ways: (1) live-path HTTP (real route + auth + editor render, ephemeral SX_SERVING_JIT=1),
(2) a focused in-process eval of the create-relation core (exists/is-a/kind-spec/symmetric/
registry-len = true,true,true,true,5). Prior full run was 140/140; changes since are purely
additive (handler + form + route + 3 tests). Re-run the blog suite when the box is quiet.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
source.sx: live-source adapter resolving Q-M4 (internal-data query, not direct PG).
Injected fetch-fn transport port (hexagonal seam); parse-row maps a blog post-row to
the importer post dict and parses the :lexical JSON string via dream-json-parse.
End-to-end drivers: backfill! (enumerate->fetch->import) and sync-verify
(enumerate->fetch->verify), + backfill-ids! explicit-id fallback.
Tests mock the transport against the documented response contract incl. a real lexical
JSON string. README flags the one blog-side gap (add a published-posts enumeration
query) + production fetch_data wiring (lives in lib/host). source 20/20; total 75/75.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
T5 — send_after addresses a registered atom name; the delayed message
lands in that process's mailbox (destination resolved at fire time,
dead/unregistered targets drop silently).
T6 — gen_server loop now handles the {reply,R,S,T} / {noreply,S,T}
timeout-bearing callback returns by scheduling {timeout} to itself via
send_after; handle_info({timeout}, S) fires when no other message
arrives first. Sanity-checks the library hookup.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
T3 — concurrent timers fire in deadline order, not schedule order
(scheduler jumps the clock to the earliest pending deadline each
time the runnable queue drains). T4 — cancel_timer on an
already-fired timer returns the atom false.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Logical-clock timer wheel in the scheduler. send_after schedules a
message-delivery event at an absolute deadline (clock + Time ms);
cancel_timer marks a live timer cancelled and reports remaining ms,
or false. Time advances only when the runnable queue drains, jumping
to the earliest pending deadline (deterministic, no wall clock).
monotonic_time/0,1 exposes the logical ms clock.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closes the 'types define the UI' loop and adds the editor's create half.
8c (render template): a type declares a :template — a parameterised SX tree (stored as
source) with (field "name") placeholders that resolve to the instance's field-values at
render. host/blog-template-of / --set-template! / --instantiate (pure tree-walk) /
--typed-block (per the post's types, parse+instantiate, pre-fetched in the handler).
host/blog-post renders it above the body. Article seeded a subtitle standfirst template.
So ONE field definition now drives BOTH the edit form AND the rendered page.
create-type (metamodel editor surface 1): POST /meta/new-type creates a published post
subtype-of "type" -> appears in host/blog-type-defs / the /meta Types list, ready to be
given fields/schema/template. Guarded (unauthed -> login, not created). /meta gains a
'+ Type' form. You can now DEFINE A TYPE THROUGH THE UI.
Verified live-path: typed post's subtitle renders on its page; create 'Recipe' via the
form -> Types(4). Blog suite 140/140.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- live-check.sh + run-picker-check.sh now set SX_SERVING_JIT=1 to MATCH THE CONTAINER:
that env gates the http-listen IO resolver, so without it perform-heavy paths (the is-a/
tags picker's reach-down BFS) falsely raise VmSuspended -> 500 in the harness while the
live site is fine (confirmed live is-a picker = 200). Harness must mirror what the
container runs.
- conformance.sh: 600s -> 1200s cap (overridable via SX_CONF_TIMEOUT). A sibling loop at
load ~6 pushed the Datalog-heavy blog suite past 600s -> false 'no suite results parsed'.
- plan: types can specify SPECIALISED EDITORS — a type's :editor slot = a content-addressed
editor component (WYSIWYG, map picker) shipped to the client like ~relate-picker. Generic
form is the default, not the ceiling; spectrum = generic -> per-field widget -> :editor.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The keystone: a type declares :fields [{name, value-type, widget}], an instance carries
:field-values, and the SAME edit form is generated from the type definitions — no per-type
code. 'The editor maps onto the types.'
8a (field model): host/blog-value-types (String/Text/URL/Int/Date/Bool -> default widget),
host/blog--widget-for (explicit > value-type default > text), host/blog-fields-of +
--set-fields! (on the type-post, like schema), --fields-summary. Article seeded with
subtitle:String + hero:URL. /meta gains a Fields column. host/blog-type-defs (the subtype-of
hierarchy = type DEFINITIONS, vs instances-of = is-a instances).
8b (instance form): host/blog-field-values-of + --set-field-values!; host/blog--fields-for-post
(union of the post's transitive types' fields, deduped); host/blog--field-inputs (one labelled
input per field, widget per value-type, pre-filled). edit-form injects the Fields section
(durable reads pre-fetched); edit-submit reads field-* inputs via host/field and stores them.
Verified live-path (ephemeral, SX_SERVING_JIT=1): relate is-a article -> field inputs appear
-> save -> values persist. Blog suite 132/132.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the vision refinement: a type drives BOTH sides of the UI from one definition —
fields {name, value-type, widget} drive the edit form (widget per value-type) AND the
render template (parameterised SX on the type-post, instantiated with field-values). An
instance is just field-values; add a field -> editor + page update, no code. kg-cards
become type-posts (the content-on-sx block vocabulary is the seed set); the editor becomes
a generic field-editor defined by the metamodel (the relation-editors already prove the
pattern). Render template = data (meta-circular); only widgets are platform pieces, selected
by value-type. Refined build order: /meta DONE -> Slice 8 typed fields (KEYSTONE) -> generic
instance form -> render template -> cards-as-types + migrate; plus create-type/create-relation
on /meta + clear-and-reseed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'see the system you've defined' page: every type-post (with its schema's required
blocks) and every relation-post (with its signature), each linking to the post that
defines it. The surface the metamodel editor hangs off (North Star UI surface 1 of 3).
- host/blog-type-defs: the type DEFINITIONS = the subtype-of hierarchy rooted at 'type'
(type + transitive subtypes). NOT host/blog-instances-of 'type' (that's the is-a
INSTANCES — typed content, not the definitions, which are linked by subtype-of).
- host/blog-meta-index (GET /meta, mounted before /:slug): pure read, all durable reads
pre-fetched into let bindings before the quasiquote (perform-in-tree = VmSuspend);
relations from the boot-populated host/blog-rel-kinds VALUE. Types + relations tables.
- Home footer links to /meta + /tags.
Verified live (ephemeral): Types (3: Type/Tag/Article, Article shows required block h1),
Relations (4: related symmetric, is-a/subtype-of/tagged directed). Blog suite 122/122.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- conformance.sh [suite] runs ONE suite (filters the SUITES array so result-parser
indices stay aligned; all MODULES still load). 'conformance.sh sxtp' = 0.3s vs ~8min.
- lib/host/live-check.sh: non-browser live smoke — boot ephemeral host, login, seed a
post (exercises form-ingest write), print status|content-type|body-head per path,
assert reads are text/sx + no JSON leak + no 5xx. The counterpart to run-picker-check.sh.
- plans/NOTE-render-diff-for-vm-ext.md: defer host_render_diff (JIT-vs-interpreter
regression oracle) to the sx-vm-extensions loop — it's their fix's oracle, not a host
feature; building it from loops/host would fork JIT-engine understanding.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
build-request-body's POST-form branch now serialises the form fields to a text/sx
body via the serialize primitive (content-type text/sx), instead of FormData ->
URLSearchParams -> urlencoded. A hydrated page posts SX; the host reads it via
host/sx-body / host/field (the server already accepts both — urlencoded stays the
no-engine / login-bootstrap fallback). Recompiled the web stack -> .sxbc.
Verified client-agnostically (no DOM, the user's preference): a new sxtp suite test
proves the wire contract serialize(engine) <-> host/sx-body(server) round-trips a
field dict losslessly, INCLUDING sx_content full of quotes/parens that would break a
naive encoder, plus host/field's content-type discrimination + urlencoded fallback
(sxtp 43/43). The DOM field-read (dom-query-all + .value) is the one irreducibly-
browser bit — left to a targeted Playwright smoke.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Greenfield SX-native pivot (NOT a strangler): the host speaks SX/SXTP end to end;
JSON only at the future ActivityPub federation edge.
- OUTPUT: host/json-status -> host/sx-status — every host/ok/host/error response is
text/sx via the serialize primitive (NOT application/json). Flips feed, relations,
blog reads. Tests assert the SX envelope ({:ok true :data ...}).
- DELETE the blog JSON CRUD /posts (POST/PUT/DELETE) + bearer-based host/blog--protect:
a pure old-contract REST mirror. Create/edit go through the HTML editor forms;
programmatic writes speak SXTP. FOLLOW-UP: no browser delete route yet (was JSON-only,
no UI) — add POST /:slug/delete + cascade edge cleanup when the metamodel UI needs it.
- INPUT: host/sx-body (sxtp.sx) parses a text/sx request body to a string-keyed dict
(parse-safe + sxtp/-normalize). feed POST + relations attach/detach read it.
- UNIFIED field reader host/fields / host/field: text/sx body OR urlencoded form by
content-type. The blog form handlers (new/edit/relate/unrelate) + login read through
it — additive, urlencoded still works (no-engine / bootstrap fallback).
Conformance 290/290 (11 suites). Retires the strangler framing in the plan; adds the
'SX all the way out' wire table. The engine half (browser posts text/sx) follows.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every object (content/type/relation post) now carries a stable :cid = hash of its
canonical, key-sorted content. The runtime has no hash primitive, so host/blog--canon
(recursive, sorts keys -> identical across processes regardless of dict insertion order)
and a tail-recursive double-hash (host/blog--hash-go / host/blog--cid-of) are built in SX.
The slug (a name) and any prior :cid are excluded -> the CID hashes content only.
git-shaped: slug = mutable name -> CID = immutable content identity.
Single choke point host/blog--write! stamps the CID on every record write; routed all
three write sites (put!, set-schema!, seed-rel!) through it. Accessors host/blog-cid and
host/blog-by-cid (reverse lookup). +6 conformance tests (blog suite 134/134). Plan: new
'Content-addressability is universal' section (CID model, git-shape, federation: types
flow across fed-sx as shared content-addressed vocabulary; structure/behaviour trust-split).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the tooling that pays for itself across the remaining slices, ranked by
ROI-per-effort: (1) host_conformance(suite) per-suite fast runner — trivial bash arg,
done by hand this session; (2) host_live_check — boot ephemeral server, authed request
sequence, return rendered HTML (generalizes run-picker-check.sh; the pre-deploy check that
catches serving-JIT divergence conformance misses); (3) host_render_diff — render a route
JIT-vs-interpreter and flag divergence (the precise detector that ends the bug class;
builds on sx_render_trace; regression oracle for the jit-bytecode-correctness loop); (4)
surface deps-check/prim-check as MCP. Plus: file the sx-tree worktree write/validate bug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the behaviour layer. Principle: behaviour is data-defined orchestration over a
small fixed vocabulary of effects; only the effect primitives + the interpreter stay code,
everything between is editable posts (meta-circular — Lifecycle/Transition/Rule/Effect are
themselves types). Guards are pure type-system (Datalog) queries; runs on flow-on-sx
(durable: wait-for webhook, after timer; saga compensation). 'Place order'/'ship' = attempt
transition T.
Sketches the effect vocabulary in four tiers — pure guards / data (graph mutations) /
domain (reserve-stock, book-seat) / integration (charge-card, create-shipment, notify,
federate; the code edge, kept small per artdag's S-expr effects) / control (wait-for, after,
emit, transition; flow primitives) — worked through store + events. The fork: declarative
core + guarded code escape-hatch (Scheme/Smalltalk on a post). Start by pinning the
vocabulary + a generic interpreter, and lift commerce-on-sx/events-on-sx from guest-code
into lifecycle+effect DATA (they already implement exactly this, just not editably).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The metamodel targets the entire rose-ash domain model, not just the blog — the finish
line of the host-on-sx strangler off Quart: define the domain schema as data instead of
porting each service's bespoke models. Records the three honest additions store/events
surface beyond a/b/c+d: (1) typed scalar ATTRIBUTES (datatype properties: price:Money,
stock:Int) alongside entity relations — a real addition, likely Slice 8; (2) behaviour/
lifecycle composes from the substrate loops (flow/commerce/events), not reinvented;
(3) integrations (payments/federation/media) stay referenced services. Structure+validation
from the metamodel, behaviour from substrates, integrations as services.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Name the destination: the host becomes a self-describing metamodel where you define a
domain (types + relations with signatures/algebra) and a working system falls out — the
blog is one seeded configuration. Most instance UI is already generic (relation editors
iterate the relations, pickers come from declares-anchors, validation from :schema), so
'define the types' is mostly a metamodel editor + a generic instance form + a
clear-and-reseed. Frames Slices 6-7 as the schema language this is for.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture the Relation<…> design from the discussion. The reframe: the parameters split
into two halves — the role SIGNATURE (shape of a tuple: per-role type a, arity b,
cardinality c) and the relation's ALGEBRA (behaviour: transitivity/symmetry/inverse/
sub-relations d). A relation is Relation<signature>; today's binary typed relations are
the degenerate 2-role case.
Slice 6: generalise :rel to a :roles signature; (a) per-role type = the declares-anchor
made explicit, (b) arity needs reification (instance-posts) for n-ary, (c) cardinality by
counting. Nominal variance, JIT caveat for n-ary role iteration.
Slice 7: declared algebraic properties with GENERIC closure (retires the hardcoded
is-a/subtype closure — OWL property characteristics); real inverse relations; sub-relations.
Decidable core stops here; defined-by-rule + cross-role predicates fenced behind the
predicate-language decision.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A type-post carries its schema in a :schema slot (a list of {:block :msg} rules — a
refinement {x : T | x has these blocks}). host/blog-schema-of reads it off the post;
the hardcoded host/blog-type-schemas table is gone. A NEW refinement type is pure
data: give a type-post a :schema and its instances are validated on save — no code
(tested with a 'guide' type requiring a 'pre' block). article's schema is migrated
onto the article post at boot (host/blog--set-schema!, a single read+write).
host/blog-put! now MERGES over the previous record, so editing a post's
title/content doesn't nuke its :schema/:rel metadata (also closes the Slice 2
'edit drops :rel' gap). schema-of reads the post (a durable read) — only the SAVE
path calls it (a write request, never a render that would VmSuspend).
conformance 299/299 (+4: article h1 enforced from the post, a new refinement type
validates its instances, schema read off the post, edit preserves :schema).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
