Files
rose-ash/plans/host-on-sx.md
giles 2713636e36
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 17s
host: hand off the native SX-island editor (browser-capable session)
The editor is the interactivity layer — it belongs on the --http island pipeline
(SSRs + hydrates islands), not the http-listen host, and needs browser/Playwright
iteration which this worktree lacks. plans/blog-editor-island.md is the handoff:
goal, architecture (docs-side island -> host /new), the live host contract
(form-urlencoded title/sx_content/status -> 303), the sx_content markup to emit
(standard tags, NOT legacy ~kg-* cards), island authoring gotchas, and pointers.
Host side is ready (ingest proven; CORS on request). Phase 5.5 marked handed off.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-19 21:04:21 +00:00

428 lines
29 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# host-on-sx: The SX web host — off Quart, onto the kernel (Dream-bound)
> **DRAFT outline.** The integration boundary that turns the subsystem libraries
> into running services, and the strangler path off Python/Quart. This is the
> dependency hub — it imports every subsystem. Decision recorded below: native
> server + SXTP **now**, `dream-on-sx` framework layer **next**, Python only at the
> external-integration edges.
The subsystems (`feed`, `search`, `acl`, `mod`, `flow`, `commerce`, `identity`,
`content`, `events`) are libraries. Something has to receive an HTTP request, route
it, call the right subsystem, and serialize the response. Today that's Python/Quart
— the one large non-SX component in the stack: separate runtime, deploy, and
failure mode. The goal is to move the web/host/domain layer onto the SX substrate
and retire Quart, **incrementally (strangler-fig), never big-bang.**
This is already underway: a native OCaml HTTP server is live in prod on
`sx.rose-ash.com` (~3ms cached, ~323 req/s, ~2MB RSS), `defhandler`/`defpage`
exist, and a partial **SXTP** protocol is specced. That is the unblocked near-term
host — no `ocaml-on-sx` dependency.
## Two layers, two timelines
1. **Now (unblocked): native server + SXTP adapter + SX handlers.** Route rose-ash
endpoints onto the SX host one at a time. Each migrated endpoint is an SX
handler dispatching to a subsystem; Quart proxies the rest until cut over.
2. **Next: `dream-on-sx` as the framework layer.** Dream gives Quart-grade
ergonomics — typed routing, middleware stacks, sessions, CSRF. It is gated on
`ocaml-on-sx` Phases 15 + minimal stdlib. **This plan is the concrete target
user that un-parks `dream-on-sx`** (see `plans/dream-on-sx.md`): "the subsystems
need an HTTP front door" is the real feature pulling Dream. Until then, do not
block migration on Dream — the native server is sufficient.
3. **Always: Python only at the edges.** External integrations — SumUp payments,
Ghost CMS, ActivityPub crypto, IPFS/Kubo — ride Python libraries today. They
stay as thin injected adapters (Python/FFI) behind subsystem interfaces until
native replacements exist. "Drop Quart" ≠ "drop every line of Python."
## Status (rolling)
`bash lib/host/conformance.sh`**171/171** (9 suites: handler, middleware, sxtp,
router, feed, relations, blog, server, ledger). **Blog now runs on the EDITOR's
content model** (`sx_content` = SX element markup, what `blog/sx/editor.sx`
emits), NOT content-on-sx CtDoc: a post is a `{slug,title,sx_content,status}`
record in the durable persist **KV**, and a post page is `render-to-html (parse
sx_content)`. Full CRUD + an editor form-ingest endpoint (`POST /new`,
form-urlencoded) + JSON API, writes auth+ACL guarded. **`render-to-html` is fast
(~0ms)** — it doesn't hit the JIT-miscompiled Smalltalk path, so blog rendering
is no longer the 2s problem (that was content-on-sx's `asHTML`).
> **Per-request IO (kernel) — FIXED.** `http-listen` handlers used to run via
> `Sx_runtime.sx_call` (bare CEK, no IO resolution), so a handler doing a durable
> `persist/read` returned an unresolved suspension. Fixed in `sx_server.ml`: the
> handler now runs through `cek_run_with_io` (`Sx_ref.continue_with_call` →
> `cek_run_with_io`), the same IO-driving runner the REPL uses — it resolves
> persist ops via `Sx_persist_store.handle_op` between CEK steps. Verified:
> handlers do per-request durable reads + writes (incl. 10 concurrent, 15 events
> on disk, no corruption); handler errors don't crash the server. NOTE: this is
> the per-request *IO* fix; it does NOT speed up the interpreted Smalltalk render
> (`/welcome/` still ~2s) — that's a separate concern, addressed by caching the
> rendered HTML at boot. (Pre-existing: an erroring handler closes the connection
> with no response instead of a 500 — worth improving later.)
>
> **Render speed (separate from IO) — NOT precompiled.** `/welcome/` is ~2s because
> the interpreted Smalltalk-on-SX render runs on the tree-walking CEK: the JIT hook
> (`register_jit_hook`) is installed only in `--http` page mode, not the epoch/
> http-listen serving mode (`make_server_env`), so zero `[jit]` activity. Enabling
> it in that mode breaks correctness (router 3/6, feed 4/11, … — the known JIT-
> bytecode bug on complex nested ASTs, which the Smalltalk evaluator is). So the
> render is slow until the JIT compiler is fixed (big win, broad payoff — its own
> loop) or the Smalltalk interpreter is optimised. Blog is FULLY DYNAMIC (reads
> store + renders per request, no cache) — slowness is honest, not hidden. Phases 1 & 2 DONE; Phase 3 cut-over
landed (50% off Quart). **The host now serves live HTTP**`lib/host/server.sx`
bridges the native `http-listen` server to the Dream app and `lib/host/serve.sh`
boots it (verified: GET /health, /feed, /feed?actor=, relations get-children/
get-parents all serve real JSON on a host port; unknown→404). Remaining: golden
harness vs live Quart, internal-HMAC middleware, docker stack + Caddy subdomain.
## Ground rules
- **Scope:** `lib/host/**` and `plans/host-on-sx.md`. May **import** every subsystem
+ the kernel's server/SXTP surface. Do **not** edit `spec/`, `hosts/`, `shared/`,
or subsystem internals — wire to their public APIs only. Host-primitive / server
changes belong in `hosts/` (out of scope) → Blockers.
- **Architecture:** a route maps (method, path) → handler; a handler is an SX fn
`request -> response` that calls subsystem APIs; middleware is composed handlers
(auth via `identity`, permission via `acl`, mute via subsystem prefs). SXTP is the
wire format between host and subsystem-as-service.
- **Migration discipline:** each endpoint moved must be behavior-equivalent to its
Quart original (golden-response test before flip). Keep a migration ledger.
- **Commits:** one feature per commit. Progress log + tick boxes.
## Architecture sketch
```
HTTP request HTTP response
│ ▲
▼ │
native OCaml http server (prod) ──────► lib/host/router.sx
(hosts/ — out of scope) — (method,path) → handler
│ ▲
▼ │
lib/host/middleware.sx lib/host/handler.sx
— auth(identity) ∘ acl ∘ mute ∘ ... — request → subsystem call → response
│ ▲
▼ │
lib/host/sxtp.sx subsystem APIs (feed/search/commerce/…)
— wire format, host↔service — called via public interfaces
└── external edges: SumUp / Ghost / AP / IPFS → injected Python/FFI adapters
```
## Phase 1 — Router + handler + one real endpoint
- [x] `router.sx``host/make-app` assembles per-domain route groups + a built-in
`/health` probe into one Dream router (reuses Dream's `dr/flatten-routes`)
- [x] `handler.sx` — JSON envelope (`host/ok`/`host/ok-status`/`host/error`),
status-carrying `host/json-status` (Dream's `dream-json` is 200-only), and
`host/query-int`. A host handler IS a Dream handler (request -> response).
- [x] migrate ONE read endpoint: `GET /feed` (`lib/host/feed.sx`) reads
`feed/all` + stream combinators, serialises recent-first; `?actor=` filter,
`?limit=` cap. Golden test asserts body == subsystem recent stream + envelope.
- [x] `conformance.sh` (mirrors `lib/dream`'s runner) — 28/28
## Phase 2 — Middleware + SXTP
- [x] `middleware.sx` — composable layers as `handler->handler`: `host/wrap-errors`
(JSON 500), `host/require-auth` (bearer -> principal, JSON 401, INJECTED token
resolver), `host/require-permission` (ACL `acl/permit?` gate, JSON 403,
INJECTED resource extractor), `host/pipeline` (first = outermost). Reuses
Dream's `dream-bearer-token` + `dream-catch-with`; calls lib/acl public API.
Mute/prefs layer deferred (no blocker, add when a domain needs it).
- [x] `sxtp.sx` — host↔subsystem wire format (per `applications/sxtp/spec.sx`).
Message algebra (`sxtp/request`/`response`/`condition`/`event` + status
helpers `sxtp/ok`/`created`/`not-found`/`forbidden`/`invalid`/`fail`) as
string-keyed dicts; verb/status/type as symbols (ride the wire bare). Codec:
`sxtp/serialize` (dict → `text/sx` list form, deterministic field order,
nested messages in their own list form, no `:msg` leak) and `sxtp/parse`
(`text/sx` → dict, deep keyword-token→string normaliser). Dream bridge:
`sxtp/from-dream` (HTTP req → SXTP req, method→verb, query→params) and
`sxtp/to-dream` (SXTP resp → HTTP resp, status→code, body→`text/sx`).
- [x] migrate a write endpoint (auth + permission + action): `POST /feed`
(`host/feed-write-routes resolve`) — auth ∘ ACL("post","feed") ∘ wrap-errors
over `host/feed-create`, which parses the JSON body and `feed/post`s it (201);
non-object body -> 400. Created activity is readable back via `GET /feed`.
## Phase 3 — Strangler migration ledger
- [x] enumerate Quart endpoints; track migrated vs proxied — `ledger.sx`: a
catalogue of every endpoint (domain, method, path, Quart original, status
`:native`/`:migrated`/`:proxied`, SX handler) + queries (by-status/by-domain,
`host/ledger-find`, `host/ledger-served?`, distinct domains) and
`host/ledger-coverage` (off-Quart % = (migrated+native)/total). Seeded with
the live state: feed reads+writes migrated, `/health` native, the
internal-only `relations`/`likes` data+action endpoints proxied.
- [ ] golden-response harness vs the live Quart responses
- [x] cut over a whole domain (`relations`) as proof — the CONTAINER relations are
fully on the host (`lib/host/relations.sx`): reads `GET .../get-children` +
`/get-parents``relations/children`/`parents`; writes `POST
.../attach-child` + `/detach-child``relations/relate`/`unrelate`, behind
the auth+ACL pipeline (mirrors POST /feed). Node model: graph atom = symbol
`"type:id"`, edge = relation-type; `child`/`parent-type` params filter by
`"type:"` prefix. Closed-loop test: attach → visible via get-children →
detach → gone. The TYPED actions (`relate`/`unrelate`/`can-relate`) stay
proxied by design — registry + cardinality validation lib/relations lacks.
## Phase 4 — Live wiring + Dream framework layer
- [x] native `http-listen` ↔ Dream-app bridge (`lib/host/server.sx`:
`host/native-handler`/`host/serve`) + `lib/host/serve.sh` launcher. Serves
real HTTP on a host port — verified live (health/feed/relations reads + 404).
- [x] promote into the docker stack + a Caddy subdomain — **LIVE at
`https://blog.rose-ash.com`** (reusing a down Quart subdomain). New compose
service `sx_host` (`docker-compose.dev-sx-host.yml`, container
`sx-dev-sx_host-1`) runs `serve.sh` on `externalnet`; Caddy reverse-proxies
`blog.rose-ash.com``sx-dev-sx_host-1:8000`. Required a `hosts/` fix:
`http-listen` bound `inet_addr_loopback` only — added `SX_HTTP_HOST` env
(default loopback; stack sets `0.0.0.0`) in `sx_server.ml`, rebuilt this
worktree's binary. Verified: `/health`, `/feed`, relations reads serve real
JSON through Cloudflare→Caddy; `/` 404 (no root route yet). `rose-ash.com`
untouched. (Inode-pinned bind-mount gotcha: editing `/root/caddy/Caddyfile`
via a tool swaps its inode so the container kept the old content — loaded live
via reload-from-non-bind-path, then RECONCILED by restarting Caddy so the
bind re-points to the corrected file. Verified post-restart: blog serves, and
`sx.rose-ash.com`/`rose-ash.com` survived.)
- [x] blog published-post read endpoint — `lib/host/blog.sx`: `GET /<slug>/`
renders a content-on-sx `CtDoc` to HTML via `content/html` (anonymous,
world-visible). In-memory slug→doc registry now (swap `host/blog-lookup` for
a persist-backed content stream later, handler/route unchanged). `:slug`
catch-all mounted LAST so domain routes win. **LIVE**: `blog.rose-ash.com/
welcome/` renders real HTML through Caddy. Needs Smalltalk+persist+content
preloads + `(st-bootstrap-classes!)`+`(content/bootstrap!)` (self-bootstraps
at load).
- [ ] proxy-to-Quart fallback for un-migrated paths (strangler requirement before
a real subdomain fronts users).
- [ ] internal-HMAC middleware on `/internal/*` (service-to-service auth; protocol
checks native, signature check needs an HMAC-SHA256 kernel prim — absent today).
- [ ] (gated) adopt `dream-on-sx` session/CSRF ergonomics; re-home external
adapters as native where replacements land.
## Phase 5 — Generic interactive SX-page serving (host SSR)
**The generic gap.** A host serves three classes: (1) JSON/data endpoints —
DONE; (2) static content pages — DONE (`render-to-html` on *parsed* markup, e.g.
blog post `sx_content`); (3) **interactive UI pages** — component/island trees
with attributes + client behaviour — **the host cannot do this at all.** The
"editor problem" is one instance; dashboards, account, market-browse, any admin
screen are the same gap. The capability — not the editor — is the deliverable.
**Why `render-to-html` alone is insufficient (proven).** `render-to-html` on
parsed markup handles attributes (`<div id="x">`); but an *evaluated* component
tree mangles them (`(form :id ..)``<form>idpost-new-form…`) because in the
host preload tags don't collect keyword args as attrs. The `--http` docs server
already does this correctly via its component-render + shell pipeline. So: reuse
that pipeline, don't reinvent or patch per-component.
**Reuse, don't rebuild.** The kernel already has: `~shared:shell/sx-page-shell`
(emits `<!doctype>` + inlined component/island defs in `<script type="text/sx">`
+ CSS + `sx-browser.js` + page SX for hydration), `http_inject_shell_statics`
(gathers defs/CSS/asset-hashes into the env), and `http_render_page`. These power
`sx.rose-ash.com`. The job is to make them reachable from the `http-listen`
serving path.
Sub-steps (each independently gated/verified):
- [x] **5.1 Page render from a host handler.** DONE. Kernel: a `render-page`
primitive (sx_server.ml, persistent mode) renders an UNEVALUATED SX
expression with the server env via `sx_render_to_html` — render-to-html
expands defcomp components + collects keyword attrs itself; SX handlers
can't reach the server env, so the prim supplies it. Host: `lib/host/page.sx`
`host/page` (expr → HTML response) + `host/page-route` (mount on a GET
path). Gate MET: `~editor/form` renders correct HTML (`<form method="post"
class=.. id="post-new-form">…`), and the `page` suite (8 tests) proves a
generic attributed+nested component renders right (no `:class`-as-text
mangling). Root cause confirmed: bare render-to-html on an *evaluated* tree
mangles attrs; `render-page` renders the *unevaluated* expr so expansion +
attr-collection happen in render-to-html.
- [ ] **5.2 Shell statics + aser SSR (the real dynamic-page path).** `render-page`
(5.1) renders STATIC component trees, but is NOT the full evaluator —
dynamic-logic bodies fail (proven: a component doing `(map fn items)` over
`(unquote data)` → "Not callable: nil"). Clean dynamic component pages
(a posts loop) + island pages therefore need the **aser** pipeline (evaluate
control flow, serialise tags) + `http_inject_shell_statics` (component defs /
CSS / asset hashes) + `~shared:shell/sx-page-shell`. Gate: a page with a data
loop renders, and a full shell emits with defs inlined.
NOTE (2026-06-19): the legacy-editor stopgaps (kg-compat aliases, `./blog`
mount, legacy `sx-editor.js` + hardcoded asset URLs at `/new`, the
`~editor/sx-editor-styles` reuse) were REVERTED — they were debt to revive
stale code. `/new` is now a clean minimal form; host pages still use minimal
shell HTML until the aser path lands. Posts render via per-block guarded
`render-page`; unsupported editor cards (e.g. `~kg-md`) show placeholders by
design (no alias shim).
- [ ] **5.3 Static-asset serving.** Serve `/scripts/*.js`, `/*.css`, `/wasm/*`
from `shared/static`. Host has none today — needs a kernel file-serving
route in the `http-listen` server (or a file-read prim + SX static handler).
Interim option to defer: reference assets by absolute URL from the existing
static host. Gate: `sx-browser.js`/CSS load for a host-served page.
- [ ] **5.4 Island hydration.** Confirm a trivial island page boots + hydrates
client-side (sx-browser.js) when served by the host. Gate: a counter island
increments in the browser.
- [~] **5.5 Editor POC — HANDED OFF.** The native SX-island editor is the
interactivity layer; per the architecture it lives on the `--http` island
pipeline (not the host) and needs browser/Playwright iteration (absent in
this worktree). Handoff brief: `plans/blog-editor-island.md`. The host side
is READY: `POST /new` ingest is live + proven (form-urlencoded
title/sx_content/status → 303); CORS can be added on request if the editor
uses fetch. Decision: don't port island hydration into the host; the editor
is a docs-side island that publishes to the host.
**Note:** component SSR is interpreted → slow until the `sx-vm-extensions` JIT
loop lands; correctness first, speed follows. Scope spans `hosts/` (page-render
exposure + static serving) + `lib/host` (page route type + page handlers).
**Modern editor — language.** A WYSIWYG editor is a *reactive UI*, so it should be
an **SX reactive island** (`defisland` + signals/lakes — the platform's native UI
primitive), NOT a guest language (Datalog/Prolog/APL/Haskell are logic/data/array
— wrong tool) and NOT a JS lib (Lexical/Koenig, the legacy baggage). The document
*model* it edits is **content-on-sx** (structured blocks, CvRDT-ready for
collaboration). So: **SX islands for the UI, content-on-sx for the model** — SX
all the way down, dogfooding the reactive runtime + content-on-sx + this new
page-serving capability. (Legacy `blog/sx/editor.sx` is Lexical/Koenig/Quart-CSRF
era — replace, don't resurrect; the `POST /new` ingest already speaks the
`sx_content` contract any new editor emits.)
## Progress log
- **Phase 1 (DONE, 28/28).** `lib/host/{handler,router,feed}.sx` + three test
suites + `conformance.sh`. The host is a thin wiring layer: a host handler is a
Dream handler that calls a subsystem public API and serialises the result via a
shared JSON envelope. First migrated endpoint: `GET /feed`.
- **Decision — build on Dream from Phase 1, not a throwaway native model.** The
plan front-matter gated Dream to Phase 4, but `dream-on-sx` is merged
(commit fe958bda) and its gate (`ocaml-on-sx` P15+P6) is green (480/480), so
reinventing request/response + routing would be pure duplication. Host reuses
Dream's `types.sx` (request/response dicts), `json.sx` (encode), and
`router.sx` (`dream-router`/`dream-get`/`dr/flatten-routes`). Phase 4's
"adopt Dream ergonomics" is therefore largely already satisfied; what remains
for Phase 4 is the live wiring against the real OCaml HTTP server + session.
- The OCaml server handing a `dream-request`-shaped dict to SX handlers is a
`hosts/` change (out of scope) — tracked under Blockers as the eventual
live-wiring step. For now the host layer is exercised purely via conformance.
- **Phase 2 (middleware + write endpoint DONE, 43/43).** `lib/host/middleware.sx`
+ a guarded `POST /feed`. Middleware is plain function composition over Dream's
primitives; auth/permission *policy* is injected (token resolver, resource
extractor) so the layer is policy-free and testable. ACL authorisation runs
against lib/acl's public `acl/permit?` (string atoms work — no symbol coercion
needed). The write path proves the auth ∘ permission ∘ action stack end-to-end:
401 unauth, 403 unpermitted, 201 + readback on success, 400 on bad body.
- **Phase 2 COMPLETE (82/82).** `lib/host/sxtp.sx` adds the SXTP codec + Dream
bridge (39-test suite). Key representation calls, learned by probing the runtime:
keywords are strings at eval time but the `serialize` primitive renders
string-keyed dicts back as `{:k v}` and symbols bare — so messages are
string-keyed dicts with verb/status/type as symbols, and a small str-based
emitter produces wire-faithful list form. `parse` needs a deep normaliser
because parsed keyword tokens are a distinct type (not `=` to string literals).
`unquote-splicing` is unreliable here, so the serializer is str-based, not
quasiquote-based.
- **Next: Phase 3 — strangler migration ledger.** Enumerate the Quart endpoints
(use the `rose-ash-services` `svc_routes` MCP tool), track migrated vs proxied,
and stand up a golden-response harness against the live Quart responses. Then
cut over the smallest whole domain (`likes` or `relations`) as proof.
- **Phase 3 — ledger module (DONE, 107/107).** `lib/host/ledger.sx` + a 25-test
suite. Enumerated the endpoint surface via the `rose-ash-services` MCP
(`svc_routes`/`svc_queries`/`svc_actions`): `likes` and `relations` have **no
public blueprint routes** — they're internal-only, exposed as
`/internal/data/{query}` + `/internal/actions/{action}` (HMAC-signed). The
ledger is a pure-data catalogue keyed by (domain, method, path) carrying each
endpoint's Quart original, status, and serving SX handler; coverage reports the
off-Quart percentage. Cut-over target chosen: **`relations`** (already has a real
SX subsystem `lib/relations` — children/parents reads + relate/unrelate writes
map straight onto its public API); `likes` stays proxied (no SX lib to dispatch
to). NEXT: migrate the `relations` read endpoints onto host handlers (flip their
ledger status to `:migrated`) with golden tests.
- **Phase 3 — relations READ cut-over (DONE, 121/121).** `lib/host/relations.sx`
+ a 13-test golden suite; ledger flipped (off-Quart coverage 27% → 45%). The two
internal read queries (`get-children`, `get-parents`) now dispatch to the
`lib/relations` Datalog graph. Bridge: the Quart `(type, id)` node key maps to a
graph atom `(string->symbol "type:id")` with relation-type as the edge kind;
optional `child-type`/`parent-type` params filter the result list by `"type:"`
prefix (verified live: composite-string nodes round-trip through
`relations/relate``relations/children`). Golden discipline: `relations` is
internal-only (no public Quart route — confirmed via `svc_routes`), so the golden
is a **pinned fixture** (a known graph loaded in-test, asserted as
`subsystem-call + envelope`) rather than a live Quart capture. Reads are
unguarded for now — the signed-internal-auth gate is a separate middleware layer,
same as the feed reads. NEXT: relations WRITE actions (`relate`/`unrelate`)
behind the auth+ACL pipeline (mirroring POST /feed).
- **Phase 3 — relations WRITE cut-over (DONE, 132/132).** `lib/host/relations.sx`
gains `host/relations-attach`/`-detach` (`POST .../attach-child` + `/detach-child`)
and `host/relations-write-routes` — the write side of the container reads,
dispatching to `relations/relate`/`unrelate` over the same `"type:id"` node
model so an attach is immediately visible through `get-children`. Each runs
behind the host pipeline `wrap-errors ∘ require-auth ∘ require-permission`
(`"relate"`/`"unrelate"` on `"relations"`) — exactly the POST /feed stack. The
relations test suite proves the closed loop end-to-end: 401 unauth, 403 authed-
but-unpermitted (graph unchanged), 201 attach → child visible via the migrated
read → 200 detach → child gone; 400 on bad/short payloads. The ledger now models
the full relations surface (7 endpoints): container reads+writes `:migrated`,
typed `relate`/`unrelate`/`can-relate` `:proxied` (registry/cardinality
validation not in lib/relations). Off-Quart coverage 45% → **50%** (7/14).
`relations` is the first whole *coherent feature* (container relations) fully
off Quart. NEXT: golden-response harness vs live Quart, then survey the next
domain (blog/likes proxied — likes needs an SX subsystem first).
- **Phase 4 — live wiring bridge (DONE, 145/145).** `lib/host/server.sx` adapts the
native `http-listen` contract (string-keyed req `{"method" "path" "query"
"headers" "body"}``{:status :headers :body}`) to the Dream app: `host/-native
->dream` reassembles `path`+`query` into a target `dream-request` parses;
`host/-dream->native` is near-identity (dream-response is already `{:body
:headers :status}`). `host/serve port groups` = `http-listen` over
`host/native-handler (host/make-app groups)`. `lib/host/serve.sh` boots the full
module set (mirrors conformance) and serves in the foreground (container-entry
shaped). **Verified live** on a host port: `/health` 200 JSON, `/feed` recent-
first seeded activities, `/feed?actor=` filtered, relations `get-children`/`get-
parents` real JSON, unknown→404. Demo run was a standalone `sx_server.exe`
process (NOT the docker stack) — killed by its own PID, never `pkill` (siblings
share the binary). The standing "live wiring is a hosts/ change" Blocker is
resolved for the SX side: the bridge is pure SX in `lib/host`; only the *launch*
(docker stack + Caddy) remains. NEXT: golden harness, internal-HMAC, then promote
into the stack behind a fresh subdomain.
## SX gotchas + how this loop guards against them
The SX dev experience has real footguns. Most are statically detectable; the
tools exist (`sx_validate`, `deps-check`, `sx_format_check`) but must be *gated*.
Hit/relevant here:
- **Reserved-name shadowing** — `guard`/`bind`/`conj`/`disj` are special forms or
host primitives; a local binding of that name is silently shadowed by the form.
(`(let ((guard ...)))` made `(guard handler)` invoke the R7RS `guard` special
form → `first: expected list`.) Fix: namespace-prefix every helper
(`host/blog--protect`, never `guard`).
- **Silent test truncation** — a test file that errors mid-load returns only the
tests that ran before the error, reporting a FALSE GREEN ("blog 13 passed, 0
failed" while 16 CRUD tests never ran). **GUARDED**: `conformance.sh` now greps
the run output for `Undefined symbol` / `Unhandled exception` / `expected list,
got` / `[load] … error` and aborts loudly before the tally can hide it.
- **`let` is parallel** (bindings can't see each other), **bodies need `(do …)`**
(only the last expr evaluates), **`append!` no-ops on map/rest-derived lists**,
**parsed keyword tokens ≠ string literals**. These produce wrong *results*, so
test coverage catches them as red (not silent) — provided the runner is honest,
which the truncation guard now ensures.
Prevention ladder: parse (`sx_validate` after every edit) → unresolved/shadowed
symbols (`deps-check`, candidate pre-commit gate) → fail-loud runner (done) →
behavioural tests. A `deps-check`-style "binding shadows a special form" lint
would catch the reserved-name class before runtime — a worthwhile follow-up.
## ⚠ Experimental: unguarded create live on blog.rose-ash.com
`host/blog-open-create-routes` mounts **`POST /new` with NO auth** (create-only,
error-trapped) so the SX editor can publish end-to-end. **Validated live**: an
editor-style form POST → 303 → the post renders at `/<slug>/` and lists on `/`.
This is a deliberate, short-lived public write hole (create-only — no PUT/DELETE
exposed; obscure subdomain). **MUST be gated before real use** — Caddy basicauth
on `/new` (the `/root/caddy/auth` dir exists) or session auth once identity lands.
Swap `host/blog-open-create-routes``host/blog-write-routes <resolver>` to gate.
## Blockers
- **Live wiring to the native OCaml HTTP server** (Phase 3/4): the prod server in
`hosts/` must hand SX handlers a `dream-request` dict and serialise the returned
`dream-response`. That is a `hosts/` change (out of scope for this loop, which is
`lib/host/**` only). Until then, endpoints are verified via `conformance.sh`, not
HTTP. Not blocking Phase 2 (middleware + SXTP + a write endpoint).
- **Worktree tooling:** in this `loops/host` worktree every sx-tree *write* tool
(`sx_write_file`, `sx_replace_node`, …) raises `yojson "Expected string, got
null"` at the MCP layer — same class as the `loops/dream` worktree gotcha, but
here even `sx_write_file` fails. Read-side sx-tree tools work. New `.sx` files
were created with the `Write` tool (the .sx hook is inactive in this worktree)
and each validated afterwards with `sx_validate` to keep the parse guarantee.