From dbcbc39ebe0289a13cf3885d0b0e2315d542b818 Mon Sep 17 00:00:00 2001 From: giles Date: Sun, 28 Jun 2026 20:53:06 +0000 Subject: [PATCH] =?UTF-8?q?host:=20blog=20SPA=20scaffolding=20(WASM=20kern?= =?UTF-8?q?el)=20=E2=80=94=20server=20side=20complete,=20boost=20blocked?= =?UTF-8?q?=20on=20bundle=20rebuild?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Turn the blog into a SPA using the SX-htmx engine (web/engine.sx) booting the WASM OCaml kernel (same evaluator as the server) in-browser, with sx-boost fragment-swapping every link into #content. Server side DONE + verified: - lib/host/static.sx: GET /static/** serves shared/static via the file-read primitive (ctype by ext, traversal-guarded, 404 on missing). Wired into serve.sh (module + route group). Tested: kernel JS + .wasm binary-exact. - host/blog--page is now the SPA shell: full page = WASM boot scripts + sx-boost=#content wrapper + #content; on SX-Request:true returns ONLY the inner content fragment for the engine to swap. All 13 handlers thread req. - docker-compose mounts ./shared/static. - lib/host/playwright/spa-check.{spec.js,run-spa-check.sh}: boot/boost/swap/back. Client side: the WASM kernel BOOTS (SxKernel object, data-sx-ready=true, web stack loads). BLOCKER: the bundled .sxbc throw 'VM: unknown opcode 0' vs this worktree's kernel -> .sx source fallback -> boot.sx source fails 'Expected list, got string' -> process-boosted never binds links (boosted 0/N). Fix = rebuild a consistent WASM bundle (recompile .sxbc against the kernel via scripts/sx-build-all.sh); the browser wasm target isn't built here yet. See plans/host-spa.md. Live NOT redeployed (stays on pre-SPA process). Co-Authored-By: Claude Opus 4.8 --- docker-compose.dev-sx-host.yml | 3 ++ lib/host/blog.sx | 61 ++++++++++++++++-------- lib/host/playwright/run-spa-check.sh | 68 +++++++++++++++++++++++++++ lib/host/playwright/spa-check.spec.js | 65 +++++++++++++++++++++++++ lib/host/serve.sh | 3 +- lib/host/static.sx | 52 ++++++++++++++++++++ plans/host-spa.md | 67 ++++++++++++++++++++++++++ 7 files changed, 298 insertions(+), 21 deletions(-) create mode 100644 lib/host/playwright/run-spa-check.sh create mode 100644 lib/host/playwright/spa-check.spec.js create mode 100644 lib/host/static.sx create mode 100644 plans/host-spa.md diff --git a/docker-compose.dev-sx-host.yml b/docker-compose.dev-sx-host.yml index 0e30e483..9aad03ec 100644 --- a/docker-compose.dev-sx-host.yml +++ b/docker-compose.dev-sx-host.yml @@ -39,6 +39,9 @@ services: - ./spec:/app/spec:ro - ./lib:/app/lib:ro - ./web:/app/web:ro + # Client assets for the blog SPA: the WASM OCaml kernel + sx-platform + the + # web-stack modules, served by lib/host/static.sx at /static/**. + - ./shared/static:/app/shared/static:ro # OCaml server binary — this worktree's build (has the SX_HTTP_HOST bind fix) - ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro # Durable persist store (the SX op-log/kv on disk) — survives restarts. diff --git a/lib/host/blog.sx b/lib/host/blog.sx index 222e54f7..8a668999 100644 --- a/lib/host/blog.sx +++ b/lib/host/blog.sx @@ -404,14 +404,35 @@ ;; builds the tree (running any dynamic logic in the full evaluator, e.g. a posts ;; loop) and render-page renders the static result — no embedded HTML strings, ;; only the doctype prefix render-to-html doesn't emit. `body` is an SX node. +;; SPA shell. The blog is a single-page app: the page boots the WASM OCaml kernel +;; (the SAME evaluator as the server) + the SX-htmx engine (web/engine.sx), and +;; `sx-boost="#content"` turns every in-page link/form into a fragment swap into +;; #content — no full reloads, history handled. A boosted request carries the +;; SX-Request:true header; we then return ONLY the inner content (so the engine +;; swaps it straight into #content). A direct / no-JS request gets the full shell, +;; so the blog degrades gracefully to plain server-rendered pages. +(define host/blog--spa-req? (fn (req) (= (dream-header req "sx-request") "true"))) + (define host/blog--page - (fn (title body) - (str "" - (render-page - (quasiquote - (html - (head (meta :charset "utf-8") (title (unquote title))) - (body (unquote body)))))))) + (fn (req title body) + (if (host/blog--spa-req? req) + ;; fragment: inner content only — engine swaps it into #content + (render-page body) + ;; full SPA shell: WASM kernel + platform + boosted #content + (str "" + (render-page + (quasiquote + (html + (head (meta :charset "utf-8") (title (unquote title)) + (script :src "/static/wasm/sx_browser.bc.wasm.js") + (script :src "/static/wasm/sx-platform.js")) + (body + ;; sx-boost must be on a DESCENDANT of (process-boosted + ;; queries [sx-boost] WITHIN the body, so it can't sit on body + ;; itself). The wrapper boosts every link/form inside, targeting + ;; #content; #content is the swap target. + (div :sx-boost "#content" + (div :id "content" (unquote body))))))))))) ;; ── registry-driven relation rendering (post page) ────────────────── ;; One labelled block of links from records ({:slug :title}), or "" when empty. @@ -576,7 +597,7 @@ (relations (host/blog--relations-or-hint slug (not (nil? principal)))) (auth-foot (host/auth-footer req))) (dream-html - (host/blog--page (get r :title) + (host/blog--page req (get r :title) (quasiquote (div (article (raw! (unquote body-html))) @@ -590,7 +611,7 @@ " · " (unquote auth-foot)))))))) (dream-html-status 404 - (host/blog--page "Not found" + (host/blog--page req "Not found" (quasiquote (div (h1 "404") (p (unquote (str "No published post: " slug)))))))))))) @@ -612,7 +633,7 @@ ;; quasiquote (a perform during tree-build raises VmSuspended). (auth-foot (host/auth-footer req))) (dream-html - (host/blog--page "Blog" + (host/blog--page req "Blog" (quasiquote (div (h1 "Posts") (unquote listing) @@ -637,7 +658,7 @@ (unquote (get p :title)))))) recs))) (dream-html - (host/blog--page "Tags" + (host/blog--page req "Tags" (quasiquote (div (h1 "Tags") (unquote (if (> (len recs) 0) @@ -657,7 +678,7 @@ (dream-response 200 {:content-type "text/plain; charset=utf-8"} (or (get r :sx-content) "")) (dream-html-status 404 - (host/blog--page "Not found" + (host/blog--page req "Not found" (quasiquote (div (h1 "404") (p (unquote (str "No post: " slug)))))))))))) ;; ── create page (GET /new) — clean minimal form as an SX tree ─────── @@ -666,7 +687,7 @@ (define host/blog-new-form (fn (req) (dream-html - (host/blog--page "New post" + (host/blog--page req "New post" (quasiquote (div (h1 "New post") @@ -706,12 +727,12 @@ (cond ((or (nil? title) (= title "")) (dream-html-status 400 - (host/blog--page "Error" + (host/blog--page req "Error" (quasiquote (div (h1 "Error") (p "Title is required.") (p (a :href "/new" "Back"))))))) ((not (host/blog-content-ok? sx-content)) (dream-html-status 400 - (host/blog--page "Error" + (host/blog--page req "Error" (quasiquote (div (h1 "Error") (p "Post body is not valid SX markup.") (p (a :href "/new" "Back"))))))) (else @@ -793,7 +814,7 @@ (kind (or (dream-form-field req "kind") "related"))) (if (nil? (host/blog-get slug)) (dream-html-status 404 - (host/blog--page "Not found" + (host/blog--page req "Not found" (quasiquote (div (h1 "404") (p (unquote (str "No post: " slug))))))) (begin (when (and other (not (= other "")) (not (= other slug)) @@ -823,7 +844,7 @@ (let ((r (host/blog-get slug))) (if (nil? r) (dream-html-status 404 - (host/blog--page "Not found" + (host/blog--page req "Not found" (quasiquote (div (h1 "404") (p (unquote (str "No post: " slug))))))) (let ((status (get r :status))) ;; the relation editors + tag toggle do durable reads — compute them @@ -836,7 +857,7 @@ (quasiquote (option :value (unquote val) :selected "selected" (unquote label))) (quasiquote (option :value (unquote val) (unquote label))))))) (dream-html - (host/blog--page (str "Edit: " (get r :title)) + (host/blog--page req (str "Edit: " (get r :title)) (quasiquote (div (h1 (unquote (str "Edit: " (get r :title)))) @@ -868,7 +889,7 @@ (let ((slug (dream-param req "slug")) (r (host/blog-get (dream-param req "slug")))) (if (nil? r) (dream-html-status 404 - (host/blog--page "Not found" + (host/blog--page req "Not found" (quasiquote (div (h1 "404") (p (unquote (str "No post: " slug))))))) (let ((title (or (dream-form-field req "title") (get r :title))) (sx-content (or (dream-form-field req "sx_content") "")) @@ -884,7 +905,7 @@ (dream-redirect (str "/" slug "/"))) (let ((issue-items (map (fn (i) (quasiquote (li (unquote i)))) issues))) (dream-html-status 400 - (host/blog--page "Cannot save" + (host/blog--page req "Cannot save" (quasiquote (div (h1 "Cannot save") (p "This post can't be saved yet:") diff --git a/lib/host/playwright/run-spa-check.sh b/lib/host/playwright/run-spa-check.sh new file mode 100644 index 00000000..ff3044e3 --- /dev/null +++ b/lib/host/playwright/run-spa-check.sh @@ -0,0 +1,68 @@ +#!/usr/bin/env bash +# Browser check for the blog SPA. Spins up an EPHEMERAL host server (this +# worktree's binary + lib, a temp persist dir), seeds a couple of posts, runs +# lib/host/playwright/spa-check.spec.js in the main worktree's Playwright, then +# tears everything down. Verifies the WASM OCaml kernel boots in-browser and +# sx-boost turns the blog into a SPA. No live-site dependency. +# +# bash lib/host/playwright/run-spa-check.sh +# +# Requires: the OCaml binary built (hosts/ocaml/_build/default/bin/sx_server.exe) +# and Playwright + chromium in /root/rose-ash (the architecture worktree). +set -uo pipefail +cd "$(git rev-parse --show-toplevel)" +ROOT=$(pwd) + +PORT="${SPA_PORT:-8914}" +PW_DIR="${PW_DIR:-/root/rose-ash}" # worktree that has node_modules + chromium +USER="admin" +PASS="spa-check-pw" +SECRET="spa-check-secret" +PDIR=$(mktemp -d) +JAR=$(mktemp) +SPEC_SRC="lib/host/playwright/spa-check.spec.js" +SPEC_DST="$PW_DIR/tests/playwright/_spa-check.spec.js" +SERVE_LOG=$(mktemp) + +cleanup() { + [ -n "${SVPID:-}" ] && kill "$SVPID" 2>/dev/null + local pid + pid=$(ss -lptn "sport = :$PORT" 2>/dev/null | grep -oE 'pid=[0-9]+' | head -1 | cut -d= -f2) + [ -n "$pid" ] && kill "$pid" 2>/dev/null + rm -f "$SPEC_DST" "$JAR" "$SERVE_LOG" + rm -rf "$PDIR" +} +trap cleanup EXIT + +echo "== starting ephemeral host server on :$PORT (persist=$PDIR) ==" +HOST_PORT="$PORT" SX_PERSIST_DIR="$PDIR" \ + SX_ADMIN_USER="$USER" SX_ADMIN_PASSWORD="$PASS" SX_SESSION_SECRET="$SECRET" \ + bash lib/host/serve.sh >"$SERVE_LOG" 2>&1 & +SVPID=$! + +for i in $(seq 1 60); do + curl -sf -o /dev/null "http://127.0.0.1:$PORT/health" 2>/dev/null && break + sleep 1 + [ "$i" = "60" ] && { echo "server never came up:"; cat "$SERVE_LOG"; exit 1; } +done +echo "== server up ==" + +echo "== seeding posts ==" +curl -s -c "$JAR" -o /dev/null -X POST "http://127.0.0.1:$PORT/login" \ + --data "username=$USER&password=$PASS" +for t in "Alpha Post" "Beta Post"; do + curl -s -b "$JAR" -o /dev/null -X POST "http://127.0.0.1:$PORT/new" \ + --data "title=$t&sx_content=(article (h1 \"$t\") (p \"body\"))&status=published" +done +echo "== seeded ($(curl -s "http://127.0.0.1:$PORT/posts" | grep -o '"slug"' | wc -l) posts) ==" + +echo "== running Playwright ==" +cp "$ROOT/$SPEC_SRC" "$SPEC_DST" +cd "$PW_DIR" +SX_TEST_URL="http://127.0.0.1:$PORT" \ + node_modules/.bin/playwright test _spa-check.spec.js --workers=1 \ + --config tests/playwright/playwright.config.js +RC=$? + +echo "== done (exit $RC) ==" +exit $RC diff --git a/lib/host/playwright/spa-check.spec.js b/lib/host/playwright/spa-check.spec.js new file mode 100644 index 00000000..428481c7 --- /dev/null +++ b/lib/host/playwright/spa-check.spec.js @@ -0,0 +1,65 @@ +// Browser check for the blog SPA (lib/host/blog.sx + lib/host/static.sx). Runs +// against an ephemeral host server seeded with a couple of posts by +// run-spa-check.sh, which copies this spec into the Playwright env and sets +// SX_TEST_URL. Verifies the WASM OCaml kernel boots in the browser, the SX-htmx +// engine activates sx-boost on #content's links, and clicking a link does a +// fragment swap (no full page reload) with history — i.e. it's a real SPA. +const { test, expect } = require('playwright/test'); + +// boot-init sets data-sx-ready="true" on once the WASM kernel + web stack +// have loaded and the page has been processed. WASM compile + ~25 asset fetches, +// so allow generous time. +async function waitReady(page) { + await expect(page.locator('html[data-sx-ready="true"]')).toHaveCount(1, { timeout: 45000 }); +} + +// a post link in the listing (trailing slash); skip /new, /login, /tags. +const POSTLINK = '#content a[href$="/"]'; + +test.describe('blog SPA', () => { + test('WASM kernel boots and marks the document ready', async ({ page }) => { + const errors = []; + page.on('console', (m) => { if (m.type() === 'error') errors.push(m.text()); }); + page.on('pageerror', (e) => errors.push(String(e))); + await page.goto('/'); + await waitReady(page); + // the shell shipped the WASM loaders + expect(await page.locator('script[src*="sx_browser.bc.wasm.js"]').count()).toBe(1); + expect(await page.locator('script[src*="sx-platform.js"]').count()).toBe(1); + // no boot-time JS errors + expect(errors, errors.join('\n')).toEqual([]); + }); + + test('links inside #content get boosted', async ({ page }) => { + await page.goto('/'); + await waitReady(page); + // the engine marks a boosted element with data-sx-bound containing "boost" + await expect(page.locator(POSTLINK).first()).toHaveAttribute('data-sx-bound', /boost/, { timeout: 15000 }); + }); + + test('clicking a link does a fragment swap — no full reload, URL updates', async ({ page }) => { + await page.goto('/'); + await waitReady(page); + // sentinel survives ONLY if there is no full-page reload + await page.evaluate(() => { window.__noReload = true; }); + const link = page.locator(POSTLINK).first(); + const href = await link.getAttribute('href'); + await link.click(); + await page.waitForURL((u) => u.pathname === href, { timeout: 15000 }); + expect(await page.evaluate(() => window.__noReload)).toBe(true); // no reload + // content was swapped into #content (a post page carries the post footer) + await expect(page.locator('#content')).toContainText(/all posts/i, { timeout: 15000 }); + }); + + test('back button restores the listing', async ({ page }) => { + await page.goto('/'); + await waitReady(page); + const link = page.locator(POSTLINK).first(); + const href = await link.getAttribute('href'); + await link.click(); + await page.waitForURL((u) => u.pathname === href, { timeout: 15000 }); + await page.goBack(); + await page.waitForURL((u) => u.pathname === '/', { timeout: 15000 }); + await expect(page.locator('#content h1')).toContainText('Posts'); + }); +}); diff --git a/lib/host/serve.sh b/lib/host/serve.sh index 472bc090..21f0b302 100755 --- a/lib/host/serve.sh +++ b/lib/host/serve.sh @@ -81,6 +81,7 @@ MODULES=( "lib/host/auth.sx" "lib/host/sxtp.sx" "lib/host/router.sx" + "lib/host/static.sx" "lib/host/feed.sx" "lib/host/relations.sx" "lib/host/blog.sx" @@ -157,5 +158,5 @@ EPOCH=1 # middleware, so a browser logs in then publishes. The bearer resolver is a stub # (no API tokens configured) — browser session is the live auth path for now. # blog-routes LAST — its GET /:slug catch-all must not shadow the rest. - echo "(eval \"(host/serve $PORT (list host/feed-routes host/relations-routes (host/blog-write-routes (fn (tok) nil)) host/blog-routes))\")" + echo "(eval \"(host/serve $PORT (list host/static-routes host/feed-routes host/relations-routes (host/blog-write-routes (fn (tok) nil)) host/blog-routes))\")" } | exec "$SX_SERVER" diff --git a/lib/host/static.sx b/lib/host/static.sx new file mode 100644 index 00000000..eb176bb6 --- /dev/null +++ b/lib/host/static.sx @@ -0,0 +1,52 @@ +;; lib/host/static.sx — serve the client kernel + assets so the blog can boot the +;; SX-htmx hypermedia engine (web/engine.sx) and run as a SPA. The native +;; http-listen host reads files with the `file-read` primitive (no perform), so +;; GET /static/** maps to a file under the static root (default "shared/static", +;; resolved against the server cwd — mount ./shared/static there in the container). +;; Depends on lib/dream/types.sx (dream-response/-html-status/-param) + router. + +(define host/static-root "shared/static") +(define host/static-use-root! (fn (r) (set! host/static-root r))) + +;; content-type by file extension; default to octet-stream. +(define host/static--ctype + (fn (path) + (cond + ((ends-with? path ".js") "application/javascript; charset=utf-8") + ((ends-with? path ".mjs") "application/javascript; charset=utf-8") + ((ends-with? path ".css") "text/css; charset=utf-8") + ((ends-with? path ".json") "application/json; charset=utf-8") + ((ends-with? path ".map") "application/json; charset=utf-8") + ((ends-with? path ".svg") "image/svg+xml") + ((ends-with? path ".png") "image/png") + ((ends-with? path ".woff2") "font/woff2") + ((ends-with? path ".wasm") "application/wasm") + (true "application/octet-stream")))) + +;; reject empty, absolute, or traversal paths. +(define host/static--safe? + (fn (rel) + (and (> (len rel) 0) + (not (starts-with? rel "/")) + (not (string-contains? rel ".."))))) + +;; Serve one asset by its path relative to the static root. file-read THROWS on a +;; missing file, so gate on file-exists? first and return a 404 instead. +(define host/static-serve + (fn (rel) + (if (not (host/static--safe? rel)) + (dream-html-status 403 "Forbidden") + (let ((path (str host/static-root "/" rel))) + (if (not (file-exists? path)) + (dream-html-status 404 "Not Found") + (dream-response 200 + {:content-type (host/static--ctype rel) + :cache-control "public, max-age=3600"} + (file-read path))))))) + +;; Route group: GET /static/** -> file under the static root. A plain route LIST +;; (like host/feed-routes); host/serve combines + flattens the groups itself. +(define host/static-routes + (list + (dream-get "/static/**" + (fn (req) (host/static-serve (dream-param req "**")))))) diff --git a/plans/host-spa.md b/plans/host-spa.md new file mode 100644 index 00000000..c7533fd9 --- /dev/null +++ b/plans/host-spa.md @@ -0,0 +1,67 @@ +# Host blog → SPA via the SX-htmx engine (WASM OCaml kernel) + +Turn the blog (lib/host/blog.sx) into a single-page app using the in-repo SX +hypermedia engine (web/engine.sx — "our htmx"): boot the **WASM OCaml kernel** +(the same evaluator the server runs) in the browser, and `sx-boost` every +link/form into a fragment swap into `#content` — no full reloads, history kept, +graceful degradation to plain server-rendered pages with no JS. + +## Status + +**DONE — server side (verified, all green):** +- `lib/host/static.sx` — `GET /static/**` serves files under `shared/static` via + the `file-read` primitive (content-type by extension, path-traversal guarded, + 404 on missing). Mounted in serve.sh + the route list. Tested: kernel JS 200 + + correct ctype + exact bytes; `.wasm` binary-exact with `application/wasm`; + traversal/missing → 404. +- `lib/host/blog.sx` `host/blog--page` is now the SPA shell: full page = WASM boot + scripts (`/static/wasm/sx_browser.bc.wasm.js` + `sx-platform.js`) + a + `sx-boost="#content"` wrapper div + `#content`. On the `SX-Request: true` header + (a boosted nav) it returns ONLY the inner content (fragment) so the engine swaps + it into `#content`. All 13 page handlers thread `req`. Tested: full page carries + scripts+boost+#content; `SX-Request` returns the bare fragment. +- `docker-compose.dev-sx-host.yml` mounts `./shared/static` so the live container + can serve the kernel. +- `lib/host/playwright/spa-check.spec.js` + `run-spa-check.sh` — browser check + (boot, boost, fragment swap, back button). + +**DONE — client side, partial:** +- The WASM kernel BOOTS in a headless browser: `globalThis.SxKernel` is an object, + `` is set, the web-stack modules load. +- Fixed: this worktree's `shared/static/wasm/sx_browser.bc.wasm.assets/` was + missing 5 of 11 `.wasm` units (`sx-`, `unix-`, `re-`, `start-`, + `dune__exe__Sx_browser-`); copied the complete set from the main worktree. + +**BLOCKER — boost does not activate (`boosted links: 0 / N`):** +- The bundled `.sxbc` bytecode throws `VM: unknown opcode 0` against this + worktree's `sx_browser.bc.wasm.js` kernel, so sx-platform.js falls back to `.sx` + source for every web-stack module. Source fallback works for all modules EXCEPT + `boot.sx`, which then fails with `Expected list, got string` — so the boot + sequence that wires `process-elements → process-boosted` doesn't complete and no + link gets `_sxBoundboost`. +- Root cause: the `.sxbc` in `shared/static/wasm/sx/` are out of sync with the + WASM kernel (sx.rose-ash.com avoids this because its Docker image ships a + consistent bundle and it navigates via client-router page-routes, not boost). + +## Next step — rebuild a consistent WASM bundle + +`scripts/sx-build-all.sh` does: build the browser wasm target → sync web `.sx` +into `hosts/ocaml/browser/dist/sx/` → `node hosts/ocaml/browser/compile-modules.js` +(recompiles `.sxbc` via the native sx_server binary) → copy into +`shared/static/wasm/`. The browser wasm target is NOT built in this worktree +(`hosts/ocaml/_build/default/browser/` is empty), so this needs the +`wasm_of_ocaml` toolchain set up first. Once the `.sxbc` match the kernel, the +bytecode path loads (no source fallback), `boot.sx` runs, and `process-boosted` +binds the links — then the SPA Playwright check should pass. + +Alternatively: build the browser kernel in the main worktree (which has the +pipeline) and copy a consistent `sx_browser.bc.wasm.js` + assets + `.sxbc` set +into this worktree's `shared/static/wasm/`. + +## Deploy note + +The live container is NOT redeployed with the SPA shell yet — it keeps running the +pre-SPA `blog.sx` in memory (the native host doesn't hot-reload). Don't recreate +the container until the bundle is consistent and the SPA Playwright check is green, +to avoid shipping a kernel that boots but doesn't boost. (Even if it is recreated, +pages degrade gracefully: links still do normal full-page nav.)