giles
05c0a0b01a
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 22s
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
122 lines
7.3 KiB
Markdown
122 lines
7.3 KiB
Markdown
# Host blog → SPA via the SX-htmx engine (WASM OCaml kernel)
|
|
|
|
Turn the blog (lib/host/blog.sx) into a single-page app using the in-repo SX
|
|
hypermedia engine (web/engine.sx — "our htmx"): boot the **WASM OCaml kernel**
|
|
(the same evaluator the server runs) in the browser, and `sx-boost` every
|
|
link/form into a fragment swap into `#content` — no full reloads, history kept,
|
|
graceful degradation to plain server-rendered pages with no JS.
|
|
|
|
## Status
|
|
|
|
**DONE — server side (verified, all green):**
|
|
- `lib/host/static.sx` — `GET /static/**` serves files under `shared/static` via
|
|
the `file-read` primitive (content-type by extension, path-traversal guarded,
|
|
404 on missing). Mounted in serve.sh + the route list. Tested: kernel JS 200 +
|
|
correct ctype + exact bytes; `.wasm` binary-exact with `application/wasm`;
|
|
traversal/missing → 404.
|
|
- `lib/host/blog.sx` `host/blog--page` is now the SPA shell: full page = WASM boot
|
|
scripts (`/static/wasm/sx_browser.bc.wasm.js` + `sx-platform.js`) + a
|
|
`sx-boost="#content"` wrapper div + `#content`. On the `SX-Request: true` header
|
|
(a boosted nav) it returns ONLY the inner content (fragment) so the engine swaps
|
|
it into `#content`. All 13 page handlers thread `req`. Tested: full page carries
|
|
scripts+boost+#content; `SX-Request` returns the bare fragment.
|
|
- `docker-compose.dev-sx-host.yml` mounts `./shared/static` so the live container
|
|
can serve the kernel.
|
|
- `lib/host/playwright/spa-check.spec.js` + `run-spa-check.sh` — browser check
|
|
(boot, boost, fragment swap, back button).
|
|
|
|
**DONE — client side, partial:**
|
|
- The WASM kernel BOOTS in a headless browser: `globalThis.SxKernel` is an object,
|
|
`<html data-sx-ready="true">` is set, the web-stack modules load.
|
|
- Fixed: this worktree's `shared/static/wasm/sx_browser.bc.wasm.assets/` was
|
|
missing 5 of 11 `.wasm` units (`sx-`, `unix-`, `re-`, `start-`,
|
|
`dune__exe__Sx_browser-`); copied the complete set from the main worktree.
|
|
|
|
**BLOCKER — boost does not activate (`boosted links: 0 / N`):**
|
|
- The bundled `.sxbc` bytecode throws `VM: unknown opcode 0` against this
|
|
worktree's `sx_browser.bc.wasm.js` kernel, so sx-platform.js falls back to `.sx`
|
|
source for every web-stack module. Source fallback works for all modules EXCEPT
|
|
`boot.sx`, which then fails with `Expected list, got string` — so the boot
|
|
sequence that wires `process-elements → process-boosted` doesn't complete and no
|
|
link gets `_sxBoundboost`.
|
|
- Root cause: the `.sxbc` in `shared/static/wasm/sx/` are out of sync with the
|
|
WASM kernel (sx.rose-ash.com avoids this because its Docker image ships a
|
|
consistent bundle and it navigates via client-router page-routes, not boost).
|
|
|
|
## UPDATE 2026-06-29 — kernel BOOT crash fixed (crypto WASM-safe)
|
|
|
|
The boot crash was NOT the build pipeline — it was the kernel's crypto stack
|
|
assuming 63-bit native int. On the web targets (js_of_ocaml 32-bit, wasm_of_ocaml
|
|
31-bit) sha2/cbor/cid/ed25519 truncated, and ed25519 precomputes `sqrtm1` +
|
|
`base_point` AT MODULE INIT via a base-2^26 bignum whose 52-bit products overflow
|
|
→ `Char.chr(-4)` crash on load. Fixed in `fce9e0c6` (sx_sha2 Int32 rounds +
|
|
Int64 length, sx_cbor Int64 width-select, sx_cid bounded base32, sx_ed25519 Int64
|
|
bignum mul/div_small). Verified: NIST/CID vectors match native↔js↔wasm; native
|
|
conformance 271/271; **the freshly-built browser kernel now BOOTS** (SxKernel
|
|
live, data-sx-ready=true, crypto-sha256 correct on js + wasm).
|
|
|
|
REMAINING for boost (separate layer — web-stack loading, NOT crypto). Two
|
|
compounding roots, both fully diagnosed:
|
|
|
|
1. **`.sxbc` carry NIL bytecode.** `compile-modules.js` (via the native binary)
|
|
emits `:bytecode (nil nil nil …)` placeholders, not real bytecode — so the
|
|
SX-level `vm.sx` interpreter reads nil → `VM: unknown opcode 0`, and the web
|
|
stack falls back to `.sx` source for every module. (Confirmed by inspecting a
|
|
freshly-compiled `dom.sxbc`.) The native compiler isn't producing bytecode in
|
|
this path.
|
|
|
|
2. **Source-fallback can't resolve manifest-mapped libraries.** With imports
|
|
stripped, all 23 `boot.sx` body forms load clean — the `Expected list, got
|
|
string` is from an `import`. `boot.sx` imports `(sx signals-web)`, but that
|
|
library is *defined inside `signals.sx`* (the file→library names don't match;
|
|
the module-manifest maps `"sx signals-web" → signals.sxbc`). The `.sx`
|
|
source-fallback resolver maps a library to a like-named FILE, looks for a
|
|
non-existent `signals-web.sx`, and the failed resolution returns a string into
|
|
a list op → the error → `boot.sx` never loads → `process-boosted` never runs →
|
|
boost 0/N. (A `signals-web.sx` bridge that imports signals was NOT sufficient
|
|
— there is at least one more such mismatch among the imports.)
|
|
|
|
THE CLEAN FIX is a proper bundle rebuild via `scripts/sx-build-all.sh` so the
|
|
`.sxbc` carry real bytecode and the manifest-driven path loads everything (no
|
|
source fallback, so root #2 never triggers) — gated on fixing root #1 (why
|
|
`compile-modules.js` emits nil bytecode). Alternatively, make the source-fallback
|
|
resolver manifest-aware. Neither is a quick edit; it's a web-stack build-tooling
|
|
sub-project. The kernel itself is now correct and boots.
|
|
|
|
## Rebuild attempt (2026-06-28) — FAILED, reverted (superseded by the fix above)
|
|
|
|
Tried it: `dune build browser/sx_browser.bc.wasm.js` succeeded (with many
|
|
`integer-overflow` warnings — "generated code might be incorrect"), and
|
|
`node hosts/ocaml/browser/compile-modules.js shared/static/wasm` recompiled all
|
|
35 `.sxbc` cleanly. But the freshly-built kernel **crashes on init** in the
|
|
browser: `Fatal error: exception Invalid_argument("Char.chr")` — so `SxKernel`
|
|
never initialises (worse than before). The integer-overflow truncation during
|
|
wasm codegen is the likely culprit (a SHA/char constant). Reverted
|
|
`shared/static/wasm/` to the main-worktree bundle (which boots cleanly —
|
|
verified SxKernel + data-sx-ready). So a naive in-worktree rebuild is NOT the
|
|
fix; the wasm build itself needs investigating (wasm_of_ocaml version? the merged
|
|
sx-vm-extensions/resolver changes interacting with codegen?).
|
|
|
|
## Next step — rebuild a consistent WASM bundle
|
|
|
|
`scripts/sx-build-all.sh` does: build the browser wasm target → sync web `.sx`
|
|
into `hosts/ocaml/browser/dist/sx/` → `node hosts/ocaml/browser/compile-modules.js`
|
|
(recompiles `.sxbc` via the native sx_server binary) → copy into
|
|
`shared/static/wasm/`. The browser wasm target is NOT built in this worktree
|
|
(`hosts/ocaml/_build/default/browser/` is empty), so this needs the
|
|
`wasm_of_ocaml` toolchain set up first. Once the `.sxbc` match the kernel, the
|
|
bytecode path loads (no source fallback), `boot.sx` runs, and `process-boosted`
|
|
binds the links — then the SPA Playwright check should pass.
|
|
|
|
Alternatively: build the browser kernel in the main worktree (which has the
|
|
pipeline) and copy a consistent `sx_browser.bc.wasm.js` + assets + `.sxbc` set
|
|
into this worktree's `shared/static/wasm/`.
|
|
|
|
## Deploy note
|
|
|
|
The live container is NOT redeployed with the SPA shell yet — it keeps running the
|
|
pre-SPA `blog.sx` in memory (the native host doesn't hot-reload). Don't recreate
|
|
the container until the bundle is consistent and the SPA Playwright check is green,
|
|
to avoid shipping a kernel that boots but doesn't boost. (Even if it is recreated,
|
|
pages degrade gracefully: links still do normal full-page nav.)
|