Content-addressed on-demand loading: Merkle DAG for all browser assets

Replace the monolithic 500KB <script data-components> block with a 25KB
JSON manifest mapping names to content hashes. Every definition —
components, islands, macros, client libraries, bytecode modules, and
WASM binaries — is now content-addressed and loaded on demand.

Server (sx_server.ml):
- build_hash_index: Merkle DAG over all definitions — topological sort,
  hash leaves first, component refs become @h:{hash} in instantiated form
- /sx/h/{hash} endpoint: serves definitions with Cache-Control: immutable
- Per-page manifest in <script data-sx-manifest> with defs + modules + boot
- Client library .sx files hashed as whole units (tw.sx, tw-layout.sx, etc.)
- .sxbc modules and WASM kernel hashed individually

Browser (sx-platform.js):
- Content-addressed boot: inline script loads kernel + platform by hash
- loadDefinitionByHash: recursive dep resolution with @h: rewriting
- resolveHash: 3-tier cache (memory → localStorage → fetch /sx/h/{hash})
- __resolve-symbol extended for manifest-based component + library loading
- Cache API wrapper intercepts .wasm fetches for offline caching
- Eager pre-loading of plain symbol deps for CEK evaluator compatibility

Shell template (shell.sx):
- Monolithic <script data-components> removed
- data-sx-manifest script with full hash manifest
- Inline bootstrap replaces <script src="...?v="> with CID-based loading

Second visit loads zero bytes from network. Changed content gets a new
hash — only that item refetched (Merkle propagation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-14 10:14:39 +00:00
parent 2cba359fdf
commit 1d83ccba3c
4 changed files with 801 additions and 44 deletions

View File

@@ -7,6 +7,7 @@
(sx-css :as string?)
(component-hash :as string?)
(component-defs :as string?)
(component-manifest :as string?)
(pages-sx :as string?)
(page-sx :as string?)
(body-html :as string?)
@@ -61,11 +62,12 @@
(style
(raw!
"[data-sx-island] button,[data-sx-island] a,[data-sx-island] [role=button]{cursor:pointer}"))
(script
:type "text/sx"
:data-components true
:data-hash component-hash
(raw! (or component-defs "")))
(when
component-manifest
(script
:type "application/json"
:data-sx-manifest true
(raw! component-manifest)))
(when
init-sx
(script :type "text/sx" :data-init true (raw! init-sx)))
@@ -74,12 +76,6 @@
:type "text/sx"
:data-mount "#sx-root"
(raw! (or page-sx "")))
(<>
(script
:src (str
asset-url
"/wasm/sx_browser.bc.wasm.js?v="
(or wasm-hash "0")))
(script
:src (str asset-url "/wasm/sx-platform.js?v=" (or platform-hash "0"))
:data-sxbc-hash (or sxbc-hash "0")))))))
(script
(raw!
"\n(function(){\n var m=document.querySelector('[data-sx-manifest]');\n if(!m)return;\n var j=JSON.parse(m.textContent);\n\n // Cache API wrapper — intercept .wasm fetches for offline caching.\n if(typeof caches!=='undefined'){\n var _fetch=window.fetch;\n var CACHE='sx-wasm-v1';\n window.fetch=function(input,init){\n var url=(typeof input==='string')?input:\n (input instanceof URL)?input.href:\n (input&&input.url)||'';\n if(url.indexOf('.wasm')!==-1){\n return caches.open(CACHE).then(function(c){\n return c.match(url).then(function(r){\n if(r)return r;\n return _fetch(input,init).then(function(resp){\n if(resp.ok)c.put(url,resp.clone());\n return resp;\n });\n });\n });\n }\n return _fetch(input,init);\n };\n }\n\n // Content-addressed boot: load kernel + platform by hash\n if(!j.boot)return;\n j.boot.forEach(function(h){\n var s=document.createElement('script');\n s.src='/sx/h/'+h;\n document.head.appendChild(s);\n });\n})();\n"))))))