coop/rose-ash
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix empty code blocks: rename ~docs/code param, fix batched IO dispatch

Browse Source 
Two bugs caused code blocks to render empty across the site:

1. ~docs/code component had parameter named `code` which collided with
   the HTML <code> tag name. Renamed to `src` and updated all 57
   callers. Added font-mono class for explicit monospace.

2. Batched IO dispatch in ocaml_bridge.py only skipped one leading
   number (batch ID) but the format has two (epoch + ID):
   (io-request EPOCH ID "name" args...). Changed to skip all leading
   numbers so the string name is correctly found. This fixes highlight
   and other batchable helpers returning empty results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
giles
2026-03-25 18:08:40 +00:00
parent 739d04672b
commit 6f96452f70
58 changed files with 440 additions and 437 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
sx/sx/plans/art-dag-sx.sx
View File

@@ -23,7 +23,7 @@
(~docs/section :title "Recipes as SX" :id "recipes-as-sx"
(p "Art DAG recipes are already s-expression effect chains. Currently executed by L1 Celery workers via Python. The SX version: recipes " (em "are") " SX programs. They evaluate in an environment that has media processing primitives. A recipe doesn't \"call\" a GPU function " (em "- ") "it evaluates in an environment where " (code "gpu-exec") " is a primitive.")
(~docs/code :code (highlight ";; A recipe is an SX program in a media-processing environment\n(define composite-layers\n (fn (base-cid overlay-cid blend)\n (let ((base (resolve-cid base-cid))\n (overlay (resolve-cid overlay-cid)))\n (gpu-exec :op \"composite\"\n :layers (list base overlay)\n :blend blend\n :output :stream))))" "lisp"))
(~docs/code :src (highlight ";; A recipe is an SX program in a media-processing environment\n(define composite-layers\n (fn (base-cid overlay-cid blend)\n (let ((base (resolve-cid base-cid))\n (overlay (resolve-cid overlay-cid)))\n (gpu-exec :op \"composite\"\n :layers (list base overlay)\n :blend blend\n :output :stream))))" "lisp"))
(p "This isn't a DSL embedded in Python. It's SX, the same language that renders pages and defines components. The recipe author uses " (code "let") ", " (code "fn") ", " (code "map") ", " (code "if") " " (em "- ") "the full language. The only difference is what primitives are available. " (code "resolve-cid") " fetches content-addressed data. " (code "gpu-exec") " dispatches GPU operations. These are primitives, not library calls. They exist in the environment the same way " (code "+") " and " (code "str") " exist.")
(p "The recipe is data. It's an s-expression. You can parse it, analyze it, transform it, serialize it, hash it, store it, transmit it. You can inspect a recipe's dependency graph the same way " (code "deps.sx") " inspects component dependencies. You can type-check a recipe the same way " (code "typed-sx") " type-checks components. The tools already exist. They just need a new set of primitives to reason about."))
@@ -33,7 +33,7 @@
(~docs/section :title "Split execution" :id "split-execution"
(p "Some recipe steps run against cached data (fast, local). Others need GPU. Others need live input. The evaluator doesn't dispatch " (em "- ") "the boundary declarations do. " (code "(with-boundary (gpu-compute) ...)") " migrates to a GPU-capable host. " (code "(with-boundary (live-ingest) ...)") " opens WebRTC feeds. The recipe author doesn't manage infrastructure " (em "- ") "they declare capabilities, and execution flows to where they exist.")
(~docs/code :code (highlight "(define live-composite\n (fn (recipe-cid camera-count)\n (let ((recipe (resolve-cid recipe-cid)))\n ;; Phase 1: cached data (local, fast)\n (let ((base-layers (map resolve-cid (get recipe \"layers\"))))\n ;; Phase 2: GPU processing\n (with-boundary (gpu-compute)\n (let ((composed (gpu-exec :op \"composite\"\n :layers base-layers\n :blend (get recipe \"blend\"))))\n ;; Phase 3: live feeds\n (with-boundary (live-ingest)\n (let ((feeds (map (fn (i)\n (open-feed :protocol \"webrtc\"\n :label (str \"camera-\" i)))\n (range 0 camera-count))))\n ;; Phase 4: encode and stream\n (with-boundary (encoding)\n (encode-stream\n :sources (concat (list composed) feeds)\n :codec \"h264\"\n :output :stream))))))))))" "lisp"))
(~docs/code :src (highlight "(define live-composite\n (fn (recipe-cid camera-count)\n (let ((recipe (resolve-cid recipe-cid)))\n ;; Phase 1: cached data (local, fast)\n (let ((base-layers (map resolve-cid (get recipe \"layers\"))))\n ;; Phase 2: GPU processing\n (with-boundary (gpu-compute)\n (let ((composed (gpu-exec :op \"composite\"\n :layers base-layers\n :blend (get recipe \"blend\"))))\n ;; Phase 3: live feeds\n (with-boundary (live-ingest)\n (let ((feeds (map (fn (i)\n (open-feed :protocol \"webrtc\"\n :label (str \"camera-\" i)))\n (range 0 camera-count))))\n ;; Phase 4: encode and stream\n (with-boundary (encoding)\n (encode-stream\n :sources (concat (list composed) feeds)\n :codec \"h264\"\n :output :stream))))))))))" "lisp"))
(p "Four phases. Four capability requirements. The program reads linearly " (em "- ") "resolve cached layers, composite on GPU, open live feeds, encode output. But execution migrates across hosts as needed. The " (code "with-boundary") " blocks are the seams. Everything inside a boundary block runs on a host that provides those capabilities. Everything outside runs wherever the program started.")
(p "This is the same mechanism described in the generative SX plan's environment migration section. " (code "with-boundary") " serializes the environment (" (code "env-snapshot") "), ships the pending expression to a capable host, and execution continues there. The recipe author writes a linear program. The runtime makes it distributed."))
@@ -56,7 +56,7 @@
(~docs/section :title "Feed generation" :id "feed-generation"
(p "The executing program creates new endpoints as a side effect. " (code "(open-feed ...)") " doesn't return data " (em "- ") "it returns a connectable endpoint. WebRTC peers, SSE streams, WebSocket channels. These are generative acts: the endpoint didn't exist before the recipe ran. The program grew its own input surface.")
(p "When the recipe completes or the island disposes, feeds are cleaned up via the disposal mechanism. An " (code "effect") " in island scope that opens a feed returns a cleanup function. When the island unmounts, the cleanup runs, the feed closes, the endpoint disappears. The lifecycle is automatic.")
(~docs/code :code (highlight ";; A feed is a connectable endpoint, not raw data\n(define create-camera-mosaic\n (fn (camera-ids)\n ;; Each open-feed returns a connectable URL, not bytes\n (let ((feeds (map (fn (id)\n (open-feed :protocol \"webrtc\"\n :label (str \"cam-\" id)\n :quality \"720p\"))\n camera-ids)))\n ;; The mosaic recipe composes feeds as inputs\n (gpu-exec :op \"mosaic\"\n :inputs feeds\n :layout \"grid\"\n :output (open-feed :protocol \"sse\"\n :label \"mosaic-output\"\n :format \"mjpeg\")))))" "lisp"))
(~docs/code :src (highlight ";; A feed is a connectable endpoint, not raw data\n(define create-camera-mosaic\n (fn (camera-ids)\n ;; Each open-feed returns a connectable URL, not bytes\n (let ((feeds (map (fn (id)\n (open-feed :protocol \"webrtc\"\n :label (str \"cam-\" id)\n :quality \"720p\"))\n camera-ids)))\n ;; The mosaic recipe composes feeds as inputs\n (gpu-exec :op \"mosaic\"\n :inputs feeds\n :layout \"grid\"\n :output (open-feed :protocol \"sse\"\n :label \"mosaic-output\"\n :format \"mjpeg\")))))" "lisp"))
(p "The output is itself a feed. A client connects to the mosaic output URL and receives composed frames. The feeds are the program's I/O surface " (em "- ") "they exist because the program created them, and they die when the program stops. No static route configuration. No service mesh. The program declares what it needs and creates what it produces."))
;; =====================================================================
@@ -65,7 +65,7 @@
(~docs/section :title "The client boundary" :id "client-boundary"
(p "The browser is just another execution environment with its own primitive set: " (code "render-to-dom") ", " (code "signal") ", " (code "deref") ", " (code "open-feed") " (as WebRTC consumer). A streaming art-dag response arrives as SX wire format. The client evaluates it in island scope " (em "- ") "signals bind to stream frames, " (code "reactive-list") " renders feed thumbnails, " (code "computed") " derives overlay parameters. The server pushes frames; the client renders them reactively. No special video player " (em "- ") "just signals and DOM.")
(~docs/code :code (highlight "(defisland ~plans/art-dag-sx/live-canvas ()\n (let ((frames (signal nil))\n (feed-url (signal nil)))\n ;; Connect to stream when URL arrives\n (effect (fn ()\n (when (deref feed-url)\n (connect-stream (deref feed-url)\n :on-frame (fn (f) (reset! frames f))))))\n (div :class \"relative aspect-video bg-black rounded\"\n (when (deref frames)\n (canvas :width 1920 :height 1080\n :draw (fn (ctx)\n (draw-frame ctx (deref frames)))))\n (when (not (deref frames))\n (p :class \"absolute inset-0 flex items-center justify-center text-white/50\"\n \"Waiting for stream...\")))))" "lisp"))
(~docs/code :src (highlight "(defisland ~plans/art-dag-sx/live-canvas ()\n (let ((frames (signal nil))\n (feed-url (signal nil)))\n ;; Connect to stream when URL arrives\n (effect (fn ()\n (when (deref feed-url)\n (connect-stream (deref feed-url)\n :on-frame (fn (f) (reset! frames f))))))\n (div :class \"relative aspect-video bg-black rounded\"\n (when (deref frames)\n (canvas :width 1920 :height 1080\n :draw (fn (ctx)\n (draw-frame ctx (deref frames)))))\n (when (not (deref frames))\n (p :class \"absolute inset-0 flex items-center justify-center text-white/50\"\n \"Waiting for stream...\")))))" "lisp"))
(p "The island is reactive. When " (code "frames") " updates, only the canvas redraws. When " (code "feed-url") " updates, the effect reconnects. No polling loop. No WebSocket message handler parsing JSON. The stream is a signal source. The DOM is a signal consumer. The reactive graph connects them.")
(p "This is the same island architecture from the reactive islands plan " (em "- ") "signals, effects, computed, disposal. The only difference is the data source. Instead of an HTMX response mutating a signal, a WebRTC stream mutates a signal. The rendering pipeline doesn't know or care where the data comes from. It reacts."))
@@ -76,7 +76,7 @@
(~docs/section :title "L1/L2 integration" :id "l1-l2"
(p "L1 is the compute layer (Celery workers, GPU nodes). L2 is the registry (ActivityPub, recipe discovery). In SX terms: L1 hosts provide " (code "gpu-exec") ", " (code "encode-stream") ", " (code "resolve-cid-local") ". L2 hosts provide " (code "discover-recipe") ", " (code "publish-recipe") ", " (code "federate-activity") ".")
(p "An SX program that needs both crosses boundaries as needed " (em "- ") "fetch recipe metadata from L2, execute it on L1, publish results back to L2.")
(~docs/code :code (highlight ";; A full pipeline crossing L1 and L2 boundaries\n(define render-and-publish\n (fn (recipe-name output-label)\n ;; L2: discover the recipe\n (with-boundary (registry)\n (let ((recipe-cid (discover-recipe :name recipe-name\n :version \"latest\")))\n ;; L1: execute the recipe\n (with-boundary (gpu-compute)\n (let ((result-cid (gpu-exec :recipe (resolve-cid recipe-cid)\n :output :cid)))\n ;; L2: publish the result\n (with-boundary (registry)\n (publish-recipe\n :name output-label\n :input-cid recipe-cid\n :output-cid result-cid\n :activity \"Create\"))))))))" "lisp"))
(~docs/code :src (highlight ";; A full pipeline crossing L1 and L2 boundaries\n(define render-and-publish\n (fn (recipe-name output-label)\n ;; L2: discover the recipe\n (with-boundary (registry)\n (let ((recipe-cid (discover-recipe :name recipe-name\n :version \"latest\")))\n ;; L1: execute the recipe\n (with-boundary (gpu-compute)\n (let ((result-cid (gpu-exec :recipe (resolve-cid recipe-cid)\n :output :cid)))\n ;; L2: publish the result\n (with-boundary (registry)\n (publish-recipe\n :name output-label\n :input-cid recipe-cid\n :output-cid result-cid\n :activity \"Create\"))))))))" "lisp"))
(p "Three boundary crossings. L2 to find the recipe. L1 to execute it. L2 to publish the result. The program reads as a linear sequence of operations. The runtime handles the dispatch " (em "- ") "which host provides " (code "discover-recipe") ", which host provides " (code "gpu-exec") ", which host provides " (code "publish-recipe") ". The program author doesn't configure endpoints or manage connections. They declare capabilities."))
;; =====================================================================

4
sx/sx/plans/async-eval-convergence.sx
View File

@@ -43,12 +43,12 @@
(h4 :class "font-semibold mt-4 mb-2" "Phase 1 — Async call hook")
(p "The bootstrapped evaluator calls primitives via " (code "apply(fn, args)") ". In the Python host, " (code "apply") " is a platform primitive. Replace it with an async-aware version:")
(~docs/code :code (highlight "(define apply-fn\n (fn (f args)\n ;; Platform provides: if f returns a coroutine, await it\n (apply-maybe-async f args)))" "lisp"))
(~docs/code :src (highlight "(define apply-fn\n (fn (f args)\n ;; Platform provides: if f returns a coroutine, await it\n (apply-maybe-async f args)))" "lisp"))
(p "The bootstrapper emits " (code "apply_maybe_async") " as a Python " (code "async def") " that checks if the result is a coroutine and awaits it if so. Pure functions return immediately. IO primitives return coroutines that get awaited. " (strong "Zero overhead for pure calls") " — just an " (code "isinstance") " check.")
(h4 :class "font-semibold mt-4 mb-2" "Phase 2 — Async trampoline")
(p "The spec's trampoline loop resolves thunks synchronously. The Python bootstrapper emits an " (code "async def trampoline") " variant that can await thunks whose bodies contain IO calls. The trampoline structure is identical — only the " (code "await") " keyword is added.")
(~docs/code :code (highlight "# Bootstrapper emits this for Python async target\nasync def trampoline(val):\n while isinstance(val, Thunk):\n val = await eval_expr(val.expr, val.env)\n return val" "python"))
(~docs/code :src (highlight "# Bootstrapper emits this for Python async target\nasync def trampoline(val):\n while isinstance(val, Thunk):\n val = await eval_expr(val.expr, val.env)\n return val" "python"))
(h4 :class "font-semibold mt-4 mb-2" "Phase 3 — Aser as spec module")
(p "The " (code "_aser") " rendering mode (evaluate control flow, serialize HTML/components as SX source) should be specced as a module in " (code "render.sx") " alongside " (code "render-to-html") " and " (code "render-to-dom") ". It's currently hand-written Python because it predates the spec, but its logic is pure SX: walk the AST, eval certain forms, serialize others.")

38
sx/sx/plans/cek-reactive.sx
View File

@@ -26,7 +26,7 @@
(p "Currently wrapped in explicit " (code "effect") " calls. With deref-as-shift, "
"the continuation captures this automatically:")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; User writes:\n(div :class (str \"count-\" (deref counter))\n (str \"Value: \" (deref counter)))\n\n;; Renderer internally wraps each expression:\n(div :class (reactive-reset update-attr-fn (str \"count-\" (deref counter)))\n (reactive-reset update-text-fn (str \"Value: \" (deref counter))))\n\n;; When (deref counter) hits a signal inside reactive-reset:\n;; 1. Shift: capture continuation (str \"count-\" [HOLE])\n;; 2. Register continuation as signal subscriber\n;; 3. Return current value for initial render\n;; When counter changes:\n;; Re-invoke continuation with new value → update-fn updates DOM"
"lisp")))
@@ -42,17 +42,17 @@
(~docs/subsection :title "1a. platform_js.py — SPEC_MODULES + platform code"
(p "Add to " (code "SPEC_MODULES") " dict:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"\"frames\": (\"frames.sx\", \"frames (CEK continuation frames)\"),\n\"cek\": (\"cek.sx\", \"cek (explicit CEK machine evaluator)\"),\n\n# Add ordering (new constant):\nSPEC_MODULE_ORDER = [\"deps\", \"frames\", \"page-helpers\", \"router\", \"signals\", \"cek\"]"
"python"))
(p "Add " (code "PLATFORM_CEK_JS") " constant (mirrors " (code "PLATFORM_CEK_PY") "):")
(~docs/code :code (highlight
(~docs/code :src (highlight
"// Primitive aliases used by cek.sx\nvar inc = PRIMITIVES[\"inc\"];\nvar dec = PRIMITIVES[\"dec\"];\nvar zip_pairs = PRIMITIVES[\"zip-pairs\"];\n\nfunction makeCekContinuation(captured, restKont) {\n var c = new Continuation(function(v) { return v !== undefined ? v : NIL; });\n c._cek_data = {\"captured\": captured, \"rest-kont\": restKont};\n return c;\n}\nfunction continuationData(c) {\n return (c && c._cek_data) ? c._cek_data : {};\n}"
"javascript"))
(p "Add " (code "CEK_FIXUPS_JS") " — iterative " (code "cek-run") " override:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"cekRun = function(state) {\n while (!cekTerminal_p(state)) { state = cekStep(state); }\n return cekValue(state);\n};"
"javascript")))
@@ -68,12 +68,12 @@
(~docs/subsection :title "1c. js.sx — RENAMES for predicate functions"
(p "Default mangling handles most names. Only add RENAMES where " (code "?")
" suffix needs clean JS names:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"\"cek-terminal?\" \"cekTerminalP\"\n\"kont-empty?\" \"kontEmptyP\"\n\"make-cek-continuation\" \"makeCekContinuation\"\n\"continuation-data\" \"continuationData\""
"lisp")))
(~docs/subsection :title "1d. bootstrap_py.py — RENAMES for CEK predicates"
(~docs/code :code (highlight
(~docs/code :src (highlight
"\"cek-terminal?\": \"cek_terminal_p\",\n\"kont-empty?\": \"kont_empty_p\",\n\"make-cek-continuation\": \"make_cek_continuation\",\n\"continuation-data\": \"continuation_data\","
"python")))
@@ -92,18 +92,18 @@
(p "New frame types in " (code "frames.sx") " that enable deref-as-shift.")
(~docs/subsection :title "2a. New frame constructors"
(~docs/code :code (highlight
(~docs/code :src (highlight
";; ReactiveResetFrame: delimiter for reactive deref-as-shift\n;; Carries an update-fn that gets called with new values on re-render.\n(define make-reactive-reset-frame\n (fn (env update-fn first-render?)\n {:type \"reactive-reset\" :env env :update-fn update-fn\n :first-render first-render?}))\n\n;; DerefFrame: awaiting evaluation of deref's argument\n(define make-deref-frame\n (fn (env)\n {:type \"deref\" :env env}))"
"lisp")))
(~docs/subsection :title "2b. Update kont-capture-to-reset"
(p "Must stop at EITHER " (code "\"reset\"") " OR " (code "\"reactive-reset\"") ":")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define kont-capture-to-reset\n (fn (kont)\n (define scan\n (fn (k captured)\n (if (empty? k)\n (error \"shift without enclosing reset\")\n (let ((frame (first k)))\n (if (or (= (frame-type frame) \"reset\")\n (= (frame-type frame) \"reactive-reset\"))\n (list captured (rest k))\n (scan (rest k) (append captured (list frame))))))))\n (scan kont (list))))"
"lisp")))
(~docs/subsection :title "2c. Helpers to scan for ReactiveResetFrame"
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define has-reactive-reset-frame?\n (fn (kont)\n (if (empty? kont) false\n (if (= (frame-type (first kont)) \"reactive-reset\") true\n (has-reactive-reset-frame? (rest kont))))))\n\n;; Returns 3 values: (captured, frame, rest)\n(define kont-capture-to-reactive-reset\n (fn (kont)\n (define scan\n (fn (k captured)\n (if (empty? k)\n (error \"reactive deref without enclosing reactive-reset\")\n (let ((frame (first k)))\n (if (= (frame-type frame) \"reactive-reset\")\n (list captured frame (rest k))\n (scan (rest k) (append captured (list frame))))))))\n (scan kont (list))))"
"lisp"))))
@@ -118,30 +118,30 @@
(~docs/subsection :title "3a. Add to special form dispatch in cek.sx"
(p "In the dispatch table (around where " (code "reset") " and " (code "shift") " are):")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(= name \"deref\") (step-sf-deref args env kont)"
"lisp")))
(~docs/subsection :title "3b. step-sf-deref"
(p "Evaluates the argument first (push DerefFrame), then decides whether to shift:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define step-sf-deref\n (fn (args env kont)\n (make-cek-state\n (first args) env\n (kont-push (make-deref-frame env) kont))))"
"lisp")))
(~docs/subsection :title "3c. Handle DerefFrame in step-continue"
(p "When the deref argument is evaluated, decide: shift or return.")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(= ft \"deref\")\n (let ((val value)\n (fenv (get frame \"env\")))\n (if (not (signal? val))\n ;; Not a signal: pass through\n (make-cek-value val fenv rest-k)\n ;; Signal: check for ReactiveResetFrame\n (if (has-reactive-reset-frame? rest-k)\n ;; Perform reactive shift\n (reactive-shift-deref val fenv rest-k)\n ;; No reactive-reset: normal deref (scope-based tracking)\n (do\n (let ((ctx (context \"sx-reactive\" nil)))\n (when ctx\n (let ((dep-list (get ctx \"deps\"))\n (notify-fn (get ctx \"notify\")))\n (when (not (contains? dep-list val))\n (append! dep-list val)\n (signal-add-sub! val notify-fn)))))\n (make-cek-value (signal-value val) fenv rest-k)))))"
"lisp")))
(~docs/subsection :title "3d. reactive-shift-deref — the heart"
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define reactive-shift-deref\n (fn (sig env kont)\n (let ((scan-result (kont-capture-to-reactive-reset kont))\n (captured-frames (first scan-result))\n (reset-frame (nth scan-result 1))\n (remaining-kont (nth scan-result 2))\n (update-fn (get reset-frame \"update-fn\")))\n ;; Sub-scope for nested subscriber cleanup on re-invocation\n (let ((sub-disposers (list)))\n (let ((subscriber\n (fn ()\n ;; Dispose previous nested subscribers\n (for-each (fn (d) (invoke d)) sub-disposers)\n (set! sub-disposers (list))\n ;; Re-invoke: push fresh ReactiveResetFrame (first-render=false)\n (let ((new-reset (make-reactive-reset-frame env update-fn false))\n (new-kont (concat captured-frames\n (list new-reset)\n remaining-kont)))\n (with-island-scope\n (fn (d) (append! sub-disposers d))\n (fn ()\n (cek-run\n (make-cek-value (signal-value sig) env new-kont))))))))\n ;; Register subscriber\n (signal-add-sub! sig subscriber)\n ;; Register cleanup with island scope\n (register-in-scope\n (fn ()\n (signal-remove-sub! sig subscriber)\n (for-each (fn (d) (invoke d)) sub-disposers)))\n ;; Return current value for initial render\n (make-cek-value (signal-value sig) env remaining-kont))))))"
"lisp")))
(~docs/subsection :title "3e. Handle ReactiveResetFrame in step-continue"
(p "When expression completes normally (or after re-invocation):")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(= ft \"reactive-reset\")\n (let ((update-fn (get frame \"update-fn\"))\n (first? (get frame \"first-render\")))\n ;; On re-render (not first), call update-fn with new value\n (when (and update-fn (not first?))\n (invoke update-fn value))\n (make-cek-value value env rest-k))"
"lisp"))
(p (strong "Key:") " On first render, update-fn is NOT called — the value flows back to the caller "
@@ -157,24 +157,24 @@
(p "Add CEK reactive path alongside existing effect-based path, controlled by opt-in flag.")
(~docs/subsection :title "4a. Opt-in flag"
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define *use-cek-reactive* false)\n(define enable-cek-reactive! (fn () (set! *use-cek-reactive* true)))"
"lisp")))
(~docs/subsection :title "4b. CEK reactive attribute binding"
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define cek-reactive-attr\n (fn (el attr-name expr env)\n (let ((update-fn (fn (val)\n (cond\n (or (nil? val) (= val false)) (dom-remove-attr el attr-name)\n (= val true) (dom-set-attr el attr-name \"\")\n :else (dom-set-attr el attr-name (str val))))))\n ;; Mark for morph protection\n (let ((existing (or (dom-get-attr el \"data-sx-reactive-attrs\") \"\"))\n (updated (if (empty? existing) attr-name (str existing \",\" attr-name))))\n (dom-set-attr el \"data-sx-reactive-attrs\" updated))\n ;; Initial render via CEK with ReactiveResetFrame\n (let ((initial (cek-run\n (make-cek-state expr env\n (list (make-reactive-reset-frame env update-fn true))))))\n (invoke update-fn initial)))))"
"lisp")))
(~docs/subsection :title "4c. Modify render-dom-element dispatch"
(p "In attribute processing, add conditional:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(context \"sx-island-scope\" nil)\n (if *use-cek-reactive*\n (cek-reactive-attr el attr-name attr-expr env)\n (reactive-attr el attr-name\n (fn () (trampoline (eval-expr attr-expr env)))))"
"lisp"))
(p "Similarly for text positions and conditional rendering."))
(~docs/subsection :title "4d. CEK reactive text"
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define cek-reactive-text\n (fn (expr env)\n (let ((node (create-text-node \"\"))\n (update-fn (fn (val)\n (dom-set-text-content node (str val)))))\n (let ((initial (cek-run\n (make-cek-state expr env\n (list (make-reactive-reset-frame env update-fn true))))))\n (dom-set-text-content node (str initial))\n node))))"
"lisp")))
@@ -224,7 +224,7 @@
(~docs/section :title "Multi-Deref Handling" :id "multi-deref"
(~docs/code :code (highlight "(str (deref first-name) \" \" (deref last-name))" "lisp"))
(~docs/code :src (highlight "(str (deref first-name) \" \" (deref last-name))" "lisp"))
(ol :class "list-decimal pl-6 mb-6 space-y-3"
(li (strong "Initial render:") " First " (code "deref") " hits signal → shifts, captures "

24
sx/sx/plans/content-addressed-components.sx
View File

@@ -83,7 +83,7 @@
(~docs/subsection :title "Implementation"
(p "New spec function in a " (code "canonical.sx") " module:")
(~docs/code :code (highlight "(define canonical-serialize\n (fn (node)\n ;; Produce a canonical s-expression string from an AST node.\n ;; Deterministic: same AST always produces same output.\n ;; Used for CID computation — NOT for human-readable output.\n (case (type-of node)\n \"list\"\n (str \"(\" (join \" \" (map canonical-serialize node)) \")\")\n \"dict\"\n (let ((sorted-keys (sort (keys node))))\n (str \"{\" (join \" \"\n (map (fn (k)\n (str \":\" k \" \" (canonical-serialize (get node k))))\n sorted-keys)) \"}\"))\n \"string\"\n (str '\"' (escape-canonical node) '\"')\n \"number\"\n (canonical-number node)\n \"symbol\"\n (symbol-name node)\n \"keyword\"\n (str \":\" (keyword-name node))\n \"boolean\"\n (if node \"true\" \"false\")\n \"nil\"\n \"nil\")))" "lisp"))
(~docs/code :src (highlight "(define canonical-serialize\n (fn (node)\n ;; Produce a canonical s-expression string from an AST node.\n ;; Deterministic: same AST always produces same output.\n ;; Used for CID computation — NOT for human-readable output.\n (case (type-of node)\n \"list\"\n (str \"(\" (join \" \" (map canonical-serialize node)) \")\")\n \"dict\"\n (let ((sorted-keys (sort (keys node))))\n (str \"{\" (join \" \"\n (map (fn (k)\n (str \":\" k \" \" (canonical-serialize (get node k))))\n sorted-keys)) \"}\"))\n \"string\"\n (str '\"' (escape-canonical node) '\"')\n \"number\"\n (canonical-number node)\n \"symbol\"\n (symbol-name node)\n \"keyword\"\n (str \":\" (keyword-name node))\n \"boolean\"\n (if node \"true\" \"false\")\n \"nil\"\n \"nil\")))" "lisp"))
(p "This function must be bootstrapped to both Python and JS — the server computes CIDs at registration time, the client verifies them on fetch.")
(p "The canonical serializer is distinct from " (code "serialize()") " for display. " (code "serialize(pretty=True)") " remains for human-readable output. " (code "canonical-serialize") " is for hashing only.")))
@@ -103,11 +103,11 @@
(li (strong "Hash function:") " SHA3-256 (already used by artdag for content addressing)")
(li (strong "Codec:") " raw (the content is the canonical SX source bytes, not a DAG-PB wrapper)")
(li (strong "Base encoding:") " base32lower for URL-safe representation (" (code "bafy...") " prefix)"))
(~docs/code :code (highlight ";; CID computation pipeline\n(define component-cid\n (fn (component)\n ;; 1. Reconstruct full defcomp form\n ;; 2. Canonical serialize\n ;; 3. SHA3-256 hash\n ;; 4. Wrap as CIDv1\n (let ((source (canonical-serialize\n (list 'defcomp\n (symbol (str \"~\" (component-name component)))\n (component-params-list component)\n (component-body component)))))\n (cid-v1 :sha3-256 :raw (encode-utf8 source)))))" "lisp")))
(~docs/code :src (highlight ";; CID computation pipeline\n(define component-cid\n (fn (component)\n ;; 1. Reconstruct full defcomp form\n ;; 2. Canonical serialize\n ;; 3. SHA3-256 hash\n ;; 4. Wrap as CIDv1\n (let ((source (canonical-serialize\n (list 'defcomp\n (symbol (str \"~\" (component-name component)))\n (component-params-list component)\n (component-body component)))))\n (cid-v1 :sha3-256 :raw (encode-utf8 source)))))" "lisp")))
(~docs/subsection :title "Where CIDs Live"
(p "Each " (code "Component") " object gains a " (code "cid") " field, computed at registration time:")
(~docs/code :code (highlight ";; types.py extension\n@dataclass\nclass Component:\n name: str\n params: list[str]\n has_children: bool\n body: Any\n closure: dict[str, Any]\n css_classes: set[str]\n deps: set[str] # by name\n io_refs: set[str]\n cid: str | None = None # computed after registration\n dep_cids: dict[str, str] | None = None # name → CID" "python"))
(~docs/code :src (highlight ";; types.py extension\n@dataclass\nclass Component:\n name: str\n params: list[str]\n has_children: bool\n body: Any\n closure: dict[str, Any]\n css_classes: set[str]\n deps: set[str] # by name\n io_refs: set[str]\n cid: str | None = None # computed after registration\n dep_cids: dict[str, str] | None = None # name → CID" "python"))
(p "After " (code "compute_all_deps()") " runs, a new " (code "compute_all_cids()") " pass fills in CIDs for every component. Dependency CIDs are also recorded — when a component references " (code "~card") ", we store both the name and card's CID."))
(~docs/subsection :title "CID Stability"
@@ -130,7 +130,7 @@
(p :class "text-violet-800" "Metadata that travels with a CID — what a component needs, what it provides, whether it's safe to run. Enough information to resolve, validate, and render without fetching the source first."))
(~docs/subsection :title "Manifest Structure"
(~docs/code :code (highlight ";; Component manifest — published alongside the source\n(SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :source-bytes 847\n :params (:title :price :image-url)\n :has-children true\n :pure true\n :deps (\n {:name \"~card\" :cid \"bafy...card\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"}\n {:name \"~lazy-image\" :cid \"bafy...lazyimg\"})\n :css-atoms (:border :rounded :p-4 :text-sm :font-bold\n :text-green-700 :line-through :text-stone-400)\n :author \"https://rose-ash.com/apps/market\"\n :published \"2026-03-06T14:30:00Z\")" "lisp"))
(~docs/code :src (highlight ";; Component manifest — published alongside the source\n(SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :source-bytes 847\n :params (:title :price :image-url)\n :has-children true\n :pure true\n :deps (\n {:name \"~card\" :cid \"bafy...card\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"}\n {:name \"~lazy-image\" :cid \"bafy...lazyimg\"})\n :css-atoms (:border :rounded :p-4 :text-sm :font-bold\n :text-green-700 :line-through :text-stone-400)\n :author \"https://rose-ash.com/apps/market\"\n :published \"2026-03-06T14:30:00Z\")" "lisp"))
(p "Key fields:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code ":cid") " — content address of the canonical serialized source")
@@ -163,11 +163,11 @@
(li "Creates/updates " (code "IPFSPin") " record with " (code "pin_type=\"component\""))
(li "Publishes manifest to IPFS (separate CID)")
(li "Optionally announces via AP outbox for federated discovery"))
(~docs/code :code (highlight ";; IPFSPin usage for components\nIPFSPin(\n content_hash=\"sha3-256:abcdef...\",\n ipfs_cid=\"bafy...productcard\",\n pin_type=\"component\",\n source_type=\"market\", # which service defined it\n metadata={\n \"name\": \"~product-card\",\n \"manifest_cid\": \"bafy...manifest\",\n \"deps\": [\"bafy...card\", \"bafy...pricetag\"],\n \"pure\": True\n }\n)" "python")))
(~docs/code :src (highlight ";; IPFSPin usage for components\nIPFSPin(\n content_hash=\"sha3-256:abcdef...\",\n ipfs_cid=\"bafy...productcard\",\n pin_type=\"component\",\n source_type=\"market\", # which service defined it\n metadata={\n \"name\": \"~product-card\",\n \"manifest_cid\": \"bafy...manifest\",\n \"deps\": [\"bafy...card\", \"bafy...pricetag\"],\n \"pure\": True\n }\n)" "python")))
(~docs/subsection :title "Client-Side: Resolution"
(p "New spec module " (code "resolve.sx") " — the client-side component resolution pipeline:")
(~docs/code :code (highlight "(define resolve-component-by-cid\n (fn (cid callback)\n ;; Resolution cascade:\n ;; 1. Check component env (already loaded?)\n ;; 2. Check localStorage (keyed by CID = cache-forever)\n ;; 3. Check origin server (/sx/components?cid=bafy...)\n ;; 4. Fetch from IPFS gateway\n ;; 5. Verify hash matches CID\n ;; 6. Parse, validate purity, register, callback\n (let ((cached (local-storage-get (str \"sx-cid:\" cid))))\n (if cached\n (do\n (register-component-source cached)\n (callback true))\n (fetch-component-by-cid cid\n (fn (source)\n (if (verify-cid cid source)\n (do\n (local-storage-set (str \"sx-cid:\" cid) source)\n (register-component-source source)\n (callback true))\n (do\n (log-warn (str \"sx:cid verification failed \" cid))\n (callback false)))))))))" "lisp"))
(~docs/code :src (highlight "(define resolve-component-by-cid\n (fn (cid callback)\n ;; Resolution cascade:\n ;; 1. Check component env (already loaded?)\n ;; 2. Check localStorage (keyed by CID = cache-forever)\n ;; 3. Check origin server (/sx/components?cid=bafy...)\n ;; 4. Fetch from IPFS gateway\n ;; 5. Verify hash matches CID\n ;; 6. Parse, validate purity, register, callback\n (let ((cached (local-storage-get (str \"sx-cid:\" cid))))\n (if cached\n (do\n (register-component-source cached)\n (callback true))\n (fetch-component-by-cid cid\n (fn (source)\n (if (verify-cid cid source)\n (do\n (local-storage-set (str \"sx-cid:\" cid) source)\n (register-component-source source)\n (callback true))\n (do\n (log-warn (str \"sx:cid verification failed \" cid))\n (callback false)))))))))" "lisp"))
(p "The cache-forever semantics are the key insight: because CIDs are content-addressed, a cached component " (strong "can never be stale") ". If the source changes, it gets a new CID. Old CIDs remain valid forever. There is no cache invalidation problem."))
(~docs/subsection :title "Resolution Cascade"
@@ -219,7 +219,7 @@
(~docs/subsection :title "Guarantee 1: Purity is Structural"
(p "SX boundary enforcement isn't a runtime sandbox — it's a registration-time structural check. When a component is loaded from IPFS and parsed, " (code "compute_all_io_refs()") " walks its entire AST and transitive dependencies. If " (em "any") " node references an IO primitive, the component is classified as IO-dependent and " (strong "rejected for untrusted registration."))
(p "This means the evaluator literally doesn't have IO primitives in scope when running an IPFS-loaded component. It's not that we catch IO calls — the names don't resolve. There's nothing to catch.")
(~docs/code :code (highlight "(define register-untrusted-component\n (fn (source origin)\n ;; Parse the defcomp from source\n ;; Run compute-all-io-refs on the parsed component\n ;; If io_refs is non-empty → REJECT\n ;; If pure → register in env with :origin metadata\n (let ((comp (parse-component source)))\n (if (not (component-pure? comp))\n (do\n (log-warn (str \"sx:reject IO component from \" origin))\n nil)\n (do\n (register-component comp)\n (log-info (str \"sx:registered \" (component-name comp)\n \" from \" origin))\n comp)))))" "lisp")))
(~docs/code :src (highlight "(define register-untrusted-component\n (fn (source origin)\n ;; Parse the defcomp from source\n ;; Run compute-all-io-refs on the parsed component\n ;; If io_refs is non-empty → REJECT\n ;; If pure → register in env with :origin metadata\n (let ((comp (parse-component source)))\n (if (not (component-pure? comp))\n (do\n (log-warn (str \"sx:reject IO component from \" origin))\n nil)\n (do\n (register-component comp)\n (log-info (str \"sx:registered \" (component-name comp)\n \" from \" origin))\n comp)))))" "lisp")))
(~docs/subsection :title "Guarantee 2: Content Verification"
(p "The CID IS the hash. When you fetch " (code "bafy...abc") " from any source — IPFS gateway, origin server, peer — you hash the response and compare. If it doesn't match, you reject it. No MITM attack can alter the content without changing the CID.")
@@ -279,17 +279,17 @@
(~docs/subsection :title "CID References in Page Registry"
(p "The page registry (shipped to the client as " (code "<script type=\"text/sx-pages\">") ") currently lists deps by name. Extend to include CIDs:")
(~docs/code :code (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :deps ({:name \"~essay-foo\" :cid \"bafy...essay\"}\n {:name \"~doc-code\" :cid \"bafy...doccode\"})\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(~docs/code :src (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :deps ({:name \"~essay-foo\" :cid \"bafy...essay\"}\n {:name \"~doc-code\" :cid \"bafy...doccode\"})\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(p "The " (a :href "/sx/(etc.(plan.predictive-prefetch))" :class "text-violet-700 underline" "predictive prefetch system") " uses these CIDs to fetch components from the resolution cascade rather than only from the origin server's " (code "/sx/components") " endpoint."))
(~docs/subsection :title "SX Response Component Headers"
(p "Currently, " (code "SX-Components") " header lists loaded component names. Extend to support CIDs:")
(~docs/code :code (highlight "Request:\nSX-Components: ~card:bafy...card,~plans/environment-images/nav:bafy...nav\n\nResponse:\nSX-Component-CIDs: ~plans/content-addressed-components/essay-foo:bafy...essay,~docs/code:bafy...doccode\n\n;; Response body only includes defs the client doesn't have\n(defcomp ~plans/content-addressed-components/essay-foo ...)" "http"))
(~docs/code :src (highlight "Request:\nSX-Components: ~card:bafy...card,~plans/environment-images/nav:bafy...nav\n\nResponse:\nSX-Component-CIDs: ~plans/content-addressed-components/essay-foo:bafy...essay,~docs/code:bafy...doccode\n\n;; Response body only includes defs the client doesn't have\n(defcomp ~plans/content-addressed-components/essay-foo ...)" "http"))
(p "The client can then verify received components match their declared CIDs. If the origin server is compromised, CID verification catches the tampered response."))
(~docs/subsection :title "Federated Content"
(p "When an ActivityPub activity arrives with SX content, it declares component requirements by CID:")
(~docs/code :code (highlight "(Create\n :actor \"https://other-instance.com/users/bob\"\n :object (Note\n :content (~product-card :title \"Bob's Widget\" :price 29.99)\n :requires (list\n {:name \"~product-card\" :cid \"bafy...prodcard\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"})))" "lisp"))
(~docs/code :src (highlight "(Create\n :actor \"https://other-instance.com/users/bob\"\n :object (Note\n :content (~product-card :title \"Bob's Widget\" :price 29.99)\n :requires (list\n {:name \"~product-card\" :cid \"bafy...prodcard\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"})))" "lisp"))
(p "The receiving browser resolves required components through the cascade. If Bob's instance is down, the components are still fetchable from IPFS. The content is self-describing and self-resolving.")))
;; -----------------------------------------------------------------------
@@ -304,12 +304,12 @@
(~docs/subsection :title "Component Registry as AP Actor"
(p "Each server exposes a component registry actor:")
(~docs/code :code (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(~docs/code :src (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(p "Follow the registry to receive component updates. The outbox is a chronological feed of Create/Update/Delete activities for components. 'Update' means a new CID for the same name — consumers decide whether to adopt it."))
(~docs/subsection :title "Discovery Protocol"
(p "Webfinger-style lookup for components by name:")
(~docs/code :code (highlight "GET /.well-known/sx-component?name=~product-card\n\n{\n \"name\": \"~product-card\",\n \"cid\": \"bafy...prodcard\",\n \"manifest_cid\": \"bafy...manifest\",\n \"gateway\": \"https://rose-ash.com/ipfs/\",\n \"author\": \"https://rose-ash.com/apps/market\"\n}" "http"))
(~docs/code :src (highlight "GET /.well-known/sx-component?name=~product-card\n\n{\n \"name\": \"~product-card\",\n \"cid\": \"bafy...prodcard\",\n \"manifest_cid\": \"bafy...manifest\",\n \"gateway\": \"https://rose-ash.com/ipfs/\",\n \"author\": \"https://rose-ash.com/apps/market\"\n}" "http"))
(p "This is an optional convenience — any consumer that knows the CID can skip discovery and fetch directly from IPFS. Discovery answers the question: " (em "\"what's the current version of ~product-card on rose-ash.com?\""))
)

14
sx/sx/plans/environment-images.sx
View File

@@ -66,7 +66,7 @@
(~docs/section :title "Image Format" :id "format"
(p "The image is itself an s-expression — the same format the spec is written in. This means the image can be parsed by the same parser, inspected by the same tools, and content-addressed by the same canonical serializer.")
(~docs/code :code (highlight "(sx-image\n :version 1\n :spec-cids {:eval \"bafy...eval\"\n :render \"bafy...render\"\n :parser \"bafy...parser\"\n :primitives \"bafy...prims\"\n :boundary \"bafy...boundary\"\n :signals \"bafy...signals\"}\n\n :components (\n (defcomp ~plans/environment-images/card (&key title subtitle &rest children)\n (div :class \"card\" (h2 title) (when subtitle (p subtitle)) children))\n (defcomp ~plans/environment-images/nav (&key items current)\n (nav :class \"nav\" (map (fn (item) ...) items)))\n ;; ... all registered components\n )\n\n :macros (\n (defmacro when (test &rest body)\n (list 'if test (cons 'begin body) nil))\n ;; ... all macros\n )\n\n :bindings (\n (define void-elements (list \"area\" \"base\" \"br\" \"col\" ...))\n (define boolean-attrs (list \"checked\" \"disabled\" ...))\n ;; ... all top-level defines\n )\n\n :primitive-names (\"str\" \"+\" \"-\" \"*\" \"/\" \"=\" \"<\" \">\" ...)\n :io-names (\"fetch-data\" \"call-action\" \"app-url\" ...))" "lisp"))
(~docs/code :src (highlight "(sx-image\n :version 1\n :spec-cids {:eval \"bafy...eval\"\n :render \"bafy...render\"\n :parser \"bafy...parser\"\n :primitives \"bafy...prims\"\n :boundary \"bafy...boundary\"\n :signals \"bafy...signals\"}\n\n :components (\n (defcomp ~plans/environment-images/card (&key title subtitle &rest children)\n (div :class \"card\" (h2 title) (when subtitle (p subtitle)) children))\n (defcomp ~plans/environment-images/nav (&key items current)\n (nav :class \"nav\" (map (fn (item) ...) items)))\n ;; ... all registered components\n )\n\n :macros (\n (defmacro when (test &rest body)\n (list 'if test (cons 'begin body) nil))\n ;; ... all macros\n )\n\n :bindings (\n (define void-elements (list \"area\" \"base\" \"br\" \"col\" ...))\n (define boolean-attrs (list \"checked\" \"disabled\" ...))\n ;; ... all top-level defines\n )\n\n :primitive-names (\"str\" \"+\" \"-\" \"*\" \"/\" \"=\" \"<\" \">\" ...)\n :io-names (\"fetch-data\" \"call-action\" \"app-url\" ...))" "lisp"))
(p "The " (code ":spec-cids") " field is the key. It links this image back to the exact spec that produced it. Anyone can verify the image by:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
@@ -83,7 +83,7 @@
(p "The image CID is computed by canonical-serializing the entire " (code "(sx-image ...)") " form and hashing it. Same process as component CIDs, just applied to the whole environment.")
(p "The relationship between spec CIDs and image CID is deterministic:")
(~docs/code :code (highlight ";; The image CID is a pure function of the spec CIDs\n;; (assuming a deterministic evaluator, which SX guarantees)\n(define image-cid-from-specs\n (fn (spec-cids)\n ;; 1. Fetch each spec file by CID\n ;; 2. Evaluate all specs in a fresh environment\n ;; 3. Extract components, macros, bindings\n ;; 4. Build (sx-image ...) form\n ;; 5. Canonical serialize\n ;; 6. Hash → CID\n ))" "lisp"))
(~docs/code :src (highlight ";; The image CID is a pure function of the spec CIDs\n;; (assuming a deterministic evaluator, which SX guarantees)\n(define image-cid-from-specs\n (fn (spec-cids)\n ;; 1. Fetch each spec file by CID\n ;; 2. Evaluate all specs in a fresh environment\n ;; 3. Extract components, macros, bindings\n ;; 4. Build (sx-image ...) form\n ;; 5. Canonical serialize\n ;; 6. Hash → CID\n ))" "lisp"))
(p "This means you can compute the expected image CID from the spec CIDs " (em "without") " having the image. If someone hands you an image claiming to be from spec " (code "bafy...eval") ", you can verify it by re-evaluating the spec and comparing CIDs. The image is a verifiable cache.")
(p "In practice, you'd only do this verification once per spec version. After that, the image CID is trusted by content-addressing — same bytes, same hash, forever."))
@@ -95,7 +95,7 @@
(~docs/section :title "Endpoint Provenance" :id "provenance"
(p "Every served page gains a provenance header linking it to the spec that rendered it:")
(~docs/code :code (highlight "HTTP/1.1 200 OK\nContent-Type: text/html\nSX-Spec: bafy...eval,bafy...render,bafy...parser,bafy...prims\nSX-Image: bafy...image\nSX-Page-Components: ~plans/environment-images/card:bafy...card,~plans/environment-images/nav:bafy...nav" "http"))
(~docs/code :src (highlight "HTTP/1.1 200 OK\nContent-Type: text/html\nSX-Spec: bafy...eval,bafy...render,bafy...parser,bafy...prims\nSX-Image: bafy...image\nSX-Page-Components: ~plans/environment-images/card:bafy...card,~plans/environment-images/nav:bafy...nav" "http"))
(p "Three levels of verification:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -134,7 +134,7 @@
(li "If no: evaluate spec files from source, build image, cache it")
(li "Register IO primitives and page helpers (host-specific, not in image)")
(li "Ready to serve"))
(~docs/code :code (highlight "(define load-environment\n (fn (spec-cids image-cache-dir)\n (let ((expected-image-cid (image-cid-for-specs spec-cids))\n (cached-path (str image-cache-dir \"/\" expected-image-cid \".sx\")))\n (if (file-exists? cached-path)\n ;; Fast path: deserialize\n (let ((image (parse (read-file cached-path))))\n (if (= (verify-image-cid image) expected-image-cid)\n (deserialize-image image)\n ;; Cache corrupted — rebuild\n (build-and-cache-image spec-cids image-cache-dir)))\n ;; Cold path: evaluate from source\n (build-and-cache-image spec-cids image-cache-dir)))))" "lisp")))
(~docs/code :src (highlight "(define load-environment\n (fn (spec-cids image-cache-dir)\n (let ((expected-image-cid (image-cid-for-specs spec-cids))\n (cached-path (str image-cache-dir \"/\" expected-image-cid \".sx\")))\n (if (file-exists? cached-path)\n ;; Fast path: deserialize\n (let ((image (parse (read-file cached-path))))\n (if (= (verify-image-cid image) expected-image-cid)\n (deserialize-image image)\n ;; Cache corrupted — rebuild\n (build-and-cache-image spec-cids image-cache-dir)))\n ;; Cold path: evaluate from source\n (build-and-cache-image spec-cids image-cache-dir)))))" "lisp")))
(~docs/subsection :title "Client Boot"
(p "The client already caches component definitions in localStorage keyed by bundle hash. Images extend this: cache the entire evaluated environment, not just individual components.")
@@ -153,7 +153,7 @@
(~docs/section :title "Standalone HTML Bundles" :id "standalone"
(p "An image can be inlined into a single HTML document, producing a fully self-contained application with no server dependency:")
(~docs/code :code (highlight "<!doctype html>\n<html>\n<head>\n <script type=\"text/sx-image\">\n (sx-image\n :version 1\n :spec-cids {...}\n :components (...)\n :macros (...)\n :bindings (...))\n </script>\n <script type=\"text/sx-pages\">\n (defpage home :path \"/\" :content (~home-page))\n </script>\n <script src=\"sx-ref.js\"></script>\n</head>\n<body>\n <div id=\"app\"></div>\n <script>\n // Deserialize image, register components, render\n Sx.bootFromImage(document.querySelector('[type=\"text/sx-image\"]'))\n </script>\n</body>\n</html>" "html"))
(~docs/code :src (highlight "<!doctype html>\n<html>\n<head>\n <script type=\"text/sx-image\">\n (sx-image\n :version 1\n :spec-cids {...}\n :components (...)\n :macros (...)\n :bindings (...))\n </script>\n <script type=\"text/sx-pages\">\n (defpage home :path \"/\" :content (~home-page))\n </script>\n <script src=\"sx-ref.js\"></script>\n</head>\n<body>\n <div id=\"app\"></div>\n <script>\n // Deserialize image, register components, render\n Sx.bootFromImage(document.querySelector('[type=\"text/sx-image\"]'))\n </script>\n</body>\n</html>" "html"))
(p "This document is its own CID. Pin it to IPFS and it's a permanent, executable, verifiable application. No origin server, no CDN, no DNS. The content network is the deployment target.")
@@ -168,7 +168,7 @@
(~docs/section :title "Namespaced Environments" :id "namespaces"
(p "As the component library grows across services, a flat environment risks name collisions. Images provide a natural boundary for namespace scoping.")
(~docs/code :code (highlight "(sx-image\n :version 1\n :namespace \"market\"\n :spec-cids {...}\n :extends \"bafy...shared-image\" ;; inherits shared components\n :components (\n ;; market-specific components\n (defcomp ~plans/environment-images/product-card ...)\n (defcomp ~plans/environment-images/price-tag ...)\n ))" "lisp"))
(~docs/code :src (highlight "(sx-image\n :version 1\n :namespace \"market\"\n :spec-cids {...}\n :extends \"bafy...shared-image\" ;; inherits shared components\n :components (\n ;; market-specific components\n (defcomp ~plans/environment-images/product-card ...)\n (defcomp ~plans/environment-images/price-tag ...)\n ))" "lisp"))
(p "Resolution: " (code "market/~plans/environment-images/product-card") " → look in market image first, then fall through to the shared image (via " (code ":extends") "). Each service produces its own image, layered on top of the shared base.")
@@ -205,7 +205,7 @@
(~docs/section :title "The Verification Chain" :id "chain"
(p "The full provenance chain from served page back to source:")
(~docs/code :code (highlight ";; 1. Page served with provenance headers\n;;\n;; SX-Spec: bafy...eval,bafy...render,...\n;; SX-Image: bafy...market-image\n;; SX-Page: (defpage product :path \"/products/<slug>\" ...)\n;;\n;; 2. Verify image → spec\n;;\n;; Fetch specs by CID → evaluate → build image → compare CID\n;; If match: the image was correctly produced from these specs\n;;\n;; 3. Verify page → image\n;;\n;; Deserialize image → evaluate page defn → render\n;; If output matches served HTML: the page was correctly rendered\n;;\n;; 4. Trust chain terminates at the spec\n;;\n;; The spec is self-hosting (eval.sx evaluates itself)\n;; The spec's CID is its identity\n;; No external trust anchor needed beyond the hash function" "lisp"))
(~docs/code :src (highlight ";; 1. Page served with provenance headers\n;;\n;; SX-Spec: bafy...eval,bafy...render,...\n;; SX-Image: bafy...market-image\n;; SX-Page: (defpage product :path \"/products/<slug>\" ...)\n;;\n;; 2. Verify image → spec\n;;\n;; Fetch specs by CID → evaluate → build image → compare CID\n;; If match: the image was correctly produced from these specs\n;;\n;; 3. Verify page → image\n;;\n;; Deserialize image → evaluate page defn → render\n;; If output matches served HTML: the page was correctly rendered\n;;\n;; 4. Trust chain terminates at the spec\n;;\n;; The spec is self-hosting (eval.sx evaluates itself)\n;; The spec's CID is its identity\n;; No external trust anchor needed beyond the hash function" "lisp"))
(p "This is stronger than code signing. Code signing says " (em "\"this entity vouches for this binary.\"") " Content addressing says " (em "\"this binary is the deterministic output of this source.\"") " No entity needed. No certificate authority. No revocation lists. Just math."))

32
sx/sx/plans/foundations.sx
View File

@@ -20,7 +20,7 @@
(p "Every layer is definable in terms of the one below. "
"No layer can be decomposed without the layer beneath it.")
(~docs/code :code
(~docs/code :src
(str "Layer 0: CEK machine (expression + environment + continuation) \u2714 DONE\n"
"Layer 1: Continuations (shift / reset \u2014 delimited capture) \u2714 DONE\n"
"Layer 2: Algebraic effects (operations + handlers) \u2714 DONE\n"
@@ -117,7 +117,7 @@
(p "All higher-order forms step element-by-element through the CEK machine:")
(~docs/code :code
(~docs/code :src
(str ";; map pushes a MapFrame, calls f on each element via continue-with-call\n"
"(map (fn (x) (* 2 (deref counter))) items)\n"
"\n"
@@ -138,7 +138,7 @@
(p "Delimited continuations (Felleisen 1988, Danvy & Filinski 1990) "
"expose the K register as a first-class value:")
(~docs/code :code
(~docs/code :src
(str ";; reset marks a point in the continuation\n"
"(reset\n"
" (+ 1 (shift k ;; k = \"the rest up to reset\"\n"
@@ -154,7 +154,7 @@
(p "The reactive payoff. " (code "deref") " inside a " (code "reactive-reset") " boundary "
"is shift/reset applied to signals:")
(~docs/code :code
(~docs/code :src
(str ";; User writes:\n"
"(div :class (str \"count-\" (deref counter))\n"
" (str \"Value: \" (deref counter)))\n"
@@ -258,7 +258,7 @@
(p "Fixed-size mutable byte arrays. A small primitive surface that unlocks "
"binary protocol parsing, image headers, wire formats, and efficient CID computation:")
(~docs/code :code
(~docs/code :src
(str ";; Create and write\n"
"(let ((buf (make-buffer 256)))\n"
" (buffer-write-u8! buf 0 #xFF)\n"
@@ -317,7 +317,7 @@
"On compiled hosts (OCaml, Rust), they compile to unboxed records — no dict overhead, "
"no hash lookups, no string key comparisons:")
(~docs/code :code
(~docs/code :src
(str ";; Define a struct\n"
"(defstruct point (x : number) (y : number))\n"
"\n"
@@ -396,7 +396,7 @@
"If " (code "~bank/payment-form") " calls " (code "~bank/amount-input") " calls "
(code "~ui/text-field") ", all three definitions are part of the CID:")
(~docs/code :code
(~docs/code :src
(str ";; Shallow CID — just this component's definition\n"
"(freeze-to-cid ~bank/payment-form) ;; => bafyrei..abc\n"
"\n"
@@ -413,7 +413,7 @@
"Canonical SX: no comments, no redundant whitespace, deterministic key ordering in dicts, "
"normalized number representation:")
(~docs/code :code
(~docs/code :src
(str ";; These must produce the same CID on JS, Python, and OCaml:\n"
"(canonical-sx '(div :class \"card\" (p \"hello\")))\n"
";; => \"(div :class \\\"card\\\" (p \\\"hello\\\"))\"\n"
@@ -425,7 +425,7 @@
(p "The client-side verification flow:")
(~docs/code :code
(~docs/code :src
(str ";; Server sends component + CID via aser wire format\n"
";; Browser receives, independently computes CID, compares\n"
"\n"
@@ -449,7 +449,7 @@
(p "Publishers declare expected CIDs via a well-known manifest:")
(~docs/code :code
(~docs/code :src
(str ";; .well-known/sx-manifest.json\n"
"{\"version\": 1,\n"
" \"components\": {\n"
@@ -514,7 +514,7 @@
(p "The fundamental primitive. Create a new CEK machine from a thunk, "
"run it concurrently, return a signal that resolves with the result:")
(~docs/code :code
(~docs/code :src
(str "(let ((result (spawn (fn () (* 6 7)))))
"
" ;; result is a signal \u2014 nil until the computation completes
@@ -534,7 +534,7 @@
(p "The default implementation is synchronous (no actual concurrency). "
"The host overrides " (code "spawn-impl") " for real concurrency:")
(~docs/code :code
(~docs/code :src
(str ";; Host-provided spawn (Web Worker)
"
"(set! spawn-impl
@@ -554,7 +554,7 @@
(p "Typed communication pipes between concurrent computations. "
"Send is non-blocking. Receive suspends via " (code "shift") " until data arrives:")
(~docs/code :code
(~docs/code :src
(str ";; Create a channel
"
"(make-channel \"results\")
@@ -599,7 +599,7 @@
(p "Spawn multiple computations, collect all results. "
"The spec primitive, not a library pattern:")
(~docs/code :code
(~docs/code :src
(str ";; Fork: run N computations concurrently
"
"(let ((results (fork-join (list
@@ -633,7 +633,7 @@
(li (strong "Cooperative") " \u2014 machines yield explicitly via " (code "yield!"))
(li (strong "DAG-ordered") " \u2014 step machines whose inputs are ready (Art DAG)"))
(~docs/code :code
(~docs/code :src
(str ";; Cooperative scheduling
"
"(spawn (fn ()
@@ -676,7 +676,7 @@
(li "Distribute: send input-CID to whichever worker has the data")
(li "Verify: any machine can re-run from input-CID and check output-CID matches"))
(~docs/code :code
(~docs/code :src
(str ";; Content-addressed spawn
"
"(let ((input-cid (freeze-to-cid \"job\")))

20
sx/sx/plans/generative-sx.sx
View File

@@ -32,7 +32,7 @@
(~docs/subsection :title "Homoiconicity"
(p "SX code is SX data. " (code "parse") " takes a string and returns a list. " (code "aser") " takes a list and returns a string. These round-trip perfectly. The program can read its own source as naturally as it reads a config file, because they're the same format.")
(~docs/code :code (highlight ";; Code is data\n(define source \"(+ 1 2)\")\n(define ast (parse source)) ;; → (list '+ 1 2)\n(define result (eval-expr ast env)) ;; → 3\n\n;; Data is code\n(define spec '(define greet (fn (name) (str \"Hello, \" name \"!\"))))\n(eval-expr spec env)\n(greet \"world\") ;; → \"Hello, world!\"" "lisp")))
(~docs/code :src (highlight ";; Code is data\n(define source \"(+ 1 2)\")\n(define ast (parse source)) ;; → (list '+ 1 2)\n(define result (eval-expr ast env)) ;; → 3\n\n;; Data is code\n(define spec '(define greet (fn (name) (str \"Hello, \" name \"!\"))))\n(eval-expr spec env)\n(greet \"world\") ;; → \"Hello, world!\"" "lisp")))
(~docs/subsection :title "Runtime eval"
(p (code "eval-expr") " is available at runtime, not just boot. " (code "data-init") " scripts already use it. Any SX string can become running code at any point in the program's execution. This is not " (code "eval()") " bolted onto a language that doesn't want it — it's the " (em "primary mechanism") " of the language."))
@@ -54,7 +54,7 @@
(~docs/section :title "The generative pattern" :id "pattern"
(p "A generative SX program starts with a seed and grows by evaluating its own output.")
(~docs/code :code (highlight ";; The core loop\n(define run\n (fn (env source)\n (let ((ast (parse source))\n (result (eval-expr ast env)))\n ;; result might be:\n ;; a value → done, return it\n ;; a string → new SX source, evaluate it (grow)\n ;; a list of defs → new definitions to add to env\n ;; a dict → {source: \"...\", env-patch: {...}} (grow + configure)\n (cond\n (string? result)\n (run env result) ;; evaluate the output\n (and (dict? result) (has-key? result \"source\"))\n (let ((patched (env-merge env (get result \"env-patch\"))))\n (run patched (get result \"source\")))\n :else\n result))))" "lisp"))
(~docs/code :src (highlight ";; The core loop\n(define run\n (fn (env source)\n (let ((ast (parse source))\n (result (eval-expr ast env)))\n ;; result might be:\n ;; a value → done, return it\n ;; a string → new SX source, evaluate it (grow)\n ;; a list of defs → new definitions to add to env\n ;; a dict → {source: \"...\", env-patch: {...}} (grow + configure)\n (cond\n (string? result)\n (run env result) ;; evaluate the output\n (and (dict? result) (has-key? result \"source\"))\n (let ((patched (env-merge env (get result \"env-patch\"))))\n (run patched (get result \"source\")))\n :else\n result))))" "lisp"))
(p "The program evaluates source. If the result is more source, it evaluates that too. Each iteration can extend the environment — add new functions, new macros, new primitives. The environment grows. The program becomes capable of things it couldn't do at the start.")
@@ -96,29 +96,29 @@
(~docs/subsection :title "1. The spec that compiles itself"
(p "Currently: " (code "bootstrap_js.py") " (Python) reads " (code "eval.sx") " (SX) and emits " (code "sx-browser.js") " (JavaScript). Three languages. Two of them are hosts, one is the spec.")
(p "Generative version: " (code "eval.sx") " evaluates itself with a code-generation adapter. The evaluator walks its own AST and emits the target language. No Python bootstrapper. No JavaScript template. The spec " (em "is") " the compiler.")
(~docs/code :code (highlight ";; bootstrap.sx — the spec compiles itself\n;;\n;; Load the codegen adapter for the target\n(define emit (load-adapter target)) ;; target = \"js\" | \"py\" | \"go\" | ...\n;;\n;; Read the spec files\n(define spec-source (read-file \"eval.sx\"))\n(define spec-ast (parse spec-source))\n;;\n;; Walk the AST and emit target code\n;; The walker IS the evaluator — eval.sx evaluating eval.sx\n;; with emit instead of execute\n(define target-code\n (eval-expr spec-ast\n (env-extend codegen-env\n ;; Override define, fn, if, etc. to emit instead of execute\n (codegen-special-forms emit))))\n;;\n(write-file (str \"sx-ref.\" (target-extension target)) target-code)" "lisp"))
(~docs/code :src (highlight ";; bootstrap.sx — the spec compiles itself\n;;\n;; Load the codegen adapter for the target\n(define emit (load-adapter target)) ;; target = \"js\" | \"py\" | \"go\" | ...\n;;\n;; Read the spec files\n(define spec-source (read-file \"eval.sx\"))\n(define spec-ast (parse spec-source))\n;;\n;; Walk the AST and emit target code\n;; The walker IS the evaluator — eval.sx evaluating eval.sx\n;; with emit instead of execute\n(define target-code\n (eval-expr spec-ast\n (env-extend codegen-env\n ;; Override define, fn, if, etc. to emit instead of execute\n (codegen-special-forms emit))))\n;;\n(write-file (str \"sx-ref.\" (target-extension target)) target-code)" "lisp"))
(p "This is the bootstrapper rewritten as a generative program. The spec reads itself, walks itself, and writes the output language. Adding a new target means writing a new " (code "load-adapter") " — a set of emitters for the SX special forms. The walker doesn't change. The spec doesn't change. Only the output format changes.")
(p "The current bootstrappers (" (code "bootstrap_js.py") ", " (code "bootstrap_py.py") ") would become the first two adapters. Future targets (Go, Rust, WASM) are additional adapters, written in SX and bootstrapped like everything else."))
(~docs/subsection :title "2. The program that discovers its dependencies"
(p (code "deps.sx") " analyzes component dependency graphs. It walks component ASTs, finds " (code "~plans/content-addressed-components/name") " references, computes transitive closures. This is the analytic mode — SX analyzing SX.")
(p "The generative version: a program that discovers it needs a component, searches for its definition (local files, IPFS, a registry), loads it, evaluates it, and continues. The program grows its own capability set at runtime.")
(~docs/code :code (highlight ";; A program that discovers and loads what it needs\n(define render-page\n (fn (page-name)\n (let ((page-def (lookup-component page-name)))\n (when (nil? page-def)\n ;; Component not found locally — fetch from registry\n (let ((source (fetch-component-source page-name)))\n ;; Evaluate the definition — it joins the environment\n (eval-expr (parse source) env)\n ;; Now it exists\n (set! page-def (lookup-component page-name))))\n ;; Render with all dependencies resolved\n (render-to-html (list page-def)))))" "lisp"))
(~docs/code :src (highlight ";; A program that discovers and loads what it needs\n(define render-page\n (fn (page-name)\n (let ((page-def (lookup-component page-name)))\n (when (nil? page-def)\n ;; Component not found locally — fetch from registry\n (let ((source (fetch-component-source page-name)))\n ;; Evaluate the definition — it joins the environment\n (eval-expr (parse source) env)\n ;; Now it exists\n (set! page-def (lookup-component page-name))))\n ;; Render with all dependencies resolved\n (render-to-html (list page-def)))))" "lisp"))
(p "This already happens in the browser. " (code "sx_response") " prepends missing component definitions as a " (code "data-components") " script block. The client evaluates them and they join the environment. The generative version makes this explicit: the program tells you what it needs, you give it source, it evaluates it, it grows."))
(~docs/subsection :title "3. The test suite that writes tests"
(p "Given a function's signature and a set of properties (" (code "prove.sx") " already has the property language), generate test cases that verify the properties. The program reads its own function definitions, generates SX expressions that test them, and evaluates those expressions.")
(~docs/code :code (highlight ";; Given: a function and properties about it\n(define-property string-reverse-involutory\n :forall ((s string?))\n :holds (= (reverse (reverse s)) s))\n\n;; Generate: test cases from the property\n;; The program reads the property, generates test source, evals it\n(define tests (generate-tests string-reverse-involutory))\n;; tests = list of (assert (= (reverse (reverse \"hello\")) \"hello\"))\n;; (assert (= (reverse (reverse \"\")) \"\"))\n;; (assert (= (reverse (reverse \"a\")) \"a\"))\n;; ... (random strings, edge cases)\n(for-each (fn (t) (eval-expr t env)) tests)" "lisp"))
(~docs/code :src (highlight ";; Given: a function and properties about it\n(define-property string-reverse-involutory\n :forall ((s string?))\n :holds (= (reverse (reverse s)) s))\n\n;; Generate: test cases from the property\n;; The program reads the property, generates test source, evals it\n(define tests (generate-tests string-reverse-involutory))\n;; tests = list of (assert (= (reverse (reverse \"hello\")) \"hello\"))\n;; (assert (= (reverse (reverse \"\")) \"\"))\n;; (assert (= (reverse (reverse \"a\")) \"a\"))\n;; ... (random strings, edge cases)\n(for-each (fn (t) (eval-expr t env)) tests)" "lisp"))
(p "The program analyzed itself (read the property), generated new SX (the test cases), and evaluated it (ran the tests). Three modes — analytic, synthetic, generative — in sequence."))
(~docs/subsection :title "4. The server that extends its own API"
(p "An SX server receives a request it doesn't know how to handle. Instead of returning 404, it examines the request, generates a handler, evaluates it, and handles the request.")
(~docs/code :code (highlight ";; A route handler that generates new route handlers\n(define handle-unknown-route\n (fn (path params)\n ;; Analyze what was requested\n (let ((segments (split path \"/\"))\n (resource (nth segments 1))\n (action (nth segments 2)))\n ;; Check if a schema exists for this resource\n (let ((schema (lookup-schema resource)))\n (when schema\n ;; Generate a CRUD handler from the schema\n (let ((handler-source (generate-crud-handler resource action schema)))\n ;; Evaluate it — the handler now exists\n (eval-expr (parse handler-source) env)\n ;; Route future requests to the generated handler\n (register-route path (env-get env (str resource \"-\" action)))\n ;; Handle this request with the new handler\n ((env-get env (str resource \"-\" action)) params)))))))" "lisp"))
(~docs/code :src (highlight ";; A route handler that generates new route handlers\n(define handle-unknown-route\n (fn (path params)\n ;; Analyze what was requested\n (let ((segments (split path \"/\"))\n (resource (nth segments 1))\n (action (nth segments 2)))\n ;; Check if a schema exists for this resource\n (let ((schema (lookup-schema resource)))\n (when schema\n ;; Generate a CRUD handler from the schema\n (let ((handler-source (generate-crud-handler resource action schema)))\n ;; Evaluate it — the handler now exists\n (eval-expr (parse handler-source) env)\n ;; Route future requests to the generated handler\n (register-route path (env-get env (str resource \"-\" action)))\n ;; Handle this request with the new handler\n ((env-get env (str resource \"-\" action)) params)))))))" "lisp"))
(p "This is not code generation in the Rails scaffolding sense — those generate files you then edit. This generates running code at runtime. The handler didn't exist. Now it does. The server grew."))
(~docs/subsection :title "5. The macro system that learns idioms"
(p "A generative macro system that detects repeated patterns in code and synthesizes macros to capture them. The program watches itself being written and abstracts its own patterns.")
(~docs/code :code (highlight ";; The program notices this pattern appearing repeatedly:\n;; (div :class \"card\" (h2 title) (p body) children...)\n;;\n;; It generates:\n(defmacro ~card (title body &rest children)\n (div :class \"card\"\n (h2 ,title)\n (p ,body)\n ,@children))\n;;\n;; And rewrites its own source to use the new macro.\n;; This is an SX program that:\n;; 1. Analyzed its own AST (found repeated subtrees)\n;; 2. Synthesized a macro (extracted the pattern)\n;; 3. Evaluated the macro definition (extended env)\n;; 4. Rewrote its own source (used the macro)\n;; Four generative steps." "lisp"))
(~docs/code :src (highlight ";; The program notices this pattern appearing repeatedly:\n;; (div :class \"card\" (h2 title) (p body) children...)\n;;\n;; It generates:\n(defmacro ~card (title body &rest children)\n (div :class \"card\"\n (h2 ,title)\n (p ,body)\n ,@children))\n;;\n;; And rewrites its own source to use the new macro.\n;; This is an SX program that:\n;; 1. Analyzed its own AST (found repeated subtrees)\n;; 2. Synthesized a macro (extracted the pattern)\n;; 3. Evaluated the macro definition (extended env)\n;; 4. Rewrote its own source (used the macro)\n;; Four generative steps." "lisp"))
(p "The connection to the Art DAG: each version of the source is content-addressed. The original (before macros) and the refactored (after macros) are both immutable nodes. The generative step is an edge in the DAG. You can always inspect what the program was before it rewrote itself.")))
;; =====================================================================
@@ -128,7 +128,7 @@
(~docs/section :title "The seed" :id "seed"
(p "A generative SX program starts with a seed. The seed must contain enough to bootstrap the generative loop: a parser, an evaluator, and the " (code "run") " function. Everything else is grown.")
(~docs/code :code (highlight ";; seed.sx — the minimal generative program\n;;\n;; This file contains:\n;; - The SX parser (parse)\n;; - The SX evaluator (eval-expr)\n;; - The generative loop (run)\n;; - A source acquisition function (next-source)\n;;\n;; Everything else — primitives, rendering, networking, persistence —\n;; is loaded by the program as it discovers it needs them.\n\n(define run\n (fn (env)\n (let ((source (next-source env)))\n (when source\n (let ((result (eval-expr (parse source) env)))\n (run env))))))\n\n;; Start with a bare environment\n(run (env-extend (dict\n \"parse\" parse\n \"eval-expr\" eval-expr\n \"next-source\" initial-source-fn)))" "lisp"))
(~docs/code :src (highlight ";; seed.sx — the minimal generative program\n;;\n;; This file contains:\n;; - The SX parser (parse)\n;; - The SX evaluator (eval-expr)\n;; - The generative loop (run)\n;; - A source acquisition function (next-source)\n;;\n;; Everything else — primitives, rendering, networking, persistence —\n;; is loaded by the program as it discovers it needs them.\n\n(define run\n (fn (env)\n (let ((source (next-source env)))\n (when source\n (let ((result (eval-expr (parse source) env)))\n (run env))))))\n\n;; Start with a bare environment\n(run (env-extend (dict\n \"parse\" parse\n \"eval-expr\" eval-expr\n \"next-source\" initial-source-fn)))" "lisp"))
(p "The seed is a quine that doesn't just reproduce itself — it " (em "extends") " itself. Each call to " (code "next-source") " returns new SX that the seed evaluates in its own environment. The environment grows. The seed's capabilities grow. But the seed itself never changes — it's the fixed point of the generative process.")
@@ -192,7 +192,7 @@
(~docs/subsection :title "Correct quotation and splicing"
(p "Quasiquote (" (code "`") "), unquote (" (code ",") "), and unquote-splicing (" (code ",@") ") must work correctly for programmatic code construction. The host needs these as first-class operations, not string concatenation.")
(p "A generative program builds code by template:")
(~docs/code :code (highlight ";; The generative program builds new definitions by template\n(define gen-handler\n (fn (name params body)\n `(define ,name\n (fn ,params\n ,@body))))\n\n;; gen-handler produces an AST, not a string\n;; The AST can be inspected, transformed, hashed, then evaluated\n(eval-in (gen-handler 'greet '(name) '((str \"Hello, \" name))) env)" "lisp"))
(~docs/code :src (highlight ";; The generative program builds new definitions by template\n(define gen-handler\n (fn (name params body)\n `(define ,name\n (fn ,params\n ,@body))))\n\n;; gen-handler produces an AST, not a string\n;; The AST can be inspected, transformed, hashed, then evaluated\n(eval-in (gen-handler 'greet '(name) '((str \"Hello, \" name))) env)" "lisp"))
(p "String concatenation would work — " (code "(str \"(define \" name \" ...)\")") " — but it's fragile, unstructured, and can't be inspected before evaluation. Quasiquote produces an AST. The generative program works with " (em "structure") ", not text."))
(~docs/subsection :title "Tail-call optimization"
@@ -229,7 +229,7 @@
(~docs/subsection :title "with-boundary as migration point"
(p "Execution migrates to where the primitives are. When the evaluator hits a " (code "with-boundary") " block requiring primitives the current host doesn't have, it serializes state (" (code "env-snapshot") "), ships the pending expression plus environment to a host that has them, and execution continues there. The block is the unit of migration, not individual primitive calls.")
(~docs/code :code (highlight "(with-boundary (media-processing encoding)\n (let ((frames (gpu-exec recipe cached-layers)))\n (encode-stream frames :codec \"h264\"\n :on-input-needed (fn (slot)\n (with-boundary (live-ingest)\n (open-feed :protocol \"webrtc\" :slot slot))))))" "lisp"))
(~docs/code :src (highlight "(with-boundary (media-processing encoding)\n (let ((frames (gpu-exec recipe cached-layers)))\n (encode-stream frames :codec \"h264\"\n :on-input-needed (fn (slot)\n (with-boundary (live-ingest)\n (open-feed :protocol \"webrtc\" :slot slot))))))" "lisp"))
(p "This program starts wherever it starts. When it hits " (code "(with-boundary (media-processing encoding) ...)") ", the evaluator checks: does the current host provide " (code "gpu-exec") " and " (code "encode-stream") "? If yes, evaluate in place. If no, snapshot the environment, serialize the pending expression, and dispatch to a host that does. Inside the encoding block, " (code ":on-input-needed") " triggers a nested migration — the " (code "(with-boundary (live-ingest) ...)") " block dispatches to an ingest server that provides " (code "open-feed") ".")
(p "The program doesn't know where it runs. It declares what it needs. The runtime figures out " (em "where") "."))

16
sx/sx/plans/isolated-evaluator.sx
View File

@@ -152,7 +152,7 @@
(p (strong "Goal:") " All real-world-touching JavaScript lives in one standalone module. The evaluator never directly accesses " (code "document") ", " (code "window") ", " (code "fetch") ", " (code "localStorage") ", " (code "history") ", etc.")
(h4 :class "font-semibold mt-4 mb-2" "Architecture")
(~docs/code :code (highlight " \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 sx-platform.js \u2502 \u2190 DOM, fetch, timers, storage\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 sx-evaluator.js \u2502 \u2502 sx-wasm-shim.js \u2502\n \u2502 (isolated JS) \u2502 \u2502 (WASM instance \u2502\n \u2502 \u2502 \u2502 + handle table) \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518" "text"))
(~docs/code :src (highlight " \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 sx-platform.js \u2502 \u2190 DOM, fetch, timers, storage\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 \u2502\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510 \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n \u2502 sx-evaluator.js \u2502 \u2502 sx-wasm-shim.js \u2502\n \u2502 (isolated JS) \u2502 \u2502 (WASM instance \u2502\n \u2502 \u2502 \u2502 + handle table) \u2502\n \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518 \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518" "text"))
(h4 :class "font-semibold mt-4 mb-2" "What moves into sx-platform.js")
(p "Extracted from " (code "platform_js.py") " string constants:")
@@ -208,15 +208,15 @@
(h4 :class "font-semibold mt-4 mb-2" "Core-only build")
(p "The bootstrapper already supports selecting which modules to compile. A core-only build:")
(~docs/code :code (highlight "# In run_js_sx.py \u2014 core-only build\ncompile_ref_to_js(\n adapters=[\"parser\", \"html\", \"sx\", \"dom\"], # core spec + adapters\n modules=None, # no signals, engine, orchestration, boot\n extensions=None, # no continuations\n spec_modules=None # no deps, router, cek, frames, page-helpers\n)" "python"))
(~docs/code :src (highlight "# In run_js_sx.py \u2014 core-only build\ncompile_ref_to_js(\n adapters=[\"parser\", \"html\", \"sx\", \"dom\"], # core spec + adapters\n modules=None, # no signals, engine, orchestration, boot\n extensions=None, # no continuations\n spec_modules=None # no deps, router, cek, frames, page-helpers\n)" "python"))
(h4 :class "font-semibold mt-4 mb-2" "Web framework loading")
(p "Web framework " (code ".sx") " files ship as " (code "<script type=\"text/sx-lib\">") " blocks. The platform boot shim evaluates them before component scripts:")
(~docs/code :code (highlight "<script src=\"/static/scripts/sx-platform.js\"></script>\n<script src=\"/static/scripts/sx-evaluator.js\"></script>\n<script type=\"text/sx-lib\">\n ;; concatenated web/ framework: signals, deps, router,\n ;; engine, orchestration, boot\n</script>\n<script type=\"text/sx\" data-components>\n ;; page component definitions\n</script>" "html"))
(~docs/code :src (highlight "<script src=\"/static/scripts/sx-platform.js\"></script>\n<script src=\"/static/scripts/sx-evaluator.js\"></script>\n<script type=\"text/sx-lib\">\n ;; concatenated web/ framework: signals, deps, router,\n ;; engine, orchestration, boot\n</script>\n<script type=\"text/sx\" data-components>\n ;; page component definitions\n</script>" "html"))
(h4 :class "font-semibold mt-4 mb-2" "Boot chicken-and-egg")
(p (code "boot.sx") " orchestrates the boot sequence but is itself web framework code. Solution: thin native boot shim (~30 lines) in " (code "sx-platform.js") ":")
(~docs/code :code (highlight "SxPlatform.boot = function(evaluator) {\n // 1. Evaluate web framework .sx libraries\n var libs = document.querySelectorAll('script[type=\"text/sx-lib\"]');\n for (var i = 0; i < libs.length; i++) {\n evaluator.evalSource(libs[i].textContent);\n }\n // 2. Call boot-init (defined in boot.sx)\n evaluator.callFunction('boot-init');\n};" "javascript"))
(~docs/code :src (highlight "SxPlatform.boot = function(evaluator) {\n // 1. Evaluate web framework .sx libraries\n var libs = document.querySelectorAll('script[type=\"text/sx-lib\"]');\n for (var i = 0; i < libs.length; i++) {\n evaluator.evalSource(libs[i].textContent);\n }\n // 2. Call boot-init (defined in boot.sx)\n evaluator.callFunction('boot-init');\n};" "javascript"))
(h4 :class "font-semibold mt-4 mb-2" "Performance")
(p "Parsing + evaluating ~5,000 lines of web framework " (code ".sx") " at startup takes ~10\u201350ms. After " (code "define") ", functions are Lambda objects dispatched identically to compiled functions. " (strong "Zero ongoing performance difference.")))
@@ -230,14 +230,14 @@
(p (strong "Goal:") " Rust evaluator calls " (code "sx-platform.js") " via wasm-bindgen imports. Handle table bridges DOM references.")
(h4 :class "font-semibold mt-4 mb-2" "Handle table (JS-side)")
(~docs/code :code (highlight "// In sx-wasm-shim.js\nconst handles = [null]; // index 0 = null handle\nfunction allocHandle(obj) { handles.push(obj); return handles.length - 1; }\nfunction getHandle(id) { return handles[id]; }\nfunction freeHandle(id) { handles[id] = null; }" "javascript"))
(~docs/code :src (highlight "// In sx-wasm-shim.js\nconst handles = [null]; // index 0 = null handle\nfunction allocHandle(obj) { handles.push(obj); return handles.length - 1; }\nfunction getHandle(id) { return handles[id]; }\nfunction freeHandle(id) { handles[id] = null; }" "javascript"))
(p "DOM nodes are JS objects. The handle table maps " (code "u32") " IDs to JS objects. Rust stores " (code "Value::Handle(u32)") " and passes the " (code "u32") " to imported JS functions.")
(h4 :class "font-semibold mt-4 mb-2" "Value::Handle in Rust")
(~docs/code :code (highlight "// In platform.rs\npub enum Value {\n // ... existing variants ...\n Handle(u32), // opaque reference to JS-side object\n}" "rust"))
(~docs/code :src (highlight "// In platform.rs\npub enum Value {\n // ... existing variants ...\n Handle(u32), // opaque reference to JS-side object\n}" "rust"))
(h4 :class "font-semibold mt-4 mb-2" "WASM imports from platform")
(~docs/code :code (highlight "#[wasm_bindgen(module = \"/sx-platform-wasm.js\")]\nextern \"C\" {\n fn platform_create_element(tag: &str) -> u32;\n fn platform_create_text_node(text: &str) -> u32;\n fn platform_set_attr(handle: u32, name: &str, value: &str);\n fn platform_append_child(parent: u32, child: u32);\n fn platform_add_event_listener(handle: u32, event: &str, callback_id: u32);\n // ... ~50 DOM primitives\n}" "rust"))
(~docs/code :src (highlight "#[wasm_bindgen(module = \"/sx-platform-wasm.js\")]\nextern \"C\" {\n fn platform_create_element(tag: &str) -> u32;\n fn platform_create_text_node(text: &str) -> u32;\n fn platform_set_attr(handle: u32, name: &str, value: &str);\n fn platform_append_child(parent: u32, child: u32);\n fn platform_add_event_listener(handle: u32, event: &str, callback_id: u32);\n // ... ~50 DOM primitives\n}" "rust"))
(h4 :class "font-semibold mt-4 mb-2" "Callback table for events")
(p "When Rust creates an event handler (a Lambda), it stores it in a callback table and gets a " (code "u32") " ID. JS " (code "addEventListener") " wraps it: when the event fires, JS calls into WASM with the callback ID. Rust looks up the Lambda and evaluates it.")
@@ -268,7 +268,7 @@
(li "Page is interactive"))
(h4 :class "font-semibold mt-4 mb-2" "Library dependency order")
(~docs/code :code (highlight "signals.sx \u2190 no SX deps (uses only core primitives)\ndeps.sx \u2190 no SX deps\nframes.sx \u2190 no SX deps\nrouter.sx \u2190 no SX deps\npage-helpers.sx \u2190 no SX deps\nengine.sx \u2190 uses render.sx (core), adapter-dom.sx (core)\norchestration.sx \u2190 depends on engine.sx\nboot.sx \u2190 depends on orchestration.sx" "text")))
(~docs/code :src (highlight "signals.sx \u2190 no SX deps (uses only core primitives)\ndeps.sx \u2190 no SX deps\nframes.sx \u2190 no SX deps\nrouter.sx \u2190 no SX deps\npage-helpers.sx \u2190 no SX deps\nengine.sx \u2190 uses render.sx (core), adapter-dom.sx (core)\norchestration.sx \u2190 depends on engine.sx\nboot.sx \u2190 depends on orchestration.sx" "text")))
;; -----------------------------------------------------------------------

24
sx/sx/plans/isomorphic.sx
View File

@@ -50,7 +50,7 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Transitive closure (deps.sx)")
(p "9 functions that walk the component graph. The core:")
(~docs/code :code (highlight "(define (transitive-deps name env)\n (let ((key (if (starts-with? name \"~\") name\n (concat \"~\" name)))\n (seen (set-create)))\n (transitive-deps-walk key env seen)\n (set-remove seen key)))" "lisp"))
(~docs/code :src (highlight "(define (transitive-deps name env)\n (let ((key (if (starts-with? name \"~\") name\n (concat \"~\" name)))\n (seen (set-create)))\n (transitive-deps-walk key env seen)\n (set-remove seen key)))" "lisp"))
(p (code "scan-refs") " walks a component body AST collecting " (code "~") " symbols. "
(code "transitive-deps") " follows references recursively through the env. "
(code "compute-all-deps") " batch-computes and caches deps for every component. "
@@ -58,7 +58,7 @@
(div
(h4 :class "font-semibold text-stone-700" "2. Page scanning")
(~docs/code :code (highlight "(define (components-needed page-source env)\n (let ((direct (scan-components-from-source page-source))\n (all-needed (set-create)))\n (for-each (fn (name) ...\n (set-add! all-needed name)\n (set-union! all-needed (component-deps comp)))\n direct)\n all-needed))" "lisp"))
(~docs/code :src (highlight "(define (components-needed page-source env)\n (let ((direct (scan-components-from-source page-source))\n (all-needed (set-create)))\n (for-each (fn (name) ...\n (set-add! all-needed name)\n (set-union! all-needed (component-deps comp)))\n direct)\n all-needed))" "lisp"))
(p (code "scan-components-from-source") " finds " (code "(~plans/content-addressed-components/name") " patterns in serialized SX via regex. " (code "components-needed") " combines scanning with the cached transitive closure to produce the minimal component set for a page."))
(div
@@ -110,12 +110,12 @@
(div
(h4 :class "font-semibold text-stone-700" "1. IO scanning")
(p (code "scan-io-refs") " walks an AST node, collecting symbol names that match an IO name set. The IO set is provided by the host from boundary declarations (all three tiers: core IO, deployment IO, page helpers).")
(~docs/code :code (highlight "(define scan-io-refs\n (fn (node io-names)\n (let ((refs (list)))\n (scan-io-refs-walk node io-names refs)\n refs)))" "lisp")))
(~docs/code :src (highlight "(define scan-io-refs\n (fn (node io-names)\n (let ((refs (list)))\n (scan-io-refs-walk node io-names refs)\n refs)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. Transitive IO closure")
(p (code "transitive-io-refs") " follows component deps recursively, unioning IO refs from all reachable components and macros. Cycle-safe via seen-set.")
(~docs/code :code (highlight "(define transitive-io-refs\n (fn (name env io-names)\n ;; Walk deps, scan each body for IO refs,\n ;; union all refs transitively.\n ...))" "lisp")))
(~docs/code :src (highlight "(define transitive-io-refs\n (fn (name env io-names)\n ;; Walk deps, scan each body for IO refs,\n ;; union all refs transitively.\n ...))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "3. Batch computation")
@@ -168,13 +168,13 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Route matching spec (router.sx)")
(p "New spec module with pure functions for Flask-style route pattern matching:")
(~docs/code :code (highlight "(define split-path-segments ;; \"/language/docs/hello\" → (\"docs\" \"hello\")\n(define parse-route-pattern ;; \"/language/docs/<slug>\" → segment descriptors\n(define match-route-segments ;; segments + pattern → params dict or nil\n(define find-matching-route ;; path + route table → first match" "lisp"))
(~docs/code :src (highlight "(define split-path-segments ;; \"/language/docs/hello\" → (\"docs\" \"hello\")\n(define parse-route-pattern ;; \"/language/docs/<slug>\" → segment descriptors\n(define match-route-segments ;; segments + pattern → params dict or nil\n(define find-matching-route ;; path + route table → first match" "lisp"))
(p "No platform interface needed — uses only pure string and list primitives. Bootstrapped to both hosts via " (code "--spec-modules deps,router") "."))
(div
(h4 :class "font-semibold text-stone-700" "2. Page registry")
(p "Server serializes defpage metadata as SX dict literals inside " (code "<script type=\"text/sx-pages\">") ". Each entry carries name, path pattern, auth level, has-data flag, serialized content expression, and closure values.")
(~docs/code :code (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(~docs/code :src (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(p "boot.sx processes these at startup using the SX parser — the same " (code "parse") " function from parser.sx — building route entries with parsed patterns into the " (code "_page-routes") " table. No JSON dependency."))
(div
@@ -243,7 +243,7 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Abstract resolve-page-data")
(p "Spec-level primitive in orchestration.sx. The spec says \"I need data for this page\" — platform provides transport:")
(~docs/code :code (highlight "(resolve-page-data page-name params\n (fn (data)\n ;; data is a dict — merge into env and render\n (let ((env (merge closure params data))\n (rendered (try-eval-content content-src env)))\n (swap-rendered-content target rendered pathname))))" "lisp"))
(~docs/code :src (highlight "(resolve-page-data page-name params\n (fn (data)\n ;; data is a dict — merge into env and render\n (let ((env (merge closure params data))\n (rendered (try-eval-content content-src env)))\n (swap-rendered-content target rendered pathname))))" "lisp"))
(p "Browser platform: HTTP fetch to " (code "/sx/data/<page-name>") ". Future platforms could use IPC, cache, WebSocket, etc."))
(div
@@ -353,7 +353,7 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Suspense component")
(p "When streaming, the server renders the page with " (code "~shared:pages/suspense") " placeholders instead of awaiting IO:")
(~docs/code :code (highlight "(~app-body\n :header-rows (~shared:pages/suspense :id \"stream-headers\"\n :fallback (div :class \"h-12 bg-stone-200 animate-pulse\"))\n :content (~shared:pages/suspense :id \"stream-content\"\n :fallback (div :class \"p-8 animate-pulse\" ...)))" "lisp")))
(~docs/code :src (highlight "(~app-body\n :header-rows (~shared:pages/suspense :id \"stream-headers\"\n :fallback (div :class \"h-12 bg-stone-200 animate-pulse\"))\n :content (~shared:pages/suspense :id \"stream-content\"\n :fallback (div :class \"p-8 animate-pulse\" ...)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. Chunked transfer")
@@ -366,7 +366,7 @@
(div
(h4 :class "font-semibold text-stone-700" "3. Client resolution")
(p "Each resolution chunk is an inline script:")
(~docs/code :code (highlight "<script>\n window.__sxResolve(\"stream-content\",\n \"(~article :title \\\"Hello\\\")\")\n</script>" "html"))
(~docs/code :src (highlight "<script>\n window.__sxResolve(\"stream-content\",\n \"(~article :title \\\"Hello\\\")\")\n</script>" "html"))
(p "The client parses the SX, renders to DOM, and replaces the suspense placeholder's children."))
(div
@@ -429,7 +429,7 @@
(p :class "text-green-800 text-sm" "Components declare where they prefer to render. The spec combines affinity with IO analysis to produce a per-component render target decision."))
(p "Affinity annotations let component authors express rendering preferences:")
(~docs/code :code (highlight "(defcomp ~plans/isomorphic/product-grid (&key products)\n :affinity :client ;; interactive, prefer client rendering\n (div ...))\n\n(defcomp ~plans/isomorphic/auth-menu (&key user)\n :affinity :server ;; auth-sensitive, always server\n (div ...))\n\n(defcomp ~plans/isomorphic/card (&key title)\n ;; no annotation = :affinity :auto (default)\n ;; runtime decides from IO analysis\n (div ...))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/isomorphic/product-grid (&key products)\n :affinity :client ;; interactive, prefer client rendering\n (div ...))\n\n(defcomp ~plans/isomorphic/auth-menu (&key user)\n :affinity :server ;; auth-sensitive, always server\n (div ...))\n\n(defcomp ~plans/isomorphic/card (&key title)\n ;; no annotation = :affinity :auto (default)\n ;; runtime decides from IO analysis\n (div ...))" "lisp"))
(p "The " (code "render-target") " function in deps.sx combines affinity with IO analysis:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -468,7 +468,7 @@
(p "Given component tree + IO dependency graph + affinity annotations, decide per-component: server-expand, client-render, or stream. Planning step cached at registration, recomputed on component change.")
(p (code "page-render-plan") " in deps.sx computes per-page boundary decisions:")
(~docs/code :code (highlight "(page-render-plan page-source env io-names)\n;; Returns:\n;; {:components {~plans/content-addressed-components/name \"server\"|\"client\" ...}\n;; :server (list of server-expanded names)\n;; :client (list of client-rendered names)\n;; :io-deps (IO primitives needed by server components)}" "lisp"))
(~docs/code :src (highlight "(page-render-plan page-source env io-names)\n;; Returns:\n;; {:components {~plans/content-addressed-components/name \"server\"|\"client\" ...}\n;; :server (list of server-expanded names)\n;; :client (list of client-rendered names)\n;; :io-deps (IO primitives needed by server components)}" "lisp"))
(~docs/subsection :title "Integration Points"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -495,7 +495,7 @@
(~docs/subsection :title "Cache Invalidation"
(p "Component authors can declare cache invalidation on elements that trigger mutations:")
(~docs/code :code (highlight ";; Clear specific page's cache after successful action\n(form :sx-post \"/cart/remove\"\n :sx-cache-invalidate \"cart-page\"\n ...)\n\n;; Clear ALL page caches after action\n(button :sx-post \"/admin/reset\"\n :sx-cache-invalidate \"*\")" "lisp"))
(~docs/code :src (highlight ";; Clear specific page's cache after successful action\n(form :sx-post \"/cart/remove\"\n :sx-cache-invalidate \"cart-page\"\n ...)\n\n;; Clear ALL page caches after action\n(button :sx-post \"/admin/reset\"\n :sx-cache-invalidate \"*\")" "lisp"))
(p "The server can also control client cache via response headers:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "SX-Cache-Invalidate: page-name") " — clear cache for a page")

12
sx/sx/plans/js-bootstrapper.sx
View File

@@ -33,7 +33,7 @@
(~docs/subsection :title "Mode 1: Spec Bootstrapper"
(p "Same job as " (code "bootstrap_js.py") ". Read spec " (code ".sx") " files, "
"emit " (code "sx-ref.js") ".")
(~docs/code :code (highlight ";; Translate eval.sx to JavaScript
(~docs/code :src (highlight ";; Translate eval.sx to JavaScript
(js-translate-file (parse-file \"eval.sx\"))
;; → \"function evalExpr(expr, env) { ... }\"
@@ -49,7 +49,7 @@
"Given a component tree that the server has already evaluated (data fetched, "
"conditionals resolved, loops expanded), " (code "js.sx") " compiles the "
"resulting DOM description into a JavaScript program that builds the same DOM.")
(~docs/code :code (highlight ";; Server evaluates the page (fetches data, expands components)
(~docs/code :src (highlight ";; Server evaluates the page (fetches data, expands components)
;; Result is a resolved SX tree: (div :class \"...\" (h1 \"Hello\") ...)
;; js.sx compiles that tree to standalone JS
@@ -247,20 +247,20 @@
(~docs/subsection :title "What Gets Compiled"
(p "A resolved SX tree like:")
(~docs/code :code (highlight "(div :class \"container\"
(~docs/code :src (highlight "(div :class \"container\"
(h1 \"Hello\")
(ul (map (fn (item)
(li :class \"item\" (get item \"name\")))
items)))" "lisp"))
(p "After server-side evaluation (with " (code "items") " = "
(code "[{\"name\": \"Alice\"}, {\"name\": \"Bob\"}]") "):")
(~docs/code :code (highlight "(div :class \"container\"
(~docs/code :src (highlight "(div :class \"container\"
(h1 \"Hello\")
(ul
(li :class \"item\" \"Alice\")
(li :class \"item\" \"Bob\")))" "lisp"))
(p "Compiles to:")
(~docs/code :code (highlight "var _0 = document.createElement('div');
(~docs/code :src (highlight "var _0 = document.createElement('div');
_0.className = 'container';
var _1 = document.createElement('h1');
_1.textContent = 'Hello';
@@ -416,7 +416,7 @@ _0.appendChild(_2);" "javascript")))
(li "Runtime slicing: analyze tree → include only necessary runtime modules")))
(~docs/subsection :title "Phase 5: Verification"
(~docs/code :code (highlight "# Mode 1: spec bootstrapper parity
(~docs/code :src (highlight "# Mode 1: spec bootstrapper parity
python bootstrap_js.py > sx-ref-g0.js
python run_js_sx.py > sx-ref-g1.js
diff sx-ref-g0.js sx-ref-g1.js # must be empty

10
sx/sx/plans/live-streaming.sx
View File

@@ -30,17 +30,17 @@
(~docs/subsection :title "SSE Protocol"
(p "A " (code "~live") " component declares a persistent connection to an SSE endpoint:")
(~docs/code :code (highlight "(~live :src \"/api/stream/dashboard\"\n (~shared:pages/suspense :id \"cpu\" :fallback (span \"Loading...\"))\n (~shared:pages/suspense :id \"memory\" :fallback (span \"Loading...\"))\n (~shared:pages/suspense :id \"requests\" :fallback (span \"Loading...\")))" "lisp"))
(~docs/code :src (highlight "(~live :src \"/api/stream/dashboard\"\n (~shared:pages/suspense :id \"cpu\" :fallback (span \"Loading...\"))\n (~shared:pages/suspense :id \"memory\" :fallback (span \"Loading...\"))\n (~shared:pages/suspense :id \"requests\" :fallback (span \"Loading...\")))" "lisp"))
(p "The server SSE endpoint yields SX resolve events:")
(~docs/code :code (highlight "async def dashboard_stream():\n while True:\n stats = await get_system_stats()\n yield sx_sse_event(\"cpu\", f'(~stat-badge :value \"{stats.cpu}%\")')\n yield sx_sse_event(\"memory\", f'(~stat-badge :value \"{stats.mem}%\")')\n await asyncio.sleep(1)" "python"))
(~docs/code :src (highlight "async def dashboard_stream():\n while True:\n stats = await get_system_stats()\n yield sx_sse_event(\"cpu\", f'(~stat-badge :value \"{stats.cpu}%\")')\n yield sx_sse_event(\"memory\", f'(~stat-badge :value \"{stats.mem}%\")')\n await asyncio.sleep(1)" "python"))
(p "SSE wire format — each event is a suspense resolve:")
(~docs/code :code (highlight "event: sx-resolve\ndata: {\"id\": \"cpu\", \"sx\": \"(~stat-badge :value \\\"42%\\\")\"}\n\nevent: sx-resolve\ndata: {\"id\": \"memory\", \"sx\": \"(~stat-badge :value \\\"68%\\\")\"}" "text")))
(~docs/code :src (highlight "event: sx-resolve\ndata: {\"id\": \"cpu\", \"sx\": \"(~stat-badge :value \\\"42%\\\")\"}\n\nevent: sx-resolve\ndata: {\"id\": \"memory\", \"sx\": \"(~stat-badge :value \\\"68%\\\")\"}" "text")))
(~docs/subsection :title "WebSocket Protocol"
(p "A " (code "~ws") " component establishes a bidirectional channel:")
(~docs/code :code (highlight "(~ws :src \"/ws/chat\"\n :on-message handle-chat-message\n (~shared:pages/suspense :id \"messages\" :fallback (div \"Connecting...\"))\n (~shared:pages/suspense :id \"typing\" :fallback (span)))" "lisp"))
(~docs/code :src (highlight "(~ws :src \"/ws/chat\"\n :on-message handle-chat-message\n (~shared:pages/suspense :id \"messages\" :fallback (div \"Connecting...\"))\n (~shared:pages/suspense :id \"typing\" :fallback (span)))" "lisp"))
(p "Client can send SX expressions back:")
(~docs/code :code (highlight ";; Client sends:\n(sx-send ws-conn '(chat-message :text \"hello\" :user \"alice\"))\n\n;; Server receives, broadcasts to all connected clients:\n;; event: sx-resolve for \"messages\" suspense" "lisp")))
(~docs/code :src (highlight ";; Client sends:\n(sx-send ws-conn '(chat-message :text \"hello\" :user \"alice\"))\n\n;; Server receives, broadcasts to all connected clients:\n;; event: sx-resolve for \"messages\" suspense" "lisp")))
(~docs/subsection :title "Shared Resolution Mechanism"
(p "All three transports use the same client-side resolution:")

32
sx/sx/plans/mother-language.sx
View File

@@ -167,7 +167,7 @@
" + " (code "primitives.sx") " + " (code "eval.sx") ", emits OCaml source. "
"Same pattern as the existing Rust/Python/JS bootstrappers.")
(p "The OCaml output is a standalone module:")
(~docs/code :code (highlight "type value =\n | Nil | Bool of bool | Num of float | Str of string\n | Sym of string | Kw of string\n | List of value list | Dict of (value * value) list\n | Lambda of params * value list * env\n | Component of string * params * value list * env\n | Handle of int (* opaque FFI reference *)\n\ntype frame =\n | IfFrame of value list * value list * env\n | ArgFrame of value list * value list * env\n | MapFrame of value * value list * value list * env\n | ReactiveResetFrame of value\n | DerefFrame of value\n (* ... 20+ frame types from frames.sx *)\n\ntype kont = frame list\ntype state = value * env * kont\n\nlet step ((ctrl, env, kont) : state) : state =\n match ctrl with\n | Lit v -> continue_val v kont\n | Var name -> continue_val (Env.find name env) kont\n | App (f, args) -> (f, env, ArgFrame(args, [], env) :: kont)\n | ..." "ocaml"))
(~docs/code :src (highlight "type value =\n | Nil | Bool of bool | Num of float | Str of string\n | Sym of string | Kw of string\n | List of value list | Dict of (value * value) list\n | Lambda of params * value list * env\n | Component of string * params * value list * env\n | Handle of int (* opaque FFI reference *)\n\ntype frame =\n | IfFrame of value list * value list * env\n | ArgFrame of value list * value list * env\n | MapFrame of value * value list * value list * env\n | ReactiveResetFrame of value\n | DerefFrame of value\n (* ... 20+ frame types from frames.sx *)\n\ntype kont = frame list\ntype state = value * env * kont\n\nlet step ((ctrl, env, kont) : state) : state =\n match ctrl with\n | Lit v -> continue_val v kont\n | Var name -> continue_val (Env.find name env) kont\n | App (f, args) -> (f, env, ArgFrame(args, [], env) :: kont)\n | ..." "ocaml"))
(h4 :class "font-semibold mt-6 mb-2" "Phase 2: Native + WASM builds")
(p "Compile the OCaml output to:")
@@ -186,7 +186,7 @@
(h4 :class "font-semibold mt-6 mb-2" "Phase 4: SX linearity checking")
(p "Extend " (code "types.sx") " with quantity annotations:")
(~docs/code :code (highlight ";; Quantity annotations on types\n(define-type (Signal a) :quantity :affine) ;; use at most once per scope\n(define-type (Channel a) :quantity :linear) ;; must be consumed exactly once\n\n;; Effect declarations with linearity\n(define-io-primitive \"send-message\"\n :params (channel message)\n :quantity :linear\n :effects [io]\n :doc \"Must be handled exactly once.\")\n\n;; The type checker (specced in .sx, compiled to OCaml) validates\n;; linearity at component registration time. Runtime enforcement\n;; by OCaml's one-shot continuations is the safety net." "lisp"))
(~docs/code :src (highlight ";; Quantity annotations on types\n(define-type (Signal a) :quantity :affine) ;; use at most once per scope\n(define-type (Channel a) :quantity :linear) ;; must be consumed exactly once\n\n;; Effect declarations with linearity\n(define-io-primitive \"send-message\"\n :params (channel message)\n :quantity :linear\n :effects [io]\n :doc \"Must be handled exactly once.\")\n\n;; The type checker (specced in .sx, compiled to OCaml) validates\n;; linearity at component registration time. Runtime enforcement\n;; by OCaml's one-shot continuations is the safety net." "lisp"))
(p "The type checker runs at spec-validation time. The compiled evaluator "
"executes already-verified code. SX's type system provides the linearity "
"guarantees, not the host language.")
@@ -200,7 +200,7 @@
(li "OCaml becomes the bootstrap language only \u2014 "
"needed once to get the self-hosting loop started, then never touched again"))
(~docs/code :code (highlight ";; The bootstrap chain\n\nStep 0: Python evaluator (existing)\n \u2193 evaluates bootstrap_ml.py\nStep 1: OCaml evaluator (compiled from spec by Python)\n \u2193 evaluates compiler.sx\nStep 2: SX compiler (compiled from .sx by OCaml evaluator)\n \u2193 compiles itself\nStep 3: SX compiler (compiled by itself)\n \u2193 compiles everything\n \u2193 emits native, WASM, JS from .sx spec\n \u2193 OCaml is no longer in the chain" "text"))
(~docs/code :src (highlight ";; The bootstrap chain\n\nStep 0: Python evaluator (existing)\n \u2193 evaluates bootstrap_ml.py\nStep 1: OCaml evaluator (compiled from spec by Python)\n \u2193 evaluates compiler.sx\nStep 2: SX compiler (compiled from .sx by OCaml evaluator)\n \u2193 compiles itself\nStep 3: SX compiler (compiled by itself)\n \u2193 compiles everything\n \u2193 emits native, WASM, JS from .sx spec\n \u2193 OCaml is no longer in the chain" "text"))
(p "At Step 3, the only language is SX. The compiler reads " (code ".sx") " and emits machine code. "
"OCaml was the scaffolding. The scaffolding comes down."))
@@ -314,7 +314,7 @@
"The client receives it and " (strong "compiles to WASM and executes") ". "
"Not interprets. Not dispatches bytecodes. Compiles.")
(~docs/code :code (highlight "Server sends: (defcomp ~card (&key title) (div :class \"card\" (h2 title)))\n\nClient does:\n 1. Parse SX source (fast \u2014 it's s-expressions)\n 2. Hash AST \u2192 CID\n 3. Cache hit? Call the already-compiled WASM function\n 4. Cache miss? Compile to WASM, cache by CID, call it\n\nStep 4 is the JIT. The compiler (itself WASM) emits a WASM\nfunction, instantiates it via WebAssembly.instantiate, caches\nthe module by CID. Next time: direct function call, zero overhead." "text"))
(~docs/code :src (highlight "Server sends: (defcomp ~card (&key title) (div :class \"card\" (h2 title)))\n\nClient does:\n 1. Parse SX source (fast \u2014 it's s-expressions)\n 2. Hash AST \u2192 CID\n 3. Cache hit? Call the already-compiled WASM function\n 4. Cache miss? Compile to WASM, cache by CID, call it\n\nStep 4 is the JIT. The compiler (itself WASM) emits a WASM\nfunction, instantiates it via WebAssembly.instantiate, caches\nthe module by CID. Next time: direct function call, zero overhead." "text"))
(p "This is what V8 does with JavaScript. What LuaJIT does with Lua. "
"The difference: SX's semantics are simpler (no prototype chains, no " (code "this")
@@ -324,7 +324,7 @@
(h4 :class "font-semibold mt-4 mb-2" "The compilation tiers")
(~docs/code :code (highlight "Tier 0: .sx source \u2192 tree-walking CEK (correct, slow \u2014 current)\nTier 1: .sx source \u2192 bytecodes \u2192 dispatch loop (correct, fast)\nTier 2: .sx source \u2192 WASM functions \u2192 execute (correct, fastest)\nTier 3: .sx source \u2192 native machine code (ahead-of-time, maximum)" "text"))
(~docs/code :src (highlight "Tier 0: .sx source \u2192 tree-walking CEK (correct, slow \u2014 current)\nTier 1: .sx source \u2192 bytecodes \u2192 dispatch loop (correct, fast)\nTier 2: .sx source \u2192 WASM functions \u2192 execute (correct, fastest)\nTier 3: .sx source \u2192 native machine code (ahead-of-time, maximum)" "text"))
(p "Each tier is faster. Tier 1 (bytecodes) is the "
(a :href "/sx/(etc.(plan.wasm-bytecode-vm))" "WASM Bytecode VM")
@@ -337,7 +337,7 @@
(p "The server can compile too. Instead of sending SX source for the client to JIT, "
"send precompiled WASM:")
(~docs/code :code (highlight ";; Option A: send SX source, client JIT compiles\nContent-Type: text/sx\n\n(div :class \"card\" (h2 \"hello\"))\n\n;; Option B: send precompiled WASM, client instantiates directly\nContent-Type: application/wasm\nX-Sx-Cid: bafyrei...\n\n<binary WASM module>" "text"))
(~docs/code :src (highlight ";; Option A: send SX source, client JIT compiles\nContent-Type: text/sx\n\n(div :class \"card\" (h2 \"hello\"))\n\n;; Option B: send precompiled WASM, client instantiates directly\nContent-Type: application/wasm\nX-Sx-Cid: bafyrei...\n\n<binary WASM module>" "text"))
(p "Option B skips parsing and compilation entirely. The client instantiates "
"the WASM module and calls it. The server did all the work.")
@@ -346,7 +346,7 @@
(p "Every " (code ".sx") " expression has a CID. Every compiled artifact has a CID. "
"The mapping is deterministic \u2014 the compiler is a pure function:")
(~docs/code :code (highlight "source CID \u2192 compiled WASM CID\nbafyrei... \u2192 bafyrei...\n\nThis mapping is cacheable everywhere:\n\u2022 Browser cache \u2014 first visitor compiles, second visitor gets cached WASM\n\u2022 CDN \u2014 compiled artifacts served at the edge\n\u2022 IPFS \u2014 content-addressed by definition, globally deduplicated\n\u2022 Local disk \u2014 offline apps work from cached compiled components" "text"))
(~docs/code :src (highlight "source CID \u2192 compiled WASM CID\nbafyrei... \u2192 bafyrei...\n\nThis mapping is cacheable everywhere:\n\u2022 Browser cache \u2014 first visitor compiles, second visitor gets cached WASM\n\u2022 CDN \u2014 compiled artifacts served at the edge\n\u2022 IPFS \u2014 content-addressed by definition, globally deduplicated\n\u2022 Local disk \u2014 offline apps work from cached compiled components" "text"))
(h4 :class "font-semibold mt-4 mb-2" "Entire apps as machine code")
(p "The entire application can be ahead-of-time compiled to a WASM binary. "
@@ -359,7 +359,7 @@
(h4 :class "font-semibold mt-4 mb-2" "The architecture")
(~docs/code :code (highlight "sx-platform.js \u2190 DOM, fetch, timers (the real world)\n \u2191 calls\nsx-compiler.wasm \u2190 the SX compiler (itself compiled to WASM)\n \u2191 compiles\n.sx source \u2190 received from server / cache / inline\n \u2193 emits\nnative WASM functions \u2190 cached by CID, instantiated on demand\n \u2193 executes\nactual DOM mutations via platform primitives" "text"))
(~docs/code :src (highlight "sx-platform.js \u2190 DOM, fetch, timers (the real world)\n \u2191 calls\nsx-compiler.wasm \u2190 the SX compiler (itself compiled to WASM)\n \u2191 compiles\n.sx source \u2190 received from server / cache / inline\n \u2193 emits\nnative WASM functions \u2190 cached by CID, instantiated on demand\n \u2193 executes\nactual DOM mutations via platform primitives" "text"))
(p "The compiler is WASM. The code it produces is WASM. "
"It's compiled code all the way down. "
@@ -410,19 +410,19 @@
(h4 :class "font-semibold mt-4 mb-2" "Content-addressed tamper detection")
(p "The server sends both SX source and precompiled WASM CID. The client can verify:")
(~docs/code :code (highlight ";; Server sends:\nContent-Type: application/wasm\nX-Sx-Source-Cid: bafyrei..source\nX-Sx-Compiled-Cid: bafyrei..compiled\n\n;; Client verifies (optional, configurable):\n1. Hash the WASM binary \u2192 matches X-Sx-Compiled-Cid?\n2. Compile source locally \u2192 produces same compiled CID?\n3. Check manifest of pinned CIDs \u2192 CID is expected?\n\n;; Any mismatch = tampered = reject" "text"))
(~docs/code :src (highlight ";; Server sends:\nContent-Type: application/wasm\nX-Sx-Source-Cid: bafyrei..source\nX-Sx-Compiled-Cid: bafyrei..compiled\n\n;; Client verifies (optional, configurable):\n1. Hash the WASM binary \u2192 matches X-Sx-Compiled-Cid?\n2. Compile source locally \u2192 produces same compiled CID?\n3. Check manifest of pinned CIDs \u2192 CID is expected?\n\n;; Any mismatch = tampered = reject" "text"))
(h4 :class "font-semibold mt-4 mb-2" "Capability attenuation per component")
(p "The platform scopes capabilities per evaluator instance. "
"App shell gets full access. Third-party or user-generated content gets the minimum:")
(~docs/code :code (highlight "// Full capabilities for the app shell\nplatform.registerAll(appShellCompiler);\n\n// Restricted for user-generated content\nplatform.registerSubset(userContentCompiler, {\n allow: [\"dom-create-element\", \"dom-set-attr\", \"dom-append\",\n \"dom-create-text-node\", \"dom-set-text\"],\n deny: [\"fetch\", \"localStorage\", \"dom-listen\",\n \"dom-set-inner-html\", \"eval\"]\n});\n\n// The restricted compiler's WASM module literally doesn't\n// have imports for the denied functions. Not just blocked\n// at runtime \u2014 absent from the binary." "javascript"))
(~docs/code :src (highlight "// Full capabilities for the app shell\nplatform.registerAll(appShellCompiler);\n\n// Restricted for user-generated content\nplatform.registerSubset(userContentCompiler, {\n allow: [\"dom-create-element\", \"dom-set-attr\", \"dom-append\",\n \"dom-create-text-node\", \"dom-set-text\"],\n deny: [\"fetch\", \"localStorage\", \"dom-listen\",\n \"dom-set-inner-html\", \"eval\"]\n});\n\n// The restricted compiler's WASM module literally doesn't\n// have imports for the denied functions. Not just blocked\n// at runtime \u2014 absent from the binary." "javascript"))
(h4 :class "font-semibold mt-4 mb-2" "Component manifests")
(p "The app ships with a manifest of expected CIDs for its core components. "
"Like subresource integrity (SRI) but for compiled code:")
(~docs/code :code (highlight ";; Component manifest (shipped with the app, signed)\n{\n \"~card\": \"bafyrei..abc\"\n \"~header\": \"bafyrei..def\"\n \"~nav-item\": \"bafyrei..ghi\"\n}\n\n;; On navigation: server sends component update\n;; Client compiles \u2192 checks CID against manifest\n;; Match = trusted, execute\n;; Mismatch = tampered, reject and report" "text"))
(~docs/code :src (highlight ";; Component manifest (shipped with the app, signed)\n{\n \"~card\": \"bafyrei..abc\"\n \"~header\": \"bafyrei..def\"\n \"~nav-item\": \"bafyrei..ghi\"\n}\n\n;; On navigation: server sends component update\n;; Client compiles \u2192 checks CID against manifest\n;; Match = trusted, execute\n;; Mismatch = tampered, reject and report" "text"))
(p "The security model is " (em "structural") ", not bolt-on. "
"WASM isolation, platform capabilities, content-addressed verification, "
@@ -444,7 +444,7 @@
(h4 :class "font-semibold mt-4 mb-2" "The rendering pipeline")
(~docs/code :code (highlight "Crawler visits:\n GET /page\n \u2192 Server compiles SX (native OCaml)\n \u2192 render-to-html (adapter-html.sx)\n \u2192 Full static HTML with semantic markup\n \u2192 Google indexes it\n\nUser first visit:\n GET /page\n \u2192 Server renders HTML (same as crawler)\n \u2192 Browser displays immediately (no JS needed)\n \u2192 Client loads sx-compiler.wasm + sx-platform.js\n \u2192 Hydrates: attaches event handlers, activates islands\n \u2192 Page is interactive\n\nUser navigates (SPA):\n sx-get /next-page\n \u2192 Server sends SX wire format (aser)\n \u2192 Client compiles + renders via WASM\n \u2192 Morph engine patches the DOM" "text"))
(~docs/code :src (highlight "Crawler visits:\n GET /page\n \u2192 Server compiles SX (native OCaml)\n \u2192 render-to-html (adapter-html.sx)\n \u2192 Full static HTML with semantic markup\n \u2192 Google indexes it\n\nUser first visit:\n GET /page\n \u2192 Server renders HTML (same as crawler)\n \u2192 Browser displays immediately (no JS needed)\n \u2192 Client loads sx-compiler.wasm + sx-platform.js\n \u2192 Hydrates: attaches event handlers, activates islands\n \u2192 Page is interactive\n\nUser navigates (SPA):\n sx-get /next-page\n \u2192 Server sends SX wire format (aser)\n \u2192 Client compiles + renders via WASM\n \u2192 Morph engine patches the DOM" "text"))
(p "The server and client have the " (em "same compiler") " from the " (em "same spec") ". "
(code "adapter-html.sx") " produces HTML strings. "
@@ -465,7 +465,7 @@
(p "The server can prerender every page to static HTML, hash it, "
"and cache it at the CDN edge:")
(~docs/code :code (highlight "Page source CID \u2192 Rendered HTML CID\nbafyrei..source \u2192 bafyrei..html\n\n\u2022 Crawler hits CDN \u2192 instant HTML, no server round-trip\n\u2022 Page content changes \u2192 new source CID \u2192 new HTML CID \u2192 CDN invalidated\n\u2022 Same CID = same HTML forever \u2192 infinite cache, zero revalidation" "text"))
(~docs/code :src (highlight "Page source CID \u2192 Rendered HTML CID\nbafyrei..source \u2192 bafyrei..html\n\n\u2022 Crawler hits CDN \u2192 instant HTML, no server round-trip\n\u2022 Page content changes \u2192 new source CID \u2192 new HTML CID \u2192 CDN invalidated\n\u2022 Same CID = same HTML forever \u2192 infinite cache, zero revalidation" "text"))
(p "This is the same content-addressed caching as compiled WASM, "
"applied to the HTML output. Both the compiled client code and "
@@ -534,11 +534,11 @@
"Tag names, class strings, attribute keys \u2014 all baked into the compiled component. "
"The only payload is the data that differs between instances.")
(~docs/code :code (highlight ";; HTML response: ~450 bytes\n<div class=\"card\"><h2>Hello</h2><p class=\"text-stone-600\">World</p></div>\n\n;; SX source response: ~60 bytes\n(~card :title \"Hello\" :body \"World\")\n\n;; Bytecode response: ~18 bytes\n[CALL_CID bafyrei..card] [STR \"Hello\"] [STR \"World\"]\n\n;; Gzipped bytecode: ~12 bytes" "text"))
(~docs/code :src (highlight ";; HTML response: ~450 bytes\n<div class=\"card\"><h2>Hello</h2><p class=\"text-stone-600\">World</p></div>\n\n;; SX source response: ~60 bytes\n(~card :title \"Hello\" :body \"World\")\n\n;; Bytecode response: ~18 bytes\n[CALL_CID bafyrei..card] [STR \"Hello\"] [STR \"World\"]\n\n;; Gzipped bytecode: ~12 bytes" "text"))
(p "For a list of 50 cards:")
(~docs/code :code (highlight "HTML: 50\u00d7 <div class=\"card\"><h2>...</h2>...</div> ~22 KB\nSX source: 50\u00d7 (~card :title \"...\" :body \"...\") ~3 KB\nBytecode: [CALL_CID] + 50\u00d7 [STR, STR] ~800 bytes\nGzipped bytecode: ~400 bytes" "text"))
(~docs/code :src (highlight "HTML: 50\u00d7 <div class=\"card\"><h2>...</h2>...</div> ~22 KB\nSX source: 50\u00d7 (~card :title \"...\" :body \"...\") ~3 KB\nBytecode: [CALL_CID] + 50\u00d7 [STR, STR] ~800 bytes\nGzipped bytecode: ~400 bytes" "text"))
(p "The markup structure is in the compiled component. "
"The class strings are in the compiled component. "
@@ -548,7 +548,7 @@
(h4 :class "font-semibold mt-4 mb-2" "Content negotiation")
(p "The client advertises what it has via request headers:")
(~docs/code :code (highlight ";; Client tells server what's cached\nAccept: application/sx-bytecode, text/sx, text/html\nX-Sx-Cached-Components: bafyrei..card, bafyrei..header, bafyrei..nav\n\n;; Server picks the smallest response:\n;; - Client has ~card cached? Send bytecode (data only)\n;; - Client has compiler but not ~card? Send SX source\n;; - Client has nothing? Send HTML" "text"))
(~docs/code :src (highlight ";; Client tells server what's cached\nAccept: application/sx-bytecode, text/sx, text/html\nX-Sx-Cached-Components: bafyrei..card, bafyrei..header, bafyrei..nav\n\n;; Server picks the smallest response:\n;; - Client has ~card cached? Send bytecode (data only)\n;; - Client has compiler but not ~card? Send SX source\n;; - Client has nothing? Send HTML" "text"))
(p "The server and client negotiate the optimal response format automatically. "
"First visit is HTML (full progressive enhancement). "

22
sx/sx/plans/nav-redesign.sx
View File

@@ -19,7 +19,7 @@
(~docs/subsection :title "Structure"
(p "One vertical column, centered. Each level is a row.")
(~docs/code :code (highlight ";; Home (nothing selected)\n;;\n;; [ sx ]\n;;\n;; Docs CSSX Reference Protocols Examples\n;; Essays Philosophy Specs Bootstrappers\n;; Testing Isomorphism Plans Reactive Islands\n\n\n;; Section selected (e.g. Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; Status Reader Macros Theorem Prover\n;; Self-Hosting JS Bootstrapper SX-Activity\n;; Predictive Prefetching Content-Addressed\n;; Environment Images Runtime Slicing Typed SX\n;; Fragment Protocol ...\n\n\n;; Page selected (e.g. Typed SX under Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; < Typed SX >\n;;\n;; [ page content here ]" "lisp")))
(~docs/code :src (highlight ";; Home (nothing selected)\n;;\n;; [ sx ]\n;;\n;; Docs CSSX Reference Protocols Examples\n;; Essays Philosophy Specs Bootstrappers\n;; Testing Isomorphism Plans Reactive Islands\n\n\n;; Section selected (e.g. Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; Status Reader Macros Theorem Prover\n;; Self-Hosting JS Bootstrapper SX-Activity\n;; Predictive Prefetching Content-Addressed\n;; Environment Images Runtime Slicing Typed SX\n;; Fragment Protocol ...\n\n\n;; Page selected (e.g. Typed SX under Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; < Typed SX >\n;;\n;; [ page content here ]" "lisp")))
(~docs/subsection :title "Rules"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
@@ -65,7 +65,7 @@
(~docs/subsection :title "Arrows"
(p "Left and right arrows are inline with the selected item name. They navigate to the previous/next sibling in the current list. Keyboard accessible: left/right arrow keys when the row is focused.")
(~docs/code :code (highlight ";; Arrow rendering\n;;\n;; < Plans >\n;;\n;; < is a link to /plans/content-addressed-components\n;; (the previous sibling in plans-nav-items)\n;; > is a link to /plans/fragment-protocol\n;; (the next sibling)\n;; \"Plans\" is a link to /plans/ (the section index)\n;;\n;; At the edges, the arrow wraps:\n;; first item: < wraps to last\n;; last item: > wraps to first" "lisp")))
(~docs/code :src (highlight ";; Arrow rendering\n;;\n;; < Plans >\n;;\n;; < is a link to /plans/content-addressed-components\n;; (the previous sibling in plans-nav-items)\n;; > is a link to /plans/fragment-protocol\n;; (the next sibling)\n;; \"Plans\" is a link to /plans/ (the section index)\n;;\n;; At the edges, the arrow wraps:\n;; first item: < wraps to last\n;; last item: > wraps to first" "lisp")))
(~docs/subsection :title "Transitions"
(p "Selecting an item: the list fades/collapses, the selected item moves to breadcrumb position, children appear below. This is an L0 morph — the server renders the new state, the client morphs. No JS animation library needed, just CSS transitions on the morph targets.")
@@ -78,11 +78,11 @@
(~docs/section :title "Data Model" :id "data"
(p "The current nav data is flat — each section has its own " (code "define") ". The new model is a single tree:")
(~docs/code :code (highlight "(define sx-nav-tree\n {:label \"sx\"\n :href \"/\"\n :children (list\n {:label \"Docs\"\n :href \"/language/docs/introduction\"\n :children docs-nav-items}\n {:label \"CSSX\"\n :href \"/applications/cssx/\"\n :children cssx-nav-items}\n {:label \"Reference\"\n :href \"/reference/\"\n :children reference-nav-items}\n {:label \"Protocols\"\n :href \"/applications/protocols/wire-format\"\n :children protocols-nav-items}\n {:label \"Examples\"\n :href \"/examples/click-to-load\"\n :children examples-nav-items}\n {:label \"Essays\"\n :href \"/etc/essays/\"\n :children essays-nav-items}\n {:label \"Philosophy\"\n :href \"/etc/philosophy/sx-manifesto\"\n :children philosophy-nav-items}\n {:label \"Specs\"\n :href \"/language/specs/\"\n :children specs-nav-items}\n {:label \"Bootstrappers\"\n :href \"/language/bootstrappers/\"\n :children bootstrappers-nav-items}\n {:label \"Testing\"\n :href \"/language/testing/\"\n :children testing-nav-items}\n {:label \"Isomorphism\"\n :href \"/geography/isomorphism/\"\n :children isomorphism-nav-items}\n {:label \"Plans\"\n :href \"/etc/plans/\"\n :children plans-nav-items}\n {:label \"Reactive Islands\"\n :href \"/reactive-islands/\"\n :children reactive-islands-nav-items})})" "lisp"))
(~docs/code :src (highlight "(define sx-nav-tree\n {:label \"sx\"\n :href \"/\"\n :children (list\n {:label \"Docs\"\n :href \"/language/docs/introduction\"\n :children docs-nav-items}\n {:label \"CSSX\"\n :href \"/applications/cssx/\"\n :children cssx-nav-items}\n {:label \"Reference\"\n :href \"/reference/\"\n :children reference-nav-items}\n {:label \"Protocols\"\n :href \"/applications/protocols/wire-format\"\n :children protocols-nav-items}\n {:label \"Examples\"\n :href \"/examples/click-to-load\"\n :children examples-nav-items}\n {:label \"Essays\"\n :href \"/etc/essays/\"\n :children essays-nav-items}\n {:label \"Philosophy\"\n :href \"/etc/philosophy/sx-manifesto\"\n :children philosophy-nav-items}\n {:label \"Specs\"\n :href \"/language/specs/\"\n :children specs-nav-items}\n {:label \"Bootstrappers\"\n :href \"/language/bootstrappers/\"\n :children bootstrappers-nav-items}\n {:label \"Testing\"\n :href \"/language/testing/\"\n :children testing-nav-items}\n {:label \"Isomorphism\"\n :href \"/geography/isomorphism/\"\n :children isomorphism-nav-items}\n {:label \"Plans\"\n :href \"/etc/plans/\"\n :children plans-nav-items}\n {:label \"Reactive Islands\"\n :href \"/reactive-islands/\"\n :children reactive-islands-nav-items})})" "lisp"))
(p "The existing per-section lists (" (code "docs-nav-items") ", " (code "plans-nav-items") ", etc.) remain unchanged — they just become the " (code ":children") " of tree nodes. Sub-sections that have their own sub-items can nest further:")
(~docs/code :code (highlight ";; Future: deeper nesting\n{:label \"Plans\"\n :href \"/etc/plans/\"\n :children (list\n {:label \"Status\" :href \"/etc/plans/status\"}\n {:label \"Bootstrappers\" :href \"/etc/plans/self-hosting-bootstrapper\"\n :children (list\n {:label \"py.sx\" :href \"/etc/plans/self-hosting-bootstrapper\"}\n {:label \"js.sx\" :href \"/etc/plans/js-bootstrapper\"})}\n ;; ...\n )}" "lisp"))
(~docs/code :src (highlight ";; Future: deeper nesting\n{:label \"Plans\"\n :href \"/etc/plans/\"\n :children (list\n {:label \"Status\" :href \"/etc/plans/status\"}\n {:label \"Bootstrappers\" :href \"/etc/plans/self-hosting-bootstrapper\"\n :children (list\n {:label \"py.sx\" :href \"/etc/plans/self-hosting-bootstrapper\"}\n {:label \"js.sx\" :href \"/etc/plans/js-bootstrapper\"})}\n ;; ...\n )}" "lisp"))
(p "The tree depth is unlimited. The nav component recurses."))
@@ -94,19 +94,19 @@
(p "Three new components replace the entire menu bar system:")
(~docs/subsection :title "~plans/nav-redesign/logo"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/logo ()\n (a :href \"/\"\n :sx-get \"/\" :sx-target \"#main-panel\" :sx-select \"#main-panel\"\n :sx-swap \"outerHTML\" :sx-push-url \"true\"\n :class \"block text-center py-4\"\n (span :class \"text-2xl font-bold text-violet-700\" \"sx\")))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/nav-redesign/logo ()\n (a :href \"/\"\n :sx-get \"/\" :sx-target \"#main-panel\" :sx-select \"#main-panel\"\n :sx-swap \"outerHTML\" :sx-push-url \"true\"\n :class \"block text-center py-4\"\n (span :class \"text-2xl font-bold text-violet-700\" \"sx\")))" "lisp"))
(p "Always at the top. Always centered. The anchor."))
(~docs/subsection :title "~plans/nav-redesign/nav-breadcrumb"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/nav-breadcrumb (&key path siblings level)\n ;; Renders one breadcrumb row: < Label >\n ;; path = the nav tree node for this level\n ;; siblings = list of sibling nodes (for arrow nav)\n ;; level = depth (controls text size/color)\n (let ((idx (find-index siblings path))\n (prev (nth siblings (mod (- idx 1) (len siblings))))\n (next (nth siblings (mod (+ idx 1) (len siblings)))))\n (div :class (str \"flex items-center justify-center gap-3 py-1\"\n (nav-level-classes level))\n (a :href (get prev \"href\")\n :sx-get (get prev \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Previous\"\n \"<\")\n (a :href (get path \"href\")\n :sx-get (get path \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"font-medium\"\n (get path \"label\"))\n (a :href (get next \"href\")\n :sx-get (get next \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Next\"\n \">\"))))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/nav-redesign/nav-breadcrumb (&key path siblings level)\n ;; Renders one breadcrumb row: < Label >\n ;; path = the nav tree node for this level\n ;; siblings = list of sibling nodes (for arrow nav)\n ;; level = depth (controls text size/color)\n (let ((idx (find-index siblings path))\n (prev (nth siblings (mod (- idx 1) (len siblings))))\n (next (nth siblings (mod (+ idx 1) (len siblings)))))\n (div :class (str \"flex items-center justify-center gap-3 py-1\"\n (nav-level-classes level))\n (a :href (get prev \"href\")\n :sx-get (get prev \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Previous\"\n \"<\")\n (a :href (get path \"href\")\n :sx-get (get path \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"font-medium\"\n (get path \"label\"))\n (a :href (get next \"href\")\n :sx-get (get next \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Next\"\n \">\"))))" "lisp"))
(p "One row per selected level. Shows the current node with left/right arrows to siblings."))
(~docs/subsection :title "~plans/nav-redesign/nav-list"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/nav-list (&key items level)\n ;; Renders a wrapped list of links — the children of the current level\n (div :class (str \"flex flex-wrap justify-center gap-x-4 gap-y-2 py-2\"\n (nav-level-classes level))\n (map (fn (item)\n (a :href (get item \"href\")\n :sx-get (get item \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"hover:text-violet-700 transition-colors\"\n (get item \"label\")))\n items)))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/nav-redesign/nav-list (&key items level)\n ;; Renders a wrapped list of links — the children of the current level\n (div :class (str \"flex flex-wrap justify-center gap-x-4 gap-y-2 py-2\"\n (nav-level-classes level))\n (map (fn (item)\n (a :href (get item \"href\")\n :sx-get (get item \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"hover:text-violet-700 transition-colors\"\n (get item \"label\")))\n items)))" "lisp"))
(p "The children of the current level, rendered as a centered wrapped list of plain links."))
(~docs/subsection :title "~plans/nav-redesign/nav — the composition"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/nav (&key trail children-items level)\n ;; trail = list of {node, siblings} from root to current\n ;; children-items = children of the deepest selected node\n ;; level = depth of children\n (div :class \"max-w-3xl mx-auto px-4\"\n ;; Logo\n (~plans/nav-redesign/logo)\n ;; Breadcrumb trail (one row per selected ancestor)\n (map-indexed (fn (i crumb)\n (~nav-breadcrumb\n :path (get crumb \"node\")\n :siblings (get crumb \"siblings\")\n :level (+ i 1)))\n trail)\n ;; Children of the deepest selected node\n (when children-items\n (~plans/nav-redesign/nav-list :items children-items :level level))))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/nav-redesign/nav (&key trail children-items level)\n ;; trail = list of {node, siblings} from root to current\n ;; children-items = children of the deepest selected node\n ;; level = depth of children\n (div :class \"max-w-3xl mx-auto px-4\"\n ;; Logo\n (~plans/nav-redesign/logo)\n ;; Breadcrumb trail (one row per selected ancestor)\n (map-indexed (fn (i crumb)\n (~nav-breadcrumb\n :path (get crumb \"node\")\n :siblings (get crumb \"siblings\")\n :level (+ i 1)))\n trail)\n ;; Children of the deepest selected node\n (when children-items\n (~plans/nav-redesign/nav-list :items children-items :level level))))" "lisp"))
(p "That's the entire navigation. Three small components composed. No bars, no dropdowns, no mobile variants.")))
;; -----------------------------------------------------------------------
@@ -116,7 +116,7 @@
(~docs/section :title "Path Resolution" :id "resolution"
(p "Given a URL path, compute the breadcrumb trail and children. This is a tree walk:")
(~docs/code :code (highlight "(define resolve-nav-path\n (fn (tree current-href)\n ;; Walk sx-nav-tree, find the node matching current-href,\n ;; return the trail of ancestors + current children.\n ;;\n ;; Returns: {:trail (list of {:node N :siblings S})\n ;; :children (list) or nil\n ;; :depth number}\n ;;\n ;; Example: current-href = \"/etc/plans/typed-sx\"\n ;; → trail: [{:node Plans :siblings [Docs, CSSX, ...]}\n ;; {:node Typed-SX :siblings [Status, Reader-Macros, ...]}]\n ;; → children: nil (leaf node)\n ;; → depth: 2\n (let ((result (walk-nav-tree tree current-href (list))))\n result)))" "lisp"))
(~docs/code :src (highlight "(define resolve-nav-path\n (fn (tree current-href)\n ;; Walk sx-nav-tree, find the node matching current-href,\n ;; return the trail of ancestors + current children.\n ;;\n ;; Returns: {:trail (list of {:node N :siblings S})\n ;; :children (list) or nil\n ;; :depth number}\n ;;\n ;; Example: current-href = \"/etc/plans/typed-sx\"\n ;; → trail: [{:node Plans :siblings [Docs, CSSX, ...]}\n ;; {:node Typed-SX :siblings [Status, Reader-Macros, ...]}]\n ;; → children: nil (leaf node)\n ;; → depth: 2\n (let ((result (walk-nav-tree tree current-href (list))))\n result)))" "lisp"))
(p "This runs server-side (it's a pure function, no IO). The layout component calls it with the current URL and passes the result to " (code "~plans/nav-redesign/nav") ". Same pattern as the current " (code "find-current") " but produces a richer result.")
@@ -181,11 +181,11 @@
(~docs/section :title "Layout Simplification" :id "layout"
(p "The defpage layout declarations currently specify section, sub-label, sub-href, sub-nav, selected — five params to configure two menu bars. The new layout takes one param: the nav trail.")
(~docs/code :code (highlight ";; Current (verbose, configures two bars)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-section\n :section \"Plans\"\n :sub-label \"Plans\"\n :sub-href \"/etc/plans/\"\n :sub-nav (~nav-data/section-nav :items plans-nav-items\n :current (find-current plans-nav-items slug))\n :selected (or (find-current plans-nav-items slug) \"\"))\n :content (...))\n\n;; New (one param, nav computed from URL)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-docs :path (str \"/etc/plans/\" slug))\n :content (...))" "lisp"))
(~docs/code :src (highlight ";; Current (verbose, configures two bars)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-section\n :section \"Plans\"\n :sub-label \"Plans\"\n :sub-href \"/etc/plans/\"\n :sub-nav (~nav-data/section-nav :items plans-nav-items\n :current (find-current plans-nav-items slug))\n :selected (or (find-current plans-nav-items slug) \"\"))\n :content (...))\n\n;; New (one param, nav computed from URL)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-docs :path (str \"/etc/plans/\" slug))\n :content (...))" "lisp"))
(p "The layout component computes the nav trail internally from the path and the nav tree. No more passing section names, sub-labels, or pre-built nav components through layout params.")
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/docs-layout-full (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~root-header-auto)\n (~sx-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\")))))\n\n(defcomp ~plans/nav-redesign/docs-layout-oob (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~oob-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\"))\n (~root-header-auto true))))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/nav-redesign/docs-layout-full (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~root-header-auto)\n (~sx-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\")))))\n\n(defcomp ~plans/nav-redesign/docs-layout-oob (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~oob-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\"))\n (~root-header-auto true))))" "lisp"))
(p "Two layout components instead of twelve. Every defpage in docs.sx simplifies from five layout params to one."))

30
sx/sx/plans/predictive-prefetch.sx
View File

@@ -96,17 +96,17 @@
(~docs/subsection :title "Eager Bundle"
(p "The server already computes per-page component bundles. For key navigation paths — the main nav bar, section nav — the server can include " (em "linked routes' components") " in the initial bundle, not just the current page's.")
(~docs/code :code (highlight ";; defpage metadata declares eager prefetch targets\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :auth :public\n :prefetch :eager ;; bundle deps for all linked pure routes\n :content (case slug ...))" "lisp"))
(~docs/code :src (highlight ";; defpage metadata declares eager prefetch targets\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :auth :public\n :prefetch :eager ;; bundle deps for all linked pure routes\n :content (case slug ...))" "lisp"))
(p "Implementation: " (code "components_for_page()") " already scans the page SX for component refs. Extend it to also scan for " (code "href") " attributes, match them against the page registry, and include those pages' deps in the bundle. The cost is a larger initial payload; the benefit is zero-latency navigation within a section."))
(~docs/subsection :title "Idle Timer"
(p "After page load and initial render, use " (code "requestIdleCallback") " (or a fallback " (code "setTimeout") ") to scan visible nav links and batch-prefetch their missing components in a single request.")
(~docs/code :code (highlight "(define prefetch-visible-links-on-idle\n (fn ()\n (request-idle-callback\n (fn ()\n (let ((links (dom-query-all \"a[href][sx-get]\"))\n (all-missing (list)))\n (for-each\n (fn (link)\n (let ((missing (compute-missing-deps\n (url-pathname (dom-get-attr link \"href\")))))\n (when missing\n (for-each (fn (d) (append! all-missing d))\n missing))))\n links)\n (when (not (empty? all-missing))\n (prefetch-components (dedupe all-missing))))))))" "lisp"))
(~docs/code :src (highlight "(define prefetch-visible-links-on-idle\n (fn ()\n (request-idle-callback\n (fn ()\n (let ((links (dom-query-all \"a[href][sx-get]\"))\n (all-missing (list)))\n (for-each\n (fn (link)\n (let ((missing (compute-missing-deps\n (url-pathname (dom-get-attr link \"href\")))))\n (when missing\n (for-each (fn (d) (append! all-missing d))\n missing))))\n links)\n (when (not (empty? all-missing))\n (prefetch-components (dedupe all-missing))))))))" "lisp"))
(p "Called once from " (code "boot-init") " after initial processing. Batches all missing deps into one network request. Low priority — browser handles it when idle."))
(~docs/subsection :title "Mouse Approach (Trajectory Prediction)"
(p "Don't wait for the cursor to reach the link — predict where it's heading. Track the last few " (code "mousemove") " events, extrapolate the trajectory, and if it points toward a link, start prefetching before the hover event fires.")
(~docs/code :code (highlight "(define bind-approach-prefetch\n (fn (container)\n ;; Track mouse trajectory within a nav container.\n ;; On each mousemove, extrapolate position ~200ms ahead.\n ;; If projected point intersects a link's bounding box,\n ;; prefetch that link's route deps.\n (let ((last-x 0) (last-y 0) (last-t 0)\n (prefetched (dict)))\n (dom-add-listener container \"mousemove\"\n (fn (e)\n (let ((now (timestamp))\n (dt (- now last-t)))\n (when (> dt 16) ;; ~60fps throttle\n (let ((vx (/ (- (event-x e) last-x) dt))\n (vy (/ (- (event-y e) last-y) dt))\n (px (+ (event-x e) (* vx 200)))\n (py (+ (event-y e) (* vy 200)))\n (target (dom-element-at-point px py)))\n (when (and target (dom-has-attr? target \"href\")\n (not (get prefetched\n (dom-get-attr target \"href\"))))\n (let ((href (dom-get-attr target \"href\")))\n (set! prefetched\n (merge prefetched {href true}))\n (prefetch-route-deps\n (url-pathname href)))))\n (set! last-x (event-x e))\n (set! last-y (event-y e))\n (set! last-t now))))))))" "lisp"))
(~docs/code :src (highlight "(define bind-approach-prefetch\n (fn (container)\n ;; Track mouse trajectory within a nav container.\n ;; On each mousemove, extrapolate position ~200ms ahead.\n ;; If projected point intersects a link's bounding box,\n ;; prefetch that link's route deps.\n (let ((last-x 0) (last-y 0) (last-t 0)\n (prefetched (dict)))\n (dom-add-listener container \"mousemove\"\n (fn (e)\n (let ((now (timestamp))\n (dt (- now last-t)))\n (when (> dt 16) ;; ~60fps throttle\n (let ((vx (/ (- (event-x e) last-x) dt))\n (vy (/ (- (event-y e) last-y) dt))\n (px (+ (event-x e) (* vx 200)))\n (py (+ (event-y e) (* vy 200)))\n (target (dom-element-at-point px py)))\n (when (and target (dom-has-attr? target \"href\")\n (not (get prefetched\n (dom-get-attr target \"href\"))))\n (let ((href (dom-get-attr target \"href\")))\n (set! prefetched\n (merge prefetched {href true}))\n (prefetch-route-deps\n (url-pathname href)))))\n (set! last-x (event-x e))\n (set! last-y (event-y e))\n (set! last-t now))))))))" "lisp"))
(p "This is the most speculative strategy — best suited for dense navigation areas (section sidebars, nav bars) where the cursor trajectory is a strong predictor. The " (code "prefetched") " dict prevents duplicate fetches within the same container interaction."))
(~docs/subsection :title "Components + Data (Hybrid Prefetch)"
@@ -118,12 +118,12 @@
(li "Server returns " (em "only data") " (JSON or SX bindings), not the full rendered page")
(li "Client evaluates the content expression with prefetched components + fetched data")
(li "Result: faster than full server render, no redundant component transfer"))
(~docs/code :code (highlight ";; Declarative: prefetch components, fetch data on click\n(defpage reference-page\n :path \"/reference/<slug>\"\n :auth :public\n :prefetch :components ;; prefetch components, data stays server-fetched\n :data (reference-data slug)\n :content (~reference/attrs-content :attrs attrs))\n\n;; On click, client-side flow:\n;; 1. Components already prefetched (from hover/idle)\n;; 2. GET /reference/attributes → server returns data bindings\n;; 3. Client evals (reference-data slug) result + content expr\n;; 4. Renders locally with cached components" "lisp"))
(~docs/code :src (highlight ";; Declarative: prefetch components, fetch data on click\n(defpage reference-page\n :path \"/reference/<slug>\"\n :auth :public\n :prefetch :components ;; prefetch components, data stays server-fetched\n :data (reference-data slug)\n :content (~reference/attrs-content :attrs attrs))\n\n;; On click, client-side flow:\n;; 1. Components already prefetched (from hover/idle)\n;; 2. GET /reference/attributes → server returns data bindings\n;; 3. Client evals (reference-data slug) result + content expr\n;; 4. Renders locally with cached components" "lisp"))
(p "This is a stepping stone toward full Phase 4 (client IO bridge) of the isomorphic roadmap — it achieves partial client rendering for data pages without needing a general-purpose client async evaluator. The server is a data service, the client is the renderer."))
(~docs/subsection :title "Declarative Configuration"
(p "All strategies configured via " (code "defpage") " metadata and " (code "sx-prefetch") " attributes on links/containers:")
(~docs/code :code (highlight ";; Page-level: what to prefetch for routes linking TO this page\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :prefetch :eager) ;; bundle with linking page\n\n(defpage reference-page\n :path \"/reference/<slug>\"\n :prefetch :components) ;; prefetch components, data on click\n\n;; Link-level: override per-link\n(a :href \"/language/docs/components\"\n :sx-prefetch \"idle\") ;; prefetch after page idle\n\n;; Container-level: approach prediction for nav areas\n(nav :sx-prefetch \"approach\"\n (a :href \"/language/docs/\") (a :href \"/reference/\") ...)" "lisp"))
(~docs/code :src (highlight ";; Page-level: what to prefetch for routes linking TO this page\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :prefetch :eager) ;; bundle with linking page\n\n(defpage reference-page\n :path \"/reference/<slug>\"\n :prefetch :components) ;; prefetch components, data on click\n\n;; Link-level: override per-link\n(a :href \"/language/docs/components\"\n :sx-prefetch \"idle\") ;; prefetch after page idle\n\n;; Container-level: approach prediction for nav areas\n(nav :sx-prefetch \"approach\"\n (a :href \"/language/docs/\") (a :href \"/reference/\") ...)" "lisp"))
(p "Priority cascade: explicit " (code "sx-prefetch") " on link > " (code ":prefetch") " on target defpage > default (hover). The system never prefetches the same components twice — " (code "_prefetch-pending") " and " (code "loaded-component-names") " handle dedup.")))
;; -----------------------------------------------------------------------
@@ -136,7 +136,7 @@
(~docs/subsection :title "Phase 1: Component Fetch Endpoint (Python)"
(p "A new " (strong "public") " endpoint (not " (code "/internal/") " — the client's browser calls it) that returns component definitions by name.")
(~docs/code :code (highlight "GET /<service-prefix>/sx/components?names=~plans/predictive-prefetch/card,~essay-foo\n\nResponse (text/sx):\n(defcomp ~plans/predictive-prefetch/card (&key title &rest children)\n (div :class \"border rounded p-4\" (h2 title) children))\n(defcomp ~plans/predictive-prefetch/essay-foo (&key id)\n (div (~plans/predictive-prefetch/card :title id)))" "http"))
(~docs/code :src (highlight "GET /<service-prefix>/sx/components?names=~plans/predictive-prefetch/card,~essay-foo\n\nResponse (text/sx):\n(defcomp ~plans/predictive-prefetch/card (&key title &rest children)\n (div :class \"border rounded p-4\" (h2 title) children))\n(defcomp ~plans/predictive-prefetch/essay-foo (&key id)\n (div (~plans/predictive-prefetch/card :title id)))" "http"))
(p "The server resolves transitive deps via " (code "deps.py") ", subtracts anything listed in the " (code "SX-Components") " request header (already loaded), serializes and returns. This is essentially " (code "components_for_request()") " driven by an explicit " (code "?names=") " param.")
(p "Cache-friendly: the response is a pure function of component hash + requested names. " (code "Cache-Control: public, max-age=3600") " with the component hash as ETag."))
@@ -147,41 +147,41 @@
(div
(h4 :class "font-semibold text-stone-700" "1. compute-missing-deps")
(p "Given a pathname, find the page, return dep names not in " (code "loaded-component-names") ". Returns nil if page not found or has data (can't client-route anyway).")
(~docs/code :code (highlight "(define compute-missing-deps\n (fn (pathname)\n (let ((match (find-matching-route pathname _page-routes)))\n (when (and match (not (get match \"has-data\")))\n (let ((deps (or (get match \"deps\") (list)))\n (loaded (loaded-component-names)))\n (filter (fn (d) (not (contains? loaded d))) deps))))))" "lisp")))
(~docs/code :src (highlight "(define compute-missing-deps\n (fn (pathname)\n (let ((match (find-matching-route pathname _page-routes)))\n (when (and match (not (get match \"has-data\")))\n (let ((deps (or (get match \"deps\") (list)))\n (loaded (loaded-component-names)))\n (filter (fn (d) (not (contains? loaded d))) deps))))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. prefetch-components")
(p "Fetch component definitions from the server for a list of names. Deduplicates in-flight requests. On success, parses and registers the returned definitions into the component env.")
(~docs/code :code (highlight "(define _prefetch-pending (dict))\n\n(define prefetch-components\n (fn (names)\n (let ((key (join \",\" (sort names))))\n (when (not (get _prefetch-pending key))\n (set! _prefetch-pending\n (merge _prefetch-pending {key true}))\n (fetch-components-from-server names\n (fn (sx-text)\n (sx-process-component-text sx-text)\n (dict-remove! _prefetch-pending key)))))))" "lisp")))
(~docs/code :src (highlight "(define _prefetch-pending (dict))\n\n(define prefetch-components\n (fn (names)\n (let ((key (join \",\" (sort names))))\n (when (not (get _prefetch-pending key))\n (set! _prefetch-pending\n (merge _prefetch-pending {key true}))\n (fetch-components-from-server names\n (fn (sx-text)\n (sx-process-component-text sx-text)\n (dict-remove! _prefetch-pending key)))))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "3. prefetch-route-deps")
(p "High-level composition: compute missing deps for a route, fetch if any.")
(~docs/code :code (highlight "(define prefetch-route-deps\n (fn (pathname)\n (let ((missing (compute-missing-deps pathname)))\n (when (and missing (not (empty? missing)))\n (log-info (str \"sx:prefetch \"\n (len missing) \" components for \" pathname))\n (prefetch-components missing)))))" "lisp")))
(~docs/code :src (highlight "(define prefetch-route-deps\n (fn (pathname)\n (let ((missing (compute-missing-deps pathname)))\n (when (and missing (not (empty? missing)))\n (log-info (str \"sx:prefetch \"\n (len missing) \" components for \" pathname))\n (prefetch-components missing)))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "4. Trigger: link hover")
(p "On mouseover of a boosted link, prefetch its route's missing components. Debounced 150ms to avoid fetching on quick mouse-throughs.")
(~docs/code :code (highlight "(define bind-prefetch-on-hover\n (fn (link)\n (let ((timer nil))\n (dom-add-listener link \"mouseover\"\n (fn (e)\n (clear-timeout timer)\n (set! timer (set-timeout\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n 150))))\n (dom-add-listener link \"mouseout\"\n (fn (e) (clear-timeout timer))))))" "lisp")))
(~docs/code :src (highlight "(define bind-prefetch-on-hover\n (fn (link)\n (let ((timer nil))\n (dom-add-listener link \"mouseover\"\n (fn (e)\n (clear-timeout timer)\n (set! timer (set-timeout\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n 150))))\n (dom-add-listener link \"mouseout\"\n (fn (e) (clear-timeout timer))))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "5. Trigger: viewport intersection (opt-in)")
(p "More aggressive strategy: when a link scrolls into view, prefetch its route's deps. Opt-in via " (code "sx-prefetch=\"visible\"") " attribute.")
(~docs/code :code (highlight "(define bind-prefetch-on-visible\n (fn (link)\n (observe-intersection link\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n true 0)))" "lisp")))
(~docs/code :src (highlight "(define bind-prefetch-on-visible\n (fn (link)\n (observe-intersection link\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n true 0)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "6. Integration into process-elements")
(p "During the existing hydration pass, for each boosted link:")
(~docs/code :code (highlight ";; In process-elements, after binding boost behavior:\n(when (and (should-boost-link? link)\n (dom-get-attr link \"href\"))\n (bind-prefetch-on-hover link))\n\n;; Explicit viewport prefetch:\n(when (dom-has-attr? link \"sx-prefetch\")\n (bind-prefetch-on-visible link))" "lisp")))))
(~docs/code :src (highlight ";; In process-elements, after binding boost behavior:\n(when (and (should-boost-link? link)\n (dom-get-attr link \"href\"))\n (bind-prefetch-on-hover link))\n\n;; Explicit viewport prefetch:\n(when (dom-has-attr? link \"sx-prefetch\")\n (bind-prefetch-on-visible link))" "lisp")))))
(~docs/subsection :title "Phase 3: Boundary Declaration"
(p "Two new IO primitives in " (code "boundary.sx") " (browser-only):")
(~docs/code :code (highlight ";; IO primitives (browser-only)\n(io fetch-components-from-server (names callback) -> void)\n(io sx-process-component-text (sx-text) -> void)" "lisp"))
(~docs/code :src (highlight ";; IO primitives (browser-only)\n(io fetch-components-from-server (names callback) -> void)\n(io sx-process-component-text (sx-text) -> void)" "lisp"))
(p "These are thin wrappers around " (code "fetch()") " + the existing component script processing logic already in the boundary adapter."))
(~docs/subsection :title "Phase 4: Bootstrap"
(p (code "bootstrap_js.py") " picks up the new functions from the spec and emits them into " (code "sx-browser.js") ". The two new boundary IO functions get implemented in the JS boundary adapter — the hand-written glue code that the bootstrapper doesn't generate.")
(~docs/code :code (highlight "// fetch-components-from-server: calls the endpoint\nfunction fetchComponentsFromServer(names, callback) {\n const url = `${routePrefix}/sx/components?names=${names.join(\",\")}`;\n const headers = {\n \"SX-Components\": loadedComponentNames().join(\",\")\n };\n fetch(url, { headers })\n .then(r => r.ok ? r.text() : \"\")\n .then(text => callback(text))\n .catch(() => {}); // silent fail — prefetch is best-effort\n}\n\n// sx-process-component-text: parse defcomp/defmacro into env\nfunction sxProcessComponentText(sxText) {\n if (!sxText) return;\n const frag = document.createElement(\"div\");\n frag.innerHTML =\n `<script type=\"text/sx\" data-components>${sxText}<\\/script>`;\n Sx.processScripts(frag);\n}" "javascript"))))
(~docs/code :src (highlight "// fetch-components-from-server: calls the endpoint\nfunction fetchComponentsFromServer(names, callback) {\n const url = `${routePrefix}/sx/components?names=${names.join(\",\")}`;\n const headers = {\n \"SX-Components\": loadedComponentNames().join(\",\")\n };\n fetch(url, { headers })\n .then(r => r.ok ? r.text() : \"\")\n .then(text => callback(text))\n .catch(() => {}); // silent fail — prefetch is best-effort\n}\n\n// sx-process-component-text: parse defcomp/defmacro into env\nfunction sxProcessComponentText(sxText) {\n if (!sxText) return;\n const frag = document.createElement(\"div\");\n frag.innerHTML =\n `<script type=\"text/sx\" data-components>${sxText}<\\/script>`;\n Sx.processScripts(frag);\n}" "javascript"))))
;; -----------------------------------------------------------------------
;; Request flow
@@ -189,7 +189,7 @@
(~docs/section :title "Request Flow" :id "request-flow"
(p "End-to-end example: user hovers a link, components prefetch, click goes client-side.")
(~docs/code :code (highlight "User hovers link \"/language/docs/sx-manifesto\"\n |\n +-- bind-prefetch-on-hover fires (150ms debounce)\n |\n +-- compute-missing-deps(\"/language/docs/sx-manifesto\")\n | +-- find-matching-route -> page with deps:\n | | [\"~essay-sx-manifesto\", \"~doc-code\"]\n | +-- loaded-component-names -> [\"~nav\", \"~footer\", \"~doc-code\"]\n | +-- missing: [\"~essay-sx-manifesto\"]\n |\n +-- prefetch-components([\"~essay-sx-manifesto\"])\n | +-- GET /sx/components?names=~essay-sx-manifesto\n | | Headers: SX-Components: ~plans/environment-images/nav,~footer,~doc-code\n | +-- Server resolves transitive deps\n | | (also needs ~plans/predictive-prefetch/rich-text, subtracts already-loaded)\n | +-- Response:\n | (defcomp ~plans/predictive-prefetch/essay-sx-manifesto ...) \n | (defcomp ~plans/predictive-prefetch/rich-text ...)\n |\n +-- sx-process-component-text registers defcomps in env\n |\n +-- User clicks link\n +-- try-client-route(\"/language/docs/sx-manifesto\")\n +-- has-all-deps? -> true (prefetched!)\n +-- eval content -> DOM\n +-- Client-side render, no server roundtrip" "text")))
(~docs/code :src (highlight "User hovers link \"/language/docs/sx-manifesto\"\n |\n +-- bind-prefetch-on-hover fires (150ms debounce)\n |\n +-- compute-missing-deps(\"/language/docs/sx-manifesto\")\n | +-- find-matching-route -> page with deps:\n | | [\"~essay-sx-manifesto\", \"~doc-code\"]\n | +-- loaded-component-names -> [\"~nav\", \"~footer\", \"~doc-code\"]\n | +-- missing: [\"~essay-sx-manifesto\"]\n |\n +-- prefetch-components([\"~essay-sx-manifesto\"])\n | +-- GET /sx/components?names=~essay-sx-manifesto\n | | Headers: SX-Components: ~plans/environment-images/nav,~footer,~doc-code\n | +-- Server resolves transitive deps\n | | (also needs ~plans/predictive-prefetch/rich-text, subtracts already-loaded)\n | +-- Response:\n | (defcomp ~plans/predictive-prefetch/essay-sx-manifesto ...) \n | (defcomp ~plans/predictive-prefetch/rich-text ...)\n |\n +-- sx-process-component-text registers defcomps in env\n |\n +-- User clicks link\n +-- try-client-route(\"/language/docs/sx-manifesto\")\n +-- has-all-deps? -> true (prefetched!)\n +-- eval content -> DOM\n +-- Client-side render, no server roundtrip" "text")))
;; -----------------------------------------------------------------------
;; File changes

18
sx/sx/plans/reader-macros.sx
View File

@@ -13,19 +13,19 @@
(~docs/subsection :title "Dispatch Character: #"
(p "Lisp tradition. " (code "#") " is NOT in " (code "ident-start") " or " (code "ident-char") " — completely free. Pattern:")
(~docs/code :code (highlight "#;expr → (read and discard expr, return next)\n#|...| → raw string literal\n#'expr → (quote expr)" "lisp")))
(~docs/code :src (highlight "#;expr → (read and discard expr, return next)\n#|...| → raw string literal\n#'expr → (quote expr)" "lisp")))
(~docs/subsection :title "#; — Datum comment"
(p "Scheme/Racket standard. Reads and discards the next expression. Preserves balanced parens.")
(~docs/code :code (highlight "(list 1 #;(this is commented out) 2 3) → (list 1 2 3)" "lisp")))
(~docs/code :src (highlight "(list 1 #;(this is commented out) 2 3) → (list 1 2 3)" "lisp")))
(~docs/subsection :title "#|...| — Raw string"
(p "No escape processing. Everything between " (code "#|") " and " (code "|") " is literal. Enables inline markdown, regex patterns, code examples.")
(~docs/code :code (highlight "(~md #|## Title\n\nSome **bold** text with \"quotes\" and \\backslashes.|)" "lisp")))
(~docs/code :src (highlight "(~md #|## Title\n\nSome **bold** text with \"quotes\" and \\backslashes.|)" "lisp")))
(~docs/subsection :title "#' — Quote shorthand"
(p "Currently no single-char quote (" (code "`") " is quasiquote).")
(~docs/code :code (highlight "#'my-function → (quote my-function)" "lisp")))
(~docs/code :src (highlight "#'my-function → (quote my-function)" "lisp")))
(~docs/subsection :title "Extensible dispatch: #name"
(p "User-defined reader macros via " (code "#name expr") ". The parser reads an identifier after " (code "#") ", looks up a handler in the reader macro registry, and calls it with the next parsed expression. See the " (a :href "/sx/(etc.(plan.reader-macro-demo))" :class "text-violet-600 hover:underline" "#z3 demo") " for a working example that translates SX spec declarations to SMT-LIB.")))
@@ -39,15 +39,15 @@
(~docs/subsection :title "1. Spec: parser.sx"
(p "Add " (code "#") " dispatch to " (code "read-expr") " (after the " (code ",") "/" (code ",@") " case, before number). Add " (code "read-raw-string") " helper function.")
(~docs/code :code (highlight ";; Reader macro dispatch\n(= ch \"#\")\n (do (set! pos (inc pos))\n (if (>= pos len-src)\n (error \"Unexpected end of input after #\")\n (let ((dispatch-ch (nth source pos)))\n (cond\n ;; #; — datum comment: read and discard next expr\n (= dispatch-ch \";\")\n (do (set! pos (inc pos))\n (read-expr) ;; read and discard\n (read-expr)) ;; return the NEXT expr\n\n ;; #| — raw string\n (= dispatch-ch \"|\")\n (do (set! pos (inc pos))\n (read-raw-string))\n\n ;; #' — quote shorthand\n (= dispatch-ch \"'\")\n (do (set! pos (inc pos))\n (list (make-symbol \"quote\") (read-expr)))\n\n :else\n (error (str \"Unknown reader macro: #\" dispatch-ch))))))" "lisp"))
(~docs/code :src (highlight ";; Reader macro dispatch\n(= ch \"#\")\n (do (set! pos (inc pos))\n (if (>= pos len-src)\n (error \"Unexpected end of input after #\")\n (let ((dispatch-ch (nth source pos)))\n (cond\n ;; #; — datum comment: read and discard next expr\n (= dispatch-ch \";\")\n (do (set! pos (inc pos))\n (read-expr) ;; read and discard\n (read-expr)) ;; return the NEXT expr\n\n ;; #| — raw string\n (= dispatch-ch \"|\")\n (do (set! pos (inc pos))\n (read-raw-string))\n\n ;; #' — quote shorthand\n (= dispatch-ch \"'\")\n (do (set! pos (inc pos))\n (list (make-symbol \"quote\") (read-expr)))\n\n :else\n (error (str \"Unknown reader macro: #\" dispatch-ch))))))" "lisp"))
(p "The " (code "read-raw-string") " helper:")
(~docs/code :code (highlight "(define read-raw-string\n (fn ()\n (let ((buf \"\"))\n (define raw-loop\n (fn ()\n (if (>= pos len-src)\n (error \"Unterminated raw string\")\n (let ((ch (nth source pos)))\n (if (= ch \"|\")\n (do (set! pos (inc pos)) nil) ;; done\n (do (set! buf (str buf ch))\n (set! pos (inc pos))\n (raw-loop)))))))\n (raw-loop)\n buf)))" "lisp")))
(~docs/code :src (highlight "(define read-raw-string\n (fn ()\n (let ((buf \"\"))\n (define raw-loop\n (fn ()\n (if (>= pos len-src)\n (error \"Unterminated raw string\")\n (let ((ch (nth source pos)))\n (if (= ch \"|\")\n (do (set! pos (inc pos)) nil) ;; done\n (do (set! buf (str buf ch))\n (set! pos (inc pos))\n (raw-loop)))))))\n (raw-loop)\n buf)))" "lisp")))
(~docs/subsection :title "2. Python: parser.py"
(p "Add " (code "#") " dispatch to " (code "_parse_expr()") " (after " (code ",") "/" (code ",@") " handling ~line 252). Add " (code "_read_raw_string()") " method to Tokenizer.")
(~docs/code :code (highlight "# In _parse_expr(), after the comma/splice-unquote block:\nif raw == \"#\":\n tok._advance(1) # consume the #\n if tok.pos >= len(tok.text):\n raise ParseError(\"Unexpected end of input after #\",\n tok.pos, tok.line, tok.col)\n dispatch = tok.text[tok.pos]\n if dispatch == \";\":\n tok._advance(1)\n _parse_expr(tok) # read and discard\n return _parse_expr(tok) # return next\n if dispatch == \"|\":\n tok._advance(1)\n return tok._read_raw_string()\n if dispatch == \"'\":\n tok._advance(1)\n return [Symbol(\"quote\"), _parse_expr(tok)]\n raise ParseError(f\"Unknown reader macro: #{dispatch}\",\n tok.pos, tok.line, tok.col)" "python"))
(~docs/code :src (highlight "# In _parse_expr(), after the comma/splice-unquote block:\nif raw == \"#\":\n tok._advance(1) # consume the #\n if tok.pos >= len(tok.text):\n raise ParseError(\"Unexpected end of input after #\",\n tok.pos, tok.line, tok.col)\n dispatch = tok.text[tok.pos]\n if dispatch == \";\":\n tok._advance(1)\n _parse_expr(tok) # read and discard\n return _parse_expr(tok) # return next\n if dispatch == \"|\":\n tok._advance(1)\n return tok._read_raw_string()\n if dispatch == \"'\":\n tok._advance(1)\n return [Symbol(\"quote\"), _parse_expr(tok)]\n raise ParseError(f\"Unknown reader macro: #{dispatch}\",\n tok.pos, tok.line, tok.col)" "python"))
(p "The " (code "_read_raw_string()") " method on Tokenizer:")
(~docs/code :code (highlight "def _read_raw_string(self) -> str:\n buf = []\n while self.pos < len(self.text):\n ch = self.text[self.pos]\n if ch == \"|\":\n self._advance(1)\n return \"\".join(buf)\n buf.append(ch)\n self._advance(1)\n raise ParseError(\"Unterminated raw string\",\n self.pos, self.line, self.col)" "python")))
(~docs/code :src (highlight "def _read_raw_string(self) -> str:\n buf = []\n while self.pos < len(self.text):\n ch = self.text[self.pos]\n if ch == \"|\":\n self._advance(1)\n return \"\".join(buf)\n buf.append(ch)\n self._advance(1)\n raise ParseError(\"Unterminated raw string\",\n self.pos, self.line, self.col)" "python")))
(~docs/subsection :title "3. JS: auto-transpiled"
(p "JS parser comes from bootstrap of parser.sx — spec change handles it automatically."))
@@ -57,7 +57,7 @@
(~docs/subsection :title "5. Grammar update"
(p "Add reader macro syntax to the grammar comment at the top of parser.sx:")
(~docs/code :code (highlight ";; reader → '#;' expr (datum comment)\n;; | '#|' raw-chars '|' (raw string)\n;; | \"#'\" expr (quote shorthand)" "lisp"))))
(~docs/code :src (highlight ";; reader → '#;' expr (datum comment)\n;; | '#|' raw-chars '|' (raw string)\n;; | \"#'\" expr (quote shorthand)" "lisp"))))
;; -----------------------------------------------------------------------
;; Files

14
sx/sx/plans/runtime-slicing.sx
View File

@@ -85,7 +85,7 @@
(p "Per the " (a :href "/sx/(etc.(plan.self-hosting-bootstrapper))" :class "text-violet-700 underline" "self-hosting principle") ", the slicer is not a build tool — it's a spec module. " (code "slice.sx") " analyzes the spec's own dependency graph and determines which defines belong to which tier.")
(p (code "js.sx") " (the self-hosting SX-to-JavaScript translator) already compiles the full spec. Slicing is a filter: " (code "js.sx") " translates only the defines that " (code "slice.sx") " selects for a given tier.")
(~docs/code :code (highlight ";; slice.sx — determine which defines each tier needs\n;;\n;; Input: the full list of defines from all spec files\n;; Output: a filtered list for the requested tier\n\n(define tier-deps\n ;; Which spec modules each tier requires\n {:L0 (list \"engine\" \"boot-partial\")\n :L1 (list \"engine\" \"boot-partial\" \"dom-partial\")\n :L2 (list \"engine\" \"boot-partial\" \"dom-partial\"\n \"signals\" \"dom-island\")\n :L3 (list \"eval\" \"render\" \"parser\"\n \"engine\" \"orchestration\" \"boot\"\n \"dom\" \"signals\" \"router\")})\n\n(define slice-defines\n (fn (tier all-defines)\n ;; 1. Get the module list for this tier\n ;; 2. Walk each define's dependency references\n ;; 3. Include a define only if ALL its deps are\n ;; satisfiable within the tier's module set\n ;; 4. Return the filtered define list\n (let ((modules (get tier-deps tier)))\n (filter\n (fn (d) (tier-satisfies? modules (define-deps d)))\n all-defines))))" "lisp"))
(~docs/code :src (highlight ";; slice.sx — determine which defines each tier needs\n;;\n;; Input: the full list of defines from all spec files\n;; Output: a filtered list for the requested tier\n\n(define tier-deps\n ;; Which spec modules each tier requires\n {:L0 (list \"engine\" \"boot-partial\")\n :L1 (list \"engine\" \"boot-partial\" \"dom-partial\")\n :L2 (list \"engine\" \"boot-partial\" \"dom-partial\"\n \"signals\" \"dom-island\")\n :L3 (list \"eval\" \"render\" \"parser\"\n \"engine\" \"orchestration\" \"boot\"\n \"dom\" \"signals\" \"router\")})\n\n(define slice-defines\n (fn (tier all-defines)\n ;; 1. Get the module list for this tier\n ;; 2. Walk each define's dependency references\n ;; 3. Include a define only if ALL its deps are\n ;; satisfiable within the tier's module set\n ;; 4. Return the filtered define list\n (let ((modules (get tier-deps tier)))\n (filter\n (fn (d) (tier-satisfies? modules (define-deps d)))\n all-defines))))" "lisp"))
(p "The pipeline becomes:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
@@ -103,7 +103,7 @@
(~docs/section :title "Define-Level Dependency Analysis" :id "deps"
(p "The slicer needs to know which defines reference which other defines. This is a simpler version of what " (code "deps.sx") " does for components — but at the define level, not the component level.")
(~docs/code :code (highlight ";; Walk a define's body AST, collect all symbol references\n;; that resolve to other top-level defines.\n;;\n;; (define morph-children\n;; (fn (old-node new-node)\n;; ...uses morph-node, morph-attrs, create-element...))\n;;\n;; → deps: #{morph-node morph-attrs create-element}\n\n(define define-refs\n (fn (body all-define-names)\n ;; Collect all symbols in body that appear in all-define-names\n (let ((refs (make-set)))\n (walk-ast body\n (fn (node)\n (when (and (symbol? node)\n (set-has? all-define-names (symbol-name node)))\n (set-add! refs (symbol-name node)))))\n refs)))" "lisp"))
(~docs/code :src (highlight ";; Walk a define's body AST, collect all symbol references\n;; that resolve to other top-level defines.\n;;\n;; (define morph-children\n;; (fn (old-node new-node)\n;; ...uses morph-node, morph-attrs, create-element...))\n;;\n;; → deps: #{morph-node morph-attrs create-element}\n\n(define define-refs\n (fn (body all-define-names)\n ;; Collect all symbols in body that appear in all-define-names\n (let ((refs (make-set)))\n (walk-ast body\n (fn (node)\n (when (and (symbol? node)\n (set-has? all-define-names (symbol-name node)))\n (set-add! refs (symbol-name node)))))\n refs)))" "lisp"))
(p "From these per-define deps, we build the full dependency graph and compute transitive closures. A tier's define set is the transitive closure of its entry points:")
@@ -139,7 +139,7 @@
(p "The bootstrapped code (from js.sx) is only half the story. Each define also depends on platform primitives — the hand-written JS glue that js.sx doesn't produce. These live in " (code "bootstrap_js.py") "'s PLATFORM_* blocks.")
(p "The slicer must track platform deps too:")
(~docs/code :code (highlight ";; Platform primitives referenced by tier\n;;\n;; L0 needs: createElement, setAttribute, morphAttrs,\n;; fetch (for sx-get/post), pushState, replaceState\n;;\n;; L1 adds: classList.toggle, addEventListener\n;;\n;; L2 adds: createComment, createTextNode, domRemove,\n;; domChildNodes — the reactive DOM primitives\n;;\n;; L3 adds: everything in PLATFORM_PARSER_JS,\n;; full DOM adapter, async IO bridge\n\n(define platform-deps\n {:L0 (list :dom-morph :fetch :history)\n :L1 (list :dom-morph :fetch :history :dom-events)\n :L2 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors)\n :L3 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors\n :parser :evaluator :async-io)})" "lisp"))
(~docs/code :src (highlight ";; Platform primitives referenced by tier\n;;\n;; L0 needs: createElement, setAttribute, morphAttrs,\n;; fetch (for sx-get/post), pushState, replaceState\n;;\n;; L1 adds: classList.toggle, addEventListener\n;;\n;; L2 adds: createComment, createTextNode, domRemove,\n;; domChildNodes — the reactive DOM primitives\n;;\n;; L3 adds: everything in PLATFORM_PARSER_JS,\n;; full DOM adapter, async IO bridge\n\n(define platform-deps\n {:L0 (list :dom-morph :fetch :history)\n :L1 (list :dom-morph :fetch :history :dom-events)\n :L2 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors)\n :L3 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors\n :parser :evaluator :async-io)})" "lisp"))
(p "The platform JS blocks in " (code "bootstrap_js.py") " are already organized by adapter (" (code "PLATFORM_DOM_JS") ", " (code "PLATFORM_ENGINE_PURE_JS") ", etc). Slicing further subdivides these into the minimal set each tier needs.")
(p "This subdivision also happens in SX: " (code "slice.sx") " declares which platform blocks each tier requires. " (code "js.sx") " doesn't need to change — it translates defines. The bootstrapper script reads the slice spec and assembles the platform preamble accordingly."))
@@ -152,11 +152,11 @@
(p "The simplest approach: one file per tier. The server knows each page's tier (from " (code "defpage") " metadata or component analysis) and serves the right script tag.")
(p "Better: a base file (L0) that all pages load, plus tier deltas loaded on demand.")
(~docs/code :code (highlight ";; Server emits the appropriate script for the page's tier\n;;\n;; L0 page (blog post, product listing):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;;\n;; L2 page (reactive island):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;; <script src=\"/static/scripts/sx-L2-delta.js\"></script>\n;;\n;; Client-side navigation from L0 → L2:\n;; 1. L0 runtime handles the swap\n;; 2. New page declares tier=L2 in response header\n;; 3. L0 runtime loads sx-L2-delta.js dynamically\n;; 4. Island hydration proceeds\n\n(define page-tier\n (fn (page)\n ;; Analyze the page's component tree\n ;; If any component is defisland → L2\n ;; If any component uses on-event/toggle! → L1\n ;; Otherwise → L0\n (cond\n ((page-has-islands? page) :L2)\n ((page-has-dom-ops? page) :L1)\n (true :L0))))" "lisp"))
(~docs/code :src (highlight ";; Server emits the appropriate script for the page's tier\n;;\n;; L0 page (blog post, product listing):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;;\n;; L2 page (reactive island):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;; <script src=\"/static/scripts/sx-L2-delta.js\"></script>\n;;\n;; Client-side navigation from L0 → L2:\n;; 1. L0 runtime handles the swap\n;; 2. New page declares tier=L2 in response header\n;; 3. L0 runtime loads sx-L2-delta.js dynamically\n;; 4. Island hydration proceeds\n\n(define page-tier\n (fn (page)\n ;; Analyze the page's component tree\n ;; If any component is defisland → L2\n ;; If any component uses on-event/toggle! → L1\n ;; Otherwise → L0\n (cond\n ((page-has-islands? page) :L2)\n ((page-has-dom-ops? page) :L1)\n (true :L0))))" "lisp"))
(~docs/subsection :title "SX-Tier Response Header"
(p "The server includes the page's tier in the response:")
(~docs/code :code (highlight "HTTP/1.1 200 OK\nSX-Tier: L0\nSX-Components: ~card:bafy...,~plans/environment-images/nav:bafy...\n\n;; or for an island page:\nSX-Tier: L2\nSX-Components: ~counter-island:bafy..." "http"))
(~docs/code :src (highlight "HTTP/1.1 200 OK\nSX-Tier: L0\nSX-Components: ~card:bafy...,~plans/environment-images/nav:bafy...\n\n;; or for an island page:\nSX-Tier: L2\nSX-Components: ~counter-island:bafy..." "http"))
(p "On client-side navigation, the engine reads " (code "SX-Tier") " from the response. If the new page requires a higher tier than currently loaded, it fetches the delta script before processing the swap. The delta script registers its additional primitives and the swap proceeds."))
(~docs/subsection :title "Cache Behavior"
@@ -170,7 +170,7 @@
(~docs/section :title "Automatic Tier Detection" :id "auto-detect"
(p (code "deps.sx") " already classifies components as pure or IO-dependent. Extend it to classify pages by tier:")
(~docs/code :code (highlight ";; Extend deps.sx with tier analysis\n;;\n;; Walk the page's component tree:\n;; - Any defisland → L2 minimum\n;; - Any on-event, toggle!, set-attr! call → L1 minimum \n;; - Any client-eval'd component (SX wire + defcomp) → L3\n;; - Otherwise → L0\n;;\n;; The tier is the MAX of all components' requirements.\n\n(define component-tier\n (fn (comp)\n (cond\n ((island? comp) :L2)\n ((has-dom-ops? (component-body comp)) :L1)\n (true :L0))))\n\n(define page-tier\n (fn (page-def)\n (let ((comp-tiers (map component-tier\n (page-all-components page-def))))\n (max-tier comp-tiers))))" "lisp"))
(~docs/code :src (highlight ";; Extend deps.sx with tier analysis\n;;\n;; Walk the page's component tree:\n;; - Any defisland → L2 minimum\n;; - Any on-event, toggle!, set-attr! call → L1 minimum \n;; - Any client-eval'd component (SX wire + defcomp) → L3\n;; - Otherwise → L0\n;;\n;; The tier is the MAX of all components' requirements.\n\n(define component-tier\n (fn (comp)\n (cond\n ((island? comp) :L2)\n ((has-dom-ops? (component-body comp)) :L1)\n (true :L0))))\n\n(define page-tier\n (fn (page-def)\n (let ((comp-tiers (map component-tier\n (page-all-components page-def))))\n (max-tier comp-tiers))))" "lisp"))
(p "This runs at registration time (same phase as " (code "compute_all_deps") "). Each " (code "PageDef") " gains a " (code "tier") " field. The server uses it to select the script tag. No manual annotation needed — the tier is derived from what the page actually uses."))
@@ -236,7 +236,7 @@
(~docs/section :title "Build Pipeline" :id "pipeline"
(p "The pipeline uses the same tools that already exist — " (code "js.sx") " for translation, " (code "bootstrap_js.py") " for platform assembly — but feeds them filtered define lists.")
(~docs/code :code (highlight ";; Build all tiers\n;;\n;; 1. Load all spec .sx files\n;; 2. Extract all defines (same as current bootstrap)\n;; 3. Run slice.sx to partition defines by tier\n;; 4. For each tier:\n;; a. js.sx translates the tier's define list\n;; b. Platform assembler wraps with minimal platform JS\n;; c. Output: sx-L{n}.js\n;; 5. Compute deltas: L1-delta = L1 - L0, L2-delta = L2 - L1, etc.\n\n;; The bootstrapper script orchestrates this:\n;;\n;; python bootstrap_js.py --tier L0 -o sx-L0.js\n;; python bootstrap_js.py --tier L1 --delta -o sx-L1-delta.js\n;; python bootstrap_js.py --tier L2 --delta -o sx-L2-delta.js\n;; python bootstrap_js.py -o sx-browser.js # full (L3, backward compat)" "lisp"))
(~docs/code :src (highlight ";; Build all tiers\n;;\n;; 1. Load all spec .sx files\n;; 2. Extract all defines (same as current bootstrap)\n;; 3. Run slice.sx to partition defines by tier\n;; 4. For each tier:\n;; a. js.sx translates the tier's define list\n;; b. Platform assembler wraps with minimal platform JS\n;; c. Output: sx-L{n}.js\n;; 5. Compute deltas: L1-delta = L1 - L0, L2-delta = L2 - L1, etc.\n\n;; The bootstrapper script orchestrates this:\n;;\n;; python bootstrap_js.py --tier L0 -o sx-L0.js\n;; python bootstrap_js.py --tier L1 --delta -o sx-L1-delta.js\n;; python bootstrap_js.py --tier L2 --delta -o sx-L2-delta.js\n;; python bootstrap_js.py -o sx-browser.js # full (L3, backward compat)" "lisp"))
(p "The " (code "--delta") " flag emits only the defines not present in the previous tier. The delta file calls " (code "Sx.extend()") " to register its additions into the already-loaded runtime.")

8
sx/sx/plans/rust-wasm-host.sx
View File

@@ -49,13 +49,13 @@
(h4 :class "font-semibold mt-4 mb-2" "Shared platform layer")
(p (code "platform_js.py") " already contains all DOM, browser, fetch, timer, and storage implementations — bootstrapped from " (code "boundary.sx") ". These are pure JavaScript functions that call browser APIs. They don't depend on the evaluator.")
(p "Extract them into a standalone " (code "sx-platform.js") " module. Both " (code "sx-browser.js") " (the current JS evaluator) and the new " (code "sx-wasm-shim.js") " import from the same platform module:")
(~docs/code :code (highlight " ┌─────────────────┐\n │ sx-platform.js │ ← DOM, fetch, timers, storage\n └────────┬────────┘\n │\n ┌──────────────┼──────────────┐\n │ │\n ┌─────────┴─────────┐ ┌────────┴────────┐\n │ sx-browser.js │ │ sx-wasm-shim.js │\n │ (JS tree-walker) │ │ (WASM instance │\n │ │ │ + handle table) │\n └────────────────────┘ └─────────────────┘" "text"))
(~docs/code :src (highlight " ┌─────────────────┐\n │ sx-platform.js │ ← DOM, fetch, timers, storage\n └────────┬────────┘\n │\n ┌──────────────┼──────────────┐\n │ │\n ┌─────────┴─────────┐ ┌────────┴────────┐\n │ sx-browser.js │ │ sx-wasm-shim.js │\n │ (JS tree-walker) │ │ (WASM instance │\n │ │ │ + handle table) │\n └────────────────────┘ └─────────────────┘" "text"))
(p "One codebase for all browser primitives. Bug fixes apply to both targets. The evaluator is the only thing that changes — JS tree-walker vs Rust/WASM tree-walker.")
(h4 :class "font-semibold mt-4 mb-2" "Opaque handle table")
(p "Rust/WASM can't hold DOM node references directly. Instead, Rust values use " (code "Value::Handle(u32)") " — an opaque integer that indexes into a JavaScript-side handle table:")
(~docs/code :code (highlight "// JS side (in sx-wasm-shim.js)\nconst handles = []; // handle_id → DOM node\n\nfunction allocHandle(node) {\n const id = handles.length;\n handles.push(node);\n return id;\n}\n\nfunction getHandle(id) { return handles[id]; }\nfunction freeHandle(id) { handles[id] = null; }" "javascript"))
(~docs/code :code (highlight "// Rust side\n#[derive(Clone, Debug)]\nenum Value {\n Nil,\n Bool(bool),\n Number(f64),\n Str(String),\n Symbol(String),\n Keyword(String),\n List(Vec<Value>),\n Dict(Vec<(Value, Value)>),\n Lambda(Rc<Closure>),\n Handle(u32), // ← opaque DOM node reference\n}" "rust"))
(~docs/code :src (highlight "// JS side (in sx-wasm-shim.js)\nconst handles = []; // handle_id → DOM node\n\nfunction allocHandle(node) {\n const id = handles.length;\n handles.push(node);\n return id;\n}\n\nfunction getHandle(id) { return handles[id]; }\nfunction freeHandle(id) { handles[id] = null; }" "javascript"))
(~docs/code :src (highlight "// Rust side\n#[derive(Clone, Debug)]\nenum Value {\n Nil,\n Bool(bool),\n Number(f64),\n Str(String),\n Symbol(String),\n Keyword(String),\n List(Vec<Value>),\n Dict(Vec<(Value, Value)>),\n Lambda(Rc<Closure>),\n Handle(u32), // ← opaque DOM node reference\n}" "rust"))
(p "When Rust calls a DOM primitive (e.g. " (code "createElement") "), it gets back a " (code "Handle(id)") ". When it passes that handle to " (code "appendChild") ", the JS shim looks up the real node. Rust never sees a DOM node — only integer IDs.")
(h4 :class "font-semibold mt-4 mb-2" "The JS shim is thin")
@@ -223,7 +223,7 @@
(li (strong "Step 2") " — " (code "sx-browser.js") " imports from " (code "sx-platform.js") " instead of containing the implementations inline")
(li (strong "Step 3") " — " (code "sx-wasm-shim.js") " imports from the same " (code "sx-platform.js") " and wires the functions as WASM imports"))
(p "Result: one implementation of every browser primitive, shared by both evaluator targets. Fix a bug in " (code "sx-platform.js") " and both JS and WASM evaluators get the fix.")
(~docs/code :code (highlight "// sx-platform.js (extracted from platform_js.py output)\nexport function createElement(tag) {\n return document.createElement(tag);\n}\nexport function setAttribute(el, key, val) {\n el.setAttribute(key, val);\n}\nexport function appendChild(parent, child) {\n parent.appendChild(child);\n}\nexport function addEventListener(el, event, callback) {\n el.addEventListener(event, callback);\n}\n// ... ~150 more browser primitives\n\n// sx-browser.js\nimport * as platform from './sx-platform.js';\n// Uses platform functions directly — evaluator is JS\n\n// sx-wasm-shim.js\nimport * as platform from './sx-platform.js';\n// Wraps platform functions for WASM import — evaluator is Rust" "javascript")))
(~docs/code :src (highlight "// sx-platform.js (extracted from platform_js.py output)\nexport function createElement(tag) {\n return document.createElement(tag);\n}\nexport function setAttribute(el, key, val) {\n el.setAttribute(key, val);\n}\nexport function appendChild(parent, child) {\n parent.appendChild(child);\n}\nexport function addEventListener(el, event, callback) {\n el.addEventListener(event, callback);\n}\n// ... ~150 more browser primitives\n\n// sx-browser.js\nimport * as platform from './sx-platform.js';\n// Uses platform functions directly — evaluator is JS\n\n// sx-wasm-shim.js\nimport * as platform from './sx-platform.js';\n// Wraps platform functions for WASM import — evaluator is Rust" "javascript")))
;; -----------------------------------------------------------------------
;; Interaction with other plans

18
sx/sx/plans/scoped-effects.sx
View File

@@ -127,7 +127,7 @@
(li (strong "Accumulator") " — data flowing " (em "upward") " from descendants (writable via " (code "emit!") ")")
(li (strong "Propagation mode") " — " (em "when") " effects on this scope are realised"))
(~docs/code :code (highlight "(scope \"name\"\n :value expr ;; downward (readable by descendants)\n :propagation :render ;; :render | :reactive | :morph\n body...)" "lisp"))
(~docs/code :src (highlight "(scope \"name\"\n :value expr ;; downward (readable by descendants)\n :propagation :render ;; :render | :reactive | :morph\n body...)" "lisp"))
(p "Every existing mechanism is a scope with a specific configuration:")
@@ -164,7 +164,7 @@
(td :class "py-2" ":reactive (cross-scope)"))))
(~docs/subsection :title "Scopes compose by nesting"
(~docs/code :code (highlight ";; An island with a theme context and a morphable lake\n(scope \"my-island\" :propagation :reactive\n (let ((colour (signal \"violet\")))\n (scope \"theme\" :value {:primary colour} :propagation :render\n (div\n (h1 :style (str \"color:\" (deref (get (context \"theme\") :primary)))\n \"Themed heading\")\n (scope \"product-details\" :propagation :morph\n ;; Server morphs this on navigation\n ;; Reactive attrs on the h1 are protected\n (~product-card :id 42))))))" "lisp"))
(~docs/code :src (highlight ";; An island with a theme context and a morphable lake\n(scope \"my-island\" :propagation :reactive\n (let ((colour (signal \"violet\")))\n (scope \"theme\" :value {:primary colour} :propagation :render\n (div\n (h1 :style (str \"color:\" (deref (get (context \"theme\") :primary)))\n \"Themed heading\")\n (scope \"product-details\" :propagation :morph\n ;; Server morphs this on navigation\n ;; Reactive attrs on the h1 are protected\n (~product-card :id 42))))))" "lisp"))
(p "Three scopes, three propagation modes, nested naturally. "
"The reactive scope manages signal lifecycle. "
"The render scope provides downward context. "
@@ -213,7 +213,7 @@
(p "The (Down ↓, Reactive) cell — " (strong "reactive context") " — "
"is the most interesting gap. It's React's Context + signals, but without "
"the re-render avalanche that makes React Context slow.")
(~docs/code :code (highlight ";; Reactive context: value is a signal, propagation is reactive\n(scope \"theme\" :value (signal {:primary \"violet\"}) :propagation :reactive\n ;; Any descendant reads with context + deref\n ;; Only the specific DOM node that uses the value updates\n (h1 :style (str \"color:\" (get (deref (context \"theme\")) :primary))\n \"This h1 updates when the theme signal changes\")\n ;; Deep nesting doesn't matter — it's O(1) per subscriber\n (~deeply-nested-component-tree))" "lisp"))
(~docs/code :src (highlight ";; Reactive context: value is a signal, propagation is reactive\n(scope \"theme\" :value (signal {:primary \"violet\"}) :propagation :reactive\n ;; Any descendant reads with context + deref\n ;; Only the specific DOM node that uses the value updates\n (h1 :style (str \"color:\" (get (deref (context \"theme\")) :primary))\n \"This h1 updates when the theme signal changes\")\n ;; Deep nesting doesn't matter — it's O(1) per subscriber\n (~deeply-nested-component-tree))" "lisp"))
(p "React re-renders " (em "every") " component that reads a changed context. "
"SX's reactive context updates " (em "only") " the DOM nodes that " (code "deref") " the signal. "
"Same API, fundamentally different performance characteristics.")))
@@ -280,7 +280,7 @@
(~docs/subsection :title "The effect hierarchy"
(p "Effects form a hierarchy. Lower-level effects are handled by higher-level scopes:")
(~docs/code :code (highlight ";; Effect hierarchy (innermost handled first)\n;;\n;; spread → element scope handles (merge attrs)\n;; emit! → nearest provide handles (accumulate)\n;; signal write → reactive scope handles (notify subscribers)\n;; morph → morph scope handles (diff + patch)\n;;\n;; If no handler is found, the effect bubbles up.\n;; Unhandled effects at the root are errors (like uncaught exceptions).\n;;\n;; This is why (emit! \"cssx\" rule) without a provider\n;; should error — there's no handler for the effect." "lisp"))
(~docs/code :src (highlight ";; Effect hierarchy (innermost handled first)\n;;\n;; spread → element scope handles (merge attrs)\n;; emit! → nearest provide handles (accumulate)\n;; signal write → reactive scope handles (notify subscribers)\n;; morph → morph scope handles (diff + patch)\n;;\n;; If no handler is found, the effect bubbles up.\n;; Unhandled effects at the root are errors (like uncaught exceptions).\n;;\n;; This is why (emit! \"cssx\" rule) without a provider\n;; should error — there's no handler for the effect." "lisp"))
(p "The current " (code "collect!") " is a global accumulator — effectively an "
"implicit top-level handler. Under the scope model, it would be an explicit "
(code "provide") " in the layout that handles " (code "\"cssx\"") " effects.")))
@@ -307,7 +307,7 @@
(~docs/subsection :title "2. Effect composition"
(p "Because scopes nest, effects compose naturally:")
(~docs/code :code (highlight ";; Animation scope inside a reactive scope\n(scope \"island\" :propagation :reactive\n (let ((items (signal large-list)))\n (scope \"smooth\" :propagation :animation\n ;; Updates to items are batched per frame\n ;; The reactive scope tracks deps\n ;; The animation scope throttles DOM writes\n (for-each (fn (item)\n (div (get item \"name\"))) (deref items)))))" "lisp"))
(~docs/code :src (highlight ";; Animation scope inside a reactive scope\n(scope \"island\" :propagation :reactive\n (let ((items (signal large-list)))\n (scope \"smooth\" :propagation :animation\n ;; Updates to items are batched per frame\n ;; The reactive scope tracks deps\n ;; The animation scope throttles DOM writes\n (for-each (fn (item)\n (div (get item \"name\"))) (deref items)))))" "lisp"))
(p "The reactive scope notifies when " (code "items") " changes. "
"The animation scope batches the resulting DOM writes to the next frame. "
"Neither scope knows about the other. They compose by nesting."))
@@ -315,7 +315,7 @@
(~docs/subsection :title "3. Server/client as effect boundary"
(p "The deepest consequence: the server/client boundary becomes " (em "just another "
"scope boundary") ". What crosses it is determined by the propagation mode:")
(~docs/code :code (highlight ";; The server renders this:\n(scope \"page\" :propagation :render\n (scope \"header\" :propagation :reactive\n ;; Client takes over — reactive scope\n (island-body...))\n (scope \"content\" :propagation :morph\n ;; Server controls — morphed on navigation\n (page-content...))\n (scope \"footer\" :propagation :render\n ;; Static — rendered once, never changes\n (footer...)))" "lisp"))
(~docs/code :src (highlight ";; The server renders this:\n(scope \"page\" :propagation :render\n (scope \"header\" :propagation :reactive\n ;; Client takes over — reactive scope\n (island-body...))\n (scope \"content\" :propagation :morph\n ;; Server controls — morphed on navigation\n (page-content...))\n (scope \"footer\" :propagation :render\n ;; Static — rendered once, never changes\n (footer...)))" "lisp"))
(p "The renderer walks the tree. When it hits a reactive scope, it serialises "
"state and emits a hydration marker. When it hits a morph scope, it emits "
"a lake marker. When it hits a render scope, it just renders. "
@@ -326,7 +326,7 @@
(p "Named stores (" (code "def-store/use-store") ") are scopes that transcend "
"the render tree. Two islands sharing a signal is two reactive scopes "
"referencing the same named scope:")
(~docs/code :code (highlight ";; Two islands, one shared scope\n(scope \"cart\" :value (signal []) :propagation :reactive :global true\n ;; Any island can read/write the cart\n ;; The scope transcends the render tree\n ;; Signal propagation handles cross-island updates)" "lisp"))
(~docs/code :src (highlight ";; Two islands, one shared scope\n(scope \"cart\" :value (signal []) :propagation :reactive :global true\n ;; Any island can read/write the cart\n ;; The scope transcends the render tree\n ;; Signal propagation handles cross-island updates)" "lisp"))
(p "The " (code ":global") " flag lifts the scope out of the tree hierarchy "
"into a named registry. " (code "def-store") " is syntax sugar for this.")))
@@ -357,7 +357,7 @@
(code "collect!") " creates a lazy root scope with deduplication. "
"All adapters use " (code "scope-push!/scope-pop!") " directly.")
(p "The unified platform structure:")
(~docs/code :code (highlight "_scope_stacks = {} ;; {name: [{value, emitted: [], dedup: bool}]}" "python"))
(~docs/code :src (highlight "_scope_stacks = {} ;; {name: [{value, emitted: [], dedup: bool}]}" "python"))
(p "See "
(a :href "/sx/(geography.(scopes))" :class "text-violet-600 hover:underline" "scopes article")
"."))
@@ -365,7 +365,7 @@
(~docs/subsection :title "Phase 3: effect handlers (future)"
(p "Make propagation modes extensible. A " (code ":propagation") " value is a "
"handler function that determines when and how effects are realised:")
(~docs/code :code (highlight ";; Custom propagation mode\n(define :debounced\n (fn (emit-fn delay)\n ;; Returns a handler that debounces effect realisation\n (let ((timer nil))\n (fn (effect)\n (when timer (clear-timeout timer))\n (set! timer (set-timeout\n (fn () (emit-fn effect)) delay))))))\n\n;; Use it\n(scope \"search\" :propagation (:debounced 300)\n ;; Effects in this scope are debounced by 300ms\n (input :on-input (fn (e)\n (emit! \"search\" (get-value e)))))" "lisp"))
(~docs/code :src (highlight ";; Custom propagation mode\n(define :debounced\n (fn (emit-fn delay)\n ;; Returns a handler that debounces effect realisation\n (let ((timer nil))\n (fn (effect)\n (when timer (clear-timeout timer))\n (set! timer (set-timeout\n (fn () (emit-fn effect)) delay))))))\n\n;; Use it\n(scope \"search\" :propagation (:debounced 300)\n ;; Effects in this scope are debounced by 300ms\n (input :on-input (fn (e)\n (emit! \"search\" (get-value e)))))" "lisp"))
(p (strong "Delivers: ") "user-defined propagation modes (animation, debounce, throttle, "
"worker, stream), effect composition by nesting, "
"the full algebraic effects model.")))

2
sx/sx/plans/self-hosting-bootstrapper.sx
View File

@@ -176,7 +176,7 @@
(p "Python closures can read but not rebind outer variables. "
(code "py.sx") " detects " (code "set!") " targets that cross lambda boundaries "
"and routes them through a " (code "_cells") " dict:")
(~docs/code :code (highlight ";; SX ;; Python
(~docs/code :src (highlight ";; SX ;; Python
(define counter def counter():
(fn () _cells = {}
(let ((n 0)) _cells['n'] = 0

12
sx/sx/plans/spec-explorer.sx
View File

@@ -97,7 +97,7 @@
(p "The explorer shows effect badges on each function card, and the stats bar aggregates them across the whole file. Pure functions (green) are the nucleus — no side effects, fully deterministic, safe to cache, reorder, or parallelise.")
(~docs/code :code (highlight "(define signal :effects []\n (fn ((initial-value :as any))\n (make-signal initial-value)))\n\n(define reset! :effects [mutation]\n (fn ((s :as signal) value)\n (when (signal? s)\n (let ((old (signal-value s)))\n (when (not (identical? old value))\n (signal-set-value! s value)\n (notify-subscribers s))))))" "sx")))
(~docs/code :src (highlight "(define signal :effects []\n (fn ((initial-value :as any))\n (make-signal initial-value)))\n\n(define reset! :effects [mutation]\n (fn ((s :as signal) value)\n (when (signal? s)\n (let ((old (signal-value s)))\n (when (not (identical? old value))\n (signal-set-value! s value)\n (notify-subscribers s))))))" "sx")))
;; -----------------------------------------------------------------------
;; Bootstrapper translations
@@ -107,15 +107,15 @@
(p "Each function is translated by the actual bootstrappers that build the production runtime. The same " (code "signal") " function shown in three target languages:")
(~docs/subsection :title "Python (via bootstrap_py.py)"
(~docs/code :code (highlight "def signal(initial_value):\n return make_signal(initial_value)" "python"))
(~docs/code :src (highlight "def signal(initial_value):\n return make_signal(initial_value)" "python"))
(p :class "text-sm text-stone-500" (code "PyEmitter._emit_define()") " — the exact same code path that generates " (code "sx_ref.py") "."))
(~docs/subsection :title "JavaScript (via js.sx)"
(~docs/code :code (highlight "var signal = function(initial_value) {\n return make_signal(initial_value);\n};" "javascript"))
(~docs/code :src (highlight "var signal = function(initial_value) {\n return make_signal(initial_value);\n};" "javascript"))
(p :class "text-sm text-stone-500" (code "js-emit-define") " — the self-hosting JS bootstrapper, written in SX, evaluated by the Python evaluator."))
(~docs/subsection :title "Z3 / SMT-LIB (via z3.sx)"
(~docs/code :code (highlight "; signal — Create a reactive signal container with an initial value.\n(declare-fun signal (Value) Value)" "lisp"))
(~docs/code :src (highlight "; signal — Create a reactive signal container with an initial value.\n(declare-fun signal (Value) Value)" "lisp"))
(p :class "text-sm text-stone-500" (code "z3-translate") " — the first self-hosted bootstrapper, translating spec declarations to verification conditions for theorem provers.")))
;; -----------------------------------------------------------------------
@@ -127,12 +127,12 @@
(~docs/subsection :title "Tests"
(p "Test files (" (code "test-signals.sx") ", " (code "test-eval.sx") ", etc.) use the " (code "defsuite") "/" (code "deftest") " framework. The explorer matches tests to functions by suite name and shows them on the function card.")
(~docs/code :code (highlight "(defsuite \"signal basics\"\n (deftest \"creates signal with value\"\n (let ((s (signal 42)))\n (assert-equal (deref s) 42)))\n (deftest \"reset changes value\"\n (let ((s (signal 0)))\n (reset! s 99)\n (assert-equal (deref s) 99))))" "sx")))
(~docs/code :src (highlight "(defsuite \"signal basics\"\n (deftest \"creates signal with value\"\n (let ((s (signal 42)))\n (assert-equal (deref s) 42)))\n (deftest \"reset changes value\"\n (let ((s (signal 0)))\n (reset! s 99)\n (assert-equal (deref s) 99))))" "sx")))
(~docs/subsection :title "Proofs"
(p (code "prove.sx") " verifies algebraic properties of SX primitives by bounded model checking. For each " (code "define-primitive") " with a " (code ":body") ", " (code "prove-translate") " translates to SMT-LIB and verifies satisfiability by construction.")
(p "Properties from the " (code "sx-properties") " library are matched to functions and shown on their cards:")
(~docs/code :code (highlight ";; prove.sx property: +-commutative\n{:name \"+-commutative\"\n :vars (list \"a\" \"b\")\n :test (fn (a b) (= (+ a b) (+ b a)))\n :holds '(= (+ a b) (+ b a))}\n\n;; Result: verified — 1,681 ground instances tested" "sx"))))
(~docs/code :src (highlight ";; prove.sx property: +-commutative\n{:name \"+-commutative\"\n :vars (list \"a\" \"b\")\n :test (fn (a b) (= (+ a b) (+ b a)))\n :holds '(= (+ a b) (+ b a))}\n\n;; Result: verified — 1,681 ground instances tested" "sx"))))
;; -----------------------------------------------------------------------
;; Architecture

32
sx/sx/plans/sx-activity.sx
View File

@@ -31,12 +31,12 @@
(~docs/subsection :title "The Problem"
(p "JSON-LD activities are verbose and require context resolution:")
(~docs/code :code (highlight "{\"@context\": \"https://www.w3.org/ns/activitystreams\",\n \"type\": \"Create\",\n \"actor\": \"https://example.com/users/alice\",\n \"object\": {\n \"type\": \"Note\",\n \"content\": \"<p>Hello world</p>\",\n \"attributedTo\": \"https://example.com/users/alice\"\n }}" "json"))
(~docs/code :src (highlight "{\"@context\": \"https://www.w3.org/ns/activitystreams\",\n \"type\": \"Create\",\n \"actor\": \"https://example.com/users/alice\",\n \"object\": {\n \"type\": \"Note\",\n \"content\": \"<p>Hello world</p>\",\n \"attributedTo\": \"https://example.com/users/alice\"\n }}" "json"))
(p "Every consumer parses JSON, resolves @context, extracts fields, then builds their own UI around the raw data. The content is HTML embedded in a JSON string — two formats nested, neither evaluable."))
(~docs/subsection :title "SX Activity Format"
(p "The same activity as SX:")
(~docs/code :code (highlight "(Create\n :actor \"https://example.com/users/alice\"\n :published \"2026-03-06T12:00:00Z\"\n :object (Note\n :attributed-to \"https://example.com/users/alice\"\n :content (p \"Hello world\")\n :media-type \"text/sx\"))" "lisp"))
(~docs/code :src (highlight "(Create\n :actor \"https://example.com/users/alice\"\n :published \"2026-03-06T12:00:00Z\"\n :object (Note\n :attributed-to \"https://example.com/users/alice\"\n :content (p \"Hello world\")\n :media-type \"text/sx\"))" "lisp"))
(p "The content isn't a string containing markup — it " (em "is") " markup. The receiving server can evaluate it directly. The Note's content is a renderable SX expression."))
(~docs/subsection :title "Approach"
@@ -44,7 +44,7 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Activity vocabulary in SX")
(p "Map ActivityStreams types to SX symbols. Activities are lists with a type head and keyword properties:")
(~docs/code :code (highlight ";; Core activity types\n(Create :actor ... :object ...)\n(Update :actor ... :object ...)\n(Delete :actor ... :object ...)\n(Follow :actor ... :object ...)\n(Like :actor ... :object ...)\n(Announce :actor ... :object ...)\n\n;; Object types\n(Note :content ... :attributed-to ...)\n(Article :name ... :content ... :summary ...)\n(Image :url ... :media-type ... :cid ...)\n(Collection :total-items ... :items ...)" "lisp")))
(~docs/code :src (highlight ";; Core activity types\n(Create :actor ... :object ...)\n(Update :actor ... :object ...)\n(Delete :actor ... :object ...)\n(Follow :actor ... :object ...)\n(Like :actor ... :object ...)\n(Announce :actor ... :object ...)\n\n;; Object types\n(Note :content ... :attributed-to ...)\n(Article :name ... :content ... :summary ...)\n(Image :url ... :media-type ... :cid ...)\n(Collection :total-items ... :items ...)" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. Content negotiation")
@@ -84,13 +84,13 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Component CID computation")
(p "Each " (code "defcomp") " definition gets a content address:")
(~docs/code :code (highlight ";; Component source\n(defcomp ~plans/sx-activity/card (&key title &rest children)\n (div :class \"border rounded p-4\"\n (h2 title) children))\n\n;; CID = SHA3-256 of canonical serialized form\n;; → bafy...abc123\n;; Stored: ipfs://bafy...abc123 → component source" "lisp"))
(~docs/code :src (highlight ";; Component source\n(defcomp ~plans/sx-activity/card (&key title &rest children)\n (div :class \"border rounded p-4\"\n (h2 title) children))\n\n;; CID = SHA3-256 of canonical serialized form\n;; → bafy...abc123\n;; Stored: ipfs://bafy...abc123 → component source" "lisp"))
(p "Canonical form: normalize whitespace, sort keyword args alphabetically, strip comments. Same component always produces same CID regardless of formatting."))
(div
(h4 :class "font-semibold text-stone-700" "2. Component references in activities")
(p "Activities declare which components they need by CID:")
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :requires (list\n \"bafy...card\" ;; ~plans/sx-activity/card component\n \"bafy...avatar\") ;; ~shared:misc/avatar component\n :object (Note\n :content (~plans/sx-activity/card :title \"Hello\"\n (~shared:misc/avatar :src \"ipfs://bafy...photo\")\n (p \"This renders with the card component.\"))))" "lisp"))
(~docs/code :src (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :requires (list\n \"bafy...card\" ;; ~plans/sx-activity/card component\n \"bafy...avatar\") ;; ~shared:misc/avatar component\n :object (Note\n :content (~plans/sx-activity/card :title \"Hello\"\n (~shared:misc/avatar :src \"ipfs://bafy...photo\")\n (p \"This renders with the card component.\"))))" "lisp"))
(p "The receiving browser fetches missing components from IPFS, verifies CIDs, registers them, then renders the content."))
(div
@@ -106,7 +106,7 @@
(div
(h4 :class "font-semibold text-stone-700" "4. Component publication")
(p "Server-side: on component registration, compute CID and pin to IPFS. Track in " (code "IPFSPin") " model (already exists). Publish component availability via AP outbox:")
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/apps/market\"\n :object (SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :version \"1.0.0\"\n :deps (list \"bafy...card\" \"bafy...price-tag\")))" "lisp")))))
(~docs/code :src (highlight "(Create\n :actor \"https://rose-ash.com/apps/market\"\n :object (SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :version \"1.0.0\"\n :deps (list \"bafy...card\" \"bafy...price-tag\")))" "lisp")))))
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -133,18 +133,18 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Media CID pipeline")
(p "On upload: hash content → pin to IPFS → store CID in database. Activities reference media by CID alongside URL fallback:")
(~docs/code :code (highlight "(Image\n :cid \"bafy...photo123\"\n :url \"https://rose-ash.com/media/photo.jpg\" ;; fallback\n :media-type \"image/jpeg\"\n :width 1200 :height 800)" "lisp")))
(~docs/code :src (highlight "(Image\n :cid \"bafy...photo123\"\n :url \"https://rose-ash.com/media/photo.jpg\" ;; fallback\n :media-type \"image/jpeg\"\n :width 1200 :height 800)" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. DAG output federation")
(p "artdag processing results (rendered video, processed images) already have CIDs. Federate them as activities:")
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Artwork\n :name \"Sunset Remix\"\n :cid \"bafy...artwork\"\n :dag-cid \"bafy...dag\" ;; full DAG for reproduction\n :media-type \"video/mp4\"\n :sources (list\n (Image :cid \"bafy...src1\" :attribution \"...\")\n (Image :cid \"bafy...src2\" :attribution \"...\"))))" "lisp"))
(~docs/code :src (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Artwork\n :name \"Sunset Remix\"\n :cid \"bafy...artwork\"\n :dag-cid \"bafy...dag\" ;; full DAG for reproduction\n :media-type \"video/mp4\"\n :sources (list\n (Image :cid \"bafy...src1\" :attribution \"...\")\n (Image :cid \"bafy...src2\" :attribution \"...\"))))" "lisp"))
(p "The " (code ":dag-cid") " lets anyone re-execute the processing pipeline. The artwork is both a result and a reproducible recipe."))
(div
(h4 :class "font-semibold text-stone-700" "3. Shared SX content store")
(p "Not just components and media — full page content can be content-addressed. An Article's body is SX, pinned to IPFS:")
(~docs/code :code (highlight "(Article\n :name \"Why S-Expressions\"\n :content-cid \"bafy...article-body\" ;; SX source on IPFS\n :requires (list \"bafy...doc-page\" \"bafy...code-block\")\n :summary \"Why SX uses s-expressions instead of HTML.\")" "lisp"))
(~docs/code :src (highlight "(Article\n :name \"Why S-Expressions\"\n :content-cid \"bafy...article-body\" ;; SX source on IPFS\n :requires (list \"bafy...doc-page\" \"bafy...code-block\")\n :summary \"Why SX uses s-expressions instead of HTML.\")" "lisp"))
(p "The content outlives the server. Anyone with the CID can fetch, parse, and render the article with its original components."))
(div
@@ -177,12 +177,12 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Component collections as AP actors")
(p "Each server exposes a component registry actor:")
(~docs/code :code (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(~docs/code :src (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(p "Follow the registry to receive component updates. The outbox is a chronological feed of Create/Update/Delete activities for components."))
(div
(h4 :class "font-semibold text-stone-700" "2. Component metadata")
(~docs/code :code (highlight "(SxComponent\n :name \"~data-table\"\n :cid \"bafy...datatable\"\n :version \"2.1.0\"\n :deps (list \"bafy...sortable\" \"bafy...paginator\")\n :params (list\n (dict :name \"rows\" :type \"list\" :required true)\n (dict :name \"columns\" :type \"list\" :required true)\n (dict :name \"sortable\" :type \"boolean\" :default false))\n :css-atoms (list :border :rounded :p-4 :text-sm)\n :preview-cid \"bafy...screenshot\"\n :license \"MIT\")" "lisp"))
(~docs/code :src (highlight "(SxComponent\n :name \"~data-table\"\n :cid \"bafy...datatable\"\n :version \"2.1.0\"\n :deps (list \"bafy...sortable\" \"bafy...paginator\")\n :params (list\n (dict :name \"rows\" :type \"list\" :required true)\n (dict :name \"columns\" :type \"list\" :required true)\n (dict :name \"sortable\" :type \"boolean\" :default false))\n :css-atoms (list :border :rounded :p-4 :text-sm)\n :preview-cid \"bafy...screenshot\"\n :license \"MIT\")" "lisp"))
(p "Dependencies are transitive CID references. CSS atoms declare which CSSX rules the component needs. Preview CID is a screenshot for registry browsing."))
(div
@@ -197,7 +197,7 @@
(div
(h4 :class "font-semibold text-stone-700" "4. Version resolution")
(p "Components are immutable (CID = identity). \"Updating\" a component publishes a new CID. Activities reference specific CIDs, so old content always renders correctly. The registry tracks version history:")
(~docs/code :code (highlight "(Update\n :actor \"https://rose-ash.com/sx-registry\"\n :object (SxComponent\n :name \"~card\"\n :cid \"bafy...card-v2\" ;; new version\n :replaces \"bafy...card-v1\" ;; previous version\n :changelog \"Added subtitle slot\"))" "lisp")))))
(~docs/code :src (highlight "(Update\n :actor \"https://rose-ash.com/sx-registry\"\n :object (SxComponent\n :name \"~card\"\n :cid \"bafy...card-v2\" ;; new version\n :replaces \"bafy...card-v1\" ;; previous version\n :changelog \"Added subtitle slot\"))" "lisp")))))
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -232,7 +232,7 @@
(div
(h4 :class "font-semibold text-stone-700" "2. Provenance chain in activities")
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Note :content (p \"Hello\") :cid \"bafy...note\")\n :provenance (Anchor\n :tree-cid \"bafy...merkle-tree\"\n :leaf-index 42\n :ots-cid \"bafy...ots-proof\"\n :btc-txid \"abc123...def\"\n :btc-block 890123\n :anchored-at \"2026-03-06T12:00:00Z\"))" "lisp"))
(~docs/code :src (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Note :content (p \"Hello\") :cid \"bafy...note\")\n :provenance (Anchor\n :tree-cid \"bafy...merkle-tree\"\n :leaf-index 42\n :ots-cid \"bafy...ots-proof\"\n :btc-txid \"abc123...def\"\n :btc-block 890123\n :anchored-at \"2026-03-06T12:00:00Z\"))" "lisp"))
(p "Any party can verify: fetch the OTS proof from IPFS, check the Merkle path from the activity's CID to the tree root, confirm the tree root is committed in the Bitcoin block."))
(div
@@ -246,7 +246,7 @@
(div
(h4 :class "font-semibold text-stone-700" "4. Verification UI")
(p "Client-side provenance badge on federated content:")
(~docs/code :code (highlight "(defcomp ~plans/sx-activity/provenance-badge (&key anchor)\n (when anchor\n (details :class \"inline text-xs text-stone-400\"\n (summary \"✓ Anchored\")\n (dl :class \"mt-1 space-y-1\"\n (dt \"Bitcoin block\") (dd (get anchor \"btc-block\"))\n (dt \"Timestamp\") (dd (get anchor \"anchored-at\"))\n (dt \"Proof\") (dd (a :href (str \"ipfs://\" (get anchor \"ots-cid\"))\n \"OTS proof\"))))))" "lisp")))))
(~docs/code :src (highlight "(defcomp ~plans/sx-activity/provenance-badge (&key anchor)\n (when anchor\n (details :class \"inline text-xs text-stone-400\"\n (summary \"✓ Anchored\")\n (dl :class \"mt-1 space-y-1\"\n (dt \"Bitcoin block\") (dd (get anchor \"btc-block\"))\n (dt \"Timestamp\") (dd (get anchor \"anchored-at\"))\n (dt \"Proof\") (dd (a :href (str \"ipfs://\" (get anchor \"ots-cid\"))\n \"OTS proof\"))))))" "lisp")))))
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -282,7 +282,7 @@
(div
(h4 :class "font-semibold text-stone-700" "Parsers and transforms")
(p "A markdown parser is just an SX function: takes a string, returns an SX tree. Publish it to IPFS. Now anyone can use your markdown dialect. Same for: syntax highlighters, BBCode parsers, wiki markup, LaTeX subsets, CSV-to-table converters, JSON-to-SX adapters. " (strong "The parser ecosystem becomes shared infrastructure."))
(~docs/code :code (highlight ";; A markdown parser, published to IPFS\n;; CID: bafy...md-parser\n(define parse-markdown\n (fn (source)\n ;; tokenize → build AST → return SX tree\n ;; (parse-markdown \"# Hello\\n**bold**\")\n ;; → (h1 \"Hello\") (p (strong \"bold\"))\n ...))\n\n;; Anyone can use it in their components\n(defcomp ~plans/sx-activity/blog-post (&key markdown-source)\n (div :class \"prose\"\n (parse-markdown markdown-source)))" "lisp")))
(~docs/code :src (highlight ";; A markdown parser, published to IPFS\n;; CID: bafy...md-parser\n(define parse-markdown\n (fn (source)\n ;; tokenize → build AST → return SX tree\n ;; (parse-markdown \"# Hello\\n**bold**\")\n ;; → (h1 \"Hello\") (p (strong \"bold\"))\n ...))\n\n;; Anyone can use it in their components\n(defcomp ~plans/sx-activity/blog-post (&key markdown-source)\n (div :class \"prose\"\n (parse-markdown markdown-source)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "Server-side and client-side logic")
@@ -350,7 +350,7 @@
(~docs/subsection :title "Serverless applications on IPFS"
(p "The logical conclusion: " (strong "entire web applications hosted on IPFS with no server at all."))
(p "An SX application is a tree of content-addressed artifacts: a root page definition, component dependencies, media, stylesheets, parsers, transforms. Pin the root CID to IPFS and the application is live. No server, no DNS, no hosting provider, no deployment pipeline. Someone gives you a CID, you paste it into an SX-aware browser, and the application runs.")
(~docs/code :code (highlight ";; An entire blog — one CID\n;; ipfs://bafy...my-blog\n(defpage blog-home\n :path \"/\"\n :requires (list\n \"bafy...article-layout\" ;; layout component\n \"bafy...md-parser\" ;; markdown parser\n \"bafy...syntax-highlight\" ;; code highlighting\n \"bafy...nav-component\") ;; navigation\n :content\n (~article-layout\n :title \"My Blog\"\n :nav (~nav-component\n :items (list\n (dict :label \"Post 1\" :cid \"bafy...post-1\")\n (dict :label \"Post 2\" :cid \"bafy...post-2\")))\n :body (~markdown-page\n :source-cid \"bafy...homepage-md\")))" "lisp"))
(~docs/code :src (highlight ";; An entire blog — one CID\n;; ipfs://bafy...my-blog\n(defpage blog-home\n :path \"/\"\n :requires (list\n \"bafy...article-layout\" ;; layout component\n \"bafy...md-parser\" ;; markdown parser\n \"bafy...syntax-highlight\" ;; code highlighting\n \"bafy...nav-component\") ;; navigation\n :content\n (~article-layout\n :title \"My Blog\"\n :nav (~nav-component\n :items (list\n (dict :label \"Post 1\" :cid \"bafy...post-1\")\n (dict :label \"Post 2\" :cid \"bafy...post-2\")))\n :body (~markdown-page\n :source-cid \"bafy...homepage-md\")))" "lisp"))
(p "What this looks like in practice:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Personal sites: ") "A portfolio or blog is a handful of SX files + media. Pin to IPFS. Share the CID. No hosting costs, no domain renewal, no SSL certificates. The site is permanent.")

4
sx/sx/plans/sx-ci.sx
View File

@@ -17,7 +17,7 @@
(~docs/section :title "Design" :id "design"
(p :class "text-stone-600"
"Pipeline definitions are " (code ".sx") " files. A minimal Python CLI runner evaluates them using " (code "sx_ref.py") ". CI-specific IO primitives (shell execution, Docker, git) are boundary-declared and only available to the pipeline runner — never to web components.")
(~docs/code :code (highlight ";; pipeline/deploy.sx\n(let ((targets (if (= (length ARGS) 0)\n (~plans/sx-ci/detect-changed :base \"HEAD~1\")\n (filter (fn (svc) (some (fn (a) (= a (get svc \"name\"))) ARGS))\n services))))\n (when (= (length targets) 0)\n (log-step \"No changes detected\")\n (exit 0))\n\n (log-step (str \"Deploying: \" (join \" \" (map (fn (s) (get s \"name\")) targets))))\n\n ;; Tests first\n (~unit-tests)\n (~sx-spec-tests)\n\n ;; Build, push, restart\n (for-each (fn (svc) (~plans/sx-ci/build-service :service svc)) targets)\n (for-each (fn (svc) (~restart-service :service svc)) targets)\n\n (log-step \"Deploy complete\"))" "lisp"))
(~docs/code :src (highlight ";; pipeline/deploy.sx\n(let ((targets (if (= (length ARGS) 0)\n (~plans/sx-ci/detect-changed :base \"HEAD~1\")\n (filter (fn (svc) (some (fn (a) (= a (get svc \"name\"))) ARGS))\n services))))\n (when (= (length targets) 0)\n (log-step \"No changes detected\")\n (exit 0))\n\n (log-step (str \"Deploying: \" (join \" \" (map (fn (s) (get s \"name\")) targets))))\n\n ;; Tests first\n (~unit-tests)\n (~sx-spec-tests)\n\n ;; Build, push, restart\n (for-each (fn (svc) (~plans/sx-ci/build-service :service svc)) targets)\n (for-each (fn (svc) (~restart-service :service svc)) targets)\n\n (log-step \"Deploy complete\"))" "lisp"))
(p :class "text-stone-600"
"Pipeline steps are components. " (code "~unit-tests") ", " (code "~plans/sx-ci/build-service") ", " (code "~plans/sx-ci/detect-changed") " are " (code "defcomp") " definitions that compose by nesting — the same mechanism used for page layouts, navigation, and every other piece of the system."))
@@ -73,7 +73,7 @@
(~docs/section :title "Reusable Steps" :id "steps"
(p :class "text-stone-600"
"Pipeline steps are components — same " (code "defcomp") " as UI components, same " (code "&key") " params, same composition by nesting:")
(~docs/code :code (highlight "(defcomp ~plans/sx-ci/detect-changed (&key base)\n (let ((files (git-diff-files (or base \"HEAD~1\") \"HEAD\")))\n (if (some (fn (f) (starts-with? f \"shared/\")) files)\n services\n (filter (fn (svc)\n (some (fn (f) (starts-with? f (str (get svc \"dir\") \"/\"))) files))\n services))))\n\n(defcomp ~plans/sx-ci/build-service (&key service)\n (let ((name (get service \"name\"))\n (tag (str registry \"/\" name \":latest\")))\n (log-step (str \"Building \" name))\n (docker-build :file (str (get service \"dir\") \"/Dockerfile\") :tag tag :context \".\")\n (docker-push tag)))\n\n(defcomp ~plans/sx-ci/bootstrap-check ()\n (log-step \"Checking bootstrapped files are up to date\")\n (shell-run! \"python shared/sx/ref/bootstrap_js.py\")\n (shell-run! \"python shared/sx/ref/bootstrap_py.py\")\n (let ((diff (shell-run \"git diff --name-only shared/static/scripts/sx-ref.js shared/sx/ref/sx_ref.py\")))\n (when (not (= (get diff \"stdout\") \"\"))\n (fail! \"Bootstrapped files are stale — rebootstrap and commit\"))))" "lisp"))
(~docs/code :src (highlight "(defcomp ~plans/sx-ci/detect-changed (&key base)\n (let ((files (git-diff-files (or base \"HEAD~1\") \"HEAD\")))\n (if (some (fn (f) (starts-with? f \"shared/\")) files)\n services\n (filter (fn (svc)\n (some (fn (f) (starts-with? f (str (get svc \"dir\") \"/\"))) files))\n services))))\n\n(defcomp ~plans/sx-ci/build-service (&key service)\n (let ((name (get service \"name\"))\n (tag (str registry \"/\" name \":latest\")))\n (log-step (str \"Building \" name))\n (docker-build :file (str (get service \"dir\") \"/Dockerfile\") :tag tag :context \".\")\n (docker-push tag)))\n\n(defcomp ~plans/sx-ci/bootstrap-check ()\n (log-step \"Checking bootstrapped files are up to date\")\n (shell-run! \"python shared/sx/ref/bootstrap_js.py\")\n (shell-run! \"python shared/sx/ref/bootstrap_py.py\")\n (let ((diff (shell-run \"git diff --name-only shared/static/scripts/sx-ref.js shared/sx/ref/sx_ref.py\")))\n (when (not (= (get diff \"stdout\") \"\"))\n (fail! \"Bootstrapped files are stale — rebootstrap and commit\"))))" "lisp"))
(p :class "text-stone-600"
"Compare this to GitHub Actions YAML, where \"reuse\" means composite actions with " (code "uses:") " references, input/output mappings, shell script blocks inside YAML strings, and a " (a :href "https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions" :class "text-violet-600 hover:underline" "100-page syntax reference") ". SX pipeline reuse is function composition. That is all it has ever been."))

26
sx/sx/plans/sx-protocol.sx
View File

@@ -51,9 +51,9 @@
(h3 :class "text-lg font-semibold mt-6 mb-2" "URLs as S-Expressions")
(p "A conventional URL:")
(~docs/code :code (highlight "https://site.com/blog/my-post?filter=published&sort=date" "text"))
(~docs/code :src (highlight "https://site.com/blog/my-post?filter=published&sort=date" "text"))
(p "As an SX expression:")
(~docs/code :code (highlight "(get.site.com.(blog.(my-post.(filter.published.sort.date))))" "lisp"))
(~docs/code :src (highlight "(get.site.com.(blog.(my-post.(filter.published.sort.date))))" "lisp"))
(ul
(li "The protocol/verb is the first atom: " (code "get"))
(li "The domain follows: " (code "site.com"))
@@ -64,11 +64,11 @@
(p "Lisp conventionally uses spaces as separators. In URLs, spaces become " (code "%20")
". SX uses dots instead, which are URL-safe and semantically meaningful — a dot between "
"two atoms is a " (strong "cons pair") ", the fundamental unit of Lisp structure.")
(~docs/code :code (highlight ";; Clean, URL-safe, valid Lisp\n(blog.(filter.published).(sort.date.desc))" "lisp"))
(~docs/code :src (highlight ";; Clean, URL-safe, valid Lisp\n(blog.(filter.published).(sort.date.desc))" "lisp"))
(h3 :class "text-lg font-semibold mt-6 mb-2" "Verbs Are Just Atoms")
(p "HTTP methods are not special syntax — they are simply the first element of the expression:")
(~docs/code :code (highlight "(get.site.com.(post.my-first-post)) ; read\n(post.site.com.(submit-post.(title.hello))) ; write\n(ws.site.com.(live-feed)) ; websocket / subscribe" "lisp"))
(~docs/code :src (highlight "(get.site.com.(post.my-first-post)) ; read\n(post.site.com.(submit-post.(title.hello))) ; write\n(ws.site.com.(live-feed)) ; websocket / subscribe" "lisp"))
(p "No special protocol prefixes. No " (code "https://") " vs " (code "wss://")
". The verb is data, like everything else."))
@@ -81,7 +81,7 @@
(h3 :class "text-lg font-semibold mt-6 mb-2" "Queries Are URLs")
(p "Because SX expressions are URLs, every query is a GET request:")
(~docs/code :code (highlight ";; This is a URL and a query simultaneously\n(get.site.com.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(~docs/code :src (highlight ";; This is a URL and a query simultaneously\n(get.site.com.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(ul
(li "Fully cacheable by CDNs")
(li "Bookmarkable and shareable")
@@ -91,19 +91,19 @@
(h3 :class "text-lg font-semibold mt-6 mb-2" "Responses Include Rendering")
(p "GraphQL returns data. Graph-SX returns " (strong "hypermedia")
" — data and its presentation in the same expression:")
(~docs/code :code (highlight ";; GraphQL response (dead data)\n{\"title\": \"My Post\", \"body\": \"Hello world\"}\n\n;; Graph-SX response (live hypermedia)\n(article\n (h1 \"My Post\")\n (p \"Hello world\")\n (a (href (get.site.com.(post.next-post))) \"Next\"))" "lisp"))
(~docs/code :src (highlight ";; GraphQL response (dead data)\n{\"title\": \"My Post\", \"body\": \"Hello world\"}\n\n;; Graph-SX response (live hypermedia)\n(article\n (h1 \"My Post\")\n (p \"Hello world\")\n (a (href (get.site.com.(post.next-post))) \"Next\"))" "lisp"))
(p "The server returns what the resource " (strong "is") " and how to "
(strong "present") " it in one unified structure. There is no separate rendering layer.")
(h3 :class "text-lg font-semibold mt-6 mb-2" "Queries Are Transformations")
(p "Because SX is a full programming language, the query and the transformation "
"are the same expression:")
(~docs/code :code (highlight ";; Fetch, filter, and transform in one expression\n(map (lambda (p) (title p))\n (filter published?\n (posts (after \"2025\"))))" "lisp"))
(~docs/code :src (highlight ";; Fetch, filter, and transform in one expression\n(map (lambda (p) (title p))\n (filter published?\n (posts (after \"2025\"))))" "lisp"))
(p "No separate processing step. No client-side data manipulation layer."))
(~docs/section :title "Components" :id "components"
(p "SX supports server-side composable components via the " (code "~") " prefix convention:")
(~docs/code :code (highlight "(~get.everything-under-the-sun)" "lisp"))
(~docs/code :src (highlight "(~get.everything-under-the-sun)" "lisp"))
(p "A " (code "~component") " is a named server-side function that:")
(ol
(li "Receives the expression as arguments")
@@ -111,19 +111,19 @@
(li "Processes and composes results")
(li "Returns hypermedia"))
(p "Components compose naturally:")
(~docs/code :code (highlight "(~page.home\n (~hero.banner)\n (~get.latest-posts.(limit.5))\n (~get.featured.(filter.pinned)))" "lisp"))
(~docs/code :src (highlight "(~page.home\n (~hero.banner)\n (~get.latest-posts.(limit.5))\n (~get.featured.(filter.pinned)))" "lisp"))
(p "This is equivalent to React Server Components — but without a framework, "
"without a build step, and without leaving Lisp."))
(~docs/section :title "Cross-Domain Composition" :id "cross-domain"
(p "Because domain and verb are just atoms, cross-domain queries are structurally "
"identical to local ones:")
(~docs/code :code (highlight ";; Local\n(post.my-first-post)\n\n;; Remote — identical structure, qualified\n(get.site.com.(post.my-first-post))\n\n;; Composed across domains\n(~render\n (get.site.com.(post.my-first-post))\n (get.cdn.com.(image.hero)))" "lisp"))
(~docs/code :src (highlight ";; Local\n(post.my-first-post)\n\n;; Remote — identical structure, qualified\n(get.site.com.(post.my-first-post))\n\n;; Composed across domains\n(~render\n (get.site.com.(post.my-first-post))\n (get.cdn.com.(image.hero)))" "lisp"))
(p "Network calls are function calls. Remote resources are just namespaced expressions."))
(~docs/section :title "Self-Describing and Introspectable" :id "introspectable"
(p "Because the site is implemented in SX and served as SX, every page is introspectable:")
(~docs/code :code (highlight "(get.sx.dev.(about)) ; the about page\n(get.sx.dev.(source.(about))) ; the SX source for the about page\n(get.sx.dev.(eval.(source.about))) ; re-evaluate it live" "lisp"))
(~docs/code :src (highlight "(get.sx.dev.(about)) ; the about page\n(get.sx.dev.(source.(about))) ; the SX source for the about page\n(get.sx.dev.(eval.(source.about))) ; re-evaluate it live" "lisp"))
(p "The site is its own documentation. The source is always one expression away."))
(~docs/section :title "Comparison" :id "comparison"
@@ -185,13 +185,13 @@
(p "The logical conclusion of SX is a " (strong "new internet protocol")
" in which the URL, the HTTP verb, the query language, the response format, "
"and the rendering layer are all unified under one evaluable expression format.")
(~docs/code :code (highlight ";; The entire network request — protocol, domain, verb, query, all one expression\n(get.sx.dev.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(~docs/code :src (highlight ";; The entire network request — protocol, domain, verb, query, all one expression\n(get.sx.dev.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(p "HTTP becomes one possible implementation of a more general principle:")
(blockquote :class "border-l-4 border-violet-300 pl-4 italic text-stone-600 my-4"
(p (strong "Evaluate this expression. Return an expression."))))
(~docs/section :title "Reference Implementation" :id "reference"
(p "SX is implemented in SX. The reference implementation is self-hosting and available at:")
(~docs/code :code (highlight "(get.sx.dev.(source.evaluator))" "lisp"))
(~docs/code :src (highlight "(get.sx.dev.(source.evaluator))" "lisp"))
(p :class "text-sm text-stone-500 mt-4 italic"
"This proposal was written in conversation with Claude (Anthropic). The ideas are the author's own."))))

18
sx/sx/plans/sx-pub.sx
View File

@@ -21,10 +21,10 @@
(~docs/subsection :title "1. The Rendering Gap"
(p "An ActivityPub Note looks like this:")
(~docs/code :code "{\"type\": \"Note\",\n \"content\": \"<p>Hello <strong>world</strong></p>\",\n \"attributedTo\": \"https://example.com/users/alice\"}")
(~docs/code :src "{\"type\": \"Note\",\n \"content\": \"<p>Hello <strong>world</strong></p>\",\n \"attributedTo\": \"https://example.com/users/alice\"}")
(p "The content is an HTML string embedded in a JSON string. Two formats nested, neither evaluable. Every client — Mastodon, Pleroma, Misskey, Lemmy — parses this independently and wraps it in their own UI. A poll renders differently everywhere. An event looks different everywhere. There is no shared component model because the protocol has no concept of components.")
(p "In sx-pub, the content " (em "is") " the component:")
(~docs/code :code "(Note\n :attributed-to \"https://pub.sx-web.org/pub/actor\"\n :content (p \"Hello \" (strong \"world\")))")
(~docs/code :src "(Note\n :attributed-to \"https://pub.sx-web.org/pub/actor\"\n :content (p \"Hello \" (strong \"world\")))")
(p "No HTML-in-JSON. No parsing step. The receiver evaluates the s-expression and gets rendered output. The content carries its own rendering semantics."))
(~docs/subsection :title "2. Inert Data vs. Evaluable Programs"
@@ -63,23 +63,23 @@
(p "Everything is s-expressions. No JSON-LD, no @context resolution, no nested HTML-in-JSON strings.")
(~docs/subsection :title "Actor"
(~docs/code :code "(SxActor\n :id \"https://pub.sx-web.org/pub/actor\"\n :type \"SxPublisher\"\n :name \"sx\"\n :summary \"SX language — specs, platforms, components\"\n :public-key-pem \"-----BEGIN PUBLIC KEY-----\\n...\"\n :inbox \"/pub/inbox\"\n :outbox \"/pub/outbox\"\n :followers \"/pub/followers\"\n :following \"/pub/following\"\n :collections (list\n (SxCollection :name \"core-specs\" :href \"/pub/core-specs\")\n (SxCollection :name \"platforms\" :href \"/pub/platforms\")\n (SxCollection :name \"components\" :href \"/pub/components\")))"))
(~docs/code :src "(SxActor\n :id \"https://pub.sx-web.org/pub/actor\"\n :type \"SxPublisher\"\n :name \"sx\"\n :summary \"SX language — specs, platforms, components\"\n :public-key-pem \"-----BEGIN PUBLIC KEY-----\\n...\"\n :inbox \"/pub/inbox\"\n :outbox \"/pub/outbox\"\n :followers \"/pub/followers\"\n :following \"/pub/following\"\n :collections (list\n (SxCollection :name \"core-specs\" :href \"/pub/core-specs\")\n (SxCollection :name \"platforms\" :href \"/pub/platforms\")\n (SxCollection :name \"components\" :href \"/pub/components\")))"))
(~docs/subsection :title "Publish Activity"
(p "When content is published, it's pinned to IPFS and announced to followers:")
(~docs/code :code "(Publish\n :actor \"https://pub.sx-web.org/pub/actor\"\n :published \"2026-03-24T12:00:00Z\"\n :object (SxDocument\n :name \"evaluator\"\n :path \"/pub/core-specs/evaluator\"\n :cid \"bafy...eval123\"\n :content-type \"text/sx\"\n :size 48000\n :hash \"sha3-256:abc123...\"\n :requires (list \"bafy...parser\" \"bafy...primitives\")\n :summary \"CEK machine evaluator — frames, utils, step function\"))")
(~docs/code :src "(Publish\n :actor \"https://pub.sx-web.org/pub/actor\"\n :published \"2026-03-24T12:00:00Z\"\n :object (SxDocument\n :name \"evaluator\"\n :path \"/pub/core-specs/evaluator\"\n :cid \"bafy...eval123\"\n :content-type \"text/sx\"\n :size 48000\n :hash \"sha3-256:abc123...\"\n :requires (list \"bafy...parser\" \"bafy...primitives\")\n :summary \"CEK machine evaluator — frames, utils, step function\"))")
(p "The " (code ":requires") " field declares CID dependencies — a component that imports the parser declares that dependency. Receivers can pin the full dependency graph."))
(~docs/subsection :title "Follow — Subscribe to a Remote Server"
(~docs/code :code "(Follow\n :actor \"https://pub.sx-web.org/pub/actor\"\n :object \"https://other.example.com/pub/actor\")")
(~docs/code :src "(Follow\n :actor \"https://pub.sx-web.org/pub/actor\"\n :object \"https://other.example.com/pub/actor\")")
(p "When accepted, the remote server delivers " (code "(Publish ...)") " activities to our inbox. We pin their CIDs locally."))
(~docs/subsection :title "Announce — Mirror Remote Content"
(~docs/code :code "(Announce\n :actor \"https://pub.sx-web.org/pub/actor\"\n :object \"bafy...remote-cid\")")
(~docs/code :src "(Announce\n :actor \"https://pub.sx-web.org/pub/actor\"\n :object \"bafy...remote-cid\")")
(p "Re-publish content from a followed server to our own followers. The CID is already pinned — we're just signalling that we endorse it."))
(~docs/subsection :title "Anchor — Blockchain Provenance Record"
(~docs/code :code "(Anchor\n :tree-cid \"bafy...merkle-tree\"\n :leaf-index 42\n :ots-cid \"bafy...ots-proof\"\n :btc-txid \"abc123...def\"\n :btc-block 890123\n :anchored-at \"2026-03-24T12:00:00Z\")")
(~docs/code :src "(Anchor\n :tree-cid \"bafy...merkle-tree\"\n :leaf-index 42\n :ots-cid \"bafy...ots-proof\"\n :btc-txid \"abc123...def\"\n :btc-block 890123\n :anchored-at \"2026-03-24T12:00:00Z\")")
(p "Embedded in every " (code "(Publish ...)") " activity. Receivers can verify: the Merkle tree was anchored in that Bitcoin block, and the CID is a leaf in that tree. Inclusion proof is verifiable offline.")))
;; -----------------------------------------------------------------------
@@ -195,7 +195,7 @@
(li "Construct " (code "(Publish ...)") " with anchor proof embedded")
(li "Deliver to follower inboxes — " (strong "only now") " is it public"))
(p :class "text-sm text-stone-500 mt-2 italic" "This means publishing isn't instant — there's a delay while the anchor confirms. That's a feature: it forces a deliberate pace and makes provenance airtight.")
(~docs/code :code "(Publish\n :actor \"https://pub.sx-web.org/pub/actor\"\n :published \"2026-03-24T12:00:00Z\"\n :object (SxDocument\n :name \"evaluator\"\n :cid \"bafy...eval123\"\n :anchor (Anchor\n :tree-cid \"bafy...merkle\"\n :ots-cid \"bafy...proof\"\n :btc-txid \"abc123...\"\n :btc-block 890123)))")
(~docs/code :src "(Publish\n :actor \"https://pub.sx-web.org/pub/actor\"\n :published \"2026-03-24T12:00:00Z\"\n :object (SxDocument\n :name \"evaluator\"\n :cid \"bafy...eval123\"\n :anchor (Anchor\n :tree-cid \"bafy...merkle\"\n :ots-cid \"bafy...proof\"\n :btc-txid \"abc123...\"\n :btc-block 890123)))")
(p :class "text-sm text-stone-600" "Receivers verify: the CID was anchored in that Bitcoin block " (em "before") " the Publish was sent. No way to backdate."))
(~docs/subsection :title "Follow"
@@ -233,7 +233,7 @@
(~docs/section :title "CID ↔ Path Translation" :id "cid-path"
(p "Published SX refers to dependencies by CID — the immutable, universal identifier. But humans browse by path.")
(~docs/code :code ";; Canonical SX (stored in IPFS) references by CID\n(defcomp ~my-app/dashboard (&key data)\n ;; This component depends on a chart library published at another CID\n (let ((chart-lib (require \"bafy...chart-lib-cid\")))\n (div :class \"dashboard\"\n (chart-lib/bar-chart :data data))))\n\n;; But on pub.sx-web.org you browse by path:\n;; /pub/components/dashboard\n;; /pub/core-specs/evaluator\n;; /pub/platforms/javascript\n;;\n;; The DB maps: path → CID → IPFS content\n;; Following a remote server maps: their-path → CID → our local pin")
(~docs/code :src ";; Canonical SX (stored in IPFS) references by CID\n(defcomp ~my-app/dashboard (&key data)\n ;; This component depends on a chart library published at another CID\n (let ((chart-lib (require \"bafy...chart-lib-cid\")))\n (div :class \"dashboard\"\n (chart-lib/bar-chart :data data))))\n\n;; But on pub.sx-web.org you browse by path:\n;; /pub/components/dashboard\n;; /pub/core-specs/evaluator\n;; /pub/platforms/javascript\n;;\n;; The DB maps: path → CID → IPFS content\n;; Following a remote server maps: their-path → CID → our local pin")
(p "When you follow another sx-pub server and they publish a component, you get the " (code "(Publish ...)") " activity with the CID. Your server pins it. Now " (code "(require \"bafy...\")") " resolves locally — no network round-trip, no central registry, no package manager."))

24
sx/sx/plans/sx-urls.sx
View File

@@ -13,14 +13,14 @@
(code "/sx/(language.(doc.introduction))") ". Dots replace spaces as the URL-friendly "
"separator — they are unreserved in RFC 3986, never percent-encoded, and visually clean. "
"The parser treats dot as whitespace: " (code "s/./ /") " before parsing as SX.")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; Current → SX URLs (dots = spaces)\n/language/specs/signals → /(language.(spec.signals))\n/language/specs/explore/signals → /(language.(spec.(explore.signals)))\n/language/docs/introduction → /(language.(doc.introduction))\n/etc/plans/spec-explorer → /(etc.(plan.spec-explorer))\n\n;; Direct component access — any defcomp is addressable\n/(~essays/sx-sucks/essay-sx-sucks)\n/(~plans/sx-urls/plan-sx-urls-content)\n/(~analyzer/bundle-analyzer-content)"
"lisp")))
(~docs/section :title "Scoping — The 30-Year Ambiguity, Fixed" :id "scoping"
(p "REST URLs have an inherent ambiguity: does a filter/parameter apply to "
"the last segment, or the whole path? Consider:")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; REST — ambiguous:\n/users/123/posts?filter=published\n;; Is the filter scoped to posts? Or to the user? Or the whole query?\n;; Nobody knows. Conventions vary. Documentation required.\n\n;; SX URLs — explicit scoping via nesting:\n/(hello.(sailor.(filter.hhh))) ;; filter scoped to sailor\n/(hello.sailor.(filter.hhh)) ;; filter scoped to hello\n\n;; These mean different things, both expressible.\n;; Parens make scope visible. No ambiguity. No documentation needed."
"lisp"))
(p "This is not a minor syntactic preference. REST has never been able to express "
@@ -37,7 +37,7 @@
"Dots are unreserved in RFC 3986, never encoded, and read naturally as \"drill down.\"")
(p "The rule is simple: " (strong "dot = space, nothing more") ". "
"Parens carry all the structural meaning. Dots are syntactic sugar for URLs only:")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; These are identical after dot→space transform:\n/(language.(doc.introduction)) → (language (doc introduction))\n/(geography.(hypermedia.(reference.attributes)))\n → (geography (hypermedia (reference attributes)))\n\n;; Parens are still required for nesting:\n/(language.doc.introduction) → (language doc introduction)\n;; = language(\"doc\", \"introduction\") — WRONG\n\n;; Correct nesting:\n/(language.(doc.introduction)) → (language (doc introduction))\n;; = language(doc(\"introduction\")) — RIGHT"
"lisp"))
(p "The server's URL handler does one thing before parsing: "
@@ -45,7 +45,7 @@
(~docs/section :title "The Lisp Tax" :id "parens"
(p "People will hate the parentheses. But consider what developers already accept:")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; Developers happily write this every day:\nhttps://api.site.com/v2/users/123/posts?filter=published&sort=date&order=desc&limit=10&offset=20\n\n;; And they would complain about this?\nhttps://site.com/(users.(posts.123.(filter.published.sort.date.limit.10)))\n\n;; The second is shorter, structured, unambiguous, and composable."
"lisp"))
(p "The real question: who is reading these URLs?")
@@ -60,7 +60,7 @@
(~docs/section :title "The Site Is a REPL" :id "repl"
(p "The address bar becomes the input line of a REPL. The page is the output.")
(~docs/code :code (highlight
(~docs/code :src (highlight
"/sx/(about) ;; renders the about page\n/(source.(about)) ;; returns the SX source for the about page\n/(eval.(source.(about))) ;; re-evaluates it live\n\n;; The killer demo:\n/(eval.(map.double.(list.1.2.3))) ;; actually returns (2 4 6)\n\n;; The website IS a REPL. The address bar IS the input."
"lisp"))
(p "You do not need to explain what SX is. You show someone a URL and they "
@@ -72,7 +72,7 @@
(p "The " (code "~") " sigil means \"find and execute this component.\" "
"Components can make onward queries, process results, and return composed content — "
"like server-side includes but Lispy and composable.")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; ~get is a component that fetches, processes, and returns\n/(~get.everything-under-the-sun)\n\n;; The flow:\n;; 1. Server finds ~get component in env\n;; 2. ~get makes onward queries\n;; 3. Processes and transforms results\n;; 4. Returns composed hypermedia\n\n;; Because it's all SX, you nest and compose:\n/(~page.home\n .(~hero.banner)\n .(~get.latest-posts.(limit.5))\n .(~get.featured.(filter.pinned)))\n\n;; Each ~component is independently:\n;; - cacheable (by its expression)\n;; - reusable (same component, different args)\n;; - testable (evaluate in isolation)"
"lisp"))
(p "This is what React Server Components are trying to do — server-side data resolution "
@@ -167,7 +167,7 @@
(p "Any " (code "defcomp") " is directly addressable via its " (code "~plans/content-addressed-components/name") ". "
"The URL evaluator sees " (code "~essays/sx-sucks/essay-sx-sucks") ", looks it up in the component env, "
"evaluates it, wraps in " (code "~layouts/doc") ", and returns.")
(~docs/code :code (highlight
(~docs/code :src (highlight
";; Page functions are convenience wrappers:\n/(etc.(essay.sx-sucks)) ;; dispatches via case statement\n\n;; But you can bypass them entirely:\n/(~essays/sx-sucks/essay-sx-sucks) ;; direct component — no routing needed\n\n;; Every defcomp is instantly URL-accessible:\n/(~plans/sx-urls/plan-sx-urls-content) ;; this very page\n/(~analyzer/bundle-analyzer-content) ;; tools\n/(~docs-content/docs-evaluator-content) ;; docs"
"lisp"))
(p "New components are instantly URL-accessible without routing wiring. "
@@ -217,26 +217,26 @@
"is parsed as SX and evaluated with a " (strong "soft eval") ": "
"known function names are called; unknown symbols self-evaluate to their name as a string; "
"components (" (code "~plans/content-addressed-components/name") ") are looked up in the component env.")
(~docs/code :code (highlight
(~docs/code :src (highlight
"/sx/(language.(doc.introduction))\n\n;; After dot→space: (language (doc introduction))\n;; 1. Eval `introduction` → not a known function → \"introduction\"\n;; 2. Eval (doc \"introduction\") → call doc(\"introduction\") → page content\n;; 3. Eval (language content) → call language(content) → passes through\n;; 4. Router wraps result in (~layouts/doc :path \"(language (doc introduction))\" ...)\n\n/(~essays/sx-sucks/essay-sx-sucks)\n;; 1. Eval ~essays/sx-sucks/essay-sx-sucks → component lookup → evaluate → content\n;; 2. Router wraps in ~layouts/doc"
"lisp"))
(~docs/subsection :title "Section Functions"
(p "Structural functions that encode hierarchy and pass through content:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define language\n (fn (&rest args)\n (if (empty? args) (language-index-content) (first args))))\n\n(define geography\n (fn (&rest args)\n (if (empty? args) (geography-index-content) (first args))))\n\n;; Sub-sections also pass through\n(define hypermedia\n (fn (&rest args)\n (if (empty? args) (hypermedia-index-content) (first args))))"
"lisp")))
(~docs/subsection :title "Page Functions"
(p "Leaf functions that dispatch to content components. "
"Data-dependent pages call helpers directly — the async evaluator handles IO:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"(define doc\n (fn (&rest args)\n (let ((slug (first-or-nil args)))\n (if (nil? slug)\n (~docs-content/docs-introduction-content)\n (case slug\n \"introduction\" (~docs-content/docs-introduction-content)\n \"getting-started\" (~docs-content/docs-getting-started-content)\n ...)))))\n\n(define bootstrapper\n (fn (&rest args)\n (let ((slug (first-or-nil args))\n (data (when slug (bootstrapper-data slug))))\n (if (nil? slug)\n (~specs/bootstrappers-index-content)\n (if (get data \"bootstrapper-not-found\")\n (~specs/not-found :slug slug)\n (case slug\n \"python\" (~specs/bootstrapper-py-content ...)\n ...))))))"
"lisp"))))
(~docs/section :title "The Catch-All Route" :id "route"
(p "The entire routing layer becomes one handler:")
(~docs/code :code (highlight
(~docs/code :src (highlight
"@app.get(\"/\")\nasync def sx_home():\n return await eval_sx_url(\"/\")\n\n@app.get(\"/<path:expr>\")\nasync def sx_eval_route(expr):\n return await eval_sx_url(f\"/{expr}\")"
"python"))
(p (code "eval_sx_url") " in seven steps:")
@@ -252,7 +252,7 @@
"so they match first."))
(~docs/section :title "Composability" :id "composability"
(~docs/code :code (highlight
(~docs/code :src (highlight
";; Direct component access\n/(~essays/sx-sucks/essay-sx-sucks)\n/(~specs-explorer/spec-explorer-content)\n\n;; URL special forms\n/(source.(~essays/sx-sucks/essay-sx-sucks)) ;; view defcomp source\n/(inspect.(language.(doc.primitives))) ;; deps, render plan\n/(diff.(language.(spec.signals)).(language.(spec.eval))) ;; side by side\n/(eval.(map.double.(list.1.2.3))) ;; REPL in the URL bar\n\n;; Components as query resolvers\n/(~page.home\n .(~hero.banner)\n .(~get.latest-posts.(limit.5))\n .(~get.featured.(filter.pinned)))\n\n;; Scoping is explicit\n/(users.(posts.123.(filter.published))) ;; filter scoped to posts\n/(users.posts.123.(filter.published)) ;; filter scoped to users\n\n;; Cross-service (future)\n/(market.(product.42.:fields.(name.price)))\n/(subscribe.(etc.(plan.status)))"
"lisp")))

8
sx/sx/plans/theorem-prover.sx
View File

@@ -45,7 +45,7 @@
(~docs/section :title "Phase 1: Definitional satisfiability" :id "phase1"
(p :class "text-stone-600"
"Every " (code "define-primitive") " with a " (code ":body") " produces a " (code "forall") " assertion in SMT-LIB. For example, " (code "(define-primitive \"inc\" :params (n) :body (+ n 1))") " becomes:")
(~docs/code :code (highlight "; inc\n(declare-fun inc (Int) Int)\n(assert (forall (((n Int)))\n (= (inc n) (+ n 1))))\n(check-sat)" "lisp"))
(~docs/code :src (highlight "; inc\n(declare-fun inc (Int) Int)\n(assert (forall (((n Int)))\n (= (inc n) (+ n 1))))\n(check-sat)" "lisp"))
(p :class "text-stone-600"
"This is satisfiable by construction: define " (code "inc(n) = n + 1") " and the assertion holds. " (code "prove.sx") " verifies this mechanically for every primitive — it parses the SMT-LIB, extracts the definition, builds a model, and evaluates it with test values.")
@@ -168,7 +168,7 @@
"Each property also generates SMT-LIB for unbounded verification by an external solver. The strategy: assert the " (em "negation") " of the universal property. If Z3 returns " (code "unsat") ", the property holds for " (em "all") " integers — not just the bounded domain.")
(p :class "text-stone-600"
(code "prove.sx") " reuses " (code "z3-expr") " from " (code "z3.sx") " to translate the property AST to SMT-LIB. Properties with preconditions use " (code "=>") " (implication). The same SX expression is both the bounded test and the formal verification condition.")
(~docs/code :code (highlight smtlib-sample "lisp")))
(~docs/code :src (highlight smtlib-sample "lisp")))
;; --- What it tells us ---
(~docs/section :title "What this tells us about SX" :id "implications"
@@ -245,9 +245,9 @@
(code "prove-search") " walks tuples looking for counterexamples. "
(code "sx-properties") " declares 34 algebraic laws as test functions with quoted ASTs. "
(code "prove-property-smtlib") " translates properties to SMT-LIB verification conditions via " (code "z3-expr") ".")
(~docs/code :code (highlight prove-source "lisp")))
(~docs/code :src (highlight prove-source "lisp")))
(~docs/section :title "The translator: z3.sx" :id "z3-source"
(p :class "text-stone-600"
"The translator that " (code "prove.sx") " depends on. SX expressions that walk other SX expressions and emit SMT-LIB strings. Both files together: ~760 lines of s-expressions, no host language logic.")
(~docs/code :code (highlight z3-source "lisp")))))
(~docs/code :src (highlight z3-source "lisp")))))

42
sx/sx/plans/typed-sx.sx
View File

@@ -61,20 +61,20 @@
(p "Small, practical, no type theory PhD required.")
(~docs/subsection :title "Base types"
(~docs/code :code (highlight ";; Atomic types\nnumber string boolean nil symbol keyword element\n\n;; Nullable (already used in boundary.sx)\nstring? ;; (or string nil)\ndict? ;; (or dict nil)\nnumber? ;; (or number nil)\n\n;; The top type — anything goes\nany\n\n;; The bottom type — never returns (e.g. abort)\nnever" "lisp")))
(~docs/code :src (highlight ";; Atomic types\nnumber string boolean nil symbol keyword element\n\n;; Nullable (already used in boundary.sx)\nstring? ;; (or string nil)\ndict? ;; (or dict nil)\nnumber? ;; (or number nil)\n\n;; The top type — anything goes\nany\n\n;; The bottom type — never returns (e.g. abort)\nnever" "lisp")))
(~docs/subsection :title "Compound types"
(~docs/code :code (highlight ";; Collections with element types\n(list-of number) ;; list where every element is a number\n(list-of string) ;; list of strings\n(list-of any) ;; list (same as untyped)\n(dict-of string number) ;; dict with string keys, number values\n(dict-of string any) ;; dict with string keys (typical kwargs)\n\n;; Union types\n(or string number) ;; either string or number\n(or string nil) ;; same as string?\n\n;; Function types\n(-> number number) ;; number → number\n(-> string string boolean) ;; (string, string) → boolean\n(-> (list-of any) number) ;; list → number\n(-> &rest any number) ;; variadic → number" "lisp")))
(~docs/code :src (highlight ";; Collections with element types\n(list-of number) ;; list where every element is a number\n(list-of string) ;; list of strings\n(list-of any) ;; list (same as untyped)\n(dict-of string number) ;; dict with string keys, number values\n(dict-of string any) ;; dict with string keys (typical kwargs)\n\n;; Union types\n(or string number) ;; either string or number\n(or string nil) ;; same as string?\n\n;; Function types\n(-> number number) ;; number → number\n(-> string string boolean) ;; (string, string) → boolean\n(-> (list-of any) number) ;; list → number\n(-> &rest any number) ;; variadic → number" "lisp")))
(~docs/subsection :title "Component types"
(~docs/code :code (highlight ";; Component type is its keyword signature\n;; Derived automatically from defcomp — no annotation needed\n\n(comp :title string :price number &rest element)\n;; keyword args children type\n\n;; This is NOT a new syntax for defcomp.\n;; It's the TYPE that a defcomp declaration produces.\n;; The checker infers it from parse-comp-params + annotations." "lisp")))
(~docs/code :src (highlight ";; Component type is its keyword signature\n;; Derived automatically from defcomp — no annotation needed\n\n(comp :title string :price number &rest element)\n;; keyword args children type\n\n;; This is NOT a new syntax for defcomp.\n;; It's the TYPE that a defcomp declaration produces.\n;; The checker infers it from parse-comp-params + annotations." "lisp")))
(p "That's the core. No higher-kinded types, no dependent types, no type classes. Just: what goes in, what comes out, can it be nil.")
(~docs/subsection :title "User-defined types"
(~docs/code :code (highlight ";; Type alias — a name for an existing type\n(deftype price number)\n(deftype html-string string)\n\n;; Union — one of several types\n(deftype renderable (union string number nil))\n(deftype key-type (union string keyword))\n\n;; Record — typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number})\n\n;; Parameterized — generic over a type variable\n(deftype (maybe a) (union nil a))\n(deftype (list-of-pairs a b) (list-of (dict-of a b)))\n(deftype (result a e) (union (ok a) (err e)))" "lisp"))
(~docs/code :src (highlight ";; Type alias — a name for an existing type\n(deftype price number)\n(deftype html-string string)\n\n;; Union — one of several types\n(deftype renderable (union string number nil))\n(deftype key-type (union string keyword))\n\n;; Record — typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number})\n\n;; Parameterized — generic over a type variable\n(deftype (maybe a) (union nil a))\n(deftype (list-of-pairs a b) (list-of (dict-of a b)))\n(deftype (result a e) (union (ok a) (err e)))" "lisp"))
(p (code "deftype") " is a declaration form — zero runtime cost, purely for the checker. The type registry resolves user-defined type names during " (code "subtype?") " and " (code "infer-type") ". Records enable typed keyword args for components:")
(~docs/code :code (highlight ";; Define a prop shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :on-click (-> any nil)?})\n\n;; Use it in a component\n(defcomp ~plans/typed-sx/product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n (span (format-decimal (get props :price) 2))\n children))\n\n;; Checker validates dict literals against record shape:\n(~plans/typed-sx/product-card :props {:title \"Widget\" :price \"oops\"})\n;; ^^^^^^\n;; ERROR: :price expects number, got string" "lisp"))))
(~docs/code :src (highlight ";; Define a prop shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :on-click (-> any nil)?})\n\n;; Use it in a component\n(defcomp ~plans/typed-sx/product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n (span (format-decimal (get props :price) 2))\n children))\n\n;; Checker validates dict literals against record shape:\n(~plans/typed-sx/product-card :props {:title \"Widget\" :price \"oops\"})\n;; ^^^^^^\n;; ERROR: :price expects number, got string" "lisp"))))
;; -----------------------------------------------------------------------
;; Annotation syntax
@@ -84,15 +84,15 @@
(p "Annotations are optional. Three places they can appear:")
(~docs/subsection :title "1. Component params"
(~docs/code :code (highlight ";; Current (unchanged, still works)\n(defcomp ~plans/typed-sx/product-card (&key title price image-url &rest children)\n (div ...))\n\n;; Annotated — colon after param name\n(defcomp ~plans/typed-sx/product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n (div ...))\n\n;; Parenthesized pairs: (name : type)\n;; Unannotated params default to `any`\n;; &rest children is always (list-of element)" "lisp"))
(~docs/code :src (highlight ";; Current (unchanged, still works)\n(defcomp ~plans/typed-sx/product-card (&key title price image-url &rest children)\n (div ...))\n\n;; Annotated — colon after param name\n(defcomp ~plans/typed-sx/product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n (div ...))\n\n;; Parenthesized pairs: (name : type)\n;; Unannotated params default to `any`\n;; &rest children is always (list-of element)" "lisp"))
(p "The " (code "(name : type)") " syntax is unambiguous — a 3-element list where the second element is the symbol " (code ":") ". The parser already handles lists inside parameter lists. " (code "parse-comp-params") " gains a branch: if a param is a list of length 3 with " (code ":") " in the middle, extract name and type."))
(~docs/subsection :title "2. Define/lambda return types"
(~docs/code :code (highlight ";; Current (unchanged)\n(define total-price\n (fn (items)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))\n\n;; Annotated — :returns after params\n(define total-price\n (fn ((items : (list-of dict)) :returns number)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))" "lisp"))
(~docs/code :src (highlight ";; Current (unchanged)\n(define total-price\n (fn (items)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))\n\n;; Annotated — :returns after params\n(define total-price\n (fn ((items : (list-of dict)) :returns number)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))" "lisp"))
(p (code ":returns") " is already the convention in " (code "primitives.sx") " and " (code "boundary.sx") ". Same keyword, same position (after params), same meaning."))
(~docs/subsection :title "3. Let bindings"
(~docs/code :code (highlight ";; Current (unchanged)\n(let ((x (compute-value)))\n (+ x 1))\n\n;; Annotated\n(let (((x : number) (compute-value)))\n (+ x 1))\n\n;; Usually unnecessary — the checker infers let binding\n;; types from the right-hand side. Only annotate when\n;; the inference is ambiguous (e.g. the RHS returns `any`)." "lisp"))
(~docs/code :src (highlight ";; Current (unchanged)\n(let ((x (compute-value)))\n (+ x 1))\n\n;; Annotated\n(let (((x : number) (compute-value)))\n (+ x 1))\n\n;; Usually unnecessary — the checker infers let binding\n;; types from the right-hand side. Only annotate when\n;; the inference is ambiguous (e.g. the RHS returns `any`)." "lisp"))
(p "All annotations are syntactically backward-compatible. Unannotated code parses and runs identically. The annotations are simply ignored by evaluators that don't have the type checker loaded."))
@@ -145,7 +145,7 @@
(p "The practical sweet spot: " (strong "annotate component params, nothing else.") " Components are the public API — the boundary between independent pieces of code. Their params are the contract. Internal lambdas and let bindings benefit less from annotations because the checker can infer their types from context.")
(~docs/code :code (highlight ";; Sweet spot: annotate the interface, infer the rest\n(defcomp ~plans/typed-sx/price-display (&key (price : number)\n (currency : string)\n (sale-price : number?))\n ;; Everything below is inferred:\n ;; formatted → string (str returns string)\n ;; discount → number (- returns number)\n ;; has-sale → boolean (and returns boolean)\n (let ((formatted (str currency (format-number price 2)))\n (has-sale (and sale-price (< sale-price price)))\n (discount (if has-sale\n (round (* 100 (/ (- price sale-price) price)))\n 0)))\n (div :class \"price\"\n (span :class (if has-sale \"line-through text-stone-400\" \"font-bold\")\n formatted)\n (when has-sale\n (span :class \"text-green-700 font-bold ml-2\"\n (str currency (format-number sale-price 2))\n (span :class \"text-xs ml-1\" (str \"(-\" discount \"%)\")))))))" "lisp")))
(~docs/code :src (highlight ";; Sweet spot: annotate the interface, infer the rest\n(defcomp ~plans/typed-sx/price-display (&key (price : number)\n (currency : string)\n (sale-price : number?))\n ;; Everything below is inferred:\n ;; formatted → string (str returns string)\n ;; discount → number (- returns number)\n ;; has-sale → boolean (and returns boolean)\n (let ((formatted (str currency (format-number price 2)))\n (has-sale (and sale-price (< sale-price price)))\n (discount (if has-sale\n (round (* 100 (/ (- price sale-price) price)))\n 0)))\n (div :class \"price\"\n (span :class (if has-sale \"line-through text-stone-400\" \"font-bold\")\n formatted)\n (when has-sale\n (span :class \"text-green-700 font-bold ml-2\"\n (str currency (format-number sale-price 2))\n (span :class \"text-xs ml-1\" (str \"(-\" discount \"%)\")))))))" "lisp")))
;; -----------------------------------------------------------------------
;; Error reporting
@@ -154,7 +154,7 @@
(~docs/section :title "Error Reporting" :id "errors"
(p "Type errors are reported at registration time with source location, expected type, actual type, and the full call chain.")
(~docs/code :code (highlight ";; Example error output:\n;;\n;; TYPE ERROR in ~checkout-summary (checkout.sx:34)\n;;\n;; (str \"Total: \" (compute-total items))\n;; ^^^^^^^^^^^^^^^^^\n;; Argument 2 of `str` expects: string\n;; Got: number (from compute-total :returns number)\n;;\n;; Fix: (str \"Total: \" (str (compute-total items)))\n;;\n;;\n;; TYPE ERROR in ~reactive-islands/event-bridge/product-page (products.sx:12)\n;;\n;; (~plans/typed-sx/product-card :title product-name :price \"29.99\")\n;; ^^^^^^\n;; Keyword :price of ~plans/typed-sx/product-card expects: number\n;; Got: string (literal \"29.99\")\n;;\n;; Fix: (~plans/typed-sx/product-card :title product-name :price 29.99)" "lisp"))
(~docs/code :src (highlight ";; Example error output:\n;;\n;; TYPE ERROR in ~checkout-summary (checkout.sx:34)\n;;\n;; (str \"Total: \" (compute-total items))\n;; ^^^^^^^^^^^^^^^^^\n;; Argument 2 of `str` expects: string\n;; Got: number (from compute-total :returns number)\n;;\n;; Fix: (str \"Total: \" (str (compute-total items)))\n;;\n;;\n;; TYPE ERROR in ~reactive-islands/event-bridge/product-page (products.sx:12)\n;;\n;; (~plans/typed-sx/product-card :title product-name :price \"29.99\")\n;; ^^^^^^\n;; Keyword :price of ~plans/typed-sx/product-card expects: number\n;; Got: string (literal \"29.99\")\n;;\n;; Fix: (~plans/typed-sx/product-card :title product-name :price 29.99)" "lisp"))
(p "Severity levels:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -186,7 +186,7 @@
(~docs/section :title "Nil Narrowing" :id "nil"
(p "The most common real-world type error in SX: passing a possibly-nil value where a non-nil is required. " (code "get") " returns " (code "any") " (might be nil). " (code "current-user") " returns " (code "dict?") " (explicitly nullable). Piping these into " (code "str") " or arithmetic without checking is the #1 source of runtime errors.")
(~docs/code :code (highlight ";; Before: runtime error if user is nil\n(defcomp ~plans/typed-sx/greeting (&key (user : dict?))\n (h1 (str \"Hello, \" (get user \"name\"))))\n ;; ^^^ TYPE WARNING: user is dict?, get needs non-nil first arg\n\n;; After: checker enforces nil handling\n(defcomp ~plans/typed-sx/greeting (&key (user : dict?))\n (if user\n (h1 (str \"Hello, \" (get user \"name\")))\n ;; In this branch, checker narrows user to `dict` (not nil)\n (h1 \"Hello, guest\")))\n ;; No warning — nil case handled" "lisp"))
(~docs/code :src (highlight ";; Before: runtime error if user is nil\n(defcomp ~plans/typed-sx/greeting (&key (user : dict?))\n (h1 (str \"Hello, \" (get user \"name\"))))\n ;; ^^^ TYPE WARNING: user is dict?, get needs non-nil first arg\n\n;; After: checker enforces nil handling\n(defcomp ~plans/typed-sx/greeting (&key (user : dict?))\n (if user\n (h1 (str \"Hello, \" (get user \"name\")))\n ;; In this branch, checker narrows user to `dict` (not nil)\n (h1 \"Hello, guest\")))\n ;; No warning — nil case handled" "lisp"))
(p "Narrowing rules:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -203,7 +203,7 @@
(~docs/section :title "Component Signature Verification" :id "signatures"
(p "The highest-value check: verifying that component call sites match declared signatures. This is where most bugs live.")
(~docs/code :code (highlight ";; Definition\n(defcomp ~plans/typed-sx/product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n ...)\n\n;; Call site checks:\n(~plans/typed-sx/product-card :title \"Widget\" :price 29.99) ;; OK\n(~plans/typed-sx/product-card :title \"Widget\") ;; ERROR: :price required\n(~plans/typed-sx/product-card :title 42 :price 29.99) ;; ERROR: :title expects string\n(~plans/typed-sx/product-card :title \"Widget\" :price 29.99\n (p \"Description\") (p \"Details\")) ;; OK: children\n(~plans/typed-sx/product-card :titel \"Widget\" :price 29.99) ;; WARNING: :titel unknown\n ;; (did you mean :title?)" "lisp"))
(~docs/code :src (highlight ";; Definition\n(defcomp ~plans/typed-sx/product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n ...)\n\n;; Call site checks:\n(~plans/typed-sx/product-card :title \"Widget\" :price 29.99) ;; OK\n(~plans/typed-sx/product-card :title \"Widget\") ;; ERROR: :price required\n(~plans/typed-sx/product-card :title 42 :price 29.99) ;; ERROR: :title expects string\n(~plans/typed-sx/product-card :title \"Widget\" :price 29.99\n (p \"Description\") (p \"Details\")) ;; OK: children\n(~plans/typed-sx/product-card :titel \"Widget\" :price 29.99) ;; WARNING: :titel unknown\n ;; (did you mean :title?)" "lisp"))
(p "The checker walks every component call in every component body. For each call:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
@@ -222,7 +222,7 @@
(~docs/section :title "Thread-First Type Flow" :id "thread-first"
(p "The " (code "->") " (thread-first) form is SX's primary composition operator. Type checking it means verifying each step's output matches the next step's input:")
(~docs/code :code (highlight ";; (-> items\n;; (filter active?) ;; (list-of dict) → (list-of dict)\n;; (map name) ;; (list-of dict) → (list-of string)\n;; (join \", \")) ;; (list-of string) → string\n;;\n;; Type flow: (list-of dict) → (list-of dict) → (list-of string) → string\n;; Each step's output is the next step's first argument.\n\n;; ERROR example:\n;; (-> items\n;; (filter active?)\n;; (join \", \") ;; join expects (list-of string),\n;; (map name)) ;; got string — wrong order!\n;;\n;; TYPE ERROR: step 3 (map) expects (list-of any) as first arg\n;; got: string (from join)" "lisp"))
(~docs/code :src (highlight ";; (-> items\n;; (filter active?) ;; (list-of dict) → (list-of dict)\n;; (map name) ;; (list-of dict) → (list-of string)\n;; (join \", \")) ;; (list-of string) → string\n;;\n;; Type flow: (list-of dict) → (list-of dict) → (list-of string) → string\n;; Each step's output is the next step's first argument.\n\n;; ERROR example:\n;; (-> items\n;; (filter active?)\n;; (join \", \") ;; join expects (list-of string),\n;; (map name)) ;; got string — wrong order!\n;;\n;; TYPE ERROR: step 3 (map) expects (list-of any) as first arg\n;; got: string (from join)" "lisp"))
(p "The checker threads the inferred type through each step. If any step's input type doesn't match the previous step's output type, it reports the exact point where the pipeline breaks."))
@@ -271,19 +271,19 @@
(p (code "deftype") " introduces named types — aliases, unions, records, and parameterized types. All are declaration-only, zero runtime cost, resolved at check time.")
(~docs/subsection :title "Type aliases"
(~docs/code :code (highlight ";; Simple name for an existing type\n(deftype price number)\n(deftype html-string string)\n(deftype user-id (or string number))\n\n;; Use anywhere a type is expected\n(defcomp ~plans/typed-sx/price-tag (&key (amount :as price) (label :as string))\n (span :class \"price\" (str label \": $\" (format-decimal amount 2))))" "lisp"))
(~docs/code :src (highlight ";; Simple name for an existing type\n(deftype price number)\n(deftype html-string string)\n(deftype user-id (or string number))\n\n;; Use anywhere a type is expected\n(defcomp ~plans/typed-sx/price-tag (&key (amount :as price) (label :as string))\n (span :class \"price\" (str label \": $\" (format-decimal amount 2))))" "lisp"))
(p "Aliases are transparent — " (code "price") " IS " (code "number") " for all checking purposes. They exist for documentation and domain semantics."))
(~docs/subsection :title "Union types"
(~docs/code :code (highlight ";; Named unions\n(deftype renderable (union string number nil component))\n(deftype key-type (union string keyword))\n(deftype falsy (union nil false))\n\n;; The checker narrows unions in branches:\n(define handle-input\n (fn ((val :as (union string number)))\n (if (string? val)\n (upper val) ;; narrowed to string — upper is valid\n (+ val 1)))) ;; narrowed to number — + is valid" "lisp"))
(~docs/code :src (highlight ";; Named unions\n(deftype renderable (union string number nil component))\n(deftype key-type (union string keyword))\n(deftype falsy (union nil false))\n\n;; The checker narrows unions in branches:\n(define handle-input\n (fn ((val :as (union string number)))\n (if (string? val)\n (upper val) ;; narrowed to string — upper is valid\n (+ val 1)))) ;; narrowed to number — + is valid" "lisp"))
(p "Union types compose with narrowing — " (code "if (string? x)") " in the then-branch narrows " (code "(union string number)") " to " (code "string") ". Same flow typing that already works for nullable."))
(~docs/subsection :title "Record types (typed dicts)"
(~docs/code :code (highlight ";; Typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :tags (list-of string)})\n\n;; Checker validates dict literals against shape:\n{:title \"Widget\" :price \"oops\"}\n;; ERROR: :price expects number, got string\n\n{:title \"Widget\" :price 29.99}\n;; WARNING: missing :tags (required field)\n\n;; Record types enable typed component props:\n(defcomp ~plans/typed-sx/product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n children))" "lisp"))
(~docs/code :src (highlight ";; Typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :tags (list-of string)})\n\n;; Checker validates dict literals against shape:\n{:title \"Widget\" :price \"oops\"}\n;; ERROR: :price expects number, got string\n\n{:title \"Widget\" :price 29.99}\n;; WARNING: missing :tags (required field)\n\n;; Record types enable typed component props:\n(defcomp ~plans/typed-sx/product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n children))" "lisp"))
(p "Records are the big win. Components pass dicts everywhere — config, props, context. A record type makes " (code "get") " on a known-shape dict return the field's type instead of " (code "any") ". This is where " (code "deftype") " pays for itself."))
(~docs/subsection :title "Parameterized types"
(~docs/code :code (highlight ";; Generic over type variables\n(deftype (maybe a) (union nil a))\n(deftype (result a e) (union {:ok a} {:err e}))\n(deftype (pair a b) {:fst a :snd b})\n\n;; Used in signatures:\n(define find-user : (-> number (maybe user-record))\n (fn (id) ...))\n\n;; Checker instantiates: (maybe user-record) = (union nil user-record)\n;; So the caller must handle nil." "lisp"))
(~docs/code :src (highlight ";; Generic over type variables\n(deftype (maybe a) (union nil a))\n(deftype (result a e) (union {:ok a} {:err e}))\n(deftype (pair a b) {:fst a :snd b})\n\n;; Used in signatures:\n(define find-user : (-> number (maybe user-record))\n (fn (id) ...))\n\n;; Checker instantiates: (maybe user-record) = (union nil user-record)\n;; So the caller must handle nil." "lisp"))
(p "Parameterized types are substitution-based — " (code "(maybe string)") " expands to " (code "(union nil string)") " at check time. No inference of type parameters (that would require Hindley-Milner). You write " (code "(maybe string)") " explicitly, the checker substitutes and verifies.")))
;; -----------------------------------------------------------------------
@@ -294,7 +294,7 @@
(p "The pragmatic middle: static effect " (em "checking") " without algebraic effect " (em "handlers") ". Functions declare what side effects they use. The checker enforces that effects don't leak across boundaries. No continuations, no runtime cost.")
(~docs/subsection :title "Effect declarations"
(~docs/code :code (highlight ";; Declare named effects\n(defeffect io) ;; Database, HTTP, file system\n(defeffect dom) ;; Browser DOM manipulation\n(defeffect async) ;; Asynchronous operations\n(defeffect state) ;; Mutable state (set!, dict-set!, append!)\n\n;; Functions declare their effects in brackets\n(define fetch-user : (-> number user) [io async]\n (fn (id) (query \"SELECT * FROM users WHERE id = $1\" id)))\n\n(define toggle-class : (-> element string nil) [dom]\n (fn (el cls) (set-attr! el :class cls)))\n\n;; Pure by default — no annotation means no effects\n(define add-prices : (-> (list-of number) number)\n (fn (prices) (reduce + 0 prices)))" "lisp")))
(~docs/code :src (highlight ";; Declare named effects\n(defeffect io) ;; Database, HTTP, file system\n(defeffect dom) ;; Browser DOM manipulation\n(defeffect async) ;; Asynchronous operations\n(defeffect state) ;; Mutable state (set!, dict-set!, append!)\n\n;; Functions declare their effects in brackets\n(define fetch-user : (-> number user) [io async]\n (fn (id) (query \"SELECT * FROM users WHERE id = $1\" id)))\n\n(define toggle-class : (-> element string nil) [dom]\n (fn (el cls) (set-attr! el :class cls)))\n\n;; Pure by default — no annotation means no effects\n(define add-prices : (-> (list-of number) number)\n (fn (prices) (reduce + 0 prices)))" "lisp")))
(~docs/subsection :title "What it checks"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
@@ -327,14 +327,14 @@
(p "This is exactly the information " (code "deps.sx") " already computes — which components have IO refs. Effects promote it from a runtime classification to a static type-level property. Pure components get an ironclad guarantee: memoize, cache, SSR anywhere, serialize for client — provably safe."))
(~docs/subsection :title "Effect propagation"
(~docs/code :code (highlight ";; Effects propagate through calls:\n(define fetch-prices : (-> (list-of number)) [io async]\n (fn () (query \"SELECT price FROM products\")))\n\n(define render-total : (-> element) [io async] ;; must declare, calls fetch-prices\n (fn ()\n (let ((prices (fetch-prices)))\n (span (str \"$\" (reduce + 0 prices))))))\n\n;; ERROR if you forget:\n(define render-total : (-> element) ;; no effects declared\n (fn ()\n (let ((prices (fetch-prices))) ;; ERROR: calls [io async] from pure context\n (span (str \"$\" (reduce + 0 prices))))))" "lisp"))
(~docs/code :src (highlight ";; Effects propagate through calls:\n(define fetch-prices : (-> (list-of number)) [io async]\n (fn () (query \"SELECT price FROM products\")))\n\n(define render-total : (-> element) [io async] ;; must declare, calls fetch-prices\n (fn ()\n (let ((prices (fetch-prices)))\n (span (str \"$\" (reduce + 0 prices))))))\n\n;; ERROR if you forget:\n(define render-total : (-> element) ;; no effects declared\n (fn ()\n (let ((prices (fetch-prices))) ;; ERROR: calls [io async] from pure context\n (span (str \"$\" (reduce + 0 prices))))))" "lisp"))
(p "The checker walks call graphs and verifies that every function's declared effects are a superset of its callees' effects. This is transitive — if A calls B calls C, and C has " (code "[io]") ", then A must also declare " (code "[io]") "."))
(~docs/subsection :title "Gradual effects"
(p "Like gradual types, effects are opt-in. Unannotated functions are assumed to have " (em "all") " effects — they can call anything, and anything can call them. This is safe (no false positives) but provides no guarantees. As you annotate more functions, the checker catches more violations.")
(p "The practical sweet spot: annotate " (code "defcomp") " declarations (they're the public API) and let the checker verify that pure components don't accidentally depend on IO. Internal helpers can stay unannotated until they matter.")
(~docs/code :code (highlight ";; Annotated component — checker enforces purity\n(defcomp ~plans/typed-sx/price-display [pure] (&key (price :as number))\n (span :class \"price\" (str \"$\" (format-decimal price 2))))\n\n;; ERROR: pure component calls IO\n(defcomp ~plans/typed-sx/price-display [pure] (&key (product-id :as number))\n (let ((product (fetch-product product-id))) ;; ERROR: [io] in [pure] context\n (span :class \"price\" (str \"$\" (get product \"price\")))))" "lisp")))
(~docs/code :src (highlight ";; Annotated component — checker enforces purity\n(defcomp ~plans/typed-sx/price-display [pure] (&key (price :as number))\n (span :class \"price\" (str \"$\" (format-decimal price 2))))\n\n;; ERROR: pure component calls IO\n(defcomp ~plans/typed-sx/price-display [pure] (&key (product-id :as number))\n (let ((product (fetch-product product-id))) ;; ERROR: [io] in [pure] context\n (span :class \"price\" (str \"$\" (get product \"price\")))))" "lisp")))
(~docs/subsection :title "Relationship to deps.sx and boundary.sx"
(p "Effects don't replace the existing systems — they formalize them:")
@@ -397,7 +397,7 @@
(~docs/subsection :title "Phase 5: Typed Primitives (done)"
(p "Add param types to " (code "primitives.sx") " declarations.")
(~docs/code :code (highlight ";; Current\n(define-primitive \"+\"\n :params (&rest args)\n :returns \"number\"\n :doc \"Sum all arguments.\")\n\n;; Extended\n(define-primitive \"+\"\n :params (&rest (args : number))\n :returns \"number\"\n :doc \"Sum all arguments.\")" "lisp"))
(~docs/code :src (highlight ";; Current\n(define-primitive \"+\"\n :params (&rest args)\n :returns \"number\"\n :doc \"Sum all arguments.\")\n\n;; Extended\n(define-primitive \"+\"\n :params (&rest (args : number))\n :returns \"number\"\n :doc \"Sum all arguments.\")" "lisp"))
(p "This is the biggest payoff for effort: ~80 primitives gain param types, enabling the checker to catch every mistyped primitive call across the entire codebase."))
(~docs/subsection :title "Phase 6: User-Defined Types (deftype)"

8
sx/sx/plans/wasm-bytecode-vm.sx
View File

@@ -31,7 +31,7 @@
(h4 :class "font-semibold mt-4 mb-2" "1. Bytecode format — bytecode.sx")
(p "A spec for the bytecode instruction set. Stack-based VM (simpler than register-based, natural fit for s-expressions). Instructions:")
(~docs/code :code (highlight ";; Core instructions\nPUSH_CONST idx ;; push constant from pool\nPUSH_NIL ;; push nil\nPUSH_TRUE / PUSH_FALSE\nLOOKUP idx ;; look up symbol by index\nSET idx ;; define/set symbol\nCALL n ;; call top-of-stack with n args\nTAIL_CALL n ;; tail call (TCO)\nRETURN\nJUMP offset ;; unconditional jump\nJUMP_IF_FALSE offset ;; conditional jump\nMAKE_LAMBDA idx n_params ;; create closure\nMAKE_LIST n ;; collect n stack values into list\nMAKE_DICT n ;; collect 2n stack values into dict\nPOP ;; discard top\nDUP ;; duplicate top" "lisp"))
(~docs/code :src (highlight ";; Core instructions\nPUSH_CONST idx ;; push constant from pool\nPUSH_NIL ;; push nil\nPUSH_TRUE / PUSH_FALSE\nLOOKUP idx ;; look up symbol by index\nSET idx ;; define/set symbol\nCALL n ;; call top-of-stack with n args\nTAIL_CALL n ;; tail call (TCO)\nRETURN\nJUMP offset ;; unconditional jump\nJUMP_IF_FALSE offset ;; conditional jump\nMAKE_LAMBDA idx n_params ;; create closure\nMAKE_LIST n ;; collect n stack values into list\nMAKE_DICT n ;; collect 2n stack values into dict\nPOP ;; discard top\nDUP ;; duplicate top" "lisp"))
(p "Bytecode modules contain: a " (strong "constant pool") " (strings, numbers, symbols), a " (strong "code section") " (instruction bytes), and a " (strong "metadata section") " (source maps, component/island declarations for the host to register).")
(h4 :class "font-semibold mt-4 mb-2" "2. Compiler — compile.sx")
@@ -59,11 +59,11 @@
(h4 :class "font-semibold mt-4 mb-2" "Strategy A: Direct calls")
(p "Each DOM operation (" (code "createElement") ", " (code "setAttribute") ", " (code "appendChild") ") is a separate WASM→JS call. Simple, works, but ~50ns overhead per call. For a page with 1,000 DOM operations, that's ~50μs — negligible.")
(~docs/code :code (highlight "// JS side — imported by WASM\nfunction domCreateElement(tag_ptr, tag_len) {\n const tag = readString(tag_ptr, tag_len);\n return storeHandle(document.createElement(tag));\n}\n\n// Rust side\nextern \"C\" { fn dom_create_element(tag: *const u8, len: u32) -> u32; }" "javascript"))
(~docs/code :src (highlight "// JS side — imported by WASM\nfunction domCreateElement(tag_ptr, tag_len) {\n const tag = readString(tag_ptr, tag_len);\n return storeHandle(document.createElement(tag));\n}\n\n// Rust side\nextern \"C\" { fn dom_create_element(tag: *const u8, len: u32) -> u32; }" "javascript"))
(h4 :class "font-semibold mt-4 mb-2" "Strategy B: Command buffer")
(p "Batch DOM operations in WASM memory as a command buffer. Flush to JS in one call. JS walks the buffer and applies all operations. Fewer boundary crossings, but more complex.")
(~docs/code :code (highlight ";; Command buffer format (in shared WASM memory)\n;; [CREATE_ELEMENT, tag_idx, handle_out]\n;; [SET_ATTR, handle, key_idx, val_idx]\n;; [APPEND_CHILD, parent_handle, child_handle]\n;; [SET_TEXT, handle, text_idx]\n;; Then: (flush-dom-commands)" "lisp"))
(~docs/code :src (highlight ";; Command buffer format (in shared WASM memory)\n;; [CREATE_ELEMENT, tag_idx, handle_out]\n;; [SET_ATTR, handle, key_idx, val_idx]\n;; [APPEND_CHILD, parent_handle, child_handle]\n;; [SET_TEXT, handle, text_idx]\n;; Then: (flush-dom-commands)" "lisp"))
(p "Strategy A is simpler and sufficient for SX workloads. Strategy B is an optimisation if profiling shows the boundary crossing matters. " (strong "Start with A, measure, switch to B only if needed.")))
;; -----------------------------------------------------------------------
@@ -170,7 +170,7 @@
(~docs/section :title "Dual Target: JS or WASM from the Same Spec" :id "dual-target"
(p "The key insight: this is " (strong "not a replacement") " for the JS evaluator. It's " (strong "another compilation target from the same spec") ". The existing bootstrapper pipeline already proves this pattern:")
(~docs/code :code (highlight "eval.sx ──→ bootstrap_js.py ──→ sx-ref.js (browser, JS eval)\n ──→ bootstrap_py.py ──→ sx_ref.py (server, Python eval)\n ──→ bootstrap_rs.py ──→ sx-vm.wasm (browser, WASM eval) ← new" "text"))
(~docs/code :src (highlight "eval.sx ──→ bootstrap_js.py ──→ sx-ref.js (browser, JS eval)\n ──→ bootstrap_py.py ──→ sx_ref.py (server, Python eval)\n ──→ bootstrap_rs.py ──→ sx-vm.wasm (browser, WASM eval) ← new" "text"))
(p "All three outputs have identical semantics because they're compiled from the same source. The choice of which to use is a " (strong "deployment decision") ", not an architectural one:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (strong "JS-only") " — current default. Works everywhere. Zero WASM dependency. Ship sx-browser.js + SX source text.")