coop/rose-ash
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rename all 1,169 components to path-based names with namespace support

Browse Source 
Component names now reflect filesystem location using / as path separator
and : as namespace separator for shared components:
  ~sx-header → ~layouts/header
  ~layout-app-body → ~shared:layout/app-body
  ~blog-admin-dashboard → ~admin/dashboard

209 files, 4,941 replacements across all services.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
giles
2026-03-12 22:00:12 +00:00
parent de80d921e9
commit b0920a1121
209 changed files with 4620 additions and 4620 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
sx/sx/plans/art-dag-sx.sx
View File

@@ -2,8 +2,8 @@
;; Art DAG on SX — SX endpoints as portals into media processing environments
;; ---------------------------------------------------------------------------
(defcomp ~plan-art-dag-sx-content ()
(~doc-page :title "Art DAG on SX"
(defcomp ~plans/art-dag-sx/plan-art-dag-sx-content ()
(~docs/page :title "Art DAG on SX"
(p :class "text-stone-500 text-sm italic mb-8"
"An SX endpoint is a portal into an execution environment. The URL is the entry point; the boundary declaration determines what world you enter.")
@@ -12,7 +12,7 @@
;; I. The endpoint as portal
;; =====================================================================
(~doc-section :title "The endpoint as portal" :id "endpoint-as-portal"
(~docs/section :title "The endpoint as portal" :id "endpoint-as-portal"
(p "An SX endpoint isn't a static route handler. It's a portal into an execution environment. A blog endpoint has " (code "query-posts") ", " (code "render-markdown") ". An art-dag endpoint has " (code "fetch-recipe") ", " (code "resolve-cid") ", " (code "gpu-exec") ", " (code "encode-stream") ", " (code "open-feed") ". Same evaluator. Different primitives. Different capabilities.")
(p "The URL is the entry point. The boundary declaration determines what world you enter. When you hit " (code "/art/render/Qm...") ", the evaluator boots with media-processing primitives. When you hit " (code "/blog/post/hello") ", the evaluator boots with content primitives. The SX code looks the same either way " (em "- ") "function calls, let bindings, conditionals. But the set of primitives available changes everything about what the program can do.")
(p "This is the same principle as the browser/server split. The browser has " (code "render-to-dom") " and " (code "signal") ". The server has " (code "query-db") " and " (code "render-to-html") ". Neither is a restricted version of the other " (em "- ") "they are different environments with different capabilities. The art-dag environment is a third world: " (code "gpu-exec") ", " (code "resolve-cid") ", " (code "encode-stream") ". Same language. Different universe."))
@@ -21,9 +21,9 @@
;; II. Recipes as SX
;; =====================================================================
(~doc-section :title "Recipes as SX" :id "recipes-as-sx"
(~docs/section :title "Recipes as SX" :id "recipes-as-sx"
(p "Art DAG recipes are already s-expression effect chains. Currently executed by L1 Celery workers via Python. The SX version: recipes " (em "are") " SX programs. They evaluate in an environment that has media processing primitives. A recipe doesn't \"call\" a GPU function " (em "- ") "it evaluates in an environment where " (code "gpu-exec") " is a primitive.")
(~doc-code :code (highlight ";; A recipe is an SX program in a media-processing environment\n(define composite-layers\n (fn (base-cid overlay-cid blend)\n (let ((base (resolve-cid base-cid))\n (overlay (resolve-cid overlay-cid)))\n (gpu-exec :op \"composite\"\n :layers (list base overlay)\n :blend blend\n :output :stream))))" "lisp"))
(~docs/code :code (highlight ";; A recipe is an SX program in a media-processing environment\n(define composite-layers\n (fn (base-cid overlay-cid blend)\n (let ((base (resolve-cid base-cid))\n (overlay (resolve-cid overlay-cid)))\n (gpu-exec :op \"composite\"\n :layers (list base overlay)\n :blend blend\n :output :stream))))" "lisp"))
(p "This isn't a DSL embedded in Python. It's SX, the same language that renders pages and defines components. The recipe author uses " (code "let") ", " (code "fn") ", " (code "map") ", " (code "if") " " (em "- ") "the full language. The only difference is what primitives are available. " (code "resolve-cid") " fetches content-addressed data. " (code "gpu-exec") " dispatches GPU operations. These are primitives, not library calls. They exist in the environment the same way " (code "+") " and " (code "str") " exist.")
(p "The recipe is data. It's an s-expression. You can parse it, analyze it, transform it, serialize it, hash it, store it, transmit it. You can inspect a recipe's dependency graph the same way " (code "deps.sx") " inspects component dependencies. You can type-check a recipe the same way " (code "typed-sx") " type-checks components. The tools already exist. They just need a new set of primitives to reason about."))
@@ -31,9 +31,9 @@
;; III. Split execution
;; =====================================================================
(~doc-section :title "Split execution" :id "split-execution"
(~docs/section :title "Split execution" :id "split-execution"
(p "Some recipe steps run against cached data (fast, local). Others need GPU. Others need live input. The evaluator doesn't dispatch " (em "- ") "the boundary declarations do. " (code "(with-boundary (gpu-compute) ...)") " migrates to a GPU-capable host. " (code "(with-boundary (live-ingest) ...)") " opens WebRTC feeds. The recipe author doesn't manage infrastructure " (em "- ") "they declare capabilities, and execution flows to where they exist.")
(~doc-code :code (highlight "(define live-composite\n (fn (recipe-cid camera-count)\n (let ((recipe (resolve-cid recipe-cid)))\n ;; Phase 1: cached data (local, fast)\n (let ((base-layers (map resolve-cid (get recipe \"layers\"))))\n ;; Phase 2: GPU processing\n (with-boundary (gpu-compute)\n (let ((composed (gpu-exec :op \"composite\"\n :layers base-layers\n :blend (get recipe \"blend\"))))\n ;; Phase 3: live feeds\n (with-boundary (live-ingest)\n (let ((feeds (map (fn (i)\n (open-feed :protocol \"webrtc\"\n :label (str \"camera-\" i)))\n (range 0 camera-count))))\n ;; Phase 4: encode and stream\n (with-boundary (encoding)\n (encode-stream\n :sources (concat (list composed) feeds)\n :codec \"h264\"\n :output :stream))))))))))" "lisp"))
(~docs/code :code (highlight "(define live-composite\n (fn (recipe-cid camera-count)\n (let ((recipe (resolve-cid recipe-cid)))\n ;; Phase 1: cached data (local, fast)\n (let ((base-layers (map resolve-cid (get recipe \"layers\"))))\n ;; Phase 2: GPU processing\n (with-boundary (gpu-compute)\n (let ((composed (gpu-exec :op \"composite\"\n :layers base-layers\n :blend (get recipe \"blend\"))))\n ;; Phase 3: live feeds\n (with-boundary (live-ingest)\n (let ((feeds (map (fn (i)\n (open-feed :protocol \"webrtc\"\n :label (str \"camera-\" i)))\n (range 0 camera-count))))\n ;; Phase 4: encode and stream\n (with-boundary (encoding)\n (encode-stream\n :sources (concat (list composed) feeds)\n :codec \"h264\"\n :output :stream))))))))))" "lisp"))
(p "Four phases. Four capability requirements. The program reads linearly " (em "- ") "resolve cached layers, composite on GPU, open live feeds, encode output. But execution migrates across hosts as needed. The " (code "with-boundary") " blocks are the seams. Everything inside a boundary block runs on a host that provides those capabilities. Everything outside runs wherever the program started.")
(p "This is the same mechanism described in the generative SX plan's environment migration section. " (code "with-boundary") " serializes the environment (" (code "env-snapshot") "), ships the pending expression to a capable host, and execution continues there. The recipe author writes a linear program. The runtime makes it distributed."))
@@ -41,7 +41,7 @@
;; IV. Content-addressed everything
;; =====================================================================
(~doc-section :title "Content-addressed everything" :id "content-addressed"
(~docs/section :title "Content-addressed everything" :id "content-addressed"
(p "Recipes are CIDs. Effects are CIDs. Intermediate frames are content-addressed. The execution DAG is a content-addressed graph " (em "- ") "every step can be verified, cached, or replayed.")
(p "A composite of three layers with a specific blend mode always produces the same CID. Caching is automatic: if the CID exists locally, skip the computation. This is the Art DAG's existing model " (em "- ") "SHA3-256 hashes identify everything. SX makes it explicit: the recipe source itself is content-addressed. Two users who write the same recipe get the same CID. They're running the same program.")
(div :class "rounded border border-stone-200 bg-stone-50 p-4 my-4"
@@ -53,19 +53,19 @@
;; V. Feed generation
;; =====================================================================
(~doc-section :title "Feed generation" :id "feed-generation"
(~docs/section :title "Feed generation" :id "feed-generation"
(p "The executing program creates new endpoints as a side effect. " (code "(open-feed ...)") " doesn't return data " (em "- ") "it returns a connectable endpoint. WebRTC peers, SSE streams, WebSocket channels. These are generative acts: the endpoint didn't exist before the recipe ran. The program grew its own input surface.")
(p "When the recipe completes or the island disposes, feeds are cleaned up via the disposal mechanism. An " (code "effect") " in island scope that opens a feed returns a cleanup function. When the island unmounts, the cleanup runs, the feed closes, the endpoint disappears. The lifecycle is automatic.")
(~doc-code :code (highlight ";; A feed is a connectable endpoint, not raw data\n(define create-camera-mosaic\n (fn (camera-ids)\n ;; Each open-feed returns a connectable URL, not bytes\n (let ((feeds (map (fn (id)\n (open-feed :protocol \"webrtc\"\n :label (str \"cam-\" id)\n :quality \"720p\"))\n camera-ids)))\n ;; The mosaic recipe composes feeds as inputs\n (gpu-exec :op \"mosaic\"\n :inputs feeds\n :layout \"grid\"\n :output (open-feed :protocol \"sse\"\n :label \"mosaic-output\"\n :format \"mjpeg\")))))" "lisp"))
(~docs/code :code (highlight ";; A feed is a connectable endpoint, not raw data\n(define create-camera-mosaic\n (fn (camera-ids)\n ;; Each open-feed returns a connectable URL, not bytes\n (let ((feeds (map (fn (id)\n (open-feed :protocol \"webrtc\"\n :label (str \"cam-\" id)\n :quality \"720p\"))\n camera-ids)))\n ;; The mosaic recipe composes feeds as inputs\n (gpu-exec :op \"mosaic\"\n :inputs feeds\n :layout \"grid\"\n :output (open-feed :protocol \"sse\"\n :label \"mosaic-output\"\n :format \"mjpeg\")))))" "lisp"))
(p "The output is itself a feed. A client connects to the mosaic output URL and receives composed frames. The feeds are the program's I/O surface " (em "- ") "they exist because the program created them, and they die when the program stops. No static route configuration. No service mesh. The program declares what it needs and creates what it produces."))
;; =====================================================================
;; VI. The client boundary
;; =====================================================================
(~doc-section :title "The client boundary" :id "client-boundary"
(~docs/section :title "The client boundary" :id "client-boundary"
(p "The browser is just another execution environment with its own primitive set: " (code "render-to-dom") ", " (code "signal") ", " (code "deref") ", " (code "open-feed") " (as WebRTC consumer). A streaming art-dag response arrives as SX wire format. The client evaluates it in island scope " (em "- ") "signals bind to stream frames, " (code "reactive-list") " renders feed thumbnails, " (code "computed") " derives overlay parameters. The server pushes frames; the client renders them reactively. No special video player " (em "- ") "just signals and DOM.")
(~doc-code :code (highlight "(defisland ~live-canvas ()\n (let ((frames (signal nil))\n (feed-url (signal nil)))\n ;; Connect to stream when URL arrives\n (effect (fn ()\n (when (deref feed-url)\n (connect-stream (deref feed-url)\n :on-frame (fn (f) (reset! frames f))))))\n (div :class \"relative aspect-video bg-black rounded\"\n (when (deref frames)\n (canvas :width 1920 :height 1080\n :draw (fn (ctx)\n (draw-frame ctx (deref frames)))))\n (when (not (deref frames))\n (p :class \"absolute inset-0 flex items-center justify-center text-white/50\"\n \"Waiting for stream...\")))))" "lisp"))
(~docs/code :code (highlight "(defisland ~plans/art-dag-sx/live-canvas ()\n (let ((frames (signal nil))\n (feed-url (signal nil)))\n ;; Connect to stream when URL arrives\n (effect (fn ()\n (when (deref feed-url)\n (connect-stream (deref feed-url)\n :on-frame (fn (f) (reset! frames f))))))\n (div :class \"relative aspect-video bg-black rounded\"\n (when (deref frames)\n (canvas :width 1920 :height 1080\n :draw (fn (ctx)\n (draw-frame ctx (deref frames)))))\n (when (not (deref frames))\n (p :class \"absolute inset-0 flex items-center justify-center text-white/50\"\n \"Waiting for stream...\")))))" "lisp"))
(p "The island is reactive. When " (code "frames") " updates, only the canvas redraws. When " (code "feed-url") " updates, the effect reconnects. No polling loop. No WebSocket message handler parsing JSON. The stream is a signal source. The DOM is a signal consumer. The reactive graph connects them.")
(p "This is the same island architecture from the reactive islands plan " (em "- ") "signals, effects, computed, disposal. The only difference is the data source. Instead of an HTMX response mutating a signal, a WebRTC stream mutates a signal. The rendering pipeline doesn't know or care where the data comes from. It reacts."))
@@ -73,17 +73,17 @@
;; VII. L1/L2 integration
;; =====================================================================
(~doc-section :title "L1/L2 integration" :id "l1-l2"
(~docs/section :title "L1/L2 integration" :id "l1-l2"
(p "L1 is the compute layer (Celery workers, GPU nodes). L2 is the registry (ActivityPub, recipe discovery). In SX terms: L1 hosts provide " (code "gpu-exec") ", " (code "encode-stream") ", " (code "resolve-cid-local") ". L2 hosts provide " (code "discover-recipe") ", " (code "publish-recipe") ", " (code "federate-activity") ".")
(p "An SX program that needs both crosses boundaries as needed " (em "- ") "fetch recipe metadata from L2, execute it on L1, publish results back to L2.")
(~doc-code :code (highlight ";; A full pipeline crossing L1 and L2 boundaries\n(define render-and-publish\n (fn (recipe-name output-label)\n ;; L2: discover the recipe\n (with-boundary (registry)\n (let ((recipe-cid (discover-recipe :name recipe-name\n :version \"latest\")))\n ;; L1: execute the recipe\n (with-boundary (gpu-compute)\n (let ((result-cid (gpu-exec :recipe (resolve-cid recipe-cid)\n :output :cid)))\n ;; L2: publish the result\n (with-boundary (registry)\n (publish-recipe\n :name output-label\n :input-cid recipe-cid\n :output-cid result-cid\n :activity \"Create\"))))))))" "lisp"))
(~docs/code :code (highlight ";; A full pipeline crossing L1 and L2 boundaries\n(define render-and-publish\n (fn (recipe-name output-label)\n ;; L2: discover the recipe\n (with-boundary (registry)\n (let ((recipe-cid (discover-recipe :name recipe-name\n :version \"latest\")))\n ;; L1: execute the recipe\n (with-boundary (gpu-compute)\n (let ((result-cid (gpu-exec :recipe (resolve-cid recipe-cid)\n :output :cid)))\n ;; L2: publish the result\n (with-boundary (registry)\n (publish-recipe\n :name output-label\n :input-cid recipe-cid\n :output-cid result-cid\n :activity \"Create\"))))))))" "lisp"))
(p "Three boundary crossings. L2 to find the recipe. L1 to execute it. L2 to publish the result. The program reads as a linear sequence of operations. The runtime handles the dispatch " (em "- ") "which host provides " (code "discover-recipe") ", which host provides " (code "gpu-exec") ", which host provides " (code "publish-recipe") ". The program author doesn't configure endpoints or manage connections. They declare capabilities."))
;; =====================================================================
;; VIII. The primitive sets
;; =====================================================================
(~doc-section :title "The primitive sets" :id "primitive-sets"
(~docs/section :title "The primitive sets" :id "primitive-sets"
(p "Each execution environment provides its own set of primitives. The language is the same everywhere. The capabilities differ.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"

24
sx/sx/plans/async-eval-convergence.sx
View File

@@ -2,10 +2,10 @@
;; Async Evaluator Convergence — Bootstrap async_eval.py from Spec
;; ---------------------------------------------------------------------------
(defcomp ~plan-async-eval-convergence-content ()
(~doc-page :title "Async Evaluator Convergence"
(defcomp ~plans/async-eval-convergence/plan-async-eval-convergence-content ()
(~docs/page :title "Async Evaluator Convergence"
(~doc-section :title "The Problem" :id "problem"
(~docs/section :title "The Problem" :id "problem"
(p "There are currently " (strong "three") " lambda call implementations that must be kept in sync:")
(ol :class "list-decimal list-inside space-y-2 mt-2"
(li (code "shared/sx/ref/eval.sx") " — the canonical spec, bootstrapped to " (code "sx-ref.js") " and " (code "sx_ref.py"))
@@ -14,7 +14,7 @@
(p "Every semantic change to the evaluator — lenient lambda arity, new special forms, calling convention tweaks — must be applied to all three. The spec is authoritative but " (code "async_eval.py") " is what actually serves pages. This is a maintenance hazard and a source of subtle divergence bugs.")
(p "The lenient arity change (lambda params pad missing args with nil instead of erroring) exposed this: the spec and sync evaluator were updated, but " (code "async_eval.py") " still had strict arity checking, causing production crashes."))
(~doc-section :title "Why async_eval.py Exists" :id "why"
(~docs/section :title "Why async_eval.py Exists" :id "why"
(p "The async evaluator exists because SX page rendering needs to:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (strong "Await IO primitives") " — page helpers like " (code "highlight") ", " (code "reference-data") ", " (code "component-source") " call Python async functions (DB queries, HTTP fetches). The spec evaluator is synchronous.")
@@ -26,7 +26,7 @@
;; Architecture
;; -----------------------------------------------------------------------
(~doc-section :title "Target Architecture" :id "architecture"
(~docs/section :title "Target Architecture" :id "architecture"
(p "The goal is to " (strong "eliminate hand-written evaluator code entirely") ". All evaluation semantics come from the spec via bootstrapping. The host provides only:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (strong "Platform primitives") " — type constructors, env operations, DOM/HTML primitives")
@@ -38,17 +38,17 @@
;; Approach: Async Adapter Layer
;; -----------------------------------------------------------------------
(~doc-section :title "Approach: Async Adapter Layer" :id "approach"
(~docs/section :title "Approach: Async Adapter Layer" :id "approach"
(p "Rather than making the spec itself async (which would pollute it with Python-specific concerns), introduce a thin adapter layer between the bootstrapped evaluator and the IO boundary:")
(h4 :class "font-semibold mt-4 mb-2" "Phase 1 — Async call hook")
(p "The bootstrapped evaluator calls primitives via " (code "apply(fn, args)") ". In the Python host, " (code "apply") " is a platform primitive. Replace it with an async-aware version:")
(~doc-code :code (highlight "(define apply-fn\n (fn (f args)\n ;; Platform provides: if f returns a coroutine, await it\n (apply-maybe-async f args)))" "lisp"))
(~docs/code :code (highlight "(define apply-fn\n (fn (f args)\n ;; Platform provides: if f returns a coroutine, await it\n (apply-maybe-async f args)))" "lisp"))
(p "The bootstrapper emits " (code "apply_maybe_async") " as a Python " (code "async def") " that checks if the result is a coroutine and awaits it if so. Pure functions return immediately. IO primitives return coroutines that get awaited. " (strong "Zero overhead for pure calls") " — just an " (code "isinstance") " check.")
(h4 :class "font-semibold mt-4 mb-2" "Phase 2 — Async trampoline")
(p "The spec's trampoline loop resolves thunks synchronously. The Python bootstrapper emits an " (code "async def trampoline") " variant that can await thunks whose bodies contain IO calls. The trampoline structure is identical — only the " (code "await") " keyword is added.")
(~doc-code :code (highlight "# Bootstrapper emits this for Python async target\nasync def trampoline(val):\n while isinstance(val, Thunk):\n val = await eval_expr(val.expr, val.env)\n return val" "python"))
(~docs/code :code (highlight "# Bootstrapper emits this for Python async target\nasync def trampoline(val):\n while isinstance(val, Thunk):\n val = await eval_expr(val.expr, val.env)\n return val" "python"))
(h4 :class "font-semibold mt-4 mb-2" "Phase 3 — Aser as spec module")
(p "The " (code "_aser") " rendering mode (evaluate control flow, serialize HTML/components as SX source) should be specced as a module in " (code "render.sx") " alongside " (code "render-to-html") " and " (code "render-to-dom") ". It's currently hand-written Python because it predates the spec, but its logic is pure SX: walk the AST, eval certain forms, serialize others.")
@@ -61,7 +61,7 @@
;; What changes in the bootstrapper
;; -----------------------------------------------------------------------
(~doc-section :title "Bootstrapper Changes" :id "bootstrapper"
(~docs/section :title "Bootstrapper Changes" :id "bootstrapper"
(p "The Python bootstrapper (" (code "bootstrap_py.py") ") gains a new emit mode: " (code "--async") ". This emits:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (code "async def eval_expr") " instead of " (code "def eval_expr"))
@@ -74,7 +74,7 @@
;; Migration path
;; -----------------------------------------------------------------------
(~doc-section :title "Migration Path" :id "migration"
(~docs/section :title "Migration Path" :id "migration"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -107,14 +107,14 @@
;; Principles
;; -----------------------------------------------------------------------
(~doc-section :title "Principles" :id "principles"
(~docs/section :title "Principles" :id "principles"
(ul :class "list-disc list-inside space-y-2"
(li (strong "The spec is the single source of truth.") " All SX evaluation semantics live in .sx files. Host code implements platform primitives, not evaluation rules.")
(li (strong "Async is a host concern, not a language concern.") " The spec is synchronous. The Python bootstrapper emits async wrappers. The JS bootstrapper emits sync code. The spec doesn't know or care.")
(li (strong "Shadow-compare before switching.") " Every migration step runs both paths in parallel and asserts identical output. No big-bang cutover.")
(li (strong "Aser is just another render mode.") " It belongs in render.sx alongside render-to-html and render-to-dom. It's not special — it's the 'evaluate some, serialize the rest' mode.")))
(~doc-section :title "Outcome" :id "outcome"
(~docs/section :title "Outcome" :id "outcome"
(p "After convergence:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li "One evaluator implementation (the spec), bootstrapped to every host")

98
sx/sx/plans/content-addressed-components.sx
View File

@@ -2,10 +2,10 @@
;; Content-Addressed Components
;; ---------------------------------------------------------------------------
(defcomp ~plan-content-addressed-components-content ()
(~doc-page :title "Content-Addressed Components"
(defcomp ~plans/content-addressed-components/plan-content-addressed-components-content ()
(~docs/page :title "Content-Addressed Components"
(~doc-section :title "The Premise" :id "premise"
(~docs/section :title "The Premise" :id "premise"
(p "SX components are pure functions. Boundary enforcement guarantees it — a component cannot call IO primitives, make network requests, access cookies, or touch the filesystem. " (code "Component.is_pure") " is a structural property, verified at registration time by scanning the transitive closure of IO references via " (code "deps.sx") ".")
(p "Pure functions have a remarkable property: " (strong "their identity is their content.") " Two components that produce the same serialized form are the same component, regardless of who wrote them or where they're hosted. This means we can content-address them — compute a cryptographic hash of the canonical serialized form, and that hash " (em "is") " the component's identity.")
(p "Content addressing turns components into shared infrastructure. Define " (code "~card") " once, pin it to IPFS, and every SX application on the planet can use it by CID. No package registry, no npm install, no version conflicts. The CID " (em "is") " the version. The hash " (em "is") " the trust. Boundary enforcement " (em "is") " the sandbox.")
@@ -15,7 +15,7 @@
;; Current State
;; -----------------------------------------------------------------------
(~doc-section :title "Current State" :id "current-state"
(~docs/section :title "Current State" :id "current-state"
(p "What already exists and what's missing.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -62,28 +62,28 @@
;; Canonical Serialization
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 1: Canonical Serialization" :id "canonical-serialization"
(~docs/section :title "Phase 1: Canonical Serialization" :id "canonical-serialization"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "The foundation")
(p :class "text-violet-800" "Same component must always produce the same bytes, regardless of original formatting, whitespace, or comment placement. Without this, content addressing is meaningless."))
(~doc-subsection :title "The Problem"
(~docs/subsection :title "The Problem"
(p "Currently " (code "serialize(body, pretty=True)") " produces readable SX source from the parsed AST. But serialization isn't fully canonical — it depends on the internal representation order, and there's no normalization pass. Two semantically identical components formatted differently would produce different hashes.")
(p "We need a " (strong "canonical form") " that strips all variance:"))
(~doc-subsection :title "Canonical Form Rules"
(~docs/subsection :title "Canonical Form Rules"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li (strong "Strip comments.") " Comments are parsing artifacts, not part of the AST. The serializer already ignores them (it works from the parsed tree), but any future comment-preserving parser must not affect canonical output.")
(li (strong "Normalize whitespace.") " Single space between tokens, newline before each top-level form in a body. No trailing whitespace. No blank lines.")
(li (strong "Sort keyword arguments alphabetically.") " In component calls: " (code "(~card :class \"x\" :title \"y\")") " not " (code "(~card :title \"y\" :class \"x\")") ". In dict literals: " (code "{:a 1 :b 2}") " not " (code "{:b 2 :a 1}") ".")
(li (strong "Normalize string escapes.") " Use " (code "\\n") " not literal newlines in strings. Escape only what must be escaped.")
(li (strong "Normalize numbers.") " " (code "1.0") " not " (code "1.00") " or " (code "1.") ". " (code "42") " not " (code "042") ".")
(li (strong "Include the full definition form.") " Hash the complete " (code "(defcomp ~name (params) body)") ", not just the body. The name and parameter signature are part of the component's identity.")))
(li (strong "Include the full definition form.") " Hash the complete " (code "(defcomp ~plans/content-addressed-components/name (params) body)") ", not just the body. The name and parameter signature are part of the component's identity.")))
(~doc-subsection :title "Implementation"
(~docs/subsection :title "Implementation"
(p "New spec function in a " (code "canonical.sx") " module:")
(~doc-code :code (highlight "(define canonical-serialize\n (fn (node)\n ;; Produce a canonical s-expression string from an AST node.\n ;; Deterministic: same AST always produces same output.\n ;; Used for CID computation — NOT for human-readable output.\n (case (type-of node)\n \"list\"\n (str \"(\" (join \" \" (map canonical-serialize node)) \")\")\n \"dict\"\n (let ((sorted-keys (sort (keys node))))\n (str \"{\" (join \" \"\n (map (fn (k)\n (str \":\" k \" \" (canonical-serialize (get node k))))\n sorted-keys)) \"}\"))\n \"string\"\n (str '\"' (escape-canonical node) '\"')\n \"number\"\n (canonical-number node)\n \"symbol\"\n (symbol-name node)\n \"keyword\"\n (str \":\" (keyword-name node))\n \"boolean\"\n (if node \"true\" \"false\")\n \"nil\"\n \"nil\")))" "lisp"))
(~docs/code :code (highlight "(define canonical-serialize\n (fn (node)\n ;; Produce a canonical s-expression string from an AST node.\n ;; Deterministic: same AST always produces same output.\n ;; Used for CID computation — NOT for human-readable output.\n (case (type-of node)\n \"list\"\n (str \"(\" (join \" \" (map canonical-serialize node)) \")\")\n \"dict\"\n (let ((sorted-keys (sort (keys node))))\n (str \"{\" (join \" \"\n (map (fn (k)\n (str \":\" k \" \" (canonical-serialize (get node k))))\n sorted-keys)) \"}\"))\n \"string\"\n (str '\"' (escape-canonical node) '\"')\n \"number\"\n (canonical-number node)\n \"symbol\"\n (symbol-name node)\n \"keyword\"\n (str \":\" (keyword-name node))\n \"boolean\"\n (if node \"true\" \"false\")\n \"nil\"\n \"nil\")))" "lisp"))
(p "This function must be bootstrapped to both Python and JS — the server computes CIDs at registration time, the client verifies them on fetch.")
(p "The canonical serializer is distinct from " (code "serialize()") " for display. " (code "serialize(pretty=True)") " remains for human-readable output. " (code "canonical-serialize") " is for hashing only.")))
@@ -91,26 +91,26 @@
;; CID Computation
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 2: CID Computation" :id "cid-computation"
(~docs/section :title "Phase 2: CID Computation" :id "cid-computation"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Every component gets a stable, unique content identifier. Same source → same CID, always. Different source → different CID, always."))
(~doc-subsection :title "CID Format"
(~docs/subsection :title "CID Format"
(p "Use " (a :href "https://github.com/multiformats/cid" :class "text-violet-700 underline" "CIDv1") " with:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Hash function:") " SHA3-256 (already used by artdag for content addressing)")
(li (strong "Codec:") " raw (the content is the canonical SX source bytes, not a DAG-PB wrapper)")
(li (strong "Base encoding:") " base32lower for URL-safe representation (" (code "bafy...") " prefix)"))
(~doc-code :code (highlight ";; CID computation pipeline\n(define component-cid\n (fn (component)\n ;; 1. Reconstruct full defcomp form\n ;; 2. Canonical serialize\n ;; 3. SHA3-256 hash\n ;; 4. Wrap as CIDv1\n (let ((source (canonical-serialize\n (list 'defcomp\n (symbol (str \"~\" (component-name component)))\n (component-params-list component)\n (component-body component)))))\n (cid-v1 :sha3-256 :raw (encode-utf8 source)))))" "lisp")))
(~docs/code :code (highlight ";; CID computation pipeline\n(define component-cid\n (fn (component)\n ;; 1. Reconstruct full defcomp form\n ;; 2. Canonical serialize\n ;; 3. SHA3-256 hash\n ;; 4. Wrap as CIDv1\n (let ((source (canonical-serialize\n (list 'defcomp\n (symbol (str \"~\" (component-name component)))\n (component-params-list component)\n (component-body component)))))\n (cid-v1 :sha3-256 :raw (encode-utf8 source)))))" "lisp")))
(~doc-subsection :title "Where CIDs Live"
(~docs/subsection :title "Where CIDs Live"
(p "Each " (code "Component") " object gains a " (code "cid") " field, computed at registration time:")
(~doc-code :code (highlight ";; types.py extension\n@dataclass\nclass Component:\n name: str\n params: list[str]\n has_children: bool\n body: Any\n closure: dict[str, Any]\n css_classes: set[str]\n deps: set[str] # by name\n io_refs: set[str]\n cid: str | None = None # computed after registration\n dep_cids: dict[str, str] | None = None # name → CID" "python"))
(~docs/code :code (highlight ";; types.py extension\n@dataclass\nclass Component:\n name: str\n params: list[str]\n has_children: bool\n body: Any\n closure: dict[str, Any]\n css_classes: set[str]\n deps: set[str] # by name\n io_refs: set[str]\n cid: str | None = None # computed after registration\n dep_cids: dict[str, str] | None = None # name → CID" "python"))
(p "After " (code "compute_all_deps()") " runs, a new " (code "compute_all_cids()") " pass fills in CIDs for every component. Dependency CIDs are also recorded — when a component references " (code "~card") ", we store both the name and card's CID."))
(~doc-subsection :title "CID Stability"
(~docs/subsection :title "CID Stability"
(p "A component's CID changes when and only when its " (strong "semantics") " change:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Reformatting the " (code ".sx") " source file → same AST → same canonical form → " (strong "same CID"))
@@ -123,14 +123,14 @@
;; Component Manifest
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 3: Component Manifest" :id "manifest"
(~docs/section :title "Phase 3: Component Manifest" :id "manifest"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Metadata that travels with a CID — what a component needs, what it provides, whether it's safe to run. Enough information to resolve, validate, and render without fetching the source first."))
(~doc-subsection :title "Manifest Structure"
(~doc-code :code (highlight ";; Component manifest — published alongside the source\n(SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :source-bytes 847\n :params (:title :price :image-url)\n :has-children true\n :pure true\n :deps (\n {:name \"~card\" :cid \"bafy...card\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"}\n {:name \"~lazy-image\" :cid \"bafy...lazyimg\"})\n :css-atoms (:border :rounded :p-4 :text-sm :font-bold\n :text-green-700 :line-through :text-stone-400)\n :author \"https://rose-ash.com/apps/market\"\n :published \"2026-03-06T14:30:00Z\")" "lisp"))
(~docs/subsection :title "Manifest Structure"
(~docs/code :code (highlight ";; Component manifest — published alongside the source\n(SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :source-bytes 847\n :params (:title :price :image-url)\n :has-children true\n :pure true\n :deps (\n {:name \"~card\" :cid \"bafy...card\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"}\n {:name \"~lazy-image\" :cid \"bafy...lazyimg\"})\n :css-atoms (:border :rounded :p-4 :text-sm :font-bold\n :text-green-700 :line-through :text-stone-400)\n :author \"https://rose-ash.com/apps/market\"\n :published \"2026-03-06T14:30:00Z\")" "lisp"))
(p "Key fields:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code ":cid") " — content address of the canonical serialized source")
@@ -140,7 +140,7 @@
(li (code ":params") " — parameter signature for tooling, documentation, IDE support")
(li (code ":author") " — who published this. AP actor URL, verifiable via HTTP Signatures")))
(~doc-subsection :title "Manifest CID"
(~docs/subsection :title "Manifest CID"
(p "The manifest itself is content-addressed. But the manifest CID is " (em "not") " the component CID — they're separate objects. The component CID is derived from the source alone (pure content). The manifest CID includes metadata that could change (author, publication date) without changing the component.")
(p "Resolution order: manifest CID → manifest → component CID → component source. Or shortcut: component CID → source directly, if you already know what you need.")))
@@ -148,13 +148,13 @@
;; IPFS Storage & Resolution
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 4: IPFS Storage & Resolution" :id "ipfs"
(~docs/section :title "Phase 4: IPFS Storage & Resolution" :id "ipfs"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Components live on IPFS. Any browser can fetch them by CID. No origin server needed. No CDN. No DNS. The content network IS the distribution network."))
(~doc-subsection :title "Server-Side: Publication"
(~docs/subsection :title "Server-Side: Publication"
(p "On component registration (startup or hot-reload), the server:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
(li "Computes canonical form and CID")
@@ -163,14 +163,14 @@
(li "Creates/updates " (code "IPFSPin") " record with " (code "pin_type=\"component\""))
(li "Publishes manifest to IPFS (separate CID)")
(li "Optionally announces via AP outbox for federated discovery"))
(~doc-code :code (highlight ";; IPFSPin usage for components\nIPFSPin(\n content_hash=\"sha3-256:abcdef...\",\n ipfs_cid=\"bafy...productcard\",\n pin_type=\"component\",\n source_type=\"market\", # which service defined it\n metadata={\n \"name\": \"~product-card\",\n \"manifest_cid\": \"bafy...manifest\",\n \"deps\": [\"bafy...card\", \"bafy...pricetag\"],\n \"pure\": True\n }\n)" "python")))
(~docs/code :code (highlight ";; IPFSPin usage for components\nIPFSPin(\n content_hash=\"sha3-256:abcdef...\",\n ipfs_cid=\"bafy...productcard\",\n pin_type=\"component\",\n source_type=\"market\", # which service defined it\n metadata={\n \"name\": \"~product-card\",\n \"manifest_cid\": \"bafy...manifest\",\n \"deps\": [\"bafy...card\", \"bafy...pricetag\"],\n \"pure\": True\n }\n)" "python")))
(~doc-subsection :title "Client-Side: Resolution"
(~docs/subsection :title "Client-Side: Resolution"
(p "New spec module " (code "resolve.sx") " — the client-side component resolution pipeline:")
(~doc-code :code (highlight "(define resolve-component-by-cid\n (fn (cid callback)\n ;; Resolution cascade:\n ;; 1. Check component env (already loaded?)\n ;; 2. Check localStorage (keyed by CID = cache-forever)\n ;; 3. Check origin server (/sx/components?cid=bafy...)\n ;; 4. Fetch from IPFS gateway\n ;; 5. Verify hash matches CID\n ;; 6. Parse, validate purity, register, callback\n (let ((cached (local-storage-get (str \"sx-cid:\" cid))))\n (if cached\n (do\n (register-component-source cached)\n (callback true))\n (fetch-component-by-cid cid\n (fn (source)\n (if (verify-cid cid source)\n (do\n (local-storage-set (str \"sx-cid:\" cid) source)\n (register-component-source source)\n (callback true))\n (do\n (log-warn (str \"sx:cid verification failed \" cid))\n (callback false)))))))))" "lisp"))
(~docs/code :code (highlight "(define resolve-component-by-cid\n (fn (cid callback)\n ;; Resolution cascade:\n ;; 1. Check component env (already loaded?)\n ;; 2. Check localStorage (keyed by CID = cache-forever)\n ;; 3. Check origin server (/sx/components?cid=bafy...)\n ;; 4. Fetch from IPFS gateway\n ;; 5. Verify hash matches CID\n ;; 6. Parse, validate purity, register, callback\n (let ((cached (local-storage-get (str \"sx-cid:\" cid))))\n (if cached\n (do\n (register-component-source cached)\n (callback true))\n (fetch-component-by-cid cid\n (fn (source)\n (if (verify-cid cid source)\n (do\n (local-storage-set (str \"sx-cid:\" cid) source)\n (register-component-source source)\n (callback true))\n (do\n (log-warn (str \"sx:cid verification failed \" cid))\n (callback false)))))))))" "lisp"))
(p "The cache-forever semantics are the key insight: because CIDs are content-addressed, a cached component " (strong "can never be stale") ". If the source changes, it gets a new CID. Old CIDs remain valid forever. There is no cache invalidation problem."))
(~doc-subsection :title "Resolution Cascade"
(~docs/subsection :title "Resolution Cascade"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -210,22 +210,22 @@
;; Security Model
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 5: Security Model" :id "security"
(~docs/section :title "Phase 5: Security Model" :id "security"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "The hard part")
(p :class "text-violet-800" "Loading code from the network is the web's original sin. Content-addressed components are safe because of three structural guarantees — not policies, not trust, not sandboxes that can be escaped."))
(~doc-subsection :title "Guarantee 1: Purity is Structural"
(~docs/subsection :title "Guarantee 1: Purity is Structural"
(p "SX boundary enforcement isn't a runtime sandbox — it's a registration-time structural check. When a component is loaded from IPFS and parsed, " (code "compute_all_io_refs()") " walks its entire AST and transitive dependencies. If " (em "any") " node references an IO primitive, the component is classified as IO-dependent and " (strong "rejected for untrusted registration."))
(p "This means the evaluator literally doesn't have IO primitives in scope when running an IPFS-loaded component. It's not that we catch IO calls — the names don't resolve. There's nothing to catch.")
(~doc-code :code (highlight "(define register-untrusted-component\n (fn (source origin)\n ;; Parse the defcomp from source\n ;; Run compute-all-io-refs on the parsed component\n ;; If io_refs is non-empty → REJECT\n ;; If pure → register in env with :origin metadata\n (let ((comp (parse-component source)))\n (if (not (component-pure? comp))\n (do\n (log-warn (str \"sx:reject IO component from \" origin))\n nil)\n (do\n (register-component comp)\n (log-info (str \"sx:registered \" (component-name comp)\n \" from \" origin))\n comp)))))" "lisp")))
(~docs/code :code (highlight "(define register-untrusted-component\n (fn (source origin)\n ;; Parse the defcomp from source\n ;; Run compute-all-io-refs on the parsed component\n ;; If io_refs is non-empty → REJECT\n ;; If pure → register in env with :origin metadata\n (let ((comp (parse-component source)))\n (if (not (component-pure? comp))\n (do\n (log-warn (str \"sx:reject IO component from \" origin))\n nil)\n (do\n (register-component comp)\n (log-info (str \"sx:registered \" (component-name comp)\n \" from \" origin))\n comp)))))" "lisp")))
(~doc-subsection :title "Guarantee 2: Content Verification"
(~docs/subsection :title "Guarantee 2: Content Verification"
(p "The CID IS the hash. When you fetch " (code "bafy...abc") " from any source — IPFS gateway, origin server, peer — you hash the response and compare. If it doesn't match, you reject it. No MITM attack can alter the content without changing the CID.")
(p "This is stronger than HTTPS. HTTPS trusts the certificate authority, the DNS resolver, and the server operator. Content addressing trusts " (em "mathematics") ". The hash either matches or it doesn't."))
(~doc-subsection :title "Guarantee 3: Evaluation Limits"
(~docs/subsection :title "Guarantee 3: Evaluation Limits"
(p "Pure doesn't mean terminating. A component could contain an infinite loop or exponential recursion. SX evaluators enforce step limits:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Max eval steps:") " configurable per context. Untrusted components get a lower limit than local ones.")
@@ -233,7 +233,7 @@
(li (strong "Max output size:") " prevents a component from producing gigabytes of DOM nodes."))
(p "Exceeding any limit halts evaluation and returns an error node. The worst case is wasted CPU — never data exfiltration, never unauthorized IO."))
(~doc-subsection :title "Trust Tiers"
(~docs/subsection :title "Trust Tiers"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -258,7 +258,7 @@
(td :class "px-3 py-2 text-stone-700" "Pure only (IO rejected)")
(td :class "px-3 py-2 text-stone-600" "Strict limits"))))))
(~doc-subsection :title "What Can Go Wrong"
(~docs/subsection :title "What Can Go Wrong"
(p "Honest accounting of the attack surface:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Visual spoofing:") " A malicious component could render UI that looks like a login form. Mitigation: untrusted components render inside a visually distinct container with origin attribution.")
@@ -271,49 +271,49 @@
;; Wire Format & Prefetch Integration
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 6: Wire Format & Prefetch Integration" :id "wire-format"
(~docs/section :title "Phase 6: Wire Format & Prefetch Integration" :id "wire-format"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Pages and SX responses reference components by CID. The prefetch system resolves them from the most efficient source. Components become location-independent."))
(~doc-subsection :title "CID References in Page Registry"
(~docs/subsection :title "CID References in Page Registry"
(p "The page registry (shipped to the client as " (code "<script type=\"text/sx-pages\">") ") currently lists deps by name. Extend to include CIDs:")
(~doc-code :code (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :deps ({:name \"~essay-foo\" :cid \"bafy...essay\"}\n {:name \"~doc-code\" :cid \"bafy...doccode\"})\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(~docs/code :code (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :deps ({:name \"~essay-foo\" :cid \"bafy...essay\"}\n {:name \"~doc-code\" :cid \"bafy...doccode\"})\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(p "The " (a :href "/sx/(etc.(plan.predictive-prefetch))" :class "text-violet-700 underline" "predictive prefetch system") " uses these CIDs to fetch components from the resolution cascade rather than only from the origin server's " (code "/sx/components") " endpoint."))
(~doc-subsection :title "SX Response Component Headers"
(~docs/subsection :title "SX Response Component Headers"
(p "Currently, " (code "SX-Components") " header lists loaded component names. Extend to support CIDs:")
(~doc-code :code (highlight "Request:\nSX-Components: ~card:bafy...card,~nav:bafy...nav\n\nResponse:\nSX-Component-CIDs: ~essay-foo:bafy...essay,~doc-code:bafy...doccode\n\n;; Response body only includes defs the client doesn't have\n(defcomp ~essay-foo ...)" "http"))
(~docs/code :code (highlight "Request:\nSX-Components: ~card:bafy...card,~plans/environment-images/nav:bafy...nav\n\nResponse:\nSX-Component-CIDs: ~plans/content-addressed-components/essay-foo:bafy...essay,~docs/code:bafy...doccode\n\n;; Response body only includes defs the client doesn't have\n(defcomp ~plans/content-addressed-components/essay-foo ...)" "http"))
(p "The client can then verify received components match their declared CIDs. If the origin server is compromised, CID verification catches the tampered response."))
(~doc-subsection :title "Federated Content"
(~docs/subsection :title "Federated Content"
(p "When an ActivityPub activity arrives with SX content, it declares component requirements by CID:")
(~doc-code :code (highlight "(Create\n :actor \"https://other-instance.com/users/bob\"\n :object (Note\n :content (~product-card :title \"Bob's Widget\" :price 29.99)\n :requires (list\n {:name \"~product-card\" :cid \"bafy...prodcard\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"})))" "lisp"))
(~docs/code :code (highlight "(Create\n :actor \"https://other-instance.com/users/bob\"\n :object (Note\n :content (~product-card :title \"Bob's Widget\" :price 29.99)\n :requires (list\n {:name \"~product-card\" :cid \"bafy...prodcard\"}\n {:name \"~price-tag\" :cid \"bafy...pricetag\"})))" "lisp"))
(p "The receiving browser resolves required components through the cascade. If Bob's instance is down, the components are still fetchable from IPFS. The content is self-describing and self-resolving.")))
;; -----------------------------------------------------------------------
;; Component Sharing & Discovery
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 7: Sharing & Discovery" :id "sharing"
(~docs/section :title "Phase 7: Sharing & Discovery" :id "sharing"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Servers publish component collections via AP. Other servers follow them. Like npm, but federated, content-addressed, and structurally safe."))
(~doc-subsection :title "Component Registry as AP Actor"
(~docs/subsection :title "Component Registry as AP Actor"
(p "Each server exposes a component registry actor:")
(~doc-code :code (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(~docs/code :code (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(p "Follow the registry to receive component updates. The outbox is a chronological feed of Create/Update/Delete activities for components. 'Update' means a new CID for the same name — consumers decide whether to adopt it."))
(~doc-subsection :title "Discovery Protocol"
(~docs/subsection :title "Discovery Protocol"
(p "Webfinger-style lookup for components by name:")
(~doc-code :code (highlight "GET /.well-known/sx-component?name=~product-card\n\n{\n \"name\": \"~product-card\",\n \"cid\": \"bafy...prodcard\",\n \"manifest_cid\": \"bafy...manifest\",\n \"gateway\": \"https://rose-ash.com/ipfs/\",\n \"author\": \"https://rose-ash.com/apps/market\"\n}" "http"))
(~docs/code :code (highlight "GET /.well-known/sx-component?name=~product-card\n\n{\n \"name\": \"~product-card\",\n \"cid\": \"bafy...prodcard\",\n \"manifest_cid\": \"bafy...manifest\",\n \"gateway\": \"https://rose-ash.com/ipfs/\",\n \"author\": \"https://rose-ash.com/apps/market\"\n}" "http"))
(p "This is an optional convenience — any consumer that knows the CID can skip discovery and fetch directly from IPFS. Discovery answers the question: " (em "\"what's the current version of ~product-card on rose-ash.com?\""))
)
(~doc-subsection :title "Name Resolution"
(~docs/subsection :title "Name Resolution"
(p "Names are human-friendly aliases for CIDs. The same name on different servers can refer to different components (different CIDs). Conflict resolution is simple:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Local wins:") " If the server defines " (code "~card") ", that definition takes precedence over any federated " (code "~card") ".")
@@ -324,7 +324,7 @@
;; Spec modules
;; -----------------------------------------------------------------------
(~doc-section :title "Spec Modules" :id "spec-modules"
(~docs/section :title "Spec Modules" :id "spec-modules"
(p "Per the SX host architecture principle, all content-addressing logic is specced in " (code ".sx") " files and bootstrapped:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -351,7 +351,7 @@
;; Critical files
;; -----------------------------------------------------------------------
(~doc-section :title "Critical Files" :id "critical-files"
(~docs/section :title "Critical Files" :id "critical-files"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -404,7 +404,7 @@
;; Relationship
;; -----------------------------------------------------------------------
(~doc-section :title "Relationships" :id "relationships"
(~docs/section :title "Relationships" :id "relationships"
(p "This plan is the foundation for several other plans and roadmaps:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (a :href "/sx/(etc.(plan.sx-activity))" :class "text-violet-700 underline" "SX-Activity") " Phase 2 (content-addressed components on IPFS) is a summary of this plan. This plan supersedes that section with full detail.")

68
sx/sx/plans/environment-images.sx
View File

@@ -2,10 +2,10 @@
;; Content-Addressed Environment Images
;; ---------------------------------------------------------------------------
(defcomp ~plan-environment-images-content ()
(~doc-page :title "Content-Addressed Environment Images"
(defcomp ~plans/environment-images/plan-environment-images-content ()
(~docs/page :title "Content-Addressed Environment Images"
(~doc-section :title "The Idea" :id "idea"
(~docs/section :title "The Idea" :id "idea"
(p "Every served SX endpoint should point back to its spec. The spec CIDs identify the exact evaluator, renderer, parser, and primitives that produced the output. This makes every endpoint " (strong "fully executable") " — anyone with the CIDs can independently reproduce the result.")
(p "But evaluating spec files from source on every cold start is wasteful. The specs are pure — same source always produces the same evaluated environment. So we can serialize the " (em "evaluated") " environment as a content-addressed image: all defcomps, defmacros, bound symbols, resolved closures frozen into a single artifact. The image CID is a function of its contents. Load the image, skip evaluation, get the same result.")
(p "The chain becomes:")
@@ -22,7 +22,7 @@
;; What gets serialized
;; -----------------------------------------------------------------------
(~doc-section :title "What Gets Serialized" :id "what"
(~docs/section :title "What Gets Serialized" :id "what"
(p "An environment image is a snapshot of everything produced by evaluating the spec files. Not the source — the result.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -63,10 +63,10 @@
;; Image format
;; -----------------------------------------------------------------------
(~doc-section :title "Image Format" :id "format"
(~docs/section :title "Image Format" :id "format"
(p "The image is itself an s-expression — the same format the spec is written in. This means the image can be parsed by the same parser, inspected by the same tools, and content-addressed by the same canonical serializer.")
(~doc-code :code (highlight "(sx-image\n :version 1\n :spec-cids {:eval \"bafy...eval\"\n :render \"bafy...render\"\n :parser \"bafy...parser\"\n :primitives \"bafy...prims\"\n :boundary \"bafy...boundary\"\n :signals \"bafy...signals\"}\n\n :components (\n (defcomp ~card (&key title subtitle &rest children)\n (div :class \"card\" (h2 title) (when subtitle (p subtitle)) children))\n (defcomp ~nav (&key items current)\n (nav :class \"nav\" (map (fn (item) ...) items)))\n ;; ... all registered components\n )\n\n :macros (\n (defmacro when (test &rest body)\n (list 'if test (cons 'begin body) nil))\n ;; ... all macros\n )\n\n :bindings (\n (define void-elements (list \"area\" \"base\" \"br\" \"col\" ...))\n (define boolean-attrs (list \"checked\" \"disabled\" ...))\n ;; ... all top-level defines\n )\n\n :primitive-names (\"str\" \"+\" \"-\" \"*\" \"/\" \"=\" \"<\" \">\" ...)\n :io-names (\"fetch-data\" \"call-action\" \"app-url\" ...))" "lisp"))
(~docs/code :code (highlight "(sx-image\n :version 1\n :spec-cids {:eval \"bafy...eval\"\n :render \"bafy...render\"\n :parser \"bafy...parser\"\n :primitives \"bafy...prims\"\n :boundary \"bafy...boundary\"\n :signals \"bafy...signals\"}\n\n :components (\n (defcomp ~plans/environment-images/card (&key title subtitle &rest children)\n (div :class \"card\" (h2 title) (when subtitle (p subtitle)) children))\n (defcomp ~plans/environment-images/nav (&key items current)\n (nav :class \"nav\" (map (fn (item) ...) items)))\n ;; ... all registered components\n )\n\n :macros (\n (defmacro when (test &rest body)\n (list 'if test (cons 'begin body) nil))\n ;; ... all macros\n )\n\n :bindings (\n (define void-elements (list \"area\" \"base\" \"br\" \"col\" ...))\n (define boolean-attrs (list \"checked\" \"disabled\" ...))\n ;; ... all top-level defines\n )\n\n :primitive-names (\"str\" \"+\" \"-\" \"*\" \"/\" \"=\" \"<\" \">\" ...)\n :io-names (\"fetch-data\" \"call-action\" \"app-url\" ...))" "lisp"))
(p "The " (code ":spec-cids") " field is the key. It links this image back to the exact spec that produced it. Anyone can verify the image by:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
@@ -79,11 +79,11 @@
;; Image CID
;; -----------------------------------------------------------------------
(~doc-section :title "Image CID" :id "image-cid"
(~docs/section :title "Image CID" :id "image-cid"
(p "The image CID is computed by canonical-serializing the entire " (code "(sx-image ...)") " form and hashing it. Same process as component CIDs, just applied to the whole environment.")
(p "The relationship between spec CIDs and image CID is deterministic:")
(~doc-code :code (highlight ";; The image CID is a pure function of the spec CIDs\n;; (assuming a deterministic evaluator, which SX guarantees)\n(define image-cid-from-specs\n (fn (spec-cids)\n ;; 1. Fetch each spec file by CID\n ;; 2. Evaluate all specs in a fresh environment\n ;; 3. Extract components, macros, bindings\n ;; 4. Build (sx-image ...) form\n ;; 5. Canonical serialize\n ;; 6. Hash → CID\n ))" "lisp"))
(~docs/code :code (highlight ";; The image CID is a pure function of the spec CIDs\n;; (assuming a deterministic evaluator, which SX guarantees)\n(define image-cid-from-specs\n (fn (spec-cids)\n ;; 1. Fetch each spec file by CID\n ;; 2. Evaluate all specs in a fresh environment\n ;; 3. Extract components, macros, bindings\n ;; 4. Build (sx-image ...) form\n ;; 5. Canonical serialize\n ;; 6. Hash → CID\n ))" "lisp"))
(p "This means you can compute the expected image CID from the spec CIDs " (em "without") " having the image. If someone hands you an image claiming to be from spec " (code "bafy...eval") ", you can verify it by re-evaluating the spec and comparing CIDs. The image is a verifiable cache.")
(p "In practice, you'd only do this verification once per spec version. After that, the image CID is trusted by content-addressing — same bytes, same hash, forever."))
@@ -92,10 +92,10 @@
;; Endpoint provenance
;; -----------------------------------------------------------------------
(~doc-section :title "Endpoint Provenance" :id "provenance"
(~docs/section :title "Endpoint Provenance" :id "provenance"
(p "Every served page gains a provenance header linking it to the spec that rendered it:")
(~doc-code :code (highlight "HTTP/1.1 200 OK\nContent-Type: text/html\nSX-Spec: bafy...eval,bafy...render,bafy...parser,bafy...prims\nSX-Image: bafy...image\nSX-Page-Components: ~card:bafy...card,~nav:bafy...nav" "http"))
(~docs/code :code (highlight "HTTP/1.1 200 OK\nContent-Type: text/html\nSX-Spec: bafy...eval,bafy...render,bafy...parser,bafy...prims\nSX-Image: bafy...image\nSX-Page-Components: ~plans/environment-images/card:bafy...card,~plans/environment-images/nav:bafy...nav" "http"))
(p "Three levels of verification:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -107,7 +107,7 @@
(tbody
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-semibold text-stone-800" "Component")
(td :class "px-3 py-2 text-stone-700" "Fetch " (code "~card") " by CID, verify hash")
(td :class "px-3 py-2 text-stone-700" "Fetch " (code "~plans/environment-images/card") " by CID, verify hash")
(td :class "px-3 py-2 text-stone-600" "Trust the evaluator"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-semibold text-stone-800" "Image")
@@ -124,19 +124,19 @@
;; Cold start optimization
;; -----------------------------------------------------------------------
(~doc-section :title "Cold Start: Images as Cache" :id "cold-start"
(~docs/section :title "Cold Start: Images as Cache" :id "cold-start"
(p "The practical motivation: evaluating all spec files + service components on every server restart is slow. An image eliminates this.")
(~doc-subsection :title "Server Startup"
(~docs/subsection :title "Server Startup"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li "Check if a cached image exists for the current spec CIDs")
(li "If yes: deserialize the image (fast — parsing a single file, no evaluation)")
(li "If no: evaluate spec files from source, build image, cache it")
(li "Register IO primitives and page helpers (host-specific, not in image)")
(li "Ready to serve"))
(~doc-code :code (highlight "(define load-environment\n (fn (spec-cids image-cache-dir)\n (let ((expected-image-cid (image-cid-for-specs spec-cids))\n (cached-path (str image-cache-dir \"/\" expected-image-cid \".sx\")))\n (if (file-exists? cached-path)\n ;; Fast path: deserialize\n (let ((image (parse (read-file cached-path))))\n (if (= (verify-image-cid image) expected-image-cid)\n (deserialize-image image)\n ;; Cache corrupted — rebuild\n (build-and-cache-image spec-cids image-cache-dir)))\n ;; Cold path: evaluate from source\n (build-and-cache-image spec-cids image-cache-dir)))))" "lisp")))
(~docs/code :code (highlight "(define load-environment\n (fn (spec-cids image-cache-dir)\n (let ((expected-image-cid (image-cid-for-specs spec-cids))\n (cached-path (str image-cache-dir \"/\" expected-image-cid \".sx\")))\n (if (file-exists? cached-path)\n ;; Fast path: deserialize\n (let ((image (parse (read-file cached-path))))\n (if (= (verify-image-cid image) expected-image-cid)\n (deserialize-image image)\n ;; Cache corrupted — rebuild\n (build-and-cache-image spec-cids image-cache-dir)))\n ;; Cold path: evaluate from source\n (build-and-cache-image spec-cids image-cache-dir)))))" "lisp")))
(~doc-subsection :title "Client Boot"
(~docs/subsection :title "Client Boot"
(p "The client already caches component definitions in localStorage keyed by bundle hash. Images extend this: cache the entire evaluated environment, not just individual components.")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
(li "Page ships " (code "SX-Image") " header with image CID")
@@ -150,10 +150,10 @@
;; Standalone HTML
;; -----------------------------------------------------------------------
(~doc-section :title "Standalone HTML Bundles" :id "standalone"
(~docs/section :title "Standalone HTML Bundles" :id "standalone"
(p "An image can be inlined into a single HTML document, producing a fully self-contained application with no server dependency:")
(~doc-code :code (highlight "<!doctype html>\n<html>\n<head>\n <script type=\"text/sx-image\">\n (sx-image\n :version 1\n :spec-cids {...}\n :components (...)\n :macros (...)\n :bindings (...))\n </script>\n <script type=\"text/sx-pages\">\n (defpage home :path \"/\" :content (~home-page))\n </script>\n <script src=\"sx-ref.js\"></script>\n</head>\n<body>\n <div id=\"app\"></div>\n <script>\n // Deserialize image, register components, render\n Sx.bootFromImage(document.querySelector('[type=\"text/sx-image\"]'))\n </script>\n</body>\n</html>" "html"))
(~docs/code :code (highlight "<!doctype html>\n<html>\n<head>\n <script type=\"text/sx-image\">\n (sx-image\n :version 1\n :spec-cids {...}\n :components (...)\n :macros (...)\n :bindings (...))\n </script>\n <script type=\"text/sx-pages\">\n (defpage home :path \"/\" :content (~home-page))\n </script>\n <script src=\"sx-ref.js\"></script>\n</head>\n<body>\n <div id=\"app\"></div>\n <script>\n // Deserialize image, register components, render\n Sx.bootFromImage(document.querySelector('[type=\"text/sx-image\"]'))\n </script>\n</body>\n</html>" "html"))
(p "This document is its own CID. Pin it to IPFS and it's a permanent, executable, verifiable application. No origin server, no CDN, no DNS. The content network is the deployment target.")
@@ -165,12 +165,12 @@
;; Namespace scoping
;; -----------------------------------------------------------------------
(~doc-section :title "Namespaced Environments" :id "namespaces"
(~docs/section :title "Namespaced Environments" :id "namespaces"
(p "As the component library grows across services, a flat environment risks name collisions. Images provide a natural boundary for namespace scoping.")
(~doc-code :code (highlight "(sx-image\n :version 1\n :namespace \"market\"\n :spec-cids {...}\n :extends \"bafy...shared-image\" ;; inherits shared components\n :components (\n ;; market-specific components\n (defcomp ~product-card ...)\n (defcomp ~price-tag ...)\n ))" "lisp"))
(~docs/code :code (highlight "(sx-image\n :version 1\n :namespace \"market\"\n :spec-cids {...}\n :extends \"bafy...shared-image\" ;; inherits shared components\n :components (\n ;; market-specific components\n (defcomp ~plans/environment-images/product-card ...)\n (defcomp ~plans/environment-images/price-tag ...)\n ))" "lisp"))
(p "Resolution: " (code "market/~product-card") " → look in market image first, then fall through to the shared image (via " (code ":extends") "). Each service produces its own image, layered on top of the shared base.")
(p "Resolution: " (code "market/~plans/environment-images/product-card") " → look in market image first, then fall through to the shared image (via " (code ":extends") "). Each service produces its own image, layered on top of the shared base.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
@@ -181,19 +181,19 @@
(tbody
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "bafy...shared")
(td :class "px-3 py-2 text-stone-700" "~card, ~nav, ~section-nav, ~doc-page, ~doc-code — shared components from " (code "shared/sx/templates/"))
(td :class "px-3 py-2 text-stone-700" "~plans/environment-images/card, ~plans/environment-images/nav, ~nav-data/section-nav, ~docs/page, ~docs/code — shared components from " (code "shared/sx/templates/"))
(td :class "px-3 py-2 text-stone-600" "None (root)"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "bafy...blog")
(td :class "px-3 py-2 text-stone-700" "~post-card, ~post-body, ~tag-list — blog-specific from " (code "blog/sx/"))
(td :class "px-3 py-2 text-stone-700" "~shared:cards/post-card, ~post-body, ~tag-list — blog-specific from " (code "blog/sx/"))
(td :class "px-3 py-2 text-stone-600" "bafy...shared"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "bafy...market")
(td :class "px-3 py-2 text-stone-700" "~product-card, ~price-tag, ~cart-mini — market-specific from " (code "market/sx/"))
(td :class "px-3 py-2 text-stone-700" "~plans/environment-images/product-card, ~plans/environment-images/price-tag, ~shared:fragments/cart-mini — market-specific from " (code "market/sx/"))
(td :class "px-3 py-2 text-stone-600" "bafy...shared"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "bafy...sx-docs")
(td :class "px-3 py-2 text-stone-700" "~doc-section, ~example-source, plans, essays — sx docs from " (code "sx/sx/"))
(td :class "px-3 py-2 text-stone-700" "~docs/section, ~examples/source, plans, essays — sx docs from " (code "sx/sx/"))
(td :class "px-3 py-2 text-stone-600" "bafy...shared")))))
(p "The " (code ":extends") " field is a CID, not a name. Image composition is content-addressed: changing the shared image produces a new shared CID, which invalidates all service images that extend it. Exactly the right cascading behavior."))
@@ -202,10 +202,10 @@
;; Spec → Image → Page chain
;; -----------------------------------------------------------------------
(~doc-section :title "The Verification Chain" :id "chain"
(~docs/section :title "The Verification Chain" :id "chain"
(p "The full provenance chain from served page back to source:")
(~doc-code :code (highlight ";; 1. Page served with provenance headers\n;;\n;; SX-Spec: bafy...eval,bafy...render,...\n;; SX-Image: bafy...market-image\n;; SX-Page: (defpage product :path \"/products/<slug>\" ...)\n;;\n;; 2. Verify image → spec\n;;\n;; Fetch specs by CID → evaluate → build image → compare CID\n;; If match: the image was correctly produced from these specs\n;;\n;; 3. Verify page → image\n;;\n;; Deserialize image → evaluate page defn → render\n;; If output matches served HTML: the page was correctly rendered\n;;\n;; 4. Trust chain terminates at the spec\n;;\n;; The spec is self-hosting (eval.sx evaluates itself)\n;; The spec's CID is its identity\n;; No external trust anchor needed beyond the hash function" "lisp"))
(~docs/code :code (highlight ";; 1. Page served with provenance headers\n;;\n;; SX-Spec: bafy...eval,bafy...render,...\n;; SX-Image: bafy...market-image\n;; SX-Page: (defpage product :path \"/products/<slug>\" ...)\n;;\n;; 2. Verify image → spec\n;;\n;; Fetch specs by CID → evaluate → build image → compare CID\n;; If match: the image was correctly produced from these specs\n;;\n;; 3. Verify page → image\n;;\n;; Deserialize image → evaluate page defn → render\n;; If output matches served HTML: the page was correctly rendered\n;;\n;; 4. Trust chain terminates at the spec\n;;\n;; The spec is self-hosting (eval.sx evaluates itself)\n;; The spec's CID is its identity\n;; No external trust anchor needed beyond the hash function" "lisp"))
(p "This is stronger than code signing. Code signing says " (em "\"this entity vouches for this binary.\"") " Content addressing says " (em "\"this binary is the deterministic output of this source.\"") " No entity needed. No certificate authority. No revocation lists. Just math."))
@@ -213,9 +213,9 @@
;; Implementation phases
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(~doc-subsection :title "Phase 1: Image Serialization"
(~docs/subsection :title "Phase 1: Image Serialization"
(p "Spec module " (code "image.sx") " — serialize and deserialize evaluated environments.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "serialize-environment") " — walk the env, extract components/macros/bindings, produce " (code "(sx-image ...)") " form")
@@ -223,14 +223,14 @@
(li (code "image-cid") " — canonical-serialize the image form, hash → CID")
(li "Must handle closure serialization — component closures reference other components by name, which must be re-linked on deserialization")))
(~doc-subsection :title "Phase 2: Spec Provenance"
(~docs/subsection :title "Phase 2: Spec Provenance"
(p "Compute CIDs for all spec files at startup. Attach to environment metadata.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Hash each spec file's canonical source at load time")
(li "Store in env metadata as " (code ":spec-cids") " dict")
(li "Include in image serialization")))
(~doc-subsection :title "Phase 3: Server-Side Caching"
(~docs/subsection :title "Phase 3: Server-Side Caching"
(p "Cache images on disk keyed by spec CIDs. Skip evaluation on warm restart.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "On startup: compute spec CIDs → derive expected image CID → check cache")
@@ -238,7 +238,7 @@
(li "Cache miss: evaluate specs, serialize image, write cache")
(li "Any spec file change → new spec CID → new image CID → cache miss → rebuild")))
(~doc-subsection :title "Phase 4: Client Images"
(~docs/subsection :title "Phase 4: Client Images"
(p "Ship image CID in response headers. Client caches full env in localStorage.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "SX-Image") " response header with image CID")
@@ -246,7 +246,7 @@
(li "Cache hit: deserialize, skip per-component fetch/parse")
(li "Cache miss: fetch image (single request), deserialize, cache")))
(~doc-subsection :title "Phase 5: Standalone Export"
(~docs/subsection :title "Phase 5: Standalone Export"
(p "Generate self-contained HTML with inlined image. Pin to IPFS.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Inline " (code "(sx-image ...)") " as " (code "<script type=\"text/sx-image\">"))
@@ -254,7 +254,7 @@
(li "Include sx-ref.js (or link to its CID)")
(li "The resulting HTML is a complete application — pin its CID to IPFS")))
(~doc-subsection :title "Phase 6: Namespaced Images"
(~docs/subsection :title "Phase 6: Namespaced Images"
(p "Per-service images with " (code ":extends") " for layered composition.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Shared image: components from " (code "shared/sx/templates/"))
@@ -266,7 +266,7 @@
;; Dependencies
;; -----------------------------------------------------------------------
(~doc-section :title "Dependencies" :id "dependencies"
(~docs/section :title "Dependencies" :id "dependencies"
(p "What must exist before this plan can execute:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"

12
sx/sx/plans/fragment-protocol.sx
View File

@@ -2,14 +2,14 @@
;; Fragment Protocol
;; ---------------------------------------------------------------------------
(defcomp ~plan-fragment-protocol-content ()
(~doc-page :title "Fragment Protocol"
(defcomp ~plans/fragment-protocol/plan-fragment-protocol-content ()
(~docs/page :title "Fragment Protocol"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "Fragment endpoints return raw sexp source (e.g., " (code "(~blog-nav-wrapper :items ...)") "). The consuming service embeds this in its page sexp, which the client evaluates. But service-specific components like " (code "~blog-nav-wrapper") " are only in that service's component env — not in the consumer's. So the consumer's " (code "client_components_tag()") " never sends them to the client, causing \"Unknown component\" errors.")
(p "The fix: transfer component definitions alongside fragments. Services tell the provider what they already have; the provider sends only what's missing."))
(~doc-section :title "What exists" :id "exists"
(~docs/section :title "What exists" :id "exists"
(div :class "rounded border border-green-200 bg-green-50 p-4"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Fragment GET infrastructure works (" (code "shared/infrastructure/fragments.py") ")")
@@ -17,7 +17,7 @@
(li "Content type negotiation for text/html and text/sx responses")
(li "Fragment caching and composition in page rendering"))))
(~doc-section :title "What remains" :id "remains"
(~docs/section :title "What remains" :id "remains"
(div :class "rounded border border-amber-200 bg-amber-50 p-4"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "POST sexp protocol: ") "Switch from GET to POST with structured sexp body containing " (code ":components") " list of what consumer already has")
@@ -26,7 +26,7 @@
(li (strong "Register received defs: ") "Consumer parses " (code ":defs") " from response and registers into its " (code "_COMPONENT_ENV"))
(li (strong "Shared blueprint factory: ") (code "create_fragment_blueprint(handlers)") " to deduplicate the identical fragment endpoint pattern across 8 services"))))
(~doc-section :title "Files to modify" :id "files"
(~docs/section :title "Files to modify" :id "files"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"

116
sx/sx/plans/generative-sx.sx
View File

@@ -2,8 +2,8 @@
;; Generative SX — programs that write themselves as they run
;; ---------------------------------------------------------------------------
(defcomp ~plan-generative-sx-content ()
(~doc-page :title "Generative SX"
(defcomp ~plans/generative-sx/plan-generative-sx-content ()
(~docs/page :title "Generative SX"
(p :class "text-stone-500 text-sm italic mb-8"
"In the browser, SX is a program that modifies itself in response to external stimuli. Outside the browser, it becomes a program that writes itself as it runs.")
@@ -12,7 +12,7 @@
;; I. The observation
;; =====================================================================
(~doc-section :title "The observation" :id "observation"
(~docs/section :title "The observation" :id "observation"
(p "The marshes work made something visible. A server response arrives carrying " (code "(reset! (use-store \"price\") 14.99)") " inside a " (code "data-init") " script. The SX evaluator parses this string, evaluates it in its own environment, and mutates its own signal graph. The program accepted new source at runtime and changed itself.")
(p "This isn't metaprogramming. Macros expand at compile time — they transform source before evaluation. This is different: the program is " (em "already running") " when it receives new code, evaluates it, and continues with an extended state. The DOM is just the boundary. The signal graph is just the state. The mechanism is general:")
(ol :class "list-decimal list-inside space-y-2 text-stone-600"
@@ -27,34 +27,34 @@
;; II. What already exists
;; =====================================================================
(~doc-section :title "What already exists" :id "exists"
(~docs/section :title "What already exists" :id "exists"
(p "The pieces are already built. They just haven't been connected into the generative pattern.")
(~doc-subsection :title "Homoiconicity"
(~docs/subsection :title "Homoiconicity"
(p "SX code is SX data. " (code "parse") " takes a string and returns a list. " (code "aser") " takes a list and returns a string. These round-trip perfectly. The program can read its own source as naturally as it reads a config file, because they're the same format.")
(~doc-code :code (highlight ";; Code is data\n(define source \"(+ 1 2)\")\n(define ast (parse source)) ;; → (list '+ 1 2)\n(define result (eval-expr ast env)) ;; → 3\n\n;; Data is code\n(define spec '(define greet (fn (name) (str \"Hello, \" name \"!\"))))\n(eval-expr spec env)\n(greet \"world\") ;; → \"Hello, world!\"" "lisp")))
(~docs/code :code (highlight ";; Code is data\n(define source \"(+ 1 2)\")\n(define ast (parse source)) ;; → (list '+ 1 2)\n(define result (eval-expr ast env)) ;; → 3\n\n;; Data is code\n(define spec '(define greet (fn (name) (str \"Hello, \" name \"!\"))))\n(eval-expr spec env)\n(greet \"world\") ;; → \"Hello, world!\"" "lisp")))
(~doc-subsection :title "Runtime eval"
(~docs/subsection :title "Runtime eval"
(p (code "eval-expr") " is available at runtime, not just boot. " (code "data-init") " scripts already use it. Any SX string can become running code at any point in the program's execution. This is not " (code "eval()") " bolted onto a language that doesn't want it — it's the " (em "primary mechanism") " of the language."))
(~doc-subsection :title "The environment model"
(~docs/subsection :title "The environment model"
(p (code "env-extend") " creates a child scope. " (code "env-set!") " adds to the current scope. " (code "define") " creates new bindings. New definitions don't replace old ones — they extend the environment. The program grows monotonically. Every previous state is still reachable through scope chains.")
(p "This is the critical property. A generative program doesn't destroy what it was — it " (em "becomes more") ". Each generation includes everything before it plus what it just wrote."))
(~doc-subsection :title "The bootstrapper"
(~docs/subsection :title "The bootstrapper"
(p "The bootstrapper reads " (code "eval.sx") " — the evaluator's definition of itself — and emits JavaScript or Python that " (em "is") " that evaluator. The spec writes itself into a host language. This is already a generative program, frozen at build time: read source → transform → emit target. Generative SX unfreezes this. The transformation happens " (em "while the program runs") ", not before."))
(~doc-subsection :title "Content-addressed identity"
(~docs/subsection :title "Content-addressed identity"
(p "From the Art DAG: all data identified by SHA3-256 hashes. If a program fragment is identified by its hash, then \"writing yourself\" means producing new hashes. The history is immutable. You can always go back. A generative program isn't a mutating blob — it's a DAG of versioned states.")))
;; =====================================================================
;; III. The generative pattern
;; =====================================================================
(~doc-section :title "The generative pattern" :id "pattern"
(~docs/section :title "The generative pattern" :id "pattern"
(p "A generative SX program starts with a seed and grows by evaluating its own output.")
(~doc-code :code (highlight ";; The core loop\n(define run\n (fn (env source)\n (let ((ast (parse source))\n (result (eval-expr ast env)))\n ;; result might be:\n ;; a value → done, return it\n ;; a string → new SX source, evaluate it (grow)\n ;; a list of defs → new definitions to add to env\n ;; a dict → {source: \"...\", env-patch: {...}} (grow + configure)\n (cond\n (string? result)\n (run env result) ;; evaluate the output\n (and (dict? result) (has-key? result \"source\"))\n (let ((patched (env-merge env (get result \"env-patch\"))))\n (run patched (get result \"source\")))\n :else\n result))))" "lisp"))
(~docs/code :code (highlight ";; The core loop\n(define run\n (fn (env source)\n (let ((ast (parse source))\n (result (eval-expr ast env)))\n ;; result might be:\n ;; a value → done, return it\n ;; a string → new SX source, evaluate it (grow)\n ;; a list of defs → new definitions to add to env\n ;; a dict → {source: \"...\", env-patch: {...}} (grow + configure)\n (cond\n (string? result)\n (run env result) ;; evaluate the output\n (and (dict? result) (has-key? result \"source\"))\n (let ((patched (env-merge env (get result \"env-patch\"))))\n (run patched (get result \"source\")))\n :else\n result))))" "lisp"))
(p "The program evaluates source. If the result is more source, it evaluates that too. Each iteration can extend the environment — add new functions, new macros, new primitives. The environment grows. The program becomes capable of things it couldn't do at the start.")
@@ -62,7 +62,7 @@
(p :class "text-violet-900 font-medium" "This is not eval-in-a-loop")
(p :class "text-violet-800 text-sm" "A REPL evaluates user input in a persistent environment. That's interactive, not generative. The generative pattern is different: the program itself decides what to evaluate next. No user in the loop. The output of one evaluation becomes the input to the next. The program writes itself."))
(~doc-subsection :title "Three modes"
(~docs/subsection :title "Three modes"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -91,44 +91,44 @@
;; IV. Concrete manifestations
;; =====================================================================
(~doc-section :title "Concrete manifestations" :id "manifestations"
(~docs/section :title "Concrete manifestations" :id "manifestations"
(~doc-subsection :title "1. The spec that compiles itself"
(~docs/subsection :title "1. The spec that compiles itself"
(p "Currently: " (code "bootstrap_js.py") " (Python) reads " (code "eval.sx") " (SX) and emits " (code "sx-browser.js") " (JavaScript). Three languages. Two of them are hosts, one is the spec.")
(p "Generative version: " (code "eval.sx") " evaluates itself with a code-generation adapter. The evaluator walks its own AST and emits the target language. No Python bootstrapper. No JavaScript template. The spec " (em "is") " the compiler.")
(~doc-code :code (highlight ";; bootstrap.sx — the spec compiles itself\n;;\n;; Load the codegen adapter for the target\n(define emit (load-adapter target)) ;; target = \"js\" | \"py\" | \"go\" | ...\n;;\n;; Read the spec files\n(define spec-source (read-file \"eval.sx\"))\n(define spec-ast (parse spec-source))\n;;\n;; Walk the AST and emit target code\n;; The walker IS the evaluator — eval.sx evaluating eval.sx\n;; with emit instead of execute\n(define target-code\n (eval-expr spec-ast\n (env-extend codegen-env\n ;; Override define, fn, if, etc. to emit instead of execute\n (codegen-special-forms emit))))\n;;\n(write-file (str \"sx-ref.\" (target-extension target)) target-code)" "lisp"))
(~docs/code :code (highlight ";; bootstrap.sx — the spec compiles itself\n;;\n;; Load the codegen adapter for the target\n(define emit (load-adapter target)) ;; target = \"js\" | \"py\" | \"go\" | ...\n;;\n;; Read the spec files\n(define spec-source (read-file \"eval.sx\"))\n(define spec-ast (parse spec-source))\n;;\n;; Walk the AST and emit target code\n;; The walker IS the evaluator — eval.sx evaluating eval.sx\n;; with emit instead of execute\n(define target-code\n (eval-expr spec-ast\n (env-extend codegen-env\n ;; Override define, fn, if, etc. to emit instead of execute\n (codegen-special-forms emit))))\n;;\n(write-file (str \"sx-ref.\" (target-extension target)) target-code)" "lisp"))
(p "This is the bootstrapper rewritten as a generative program. The spec reads itself, walks itself, and writes the output language. Adding a new target means writing a new " (code "load-adapter") " — a set of emitters for the SX special forms. The walker doesn't change. The spec doesn't change. Only the output format changes.")
(p "The current bootstrappers (" (code "bootstrap_js.py") ", " (code "bootstrap_py.py") ") would become the first two adapters. Future targets (Go, Rust, WASM) are additional adapters, written in SX and bootstrapped like everything else."))
(~doc-subsection :title "2. The program that discovers its dependencies"
(p (code "deps.sx") " analyzes component dependency graphs. It walks component ASTs, finds " (code "~name") " references, computes transitive closures. This is the analytic mode — SX analyzing SX.")
(~docs/subsection :title "2. The program that discovers its dependencies"
(p (code "deps.sx") " analyzes component dependency graphs. It walks component ASTs, finds " (code "~plans/content-addressed-components/name") " references, computes transitive closures. This is the analytic mode — SX analyzing SX.")
(p "The generative version: a program that discovers it needs a component, searches for its definition (local files, IPFS, a registry), loads it, evaluates it, and continues. The program grows its own capability set at runtime.")
(~doc-code :code (highlight ";; A program that discovers and loads what it needs\n(define render-page\n (fn (page-name)\n (let ((page-def (lookup-component page-name)))\n (when (nil? page-def)\n ;; Component not found locally — fetch from registry\n (let ((source (fetch-component-source page-name)))\n ;; Evaluate the definition — it joins the environment\n (eval-expr (parse source) env)\n ;; Now it exists\n (set! page-def (lookup-component page-name))))\n ;; Render with all dependencies resolved\n (render-to-html (list page-def)))))" "lisp"))
(~docs/code :code (highlight ";; A program that discovers and loads what it needs\n(define render-page\n (fn (page-name)\n (let ((page-def (lookup-component page-name)))\n (when (nil? page-def)\n ;; Component not found locally — fetch from registry\n (let ((source (fetch-component-source page-name)))\n ;; Evaluate the definition — it joins the environment\n (eval-expr (parse source) env)\n ;; Now it exists\n (set! page-def (lookup-component page-name))))\n ;; Render with all dependencies resolved\n (render-to-html (list page-def)))))" "lisp"))
(p "This already happens in the browser. " (code "sx_response") " prepends missing component definitions as a " (code "data-components") " script block. The client evaluates them and they join the environment. The generative version makes this explicit: the program tells you what it needs, you give it source, it evaluates it, it grows."))
(~doc-subsection :title "3. The test suite that writes tests"
(~docs/subsection :title "3. The test suite that writes tests"
(p "Given a function's signature and a set of properties (" (code "prove.sx") " already has the property language), generate test cases that verify the properties. The program reads its own function definitions, generates SX expressions that test them, and evaluates those expressions.")
(~doc-code :code (highlight ";; Given: a function and properties about it\n(define-property string-reverse-involutory\n :forall ((s string?))\n :holds (= (reverse (reverse s)) s))\n\n;; Generate: test cases from the property\n;; The program reads the property, generates test source, evals it\n(define tests (generate-tests string-reverse-involutory))\n;; tests = list of (assert (= (reverse (reverse \"hello\")) \"hello\"))\n;; (assert (= (reverse (reverse \"\")) \"\"))\n;; (assert (= (reverse (reverse \"a\")) \"a\"))\n;; ... (random strings, edge cases)\n(for-each (fn (t) (eval-expr t env)) tests)" "lisp"))
(~docs/code :code (highlight ";; Given: a function and properties about it\n(define-property string-reverse-involutory\n :forall ((s string?))\n :holds (= (reverse (reverse s)) s))\n\n;; Generate: test cases from the property\n;; The program reads the property, generates test source, evals it\n(define tests (generate-tests string-reverse-involutory))\n;; tests = list of (assert (= (reverse (reverse \"hello\")) \"hello\"))\n;; (assert (= (reverse (reverse \"\")) \"\"))\n;; (assert (= (reverse (reverse \"a\")) \"a\"))\n;; ... (random strings, edge cases)\n(for-each (fn (t) (eval-expr t env)) tests)" "lisp"))
(p "The program analyzed itself (read the property), generated new SX (the test cases), and evaluated it (ran the tests). Three modes — analytic, synthetic, generative — in sequence."))
(~doc-subsection :title "4. The server that extends its own API"
(~docs/subsection :title "4. The server that extends its own API"
(p "An SX server receives a request it doesn't know how to handle. Instead of returning 404, it examines the request, generates a handler, evaluates it, and handles the request.")
(~doc-code :code (highlight ";; A route handler that generates new route handlers\n(define handle-unknown-route\n (fn (path params)\n ;; Analyze what was requested\n (let ((segments (split path \"/\"))\n (resource (nth segments 1))\n (action (nth segments 2)))\n ;; Check if a schema exists for this resource\n (let ((schema (lookup-schema resource)))\n (when schema\n ;; Generate a CRUD handler from the schema\n (let ((handler-source (generate-crud-handler resource action schema)))\n ;; Evaluate it — the handler now exists\n (eval-expr (parse handler-source) env)\n ;; Route future requests to the generated handler\n (register-route path (env-get env (str resource \"-\" action)))\n ;; Handle this request with the new handler\n ((env-get env (str resource \"-\" action)) params)))))))" "lisp"))
(~docs/code :code (highlight ";; A route handler that generates new route handlers\n(define handle-unknown-route\n (fn (path params)\n ;; Analyze what was requested\n (let ((segments (split path \"/\"))\n (resource (nth segments 1))\n (action (nth segments 2)))\n ;; Check if a schema exists for this resource\n (let ((schema (lookup-schema resource)))\n (when schema\n ;; Generate a CRUD handler from the schema\n (let ((handler-source (generate-crud-handler resource action schema)))\n ;; Evaluate it — the handler now exists\n (eval-expr (parse handler-source) env)\n ;; Route future requests to the generated handler\n (register-route path (env-get env (str resource \"-\" action)))\n ;; Handle this request with the new handler\n ((env-get env (str resource \"-\" action)) params)))))))" "lisp"))
(p "This is not code generation in the Rails scaffolding sense — those generate files you then edit. This generates running code at runtime. The handler didn't exist. Now it does. The server grew."))
(~doc-subsection :title "5. The macro system that learns idioms"
(~docs/subsection :title "5. The macro system that learns idioms"
(p "A generative macro system that detects repeated patterns in code and synthesizes macros to capture them. The program watches itself being written and abstracts its own patterns.")
(~doc-code :code (highlight ";; The program notices this pattern appearing repeatedly:\n;; (div :class \"card\" (h2 title) (p body) children...)\n;;\n;; It generates:\n(defmacro ~card (title body &rest children)\n (div :class \"card\"\n (h2 ,title)\n (p ,body)\n ,@children))\n;;\n;; And rewrites its own source to use the new macro.\n;; This is an SX program that:\n;; 1. Analyzed its own AST (found repeated subtrees)\n;; 2. Synthesized a macro (extracted the pattern)\n;; 3. Evaluated the macro definition (extended env)\n;; 4. Rewrote its own source (used the macro)\n;; Four generative steps." "lisp"))
(~docs/code :code (highlight ";; The program notices this pattern appearing repeatedly:\n;; (div :class \"card\" (h2 title) (p body) children...)\n;;\n;; It generates:\n(defmacro ~card (title body &rest children)\n (div :class \"card\"\n (h2 ,title)\n (p ,body)\n ,@children))\n;;\n;; And rewrites its own source to use the new macro.\n;; This is an SX program that:\n;; 1. Analyzed its own AST (found repeated subtrees)\n;; 2. Synthesized a macro (extracted the pattern)\n;; 3. Evaluated the macro definition (extended env)\n;; 4. Rewrote its own source (used the macro)\n;; Four generative steps." "lisp"))
(p "The connection to the Art DAG: each version of the source is content-addressed. The original (before macros) and the refactored (after macros) are both immutable nodes. The generative step is an edge in the DAG. You can always inspect what the program was before it rewrote itself.")))
;; =====================================================================
;; V. The seed
;; =====================================================================
(~doc-section :title "The seed" :id "seed"
(~docs/section :title "The seed" :id "seed"
(p "A generative SX program starts with a seed. The seed must contain enough to bootstrap the generative loop: a parser, an evaluator, and the " (code "run") " function. Everything else is grown.")
(~doc-code :code (highlight ";; seed.sx — the minimal generative program\n;;\n;; This file contains:\n;; - The SX parser (parse)\n;; - The SX evaluator (eval-expr)\n;; - The generative loop (run)\n;; - A source acquisition function (next-source)\n;;\n;; Everything else — primitives, rendering, networking, persistence —\n;; is loaded by the program as it discovers it needs them.\n\n(define run\n (fn (env)\n (let ((source (next-source env)))\n (when source\n (let ((result (eval-expr (parse source) env)))\n (run env))))))\n\n;; Start with a bare environment\n(run (env-extend (dict\n \"parse\" parse\n \"eval-expr\" eval-expr\n \"next-source\" initial-source-fn)))" "lisp"))
(~docs/code :code (highlight ";; seed.sx — the minimal generative program\n;;\n;; This file contains:\n;; - The SX parser (parse)\n;; - The SX evaluator (eval-expr)\n;; - The generative loop (run)\n;; - A source acquisition function (next-source)\n;;\n;; Everything else — primitives, rendering, networking, persistence —\n;; is loaded by the program as it discovers it needs them.\n\n(define run\n (fn (env)\n (let ((source (next-source env)))\n (when source\n (let ((result (eval-expr (parse source) env)))\n (run env))))))\n\n;; Start with a bare environment\n(run (env-extend (dict\n \"parse\" parse\n \"eval-expr\" eval-expr\n \"next-source\" initial-source-fn)))" "lisp"))
(p "The seed is a quine that doesn't just reproduce itself — it " (em "extends") " itself. Each call to " (code "next-source") " returns new SX that the seed evaluates in its own environment. The environment grows. The seed's capabilities grow. But the seed itself never changes — it's the fixed point of the generative process.")
@@ -140,19 +140,19 @@
;; VI. Growth constraints
;; =====================================================================
(~doc-section :title "Growth constraints" :id "constraints"
(~docs/section :title "Growth constraints" :id "constraints"
(p "Unconstrained self-modification is dangerous. A program that can rewrite any part of itself can rewrite its own safety checks. Generative SX needs growth constraints — rules about what the program can and cannot do to itself.")
(~doc-subsection :title "The boundary"
(~docs/subsection :title "The boundary"
(p "The boundary system (" (code "boundary.sx") ") already enforces this. Pure primitives can't do IO. IO primitives can't escape their declared capabilities. Components are classified as pure or IO-dependent. The boundary is checked at registration time — " (code "SX_BOUNDARY_STRICT=1") " means violations crash at startup.")
(p "For generative programs, the boundary extends: generated code is subject to the same constraints as hand-written code. A generative program can't synthesize an IO primitive — it can only compose existing ones. It can't bypass the boundary by generating code that accesses raw platform APIs. The sandbox applies to generated code exactly as it applies to original code.")
(p "This is the key safety property: " (strong "generative SX is sandboxed generative SX") ". The generated code runs in the same evaluator with the same restrictions. No escape hatches."))
(~doc-subsection :title "Content addressing as audit trail"
(~docs/subsection :title "Content addressing as audit trail"
(p "Every piece of generated code is content-addressed. The SHA3-256 hash of the generated source is its identity. You can trace any piece of running code back to the generation step that produced it, the input that triggered that step, and the state of the environment at that point.")
(p "This makes generative programs auditable. \"Where did this function come from?\" has a definite answer: it was generated by " (em "this") " code, from " (em "this") " input, at " (em "this") " point in the generative sequence. The DAG of content hashes is the program's autobiography."))
(~doc-subsection :title "Monotonic growth"
(~docs/subsection :title "Monotonic growth"
(p "The environment model is append-only. " (code "define") " creates new bindings; it doesn't destroy old ones (inner scopes shadow, but the outer binding persists). " (code "env-extend") " creates a child — the parent is immutable.")
(p "A generative program can extend its environment but cannot shrink it. It can add new functions but cannot delete existing ones. It can shadow a function with a new definition but cannot destroy the original. This means the program's history is preserved in its scope chain — you can always inspect what it was before any given generation step.")
(p "Destructive operations (" (code "set!") ") are confined to mutable cells explicitly created for that purpose. The generative loop itself operates on immutable environments extended with each step.")))
@@ -161,14 +161,14 @@
;; VII. Host properties
;; =====================================================================
(~doc-section :title "Host properties" :id "host-properties"
(~docs/section :title "Host properties" :id "host-properties"
(p "A generative SX program runs on a host — JavaScript, Python, Go, bare metal. The host must provide specific properties or the generative loop breaks. These aren't preferences. They're " (em "requirements") ". A host that violates any of them can't run generative SX correctly.")
(~doc-subsection :title "Lossless parse/serialize round-trip"
(~docs/subsection :title "Lossless parse/serialize round-trip"
(p "The host must implement " (code "parse") " and " (code "aser") " such that " (code "(aser (parse source))") " produces semantically equivalent source. Generated code passes through " (code "parse → transform → serialize → parse") " cycles. If the round-trip is lossy — if whitespace, keyword order, or nested structure is corrupted — the generative loop silently degrades. After enough iterations, the program isn't what it thinks it is.")
(p "This is homoiconicity at the implementation level, not just the language level. The host's data structures must faithfully represent the full AST, and the serializer must faithfully reproduce it."))
(~doc-subsection :title "Runtime eval with first-class environments"
(~docs/subsection :title "Runtime eval with first-class environments"
(p (code "eval-in") " requires evaluating arbitrary expressions in arbitrary environments at runtime. The host must support:")
(ul :class "list-disc pl-5 space-y-1 text-stone-600"
(li "Creating new environments (" (code "env-extend") ")")
@@ -177,37 +177,37 @@
(li "Passing environments as values — storing them in variables, returning them from functions"))
(p "Environments aren't implementation detail in a generative program. They're the " (em "state") ". The running environment at generation step N is the complete description of what the program knows. The host must treat environments as first-class values, not hidden interpreter internals."))
(~doc-subsection :title "Monotonic environment growth"
(~docs/subsection :title "Monotonic environment growth"
(p "A generative program that can " (code "undefine") " things becomes unpredictable. If generation step N+1 removes a function that step N defined, step N+2 might reference the missing function and fail — or worse, silently bind to a different function in an outer scope.")
(p "The host must enforce that environments grow monotonically. New bindings append. Existing bindings in a given scope are immutable once set (or explicitly versioned). " (code "env-extend") " creates children; it never mutates the parent. This makes the generative loop convergent — each step strictly increases the program's capabilities, never decreases them."))
(~doc-subsection :title "Content-addressed storage"
(~docs/subsection :title "Content-addressed storage"
(p "Every generated fragment gets a SHA3-256 identity. The host needs native or near-native hashing and a content-addressed store — an in-memory dict at minimum, IPFS at scale. This provides the audit trail: you can always answer \"where did this code come from?\" by walking the hash chain back to the generation step that produced it.")
(p "Without content addressing, generative programs are opaque. You can't diff two versions of a generated function. You can't roll back to a previous generation. You can't verify that two nodes in a seed network generated the same code from the same input. Content addressing makes the generative process " (em "inspectable") "."))
(~doc-subsection :title "Boundary enforcement on generated code"
(~docs/subsection :title "Boundary enforcement on generated code"
(p "Generated code must pass through the same boundary validation as hand-written code. If " (code "write-file") " is a Tier 2 IO primitive, a generated expression can't call it unless the evaluation context permits Tier 2.")
(p "The host must enforce this " (em "at eval time") ", not just at definition time — because generated code didn't exist at definition time. Every call to " (code "eval-in") " must check the boundary. Every primitive invoked by generated code must verify its tier. There is no \"trusted generated code\" — all code is untrusted until the boundary clears it."))
(~doc-subsection :title "Correct quotation and splicing"
(~docs/subsection :title "Correct quotation and splicing"
(p "Quasiquote (" (code "`") "), unquote (" (code ",") "), and unquote-splicing (" (code ",@") ") must work correctly for programmatic code construction. The host needs these as first-class operations, not string concatenation.")
(p "A generative program builds code by template:")
(~doc-code :code (highlight ";; The generative program builds new definitions by template\n(define gen-handler\n (fn (name params body)\n `(define ,name\n (fn ,params\n ,@body))))\n\n;; gen-handler produces an AST, not a string\n;; The AST can be inspected, transformed, hashed, then evaluated\n(eval-in (gen-handler 'greet '(name) '((str \"Hello, \" name))) env)" "lisp"))
(~docs/code :code (highlight ";; The generative program builds new definitions by template\n(define gen-handler\n (fn (name params body)\n `(define ,name\n (fn ,params\n ,@body))))\n\n;; gen-handler produces an AST, not a string\n;; The AST can be inspected, transformed, hashed, then evaluated\n(eval-in (gen-handler 'greet '(name) '((str \"Hello, \" name))) env)" "lisp"))
(p "String concatenation would work — " (code "(str \"(define \" name \" ...)\")") " — but it's fragile, unstructured, and can't be inspected before evaluation. Quasiquote produces an AST. The generative program works with " (em "structure") ", not text."))
(~doc-subsection :title "Tail-call optimization"
(~docs/subsection :title "Tail-call optimization"
(p "The generative loop is inherently recursive: eval produces source, which is eval'd, which may produce more source. Without TCO, the loop blows the stack after enough iterations. The trampoline/thunk mechanism in the spec handles this, but the host must implement it efficiently.")
(p "This is not optional. A generative program that can only recurse a few thousand times before crashing is not a generative program — it's a demo. The self-compiling spec (Phase 1) alone requires walking every node of " (code "eval.sx") ", which is thousands of recursive calls."))
(~doc-subsection :title "Deterministic evaluation order"
(~docs/subsection :title "Deterministic evaluation order"
(p "If two hosts evaluate the same generative program and get different results because of evaluation order, the content hashes diverge. The programs are no longer equivalent. They can't federate (Phase 5), can't verify each other's output, can't share generated code.")
(p "The host must guarantee: dict iteration order is deterministic (insertion order). Argument evaluation is left-to-right. Effect sequencing follows definition order. No observable nondeterminism in pure evaluation. This is what makes generative programs " (em "reproducible") " — same seed, same input, same output, regardless of host."))
(~doc-subsection :title "Serializable state"
(~docs/subsection :title "Serializable state"
(p "For Phase 4 (self-extending server) and Phase 5 (seed network), a generative program needs to pause, serialize its state, and resume elsewhere. The host needs the ability to serialize an environment + pending expression as data.")
(p "This doesn't require first-class continuations (though those work). It requires that everything in the environment is serializable: functions serialize as their source AST, signals as their current value, environments as nested dicts. The " (code "env-snapshot") " primitive provides this. The host must ensure nothing in the environment is opaque — no host-language closures that can't be serialized, no hidden mutable state that isn't captured by the snapshot."))
(~doc-subsection :title "IO isolation"
(~docs/subsection :title "IO isolation"
(p "The generative primitives (" (code "read-file") ", " (code "write-file") ", " (code "list-files") ") are the " (em "only") " way generated code touches the outside world. The host must be able to intercept, log, and deny all IO. There is no escape hatch through FFI or native calls.")
(p "This is what makes generative programs auditable. If the host allows generated code to call raw " (code "fs.writeFileSync") " or " (code "os.system") ", the boundary is meaningless. The host must virtualize all IO through the declared primitives. Generated code that tries to escape the sandbox hits the boundary, not the OS."))
@@ -220,29 +220,29 @@
;; VIII. Environment migration
;; =====================================================================
(~doc-section :title "Environment migration" :id "env-migration"
(~docs/section :title "Environment migration" :id "env-migration"
(p "SX endpoints tunnel into different execution environments with different primitive sets. A browser has " (code "render-to-dom") " but no " (code "gpu-exec") ". A render node has " (code "gpu-exec") " but no " (code "fetch-fragment") ". An ingest server has " (code "open-feed") " but neither. The boundary isn't just a restriction — it's a " (em "capability declaration") ". It tells you what an environment " (em "can do") ".")
(~doc-subsection :title "Boundary as capability declaration"
(~docs/subsection :title "Boundary as capability declaration"
(p "Every environment declares its boundary: the set of primitives it provides. SX source is portable across any environment that satisfies its primitive requirements. If a program only uses pure primitives (Tier 0), it runs anywhere. If it calls " (code "gpu-exec") ", it needs an environment that provides " (code "gpu-exec") ". The boundary is a type signature on the environment itself — not \"what can this code do\" but \"what must the host provide.\"")
(p "This inverts the usual framing. The boundary doesn't " (em "forbid") " — it " (em "requires") ". A generated program that calls " (code "encode-stream") " is declaring a hardware dependency. The boundary system doesn't block the call — it routes the program to a host that can satisfy it."))
(~doc-subsection :title "with-boundary as migration point"
(~docs/subsection :title "with-boundary as migration point"
(p "Execution migrates to where the primitives are. When the evaluator hits a " (code "with-boundary") " block requiring primitives the current host doesn't have, it serializes state (" (code "env-snapshot") "), ships the pending expression plus environment to a host that has them, and execution continues there. The block is the unit of migration, not individual primitive calls.")
(~doc-code :code (highlight "(with-boundary (media-processing encoding)\n (let ((frames (gpu-exec recipe cached-layers)))\n (encode-stream frames :codec \"h264\"\n :on-input-needed (fn (slot)\n (with-boundary (live-ingest)\n (open-feed :protocol \"webrtc\" :slot slot))))))" "lisp"))
(~docs/code :code (highlight "(with-boundary (media-processing encoding)\n (let ((frames (gpu-exec recipe cached-layers)))\n (encode-stream frames :codec \"h264\"\n :on-input-needed (fn (slot)\n (with-boundary (live-ingest)\n (open-feed :protocol \"webrtc\" :slot slot))))))" "lisp"))
(p "This program starts wherever it starts. When it hits " (code "(with-boundary (media-processing encoding) ...)") ", the evaluator checks: does the current host provide " (code "gpu-exec") " and " (code "encode-stream") "? If yes, evaluate in place. If no, snapshot the environment, serialize the pending expression, and dispatch to a host that does. Inside the encoding block, " (code ":on-input-needed") " triggers a nested migration — the " (code "(with-boundary (live-ingest) ...)") " block dispatches to an ingest server that provides " (code "open-feed") ".")
(p "The program doesn't know where it runs. It declares what it needs. The runtime figures out " (em "where") "."))
(~doc-subsection :title "Declaration, not discovery"
(~docs/subsection :title "Declaration, not discovery"
(p "Boundary requirements are declared at scope boundaries, not discovered at call time. This is the critical constraint. A generative program that synthesizes a " (code "with-boundary") " block is declaring — at generation time — what the block will need. The declaration is inspectable before execution. You can analyze a generated program's boundary requirements without running it.")
(p "This gives constraint checking on generated code. A generative loop that produces a " (code "with-boundary") " block must produce a valid boundary declaration. If the generated block calls " (code "gpu-exec") " but doesn't declare " (code "media-processing") ", the boundary checker rejects it — at generation time, not at runtime. The program must say what it needs before it needs it."))
(~doc-subsection :title "Nested migration"
(~docs/subsection :title "Nested migration"
(p "Nested " (code "with-boundary") " blocks are nested migrations. The program walks the capability graph, carrying its state, accumulating content-addressed history. Each migration is an edge in the DAG — the source environment, the target environment, the serialized state, the pending expression. All content-addressed. All auditable.")
(p "A three-level nesting — browser to render node to ingest server — is three migrations. The browser evaluates the outer expression, hits a " (code "with-boundary") " requiring GPU, migrates to the render node. The render node evaluates until it hits a " (code "with-boundary") " requiring live ingest, migrates to the ingest server. Each migration carries the accumulated environment. Each return ships results back up the chain.")
(p "The nesting depth is bounded by the capability graph. If there are four distinct environment types, the maximum nesting is four. In practice, most programs need one or two migrations. The deep nesting is there for generative programs that discover capabilities as they run."))
(~doc-subsection :title "Environment chaining"
(~docs/subsection :title "Environment chaining"
(p "Split execution — cached layers on one host, GPU rendering on another, encoding on a third — is just environment chaining. The evaluator runs in one environment until it hits a primitive requiring a different one. The primitive " (em "is") " the dispatch.")
(p "This collapses the distinction between \"local function call\" and \"remote service invocation.\" From the SX program's perspective, " (code "gpu-exec") " is a primitive. Whether it runs on the local GPU or a remote render farm is an environment configuration detail, not a language-level concern. The " (code "with-boundary") " block declares the requirement. The runtime satisfies it. The program doesn't care how.")
(p "Environment chaining also explains the Art DAG's three-phase execution pattern (analyze, plan, execute). Each phase runs in a different environment with different primitives. The analyze phase needs " (code "content-hash") " and " (code "list-files") ". The plan phase needs " (code "env-snapshot") " and scheduling primitives. The execute phase needs " (code "gpu-exec") " and storage primitives. Three " (code "with-boundary") " blocks. Three environments. One program.")))
@@ -251,9 +251,9 @@
;; IX. Implementation phases
;; =====================================================================
(~doc-section :title "Implementation phases" :id "phases"
(~docs/section :title "Implementation phases" :id "phases"
(~doc-subsection :title "Phase 0: Generative primitives"
(~docs/subsection :title "Phase 0: Generative primitives"
(p "Add the minimal set of primitives needed for a generative loop. These are IO primitives — they cross the boundary.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
@@ -288,7 +288,7 @@
(td :class "px-3 py-2 text-stone-600" "SHA3-256 hash of source string")))))
(p "These are the building blocks. The generative loop composes them. The primitives themselves are minimal — no networking, no databases, no UI. Just: read, write, evaluate, inspect, hash."))
(~doc-subsection :title "Phase 1: Self-compiling spec"
(~docs/subsection :title "Phase 1: Self-compiling spec"
(p "Rewrite " (code "bootstrap_js.py") " as " (code "bootstrap.sx") ". The bootstrapper becomes an SX program that reads the spec files and emits target code.")
(ol :class "list-decimal list-inside space-y-2 text-stone-600"
(li "Write " (code "codegen-js.sx") " — JavaScript code generation adapter (emit JS from SX AST)")
@@ -298,19 +298,19 @@
(li "Retire the Python bootstrappers"))
(p "This is the first real generative program: SX reading SX and writing JavaScript. The same program, with a different adapter, writes Python. Or Go. Or WASM. The spec doesn't change. Only the adapter changes."))
(~doc-subsection :title "Phase 2: Generative deps"
(~docs/subsection :title "Phase 2: Generative deps"
(p "Rewrite " (code "deps.sx") " as a generative program. Instead of computing a static dependency graph, it runs continuously: watch for new component definitions, update the graph, re-emit optimized bundles.")
(p "This is the deps analyzer turned inside out. Instead of \"analyze all components, output a graph,\" it's \"when a new component appears, update the running graph.\" The dependency analysis is an ongoing computation, not a one-shot pass."))
(~doc-subsection :title "Phase 3: Generative testing"
(~docs/subsection :title "Phase 3: Generative testing"
(p "Connect " (code "prove.sx") " to the generative loop. When a new function is defined, automatically generate property tests, run them, report failures. When a function changes, regenerate and rerun only the affected tests.")
(p "The test suite is not a separate artifact — it's a side effect of the generative process. Every function that enters the environment is tested. The tests are generated from properties, not hand-written. The program verifies itself as it grows."))
(~doc-subsection :title "Phase 4: The self-extending server"
(~docs/subsection :title "Phase 4: The self-extending server"
(p "An SX server with a generative core. New routes, handlers, and middleware can be added at runtime by evaluating SX source. The server's API surface is a living environment that grows with use.")
(p "Not a scripting layer bolted onto a framework — the server " (em "is") " a generative SX program. Its routes are SX definitions. Its middleware is SX functions. Adding a new endpoint means evaluating a new " (code "defhandler") " in the running environment."))
(~doc-subsection :title "Phase 5: The seed network"
(~docs/subsection :title "Phase 5: The seed network"
(p "Multiple generative SX programs exchanging source. Each node runs a seed. When node A discovers a capability it lacks, it requests the source from node B. Node B's generated code is content-addressed — A can verify it, evaluate it, and grow.")
(p "This is SX-Activity applied to generative programs. The wire format is SX. The content is SX. The evaluation is SX. The programs share source, not data. They grow together.")))
@@ -318,7 +318,7 @@
;; X. The strange loop
;; =====================================================================
(~doc-section :title "The strange loop" :id "strange-loop"
(~docs/section :title "The strange loop" :id "strange-loop"
(p "Hofstadter's strange loop: a hierarchy of levels where the top level reaches back down and affects the bottom level. In a generative SX program:")
(ul :class "list-disc pl-5 space-y-2 text-stone-600"
(li "The bottom level is the evaluator — it evaluates expressions")

12
sx/sx/plans/glue-decoupling.sx
View File

@@ -2,14 +2,14 @@
;; Glue Decoupling
;; ---------------------------------------------------------------------------
(defcomp ~plan-glue-decoupling-content ()
(~doc-page :title "Cross-App Decoupling via Glue Services"
(defcomp ~plans/glue-decoupling/plan-glue-decoupling-content ()
(~docs/page :title "Cross-App Decoupling via Glue Services"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "All cross-domain FK constraints have been dropped (with pragmatic exceptions for OrderItem.product_id and CartItem). Cross-domain writes go through internal HTTP and activity bus. However, " (strong "25+ cross-app model imports remain") " — apps still import from each other's models/ directories. This means every app needs every other app's code on disk to start.")
(p "The goal: eliminate all cross-app model imports. Every app only imports from its own models/, from shared/, and from a new glue/ service layer."))
(~doc-section :title "Current state" :id "current"
(~docs/section :title "Current state" :id "current"
(p "Apps are partially decoupled via HTTP interfaces (fetch_data, call_action, send_internal_activity) and DTOs. The Cart microservice split (relations, likes, orders) is complete. But direct model imports persist in:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Cart") " — 9 files importing from market, events, blog")
@@ -17,7 +17,7 @@
(li (strong "Events") " — 5 files importing from blog, market, cart")
(li (strong "Market") " — 1 file importing from blog")))
(~doc-section :title "What remains" :id "remains"
(~docs/section :title "What remains" :id "remains"
(div :class "space-y-3"
(div :class "rounded border border-stone-200 p-3"
(h4 :class "font-semibold text-stone-700" "1. glue/services/pages.py")
@@ -41,7 +41,7 @@
(h4 :class "font-semibold text-stone-700" "7. Model registration + cleanup")
(p :class "text-sm text-stone-600" "register_models() in glue/setup.py, update all app.py files, delete moved service files."))))
(~doc-section :title "Docker consideration" :id "docker"
(~docs/section :title "Docker consideration" :id "docker"
(p :class "text-stone-600" "For glue services to work in Docker (single app per container), model files from other apps must be importable. Recommended: try/except at import time — glue services that can't import a model raise ImportError at call time, which only happens if called from the wrong app."))))
;; ---------------------------------------------------------------------------

4
sx/sx/plans/index.sx
View File

@@ -4,8 +4,8 @@
;; Plans index page
;; ---------------------------------------------------------------------------
(defcomp ~plans-index-content ()
(~doc-page :title "Plans"
(defcomp ~plans/index/plans-index-content ()
(~docs/page :title "Plans"
(div :class "space-y-4"
(p :class "text-lg text-stone-600 mb-4"
"Architecture roadmaps and implementation plans for SX.")

156
sx/sx/plans/isomorphic.sx
View File

@@ -2,14 +2,14 @@
;; Isomorphic Architecture Roadmap
;; ---------------------------------------------------------------------------
(defcomp ~plan-isomorphic-content ()
(~doc-page :title "Isomorphic Architecture Roadmap"
(defcomp ~plans/isomorphic/plan-isomorphic-content ()
(~docs/page :title "Isomorphic Architecture Roadmap"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "SX has a working server-client pipeline: server evaluates pages with IO (DB, fragments), serializes as SX wire format, client parses and renders to DOM. The language and primitives are already isomorphic " (em "— same spec, same semantics, both sides.") " What's missing is the " (strong "plumbing") " that makes the boundary between server and client a sliding window rather than a fixed wall.")
(p "The key insight: " (strong "s-expressions can partially unfold on the server after IO, then finish unfolding on the client.") " The system knows which components have data fetches (via IO detection in " (a :href "/sx/(language.(spec.deps))" :class "text-violet-700 underline" "deps.sx") "), resolves those server-side, and sends the rest as pure SX for client rendering. The boundary slides automatically based on what each component actually needs."))
(~doc-section :title "Current State" :id "current-state"
(~docs/section :title "Current State" :id "current-state"
(ul :class "space-y-2 text-stone-700 list-disc pl-5"
(li (strong "Primitive parity: ") "100%. ~80 pure primitives, same names/semantics, JS and Python.")
(li (strong "eval/parse/render: ") "Complete both sides. sx-ref.js has eval, parse, render-to-html, render-to-dom, aser.")
@@ -21,13 +21,13 @@
(li (strong "IO detection: ") "deps.sx classifies every component as pure or IO-dependent. Server expands IO components, serializes pure ones for client.")
(li (strong "Client-side routing: ") "router.sx matches URL patterns. Pure pages render instantly without server roundtrips. Pages with :data fall through to server transparently.")
(li (strong "Client IO proxy: ") "IO primitives registered on the client call back to the server via fetch. Components with IO deps can render client-side.")
(li (strong "Streaming/suspense: ") "defpage :stream true enables chunked HTML. ~suspense placeholders show loading skeletons; __sxResolve() fills in content as IO completes.")))
(li (strong "Streaming/suspense: ") "defpage :stream true enables chunked HTML. ~shared:pages/suspense placeholders show loading skeletons; __sxResolve() fills in content as IO completes.")))
;; -----------------------------------------------------------------------
;; Phase 1
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 1: Component Distribution & Dependency Analysis" :id "phase-1"
(~docs/section :title "Phase 1: Component Distribution & Dependency Analysis" :id "phase-1"
(div :class "rounded border border-green-300 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -37,10 +37,10 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "Per-page component bundles instead of sending every definition to every page. Smaller payloads, faster boot, better cache hit rates."))
(~doc-subsection :title "The Problem"
(~docs/subsection :title "The Problem"
(p "The page boot payload serializes every component definition in the environment. A page that uses 5 components still receives all 50+. No mechanism determines which components a page actually needs — the boundary between \"loaded\" and \"used\" is invisible."))
(~doc-subsection :title "Implementation"
(~docs/subsection :title "Implementation"
(p "The dependency analysis algorithm is defined in "
(a :href "/sx/(language.(spec.deps))" :class "text-violet-700 underline" "deps.sx")
@@ -50,7 +50,7 @@
(div
(h4 :class "font-semibold text-stone-700" "1. Transitive closure (deps.sx)")
(p "9 functions that walk the component graph. The core:")
(~doc-code :code (highlight "(define (transitive-deps name env)\n (let ((key (if (starts-with? name \"~\") name\n (concat \"~\" name)))\n (seen (set-create)))\n (transitive-deps-walk key env seen)\n (set-remove seen key)))" "lisp"))
(~docs/code :code (highlight "(define (transitive-deps name env)\n (let ((key (if (starts-with? name \"~\") name\n (concat \"~\" name)))\n (seen (set-create)))\n (transitive-deps-walk key env seen)\n (set-remove seen key)))" "lisp"))
(p (code "scan-refs") " walks a component body AST collecting " (code "~") " symbols. "
(code "transitive-deps") " follows references recursively through the env. "
(code "compute-all-deps") " batch-computes and caches deps for every component. "
@@ -58,8 +58,8 @@
(div
(h4 :class "font-semibold text-stone-700" "2. Page scanning")
(~doc-code :code (highlight "(define (components-needed page-source env)\n (let ((direct (scan-components-from-source page-source))\n (all-needed (set-create)))\n (for-each (fn (name) ...\n (set-add! all-needed name)\n (set-union! all-needed (component-deps comp)))\n direct)\n all-needed))" "lisp"))
(p (code "scan-components-from-source") " finds " (code "(~name") " patterns in serialized SX via regex. " (code "components-needed") " combines scanning with the cached transitive closure to produce the minimal component set for a page."))
(~docs/code :code (highlight "(define (components-needed page-source env)\n (let ((direct (scan-components-from-source page-source))\n (all-needed (set-create)))\n (for-each (fn (name) ...\n (set-add! all-needed name)\n (set-union! all-needed (component-deps comp)))\n direct)\n all-needed))" "lisp"))
(p (code "scan-components-from-source") " finds " (code "(~plans/content-addressed-components/name") " patterns in serialized SX via regex. " (code "components-needed") " combines scanning with the cached transitive closure to produce the minimal component set for a page."))
(div
(h4 :class "font-semibold text-stone-700" "3. Per-page CSS scoping")
@@ -74,13 +74,13 @@
(li (code "env-components") " — enumerate all component entries in an environment")
(li (code "regex-find-all") " / " (code "scan-css-classes") " — host-native regex and CSS scanning")))))
(~doc-subsection :title "Spec module"
(~docs/subsection :title "Spec module"
(p "deps.sx is loaded as a " (strong "spec module") " — an optional extension to the core spec. The bootstrapper flag " (code "--spec-modules deps") " includes it in the generated output alongside the core evaluator, parser, and renderer. Phase 2 IO detection was added to the same module — same bootstrapping mechanism, no architecture changes needed.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/ref/deps.sx — canonical spec (14 functions, 8 platform declarations)")
(li "Bootstrapped to all host targets via --spec-modules deps")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "15 dedicated tests: scan, transitive closure, circular deps, compute-all, components-needed")
(li "Bootstrapped output verified on both host targets")
@@ -91,7 +91,7 @@
;; Phase 2
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 2: Smart Server/Client Boundary — IO Detection" :id "phase-2"
(~docs/section :title "Phase 2: Smart Server/Client Boundary — IO Detection" :id "phase-2"
(div :class "rounded border border-green-300 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -101,7 +101,7 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "Automatic IO detection and selective expansion. Server expands IO-dependent components, serializes pure ones for client. Per-component intelligence replaces global toggle."))
(~doc-subsection :title "IO Detection in the Spec"
(~docs/subsection :title "IO Detection in the Spec"
(p "Five new functions in "
(a :href "/sx/(language.(spec.deps))" :class "text-violet-700 underline" "deps.sx")
" extend the Phase 1 walker to detect IO primitive references:")
@@ -110,12 +110,12 @@
(div
(h4 :class "font-semibold text-stone-700" "1. IO scanning")
(p (code "scan-io-refs") " walks an AST node, collecting symbol names that match an IO name set. The IO set is provided by the host from boundary declarations (all three tiers: core IO, deployment IO, page helpers).")
(~doc-code :code (highlight "(define scan-io-refs\n (fn (node io-names)\n (let ((refs (list)))\n (scan-io-refs-walk node io-names refs)\n refs)))" "lisp")))
(~docs/code :code (highlight "(define scan-io-refs\n (fn (node io-names)\n (let ((refs (list)))\n (scan-io-refs-walk node io-names refs)\n refs)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. Transitive IO closure")
(p (code "transitive-io-refs") " follows component deps recursively, unioning IO refs from all reachable components and macros. Cycle-safe via seen-set.")
(~doc-code :code (highlight "(define transitive-io-refs\n (fn (name env io-names)\n ;; Walk deps, scan each body for IO refs,\n ;; union all refs transitively.\n ...))" "lisp")))
(~docs/code :code (highlight "(define transitive-io-refs\n (fn (name env io-names)\n ;; Walk deps, scan each body for IO refs,\n ;; union all refs transitively.\n ...))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "3. Batch computation")
@@ -125,7 +125,7 @@
(h4 :class "font-semibold text-stone-700" "4. Component metadata")
(p "Each component now carries " (code "io_refs") " (transitive IO primitive names) alongside " (code "deps") " and " (code "css_classes") ". The derived " (code "is_pure") " property is true when " (code "io_refs") " is empty — the component can render anywhere without server data."))))
(~doc-subsection :title "Selective Expansion"
(~docs/subsection :title "Selective Expansion"
(p "The partial evaluator " (code "_aser") " now uses per-component IO metadata instead of a global toggle:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "IO-dependent") " → expand server-side (IO must resolve)")
@@ -133,13 +133,13 @@
(li (strong "Layout slot context") " → all components still expand (backwards compat via " (code "_expand_components") " context var)"))
(p "A component calling " (code "(highlight ...)") " or " (code "(query ...)") " is IO-dependent. A component with only HTML tags and string ops is pure."))
(~doc-subsection :title "Platform interface additions"
(~docs/subsection :title "Platform interface additions"
(p "Two new platform functions each host implements:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "(component-io-refs c) → cached IO ref list")
(li "(component-set-io-refs! c refs) → cache IO refs on component")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Components calling (query ...) or (highlight ...) classified IO-dependent")
(li "Pure components (HTML-only) classified pure with empty io_refs")
@@ -151,7 +151,7 @@
;; Phase 3
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 3: Client-Side Routing (SPA Mode)" :id "phase-3"
(~docs/section :title "Phase 3: Client-Side Routing (SPA Mode)" :id "phase-3"
(div :class "rounded border border-green-300 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -161,20 +161,20 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "After initial page load, pure pages render instantly without server roundtrips. Client matches routes locally, evaluates content expressions with cached components, and only falls back to server for pages with :data dependencies."))
(~doc-subsection :title "Architecture"
(~docs/subsection :title "Architecture"
(p "Three-layer approach: spec defines pure route matching, page registry bridges server metadata to client, orchestration intercepts navigation for try-first/fallback.")
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Route matching spec (router.sx)")
(p "New spec module with pure functions for Flask-style route pattern matching:")
(~doc-code :code (highlight "(define split-path-segments ;; \"/language/docs/hello\" → (\"docs\" \"hello\")\n(define parse-route-pattern ;; \"/language/docs/<slug>\" → segment descriptors\n(define match-route-segments ;; segments + pattern → params dict or nil\n(define find-matching-route ;; path + route table → first match" "lisp"))
(~docs/code :code (highlight "(define split-path-segments ;; \"/language/docs/hello\" → (\"docs\" \"hello\")\n(define parse-route-pattern ;; \"/language/docs/<slug>\" → segment descriptors\n(define match-route-segments ;; segments + pattern → params dict or nil\n(define find-matching-route ;; path + route table → first match" "lisp"))
(p "No platform interface needed — uses only pure string and list primitives. Bootstrapped to both hosts via " (code "--spec-modules deps,router") "."))
(div
(h4 :class "font-semibold text-stone-700" "2. Page registry")
(p "Server serializes defpage metadata as SX dict literals inside " (code "<script type=\"text/sx-pages\">") ". Each entry carries name, path pattern, auth level, has-data flag, serialized content expression, and closure values.")
(~doc-code :code (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(~docs/code :code (highlight "{:name \"docs-page\" :path \"/language/docs/<slug>\"\n :auth \"public\" :has-data false\n :content \"(case slug ...)\" :closure {}}" "lisp"))
(p "boot.sx processes these at startup using the SX parser — the same " (code "parse") " function from parser.sx — building route entries with parsed patterns into the " (code "_page-routes") " table. No JSON dependency."))
(div
@@ -188,7 +188,7 @@
(li "If anything fails (no match, has data, eval error): transparent fallback to server fetch"))
(p (code "handle-popstate") " also tries client routing before server fetch on back/forward."))))
(~doc-subsection :title "What becomes client-routable"
(~docs/subsection :title "What becomes client-routable"
(p "All pages with content expressions — most of this docs app. Pure pages render instantly; :data pages fetch data then render client-side (Phase 4):")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "/") ", " (code "/language/docs/") ", " (code "/language/docs/<slug>") " (most slugs), " (code "/applications/protocols/") ", " (code "/applications/protocols/<slug>"))
@@ -202,11 +202,11 @@
(li (code "/geography/isomorphism/bundle-analyzer") " (has " (code ":data (bundle-analyzer-data)") ")")
(li (code "/geography/isomorphism/data-test") " (has " (code ":data (data-test-data)") " — " (a :href "/sx/(geography.(isomorphism.data-test))" :class "text-violet-700 underline" "Phase 4 demo") ")")))
(~doc-subsection :title "Try-first/fallback design"
(~docs/subsection :title "Try-first/fallback design"
(p "Client routing uses a try-first approach: attempt local evaluation in a try/catch, fall back to server fetch on any failure. This avoids needing perfect static analysis of content expressions — if a content expression calls a page helper the client doesn't have, the eval throws, and the server handles it transparently.")
(p "Console messages provide visibility: " (code "sx:route client /essays/why-sexps") " vs " (code "sx:route server /specs/eval") "."))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/ref/router.sx — route pattern matching spec")
(li "shared/sx/ref/boot.sx — process page registry scripts")
@@ -215,7 +215,7 @@
(li "shared/sx/ref/bootstrap_py.py — router spec module (parity)")
(li "shared/sx/helpers.py — page registry SX serialization")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Pure page navigation: zero server requests, console shows \"sx:route client\"")
(li "IO/data page fallback: falls through to server fetch transparently")
@@ -227,7 +227,7 @@
;; Phase 4
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 4: Client Async & IO Bridge" :id "phase-4"
(~docs/section :title "Phase 4: Client Async & IO Bridge" :id "phase-4"
(div :class "rounded border border-green-300 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -236,14 +236,14 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "Client fetches server-evaluated data and renders :data pages locally. Data cached with TTL to avoid redundant fetches on back/forward navigation. All IO stays server-side — no continuations needed."))
(~doc-subsection :title "Architecture"
(~docs/subsection :title "Architecture"
(p "Separates IO from rendering. Server evaluates :data expression (async, with DB/service access), serializes result as SX wire format. Client fetches pre-evaluated data, parses it, merges into env, renders pure :content client-side.")
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Abstract resolve-page-data")
(p "Spec-level primitive in orchestration.sx. The spec says \"I need data for this page\" — platform provides transport:")
(~doc-code :code (highlight "(resolve-page-data page-name params\n (fn (data)\n ;; data is a dict — merge into env and render\n (let ((env (merge closure params data))\n (rendered (try-eval-content content-src env)))\n (swap-rendered-content target rendered pathname))))" "lisp"))
(~docs/code :code (highlight "(resolve-page-data page-name params\n (fn (data)\n ;; data is a dict — merge into env and render\n (let ((env (merge closure params data))\n (rendered (try-eval-content content-src env)))\n (swap-rendered-content target rendered pathname))))" "lisp"))
(p "Browser platform: HTTP fetch to " (code "/sx/data/<page-name>") ". Future platforms could use IPC, cache, WebSocket, etc."))
(div
@@ -260,7 +260,7 @@
(li "After TTL: stale entry evicted, fresh fetch on next visit"))
(p "Try it: navigate to the " (a :href "/sx/(geography.(isomorphism.data-test))" :class "text-violet-700 underline" "data test page") ", go back, return within 30s — the server-time stays the same (cached). Wait 30s+ and return — new time (fresh fetch)."))))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/ref/orchestration.sx — resolve-page-data spec, data cache")
(li "shared/sx/ref/bootstrap_js.py — platform resolvePageData (HTTP fetch)")
@@ -269,7 +269,7 @@
(li "sx/sx/data-test.sx — test component")
(li "shared/sx/tests/test_page_data.py — 30 unit tests")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "30 unit tests: serialize roundtrip, kebab-case, deps, full pipeline simulation, cache TTL")
(li "Console: " (code "sx:route client+data") " on first visit, " (code "sx:route client+cache") " on return within 30s")
@@ -280,7 +280,7 @@
;; Phase 5
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 5: Client IO Proxy" :id "phase-5"
(~docs/section :title "Phase 5: Client IO Proxy" :id "phase-5"
(div :class "rounded border border-green-300 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -288,7 +288,7 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "Components with IO dependencies render client-side. IO primitives are proxied to the server — the client evaluator calls them like normal functions, the proxy fetches results via HTTP, the async DOM renderer awaits the promises and continues."))
(~doc-subsection :title "How it works"
(~docs/subsection :title "How it works"
(p "Instead of async-aware continuations (originally planned), Phase 5 was solved by combining three mechanisms that emerged from Phases 3-4:")
(div :class "space-y-4"
@@ -304,18 +304,18 @@
(h4 :class "font-semibold text-stone-700" "3. Async DOM renderer")
(p (code "asyncRenderToDom") " walks the expression tree and handles Promises transparently. When a subexpression returns a Promise (from an IO proxy call), the renderer awaits it and continues building the DOM tree. No continuations needed — JavaScript's native Promise mechanism provides the suspension."))))
(~doc-subsection :title "Why continuations weren't needed"
(~docs/subsection :title "Why continuations weren't needed"
(p "The original Phase 5 plan called for async-aware shift/reset or a CPS transform of the evaluator. In practice, JavaScript's Promise mechanism provided the same capability: the async DOM renderer naturally suspends when it encounters a Promise and resumes when it resolves.")
(p "Delimited continuations remain valuable for Phase 6 (streaming/suspense on the " (em "server") " side, where Python doesn't have native Promise-based suspension in the evaluator). But for client-side IO, Promises + async render were sufficient."))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/ref/orchestration.sx — registerIoDeps, IO proxy registration")
(li "shared/sx/ref/bootstrap_js.py — asyncRenderToDom, IO proxy HTTP transport")
(li "shared/sx/helpers.py — io_deps in page registry entries")
(li "shared/sx/deps.py — transitive IO ref computation")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Navigate to any page with IO deps (e.g. /testing/eval) — console shows IO proxy calls")
(li "Components using " (code "highlight") " render correctly via proxy")
@@ -326,7 +326,7 @@
;; Phase 6
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 6: Streaming & Suspense" :id "phase-6"
(~docs/section :title "Phase 6: Streaming & Suspense" :id "phase-6"
(div :class "rounded border border-green-300 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -335,9 +335,9 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "Server streams partially-evaluated SX as IO resolves. Client renders available subtrees immediately with loading skeletons, fills in suspended parts as data arrives."))
(~doc-subsection :title "What was built"
(~docs/subsection :title "What was built"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "~suspense") " component — renders fallback content with a stable DOM ID, replaced when resolution arrives")
(li (code "~shared:pages/suspense") " component — renders fallback content with a stable DOM ID, replaced when resolution arrives")
(li (code "defpage :stream true") " — opts a page into streaming response mode")
(li (code "defpage :fallback expr") " — custom loading skeleton for streaming pages")
(li (code "execute_page_streaming()") " — Quart async generator response that yields HTML chunks")
@@ -347,13 +347,13 @@
(li "Concurrent IO: data eval + header eval run in parallel via " (code "asyncio.create_task"))
(li "Completion-order streaming: whichever IO finishes first gets sent first via " (code "asyncio.wait(FIRST_COMPLETED)"))))
(~doc-subsection :title "Architecture"
(~docs/subsection :title "Architecture"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Suspense component")
(p "When streaming, the server renders the page with " (code "~suspense") " placeholders instead of awaiting IO:")
(~doc-code :code (highlight "(~app-body\n :header-rows (~suspense :id \"stream-headers\"\n :fallback (div :class \"h-12 bg-stone-200 animate-pulse\"))\n :content (~suspense :id \"stream-content\"\n :fallback (div :class \"p-8 animate-pulse\" ...)))" "lisp")))
(p "When streaming, the server renders the page with " (code "~shared:pages/suspense") " placeholders instead of awaiting IO:")
(~docs/code :code (highlight "(~app-body\n :header-rows (~shared:pages/suspense :id \"stream-headers\"\n :fallback (div :class \"h-12 bg-stone-200 animate-pulse\"))\n :content (~shared:pages/suspense :id \"stream-content\"\n :fallback (div :class \"p-8 animate-pulse\" ...)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. Chunked transfer")
@@ -366,19 +366,19 @@
(div
(h4 :class "font-semibold text-stone-700" "3. Client resolution")
(p "Each resolution chunk is an inline script:")
(~doc-code :code (highlight "<script>\n window.__sxResolve(\"stream-content\",\n \"(~article :title \\\"Hello\\\")\")\n</script>" "html"))
(~docs/code :code (highlight "<script>\n window.__sxResolve(\"stream-content\",\n \"(~article :title \\\"Hello\\\")\")\n</script>" "html"))
(p "The client parses the SX, renders to DOM, and replaces the suspense placeholder's children."))
(div
(h4 :class "font-semibold text-stone-700" "4. Concurrent IO")
(p "Data evaluation and header construction run in parallel. " (code "asyncio.wait(FIRST_COMPLETED)") " yields resolution chunks in whatever order IO completes — no artificial sequencing."))))
(~doc-subsection :title "Continuation foundation"
(~docs/subsection :title "Continuation foundation"
(p "Delimited continuations (" (code "reset") "/" (code "shift") ") are implemented in the Python evaluator (async_eval.py lines 586-624) and available as special forms. Phase 6 uses the simpler pattern of concurrent IO + completion-order streaming, but the continuation machinery is in place for Phase 7's more sophisticated evaluation-level suspension."))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/templates/pages.sx — ~suspense component definition")
(li "shared/sx/templates/pages.sx — ~shared:pages/suspense component definition")
(li "shared/sx/types.py — PageDef.stream, PageDef.fallback_expr fields")
(li "shared/sx/evaluator.py — defpage :stream/:fallback parsing")
(li "shared/sx/pages.py — execute_page_streaming(), streaming route mounting")
@@ -391,7 +391,7 @@
(li "sx/sxc/pages/docs.sx — streaming-demo defpage")
(li "sx/sxc/pages/helpers.py — streaming-demo-data page helper")))
(~doc-subsection :title "Demonstration"
(~docs/subsection :title "Demonstration"
(p "The " (a :href "/sx/(geography.(isomorphism.streaming))" :class "text-violet-700 underline" "streaming demo page") " exercises the full pipeline:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
(li "Navigate to " (a :href "/sx/(geography.(isomorphism.streaming))" :class "text-violet-700 underline" "/sx/(geography.(isomorphism.streaming))"))
@@ -400,10 +400,10 @@
(li "Open the Network tab — observe " (code "Transfer-Encoding: chunked") " on the document response")
(li "The document response shows multiple chunks arriving over time: shell first, then resolution scripts")))
(~doc-subsection :title "What to verify"
(~docs/subsection :title "What to verify"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Instant shell: ") "The page HTML arrives immediately — no waiting for the 1.5s data fetch")
(li (strong "Suspense placeholders: ") "The " (code "~suspense") " component renders a " (code "data-suspense") " wrapper with animated fallback content")
(li (strong "Suspense placeholders: ") "The " (code "~shared:pages/suspense") " component renders a " (code "data-suspense") " wrapper with animated fallback content")
(li (strong "Resolution: ") "The " (code "__sxResolve()") " inline script replaces the placeholder with real rendered content")
(li (strong "Chunked encoding: ") "Network tab shows the document as a chunked response with multiple frames")
(li (strong "Concurrent IO: ") "Header and content resolve independently — whichever finishes first appears first")
@@ -413,7 +413,7 @@
;; Phase 7
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 7: Full Isomorphism" :id "phase-7"
(~docs/section :title "Phase 7: Full Isomorphism" :id "phase-7"
(div :class "rounded border border-green-200 bg-green-50 p-4 mb-4"
(div :class "flex items-center gap-2 mb-2"
@@ -421,7 +421,7 @@
(p :class "text-green-900 font-medium" "What it enables")
(p :class "text-green-800" "Same SX code runs on either side. Runtime chooses optimal split via affinity annotations and render plans. Client data cache managed via invalidation headers and server-driven updates. Cross-host isomorphism verified by 61 automated tests."))
(~doc-subsection :title "7a. Affinity Annotations & Render Target"
(~docs/subsection :title "7a. Affinity Annotations & Render Target"
(div :class "rounded border border-green-300 bg-green-50 p-3 mb-4"
(div :class "flex items-center gap-2 mb-1"
@@ -429,7 +429,7 @@
(p :class "text-green-800 text-sm" "Components declare where they prefer to render. The spec combines affinity with IO analysis to produce a per-component render target decision."))
(p "Affinity annotations let component authors express rendering preferences:")
(~doc-code :code (highlight "(defcomp ~product-grid (&key products)\n :affinity :client ;; interactive, prefer client rendering\n (div ...))\n\n(defcomp ~auth-menu (&key user)\n :affinity :server ;; auth-sensitive, always server\n (div ...))\n\n(defcomp ~card (&key title)\n ;; no annotation = :affinity :auto (default)\n ;; runtime decides from IO analysis\n (div ...))" "lisp"))
(~docs/code :code (highlight "(defcomp ~plans/isomorphic/product-grid (&key products)\n :affinity :client ;; interactive, prefer client rendering\n (div ...))\n\n(defcomp ~plans/isomorphic/auth-menu (&key user)\n :affinity :server ;; auth-sensitive, always server\n (div ...))\n\n(defcomp ~plans/isomorphic/card (&key title)\n ;; no annotation = :affinity :auto (default)\n ;; runtime decides from IO analysis\n (div ...))" "lisp"))
(p "The " (code "render-target") " function in deps.sx combines affinity with IO analysis:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -439,7 +439,7 @@
(p "The server's partial evaluator (" (code "_aser") ") uses " (code "render_target") " instead of the previous " (code "is_pure") " check. Components with " (code ":affinity :client") " are serialized for client rendering even if they call IO primitives — the IO proxy (Phase 5) handles the calls.")
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/ref/eval.sx — defcomp annotation parsing, defcomp-kwarg helper")
(li "shared/sx/ref/deps.sx — render-target function, platform interface")
@@ -451,14 +451,14 @@
(li "shared/sx/ref/test-eval.sx — 4 new defcomp affinity tests")
(li "shared/sx/ref/test-deps.sx — 6 new render-target tests")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "269 spec tests pass (10 new: 4 eval + 6 deps)")
(li "79 Python unit tests pass")
(li "Bootstrapped to both hosts (sx_ref.py + sx-browser.js)")
(li "Backward compatible: existing defcomp without :affinity defaults to \"auto\""))))
(~doc-subsection :title "7b. Runtime Boundary Optimizer"
(~docs/subsection :title "7b. Runtime Boundary Optimizer"
(div :class "rounded border border-green-300 bg-green-50 p-3 mb-4"
(div :class "flex items-center gap-2 mb-1"
@@ -468,9 +468,9 @@
(p "Given component tree + IO dependency graph + affinity annotations, decide per-component: server-expand, client-render, or stream. Planning step cached at registration, recomputed on component change.")
(p (code "page-render-plan") " in deps.sx computes per-page boundary decisions:")
(~doc-code :code (highlight "(page-render-plan page-source env io-names)\n;; Returns:\n;; {:components {~name \"server\"|\"client\" ...}\n;; :server (list of server-expanded names)\n;; :client (list of client-rendered names)\n;; :io-deps (IO primitives needed by server components)}" "lisp"))
(~docs/code :code (highlight "(page-render-plan page-source env io-names)\n;; Returns:\n;; {:components {~plans/content-addressed-components/name \"server\"|\"client\" ...}\n;; :server (list of server-expanded names)\n;; :client (list of client-rendered names)\n;; :io-deps (IO primitives needed by server components)}" "lisp"))
(~doc-subsection :title "Integration Points"
(~docs/subsection :title "Integration Points"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "shared/sx/ref/deps.sx") " — " (code "page-render-plan") " spec function")
(li (code "shared/sx/deps.py") " — Python wrapper, dispatches to bootstrapped code")
@@ -478,13 +478,13 @@
(li (code "shared/sx/helpers.py") " — " (code "_build_pages_sx()") " includes " (code ":render-plan") " in client page registry")
(li (code "shared/sx/types.py") " — " (code "PageDef.render_plan") " field")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "5 new spec tests (page-render-plan suite)")
(li "Render plans visible on " (a :href "/sx/(geography.(isomorphism.affinity))" "affinity demo page"))
(li "Client page registry includes :render-plan for each page"))))
(~doc-subsection :title "7c. Cache Invalidation & Optimistic Data Updates"
(~docs/subsection :title "7c. Cache Invalidation & Optimistic Data Updates"
(div :class "rounded border border-green-300 bg-green-50 p-3 mb-4"
(div :class "flex items-center gap-2 mb-1"
@@ -493,15 +493,15 @@
(p "The client-side page data cache (30-second TTL) now supports cache invalidation, server-driven updates, and optimistic mutations. The client predicts the result of a mutation, immediately re-renders with the predicted data, and confirms or reverts when the server responds.")
(~doc-subsection :title "Cache Invalidation"
(~docs/subsection :title "Cache Invalidation"
(p "Component authors can declare cache invalidation on elements that trigger mutations:")
(~doc-code :code (highlight ";; Clear specific page's cache after successful action\n(form :sx-post \"/cart/remove\"\n :sx-cache-invalidate \"cart-page\"\n ...)\n\n;; Clear ALL page caches after action\n(button :sx-post \"/admin/reset\"\n :sx-cache-invalidate \"*\")" "lisp"))
(~docs/code :code (highlight ";; Clear specific page's cache after successful action\n(form :sx-post \"/cart/remove\"\n :sx-cache-invalidate \"cart-page\"\n ...)\n\n;; Clear ALL page caches after action\n(button :sx-post \"/admin/reset\"\n :sx-cache-invalidate \"*\")" "lisp"))
(p "The server can also control client cache via response headers:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "SX-Cache-Invalidate: page-name") " — clear cache for a page")
(li (code "SX-Cache-Update: page-name") " — replace cache with the response body (SX-format data)")))
(~doc-subsection :title "Optimistic Mutations"
(~docs/subsection :title "Optimistic Mutations"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "optimistic-cache-update") " — applies a mutator function to cached data, saves a snapshot for rollback")
(li (strong "optimistic-cache-revert") " — restores the pre-mutation snapshot if the server rejects")
@@ -509,19 +509,19 @@
(li (strong "submit-mutation") " — orchestration function: predict, submit, confirm/revert")
(li (strong "/sx/action/<name>") " — server endpoint for processing mutations (POST, returns SX wire format)")))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/ref/orchestration.sx — cache management + optimistic cache functions + submit-mutation spec")
(li "shared/sx/ref/engine.sx — SX-Cache-Invalidate, SX-Cache-Update response headers")
(li "shared/sx/pages.py — mount_action_endpoint for /sx/action/<name>")
(li "sx/sx/optimistic-demo.sx — live demo component")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Live demo at " (a :href "/sx/(geography.(isomorphism.optimistic))" :class "text-violet-600 hover:underline" "/sx/(geography.(isomorphism.optimistic))"))
(li "Console log: " (code "sx:optimistic confirmed") " / " (code "sx:optimistic reverted")))))
(~doc-subsection :title "7d. Offline Data Layer"
(~docs/subsection :title "7d. Offline Data Layer"
(div :class "rounded border border-green-300 bg-green-50 p-3 mb-4"
(div :class "flex items-center gap-2 mb-1"
@@ -535,7 +535,7 @@
(li (strong "/static/* ") "— stale-while-revalidate via Cache API. Serves cached assets immediately, updates in background.")
(li (strong "Offline mutations") " — " (code "offline-aware-mutation") " routes to " (code "submit-mutation") " when online, " (code "offline-queue-mutation") " when offline. " (code "offline-sync") " replays the queue on reconnect."))
(~doc-subsection :title "How It Works"
(~docs/subsection :title "How It Works"
(ol :class "list-decimal list-inside text-stone-700 space-y-2"
(li "On boot, " (code "sx-browser.js") " registers the SW at " (code "/sx-sw.js") " (root scope)")
(li "SW intercepts fetch events and routes by URL pattern")
@@ -544,20 +544,20 @@
(li "Cache invalidation propagates: element attr / response header → in-memory cache → SW message → IndexedDB")
(li "Offline mutations queue locally, replay on reconnect via " (code "offline-sync"))))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/static/scripts/sx-sw.js — Service Worker (network-first + stale-while-revalidate)")
(li "shared/sx/ref/orchestration.sx — offline queue, sync, connectivity tracking, sw-post-message")
(li "shared/sx/pages.py — mount_service_worker() serves SW at /sx-sw.js")
(li "sx/sx/offline-demo.sx — live demo component")))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Live demo at " (a :href "/sx/(geography.(isomorphism.offline))" :class "text-violet-600 hover:underline" "/sx/(geography.(isomorphism.offline))"))
(li "Test with DevTools Network → Offline mode")
(li "Console log: " (code "sx:offline queued") ", " (code "sx:offline syncing") ", " (code "sx:offline synced")))))
(~doc-subsection :title "7e. Isomorphic Testing"
(~docs/subsection :title "7e. Isomorphic Testing"
(div :class "rounded border border-green-300 bg-green-50 p-3 mb-4"
(div :class "flex items-center gap-2 mb-1"
@@ -569,12 +569,12 @@
(li "37 eval tests: arithmetic, comparison, strings, collections, logic, let/lambda, higher-order, dict, keywords, cond/case")
(li "24 render tests: elements, attributes, nesting, void elements, boolean attrs, conditionals, map, components, affinity, HTML escaping"))
(~doc-subsection :title "Files"
(~docs/subsection :title "Files"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "shared/sx/tests/test_isomorphic.py — cross-host test suite")
(li "Run: " (code "python3 -m pytest shared/sx/tests/test_isomorphic.py -q")))))
(~doc-subsection :title "7f. Universal Page Descriptor"
(~docs/subsection :title "7f. Universal Page Descriptor"
(div :class "rounded border border-green-300 bg-green-50 p-3 mb-4"
(div :class "flex items-center gap-2 mb-1"
@@ -595,9 +595,9 @@
;; Cross-Cutting Concerns
;; -----------------------------------------------------------------------
(~doc-section :title "Cross-Cutting Concerns" :id "cross-cutting"
(~docs/section :title "Cross-Cutting Concerns" :id "cross-cutting"
(~doc-subsection :title "Error Reporting"
(~docs/subsection :title "Error Reporting"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Phase 1: \"Unknown component\" includes which page expected it and what bundle was sent")
(li "Phase 2: Server logs which components expanded server-side vs sent to client")
@@ -605,7 +605,7 @@
(li "Phase 4: Client data errors include page name, params, server response status")
(li "Source location tracking in parser → propagate through eval → include in error messages")))
(~doc-subsection :title "Backward Compatibility"
(~docs/subsection :title "Backward Compatibility"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Pages without annotations behave as today")
(li "SX-Request / SX-Components / SX-Css header protocol continues")
@@ -613,14 +613,14 @@
(li "_expand_components continues as override")
(li "Each phase is opt-in: disable → identical to previous behavior")))
(~doc-subsection :title "Spec Integrity"
(~docs/subsection :title "Spec Integrity"
(p "All new behavior specified in .sx files under shared/sx/ref/ before implementation. Bootstrappers transpile from spec. This ensures JS and Python stay in sync.")))
;; -----------------------------------------------------------------------
;; Critical Files
;; -----------------------------------------------------------------------
(~doc-section :title "Critical Files" :id "critical-files"
(~docs/section :title "Critical Files" :id "critical-files"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"

66
sx/sx/plans/js-bootstrapper.sx
View File

@@ -2,14 +2,14 @@
;; js.sx — Self-Hosting JavaScript Bootstrapper
;; ---------------------------------------------------------------------------
(defcomp ~plan-js-bootstrapper-content ()
(~doc-page :title "js.sx — JavaScript Bootstrapper"
(defcomp ~plans/js-bootstrapper/plan-js-bootstrapper-content ()
(~docs/page :title "js.sx — JavaScript Bootstrapper"
;; -----------------------------------------------------------------------
;; Overview
;; -----------------------------------------------------------------------
(~doc-section :title "Overview" :id "overview"
(~docs/section :title "Overview" :id "overview"
(p (code "bootstrap_js.py") " is a 4,361-line Python program that reads the "
(code ".sx") " spec files and emits " (code "sx-ref.js") " — the entire "
"browser runtime. Parser, evaluator, three rendering adapters (HTML, SX wire, DOM), "
@@ -28,12 +28,12 @@
;; Two Modes
;; -----------------------------------------------------------------------
(~doc-section :title "Two Compilation Modes" :id "modes"
(~docs/section :title "Two Compilation Modes" :id "modes"
(~doc-subsection :title "Mode 1: Spec Bootstrapper"
(~docs/subsection :title "Mode 1: Spec Bootstrapper"
(p "Same job as " (code "bootstrap_js.py") ". Read spec " (code ".sx") " files, "
"emit " (code "sx-ref.js") ".")
(~doc-code :code (highlight ";; Translate eval.sx to JavaScript
(~docs/code :code (highlight ";; Translate eval.sx to JavaScript
(js-translate-file (parse-file \"eval.sx\"))
;; → \"function evalExpr(expr, env) { ... }\"
@@ -44,12 +44,12 @@
(p "The output is identical to " (code "python bootstrap_js.py") ". "
"Verification: " (code "diff <(python bootstrap_js.py) <(python run_js_sx.py)") "."))
(~doc-subsection :title "Mode 2: Component Compiler"
(~docs/subsection :title "Mode 2: Component Compiler"
(p "Server-side SX evaluation + " (code "js.sx") " translation = static JS output. "
"Given a component tree that the server has already evaluated (data fetched, "
"conditionals resolved, loops expanded), " (code "js.sx") " compiles the "
"resulting DOM description into a JavaScript program that builds the same DOM.")
(~doc-code :code (highlight ";; Server evaluates the page (fetches data, expands components)
(~docs/code :code (highlight ";; Server evaluates the page (fetches data, expands components)
;; Result is a resolved SX tree: (div :class \"...\" (h1 \"Hello\") ...)
;; js.sx compiles that tree to standalone JS
@@ -70,7 +70,7 @@
;; Architecture
;; -----------------------------------------------------------------------
(~doc-section :title "Architecture" :id "architecture"
(~docs/section :title "Architecture" :id "architecture"
(p "The JS bootstrapper has more moving parts than the Python one because "
"JavaScript is the " (em "client") " host. The browser runtime includes "
"things Python never needs:")
@@ -141,11 +141,11 @@
;; Translation Rules
;; -----------------------------------------------------------------------
(~doc-section :title "Translation Rules" :id "translation"
(~docs/section :title "Translation Rules" :id "translation"
(p (code "js.sx") " shares the same pattern as " (code "py.sx") " — expression translator, "
"statement translator, name mangling — but with JavaScript-specific mappings:")
(~doc-subsection :title "Name Mangling"
(~docs/subsection :title "Name Mangling"
(p "SX uses kebab-case. JavaScript uses camelCase.")
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
@@ -180,7 +180,7 @@
(td :class "px-4 py-2 font-mono" "delete_")
(td :class "px-4 py-2" "JS reserved word escape"))))))
(~doc-subsection :title "Special Forms → JavaScript"
(~docs/subsection :title "Special Forms → JavaScript"
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
(thead :class "bg-stone-50"
@@ -216,7 +216,7 @@
(td :class "px-4 py-2 font-mono" "&rest args")
(td :class "px-4 py-2 font-mono" "...args (rest params)"))))))
(~doc-subsection :title "JavaScript Advantages"
(~docs/subsection :title "JavaScript Advantages"
(p "JavaScript is easier to target than Python in two key ways:")
(ul :class "list-disc pl-6 space-y-2 text-stone-700"
(li (strong "No mutation problem. ")
@@ -232,7 +232,7 @@
;; Component Compilation
;; -----------------------------------------------------------------------
(~doc-section :title "Component Compilation" :id "component-compiler"
(~docs/section :title "Component Compilation" :id "component-compiler"
(p "Mode 2 is the interesting one. The server already evaluates SX page "
"definitions — it fetches data, resolves conditionals, expands components, "
"and produces a complete DOM description as an SX tree. Currently this tree "
@@ -245,22 +245,22 @@
"to evaluate. It's just a description of DOM nodes. " (code "js.sx")
" walks this tree and emits imperative JavaScript that constructs the same DOM.")
(~doc-subsection :title "What Gets Compiled"
(~docs/subsection :title "What Gets Compiled"
(p "A resolved SX tree like:")
(~doc-code :code (highlight "(div :class \"container\"
(~docs/code :code (highlight "(div :class \"container\"
(h1 \"Hello\")
(ul (map (fn (item)
(li :class \"item\" (get item \"name\")))
items)))" "lisp"))
(p "After server-side evaluation (with " (code "items") " = "
(code "[{\"name\": \"Alice\"}, {\"name\": \"Bob\"}]") "):")
(~doc-code :code (highlight "(div :class \"container\"
(~docs/code :code (highlight "(div :class \"container\"
(h1 \"Hello\")
(ul
(li :class \"item\" \"Alice\")
(li :class \"item\" \"Bob\")))" "lisp"))
(p "Compiles to:")
(~doc-code :code (highlight "var _0 = document.createElement('div');
(~docs/code :code (highlight "var _0 = document.createElement('div');
_0.className = 'container';
var _1 = document.createElement('h1');
_1.textContent = 'Hello';
@@ -276,7 +276,7 @@ _4.textContent = 'Bob';
_2.appendChild(_4);
_0.appendChild(_2);" "javascript")))
(~doc-subsection :title "Why Not Just Use HTML?"
(~docs/subsection :title "Why Not Just Use HTML?"
(p "HTML already does this — " (code "innerHTML") " parses and builds DOM. "
"Why compile to JS instead?")
(ul :class "list-disc pl-6 space-y-2 text-stone-700"
@@ -301,7 +301,7 @@ _0.appendChild(_2);" "javascript")))
"browser, Node, Deno, Bun. Server-side rendered pages become "
"testable JavaScript programs.")))
(~doc-subsection :title "Hybrid Mode"
(~docs/subsection :title "Hybrid Mode"
(p "Not every page is fully static. Some parts are server-rendered, "
"some are interactive. " (code "js.sx") " handles this with a hybrid approach:")
(ul :class "list-disc pl-6 space-y-2 text-stone-700"
@@ -320,7 +320,7 @@ _0.appendChild(_2);" "javascript")))
;; The Bootstrap Chain
;; -----------------------------------------------------------------------
(~doc-section :title "The Bootstrap Chain" :id "chain"
(~docs/section :title "The Bootstrap Chain" :id "chain"
(p "With both " (code "py.sx") " and " (code "js.sx") ", the full picture:")
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
@@ -370,9 +370,9 @@ _0.appendChild(_2);" "javascript")))
;; Implementation Plan
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(~doc-subsection :title "Phase 1: Expression Translator"
(~docs/subsection :title "Phase 1: Expression Translator"
(p "Core SX-to-JavaScript expression translation.")
(ul :class "list-disc pl-6 space-y-1 text-stone-700"
(li (code "js-mangle") " — SX name → JavaScript identifier (RENAMES + kebab→camelCase)")
@@ -386,7 +386,7 @@ _0.appendChild(_2);" "javascript")))
(code "===") ", " (code "!==") ", " (code "%"))
(li (code "&rest") " → " (code "...args") " (rest parameters)")))
(~doc-subsection :title "Phase 2: Statement Translator"
(~docs/subsection :title "Phase 2: Statement Translator"
(p "Top-level and function body statement emission.")
(ul :class "list-disc pl-6 space-y-1 text-stone-700"
(li (code "js-statement") " — emit as JavaScript statement")
@@ -396,7 +396,7 @@ _0.appendChild(_2);" "javascript")))
(li (code "do") "/" (code "begin") " → comma expression or block")
(li "Function bodies with multiple expressions → explicit " (code "return"))))
(~doc-subsection :title "Phase 3: Spec Bootstrapper"
(~docs/subsection :title "Phase 3: Spec Bootstrapper"
(p "Process spec files identically to " (code "bootstrap_js.py") ".")
(ul :class "list-disc pl-6 space-y-1 text-stone-700"
(li (code "js-extract-defines") " — parse .sx source, collect top-level defines")
@@ -405,7 +405,7 @@ _0.appendChild(_2);" "javascript")))
(li "Dependency resolution: engine requires dom, boot requires engine + parser")
(li "Static sections (IIFE wrapper, platform interface) stay as string templates")))
(~doc-subsection :title "Phase 4: Component Compiler"
(~docs/subsection :title "Phase 4: Component Compiler"
(p "Ahead-of-time compilation of evaluated SX trees to JavaScript.")
(ul :class "list-disc pl-6 space-y-1 text-stone-700"
(li (code "js-compile-element") " — emit " (code "createElement") " + attribute setting")
@@ -415,8 +415,8 @@ _0.appendChild(_2);" "javascript")))
(li (code "js-compile-fragment") " — emit " (code "DocumentFragment") " construction")
(li "Runtime slicing: analyze tree → include only necessary runtime modules")))
(~doc-subsection :title "Phase 5: Verification"
(~doc-code :code (highlight "# Mode 1: spec bootstrapper parity
(~docs/subsection :title "Phase 5: Verification"
(~docs/code :code (highlight "# Mode 1: spec bootstrapper parity
python bootstrap_js.py > sx-ref-g0.js
python run_js_sx.py > sx-ref-g1.js
diff sx-ref-g0.js sx-ref-g1.js # must be empty
@@ -430,7 +430,7 @@ python test_js_compile.py # renders both, diffs DOM" "bash")))
;; Comparison with py.sx
;; -----------------------------------------------------------------------
(~doc-section :title "Comparison with py.sx" :id "comparison"
(~docs/section :title "Comparison with py.sx" :id "comparison"
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
(thead :class "bg-stone-50"
@@ -472,8 +472,8 @@ python test_js_compile.py # renders both, diffs DOM" "bash")))
;; Implications
;; -----------------------------------------------------------------------
(~doc-section :title "Implications" :id "implications"
(~doc-subsection :title "Zero-Runtime Static Sites"
(~docs/section :title "Implications" :id "implications"
(~docs/subsection :title "Zero-Runtime Static Sites"
(p "A static page written in SX compiles to a JavaScript program with "
"no SX runtime dependency. The output is just DOM API calls — "
(code "createElement") ", " (code "appendChild") ", " (code "textContent")
@@ -484,7 +484,7 @@ python test_js_compile.py # renders both, diffs DOM" "bash")))
"The server returns a CID. The browser fetches and executes pre-compiled JavaScript. "
"No parser, no evaluator, no network round-trip for component definitions."))
(~doc-subsection :title "Progressive Enhancement Layers"
(~docs/subsection :title "Progressive Enhancement Layers"
(p "The component compiler naturally supports progressive enhancement:")
(ol :class "list-decimal pl-6 space-y-1 text-stone-700"
(li (strong "HTML") " — server renders to HTML string. No JS needed. Works everywhere.")
@@ -498,7 +498,7 @@ python test_js_compile.py # renders both, diffs DOM" "bash")))
"A single-page app needs layer 3. A real-time dashboard needs layer 4. "
(code "js.sx") " makes layer 2 possible — it didn't exist before."))
(~doc-subsection :title "The Bootstrap Completion"
(~docs/subsection :title "The Bootstrap Completion"
(p "With " (code "py.sx") " and " (code "js.sx") " both written in SX:")
(ul :class "list-disc pl-6 space-y-2 text-stone-700"
(li "The " (em "spec") " defines SX semantics (" (code "eval.sx") ", " (code "render.sx") ", ...)")

40
sx/sx/plans/live-streaming.sx
View File

@@ -2,12 +2,12 @@
;; Live Streaming — SSE & WebSocket
;; ---------------------------------------------------------------------------
(defcomp ~plan-live-streaming-content ()
(~doc-page :title "Live Streaming"
(defcomp ~plans/live-streaming/plan-live-streaming-content ()
(~docs/page :title "Live Streaming"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "SX streaming currently uses chunked transfer encoding: the server sends an HTML shell with "
(code "~suspense") " placeholders, then resolves each one via inline "
(code "~shared:pages/suspense") " placeholders, then resolves each one via inline "
(code "<script>__sxResolve(id, sx)</script>") " chunks as IO completes. "
"Once the response finishes, the connection closes. Each slot resolves exactly once.")
(p "This is powerful for initial page load but doesn't support live updates "
@@ -16,9 +16,9 @@
(p "The key insight: the client already has " (code "Sx.resolveSuspense(id, sxSource)") " which replaces "
"DOM content by suspense ID. A persistent connection just needs to keep calling it."))
(~doc-section :title "Design" :id "design"
(~docs/section :title "Design" :id "design"
(~doc-subsection :title "Transport Hierarchy"
(~docs/subsection :title "Transport Hierarchy"
(p "Three tiers, progressively more capable:")
(ol :class "list-decimal list-inside space-y-2 text-stone-700 text-sm"
(li (strong "Chunked streaming") " (done) — single HTTP response, each suspense resolves once. "
@@ -28,21 +28,21 @@
(li (strong "WebSocket") " — bidirectional, client can send events back. "
"Best for: chat, collaborative editing, interactive applications.")))
(~doc-subsection :title "SSE Protocol"
(~docs/subsection :title "SSE Protocol"
(p "A " (code "~live") " component declares a persistent connection to an SSE endpoint:")
(~doc-code :code (highlight "(~live :src \"/api/stream/dashboard\"\n (~suspense :id \"cpu\" :fallback (span \"Loading...\"))\n (~suspense :id \"memory\" :fallback (span \"Loading...\"))\n (~suspense :id \"requests\" :fallback (span \"Loading...\")))" "lisp"))
(~docs/code :code (highlight "(~live :src \"/api/stream/dashboard\"\n (~shared:pages/suspense :id \"cpu\" :fallback (span \"Loading...\"))\n (~shared:pages/suspense :id \"memory\" :fallback (span \"Loading...\"))\n (~shared:pages/suspense :id \"requests\" :fallback (span \"Loading...\")))" "lisp"))
(p "The server SSE endpoint yields SX resolve events:")
(~doc-code :code (highlight "async def dashboard_stream():\n while True:\n stats = await get_system_stats()\n yield sx_sse_event(\"cpu\", f'(~stat-badge :value \"{stats.cpu}%\")')\n yield sx_sse_event(\"memory\", f'(~stat-badge :value \"{stats.mem}%\")')\n await asyncio.sleep(1)" "python"))
(~docs/code :code (highlight "async def dashboard_stream():\n while True:\n stats = await get_system_stats()\n yield sx_sse_event(\"cpu\", f'(~stat-badge :value \"{stats.cpu}%\")')\n yield sx_sse_event(\"memory\", f'(~stat-badge :value \"{stats.mem}%\")')\n await asyncio.sleep(1)" "python"))
(p "SSE wire format — each event is a suspense resolve:")
(~doc-code :code (highlight "event: sx-resolve\ndata: {\"id\": \"cpu\", \"sx\": \"(~stat-badge :value \\\"42%\\\")\"}\n\nevent: sx-resolve\ndata: {\"id\": \"memory\", \"sx\": \"(~stat-badge :value \\\"68%\\\")\"}" "text")))
(~docs/code :code (highlight "event: sx-resolve\ndata: {\"id\": \"cpu\", \"sx\": \"(~stat-badge :value \\\"42%\\\")\"}\n\nevent: sx-resolve\ndata: {\"id\": \"memory\", \"sx\": \"(~stat-badge :value \\\"68%\\\")\"}" "text")))
(~doc-subsection :title "WebSocket Protocol"
(~docs/subsection :title "WebSocket Protocol"
(p "A " (code "~ws") " component establishes a bidirectional channel:")
(~doc-code :code (highlight "(~ws :src \"/ws/chat\"\n :on-message handle-chat-message\n (~suspense :id \"messages\" :fallback (div \"Connecting...\"))\n (~suspense :id \"typing\" :fallback (span)))" "lisp"))
(~docs/code :code (highlight "(~ws :src \"/ws/chat\"\n :on-message handle-chat-message\n (~shared:pages/suspense :id \"messages\" :fallback (div \"Connecting...\"))\n (~shared:pages/suspense :id \"typing\" :fallback (span)))" "lisp"))
(p "Client can send SX expressions back:")
(~doc-code :code (highlight ";; Client sends:\n(sx-send ws-conn '(chat-message :text \"hello\" :user \"alice\"))\n\n;; Server receives, broadcasts to all connected clients:\n;; event: sx-resolve for \"messages\" suspense" "lisp")))
(~docs/code :code (highlight ";; Client sends:\n(sx-send ws-conn '(chat-message :text \"hello\" :user \"alice\"))\n\n;; Server receives, broadcasts to all connected clients:\n;; event: sx-resolve for \"messages\" suspense" "lisp")))
(~doc-subsection :title "Shared Resolution Mechanism"
(~docs/subsection :title "Shared Resolution Mechanism"
(p "All three transports use the same client-side resolution:")
(ul :class "list-disc list-inside space-y-1 text-stone-600 text-sm"
(li (code "Sx.resolveSuspense(id, sxSource)") " — already exists, parses SX and renders to DOM")
@@ -51,9 +51,9 @@
(li "The component env (defs needed for rendering) can be sent once on connection open")
(li "Subsequent events only need the SX expression — lightweight wire format"))))
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(~doc-subsection :title "Phase 1: SSE Infrastructure"
(~docs/subsection :title "Phase 1: SSE Infrastructure"
(ol :class "list-decimal list-inside space-y-2 text-stone-700 text-sm"
(li "Add " (code "~live") " component to " (code "shared/sx/templates/") " — renders child suspense placeholders, "
"emits " (code "data-sx-live") " attribute with SSE endpoint URL")
@@ -62,28 +62,28 @@
(li "Add " (code "sx_sse_event(id, sx)") " helper for Python SSE endpoints — formats SSE wire protocol")
(li "Add " (code "sse_stream()") " Quart helper — returns async generator Response with correct headers")))
(~doc-subsection :title "Phase 2: Defpage Integration"
(~docs/subsection :title "Phase 2: Defpage Integration"
(ol :class "list-decimal list-inside space-y-2 text-stone-700 text-sm"
(li "New " (code ":live") " defpage slot — declares SSE endpoint + suspense bindings")
(li "Auto-mount SSE endpoint alongside the page route")
(li "Component defs sent as first SSE event on connection open")
(li "Automatic reconnection with exponential backoff")))
(~doc-subsection :title "Phase 3: WebSocket"
(~docs/subsection :title "Phase 3: WebSocket"
(ol :class "list-decimal list-inside space-y-2 text-stone-700 text-sm"
(li "Add " (code "~ws") " component — bidirectional channel with send/receive")
(li "Add " (code "sx-ws.js") " client module — WebSocket management, message routing")
(li "Server-side: Quart WebSocket handlers that receive and broadcast SX events")
(li "Client-side: " (code "sx-send") " primitive for sending SX expressions to server")))
(~doc-subsection :title "Phase 4: Spec & Boundary"
(~docs/subsection :title "Phase 4: Spec & Boundary"
(ol :class "list-decimal list-inside space-y-2 text-stone-700 text-sm"
(li "Spec " (code "~live") " and " (code "~ws") " in " (code "render.sx") " (how they render in each mode)")
(li "Add SSE/WS IO primitives to " (code "boundary.sx"))
(li "Bootstrap SSE/WS connection management into " (code "sx-ref.js"))
(li "Spec-level tests for resolve, reconnection, and message routing"))))
(~doc-section :title "Files" :id "files"
(~docs/section :title "Files" :id "files"
(table :class "w-full text-left border-collapse"
(thead
(tr :class "border-b border-stone-200"

94
sx/sx/plans/nav-redesign.sx
View File

@@ -2,10 +2,10 @@
;; Navigation Redesign — SX Docs
;; ---------------------------------------------------------------------------
(defcomp ~plan-nav-redesign-content ()
(~doc-page :title "Navigation Redesign"
(defcomp ~plans/nav-redesign/plan-nav-redesign-content ()
(~docs/page :title "Navigation Redesign"
(~doc-section :title "The Problem" :id "problem"
(~docs/section :title "The Problem" :id "problem"
(p "The current navigation is a horizontal menu bar system: root bar, sx bar, sub-section bar. 13 top-level sections crammed into a scrolling horizontal row. Hover to see dropdowns. Click a section, get a second bar underneath. Click a page, get a third bar. Three stacked bars eating vertical space on every page.")
(p "It's a conventional web pattern and it's bad for this site. SX docs has a deep hierarchy — sections contain subsections contain pages. Horizontal bars can't express depth. They flatten everything into one level and hide the rest behind hover states that don't work on mobile, that obscure content, that require spatial memory of where things are.")
(p "The new nav is vertical, hierarchical, and infinite. No dropdowns. No menu bars. Just a centered breadcrumb trail that expands downward as you drill in."))
@@ -14,14 +14,14 @@
;; Design
;; -----------------------------------------------------------------------
(~doc-section :title "Design" :id "design"
(~docs/section :title "Design" :id "design"
(~doc-subsection :title "Structure"
(~docs/subsection :title "Structure"
(p "One vertical column, centered. Each level is a row.")
(~doc-code :code (highlight ";; Home (nothing selected)\n;;\n;; [ sx ]\n;;\n;; Docs CSSX Reference Protocols Examples\n;; Essays Philosophy Specs Bootstrappers\n;; Testing Isomorphism Plans Reactive Islands\n\n\n;; Section selected (e.g. Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; Status Reader Macros Theorem Prover\n;; Self-Hosting JS Bootstrapper SX-Activity\n;; Predictive Prefetching Content-Addressed\n;; Environment Images Runtime Slicing Typed SX\n;; Fragment Protocol ...\n\n\n;; Page selected (e.g. Typed SX under Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; < Typed SX >\n;;\n;; [ page content here ]" "lisp")))
(~docs/code :code (highlight ";; Home (nothing selected)\n;;\n;; [ sx ]\n;;\n;; Docs CSSX Reference Protocols Examples\n;; Essays Philosophy Specs Bootstrappers\n;; Testing Isomorphism Plans Reactive Islands\n\n\n;; Section selected (e.g. Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; Status Reader Macros Theorem Prover\n;; Self-Hosting JS Bootstrapper SX-Activity\n;; Predictive Prefetching Content-Addressed\n;; Environment Images Runtime Slicing Typed SX\n;; Fragment Protocol ...\n\n\n;; Page selected (e.g. Typed SX under Plans)\n;;\n;; [ sx ]\n;;\n;; < Plans >\n;;\n;; < Typed SX >\n;;\n;; [ page content here ]" "lisp")))
(~doc-subsection :title "Rules"
(~docs/subsection :title "Rules"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li (strong "Logo at top, centered.") " Always visible. Click = home. The only fixed element.")
(li (strong "Level 1: section list.") " Shown on home page as a wrapped, centered list of links. This is the full menu — no hiding, no hamburger.")
@@ -36,8 +36,8 @@
;; Visual language
;; -----------------------------------------------------------------------
(~doc-section :title "Visual Language" :id "visual"
(~doc-subsection :title "Levels"
(~docs/section :title "Visual Language" :id "visual"
(~docs/subsection :title "Levels"
(p "Each level has decreasing visual weight:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
@@ -63,11 +63,11 @@
(td :class "px-3 py-2 text-stone-700" "Same as subsection")
(td :class "px-3 py-2 text-stone-600" "Same as subsection"))))))
(~doc-subsection :title "Arrows"
(~docs/subsection :title "Arrows"
(p "Left and right arrows are inline with the selected item name. They navigate to the previous/next sibling in the current list. Keyboard accessible: left/right arrow keys when the row is focused.")
(~doc-code :code (highlight ";; Arrow rendering\n;;\n;; < Plans >\n;;\n;; < is a link to /plans/content-addressed-components\n;; (the previous sibling in plans-nav-items)\n;; > is a link to /plans/fragment-protocol\n;; (the next sibling)\n;; \"Plans\" is a link to /plans/ (the section index)\n;;\n;; At the edges, the arrow wraps:\n;; first item: < wraps to last\n;; last item: > wraps to first" "lisp")))
(~docs/code :code (highlight ";; Arrow rendering\n;;\n;; < Plans >\n;;\n;; < is a link to /plans/content-addressed-components\n;; (the previous sibling in plans-nav-items)\n;; > is a link to /plans/fragment-protocol\n;; (the next sibling)\n;; \"Plans\" is a link to /plans/ (the section index)\n;;\n;; At the edges, the arrow wraps:\n;; first item: < wraps to last\n;; last item: > wraps to first" "lisp")))
(~doc-subsection :title "Transitions"
(~docs/subsection :title "Transitions"
(p "Selecting an item: the list fades/collapses, the selected item moves to breadcrumb position, children appear below. This is an L0 morph — the server renders the new state, the client morphs. No JS animation library needed, just CSS transitions on the morph targets.")
(p "Going up: click an ancestor in the breadcrumb. Its children (the level below) expand back into a list. Reverse of the drill-down.")))
@@ -75,14 +75,14 @@
;; Data model
;; -----------------------------------------------------------------------
(~doc-section :title "Data Model" :id "data"
(~docs/section :title "Data Model" :id "data"
(p "The current nav data is flat — each section has its own " (code "define") ". The new model is a single tree:")
(~doc-code :code (highlight "(define sx-nav-tree\n {:label \"sx\"\n :href \"/\"\n :children (list\n {:label \"Docs\"\n :href \"/language/docs/introduction\"\n :children docs-nav-items}\n {:label \"CSSX\"\n :href \"/applications/cssx/\"\n :children cssx-nav-items}\n {:label \"Reference\"\n :href \"/reference/\"\n :children reference-nav-items}\n {:label \"Protocols\"\n :href \"/applications/protocols/wire-format\"\n :children protocols-nav-items}\n {:label \"Examples\"\n :href \"/examples/click-to-load\"\n :children examples-nav-items}\n {:label \"Essays\"\n :href \"/etc/essays/\"\n :children essays-nav-items}\n {:label \"Philosophy\"\n :href \"/etc/philosophy/sx-manifesto\"\n :children philosophy-nav-items}\n {:label \"Specs\"\n :href \"/language/specs/\"\n :children specs-nav-items}\n {:label \"Bootstrappers\"\n :href \"/language/bootstrappers/\"\n :children bootstrappers-nav-items}\n {:label \"Testing\"\n :href \"/language/testing/\"\n :children testing-nav-items}\n {:label \"Isomorphism\"\n :href \"/geography/isomorphism/\"\n :children isomorphism-nav-items}\n {:label \"Plans\"\n :href \"/etc/plans/\"\n :children plans-nav-items}\n {:label \"Reactive Islands\"\n :href \"/reactive-islands/\"\n :children reactive-islands-nav-items})})" "lisp"))
(~docs/code :code (highlight "(define sx-nav-tree\n {:label \"sx\"\n :href \"/\"\n :children (list\n {:label \"Docs\"\n :href \"/language/docs/introduction\"\n :children docs-nav-items}\n {:label \"CSSX\"\n :href \"/applications/cssx/\"\n :children cssx-nav-items}\n {:label \"Reference\"\n :href \"/reference/\"\n :children reference-nav-items}\n {:label \"Protocols\"\n :href \"/applications/protocols/wire-format\"\n :children protocols-nav-items}\n {:label \"Examples\"\n :href \"/examples/click-to-load\"\n :children examples-nav-items}\n {:label \"Essays\"\n :href \"/etc/essays/\"\n :children essays-nav-items}\n {:label \"Philosophy\"\n :href \"/etc/philosophy/sx-manifesto\"\n :children philosophy-nav-items}\n {:label \"Specs\"\n :href \"/language/specs/\"\n :children specs-nav-items}\n {:label \"Bootstrappers\"\n :href \"/language/bootstrappers/\"\n :children bootstrappers-nav-items}\n {:label \"Testing\"\n :href \"/language/testing/\"\n :children testing-nav-items}\n {:label \"Isomorphism\"\n :href \"/geography/isomorphism/\"\n :children isomorphism-nav-items}\n {:label \"Plans\"\n :href \"/etc/plans/\"\n :children plans-nav-items}\n {:label \"Reactive Islands\"\n :href \"/reactive-islands/\"\n :children reactive-islands-nav-items})})" "lisp"))
(p "The existing per-section lists (" (code "docs-nav-items") ", " (code "plans-nav-items") ", etc.) remain unchanged — they just become the " (code ":children") " of tree nodes. Sub-sections that have their own sub-items can nest further:")
(~doc-code :code (highlight ";; Future: deeper nesting\n{:label \"Plans\"\n :href \"/etc/plans/\"\n :children (list\n {:label \"Status\" :href \"/etc/plans/status\"}\n {:label \"Bootstrappers\" :href \"/etc/plans/self-hosting-bootstrapper\"\n :children (list\n {:label \"py.sx\" :href \"/etc/plans/self-hosting-bootstrapper\"}\n {:label \"js.sx\" :href \"/etc/plans/js-bootstrapper\"})}\n ;; ...\n )}" "lisp"))
(~docs/code :code (highlight ";; Future: deeper nesting\n{:label \"Plans\"\n :href \"/etc/plans/\"\n :children (list\n {:label \"Status\" :href \"/etc/plans/status\"}\n {:label \"Bootstrappers\" :href \"/etc/plans/self-hosting-bootstrapper\"\n :children (list\n {:label \"py.sx\" :href \"/etc/plans/self-hosting-bootstrapper\"}\n {:label \"js.sx\" :href \"/etc/plans/js-bootstrapper\"})}\n ;; ...\n )}" "lisp"))
(p "The tree depth is unlimited. The nav component recurses."))
@@ -90,35 +90,35 @@
;; Components
;; -----------------------------------------------------------------------
(~doc-section :title "Components" :id "components"
(~docs/section :title "Components" :id "components"
(p "Three new components replace the entire menu bar system:")
(~doc-subsection :title "~sx-logo"
(~doc-code :code (highlight "(defcomp ~sx-logo ()\n (a :href \"/\"\n :sx-get \"/\" :sx-target \"#main-panel\" :sx-select \"#main-panel\"\n :sx-swap \"outerHTML\" :sx-push-url \"true\"\n :class \"block text-center py-4\"\n (span :class \"text-2xl font-bold text-violet-700\" \"sx\")))" "lisp"))
(~docs/subsection :title "~plans/nav-redesign/logo"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/logo ()\n (a :href \"/\"\n :sx-get \"/\" :sx-target \"#main-panel\" :sx-select \"#main-panel\"\n :sx-swap \"outerHTML\" :sx-push-url \"true\"\n :class \"block text-center py-4\"\n (span :class \"text-2xl font-bold text-violet-700\" \"sx\")))" "lisp"))
(p "Always at the top. Always centered. The anchor."))
(~doc-subsection :title "~nav-breadcrumb"
(~doc-code :code (highlight "(defcomp ~nav-breadcrumb (&key path siblings level)\n ;; Renders one breadcrumb row: < Label >\n ;; path = the nav tree node for this level\n ;; siblings = list of sibling nodes (for arrow nav)\n ;; level = depth (controls text size/color)\n (let ((idx (find-index siblings path))\n (prev (nth siblings (mod (- idx 1) (len siblings))))\n (next (nth siblings (mod (+ idx 1) (len siblings)))))\n (div :class (str \"flex items-center justify-center gap-3 py-1\"\n (nav-level-classes level))\n (a :href (get prev \"href\")\n :sx-get (get prev \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Previous\"\n \"<\")\n (a :href (get path \"href\")\n :sx-get (get path \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"font-medium\"\n (get path \"label\"))\n (a :href (get next \"href\")\n :sx-get (get next \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Next\"\n \">\"))))" "lisp"))
(~docs/subsection :title "~plans/nav-redesign/nav-breadcrumb"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/nav-breadcrumb (&key path siblings level)\n ;; Renders one breadcrumb row: < Label >\n ;; path = the nav tree node for this level\n ;; siblings = list of sibling nodes (for arrow nav)\n ;; level = depth (controls text size/color)\n (let ((idx (find-index siblings path))\n (prev (nth siblings (mod (- idx 1) (len siblings))))\n (next (nth siblings (mod (+ idx 1) (len siblings)))))\n (div :class (str \"flex items-center justify-center gap-3 py-1\"\n (nav-level-classes level))\n (a :href (get prev \"href\")\n :sx-get (get prev \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Previous\"\n \"<\")\n (a :href (get path \"href\")\n :sx-get (get path \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"font-medium\"\n (get path \"label\"))\n (a :href (get next \"href\")\n :sx-get (get next \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"text-stone-400 hover:text-violet-600\"\n :aria-label \"Next\"\n \">\"))))" "lisp"))
(p "One row per selected level. Shows the current node with left/right arrows to siblings."))
(~doc-subsection :title "~nav-list"
(~doc-code :code (highlight "(defcomp ~nav-list (&key items level)\n ;; Renders a wrapped list of links — the children of the current level\n (div :class (str \"flex flex-wrap justify-center gap-x-4 gap-y-2 py-2\"\n (nav-level-classes level))\n (map (fn (item)\n (a :href (get item \"href\")\n :sx-get (get item \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"hover:text-violet-700 transition-colors\"\n (get item \"label\")))\n items)))" "lisp"))
(~docs/subsection :title "~plans/nav-redesign/nav-list"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/nav-list (&key items level)\n ;; Renders a wrapped list of links — the children of the current level\n (div :class (str \"flex flex-wrap justify-center gap-x-4 gap-y-2 py-2\"\n (nav-level-classes level))\n (map (fn (item)\n (a :href (get item \"href\")\n :sx-get (get item \"href\") :sx-target \"#main-panel\"\n :sx-select \"#main-panel\" :sx-swap \"outerHTML\"\n :sx-push-url \"true\"\n :class \"hover:text-violet-700 transition-colors\"\n (get item \"label\")))\n items)))" "lisp"))
(p "The children of the current level, rendered as a centered wrapped list of plain links."))
(~doc-subsection :title "~sx-nav — the composition"
(~doc-code :code (highlight "(defcomp ~sx-nav (&key trail children-items level)\n ;; trail = list of {node, siblings} from root to current\n ;; children-items = children of the deepest selected node\n ;; level = depth of children\n (div :class \"max-w-3xl mx-auto px-4\"\n ;; Logo\n (~sx-logo)\n ;; Breadcrumb trail (one row per selected ancestor)\n (map-indexed (fn (i crumb)\n (~nav-breadcrumb\n :path (get crumb \"node\")\n :siblings (get crumb \"siblings\")\n :level (+ i 1)))\n trail)\n ;; Children of the deepest selected node\n (when children-items\n (~nav-list :items children-items :level level))))" "lisp"))
(~docs/subsection :title "~plans/nav-redesign/nav — the composition"
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/nav (&key trail children-items level)\n ;; trail = list of {node, siblings} from root to current\n ;; children-items = children of the deepest selected node\n ;; level = depth of children\n (div :class \"max-w-3xl mx-auto px-4\"\n ;; Logo\n (~plans/nav-redesign/logo)\n ;; Breadcrumb trail (one row per selected ancestor)\n (map-indexed (fn (i crumb)\n (~nav-breadcrumb\n :path (get crumb \"node\")\n :siblings (get crumb \"siblings\")\n :level (+ i 1)))\n trail)\n ;; Children of the deepest selected node\n (when children-items\n (~plans/nav-redesign/nav-list :items children-items :level level))))" "lisp"))
(p "That's the entire navigation. Three small components composed. No bars, no dropdowns, no mobile variants.")))
;; -----------------------------------------------------------------------
;; Path resolution
;; -----------------------------------------------------------------------
(~doc-section :title "Path Resolution" :id "resolution"
(~docs/section :title "Path Resolution" :id "resolution"
(p "Given a URL path, compute the breadcrumb trail and children. This is a tree walk:")
(~doc-code :code (highlight "(define resolve-nav-path\n (fn (tree current-href)\n ;; Walk sx-nav-tree, find the node matching current-href,\n ;; return the trail of ancestors + current children.\n ;;\n ;; Returns: {:trail (list of {:node N :siblings S})\n ;; :children (list) or nil\n ;; :depth number}\n ;;\n ;; Example: current-href = \"/etc/plans/typed-sx\"\n ;; → trail: [{:node Plans :siblings [Docs, CSSX, ...]}\n ;; {:node Typed-SX :siblings [Status, Reader-Macros, ...]}]\n ;; → children: nil (leaf node)\n ;; → depth: 2\n (let ((result (walk-nav-tree tree current-href (list))))\n result)))" "lisp"))
(~docs/code :code (highlight "(define resolve-nav-path\n (fn (tree current-href)\n ;; Walk sx-nav-tree, find the node matching current-href,\n ;; return the trail of ancestors + current children.\n ;;\n ;; Returns: {:trail (list of {:node N :siblings S})\n ;; :children (list) or nil\n ;; :depth number}\n ;;\n ;; Example: current-href = \"/etc/plans/typed-sx\"\n ;; → trail: [{:node Plans :siblings [Docs, CSSX, ...]}\n ;; {:node Typed-SX :siblings [Status, Reader-Macros, ...]}]\n ;; → children: nil (leaf node)\n ;; → depth: 2\n (let ((result (walk-nav-tree tree current-href (list))))\n result)))" "lisp"))
(p "This runs server-side (it's a pure function, no IO). The layout component calls it with the current URL and passes the result to " (code "~sx-nav") ". Same pattern as the current " (code "find-current") " but produces a richer result.")
(p "This runs server-side (it's a pure function, no IO). The layout component calls it with the current URL and passes the result to " (code "~plans/nav-redesign/nav") ". Same pattern as the current " (code "find-current") " but produces a richer result.")
(p "For sx-get navigations (HTMX swaps), the server re-renders the nav with the new path. The morph diffs the old and new nav — breadcrumb rows appear/disappear, the list changes. CSS transitions handle the visual."))
@@ -126,7 +126,7 @@
;; What goes away
;; -----------------------------------------------------------------------
(~doc-section :title "What Goes Away" :id "removal"
(~docs/section :title "What Goes Away" :id "removal"
(p "Significant deletion:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
@@ -136,7 +136,7 @@
(th :class "px-3 py-2 font-medium text-stone-600" "Why")))
(tbody
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~menu-row-sx")
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~shared:layout/menu-row-sx")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "shared/sx/templates/layout.sx")
(td :class "px-3 py-2 text-stone-600" "Horizontal bar with colour levels — replaced by breadcrumb rows"))
(tr :class "border-b border-stone-100"
@@ -150,23 +150,23 @@
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~sx-main-nav")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "sx/sx/layouts.sx")
(td :class "px-3 py-2 text-stone-600" "Horizontal nav list — replaced by ~nav-list"))
(td :class "px-3 py-2 text-stone-600" "Horizontal nav list — replaced by ~plans/nav-redesign/nav-list"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~section-nav")
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~nav-data/section-nav")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "sx/sx/nav-data.sx")
(td :class "px-3 py-2 text-stone-600" "Sub-nav builder — replaced by ~nav-list"))
(td :class "px-3 py-2 text-stone-600" "Sub-nav builder — replaced by ~plans/nav-redesign/nav-list"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~nav-link")
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~shared:layout/nav-link")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "shared/sx/templates/layout.sx")
(td :class "px-3 py-2 text-stone-600" "Complex link with aria-selected + submenu wrapper — replaced by plain a tags"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~mobile-menu-section")
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "~shared:layout/mobile-menu-section")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "shared/sx/templates/layout.sx")
(td :class "px-3 py-2 text-stone-600" "Separate mobile menu — new nav is inherently responsive"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "6 layout variants")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "sx/sx/layouts.sx")
(td :class "px-3 py-2 text-stone-600" "full/oob/mobile × home/section — replaced by one layout with ~sx-nav"))
(td :class "px-3 py-2 text-stone-600" "full/oob/mobile × home/section — replaced by one layout with ~plans/nav-redesign/nav"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" ".nav-group CSS")
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "shared/sx/templates/shell.sx")
@@ -178,14 +178,14 @@
;; Layout simplification
;; -----------------------------------------------------------------------
(~doc-section :title "Layout Simplification" :id "layout"
(~docs/section :title "Layout Simplification" :id "layout"
(p "The defpage layout declarations currently specify section, sub-label, sub-href, sub-nav, selected — five params to configure two menu bars. The new layout takes one param: the nav trail.")
(~doc-code :code (highlight ";; Current (verbose, configures two bars)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-section\n :section \"Plans\"\n :sub-label \"Plans\"\n :sub-href \"/etc/plans/\"\n :sub-nav (~section-nav :items plans-nav-items\n :current (find-current plans-nav-items slug))\n :selected (or (find-current plans-nav-items slug) \"\"))\n :content (...))\n\n;; New (one param, nav computed from URL)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-docs :path (str \"/etc/plans/\" slug))\n :content (...))" "lisp"))
(~docs/code :code (highlight ";; Current (verbose, configures two bars)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-section\n :section \"Plans\"\n :sub-label \"Plans\"\n :sub-href \"/etc/plans/\"\n :sub-nav (~nav-data/section-nav :items plans-nav-items\n :current (find-current plans-nav-items slug))\n :selected (or (find-current plans-nav-items slug) \"\"))\n :content (...))\n\n;; New (one param, nav computed from URL)\n(defpage plan-page\n :path \"/etc/plans/<slug>\"\n :layout (:sx-docs :path (str \"/etc/plans/\" slug))\n :content (...))" "lisp"))
(p "The layout component computes the nav trail internally from the path and the nav tree. No more passing section names, sub-labels, or pre-built nav components through layout params.")
(~doc-code :code (highlight "(defcomp ~sx-docs-layout-full (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~root-header-auto)\n (~sx-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\")))))\n\n(defcomp ~sx-docs-layout-oob (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~oob-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\"))\n (~root-header-auto true))))" "lisp"))
(~docs/code :code (highlight "(defcomp ~plans/nav-redesign/docs-layout-full (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~root-header-auto)\n (~sx-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\")))))\n\n(defcomp ~plans/nav-redesign/docs-layout-oob (&key path)\n (let ((nav-state (resolve-nav-path sx-nav-tree path)))\n (<> (~oob-nav\n :trail (get nav-state \"trail\")\n :children-items (get nav-state \"children\")\n :level (get nav-state \"depth\"))\n (~root-header-auto true))))" "lisp"))
(p "Two layout components instead of twelve. Every defpage in docs.sx simplifies from five layout params to one."))
@@ -193,7 +193,7 @@
;; Scope
;; -----------------------------------------------------------------------
(~doc-section :title "Scope" :id "scope"
(~docs/section :title "Scope" :id "scope"
(div :class "rounded border border-amber-200 bg-amber-50 p-4 mb-4"
(p :class "text-amber-900 font-medium" "SX docs only — for now")
(p :class "text-amber-800" "This redesign applies to the SX docs app (" (code "sx/") "). The other services (blog, market, events, etc.) keep their current navigation. If the pattern proves out, it can migrate to shared infrastructure and replace the root menu system too."))
@@ -217,29 +217,29 @@
;; Implementation
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(~doc-subsection :title "Phase 1: Nav tree + resolution"
(~docs/subsection :title "Phase 1: Nav tree + resolution"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Add " (code "sx-nav-tree") " to " (code "nav-data.sx") " — compose existing " (code "*-nav-items") " lists into a tree")
(li "Write " (code "resolve-nav-path") " — pure function, tree walk, returns trail + children")
(li "Test: given a path, produces the correct breadcrumb trail and child list")))
(~doc-subsection :title "Phase 2: New components"
(~docs/subsection :title "Phase 2: New components"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Write " (code "~sx-logo") ", " (code "~nav-breadcrumb") ", " (code "~nav-list") ", " (code "~sx-nav"))
(li "Write " (code "~sx-docs-layout-full") " and " (code "~sx-docs-layout-oob"))
(li "Write " (code "~plans/nav-redesign/logo") ", " (code "~plans/nav-redesign/nav-breadcrumb") ", " (code "~plans/nav-redesign/nav-list") ", " (code "~plans/nav-redesign/nav"))
(li "Write " (code "~plans/nav-redesign/docs-layout-full") " and " (code "~plans/nav-redesign/docs-layout-oob"))
(li "Register new layout in " (code "layouts.py"))
(li "Test with one defpage first — verify morph transitions work")))
(~doc-subsection :title "Phase 3: Migrate all defpages"
(~docs/subsection :title "Phase 3: Migrate all defpages"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Update every defpage in " (code "docs.sx") " to use " (code ":layout (:sx-docs :path ...)"))
(li "This is mechanical — replace the 5-param layout block with 1-param")))
(~doc-subsection :title "Phase 4: Delete old components"
(~docs/subsection :title "Phase 4: Delete old components"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Delete " (code "~sx-main-nav") ", " (code "~sx-header-row") ", " (code "~sx-sub-row") ", " (code "~section-nav"))
(li "Delete " (code "~sx-main-nav") ", " (code "~sx-header-row") ", " (code "~sx-sub-row") ", " (code "~nav-data/section-nav"))
(li "Delete all 12 SX layout variants from " (code "layouts.sx"))
(li "Delete old layout registrations from " (code "layouts.py"))
(li "Remove " (code ".nav-group") " CSS if no other service uses it"))))))

70
sx/sx/plans/predictive-prefetch.sx
View File

@@ -2,15 +2,15 @@
;; Predictive Component Prefetching
;; ---------------------------------------------------------------------------
(defcomp ~plan-predictive-prefetch-content ()
(~doc-page :title "Predictive Component Prefetching"
(defcomp ~plans/predictive-prefetch/plan-predictive-prefetch-content ()
(~docs/page :title "Predictive Component Prefetching"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "Phase 3 of the isomorphic roadmap added client-side routing with component dependency checking. When a user clicks a link, " (code "try-client-route") " checks " (code "has-all-deps?") " — if the target page needs components not yet loaded, the client falls back to a server fetch. This works correctly but misses an opportunity: " (strong "we can prefetch those missing components before the click happens."))
(p "The page registry already carries " (code ":deps") " metadata for every page. The client already knows which components are loaded via " (code "loaded-component-names") ". The gap is a mechanism to " (em "proactively") " resolve the difference — fetching missing component definitions so that by the time the user clicks, client-side routing succeeds.")
(p "But this goes beyond just hover-to-prefetch. The full spectrum includes: bundling linked routes' components with the initial page load, batch-prefetching after idle, predicting mouse trajectory toward links, and even splitting the component/data fetch so that " (code ":data") " pages can prefetch their components and only fetch data on click. Each strategy trades bandwidth for latency, and pages should be able to declare which tradeoff they want."))
(~doc-section :title "Current State" :id "current-state"
(~docs/section :title "Current State" :id "current-state"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -47,7 +47,7 @@
;; Prefetch strategies
;; -----------------------------------------------------------------------
(~doc-section :title "Prefetch Strategies" :id "strategies"
(~docs/section :title "Prefetch Strategies" :id "strategies"
(p "Prefetching is a spectrum from conservative to aggressive. The system should support all of these, configured declaratively per link or per page via " (code "defpage") " metadata and " (code "sx-prefetch") " attributes.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -94,22 +94,22 @@
(td :class "px-3 py-2 text-stone-700" "Components " (em "and") " page data for " (code ":data") " pages")
(td :class "px-3 py-2 text-stone-600" "Zero for components; data fetch may still be in flight")))))
(~doc-subsection :title "Eager Bundle"
(~docs/subsection :title "Eager Bundle"
(p "The server already computes per-page component bundles. For key navigation paths — the main nav bar, section nav — the server can include " (em "linked routes' components") " in the initial bundle, not just the current page's.")
(~doc-code :code (highlight ";; defpage metadata declares eager prefetch targets\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :auth :public\n :prefetch :eager ;; bundle deps for all linked pure routes\n :content (case slug ...))" "lisp"))
(~docs/code :code (highlight ";; defpage metadata declares eager prefetch targets\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :auth :public\n :prefetch :eager ;; bundle deps for all linked pure routes\n :content (case slug ...))" "lisp"))
(p "Implementation: " (code "components_for_page()") " already scans the page SX for component refs. Extend it to also scan for " (code "href") " attributes, match them against the page registry, and include those pages' deps in the bundle. The cost is a larger initial payload; the benefit is zero-latency navigation within a section."))
(~doc-subsection :title "Idle Timer"
(~docs/subsection :title "Idle Timer"
(p "After page load and initial render, use " (code "requestIdleCallback") " (or a fallback " (code "setTimeout") ") to scan visible nav links and batch-prefetch their missing components in a single request.")
(~doc-code :code (highlight "(define prefetch-visible-links-on-idle\n (fn ()\n (request-idle-callback\n (fn ()\n (let ((links (dom-query-all \"a[href][sx-get]\"))\n (all-missing (list)))\n (for-each\n (fn (link)\n (let ((missing (compute-missing-deps\n (url-pathname (dom-get-attr link \"href\")))))\n (when missing\n (for-each (fn (d) (append! all-missing d))\n missing))))\n links)\n (when (not (empty? all-missing))\n (prefetch-components (dedupe all-missing))))))))" "lisp"))
(~docs/code :code (highlight "(define prefetch-visible-links-on-idle\n (fn ()\n (request-idle-callback\n (fn ()\n (let ((links (dom-query-all \"a[href][sx-get]\"))\n (all-missing (list)))\n (for-each\n (fn (link)\n (let ((missing (compute-missing-deps\n (url-pathname (dom-get-attr link \"href\")))))\n (when missing\n (for-each (fn (d) (append! all-missing d))\n missing))))\n links)\n (when (not (empty? all-missing))\n (prefetch-components (dedupe all-missing))))))))" "lisp"))
(p "Called once from " (code "boot-init") " after initial processing. Batches all missing deps into one network request. Low priority — browser handles it when idle."))
(~doc-subsection :title "Mouse Approach (Trajectory Prediction)"
(~docs/subsection :title "Mouse Approach (Trajectory Prediction)"
(p "Don't wait for the cursor to reach the link — predict where it's heading. Track the last few " (code "mousemove") " events, extrapolate the trajectory, and if it points toward a link, start prefetching before the hover event fires.")
(~doc-code :code (highlight "(define bind-approach-prefetch\n (fn (container)\n ;; Track mouse trajectory within a nav container.\n ;; On each mousemove, extrapolate position ~200ms ahead.\n ;; If projected point intersects a link's bounding box,\n ;; prefetch that link's route deps.\n (let ((last-x 0) (last-y 0) (last-t 0)\n (prefetched (dict)))\n (dom-add-listener container \"mousemove\"\n (fn (e)\n (let ((now (timestamp))\n (dt (- now last-t)))\n (when (> dt 16) ;; ~60fps throttle\n (let ((vx (/ (- (event-x e) last-x) dt))\n (vy (/ (- (event-y e) last-y) dt))\n (px (+ (event-x e) (* vx 200)))\n (py (+ (event-y e) (* vy 200)))\n (target (dom-element-at-point px py)))\n (when (and target (dom-has-attr? target \"href\")\n (not (get prefetched\n (dom-get-attr target \"href\"))))\n (let ((href (dom-get-attr target \"href\")))\n (set! prefetched\n (merge prefetched {href true}))\n (prefetch-route-deps\n (url-pathname href)))))\n (set! last-x (event-x e))\n (set! last-y (event-y e))\n (set! last-t now))))))))" "lisp"))
(~docs/code :code (highlight "(define bind-approach-prefetch\n (fn (container)\n ;; Track mouse trajectory within a nav container.\n ;; On each mousemove, extrapolate position ~200ms ahead.\n ;; If projected point intersects a link's bounding box,\n ;; prefetch that link's route deps.\n (let ((last-x 0) (last-y 0) (last-t 0)\n (prefetched (dict)))\n (dom-add-listener container \"mousemove\"\n (fn (e)\n (let ((now (timestamp))\n (dt (- now last-t)))\n (when (> dt 16) ;; ~60fps throttle\n (let ((vx (/ (- (event-x e) last-x) dt))\n (vy (/ (- (event-y e) last-y) dt))\n (px (+ (event-x e) (* vx 200)))\n (py (+ (event-y e) (* vy 200)))\n (target (dom-element-at-point px py)))\n (when (and target (dom-has-attr? target \"href\")\n (not (get prefetched\n (dom-get-attr target \"href\"))))\n (let ((href (dom-get-attr target \"href\")))\n (set! prefetched\n (merge prefetched {href true}))\n (prefetch-route-deps\n (url-pathname href)))))\n (set! last-x (event-x e))\n (set! last-y (event-y e))\n (set! last-t now))))))))" "lisp"))
(p "This is the most speculative strategy — best suited for dense navigation areas (section sidebars, nav bars) where the cursor trajectory is a strong predictor. The " (code "prefetched") " dict prevents duplicate fetches within the same container interaction."))
(~doc-subsection :title "Components + Data (Hybrid Prefetch)"
(~docs/subsection :title "Components + Data (Hybrid Prefetch)"
(p "The most interesting strategy. For pages with " (code ":data") " dependencies, current behavior is full server fallback. But the page's " (em "components") " are still pure and prefetchable. If we prefetch components ahead of time, the click only needs to fetch " (em "data") " — a much smaller, faster response.")
(p "This creates a new rendering path:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
@@ -118,84 +118,84 @@
(li "Server returns " (em "only data") " (JSON or SX bindings), not the full rendered page")
(li "Client evaluates the content expression with prefetched components + fetched data")
(li "Result: faster than full server render, no redundant component transfer"))
(~doc-code :code (highlight ";; Declarative: prefetch components, fetch data on click\n(defpage reference-page\n :path \"/reference/<slug>\"\n :auth :public\n :prefetch :components ;; prefetch components, data stays server-fetched\n :data (reference-data slug)\n :content (~reference-attrs-content :attrs attrs))\n\n;; On click, client-side flow:\n;; 1. Components already prefetched (from hover/idle)\n;; 2. GET /reference/attributes → server returns data bindings\n;; 3. Client evals (reference-data slug) result + content expr\n;; 4. Renders locally with cached components" "lisp"))
(~docs/code :code (highlight ";; Declarative: prefetch components, fetch data on click\n(defpage reference-page\n :path \"/reference/<slug>\"\n :auth :public\n :prefetch :components ;; prefetch components, data stays server-fetched\n :data (reference-data slug)\n :content (~reference/attrs-content :attrs attrs))\n\n;; On click, client-side flow:\n;; 1. Components already prefetched (from hover/idle)\n;; 2. GET /reference/attributes → server returns data bindings\n;; 3. Client evals (reference-data slug) result + content expr\n;; 4. Renders locally with cached components" "lisp"))
(p "This is a stepping stone toward full Phase 4 (client IO bridge) of the isomorphic roadmap — it achieves partial client rendering for data pages without needing a general-purpose client async evaluator. The server is a data service, the client is the renderer."))
(~doc-subsection :title "Declarative Configuration"
(~docs/subsection :title "Declarative Configuration"
(p "All strategies configured via " (code "defpage") " metadata and " (code "sx-prefetch") " attributes on links/containers:")
(~doc-code :code (highlight ";; Page-level: what to prefetch for routes linking TO this page\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :prefetch :eager) ;; bundle with linking page\n\n(defpage reference-page\n :path \"/reference/<slug>\"\n :prefetch :components) ;; prefetch components, data on click\n\n;; Link-level: override per-link\n(a :href \"/language/docs/components\"\n :sx-prefetch \"idle\") ;; prefetch after page idle\n\n;; Container-level: approach prediction for nav areas\n(nav :sx-prefetch \"approach\"\n (a :href \"/language/docs/\") (a :href \"/reference/\") ...)" "lisp"))
(~docs/code :code (highlight ";; Page-level: what to prefetch for routes linking TO this page\n(defpage docs-page\n :path \"/language/docs/<slug>\"\n :prefetch :eager) ;; bundle with linking page\n\n(defpage reference-page\n :path \"/reference/<slug>\"\n :prefetch :components) ;; prefetch components, data on click\n\n;; Link-level: override per-link\n(a :href \"/language/docs/components\"\n :sx-prefetch \"idle\") ;; prefetch after page idle\n\n;; Container-level: approach prediction for nav areas\n(nav :sx-prefetch \"approach\"\n (a :href \"/language/docs/\") (a :href \"/reference/\") ...)" "lisp"))
(p "Priority cascade: explicit " (code "sx-prefetch") " on link > " (code ":prefetch") " on target defpage > default (hover). The system never prefetches the same components twice — " (code "_prefetch-pending") " and " (code "loaded-component-names") " handle dedup.")))
;; -----------------------------------------------------------------------
;; Design
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation Design" :id "design"
(~docs/section :title "Implementation Design" :id "design"
(p "Per the SX host architecture principle: all SX-specific logic goes in " (code ".sx") " spec files and gets bootstrapped. The prefetch logic — scanning links, computing missing deps, managing the component cache — must be specced in " (code ".sx") ", not written directly in JS or Python.")
(~doc-subsection :title "Phase 1: Component Fetch Endpoint (Python)"
(~docs/subsection :title "Phase 1: Component Fetch Endpoint (Python)"
(p "A new " (strong "public") " endpoint (not " (code "/internal/") " — the client's browser calls it) that returns component definitions by name.")
(~doc-code :code (highlight "GET /<service-prefix>/sx/components?names=~card,~essay-foo\n\nResponse (text/sx):\n(defcomp ~card (&key title &rest children)\n (div :class \"border rounded p-4\" (h2 title) children))\n(defcomp ~essay-foo (&key id)\n (div (~card :title id)))" "http"))
(~docs/code :code (highlight "GET /<service-prefix>/sx/components?names=~plans/predictive-prefetch/card,~essay-foo\n\nResponse (text/sx):\n(defcomp ~plans/predictive-prefetch/card (&key title &rest children)\n (div :class \"border rounded p-4\" (h2 title) children))\n(defcomp ~plans/predictive-prefetch/essay-foo (&key id)\n (div (~plans/predictive-prefetch/card :title id)))" "http"))
(p "The server resolves transitive deps via " (code "deps.py") ", subtracts anything listed in the " (code "SX-Components") " request header (already loaded), serializes and returns. This is essentially " (code "components_for_request()") " driven by an explicit " (code "?names=") " param.")
(p "Cache-friendly: the response is a pure function of component hash + requested names. " (code "Cache-Control: public, max-age=3600") " with the component hash as ETag."))
(~doc-subsection :title "Phase 2: Client Prefetch Logic (SX spec)"
(~docs/subsection :title "Phase 2: Client Prefetch Logic (SX spec)"
(p "New functions in " (code "orchestration.sx") " (or a new " (code "prefetch.sx") " if scope warrants):")
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. compute-missing-deps")
(p "Given a pathname, find the page, return dep names not in " (code "loaded-component-names") ". Returns nil if page not found or has data (can't client-route anyway).")
(~doc-code :code (highlight "(define compute-missing-deps\n (fn (pathname)\n (let ((match (find-matching-route pathname _page-routes)))\n (when (and match (not (get match \"has-data\")))\n (let ((deps (or (get match \"deps\") (list)))\n (loaded (loaded-component-names)))\n (filter (fn (d) (not (contains? loaded d))) deps))))))" "lisp")))
(~docs/code :code (highlight "(define compute-missing-deps\n (fn (pathname)\n (let ((match (find-matching-route pathname _page-routes)))\n (when (and match (not (get match \"has-data\")))\n (let ((deps (or (get match \"deps\") (list)))\n (loaded (loaded-component-names)))\n (filter (fn (d) (not (contains? loaded d))) deps))))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. prefetch-components")
(p "Fetch component definitions from the server for a list of names. Deduplicates in-flight requests. On success, parses and registers the returned definitions into the component env.")
(~doc-code :code (highlight "(define _prefetch-pending (dict))\n\n(define prefetch-components\n (fn (names)\n (let ((key (join \",\" (sort names))))\n (when (not (get _prefetch-pending key))\n (set! _prefetch-pending\n (merge _prefetch-pending {key true}))\n (fetch-components-from-server names\n (fn (sx-text)\n (sx-process-component-text sx-text)\n (dict-remove! _prefetch-pending key)))))))" "lisp")))
(~docs/code :code (highlight "(define _prefetch-pending (dict))\n\n(define prefetch-components\n (fn (names)\n (let ((key (join \",\" (sort names))))\n (when (not (get _prefetch-pending key))\n (set! _prefetch-pending\n (merge _prefetch-pending {key true}))\n (fetch-components-from-server names\n (fn (sx-text)\n (sx-process-component-text sx-text)\n (dict-remove! _prefetch-pending key)))))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "3. prefetch-route-deps")
(p "High-level composition: compute missing deps for a route, fetch if any.")
(~doc-code :code (highlight "(define prefetch-route-deps\n (fn (pathname)\n (let ((missing (compute-missing-deps pathname)))\n (when (and missing (not (empty? missing)))\n (log-info (str \"sx:prefetch \"\n (len missing) \" components for \" pathname))\n (prefetch-components missing)))))" "lisp")))
(~docs/code :code (highlight "(define prefetch-route-deps\n (fn (pathname)\n (let ((missing (compute-missing-deps pathname)))\n (when (and missing (not (empty? missing)))\n (log-info (str \"sx:prefetch \"\n (len missing) \" components for \" pathname))\n (prefetch-components missing)))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "4. Trigger: link hover")
(p "On mouseover of a boosted link, prefetch its route's missing components. Debounced 150ms to avoid fetching on quick mouse-throughs.")
(~doc-code :code (highlight "(define bind-prefetch-on-hover\n (fn (link)\n (let ((timer nil))\n (dom-add-listener link \"mouseover\"\n (fn (e)\n (clear-timeout timer)\n (set! timer (set-timeout\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n 150))))\n (dom-add-listener link \"mouseout\"\n (fn (e) (clear-timeout timer))))))" "lisp")))
(~docs/code :code (highlight "(define bind-prefetch-on-hover\n (fn (link)\n (let ((timer nil))\n (dom-add-listener link \"mouseover\"\n (fn (e)\n (clear-timeout timer)\n (set! timer (set-timeout\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n 150))))\n (dom-add-listener link \"mouseout\"\n (fn (e) (clear-timeout timer))))))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "5. Trigger: viewport intersection (opt-in)")
(p "More aggressive strategy: when a link scrolls into view, prefetch its route's deps. Opt-in via " (code "sx-prefetch=\"visible\"") " attribute.")
(~doc-code :code (highlight "(define bind-prefetch-on-visible\n (fn (link)\n (observe-intersection link\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n true 0)))" "lisp")))
(~docs/code :code (highlight "(define bind-prefetch-on-visible\n (fn (link)\n (observe-intersection link\n (fn () (prefetch-route-deps\n (url-pathname (dom-get-attr link \"href\"))))\n true 0)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "6. Integration into process-elements")
(p "During the existing hydration pass, for each boosted link:")
(~doc-code :code (highlight ";; In process-elements, after binding boost behavior:\n(when (and (should-boost-link? link)\n (dom-get-attr link \"href\"))\n (bind-prefetch-on-hover link))\n\n;; Explicit viewport prefetch:\n(when (dom-has-attr? link \"sx-prefetch\")\n (bind-prefetch-on-visible link))" "lisp")))))
(~docs/code :code (highlight ";; In process-elements, after binding boost behavior:\n(when (and (should-boost-link? link)\n (dom-get-attr link \"href\"))\n (bind-prefetch-on-hover link))\n\n;; Explicit viewport prefetch:\n(when (dom-has-attr? link \"sx-prefetch\")\n (bind-prefetch-on-visible link))" "lisp")))))
(~doc-subsection :title "Phase 3: Boundary Declaration"
(~docs/subsection :title "Phase 3: Boundary Declaration"
(p "Two new IO primitives in " (code "boundary.sx") " (browser-only):")
(~doc-code :code (highlight ";; IO primitives (browser-only)\n(io fetch-components-from-server (names callback) -> void)\n(io sx-process-component-text (sx-text) -> void)" "lisp"))
(~docs/code :code (highlight ";; IO primitives (browser-only)\n(io fetch-components-from-server (names callback) -> void)\n(io sx-process-component-text (sx-text) -> void)" "lisp"))
(p "These are thin wrappers around " (code "fetch()") " + the existing component script processing logic already in the boundary adapter."))
(~doc-subsection :title "Phase 4: Bootstrap"
(~docs/subsection :title "Phase 4: Bootstrap"
(p (code "bootstrap_js.py") " picks up the new functions from the spec and emits them into " (code "sx-browser.js") ". The two new boundary IO functions get implemented in the JS boundary adapter — the hand-written glue code that the bootstrapper doesn't generate.")
(~doc-code :code (highlight "// fetch-components-from-server: calls the endpoint\nfunction fetchComponentsFromServer(names, callback) {\n const url = `${routePrefix}/sx/components?names=${names.join(\",\")}`;\n const headers = {\n \"SX-Components\": loadedComponentNames().join(\",\")\n };\n fetch(url, { headers })\n .then(r => r.ok ? r.text() : \"\")\n .then(text => callback(text))\n .catch(() => {}); // silent fail — prefetch is best-effort\n}\n\n// sx-process-component-text: parse defcomp/defmacro into env\nfunction sxProcessComponentText(sxText) {\n if (!sxText) return;\n const frag = document.createElement(\"div\");\n frag.innerHTML =\n `<script type=\"text/sx\" data-components>${sxText}<\\/script>`;\n Sx.processScripts(frag);\n}" "javascript"))))
(~docs/code :code (highlight "// fetch-components-from-server: calls the endpoint\nfunction fetchComponentsFromServer(names, callback) {\n const url = `${routePrefix}/sx/components?names=${names.join(\",\")}`;\n const headers = {\n \"SX-Components\": loadedComponentNames().join(\",\")\n };\n fetch(url, { headers })\n .then(r => r.ok ? r.text() : \"\")\n .then(text => callback(text))\n .catch(() => {}); // silent fail — prefetch is best-effort\n}\n\n// sx-process-component-text: parse defcomp/defmacro into env\nfunction sxProcessComponentText(sxText) {\n if (!sxText) return;\n const frag = document.createElement(\"div\");\n frag.innerHTML =\n `<script type=\"text/sx\" data-components>${sxText}<\\/script>`;\n Sx.processScripts(frag);\n}" "javascript"))))
;; -----------------------------------------------------------------------
;; Request flow
;; -----------------------------------------------------------------------
(~doc-section :title "Request Flow" :id "request-flow"
(~docs/section :title "Request Flow" :id "request-flow"
(p "End-to-end example: user hovers a link, components prefetch, click goes client-side.")
(~doc-code :code (highlight "User hovers link \"/language/docs/sx-manifesto\"\n |\n +-- bind-prefetch-on-hover fires (150ms debounce)\n |\n +-- compute-missing-deps(\"/language/docs/sx-manifesto\")\n | +-- find-matching-route -> page with deps:\n | | [\"~essay-sx-manifesto\", \"~doc-code\"]\n | +-- loaded-component-names -> [\"~nav\", \"~footer\", \"~doc-code\"]\n | +-- missing: [\"~essay-sx-manifesto\"]\n |\n +-- prefetch-components([\"~essay-sx-manifesto\"])\n | +-- GET /sx/components?names=~essay-sx-manifesto\n | | Headers: SX-Components: ~nav,~footer,~doc-code\n | +-- Server resolves transitive deps\n | | (also needs ~rich-text, subtracts already-loaded)\n | +-- Response:\n | (defcomp ~essay-sx-manifesto ...) \n | (defcomp ~rich-text ...)\n |\n +-- sx-process-component-text registers defcomps in env\n |\n +-- User clicks link\n +-- try-client-route(\"/language/docs/sx-manifesto\")\n +-- has-all-deps? -> true (prefetched!)\n +-- eval content -> DOM\n +-- Client-side render, no server roundtrip" "text")))
(~docs/code :code (highlight "User hovers link \"/language/docs/sx-manifesto\"\n |\n +-- bind-prefetch-on-hover fires (150ms debounce)\n |\n +-- compute-missing-deps(\"/language/docs/sx-manifesto\")\n | +-- find-matching-route -> page with deps:\n | | [\"~essay-sx-manifesto\", \"~doc-code\"]\n | +-- loaded-component-names -> [\"~nav\", \"~footer\", \"~doc-code\"]\n | +-- missing: [\"~essay-sx-manifesto\"]\n |\n +-- prefetch-components([\"~essay-sx-manifesto\"])\n | +-- GET /sx/components?names=~essay-sx-manifesto\n | | Headers: SX-Components: ~plans/environment-images/nav,~footer,~doc-code\n | +-- Server resolves transitive deps\n | | (also needs ~plans/predictive-prefetch/rich-text, subtracts already-loaded)\n | +-- Response:\n | (defcomp ~plans/predictive-prefetch/essay-sx-manifesto ...) \n | (defcomp ~plans/predictive-prefetch/rich-text ...)\n |\n +-- sx-process-component-text registers defcomps in env\n |\n +-- User clicks link\n +-- try-client-route(\"/language/docs/sx-manifesto\")\n +-- has-all-deps? -> true (prefetched!)\n +-- eval content -> DOM\n +-- Client-side render, no server roundtrip" "text")))
;; -----------------------------------------------------------------------
;; File changes
;; -----------------------------------------------------------------------
(~doc-section :title "File Changes" :id "file-changes"
(~docs/section :title "File Changes" :id "file-changes"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -228,14 +228,14 @@
;; Non-goals & rollout
;; -----------------------------------------------------------------------
(~doc-section :title "Non-Goals (This Phase)" :id "non-goals"
(~docs/section :title "Non-Goals (This Phase)" :id "non-goals"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Analytics-driven prediction") " — no ML models or click-frequency heuristics. Trajectory prediction uses geometry, not statistics.")
(li (strong "Cross-service prefetch") " — components are per-service. A link to a different service domain is always a server navigation.")
(li (strong "Service worker caching") " — could layer on later, but basic fetch + in-memory registration is sufficient.")
(li (strong "Full client-side data evaluation") " — the components+data strategy fetches data from the server, it doesn't replicate server IO on the client. That's Phase 4 of the isomorphic roadmap.")))
(~doc-section :title "Rollout" :id "rollout"
(~docs/section :title "Rollout" :id "rollout"
(p "Incremental, each step independently valuable:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li (strong "Component endpoint") " — purely additive. Refactor " (code "components_for_request()") " to accept explicit " (code "?names=") " param.")
@@ -246,7 +246,7 @@
(li (strong "Eager bundles") " — extend " (code "components_for_page()") " to include linked routes' deps. Heavier initial payload, zero-latency nav.")
(li (strong "Components + data split") " — new server response mode returning data bindings only. Client renders with prefetched components. Bridges toward Phase 4.")))
(~doc-section :title "Relationship to Isomorphic Roadmap" :id "relationship"
(~docs/section :title "Relationship to Isomorphic Roadmap" :id "relationship"
(p "This plan sits between Phase 3 (client-side routing) and Phase 4 (client async & IO bridge) of the "
(a :href "/sx/(etc.(plan.isomorphic-architecture))" :class "text-violet-700 underline" "isomorphic architecture roadmap")
". It extends Phase 3 by making more navigations go client-side without needing any IO bridge — purely by ensuring component definitions are available before they're needed.")

58
sx/sx/plans/reader-macro-demo.sx
View File

@@ -2,19 +2,19 @@
;; Reader Macro Demo: #z3 — SX Spec to SMT-LIB (live translation via z3.sx)
;; ---------------------------------------------------------------------------
(defcomp ~z3-example (&key (sx-source :as string) (smt-output :as string))
(defcomp ~plans/reader-macro-demo/z3-example (&key (sx-source :as string) (smt-output :as string))
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
(div
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
(~doc-code :code (highlight sx-source "lisp")))
(~docs/code :code (highlight sx-source "lisp")))
(div
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output (live from z3.sx)")
(~doc-code :code (highlight smt-output "lisp")))))
(~docs/code :code (highlight smt-output "lisp")))))
(defcomp ~plan-reader-macro-demo-content ()
(~doc-page :title "Reader Macro Demo: #z3"
(defcomp ~plans/reader-macro-demo/plan-reader-macro-demo-content ()
(~docs/page :title "Reader Macro Demo: #z3"
(~doc-section :title "The idea" :id "idea"
(~docs/section :title "The idea" :id "idea"
(p :class "text-stone-600"
"SX spec files (" (code "primitives.sx") ", " (code "eval.sx") ") are machine-readable declarations. Reader macros transform these at parse time. " (code "#z3") " reads a " (code "define-primitive") " declaration and emits " (a :href "https://smtlib.cs.uiowa.edu/" :class "text-violet-600 hover:underline" "SMT-LIB") " — the standard input language for " (a :href "https://github.com/Z3Prover/z3" :class "text-violet-600 hover:underline" "Z3") " and other theorem provers.")
(p :class "text-stone-600"
@@ -24,67 +24,67 @@
(p :class "text-sm text-violet-700"
"The translator is written in SX itself (" (code "z3.sx") "). The SX evaluator executes " (code "z3.sx") " against the spec to produce SMT-LIB. Same pattern as " (code "bootstrap_js.py") " and " (code "bootstrap_py.py") ", but the transformation logic is an s-expression program transforming other s-expressions. All examples on this page are " (em "live output") " — not hardcoded strings.")))
(~doc-section :title "Arithmetic primitives" :id "arithmetic"
(~docs/section :title "Arithmetic primitives" :id "arithmetic"
(p :class "text-stone-600"
"Primitives with " (code ":body") " generate " (code "forall") " assertions. Z3 can verify the definition is satisfiable.")
(~doc-subsection :title "inc"
(~z3-example
(~docs/subsection :title "inc"
(~plans/reader-macro-demo/z3-example
:sx-source "(define-primitive \"inc\"\n :params (n)\n :returns \"number\"\n :doc \"Increment by 1.\"\n :body (+ n 1))"
:smt-output #z3(define-primitive "inc" :params (n) :returns "number" :doc "Increment by 1." :body (+ n 1))))
(~doc-subsection :title "clamp"
(~docs/subsection :title "clamp"
(p :class "text-stone-600 mb-2"
(code "max") " and " (code "min") " have no SMT-LIB equivalent — translated to " (code "ite") " (if-then-else).")
(~z3-example
(~plans/reader-macro-demo/z3-example
:sx-source "(define-primitive \"clamp\"\n :params (x lo hi)\n :returns \"number\"\n :doc \"Clamp x to range [lo, hi].\"\n :body (max lo (min hi x)))"
:smt-output #z3(define-primitive "clamp" :params (x lo hi) :returns "number" :doc "Clamp x to range [lo, hi]." :body (max lo (min hi x))))))
(~doc-section :title "Predicates" :id "predicates"
(~docs/section :title "Predicates" :id "predicates"
(p :class "text-stone-600"
"Predicates return " (code "Bool") " in SMT-LIB. " (code "mod") " and " (code "=") " are identity translations — same syntax in both languages.")
(~doc-subsection :title "odd?"
(~z3-example
(~docs/subsection :title "odd?"
(~plans/reader-macro-demo/z3-example
:sx-source "(define-primitive \"odd?\"\n :params (n)\n :returns \"boolean\"\n :doc \"True if n is odd.\"\n :body (= (mod n 2) 1))"
:smt-output #z3(define-primitive "odd?" :params (n) :returns "boolean" :doc "True if n is odd." :body (= (mod n 2) 1))))
(~doc-subsection :title "!= (inequality)"
(~z3-example
(~docs/subsection :title "!= (inequality)"
(~plans/reader-macro-demo/z3-example
:sx-source "(define-primitive \"!=\"\n :params (a b)\n :returns \"boolean\"\n :doc \"Inequality.\"\n :body (not (= a b)))"
:smt-output #z3(define-primitive "!=" :params (a b) :returns "boolean" :doc "Inequality." :body (not (= a b))))))
(~doc-section :title "Variadics and bodyless" :id "variadics"
(~docs/section :title "Variadics and bodyless" :id "variadics"
(p :class "text-stone-600"
"Variadic primitives (" (code "&rest") ") are declared as uninterpreted functions — Z3 can reason about their properties but not their implementation. Primitives without " (code ":body") " get only a declaration.")
(~doc-subsection :title "+ (variadic)"
(~z3-example
(~docs/subsection :title "+ (variadic)"
(~plans/reader-macro-demo/z3-example
:sx-source "(define-primitive \"+\"\n :params (&rest args)\n :returns \"number\"\n :doc \"Sum all arguments.\")"
:smt-output #z3(define-primitive "+" :params (&rest args) :returns "number" :doc "Sum all arguments.")))
(~doc-subsection :title "nil? (no body)"
(~z3-example
(~docs/subsection :title "nil? (no body)"
(~plans/reader-macro-demo/z3-example
:sx-source "(define-primitive \"nil?\"\n :params (x)\n :returns \"boolean\"\n :doc \"True if x is nil/null/None.\")"
:smt-output #z3(define-primitive "nil?" :params (x) :returns "boolean" :doc "True if x is nil/null/None."))))
(~doc-section :title "Translate entire spec files" :id "full-specs"
(~docs/section :title "Translate entire spec files" :id "full-specs"
(p :class "text-stone-600"
"The translator can process entire spec files. " (code "z3-translate-file") " in " (code "z3.sx") " filters for " (code "define-primitive") ", " (code "define-io-primitive") ", and " (code "define-special-form") " declarations, translates each, and concatenates the output.")
(p :class "text-stone-600"
"Below is the live SMT-LIB output from translating the full " (code "primitives.sx") " — all 87 primitive declarations. The composition is pure SX: " (code "(z3-translate-file (sx-parse (read-spec-file \"primitives.sx\")))") " — read the file, parse it, translate it. No Python glue.")
(~doc-subsection :title "primitives.sx (87 primitives)"
(~doc-code :code (highlight (z3-translate-file (sx-parse (read-spec-file "primitives.sx"))) "lisp"))))
(~docs/subsection :title "primitives.sx (87 primitives)"
(~docs/code :code (highlight (z3-translate-file (sx-parse (read-spec-file "primitives.sx"))) "lisp"))))
(~doc-section :title "The translator: z3.sx" :id "z3-source"
(~docs/section :title "The translator: z3.sx" :id "z3-source"
(p :class "text-stone-600"
"The entire translator is a single SX file — s-expressions that walk other s-expressions and emit strings. No host language logic. The same file runs in Python (server) and could run in JavaScript (browser) via the bootstrapped evaluator.")
(~doc-code :code (highlight (read-spec-file "z3.sx") "lisp"))
(~docs/code :code (highlight (read-spec-file "z3.sx") "lisp"))
(p :class "text-stone-600 mt-4"
"359 lines. The key functions: " (code "z3-sort") " maps SX types to SMT-LIB sorts. " (code "z3-expr") " recursively translates expressions — identity ops pass through unchanged, " (code "max") "/" (code "min") " become " (code "ite") ", predicates get renamed. " (code "z3-translate") " dispatches on form type. " (code "z3-translate-file") " filters and batch-translates."))
(~doc-section :title "The pattern: SX → anything" :id "sx-to-anything"
(~docs/section :title "The pattern: SX → anything" :id "sx-to-anything"
(p :class "text-stone-600"
"z3.sx proves the pattern: an SX program that transforms SX ASTs into a target language. The same pattern works for any target.")
(div :class "rounded border border-stone-200 bg-stone-50 p-4 mt-2 mb-4"
@@ -120,7 +120,7 @@
(p :class "text-stone-600"
"A " (code "py.sx") " wouldn't be limited to the spec. Any SX expression could be translated: " (code "#py(map (fn (x) (* x x)) items)") " → " (code "list(map(lambda x: x * x, items))") ". The bootstrappers (" (code "bootstrap_js.py") ", " (code "bootstrap_py.py") ") are Python programs that do this for the full spec. " (code "py.sx") " would be the same thing, written in SX — a self-hosting bootstrapper."))
(~doc-section :title "How it works" :id "how-it-works"
(~docs/section :title "How it works" :id "how-it-works"
(p :class "text-stone-600"
"The " (code "#z3") " reader macro is registered before parsing. When the parser hits " (code "#z3(define-primitive ...)") ", it:")
(ol :class "list-decimal pl-6 space-y-2 text-stone-600"
@@ -135,7 +135,7 @@
(p :class "text-stone-600"
"The handler is a pure function from AST to value. No side effects. No mutation. Reader macros are " (em "syntax transformations") " — they happen before evaluation, before rendering, before anything else. They are the earliest possible extension point."))
(~doc-section :title "The strange loop" :id "strange-loop"
(~docs/section :title "The strange loop" :id "strange-loop"
(p :class "text-stone-600"
"The SX specification files are simultaneously:")
(ul :class "list-disc pl-6 space-y-2 text-stone-600"

56
sx/sx/plans/reader-macros.sx
View File

@@ -2,32 +2,32 @@
;; Reader Macros
;; ---------------------------------------------------------------------------
(defcomp ~plan-reader-macros-content ()
(~doc-page :title "Reader Macros"
(defcomp ~plans/reader-macros/plan-reader-macros-content ()
(~docs/page :title "Reader Macros"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "SX has three hardcoded reader transformations: " (code "`") " → " (code "(quasiquote ...)") ", " (code ",") " → " (code "(unquote ...)") ", " (code ",@") " → " (code "(splice-unquote ...)") ". These are baked into the parser with no extensibility. The " (code "~") " prefix for components and " (code "&") " for param modifiers are just symbol characters, handled at eval time.")
(p "Reader macros add parse-time transformations triggered by a dispatch character. Motivating use case: a " (code "~md") " component that uses heredoc syntax for markdown source instead of string literals. More broadly: datum comments, raw strings, and custom literal syntax."))
(~doc-section :title "Design" :id "design"
(~docs/section :title "Design" :id "design"
(~doc-subsection :title "Dispatch Character: #"
(~docs/subsection :title "Dispatch Character: #"
(p "Lisp tradition. " (code "#") " is NOT in " (code "ident-start") " or " (code "ident-char") " — completely free. Pattern:")
(~doc-code :code (highlight "#;expr → (read and discard expr, return next)\n#|...| → raw string literal\n#'expr → (quote expr)" "lisp")))
(~docs/code :code (highlight "#;expr → (read and discard expr, return next)\n#|...| → raw string literal\n#'expr → (quote expr)" "lisp")))
(~doc-subsection :title "#; — Datum comment"
(~docs/subsection :title "#; — Datum comment"
(p "Scheme/Racket standard. Reads and discards the next expression. Preserves balanced parens.")
(~doc-code :code (highlight "(list 1 #;(this is commented out) 2 3) → (list 1 2 3)" "lisp")))
(~docs/code :code (highlight "(list 1 #;(this is commented out) 2 3) → (list 1 2 3)" "lisp")))
(~doc-subsection :title "#|...| — Raw string"
(~docs/subsection :title "#|...| — Raw string"
(p "No escape processing. Everything between " (code "#|") " and " (code "|") " is literal. Enables inline markdown, regex patterns, code examples.")
(~doc-code :code (highlight "(~md #|## Title\n\nSome **bold** text with \"quotes\" and \\backslashes.|)" "lisp")))
(~docs/code :code (highlight "(~md #|## Title\n\nSome **bold** text with \"quotes\" and \\backslashes.|)" "lisp")))
(~doc-subsection :title "#' — Quote shorthand"
(~docs/subsection :title "#' — Quote shorthand"
(p "Currently no single-char quote (" (code "`") " is quasiquote).")
(~doc-code :code (highlight "#'my-function → (quote my-function)" "lisp")))
(~docs/code :code (highlight "#'my-function → (quote my-function)" "lisp")))
(~doc-subsection :title "Extensible dispatch: #name"
(~docs/subsection :title "Extensible dispatch: #name"
(p "User-defined reader macros via " (code "#name expr") ". The parser reads an identifier after " (code "#") ", looks up a handler in the reader macro registry, and calls it with the next parsed expression. See the " (a :href "/sx/(etc.(plan.reader-macro-demo))" :class "text-violet-600 hover:underline" "#z3 demo") " for a working example that translates SX spec declarations to SMT-LIB.")))
@@ -35,35 +35,35 @@
;; Implementation
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(~doc-subsection :title "1. Spec: parser.sx"
(~docs/subsection :title "1. Spec: parser.sx"
(p "Add " (code "#") " dispatch to " (code "read-expr") " (after the " (code ",") "/" (code ",@") " case, before number). Add " (code "read-raw-string") " helper function.")
(~doc-code :code (highlight ";; Reader macro dispatch\n(= ch \"#\")\n (do (set! pos (inc pos))\n (if (>= pos len-src)\n (error \"Unexpected end of input after #\")\n (let ((dispatch-ch (nth source pos)))\n (cond\n ;; #; — datum comment: read and discard next expr\n (= dispatch-ch \";\")\n (do (set! pos (inc pos))\n (read-expr) ;; read and discard\n (read-expr)) ;; return the NEXT expr\n\n ;; #| — raw string\n (= dispatch-ch \"|\")\n (do (set! pos (inc pos))\n (read-raw-string))\n\n ;; #' — quote shorthand\n (= dispatch-ch \"'\")\n (do (set! pos (inc pos))\n (list (make-symbol \"quote\") (read-expr)))\n\n :else\n (error (str \"Unknown reader macro: #\" dispatch-ch))))))" "lisp"))
(~docs/code :code (highlight ";; Reader macro dispatch\n(= ch \"#\")\n (do (set! pos (inc pos))\n (if (>= pos len-src)\n (error \"Unexpected end of input after #\")\n (let ((dispatch-ch (nth source pos)))\n (cond\n ;; #; — datum comment: read and discard next expr\n (= dispatch-ch \";\")\n (do (set! pos (inc pos))\n (read-expr) ;; read and discard\n (read-expr)) ;; return the NEXT expr\n\n ;; #| — raw string\n (= dispatch-ch \"|\")\n (do (set! pos (inc pos))\n (read-raw-string))\n\n ;; #' — quote shorthand\n (= dispatch-ch \"'\")\n (do (set! pos (inc pos))\n (list (make-symbol \"quote\") (read-expr)))\n\n :else\n (error (str \"Unknown reader macro: #\" dispatch-ch))))))" "lisp"))
(p "The " (code "read-raw-string") " helper:")
(~doc-code :code (highlight "(define read-raw-string\n (fn ()\n (let ((buf \"\"))\n (define raw-loop\n (fn ()\n (if (>= pos len-src)\n (error \"Unterminated raw string\")\n (let ((ch (nth source pos)))\n (if (= ch \"|\")\n (do (set! pos (inc pos)) nil) ;; done\n (do (set! buf (str buf ch))\n (set! pos (inc pos))\n (raw-loop)))))))\n (raw-loop)\n buf)))" "lisp")))
(~docs/code :code (highlight "(define read-raw-string\n (fn ()\n (let ((buf \"\"))\n (define raw-loop\n (fn ()\n (if (>= pos len-src)\n (error \"Unterminated raw string\")\n (let ((ch (nth source pos)))\n (if (= ch \"|\")\n (do (set! pos (inc pos)) nil) ;; done\n (do (set! buf (str buf ch))\n (set! pos (inc pos))\n (raw-loop)))))))\n (raw-loop)\n buf)))" "lisp")))
(~doc-subsection :title "2. Python: parser.py"
(~docs/subsection :title "2. Python: parser.py"
(p "Add " (code "#") " dispatch to " (code "_parse_expr()") " (after " (code ",") "/" (code ",@") " handling ~line 252). Add " (code "_read_raw_string()") " method to Tokenizer.")
(~doc-code :code (highlight "# In _parse_expr(), after the comma/splice-unquote block:\nif raw == \"#\":\n tok._advance(1) # consume the #\n if tok.pos >= len(tok.text):\n raise ParseError(\"Unexpected end of input after #\",\n tok.pos, tok.line, tok.col)\n dispatch = tok.text[tok.pos]\n if dispatch == \";\":\n tok._advance(1)\n _parse_expr(tok) # read and discard\n return _parse_expr(tok) # return next\n if dispatch == \"|\":\n tok._advance(1)\n return tok._read_raw_string()\n if dispatch == \"'\":\n tok._advance(1)\n return [Symbol(\"quote\"), _parse_expr(tok)]\n raise ParseError(f\"Unknown reader macro: #{dispatch}\",\n tok.pos, tok.line, tok.col)" "python"))
(~docs/code :code (highlight "# In _parse_expr(), after the comma/splice-unquote block:\nif raw == \"#\":\n tok._advance(1) # consume the #\n if tok.pos >= len(tok.text):\n raise ParseError(\"Unexpected end of input after #\",\n tok.pos, tok.line, tok.col)\n dispatch = tok.text[tok.pos]\n if dispatch == \";\":\n tok._advance(1)\n _parse_expr(tok) # read and discard\n return _parse_expr(tok) # return next\n if dispatch == \"|\":\n tok._advance(1)\n return tok._read_raw_string()\n if dispatch == \"'\":\n tok._advance(1)\n return [Symbol(\"quote\"), _parse_expr(tok)]\n raise ParseError(f\"Unknown reader macro: #{dispatch}\",\n tok.pos, tok.line, tok.col)" "python"))
(p "The " (code "_read_raw_string()") " method on Tokenizer:")
(~doc-code :code (highlight "def _read_raw_string(self) -> str:\n buf = []\n while self.pos < len(self.text):\n ch = self.text[self.pos]\n if ch == \"|\":\n self._advance(1)\n return \"\".join(buf)\n buf.append(ch)\n self._advance(1)\n raise ParseError(\"Unterminated raw string\",\n self.pos, self.line, self.col)" "python")))
(~docs/code :code (highlight "def _read_raw_string(self) -> str:\n buf = []\n while self.pos < len(self.text):\n ch = self.text[self.pos]\n if ch == \"|\":\n self._advance(1)\n return \"\".join(buf)\n buf.append(ch)\n self._advance(1)\n raise ParseError(\"Unterminated raw string\",\n self.pos, self.line, self.col)" "python")))
(~doc-subsection :title "3. JS: auto-transpiled"
(~docs/subsection :title "3. JS: auto-transpiled"
(p "JS parser comes from bootstrap of parser.sx — spec change handles it automatically."))
(~doc-subsection :title "4. Rebootstrap both targets"
(~docs/subsection :title "4. Rebootstrap both targets"
(p "Run " (code "bootstrap_js.py") " and " (code "bootstrap_py.py") " to regenerate " (code "sx-ref.js") " and " (code "sx_ref.py") " from the updated parser.sx spec."))
(~doc-subsection :title "5. Grammar update"
(~docs/subsection :title "5. Grammar update"
(p "Add reader macro syntax to the grammar comment at the top of parser.sx:")
(~doc-code :code (highlight ";; reader → '#;' expr (datum comment)\n;; | '#|' raw-chars '|' (raw string)\n;; | \"#'\" expr (quote shorthand)" "lisp"))))
(~docs/code :code (highlight ";; reader → '#;' expr (datum comment)\n;; | '#|' raw-chars '|' (raw string)\n;; | \"#'\" expr (quote shorthand)" "lisp"))))
;; -----------------------------------------------------------------------
;; Files
;; -----------------------------------------------------------------------
(~doc-section :title "Files" :id "files"
(~docs/section :title "Files" :id "files"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -87,9 +87,9 @@
;; Verification
;; -----------------------------------------------------------------------
(~doc-section :title "Verification" :id "verification"
(~docs/section :title "Verification" :id "verification"
(~doc-subsection :title "Parse tests"
(~docs/subsection :title "Parse tests"
(ul :class "list-disc pl-5 text-stone-700 space-y-1 font-mono text-sm"
(li "#;(ignored) 42 → 42")
(li "(list 1 #;2 3) → (list 1 3)")
@@ -99,7 +99,7 @@
(li "# at EOF → error")
(li "#x unknown → error")))
(~doc-subsection :title "Regression"
(~docs/subsection :title "Regression"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "All existing parser tests pass after changes")
(li "Rebootstrapped JS and Python pass their test suites")

44
sx/sx/plans/runtime-slicing.sx
View File

@@ -2,10 +2,10 @@
;; Runtime Slicing
;; ---------------------------------------------------------------------------
(defcomp ~plan-runtime-slicing-content ()
(~doc-page :title "Runtime Slicing"
(defcomp ~plans/runtime-slicing/plan-runtime-slicing-content ()
(~docs/page :title "Runtime Slicing"
(~doc-section :title "The Problem" :id "problem"
(~docs/section :title "The Problem" :id "problem"
(p "sx-browser.js is the full SX client runtime — evaluator, parser, renderer, engine, morph, signals, routing, orchestration, boot. Every page loads all of it.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -39,7 +39,7 @@
;; Tiers
;; -----------------------------------------------------------------------
(~doc-section :title "Tiers" :id "tiers"
(~docs/section :title "Tiers" :id "tiers"
(p "Four tiers, matching the " (a :href "/sx/(geography.(reactive.plan))" :class "text-violet-700 underline" "reactive islands") " levels:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -81,11 +81,11 @@
;; The slicer is SX
;; -----------------------------------------------------------------------
(~doc-section :title "The Slicer is SX" :id "slicer-is-sx"
(~docs/section :title "The Slicer is SX" :id "slicer-is-sx"
(p "Per the " (a :href "/sx/(etc.(plan.self-hosting-bootstrapper))" :class "text-violet-700 underline" "self-hosting principle") ", the slicer is not a build tool — it's a spec module. " (code "slice.sx") " analyzes the spec's own dependency graph and determines which defines belong to which tier.")
(p (code "js.sx") " (the self-hosting SX-to-JavaScript translator) already compiles the full spec. Slicing is a filter: " (code "js.sx") " translates only the defines that " (code "slice.sx") " selects for a given tier.")
(~doc-code :code (highlight ";; slice.sx — determine which defines each tier needs\n;;\n;; Input: the full list of defines from all spec files\n;; Output: a filtered list for the requested tier\n\n(define tier-deps\n ;; Which spec modules each tier requires\n {:L0 (list \"engine\" \"boot-partial\")\n :L1 (list \"engine\" \"boot-partial\" \"dom-partial\")\n :L2 (list \"engine\" \"boot-partial\" \"dom-partial\"\n \"signals\" \"dom-island\")\n :L3 (list \"eval\" \"render\" \"parser\"\n \"engine\" \"orchestration\" \"boot\"\n \"dom\" \"signals\" \"router\")})\n\n(define slice-defines\n (fn (tier all-defines)\n ;; 1. Get the module list for this tier\n ;; 2. Walk each define's dependency references\n ;; 3. Include a define only if ALL its deps are\n ;; satisfiable within the tier's module set\n ;; 4. Return the filtered define list\n (let ((modules (get tier-deps tier)))\n (filter\n (fn (d) (tier-satisfies? modules (define-deps d)))\n all-defines))))" "lisp"))
(~docs/code :code (highlight ";; slice.sx — determine which defines each tier needs\n;;\n;; Input: the full list of defines from all spec files\n;; Output: a filtered list for the requested tier\n\n(define tier-deps\n ;; Which spec modules each tier requires\n {:L0 (list \"engine\" \"boot-partial\")\n :L1 (list \"engine\" \"boot-partial\" \"dom-partial\")\n :L2 (list \"engine\" \"boot-partial\" \"dom-partial\"\n \"signals\" \"dom-island\")\n :L3 (list \"eval\" \"render\" \"parser\"\n \"engine\" \"orchestration\" \"boot\"\n \"dom\" \"signals\" \"router\")})\n\n(define slice-defines\n (fn (tier all-defines)\n ;; 1. Get the module list for this tier\n ;; 2. Walk each define's dependency references\n ;; 3. Include a define only if ALL its deps are\n ;; satisfiable within the tier's module set\n ;; 4. Return the filtered define list\n (let ((modules (get tier-deps tier)))\n (filter\n (fn (d) (tier-satisfies? modules (define-deps d)))\n all-defines))))" "lisp"))
(p "The pipeline becomes:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
@@ -100,10 +100,10 @@
;; Dependency analysis
;; -----------------------------------------------------------------------
(~doc-section :title "Define-Level Dependency Analysis" :id "deps"
(~docs/section :title "Define-Level Dependency Analysis" :id "deps"
(p "The slicer needs to know which defines reference which other defines. This is a simpler version of what " (code "deps.sx") " does for components — but at the define level, not the component level.")
(~doc-code :code (highlight ";; Walk a define's body AST, collect all symbol references\n;; that resolve to other top-level defines.\n;;\n;; (define morph-children\n;; (fn (old-node new-node)\n;; ...uses morph-node, morph-attrs, create-element...))\n;;\n;; → deps: #{morph-node morph-attrs create-element}\n\n(define define-refs\n (fn (body all-define-names)\n ;; Collect all symbols in body that appear in all-define-names\n (let ((refs (make-set)))\n (walk-ast body\n (fn (node)\n (when (and (symbol? node)\n (set-has? all-define-names (symbol-name node)))\n (set-add! refs (symbol-name node)))))\n refs)))" "lisp"))
(~docs/code :code (highlight ";; Walk a define's body AST, collect all symbol references\n;; that resolve to other top-level defines.\n;;\n;; (define morph-children\n;; (fn (old-node new-node)\n;; ...uses morph-node, morph-attrs, create-element...))\n;;\n;; → deps: #{morph-node morph-attrs create-element}\n\n(define define-refs\n (fn (body all-define-names)\n ;; Collect all symbols in body that appear in all-define-names\n (let ((refs (make-set)))\n (walk-ast body\n (fn (node)\n (when (and (symbol? node)\n (set-has? all-define-names (symbol-name node)))\n (set-add! refs (symbol-name node)))))\n refs)))" "lisp"))
(p "From these per-define deps, we build the full dependency graph and compute transitive closures. A tier's define set is the transitive closure of its entry points:")
@@ -135,11 +135,11 @@
;; Platform interface slicing
;; -----------------------------------------------------------------------
(~doc-section :title "Platform Interface Slicing" :id "platform"
(~docs/section :title "Platform Interface Slicing" :id "platform"
(p "The bootstrapped code (from js.sx) is only half the story. Each define also depends on platform primitives — the hand-written JS glue that js.sx doesn't produce. These live in " (code "bootstrap_js.py") "'s PLATFORM_* blocks.")
(p "The slicer must track platform deps too:")
(~doc-code :code (highlight ";; Platform primitives referenced by tier\n;;\n;; L0 needs: createElement, setAttribute, morphAttrs,\n;; fetch (for sx-get/post), pushState, replaceState\n;;\n;; L1 adds: classList.toggle, addEventListener\n;;\n;; L2 adds: createComment, createTextNode, domRemove,\n;; domChildNodes — the reactive DOM primitives\n;;\n;; L3 adds: everything in PLATFORM_PARSER_JS,\n;; full DOM adapter, async IO bridge\n\n(define platform-deps\n {:L0 (list :dom-morph :fetch :history)\n :L1 (list :dom-morph :fetch :history :dom-events)\n :L2 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors)\n :L3 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors\n :parser :evaluator :async-io)})" "lisp"))
(~docs/code :code (highlight ";; Platform primitives referenced by tier\n;;\n;; L0 needs: createElement, setAttribute, morphAttrs,\n;; fetch (for sx-get/post), pushState, replaceState\n;;\n;; L1 adds: classList.toggle, addEventListener\n;;\n;; L2 adds: createComment, createTextNode, domRemove,\n;; domChildNodes — the reactive DOM primitives\n;;\n;; L3 adds: everything in PLATFORM_PARSER_JS,\n;; full DOM adapter, async IO bridge\n\n(define platform-deps\n {:L0 (list :dom-morph :fetch :history)\n :L1 (list :dom-morph :fetch :history :dom-events)\n :L2 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors)\n :L3 (list :dom-morph :fetch :history :dom-events\n :dom-reactive :signal-constructors\n :parser :evaluator :async-io)})" "lisp"))
(p "The platform JS blocks in " (code "bootstrap_js.py") " are already organized by adapter (" (code "PLATFORM_DOM_JS") ", " (code "PLATFORM_ENGINE_PURE_JS") ", etc). Slicing further subdivides these into the minimal set each tier needs.")
(p "This subdivision also happens in SX: " (code "slice.sx") " declares which platform blocks each tier requires. " (code "js.sx") " doesn't need to change — it translates defines. The bootstrapper script reads the slice spec and assembles the platform preamble accordingly."))
@@ -148,18 +148,18 @@
;; Progressive loading
;; -----------------------------------------------------------------------
(~doc-section :title "Progressive Loading" :id "progressive"
(~docs/section :title "Progressive Loading" :id "progressive"
(p "The simplest approach: one file per tier. The server knows each page's tier (from " (code "defpage") " metadata or component analysis) and serves the right script tag.")
(p "Better: a base file (L0) that all pages load, plus tier deltas loaded on demand.")
(~doc-code :code (highlight ";; Server emits the appropriate script for the page's tier\n;;\n;; L0 page (blog post, product listing):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;;\n;; L2 page (reactive island):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;; <script src=\"/static/scripts/sx-L2-delta.js\"></script>\n;;\n;; Client-side navigation from L0 → L2:\n;; 1. L0 runtime handles the swap\n;; 2. New page declares tier=L2 in response header\n;; 3. L0 runtime loads sx-L2-delta.js dynamically\n;; 4. Island hydration proceeds\n\n(define page-tier\n (fn (page)\n ;; Analyze the page's component tree\n ;; If any component is defisland → L2\n ;; If any component uses on-event/toggle! → L1\n ;; Otherwise → L0\n (cond\n ((page-has-islands? page) :L2)\n ((page-has-dom-ops? page) :L1)\n (true :L0))))" "lisp"))
(~docs/code :code (highlight ";; Server emits the appropriate script for the page's tier\n;;\n;; L0 page (blog post, product listing):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;;\n;; L2 page (reactive island):\n;; <script src=\"/static/scripts/sx-L0.js\"></script>\n;; <script src=\"/static/scripts/sx-L2-delta.js\"></script>\n;;\n;; Client-side navigation from L0 → L2:\n;; 1. L0 runtime handles the swap\n;; 2. New page declares tier=L2 in response header\n;; 3. L0 runtime loads sx-L2-delta.js dynamically\n;; 4. Island hydration proceeds\n\n(define page-tier\n (fn (page)\n ;; Analyze the page's component tree\n ;; If any component is defisland → L2\n ;; If any component uses on-event/toggle! → L1\n ;; Otherwise → L0\n (cond\n ((page-has-islands? page) :L2)\n ((page-has-dom-ops? page) :L1)\n (true :L0))))" "lisp"))
(~doc-subsection :title "SX-Tier Response Header"
(~docs/subsection :title "SX-Tier Response Header"
(p "The server includes the page's tier in the response:")
(~doc-code :code (highlight "HTTP/1.1 200 OK\nSX-Tier: L0\nSX-Components: ~card:bafy...,~nav:bafy...\n\n;; or for an island page:\nSX-Tier: L2\nSX-Components: ~counter-island:bafy..." "http"))
(~docs/code :code (highlight "HTTP/1.1 200 OK\nSX-Tier: L0\nSX-Components: ~card:bafy...,~plans/environment-images/nav:bafy...\n\n;; or for an island page:\nSX-Tier: L2\nSX-Components: ~counter-island:bafy..." "http"))
(p "On client-side navigation, the engine reads " (code "SX-Tier") " from the response. If the new page requires a higher tier than currently loaded, it fetches the delta script before processing the swap. The delta script registers its additional primitives and the swap proceeds."))
(~doc-subsection :title "Cache Behavior"
(~docs/subsection :title "Cache Behavior"
(p "Each tier file is content-hashed (like the current " (code "sx_js_hash") " mechanism). Cache-forever semantics. A user who visits any L0 page caches the L0 runtime permanently. If they later visit an L2 page, only the ~10KB delta downloads.")
(p "Combined with " (a :href "/sx/(etc.(plan.environment-images))" :class "text-violet-700 underline" "environment images") ": the image CID includes the tier. An L0 image is smaller than an L3 image — it contains fewer primitives, no parser state, no evaluator. The standalone HTML bundle for an L0 page is tiny.")))
@@ -167,10 +167,10 @@
;; Automatic tier detection
;; -----------------------------------------------------------------------
(~doc-section :title "Automatic Tier Detection" :id "auto-detect"
(~docs/section :title "Automatic Tier Detection" :id "auto-detect"
(p (code "deps.sx") " already classifies components as pure or IO-dependent. Extend it to classify pages by tier:")
(~doc-code :code (highlight ";; Extend deps.sx with tier analysis\n;;\n;; Walk the page's component tree:\n;; - Any defisland → L2 minimum\n;; - Any on-event, toggle!, set-attr! call → L1 minimum \n;; - Any client-eval'd component (SX wire + defcomp) → L3\n;; - Otherwise → L0\n;;\n;; The tier is the MAX of all components' requirements.\n\n(define component-tier\n (fn (comp)\n (cond\n ((island? comp) :L2)\n ((has-dom-ops? (component-body comp)) :L1)\n (true :L0))))\n\n(define page-tier\n (fn (page-def)\n (let ((comp-tiers (map component-tier\n (page-all-components page-def))))\n (max-tier comp-tiers))))" "lisp"))
(~docs/code :code (highlight ";; Extend deps.sx with tier analysis\n;;\n;; Walk the page's component tree:\n;; - Any defisland → L2 minimum\n;; - Any on-event, toggle!, set-attr! call → L1 minimum \n;; - Any client-eval'd component (SX wire + defcomp) → L3\n;; - Otherwise → L0\n;;\n;; The tier is the MAX of all components' requirements.\n\n(define component-tier\n (fn (comp)\n (cond\n ((island? comp) :L2)\n ((has-dom-ops? (component-body comp)) :L1)\n (true :L0))))\n\n(define page-tier\n (fn (page-def)\n (let ((comp-tiers (map component-tier\n (page-all-components page-def))))\n (max-tier comp-tiers))))" "lisp"))
(p "This runs at registration time (same phase as " (code "compute_all_deps") "). Each " (code "PageDef") " gains a " (code "tier") " field. The server uses it to select the script tag. No manual annotation needed — the tier is derived from what the page actually uses."))
@@ -178,7 +178,7 @@
;; What L0 actually needs
;; -----------------------------------------------------------------------
(~doc-section :title "What L0 Actually Needs" :id "l0-detail"
(~docs/section :title "What L0 Actually Needs" :id "l0-detail"
(p "L0 is the critical tier — it's what most pages load. Every byte matters. Let's be precise about what it contains:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -233,10 +233,10 @@
;; Build pipeline
;; -----------------------------------------------------------------------
(~doc-section :title "Build Pipeline" :id "pipeline"
(~docs/section :title "Build Pipeline" :id "pipeline"
(p "The pipeline uses the same tools that already exist — " (code "js.sx") " for translation, " (code "bootstrap_js.py") " for platform assembly — but feeds them filtered define lists.")
(~doc-code :code (highlight ";; Build all tiers\n;;\n;; 1. Load all spec .sx files\n;; 2. Extract all defines (same as current bootstrap)\n;; 3. Run slice.sx to partition defines by tier\n;; 4. For each tier:\n;; a. js.sx translates the tier's define list\n;; b. Platform assembler wraps with minimal platform JS\n;; c. Output: sx-L{n}.js\n;; 5. Compute deltas: L1-delta = L1 - L0, L2-delta = L2 - L1, etc.\n\n;; The bootstrapper script orchestrates this:\n;;\n;; python bootstrap_js.py --tier L0 -o sx-L0.js\n;; python bootstrap_js.py --tier L1 --delta -o sx-L1-delta.js\n;; python bootstrap_js.py --tier L2 --delta -o sx-L2-delta.js\n;; python bootstrap_js.py -o sx-browser.js # full (L3, backward compat)" "lisp"))
(~docs/code :code (highlight ";; Build all tiers\n;;\n;; 1. Load all spec .sx files\n;; 2. Extract all defines (same as current bootstrap)\n;; 3. Run slice.sx to partition defines by tier\n;; 4. For each tier:\n;; a. js.sx translates the tier's define list\n;; b. Platform assembler wraps with minimal platform JS\n;; c. Output: sx-L{n}.js\n;; 5. Compute deltas: L1-delta = L1 - L0, L2-delta = L2 - L1, etc.\n\n;; The bootstrapper script orchestrates this:\n;;\n;; python bootstrap_js.py --tier L0 -o sx-L0.js\n;; python bootstrap_js.py --tier L1 --delta -o sx-L1-delta.js\n;; python bootstrap_js.py --tier L2 --delta -o sx-L2-delta.js\n;; python bootstrap_js.py -o sx-browser.js # full (L3, backward compat)" "lisp"))
(p "The " (code "--delta") " flag emits only the defines not present in the previous tier. The delta file calls " (code "Sx.extend()") " to register its additions into the already-loaded runtime.")
@@ -248,7 +248,7 @@
;; Spec modules
;; -----------------------------------------------------------------------
(~doc-section :title "Spec Modules" :id "spec-modules"
(~docs/section :title "Spec Modules" :id "spec-modules"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -271,7 +271,7 @@
;; Relationships
;; -----------------------------------------------------------------------
(~doc-section :title "Relationships" :id "relationships"
(~docs/section :title "Relationships" :id "relationships"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (a :href "/sx/(etc.(plan.environment-images))" :class "text-violet-700 underline" "Environment Images") " — tiered images are smaller. An L0 image omits the parser, evaluator, and most primitives.")
(li (a :href "/sx/(etc.(plan.content-addressed-components))" :class "text-violet-700 underline" "Content-Addressed Components") " — component CID resolution is L3-only. L0 pages don't resolve components client-side.")

30
sx/sx/plans/self-hosting-bootstrapper.sx
View File

@@ -3,8 +3,8 @@
;; ---------------------------------------------------------------------------
;; @css bg-green-100 text-green-800 bg-green-50 border-green-200 text-green-700 text-green-600
(defcomp ~plan-self-hosting-bootstrapper-content ()
(~doc-page :title "Self-Hosting Bootstrapper"
(defcomp ~plans/self-hosting-bootstrapper/plan-self-hosting-bootstrapper-content ()
(~docs/page :title "Self-Hosting Bootstrapper"
;; -----------------------------------------------------------------------
;; Status banner
@@ -24,7 +24,7 @@
;; The Idea
;; -----------------------------------------------------------------------
(~doc-section :title "The Idea" :id "idea"
(~docs/section :title "The Idea" :id "idea"
(p "We have " (code "bootstrap_py.py") " — a Python program that reads "
(code ".sx") " spec files and emits " (code "sx_ref.py")
", a standalone Python evaluator. It's a compiler written in the host language.")
@@ -42,7 +42,7 @@
;; Results
;; -----------------------------------------------------------------------
(~doc-section :title "Results" :id "results"
(~docs/section :title "Results" :id "results"
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
(thead :class "bg-stone-50"
@@ -73,7 +73,7 @@
;; Architecture
;; -----------------------------------------------------------------------
(~doc-section :title "Architecture" :id "architecture"
(~docs/section :title "Architecture" :id "architecture"
(p "Three bootstrapper generations:")
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
@@ -112,9 +112,9 @@
;; Translation Rules
;; -----------------------------------------------------------------------
(~doc-section :title "Translation Rules" :id "translation"
(~docs/section :title "Translation Rules" :id "translation"
(~doc-subsection :title "Name Mangling"
(~docs/subsection :title "Name Mangling"
(p "SX identifiers become valid Python identifiers. "
"The RENAMES dict (200+ entries) handles explicit mappings; "
"general rules handle the rest:")
@@ -147,7 +147,7 @@
(td :class "px-4 py-2 font-mono" "type_")
(td :class "px-4 py-2" "Python reserved word escape"))))))
(~doc-subsection :title "Special Forms"
(~docs/subsection :title "Special Forms"
(p "Each SX special form maps to a Python expression pattern:")
(div :class "overflow-x-auto rounded border border-stone-200 my-4"
(table :class "w-full text-sm"
@@ -172,11 +172,11 @@
(td :class "px-4 py-2 font-mono" "(case x \"a\" 1)")
(td :class "px-4 py-2 font-mono" "_sx_case(x, [(\"a\", lambda: 1)])"))))))
(~doc-subsection :title "Mutation: set! and Cell Variables"
(~docs/subsection :title "Mutation: set! and Cell Variables"
(p "Python closures can read but not rebind outer variables. "
(code "py.sx") " detects " (code "set!") " targets that cross lambda boundaries "
"and routes them through a " (code "_cells") " dict:")
(~doc-code :code (highlight ";; SX ;; Python
(~docs/code :code (highlight ";; SX ;; Python
(define counter def counter():
(fn () _cells = {}
(let ((n 0)) _cells['n'] = 0
@@ -192,7 +192,7 @@
;; Scope
;; -----------------------------------------------------------------------
(~doc-section :title "Scope" :id "scope"
(~docs/section :title "Scope" :id "scope"
(p (code "py.sx") " is a general-purpose SX-to-Python translator. "
"The translation rules are mechanical and apply to " (em "all") " SX, "
"not just the spec subset.")
@@ -214,19 +214,19 @@
;; Implications
;; -----------------------------------------------------------------------
(~doc-section :title "Implications" :id "implications"
(~doc-subsection :title "Practical"
(~docs/section :title "Implications" :id "implications"
(~docs/subsection :title "Practical"
(p "One less Python file to maintain. Changes to the transpilation logic "
"are written in SX and tested with the SX test harness. The spec and its "
"compiler live in the same language."))
(~doc-subsection :title "Architectural"
(~docs/subsection :title "Architectural"
(p "With " (code "z3.sx") " (SMT-LIB) and " (code "py.sx") " (Python), "
"the pattern is clear: SX translators are SX programs. "
"Adding a new target language means writing one " (code ".sx") " file, "
"not a new Python compiler."))
(~doc-subsection :title "Philosophical"
(~docs/subsection :title "Philosophical"
(p "A self-hosting bootstrapper is a fixed point. The spec defines behavior. "
"The translator is itself defined in terms of that behavior. Running the "
"translator on the spec produces a program that can run the translator on "

10
sx/sx/plans/social-sharing.sx
View File

@@ -2,15 +2,15 @@
;; Social Sharing
;; ---------------------------------------------------------------------------
(defcomp ~plan-social-sharing-content ()
(~doc-page :title "Social Network Sharing"
(defcomp ~plans/social-sharing/plan-social-sharing-content ()
(~docs/page :title "Social Network Sharing"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "Rose Ash already has ActivityPub for federated social sharing. This plan adds OAuth-based sharing to mainstream social networks — Facebook, Instagram, Threads, Twitter/X, LinkedIn, and Mastodon.")
(p "All social logic lives in the " (strong "account") " microservice. Content apps get a share button that opens the account share page."))
(~doc-section :title "What remains" :id "remains"
(~doc-note "Nothing has been implemented. This is the full scope of work.")
(~docs/section :title "What remains" :id "remains"
(~docs/note "Nothing has been implemented. This is the full scope of work.")
(div :class "space-y-4"

50
sx/sx/plans/spec-explorer.sx
View File

@@ -2,10 +2,10 @@
;; Spec Explorer — The Fifth Ring
;; ---------------------------------------------------------------------------
(defcomp ~plan-spec-explorer-content ()
(~doc-page :title "Spec Explorer"
(defcomp ~plans/spec-explorer/plan-spec-explorer-content ()
(~docs/page :title "Spec Explorer"
(~doc-section :title "The Five Rings" :id "five-rings"
(~docs/section :title "The Five Rings" :id "five-rings"
(p "SX has a peculiar architecture. At its centre sits a specification — a set of s-expression files that define the language. Not a description of the language. Not documentation about the language. The specification " (em "is") " the language. It is simultaneously a formal definition and executable code. You can read it as a document or run it as a program. It does not describe how to build an SX evaluator; it " (em "is") " an SX evaluator, expressed in the language it defines.")
(p "This is the nucleus. Everything else radiates outward from it.")
@@ -46,7 +46,7 @@
;; What the explorer shows
;; -----------------------------------------------------------------------
(~doc-section :title "Per-Function Cards" :id "cards"
(~docs/section :title "Per-Function Cards" :id "cards"
(p "Each function in the spec gets a card showing all five rings:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -86,7 +86,7 @@
;; Effect system
;; -----------------------------------------------------------------------
(~doc-section :title "Effect Annotations" :id "effects"
(~docs/section :title "Effect Annotations" :id "effects"
(p "Every function in the spec now carries an " (code ":effects") " annotation declaring what kind of side effects it performs:")
(div :class "flex flex-wrap gap-3 my-4"
@@ -97,68 +97,68 @@
(p "The explorer shows effect badges on each function card, and the stats bar aggregates them across the whole file. Pure functions (green) are the nucleus — no side effects, fully deterministic, safe to cache, reorder, or parallelise.")
(~doc-code :code (highlight "(define signal :effects []\n (fn ((initial-value :as any))\n (make-signal initial-value)))\n\n(define reset! :effects [mutation]\n (fn ((s :as signal) value)\n (when (signal? s)\n (let ((old (signal-value s)))\n (when (not (identical? old value))\n (signal-set-value! s value)\n (notify-subscribers s))))))" "sx")))
(~docs/code :code (highlight "(define signal :effects []\n (fn ((initial-value :as any))\n (make-signal initial-value)))\n\n(define reset! :effects [mutation]\n (fn ((s :as signal) value)\n (when (signal? s)\n (let ((old (signal-value s)))\n (when (not (identical? old value))\n (signal-set-value! s value)\n (notify-subscribers s))))))" "sx")))
;; -----------------------------------------------------------------------
;; Bootstrapper translations
;; -----------------------------------------------------------------------
(~doc-section :title "Bootstrapper Translations" :id "translations"
(~docs/section :title "Bootstrapper Translations" :id "translations"
(p "Each function is translated by the actual bootstrappers that build the production runtime. The same " (code "signal") " function shown in three target languages:")
(~doc-subsection :title "Python (via bootstrap_py.py)"
(~doc-code :code (highlight "def signal(initial_value):\n return make_signal(initial_value)" "python"))
(~docs/subsection :title "Python (via bootstrap_py.py)"
(~docs/code :code (highlight "def signal(initial_value):\n return make_signal(initial_value)" "python"))
(p :class "text-sm text-stone-500" (code "PyEmitter._emit_define()") " — the exact same code path that generates " (code "sx_ref.py") "."))
(~doc-subsection :title "JavaScript (via js.sx)"
(~doc-code :code (highlight "var signal = function(initial_value) {\n return make_signal(initial_value);\n};" "javascript"))
(~docs/subsection :title "JavaScript (via js.sx)"
(~docs/code :code (highlight "var signal = function(initial_value) {\n return make_signal(initial_value);\n};" "javascript"))
(p :class "text-sm text-stone-500" (code "js-emit-define") " — the self-hosting JS bootstrapper, written in SX, evaluated by the Python evaluator."))
(~doc-subsection :title "Z3 / SMT-LIB (via z3.sx)"
(~doc-code :code (highlight "; signal — Create a reactive signal container with an initial value.\n(declare-fun signal (Value) Value)" "lisp"))
(~docs/subsection :title "Z3 / SMT-LIB (via z3.sx)"
(~docs/code :code (highlight "; signal — Create a reactive signal container with an initial value.\n(declare-fun signal (Value) Value)" "lisp"))
(p :class "text-sm text-stone-500" (code "z3-translate") " — the first self-hosted bootstrapper, translating spec declarations to verification conditions for theorem provers.")))
;; -----------------------------------------------------------------------
;; Testing and proving
;; -----------------------------------------------------------------------
(~doc-section :title "Tests and Proofs" :id "runtime"
(~docs/section :title "Tests and Proofs" :id "runtime"
(p "Ring 4 shows that the spec does what it claims.")
(~doc-subsection :title "Tests"
(~docs/subsection :title "Tests"
(p "Test files (" (code "test-signals.sx") ", " (code "test-eval.sx") ", etc.) use the " (code "defsuite") "/" (code "deftest") " framework. The explorer matches tests to functions by suite name and shows them on the function card.")
(~doc-code :code (highlight "(defsuite \"signal basics\"\n (deftest \"creates signal with value\"\n (let ((s (signal 42)))\n (assert-equal (deref s) 42)))\n (deftest \"reset changes value\"\n (let ((s (signal 0)))\n (reset! s 99)\n (assert-equal (deref s) 99))))" "sx")))
(~docs/code :code (highlight "(defsuite \"signal basics\"\n (deftest \"creates signal with value\"\n (let ((s (signal 42)))\n (assert-equal (deref s) 42)))\n (deftest \"reset changes value\"\n (let ((s (signal 0)))\n (reset! s 99)\n (assert-equal (deref s) 99))))" "sx")))
(~doc-subsection :title "Proofs"
(~docs/subsection :title "Proofs"
(p (code "prove.sx") " verifies algebraic properties of SX primitives by bounded model checking. For each " (code "define-primitive") " with a " (code ":body") ", " (code "prove-translate") " translates to SMT-LIB and verifies satisfiability by construction.")
(p "Properties from the " (code "sx-properties") " library are matched to functions and shown on their cards:")
(~doc-code :code (highlight ";; prove.sx property: +-commutative\n{:name \"+-commutative\"\n :vars (list \"a\" \"b\")\n :test (fn (a b) (= (+ a b) (+ b a)))\n :holds '(= (+ a b) (+ b a))}\n\n;; Result: verified — 1,681 ground instances tested" "sx"))))
(~docs/code :code (highlight ";; prove.sx property: +-commutative\n{:name \"+-commutative\"\n :vars (list \"a\" \"b\")\n :test (fn (a b) (= (+ a b) (+ b a)))\n :holds '(= (+ a b) (+ b a))}\n\n;; Result: verified — 1,681 ground instances tested" "sx"))))
;; -----------------------------------------------------------------------
;; Architecture
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(p "Three layers, three increments.")
(~doc-subsection :title "Layer 1: Python helper"
(~docs/subsection :title "Layer 1: Python helper"
(p (code "spec-explorer-data(slug)") " in " (code "helpers.py") " — parses a " (code ".sx") " spec file via " (code "parse_all()") ", extracts sections/defines/effects/params, calls each bootstrapper for per-function translations, matches tests, runs proofs.")
(p "This is the only new Python code. Everything else is SX components."))
(~doc-subsection :title "Layer 2: SX components"
(~docs/subsection :title "Layer 2: SX components"
(p (code "specs-explorer.sx") " — 12-15 " (code "defcomp") " components rendering the structured data:")
(div :class "overflow-x-auto"
(pre :class "text-xs bg-stone-100 rounded p-3"
(code "~spec-explorer-content top-level, receives parsed data\n ~spec-explorer-header filename, title, source link\n ~spec-explorer-stats aggregate badges: effects, tests, proofs\n ~spec-explorer-toc section table of contents\n ~spec-explorer-section one section with its defines\n ~spec-explorer-define one function card (all five rings)\n ~spec-effect-badge colored effect badge\n ~spec-param-list typed parameter list\n ~spec-ring-translations SX / Python / JS / Z3 panels\n ~spec-ring-bridge cross-references + platform deps\n ~spec-ring-runtime tests + proofs\n ~spec-ring-examples usage examples\n ~spec-platform-interface platform primitives table"))))
(code "~specs-explorer/spec-explorer-content top-level, receives parsed data\n ~specs-explorer/spec-explorer-header filename, title, source link\n ~specs-explorer/spec-explorer-stats aggregate badges: effects, tests, proofs\n ~spec-explorer-toc section table of contents\n ~specs-explorer/spec-explorer-section one section with its defines\n ~specs-explorer/spec-explorer-define one function card (all five rings)\n ~specs-explorer/spec-effect-badge colored effect badge\n ~specs-explorer/spec-param-list typed parameter list\n ~specs-explorer/spec-ring-translations SX / Python / JS / Z3 panels\n ~specs-explorer/spec-ring-bridge cross-references + platform deps\n ~specs-explorer/spec-ring-runtime tests + proofs\n ~spec-ring-examples usage examples\n ~specs-explorer/spec-platform-interface platform primitives table"))))
(~doc-subsection :title "Layer 3: Routing"
(~docs/subsection :title "Layer 3: Routing"
(p "New route at " (code "/language/specs/explore/<slug>") " — parallel to existing raw source at " (code "/language/specs/<slug>") ". Each spec page gets an \"Explore\" link; the explorer gets a \"Source\" link.")))
;; -----------------------------------------------------------------------
;; Increments
;; -----------------------------------------------------------------------
(~doc-section :title "Incremental Delivery" :id "increments"
(~docs/section :title "Incremental Delivery" :id "increments"
(div :class "space-y-4"
(div :class "rounded border border-stone-200 p-4"
@@ -201,7 +201,7 @@
;; The strange loop
;; -----------------------------------------------------------------------
(~doc-section :title "The Strange Loop" :id "strange-loop"
(~docs/section :title "The Strange Loop" :id "strange-loop"
(p "When you view " (code "/language/specs/explore/eval") ", what happens is this:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li "The SX evaluator — bootstrapped from " (code "eval.sx") " — evaluates the page definition.")

12
sx/sx/plans/status.sx
View File

@@ -2,8 +2,8 @@
;; Plan Status Overview
;; ---------------------------------------------------------------------------
(defcomp ~plan-status-content ()
(~doc-page :title "Plan Status"
(defcomp ~plans/status/plan-status-content ()
(~docs/page :title "Plan Status"
(p :class "text-lg text-stone-600 mb-6"
"Audit of all plans across the SX language and Rose Ash platform. Last updated March 2026.")
@@ -12,7 +12,7 @@
;; Completed
;; -----------------------------------------------------------------------
(~doc-section :title "Completed" :id "completed"
(~docs/section :title "Completed" :id "completed"
(div :class "space-y-4"
@@ -74,7 +74,7 @@
;; In Progress / Partial
;; -----------------------------------------------------------------------
(~doc-section :title "In Progress" :id "in-progress"
(~docs/section :title "In Progress" :id "in-progress"
(div :class "space-y-4"
@@ -88,7 +88,7 @@
;; Not Started
;; -----------------------------------------------------------------------
(~doc-section :title "Not Started" :id "not-started"
(~docs/section :title "Not Started" :id "not-started"
(div :class "space-y-4"
@@ -124,7 +124,7 @@
(div :class "flex items-center gap-2 mb-1"
(span :class "inline-block px-2 py-0.5 rounded text-xs font-bold bg-green-600 text-white uppercase" "Complete")
(a :href "/sx/(geography.(isomorphism))" :class "font-semibold text-stone-800 underline" "Isomorphic Phase 6: Streaming & Suspense"))
(p :class "text-sm text-stone-600" "Server streams partially-evaluated SX as IO resolves. ~suspense component renders fallbacks, inline resolution scripts fill in content. Concurrent IO via asyncio, chunked transfer encoding.")
(p :class "text-sm text-stone-600" "Server streams partially-evaluated SX as IO resolves. ~shared:pages/suspense component renders fallbacks, inline resolution scripts fill in content. Concurrent IO via asyncio, chunked transfer encoding.")
(p :class "text-sm text-stone-500 mt-1" "Demo: " (a :href "/sx/(geography.(isomorphism.streaming))" "/sx/(geography.(isomorphism.streaming))")))
(div :class "rounded border border-green-300 bg-green-50 p-4"

106
sx/sx/plans/sx-activity.sx
View File

@@ -2,15 +2,15 @@
;; SX-Activity: Federated SX over ActivityPub
;; ---------------------------------------------------------------------------
(defcomp ~plan-sx-activity-content ()
(~doc-page :title "SX-Activity"
(defcomp ~plans/sx-activity/plan-sx-activity-content ()
(~docs/page :title "SX-Activity"
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p "The web is six incompatible formats duct-taped together: HTML for structure, CSS for style, JavaScript for behavior, JSON for data, server languages for backend logic, build tools for compilation. Moving anything between layers requires serialization, template languages, API contracts, and glue code. Federation (ActivityPub) adds a seventh — JSON-LD — which is inert data that every consumer must interpret from scratch and wrap in their own UI.")
(p "SX is already one evaluable format that does all six. A component definition is simultaneously structure, style (components apply classes and respond to data), behavior (event handlers), data (the AST " (em "is") " data), server-renderable (Python evaluator), and client-renderable (JS evaluator). The pieces already exist: content-addressed DAG execution (artdag), IPFS storage with CIDs, OpenTimestamps Bitcoin anchoring, boundary-enforced sandboxing.")
(p "SX-Activity wires these together into a new web. Everything — content, UI components, markdown parsers, syntax highlighters, validation logic, media, processing pipelines — is the same executable format, stored on a content-addressed network, running within each participant's own security context. " (strong "The wire format is the programming language is the component system is the package manager.")))
(~doc-section :title "Current State" :id "current-state"
(~docs/section :title "Current State" :id "current-state"
(ul :class "space-y-2 text-stone-700 list-disc pl-5"
(li (strong "ActivityPub: ") "Full implementation — virtual per-app actors, HTTP signatures, webfinger, inbox/outbox, followers/following, delivery with idempotent logging.")
(li (strong "Activity bus: ") "Unified event bus with NOTIFY/LISTEN wakeup, at-least-once delivery, handler registry keyed by (activity_type, object_type).")
@@ -23,28 +23,28 @@
;; Phase 1: SX Wire Format for Activities
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 1: SX Wire Format for Activities" :id "phase-1"
(~docs/section :title "Phase 1: SX Wire Format for Activities" :id "phase-1"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Activities expressed as s-expressions instead of JSON-LD. Same semantics as ActivityStreams, but compact, parseable, and directly evaluable. Dual-format support for backward compatibility with existing AP servers."))
(~doc-subsection :title "The Problem"
(~docs/subsection :title "The Problem"
(p "JSON-LD activities are verbose and require context resolution:")
(~doc-code :code (highlight "{\"@context\": \"https://www.w3.org/ns/activitystreams\",\n \"type\": \"Create\",\n \"actor\": \"https://example.com/users/alice\",\n \"object\": {\n \"type\": \"Note\",\n \"content\": \"<p>Hello world</p>\",\n \"attributedTo\": \"https://example.com/users/alice\"\n }}" "json"))
(~docs/code :code (highlight "{\"@context\": \"https://www.w3.org/ns/activitystreams\",\n \"type\": \"Create\",\n \"actor\": \"https://example.com/users/alice\",\n \"object\": {\n \"type\": \"Note\",\n \"content\": \"<p>Hello world</p>\",\n \"attributedTo\": \"https://example.com/users/alice\"\n }}" "json"))
(p "Every consumer parses JSON, resolves @context, extracts fields, then builds their own UI around the raw data. The content is HTML embedded in a JSON string — two formats nested, neither evaluable."))
(~doc-subsection :title "SX Activity Format"
(~docs/subsection :title "SX Activity Format"
(p "The same activity as SX:")
(~doc-code :code (highlight "(Create\n :actor \"https://example.com/users/alice\"\n :published \"2026-03-06T12:00:00Z\"\n :object (Note\n :attributed-to \"https://example.com/users/alice\"\n :content (p \"Hello world\")\n :media-type \"text/sx\"))" "lisp"))
(~docs/code :code (highlight "(Create\n :actor \"https://example.com/users/alice\"\n :published \"2026-03-06T12:00:00Z\"\n :object (Note\n :attributed-to \"https://example.com/users/alice\"\n :content (p \"Hello world\")\n :media-type \"text/sx\"))" "lisp"))
(p "The content isn't a string containing markup — it " (em "is") " markup. The receiving server can evaluate it directly. The Note's content is a renderable SX expression."))
(~doc-subsection :title "Approach"
(~docs/subsection :title "Approach"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Activity vocabulary in SX")
(p "Map ActivityStreams types to SX symbols. Activities are lists with a type head and keyword properties:")
(~doc-code :code (highlight ";; Core activity types\n(Create :actor ... :object ...)\n(Update :actor ... :object ...)\n(Delete :actor ... :object ...)\n(Follow :actor ... :object ...)\n(Like :actor ... :object ...)\n(Announce :actor ... :object ...)\n\n;; Object types\n(Note :content ... :attributed-to ...)\n(Article :name ... :content ... :summary ...)\n(Image :url ... :media-type ... :cid ...)\n(Collection :total-items ... :items ...)" "lisp")))
(~docs/code :code (highlight ";; Core activity types\n(Create :actor ... :object ...)\n(Update :actor ... :object ...)\n(Delete :actor ... :object ...)\n(Follow :actor ... :object ...)\n(Like :actor ... :object ...)\n(Announce :actor ... :object ...)\n\n;; Object types\n(Note :content ... :attributed-to ...)\n(Article :name ... :content ... :summary ...)\n(Image :url ... :media-type ... :cid ...)\n(Collection :total-items ... :items ...)" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. Content negotiation")
@@ -58,7 +58,7 @@
(h4 :class "font-semibold text-stone-700" "4. HTTP Signatures over SX")
(p "Same RSA signature mechanism. Digest header computed over the SX body. Existing keypair infrastructure unchanged."))))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Round-trip: SX → JSON-LD → SX produces identical output")
(li "Legacy AP servers receive valid JSON-LD (Mastodon can display the post)")
@@ -69,28 +69,28 @@
;; Phase 2: Content-Addressed Components on IPFS
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 2: Content-Addressed Components on IPFS" :id "phase-2"
(~docs/section :title "Phase 2: Content-Addressed Components on IPFS" :id "phase-2"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Component definitions stored on IPFS, referenced by CID. Any server can publish components. Any browser can fetch them. No central registry — content addressing IS the registry."))
(~doc-subsection :title "The Insight"
(~docs/subsection :title "The Insight"
(p "SX components are pure functions — they take data and return markup. They can't do IO (boundary enforcement guarantees this). That means they're " (strong "safe to load from any source") ". And if they're content-addressed, the CID " (em "is") " the identity — you don't need to trust the source, you just verify the hash.")
(p "Currently, component definitions travel with each page via " (code "<script type=\"text/sx\" data-components>") ". Each server bundles its own. With IPFS, components become shared infrastructure — define once, use everywhere."))
(~doc-subsection :title "Approach"
(~docs/subsection :title "Approach"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Component CID computation")
(p "Each " (code "defcomp") " definition gets a content address:")
(~doc-code :code (highlight ";; Component source\n(defcomp ~card (&key title &rest children)\n (div :class \"border rounded p-4\"\n (h2 title) children))\n\n;; CID = SHA3-256 of canonical serialized form\n;; → bafy...abc123\n;; Stored: ipfs://bafy...abc123 → component source" "lisp"))
(~docs/code :code (highlight ";; Component source\n(defcomp ~plans/sx-activity/card (&key title &rest children)\n (div :class \"border rounded p-4\"\n (h2 title) children))\n\n;; CID = SHA3-256 of canonical serialized form\n;; → bafy...abc123\n;; Stored: ipfs://bafy...abc123 → component source" "lisp"))
(p "Canonical form: normalize whitespace, sort keyword args alphabetically, strip comments. Same component always produces same CID regardless of formatting."))
(div
(h4 :class "font-semibold text-stone-700" "2. Component references in activities")
(p "Activities declare which components they need by CID:")
(~doc-code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :requires (list\n \"bafy...card\" ;; ~card component\n \"bafy...avatar\") ;; ~avatar component\n :object (Note\n :content (~card :title \"Hello\"\n (~avatar :src \"ipfs://bafy...photo\")\n (p \"This renders with the card component.\"))))" "lisp"))
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :requires (list\n \"bafy...card\" ;; ~plans/sx-activity/card component\n \"bafy...avatar\") ;; ~shared:misc/avatar component\n :object (Note\n :content (~plans/sx-activity/card :title \"Hello\"\n (~shared:misc/avatar :src \"ipfs://bafy...photo\")\n (p \"This renders with the card component.\"))))" "lisp"))
(p "The receiving browser fetches missing components from IPFS, verifies CIDs, registers them, then renders the content."))
(div
@@ -106,9 +106,9 @@
(div
(h4 :class "font-semibold text-stone-700" "4. Component publication")
(p "Server-side: on component registration, compute CID and pin to IPFS. Track in " (code "IPFSPin") " model (already exists). Publish component availability via AP outbox:")
(~doc-code :code (highlight "(Create\n :actor \"https://rose-ash.com/apps/market\"\n :object (SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :version \"1.0.0\"\n :deps (list \"bafy...card\" \"bafy...price-tag\")))" "lisp")))))
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/apps/market\"\n :object (SxComponent\n :name \"~product-card\"\n :cid \"bafy...productcard\"\n :version \"1.0.0\"\n :deps (list \"bafy...card\" \"bafy...price-tag\")))" "lisp")))))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Component pinned to IPFS → fetchable via gateway → CID verifies")
(li "Browser renders federated post using IPFS-fetched components")
@@ -119,32 +119,32 @@
;; Phase 3: Federated Media & Content Store
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 3: Federated Media & Content Store" :id "phase-3"
(~docs/section :title "Phase 3: Federated Media & Content Store" :id "phase-3"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "All media (images, video, audio, DAG outputs) stored content-addressed on IPFS. Activities reference media by CID. No hotlinking, no broken links, no dependence on the origin server staying online."))
(~doc-subsection :title "Current Mechanism"
(~docs/subsection :title "Current Mechanism"
(p "artdag already content-addresses all DAG outputs with SHA3-256 and tracks IPFS CIDs in " (code "IPFSPin") ". But media in the web platform (blog images, product photos, event banners) is stored as regular files on the origin server. Federated posts include " (code "url") " fields pointing to the origin — if the server goes down, the media is gone."))
(~doc-subsection :title "Approach"
(~docs/subsection :title "Approach"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Media CID pipeline")
(p "On upload: hash content → pin to IPFS → store CID in database. Activities reference media by CID alongside URL fallback:")
(~doc-code :code (highlight "(Image\n :cid \"bafy...photo123\"\n :url \"https://rose-ash.com/media/photo.jpg\" ;; fallback\n :media-type \"image/jpeg\"\n :width 1200 :height 800)" "lisp")))
(~docs/code :code (highlight "(Image\n :cid \"bafy...photo123\"\n :url \"https://rose-ash.com/media/photo.jpg\" ;; fallback\n :media-type \"image/jpeg\"\n :width 1200 :height 800)" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "2. DAG output federation")
(p "artdag processing results (rendered video, processed images) already have CIDs. Federate them as activities:")
(~doc-code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Artwork\n :name \"Sunset Remix\"\n :cid \"bafy...artwork\"\n :dag-cid \"bafy...dag\" ;; full DAG for reproduction\n :media-type \"video/mp4\"\n :sources (list\n (Image :cid \"bafy...src1\" :attribution \"...\")\n (Image :cid \"bafy...src2\" :attribution \"...\"))))" "lisp"))
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Artwork\n :name \"Sunset Remix\"\n :cid \"bafy...artwork\"\n :dag-cid \"bafy...dag\" ;; full DAG for reproduction\n :media-type \"video/mp4\"\n :sources (list\n (Image :cid \"bafy...src1\" :attribution \"...\")\n (Image :cid \"bafy...src2\" :attribution \"...\"))))" "lisp"))
(p "The " (code ":dag-cid") " lets anyone re-execute the processing pipeline. The artwork is both a result and a reproducible recipe."))
(div
(h4 :class "font-semibold text-stone-700" "3. Shared SX content store")
(p "Not just components and media — full page content can be content-addressed. An Article's body is SX, pinned to IPFS:")
(~doc-code :code (highlight "(Article\n :name \"Why S-Expressions\"\n :content-cid \"bafy...article-body\" ;; SX source on IPFS\n :requires (list \"bafy...doc-page\" \"bafy...code-block\")\n :summary \"Why SX uses s-expressions instead of HTML.\")" "lisp"))
(~docs/code :code (highlight "(Article\n :name \"Why S-Expressions\"\n :content-cid \"bafy...article-body\" ;; SX source on IPFS\n :requires (list \"bafy...doc-page\" \"bafy...code-block\")\n :summary \"Why SX uses s-expressions instead of HTML.\")" "lisp"))
(p "The content outlives the server. Anyone with the CID can fetch, parse, and render the article with its original components."))
(div
@@ -156,7 +156,7 @@
(li "Large media uses IPFS streaming (chunked CIDs)")
(li "Integrates with Phase 6 of isomorphic plan (streaming/suspense)")))))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Origin server offline → content still resolvable via IPFS gateway")
(li "DAG CID → re-executing DAG produces identical output")
@@ -166,23 +166,23 @@
;; Phase 4: Component Registry & Discovery
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 4: Component Registry & Discovery" :id "phase-4"
(~docs/section :title "Phase 4: Component Registry & Discovery" :id "phase-4"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Federated component discovery. Servers publish component collections. Other servers follow component feeds. Like npm, but federated, content-addressed, and the packages are safe to run (pure functions, no IO)."))
(~doc-subsection :title "Approach"
(~docs/subsection :title "Approach"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Component collections as AP actors")
(p "Each server exposes a component registry actor:")
(~doc-code :code (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(~docs/code :code (highlight "(Service\n :id \"https://rose-ash.com/sx-registry\"\n :type \"SxComponentRegistry\"\n :name \"Rose Ash Components\"\n :outbox \"https://rose-ash.com/sx-registry/outbox\"\n :followers \"https://rose-ash.com/sx-registry/followers\")" "lisp"))
(p "Follow the registry to receive component updates. The outbox is a chronological feed of Create/Update/Delete activities for components."))
(div
(h4 :class "font-semibold text-stone-700" "2. Component metadata")
(~doc-code :code (highlight "(SxComponent\n :name \"~data-table\"\n :cid \"bafy...datatable\"\n :version \"2.1.0\"\n :deps (list \"bafy...sortable\" \"bafy...paginator\")\n :params (list\n (dict :name \"rows\" :type \"list\" :required true)\n (dict :name \"columns\" :type \"list\" :required true)\n (dict :name \"sortable\" :type \"boolean\" :default false))\n :css-atoms (list :border :rounded :p-4 :text-sm)\n :preview-cid \"bafy...screenshot\"\n :license \"MIT\")" "lisp"))
(~docs/code :code (highlight "(SxComponent\n :name \"~data-table\"\n :cid \"bafy...datatable\"\n :version \"2.1.0\"\n :deps (list \"bafy...sortable\" \"bafy...paginator\")\n :params (list\n (dict :name \"rows\" :type \"list\" :required true)\n (dict :name \"columns\" :type \"list\" :required true)\n (dict :name \"sortable\" :type \"boolean\" :default false))\n :css-atoms (list :border :rounded :p-4 :text-sm)\n :preview-cid \"bafy...screenshot\"\n :license \"MIT\")" "lisp"))
(p "Dependencies are transitive CID references. CSS atoms declare which CSSX rules the component needs. Preview CID is a screenshot for registry browsing."))
(div
@@ -197,9 +197,9 @@
(div
(h4 :class "font-semibold text-stone-700" "4. Version resolution")
(p "Components are immutable (CID = identity). \"Updating\" a component publishes a new CID. Activities reference specific CIDs, so old content always renders correctly. The registry tracks version history:")
(~doc-code :code (highlight "(Update\n :actor \"https://rose-ash.com/sx-registry\"\n :object (SxComponent\n :name \"~card\"\n :cid \"bafy...card-v2\" ;; new version\n :replaces \"bafy...card-v1\" ;; previous version\n :changelog \"Added subtitle slot\"))" "lisp")))))
(~docs/code :code (highlight "(Update\n :actor \"https://rose-ash.com/sx-registry\"\n :object (SxComponent\n :name \"~card\"\n :cid \"bafy...card-v2\" ;; new version\n :replaces \"bafy...card-v1\" ;; previous version\n :changelog \"Added subtitle slot\"))" "lisp")))))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Follow registry → receive component Create activities → components available locally")
(li "Render post using component from foreign registry → works")
@@ -209,16 +209,16 @@
;; Phase 5: Bitcoin-Anchored Provenance
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 5: Bitcoin-Anchored Provenance" :id "phase-5"
(~docs/section :title "Phase 5: Bitcoin-Anchored Provenance" :id "phase-5"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What it enables")
(p :class "text-violet-800" "Cryptographic proof that content existed at a specific time, authored by a specific actor. Leverages the existing APAnchor/OpenTimestamps infrastructure. Unforgeable, independently verifiable, survives server shutdown."))
(~doc-subsection :title "Current Mechanism"
(~docs/subsection :title "Current Mechanism"
(p "The " (code "APAnchor") " model already batches activities into Merkle trees, stores the tree on IPFS, creates an OpenTimestamps proof, and records the Bitcoin txid. This runs but isn't surfaced to users or integrated with the full activity lifecycle."))
(~doc-subsection :title "Approach"
(~docs/subsection :title "Approach"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "1. Automatic anchoring pipeline")
@@ -232,7 +232,7 @@
(div
(h4 :class "font-semibold text-stone-700" "2. Provenance chain in activities")
(~doc-code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Note :content (p \"Hello\") :cid \"bafy...note\")\n :provenance (Anchor\n :tree-cid \"bafy...merkle-tree\"\n :leaf-index 42\n :ots-cid \"bafy...ots-proof\"\n :btc-txid \"abc123...def\"\n :btc-block 890123\n :anchored-at \"2026-03-06T12:00:00Z\"))" "lisp"))
(~docs/code :code (highlight "(Create\n :actor \"https://rose-ash.com/users/alice\"\n :object (Note :content (p \"Hello\") :cid \"bafy...note\")\n :provenance (Anchor\n :tree-cid \"bafy...merkle-tree\"\n :leaf-index 42\n :ots-cid \"bafy...ots-proof\"\n :btc-txid \"abc123...def\"\n :btc-block 890123\n :anchored-at \"2026-03-06T12:00:00Z\"))" "lisp"))
(p "Any party can verify: fetch the OTS proof from IPFS, check the Merkle path from the activity's CID to the tree root, confirm the tree root is committed in the Bitcoin block."))
(div
@@ -246,9 +246,9 @@
(div
(h4 :class "font-semibold text-stone-700" "4. Verification UI")
(p "Client-side provenance badge on federated content:")
(~doc-code :code (highlight "(defcomp ~provenance-badge (&key anchor)\n (when anchor\n (details :class \"inline text-xs text-stone-400\"\n (summary \"✓ Anchored\")\n (dl :class \"mt-1 space-y-1\"\n (dt \"Bitcoin block\") (dd (get anchor \"btc-block\"))\n (dt \"Timestamp\") (dd (get anchor \"anchored-at\"))\n (dt \"Proof\") (dd (a :href (str \"ipfs://\" (get anchor \"ots-cid\"))\n \"OTS proof\"))))))" "lisp")))))
(~docs/code :code (highlight "(defcomp ~plans/sx-activity/provenance-badge (&key anchor)\n (when anchor\n (details :class \"inline text-xs text-stone-400\"\n (summary \"✓ Anchored\")\n (dl :class \"mt-1 space-y-1\"\n (dt \"Bitcoin block\") (dd (get anchor \"btc-block\"))\n (dt \"Timestamp\") (dd (get anchor \"anchored-at\"))\n (dt \"Proof\") (dd (a :href (str \"ipfs://\" (get anchor \"ots-cid\"))\n \"OTS proof\"))))))" "lisp")))))
(~doc-subsection :title "Verification"
(~docs/subsection :title "Verification"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Activity anchored → OTS proof fetchable from IPFS → Merkle path validates → txid confirms in Bitcoin")
(li "Tampered activity → Merkle proof fails → provenance badge shows ✗")
@@ -258,18 +258,18 @@
;; Phase 6: The Evaluable Web
;; -----------------------------------------------------------------------
(~doc-section :title "Phase 6: The Evaluable Web" :id "phase-6"
(~docs/section :title "Phase 6: The Evaluable Web" :id "phase-6"
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mb-4"
(p :class "text-violet-900 font-medium" "What this really is")
(p :class "text-violet-800" "Not ActivityPub-with-SX. A new web. One where everything — content, components, parsers, renderers, server logic, client logic — is the same executable format, shared on a content-addressed network, running within each participant's own security context."))
(~doc-subsection :title "The insight"
(~docs/subsection :title "The insight"
(p "The web has six layers that don't talk to each other: HTML (structure), CSS (style), JavaScript (behavior), JSON (data interchange), server frameworks (backend logic), and build tools (compilation). Each has its own syntax, its own semantics, its own ecosystem. Moving data between them requires serialization, deserialization, template languages, API contracts, type coercion, and an endless parade of glue code.")
(p "SX collapses all six into one evaluable format. A component definition is simultaneously structure, style (components apply classes and respond to data), behavior (event handlers), data (the AST is data), server-renderable (Python evaluator), and client-renderable (JS evaluator). There is no boundary between \"data\" and \"program\" — s-expressions are both.")
(p "Once that's true, " (strong "everything becomes shareable.") " Not just UI components — markdown parsers, syntax highlighters, date formatters, validation logic, layout algorithms, color systems, animation curves. Any pure function over data. All content-addressed, all on IPFS, all executable within your own security context."))
(~doc-subsection :title "What travels on the network"
(~docs/subsection :title "What travels on the network"
(div :class "space-y-4"
(div
(h4 :class "font-semibold text-stone-700" "Content")
@@ -282,7 +282,7 @@
(div
(h4 :class "font-semibold text-stone-700" "Parsers and transforms")
(p "A markdown parser is just an SX function: takes a string, returns an SX tree. Publish it to IPFS. Now anyone can use your markdown dialect. Same for: syntax highlighters, BBCode parsers, wiki markup, LaTeX subsets, CSV-to-table converters, JSON-to-SX adapters. " (strong "The parser ecosystem becomes shared infrastructure."))
(~doc-code :code (highlight ";; A markdown parser, published to IPFS\n;; CID: bafy...md-parser\n(define parse-markdown\n (fn (source)\n ;; tokenize → build AST → return SX tree\n ;; (parse-markdown \"# Hello\\n**bold**\")\n ;; → (h1 \"Hello\") (p (strong \"bold\"))\n ...))\n\n;; Anyone can use it in their components\n(defcomp ~blog-post (&key markdown-source)\n (div :class \"prose\"\n (parse-markdown markdown-source)))" "lisp")))
(~docs/code :code (highlight ";; A markdown parser, published to IPFS\n;; CID: bafy...md-parser\n(define parse-markdown\n (fn (source)\n ;; tokenize → build AST → return SX tree\n ;; (parse-markdown \"# Hello\\n**bold**\")\n ;; → (h1 \"Hello\") (p (strong \"bold\"))\n ...))\n\n;; Anyone can use it in their components\n(defcomp ~plans/sx-activity/blog-post (&key markdown-source)\n (div :class \"prose\"\n (parse-markdown markdown-source)))" "lisp")))
(div
(h4 :class "font-semibold text-stone-700" "Server-side and client-side logic")
@@ -292,7 +292,7 @@
(h4 :class "font-semibold text-stone-700" "Media and processing pipelines")
(p "Images, video, audio — all content-addressed on IPFS. But also the " (em "processing pipelines") " that created them. artdag DAGs are SX. Publish a DAG CID alongside the output CID and anyone can verify the provenance, re-render at different resolution, or fork the pipeline for their own work."))))
(~doc-subsection :title "The security model"
(~docs/subsection :title "The security model"
(p "This only works because of boundary enforcement. Every piece of SX fetched from the network runs within the receiver's security context:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Pure functions can't do IO. ") "A component from IPFS can produce markup — it cannot read your cookies, make network requests, access localStorage, or call any IO primitive. The boundary spec (boundary.sx) is enforced at registration time. This isn't a policy — it's structural. The evaluator literally doesn't have IO primitives available when running untrusted code.")
@@ -302,7 +302,7 @@
(li (strong "Provenance proves authorship. ") "Bitcoin-anchored timestamps prove who published what and when. Not \"trust me\" — independently verifiable against the Bitcoin blockchain."))
(p "This is the opposite of the npm model. npm packages run with full access to your system — a malicious package can exfiltrate secrets, install backdoors, modify the filesystem. SX components are structurally sandboxed. The worst a malicious component can do is render a " (code "(div \"haha got you\")") "."))
(~doc-subsection :title "What this replaces"
(~docs/subsection :title "What this replaces"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -347,10 +347,10 @@
(td :class "px-3 py-2 text-stone-700" "IPFS CID")
(td :class "px-3 py-2 text-stone-600" "Entire applications are content-addressed, no infrastructure needed"))))))
(~doc-subsection :title "Serverless applications on IPFS"
(~docs/subsection :title "Serverless applications on IPFS"
(p "The logical conclusion: " (strong "entire web applications hosted on IPFS with no server at all."))
(p "An SX application is a tree of content-addressed artifacts: a root page definition, component dependencies, media, stylesheets, parsers, transforms. Pin the root CID to IPFS and the application is live. No server, no DNS, no hosting provider, no deployment pipeline. Someone gives you a CID, you paste it into an SX-aware browser, and the application runs.")
(~doc-code :code (highlight ";; An entire blog — one CID\n;; ipfs://bafy...my-blog\n(defpage blog-home\n :path \"/\"\n :requires (list\n \"bafy...article-layout\" ;; layout component\n \"bafy...md-parser\" ;; markdown parser\n \"bafy...syntax-highlight\" ;; code highlighting\n \"bafy...nav-component\") ;; navigation\n :content\n (~article-layout\n :title \"My Blog\"\n :nav (~nav-component\n :items (list\n (dict :label \"Post 1\" :cid \"bafy...post-1\")\n (dict :label \"Post 2\" :cid \"bafy...post-2\")))\n :body (~markdown-page\n :source-cid \"bafy...homepage-md\")))" "lisp"))
(~docs/code :code (highlight ";; An entire blog — one CID\n;; ipfs://bafy...my-blog\n(defpage blog-home\n :path \"/\"\n :requires (list\n \"bafy...article-layout\" ;; layout component\n \"bafy...md-parser\" ;; markdown parser\n \"bafy...syntax-highlight\" ;; code highlighting\n \"bafy...nav-component\") ;; navigation\n :content\n (~article-layout\n :title \"My Blog\"\n :nav (~nav-component\n :items (list\n (dict :label \"Post 1\" :cid \"bafy...post-1\")\n (dict :label \"Post 2\" :cid \"bafy...post-2\")))\n :body (~markdown-page\n :source-cid \"bafy...homepage-md\")))" "lisp"))
(p "What this looks like in practice:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Personal sites: ") "A portfolio or blog is a handful of SX files + media. Pin to IPFS. Share the CID. No hosting costs, no domain renewal, no SSL certificates. The site is permanent.")
@@ -361,7 +361,7 @@
(p "For applications that " (em "do") " need a server — user accounts, payments, real-time collaboration, database queries — the server provides IO primitives via the existing boundary system. The SX application fetches data from the server's IO endpoints, but the application itself (all the rendering, routing, component logic) lives on IPFS. The server is a " (em "data service") ", not an application host.")
(p "This inverts the current model. Today: server hosts the application, client is a thin renderer. SX web: IPFS hosts the application, server is an optional IO provider. " (strong "The application is the content. The content is the application. Both are just s-expressions.")))
(~doc-subsection :title "The end state"
(~docs/subsection :title "The end state"
(p "A browser with an SX evaluator and an IPFS gateway is a complete web platform. Given a CID — for a page, a post, an application — it can:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Fetch the content from IPFS")
@@ -378,30 +378,30 @@
;; Cross-Cutting Concerns
;; -----------------------------------------------------------------------
(~doc-section :title "Cross-Cutting Concerns" :id "cross-cutting"
(~docs/section :title "Cross-Cutting Concerns" :id "cross-cutting"
(~doc-subsection :title "Security"
(~docs/subsection :title "Security"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Boundary enforcement is the foundation. ") "IPFS-fetched components are parsed and registered like any other component. SX_BOUNDARY_STRICT ensures they can't call IO primitives. A malicious component can produce ugly markup but can't exfiltrate data or make network requests.")
(li (strong "CID verification: ") "Content fetched from IPFS is hashed and compared to the expected CID before use. Tampered content is rejected.")
(li (strong "Signature chain: ") "Actor signatures (RSA/HTTP Signatures) prove authorship. Bitcoin anchors prove timing. Together they establish non-repudiable provenance.")
(li (strong "Resource limits: ") "Evaluation of untrusted components runs with step limits (max eval steps, max recursion depth). Infinite loops are caught and terminated.")))
(~doc-subsection :title "Backward Compatibility"
(~docs/subsection :title "Backward Compatibility"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Content negotiation ensures legacy AP servers always receive valid JSON-LD")
(li "SX-Activity is strictly opt-in — servers that don't understand it get standard AP")
(li "Existing internal activity bus unchanged — SX format is for federation, not internal events")
(li "URL fallbacks on all media references — CID is preferred, URL is fallback")))
(~doc-subsection :title "Performance"
(~docs/subsection :title "Performance"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Component CIDs cached in localStorage forever (content-addressed = immutable)")
(li "IPFS gateway responses cached with long TTL (content can't change)")
(li "Local IPFS node (if present) eliminates gateway latency")
(li "Provenance verification is lazy — badge shows unverified until user clicks to verify")))
(~doc-subsection :title "Integration with Isomorphic Architecture"
(~docs/subsection :title "Integration with Isomorphic Architecture"
(p "SX-Activity builds on the isomorphic architecture plan:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Phase 1 (component distribution) → IPFS replaces per-server bundles")
@@ -413,7 +413,7 @@
;; Critical Files
;; -----------------------------------------------------------------------
(~doc-section :title "Critical Files" :id "critical-files"
(~docs/section :title "Critical Files" :id "critical-files"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"

24
sx/sx/plans/sx-ci.sx
View File

@@ -2,26 +2,26 @@
;; SX CI Pipeline
;; ---------------------------------------------------------------------------
(defcomp ~plan-sx-ci-content ()
(~doc-page :title "SX CI Pipeline"
(defcomp ~plans/sx-ci/plan-sx-ci-content ()
(~docs/page :title "SX CI Pipeline"
(p :class "text-stone-500 text-sm italic mb-8"
"Build, test, and deploy Rose Ash using the same language the application is written in.")
(~doc-section :title "Context" :id "context"
(~docs/section :title "Context" :id "context"
(p :class "text-stone-600"
"Rose Ash currently uses shell scripts for CI: " (code "deploy.sh") " auto-detects changed services via git diff, builds Docker images, pushes to the registry, and restarts Swarm services. " (code "dev.sh") " starts the dev environment and runs tests. These work, but they are opaque imperative scripts with no reuse, no composition, and no relationship to SX.")
(p :class "text-stone-600"
"The CI pipeline is the last piece of infrastructure not expressed in s-expressions. Fixing that completes the \"one representation for everything\" claim — the same language that defines the spec, the components, the pages, the essays, and the deployment config also defines the build pipeline."))
(~doc-section :title "Design" :id "design"
(~docs/section :title "Design" :id "design"
(p :class "text-stone-600"
"Pipeline definitions are " (code ".sx") " files. A minimal Python CLI runner evaluates them using " (code "sx_ref.py") ". CI-specific IO primitives (shell execution, Docker, git) are boundary-declared and only available to the pipeline runner — never to web components.")
(~doc-code :code (highlight ";; pipeline/deploy.sx\n(let ((targets (if (= (length ARGS) 0)\n (~detect-changed :base \"HEAD~1\")\n (filter (fn (svc) (some (fn (a) (= a (get svc \"name\"))) ARGS))\n services))))\n (when (= (length targets) 0)\n (log-step \"No changes detected\")\n (exit 0))\n\n (log-step (str \"Deploying: \" (join \" \" (map (fn (s) (get s \"name\")) targets))))\n\n ;; Tests first\n (~unit-tests)\n (~sx-spec-tests)\n\n ;; Build, push, restart\n (for-each (fn (svc) (~build-service :service svc)) targets)\n (for-each (fn (svc) (~restart-service :service svc)) targets)\n\n (log-step \"Deploy complete\"))" "lisp"))
(~docs/code :code (highlight ";; pipeline/deploy.sx\n(let ((targets (if (= (length ARGS) 0)\n (~plans/sx-ci/detect-changed :base \"HEAD~1\")\n (filter (fn (svc) (some (fn (a) (= a (get svc \"name\"))) ARGS))\n services))))\n (when (= (length targets) 0)\n (log-step \"No changes detected\")\n (exit 0))\n\n (log-step (str \"Deploying: \" (join \" \" (map (fn (s) (get s \"name\")) targets))))\n\n ;; Tests first\n (~unit-tests)\n (~sx-spec-tests)\n\n ;; Build, push, restart\n (for-each (fn (svc) (~plans/sx-ci/build-service :service svc)) targets)\n (for-each (fn (svc) (~restart-service :service svc)) targets)\n\n (log-step \"Deploy complete\"))" "lisp"))
(p :class "text-stone-600"
"Pipeline steps are components. " (code "~unit-tests") ", " (code "~build-service") ", " (code "~detect-changed") " are " (code "defcomp") " definitions that compose by nesting — the same mechanism used for page layouts, navigation, and every other piece of the system."))
"Pipeline steps are components. " (code "~unit-tests") ", " (code "~plans/sx-ci/build-service") ", " (code "~plans/sx-ci/detect-changed") " are " (code "defcomp") " definitions that compose by nesting — the same mechanism used for page layouts, navigation, and every other piece of the system."))
(~doc-section :title "CI Primitives" :id "primitives"
(~docs/section :title "CI Primitives" :id "primitives"
(p :class "text-stone-600"
"New IO primitives declared in " (code "boundary.sx") ", implemented only in the CI runner context:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -70,14 +70,14 @@
(p :class "text-stone-600"
"The boundary system ensures these primitives are " (em "only") " available in the CI context. Web components cannot call " (code "shell-run!") " — the evaluator will refuse to resolve the symbol, just as it refuses to resolve any other unregistered IO primitive. The sandbox is structural, not a convention."))
(~doc-section :title "Reusable Steps" :id "steps"
(~docs/section :title "Reusable Steps" :id "steps"
(p :class "text-stone-600"
"Pipeline steps are components — same " (code "defcomp") " as UI components, same " (code "&key") " params, same composition by nesting:")
(~doc-code :code (highlight "(defcomp ~detect-changed (&key base)\n (let ((files (git-diff-files (or base \"HEAD~1\") \"HEAD\")))\n (if (some (fn (f) (starts-with? f \"shared/\")) files)\n services\n (filter (fn (svc)\n (some (fn (f) (starts-with? f (str (get svc \"dir\") \"/\"))) files))\n services))))\n\n(defcomp ~build-service (&key service)\n (let ((name (get service \"name\"))\n (tag (str registry \"/\" name \":latest\")))\n (log-step (str \"Building \" name))\n (docker-build :file (str (get service \"dir\") \"/Dockerfile\") :tag tag :context \".\")\n (docker-push tag)))\n\n(defcomp ~bootstrap-check ()\n (log-step \"Checking bootstrapped files are up to date\")\n (shell-run! \"python shared/sx/ref/bootstrap_js.py\")\n (shell-run! \"python shared/sx/ref/bootstrap_py.py\")\n (let ((diff (shell-run \"git diff --name-only shared/static/scripts/sx-ref.js shared/sx/ref/sx_ref.py\")))\n (when (not (= (get diff \"stdout\") \"\"))\n (fail! \"Bootstrapped files are stale — rebootstrap and commit\"))))" "lisp"))
(~docs/code :code (highlight "(defcomp ~plans/sx-ci/detect-changed (&key base)\n (let ((files (git-diff-files (or base \"HEAD~1\") \"HEAD\")))\n (if (some (fn (f) (starts-with? f \"shared/\")) files)\n services\n (filter (fn (svc)\n (some (fn (f) (starts-with? f (str (get svc \"dir\") \"/\"))) files))\n services))))\n\n(defcomp ~plans/sx-ci/build-service (&key service)\n (let ((name (get service \"name\"))\n (tag (str registry \"/\" name \":latest\")))\n (log-step (str \"Building \" name))\n (docker-build :file (str (get service \"dir\") \"/Dockerfile\") :tag tag :context \".\")\n (docker-push tag)))\n\n(defcomp ~plans/sx-ci/bootstrap-check ()\n (log-step \"Checking bootstrapped files are up to date\")\n (shell-run! \"python shared/sx/ref/bootstrap_js.py\")\n (shell-run! \"python shared/sx/ref/bootstrap_py.py\")\n (let ((diff (shell-run \"git diff --name-only shared/static/scripts/sx-ref.js shared/sx/ref/sx_ref.py\")))\n (when (not (= (get diff \"stdout\") \"\"))\n (fail! \"Bootstrapped files are stale — rebootstrap and commit\"))))" "lisp"))
(p :class "text-stone-600"
"Compare this to GitHub Actions YAML, where \"reuse\" means composite actions with " (code "uses:") " references, input/output mappings, shell script blocks inside YAML strings, and a " (a :href "https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions" :class "text-violet-600 hover:underline" "100-page syntax reference") ". SX pipeline reuse is function composition. That is all it has ever been."))
(~doc-section :title "Pipelines" :id "pipelines"
(~docs/section :title "Pipelines" :id "pipelines"
(p :class "text-stone-600"
"Two primary pipelines, each a single " (code ".sx") " file:")
(div :class "space-y-4"
@@ -90,7 +90,7 @@
(p :class "text-sm text-stone-600" "Auto-detect changed services (or accept explicit args), run tests, build Docker images, push to registry, restart Swarm services.")
(p :class "text-sm font-mono text-violet-700 mt-1" "python -m shared.sx.ci pipeline/deploy.sx blog market"))))
(~doc-section :title "Why this matters" :id "why"
(~docs/section :title "Why this matters" :id "why"
(p :class "text-stone-600"
"CI pipelines are the strongest test case for \"one representation for everything.\" GitHub Actions, GitLab CI, CircleCI — all use YAML. YAML is not a programming language. So every CI system reinvents conditionals (" (code "if:") " expressions evaluated as strings), iteration (" (code "matrix:") " strategies), composition (" (code "uses:") " references with input/output schemas), and error handling (" (code "continue-on-error:") " booleans) — all in a data format that was never designed for any of it.")
(p :class "text-stone-600"
@@ -98,7 +98,7 @@
(p :class "text-stone-600"
"SX pipelines use real conditionals, real functions, real composition, and real error handling — because SX is a real language. The pipeline definition and the application code are the same thing. An AI that can generate SX components can generate SX pipelines. A developer who reads SX pages can read SX deploys. The representation is universal."))
(~doc-section :title "Files" :id "files"
(~docs/section :title "Files" :id "files"
(div :class "overflow-x-auto rounded border border-stone-200"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"

22
sx/sx/plans/sx-forge.sx
View File

@@ -2,10 +2,10 @@
;; Plan: a Gitea/Forgejo-style git hosting platform where everything —
;; repositories, issues, pull requests, CI, permissions — is SX.
(defcomp ~plan-sx-forge-content ()
(~doc-page :title "sx-forge: Git Forge in SX"
(defcomp ~plans/sx-forge/plan-sx-forge-content ()
(~docs/page :title "sx-forge: Git Forge in SX"
(~doc-section :title "Vision" :id "vision"
(~docs/section :title "Vision" :id "vision"
(p "A git forge where the entire interface, configuration, and automation layer "
"is written in SX. Repositories are browsed, issues are filed, pull requests "
"are reviewed, and CI pipelines are triggered — all through SX components "
@@ -14,7 +14,7 @@
"that expand to permission checks. Repository templates are defcomps. "
"The forge doesn't use SX — it " (em "is") " SX."))
(~doc-section :title "Why" :id "why"
(~docs/section :title "Why" :id "why"
(p "Gitea/Forgejo are excellent but they're Go binaries with YAML/INI config, "
"Markdown rendering, and a template engine that's separate from the application logic. "
"Every layer speaks a different language.")
@@ -28,7 +28,7 @@
(li "Access control = SX macros that expand to permission predicates")
(li "API = SX wire format (text/sx) alongside JSON for compatibility")))
(~doc-section :title "Architecture" :id "architecture"
(~docs/section :title "Architecture" :id "architecture"
(div :class "overflow-x-auto mt-4"
(table :class "w-full text-sm text-left"
(thead
@@ -66,7 +66,7 @@
(td :class "py-2 px-3" "sx-activity (ActivityPub)")
(td :class "py-2 px-3" "Cross-instance PRs, issues, stars, forks."))))))
(~doc-section :title "Configuration as SX" :id "config"
(~docs/section :title "Configuration as SX" :id "config"
(p "Instance configuration is an SX file, not YAML or INI:")
(highlight "(define forge-config
{:name \"Rose Ash Forge\"
@@ -100,10 +100,10 @@
:merge-pr (and (ci-passed?) (approved-by? 1))
:admin (role? :admin)}})" "lisp"))
(~doc-section :title "SX Diff Viewer" :id "diff-viewer"
(~docs/section :title "SX Diff Viewer" :id "diff-viewer"
(p "Diffs rendered as SX components, not pre-formatted text:")
(highlight ";; The diff viewer is a defcomp, composable like any other
(defcomp ~diff-view (&key (diff :as dict))
(defcomp ~plans/sx-forge/diff-view (&key (diff :as dict))
(map (fn (hunk)
(~diff-hunk
:file (get hunk \"file\")
@@ -118,7 +118,7 @@
(li "Suggestion blocks — click to apply a proposed change")
(li "SX-aware diffs — show component-level changes, not just line changes")))
(~doc-section :title "Federated Forge" :id "federation"
(~docs/section :title "Federated Forge" :id "federation"
(p "sx-activity enables cross-instance collaboration:")
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li (strong "Cross-instance PRs") " — open a PR from your fork on another instance")
@@ -129,7 +129,7 @@
(p "Every issue, comment, and review is a content-addressed SX document on IPFS. "
"Federation distributes references. The content is permanent and verifiable."))
(~doc-section :title "Git Operations as IO Primitives" :id "git-ops"
(~docs/section :title "Git Operations as IO Primitives" :id "git-ops"
(p "Git operations exposed as SX IO primitives in boundary.sx:")
(highlight ";; boundary.sx additions
(io git-log (repo &key branch limit offset) list)
@@ -142,7 +142,7 @@
(io git-merge (repo source target &key strategy) dict)" "lisp")
(p "Pages use these directly — no controller layer, no ORM:"))
(~doc-section :title "Implementation Path" :id "implementation"
(~docs/section :title "Implementation Path" :id "implementation"
(ol :class "space-y-3 text-stone-600 list-decimal pl-5"
(li (strong "Phase 1: Read-only browser") " — git-tree, git-blob, git-log, git-diff as IO primitives. "
"SX components for tree view, blob view, commit log, diff view.")

50
sx/sx/plans/sx-protocol.sx
View File

@@ -1,16 +1,16 @@
;; SX Protocol — A Proposal
;; S-expressions as a universal protocol for networked hypermedia.
(defcomp ~plan-sx-protocol-content ()
(~doc-page :title "SX Protocol — A Proposal"
(defcomp ~plans/sx-protocol/plan-sx-protocol-content ()
(~docs/page :title "SX Protocol — A Proposal"
(~doc-section :title "Abstract" :id "abstract"
(~docs/section :title "Abstract" :id "abstract"
(p "SX is a Lisp dialect and a proposed universal protocol for networked hypermedia. "
"It replaces URLs, HTTP verbs, query strings, API query languages, and rendering layers "
"with a single unified concept: " (strong "the s-expression") ".")
(p "Everything is an expression. Everything is evaluable. Everything is composable."))
(~doc-section :title "The Problem With the Current Web" :id "problem"
(~docs/section :title "The Problem With the Current Web" :id "problem"
(p "The modern web stack has accumulated layers of incompatible syntax to express "
"what are fundamentally the same things:")
(table :class "w-full text-sm border-collapse mb-4"
@@ -47,13 +47,13 @@
(p "Each layer invented its own syntax. None of them compose. None of them are executable. "
"None of them are data."))
(~doc-section :title "The SX Approach" :id "approach"
(~docs/section :title "The SX Approach" :id "approach"
(h3 :class "text-lg font-semibold mt-6 mb-2" "URLs as S-Expressions")
(p "A conventional URL:")
(~doc-code :code (highlight "https://site.com/blog/my-post?filter=published&sort=date" "text"))
(~docs/code :code (highlight "https://site.com/blog/my-post?filter=published&sort=date" "text"))
(p "As an SX expression:")
(~doc-code :code (highlight "(get.site.com.(blog.(my-post.(filter.published.sort.date))))" "lisp"))
(~docs/code :code (highlight "(get.site.com.(blog.(my-post.(filter.published.sort.date))))" "lisp"))
(ul
(li "The protocol/verb is the first atom: " (code "get"))
(li "The domain follows: " (code "site.com"))
@@ -64,15 +64,15 @@
(p "Lisp conventionally uses spaces as separators. In URLs, spaces become " (code "%20")
". SX uses dots instead, which are URL-safe and semantically meaningful — a dot between "
"two atoms is a " (strong "cons pair") ", the fundamental unit of Lisp structure.")
(~doc-code :code (highlight ";; Clean, URL-safe, valid Lisp\n(blog.(filter.published).(sort.date.desc))" "lisp"))
(~docs/code :code (highlight ";; Clean, URL-safe, valid Lisp\n(blog.(filter.published).(sort.date.desc))" "lisp"))
(h3 :class "text-lg font-semibold mt-6 mb-2" "Verbs Are Just Atoms")
(p "HTTP methods are not special syntax — they are simply the first element of the expression:")
(~doc-code :code (highlight "(get.site.com.(post.my-first-post)) ; read\n(post.site.com.(submit-post.(title.hello))) ; write\n(ws.site.com.(live-feed)) ; websocket / subscribe" "lisp"))
(~docs/code :code (highlight "(get.site.com.(post.my-first-post)) ; read\n(post.site.com.(submit-post.(title.hello))) ; write\n(ws.site.com.(live-feed)) ; websocket / subscribe" "lisp"))
(p "No special protocol prefixes. No " (code "https://") " vs " (code "wss://")
". The verb is data, like everything else."))
(~doc-section :title "Graph-SX: Hypermedia Queries" :id "graph-sx"
(~docs/section :title "Graph-SX: Hypermedia Queries" :id "graph-sx"
(p "GraphQL was a major advance over REST, but it made two compromises:")
(ol
(li "Queries are sent as POST bodies, sacrificing cacheability and shareability")
@@ -81,7 +81,7 @@
(h3 :class "text-lg font-semibold mt-6 mb-2" "Queries Are URLs")
(p "Because SX expressions are URLs, every query is a GET request:")
(~doc-code :code (highlight ";; This is a URL and a query simultaneously\n(get.site.com.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(~docs/code :code (highlight ";; This is a URL and a query simultaneously\n(get.site.com.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(ul
(li "Fully cacheable by CDNs")
(li "Bookmarkable and shareable")
@@ -91,19 +91,19 @@
(h3 :class "text-lg font-semibold mt-6 mb-2" "Responses Include Rendering")
(p "GraphQL returns data. Graph-SX returns " (strong "hypermedia")
" — data and its presentation in the same expression:")
(~doc-code :code (highlight ";; GraphQL response (dead data)\n{\"title\": \"My Post\", \"body\": \"Hello world\"}\n\n;; Graph-SX response (live hypermedia)\n(article\n (h1 \"My Post\")\n (p \"Hello world\")\n (a (href (get.site.com.(post.next-post))) \"Next\"))" "lisp"))
(~docs/code :code (highlight ";; GraphQL response (dead data)\n{\"title\": \"My Post\", \"body\": \"Hello world\"}\n\n;; Graph-SX response (live hypermedia)\n(article\n (h1 \"My Post\")\n (p \"Hello world\")\n (a (href (get.site.com.(post.next-post))) \"Next\"))" "lisp"))
(p "The server returns what the resource " (strong "is") " and how to "
(strong "present") " it in one unified structure. There is no separate rendering layer.")
(h3 :class "text-lg font-semibold mt-6 mb-2" "Queries Are Transformations")
(p "Because SX is a full programming language, the query and the transformation "
"are the same expression:")
(~doc-code :code (highlight ";; Fetch, filter, and transform in one expression\n(map (lambda (p) (title p))\n (filter published?\n (posts (after \"2025\"))))" "lisp"))
(~docs/code :code (highlight ";; Fetch, filter, and transform in one expression\n(map (lambda (p) (title p))\n (filter published?\n (posts (after \"2025\"))))" "lisp"))
(p "No separate processing step. No client-side data manipulation layer."))
(~doc-section :title "Components" :id "components"
(~docs/section :title "Components" :id "components"
(p "SX supports server-side composable components via the " (code "~") " prefix convention:")
(~doc-code :code (highlight "(~get.everything-under-the-sun)" "lisp"))
(~docs/code :code (highlight "(~get.everything-under-the-sun)" "lisp"))
(p "A " (code "~component") " is a named server-side function that:")
(ol
(li "Receives the expression as arguments")
@@ -111,22 +111,22 @@
(li "Processes and composes results")
(li "Returns hypermedia"))
(p "Components compose naturally:")
(~doc-code :code (highlight "(~page.home\n (~hero.banner)\n (~get.latest-posts.(limit.5))\n (~get.featured.(filter.pinned)))" "lisp"))
(~docs/code :code (highlight "(~page.home\n (~hero.banner)\n (~get.latest-posts.(limit.5))\n (~get.featured.(filter.pinned)))" "lisp"))
(p "This is equivalent to React Server Components — but without a framework, "
"without a build step, and without leaving Lisp."))
(~doc-section :title "Cross-Domain Composition" :id "cross-domain"
(~docs/section :title "Cross-Domain Composition" :id "cross-domain"
(p "Because domain and verb are just atoms, cross-domain queries are structurally "
"identical to local ones:")
(~doc-code :code (highlight ";; Local\n(post.my-first-post)\n\n;; Remote — identical structure, qualified\n(get.site.com.(post.my-first-post))\n\n;; Composed across domains\n(~render\n (get.site.com.(post.my-first-post))\n (get.cdn.com.(image.hero)))" "lisp"))
(~docs/code :code (highlight ";; Local\n(post.my-first-post)\n\n;; Remote — identical structure, qualified\n(get.site.com.(post.my-first-post))\n\n;; Composed across domains\n(~render\n (get.site.com.(post.my-first-post))\n (get.cdn.com.(image.hero)))" "lisp"))
(p "Network calls are function calls. Remote resources are just namespaced expressions."))
(~doc-section :title "Self-Describing and Introspectable" :id "introspectable"
(~docs/section :title "Self-Describing and Introspectable" :id "introspectable"
(p "Because the site is implemented in SX and served as SX, every page is introspectable:")
(~doc-code :code (highlight "(get.sx.dev.(about)) ; the about page\n(get.sx.dev.(source.(about))) ; the SX source for the about page\n(get.sx.dev.(eval.(source.about))) ; re-evaluate it live" "lisp"))
(~docs/code :code (highlight "(get.sx.dev.(about)) ; the about page\n(get.sx.dev.(source.(about))) ; the SX source for the about page\n(get.sx.dev.(eval.(source.about))) ; re-evaluate it live" "lisp"))
(p "The site is its own documentation. The source is always one expression away."))
(~doc-section :title "Comparison" :id "comparison"
(~docs/section :title "Comparison" :id "comparison"
(table :class "w-full text-sm border-collapse mb-4"
(thead
(tr :class "border-b border-stone-300"
@@ -181,17 +181,17 @@
(td :class "py-2 pr-4 text-stone-500" "Partial")
(td :class "py-2 text-green-700" "Yes")))))
(~doc-section :title "Future Direction" :id "future"
(~docs/section :title "Future Direction" :id "future"
(p "The logical conclusion of SX is a " (strong "new internet protocol")
" in which the URL, the HTTP verb, the query language, the response format, "
"and the rendering layer are all unified under one evaluable expression format.")
(~doc-code :code (highlight ";; The entire network request — protocol, domain, verb, query, all one expression\n(get.sx.dev.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(~docs/code :code (highlight ";; The entire network request — protocol, domain, verb, query, all one expression\n(get.sx.dev.(blog.(filter.(tag.lisp)).(limit.10)))" "lisp"))
(p "HTTP becomes one possible implementation of a more general principle:")
(blockquote :class "border-l-4 border-violet-300 pl-4 italic text-stone-600 my-4"
(p (strong "Evaluate this expression. Return an expression."))))
(~doc-section :title "Reference Implementation" :id "reference"
(~docs/section :title "Reference Implementation" :id "reference"
(p "SX is implemented in SX. The reference implementation is self-hosting and available at:")
(~doc-code :code (highlight "(get.sx.dev.(source.evaluator))" "lisp"))
(~docs/code :code (highlight "(get.sx.dev.(source.evaluator))" "lisp"))
(p :class "text-sm text-stone-500 mt-4 italic"
"This proposal was written in conversation with Claude (Anthropic). The ideas are the author's own."))))

22
sx/sx/plans/sx-proxy.sx
View File

@@ -2,10 +2,10 @@
;; Plan: a Caddy-style reverse proxy where routes, TLS, middleware,
;; and load balancing are SX s-expressions manipulated by macros.
(defcomp ~plan-sx-proxy-content ()
(~doc-page :title "sx-proxy: Reverse Proxy in SX"
(defcomp ~plans/sx-proxy/plan-sx-proxy-content ()
(~docs/page :title "sx-proxy: Reverse Proxy in SX"
(~doc-section :title "Vision" :id "vision"
(~docs/section :title "Vision" :id "vision"
(p "A reverse proxy where routing rules, TLS configuration, middleware chains, "
"and load balancing policies are SX. Not a config file that gets parsed — "
"actual SX that gets evaluated. Macros generate routes from service definitions. "
@@ -14,7 +14,7 @@
"No functions, no macros, no computed values, no integration with the rest of the stack. "
"sx-proxy goes all the way: the proxy configuration " (em "is") " the application."))
(~doc-section :title "Why" :id "why"
(~docs/section :title "Why" :id "why"
(p "Every proxy config language reinvents the same features badly:")
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li "Nginx: custom DSL with if-is-evil, no real conditionals, include for composition")
@@ -25,7 +25,7 @@
"SX unifies them. The proxy reads the same service definitions "
"that the orchestrator deploys."))
(~doc-section :title "Route Definitions" :id "routes"
(~docs/section :title "Route Definitions" :id "routes"
(p "Routes as SX, with macros for common patterns:")
(highlight ";; Basic route definition
(route blog.rose-ash.com
@@ -48,7 +48,7 @@
(p "One macro generates all routes from the same service definitions "
"that sx-swarm uses for deployment. Change the service, the route updates."))
(~doc-section :title "Middleware as Composition" :id "middleware"
(~docs/section :title "Middleware as Composition" :id "middleware"
(p "Middleware chains are function composition:")
(highlight ";; Middleware are functions: request -> response -> response
(define rate-limit
@@ -82,7 +82,7 @@
"Apply different chains to different routes. "
"No nginx location blocks, no Caddy handle nesting."))
(~doc-section :title "TLS Configuration" :id "tls"
(~docs/section :title "TLS Configuration" :id "tls"
(p "TLS as SX with macro-generated cert management:")
(highlight ";; Auto TLS via ACME (like Caddy)
(define tls-auto
@@ -103,7 +103,7 @@
(defmacro with-tls (config &rest routes)
`(map (fn (r) (assoc r :tls ,config)) (list ,@routes)))" "lisp"))
(~doc-section :title "Load Balancing" :id "load-balancing"
(~docs/section :title "Load Balancing" :id "load-balancing"
(p "Load balancing policies as SX values:")
(highlight ";; Load balancing strategies
(define round-robin (lb :strategy :round-robin))
@@ -124,7 +124,7 @@
:upstream blog-pool
:tls :auto)" "lisp"))
(~doc-section :title "Dynamic Reconfiguration" :id "dynamic"
(~docs/section :title "Dynamic Reconfiguration" :id "dynamic"
(p "The proxy evaluates SX — so config can be dynamic:")
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li (strong "Hot reload") " — change the SX, proxy re-evaluates. No restart.")
@@ -149,7 +149,7 @@
\"blog-prod:8000\")
:tls :auto)" "lisp"))
(~doc-section :title "Integration with sx-swarm" :id "integration"
(~docs/section :title "Integration with sx-swarm" :id "integration"
(p "sx-proxy and sx-swarm share the same service definitions:")
(highlight ";; One source of truth
(defservice blog
@@ -169,7 +169,7 @@
"The proxy, the orchestrator, the CI, and the forge all consume "
"the same SX service definitions. No YAML-to-Caddyfile-to-Dockerfile translation."))
(~doc-section :title "Implementation Path" :id "implementation"
(~docs/section :title "Implementation Path" :id "implementation"
(ol :class "space-y-3 text-stone-600 list-decimal pl-5"
(li (strong "Phase 1: Config compiler") " — SX route definitions compiled to Caddy JSON API. "
"Use Caddy as the runtime, SX as the config language.")

22
sx/sx/plans/sx-swarm.sx
View File

@@ -2,10 +2,10 @@
;; Plan: Docker Swarm management where stack definitions, service configs,
;; and deployment logic are SX s-expressions manipulated by macros.
(defcomp ~plan-sx-swarm-content ()
(~doc-page :title "sx-swarm: Container Orchestration in SX"
(defcomp ~plans/sx-swarm/plan-sx-swarm-content ()
(~docs/page :title "sx-swarm: Container Orchestration in SX"
(~doc-section :title "Vision" :id "vision"
(~docs/section :title "Vision" :id "vision"
(p "Replace docker-compose.yml and Docker Swarm stack files with SX. "
"Service definitions are defcomps. Environment configs are macros that expand "
"differently per target (dev, staging, production). Deployments are SX pipelines "
@@ -15,7 +15,7 @@
"SX has all of these. A service definition is a value. A deployment is a function. "
"An environment override is a macro."))
(~doc-section :title "Why Not YAML" :id "why-not-yaml"
(~docs/section :title "Why Not YAML" :id "why-not-yaml"
(p "Docker Compose and Swarm stack files share a fundamental problem: "
"they're static data with ad-hoc templating bolted on. "
"Variable substitution (${VAR}), extension fields (x-), YAML anchors (&/*) — "
@@ -29,7 +29,7 @@
(li "No verification — can't check constraints before deploy"))
(p "SX solves all of these because it's a programming language, not a data format."))
(~doc-section :title "Service Definitions" :id "services"
(~docs/section :title "Service Definitions" :id "services"
(p "Services defined as SX, with macros for common patterns:")
(highlight ";; Base service macro — shared defaults
(defmacro defservice (name &rest body)
@@ -53,7 +53,7 @@
"Every service gets restart policy, logging, and network config for free. "
"Override any field by specifying it in the body."))
(~doc-section :title "Environment Macros" :id "environments"
(~docs/section :title "Environment Macros" :id "environments"
(p "Environment-specific config via macros, not file merging:")
(highlight ";; Environment macro — expands differently per target
(defmacro env-for (service)
@@ -81,7 +81,7 @@
(p "No more docker-compose.yml + docker-compose.dev.yml + docker-compose.prod.yml. "
"One definition, macros handle the rest."))
(~doc-section :title "Stack Composition" :id "composition"
(~docs/section :title "Stack Composition" :id "composition"
(p "Stacks compose like functions:")
(highlight ";; Infrastructure services shared across all stacks
(define infra-services
@@ -108,7 +108,7 @@
(volume :name \"pg-data\" :driver :local)
(volume :name \"redis-data\" :driver :local))))" "lisp"))
(~doc-section :title "Deploy as SX Pipeline" :id "deploy"
(~docs/section :title "Deploy as SX Pipeline" :id "deploy"
(p "Deployment is an sx-ci pipeline, not a shell script:")
(highlight "(define deploy-pipeline
(pipeline :name \"deploy\"
@@ -128,7 +128,7 @@
:rolling true
:health-check-interval \"5s\"))))" "lisp"))
(~doc-section :title "Swarm Operations as IO" :id "swarm-ops"
(~docs/section :title "Swarm Operations as IO" :id "swarm-ops"
(p "Swarm management exposed as SX IO primitives:")
(highlight ";; boundary.sx additions
(io swarm-deploy (stack &key target rolling) dict)
@@ -144,7 +144,7 @@
(p "These compose with sx-ci and the forge — push to forge triggers CI, "
"CI runs tests, tests pass, deploy pipeline executes, swarm updates."))
(~doc-section :title "Health and Monitoring" :id "monitoring"
(~docs/section :title "Health and Monitoring" :id "monitoring"
(p "Service health as live SX components:")
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li (strong "Dashboard") " — defcomp rendering swarm-status, auto-refreshing via sx-get polling")
@@ -154,7 +154,7 @@
(p "The monitoring dashboard is an SX page like any other. "
"No Grafana, no separate monitoring stack. Same language, same renderer, same platform."))
(~doc-section :title "Implementation Path" :id "implementation"
(~docs/section :title "Implementation Path" :id "implementation"
(ol :class "space-y-3 text-stone-600 list-decimal pl-5"
(li (strong "Phase 1: Stack definition") " — SX data structures for services, networks, volumes. "
"Compiler to docker-compose.yml / docker stack deploy format.")

86
sx/sx/plans/sx-urls.sx
View File

@@ -2,10 +2,10 @@
;; Plan: replace path-based routing with s-expression URLs where the URL
;; IS the query, the render instruction, and the address — all at once.
(defcomp ~plan-sx-urls-content ()
(~doc-page :title "SX Expression URLs"
(defcomp ~plans/sx-urls/plan-sx-urls-content ()
(~docs/page :title "SX Expression URLs"
(~doc-section :title "Vision" :id "vision"
(~docs/section :title "Vision" :id "vision"
(p "URLs become s-expressions. The entire routing layer collapses into eval. "
"Every page is a function, every URL is a function call, and the nav tree hierarchy "
"is encoded directly in the nesting of the expression.")
@@ -13,14 +13,14 @@
(code "/sx/(language.(doc.introduction))") ". Dots replace spaces as the URL-friendly "
"separator — they are unreserved in RFC 3986, never percent-encoded, and visually clean. "
"The parser treats dot as whitespace: " (code "s/./ /") " before parsing as SX.")
(~doc-code :code (highlight
";; Current → SX URLs (dots = spaces)\n/language/specs/signals → /(language.(spec.signals))\n/language/specs/explore/signals → /(language.(spec.(explore.signals)))\n/language/docs/introduction → /(language.(doc.introduction))\n/etc/plans/spec-explorer → /(etc.(plan.spec-explorer))\n\n;; Direct component access — any defcomp is addressable\n/(~essay-sx-sucks)\n/(~plan-sx-urls-content)\n/(~bundle-analyzer-content)"
(~docs/code :code (highlight
";; Current → SX URLs (dots = spaces)\n/language/specs/signals → /(language.(spec.signals))\n/language/specs/explore/signals → /(language.(spec.(explore.signals)))\n/language/docs/introduction → /(language.(doc.introduction))\n/etc/plans/spec-explorer → /(etc.(plan.spec-explorer))\n\n;; Direct component access — any defcomp is addressable\n/(~essays/sx-sucks/essay-sx-sucks)\n/(~plans/sx-urls/plan-sx-urls-content)\n/(~analyzer/bundle-analyzer-content)"
"lisp")))
(~doc-section :title "Scoping — The 30-Year Ambiguity, Fixed" :id "scoping"
(~docs/section :title "Scoping — The 30-Year Ambiguity, Fixed" :id "scoping"
(p "REST URLs have an inherent ambiguity: does a filter/parameter apply to "
"the last segment, or the whole path? Consider:")
(~doc-code :code (highlight
(~docs/code :code (highlight
";; REST — ambiguous:\n/users/123/posts?filter=published\n;; Is the filter scoped to posts? Or to the user? Or the whole query?\n;; Nobody knows. Conventions vary. Documentation required.\n\n;; SX URLs — explicit scoping via nesting:\n/(hello.(sailor.(filter.hhh))) ;; filter scoped to sailor\n/(hello.sailor.(filter.hhh)) ;; filter scoped to hello\n\n;; These mean different things, both expressible.\n;; Parens make scope visible. No ambiguity. No documentation needed."
"lisp"))
(p "This is not a minor syntactic preference. REST has never been able to express "
@@ -32,20 +32,20 @@
"What took REST 30 years of convention documents to approximate, "
"SX URLs express in the syntax itself."))
(~doc-section :title "Dots as URL-Safe Whitespace" :id "dots"
(~docs/section :title "Dots as URL-Safe Whitespace" :id "dots"
(p "Spaces in URLs are ugly — they become " (code "%20") " in copy-paste, curl, logs, and proxies. "
"Dots are unreserved in RFC 3986, never encoded, and read naturally as \"drill down.\"")
(p "The rule is simple: " (strong "dot = space, nothing more") ". "
"Parens carry all the structural meaning. Dots are syntactic sugar for URLs only:")
(~doc-code :code (highlight
(~docs/code :code (highlight
";; These are identical after dot→space transform:\n/(language.(doc.introduction)) → (language (doc introduction))\n/(geography.(hypermedia.(reference.attributes)))\n → (geography (hypermedia (reference attributes)))\n\n;; Parens are still required for nesting:\n/(language.doc.introduction) → (language doc introduction)\n;; = language(\"doc\", \"introduction\") — WRONG\n\n;; Correct nesting:\n/(language.(doc.introduction)) → (language (doc introduction))\n;; = language(doc(\"introduction\")) — RIGHT"
"lisp"))
(p "The server's URL handler does one thing before parsing: "
(code "url_expr = raw_path[1:].replace('.', ' ')") ". Then standard SX parsing takes over."))
(~doc-section :title "The Lisp Tax" :id "parens"
(~docs/section :title "The Lisp Tax" :id "parens"
(p "People will hate the parentheses. But consider what developers already accept:")
(~doc-code :code (highlight
(~docs/code :code (highlight
";; Developers happily write this every day:\nhttps://api.site.com/v2/users/123/posts?filter=published&sort=date&order=desc&limit=10&offset=20\n\n;; And they would complain about this?\nhttps://site.com/(users.(posts.123.(filter.published.sort.date.limit.10)))\n\n;; The second is shorter, structured, unambiguous, and composable."
"lisp"))
(p "The real question: who is reading these URLs?")
@@ -58,9 +58,9 @@
"Every URL on the site is a live example of SX in action. "
"Visiting a page is evaluating an expression."))
(~doc-section :title "The Site Is a REPL" :id "repl"
(~docs/section :title "The Site Is a REPL" :id "repl"
(p "The address bar becomes the input line of a REPL. The page is the output.")
(~doc-code :code (highlight
(~docs/code :code (highlight
"/sx/(about) ;; renders the about page\n/(source.(about)) ;; returns the SX source for the about page\n/(eval.(source.(about))) ;; re-evaluates it live\n\n;; The killer demo:\n/(eval.(map.double.(list.1.2.3))) ;; actually returns (2 4 6)\n\n;; The website IS a REPL. The address bar IS the input."
"lisp"))
(p "You do not need to explain what SX is. You show someone a URL and they "
@@ -68,11 +68,11 @@
"The whole site becomes a self-hosting proof of concept — "
"that is not just elegant, that is the pitch."))
(~doc-section :title "Components as Query Resolvers" :id "resolvers"
(~docs/section :title "Components as Query Resolvers" :id "resolvers"
(p "The " (code "~") " sigil means \"find and execute this component.\" "
"Components can make onward queries, process results, and return composed content — "
"like server-side includes but Lispy and composable.")
(~doc-code :code (highlight
(~docs/code :code (highlight
";; ~get is a component that fetches, processes, and returns\n/(~get.everything-under-the-sun)\n\n;; The flow:\n;; 1. Server finds ~get component in env\n;; 2. ~get makes onward queries\n;; 3. Processes and transforms results\n;; 4. Returns composed hypermedia\n\n;; Because it's all SX, you nest and compose:\n/(~page.home\n .(~hero.banner)\n .(~get.latest-posts.(limit.5))\n .(~get.featured.(filter.pinned)))\n\n;; Each ~component is independently:\n;; - cacheable (by its expression)\n;; - reusable (same component, different args)\n;; - testable (evaluate in isolation)"
"lisp"))
(p "This is what React Server Components are trying to do — server-side data resolution "
@@ -80,7 +80,7 @@
"a serialization protocol, and \"use server\" pragmas. "
"SX gets it from a sigil and an evaluator."))
(~doc-section :title "HTTP Semantics — REST Re-Aligned" :id "http"
(~docs/section :title "HTTP Semantics — REST Re-Aligned" :id "http"
(p "GraphQL uses POST for queries even though they are pure reads — "
"because queries can be long and the body feels more natural for structured data. "
"But this violates HTTP semantics: POST implies side effects, "
@@ -106,7 +106,7 @@
"This is what REST always wanted but GraphQL abandoned. "
"SX re-aligns with HTTP while being more powerful than both."))
(~doc-section :title "GraphSX — This Is a Query Language" :id "graphsx"
(~docs/section :title "GraphSX — This Is a Query Language" :id "graphsx"
(p "The SX URL scheme is not just a routing convention — it is the emergence of "
(strong "GraphSX") ": GraphQL but Lisp. The structural parallel is exact:")
(div :class "overflow-x-auto mt-4"
@@ -163,17 +163,17 @@
"GraphQL had to invent a special syntax for queries because JSON is data, not code. "
"S-expressions are both."))
(~doc-section :title "Direct Component URLs" :id "components"
(p "Any " (code "defcomp") " is directly addressable via its " (code "~name") ". "
"The URL evaluator sees " (code "~essay-sx-sucks") ", looks it up in the component env, "
"evaluates it, wraps in " (code "~sx-doc") ", and returns.")
(~doc-code :code (highlight
";; Page functions are convenience wrappers:\n/(etc.(essay.sx-sucks)) ;; dispatches via case statement\n\n;; But you can bypass them entirely:\n/(~essay-sx-sucks) ;; direct component — no routing needed\n\n;; Every defcomp is instantly URL-accessible:\n/(~plan-sx-urls-content) ;; this very page\n/(~bundle-analyzer-content) ;; tools\n/(~docs-evaluator-content) ;; docs"
(~docs/section :title "Direct Component URLs" :id "components"
(p "Any " (code "defcomp") " is directly addressable via its " (code "~plans/content-addressed-components/name") ". "
"The URL evaluator sees " (code "~essays/sx-sucks/essay-sx-sucks") ", looks it up in the component env, "
"evaluates it, wraps in " (code "~layouts/doc") ", and returns.")
(~docs/code :code (highlight
";; Page functions are convenience wrappers:\n/(etc.(essay.sx-sucks)) ;; dispatches via case statement\n\n;; But you can bypass them entirely:\n/(~essays/sx-sucks/essay-sx-sucks) ;; direct component — no routing needed\n\n;; Every defcomp is instantly URL-accessible:\n/(~plans/sx-urls/plan-sx-urls-content) ;; this very page\n/(~analyzer/bundle-analyzer-content) ;; tools\n/(~docs-content/docs-evaluator-content) ;; docs"
"lisp"))
(p "New components are instantly URL-accessible without routing wiring. "
"Debugging is trivial — render any component in isolation."))
(~doc-section :title "URL Special Forms" :id "special-forms"
(~docs/section :title "URL Special Forms" :id "special-forms"
(p "URL-level functions that transform how content is resolved or displayed:")
(div :class "overflow-x-auto mt-4"
(table :class "w-full text-sm text-left"
@@ -185,7 +185,7 @@
(tbody :class "text-stone-600"
(tr :class "border-b border-stone-100"
(td :class "py-2 px-3 font-mono text-violet-700" "source")
(td :class "py-2 px-3 font-mono text-sm" "/sx/(source.(~essay-sx-sucks))")
(td :class "py-2 px-3 font-mono text-sm" "/sx/(source.(~essays/sx-sucks/essay-sx-sucks))")
(td :class "py-2 px-3" "Show defcomp source instead of rendering"))
(tr :class "border-b border-stone-100"
(td :class "py-2 px-3 font-mono text-violet-700" "inspect")
@@ -202,7 +202,7 @@
(tr :class "border-b border-stone-100"
(td :class "py-2 px-3 font-mono text-violet-700" "raw")
(td :class "py-2 px-3 font-mono text-sm" "/sx/(raw.(~some-component))")
(td :class "py-2 px-3" "Skip ~sx-doc nav wrapping"))
(td :class "py-2 px-3" "Skip ~layouts/doc nav wrapping"))
(tr :class "border-b border-stone-100"
(td :class "py-2 px-3 font-mono text-violet-700" "eval")
(td :class "py-2 px-3 font-mono text-sm" "/sx/(eval.(map.double.(list.1.2.3)))")
@@ -212,31 +212,31 @@
(td :class "py-2 px-3 font-mono text-sm" "/sx/(json.(language.(doc.primitives)))")
(td :class "py-2 px-3" "Return data as JSON — pure query mode"))))))
(~doc-section :title "Evaluation Model" :id "eval"
(~docs/section :title "Evaluation Model" :id "eval"
(p "The URL path (after stripping " (code "/") " and replacing dots with spaces) "
"is parsed as SX and evaluated with a " (strong "soft eval") ": "
"known function names are called; unknown symbols self-evaluate to their name as a string; "
"components (" (code "~name") ") are looked up in the component env.")
(~doc-code :code (highlight
"/sx/(language.(doc.introduction))\n\n;; After dot→space: (language (doc introduction))\n;; 1. Eval `introduction` → not a known function → \"introduction\"\n;; 2. Eval (doc \"introduction\") → call doc(\"introduction\") → page content\n;; 3. Eval (language content) → call language(content) → passes through\n;; 4. Router wraps result in (~sx-doc :path \"(language (doc introduction))\" ...)\n\n/(~essay-sx-sucks)\n;; 1. Eval ~essay-sx-sucks → component lookup → evaluate → content\n;; 2. Router wraps in ~sx-doc"
"components (" (code "~plans/content-addressed-components/name") ") are looked up in the component env.")
(~docs/code :code (highlight
"/sx/(language.(doc.introduction))\n\n;; After dot→space: (language (doc introduction))\n;; 1. Eval `introduction` → not a known function → \"introduction\"\n;; 2. Eval (doc \"introduction\") → call doc(\"introduction\") → page content\n;; 3. Eval (language content) → call language(content) → passes through\n;; 4. Router wraps result in (~layouts/doc :path \"(language (doc introduction))\" ...)\n\n/(~essays/sx-sucks/essay-sx-sucks)\n;; 1. Eval ~essays/sx-sucks/essay-sx-sucks → component lookup → evaluate → content\n;; 2. Router wraps in ~layouts/doc"
"lisp"))
(~doc-subsection :title "Section Functions"
(~docs/subsection :title "Section Functions"
(p "Structural functions that encode hierarchy and pass through content:")
(~doc-code :code (highlight
(~docs/code :code (highlight
"(define language\n (fn (&rest args)\n (if (empty? args) (language-index-content) (first args))))\n\n(define geography\n (fn (&rest args)\n (if (empty? args) (geography-index-content) (first args))))\n\n;; Sub-sections also pass through\n(define hypermedia\n (fn (&rest args)\n (if (empty? args) (hypermedia-index-content) (first args))))"
"lisp")))
(~doc-subsection :title "Page Functions"
(~docs/subsection :title "Page Functions"
(p "Leaf functions that dispatch to content components. "
"Data-dependent pages call helpers directly — the async evaluator handles IO:")
(~doc-code :code (highlight
"(define doc\n (fn (&rest args)\n (let ((slug (first-or-nil args)))\n (if (nil? slug)\n (~docs-introduction-content)\n (case slug\n \"introduction\" (~docs-introduction-content)\n \"getting-started\" (~docs-getting-started-content)\n ...)))))\n\n(define bootstrapper\n (fn (&rest args)\n (let ((slug (first-or-nil args))\n (data (when slug (bootstrapper-data slug))))\n (if (nil? slug)\n (~bootstrappers-index-content)\n (if (get data \"bootstrapper-not-found\")\n (~spec-not-found :slug slug)\n (case slug\n \"python\" (~bootstrapper-py-content ...)\n ...))))))"
(~docs/code :code (highlight
"(define doc\n (fn (&rest args)\n (let ((slug (first-or-nil args)))\n (if (nil? slug)\n (~docs-content/docs-introduction-content)\n (case slug\n \"introduction\" (~docs-content/docs-introduction-content)\n \"getting-started\" (~docs-content/docs-getting-started-content)\n ...)))))\n\n(define bootstrapper\n (fn (&rest args)\n (let ((slug (first-or-nil args))\n (data (when slug (bootstrapper-data slug))))\n (if (nil? slug)\n (~specs/bootstrappers-index-content)\n (if (get data \"bootstrapper-not-found\")\n (~specs/not-found :slug slug)\n (case slug\n \"python\" (~specs/bootstrapper-py-content ...)\n ...))))))"
"lisp"))))
(~doc-section :title "The Catch-All Route" :id "route"
(~docs/section :title "The Catch-All Route" :id "route"
(p "The entire routing layer becomes one handler:")
(~doc-code :code (highlight
(~docs/code :code (highlight
"@app.get(\"/\")\nasync def sx_home():\n return await eval_sx_url(\"/\")\n\n@app.get(\"/<path:expr>\")\nasync def sx_eval_route(expr):\n return await eval_sx_url(f\"/{expr}\")"
"python"))
(p (code "eval_sx_url") " in seven steps:")
@@ -246,17 +246,17 @@
(li "Parse as SX expression")
(li "Auto-quote unknown symbols (slugs become strings)")
(li "Evaluate with components + helpers + page/section functions in env")
(li "Wrap result in " (code "~sx-doc") " with the URL expression as " (code ":path"))
(li "Wrap result in " (code "~layouts/doc") " with the URL expression as " (code ":path"))
(li "Return HTML or SX wire format depending on HTMX request"))
(p "Defhandler API endpoints and Python demo routes are registered " (em "before") " the catch-all, "
"so they match first."))
(~doc-section :title "Composability" :id "composability"
(~doc-code :code (highlight
";; Direct component access\n/(~essay-sx-sucks)\n/(~spec-explorer-content)\n\n;; URL special forms\n/(source.(~essay-sx-sucks)) ;; view defcomp source\n/(inspect.(language.(doc.primitives))) ;; deps, render plan\n/(diff.(language.(spec.signals)).(language.(spec.eval))) ;; side by side\n/(eval.(map.double.(list.1.2.3))) ;; REPL in the URL bar\n\n;; Components as query resolvers\n/(~page.home\n .(~hero.banner)\n .(~get.latest-posts.(limit.5))\n .(~get.featured.(filter.pinned)))\n\n;; Scoping is explicit\n/(users.(posts.123.(filter.published))) ;; filter scoped to posts\n/(users.posts.123.(filter.published)) ;; filter scoped to users\n\n;; Cross-service (future)\n/(market.(product.42.:fields.(name.price)))\n/(subscribe.(etc.(plan.status)))"
(~docs/section :title "Composability" :id "composability"
(~docs/code :code (highlight
";; Direct component access\n/(~essays/sx-sucks/essay-sx-sucks)\n/(~specs-explorer/spec-explorer-content)\n\n;; URL special forms\n/(source.(~essays/sx-sucks/essay-sx-sucks)) ;; view defcomp source\n/(inspect.(language.(doc.primitives))) ;; deps, render plan\n/(diff.(language.(spec.signals)).(language.(spec.eval))) ;; side by side\n/(eval.(map.double.(list.1.2.3))) ;; REPL in the URL bar\n\n;; Components as query resolvers\n/(~page.home\n .(~hero.banner)\n .(~get.latest-posts.(limit.5))\n .(~get.featured.(filter.pinned)))\n\n;; Scoping is explicit\n/(users.(posts.123.(filter.published))) ;; filter scoped to posts\n/(users.posts.123.(filter.published)) ;; filter scoped to users\n\n;; Cross-service (future)\n/(market.(product.42.:fields.(name.price)))\n/(subscribe.(etc.(plan.status)))"
"lisp")))
(~doc-section :title "Implementation Phases" :id "phases"
(~docs/section :title "Implementation Phases" :id "phases"
(div :class "space-y-4"
(div :class "rounded border border-violet-200 bg-violet-50 p-4"
(p :class "font-semibold text-violet-800 mb-2" "Phase 1: Page Functions + Catch-All Route")
@@ -292,7 +292,7 @@
(li "Delete " (code "docs.sx") " (all 46 defpages)")
(li "Grep content files for stale old-style hrefs")))))
(~doc-section :title "What Stays the Same" :id "unchanged"
(~docs/section :title "What Stays the Same" :id "unchanged"
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li (strong "Defhandler API paths") " — registered before the catch-all, match first")
(li (strong "Python demo routes") " — registered via blueprint before the catch-all")

16
sx/sx/plans/sx-web-platform.sx
View File

@@ -2,10 +2,10 @@
;; Plan: transform sx-web.org from documentation site into a live development
;; environment where content is authored, tested, and deployed in the browser.
(defcomp ~plan-sx-web-platform-content ()
(~doc-page :title "sx-web.org Development Platform"
(defcomp ~plans/sx-web-platform/plan-sx-web-platform-content ()
(~docs/page :title "sx-web.org Development Platform"
(~doc-section :title "Vision" :id "vision"
(~docs/section :title "Vision" :id "vision"
(p "sx-web.org becomes the development environment for itself. "
"Authors write essays, examples, components, and specs directly in the browser. "
"Changes are planned, staged, tested, and deployed without leaving the site. "
@@ -15,7 +15,7 @@
"The entire development lifecycle happens over the web, using the same SX primitives "
"that the platform is built from."))
(~doc-section :title "Architecture" :id "architecture"
(~docs/section :title "Architecture" :id "architecture"
(p "The platform composes existing SX subsystems into a unified workflow:")
(div :class "overflow-x-auto mt-4"
(table :class "w-full text-sm text-left"
@@ -50,7 +50,7 @@
(td :class "py-2 px-3" "Environment images")
(td :class "py-2 px-3" "Spec CID \u2192 image CID \u2192 endpoint provenance"))))))
(~doc-section :title "Embedded Claude Code" :id "claude-code"
(~docs/section :title "Embedded Claude Code" :id "claude-code"
(p "Claude Code sessions run inside the browser as reactive islands. "
"The AI has access to the full SX component environment — it can read specs, "
"write components, run tests, and propose changes. All within the user's security context.")
@@ -64,7 +64,7 @@
(li "Stage changes as content-addressed preview")
(li "Publish via sx-activity when approved")))
(~doc-section :title "Workflow" :id "workflow"
(~docs/section :title "Workflow" :id "workflow"
(p "A typical session — adding a new essay:")
(ol :class "space-y-3 text-stone-600 list-decimal pl-5"
(li (strong "Author: ") "Open Claude Code session on sx-web.org. "
@@ -80,7 +80,7 @@
(li (strong "Verify: ") "Anyone can follow the CID chain from the served page "
"back to the spec that generated the evaluator that rendered it.")))
(~doc-section :title "Content Types" :id "content-types"
(~docs/section :title "Content Types" :id "content-types"
(p "Anything that can be a defcomp can be authored on the platform:")
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li (strong "Essays") " — opinion pieces, rationales, explorations")
@@ -90,7 +90,7 @@
(li (strong "Components") " — reusable UI components shared via IPFS")
(li (strong "Tests") " — defsuite/deftest written and executed live")))
(~doc-section :title "Prerequisites" :id "prerequisites"
(~docs/section :title "Prerequisites" :id "prerequisites"
(p "Systems that must be complete before the platform can work:")
(ul :class "space-y-2 text-stone-600 list-disc pl-5"
(li (strong "Reactive islands (L2+)") " — for the editor and preview panes")

48
sx/sx/plans/theorem-prover.sx
View File

@@ -3,7 +3,7 @@
;; ---------------------------------------------------------------------------
;; Helper: render a Phase 1 result row
(defcomp ~prove-phase1-row (&key (name :as string) (status :as string))
(defcomp ~plans/theorem-prover/prove-phase1-row (&key (name :as string) (status :as string))
(tr :class "border-t border-stone-100"
(td :class "py-1.5 px-3 font-mono text-xs text-stone-700" name)
(td :class "py-1.5 px-3 text-xs"
@@ -12,7 +12,7 @@
(span :class "text-red-600 font-medium" status)))))
;; Helper: render a Phase 2 result row
(defcomp ~prove-phase2-row (&key (name :as string) (status :as string) (tested :as number) (skipped :as number) (counterexample :as string?))
(defcomp ~plans/theorem-prover/prove-phase2-row (&key (name :as string) (status :as string) (tested :as number) (skipped :as number) (counterexample :as string?))
(tr :class "border-t border-stone-100"
(td :class "py-1.5 px-3 font-mono text-xs text-stone-700" name)
(td :class "py-1.5 px-3 text-xs"
@@ -27,11 +27,11 @@
(or counterexample ""))))
(defcomp ~plan-theorem-prover-content ()
(~doc-page :title "Theorem Prover"
(defcomp ~plans/theorem-prover/plan-theorem-prover-content ()
(~docs/page :title "Theorem Prover"
;; --- Intro ---
(~doc-section :title "SX proves itself" :id "intro"
(~docs/section :title "SX proves itself" :id "intro"
(p :class "text-stone-600"
(code "prove.sx") " is a constraint solver and property prover written in SX. It takes the SX specification (" (code "primitives.sx") "), translates it to formal logic via " (code "z3.sx") ", and proves properties about the result. Every step in the pipeline is an s-expression program operating on other s-expressions.")
(p :class "text-stone-600"
@@ -42,10 +42,10 @@
"No external solver. No Python proof logic. " (code "prove.sx") " is 400+ lines of s-expressions that parse SMT-LIB, evaluate expressions, generate test domains, compute cartesian products, search for counterexamples, and produce verification conditions. The same code would work client-side via the bootstrapped JavaScript evaluator.")))
;; --- Phase 1 Results ---
(~doc-section :title "Phase 1: Definitional satisfiability" :id "phase1"
(~docs/section :title "Phase 1: Definitional satisfiability" :id "phase1"
(p :class "text-stone-600"
"Every " (code "define-primitive") " with a " (code ":body") " produces a " (code "forall") " assertion in SMT-LIB. For example, " (code "(define-primitive \"inc\" :params (n) :body (+ n 1))") " becomes:")
(~doc-code :code (highlight "; inc\n(declare-fun inc (Int) Int)\n(assert (forall (((n Int)))\n (= (inc n) (+ n 1))))\n(check-sat)" "lisp"))
(~docs/code :code (highlight "; inc\n(declare-fun inc (Int) Int)\n(assert (forall (((n Int)))\n (= (inc n) (+ n 1))))\n(check-sat)" "lisp"))
(p :class "text-stone-600"
"This is satisfiable by construction: define " (code "inc(n) = n + 1") " and the assertion holds. " (code "prove.sx") " verifies this mechanically for every primitive — it parses the SMT-LIB, extracts the definition, builds a model, and evaluates it with test values.")
@@ -57,7 +57,7 @@
(p :class "text-xs text-emerald-700 mt-1"
"Computed live in " (str phase1-ms) "ms"))
(~doc-subsection :title "Results"
(~docs/subsection :title "Results"
(div :class "overflow-x-auto rounded border border-stone-200 max-h-64 overflow-y-auto"
(table :class "w-full text-left"
(thead
@@ -66,13 +66,13 @@
(th :class "py-2 px-3 text-xs text-stone-500 font-medium" "Status")))
(tbody
(map (fn (r)
(~prove-phase1-row
(~plans/theorem-prover/prove-phase1-row
:name (get r "name")
:status (get r "status")))
phase1-results))))))
;; --- Phase 2 Results ---
(~doc-section :title "Phase 2: Algebraic properties" :id "phase2"
(~docs/section :title "Phase 2: Algebraic properties" :id "phase2"
(p :class "text-stone-600"
"Phase 1 proves internal consistency. Phase 2 proves " (em "external properties") " — mathematical laws that should hold across all inputs. Each property is defined as a test function evaluated over a bounded integer domain.")
(p :class "text-stone-600"
@@ -86,7 +86,7 @@
(p :class "text-xs text-emerald-700 mt-1"
(str phase2-total-tested " constraint evaluations in " phase2-ms "ms")))
(~doc-subsection :title "Results"
(~docs/subsection :title "Results"
(div :class "overflow-x-auto rounded border border-stone-200 max-h-96 overflow-y-auto"
(table :class "w-full text-left"
(thead
@@ -98,7 +98,7 @@
(th :class "py-2 px-3 text-xs text-stone-500 font-medium" "Counterexample")))
(tbody
(map (fn (r)
(~prove-phase2-row
(~plans/theorem-prover/prove-phase2-row
:name (get r "name")
:status (get r "status")
:tested (get r "tested")
@@ -107,7 +107,7 @@
phase2-results))))))
;; --- What the properties prove ---
(~doc-section :title "What the properties prove" :id "properties"
(~docs/section :title "What the properties prove" :id "properties"
(p :class "text-stone-600"
"34 properties across seven categories. Each encodes a mathematical law that the SX primitives must obey.")
@@ -163,23 +163,23 @@
(code "(!= a b) = (not (= a b))") "."))))
;; --- SMT-LIB output ---
(~doc-section :title "SMT-LIB verification conditions" :id "smtlib"
(~docs/section :title "SMT-LIB verification conditions" :id "smtlib"
(p :class "text-stone-600"
"Each property also generates SMT-LIB for unbounded verification by an external solver. The strategy: assert the " (em "negation") " of the universal property. If Z3 returns " (code "unsat") ", the property holds for " (em "all") " integers — not just the bounded domain.")
(p :class "text-stone-600"
(code "prove.sx") " reuses " (code "z3-expr") " from " (code "z3.sx") " to translate the property AST to SMT-LIB. Properties with preconditions use " (code "=>") " (implication). The same SX expression is both the bounded test and the formal verification condition.")
(~doc-code :code (highlight smtlib-sample "lisp")))
(~docs/code :code (highlight smtlib-sample "lisp")))
;; --- What it tells us ---
(~doc-section :title "What this tells us about SX" :id "implications"
(~docs/section :title "What this tells us about SX" :id "implications"
(p :class "text-stone-600"
"Three things, at increasing depth.")
(~doc-subsection :title "1. The spec is internally consistent"
(~docs/subsection :title "1. The spec is internally consistent"
(p :class "text-stone-600"
"Phase 1 proves every " (code "define-primitive") " with a " (code ":body") " is satisfiable. The definition doesn't contradict itself. This is necessary but weak — it's true by construction. The value is mechanical verification: no typo, no copy-paste error, no accidental negation in any of the 91 definitions."))
(~doc-subsection :title "2. The primitives obey algebraic laws"
(~docs/subsection :title "2. The primitives obey algebraic laws"
(p :class "text-stone-600"
"Phase 2 proves real mathematical properties hold across bounded domains. These aren't tautologies — they're constraints that " (em "could") " fail. "
(code "(+ a b) = (+ b a)") " could fail if " (code "+") " had a subtle bug. "
@@ -188,7 +188,7 @@
(p :class "text-stone-600"
"Bounded model checking is not a mathematical proof — it verifies over a finite domain. The SMT-LIB output bridges the gap: feed it to Z3 for a universal proof over all integers."))
(~doc-subsection :title "3. SX can reason about itself"
(~docs/subsection :title "3. SX can reason about itself"
(p :class "text-stone-600"
"The deep result. The SX evaluator executes " (code "z3.sx") ", which reads SX spec files and emits formal logic. Then the SX evaluator executes " (code "prove.sx") ", which parses that logic and proves properties about it. The specification, the translator, and the prover are all written in the same language, operating on the same data structures.")
(p :class "text-stone-600"
@@ -199,7 +199,7 @@
"The SX spec defines primitives. " (code "z3.sx") " (written in SX, using those primitives) translates the spec to formal logic. " (code "prove.sx") " (written in SX, using those same primitives) proves properties about the logic. The primitives being verified are the same primitives doing the verifying. This is not circular — it's a fixed point. If the primitives were wrong, the proofs would fail."))))
;; --- The pipeline ---
(~doc-section :title "The full pipeline" :id "pipeline"
(~docs/section :title "The full pipeline" :id "pipeline"
(div :class "rounded border border-stone-200 bg-stone-50 p-4"
(table :class "w-full text-sm"
(thead (tr :class "text-left text-stone-500"
@@ -237,7 +237,7 @@
"Steps 1-3 run on this page, live, in the SX evaluator. Step 4 requires an external Z3 installation — the SMT-LIB output above is ready to feed to it."))
;; --- Source ---
(~doc-section :title "The source: prove.sx" :id "source"
(~docs/section :title "The source: prove.sx" :id "source"
(p :class "text-stone-600"
"The entire constraint solver is a single SX file. Key sections: "
(code "smt-eval") " evaluates SMT-LIB expressions. "
@@ -245,9 +245,9 @@
(code "prove-search") " walks tuples looking for counterexamples. "
(code "sx-properties") " declares 34 algebraic laws as test functions with quoted ASTs. "
(code "prove-property-smtlib") " translates properties to SMT-LIB verification conditions via " (code "z3-expr") ".")
(~doc-code :code (highlight prove-source "lisp")))
(~docs/code :code (highlight prove-source "lisp")))
(~doc-section :title "The translator: z3.sx" :id "z3-source"
(~docs/section :title "The translator: z3.sx" :id "z3-source"
(p :class "text-stone-600"
"The translator that " (code "prove.sx") " depends on. SX expressions that walk other SX expressions and emit SMT-LIB strings. Both files together: ~760 lines of s-expressions, no host language logic.")
(~doc-code :code (highlight z3-source "lisp")))))
(~docs/code :code (highlight z3-source "lisp")))))

140
sx/sx/plans/typed-sx.sx
View File

@@ -2,10 +2,10 @@
;; Typed SX — Gradual Type System
;; ---------------------------------------------------------------------------
(defcomp ~plan-typed-sx-content ()
(~doc-page :title "Typed SX"
(defcomp ~plans/typed-sx/plan-typed-sx-content ()
(~docs/page :title "Typed SX"
(~doc-section :title "The Opportunity" :id "opportunity"
(~docs/section :title "The Opportunity" :id "opportunity"
(p "SX already has types. Every primitive in " (code "primitives.sx") " declares " (code ":returns \"number\"") " or " (code ":returns \"boolean\"") ". Every IO primitive in " (code "boundary.sx") " declares " (code ":returns \"dict?\"") " or " (code ":returns \"any\"") ". Component params are named. The information exists — nobody checks it.")
(p "A gradual type system makes this information useful. Annotations are optional. Unannotated code works exactly as before. Annotated code gets checked at registration time — zero runtime cost, errors before any request is served. The checker is a spec module (" (code "types.sx") "), bootstrapped to every host.")
(p "This is not Haskell. SX doesn't need a type system to be correct — " (a :href "/sx/(etc.(plan.theorem-prover))" :class "text-violet-700 underline" "prove.sx") " already verifies primitive properties by exhaustive search. Types serve a different purpose: they catch " (strong "composition errors") " — wrong argument passed to a component, mismatched return type piped into another function, missing keyword arg. The kind of bug you find by reading the stack trace and slapping your forehead."))
@@ -14,7 +14,7 @@
;; What already exists
;; -----------------------------------------------------------------------
(~doc-section :title "What Already Exists" :id "existing"
(~docs/section :title "What Already Exists" :id "existing"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -57,42 +57,42 @@
;; Type language
;; -----------------------------------------------------------------------
(~doc-section :title "Type Language" :id "type-language"
(~docs/section :title "Type Language" :id "type-language"
(p "Small, practical, no type theory PhD required.")
(~doc-subsection :title "Base types"
(~doc-code :code (highlight ";; Atomic types\nnumber string boolean nil symbol keyword element\n\n;; Nullable (already used in boundary.sx)\nstring? ;; (or string nil)\ndict? ;; (or dict nil)\nnumber? ;; (or number nil)\n\n;; The top type — anything goes\nany\n\n;; The bottom type — never returns (e.g. abort)\nnever" "lisp")))
(~docs/subsection :title "Base types"
(~docs/code :code (highlight ";; Atomic types\nnumber string boolean nil symbol keyword element\n\n;; Nullable (already used in boundary.sx)\nstring? ;; (or string nil)\ndict? ;; (or dict nil)\nnumber? ;; (or number nil)\n\n;; The top type — anything goes\nany\n\n;; The bottom type — never returns (e.g. abort)\nnever" "lisp")))
(~doc-subsection :title "Compound types"
(~doc-code :code (highlight ";; Collections with element types\n(list-of number) ;; list where every element is a number\n(list-of string) ;; list of strings\n(list-of any) ;; list (same as untyped)\n(dict-of string number) ;; dict with string keys, number values\n(dict-of string any) ;; dict with string keys (typical kwargs)\n\n;; Union types\n(or string number) ;; either string or number\n(or string nil) ;; same as string?\n\n;; Function types\n(-> number number) ;; number → number\n(-> string string boolean) ;; (string, string) → boolean\n(-> (list-of any) number) ;; list → number\n(-> &rest any number) ;; variadic → number" "lisp")))
(~docs/subsection :title "Compound types"
(~docs/code :code (highlight ";; Collections with element types\n(list-of number) ;; list where every element is a number\n(list-of string) ;; list of strings\n(list-of any) ;; list (same as untyped)\n(dict-of string number) ;; dict with string keys, number values\n(dict-of string any) ;; dict with string keys (typical kwargs)\n\n;; Union types\n(or string number) ;; either string or number\n(or string nil) ;; same as string?\n\n;; Function types\n(-> number number) ;; number → number\n(-> string string boolean) ;; (string, string) → boolean\n(-> (list-of any) number) ;; list → number\n(-> &rest any number) ;; variadic → number" "lisp")))
(~doc-subsection :title "Component types"
(~doc-code :code (highlight ";; Component type is its keyword signature\n;; Derived automatically from defcomp — no annotation needed\n\n(comp :title string :price number &rest element)\n;; keyword args children type\n\n;; This is NOT a new syntax for defcomp.\n;; It's the TYPE that a defcomp declaration produces.\n;; The checker infers it from parse-comp-params + annotations." "lisp")))
(~docs/subsection :title "Component types"
(~docs/code :code (highlight ";; Component type is its keyword signature\n;; Derived automatically from defcomp — no annotation needed\n\n(comp :title string :price number &rest element)\n;; keyword args children type\n\n;; This is NOT a new syntax for defcomp.\n;; It's the TYPE that a defcomp declaration produces.\n;; The checker infers it from parse-comp-params + annotations." "lisp")))
(p "That's the core. No higher-kinded types, no dependent types, no type classes. Just: what goes in, what comes out, can it be nil.")
(~doc-subsection :title "User-defined types"
(~doc-code :code (highlight ";; Type alias — a name for an existing type\n(deftype price number)\n(deftype html-string string)\n\n;; Union — one of several types\n(deftype renderable (union string number nil))\n(deftype key-type (union string keyword))\n\n;; Record — typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number})\n\n;; Parameterized — generic over a type variable\n(deftype (maybe a) (union nil a))\n(deftype (list-of-pairs a b) (list-of (dict-of a b)))\n(deftype (result a e) (union (ok a) (err e)))" "lisp"))
(~docs/subsection :title "User-defined types"
(~docs/code :code (highlight ";; Type alias — a name for an existing type\n(deftype price number)\n(deftype html-string string)\n\n;; Union — one of several types\n(deftype renderable (union string number nil))\n(deftype key-type (union string keyword))\n\n;; Record — typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number})\n\n;; Parameterized — generic over a type variable\n(deftype (maybe a) (union nil a))\n(deftype (list-of-pairs a b) (list-of (dict-of a b)))\n(deftype (result a e) (union (ok a) (err e)))" "lisp"))
(p (code "deftype") " is a declaration form — zero runtime cost, purely for the checker. The type registry resolves user-defined type names during " (code "subtype?") " and " (code "infer-type") ". Records enable typed keyword args for components:")
(~doc-code :code (highlight ";; Define a prop shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :on-click (-> any nil)?})\n\n;; Use it in a component\n(defcomp ~product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n (span (format-decimal (get props :price) 2))\n children))\n\n;; Checker validates dict literals against record shape:\n(~product-card :props {:title \"Widget\" :price \"oops\"})\n;; ^^^^^^\n;; ERROR: :price expects number, got string" "lisp"))))
(~docs/code :code (highlight ";; Define a prop shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :on-click (-> any nil)?})\n\n;; Use it in a component\n(defcomp ~plans/typed-sx/product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n (span (format-decimal (get props :price) 2))\n children))\n\n;; Checker validates dict literals against record shape:\n(~plans/typed-sx/product-card :props {:title \"Widget\" :price \"oops\"})\n;; ^^^^^^\n;; ERROR: :price expects number, got string" "lisp"))))
;; -----------------------------------------------------------------------
;; Annotation syntax
;; -----------------------------------------------------------------------
(~doc-section :title "Annotation Syntax" :id "syntax"
(~docs/section :title "Annotation Syntax" :id "syntax"
(p "Annotations are optional. Three places they can appear:")
(~doc-subsection :title "1. Component params"
(~doc-code :code (highlight ";; Current (unchanged, still works)\n(defcomp ~product-card (&key title price image-url &rest children)\n (div ...))\n\n;; Annotated — colon after param name\n(defcomp ~product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n (div ...))\n\n;; Parenthesized pairs: (name : type)\n;; Unannotated params default to `any`\n;; &rest children is always (list-of element)" "lisp"))
(~docs/subsection :title "1. Component params"
(~docs/code :code (highlight ";; Current (unchanged, still works)\n(defcomp ~plans/typed-sx/product-card (&key title price image-url &rest children)\n (div ...))\n\n;; Annotated — colon after param name\n(defcomp ~plans/typed-sx/product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n (div ...))\n\n;; Parenthesized pairs: (name : type)\n;; Unannotated params default to `any`\n;; &rest children is always (list-of element)" "lisp"))
(p "The " (code "(name : type)") " syntax is unambiguous — a 3-element list where the second element is the symbol " (code ":") ". The parser already handles lists inside parameter lists. " (code "parse-comp-params") " gains a branch: if a param is a list of length 3 with " (code ":") " in the middle, extract name and type."))
(~doc-subsection :title "2. Define/lambda return types"
(~doc-code :code (highlight ";; Current (unchanged)\n(define total-price\n (fn (items)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))\n\n;; Annotated — :returns after params\n(define total-price\n (fn ((items : (list-of dict)) :returns number)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))" "lisp"))
(~docs/subsection :title "2. Define/lambda return types"
(~docs/code :code (highlight ";; Current (unchanged)\n(define total-price\n (fn (items)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))\n\n;; Annotated — :returns after params\n(define total-price\n (fn ((items : (list-of dict)) :returns number)\n (reduce + 0 (map (fn (i) (get i \"price\")) items))))" "lisp"))
(p (code ":returns") " is already the convention in " (code "primitives.sx") " and " (code "boundary.sx") ". Same keyword, same position (after params), same meaning."))
(~doc-subsection :title "3. Let bindings"
(~doc-code :code (highlight ";; Current (unchanged)\n(let ((x (compute-value)))\n (+ x 1))\n\n;; Annotated\n(let (((x : number) (compute-value)))\n (+ x 1))\n\n;; Usually unnecessary — the checker infers let binding\n;; types from the right-hand side. Only annotate when\n;; the inference is ambiguous (e.g. the RHS returns `any`)." "lisp"))
(~docs/subsection :title "3. Let bindings"
(~docs/code :code (highlight ";; Current (unchanged)\n(let ((x (compute-value)))\n (+ x 1))\n\n;; Annotated\n(let (((x : number) (compute-value)))\n (+ x 1))\n\n;; Usually unnecessary — the checker infers let binding\n;; types from the right-hand side. Only annotate when\n;; the inference is ambiguous (e.g. the RHS returns `any`)." "lisp"))
(p "All annotations are syntactically backward-compatible. Unannotated code parses and runs identically. The annotations are simply ignored by evaluators that don't have the type checker loaded."))
@@ -100,19 +100,19 @@
;; Type checking
;; -----------------------------------------------------------------------
(~doc-section :title "Type Checking" :id "checking"
(~docs/section :title "Type Checking" :id "checking"
(p "The checker runs at registration time — after " (code "compute_all_deps") ", before serving. It walks every component's body AST and verifies that call sites match declared signatures.")
(~doc-subsection :title "What it checks"
(~docs/subsection :title "What it checks"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li (strong "Primitive calls:") " " (code "(+ \"hello\" 3)") " — " (code "+") " expects numbers, got a string. Error.")
(li (strong "Component calls:") " " (code "(~product-card :title 42)") " — " (code ":title") " declared as " (code "string") ", got " (code "number") ". Error.")
(li (strong "Missing required params:") " " (code "(~product-card :price 29.99)") " — " (code ":title") " not provided, no default. Error.")
(li (strong "Unknown keyword args:") " " (code "(~product-card :title \"Hi\" :colour \"red\")") " — " (code ":colour") " not in param list. Warning.")
(li (strong "Component calls:") " " (code "(~plans/typed-sx/product-card :title 42)") " — " (code ":title") " declared as " (code "string") ", got " (code "number") ". Error.")
(li (strong "Missing required params:") " " (code "(~plans/typed-sx/product-card :price 29.99)") " — " (code ":title") " not provided, no default. Error.")
(li (strong "Unknown keyword args:") " " (code "(~plans/typed-sx/product-card :title \"Hi\" :colour \"red\")") " — " (code ":colour") " not in param list. Warning.")
(li (strong "Nil safety:") " " (code "(+ 1 (get user \"age\"))") " — " (code "get") " returns " (code "any") " (might be nil). " (code "+") " expects " (code "number") ". Warning: possible nil.")
(li (strong "Thread-first type flow:") " " (code "(-> items (filter active?) (map name) (join \", \"))") " — checks each step's output matches the next step's input.")))
(~doc-subsection :title "What it does NOT check"
(~docs/subsection :title "What it does NOT check"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "Runtime values.") " " (code "(if condition 42 \"hello\")") " — the type is " (code "(or number string)") ". The checker doesn't know which branch executes.")
(li (strong "Dict key presence (yet).") " " (code "(get user \"name\")") " — the checker knows " (code "get") " returns " (code "any") " but doesn't track which keys a dict has. Phase 6 (" (code "deftype") " records) will enable this.")
@@ -120,7 +120,7 @@
(li (strong "Full algebraic effects.") " The effect system (Phase 7) checks static effect annotations — it does not provide algebraic effect handlers, effect polymorphism, or continuation-based effect dispatch. That door remains open for the future.")))
(~doc-subsection :title "Inference"
(~docs/subsection :title "Inference"
(p "Most types are inferred, not annotated. The checker knows:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Literal types: " (code "42") " → " (code "number") ", " (code "\"hi\"") " → " (code "string") ", " (code "true") " → " (code "boolean") ", " (code "nil") " → " (code "nil"))
@@ -135,7 +135,7 @@
;; Gradual semantics
;; -----------------------------------------------------------------------
(~doc-section :title "Gradual Semantics" :id "gradual"
(~docs/section :title "Gradual Semantics" :id "gradual"
(p "The type " (code "any") " is the escape hatch. It's compatible with everything — passes every check, accepts every value. Unannotated params are " (code "any") ". The return type of " (code "get") " is " (code "any") ". This means:")
(ul :class "list-disc pl-5 text-stone-700 space-y-2"
@@ -145,16 +145,16 @@
(p "The practical sweet spot: " (strong "annotate component params, nothing else.") " Components are the public API — the boundary between independent pieces of code. Their params are the contract. Internal lambdas and let bindings benefit less from annotations because the checker can infer their types from context.")
(~doc-code :code (highlight ";; Sweet spot: annotate the interface, infer the rest\n(defcomp ~price-display (&key (price : number)\n (currency : string)\n (sale-price : number?))\n ;; Everything below is inferred:\n ;; formatted → string (str returns string)\n ;; discount → number (- returns number)\n ;; has-sale → boolean (and returns boolean)\n (let ((formatted (str currency (format-number price 2)))\n (has-sale (and sale-price (< sale-price price)))\n (discount (if has-sale\n (round (* 100 (/ (- price sale-price) price)))\n 0)))\n (div :class \"price\"\n (span :class (if has-sale \"line-through text-stone-400\" \"font-bold\")\n formatted)\n (when has-sale\n (span :class \"text-green-700 font-bold ml-2\"\n (str currency (format-number sale-price 2))\n (span :class \"text-xs ml-1\" (str \"(-\" discount \"%)\")))))))" "lisp")))
(~docs/code :code (highlight ";; Sweet spot: annotate the interface, infer the rest\n(defcomp ~plans/typed-sx/price-display (&key (price : number)\n (currency : string)\n (sale-price : number?))\n ;; Everything below is inferred:\n ;; formatted → string (str returns string)\n ;; discount → number (- returns number)\n ;; has-sale → boolean (and returns boolean)\n (let ((formatted (str currency (format-number price 2)))\n (has-sale (and sale-price (< sale-price price)))\n (discount (if has-sale\n (round (* 100 (/ (- price sale-price) price)))\n 0)))\n (div :class \"price\"\n (span :class (if has-sale \"line-through text-stone-400\" \"font-bold\")\n formatted)\n (when has-sale\n (span :class \"text-green-700 font-bold ml-2\"\n (str currency (format-number sale-price 2))\n (span :class \"text-xs ml-1\" (str \"(-\" discount \"%)\")))))))" "lisp")))
;; -----------------------------------------------------------------------
;; Error reporting
;; -----------------------------------------------------------------------
(~doc-section :title "Error Reporting" :id "errors"
(~docs/section :title "Error Reporting" :id "errors"
(p "Type errors are reported at registration time with source location, expected type, actual type, and the full call chain.")
(~doc-code :code (highlight ";; Example error output:\n;;\n;; TYPE ERROR in ~checkout-summary (checkout.sx:34)\n;;\n;; (str \"Total: \" (compute-total items))\n;; ^^^^^^^^^^^^^^^^^\n;; Argument 2 of `str` expects: string\n;; Got: number (from compute-total :returns number)\n;;\n;; Fix: (str \"Total: \" (str (compute-total items)))\n;;\n;;\n;; TYPE ERROR in ~product-page (products.sx:12)\n;;\n;; (~product-card :title product-name :price \"29.99\")\n;; ^^^^^^\n;; Keyword :price of ~product-card expects: number\n;; Got: string (literal \"29.99\")\n;;\n;; Fix: (~product-card :title product-name :price 29.99)" "lisp"))
(~docs/code :code (highlight ";; Example error output:\n;;\n;; TYPE ERROR in ~checkout-summary (checkout.sx:34)\n;;\n;; (str \"Total: \" (compute-total items))\n;; ^^^^^^^^^^^^^^^^^\n;; Argument 2 of `str` expects: string\n;; Got: number (from compute-total :returns number)\n;;\n;; Fix: (str \"Total: \" (str (compute-total items)))\n;;\n;;\n;; TYPE ERROR in ~reactive-islands/event-bridge/product-page (products.sx:12)\n;;\n;; (~plans/typed-sx/product-card :title product-name :price \"29.99\")\n;; ^^^^^^\n;; Keyword :price of ~plans/typed-sx/product-card expects: number\n;; Got: string (literal \"29.99\")\n;;\n;; Fix: (~plans/typed-sx/product-card :title product-name :price 29.99)" "lisp"))
(p "Severity levels:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -183,10 +183,10 @@
;; Nil narrowing
;; -----------------------------------------------------------------------
(~doc-section :title "Nil Narrowing" :id "nil"
(~docs/section :title "Nil Narrowing" :id "nil"
(p "The most common real-world type error in SX: passing a possibly-nil value where a non-nil is required. " (code "get") " returns " (code "any") " (might be nil). " (code "current-user") " returns " (code "dict?") " (explicitly nullable). Piping these into " (code "str") " or arithmetic without checking is the #1 source of runtime errors.")
(~doc-code :code (highlight ";; Before: runtime error if user is nil\n(defcomp ~greeting (&key (user : dict?))\n (h1 (str \"Hello, \" (get user \"name\"))))\n ;; ^^^ TYPE WARNING: user is dict?, get needs non-nil first arg\n\n;; After: checker enforces nil handling\n(defcomp ~greeting (&key (user : dict?))\n (if user\n (h1 (str \"Hello, \" (get user \"name\")))\n ;; In this branch, checker narrows user to `dict` (not nil)\n (h1 \"Hello, guest\")))\n ;; No warning — nil case handled" "lisp"))
(~docs/code :code (highlight ";; Before: runtime error if user is nil\n(defcomp ~plans/typed-sx/greeting (&key (user : dict?))\n (h1 (str \"Hello, \" (get user \"name\"))))\n ;; ^^^ TYPE WARNING: user is dict?, get needs non-nil first arg\n\n;; After: checker enforces nil handling\n(defcomp ~plans/typed-sx/greeting (&key (user : dict?))\n (if user\n (h1 (str \"Hello, \" (get user \"name\")))\n ;; In this branch, checker narrows user to `dict` (not nil)\n (h1 \"Hello, guest\")))\n ;; No warning — nil case handled" "lisp"))
(p "Narrowing rules:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
@@ -200,10 +200,10 @@
;; Component signature verification
;; -----------------------------------------------------------------------
(~doc-section :title "Component Signature Verification" :id "signatures"
(~docs/section :title "Component Signature Verification" :id "signatures"
(p "The highest-value check: verifying that component call sites match declared signatures. This is where most bugs live.")
(~doc-code :code (highlight ";; Definition\n(defcomp ~product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n ...)\n\n;; Call site checks:\n(~product-card :title \"Widget\" :price 29.99) ;; OK\n(~product-card :title \"Widget\") ;; ERROR: :price required\n(~product-card :title 42 :price 29.99) ;; ERROR: :title expects string\n(~product-card :title \"Widget\" :price 29.99\n (p \"Description\") (p \"Details\")) ;; OK: children\n(~product-card :titel \"Widget\" :price 29.99) ;; WARNING: :titel unknown\n ;; (did you mean :title?)" "lisp"))
(~docs/code :code (highlight ";; Definition\n(defcomp ~plans/typed-sx/product-card (&key (title : string)\n (price : number)\n (image-url : string?)\n &rest children)\n ...)\n\n;; Call site checks:\n(~plans/typed-sx/product-card :title \"Widget\" :price 29.99) ;; OK\n(~plans/typed-sx/product-card :title \"Widget\") ;; ERROR: :price required\n(~plans/typed-sx/product-card :title 42 :price 29.99) ;; ERROR: :title expects string\n(~plans/typed-sx/product-card :title \"Widget\" :price 29.99\n (p \"Description\") (p \"Details\")) ;; OK: children\n(~plans/typed-sx/product-card :titel \"Widget\" :price 29.99) ;; WARNING: :titel unknown\n ;; (did you mean :title?)" "lisp"))
(p "The checker walks every component call in every component body. For each call:")
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
@@ -219,10 +219,10 @@
;; Thread-first type flow
;; -----------------------------------------------------------------------
(~doc-section :title "Thread-First Type Flow" :id "thread-first"
(~docs/section :title "Thread-First Type Flow" :id "thread-first"
(p "The " (code "->") " (thread-first) form is SX's primary composition operator. Type checking it means verifying each step's output matches the next step's input:")
(~doc-code :code (highlight ";; (-> items\n;; (filter active?) ;; (list-of dict) → (list-of dict)\n;; (map name) ;; (list-of dict) → (list-of string)\n;; (join \", \")) ;; (list-of string) → string\n;;\n;; Type flow: (list-of dict) → (list-of dict) → (list-of string) → string\n;; Each step's output is the next step's first argument.\n\n;; ERROR example:\n;; (-> items\n;; (filter active?)\n;; (join \", \") ;; join expects (list-of string),\n;; (map name)) ;; got string — wrong order!\n;;\n;; TYPE ERROR: step 3 (map) expects (list-of any) as first arg\n;; got: string (from join)" "lisp"))
(~docs/code :code (highlight ";; (-> items\n;; (filter active?) ;; (list-of dict) → (list-of dict)\n;; (map name) ;; (list-of dict) → (list-of string)\n;; (join \", \")) ;; (list-of string) → string\n;;\n;; Type flow: (list-of dict) → (list-of dict) → (list-of string) → string\n;; Each step's output is the next step's first argument.\n\n;; ERROR example:\n;; (-> items\n;; (filter active?)\n;; (join \", \") ;; join expects (list-of string),\n;; (map name)) ;; got string — wrong order!\n;;\n;; TYPE ERROR: step 3 (map) expects (list-of any) as first arg\n;; got: string (from join)" "lisp"))
(p "The checker threads the inferred type through each step. If any step's input type doesn't match the previous step's output type, it reports the exact point where the pipeline breaks."))
@@ -230,7 +230,7 @@
;; Relationship to prove.sx
;; -----------------------------------------------------------------------
(~doc-section :title "Types vs Proofs" :id "types-vs-proofs"
(~docs/section :title "Types vs Proofs" :id "types-vs-proofs"
(p (a :href "/sx/(etc.(plan.theorem-prover))" :class "text-violet-700 underline" "prove.sx") " and types.sx are complementary, not competing:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -267,43 +267,43 @@
;; User-defined types
;; -----------------------------------------------------------------------
(~doc-section :title "User-Defined Types" :id "deftype"
(~docs/section :title "User-Defined Types" :id "deftype"
(p (code "deftype") " introduces named types — aliases, unions, records, and parameterized types. All are declaration-only, zero runtime cost, resolved at check time.")
(~doc-subsection :title "Type aliases"
(~doc-code :code (highlight ";; Simple name for an existing type\n(deftype price number)\n(deftype html-string string)\n(deftype user-id (or string number))\n\n;; Use anywhere a type is expected\n(defcomp ~price-tag (&key (amount :as price) (label :as string))\n (span :class \"price\" (str label \": $\" (format-decimal amount 2))))" "lisp"))
(~docs/subsection :title "Type aliases"
(~docs/code :code (highlight ";; Simple name for an existing type\n(deftype price number)\n(deftype html-string string)\n(deftype user-id (or string number))\n\n;; Use anywhere a type is expected\n(defcomp ~plans/typed-sx/price-tag (&key (amount :as price) (label :as string))\n (span :class \"price\" (str label \": $\" (format-decimal amount 2))))" "lisp"))
(p "Aliases are transparent — " (code "price") " IS " (code "number") " for all checking purposes. They exist for documentation and domain semantics."))
(~doc-subsection :title "Union types"
(~doc-code :code (highlight ";; Named unions\n(deftype renderable (union string number nil component))\n(deftype key-type (union string keyword))\n(deftype falsy (union nil false))\n\n;; The checker narrows unions in branches:\n(define handle-input\n (fn ((val :as (union string number)))\n (if (string? val)\n (upper val) ;; narrowed to string — upper is valid\n (+ val 1)))) ;; narrowed to number — + is valid" "lisp"))
(~docs/subsection :title "Union types"
(~docs/code :code (highlight ";; Named unions\n(deftype renderable (union string number nil component))\n(deftype key-type (union string keyword))\n(deftype falsy (union nil false))\n\n;; The checker narrows unions in branches:\n(define handle-input\n (fn ((val :as (union string number)))\n (if (string? val)\n (upper val) ;; narrowed to string — upper is valid\n (+ val 1)))) ;; narrowed to number — + is valid" "lisp"))
(p "Union types compose with narrowing — " (code "if (string? x)") " in the then-branch narrows " (code "(union string number)") " to " (code "string") ". Same flow typing that already works for nullable."))
(~doc-subsection :title "Record types (typed dicts)"
(~doc-code :code (highlight ";; Typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :tags (list-of string)})\n\n;; Checker validates dict literals against shape:\n{:title \"Widget\" :price \"oops\"}\n;; ERROR: :price expects number, got string\n\n{:title \"Widget\" :price 29.99}\n;; WARNING: missing :tags (required field)\n\n;; Record types enable typed component props:\n(defcomp ~product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n children))" "lisp"))
(~docs/subsection :title "Record types (typed dicts)"
(~docs/code :code (highlight ";; Typed dict shape\n(deftype card-props\n {:title string\n :subtitle string?\n :price number\n :tags (list-of string)})\n\n;; Checker validates dict literals against shape:\n{:title \"Widget\" :price \"oops\"}\n;; ERROR: :price expects number, got string\n\n{:title \"Widget\" :price 29.99}\n;; WARNING: missing :tags (required field)\n\n;; Record types enable typed component props:\n(defcomp ~plans/typed-sx/product-card (&key (props :as card-props) &rest children)\n (div :class \"card\"\n (h2 (get props :title))\n children))" "lisp"))
(p "Records are the big win. Components pass dicts everywhere — config, props, context. A record type makes " (code "get") " on a known-shape dict return the field's type instead of " (code "any") ". This is where " (code "deftype") " pays for itself."))
(~doc-subsection :title "Parameterized types"
(~doc-code :code (highlight ";; Generic over type variables\n(deftype (maybe a) (union nil a))\n(deftype (result a e) (union {:ok a} {:err e}))\n(deftype (pair a b) {:fst a :snd b})\n\n;; Used in signatures:\n(define find-user : (-> number (maybe user-record))\n (fn (id) ...))\n\n;; Checker instantiates: (maybe user-record) = (union nil user-record)\n;; So the caller must handle nil." "lisp"))
(~docs/subsection :title "Parameterized types"
(~docs/code :code (highlight ";; Generic over type variables\n(deftype (maybe a) (union nil a))\n(deftype (result a e) (union {:ok a} {:err e}))\n(deftype (pair a b) {:fst a :snd b})\n\n;; Used in signatures:\n(define find-user : (-> number (maybe user-record))\n (fn (id) ...))\n\n;; Checker instantiates: (maybe user-record) = (union nil user-record)\n;; So the caller must handle nil." "lisp"))
(p "Parameterized types are substitution-based — " (code "(maybe string)") " expands to " (code "(union nil string)") " at check time. No inference of type parameters (that would require Hindley-Milner). You write " (code "(maybe string)") " explicitly, the checker substitutes and verifies.")))
;; -----------------------------------------------------------------------
;; Effect system
;; -----------------------------------------------------------------------
(~doc-section :title "Effect System" :id "effects"
(~docs/section :title "Effect System" :id "effects"
(p "The pragmatic middle: static effect " (em "checking") " without algebraic effect " (em "handlers") ". Functions declare what side effects they use. The checker enforces that effects don't leak across boundaries. No continuations, no runtime cost.")
(~doc-subsection :title "Effect declarations"
(~doc-code :code (highlight ";; Declare named effects\n(defeffect io) ;; Database, HTTP, file system\n(defeffect dom) ;; Browser DOM manipulation\n(defeffect async) ;; Asynchronous operations\n(defeffect state) ;; Mutable state (set!, dict-set!, append!)\n\n;; Functions declare their effects in brackets\n(define fetch-user : (-> number user) [io async]\n (fn (id) (query \"SELECT * FROM users WHERE id = $1\" id)))\n\n(define toggle-class : (-> element string nil) [dom]\n (fn (el cls) (set-attr! el :class cls)))\n\n;; Pure by default — no annotation means no effects\n(define add-prices : (-> (list-of number) number)\n (fn (prices) (reduce + 0 prices)))" "lisp")))
(~docs/subsection :title "Effect declarations"
(~docs/code :code (highlight ";; Declare named effects\n(defeffect io) ;; Database, HTTP, file system\n(defeffect dom) ;; Browser DOM manipulation\n(defeffect async) ;; Asynchronous operations\n(defeffect state) ;; Mutable state (set!, dict-set!, append!)\n\n;; Functions declare their effects in brackets\n(define fetch-user : (-> number user) [io async]\n (fn (id) (query \"SELECT * FROM users WHERE id = $1\" id)))\n\n(define toggle-class : (-> element string nil) [dom]\n (fn (el cls) (set-attr! el :class cls)))\n\n;; Pure by default — no annotation means no effects\n(define add-prices : (-> (list-of number) number)\n (fn (prices) (reduce + 0 prices)))" "lisp")))
(~doc-subsection :title "What it checks"
(~docs/subsection :title "What it checks"
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
(li (strong "Pure functions can't call effectful ones.") " A function with no effect annotation calling " (code "fetch-user") " (which has " (code "[io async]") ") is an error. The IO leaked into pure code.")
(li (strong "Components declare their effect ceiling.") " A " (code "[pure]") " component can only call pure functions. A " (code "[io]") " component can call IO but not DOM. This is the render-mode safety guarantee.")
(li (strong "Render modes enforce effect sets.") " " (code "render-to-html") " (server) allows " (code "[io]") " but not " (code "[dom]") ". " (code "render-to-dom") " (browser) allows " (code "[dom]") " but not " (code "[io]") ". " (code "aser") " (wire format) allows " (code "[io]") " for evaluation but serializes the result.")
(li (strong "Islands are the effect boundary.") " Server effects (" (code "io") ") can't cross into client island code. Client effects (" (code "dom") ") can't leak into server rendering. Currently this is convention — effects make it a proof.")))
(~doc-subsection :title "Three effect sets match three render modes"
(~docs/subsection :title "Three effect sets match three render modes"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -326,17 +326,17 @@
(p "This is exactly the information " (code "deps.sx") " already computes — which components have IO refs. Effects promote it from a runtime classification to a static type-level property. Pure components get an ironclad guarantee: memoize, cache, SSR anywhere, serialize for client — provably safe."))
(~doc-subsection :title "Effect propagation"
(~doc-code :code (highlight ";; Effects propagate through calls:\n(define fetch-prices : (-> (list-of number)) [io async]\n (fn () (query \"SELECT price FROM products\")))\n\n(define render-total : (-> element) [io async] ;; must declare, calls fetch-prices\n (fn ()\n (let ((prices (fetch-prices)))\n (span (str \"$\" (reduce + 0 prices))))))\n\n;; ERROR if you forget:\n(define render-total : (-> element) ;; no effects declared\n (fn ()\n (let ((prices (fetch-prices))) ;; ERROR: calls [io async] from pure context\n (span (str \"$\" (reduce + 0 prices))))))" "lisp"))
(~docs/subsection :title "Effect propagation"
(~docs/code :code (highlight ";; Effects propagate through calls:\n(define fetch-prices : (-> (list-of number)) [io async]\n (fn () (query \"SELECT price FROM products\")))\n\n(define render-total : (-> element) [io async] ;; must declare, calls fetch-prices\n (fn ()\n (let ((prices (fetch-prices)))\n (span (str \"$\" (reduce + 0 prices))))))\n\n;; ERROR if you forget:\n(define render-total : (-> element) ;; no effects declared\n (fn ()\n (let ((prices (fetch-prices))) ;; ERROR: calls [io async] from pure context\n (span (str \"$\" (reduce + 0 prices))))))" "lisp"))
(p "The checker walks call graphs and verifies that every function's declared effects are a superset of its callees' effects. This is transitive — if A calls B calls C, and C has " (code "[io]") ", then A must also declare " (code "[io]") "."))
(~doc-subsection :title "Gradual effects"
(~docs/subsection :title "Gradual effects"
(p "Like gradual types, effects are opt-in. Unannotated functions are assumed to have " (em "all") " effects — they can call anything, and anything can call them. This is safe (no false positives) but provides no guarantees. As you annotate more functions, the checker catches more violations.")
(p "The practical sweet spot: annotate " (code "defcomp") " declarations (they're the public API) and let the checker verify that pure components don't accidentally depend on IO. Internal helpers can stay unannotated until they matter.")
(~doc-code :code (highlight ";; Annotated component — checker enforces purity\n(defcomp ~price-display [pure] (&key (price :as number))\n (span :class \"price\" (str \"$\" (format-decimal price 2))))\n\n;; ERROR: pure component calls IO\n(defcomp ~price-display [pure] (&key (product-id :as number))\n (let ((product (fetch-product product-id))) ;; ERROR: [io] in [pure] context\n (span :class \"price\" (str \"$\" (get product \"price\")))))" "lisp")))
(~docs/code :code (highlight ";; Annotated component — checker enforces purity\n(defcomp ~plans/typed-sx/price-display [pure] (&key (price :as number))\n (span :class \"price\" (str \"$\" (format-decimal price 2))))\n\n;; ERROR: pure component calls IO\n(defcomp ~plans/typed-sx/price-display [pure] (&key (product-id :as number))\n (let ((product (fetch-product product-id))) ;; ERROR: [io] in [pure] context\n (span :class \"price\" (str \"$\" (get product \"price\")))))" "lisp")))
(~doc-subsection :title "Relationship to deps.sx and boundary.sx"
(~docs/subsection :title "Relationship to deps.sx and boundary.sx"
(p "Effects don't replace the existing systems — they formalize them:")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "boundary.sx") " declares which primitives are IO. Effects declare which " (em "functions") " use IO.")
@@ -344,7 +344,7 @@
(li "The boundary is still the source of truth for " (em "what is IO") ". Effects are the enforcement mechanism for " (em "who can use it") "."))
(p "Long term, " (code "deps.sx") "'s IO classification can be derived from effect annotations. In the short term, both coexist — effects are checked, deps are computed, both must agree."))
(~doc-subsection :title "What it does NOT do"
(~docs/subsection :title "What it does NOT do"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (strong "No algebraic effect handlers.") " You can't intercept and resume effects. This would require delimited continuations in every bootstrapper target — massive complexity for marginal UI benefit.")
(li (strong "No effect polymorphism.") " You can't write a function generic over effects (" (code "forall e. (-> a [e] b)") "). This needs higher-kinded effect types — the same complexity as type classes.")
@@ -359,9 +359,9 @@
;; Implementation
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation" :id "implementation"
(~docs/section :title "Implementation" :id "implementation"
(~doc-subsection :title "Phase 1: Type Registry (done)"
(~docs/subsection :title "Phase 1: Type Registry (done)"
(p "Build the type registry from existing declarations.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Parse " (code ":returns") " from " (code "primitives.sx") " and " (code "boundary.sx") " into a type map: " (code "primitive-name → return-type"))
@@ -369,7 +369,7 @@
(li "Compute component signatures from " (code "parse-comp-params") " + any type annotations")
(li "Store in env as metadata alongside existing component/primitive objects")))
(~doc-subsection :title "Phase 2: Type Inference Engine (done)"
(~docs/subsection :title "Phase 2: Type Inference Engine (done)"
(p "Walk AST, infer types bottom-up.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Literals → concrete types")
@@ -380,7 +380,7 @@
(li "Lambda → " (code "(-> param-types return-type)") " from body inference")
(li "Map/filter → propagate element types through the transform")))
(~doc-subsection :title "Phase 3: Type Checker (done)"
(~docs/subsection :title "Phase 3: Type Checker (done)"
(p "Compare inferred types at call sites against declared types.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li "Subtype check: " (code "number") " <: " (code "any") ", " (code "string") " <: " (code "string?") ", " (code "nil") " <: " (code "string?"))
@@ -388,19 +388,19 @@
(li "Warn on possible mismatch: " (code "any") " vs " (code "number") " (might work, might not)")
(li "Component kwarg checking: required params, unknown kwargs, type mismatches")))
(~doc-subsection :title "Phase 4: Annotation Parsing (done)"
(~docs/subsection :title "Phase 4: Annotation Parsing (done)"
(p "Extend " (code "parse-comp-params") " and " (code "sf-defcomp") " to recognize type annotations.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "(name : type)") " in param lists → extract type, store in component metadata")
(li (code ":returns type") " in lambda/fn bodies → store as declared return type")
(li "Backward compatible: unannotated params remain " (code "any"))))
(~doc-subsection :title "Phase 5: Typed Primitives (done)"
(~docs/subsection :title "Phase 5: Typed Primitives (done)"
(p "Add param types to " (code "primitives.sx") " declarations.")
(~doc-code :code (highlight ";; Current\n(define-primitive \"+\"\n :params (&rest args)\n :returns \"number\"\n :doc \"Sum all arguments.\")\n\n;; Extended\n(define-primitive \"+\"\n :params (&rest (args : number))\n :returns \"number\"\n :doc \"Sum all arguments.\")" "lisp"))
(~docs/code :code (highlight ";; Current\n(define-primitive \"+\"\n :params (&rest args)\n :returns \"number\"\n :doc \"Sum all arguments.\")\n\n;; Extended\n(define-primitive \"+\"\n :params (&rest (args : number))\n :returns \"number\"\n :doc \"Sum all arguments.\")" "lisp"))
(p "This is the biggest payoff for effort: ~80 primitives gain param types, enabling the checker to catch every mistyped primitive call across the entire codebase."))
(~doc-subsection :title "Phase 6: User-Defined Types (deftype)"
(~docs/subsection :title "Phase 6: User-Defined Types (deftype)"
(p "Extend the type system with named user-defined types.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "deftype") " special form — parsed by evaluator, stored in type registry")
@@ -412,7 +412,7 @@
(li "Extend " (code "subtype?") " — resolve named types through the registry before comparing")
(li "Test: dict literal against record shape, parameterized type instantiation, field-typed " (code "get"))))
(~doc-subsection :title "Phase 7: Static Effect System"
(~docs/subsection :title "Phase 7: Static Effect System"
(p "Add effect annotations and static checking. No handlers, no runtime cost.")
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (code "defeffect") " declaration form — registers named effects: " (code "io") ", " (code "dom") ", " (code "async") ", " (code "state"))
@@ -429,7 +429,7 @@
;; Spec module
;; -----------------------------------------------------------------------
(~doc-section :title "Spec Module" :id "spec-module"
(~docs/section :title "Spec Module" :id "spec-module"
(p (code "types.sx") " — the type checker, written in SX, bootstrapped to every host.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
@@ -481,7 +481,7 @@
;; Relationships
;; -----------------------------------------------------------------------
(~doc-section :title "Relationships" :id "relationships"
(~docs/section :title "Relationships" :id "relationships"
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
(li (a :href "/sx/(etc.(plan.theorem-prover))" :class "text-violet-700 underline" "Theorem Prover") " — prove.sx verifies primitive properties; types.sx verifies composition. Complementary.")
(li (a :href "/sx/(etc.(plan.content-addressed-components))" :class "text-violet-700 underline" "Content-Addressed Components") " — component manifests gain type signatures. A consumer knows param types before fetching the source.")

38
sx/sx/plans/wasm-bytecode-vm.sx
View File

@@ -2,10 +2,10 @@
;; WASM Bytecode VM — Compile SX to bytecode, run in Rust/WASM
;; ---------------------------------------------------------------------------
(defcomp ~plan-wasm-bytecode-vm-content ()
(~doc-page :title "WASM Bytecode VM"
(defcomp ~plans/wasm-bytecode-vm/plan-wasm-bytecode-vm-content ()
(~docs/page :title "WASM Bytecode VM"
(~doc-section :title "The Idea" :id "idea"
(~docs/section :title "The Idea" :id "idea"
(p "Currently the client-side SX runtime is a tree-walking interpreter bootstrapped to JavaScript. The server sends " (strong "SX source text") " — component definitions, page content — and the browser parses and evaluates it.")
(p "The alternative: compile SX to a " (strong "compact bytecode format") ", ship bytecode to the browser, and execute it in a " (strong "WebAssembly VM written in Rust") ". The VM calls out to JavaScript for DOM operations and I/O via standard WASM↔JS bindings.")
(p "This fits naturally into the SX host architecture. Rust becomes another bootstrapper target. The spec compiles to Rust the same way it compiles to Python and JavaScript. The WASM module is the client-side expression of that Rust target."))
@@ -14,7 +14,7 @@
;; Why
;; -----------------------------------------------------------------------
(~doc-section :title "Why" :id "why"
(~docs/section :title "Why" :id "why"
(ul :class "list-disc list-inside space-y-2"
(li (strong "Wire size") " — bytecode is far more compact than source text. No redundant whitespace, no comments, no repeated symbol names. A component bundle that's 40KB of SX source might be 8KB of bytecode.")
(li (strong "No parse overhead") " — the browser currently parses every SX source string (tokenize → AST → eval). Bytecode skips parsing entirely.")
@@ -26,12 +26,12 @@
;; Architecture
;; -----------------------------------------------------------------------
(~doc-section :title "Architecture" :id "architecture"
(~docs/section :title "Architecture" :id "architecture"
(p "Three new layers, all specced in " (code ".sx") " and bootstrapped:")
(h4 :class "font-semibold mt-4 mb-2" "1. Bytecode format — bytecode.sx")
(p "A spec for the bytecode instruction set. Stack-based VM (simpler than register-based, natural fit for s-expressions). Instructions:")
(~doc-code :code (highlight ";; Core instructions\nPUSH_CONST idx ;; push constant from pool\nPUSH_NIL ;; push nil\nPUSH_TRUE / PUSH_FALSE\nLOOKUP idx ;; look up symbol by index\nSET idx ;; define/set symbol\nCALL n ;; call top-of-stack with n args\nTAIL_CALL n ;; tail call (TCO)\nRETURN\nJUMP offset ;; unconditional jump\nJUMP_IF_FALSE offset ;; conditional jump\nMAKE_LAMBDA idx n_params ;; create closure\nMAKE_LIST n ;; collect n stack values into list\nMAKE_DICT n ;; collect 2n stack values into dict\nPOP ;; discard top\nDUP ;; duplicate top" "lisp"))
(~docs/code :code (highlight ";; Core instructions\nPUSH_CONST idx ;; push constant from pool\nPUSH_NIL ;; push nil\nPUSH_TRUE / PUSH_FALSE\nLOOKUP idx ;; look up symbol by index\nSET idx ;; define/set symbol\nCALL n ;; call top-of-stack with n args\nTAIL_CALL n ;; tail call (TCO)\nRETURN\nJUMP offset ;; unconditional jump\nJUMP_IF_FALSE offset ;; conditional jump\nMAKE_LAMBDA idx n_params ;; create closure\nMAKE_LIST n ;; collect n stack values into list\nMAKE_DICT n ;; collect 2n stack values into dict\nPOP ;; discard top\nDUP ;; duplicate top" "lisp"))
(p "Bytecode modules contain: a " (strong "constant pool") " (strings, numbers, symbols), a " (strong "code section") " (instruction bytes), and a " (strong "metadata section") " (source maps, component/island declarations for the host to register).")
(h4 :class "font-semibold mt-4 mb-2" "2. Compiler — compile.sx")
@@ -54,23 +54,23 @@
;; DOM interop
;; -----------------------------------------------------------------------
(~doc-section :title "DOM Interop" :id "dom-interop"
(~docs/section :title "DOM Interop" :id "dom-interop"
(p "The main engineering challenge. Every DOM operation crosses the WASM↔JS boundary. Two strategies:")
(h4 :class "font-semibold mt-4 mb-2" "Strategy A: Direct calls")
(p "Each DOM operation (" (code "createElement") ", " (code "setAttribute") ", " (code "appendChild") ") is a separate WASM→JS call. Simple, works, but ~50ns overhead per call. For a page with 1,000 DOM operations, that's ~50μs — negligible.")
(~doc-code :code (highlight "// JS side — imported by WASM\nfunction domCreateElement(tag_ptr, tag_len) {\n const tag = readString(tag_ptr, tag_len);\n return storeHandle(document.createElement(tag));\n}\n\n// Rust side\nextern \"C\" { fn dom_create_element(tag: *const u8, len: u32) -> u32; }" "javascript"))
(~docs/code :code (highlight "// JS side — imported by WASM\nfunction domCreateElement(tag_ptr, tag_len) {\n const tag = readString(tag_ptr, tag_len);\n return storeHandle(document.createElement(tag));\n}\n\n// Rust side\nextern \"C\" { fn dom_create_element(tag: *const u8, len: u32) -> u32; }" "javascript"))
(h4 :class "font-semibold mt-4 mb-2" "Strategy B: Command buffer")
(p "Batch DOM operations in WASM memory as a command buffer. Flush to JS in one call. JS walks the buffer and applies all operations. Fewer boundary crossings, but more complex.")
(~doc-code :code (highlight ";; Command buffer format (in shared WASM memory)\n;; [CREATE_ELEMENT, tag_idx, handle_out]\n;; [SET_ATTR, handle, key_idx, val_idx]\n;; [APPEND_CHILD, parent_handle, child_handle]\n;; [SET_TEXT, handle, text_idx]\n;; Then: (flush-dom-commands)" "lisp"))
(~docs/code :code (highlight ";; Command buffer format (in shared WASM memory)\n;; [CREATE_ELEMENT, tag_idx, handle_out]\n;; [SET_ATTR, handle, key_idx, val_idx]\n;; [APPEND_CHILD, parent_handle, child_handle]\n;; [SET_TEXT, handle, text_idx]\n;; Then: (flush-dom-commands)" "lisp"))
(p "Strategy A is simpler and sufficient for SX workloads. Strategy B is an optimisation if profiling shows the boundary crossing matters. " (strong "Start with A, measure, switch to B only if needed.")))
;; -----------------------------------------------------------------------
;; String handling
;; -----------------------------------------------------------------------
(~doc-section :title "String Handling" :id "strings"
(~docs/section :title "String Handling" :id "strings"
(p "WASM has no native string type. Strings must cross the boundary via shared " (code "ArrayBuffer") " memory. Options:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (strong "Copy on crossing") " — encode to UTF-8 in WASM linear memory, JS reads via " (code "TextDecoder") ". Simple, safe, ~1μs per string.")
@@ -82,7 +82,7 @@
;; Memory management
;; -----------------------------------------------------------------------
(~doc-section :title "Memory & Closures" :id "memory"
(~docs/section :title "Memory & Closures" :id "memory"
(p "SX values that the VM must manage:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (strong "Closures") " — lambda captures free variables. Rust: " (code "Rc<Closure>") " with captured env as " (code "Vec<Value>") ".")
@@ -95,7 +95,7 @@
;; What gets compiled
;; -----------------------------------------------------------------------
(~doc-section :title "What Gets Compiled" :id "compilation"
(~docs/section :title "What Gets Compiled" :id "compilation"
(p "Not everything needs bytecode. The compilation boundary follows the existing server/client split:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
@@ -129,7 +129,7 @@
;; Bytecode vs direct WASM compilation
;; -----------------------------------------------------------------------
(~doc-section :title "Bytecode VM vs Direct WASM Compilation" :id "vm-vs-direct"
(~docs/section :title "Bytecode VM vs Direct WASM Compilation" :id "vm-vs-direct"
(p "Two paths to WASM. The choice matters:")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
@@ -168,9 +168,9 @@
;; Dual target — same spec, runtime choice
;; -----------------------------------------------------------------------
(~doc-section :title "Dual Target: JS or WASM from the Same Spec" :id "dual-target"
(~docs/section :title "Dual Target: JS or WASM from the Same Spec" :id "dual-target"
(p "The key insight: this is " (strong "not a replacement") " for the JS evaluator. It's " (strong "another compilation target from the same spec") ". The existing bootstrapper pipeline already proves this pattern:")
(~doc-code :code (highlight "eval.sx ──→ bootstrap_js.py ──→ sx-ref.js (browser, JS eval)\n ──→ bootstrap_py.py ──→ sx_ref.py (server, Python eval)\n ──→ bootstrap_rs.py ──→ sx-vm.wasm (browser, WASM eval) ← new" "text"))
(~docs/code :code (highlight "eval.sx ──→ bootstrap_js.py ──→ sx-ref.js (browser, JS eval)\n ──→ bootstrap_py.py ──→ sx_ref.py (server, Python eval)\n ──→ bootstrap_rs.py ──→ sx-vm.wasm (browser, WASM eval) ← new" "text"))
(p "All three outputs have identical semantics because they're compiled from the same source. The choice of which to use is a " (strong "deployment decision") ", not an architectural one:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li (strong "JS-only") " — current default. Works everywhere. Zero WASM dependency. Ship sx-browser.js + SX source text.")
@@ -184,7 +184,7 @@
;; Implementation phases
;; -----------------------------------------------------------------------
(~doc-section :title "Implementation Phases" :id "phases"
(~docs/section :title "Implementation Phases" :id "phases"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
@@ -225,7 +225,7 @@
;; Interaction with existing plans
;; -----------------------------------------------------------------------
(~doc-section :title "Interaction with Other Plans" :id "interactions"
(~docs/section :title "Interaction with Other Plans" :id "interactions"
(ul :class "list-disc list-inside space-y-2"
(li (strong "Async Eval Convergence") " — must complete first. The spec must be the single evaluator before we add another target. Otherwise we'd be bootstrapping a fork.")
(li (strong "Runtime Slicing") " — the WASM module can be tiered just like the JS runtime. L0 hypermedia needs no VM at all (pure HTML). L1 DOM ops needs a minimal VM. L2 islands needs signals. The WASM module should be tree-shakeable.")
@@ -237,7 +237,7 @@
;; Principles
;; -----------------------------------------------------------------------
(~doc-section :title "Principles" :id "principles"
(~docs/section :title "Principles" :id "principles"
(ul :class "list-disc list-inside space-y-2"
(li (strong "The spec remains the single source of truth.") " The bytecode format, compiler, and VM semantics are all specced in .sx. The Rust VM is just another host, like Python and JavaScript.")
(li (strong "Bytecode is an optimisation, not a requirement.") " SX source text remains a valid wire format. The system degrades gracefully — if WASM isn't available, fall back to the JS evaluator. Progressive enhancement.")
@@ -249,7 +249,7 @@
;; Outcome
;; -----------------------------------------------------------------------
(~doc-section :title "Outcome" :id "outcome"
(~docs/section :title "Outcome" :id "outcome"
(p "After completion:")
(ul :class "list-disc list-inside space-y-2 mt-2"
(li "SX compiles to four targets: JavaScript, Python, Rust (native), Rust (WASM)")