Files
rose-ash/sx/sx/plans/generative-sx.sx
giles 1341c144da URL restructure, 404 page, trailing slash normalization, layout fixes
- Rename /reactive-islands/ → /reactive/, /reference/ → /hypermedia/reference/,
  /examples/ → /hypermedia/examples/ across all .sx and .py files
- Add 404 error page (not-found.sx) working on both server refresh and
  client-side SX navigation via orchestration.sx error response handling
- Add trailing slash redirect (GET only, excludes /api/, /static/, /internal/)
- Remove blue sky-500 header bar from SX docs layout (conditional on header-rows)
- Fix 405 on API endpoints from trailing slash redirect hitting POST/PUT/DELETE
- Fix client-side 404: orchestration.sx now swaps error response content
  instead of silently dropping it
- Add new plan files and home page component

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-10 21:30:18 +00:00

338 lines
43 KiB
Plaintext
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
;; ---------------------------------------------------------------------------
;; Generative SX — programs that write themselves as they run
;; ---------------------------------------------------------------------------
(defcomp ~plan-generative-sx-content ()
(~doc-page :title "Generative SX"
(p :class "text-stone-500 text-sm italic mb-8"
"In the browser, SX is a program that modifies itself in response to external stimuli. Outside the browser, it becomes a program that writes itself as it runs.")
;; =====================================================================
;; I. The observation
;; =====================================================================
(~doc-section :title "The observation" :id "observation"
(p "The marshes work made something visible. A server response arrives carrying " (code "(reset! (use-store \"price\") 14.99)") " inside a " (code "data-init") " script. The SX evaluator parses this string, evaluates it in its own environment, and mutates its own signal graph. The program accepted new source at runtime and changed itself.")
(p "This isn't metaprogramming. Macros expand at compile time — they transform source before evaluation. This is different: the program is " (em "already running") " when it receives new code, evaluates it, and continues with an extended state. The DOM is just the boundary. The signal graph is just the state. The mechanism is general:")
(ol :class "list-decimal list-inside space-y-2 text-stone-600"
(li "A running SX evaluator with an environment")
(li "New SX source arrives (from any external source)")
(li "The evaluator parses it and evaluates it " (em "in its own environment"))
(li "The environment grows — new definitions, new state, new capabilities")
(li "Evaluation continues with the extended environment"))
(p "In the browser, step 2 is an HTTP response. Step 5 is the next render cycle. But nothing about this mechanism requires a browser, a DOM, or HTTP. The evaluator doesn't know where the source came from. It just evaluates."))
;; =====================================================================
;; II. What already exists
;; =====================================================================
(~doc-section :title "What already exists" :id "exists"
(p "The pieces are already built. They just haven't been connected into the generative pattern.")
(~doc-subsection :title "Homoiconicity"
(p "SX code is SX data. " (code "parse") " takes a string and returns a list. " (code "aser") " takes a list and returns a string. These round-trip perfectly. The program can read its own source as naturally as it reads a config file, because they're the same format.")
(~doc-code :code (highlight ";; Code is data\n(define source \"(+ 1 2)\")\n(define ast (parse source)) ;; → (list '+ 1 2)\n(define result (eval-expr ast env)) ;; → 3\n\n;; Data is code\n(define spec '(define greet (fn (name) (str \"Hello, \" name \"!\"))))\n(eval-expr spec env)\n(greet \"world\") ;; → \"Hello, world!\"" "lisp")))
(~doc-subsection :title "Runtime eval"
(p (code "eval-expr") " is available at runtime, not just boot. " (code "data-init") " scripts already use it. Any SX string can become running code at any point in the program's execution. This is not " (code "eval()") " bolted onto a language that doesn't want it — it's the " (em "primary mechanism") " of the language."))
(~doc-subsection :title "The environment model"
(p (code "env-extend") " creates a child scope. " (code "env-set!") " adds to the current scope. " (code "define") " creates new bindings. New definitions don't replace old ones — they extend the environment. The program grows monotonically. Every previous state is still reachable through scope chains.")
(p "This is the critical property. A generative program doesn't destroy what it was — it " (em "becomes more") ". Each generation includes everything before it plus what it just wrote."))
(~doc-subsection :title "The bootstrapper"
(p "The bootstrapper reads " (code "eval.sx") " — the evaluator's definition of itself — and emits JavaScript or Python that " (em "is") " that evaluator. The spec writes itself into a host language. This is already a generative program, frozen at build time: read source → transform → emit target. Generative SX unfreezes this. The transformation happens " (em "while the program runs") ", not before."))
(~doc-subsection :title "Content-addressed identity"
(p "From the Art DAG: all data identified by SHA3-256 hashes. If a program fragment is identified by its hash, then \"writing yourself\" means producing new hashes. The history is immutable. You can always go back. A generative program isn't a mutating blob — it's a DAG of versioned states.")))
;; =====================================================================
;; III. The generative pattern
;; =====================================================================
(~doc-section :title "The generative pattern" :id "pattern"
(p "A generative SX program starts with a seed and grows by evaluating its own output.")
(~doc-code :code (highlight ";; The core loop\n(define run\n (fn (env source)\n (let ((ast (parse source))\n (result (eval-expr ast env)))\n ;; result might be:\n ;; a value → done, return it\n ;; a string → new SX source, evaluate it (grow)\n ;; a list of defs → new definitions to add to env\n ;; a dict → {source: \"...\", env-patch: {...}} (grow + configure)\n (cond\n (string? result)\n (run env result) ;; evaluate the output\n (and (dict? result) (has-key? result \"source\"))\n (let ((patched (env-merge env (get result \"env-patch\"))))\n (run patched (get result \"source\")))\n :else\n result))))" "lisp"))
(p "The program evaluates source. If the result is more source, it evaluates that too. Each iteration can extend the environment — add new functions, new macros, new primitives. The environment grows. The program becomes capable of things it couldn't do at the start.")
(div :class "rounded border border-violet-200 bg-violet-50 p-4 my-4"
(p :class "text-violet-900 font-medium" "This is not eval-in-a-loop")
(p :class "text-violet-800 text-sm" "A REPL evaluates user input in a persistent environment. That's interactive, not generative. The generative pattern is different: the program itself decides what to evaluate next. No user in the loop. The output of one evaluation becomes the input to the next. The program writes itself."))
(~doc-subsection :title "Three modes"
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
(th :class "px-3 py-2 font-medium text-stone-600" "Mode")
(th :class "px-3 py-2 font-medium text-stone-600" "Input")
(th :class "px-3 py-2 font-medium text-stone-600" "Output")
(th :class "px-3 py-2 font-medium text-stone-600" "Growth")))
(tbody
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-medium text-stone-700" "Analytic")
(td :class "px-3 py-2 text-stone-600" "SX program + data")
(td :class "px-3 py-2 text-stone-600" "Analysis results")
(td :class "px-3 py-2 text-stone-600" "None — pure function"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-medium text-stone-700" "Synthetic")
(td :class "px-3 py-2 text-stone-600" "SX program")
(td :class "px-3 py-2 text-stone-600" "New SX source")
(td :class "px-3 py-2 text-stone-600" "Code generation — new defs emitted"))
(tr
(td :class "px-3 py-2 font-medium text-stone-700" "Generative")
(td :class "px-3 py-2 text-stone-600" "SX seed")
(td :class "px-3 py-2 text-stone-600" "Running program")
(td :class "px-3 py-2 text-stone-600" "Self-extending — output evaluated as input")))))))
;; =====================================================================
;; IV. Concrete manifestations
;; =====================================================================
(~doc-section :title "Concrete manifestations" :id "manifestations"
(~doc-subsection :title "1. The spec that compiles itself"
(p "Currently: " (code "bootstrap_js.py") " (Python) reads " (code "eval.sx") " (SX) and emits " (code "sx-browser.js") " (JavaScript). Three languages. Two of them are hosts, one is the spec.")
(p "Generative version: " (code "eval.sx") " evaluates itself with a code-generation adapter. The evaluator walks its own AST and emits the target language. No Python bootstrapper. No JavaScript template. The spec " (em "is") " the compiler.")
(~doc-code :code (highlight ";; bootstrap.sx — the spec compiles itself\n;;\n;; Load the codegen adapter for the target\n(define emit (load-adapter target)) ;; target = \"js\" | \"py\" | \"go\" | ...\n;;\n;; Read the spec files\n(define spec-source (read-file \"eval.sx\"))\n(define spec-ast (parse spec-source))\n;;\n;; Walk the AST and emit target code\n;; The walker IS the evaluator — eval.sx evaluating eval.sx\n;; with emit instead of execute\n(define target-code\n (eval-expr spec-ast\n (env-extend codegen-env\n ;; Override define, fn, if, etc. to emit instead of execute\n (codegen-special-forms emit))))\n;;\n(write-file (str \"sx-ref.\" (target-extension target)) target-code)" "lisp"))
(p "This is the bootstrapper rewritten as a generative program. The spec reads itself, walks itself, and writes the output language. Adding a new target means writing a new " (code "load-adapter") " — a set of emitters for the SX special forms. The walker doesn't change. The spec doesn't change. Only the output format changes.")
(p "The current bootstrappers (" (code "bootstrap_js.py") ", " (code "bootstrap_py.py") ") would become the first two adapters. Future targets (Go, Rust, WASM) are additional adapters, written in SX and bootstrapped like everything else."))
(~doc-subsection :title "2. The program that discovers its dependencies"
(p (code "deps.sx") " analyzes component dependency graphs. It walks component ASTs, finds " (code "~name") " references, computes transitive closures. This is the analytic mode — SX analyzing SX.")
(p "The generative version: a program that discovers it needs a component, searches for its definition (local files, IPFS, a registry), loads it, evaluates it, and continues. The program grows its own capability set at runtime.")
(~doc-code :code (highlight ";; A program that discovers and loads what it needs\n(define render-page\n (fn (page-name)\n (let ((page-def (lookup-component page-name)))\n (when (nil? page-def)\n ;; Component not found locally — fetch from registry\n (let ((source (fetch-component-source page-name)))\n ;; Evaluate the definition — it joins the environment\n (eval-expr (parse source) env)\n ;; Now it exists\n (set! page-def (lookup-component page-name))))\n ;; Render with all dependencies resolved\n (render-to-html (list page-def)))))" "lisp"))
(p "This already happens in the browser. " (code "sx_response") " prepends missing component definitions as a " (code "data-components") " script block. The client evaluates them and they join the environment. The generative version makes this explicit: the program tells you what it needs, you give it source, it evaluates it, it grows."))
(~doc-subsection :title "3. The test suite that writes tests"
(p "Given a function's signature and a set of properties (" (code "prove.sx") " already has the property language), generate test cases that verify the properties. The program reads its own function definitions, generates SX expressions that test them, and evaluates those expressions.")
(~doc-code :code (highlight ";; Given: a function and properties about it\n(define-property string-reverse-involutory\n :forall ((s string?))\n :holds (= (reverse (reverse s)) s))\n\n;; Generate: test cases from the property\n;; The program reads the property, generates test source, evals it\n(define tests (generate-tests string-reverse-involutory))\n;; tests = list of (assert (= (reverse (reverse \"hello\")) \"hello\"))\n;; (assert (= (reverse (reverse \"\")) \"\"))\n;; (assert (= (reverse (reverse \"a\")) \"a\"))\n;; ... (random strings, edge cases)\n(for-each (fn (t) (eval-expr t env)) tests)" "lisp"))
(p "The program analyzed itself (read the property), generated new SX (the test cases), and evaluated it (ran the tests). Three modes — analytic, synthetic, generative — in sequence."))
(~doc-subsection :title "4. The server that extends its own API"
(p "An SX server receives a request it doesn't know how to handle. Instead of returning 404, it examines the request, generates a handler, evaluates it, and handles the request.")
(~doc-code :code (highlight ";; A route handler that generates new route handlers\n(define handle-unknown-route\n (fn (path params)\n ;; Analyze what was requested\n (let ((segments (split path \"/\"))\n (resource (nth segments 1))\n (action (nth segments 2)))\n ;; Check if a schema exists for this resource\n (let ((schema (lookup-schema resource)))\n (when schema\n ;; Generate a CRUD handler from the schema\n (let ((handler-source (generate-crud-handler resource action schema)))\n ;; Evaluate it — the handler now exists\n (eval-expr (parse handler-source) env)\n ;; Route future requests to the generated handler\n (register-route path (env-get env (str resource \"-\" action)))\n ;; Handle this request with the new handler\n ((env-get env (str resource \"-\" action)) params)))))))" "lisp"))
(p "This is not code generation in the Rails scaffolding sense — those generate files you then edit. This generates running code at runtime. The handler didn't exist. Now it does. The server grew."))
(~doc-subsection :title "5. The macro system that learns idioms"
(p "A generative macro system that detects repeated patterns in code and synthesizes macros to capture them. The program watches itself being written and abstracts its own patterns.")
(~doc-code :code (highlight ";; The program notices this pattern appearing repeatedly:\n;; (div :class \"card\" (h2 title) (p body) children...)\n;;\n;; It generates:\n(defmacro ~card (title body &rest children)\n (div :class \"card\"\n (h2 ,title)\n (p ,body)\n ,@children))\n;;\n;; And rewrites its own source to use the new macro.\n;; This is an SX program that:\n;; 1. Analyzed its own AST (found repeated subtrees)\n;; 2. Synthesized a macro (extracted the pattern)\n;; 3. Evaluated the macro definition (extended env)\n;; 4. Rewrote its own source (used the macro)\n;; Four generative steps." "lisp"))
(p "The connection to the Art DAG: each version of the source is content-addressed. The original (before macros) and the refactored (after macros) are both immutable nodes. The generative step is an edge in the DAG. You can always inspect what the program was before it rewrote itself.")))
;; =====================================================================
;; V. The seed
;; =====================================================================
(~doc-section :title "The seed" :id "seed"
(p "A generative SX program starts with a seed. The seed must contain enough to bootstrap the generative loop: a parser, an evaluator, and the " (code "run") " function. Everything else is grown.")
(~doc-code :code (highlight ";; seed.sx — the minimal generative program\n;;\n;; This file contains:\n;; - The SX parser (parse)\n;; - The SX evaluator (eval-expr)\n;; - The generative loop (run)\n;; - A source acquisition function (next-source)\n;;\n;; Everything else — primitives, rendering, networking, persistence —\n;; is loaded by the program as it discovers it needs them.\n\n(define run\n (fn (env)\n (let ((source (next-source env)))\n (when source\n (let ((result (eval-expr (parse source) env)))\n (run env))))))\n\n;; Start with a bare environment\n(run (env-extend (dict\n \"parse\" parse\n \"eval-expr\" eval-expr\n \"next-source\" initial-source-fn)))" "lisp"))
(p "The seed is a quine that doesn't just reproduce itself — it " (em "extends") " itself. Each call to " (code "next-source") " returns new SX that the seed evaluates in its own environment. The environment grows. The seed's capabilities grow. But the seed itself never changes — it's the fixed point of the generative process.")
(div :class "rounded border border-stone-200 bg-stone-50 p-4 my-4"
(p :class "text-stone-700 font-medium mb-2" "The minimal seed is the spec")
(p :class "text-stone-600 text-sm" (code "eval.sx") " + " (code "parser.sx") " + " (code "primitives.sx") " = a complete SX evaluator defined in SX. This is already the seed. The bootstrappers compile it to JavaScript and Python. A generative runtime compiles it to " (em "itself") " — the seed evaluates the seed and obtains a running evaluator that can evaluate anything, including more of itself.")))
;; =====================================================================
;; VI. Growth constraints
;; =====================================================================
(~doc-section :title "Growth constraints" :id "constraints"
(p "Unconstrained self-modification is dangerous. A program that can rewrite any part of itself can rewrite its own safety checks. Generative SX needs growth constraints — rules about what the program can and cannot do to itself.")
(~doc-subsection :title "The boundary"
(p "The boundary system (" (code "boundary.sx") ") already enforces this. Pure primitives can't do IO. IO primitives can't escape their declared capabilities. Components are classified as pure or IO-dependent. The boundary is checked at registration time — " (code "SX_BOUNDARY_STRICT=1") " means violations crash at startup.")
(p "For generative programs, the boundary extends: generated code is subject to the same constraints as hand-written code. A generative program can't synthesize an IO primitive — it can only compose existing ones. It can't bypass the boundary by generating code that accesses raw platform APIs. The sandbox applies to generated code exactly as it applies to original code.")
(p "This is the key safety property: " (strong "generative SX is sandboxed generative SX") ". The generated code runs in the same evaluator with the same restrictions. No escape hatches."))
(~doc-subsection :title "Content addressing as audit trail"
(p "Every piece of generated code is content-addressed. The SHA3-256 hash of the generated source is its identity. You can trace any piece of running code back to the generation step that produced it, the input that triggered that step, and the state of the environment at that point.")
(p "This makes generative programs auditable. \"Where did this function come from?\" has a definite answer: it was generated by " (em "this") " code, from " (em "this") " input, at " (em "this") " point in the generative sequence. The DAG of content hashes is the program's autobiography."))
(~doc-subsection :title "Monotonic growth"
(p "The environment model is append-only. " (code "define") " creates new bindings; it doesn't destroy old ones (inner scopes shadow, but the outer binding persists). " (code "env-extend") " creates a child — the parent is immutable.")
(p "A generative program can extend its environment but cannot shrink it. It can add new functions but cannot delete existing ones. It can shadow a function with a new definition but cannot destroy the original. This means the program's history is preserved in its scope chain — you can always inspect what it was before any given generation step.")
(p "Destructive operations (" (code "set!") ") are confined to mutable cells explicitly created for that purpose. The generative loop itself operates on immutable environments extended with each step.")))
;; =====================================================================
;; VII. Host properties
;; =====================================================================
(~doc-section :title "Host properties" :id "host-properties"
(p "A generative SX program runs on a host — JavaScript, Python, Go, bare metal. The host must provide specific properties or the generative loop breaks. These aren't preferences. They're " (em "requirements") ". A host that violates any of them can't run generative SX correctly.")
(~doc-subsection :title "Lossless parse/serialize round-trip"
(p "The host must implement " (code "parse") " and " (code "aser") " such that " (code "(aser (parse source))") " produces semantically equivalent source. Generated code passes through " (code "parse → transform → serialize → parse") " cycles. If the round-trip is lossy — if whitespace, keyword order, or nested structure is corrupted — the generative loop silently degrades. After enough iterations, the program isn't what it thinks it is.")
(p "This is homoiconicity at the implementation level, not just the language level. The host's data structures must faithfully represent the full AST, and the serializer must faithfully reproduce it."))
(~doc-subsection :title "Runtime eval with first-class environments"
(p (code "eval-in") " requires evaluating arbitrary expressions in arbitrary environments at runtime. The host must support:")
(ul :class "list-disc pl-5 space-y-1 text-stone-600"
(li "Creating new environments (" (code "env-extend") ")")
(li "Adding bindings to existing environments (" (code "env-set!") ")")
(li "Inspecting environment contents (" (code "env-snapshot") ")")
(li "Passing environments as values — storing them in variables, returning them from functions"))
(p "Environments aren't implementation detail in a generative program. They're the " (em "state") ". The running environment at generation step N is the complete description of what the program knows. The host must treat environments as first-class values, not hidden interpreter internals."))
(~doc-subsection :title "Monotonic environment growth"
(p "A generative program that can " (code "undefine") " things becomes unpredictable. If generation step N+1 removes a function that step N defined, step N+2 might reference the missing function and fail — or worse, silently bind to a different function in an outer scope.")
(p "The host must enforce that environments grow monotonically. New bindings append. Existing bindings in a given scope are immutable once set (or explicitly versioned). " (code "env-extend") " creates children; it never mutates the parent. This makes the generative loop convergent — each step strictly increases the program's capabilities, never decreases them."))
(~doc-subsection :title "Content-addressed storage"
(p "Every generated fragment gets a SHA3-256 identity. The host needs native or near-native hashing and a content-addressed store — an in-memory dict at minimum, IPFS at scale. This provides the audit trail: you can always answer \"where did this code come from?\" by walking the hash chain back to the generation step that produced it.")
(p "Without content addressing, generative programs are opaque. You can't diff two versions of a generated function. You can't roll back to a previous generation. You can't verify that two nodes in a seed network generated the same code from the same input. Content addressing makes the generative process " (em "inspectable") "."))
(~doc-subsection :title "Boundary enforcement on generated code"
(p "Generated code must pass through the same boundary validation as hand-written code. If " (code "write-file") " is a Tier 2 IO primitive, a generated expression can't call it unless the evaluation context permits Tier 2.")
(p "The host must enforce this " (em "at eval time") ", not just at definition time — because generated code didn't exist at definition time. Every call to " (code "eval-in") " must check the boundary. Every primitive invoked by generated code must verify its tier. There is no \"trusted generated code\" — all code is untrusted until the boundary clears it."))
(~doc-subsection :title "Correct quotation and splicing"
(p "Quasiquote (" (code "`") "), unquote (" (code ",") "), and unquote-splicing (" (code ",@") ") must work correctly for programmatic code construction. The host needs these as first-class operations, not string concatenation.")
(p "A generative program builds code by template:")
(~doc-code :code (highlight ";; The generative program builds new definitions by template\n(define gen-handler\n (fn (name params body)\n `(define ,name\n (fn ,params\n ,@body))))\n\n;; gen-handler produces an AST, not a string\n;; The AST can be inspected, transformed, hashed, then evaluated\n(eval-in (gen-handler 'greet '(name) '((str \"Hello, \" name))) env)" "lisp"))
(p "String concatenation would work — " (code "(str \"(define \" name \" ...)\")") " — but it's fragile, unstructured, and can't be inspected before evaluation. Quasiquote produces an AST. The generative program works with " (em "structure") ", not text."))
(~doc-subsection :title "Tail-call optimization"
(p "The generative loop is inherently recursive: eval produces source, which is eval'd, which may produce more source. Without TCO, the loop blows the stack after enough iterations. The trampoline/thunk mechanism in the spec handles this, but the host must implement it efficiently.")
(p "This is not optional. A generative program that can only recurse a few thousand times before crashing is not a generative program — it's a demo. The self-compiling spec (Phase 1) alone requires walking every node of " (code "eval.sx") ", which is thousands of recursive calls."))
(~doc-subsection :title "Deterministic evaluation order"
(p "If two hosts evaluate the same generative program and get different results because of evaluation order, the content hashes diverge. The programs are no longer equivalent. They can't federate (Phase 5), can't verify each other's output, can't share generated code.")
(p "The host must guarantee: dict iteration order is deterministic (insertion order). Argument evaluation is left-to-right. Effect sequencing follows definition order. No observable nondeterminism in pure evaluation. This is what makes generative programs " (em "reproducible") " — same seed, same input, same output, regardless of host."))
(~doc-subsection :title "Serializable state"
(p "For Phase 4 (self-extending server) and Phase 5 (seed network), a generative program needs to pause, serialize its state, and resume elsewhere. The host needs the ability to serialize an environment + pending expression as data.")
(p "This doesn't require first-class continuations (though those work). It requires that everything in the environment is serializable: functions serialize as their source AST, signals as their current value, environments as nested dicts. The " (code "env-snapshot") " primitive provides this. The host must ensure nothing in the environment is opaque — no host-language closures that can't be serialized, no hidden mutable state that isn't captured by the snapshot."))
(~doc-subsection :title "IO isolation"
(p "The generative primitives (" (code "read-file") ", " (code "write-file") ", " (code "list-files") ") are the " (em "only") " way generated code touches the outside world. The host must be able to intercept, log, and deny all IO. There is no escape hatch through FFI or native calls.")
(p "This is what makes generative programs auditable. If the host allows generated code to call raw " (code "fs.writeFileSync") " or " (code "os.system") ", the boundary is meaningless. The host must virtualize all IO through the declared primitives. Generated code that tries to escape the sandbox hits the boundary, not the OS."))
(div :class "rounded border border-violet-200 bg-violet-50 p-4 my-4"
(p :class "text-violet-900 font-medium" "The acid test")
(p :class "text-violet-800 text-sm" "SX already has properties 17 in the spec. They fall out of the language design. Properties 810 (deterministic evaluation, serializable state, IO isolation) are constraints on the " (em "host implementation") ", not the language. A host that violates 17 can't run the spec correctly. A host that violates 810 can run it but can't be trusted in a generative context.")
(p :class "text-violet-800 text-sm mt-2" "Phase 1 — the self-compiling spec — tests properties 17. If a host can compile the spec from the spec, it necessarily has them. The remaining three are operational guarantees verified by the seed network (Phase 5), where multiple hosts must agree on the same generative output.")))
;; =====================================================================
;; VIII. Environment migration
;; =====================================================================
(~doc-section :title "Environment migration" :id "env-migration"
(p "SX endpoints tunnel into different execution environments with different primitive sets. A browser has " (code "render-to-dom") " but no " (code "gpu-exec") ". A render node has " (code "gpu-exec") " but no " (code "fetch-fragment") ". An ingest server has " (code "open-feed") " but neither. The boundary isn't just a restriction — it's a " (em "capability declaration") ". It tells you what an environment " (em "can do") ".")
(~doc-subsection :title "Boundary as capability declaration"
(p "Every environment declares its boundary: the set of primitives it provides. SX source is portable across any environment that satisfies its primitive requirements. If a program only uses pure primitives (Tier 0), it runs anywhere. If it calls " (code "gpu-exec") ", it needs an environment that provides " (code "gpu-exec") ". The boundary is a type signature on the environment itself — not \"what can this code do\" but \"what must the host provide.\"")
(p "This inverts the usual framing. The boundary doesn't " (em "forbid") " — it " (em "requires") ". A generated program that calls " (code "encode-stream") " is declaring a hardware dependency. The boundary system doesn't block the call — it routes the program to a host that can satisfy it."))
(~doc-subsection :title "with-boundary as migration point"
(p "Execution migrates to where the primitives are. When the evaluator hits a " (code "with-boundary") " block requiring primitives the current host doesn't have, it serializes state (" (code "env-snapshot") "), ships the pending expression plus environment to a host that has them, and execution continues there. The block is the unit of migration, not individual primitive calls.")
(~doc-code :code (highlight "(with-boundary (media-processing encoding)\n (let ((frames (gpu-exec recipe cached-layers)))\n (encode-stream frames :codec \"h264\"\n :on-input-needed (fn (slot)\n (with-boundary (live-ingest)\n (open-feed :protocol \"webrtc\" :slot slot))))))" "lisp"))
(p "This program starts wherever it starts. When it hits " (code "(with-boundary (media-processing encoding) ...)") ", the evaluator checks: does the current host provide " (code "gpu-exec") " and " (code "encode-stream") "? If yes, evaluate in place. If no, snapshot the environment, serialize the pending expression, and dispatch to a host that does. Inside the encoding block, " (code ":on-input-needed") " triggers a nested migration — the " (code "(with-boundary (live-ingest) ...)") " block dispatches to an ingest server that provides " (code "open-feed") ".")
(p "The program doesn't know where it runs. It declares what it needs. The runtime figures out " (em "where") "."))
(~doc-subsection :title "Declaration, not discovery"
(p "Boundary requirements are declared at scope boundaries, not discovered at call time. This is the critical constraint. A generative program that synthesizes a " (code "with-boundary") " block is declaring — at generation time — what the block will need. The declaration is inspectable before execution. You can analyze a generated program's boundary requirements without running it.")
(p "This gives constraint checking on generated code. A generative loop that produces a " (code "with-boundary") " block must produce a valid boundary declaration. If the generated block calls " (code "gpu-exec") " but doesn't declare " (code "media-processing") ", the boundary checker rejects it — at generation time, not at runtime. The program must say what it needs before it needs it."))
(~doc-subsection :title "Nested migration"
(p "Nested " (code "with-boundary") " blocks are nested migrations. The program walks the capability graph, carrying its state, accumulating content-addressed history. Each migration is an edge in the DAG — the source environment, the target environment, the serialized state, the pending expression. All content-addressed. All auditable.")
(p "A three-level nesting — browser to render node to ingest server — is three migrations. The browser evaluates the outer expression, hits a " (code "with-boundary") " requiring GPU, migrates to the render node. The render node evaluates until it hits a " (code "with-boundary") " requiring live ingest, migrates to the ingest server. Each migration carries the accumulated environment. Each return ships results back up the chain.")
(p "The nesting depth is bounded by the capability graph. If there are four distinct environment types, the maximum nesting is four. In practice, most programs need one or two migrations. The deep nesting is there for generative programs that discover capabilities as they run."))
(~doc-subsection :title "Environment chaining"
(p "Split execution — cached layers on one host, GPU rendering on another, encoding on a third — is just environment chaining. The evaluator runs in one environment until it hits a primitive requiring a different one. The primitive " (em "is") " the dispatch.")
(p "This collapses the distinction between \"local function call\" and \"remote service invocation.\" From the SX program's perspective, " (code "gpu-exec") " is a primitive. Whether it runs on the local GPU or a remote render farm is an environment configuration detail, not a language-level concern. The " (code "with-boundary") " block declares the requirement. The runtime satisfies it. The program doesn't care how.")
(p "Environment chaining also explains the Art DAG's three-phase execution pattern (analyze, plan, execute). Each phase runs in a different environment with different primitives. The analyze phase needs " (code "content-hash") " and " (code "list-files") ". The plan phase needs " (code "env-snapshot") " and scheduling primitives. The execute phase needs " (code "gpu-exec") " and storage primitives. Three " (code "with-boundary") " blocks. Three environments. One program.")))
;; =====================================================================
;; IX. Implementation phases
;; =====================================================================
(~doc-section :title "Implementation phases" :id "phases"
(~doc-subsection :title "Phase 0: Generative primitives"
(p "Add the minimal set of primitives needed for a generative loop. These are IO primitives — they cross the boundary.")
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
(table :class "w-full text-left text-sm"
(thead (tr :class "border-b border-stone-200 bg-stone-100"
(th :class "px-3 py-2 font-medium text-stone-600" "Primitive")
(th :class "px-3 py-2 font-medium text-stone-600" "Signature")
(th :class "px-3 py-2 font-medium text-stone-600" "Description")))
(tbody
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "read-file")
(td :class "px-3 py-2 text-stone-600" "(path) → string")
(td :class "px-3 py-2 text-stone-600" "Read file contents as string"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "write-file")
(td :class "px-3 py-2 text-stone-600" "(path content) → nil")
(td :class "px-3 py-2 text-stone-600" "Write string to file"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "list-files")
(td :class "px-3 py-2 text-stone-600" "(path pattern) → list")
(td :class "px-3 py-2 text-stone-600" "Glob-match files in directory"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "eval-in")
(td :class "px-3 py-2 text-stone-600" "(source env) → any")
(td :class "px-3 py-2 text-stone-600" "Parse and evaluate SX source in given environment"))
(tr :class "border-b border-stone-100"
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "env-snapshot")
(td :class "px-3 py-2 text-stone-600" "(env) → dict")
(td :class "px-3 py-2 text-stone-600" "Serialize environment to inspectable dict"))
(tr
(td :class "px-3 py-2 font-mono text-sm text-stone-700" "content-hash")
(td :class "px-3 py-2 text-stone-600" "(source) → string")
(td :class "px-3 py-2 text-stone-600" "SHA3-256 hash of source string")))))
(p "These are the building blocks. The generative loop composes them. The primitives themselves are minimal — no networking, no databases, no UI. Just: read, write, evaluate, inspect, hash."))
(~doc-subsection :title "Phase 1: Self-compiling spec"
(p "Rewrite " (code "bootstrap_js.py") " as " (code "bootstrap.sx") ". The bootstrapper becomes an SX program that reads the spec files and emits target code.")
(ol :class "list-decimal list-inside space-y-2 text-stone-600"
(li "Write " (code "codegen-js.sx") " — JavaScript code generation adapter (emit JS from SX AST)")
(li "Write " (code "codegen-py.sx") " — Python code generation adapter")
(li "Write " (code "bootstrap.sx") " — the generative loop that loads a spec, loads an adapter, and emits")
(li "Verify: " (code "(bootstrap \"js\")") " produces identical output to " (code "bootstrap_js.py"))
(li "Retire the Python bootstrappers"))
(p "This is the first real generative program: SX reading SX and writing JavaScript. The same program, with a different adapter, writes Python. Or Go. Or WASM. The spec doesn't change. Only the adapter changes."))
(~doc-subsection :title "Phase 2: Generative deps"
(p "Rewrite " (code "deps.sx") " as a generative program. Instead of computing a static dependency graph, it runs continuously: watch for new component definitions, update the graph, re-emit optimized bundles.")
(p "This is the deps analyzer turned inside out. Instead of \"analyze all components, output a graph,\" it's \"when a new component appears, update the running graph.\" The dependency analysis is an ongoing computation, not a one-shot pass."))
(~doc-subsection :title "Phase 3: Generative testing"
(p "Connect " (code "prove.sx") " to the generative loop. When a new function is defined, automatically generate property tests, run them, report failures. When a function changes, regenerate and rerun only the affected tests.")
(p "The test suite is not a separate artifact — it's a side effect of the generative process. Every function that enters the environment is tested. The tests are generated from properties, not hand-written. The program verifies itself as it grows."))
(~doc-subsection :title "Phase 4: The self-extending server"
(p "An SX server with a generative core. New routes, handlers, and middleware can be added at runtime by evaluating SX source. The server's API surface is a living environment that grows with use.")
(p "Not a scripting layer bolted onto a framework — the server " (em "is") " a generative SX program. Its routes are SX definitions. Its middleware is SX functions. Adding a new endpoint means evaluating a new " (code "defhandler") " in the running environment."))
(~doc-subsection :title "Phase 5: The seed network"
(p "Multiple generative SX programs exchanging source. Each node runs a seed. When node A discovers a capability it lacks, it requests the source from node B. Node B's generated code is content-addressed — A can verify it, evaluate it, and grow.")
(p "This is SX-Activity applied to generative programs. The wire format is SX. The content is SX. The evaluation is SX. The programs share source, not data. They grow together.")))
;; =====================================================================
;; X. The strange loop
;; =====================================================================
(~doc-section :title "The strange loop" :id "strange-loop"
(p "Hofstadter's strange loop: a hierarchy of levels where the top level reaches back down and affects the bottom level. In a generative SX program:")
(ul :class "list-disc pl-5 space-y-2 text-stone-600"
(li "The bottom level is the evaluator — it evaluates expressions")
(li "The middle level is the program — expressions that produce values")
(li "The top level is the generator — values that are new expressions")
(li "The loop closes: new expressions are evaluated by the evaluator"))
(p "The program that writes itself is not a metaphor. It's a literal description of what happens when an SX expression evaluates to an SX string that is then parsed and evaluated. The output of evaluation becomes the input to evaluation. The program is both the writer and the written.")
(p "The browser version constrains this to DOM mutations. The server version constrains it to request handling. The unconstrained version — a bare seed with " (code "next-source") " — is a strange loop in its purest form: an evaluator that evaluates what it generates, generating what it evaluates.")
(div :class "rounded border border-stone-200 bg-stone-50 p-4 mt-6"
(p :class "text-stone-700 font-medium mb-2" "The practical consequence")
(p :class "text-stone-600 text-sm" "An SX development environment where the tools are written in SX, running inside SX, modifying SX. The editor understands the code because it " (em "is") " the code. The debugger inspects the environment because it " (em "shares") " the environment. The compiler reads the spec because the spec is in the same format as everything else. There is no impedance mismatch between any layer because there is only one layer.")))
))