Implement reader macros (#;, #|...|, #', #name) and #z3 demo
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 11m13s
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 11m13s
Reader macros in parser.sx spec, Python parser.py, and hand-written sx.js: - #; datum comment: read and discard next expression - #|...| raw string: no escape processing - #' quote shorthand: (quote expr) - #name extensible dispatch: registered handler transforms next expression #z3 reader macro demo (reader_z3.py): translates define-primitive declarations from primitives.sx into SMT-LIB verification conditions. Same source, two interpretations — bootstrappers compile to executable code, #z3 extracts proof obligations. 48 parser tests (SX spec + Python), all passing. Rebootstrapped JS+Python. Demo page at /plans/reader-macro-demo with side-by-side examples. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -155,6 +155,8 @@
|
||||
:summary "Audit of all plans — what's done, what's in progress, and what remains.")
|
||||
(dict :label "Reader Macros" :href "/plans/reader-macros"
|
||||
:summary "Extensible parse-time transformations via # dispatch — datum comments, raw strings, and quote shorthand.")
|
||||
(dict :label "Reader Macro Demo" :href "/plans/reader-macro-demo"
|
||||
:summary "Live demo: #z3 translates SX spec declarations to SMT-LIB verification conditions.")
|
||||
(dict :label "SX-Activity" :href "/plans/sx-activity"
|
||||
:summary "A new web built on SX — executable content, shared components, parsers, and logic on IPFS, provenance on Bitcoin, all running within your own security context.")
|
||||
(dict :label "Predictive Prefetching" :href "/plans/predictive-prefetch"
|
||||
|
||||
137
sx/sx/plans.sx
137
sx/sx/plans.sx
@@ -49,8 +49,9 @@
|
||||
(p "Currently no single-char quote (" (code "`") " is quasiquote).")
|
||||
(~doc-code :code (highlight "#'my-function → (quote my-function)" "lisp")))
|
||||
|
||||
(~doc-subsection :title "No user-defined reader macros (yet)"
|
||||
(p "Would require multi-pass parsing or boot-phase registration. The three built-ins cover practical needs. Extensible dispatch can come later without breaking anything.")))
|
||||
(~doc-subsection :title "Extensible dispatch: #name"
|
||||
(p "User-defined reader macros via " (code "#name expr") ". The parser reads an identifier after " (code "#") ", looks up a handler in the reader macro registry, and calls it with the next parsed expression. See the " (a :href "/plans/reader-macro-demo" :class "text-violet-600 hover:underline" "#z3 demo") " for a working example that translates SX spec declarations to SMT-LIB.")))
|
||||
|
||||
|
||||
;; -----------------------------------------------------------------------
|
||||
;; Implementation
|
||||
@@ -126,6 +127,132 @@
|
||||
(li "Rebootstrapped JS and Python pass their test suites")
|
||||
(li "JS parity: SX.parse('#|hello|') returns [\"hello\"]"))))))
|
||||
|
||||
;; ---------------------------------------------------------------------------
|
||||
;; Reader Macro Demo: #z3 — SX Spec to SMT-LIB
|
||||
;; ---------------------------------------------------------------------------
|
||||
|
||||
(defcomp ~plan-reader-macro-demo-content ()
|
||||
(~doc-page :title "Reader Macro Demo: #z3"
|
||||
|
||||
(~doc-section :title "The idea" :id "idea"
|
||||
(p :class "text-stone-600"
|
||||
"SX spec files (" (code "primitives.sx") ", " (code "eval.sx") ") are machine-readable declarations. Reader macros transform these at parse time. " (code "#z3") " reads a " (code "define-primitive") " declaration and emits " (a :href "https://smtlib.cs.uiowa.edu/" :class "text-violet-600 hover:underline" "SMT-LIB") " — the standard input language for " (a :href "https://github.com/Z3Prover/z3" :class "text-violet-600 hover:underline" "Z3") " and other theorem provers.")
|
||||
(p :class "text-stone-600"
|
||||
"Same source, two interpretations. The bootstrappers read " (code "define-primitive") " and emit executable code. " (code "#z3") " reads the same form and emits verification conditions. The specification is simultaneously a program and a proof obligation.")
|
||||
(div :class "rounded border border-violet-200 bg-violet-50 p-4 mt-4"
|
||||
(p :class "text-sm text-violet-800 font-semibold mb-2" "Key insight")
|
||||
(p :class "text-sm text-violet-700"
|
||||
"SMT-LIB is itself an s-expression language. The translation from SX to SMT-LIB is s-expressions to s-expressions — most arithmetic and boolean operators are literally the same syntax in both. The reader macro barely has to transform anything.")))
|
||||
|
||||
(~doc-section :title "Arithmetic primitives" :id "arithmetic"
|
||||
(p :class "text-stone-600"
|
||||
"Primitives with " (code ":body") " generate " (code "forall") " assertions. Z3 can verify the definition is satisfiable.")
|
||||
|
||||
(~doc-subsection :title "inc"
|
||||
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"(define-primitive \"inc\"\n :params (n)\n :returns \"number\"\n :doc \"Increment by 1.\"\n :body (+ n 1))"))
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"; inc — Increment by 1.\n(declare-fun inc (Int) Int)\n(assert (forall (((n Int)))\n (= (inc n) (+ n 1))))\n(check-sat)"))))
|
||||
|
||||
(~doc-subsection :title "clamp"
|
||||
(p :class "text-stone-600 mb-2"
|
||||
(code "max") " and " (code "min") " have no SMT-LIB equivalent — translated to " (code "ite") " (if-then-else).")
|
||||
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"(define-primitive \"clamp\"\n :params (x lo hi)\n :returns \"number\"\n :doc \"Clamp x to range [lo, hi].\"\n :body (max lo (min hi x)))"))
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"; clamp — Clamp x to range [lo, hi].\n(declare-fun clamp (Int Int Int) Int)\n(assert (forall (((x Int) (lo Int) (hi Int)))\n (= (clamp x lo hi)\n (ite (>= lo (ite (<= hi x) hi x))\n lo\n (ite (<= hi x) hi x)))))\n(check-sat)")))))
|
||||
|
||||
(~doc-section :title "Predicates" :id "predicates"
|
||||
(p :class "text-stone-600"
|
||||
"Predicates return " (code "Bool") " in SMT-LIB. " (code "mod") " and " (code "=") " are identity translations — same syntax in both languages.")
|
||||
|
||||
(~doc-subsection :title "odd?"
|
||||
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"(define-primitive \"odd?\"\n :params (n)\n :returns \"boolean\"\n :doc \"True if n is odd.\"\n :body (= (mod n 2) 1))"))
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"; odd? — True if n is odd.\n(declare-fun odd_p (Int) Bool)\n(assert (forall (((n Int)))\n (= (odd_p n) (= (mod n 2) 1))))\n(check-sat)"))))
|
||||
|
||||
(~doc-subsection :title "!= (inequality)"
|
||||
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"(define-primitive \"!=\"\n :params (a b)\n :returns \"boolean\"\n :doc \"Inequality.\"\n :body (not (= a b)))"))
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"; != — Inequality.\n(declare-fun neq (Int Int) Bool)\n(assert (forall (((a Int) (b Int)))\n (= (neq a b) (not (= a b)))))\n(check-sat)")))))
|
||||
|
||||
(~doc-section :title "Variadics and bodyless" :id "variadics"
|
||||
(p :class "text-stone-600"
|
||||
"Variadic primitives (" (code "&rest") ") are declared as uninterpreted functions — Z3 can reason about their properties but not their implementation. Primitives without " (code ":body") " get only a declaration.")
|
||||
|
||||
(~doc-subsection :title "+ (variadic)"
|
||||
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"(define-primitive \"+\"\n :params (&rest args)\n :returns \"number\"\n :doc \"Sum all arguments.\")"))
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"; + — Sum all arguments.\n; (variadic — modeled as uninterpreted)\n(declare-fun + (Int Int) Int)\n(check-sat)"))))
|
||||
|
||||
(~doc-subsection :title "nil? (no body)"
|
||||
(div :class "grid grid-cols-1 md:grid-cols-2 gap-4"
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SX Source")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"(define-primitive \"nil?\"\n :params (x)\n :returns \"boolean\"\n :doc \"True if x is nil/null/None.\")"))
|
||||
(div
|
||||
(p :class "text-xs text-stone-500 uppercase tracking-wider mb-1" "SMT-LIB Output")
|
||||
(~doc-code :lang "lisp" :code
|
||||
"; nil? — True if x is nil/null/None.\n(declare-fun nil_p (Int) Bool)\n(check-sat)")))))
|
||||
|
||||
(~doc-section :title "How it works" :id "how-it-works"
|
||||
(p :class "text-stone-600"
|
||||
"The " (code "#z3") " reader macro is registered before parsing. When the parser hits " (code "#z3(define-primitive ...)") ", it:")
|
||||
(ol :class "list-decimal pl-6 space-y-2 text-stone-600"
|
||||
(li "Reads the identifier " (code "z3") " after " (code "#") ".")
|
||||
(li "Looks up " (code "z3") " in the reader macro registry.")
|
||||
(li "Reads the next expression — the full " (code "define-primitive") " form — into an AST node.")
|
||||
(li "Passes the AST to the handler function.")
|
||||
(li "The handler walks the AST, extracts " (code ":params") ", " (code ":returns") ", " (code ":body") ", and emits SMT-LIB.")
|
||||
(li "The resulting string replaces " (code "#z3(...)") " in the parse output."))
|
||||
(~doc-code :lang "python" :code
|
||||
"# Registration — one line\nfrom shared.sx.ref.reader_z3 import register_z3_macro\nregister_z3_macro()\n\n# Then #z3 works in any SX source:\nresult = parse('#z3(define-primitive \"inc\" ...)')\n# result is the SMT-LIB string")
|
||||
(p :class "text-stone-600"
|
||||
"The handler is a pure function from AST to value. No side effects. No mutation. Reader macros are " (em "syntax transformations") " — they happen before evaluation, before rendering, before anything else. They are the earliest possible extension point."))
|
||||
|
||||
(~doc-section :title "The strange loop" :id "strange-loop"
|
||||
(p :class "text-stone-600"
|
||||
"The SX specification files are simultaneously:")
|
||||
(ul :class "list-disc pl-6 space-y-2 text-stone-600"
|
||||
(li (span :class "font-semibold" "Executable code") " — bootstrappers compile them to JavaScript and Python.")
|
||||
(li (span :class "font-semibold" "Documentation") " — this docs site renders them with syntax highlighting and prose.")
|
||||
(li (span :class "font-semibold" "Formal specifications") " — " (code "#z3") " extracts verification conditions that a theorem prover can check."))
|
||||
(p :class "text-stone-600"
|
||||
"One file. Three readings. No information lost. The " (code "define-primitive") " form in " (code "primitives.sx") " does not need to be translated, annotated, or re-expressed for any of these uses. The s-expression " (em "is") " the program, the documentation, and the proof obligation.")
|
||||
(p :class "text-stone-600"
|
||||
"And the reader macro that extracts proofs? It is itself written as a Python function that takes an SX AST and returns a string. It could be written in SX. It could be compiled by the bootstrappers. The transformation tools are made of the same material as the things they transform.")
|
||||
(p :class "text-stone-600"
|
||||
"This is not hypothetical. The examples on this page are live output from the " (code "#z3") " reader macro running against the actual " (code "primitives.sx") " declarations. The same declarations that the JavaScript bootstrapper compiles into " (code "sx-ref.js") ". The same declarations that the Python bootstrapper compiles into " (code "sx_ref.py") ". Same source. Different reader."))))
|
||||
|
||||
;; ---------------------------------------------------------------------------
|
||||
;; SX-Activity: Federated SX over ActivityPub
|
||||
;; ---------------------------------------------------------------------------
|
||||
@@ -1360,10 +1487,10 @@
|
||||
|
||||
(div :class "rounded border border-stone-200 bg-stone-50 p-4"
|
||||
(div :class "flex items-center gap-2 mb-1"
|
||||
(span :class "inline-block px-2 py-0.5 rounded text-xs font-bold bg-stone-500 text-white uppercase" "Not Started")
|
||||
(span :class "inline-block px-2 py-0.5 rounded text-xs font-bold bg-green-700 text-white uppercase" "Done")
|
||||
(a :href "/plans/reader-macros" :class "font-semibold text-stone-800 underline" "Reader Macros"))
|
||||
(p :class "text-sm text-stone-600" "# dispatch character for datum comments (#;), raw strings (#|...|), and quote shorthand (#'). Fully designed but no implementation in parser.sx or parser.py.")
|
||||
(p :class "text-sm text-stone-500 mt-1" "Remaining: spec in parser.sx, Python in parser.py, rebootstrap both targets."))
|
||||
(p :class "text-sm text-stone-600" "# dispatch in parser.sx spec, Python parser.py, hand-written sx.js. Three built-ins (#;, #|...|, #') plus extensible #name dispatch. #z3 demo translates define-primitive to SMT-LIB.")
|
||||
(p :class "text-sm text-stone-500 mt-1" "48 parser tests (SX + Python), all passing. Rebootstrapped to JS and Python."))
|
||||
|
||||
(div :class "rounded border border-stone-200 bg-stone-50 p-4"
|
||||
(div :class "flex items-center gap-2 mb-1"
|
||||
|
||||
@@ -629,6 +629,7 @@
|
||||
:content (case slug
|
||||
"status" (~plan-status-content)
|
||||
"reader-macros" (~plan-reader-macros-content)
|
||||
"reader-macro-demo" (~plan-reader-macro-demo-content)
|
||||
"sx-activity" (~plan-sx-activity-content)
|
||||
"predictive-prefetch" (~plan-predictive-prefetch-content)
|
||||
"content-addressed-components" (~plan-content-addressed-components-content)
|
||||
|
||||
Reference in New Issue
Block a user