Add sexpr.js runtime plan and comprehensive Ghost removal plan
Two planning documents for the next major architectural steps: - sexpr-js-runtime-plan: isomorphic JS s-expression runtime for client-side rendering, content-addressed component caching, and native hypermedia mutations - ghost-removal-plan: full Ghost CMS replacement covering content (Lexical→sexp), membership, newsletters, Stripe subscriptions, and media uploads Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
757
docs/sexpr-js-runtime-plan.md
Normal file
757
docs/sexpr-js-runtime-plan.md
Normal file
@@ -0,0 +1,757 @@
|
||||
# sexpr.js — Development Plan
|
||||
|
||||
**An isomorphic S-expression runtime for the web.**
|
||||
**Code is data is DOM.**
|
||||
|
||||
---
|
||||
|
||||
## Vision
|
||||
|
||||
Replace HTML as the document/wire format with S-expressions. A single JavaScript library runs on both server (Node/Deno/Bun) and client (browser). The server composes and sends S-expressions over HTTP or WebSocket. The client parses them and renders/mutates the DOM. Because S-expressions are homoiconic, hypermedia controls (fetching, swapping, transitions) are native to the format — not bolted on as special attributes.
|
||||
|
||||
The framework is not a Lisp. It is a document runtime that happens to use S-expression syntax because that syntax makes documents and commands interchangeable.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────┐
|
||||
│ sexpr.js (shared) │
|
||||
│ │
|
||||
│ ┌───────────┐ ┌───────────┐ ┌────────────────┐ │
|
||||
│ │ Parser / │ │ Component │ │ Mutation │ │
|
||||
│ │ Serializer │ │ Registry │ │ Engine │ │
|
||||
│ └───────────┘ └───────────┘ └────────────────┘ │
|
||||
│ ┌───────────┐ ┌───────────┐ ┌────────────────┐ │
|
||||
│ │ Style │ │ Event │ │ VTree Diff │ │
|
||||
│ │ Compiler │ │ System │ │ & Patch │ │
|
||||
│ └───────────┘ └───────────┘ └────────────────┘ │
|
||||
└─────────────┬───────────────────────────┬───────────┘
|
||||
│ │
|
||||
┌────────▼────────┐ ┌─────────▼─────────┐
|
||||
│ Server Adapter │ │ Client Adapter │
|
||||
│ │ │ │
|
||||
│ • renderToStr() │ │ • renderToDOM() │
|
||||
│ • diff on AST │ │ • mount/hydrate │
|
||||
│ • HTTP handler │ │ • WebSocket recv │
|
||||
│ • WS push │ │ • event dispatch │
|
||||
│ • SSR bootstrap │ │ • service worker │
|
||||
└──────────────────┘ └────────────────────┘
|
||||
```
|
||||
|
||||
The core is environment-agnostic. Thin adapters provide DOM APIs (client) or string serialization (server). Both sides share the parser, component system, style compiler, and mutation engine.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Core Runtime (Weeks 1–4)
|
||||
|
||||
The foundation. A single ES module that works in any JS environment.
|
||||
|
||||
### 1.1 Parser & Serializer
|
||||
|
||||
**Parser** — tokenizer + recursive descent, producing an AST of plain JS objects.
|
||||
|
||||
- Atoms: strings (`"hello"`), numbers (`42`, `3.14`), booleans (`#t`, `#f`), symbols (`div`, `my-component`), keywords (`:class`, `:on-click`)
|
||||
- Lists: `(tag :attr "val" children...)`
|
||||
- Comments: `; line comment` and `#| block comment |#`
|
||||
- Quasiquote / unquote: `` ` `` and `,` for template interpolation on the server
|
||||
- Streaming parser variant for large documents (SAX-style)
|
||||
|
||||
**Serializer** — AST back to S-expression string. Round-trip fidelity. Pretty-printer with configurable indentation.
|
||||
|
||||
**Deliverables:**
|
||||
- `parse(string) → AST`
|
||||
- `serialize(AST) → string`
|
||||
- `prettyPrint(AST, opts) → string`
|
||||
- Streaming: `createParser()` returning a push-based parser
|
||||
- Comprehensive test suite (edge cases: nested strings, escapes, unicode, deeply nested structures)
|
||||
- Benchmark: parse speed vs JSON.parse for equivalent data
|
||||
|
||||
### 1.2 AST Representation
|
||||
|
||||
The AST should be cheap to construct, diff, and serialize. Plain objects, not classes:
|
||||
|
||||
```javascript
|
||||
// Atoms
|
||||
{ type: 'symbol', value: 'div' }
|
||||
{ type: 'keyword', value: 'class' }
|
||||
{ type: 'string', value: 'hello' }
|
||||
{ type: 'number', value: 42 }
|
||||
{ type: 'boolean', value: true }
|
||||
|
||||
// List (the fundamental structure)
|
||||
[head, ...rest] // plain arrays — cheap, diffable, JSON-compatible
|
||||
|
||||
// Element sugar (derived during render, not stored)
|
||||
// (div :class "box" (p "hi")) →
|
||||
// [sym('div'), kw('class'), str('box'), [sym('p'), str('hi')]]
|
||||
```
|
||||
|
||||
**Design decision:** ASTs are plain arrays and objects. No custom classes. This means they serialize to JSON trivially — enabling WebSocket transmission, IndexedDB caching, and worker postMessage without structured clone overhead.
|
||||
|
||||
### 1.3 Element Rendering
|
||||
|
||||
The core render function: AST → target output.
|
||||
|
||||
**Shared logic** (environment-agnostic):
|
||||
- Parse keyword attributes from element expressions
|
||||
- Resolve component references
|
||||
- Evaluate special forms (`if`, `each`, `list`, `let`, `slot`)
|
||||
- Compile inline styles
|
||||
|
||||
**Client adapter** — `renderToDOM(ast, env) → Node`:
|
||||
- Creates real DOM nodes via `document.createElement`
|
||||
- Handles SVG namespace detection
|
||||
- Registers event handlers
|
||||
- Returns live DOM node
|
||||
|
||||
**Server adapter** — `renderToString(ast, env) → string`:
|
||||
- Produces HTML string for initial page load (SEO, fast first paint)
|
||||
- Inserts hydration markers so the client can attach without full re-render
|
||||
- Escapes text content for safety
|
||||
|
||||
### 1.4 Style System
|
||||
|
||||
Styles as S-expressions, compiled to CSS strings. Isomorphic: the same style expressions produce CSS on server (injected into `<style>` tags in HTML) and client (injected into document).
|
||||
|
||||
```scheme
|
||||
(style ".card"
|
||||
:background "#1a1a2e"
|
||||
:border-radius "8px"
|
||||
:padding "1.5rem"
|
||||
:hover (:background "#2a2a3e") ; nested pseudo-classes
|
||||
:media "(max-width: 600px)"
|
||||
(:padding "1rem"))
|
||||
|
||||
(style "@keyframes fade-in"
|
||||
(from :opacity "0")
|
||||
(to :opacity "1"))
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Nested selectors (like Sass)
|
||||
- `@media`, `@keyframes`, `@container` as nested S-expressions
|
||||
- CSS variables as regular properties
|
||||
- Optional: scoped styles per component (auto-prefix class names)
|
||||
- Output: raw CSS string or `<style>` DOM node
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Hypermedia Engine (Weeks 5–8)
|
||||
|
||||
The mutation layer. This is where homoiconicity pays off.
|
||||
|
||||
### 2.1 Mutation Primitives
|
||||
|
||||
Since commands and content share the same syntax, the server can send either — or both — in a single response:
|
||||
|
||||
```scheme
|
||||
;; Content (renders to DOM)
|
||||
(div :class "card" (p "Hello"))
|
||||
|
||||
;; Command (mutates existing DOM)
|
||||
(swap! "#card-1" :inner (p "Updated"))
|
||||
|
||||
;; Compound (multiple mutations atomically)
|
||||
(batch!
|
||||
(swap! "#notifications" :append
|
||||
(div :class "toast" "Saved!"))
|
||||
(class! "#save-btn" :remove "loading")
|
||||
(transition! "#toast" :type "slide-in"))
|
||||
```
|
||||
|
||||
**Full primitive set:**
|
||||
|
||||
| Primitive | Purpose |
|
||||
|---|---|
|
||||
| `swap!` | Replace/insert content (`:inner`, `:outer`, `:before`, `:after`, `:prepend`, `:append`, `:delete`, `:morph`) |
|
||||
| `batch!` | Execute multiple mutations atomically |
|
||||
| `class!` | Add/remove/toggle CSS classes |
|
||||
| `attr!` | Set/remove attributes |
|
||||
| `style!` | Inline style manipulation |
|
||||
| `transition!` | CSS transitions and animations |
|
||||
| `wait!` | Delay between batched mutations |
|
||||
| `dispatch!` | Fire custom events |
|
||||
|
||||
### 2.2 Request/Response Cycle
|
||||
|
||||
The equivalent of HTMX's `hx-get`, `hx-post`, etc. — but as native S-expressions:
|
||||
|
||||
```scheme
|
||||
(request!
|
||||
:method "POST"
|
||||
:url "/api/todos"
|
||||
:target "#todo-list"
|
||||
:swap inner
|
||||
:include "#todo-form" ; serialize form data
|
||||
:indicator "#spinner" ; show during request
|
||||
:confirm "Are you sure?" ; browser confirm dialog
|
||||
:timeout 5000
|
||||
:retry 3
|
||||
:on-error (swap! "#errors" :inner
|
||||
(p :class "error" "Request failed")))
|
||||
```
|
||||
|
||||
**Client-side implementation:**
|
||||
1. Serialize form data (if `:include` specified)
|
||||
2. Show indicator
|
||||
3. `fetch()` with `Accept: text/x-sexpr` header
|
||||
4. Parse response as S-expression
|
||||
5. If response is a mutation command → execute it
|
||||
6. If response is content → wrap in `swap!` using `:target` and `:swap`
|
||||
7. Hide indicator
|
||||
8. Handle errors
|
||||
|
||||
**Server-side helpers:**
|
||||
```javascript
|
||||
// Server constructs response using the same library
|
||||
const { s, sym, kw, str } = require('sexpr');
|
||||
|
||||
app.post('/api/todos', (req, res) => {
|
||||
const todo = createTodo(req.body);
|
||||
res.type('text/x-sexpr').send(
|
||||
s.serialize(
|
||||
s.batch(
|
||||
s.swap('#todo-list', 'append', todoComponent(todo)),
|
||||
s.swap('#todo-count', 'inner', s.text(`${count} remaining`)),
|
||||
s.swap('#todo-input', 'attr', { value: '' })
|
||||
)
|
||||
)
|
||||
);
|
||||
});
|
||||
```
|
||||
|
||||
### 2.3 WebSocket Channel
|
||||
|
||||
For real-time updates. The server pushes S-expression mutations over WebSocket:
|
||||
|
||||
```scheme
|
||||
;; Server → Client (push)
|
||||
(batch!
|
||||
(swap! "#user-42-status" :inner
|
||||
(span :class "online" "Online"))
|
||||
(swap! "#chat-messages" :append
|
||||
(div :class "message"
|
||||
(strong "Alice") " just joined")))
|
||||
```
|
||||
|
||||
**Protocol:**
|
||||
- Content-type negotiation: `text/x-sexpr` over HTTP, raw S-expr strings over WS
|
||||
- Client reconnects automatically with exponential backoff
|
||||
- Server can send any mutation at any time — the client just evaluates it
|
||||
- Optional: message IDs for acknowledgment, ordering guarantees
|
||||
|
||||
### 2.4 Event Binding
|
||||
|
||||
Declarative event binding that works both inline and as post-render setup:
|
||||
|
||||
```scheme
|
||||
;; Inline (in element definition)
|
||||
(button :on-click (request! :method "POST" :url "/api/like"
|
||||
:target "#like-count" :swap inner)
|
||||
"Like")
|
||||
|
||||
;; Declarative (standalone, for progressive enhancement)
|
||||
(on-event! "#search-input" "input"
|
||||
:debounce 300
|
||||
(request! :method "GET"
|
||||
:url (concat "/api/search?q=" (value event.target))
|
||||
:target "#results" :swap inner))
|
||||
|
||||
;; Keyboard shortcuts
|
||||
(on-event! "body" "keydown"
|
||||
:filter (= event.key "Escape")
|
||||
(class! "#modal" :remove "open"))
|
||||
```
|
||||
|
||||
**Event modifiers** (inspired by Vue/Svelte):
|
||||
- `:debounce 300` — debounce in ms
|
||||
- `:throttle 500` — throttle in ms
|
||||
- `:once` — fire once then unbind
|
||||
- `:prevent` — preventDefault
|
||||
- `:stop` — stopPropagation
|
||||
- `:filter (expr)` — conditional guard
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: Component System (Weeks 9–12)
|
||||
|
||||
### 3.1 Component Definition & Instantiation
|
||||
|
||||
Components are parameterized S-expression templates. Not classes. Not functions. Data.
|
||||
|
||||
```scheme
|
||||
(component "todo-item" (id text done)
|
||||
(style ".todo-item" :display "flex" :align-items "center" :gap "0.75rem")
|
||||
(style ".todo-item.done .text" :text-decoration "line-through" :opacity "0.5")
|
||||
|
||||
(li :class (if done "todo-item done" "todo-item") :id (concat "todo-" id)
|
||||
(span :class "check"
|
||||
:on-click (request! :method "POST"
|
||||
:url (concat "/api/todos/" id "/toggle")
|
||||
:target (concat "#todo-" id) :swap outer)
|
||||
(if done "◉" "○"))
|
||||
(span :class "text" text)
|
||||
(button :class "delete"
|
||||
:on-click (request! :method "DELETE"
|
||||
:url (concat "/api/todos/" id)
|
||||
:target (concat "#todo-" id) :swap delete)
|
||||
"×")))
|
||||
|
||||
;; Usage
|
||||
(use "todo-item" :id "1" :text "Buy milk" :done #f)
|
||||
```
|
||||
|
||||
**Features:**
|
||||
- Lexically scoped parameters
|
||||
- Styles co-located with component (auto-scoped or global, configurable)
|
||||
- Slots for content projection: `(slot)` for default, `(slot "header")` for named
|
||||
- Isomorphic: same component definition renders on server (to string) or client (to DOM)
|
||||
|
||||
### 3.2 Slots & Composition
|
||||
|
||||
```scheme
|
||||
(component "card" (title)
|
||||
(div :class "card"
|
||||
(div :class "card-header" (h3 title))
|
||||
(div :class "card-body" (slot)) ; default slot
|
||||
(div :class "card-footer" (slot "footer")))) ; named slot
|
||||
|
||||
(use "card" :title "My Card"
|
||||
(p "This goes in the default slot")
|
||||
(template :slot "footer"
|
||||
(button "OK") (button "Cancel")))
|
||||
```
|
||||
|
||||
### 3.3 Layouts & Pages
|
||||
|
||||
Higher-level composition for full pages:
|
||||
|
||||
```scheme
|
||||
(layout "main" ()
|
||||
(style "body" :font-family "var(--font)" :margin "0")
|
||||
(style ".layout" :display "grid" :grid-template-rows "auto 1fr auto" :min-height "100vh")
|
||||
|
||||
(div :class "layout"
|
||||
(header (use "nav-bar"))
|
||||
(main (slot))
|
||||
(footer (use "site-footer"))))
|
||||
|
||||
;; A page uses a layout
|
||||
(page "/about"
|
||||
:layout "main"
|
||||
:title "About Us"
|
||||
(section
|
||||
(h1 "About")
|
||||
(p "We replaced HTML with S-expressions.")))
|
||||
```
|
||||
|
||||
### 3.4 Server-Side Component Registry
|
||||
|
||||
On the server, components are registered globally and can be shared between routes:
|
||||
|
||||
```javascript
|
||||
const { registry, component, page, serve } = require('sexpr/server');
|
||||
|
||||
// Register components (can also load from .sexpr files)
|
||||
registry.loadDir('./components');
|
||||
|
||||
// Or inline
|
||||
registry.define('greeting', ['name'],
|
||||
s`(div :class "greeting" (h1 "Hello, " name))`
|
||||
);
|
||||
|
||||
// Route handler returns S-expressions
|
||||
app.get('/about', (req, res) => {
|
||||
res.sexpr(
|
||||
page('/about', { layout: 'main', title: 'About' },
|
||||
s`(section (h1 "About") (p "Hello world"))`)
|
||||
);
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: Virtual Tree & Diffing (Weeks 13–16)
|
||||
|
||||
### 4.1 VTree Representation
|
||||
|
||||
The parsed AST already *is* the virtual tree — no separate representation needed. This is a direct benefit of homoiconicity. While React needs `createElement()` to build a virtual DOM from JSX, our AST is the VDOM.
|
||||
|
||||
```
|
||||
S-expression string → parse() → AST ≡ VTree
|
||||
│
|
||||
renderToDOM() │ diff()
|
||||
▼
|
||||
DOM / Patches
|
||||
```
|
||||
|
||||
### 4.2 Diff Algorithm
|
||||
|
||||
O(n) same-level comparison, similar to React's reconciliation but operating on S-expression ASTs:
|
||||
|
||||
```javascript
|
||||
// Both sides can call this
|
||||
const patches = diff(oldTree, newTree);
|
||||
|
||||
// Client applies to DOM
|
||||
applyPatches(rootNode, patches);
|
||||
|
||||
// Server serializes as mutation commands
|
||||
const mutations = patchesToSexpr(patches);
|
||||
// → (batch! (swap! "#el-3" :inner (p "new")) (attr! "#el-7" :set :class "active"))
|
||||
```
|
||||
|
||||
**Patch types:**
|
||||
- `REPLACE` — replace entire node
|
||||
- `PROPS` — update attributes
|
||||
- `TEXT` — update text content
|
||||
- `INSERT` — insert child at index
|
||||
- `REMOVE` — remove child at index
|
||||
- `REORDER` — reorder children (using `:key` hints)
|
||||
|
||||
**Key insight:** Because patches are also S-expressions, the server can compute a diff and send it as a `batch!` of mutations. The client doesn't need to diff at all — it just executes the mutations. This means the server does the expensive work and the client stays thin.
|
||||
|
||||
### 4.3 Keyed Reconciliation
|
||||
|
||||
For efficient list updates:
|
||||
|
||||
```scheme
|
||||
(each todos (lambda (t)
|
||||
(use "todo-item" :key (get t "id") :text (get t "text") :done (get t "done"))))
|
||||
```
|
||||
|
||||
The `:key` attribute enables the diff algorithm to match nodes across re-renders, minimizing DOM operations for list insertions, deletions, and reorderings.
|
||||
|
||||
### 4.4 Hydration
|
||||
|
||||
Server sends pre-rendered HTML (for SEO and fast first paint). Client attaches to existing DOM without re-rendering:
|
||||
|
||||
1. Server renders S-expression → HTML string with hydration markers
|
||||
2. Browser displays HTML immediately (fast first contentful paint)
|
||||
3. Client JS loads, parses the original S-expression source (embedded in a `<script type="text/x-sexpr">` tag)
|
||||
4. Client walks the existing DOM and attaches event handlers without rebuilding it
|
||||
5. Subsequent updates go through the normal S-expression channel
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: Developer Experience (Weeks 17–20)
|
||||
|
||||
### 5.1 CLI Tool
|
||||
|
||||
```bash
|
||||
npx sexpr init my-app # scaffold project
|
||||
npx sexpr dev # dev server with hot reload
|
||||
npx sexpr build # production build
|
||||
npx sexpr serve # production server
|
||||
```
|
||||
|
||||
**Project structure:**
|
||||
```
|
||||
my-app/
|
||||
components/
|
||||
nav-bar.sexpr
|
||||
todo-item.sexpr
|
||||
card.sexpr
|
||||
pages/
|
||||
index.sexpr
|
||||
about.sexpr
|
||||
layouts/
|
||||
main.sexpr
|
||||
styles/
|
||||
theme.sexpr
|
||||
server.js
|
||||
sexpr.config.js
|
||||
```
|
||||
|
||||
### 5.2 `.sexpr` File Format
|
||||
|
||||
Single-file components with co-located styles, markup, and metadata:
|
||||
|
||||
```scheme
|
||||
; components/todo-item.sexpr
|
||||
|
||||
(meta
|
||||
:name "todo-item"
|
||||
:params (id text done)
|
||||
:description "A single todo list item with toggle and delete")
|
||||
|
||||
(style ".todo-item"
|
||||
:display "flex"
|
||||
:align-items "center"
|
||||
:gap "0.75rem")
|
||||
|
||||
(li :class (if done "todo-item done" "todo-item")
|
||||
:id (concat "todo-" id)
|
||||
:key id
|
||||
(span :class "check"
|
||||
:on-click (request! :method "POST"
|
||||
:url (concat "/api/todos/" id "/toggle")
|
||||
:target (concat "#todo-" id) :swap outer)
|
||||
(if done "◉" "○"))
|
||||
(span :class "text" text)
|
||||
(button :class "delete"
|
||||
:on-click (request! :method "DELETE"
|
||||
:url (concat "/api/todos/" id)
|
||||
:target (concat "#todo-" id) :swap delete)
|
||||
"×"))
|
||||
```
|
||||
|
||||
### 5.3 DevTools
|
||||
|
||||
**Browser extension** (or embedded panel):
|
||||
- AST inspector: visualize the S-expression tree alongside the DOM
|
||||
- Mutation log: every `swap!`, `class!`, `batch!` logged with timestamp
|
||||
- Network tab: S-expression request/response viewer (not raw text)
|
||||
- Component tree: hierarchical view of instantiated components
|
||||
- Time-travel: replay mutations forward/backward
|
||||
|
||||
**Server-side:**
|
||||
- Request logger showing S-expression responses
|
||||
- Component dependency graph
|
||||
- Hot reload: file watcher on `.sexpr` files, push updates via WebSocket
|
||||
|
||||
### 5.4 Editor Support
|
||||
|
||||
- **VS Code extension**: syntax highlighting for `.sexpr` files, bracket matching (parentheses), auto-indentation, component name completion, attribute completion for HTML tags
|
||||
- **Tree-sitter grammar**: for Neovim, Helix, Zed, etc.
|
||||
- **Prettier plugin**: auto-format `.sexpr` files
|
||||
- **LSP server**: go-to-definition for components, find-references, rename symbol
|
||||
|
||||
### 5.5 Error Handling
|
||||
|
||||
- **Parse errors**: line/column reporting with context (show the offending line)
|
||||
- **Render errors**: error boundaries like React — a component crash renders a fallback, not a blank page
|
||||
- **Network errors**: `:on-error` handler in `request!`, plus global error handler
|
||||
- **Dev mode**: verbose errors with suggestions; production mode: compact
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: Ecosystem & Production (Weeks 21–28)
|
||||
|
||||
### 6.1 Middleware & Plugins
|
||||
|
||||
**Server middleware** (Express/Koa/Hono compatible):
|
||||
|
||||
```javascript
|
||||
const { sexprMiddleware } = require('sexpr/server');
|
||||
|
||||
app.use(sexprMiddleware({
|
||||
componentsDir: './components',
|
||||
layoutsDir: './layouts',
|
||||
hotReload: process.env.NODE_ENV !== 'production'
|
||||
}));
|
||||
|
||||
// Route handlers return S-expressions directly
|
||||
app.get('/', (req, res) => {
|
||||
res.sexpr(homePage(req.user));
|
||||
});
|
||||
```
|
||||
|
||||
**Plugin system** for extending the runtime:
|
||||
|
||||
```javascript
|
||||
// A plugin that adds a (markdown "...") special form
|
||||
sexpr.plugin('markdown', {
|
||||
transform(ast) { /* convert markdown to sexpr AST */ },
|
||||
serverRender(ast) { /* render to HTML string */ },
|
||||
clientRender(ast) { /* render to DOM */ }
|
||||
});
|
||||
```
|
||||
|
||||
### 6.2 Router
|
||||
|
||||
Client-side navigation without full page reloads:
|
||||
|
||||
```scheme
|
||||
(router
|
||||
(route "/" (use "home-page"))
|
||||
(route "/about" (use "about-page"))
|
||||
(route "/todos/:id" (use "todo-detail" :id params.id))
|
||||
(route "*" (use "not-found")))
|
||||
```
|
||||
|
||||
- Intercepts `<a>` clicks for internal links
|
||||
- `pushState` / `popState` navigation
|
||||
- Server-side: same route definitions, used for SSR
|
||||
- Prefetch: `(link :href "/about" :prefetch #t "About")`
|
||||
|
||||
### 6.3 Forms
|
||||
|
||||
Declarative form handling:
|
||||
|
||||
```scheme
|
||||
(form :action "/api/users" :method "POST"
|
||||
:target "#result" :swap inner
|
||||
:validate #t ; client-side validation
|
||||
:reset-on-success #t
|
||||
|
||||
(input :type "text" :name "username"
|
||||
:required #t
|
||||
:minlength 3
|
||||
:pattern "[a-zA-Z0-9]+"
|
||||
:error-message "Alphanumeric, 3+ chars")
|
||||
|
||||
(input :type "email" :name "email" :required #t)
|
||||
|
||||
(button :type "submit" "Create User"))
|
||||
```
|
||||
|
||||
The server validates identically — same validation rules expressed as S-expressions run on both sides.
|
||||
|
||||
### 6.4 Content-Type & MIME
|
||||
|
||||
Register `text/x-sexpr` as a proper MIME type:
|
||||
|
||||
- HTTP responses: `Content-Type: text/x-sexpr; charset=utf-8`
|
||||
- `Accept` header negotiation: client sends `Accept: text/x-sexpr, text/html;q=0.9`
|
||||
- Fallback: if client doesn't accept `text/x-sexpr`, server renders to HTML (graceful degradation)
|
||||
- File extension: `.sexpr`
|
||||
|
||||
### 6.5 Caching & Service Worker
|
||||
|
||||
- **Service worker**: caches S-expression responses, serves offline
|
||||
- **Incremental cache**: cache individual components, not whole pages
|
||||
- **ETag/304**: standard HTTP caching works because responses are text
|
||||
- **Compression**: S-expressions compress well with gzip/brotli (repetitive keywords)
|
||||
|
||||
### 6.6 Security
|
||||
|
||||
- **No `eval()`**: S-expressions are parsed, not evaluated as code. The runtime only understands the defined special forms.
|
||||
- **XSS prevention**: text content is always escaped when rendered to DOM (via `textContent`, not `innerHTML`). The `raw-html` escape hatch requires explicit opt-in.
|
||||
- **CSP compatible**: no inline scripts generated. Event handlers are registered via JS, not `onclick` attributes (move away from the v1 prototype approach).
|
||||
- **CSRF**: standard token-based approach, with `(meta :csrf-token "...")` in the page head.
|
||||
|
||||
### 6.7 Accessibility
|
||||
|
||||
- S-expressions map 1:1 to semantic HTML elements — `(nav ...)`, `(main ...)`, `(article ...)`, `(aside ...)` all render to their HTML equivalents
|
||||
- ARIA attributes work naturally: `:aria-label "Close" :aria-expanded #f :role "dialog"`
|
||||
- Focus management primitives: `(focus! "#element")`, `(trap-focus! "#modal")`
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: Testing & Documentation (Weeks 25–28, overlapping)
|
||||
|
||||
### 7.1 Testing Utilities
|
||||
|
||||
```javascript
|
||||
const { render, fireEvent, waitForMutation } = require('sexpr/test');
|
||||
|
||||
test('todo toggle', async () => {
|
||||
const tree = render('(use "todo-item" :id "1" :text "Buy milk" :done #f)');
|
||||
|
||||
expect(tree.query('.todo-item')).not.toHaveClass('done');
|
||||
|
||||
fireEvent.click(tree.query('.check'));
|
||||
await waitForMutation('#todo-1');
|
||||
|
||||
expect(tree.query('.todo-item')).toHaveClass('done');
|
||||
});
|
||||
```
|
||||
|
||||
- `render()` works in Node (using JSDOM) or browser
|
||||
- Snapshot testing: compare AST snapshots, not HTML string snapshots
|
||||
- Mutation assertions: `expectMutation(swap!(...))` to test server responses
|
||||
|
||||
### 7.2 Documentation Site
|
||||
|
||||
Built with sexpr.js itself (dogfooding). Includes:
|
||||
- Tutorial: build a todo app from scratch
|
||||
- API reference: every special form, mutation primitive, configuration option
|
||||
- Cookbook: common patterns (modals, infinite scroll, real-time chat, auth flows)
|
||||
- Interactive playground: edit S-expressions, see live DOM output
|
||||
- Migration guide: "coming from HTMX" and "coming from React"
|
||||
|
||||
---
|
||||
|
||||
## Technical Decisions
|
||||
|
||||
### Why Not JSON?
|
||||
|
||||
JSON could represent the same tree structure. But:
|
||||
- S-expressions are more compact for deeply nested structures (no commas, colons, or quotes on keys)
|
||||
- The syntax is its own DSL — `:keywords`, symbols, and lists feel natural for document description
|
||||
- Comments are supported (JSON has none)
|
||||
- Human-writeable: developers will author `.sexpr` files directly
|
||||
- Cultural signal: this is a Lisp-inspired project, and the syntax communicates the homoiconicity thesis immediately
|
||||
|
||||
### Why Not Compile to WebAssembly?
|
||||
|
||||
Tempting for parser performance, but:
|
||||
- JS engines already optimize parsing hot paths well
|
||||
- WASM has overhead for DOM interaction (must cross the JS boundary anyway)
|
||||
- Staying in pure JS means the library works everywhere JS does with zero build step
|
||||
- Future option: WASM parser module for very large documents
|
||||
|
||||
### Module Format
|
||||
|
||||
- ES modules (`.mjs`) as primary
|
||||
- CommonJS (`.cjs`) build for older Node.js
|
||||
- UMD build for `<script>` tag usage (the "drop it in and it works" story)
|
||||
- TypeScript type definitions (`.d.ts`) shipped alongside
|
||||
|
||||
### Bundle Size Target
|
||||
|
||||
- Core parser + renderer: **< 5KB** gzipped
|
||||
- With mutation engine: **< 8KB** gzipped
|
||||
- Full framework (router, forms, devtools hook): **< 15KB** gzipped
|
||||
|
||||
For comparison: HTMX is ~14KB, Alpine.js is ~15KB, Preact is ~3KB.
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
| Milestone | Target | Deliverable |
|
||||
|---|---|---|
|
||||
| **M1: Parser** | Week 2 | `parse()`, `serialize()`, full test suite, benchmarks |
|
||||
| **M2: Client renderer** | Week 4 | `renderToDOM()`, styles, events — the v1 prototype, cleaned up |
|
||||
| **M3: Server renderer** | Week 6 | `renderToString()`, Express middleware, SSR bootstrap |
|
||||
| **M4: Mutations** | Week 8 | `swap!`, `batch!`, `request!`, `class!` — full hypermedia engine |
|
||||
| **M5: WebSocket** | Week 10 | Real-time server push, reconnection, protocol spec |
|
||||
| **M6: Components** | Week 12 | Component system, slots, `.sexpr` file format, registry |
|
||||
| **M7: Diffing** | Week 16 | VTree diff, keyed reconciliation, hydration |
|
||||
| **M8: CLI & DX** | Week 20 | `sexpr init/dev/build`, hot reload, VS Code extension |
|
||||
| **M9: Ecosystem** | Week 24 | Router, forms, plugins, service worker caching |
|
||||
| **M10: Launch** | Week 28 | Docs site, npm publish, example apps, announcement |
|
||||
|
||||
---
|
||||
|
||||
## Example Apps (for launch)
|
||||
|
||||
1. **Todo MVC** — the classic benchmark, fully server-driven
|
||||
2. **Real-time chat** — WebSocket mutations, presence indicators
|
||||
3. **Dashboard** — data tables, charts, polling, search
|
||||
4. **Blog** — SSR, routing, SEO, markdown integration
|
||||
5. **E-commerce product page** — forms, validation, cart mutations
|
||||
|
||||
---
|
||||
|
||||
## Open Questions
|
||||
|
||||
1. **Macro system?** Should the server support `defmacro`-style macros for transforming S-expressions before rendering? This adds power but also complexity and potential security concerns.
|
||||
|
||||
2. **TypeScript integration?** Should component params be typed? Could generate TS interfaces from `(meta :params ...)` declarations.
|
||||
|
||||
3. **Compilation?** An optional ahead-of-time compiler could pre-parse `.sexpr` files into JS AST constructors, eliminating parse time at runtime. Worth the complexity?
|
||||
|
||||
4. **CSS-in-sexpr or external stylesheets?** The current approach co-locates styles. Should there also be a way to import `.css` files directly, or should all styling go through the S-expression syntax?
|
||||
|
||||
5. **Interop with existing HTML?** Can you embed an S-expression island inside an existing HTML page (like Astro islands)? Useful for incremental adoption.
|
||||
|
||||
6. **Binary wire format?** A compact binary encoding of the AST (like MessagePack for S-expressions) could reduce bandwidth for large pages. Worth the complexity vs. gzip?
|
||||
|
||||
---
|
||||
|
||||
## Name Candidates
|
||||
|
||||
- **sexpr.js** — direct, memorable, says what it is
|
||||
- **sdom** — S-expression DOM
|
||||
- **paren** — the defining character
|
||||
- **lispr** — Lisp + render
|
||||
- **homoDOM** — homoiconic DOM
|
||||
|
||||
---
|
||||
|
||||
*The document is the program. The program is the document.*
|
||||
Reference in New Issue
Block a user