From 3e29c2a3343be8f608d966ee79c94d4561397cdd Mon Sep 17 00:00:00 2001 From: giles Date: Sat, 28 Feb 2026 14:44:39 +0000 Subject: [PATCH] Unify sexp protocol and ActivityPub extension into single spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Merges sexpr-activitypub-extension.md and sexpr-protocol-and-tiered-clients.md into sexpr-unified-protocol.md — recognising that browsing, federation, and real-time updates are all the same thing: peers exchanging s-expressions on a bidirectional stream. One format, one connection, one parser. Co-Authored-By: Claude Opus 4.6 --- docs/sexpr-activitypub-extension.md | 353 ----------- docs/sexpr-protocol-and-tiered-clients.md | 454 -------------- docs/sexpr-unified-protocol.md | 731 ++++++++++++++++++++++ 3 files changed, 731 insertions(+), 807 deletions(-) delete mode 100644 docs/sexpr-activitypub-extension.md delete mode 100644 docs/sexpr-protocol-and-tiered-clients.md create mode 100644 docs/sexpr-unified-protocol.md diff --git a/docs/sexpr-activitypub-extension.md b/docs/sexpr-activitypub-extension.md deleted file mode 100644 index 7ffef75..0000000 --- a/docs/sexpr-activitypub-extension.md +++ /dev/null @@ -1,353 +0,0 @@ -# S-expression Wire Format for ActivityPub - -**An AP extension that replaces JSON-LD with s-expressions as the native wire format.** - ---- - -## Why JSON-LD Is the Wrong Format for ActivityPub - -JSON-LD was chosen for AP because of its semantic web lineage, but in practice nobody uses the linked data features — servers just pattern-match on `type` fields and ignore the `@context`. The format is: - -- **Verbose** — deeply nested objects with string keys, quoted values, commas, colons -- **Ambiguous** — compaction/expansion rules mean the same activity can have many valid JSON representations -- **Lossy** — post content is flattened to an HTML string inside a JSON string; structure is destroyed -- **Hostile to signing** — JSON-LD canonicalization is fragile and implementation-dependent -- **Hostile to AI** — agents must parse JSON, then parse embedded HTML, then reason about structure - ---- - -## Sexp AP: The Format - -### Activities as Expressions - -Every AP activity type is a symbol — the verb is the head of the expression: - -```scheme -(Create :actor alice - (Note :id post-123 - (p "content here"))) - -(Announce :actor bob - (Create :actor alice - (Note :id post-123))) - -(Undo :actor alice - (Like :actor alice :object post-123)) -``` - -Read top to bottom: who did what to what. No `"type"` field to find, no `"object"` wrapper to unwrap. - -### Full Activity Example - -```scheme -(Create - :id "https://rose-ash.com/ap/activities/456" - :actor "https://rose-ash.com/ap/users/alice" - :published "2026-02-28T14:00:00Z" - :to ("https://www.w3.org/ns/activitystreams#Public") - :cc ("https://rose-ash.com/ap/users/alice/followers") - - (Note - :id "https://rose-ash.com/ap/posts/123" - :attributed-to "https://rose-ash.com/ap/users/alice" - :in-reply-to "https://remote.social/posts/789" - :published "2026-02-28T14:00:00Z" - :summary "Weekend market update" - - ;; Content is sexp — structure preserved, not flattened to HTML string - (section - (p "Three new vendors joining this Saturday.") - (use "vendor-list" :vendors vendors) - (use "calendar-widget" :calendar-id 42)) - - ;; Attachments are typed expressions, not opaque JSON blobs - (attachment - (Image :url "https://rose-ash.com/media/market.jpg" - :media-type "image/jpeg" - :width 1200 :height 800 - :name "Saturday market stalls")) - - ;; Tags as data - (tag - (Hashtag :name "#markets" :href "https://rose-ash.com/tags/markets") - (Mention :name "@bob@remote.social" :href "https://remote.social/users/bob")))) -``` - -### JSON-LD Equivalent (for comparison) - -```json -{ - "@context": "https://www.w3.org/ns/activitystreams", - "type": "Create", - "id": "https://rose-ash.com/ap/activities/456", - "actor": "https://rose-ash.com/ap/users/alice", - "published": "2026-02-28T14:00:00Z", - "to": ["https://www.w3.org/ns/activitystreams#Public"], - "cc": ["https://rose-ash.com/ap/users/alice/followers"], - "object": { - "type": "Note", - "id": "https://rose-ash.com/ap/posts/123", - "attributedTo": "https://rose-ash.com/ap/users/alice", - "inReplyTo": "https://remote.social/posts/789", - "published": "2026-02-28T14:00:00Z", - "summary": "Weekend market update", - "content": "

Three new vendors joining this Saturday.

", - "attachment": [ - { - "type": "Image", - "url": "https://rose-ash.com/media/market.jpg", - "mediaType": "image/jpeg", - "width": 1200, - "height": 800, - "name": "Saturday market stalls" - } - ], - "tag": [ - {"type": "Hashtag", "name": "#markets", "href": "https://rose-ash.com/tags/markets"}, - {"type": "Mention", "name": "@bob@remote.social", "href": "https://remote.social/users/bob"} - ] - } -} -``` - -The JSON-LD is twice the size. The actual content — the post body — is a flat HTML string inside a JSON string. Structure lost. - -### Collections as Lists - -An outbox page: - -```scheme -(OrderedCollectionPage - :id "https://rose-ash.com/ap/users/alice/outbox?page=1" - :part-of "https://rose-ash.com/ap/users/alice/outbox" - :next "https://rose-ash.com/ap/users/alice/outbox?page=2" - :total-items 142 - - (Create :actor alice :published "2026-02-28T14:00:00Z" - (Note :id post-3 (p "Latest post"))) - (Announce :actor alice :published "2026-02-27T09:00:00Z" - (Note :id post-2 :attributed-to bob)) - (Create :actor alice :published "2026-02-26T18:00:00Z" - (Note :id post-1 (p "Earlier post")))) -``` - -Items are children of the collection expression. No `"orderedItems": [...]` wrapper. - -### Actor Profiles - -```scheme -(Person - :id "https://rose-ash.com/ap/users/alice" - :preferred-username "alice" - :name "Alice" - :summary (p "Co-op member, market organiser") - :inbox "https://rose-ash.com/ap/users/alice/inbox" - :outbox "https://rose-ash.com/ap/users/alice/outbox" - :followers "https://rose-ash.com/ap/users/alice/followers" - :following "https://rose-ash.com/ap/users/alice/following" - :public-key (:id "https://rose-ash.com/ap/users/alice#main-key" - :owner "https://rose-ash.com/ap/users/alice" - :pem "-----BEGIN PUBLIC KEY-----\n...")) -``` - ---- - -## Content Negotiation - -Servers advertise sexp support via HTTP headers: - -``` -Accept: application/x-sexpr+ap, application/activity+json;q=0.9 -Content-Type: application/x-sexpr+ap; charset=utf-8 -``` - -- Servers that don't understand sexp get standard `application/activity+json` (JSON-LD) -- Servers that do get `application/x-sexpr+ap` — compact, structured, renderable -- Graceful degradation built in from day one - -### Dual-Content Bridging - -For JSON-LD peers, include sexp as an extension field: - -```json -{ - "@context": ["https://www.w3.org/ns/activitystreams", "https://rose-ash.com/ns/sexpr"], - "type": "Create", - "object": { - "type": "Note", - "content": "

Hello world

", - "rose:sexpr": "(p \"Hello world\")" - } -} -``` - -Remote Mastodon/Pleroma servers see standard `content` HTML. Rose-ash instances see `rose:sexpr` and render natively — richer, smaller, with embedded components. - ---- - -## Advantages - -### Deterministic Signatures - -Sexp serialization has a canonical form: sorted keywords, no trailing whitespace, single-space separators. HTTP Signatures and content hashes are reproducible without JSON-LD canonicalization — which is notoriously fragile and implementation-dependent. - -### Content Is Structure - -In JSON-LD AP, post content is `"content": "

Hello

"` — an HTML string embedded in JSON. Clients must parse JSON, extract the string, then parse the HTML. Two parse steps, structure lost at each. - -In sexp AP, the content *is* the tree. `(p "Hello")` is simultaneously the wire format, the parse result, and the renderable document. One parse step, structure preserved. - -### Components Cross the Federation - -A rose-ash instance sending `(use "product-card" :name "Sourdough" :price "3.50")` to another rose-ash instance sends a component invocation. If both instances share the same component definition (identified by content hash), the receiving instance renders a full product card — not just flat text. - -This is federated UI, not just federated data. The artdag content-addressing model applies: components are content-addressed artifacts discoverable across the federation. - -### AI-Native - -An AI agent consuming a sexp AP feed gets structured trees it can parse, reason about, and respond to — not HTML soup embedded in JSON. It can generate replies as sexp, which the server translates to AP activities. The agent's tool vocabulary (components) is the same as the federation's content vocabulary. - -### Size - -Typical `Create{Note}` activity: ~800 bytes JSON-LD vs ~400 bytes sexp. With component references instead of inline content: even smaller. Compresses better with gzip/brotli due to repetitive keywords. - ---- - -## Implementation Plan - -### Phase 1: Sexp ↔ AP Serialization Layer - -Build bidirectional translation between sexp and JSON-LD AP: - -**sexp → JSON-LD** (outbound to non-sexp peers): -- Activity type symbol → `"type"` field -- Keyword attributes → JSON object properties -- Child expressions → nested `"object"`, `"attachment"`, `"tag"` arrays -- Content sexp children → rendered to HTML string for `"content"` field -- Deterministic: same sexp always produces same JSON-LD - -**JSON-LD → sexp** (inbound from any peer): -- `"type"` → head symbol -- Object properties → keyword attributes -- `"content"` HTML → parsed to sexp (best-effort; HTML→sexp parser) -- Nested objects → child expressions - -**Files:** -- `shared/infrastructure/ap_sexpr.py` — serializer/deserializer -- `shared/sexp/tests/test_ap_sexpr.py` — round-trip tests - -### Phase 2: Content Negotiation in Federation Service - -Add `Accept` header handling to the federation service inbox/outbox: - -- Outbound: check if target server accepts `application/x-sexpr+ap` - - If yes: send sexp directly - - If no: translate to JSON-LD, include `rose:sexpr` extension field -- Inbound: accept both `application/x-sexpr+ap` and `application/activity+json` - - If sexp: parse directly - - If JSON-LD: translate to sexp internally -- WebFinger: advertise sexp support in actor metadata - -**Files:** -- `federation/bp/inbox/routes.py` — add content-type handling -- `federation/bp/outbox/routes.py` — add accept-header negotiation -- `federation/services/delivery.py` — per-peer format selection - -### Phase 3: Sexp Content in Activities - -Replace HTML content with sexp in outbound activities: - -- When publishing a post, the sexp body becomes the canonical AP content -- For sexp peers: content is sent as-is (with component references) -- For JSON-LD peers: sexp is rendered to HTML for the `"content"` field -- Inbound HTML content from remote peers: parsed to sexp for storage/display - -**Depends on:** Ghost removal (Phase 1-5 of ghost-removal-plan.md) — posts must be sexp before they can be federated as sexp. - -**Files:** -- `federation/services/publishing.py` — sexp content in Create/Update activities -- `shared/sexp/html_to_sexp.py` — HTML → sexp parser for inbound content - -### Phase 4: Component Discovery Protocol - -Enable sexp-speaking peers to discover and cache shared components: - -- Actor metadata includes a `components` endpoint URL -- `GET /ap/components/` returns a manifest: component name → content hash -- `GET /ap/components/{hash}` returns the component definition (sexp) -- Receiving instance caches component definitions by hash -- Component invocations in federated content (`use "product-card" ...`) resolve from cache - -**Files:** -- `federation/bp/components/routes.py` — component manifest + fetch endpoints -- `shared/sexp/component_manifest.py` — hash generation, manifest building - -### Phase 5: Real-Time Federation via WebSocket - -For sexp-speaking peers, offer a persistent WebSocket connection instead of HTTP POST delivery: - -- Peers negotiate WS upgrade during Follow/Accept handshake -- Activities stream as sexp over the WebSocket -- Includes mutation commands for live UI updates: - ```scheme - (swap! "#federation-feed" :prepend - (use "federation-post" - :actor "alice@remote.social" - :content (p "Just posted!") - :published "2026-02-28T14:00:00Z")) - ``` -- Fallback: HTTP POST delivery for peers that don't support WS - -**Files:** -- `federation/services/ws_delivery.py` — WebSocket connection manager -- `federation/bp/ws/routes.py` — WebSocket endpoint - -### Phase 6: Specification Document - -Write a formal extension specification: - -- MIME type: `application/x-sexpr+ap` -- Canonical serialization rules (for signatures) -- Activity type → symbol mapping (all standard AP types) -- Content negotiation protocol -- Component discovery protocol -- WebSocket streaming protocol -- JSON-LD bridging rules (`rose:sexpr` extension field) -- Security considerations (no eval, escaping rules, signature verification) - -Publish as a FEP (Fediverse Enhancement Proposal). - ---- - -## Backwards Compatibility - -The extension is fully backwards-compatible: - -| Peer Type | Outbound Behaviour | Inbound Behaviour | -|---|---|---| -| Standard AP (Mastodon, Pleroma) | JSON-LD with `rose:sexpr` extension field | Parse JSON-LD, translate to sexp internally | -| Sexp-aware AP (other rose-ash instances) | Native sexp, no translation | Parse sexp directly | -| Sexp-aware with shared components | Sexp with component references | Resolve components from cache, render natively | - -No existing AP implementation breaks. Sexp-unaware servers ignore the `rose:sexpr` field and use `content` HTML as always. Sexp-aware servers get a richer, faster, more structured exchange. - ---- - -## Relationship to Other Plans - -- **sexpr-js-runtime-plan.md** — the client runtime that renders federated sexp content in the browser -- **ghost-removal-plan.md** — posts must be sexp before they can be federated as sexp (Phase 3 dependency) -- **sexpr-ai-integration.md** — AI agents consuming/producing federated content benefit from sexp structure -- **artdag** — content-addressed component definitions use the same CID model as artdag artifacts - ---- - -## Rose-Ash Specific Benefits - -Rose-ash's virtual actor projections (blog, market, events services sharing a keypair) currently emit JSON-LD activities. With sexp AP: - -- A market actor's `Create{Product}` activity includes `(use "product-card" ...)` — followers on other rose-ash instances see a full product card, not just a text description -- A calendar actor's `Create{Event}` includes `(use "calendar-widget" ...)` — followers see an interactive calendar entry with ticket availability -- Cross-instance federation becomes rich UI exchange, not just text+image syndication - -The component registry becomes a shared vocabulary across the cooperative network. Instances that share components render each other's content natively. The fediverse becomes a distributed application platform, not just a social network. diff --git a/docs/sexpr-protocol-and-tiered-clients.md b/docs/sexpr-protocol-and-tiered-clients.md deleted file mode 100644 index 520009a..0000000 --- a/docs/sexpr-protocol-and-tiered-clients.md +++ /dev/null @@ -1,454 +0,0 @@ -# Sexp Protocol and Tiered Client Architecture - -**One server, three clients, progressive enhancement from HTML to native.** - ---- - -## Overview - -The same rose-ash application serves three tiers of client from the same route handlers and component trees: - -``` - ┌─────────────────────────────┐ - │ rose-ash server (Quart) │ - │ │ - │ Same sexp component tree │ - │ Same data, same logic │ - │ │ - ├──────────┬──────────┬────────┤ - │ HTTPS │ HTTPS │ SEXPR │ - │ HTML out │ sexp out │ native │ - └────┬─────┴────┬─────┴───┬────┘ - │ │ │ - ┌──────────▼──┐ ┌─────▼──────┐ ┌▼──────────────┐ - │ Browser │ │ Browser + │ │ Rust client │ - │ (vanilla) │ │ extension │ │ (native) │ - │ │ │ │ │ │ - │ HTML + HTMX │ │ sexpr.js │ │ sexp protocol │ - │ Full CSS │ │ over HTTPS │ │ over QUIC │ - │ ~200ms load │ │ ~80ms load │ │ ~20ms load │ - └─────────────┘ └────────────┘ └────────────────┘ - Tier 0 Tier 1 Tier 2 -``` - ---- - -## Content Negotiation - -The server does format selection on a single endpoint: - -```python -@bp.get("/markets/") -async def markets(): - # Same data, same component tree, always - data = await get_markets(g.s) - tree = sexp('(page :title "Markets" (each markets ...))', markets=data) - - accept = request.headers.get("Accept", "") - - if "application/x-sexpr" in accept: - # Tier 1 (extension) or Tier 2 (native): raw sexp - return Response(serialize(tree), content_type="application/x-sexpr") - - # Tier 0: render to HTML for vanilla browsers - html = render_to_html(tree) - return Response(html, content_type="text/html") -``` - -One route. One component tree. The output format is the only thing that changes. - ---- - -## Tier Comparison - -| | Tier 0: Browser | Tier 1: Extension | Tier 2: Rust Client | -|---|---|---|---| -| URL | `https://rose-ash.com` | `https://rose-ash.com` | `sexpr://rose-ash.com` | -| Protocol | HTTPS | HTTPS | sexpr:// over QUIC | -| Wire format | HTML | sexp over HTTP | sexp native | -| Rendering | Browser DOM | sexpr.js → DOM | Rust → GPU | -| Component cache | Browser cache (URL-keyed) | IndexedDB (hash-keyed) | Disk (hash-keyed, pre-parsed AST) | -| Real-time | HTMX polling / SSE | WebSocket sexp mutations | Native bidirectional stream | -| Bundle size | HTMX 14KB + CSS | sexpr.js ~8KB | 5MB binary, zero runtime deps | -| Page load | ~200ms | ~80ms | ~20ms | -| Memory | ~200MB per tab | ~150MB per tab | ~20MB | -| Offline | Service worker (if built) | Component cache + manifest | Full local cache | -| AI agent | Parse HTML (painful) | Parse sexp (easy) | Native sexp (trivial) | -| Federation | N/A | AP via fetch | Native AP sexp stream | - ---- - -## The Sexp Protocol (Tier 2) - -### Why Not Just HTTP - -HTTP is a request-response protocol with text headers and a body. Strip away the historical baggage and what you actually need is: - -``` -Client sends: method + path + metadata + optional body -Server sends: status + metadata + body -``` - -That's just an s-expression: - -```scheme -;; Request -(GET "/markets/" - :accept "application/x-sexpr" - :auth "bearer tok_abc123" - :if-none-match "sha3-a1b2c3") - -;; Response -(200 - :content-hash "sha3-d4e5f6" - :cache :immutable - :components ("sha3-aaa" "sha3-bbb") - - (page :title "Markets" :layout "main" - (section (h1 "Markets") - (each markets (lambda (m) - (use "vendor-card" :name (get m "name"))))))) -``` - -No header/body split. No chunked transfer encoding. No `Content-Length` because the parser knows when the expression ends (balanced parens). No `Content-Type` because everything is sexp. No MIME, no multipart, no `Transfer-Encoding`, no `Connection: keep-alive` negotiation. - -### What HTTP Gets Wrong That Sexp Fixes - -#### 1. Headers are a bad key-value format - -HTTP headers are case-insensitive, can be duplicated, have weird continuation rules, and are parsed separately from the body. In sexp, metadata is just keywords in the expression — same parser, same format, no special case. - -#### 2. Request/response is too rigid - -HTTP is strictly one request → one response. To get around this, we've bolted on WebSocket (separate protocol, upgrade handshake), Server-Sent Events (hack using chunked encoding), HTTP/2 server push (failed, being removed), and HTTP/3 QUIC streams (complex, still one-request-one-response per stream). - -A sexp protocol is **bidirectional from the start**: - -```scheme -;; Client opens connection, sends request -(GET "/feed/" :stream #t) - -;; Server responds with initial content -(200 :stream #t - (page :title "Feed" - (div :id "feed" (p "Loading...")))) - -;; Server pushes updates as they happen (same connection) -(push! (swap! "#feed" :prepend - (use "post-card" :title "New post" :author "alice"))) - -;; Client sends an action (same connection) -(POST "/like/" :body (:post-id 123)) - -;; Server responds with mutation -(push! (swap! "#like-count-123" :inner "43")) - -;; Client navigates (same connection, no new handshake) -(GET "/markets/") - -;; Server responds -(200 (page :title "Markets" ...)) -``` - -One persistent connection. Requests, responses, and pushes are all sexp expressions on the same stream. No protocol upgrade, no separate WebSocket connection, no polling. - -#### 3. Caching is overcomplicated - -HTTP caching: `Cache-Control`, `ETag`, `If-None-Match`, `If-Modified-Since`, `Vary`, `Age`, `Expires`, `Last-Modified`, `s-maxage`, `stale-while-revalidate`... a dozen headers with complex interaction rules. - -With content-addressed sexp: - -```scheme -;; Request with known hash -(GET "/markets/" :have "sha3-a1b2c3") - -;; If unchanged: -(304) - -;; If changed: -(200 :hash "sha3-d4e5f6" (page ...)) -``` - -One field. The hash *is* the cache key, the ETag, and the content address. - -Components take this further — the client sends what it already has: - -```scheme -(GET "/markets/" - :have-components ("sha3-aaa" "sha3-bbb" "sha3-ccc")) - -;; Server only sends components the client is missing -(200 - :new-components ( - (component "vendor-card" :hash "sha3-ddd" (params ...) (template ...))) - (page ...)) -``` - -#### 4. Status codes are arbitrary numbers - -```scheme -(ok (page ...)) -(redirect :to "/new-location/" :permanent #t) -(not-found :message "Page does not exist") -(error :code "auth-required" :message "Please log in" - :login-url "sexpr://rose-ash.com/login/") -``` - -The status *is* the expression. Machines pattern-match on the head symbol. Humans read it. AI agents understand it without a lookup table. - -#### 5. Forms and file uploads are a mess - -HTTP: `application/x-www-form-urlencoded` or `multipart/form-data` with boundary strings, MIME parts, content-disposition headers. - -Sexp: - -```scheme -(POST "/submit/" - :body ( - :username "alice" - :email "alice@example.com" - :avatar (file :name "photo.jpg" :type "image/jpeg" :data ))) -``` - -Structured data with inline binary. One format. - -#### 6. The protocol is self-describing - -HTTP has no introspection. You need OpenAPI/Swagger specs bolted on separately. - -```scheme -(GET "/__schema/") - -;; Response: the API describes itself -(schema - (endpoint "/" :method GET - :returns (page) - :params (:stream bool)) - (endpoint "/markets/" :method GET - :returns (page :contains (list (use "vendor-card")))) - (endpoint "/like/" :method POST - :params (:post-id int) - :returns (mutation))) -``` - -An AI agent hitting `/__schema/` learns the entire API surface as parseable sexp. No separate OpenAPI doc. The schema *is* the API. - -### Protocol Stack - -``` -┌─────────────────────────────────────────┐ -│ Application: sexp documents, mutations │ -│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ -│ Session: bidirectional sexp stream │ -│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ -│ Transport: QUIC (or TCP+TLS) │ -│ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │ -│ Network: IP │ -└─────────────────────────────────────────┘ -``` - -QUIC is ideal — multiplexed streams, built-in TLS, connection migration. Each sexp expression gets its own QUIC stream. Requests and pushes multiplex without head-of-line blocking. Connection survives network changes (mobile → wifi). - ---- - -## Rust Client Architecture - -``` -┌─────────────────────────────────────┐ -│ sexpr-client (Rust) │ -│ │ -│ ┌──────────┐ ┌─────────────────┐ │ -│ │ Parser │ │ Component │ │ -│ │ (zero- │ │ Cache │ │ -│ │ copy) │ │ (SHA3 → AST) │ │ -│ └──────────┘ └─────────────────┘ │ -│ ┌──────────┐ ┌─────────────────┐ │ -│ │ Layout │ │ Network │ │ -│ │ Engine │ │ (tokio + QUIC) │ │ -│ └──────────┘ └─────────────────┘ │ -│ ┌──────────────────────────────┐ │ -│ │ Renderer (wgpu / iced) │ │ -│ └──────────────────────────────┘ │ -└─────────────────────────────────────┘ -``` - -**Why Rust makes this fast:** - -- **Parser**: sexp parsing is trivial recursive descent. Nanoseconds per node. 100-1000x faster than JS. -- **Renderer**: Skip the DOM entirely. Render directly to GPU-backed surface via `wgpu` or `iced`. -- **Networking**: `reqwest` + `tokio` with connection pooling. Component manifests fetched in parallel. -- **Component cache**: Pre-parsed ASTs on disk, content-addressed. No parse step on cache hit. -- **Memory**: No GC pauses. 10-50MB where a browser tab uses 200-500MB. - -**What you skip by not being a browser:** -- No HTML parser (5-10ms per page) -- No CSS cascade resolution (the most expensive part of browser rendering) -- No DOM construction (sexp AST *is* the tree) -- No JavaScript engine (logic in the sexp evaluator, compiled Rust) -- No security sandbox overhead (no arbitrary JS execution) -- No 2000+ web platform APIs you don't use - -Page load: network (~50ms) → parse (~0.1ms) → layout (~2ms) → paint (~3ms) = **under 60ms**. - ---- - -## Fallback Gateway - -For users without the extension or native client, a gateway translates: - -``` -sexpr://rose-ash.com/markets/ - → Gateway fetches sexp from server - → Renders to HTML - → Serves to browser at https://gateway.rose-ash.com/markets/ -``` - -The regular web always works. The sexp protocol is an acceleration layer, not a requirement. - ---- - -## Implementation Plan - -### Phase 1: Content Negotiation (Quart) - -Add `Accept` header handling to existing Quart routes. - -- Check for `application/x-sexpr` in `Accept` header -- If present: serialize the sexp tree and return directly (skip HTML render) -- If absent: render to HTML as today -- Add `Vary: Accept` header to responses - -**Files:** -- `shared/infrastructure/factory.py` — middleware for content negotiation -- `shared/sexp/serialize.py` — canonical sexp serializer (deterministic output for caching/signing) - -**Result:** Existing routes serve sexp to any client that asks. Zero changes to route logic. - -### Phase 2: sexpr.js Client Library - -Build the browser-side JS runtime (see `sexpr-js-runtime-plan.md` for full details). - -- Parser + renderer: `parse()` → AST → `renderToDOM()` -- Mutation engine: `swap!`, `batch!`, `class!`, `request!` -- Component registry with localStorage cache (content-addressed) -- `