# The Sexp Protocol

**A unified protocol for documents, applications, and federation.**
**One format. Every boundary.**

---

## Core Insight

The sexp protocol replaces HTTP, HTML, JSON-LD, WebSocket, and ActivityPub's inbox model with a single concept: **peers exchange s-expressions on a bidirectional stream.**

There is no distinction between:
- A client requesting a page (what HTTP does)
- A server pushing a real-time update (what WebSocket does)
- A peer delivering a federated activity (what AP inbox POST does)
- A server sending a DOM mutation (what HTMX does)

They are all sexp expressions sent between two peers:

```scheme
;; Browsing (request → response)
(GET "/markets/")
(ok (page :title "Markets" ...))

;; Real-time push (server → client, unsolicited)
(push! (swap! "#feed" :prepend (use "post-card" :title "New post")))

;; Federation delivery (peer → peer)
(Create :actor "alice@rose-ash.com"
  (Note :id post-123 (p "Hello from the fediverse!")))

;; Client action (client → server)
(POST "/like/" :body (:post-id 123))

;; Mutation response (server → client)
(push! (swap! "#like-count-123" :inner "43"))
```

Same parser. Same stream. Same connection. The "type" of interaction is determined by the head symbol of the expression, not by the protocol layer.

---

## Why Replace HTTP and JSON-LD

### HTTP's Problems

HTTP is a request-response protocol with text headers and a body. Strip away the historical baggage and what you actually need is:

```
Peer A sends: verb + target + metadata + optional body
Peer B sends: status + metadata + body
```

That's just an s-expression. But HTTP adds:
- **Header/body split** — two parsing phases, different formats
- **Rigid request-response** — to work around this, we bolted on WebSocket (separate protocol, upgrade handshake), SSE (chunked encoding hack), HTTP/2 push (failed, removed), HTTP/3 QUIC streams (complex, still one-request-one-response per stream)
- **Overcomplicated caching** — `Cache-Control`, `ETag`, `If-None-Match`, `If-Modified-Since`, `Vary`, `Age`, `Expires`, `Last-Modified`, `s-maxage`, `stale-while-revalidate` — a dozen headers with complex interaction rules
- **Arbitrary status codes** — memorised numbers with no semantic meaning in the format
- **Form encoding mess** — `application/x-www-form-urlencoded` or `multipart/form-data` with boundary strings, MIME parts, content-disposition headers

### JSON-LD's Problems

JSON-LD was chosen for ActivityPub because of its semantic web lineage, but in practice nobody uses the linked data features — servers just pattern-match on `type` fields and ignore the `@context`. The format is:

- **Verbose** — deeply nested objects with string keys, quoted values, commas, colons
- **Ambiguous** — compaction/expansion rules mean the same activity can have many valid JSON representations
- **Lossy** — post content is flattened to an HTML string inside a JSON string; structure destroyed
- **Hostile to signing** — JSON-LD canonicalization is fragile and implementation-dependent
- **Hostile to AI** — agents must parse JSON, then parse embedded HTML, then reason about structure

### The Sexp Solution

One format that is:
- **Compact** — half the size of equivalent JSON-LD, no closing tags like HTML
- **Unambiguous** — one canonical serialization, deterministic for signing
- **Structural** — content *is* the tree, not a string embedded in a string
- **Parseable** — trivial recursive descent, no backtracking, nanoseconds per node
- **Bidirectional** — requests, responses, pushes, and activities all use the same syntax
- **AI-native** — agents parse, generate, and reason about sexp as naturally as tool calls

---

## Protocol Specification

### Transport

```
┌─────────────────────────────────────────┐
│  Application: sexp expressions          │
│  ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│  Session: bidirectional sexp stream     │
│  ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│  Transport: QUIC (or TCP+TLS fallback)  │
│  ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ │
│  Network: IP                            │
└─────────────────────────────────────────┘
```

QUIC is ideal — multiplexed streams, built-in TLS, connection migration. Each sexp expression gets its own QUIC stream. Requests and pushes multiplex without head-of-line blocking. Connection survives network changes (mobile → wifi).

**Framing:** Length-prefixed sexp expressions over QUIC streams. The parser knows when an expression ends (balanced parens), but length-prefixing enables efficient buffer allocation.

**URL scheme:** `sexpr://host:port/path`

### Expression Types

Every expression on the wire is one of these forms:

#### Requests (client → server)

```scheme
(GET "/markets/"
  :auth "bearer tok_abc123"
  :have "sha3-a1b2c3"
  :have-components ("sha3-aaa" "sha3-bbb"))

(POST "/like/"
  :auth "bearer tok_abc123"
  :body (:post-id 123))

(POST "/submit/"
  :body (
    :username "alice"
    :email "alice@example.com"
    :avatar (file :name "photo.jpg" :type "image/jpeg" :data <binary>)))

;; Streaming request — keep connection open for pushes
(GET "/feed/" :stream #t)
```

#### Responses (server → client)

```scheme
;; Success with content
(ok :hash "sha3-d4e5f6"
  (page :title "Markets" :layout "main"
    (section (h1 "Markets")
      (each markets (lambda (m)
        (use "vendor-card" :name (get m "name")))))))

;; Not modified (client already has current version)
(not-modified)

;; Redirect
(redirect :to "/new-location/" :permanent #t)

;; Not found
(not-found :message "Page does not exist")

;; Error
(error :code "auth-required" :message "Please log in"
  :login-url "sexpr://rose-ash.com/login/")

;; Response with new components the client is missing
(ok
  :new-components (
    (component "vendor-card" :hash "sha3-ddd" (params ...) (template ...)))
  (page ...))
```

#### Pushes (server → client, unsolicited)

```scheme
;; DOM mutation
(push! (swap! "#feed" :prepend
  (use "post-card" :title "New post" :author "alice")))

;; Batch mutation
(push! (batch!
  (swap! "#notifications" :append (div :class "toast" "Saved!"))
  (class! "#save-btn" :remove "loading")))

;; Component update (new version available)
(push! (component-update "vendor-card" :hash "sha3-eee"))
```

#### Activities (peer → peer, federation)

```scheme
;; Create
(Create
  :id "https://rose-ash.com/ap/activities/456"
  :actor "https://rose-ash.com/ap/users/alice"
  :published "2026-02-28T14:00:00Z"
  :to (:public)
  :cc ("https://rose-ash.com/ap/users/alice/followers")

  (Note
    :id "https://rose-ash.com/ap/posts/123"
    :attributed-to "https://rose-ash.com/ap/users/alice"
    :in-reply-to "https://remote.social/posts/789"
    :summary "Weekend market update"

    (section
      (p "Three new vendors joining this Saturday.")
      (use "vendor-list" :vendors vendors)
      (use "calendar-widget" :calendar-id 42))

    (attachment
      (Image :url "https://rose-ash.com/media/market.jpg"
        :media-type "image/jpeg"
        :width 1200 :height 800
        :name "Saturday market stalls"))

    (tag
      (Hashtag :name "#markets" :href "https://rose-ash.com/tags/markets")
      (Mention :name "@bob@remote.social" :href "https://remote.social/users/bob"))))

;; Follow
(Follow :actor "https://rose-ash.com/ap/users/alice"
  :object "https://remote.social/users/bob")

;; Accept (response to Follow — on the same connection)
(Accept :actor "https://remote.social/users/bob"
  (Follow :actor "https://rose-ash.com/ap/users/alice"
    :object "https://remote.social/users/bob"))

;; Like, Announce, Undo — all just expressions
(Like :actor alice :object post-123)
(Announce :actor bob (Note :id post-123))
(Undo :actor alice (Like :actor alice :object post-123))
```

**Key insight:** Activities are not "posted to an inbox" — they are expressions sent on the bidirectional stream. The peer-to-peer connection *is* the inbox. When you follow someone, their activities stream to you on the same connection you used to follow them. No inbox endpoint, no HTTP POST delivery, no polling.

#### Collections (queryable, paginated)

```scheme
;; Request an actor's outbox
(GET "/ap/users/alice/outbox" :page 1)

;; Response
(ok
  (OrderedCollectionPage
    :part-of "https://rose-ash.com/ap/users/alice/outbox"
    :next "/ap/users/alice/outbox?page=2"
    :total-items 142

    (Create :actor alice :published "2026-02-28T14:00:00Z"
      (Note :id post-3 (p "Latest post")))
    (Announce :actor alice :published "2026-02-27T09:00:00Z"
      (Note :id post-2 :attributed-to bob))
    (Create :actor alice :published "2026-02-26T18:00:00Z"
      (Note :id post-1 (p "Earlier post")))))
```

#### Actor Profiles

```scheme
(Person
  :id "https://rose-ash.com/ap/users/alice"
  :preferred-username "alice"
  :name "Alice"
  :summary (p "Co-op member, market organiser")
  :inbox "sexpr://rose-ash.com/ap/users/alice/inbox"
  :outbox "sexpr://rose-ash.com/ap/users/alice/outbox"
  :followers "sexpr://rose-ash.com/ap/users/alice/followers"
  :following "sexpr://rose-ash.com/ap/users/alice/following"
  :components "sexpr://rose-ash.com/ap/components/"
  :public-key (:id "https://rose-ash.com/ap/users/alice#main-key"
               :owner "https://rose-ash.com/ap/users/alice"
               :pem "-----BEGIN PUBLIC KEY-----\n..."))
```

#### Schema Introspection

```scheme
(GET "/__schema/")

(ok
  (schema
    (endpoint "/" :method GET
      :returns (page)
      :params (:stream bool))
    (endpoint "/markets/" :method GET
      :returns (page :contains (list (use "vendor-card"))))
    (endpoint "/like/" :method POST
      :params (:post-id int)
      :returns (mutation))
    (activity Create
      :object (Note)
      :delivers-to :followers)
    (activity Follow
      :object (Person)
      :expects (Accept))))
```

An AI agent hitting `/__schema/` learns the entire surface — pages, actions, federation activities — as parseable sexp. No separate OpenAPI doc. The schema *is* the API.

### Caching

Content-addressed, hash-based. No expiry headers, no revalidation dance.

```scheme
;; Client has a cached version
(GET "/markets/" :have "sha3-a1b2c3")

;; Unchanged
(not-modified)

;; Changed — new hash included
(ok :hash "sha3-d4e5f6" (page ...))
```

Component-level caching:

```scheme
;; Client reports which components it has
(GET "/markets/" :have-components ("sha3-aaa" "sha3-bbb"))

;; Server sends only missing components alongside the page
(ok
  :new-components (
    (component "vendor-card" :hash "sha3-ddd" (params ...) (template ...)))
  (page ...))
```

The hash *is* the cache key, the ETag, and the content address. One concept replaces twelve HTTP headers.

### Component Discovery

Actors advertise a `:components` endpoint in their profile. Peers fetch and cache component definitions by content hash:

```scheme
;; Fetch component manifest
(GET "/ap/components/")

(ok
  (component-manifest
    ("cart-mini" :hash "sha3-aaa")
    ("vendor-card" :hash "sha3-bbb")
    ("calendar-widget" :hash "sha3-ccc")
    ("post-card" :hash "sha3-ddd")))

;; Fetch a specific component by hash
(GET "/ap/components/sha3-bbb")

(ok
  (component "vendor-card" :hash "sha3-bbb"
    (params name stall image)
    (div :class "vendor-card"
      (img :src image :alt name)
      (h3 name)
      (p :class "stall" stall))))
```

When a federated activity includes `(use "vendor-card" :name "Sourdough" :stall "A12")`, the receiving peer resolves the component from its cache (by hash) or fetches it from the sender's manifest. Federated UI, not just federated data.

### Signatures

Sexp serialization has a canonical form:
- Keywords sorted alphabetically
- Single space between atoms
- No trailing whitespace
- UTF-8 encoding

This makes signatures deterministic without JSON-LD canonicalization:

```scheme
(signed :sig "base64..." :key-id "https://rose-ash.com/ap/users/alice#main-key"
  (Create :actor "https://rose-ash.com/ap/users/alice"
    (Note :id post-123 (p "Hello"))))
```

Sign the canonical serialization of the inner expression. Verify by re-serializing and checking. No compaction/expansion ambiguity.

---

## Bidirectional Stream: The Unified Model

A connection between two sexp-speaking peers carries everything on one stream:

```scheme
;; === Connection opened ===

;; Client browses
(GET "/")
(ok (page :title "Home" ...))

;; Client navigates (same connection)
(GET "/markets/")
(ok (page :title "Markets" ...))

;; Client follows a user (federation)
(Follow :actor alice :object bob)

;; Server confirms
(Accept :actor bob (Follow :actor alice :object bob))

;; Bob posts something — server pushes activity
(Create :actor bob :published "2026-02-28T15:00:00Z"
  (Note :id post-456 (p "New vendor announcement!")))

;; Client sees it rendered in real-time via mutation
(push! (swap! "#feed" :prepend
  (use "post-card" :actor "bob" :content (p "New vendor announcement!"))))

;; Client likes the post
(POST "/like/" :body (:post-id 456))

;; Server pushes mutation + delivers Like activity to bob
(push! (swap! "#like-count-456" :inner "12"))

;; Client opens a streaming feed
(GET "/feed/" :stream #t)
(ok :stream #t (page :title "Feed" ...))

;; More activities arrive over time...
(Create :actor charlie :published "2026-02-28T15:30:00Z"
  (Note :id post-789 (p "Market day tomorrow!")))
(push! (swap! "#feed" :prepend
  (use "post-card" :actor "charlie" :content (p "Market day tomorrow!"))))

;; === Connection persists ===
```

Browsing, federation, real-time updates, and user actions — all on one bidirectional stream. The distinction between "web server", "AP inbox", and "WebSocket" disappears. They were always the same thing — peers exchanging structured expressions.

---

## Backwards Compatibility

### With HTTP (Tier 0 and Tier 1)

The sexp protocol runs alongside HTTPS. The same server handles both:

```python
@bp.get("/markets/")
async def markets():
    data = await get_markets(g.s)
    tree = sexp('(page :title "Markets" ...)', markets=data)

    accept = request.headers.get("Accept", "")
    if "application/x-sexpr" in accept:
        return Response(serialize(tree), content_type="application/x-sexpr")

    html = render_to_html(tree)
    return Response(html, content_type="text/html")
```

### With ActivityPub (JSON-LD peers)

For Mastodon, Pleroma, and other JSON-LD AP servers:

| Peer Type | Outbound | Inbound |
|---|---|---|
| Standard AP (Mastodon, Pleroma) | Translate sexp → JSON-LD, include `rose:sexpr` extension field | Parse JSON-LD, translate to sexp internally |
| Sexp-aware AP (other rose-ash instances) | Native sexp on bidirectional stream | Parse sexp directly |
| Sexp-aware with shared components | Sexp with component references | Resolve from cache, render natively |

JSON-LD bridging for non-sexp peers:

```json
{
  "@context": ["https://www.w3.org/ns/activitystreams", "https://rose-ash.com/ns/sexpr"],
  "type": "Create",
  "object": {
    "type": "Note",
    "content": "<p>Hello world</p>",
    "rose:sexpr": "(p \"Hello world\")"
  }
}
```

No existing AP implementation breaks. The `rose:sexpr` field is ignored by servers that don't understand it.

### Fallback Gateway

For browsers without extension or native client:

```
sexpr://rose-ash.com/markets/
  → Gateway fetches sexp from server
  → Renders to HTML
  → Serves to browser at https://rose-ash.com/markets/
```

---

## Three Client Tiers

```
                    ┌─────────────────────────────┐
                    │     rose-ash server (Quart)  │
                    │                              │
                    │  Same sexp component tree    │
                    │  Same data, same logic       │
                    │                              │
                    ├──────────┬──────────┬────────┤
                    │ HTTPS    │ HTTPS    │ SEXPR  │
                    │ HTML out │ sexp out │ native │
                    └────┬─────┴────┬─────┴───┬────┘
                         │          │         │
              ┌──────────▼──┐ ┌─────▼──────┐ ┌▼──────────────┐
              │   Browser   │ │  Browser + │ │  Rust client   │
              │   (vanilla) │ │  extension │ │  (native)      │
              │             │ │            │ │                │
              │ HTML + HTMX │ │ sexpr.js   │ │ sexp protocol  │
              │ Full CSS    │ │ over HTTPS │ │ over QUIC      │
              │ ~200ms load │ │ ~80ms load │ │ ~20ms load     │
              └─────────────┘ └────────────┘ └────────────────┘
                 Tier 0           Tier 1          Tier 2
```

| | Tier 0: Browser | Tier 1: Extension | Tier 2: Rust Client |
|---|---|---|---|
| URL | `https://rose-ash.com` | `https://rose-ash.com` | `sexpr://rose-ash.com` |
| Protocol | HTTPS | HTTPS | sexpr:// over QUIC |
| Wire format | HTML | sexp over HTTP | sexp native stream |
| Rendering | Browser DOM | sexpr.js → DOM | Rust → GPU |
| Component cache | Browser cache (URL-keyed) | IndexedDB (hash-keyed) | Disk (hash-keyed, pre-parsed AST) |
| Real-time | HTMX polling / SSE | WebSocket sexp mutations | Native bidirectional stream |
| Federation | N/A | AP via fetch | Native sexp stream |
| Bundle size | HTMX 14KB + CSS | sexpr.js ~8KB | 5MB binary, zero runtime deps |
| Page load | ~200ms | ~80ms | ~20ms |
| Memory per page | ~200MB | ~150MB | ~20MB |
| AI integration | Parse HTML (painful) | Parse sexp (easy) | Native sexp (trivial) |

### Rust Client Architecture

```
┌─────────────────────────────────────┐
│        sexpr-client (Rust)          │
│                                     │
│  ┌──────────┐  ┌─────────────────┐  │
│  │  Parser   │  │  Component      │  │
│  │  (zero-   │  │  Cache          │  │
│  │   copy)   │  │  (SHA3 → AST)  │  │
│  └──────────┘  └─────────────────┘  │
│  ┌──────────┐  ┌─────────────────┐  │
│  │  Layout   │  │  Network        │  │
│  │  Engine   │  │  (tokio + QUIC) │  │
│  └──────────┘  └─────────────────┘  │
│  ┌──────────────────────────────┐   │
│  │  Renderer (wgpu / iced)      │   │
│  └──────────────────────────────┘   │
└─────────────────────────────────────┘
```

**Why it's fast:**
- Parser: nanoseconds per node (trivial recursive descent in Rust)
- No HTML parser, no CSS cascade, no DOM construction, no JS engine
- Render directly to GPU surface — skip the entire browser rendering pipeline
- Pre-parsed ASTs from disk cache — zero parse time on cache hit
- 10-50MB memory vs 200-500MB per browser tab

Page load: network (~50ms) → parse (~0.1ms) → layout (~2ms) → paint (~3ms) = **under 60ms**.

---

## Implementation Plan

### Phase 1: Content Negotiation (Quart)

Add `Accept` header handling to existing Quart routes.

- Check for `application/x-sexpr` in `Accept` header
- If present: serialize sexp tree, return directly
- If absent: render to HTML as today
- Add `Vary: Accept` header

**Files:**
- `shared/infrastructure/factory.py` — content negotiation middleware
- `shared/sexp/serialize.py` — canonical sexp serializer (deterministic for signing/caching)

**Result:** Existing routes serve sexp to any client that asks. Zero changes to route logic.

### Phase 2: Sexp ↔ JSON-LD Bridge

Bidirectional translation for AP federation with non-sexp peers:

**sexp → JSON-LD** (outbound):
- Activity type symbol → `"type"` field
- Keyword attributes → JSON object properties
- Content sexp → rendered to HTML for `"content"` field
- Include `rose:sexpr` extension field

**JSON-LD → sexp** (inbound):
- `"type"` → head symbol
- `"content"` HTML → parsed to sexp (best-effort)
- Nested objects → child expressions

**Files:**
- `shared/infrastructure/ap_sexpr.py` — serializer/deserializer
- `shared/sexp/html_to_sexp.py` — HTML → sexp parser for inbound content
- `shared/sexp/tests/test_ap_sexpr.py` — round-trip tests

### Phase 3: sexpr.js Client Library

Browser-side JS runtime (see `sexpr-js-runtime-plan.md`):

- Parser + renderer: `parse()` → AST → `renderToDOM()`
- Mutation engine: `swap!`, `batch!`, `class!`, `request!`
- Component registry with localStorage cache (content-addressed)
- WebSocket connection for real-time pushes

### Phase 4: Browser Extension

Package sexpr.js as a WebExtension:

- Register `sexpr://` protocol handler
- Intercept navigation → fetch via HTTPS with `Accept: application/x-sexpr`
- Parse → render to DOM
- Component cache in IndexedDB (content-addressed)

**Tech:** WebExtension API (Firefox + Chrome), JS/TS.

### Phase 5: Component Discovery

Enable peers to discover and cache shared components:

- Actor profiles include `:components` endpoint URL
- `GET /ap/components/` → manifest (name → hash)
- `GET /ap/components/{hash}` → component definition (sexp)
- Content-addressed cache on client (hash-keyed)
- Federated content with `(use "component" ...)` resolves from cache

**Files:**
- `federation/bp/components/routes.py` — manifest + fetch endpoints
- `shared/sexp/component_manifest.py` — hash generation

### Phase 6: Protocol Specification

Formal specification document:

- **Framing**: length-prefixed sexp over QUIC streams
- **Expression types**: requests, responses, pushes, activities, collections
- **Canonical serialization**: deterministic rules for signing
- **Caching**: content-hash based
- **Authentication**: token in request keywords
- **Component discovery**: manifest protocol
- **Bidirectional streaming**: lifecycle, multiplexing
- **Schema introspection**: `/__schema/` endpoint
- **JSON-LD bridging**: rules for non-sexp AP peers
- **Security**: no eval, escaping, signature verification

Publish as a FEP (Fediverse Enhancement Proposal) and standalone specification.

### Phase 7: Rust Protocol Server

QUIC server alongside Hypercorn:

- Listen on separate port (e.g., 4433)
- Parse sexp requests, route to same handler logic
- Bidirectional stream: pushes, requests, responses, activities
- Component manifest endpoint
- Federation delivery on persistent connections (replaces HTTP POST to inbox)

**Tech:** Rust, `quinn` (QUIC), `tokio`, `rustls`.

**Files:**
- `sexpr-server/src/protocol.rs` — framing, parsing, routing
- `sexpr-server/src/quic.rs` — QUIC listener + stream management
- `sexpr-server/src/federation.rs` — peer connection manager

### Phase 8: Rust Native Client

Standalone sexp document viewer:

- QUIC client for `sexpr://` URLs
- Zero-copy sexp parser (arena-allocated AST)
- Component cache on disk (SHA3-keyed)
- Layout engine (flexbox subset)
- GPU renderer via `wgpu` or `iced`
- Text rendering via `cosmic-text`
- Bidirectional stream: browsing + federation + real-time on one connection

**Tech:** Rust, `quinn`, `wgpu`/`iced`, `cosmic-text`, `tokio`.

**Files:**
- `sexpr-client/src/parser.rs` — zero-copy parser
- `sexpr-client/src/cache.rs` — content-addressed component cache
- `sexpr-client/src/layout.rs` — layout engine
- `sexpr-client/src/render.rs` — GPU renderer
- `sexpr-client/src/stream.rs` — bidirectional connection manager

### Phase 9: Fallback Gateway

HTTP proxy for browsers without extension/client:

- Accept HTTPS requests at `https://rose-ash.com`
- Fetch sexp from server internally
- Render to HTML, serve to browser
- Add `<link rel="alternate" type="application/x-sexpr" href="sexpr://...">` for discovery

---

## Migration Path

Each phase builds on the last. No phase breaks existing functionality:

```
Phase 1 (content negotiation) ── Tier 0 unchanged, sexp available
Phase 2 (JSON-LD bridge)      ── federation works with all AP peers
Phase 3 (sexpr.js)            ── Tier 1 client-side rendering
Phase 4 (extension)           ── sexpr:// URLs in browser
Phase 5 (components)          ── federated UI exchange
Phase 6 (spec)                ── formal protocol document
Phase 7 (Rust server)         ── native protocol alongside HTTPS
Phase 8 (Rust client)         ── Tier 2 native experience
Phase 9 (gateway)             ── sexpr:// accessible from any browser
```

Depends on:
- **Ghost removal** (see `ghost-removal-plan.md`) — posts must be sexp before Phases 2-3 add real value
- **sexpr.js runtime** (see `sexpr-js-runtime-plan.md`) — the JS library that powers Phases 3-4

---

## Client as Node: Cooperative Compute Mesh

### Everyone Has a Server

The original web was peer-to-peer — everyone ran a server on their workstation. Then we centralised everything into data centres because HTTP was stateless and browsers were passive. The sexp protocol with client-as-node reverses that.

Each member's Rust client is not just a viewer — it's a full peer node:
- An **ActivityPub instance** (keypair, identity, inbox/outbox)
- An **IPFS node** (storing and serving content-addressed data)
- An **artdag worker** (local GPU for media processing)
- A **sexp peer** (bidirectional streams to relay and other peers)

```
┌─────────────┐  ┌─────────────┐  ┌─────────────┐
│ Alice        │  │ Bob          │  │ Charlie      │
│ RTX 4070     │  │ M2 MacBook   │  │ RX 7900      │
│ 12GB VRAM    │  │ 16GB unified │  │ 20GB VRAM    │
│              │  │              │  │              │
│ artdag node  │  │ artdag node  │  │ artdag node  │
│ IPFS node    │  │ IPFS node    │  │ IPFS node    │
│ sexp peer    │  │ sexp peer    │  │ sexp peer    │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                 │
       └────────┬────────┴────────┬────────┘
                │                 │
       ┌────────▼─────────────────▼────────┐
       │  rose-ash relay                   │
       │                                   │
       │  • Message queue (offline inbox)  │
       │  • Capability registry            │
       │  • IPFS pinning service           │
       │  • HTTPS gateway (Tier 0)         │
       │  • Peer directory                 │
       │  • Federation bridge (JSON-LD)    │
       └───────────────────────────────────┘
```

### Offline Persistence

When a member's client goes offline, their content persists on IPFS. The relay provides two services:

**IPFS pinning** — members' CIDs are pinned by the cooperative's pinning node, ensuring content stays available even when the author's client is off. This is cheap — just disk storage, no compute.

**Message queuing** — activities addressed to an offline member are held by the relay and drained when they reconnect:

```scheme
;; Alice is offline. Bob sends her a message.
;; The relay holds it.

;; Alice's client comes online, connects to relay
(hello :actor "alice@rose-ash.com" :last-seen "2026-02-28T12:00:00Z")

;; Relay drains the queue
(queued :count 3 :since "2026-02-28T12:00:00Z"
  (Create :actor bob :published "2026-02-28T16:00:00Z"
    (Note (p "See you at the market Saturday!")))
  (Like :actor charlie :object alice-post-42)
  (Follow :actor dave :object alice))

;; Alice's client processes them, sends acknowledgment
(ack :through "2026-02-28T16:00:00Z")

;; Relay clears the queue. Now alice is live —
;; subsequent activities stream directly peer-to-peer.
```

### Cooperative GPU Sharing

Members contribute idle GPU cycles to the cooperative. The relay acts as a job matchmaker:

```scheme
;; Alice uploads a video. Her laptop has integrated graphics — too slow.
(submit-job
  :type "artdag/render"
  :recipe "bafyrecipe..."
  :input "bafyinput..."
  :requirements (:min-vram 8 :gpu #t))

;; Relay knows Charlie's RTX 7900 is online and idle.
;; Job routes to Charlie's client.
(job :id "job-789" :assigned-to charlie
  :type "artdag/render"
  :recipe "bafyrecipe..."
  :input "bafyinput...")

;; Charlie's client runs the job, pins result to IPFS
(job-complete :id "job-789"
  :output "bafyoutput..."
  :duration-ms 4200
  :worker charlie)

;; Alice gets notified
(push! (swap! "#render-status" :inner
  (use "render-complete" :cid "bafyoutput...")))
```

This is already how artdag works conceptually. The L1 server is a Celery worker that picks up rendering tasks. Replace "Celery worker on a cloud server" with "Celery worker on a member's desktop" and the architecture barely changes. The task queue just has different workers.

### Economics

| | Centralised (current) | Cooperative mesh |
|---|---|---|
| Image/video processing | Cloud GPU ($2-5/hr) | Member's local GPU (free) |
| Content storage | Server disk + S3 | IPFS (distributed) + pinning |
| Content serving | Server bandwidth | Peer-to-peer + IPFS |
| Server cost | GPU instances + storage + bandwidth | Cheap relay (CPU + disk only) |
| Scaling | More users = more cost | More members = more capacity |

The co-op's infrastructure cost drops to: **one small VPS + IPFS pinning storage.** That's it. All compute — rendering, processing, serving content — is distributed across members' machines.

More members joining makes the network faster and more capable, not more expensive. Like BitTorrent seeding, but for an entire application platform.

### The Relay Server's Role

The relay is minimal — a matchmaker and persistence layer, not a compute provider:

- **Peer directory**: who's online, their QUIC address, their GPU capabilities
- **Message queue**: hold activities for offline members
- **IPFS pinning**: persist content when authors are offline
- **HTTPS gateway**: serve HTML to Tier 0 browsers (visitors, search engines)
- **Federation bridge**: translate sexp ↔ JSON-LD for Mastodon/Pleroma peers
- **Job queue**: match GPU-intensive tasks to available peers
- **Capability registry**: what each peer can do (GPU model, VRAM, storage)

The relay does no rendering, no media processing, no content generation. Its cost stays flat regardless of member count.

### Content Flow

```
Author creates post:
  1. Edit in Rust client (local)
  2. Render media with local GPU (artdag)
  3. Pin content + media to IPFS (local node)
  4. Publish CIDs to relay (for pinning + discovery)
  5. Stream activity to connected followers (peer-to-peer)
  6. Relay queues activity for offline followers

Reader views post:
  1. Fetch sexp from author's client (if online, peer-to-peer)
  2. Or fetch from IPFS by CID (if author offline)
  3. Or fetch from relay gateway as HTML (if Tier 0 browser)
  4. Components resolved from local cache (content-addressed)
  5. Render locally (Rust GPU or sexpr.js in browser)
```

No server rendered anything. No server stored anything permanently. No server paid for GPU time. The cooperative's members are the infrastructure.

---

## Cooperative Angle

- Members install the Rust client → fast native experience, 5MB binary, no app store
- Visitors browse `https://rose-ash.com` → standard HTML, no barrier
- Federated co-ops connect via persistent sexp streams → rich UI exchange, not just text syndication
- AI agents speak the protocol natively → components as tool calls, mutations as actions
- Auto-updates via content-addressed components → no gatekeeping
- The component registry is a shared vocabulary across the cooperative network
- Members' desktops are the cloud — contributing GPU, storage, and bandwidth
- The relay server stays cheap and flat-cost regardless of growth
- The original vision of the web: everyone has a server

---

## Relationship to Other Plans

| Document | Role |
|---|---|
| `sexpr-js-runtime-plan.md` | The JS library powering Tier 1 (Phases 3-4) |
| `ghost-removal-plan.md` | Posts must be sexp before federation/client rendering adds value |
| `sexpr-ai-integration.md` | AI agents benefit from all tiers and the self-describing schema |
| artdag (`artdag/`) | The media processing engine that runs on member GPUs |

---

*The document is the program. The program is the document. The protocol is both. The network is its members.*