sx-web plan: WebSocket + WebRTC peer-to-peer architecture
Replace WebTransport-only design with three-transport model: - Browser↔server: WebSocket (works today, no infrastructure changes) - Browser↔browser: WebRTC data channels (true P2P, NAT traversal) - Server↔server: HTTP federation (existing sx-pub) Home nodes relay WebRTC signaling. Reliable channels for chat/components, unreliable channels for game state/cursor positions. Transport-agnostic protocol layer — upgrade to WebTransport later with zero SX changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -26,7 +26,7 @@
|
||||
(tr (td :class "p-2 font-medium" "Runtime") (td :class "p-2" "OCaml + Python") (td :class "p-2" "JavaScript (sx-browser.js)"))
|
||||
(tr (td :class "p-2 font-medium" "Storage") (td :class "p-2" "IPFS daemon + PostgreSQL") (td :class "p-2" "IndexedDB + Cache API"))
|
||||
(tr (td :class "p-2 font-medium" "IPFS") (td :class "p-2" "Full DHT peer, pins content") (td :class "p-2" "Verifies CIDs, caches locally"))
|
||||
(tr (td :class "p-2 font-medium" "Transport") (td :class "p-2" "HTTP/3 + WebTransport server") (td :class "p-2" "WebTransport client"))
|
||||
(tr (td :class "p-2 font-medium" "Transport") (td :class "p-2" "WebSocket + HTTP federation") (td :class "p-2" "WebSocket to home + WebRTC to peers"))
|
||||
(tr (td :class "p-2 font-medium" "Availability") (td :class "p-2" "Long-lived, always on") (td :class "p-2" "Ephemeral, online when open"))
|
||||
(tr (td :class "p-2 font-medium" "Trust") (td :class "p-2" "CID verification + blockchain anchor") (td :class "p-2" "CID verification (same guarantee)"))))
|
||||
|
||||
@@ -91,10 +91,12 @@
|
||||
;; Phase 2: WebTransport Peer Layer
|
||||
;; =====================================================================
|
||||
|
||||
(~docs/section :title "Phase 2: WebTransport Peer Layer" :id "phase-2"
|
||||
(~docs/section :title "Phase 2: Peer Network" :id "phase-2"
|
||||
|
||||
(p :class "text-stone-600 italic mb-4"
|
||||
"Bidirectional streams between browsers and servers. The protocol is SX.")
|
||||
"Three transports, one SX protocol. Browser↔server via WebSocket, "
|
||||
"browser↔browser via WebRTC data channels, server↔server via federation. "
|
||||
"The protocol layer doesn't know or care which pipe carries it.")
|
||||
|
||||
(~docs/subsection :title "2a. SX Protocol Definition"
|
||||
(p "The protocol is an SX component. Nodes fetch, evaluate, and speak it.")
|
||||
@@ -106,31 +108,48 @@
|
||||
(p "The protocol definition is itself content-addressed. Nodes pin the protocol "
|
||||
"version they speak. Protocol upgrades are published to sx-pub like any component."))
|
||||
|
||||
(~docs/subsection :title "2b. WebTransport Server (Server Nodes)"
|
||||
(p "HTTP/3 WebTransport endpoint on server nodes. "
|
||||
"Browsers connect and exchange SX messages over bidirectional streams.")
|
||||
(~docs/subsection :title "2b. Three Transports"
|
||||
|
||||
(table :class "w-full mb-6 text-sm"
|
||||
(thead
|
||||
(tr (th :class "text-left p-2" "Link") (th :class "text-left p-2" "Transport") (th :class "text-left p-2" "Why")))
|
||||
(tbody
|
||||
(tr (td :class "p-2 font-medium" "Browser ↔ Home node") (td :class "p-2" "WebSocket") (td :class "p-2" "Reliable, always-on, works today, no infrastructure changes"))
|
||||
(tr (td :class "p-2 font-medium" "Browser ↔ Browser") (td :class "p-2" "WebRTC data channel") (td :class "p-2" "True peer-to-peer, NAT traversal via ICE/STUN, UDP + TCP modes"))
|
||||
(tr (td :class "p-2 font-medium" "Server ↔ Server") (td :class "p-2" "HTTP federation") (td :class "p-2" "Existing sx-pub protocol, signed activities"))))
|
||||
|
||||
(p "The SX protocol layer is transport-agnostic. " (code "peer-send") " and " (code "peer-listen") " "
|
||||
"work identically whether the peer is connected via WebSocket, WebRTC, or HTTP. "
|
||||
"Upgrade to WebTransport when HTTP/3 infrastructure matures — zero protocol changes."))
|
||||
|
||||
(~docs/subsection :title "2c. Browser ↔ Home Node (WebSocket)"
|
||||
(p "Browser connects to home node on page load. Receives push updates, "
|
||||
"sends publish requests, queries the federation.")
|
||||
|
||||
(~docs/code :src (highlight
|
||||
";; Server-side: WebTransport handler\n;;\n;; One stream per conversation:\n;; Browser → Server: (want (list \"Qm..abc\" \"Qm..def\"))\n;; Server → Browser: (provide \"Qm..abc\" \"(defcomp ...)\")\n;; Server → Browser: (provide \"Qm..def\" \"(defisland ...)\")\n;;\n;; Push notifications:\n;; Server → Browser: (announce \"Qm..xyz\" {:author \"...\" :type :component})\n;;\n;; Peer introduction:\n;; Server → Browser: (peers (list {:id \"...\" :transport \"wt://...\"}))"
|
||||
"text"))
|
||||
";; Connect to home node — WebSocket, works today\n(define home (ws-connect (str \"wss://\" home-node \"/sx-web/ws\")))\n\n;; Same protocol regardless of transport\n(peer-send home (sx-hello (node-id) (node-pubkey)))\n(peer-listen home (fn (msg) (dispatch-protocol msg)))\n\n;; Push notifications from server:\n;; (announce \"Qm..xyz\" {:author \"...\" :type :component})\n;; Content exchange:\n;; (want (list \"Qm..abc\")) → (provide \"Qm..abc\" \"(defcomp ...)\")\n;; Peer introduction:\n;; (peers (list {:id \"...\" :offer sdp-offer}))"
|
||||
"lisp")))
|
||||
|
||||
(p "Server nodes also relay between browsers that can't connect directly. "
|
||||
"A browser publishes through its home node, which delivers to followers "
|
||||
"over existing sx-pub federation " (em "and") " pushes to connected browser peers."))
|
||||
|
||||
(~docs/subsection :title "2c. WebTransport Client (Browser Nodes)"
|
||||
(p "Browser connects to home node on page load. Receives push updates. "
|
||||
"Can also connect to peer nodes for direct exchange.")
|
||||
(~docs/subsection :title "2d. Browser ↔ Browser (WebRTC)"
|
||||
(p "Direct peer-to-peer connections between browsers. Home nodes relay "
|
||||
"the signaling (SDP offer/answer exchange), then browsers talk directly. "
|
||||
"Essential for low-latency game state, collaborative editing, and peer content exchange.")
|
||||
|
||||
(~docs/code :src (highlight
|
||||
";; Browser-side: WebTransport client\n(define sx-peer\n (fn (url)\n (let ((wt (host-call (host-global \"WebTransport\") \"new\" url))\n (ready (host-get wt \"ready\")))\n (promise-then ready\n (fn ()\n ;; Send hello with identity\n (peer-send wt (sx-hello (node-id) (node-pubkey)))\n ;; Listen for incoming messages\n (peer-listen wt (fn (msg)\n (dispatch-protocol msg))))))))\n\n;; Connect to home node on boot\n(sx-peer (str \"https://\" home-node \"/sx-web/wt\"))"
|
||||
";; Signaling: home nodes relay SDP offers\n;;\n;; Browser A → Home A (WebSocket): (rtc-offer peer-b-id sdp)\n;; Home A → Home B (federation): (rtc-relay peer-a-id sdp)\n;; Home B → Browser B (WebSocket): (rtc-offer peer-a-id sdp)\n;; Browser B → Home B: (rtc-answer peer-a-id sdp)\n;; ... relay back ...\n;; WebRTC connection established — browsers talk directly\n\n;; After signaling, same API as any peer\n(define peer-b (rtc-connect offer))\n(peer-send peer-b (signal-update \"player-x\" 42))\n(peer-listen peer-b (fn (msg) (dispatch-protocol msg)))"
|
||||
"lisp"))
|
||||
|
||||
(p "Three new platform primitives for the browser host:")
|
||||
(~docs/code :src (highlight
|
||||
";; WebRTC data channel modes — chosen per room\n;;\n;; Reliable ordered (TCP-like):\n;; Chat messages, component exchange, CID resolution\n;; Every message arrives, in order\n;;\n;; Unreliable unordered (UDP-like):\n;; Game state (player positions), cursor positions, audio levels\n;; Latest value matters, old values can drop\n;;\n;; A room can use both — reliable for commands, unreliable for state\n(define game-room (join-room home \"game-1\"\n {:reliable {:chat (signal (list)) :inventory (signal (dict))}\n :unreliable {:player-x (signal 0) :player-y (signal 0)\n :aim-angle (signal 0)}}))"
|
||||
"lisp"))
|
||||
|
||||
(p "Peer-to-peer means:")
|
||||
(ul :class "list-disc pl-6 mb-4 space-y-1"
|
||||
(li (code "wt-connect") " — open WebTransport session")
|
||||
(li (code "wt-send") " — send SX expression on stream")
|
||||
(li (code "wt-listen") " — register handler for incoming messages")))
|
||||
(li "Game state at LAN-like latency — no server round-trip")
|
||||
(li "Content exchange between nearby browsers without hitting IPFS")
|
||||
(li "Collaborative editing with sub-frame update propagation")
|
||||
(li "Works even if both home nodes are on modest hardware")
|
||||
(li "Server load scales with signaling, not with traffic")))
|
||||
|
||||
(~docs/subsection :title "2d. Shared Signals Over WebTransport"
|
||||
(p "Signals synchronized across peers. The same reactive primitive "
|
||||
@@ -159,13 +178,13 @@
|
||||
"A " (code "shared-signal") " is a signal. Existing components work unchanged. "
|
||||
"The networking is invisible to the reactive layer."))
|
||||
|
||||
(~docs/subsection :title "2e. Deliverables"
|
||||
(~docs/subsection :title "2f. Deliverables"
|
||||
(ul :class "list-disc pl-6 mb-4 space-y-1"
|
||||
(li (code "web/lib/sx-sync.sx") " — protocol definition incl. shared signals (~300 LOC)")
|
||||
(li (code "web/lib/peer.sx") " — browser peer client (~150 LOC)")
|
||||
(li (code "web/lib/peer.sx") " — transport-agnostic peer abstraction (~200 LOC)")
|
||||
(li (code "web/lib/shared-signal.sx") " — shared signal primitive (~120 LOC)")
|
||||
(li "Server: WebTransport endpoint + room relay (~300 LOC Python)")
|
||||
(li "Platform primitives: " (code "wt-connect, wt-send, wt-listen") " (~50 LOC JS)")
|
||||
(li "Platform primitives: " (code "ws-connect, rtc-connect, peer-send, peer-listen") " (~80 LOC JS)")
|
||||
(li "Server: WebSocket endpoint + signaling relay + room state (~350 LOC Python)")
|
||||
(li "Conflict resolution: LWW default + CRDT opt-in (~150 LOC SX)"))))
|
||||
|
||||
;; =====================================================================
|
||||
@@ -334,7 +353,7 @@
|
||||
(~docs/section :title "Implementation Order" :id "order"
|
||||
|
||||
(~docs/code :src (highlight
|
||||
"Phase 1 Browser Node Foundation ~400 LOC SX + ~100 LOC JS\n IndexedDB store, Service Worker, CID verification\n Prerequisite: none (extends existing sx-browser.js)\n\nPhase 2 WebTransport Peer Layer ~550 LOC SX + ~250 LOC Python\n Protocol definition, server endpoint, browser client\n Prerequisite: Phase 1 (needs content store)\n\nPhase 3 In-Browser Authoring ~550 LOC SX\n Editor, test runner, publish flow\n Prerequisite: Phase 1 + 2 (needs store + peer connection)\n\nPhase 4 Component Discovery ~750 LOC SX\n Dependency resolution, component browser, search\n Prerequisite: Phase 2 (needs federation queries)\n\nPhase 5 Node Bootstrap ~300 LOC config + scripts\n Docker image, auto-bootstrap, seed protocol\n Prerequisite: Phase 1–4 (packages everything)\n\nPhase 6 AI Composition ~400 LOC SX + embedding index\n Retrieval, composition, validation\n Prerequisite: Phase 4 (needs searchable component graph)\n\n Total new SX: ~2650 LOC\n Total new platform: ~350 LOC (JS + Python)\n Timeline: Phases are independent workstreams after Phase 1"
|
||||
"Phase 1 Browser Node Foundation ~400 LOC SX + ~100 LOC JS\n IndexedDB store, Service Worker, CID verification\n Prerequisite: none (extends existing sx-browser.js)\n\nPhase 2 Peer Network ~770 LOC SX + ~430 LOC Python/JS\n WebSocket + WebRTC + protocol + shared signals + rooms\n Prerequisite: Phase 1 (needs content store)\n\nPhase 3 In-Browser Authoring ~550 LOC SX\n Editor, test runner, publish flow\n Prerequisite: Phase 1 + 2 (needs store + peer connection)\n\nPhase 4 Component Discovery ~750 LOC SX\n Dependency resolution, component browser, search\n Prerequisite: Phase 2 (needs federation queries)\n\nPhase 5 Node Bootstrap ~300 LOC config + scripts\n Docker image, auto-bootstrap, seed protocol\n Prerequisite: Phase 1–4 (packages everything)\n\nPhase 6 AI Composition ~400 LOC SX + embedding index\n Retrieval, composition, validation\n Prerequisite: Phase 4 (needs searchable component graph)\n\n Total new SX: ~2650 LOC\n Total new platform: ~350 LOC (JS + Python)\n Timeline: Phases are independent workstreams after Phase 1"
|
||||
"text"))
|
||||
|
||||
(p "Phase 1 is the foundation — everything else depends on the browser being able to "
|
||||
|
||||
Reference in New Issue
Block a user