coop/rose-ash
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

sx-web plan: WebSocket + WebRTC peer-to-peer architecture

Browse Source 
Replace WebTransport-only design with three-transport model:
- Browser↔server: WebSocket (works today, no infrastructure changes)
- Browser↔browser: WebRTC data channels (true P2P, NAT traversal)
- Server↔server: HTTP federation (existing sx-pub)

Home nodes relay WebRTC signaling. Reliable channels for chat/components,
unreliable channels for game state/cursor positions. Transport-agnostic
protocol layer — upgrade to WebTransport later with zero SX changes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
giles
2026-03-26 12:18:47 +00:00
parent 4bb4d47d63
commit 91d5de0554
1 changed files with 44 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files

69
sx/sx/plans/sx-web.sx
View File

@@ -26,7 +26,7 @@
(tr (td :class "p-2 font-medium" "Runtime") (td :class "p-2" "OCaml + Python") (td :class "p-2" "JavaScript (sx-browser.js)")) (tr (td :class "p-2 font-medium" "Runtime") (td :class "p-2" "OCaml + Python") (td :class "p-2" "JavaScript (sx-browser.js)"))
(tr (td :class "p-2 font-medium" "Storage") (td :class "p-2" "IPFS daemon + PostgreSQL") (td :class "p-2" "IndexedDB + Cache API")) (tr (td :class "p-2 font-medium" "Storage") (td :class "p-2" "IPFS daemon + PostgreSQL") (td :class "p-2" "IndexedDB + Cache API"))
(tr (td :class "p-2 font-medium" "IPFS") (td :class "p-2" "Full DHT peer, pins content") (td :class "p-2" "Verifies CIDs, caches locally")) (tr (td :class "p-2 font-medium" "IPFS") (td :class "p-2" "Full DHT peer, pins content") (td :class "p-2" "Verifies CIDs, caches locally"))
(tr (td :class "p-2 font-medium" "Transport") (td :class "p-2" "HTTP/3 + WebTransport server") (td :class "p-2" "WebTransport client")) (tr (td :class "p-2 font-medium" "Transport") (td :class "p-2" "WebSocket + HTTP federation") (td :class "p-2" "WebSocket to home + WebRTC to peers"))
(tr (td :class "p-2 font-medium" "Availability") (td :class "p-2" "Long-lived, always on") (td :class "p-2" "Ephemeral, online when open")) (tr (td :class "p-2 font-medium" "Availability") (td :class "p-2" "Long-lived, always on") (td :class "p-2" "Ephemeral, online when open"))
(tr (td :class "p-2 font-medium" "Trust") (td :class "p-2" "CID verification + blockchain anchor") (td :class "p-2" "CID verification (same guarantee)")))) (tr (td :class "p-2 font-medium" "Trust") (td :class "p-2" "CID verification + blockchain anchor") (td :class "p-2" "CID verification (same guarantee)"))))
@@ -91,10 +91,12 @@
;; Phase 2: WebTransport Peer Layer ;; Phase 2: WebTransport Peer Layer
;; ===================================================================== ;; =====================================================================
(~docs/section :title "Phase 2: WebTransport Peer Layer" :id "phase-2" (~docs/section :title "Phase 2: Peer Network" :id "phase-2"
(p :class "text-stone-600 italic mb-4" (p :class "text-stone-600 italic mb-4"
"Bidirectional streams between browsers and servers. The protocol is SX.") "Three transports, one SX protocol. Browser↔server via WebSocket, "
"browser↔browser via WebRTC data channels, server↔server via federation. "
"The protocol layer doesn't know or care which pipe carries it.")
(~docs/subsection :title "2a. SX Protocol Definition" (~docs/subsection :title "2a. SX Protocol Definition"
(p "The protocol is an SX component. Nodes fetch, evaluate, and speak it.") (p "The protocol is an SX component. Nodes fetch, evaluate, and speak it.")
@@ -106,31 +108,48 @@
(p "The protocol definition is itself content-addressed. Nodes pin the protocol " (p "The protocol definition is itself content-addressed. Nodes pin the protocol "
"version they speak. Protocol upgrades are published to sx-pub like any component.")) "version they speak. Protocol upgrades are published to sx-pub like any component."))
(~docs/subsection :title "2b. WebTransport Server (Server Nodes)" (~docs/subsection :title "2b. Three Transports"
(p "HTTP/3 WebTransport endpoint on server nodes. "
"Browsers connect and exchange SX messages over bidirectional streams.") (table :class "w-full mb-6 text-sm"
(thead
(tr (th :class "text-left p-2" "Link") (th :class "text-left p-2" "Transport") (th :class "text-left p-2" "Why")))
(tbody
(tr (td :class "p-2 font-medium" "Browser ↔ Home node") (td :class "p-2" "WebSocket") (td :class "p-2" "Reliable, always-on, works today, no infrastructure changes"))
(tr (td :class "p-2 font-medium" "Browser ↔ Browser") (td :class "p-2" "WebRTC data channel") (td :class "p-2" "True peer-to-peer, NAT traversal via ICE/STUN, UDP + TCP modes"))
(tr (td :class "p-2 font-medium" "Server ↔ Server") (td :class "p-2" "HTTP federation") (td :class "p-2" "Existing sx-pub protocol, signed activities"))))
(p "The SX protocol layer is transport-agnostic. " (code "peer-send") " and " (code "peer-listen") " "
"work identically whether the peer is connected via WebSocket, WebRTC, or HTTP. "
"Upgrade to WebTransport when HTTP/3 infrastructure matures — zero protocol changes."))
(~docs/subsection :title "2c. Browser ↔ Home Node (WebSocket)"
(p "Browser connects to home node on page load. Receives push updates, "
"sends publish requests, queries the federation.")
(~docs/code :src (highlight (~docs/code :src (highlight
";; Server-side: WebTransport handler\n;;\n;; One stream per conversation:\n;; Browser → Server: (want (list \"Qm..abc\" \"Qm..def\"))\n;; Server → Browser: (provide \"Qm..abc\" \"(defcomp ...)\")\n;; Server → Browser: (provide \"Qm..def\" \"(defisland ...)\")\n;;\n;; Push notifications:\n;; Server → Browser: (announce \"Qm..xyz\" {:author \"...\" :type :component})\n;;\n;; Peer introduction:\n;; Server → Browser: (peers (list {:id \"...\" :transport \"wt://...\"}))" ";; Connect to home node — WebSocket, works today\n(define home (ws-connect (str \"wss://\" home-node \"/sx-web/ws\")))\n\n;; Same protocol regardless of transport\n(peer-send home (sx-hello (node-id) (node-pubkey)))\n(peer-listen home (fn (msg) (dispatch-protocol msg)))\n\n;; Push notifications from server:\n;; (announce \"Qm..xyz\" {:author \"...\" :type :component})\n;; Content exchange:\n;; (want (list \"Qm..abc\")) → (provide \"Qm..abc\" \"(defcomp ...)\")\n;; Peer introduction:\n;; (peers (list {:id \"...\" :offer sdp-offer}))"
"text")) "lisp")))
(p "Server nodes also relay between browsers that can't connect directly. " (~docs/subsection :title "2d. Browser ↔ Browser (WebRTC)"
"A browser publishes through its home node, which delivers to followers " (p "Direct peer-to-peer connections between browsers. Home nodes relay "
"over existing sx-pub federation " (em "and") " pushes to connected browser peers.")) "the signaling (SDP offer/answer exchange), then browsers talk directly. "
"Essential for low-latency game state, collaborative editing, and peer content exchange.")
(~docs/subsection :title "2c. WebTransport Client (Browser Nodes)"
(p "Browser connects to home node on page load. Receives push updates. "
"Can also connect to peer nodes for direct exchange.")
(~docs/code :src (highlight (~docs/code :src (highlight
";; Browser-side: WebTransport client\n(define sx-peer\n (fn (url)\n (let ((wt (host-call (host-global \"WebTransport\") \"new\" url))\n (ready (host-get wt \"ready\")))\n (promise-then ready\n (fn ()\n ;; Send hello with identity\n (peer-send wt (sx-hello (node-id) (node-pubkey)))\n ;; Listen for incoming messages\n (peer-listen wt (fn (msg)\n (dispatch-protocol msg))))))))\n\n;; Connect to home node on boot\n(sx-peer (str \"https://\" home-node \"/sx-web/wt\"))" ";; Signaling: home nodes relay SDP offers\n;;\n;; Browser A → Home A (WebSocket): (rtc-offer peer-b-id sdp)\n;; Home A → Home B (federation): (rtc-relay peer-a-id sdp)\n;; Home B → Browser B (WebSocket): (rtc-offer peer-a-id sdp)\n;; Browser B → Home B: (rtc-answer peer-a-id sdp)\n;; ... relay back ...\n;; WebRTC connection established — browsers talk directly\n\n;; After signaling, same API as any peer\n(define peer-b (rtc-connect offer))\n(peer-send peer-b (signal-update \"player-x\" 42))\n(peer-listen peer-b (fn (msg) (dispatch-protocol msg)))"
"lisp")) "lisp"))
(p "Three new platform primitives for the browser host:") (~docs/code :src (highlight
";; WebRTC data channel modes — chosen per room\n;;\n;; Reliable ordered (TCP-like):\n;; Chat messages, component exchange, CID resolution\n;; Every message arrives, in order\n;;\n;; Unreliable unordered (UDP-like):\n;; Game state (player positions), cursor positions, audio levels\n;; Latest value matters, old values can drop\n;;\n;; A room can use both — reliable for commands, unreliable for state\n(define game-room (join-room home \"game-1\"\n {:reliable {:chat (signal (list)) :inventory (signal (dict))}\n :unreliable {:player-x (signal 0) :player-y (signal 0)\n :aim-angle (signal 0)}}))"
"lisp"))
(p "Peer-to-peer means:")
(ul :class "list-disc pl-6 mb-4 space-y-1" (ul :class "list-disc pl-6 mb-4 space-y-1"
(li (code "wt-connect") " — open WebTransport session") (li "Game state at LAN-like latency — no server round-trip")
(li (code "wt-send") " — send SX expression on stream") (li "Content exchange between nearby browsers without hitting IPFS")
(li (code "wt-listen") " — register handler for incoming messages"))) (li "Collaborative editing with sub-frame update propagation")
(li "Works even if both home nodes are on modest hardware")
(li "Server load scales with signaling, not with traffic")))
(~docs/subsection :title "2d. Shared Signals Over WebTransport" (~docs/subsection :title "2d. Shared Signals Over WebTransport"
(p "Signals synchronized across peers. The same reactive primitive " (p "Signals synchronized across peers. The same reactive primitive "
@@ -159,13 +178,13 @@
"A " (code "shared-signal") " is a signal. Existing components work unchanged. " "A " (code "shared-signal") " is a signal. Existing components work unchanged. "
"The networking is invisible to the reactive layer.")) "The networking is invisible to the reactive layer."))
(~docs/subsection :title "2e. Deliverables" (~docs/subsection :title "2f. Deliverables"
(ul :class "list-disc pl-6 mb-4 space-y-1" (ul :class "list-disc pl-6 mb-4 space-y-1"
(li (code "web/lib/sx-sync.sx") " — protocol definition incl. shared signals (~300 LOC)") (li (code "web/lib/sx-sync.sx") " — protocol definition incl. shared signals (~300 LOC)")
(li (code "web/lib/peer.sx") " — browser peer client (~150 LOC)") (li (code "web/lib/peer.sx") " — transport-agnostic peer abstraction (~200 LOC)")
(li (code "web/lib/shared-signal.sx") " — shared signal primitive (~120 LOC)") (li (code "web/lib/shared-signal.sx") " — shared signal primitive (~120 LOC)")
(li "Server: WebTransport endpoint + room relay (~300 LOC Python)") (li "Platform primitives: " (code "ws-connect, rtc-connect, peer-send, peer-listen") " (~80 LOC JS)")
(li "Platform primitives: " (code "wt-connect, wt-send, wt-listen") " (~50 LOC JS)") (li "Server: WebSocket endpoint + signaling relay + room state (~350 LOC Python)")
(li "Conflict resolution: LWW default + CRDT opt-in (~150 LOC SX)")))) (li "Conflict resolution: LWW default + CRDT opt-in (~150 LOC SX)"))))
;; ===================================================================== ;; =====================================================================
@@ -334,7 +353,7 @@
(~docs/section :title "Implementation Order" :id "order" (~docs/section :title "Implementation Order" :id "order"
(~docs/code :src (highlight (~docs/code :src (highlight
"Phase 1 Browser Node Foundation ~400 LOC SX + ~100 LOC JS\n IndexedDB store, Service Worker, CID verification\n Prerequisite: none (extends existing sx-browser.js)\n\nPhase 2 WebTransport Peer Layer ~550 LOC SX + ~250 LOC Python\n Protocol definition, server endpoint, browser client\n Prerequisite: Phase 1 (needs content store)\n\nPhase 3 In-Browser Authoring ~550 LOC SX\n Editor, test runner, publish flow\n Prerequisite: Phase 1 + 2 (needs store + peer connection)\n\nPhase 4 Component Discovery ~750 LOC SX\n Dependency resolution, component browser, search\n Prerequisite: Phase 2 (needs federation queries)\n\nPhase 5 Node Bootstrap ~300 LOC config + scripts\n Docker image, auto-bootstrap, seed protocol\n Prerequisite: Phase 1–4 (packages everything)\n\nPhase 6 AI Composition ~400 LOC SX + embedding index\n Retrieval, composition, validation\n Prerequisite: Phase 4 (needs searchable component graph)\n\n Total new SX: ~2650 LOC\n Total new platform: ~350 LOC (JS + Python)\n Timeline: Phases are independent workstreams after Phase 1" "Phase 1 Browser Node Foundation ~400 LOC SX + ~100 LOC JS\n IndexedDB store, Service Worker, CID verification\n Prerequisite: none (extends existing sx-browser.js)\n\nPhase 2 Peer Network ~770 LOC SX + ~430 LOC Python/JS\n WebSocket + WebRTC + protocol + shared signals + rooms\n Prerequisite: Phase 1 (needs content store)\n\nPhase 3 In-Browser Authoring ~550 LOC SX\n Editor, test runner, publish flow\n Prerequisite: Phase 1 + 2 (needs store + peer connection)\n\nPhase 4 Component Discovery ~750 LOC SX\n Dependency resolution, component browser, search\n Prerequisite: Phase 2 (needs federation queries)\n\nPhase 5 Node Bootstrap ~300 LOC config + scripts\n Docker image, auto-bootstrap, seed protocol\n Prerequisite: Phase 1–4 (packages everything)\n\nPhase 6 AI Composition ~400 LOC SX + embedding index\n Retrieval, composition, validation\n Prerequisite: Phase 4 (needs searchable component graph)\n\n Total new SX: ~2650 LOC\n Total new platform: ~350 LOC (JS + Python)\n Timeline: Phases are independent workstreams after Phase 1"
"text")) "text"))
(p "Phase 1 is the foundation — everything else depends on the browser being able to " (p "Phase 1 is the foundation — everything else depends on the browser being able to "