rose-ash/docs/sexpr-internal-protocol-first.md

Internal Protocol First: Building sexpr:// on the Microservice Mesh

The internal mesh is the testing ground, playground, and proving ground for the public protocol. Build it internally, battle-test it on real traffic, then open the door.

---

Why Internal First

You control both sides. Blog, market, events, relations — they're all your code. You don't need browser vendors or standards bodies. You write the client library in Python (or Rust), you write the server handler in Quart, and you iterate until it works.

---

The Traffic Is Real

Hundreds of inter-service calls per page load — fetch_data, fetch_fragment, call_action, send_internal_activity. That's real load, real error cases, real latency requirements. Not toy examples.

---

Replace HTTP Internally First

Right now services talk over HMAC-signed HTTP. Replace that transport with sexpr:// over QUIC or even Unix sockets and you've got a working protocol implementation carrying production traffic. The public HTTP interface stays untouched — the protocol lives behind the load balancer.

---

The Verbs Already Exist

The current inter-service API maps directly to the open verb system:

Current	sexpr:// equivalent
fetch_data(service, query, params)	(GET "/internal/data/{query}" :params ...)
call_action(service, action, payload)	({action} "/internal/" :body ...)
fetch_fragment(service, fragment, params)	(GET "/internal/fragments/{fragment}" :params ...)
send_internal_activity(service, activity)	(deliver :to service :activity ...)

You're not inventing traffic patterns — you're giving the existing ones a proper wire format. call_action("relations", "relate", payload) is literally (relate ...) with the ceremony stripped away.

---

When It's Solid, Open the Door

The same protocol library that blog uses to talk to relations, a Rust client uses to talk to blog. The public interface is just the internal protocol with auth and TLS on top. No separate "public API" to maintain.

Internal (today):    blog ──HTTP+HMAC──▶ relations
Internal (sexpr://): blog ──sexpr://──▶ relations     (same protocol)
Public (sexpr://):   rust-client ──sexpr://+TLS──▶ blog  (same protocol + auth)

---

What You're Actually Building

A real protocol stack — not a spec document, but running code:

• Client library (Python first, Rust later) — connect, send sexp, receive sexp, handle streams
• Server handler (Quart integration) — accept connections, route by verb+path, return sexp
• Connection management — multiplexing, keepalive, backpressure
• Error model — (err :code 404 :message "not found") instead of HTTP status codes
• Streaming — bidirectional sexp streams for real-time (replace WebSocket + SSE)
• Auth — HMAC for internal, OAuth bearer for public (same envelope, different :auth value)

All battle-tested on your own infrastructure before anyone else ever sees it.

---

Development Sequence

1. Define the wire format — framing (length-prefixed sexp over TCP/QUIC/Unix socket)
2. Build Python client+server — drop-in replacement for fetch_data/call_action/fetch_fragment
3. Run internal mesh on sexpr:// — blog ↔ relations ↔ events ↔ market all speaking sexp natively
4. Measure — latency, throughput, error rates vs current HTTP+JSON
5. Build Rust client library — same protocol, compiled and fast
6. Open public routes — Accept: application/sexp serves the same trees internally and externally
7. Rust native client — full Tier 2 client speaking the protocol that's been running in production for months