rose-ash/docs/sexpr-microservice-wire-format.md

S-expressions as Microservice Wire Format

The strongest near-term application: structured inter-service communication, even when final output is still rendered HTML.

---

The Current Problem

Rose-ash services communicate two ways, both lossy:

fetch_data() — returns dicts

The receiver has to know what keys to expect. There's no schema, no composability. It's just "here's a bag of stuff, good luck."

fetch_fragment() — returns HTML strings

The receiving service can't inspect, filter, or transform them. It just jams the string into a template. If events returns a calendar nav fragment, blog can't reorder the items or filter by date. It's take-it-or-leave-it.

---

Sexp Between Services

Sexp gives you structured data that's also renderable. The relations service could return:

(nav :class "container-nav"
  (relation :type "page->calendar" :label "Saturday Market" :href "/events/saturday/")
  (relation :type "page->market" :label "Craft Stalls" :href "/markets/crafts/"))

The receiving service can:

• Render it straight to HTML — same as today's fragments
• Filter it — exclude: page->calendar
• Reorder, group, or transform it — tree operations, not string manipulation
• Pass it through to the client as-is — if they speak sexp
• Cache it by content hash — deterministic, structural equality

All without parsing HTML or knowing the internal structure of the sending service. The tree is the API response and the renderable output.

---

One Format, Two Purposes

Today there's a split between "data endpoints" (fetch_data → dicts) and "fragment endpoints" (fetch_fragment → HTML strings). They serve different needs:

Need	Current	With Sexp
Structured data for logic	fetch_data() → dict	Same sexp tree
Renderable HTML for templates	fetch_fragment() → HTML string	Same sexp tree, rendered at the boundary
Filtering/transforming cross-service output	Not possible (HTML is opaque)	Tree operations on sexp
Caching	Key-based (URL + params)	Content-addressed (hash the tree)
Schema/validation	None (hope the keys are right)	Tree structure is self-describing

The unification eliminates an entire class of bugs — where fetch_data and fetch_fragment for the same resource return subtly inconsistent results because they're maintained as separate code paths.

---

Nothing Changes for the User

The final output to the browser is still plain HTML. The improvement is entirely in how services talk to each other — less brittle, more composable, zero extra infrastructure. Sexp is evaluated to HTML at the outermost boundary (the route handler), exactly as it is today with the sexp template engine.

---

Migration Path

This can be adopted incrementally, one service boundary at a time:

1. Relations service already returns sexp — container-nav fragment is rendered from sexp templates
2. Next: have fragment endpoints return raw sexp instead of pre-rendered HTML, let the caller render
3. Then: unify fetch_data and fetch_fragment into a single fetch_sexp that returns structured trees
4. Finally: callers that want data extract it from the tree; callers that want HTML render the tree

Each step is backwards-compatible. Services can serve both HTML fragments and sexp simultaneously during transition.