Add doc on sexp as microservice wire format
Strongest near-term application: replace lossy dicts and opaque HTML fragments with structured trees that are both inspectable and renderable. Includes incremental migration path from current fetch_data/fetch_fragment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
74
docs/sexpr-microservice-wire-format.md
Normal file
74
docs/sexpr-microservice-wire-format.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# S-expressions as Microservice Wire Format
|
||||
|
||||
**The strongest near-term application: structured inter-service communication, even when final output is still rendered HTML.**
|
||||
|
||||
---
|
||||
|
||||
## The Current Problem
|
||||
|
||||
Rose-ash services communicate two ways, both lossy:
|
||||
|
||||
### `fetch_data()` — returns dicts
|
||||
|
||||
The receiver has to know what keys to expect. There's no schema, no composability. It's just "here's a bag of stuff, good luck."
|
||||
|
||||
### `fetch_fragment()` — returns HTML strings
|
||||
|
||||
The receiving service can't inspect, filter, or transform them. It just jams the string into a template. If events returns a calendar nav fragment, blog can't reorder the items or filter by date. It's take-it-or-leave-it.
|
||||
|
||||
---
|
||||
|
||||
## Sexp Between Services
|
||||
|
||||
Sexp gives you **structured data that's also renderable**. The relations service could return:
|
||||
|
||||
```scheme
|
||||
(nav :class "container-nav"
|
||||
(relation :type "page->calendar" :label "Saturday Market" :href "/events/saturday/")
|
||||
(relation :type "page->market" :label "Craft Stalls" :href "/markets/crafts/"))
|
||||
```
|
||||
|
||||
The receiving service can:
|
||||
|
||||
- **Render it straight to HTML** — same as today's fragments
|
||||
- **Filter it** — `exclude: page->calendar`
|
||||
- **Reorder, group, or transform it** — tree operations, not string manipulation
|
||||
- **Pass it through to the client as-is** — if they speak sexp
|
||||
- **Cache it by content hash** — deterministic, structural equality
|
||||
|
||||
All without parsing HTML or knowing the internal structure of the sending service. The tree *is* the API response *and* the renderable output.
|
||||
|
||||
---
|
||||
|
||||
## One Format, Two Purposes
|
||||
|
||||
Today there's a split between "data endpoints" (`fetch_data` → dicts) and "fragment endpoints" (`fetch_fragment` → HTML strings). They serve different needs:
|
||||
|
||||
| Need | Current | With Sexp |
|
||||
|---|---|---|
|
||||
| Structured data for logic | `fetch_data()` → dict | Same sexp tree |
|
||||
| Renderable HTML for templates | `fetch_fragment()` → HTML string | Same sexp tree, rendered at the boundary |
|
||||
| Filtering/transforming cross-service output | Not possible (HTML is opaque) | Tree operations on sexp |
|
||||
| Caching | Key-based (URL + params) | Content-addressed (hash the tree) |
|
||||
| Schema/validation | None (hope the keys are right) | Tree structure is self-describing |
|
||||
|
||||
The unification eliminates an entire class of bugs — where `fetch_data` and `fetch_fragment` for the same resource return subtly inconsistent results because they're maintained as separate code paths.
|
||||
|
||||
---
|
||||
|
||||
## Nothing Changes for the User
|
||||
|
||||
The final output to the browser is still plain HTML. The improvement is entirely in how services talk to each other — less brittle, more composable, zero extra infrastructure. Sexp is evaluated to HTML at the outermost boundary (the route handler), exactly as it is today with the sexp template engine.
|
||||
|
||||
---
|
||||
|
||||
## Migration Path
|
||||
|
||||
This can be adopted incrementally, one service boundary at a time:
|
||||
|
||||
1. **Relations service already returns sexp** — `container-nav` fragment is rendered from sexp templates
|
||||
2. **Next**: have fragment endpoints return raw sexp instead of pre-rendered HTML, let the caller render
|
||||
3. **Then**: unify `fetch_data` and `fetch_fragment` into a single `fetch_sexp` that returns structured trees
|
||||
4. **Finally**: callers that want data extract it from the tree; callers that want HTML render the tree
|
||||
|
||||
Each step is backwards-compatible. Services can serve both HTML fragments and sexp simultaneously during transition.
|
||||
Reference in New Issue
Block a user