giles
881ed2cdcc
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 57s
Strongest near-term application: replace lossy dicts and opaque HTML fragments with structured trees that are both inspectable and renderable. Includes incremental migration path from current fetch_data/fetch_fragment. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
75 lines
3.3 KiB
Markdown
75 lines
3.3 KiB
Markdown
# S-expressions as Microservice Wire Format
|
|
|
|
**The strongest near-term application: structured inter-service communication, even when final output is still rendered HTML.**
|
|
|
|
---
|
|
|
|
## The Current Problem
|
|
|
|
Rose-ash services communicate two ways, both lossy:
|
|
|
|
### `fetch_data()` — returns dicts
|
|
|
|
The receiver has to know what keys to expect. There's no schema, no composability. It's just "here's a bag of stuff, good luck."
|
|
|
|
### `fetch_fragment()` — returns HTML strings
|
|
|
|
The receiving service can't inspect, filter, or transform them. It just jams the string into a template. If events returns a calendar nav fragment, blog can't reorder the items or filter by date. It's take-it-or-leave-it.
|
|
|
|
---
|
|
|
|
## Sexp Between Services
|
|
|
|
Sexp gives you **structured data that's also renderable**. The relations service could return:
|
|
|
|
```scheme
|
|
(nav :class "container-nav"
|
|
(relation :type "page->calendar" :label "Saturday Market" :href "/events/saturday/")
|
|
(relation :type "page->market" :label "Craft Stalls" :href "/markets/crafts/"))
|
|
```
|
|
|
|
The receiving service can:
|
|
|
|
- **Render it straight to HTML** — same as today's fragments
|
|
- **Filter it** — `exclude: page->calendar`
|
|
- **Reorder, group, or transform it** — tree operations, not string manipulation
|
|
- **Pass it through to the client as-is** — if they speak sexp
|
|
- **Cache it by content hash** — deterministic, structural equality
|
|
|
|
All without parsing HTML or knowing the internal structure of the sending service. The tree *is* the API response *and* the renderable output.
|
|
|
|
---
|
|
|
|
## One Format, Two Purposes
|
|
|
|
Today there's a split between "data endpoints" (`fetch_data` → dicts) and "fragment endpoints" (`fetch_fragment` → HTML strings). They serve different needs:
|
|
|
|
| Need | Current | With Sexp |
|
|
|---|---|---|
|
|
| Structured data for logic | `fetch_data()` → dict | Same sexp tree |
|
|
| Renderable HTML for templates | `fetch_fragment()` → HTML string | Same sexp tree, rendered at the boundary |
|
|
| Filtering/transforming cross-service output | Not possible (HTML is opaque) | Tree operations on sexp |
|
|
| Caching | Key-based (URL + params) | Content-addressed (hash the tree) |
|
|
| Schema/validation | None (hope the keys are right) | Tree structure is self-describing |
|
|
|
|
The unification eliminates an entire class of bugs — where `fetch_data` and `fetch_fragment` for the same resource return subtly inconsistent results because they're maintained as separate code paths.
|
|
|
|
---
|
|
|
|
## Nothing Changes for the User
|
|
|
|
The final output to the browser is still plain HTML. The improvement is entirely in how services talk to each other — less brittle, more composable, zero extra infrastructure. Sexp is evaluated to HTML at the outermost boundary (the route handler), exactly as it is today with the sexp template engine.
|
|
|
|
---
|
|
|
|
## Migration Path
|
|
|
|
This can be adopted incrementally, one service boundary at a time:
|
|
|
|
1. **Relations service already returns sexp** — `container-nav` fragment is rendered from sexp templates
|
|
2. **Next**: have fragment endpoints return raw sexp instead of pre-rendered HTML, let the caller render
|
|
3. **Then**: unify `fetch_data` and `fetch_fragment` into a single `fetch_sexp` that returns structured trees
|
|
4. **Finally**: callers that want data extract it from the tree; callers that want HTML render the tree
|
|
|
|
Each step is backwards-compatible. Services can serve both HTML fragments and sexp simultaneously during transition.
|