From 881ed2cdccd07dafbacc558dacbfecd39e724e70 Mon Sep 17 00:00:00 2001 From: giles Date: Sat, 28 Feb 2026 15:11:05 +0000 Subject: [PATCH] Add doc on sexp as microservice wire format Strongest near-term application: replace lossy dicts and opaque HTML fragments with structured trees that are both inspectable and renderable. Includes incremental migration path from current fetch_data/fetch_fragment. Co-Authored-By: Claude Opus 4.6 --- docs/sexpr-microservice-wire-format.md | 74 ++++++++++++++++++++++++++ 1 file changed, 74 insertions(+) create mode 100644 docs/sexpr-microservice-wire-format.md diff --git a/docs/sexpr-microservice-wire-format.md b/docs/sexpr-microservice-wire-format.md new file mode 100644 index 0000000..ae2c37c --- /dev/null +++ b/docs/sexpr-microservice-wire-format.md @@ -0,0 +1,74 @@ +# S-expressions as Microservice Wire Format + +**The strongest near-term application: structured inter-service communication, even when final output is still rendered HTML.** + +--- + +## The Current Problem + +Rose-ash services communicate two ways, both lossy: + +### `fetch_data()` — returns dicts + +The receiver has to know what keys to expect. There's no schema, no composability. It's just "here's a bag of stuff, good luck." + +### `fetch_fragment()` — returns HTML strings + +The receiving service can't inspect, filter, or transform them. It just jams the string into a template. If events returns a calendar nav fragment, blog can't reorder the items or filter by date. It's take-it-or-leave-it. + +--- + +## Sexp Between Services + +Sexp gives you **structured data that's also renderable**. The relations service could return: + +```scheme +(nav :class "container-nav" + (relation :type "page->calendar" :label "Saturday Market" :href "/events/saturday/") + (relation :type "page->market" :label "Craft Stalls" :href "/markets/crafts/")) +``` + +The receiving service can: + +- **Render it straight to HTML** — same as today's fragments +- **Filter it** — `exclude: page->calendar` +- **Reorder, group, or transform it** — tree operations, not string manipulation +- **Pass it through to the client as-is** — if they speak sexp +- **Cache it by content hash** — deterministic, structural equality + +All without parsing HTML or knowing the internal structure of the sending service. The tree *is* the API response *and* the renderable output. + +--- + +## One Format, Two Purposes + +Today there's a split between "data endpoints" (`fetch_data` → dicts) and "fragment endpoints" (`fetch_fragment` → HTML strings). They serve different needs: + +| Need | Current | With Sexp | +|---|---|---| +| Structured data for logic | `fetch_data()` → dict | Same sexp tree | +| Renderable HTML for templates | `fetch_fragment()` → HTML string | Same sexp tree, rendered at the boundary | +| Filtering/transforming cross-service output | Not possible (HTML is opaque) | Tree operations on sexp | +| Caching | Key-based (URL + params) | Content-addressed (hash the tree) | +| Schema/validation | None (hope the keys are right) | Tree structure is self-describing | + +The unification eliminates an entire class of bugs — where `fetch_data` and `fetch_fragment` for the same resource return subtly inconsistent results because they're maintained as separate code paths. + +--- + +## Nothing Changes for the User + +The final output to the browser is still plain HTML. The improvement is entirely in how services talk to each other — less brittle, more composable, zero extra infrastructure. Sexp is evaluated to HTML at the outermost boundary (the route handler), exactly as it is today with the sexp template engine. + +--- + +## Migration Path + +This can be adopted incrementally, one service boundary at a time: + +1. **Relations service already returns sexp** — `container-nav` fragment is rendered from sexp templates +2. **Next**: have fragment endpoints return raw sexp instead of pre-rendered HTML, let the caller render +3. **Then**: unify `fetch_data` and `fetch_fragment` into a single `fetch_sexp` that returns structured trees +4. **Finally**: callers that want data extract it from the tree; callers that want HTML render the tree + +Each step is backwards-compatible. Services can serve both HTML fragments and sexp simultaneously during transition.