diff --git a/docs/isomorphic-sx-plan.md b/docs/isomorphic-sx-plan.md index 9ebdaf8..09f6ea9 100644 --- a/docs/isomorphic-sx-plan.md +++ b/docs/isomorphic-sx-plan.md @@ -1,446 +1,273 @@ -# Isomorphic SX Architecture Migration Plan +# SX Isomorphic Architecture Roadmap ## Context -The sx layer already renders full pages client-side — `sx_page()` ships raw sx source + component definitions to the browser, `sx.js` evaluates and renders them. Components are cached in localStorage with a hash-based invalidation protocol (cookie `sx-comp-hash` → server skips sending defs if hash matches). +SX has a working server-client pipeline: server evaluates pages with IO (DB, fragments), serializes as SX wire format, client parses and renders to DOM. The language and primitives are already isomorphic — same spec, same semantics, both sides. What's missing is the **plumbing** that makes the boundary between server and client a sliding window rather than a fixed wall. -**Key insight from the user:** Pages/routes are just components. They belong in the same component registry, cached in localStorage alongside `defcomp` definitions. On navigation, if the client's component hash is current, the server doesn't need to send any s-expression source at all — just data. The client already has the page component cached and renders it locally with fresh data from the API. +The key insight: **s-expressions can partially unfold on the server after IO, then finish unfolding on the client.** The system should be clever enough to know which downstream components have data fetches, resolve those server-side, and send the rest as pure SX for client rendering. Eventually, the client can also do IO (mapping server DB queries to REST calls), handle routing (SPA), and even work offline with cached data. -### Target Architecture +## Current State (what's solid) -``` -First visit: - Server → component defs (including page components) + page data → client caches defs in localStorage +- **Primitive parity:** 100%. ~80 pure primitives, same names/semantics, JS and Python. +- **eval/parse/render:** Complete both sides. sx-ref.js has eval, parse, render-to-html, render-to-dom, aser. +- **Engine:** engine.sx (morph, swaps, triggers, history), orchestration.sx (fetch, events), boot.sx (hydration) — all transpiled. +- **Wire format:** Server `_aser` → SX source → client parses → renders to DOM. Boundary is clean. +- **Component caching:** Hash-based localStorage for component definitions and style dictionaries. +- **CSS on-demand:** CSSX resolves keywords to CSS rules, injects only used rules. +- **Boundary enforcement:** `boundary.sx` + `SX_BOUNDARY_STRICT=1` validates all primitives/IO/helpers at registration. -Subsequent navigation (same session, hash valid): - Client has page component cached → fetches only JSON data from /api/data/ → renders locally - Server sends: { data: {...} } — zero sx source - -SSR (bots, first paint): - Server evaluates the same page component with direct DB queries → sends rendered HTML - Client hydrates (binds SxEngine handlers, no re-render) -``` - -This is React-like data fetching with an s-expression view layer instead of JSX, and the component transport is a content-addressed cache rather than a JS bundle. - -### Data Delivery Modes - -The data side is not a single pattern — it's a spectrum that can be mixed per page and per fragment: - -**Mode A: Server-bundled data** — Server evaluates the page's `:data` slot, resolves all queries (including cross-service `fetch_data` calls), returns one JSON blob. Fewest round-trips. Server aggregates. - -**Mode B: Client-fetched data** — Client evaluates `:data` slot locally. Each `(query ...)` / `(service ...)` hits the relevant service's `/api/data/` endpoint independently. More round-trips but fully decoupled — each service handles its own data. - -**Mode C: Hybrid** — Server bundles same-service data (direct DB). Client fetches cross-service data in parallel from other services' APIs. Mirrors current server pattern: own-domain = SQLAlchemy, cross-domain = `fetch_data()` HTTP. - -The same spectrum applies to **fragments** (`frag` / `fetch_fragment`): - -- **Server-composed:** Server calls `fetch_fragment()` during page evaluation, bakes result into data bundle or renders inline. -- **Client-composed:** Client's `(frag ...)` primitive fetches from the service's public fragment endpoint. Fragment returns sx source, client renders locally using cached component defs. -- **Mixed:** Stable fragments (nav, auth menu) server-composed; content-specific fragments client-fetched. - -A `(query ...)` or `(frag ...)` call resolves differently depending on execution context (server vs client) but produces the same result. The choice of mode can be per-page, per-fragment, or even per-request. - -## Delivery Order - -``` -Phase 1 (Primitive Parity) ──┐ - ├── Phase 4 (Client Data Primitives) ──┐ -Phase 3 (Public Data API) ───┘ ├── Phase 5 (Data-Only Navigation) -Phase 2 (Server-Side Rendering) ────────────────────────────────────┘ -``` - -Phases 1-3 are independent. Recommended order: **3 → 1 → 2 → 4 → 5** +## Architecture Phases --- -## Phase 1: Primitive Parity +### Phase 1: Component Distribution & Dependency Analysis -Align JS and Python primitive sets so the same component source evaluates identically on both sides. +**What it enables:** Per-page component bundles instead of sending every definition to every page. Smaller payloads, faster boot, better cache hit rates. -### 1a: Add missing pure primitives to sx.js +**The problem:** `client_components_tag()` in `shared/sx/jinja_bridge.py` serializes ALL entries in `_COMPONENT_ENV`. The `sx_page()` template sends everything or nothing based on a single global hash. No mechanism determines which components a page actually needs. -Add to `PRIMITIVES` in `shared/static/scripts/sx.js`: +**Approach:** -| Primitive | JS implementation | -|-----------|-------------------| -| `clamp` | `Math.max(lo, Math.min(hi, x))` | -| `chunk-every` | partition list into n-size sublists | -| `zip-pairs` | `[[coll[0],coll[1]], [coll[2],coll[3]], ...]` | -| `dissoc` | shallow copy without specified keys | -| `into` | target-type-aware merge | -| `format-date` | minimal strftime translator covering `%Y %m %d %b %B %H %M %S` | -| `parse-int` | `parseInt` with NaN fallback to default | -| `assert` | throw if falsy | +1. **Transitive closure analyzer** — new module `shared/sx/deps.py` + - Walk `Component.body` AST, collect all `Symbol` refs starting with `~` + - Recursively follow into their bodies + - Handle control forms (`if`/`when`/`cond`/`case`) — include ALL branches + - Handle macros — expand during walk using limited eval + - Function: `transitive_deps(name: str, env: dict) -> set[str]` + - Cache result on `Component` object (invalidate on hot-reload) -Fix existing parity gaps: `round` needs optional `ndigits`; `min`/`max` need to accept a single list arg. +2. **Runtime component scanning** — after `_aser` serializes page content, scan the SX string for `(~name` patterns (parallel to existing `scan_classes_from_sx` for CSS). Then compute transitive closure to get sub-components. -### 1b: Inject `window.__sxConfig` for server-context primitives +3. **Per-page component block** in `sx_page()` — replace all-or-nothing with page-specific bundle. Hash changes per page, localStorage cache keyed by route pattern. -Modify `sx_page()` in `shared/sx/helpers.py` to inject before sx.js: +4. **SX partial responses** — `components_for_request()` already diffs against `SX-Components` header. Enhance with transitive closure so only truly needed missing components are sent. -```js -window.__sxConfig = { - appUrls: { blog: "https://blog.rose-ash.com", ... }, - assetUrl: "https://static...", - config: { /* public subset */ }, - currentUser: { id, username, display_name, avatar } | null, - relations: [ /* serialized RelationDef list */ ] -}; -``` +**Files:** +- New: `shared/sx/deps.py` — dependency analysis +- `shared/sx/jinja_bridge.py` — per-page bundle generation, cache deps on Component +- `shared/sx/helpers.py` — modify `sx_page()` and `sx_response()` for page-specific bundles +- `shared/sx/types.py` — add `deps: set[str]` to Component +- `shared/sx/ref/boot.sx` — per-page component caching alongside global cache -Sources: `ctx` has `blog_url`, `market_url`, etc. `g.user` has user info. `shared/infrastructure/urls.py` has the URL map. - -Add JS primitives reading from `__sxConfig`: `app-url`, `asset-url`, `config`, `current-user`, `relations-from`. - -`url-for` has no JS equivalent — isomorphic code uses `app-url` instead. - -### 1c: Add `defpage` to sx.js evaluator - -Add `defpage` to `SPECIAL_FORMS`. Parse the declaration, store it in `_componentEnv` under `"page:name"` (same registry as components). The page definition includes: name, path pattern, auth requirement, layout spec, and unevaluated AST for data/content/filter/aside/menu slots. - -Since pages live in `_componentEnv`, they're automatically included in the component hash, cached in localStorage, and skipped when the hash matches. No separate `