From d3617ab7f3f5c482f26a5220539e699fe2d418cc Mon Sep 17 00:00:00 2001 From: giles Date: Sat, 7 Mar 2026 00:06:22 +0000 Subject: [PATCH] Phase 4 complete: client data cache + plan update MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add page data cache in orchestration.sx (30s TTL, keyed by page-name+params) - Cache hit path: sx:route client+cache (instant render, no fetch) - Cache miss path: sx:route client+data (fetch, cache, render) - Fix HTMX response dep computation to include :data pages - Update isomorphic-sx-plan.md: Phases 1-4 marked done with details, reorder remaining phases (continuations→Phase 5, suspense→Phase 6, optimistic updates→Phase 7) Co-Authored-By: Claude Opus 4.6 --- docs/isomorphic-sx-plan.md | 257 +++++++++++++++------------- shared/static/scripts/sx-browser.js | 40 ++++- shared/sx/pages.py | 2 +- shared/sx/ref/orchestration.sx | 81 +++++++-- 4 files changed, 241 insertions(+), 139 deletions(-) diff --git a/docs/isomorphic-sx-plan.md b/docs/isomorphic-sx-plan.md index 09f6ea9..df2d324 100644 --- a/docs/isomorphic-sx-plan.md +++ b/docs/isomorphic-sx-plan.md @@ -20,136 +20,147 @@ The key insight: **s-expressions can partially unfold on the server after IO, th --- -### Phase 1: Component Distribution & Dependency Analysis +### Phase 1: Component Distribution & Dependency Analysis — DONE **What it enables:** Per-page component bundles instead of sending every definition to every page. Smaller payloads, faster boot, better cache hit rates. -**The problem:** `client_components_tag()` in `shared/sx/jinja_bridge.py` serializes ALL entries in `_COMPONENT_ENV`. The `sx_page()` template sends everything or nothing based on a single global hash. No mechanism determines which components a page actually needs. +**Implemented:** -**Approach:** - -1. **Transitive closure analyzer** — new module `shared/sx/deps.py` - - Walk `Component.body` AST, collect all `Symbol` refs starting with `~` +1. **Transitive closure analyzer** — `shared/sx/deps.py` (now `shared/sx/ref/deps.sx`, spec-level) + - Walk component body AST, collect all `~name` refs - Recursively follow into their bodies - Handle control forms (`if`/`when`/`cond`/`case`) — include ALL branches - - Handle macros — expand during walk using limited eval - - Function: `transitive_deps(name: str, env: dict) -> set[str]` - - Cache result on `Component` object (invalidate on hot-reload) + - `components_needed(source, env) -> set[str]` -2. **Runtime component scanning** — after `_aser` serializes page content, scan the SX string for `(~name` patterns (parallel to existing `scan_classes_from_sx` for CSS). Then compute transitive closure to get sub-components. +2. **IO reference analysis** — `deps.sx` also tracks IO primitive usage + - `scan-io-refs` / `transitive-io-refs` / `component-pure?` + - Used by Phase 2 for automatic server/client boundary -3. **Per-page component block** in `sx_page()` — replace all-or-nothing with page-specific bundle. Hash changes per page, localStorage cache keyed by route pattern. +3. **Per-page component block** — `_build_pages_sx()` in `helpers.py` + - Each page entry includes `:deps` list of required components + - Client page registry carries dep info for prefetching -4. **SX partial responses** — `components_for_request()` already diffs against `SX-Components` header. Enhance with transitive closure so only truly needed missing components are sent. +4. **SX partial responses** — `components_for_request()` diffs against `SX-Components` header, sends only missing components -**Files:** -- New: `shared/sx/deps.py` — dependency analysis -- `shared/sx/jinja_bridge.py` — per-page bundle generation, cache deps on Component -- `shared/sx/helpers.py` — modify `sx_page()` and `sx_response()` for page-specific bundles -- `shared/sx/types.py` — add `deps: set[str]` to Component -- `shared/sx/ref/boot.sx` — per-page component caching alongside global cache - -**Verification:** -- Page using 5/50 components → `data-components` block contains only those 5 + transitive deps -- No "Unknown component" errors after bundle reduction -- Payload size reduction measurable +**Files:** `shared/sx/ref/deps.sx`, `shared/sx/deps.py`, `shared/sx/helpers.py`, `shared/sx/jinja_bridge.py` --- -### Phase 2: Smart Server/Client Boundary +### Phase 2: Smart Server/Client Boundary — DONE **What it enables:** Formalized partial evaluation model. Server evaluates IO, serializes pure subtrees. The system automatically knows "this component needs server data" vs "this component is pure and can render anywhere." -**Current mechanism:** `_aser` in `async_eval.py` already does partial evaluation — IO primitives are awaited and substituted, HTML tags and component calls serialize as SX. The `_expand_components` context var controls expansion. But this is a global toggle, not per-component. +**Implemented:** -**Approach:** +1. **Automatic IO detection** — `deps.sx` walks component bodies for IO primitive refs + - `compute-all-io-refs` computes transitive IO analysis for all components + - `component-pure?` returns true if no IO refs transitively -1. **Automatic IO detection** — extend Phase 1 AST walker to check for references to `IO_PRIMITIVES` names (`frag`, `query`, `service`, `current-user`, etc.) - - `has_io_deps(name: str, env: dict) -> bool` - - Computed at registration time, cached on Component +2. **Selective expansion** — `_aser` expands known components server-side via `_aser_component` + - IO-dependent components expand server-side (IO must resolve) + - Unknown components serialize for client rendering + - `_expand_components` context var controls override -2. **Component metadata** — enrich Component with analysis results: - ```python - ComponentMeta: - deps: set[str] # transitive component deps (Phase 1) - io_refs: set[str] # IO primitive names referenced - is_pure: bool # True if io_refs empty (transitively) - ``` +3. **Component metadata** — computed at registration, cached on Component objects -3. **Selective expansion** — refine `_aser` (line ~1335): instead of checking a global `_expand_components` flag, check the component's `is_pure` metadata: - - IO-dependent → expand server-side (IO must resolve) - - Pure → serialize for client (let client render) - - Explicit override: `:server true` on defcomp forces server expansion - -4. **Data manifest** for pages — `PageDef` produces a declaration of what IO the page needs, enabling Phase 3 (client can prefetch data) and Phase 5 (streaming). - -**Files:** -- `shared/sx/deps.py` — add IO analysis -- `shared/sx/types.py` — add metadata fields to Component -- `shared/sx/async_eval.py` — refine `_aser` component expansion logic -- `shared/sx/jinja_bridge.py` — compute IO metadata at registration -- `shared/sx/pages.py` — data manifest on PageDef - -**Verification:** -- Components calling `(query ...)` classified IO-dependent; pure components classified pure -- Existing pages produce identical output (regression) +**Files:** `shared/sx/ref/deps.sx`, `shared/sx/async_eval.py`, `shared/sx/jinja_bridge.py` --- -### Phase 3: Client-Side Routing (SPA Mode) +### Phase 3: Client-Side Routing (SPA Mode) — DONE -**What it enables:** After initial page load, client resolves routes locally using cached components + data. Only hits server for fresh data or unknown routes. Like Next.js client-side navigation. +**What it enables:** After initial page load, client resolves routes locally using cached components. Only hits server for fresh data or unknown routes. -**Current mechanism:** All routing is server-side via `defpage` → Quart routes. Client navigates via `sx-boost` links doing `sx-get` + morphing. Every navigation = server roundtrip. +**Implemented:** -**Approach:** +1. **Client-side page registry** — `_build_pages_sx()` serializes defpage routing info + - `