Expand prefetch plan: full strategy spectrum and components+data split
Add prefetch strategies section covering the full timing spectrum: eager bundle (initial load), idle timer (requestIdleCallback), viewport (IntersectionObserver), mouse trajectory prediction, hover, mousedown, and the components+data hybrid for :data pages. Add declarative configuration via defpage :prefetch metadata and sx-prefetch attributes. Update rollout to 7 incremental steps. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
105
sx/sx/plans.sx
105
sx/sx/plans.sx
@@ -603,7 +603,8 @@
|
||||
|
||||
(~doc-section :title "Context" :id "context"
|
||||
(p "Phase 3 of the isomorphic roadmap added client-side routing with component dependency checking. When a user clicks a link, " (code "try-client-route") " checks " (code "has-all-deps?") " — if the target page needs components not yet loaded, the client falls back to a server fetch. This works correctly but misses an opportunity: " (strong "we can prefetch those missing components before the click happens."))
|
||||
(p "The page registry already carries " (code ":deps") " metadata for every page. The client already knows which components are loaded via " (code "loaded-component-names") ". The gap is a mechanism to " (em "proactively") " resolve the difference — fetching missing component definitions so that by the time the user clicks, client-side routing succeeds."))
|
||||
(p "The page registry already carries " (code ":deps") " metadata for every page. The client already knows which components are loaded via " (code "loaded-component-names") ". The gap is a mechanism to " (em "proactively") " resolve the difference — fetching missing component definitions so that by the time the user clicks, client-side routing succeeds.")
|
||||
(p "But this goes beyond just hover-to-prefetch. The full spectrum includes: bundling linked routes' components with the initial page load, batch-prefetching after idle, predicting mouse trajectory toward links, and even splitting the component/data fetch so that " (code ":data") " pages can prefetch their components and only fetch data on click. Each strategy trades bandwidth for latency, and pages should be able to declare which tradeoff they want."))
|
||||
|
||||
(~doc-section :title "Current State" :id "current-state"
|
||||
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
|
||||
@@ -638,11 +639,94 @@
|
||||
(td :class "px-3 py-2 text-stone-700" (code "find-matching-route") " matches pathname to page entry")
|
||||
(td :class "px-3 py-2 font-mono text-sm text-violet-700" "router.sx"))))))
|
||||
|
||||
;; -----------------------------------------------------------------------
|
||||
;; Prefetch strategies
|
||||
;; -----------------------------------------------------------------------
|
||||
|
||||
(~doc-section :title "Prefetch Strategies" :id "strategies"
|
||||
(p "Prefetching is a spectrum from conservative to aggressive. The system should support all of these, configured declaratively per link or per page via " (code "defpage") " metadata and " (code "sx-prefetch") " attributes.")
|
||||
|
||||
(div :class "overflow-x-auto rounded border border-stone-200 mb-4"
|
||||
(table :class "w-full text-left text-sm"
|
||||
(thead (tr :class "border-b border-stone-200 bg-stone-100"
|
||||
(th :class "px-3 py-2 font-medium text-stone-600" "Strategy")
|
||||
(th :class "px-3 py-2 font-medium text-stone-600" "Trigger")
|
||||
(th :class "px-3 py-2 font-medium text-stone-600" "What prefetches")
|
||||
(th :class "px-3 py-2 font-medium text-stone-600" "Latency on click")))
|
||||
(tbody
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Eager bundle")
|
||||
(td :class "px-3 py-2 text-stone-700" "Initial page load")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components for linked routes included in " (code "<script data-components>"))
|
||||
(td :class "px-3 py-2 text-stone-600" "Zero — already in memory"))
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Idle timer")
|
||||
(td :class "px-3 py-2 text-stone-700" "After page settles (requestIdleCallback or setTimeout)")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components for visible nav links, batched in one request")
|
||||
(td :class "px-3 py-2 text-stone-600" "Zero if idle fetch completed"))
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Viewport")
|
||||
(td :class "px-3 py-2 text-stone-700" "Link scrolls into view (IntersectionObserver)")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components for that link's route")
|
||||
(td :class "px-3 py-2 text-stone-600" "Zero if user scrolled before clicking"))
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Mouse approach")
|
||||
(td :class "px-3 py-2 text-stone-700" "Cursor moving toward link (trajectory prediction)")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components for predicted target")
|
||||
(td :class "px-3 py-2 text-stone-600" "Near-zero — fetch starts ~200ms before hover"))
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Hover")
|
||||
(td :class "px-3 py-2 text-stone-700" "mouseover (150ms debounce)")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components for hovered link's route")
|
||||
(td :class "px-3 py-2 text-stone-600" "Low — typical hover-to-click is 300-500ms"))
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Mousedown")
|
||||
(td :class "px-3 py-2 text-stone-700" "mousedown (0ms debounce)")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components for clicked link's route")
|
||||
(td :class "px-3 py-2 text-stone-600" "~80ms — mousedown-to-click gap"))
|
||||
(tr :class "border-b border-stone-100"
|
||||
(td :class "px-3 py-2 font-semibold text-stone-800" "Components + data")
|
||||
(td :class "px-3 py-2 text-stone-700" "Any of the above")
|
||||
(td :class "px-3 py-2 text-stone-700" "Components " (em "and") " page data for " (code ":data") " pages")
|
||||
(td :class "px-3 py-2 text-stone-600" "Zero for components; data fetch may still be in flight")))))
|
||||
|
||||
(~doc-subsection :title "Eager Bundle"
|
||||
(p "The server already computes per-page component bundles. For key navigation paths — the main nav bar, section nav — the server can include " (em "linked routes' components") " in the initial bundle, not just the current page's.")
|
||||
(~doc-code :code (highlight ";; defpage metadata declares eager prefetch targets\n(defpage docs-page\n :path \"/docs/<slug>\"\n :auth :public\n :prefetch :eager ;; bundle deps for all linked pure routes\n :content (case slug ...))" "lisp"))
|
||||
(p "Implementation: " (code "components_for_page()") " already scans the page SX for component refs. Extend it to also scan for " (code "href") " attributes, match them against the page registry, and include those pages' deps in the bundle. The cost is a larger initial payload; the benefit is zero-latency navigation within a section."))
|
||||
|
||||
(~doc-subsection :title "Idle Timer"
|
||||
(p "After page load and initial render, use " (code "requestIdleCallback") " (or a fallback " (code "setTimeout") ") to scan visible nav links and batch-prefetch their missing components in a single request.")
|
||||
(~doc-code :code (highlight "(define prefetch-visible-links-on-idle\n (fn ()\n (request-idle-callback\n (fn ()\n (let ((links (dom-query-all \"a[href][sx-get]\"))\n (all-missing (list)))\n (for-each\n (fn (link)\n (let ((missing (compute-missing-deps\n (url-pathname (dom-get-attr link \"href\")))))\n (when missing\n (for-each (fn (d) (append! all-missing d))\n missing))))\n links)\n (when (not (empty? all-missing))\n (prefetch-components (dedupe all-missing))))))))" "lisp"))
|
||||
(p "Called once from " (code "boot-init") " after initial processing. Batches all missing deps into one network request. Low priority — browser handles it when idle."))
|
||||
|
||||
(~doc-subsection :title "Mouse Approach (Trajectory Prediction)"
|
||||
(p "Don't wait for the cursor to reach the link — predict where it's heading. Track the last few " (code "mousemove") " events, extrapolate the trajectory, and if it points toward a link, start prefetching before the hover event fires.")
|
||||
(~doc-code :code (highlight "(define bind-approach-prefetch\n (fn (container)\n ;; Track mouse trajectory within a nav container.\n ;; On each mousemove, extrapolate position ~200ms ahead.\n ;; If projected point intersects a link's bounding box,\n ;; prefetch that link's route deps.\n (let ((last-x 0) (last-y 0) (last-t 0)\n (prefetched (dict)))\n (dom-add-listener container \"mousemove\"\n (fn (e)\n (let ((now (timestamp))\n (dt (- now last-t)))\n (when (> dt 16) ;; ~60fps throttle\n (let ((vx (/ (- (event-x e) last-x) dt))\n (vy (/ (- (event-y e) last-y) dt))\n (px (+ (event-x e) (* vx 200)))\n (py (+ (event-y e) (* vy 200)))\n (target (dom-element-at-point px py)))\n (when (and target (dom-has-attr? target \"href\")\n (not (get prefetched\n (dom-get-attr target \"href\"))))\n (let ((href (dom-get-attr target \"href\")))\n (set! prefetched\n (merge prefetched {href true}))\n (prefetch-route-deps\n (url-pathname href)))))\n (set! last-x (event-x e))\n (set! last-y (event-y e))\n (set! last-t now))))))))" "lisp"))
|
||||
(p "This is the most speculative strategy — best suited for dense navigation areas (section sidebars, nav bars) where the cursor trajectory is a strong predictor. The " (code "prefetched") " dict prevents duplicate fetches within the same container interaction."))
|
||||
|
||||
(~doc-subsection :title "Components + Data (Hybrid Prefetch)"
|
||||
(p "The most interesting strategy. For pages with " (code ":data") " dependencies, current behavior is full server fallback. But the page's " (em "components") " are still pure and prefetchable. If we prefetch components ahead of time, the click only needs to fetch " (em "data") " — a much smaller, faster response.")
|
||||
(p "This creates a new rendering path:")
|
||||
(ol :class "list-decimal pl-5 text-stone-700 space-y-1"
|
||||
(li "Prefetch: hover/idle/viewport triggers " (code "prefetch-components") " for the target page")
|
||||
(li "Click: client has components, but page has " (code ":data") " — fetch data from server")
|
||||
(li "Server returns " (em "only data") " (JSON or SX bindings), not the full rendered page")
|
||||
(li "Client evaluates the content expression with prefetched components + fetched data")
|
||||
(li "Result: faster than full server render, no redundant component transfer"))
|
||||
(~doc-code :code (highlight ";; Declarative: prefetch components, fetch data on click\n(defpage reference-page\n :path \"/reference/<slug>\"\n :auth :public\n :prefetch :components ;; prefetch components, data stays server-fetched\n :data (reference-data slug)\n :content (~reference-attrs-content :attrs attrs))\n\n;; On click, client-side flow:\n;; 1. Components already prefetched (from hover/idle)\n;; 2. GET /reference/attributes → server returns data bindings\n;; 3. Client evals (reference-data slug) result + content expr\n;; 4. Renders locally with cached components" "lisp"))
|
||||
(p "This is a stepping stone toward full Phase 4 (client IO bridge) of the isomorphic roadmap — it achieves partial client rendering for data pages without needing a general-purpose client async evaluator. The server is a data service, the client is the renderer."))
|
||||
|
||||
(~doc-subsection :title "Declarative Configuration"
|
||||
(p "All strategies configured via " (code "defpage") " metadata and " (code "sx-prefetch") " attributes on links/containers:")
|
||||
(~doc-code :code (highlight ";; Page-level: what to prefetch for routes linking TO this page\n(defpage docs-page\n :path \"/docs/<slug>\"\n :prefetch :eager) ;; bundle with linking page\n\n(defpage reference-page\n :path \"/reference/<slug>\"\n :prefetch :components) ;; prefetch components, data on click\n\n;; Link-level: override per-link\n(a :href \"/docs/components\"\n :sx-prefetch \"idle\") ;; prefetch after page idle\n\n;; Container-level: approach prediction for nav areas\n(nav :sx-prefetch \"approach\"\n (a :href \"/docs/\") (a :href \"/reference/\") ...)" "lisp"))
|
||||
(p "Priority cascade: explicit " (code "sx-prefetch") " on link > " (code ":prefetch") " on target defpage > default (hover). The system never prefetches the same components twice — " (code "_prefetch-pending") " and " (code "loaded-component-names") " handle dedup.")))
|
||||
|
||||
;; -----------------------------------------------------------------------
|
||||
;; Design
|
||||
;; -----------------------------------------------------------------------
|
||||
|
||||
(~doc-section :title "Design" :id "design"
|
||||
(~doc-section :title "Implementation Design" :id "design"
|
||||
|
||||
(p "Per the SX host architecture principle: all SX-specific logic goes in " (code ".sx") " spec files and gets bootstrapped. The prefetch logic — scanning links, computing missing deps, managing the component cache — must be specced in " (code ".sx") ", not written directly in JS or Python.")
|
||||
|
||||
@@ -742,18 +826,21 @@
|
||||
|
||||
(~doc-section :title "Non-Goals (This Phase)" :id "non-goals"
|
||||
(ul :class "list-disc pl-5 text-stone-700 space-y-1"
|
||||
(li (strong "Predictive ML/heuristic prefetch") " — no analytics-driven \"likely next page\" predictions. Just hover + optional viewport.")
|
||||
(li (strong "Analytics-driven prediction") " — no ML models or click-frequency heuristics. Trajectory prediction uses geometry, not statistics.")
|
||||
(li (strong "Cross-service prefetch") " — components are per-service. A link to a different service domain is always a server navigation.")
|
||||
(li (strong "Service worker caching") " — could layer on later, but basic fetch + in-memory registration is sufficient.")
|
||||
(li (strong "Prefetching page data") " — pages with " (code ":data_expr") " still go to the server. Prefetching their data is a separate, larger problem (Phase 4 of isomorphic roadmap).")))
|
||||
(li (strong "Full client-side data evaluation") " — the components+data strategy fetches data from the server, it doesn't replicate server IO on the client. That's Phase 4 of the isomorphic roadmap.")))
|
||||
|
||||
(~doc-section :title "Rollout" :id "rollout"
|
||||
(p "Incremental, each step independently valuable:")
|
||||
(ol :class "list-decimal pl-5 text-stone-700 space-y-2"
|
||||
(li (strong "Implement endpoint") " — lowest risk, purely additive. Refactor " (code "components_for_request()") " to accept explicit names param.")
|
||||
(li (strong "Implement SX spec functions") " — no wiring yet, testable in isolation.")
|
||||
(li (strong "Wire hover prefetch into process-elements") " — all boosted links get it automatically.")
|
||||
(li (strong "Observe") " — console logs (" (code "sx:prefetch N components for /path") ") show what's happening. Check network tab for redundant fetches, timing.")
|
||||
(li (strong "Add viewport prefetch") " — opt-in via " (code "sx-prefetch=\"visible\"") " attribute on nav containers.")))
|
||||
(li (strong "Component endpoint") " — purely additive. Refactor " (code "components_for_request()") " to accept explicit " (code "?names=") " param.")
|
||||
(li (strong "Core spec functions") " — " (code "compute-missing-deps") ", " (code "prefetch-components") ", " (code "prefetch-route-deps") " in orchestration.sx. Testable in isolation.")
|
||||
(li (strong "Hover prefetch") " — wire " (code "bind-prefetch-on-hover") " into " (code "process-elements") ". All boosted links get it automatically. Console logs show activity.")
|
||||
(li (strong "Idle batch prefetch") " — call " (code "prefetch-visible-links-on-idle") " from " (code "boot-init") ". One request prefetches all visible nav deps after page settles.")
|
||||
(li (strong "Viewport + approach") " — opt-in via " (code "sx-prefetch") " attributes. Trajectory prediction for dense nav areas.")
|
||||
(li (strong "Eager bundles") " — extend " (code "components_for_page()") " to include linked routes' deps. Heavier initial payload, zero-latency nav.")
|
||||
(li (strong "Components + data split") " — new server response mode returning data bindings only. Client renders with prefetched components. Bridges toward Phase 4.")))
|
||||
|
||||
(~doc-section :title "Relationship to Isomorphic Roadmap" :id "relationship"
|
||||
(p "This plan sits between Phase 3 (client-side routing) and Phase 4 (client async & IO bridge) of the "
|
||||
|
||||
Reference in New Issue
Block a user