From 6b7559fcafea916130814a5fc8c4b604ad303301 Mon Sep 17 00:00:00 2001 From: giles Date: Fri, 24 Apr 2026 06:54:21 +0000 Subject: [PATCH] HS-design: E40 real fetch + before-fetch + non-2xx Co-Authored-By: Claude Opus 4.7 (1M context) --- plans/designs/e40-real-fetch.md | 256 ++++++++++++++++++++++++++++++++ 1 file changed, 256 insertions(+) create mode 100644 plans/designs/e40-real-fetch.md diff --git a/plans/designs/e40-real-fetch.md b/plans/designs/e40-real-fetch.md new file mode 100644 index 00000000..ba375121 --- /dev/null +++ b/plans/designs/e40-real-fetch.md @@ -0,0 +1,256 @@ +# E40 — Fetch non-2xx + `hyperscript:beforeFetch` + real Response + +Cluster 40 of `plans/hs-conformance-to-100.md`. Target: +7 tests. + +## 1. Failing tests + +All seven currently emit `SKIP (skip-list)` from the generator +(`SKIP_TEST_NAMES` in `tests/playwright/generate-sx-tests.py` lines 151-158). +Upstream bodies — stripped to assertions: + +| # | Name | Mock | Script | Assertion | +|---|------|------|--------|-----------| +| 1 | **Response can be converted to JSON via as JSON** | `/test` → 200, JSON `{name:"Joe"}` | `fetch /test as Response then put (it as JSON).name into me` | text = `"Joe"` | +| 2 | **as response does not throw on 404** | `/test` → 404, body `not found` | `fetch /test as response then put it.status into me` | text = `"404"` | +| 3 | **can catch an error that occurs when using fetch** | `/test` → `route.abort()` (network error) | `fetch /test catch e log e put "yay" into me` | text = `"yay"` | +| 4 | **do not throw passes through 404 response** | `/test` → 404, body `the body` | `fetch /test do not throw then put it into me` | text = `"the body"` | +| 5 | **don't throw passes through 404 response** | idem, apostrophe form | `fetch /test don't throw then put it into me` | text = `"the body"` | +| 6 | **throws on non-2xx response by default** | `/test` → 404, body `not found` | `fetch /test catch e put "caught" into me` | text = `"caught"` | +| 7 | **triggers an event just before fetching** | `/test` → 200, `yay`; listener on `hyperscript:beforeFetch` sets target class to `foo-set` | `on click fetch "/test" then put it into my.innerHTML end` | initial: no `foo-set`; after click: has `foo-set` AND text `yay` | + +## 2. Upstream semantics + +**`hyperscript:beforeFetch`** — dispatched on the element running the feature, +just before `window.fetch()` is called. Upstream `CustomEvent` detail: +`{ resolve, reject, url, options, response? }` — handlers may mutate +`options`, call `resolve(value)` to short-circuit, or `reject(err)` to fail. +The test only checks that the event fires on the element; detail fields are +not asserted. We implement the event but only `cancelable` + mutable +`options` matter for conformance. + +**Non-2xx handling** — upstream default is: when `response.ok === false`, the +command throws `FetchError` carrying the response. Two opt-outs: +- `as response` / `as Response` — yields the Response wrapper; no throw. +- `do not throw` / `don't throw` — after the conversion, if there was an HTTP + error, still return the converted body (whichever format was chosen). + +**Network errors** (route abort, DNS fail) — always thrown regardless of +`do not throw`. `catch e` handles them. + +**Response API** tests rely on: `.ok`, `.status`, `.url`, and **chained +conversion** — `(it as JSON)` when `it` is a Response parses the body as +JSON. Upstream `fetch /test as Response then … (it as JSON)` reuses the +Response's body; our wrapper must remember the raw body. + +## 3. Proposed runtime shape + +### 3a. Response wrapper (SX dict, not host Response) + +``` +{ :_hs-response true + :ok + :status + :url + :headers + :_body ;; raw text, always kept + :_json ;; raw JSON string (for re-parse on `as JSON`) + :_html } +``` + +A single dict representation is enough; the kernel already passes dicts +through as hs values. No new FFI type. + +### 3b. `hs-fetch` restructured + +Pseudocode (design only — no emission here): + +``` +(define hs-fetch + (fn (url format opts do-not-throw target) + ;; 1. Dispatch beforeFetch event on `target` element. + ;; 2. If event.defaultPrevented → return nil. + ;; 3. Call (perform (io-fetch url "response" opts)) — always fetch + ;; the full wrapper, regardless of `format`. + ;; 4. If the host op returned a :_network-error key → raise a + ;; FetchError (catchable by hyperscript `catch`). + ;; 5. If (not :ok) and (not do-not-throw) and format ≠ "response" → + ;; raise FetchError carrying the wrapper. + ;; 6. Convert by format: + ;; "response" → return the wrapper as-is + ;; "json" → (host-call JSON "parse" (:_json wrapper)) → hs-host-to-sx + ;; "html" → io-parse-html on (:_html wrapper) → fragment + ;; "text"/other → (:_body wrapper) + ;; `do not throw` only matters if (not :ok); the value is still + ;; whatever the conversion yields for the real body.)) +``` + +### 3c. Chained `as JSON` on a Response (test 1) + +`(it as Response) as JSON` — hyperscript's `as` operator needs a +Response-aware branch: if `(dict? x) and (get x :_hs-response)` and target +type is `JSON`, call `host JSON.parse` on `:_json`. Add a clause to the +existing `as` / conversion dispatcher in `runtime.sx`. + +### 3d. Error propagation + +Introduce `hs-fetch-error` — a dict `{:_hs-error :FetchError :response +:message "..."}`. Hyperscript `catch` already binds the raised value to the +catch symbol; tests `can catch …` and `throws on non-2xx …` both just +`put "yay" into me` / `"caught"` so the error's *shape* is uninspected. + +The compiler / parser need two keyword paths: + +- `do not throw` / `don't throw` — tokenizer already tokenizes `don't` as + `don` `'t` (two tokens) or as a single `don't`; verify and emit a + `:do-not-throw` flag into the fetch AST. +- `as Response` / `as response` — already handled (runtime line 763). + +### 3e. `beforeFetch` event + +Use the existing DOM event plumbing. Add a primitive call: +`(host-new "CustomEvent" "hyperscript:beforeFetch" {:detail {...} +:bubbles true :cancelable true})`, then `dispatchEvent` on the fetch +target. Listen for `defaultPrevented` before proceeding. + +## 4. Test mock strategy + +Stay in-process. Extend `tests/hs-run-filtered.js`: + +### 4a. Test-name-keyed route table + +Add a `_fetchScripts` map keyed by test name. The runner already tracks +`_current-test-name` (used for the `ask`/`answer` cluster, cluster 28). +Each entry describes the `/test` route for that test: + +```js +const _fetchScripts = { + "as response does not throw on 404": + { "/test": { status: 404, body: "not found" } }, + "do not throw passes through 404 response": + { "/test": { status: 404, body: "the body" } }, + "don't throw passes through 404 response": + { "/test": { status: 404, body: "the body" } }, + "throws on non-2xx response by default": + { "/test": { status: 404, body: "not found" } }, + "Response can be converted to JSON via as JSON": + { "/test": { status: 200, body: '{"name":"Joe"}', + contentType: "application/json" } }, + "can catch an error that occurs when using fetch": + { "/test": { networkError: true } }, + "triggers an event just before fetching": + { "/test": { status: 200, body: "yay", + contentType: "text/html" } }, +}; +``` + +`_mockFetch(url)` / the `io-fetch` branch of `_driveAsync` reads +`_fetchScripts[_currentTestName]?.[url]` first, falls back to +`_fetchRoutes[url]`. Network-error entries set `networkError: true` → the +io-fetch handler resumes with `{ _network_error: true, message: "aborted" }` +so `hs-fetch` can raise. + +### 4b. `window.addEventListener('hyperscript:beforeFetch', …)` support + +The JSDOM-lite `El` in the runner already implements +`addEventListener`/`dispatchEvent` on elements. `window` in the runner is a +plain object — we need a minimal `window.addEventListener` that records +listeners and fires when `hs-fetch` dispatches the event. Either: + +- Option A: make `window` an `El` instance (cheap, uniform). +- Option B: add a tiny event-target shim on `window` with the usual methods. + +Option A is smaller. The test-7 listener is registered on `window`, but +since `bubbles: true`, we can also dispatch on `document.body` if `window` +routing is awkward; upstream dispatches on the element and bubbles. + +### 4c. No real HTTP + +All fetches remain synchronous-resume via `_driveAsync`. Zero network. + +## 5. Test delta estimate (per-test) + +All seven are currently `SKIP` and count zero. Converting them means +(a) removing from `SKIP_TEST_NAMES`, (b) teaching the generator to emit +them, (c) runtime + mock changes. Each is mechanically independent: + +| # | Test | Risk | Est | +|---|------|------|-----| +| 1 | Response → JSON chain | Compiler: chained `as`; runtime: `:_json` re-parse | +1 | +| 2 | `as response` on 404 | Runtime: don't throw when format=response | +1 | +| 3 | catch abort | Runtime: raise on network error; mock: networkError flag | +1 | +| 4 | `do not throw` + 404 | Tokenizer/parser: multi-word modifier; runtime: flag | +1 | +| 5 | `don't throw` + 404 | Tokenizer: apostrophe contraction accepted | +1 | +| 6 | non-2xx throws default | Runtime: throw when !ok && !format=response && !do-not-throw | +1 | +| 7 | beforeFetch event | Runtime: dispatch event on target; runner: `window` listener | +1 | + +Total: +7. Plus possible knock-on fixes in neighbouring non-skipped +tests — the existing "can do a simple fetch with a response object" test +(line 7177) currently passes with a thin mock; the restructure must keep +it green. + +## 6. Risks + +- **Cluster-1 JSON unwrap collision.** `hs-fetch` today unwraps JSON via + `hs-host-to-sx raw` when `fmt = "json"`. The new wrapper keeps `:_json` + as a raw string; the existing `as json` path must now parse `:_json` + through host JSON before calling `hs-host-to-sx`. Failure mode: doubled + parse or raw-string leak into tests `can do a simple fetch w/ json*` + (currently green — regression risk). +- **Generator coverage.** Five of the seven upstream bodies use + `page.route`/`route.fulfill`/`route.abort` — the generator currently + gives up on these. Need a new translator step that recognises + `page.route('**/test', …)` and emits a `(set-fetch-script! "...")` + preamble so `_driveAsync` picks up the right response. This is a small + generator change, not a deep rewrite. +- **Request interception** for the `beforeFetch` test requires + `window.addEventListener` on a JSDOM-lite host — new surface; keep the + shim minimal (addEventListener/removeEventListener/dispatchEvent only). +- **Tokenizer for `don't`.** Apostrophe handling in the HS tokenizer is + fragile (see `hs-tokenizer.sx`). May require a targeted fix so `don't` + tokenises as one word in the fetch-modifier position. +- **Catchable errors.** Hyperscript `catch` flow in our runtime currently + catches host exceptions; confirm raising a dict via `(raise {...})` + propagates into the catch handler and binds as `e`. +- **`hyperscript:beforeFetch` bubble path.** Upstream dispatches on the + element; the test listener is on `window`. Element→document→window + bubbling in our JSDOM-lite is partial; simplest fix is to dispatch on + the element *and* invoke any `window` listeners for that event type. + +## 7. Implementation checklist (commit-sized) + +1. **Design merge** (this doc). One commit, no code. +2. **Runner: `_fetchScripts` + `_currentTestName` plumbing.** Extend + `tests/hs-run-filtered.js` only; add a self-test row in the mock + scripts. No generator / runtime changes yet. +3. **Runner: `window.addEventListener` shim** + dispatch relay from + element events. Add a smoke assertion. +4. **Generator: recognise `page.route` / `route.fulfill` / `route.abort`.** + Emit `(set-fetch-script! …)` SX form; remove matched tests from + `SKIP_TEST_NAMES` incrementally (one per commit below). +5. **Runtime: response wrapper + reshape `hs-fetch`.** Keep existing + `as json`/`as text`/naked paths green; un-skip **test 2** (`as response` + on 404) first — smallest surface. Commit: `HS: fetch response wrapper + preserves status on 404 (+1)`. +6. **Runtime: chained `as JSON` on Response.** Un-skip **test 1**. + Commit: `HS: as JSON on Response wrapper (+1)`. +7. **Tokenizer / parser: `do not throw` modifier.** Un-skip **test 4**. + Commit: `HS: fetch do-not-throw modifier (+1)`. +8. **Tokenizer: `don't throw` contraction.** Un-skip **test 5**. + Commit: `HS: fetch dont-throw contraction (+1)`. +9. **Runtime: throw on non-2xx default.** Un-skip **test 6**. Risk: the + default-behaviour change could break unrelated `fetch /test` tests if + any current routes are non-2xx. Audit `_fetchRoutes`. Commit: + `HS: fetch throws on non-2xx default (+1)`. +10. **Runtime: raise on network error + runner networkError flag.** + Un-skip **test 3**. Commit: `HS: fetch catch network error (+1)`. +11. **Runtime: dispatch `hyperscript:beforeFetch`.** Un-skip **test 7**. + Commit: `HS: hyperscript:beforeFetch event (+1)`. +12. **Sync WASM staging** after each runtime change: + `cp lib/hyperscript/runtime.sx shared/static/wasm/sx/hs-runtime.sx` + (per plan rule 4). +13. **Scoreboard + plan log** in the same commit as each code commit + (plan rule 11). + +Total commits: 1 design + 2 infrastructure + 7 test-unlocks + 1 wrap-up +scoreboard summary = ~10-11 commits. Each unlock is independently +committable and safely revertible.