HS-design: E40 real fetch + before-fetch + non-2xx

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-24 06:54:21 +00:00
parent 67d4b9dae5
commit 6b7559fcaf
1 changed files with 256 additions and 0 deletions
--- a/plans/designs/e40-real-fetch.md
+++ b/plans/designs/e40-real-fetch.md
@@ -0,0 +1,256 @@
+# E40 — Fetch non-2xx + `hyperscript:beforeFetch` + real Response
+
+Cluster 40 of `plans/hs-conformance-to-100.md`. Target: +7 tests.
+
+## 1. Failing tests
+
+All seven currently emit `SKIP (skip-list)` from the generator
+(`SKIP_TEST_NAMES` in `tests/playwright/generate-sx-tests.py` lines 151-158).
+Upstream bodies — stripped to assertions:
+
+| # | Name | Mock | Script | Assertion |
+|---|------|------|--------|-----------|
+| 1 | **Response can be converted to JSON via as JSON** | `/test` → 200, JSON `{name:"Joe"}` | `fetch /test as Response then put (it as JSON).name into me` | text = `"Joe"` |
+| 2 | **as response does not throw on 404** | `/test` → 404, body `not found` | `fetch /test as response then put it.status into me` | text = `"404"` |
+| 3 | **can catch an error that occurs when using fetch** | `/test` → `route.abort()` (network error) | `fetch /test catch e log e put "yay" into me` | text = `"yay"` |
+| 4 | **do not throw passes through 404 response** | `/test` → 404, body `the body` | `fetch /test do not throw then put it into me` | text = `"the body"` |
+| 5 | **don't throw passes through 404 response** | idem, apostrophe form | `fetch /test don't throw then put it into me` | text = `"the body"` |
+| 6 | **throws on non-2xx response by default** | `/test` → 404, body `not found` | `fetch /test catch e put "caught" into me` | text = `"caught"` |
+| 7 | **triggers an event just before fetching** | `/test` → 200, `yay`; listener on `hyperscript:beforeFetch` sets target class to `foo-set` | `on click fetch "/test" then put it into my.innerHTML end` | initial: no `foo-set`; after click: has `foo-set` AND text `yay` |
+
+## 2. Upstream semantics
+
+**`hyperscript:beforeFetch`** — dispatched on the element running the feature,
+just before `window.fetch()` is called. Upstream `CustomEvent` detail:
+`{ resolve, reject, url, options, response? }` — handlers may mutate
+`options`, call `resolve(value)` to short-circuit, or `reject(err)` to fail.
+The test only checks that the event fires on the element; detail fields are
+not asserted. We implement the event but only `cancelable` + mutable
+`options` matter for conformance.
+
+**Non-2xx handling** — upstream default is: when `response.ok === false`, the
+command throws `FetchError` carrying the response. Two opt-outs:
+- `as response` / `as Response` — yields the Response wrapper; no throw.
+- `do not throw` / `don't throw` — after the conversion, if there was an HTTP
+  error, still return the converted body (whichever format was chosen).
+
+**Network errors** (route abort, DNS fail) — always thrown regardless of
+`do not throw`. `catch e` handles them.
+
+**Response API** tests rely on: `.ok`, `.status`, `.url`, and **chained
+conversion** — `(it as JSON)` when `it` is a Response parses the body as
+JSON. Upstream `fetch /test as Response then … (it as JSON)` reuses the
+Response's body; our wrapper must remember the raw body.
+
+## 3. Proposed runtime shape
+
+### 3a. Response wrapper (SX dict, not host Response)
+
+```
+{ :_hs-response true
+  :ok       <bool>
+  :status   <int>
+  :url      <string>
+  :headers  <dict>
+  :_body    <string>      ;; raw text, always kept
+  :_json    <string|nil>  ;; raw JSON string (for re-parse on `as JSON`)
+  :_html    <string|nil> }
+```
+
+A single dict representation is enough; the kernel already passes dicts
+through as hs values. No new FFI type.
+
+### 3b. `hs-fetch` restructured
+
+Pseudocode (design only — no emission here):
+
+```
+(define hs-fetch
+  (fn (url format opts do-not-throw target)
+    ;; 1. Dispatch beforeFetch event on `target` element.
+    ;; 2. If event.defaultPrevented → return nil.
+    ;; 3. Call (perform (io-fetch url "response" opts)) — always fetch
+    ;;    the full wrapper, regardless of `format`.
+    ;; 4. If the host op returned a :_network-error key → raise a
+    ;;    FetchError (catchable by hyperscript `catch`).
+    ;; 5. If (not :ok) and (not do-not-throw) and format ≠ "response" →
+    ;;    raise FetchError carrying the wrapper.
+    ;; 6. Convert by format:
+    ;;       "response" → return the wrapper as-is
+    ;;       "json"     → (host-call JSON "parse" (:_json wrapper)) → hs-host-to-sx
+    ;;       "html"     → io-parse-html on (:_html wrapper) → fragment
+    ;;       "text"/other → (:_body wrapper)
+    ;;    `do not throw` only matters if (not :ok); the value is still
+    ;;    whatever the conversion yields for the real body.))
+```
+
+### 3c. Chained `as JSON` on a Response (test 1)
+
+`(it as Response) as JSON` — hyperscript's `as` operator needs a
+Response-aware branch: if `(dict? x) and (get x :_hs-response)` and target
+type is `JSON`, call `host JSON.parse` on `:_json`. Add a clause to the
+existing `as` / conversion dispatcher in `runtime.sx`.
+
+### 3d. Error propagation
+
+Introduce `hs-fetch-error` — a dict `{:_hs-error :FetchError :response <w>
+:message "..."}`. Hyperscript `catch` already binds the raised value to the
+catch symbol; tests `can catch …` and `throws on non-2xx …` both just
+`put "yay" into me` / `"caught"` so the error's *shape* is uninspected.
+
+The compiler / parser need two keyword paths:
+
+- `do not throw` / `don't throw` — tokenizer already tokenizes `don't` as
+  `don` `'t` (two tokens) or as a single `don't`; verify and emit a
+  `:do-not-throw` flag into the fetch AST.
+- `as Response` / `as response` — already handled (runtime line 763).
+
+### 3e. `beforeFetch` event
+
+Use the existing DOM event plumbing. Add a primitive call:
+`(host-new "CustomEvent" "hyperscript:beforeFetch" {:detail {...}
+:bubbles true :cancelable true})`, then `dispatchEvent` on the fetch
+target. Listen for `defaultPrevented` before proceeding.
+
+## 4. Test mock strategy
+
+Stay in-process. Extend `tests/hs-run-filtered.js`:
+
+### 4a. Test-name-keyed route table
+
+Add a `_fetchScripts` map keyed by test name. The runner already tracks
+`_current-test-name` (used for the `ask`/`answer` cluster, cluster 28).
+Each entry describes the `/test` route for that test:
+
+```js
+const _fetchScripts = {
+  "as response does not throw on 404":
+      { "/test": { status: 404, body: "not found" } },
+  "do not throw passes through 404 response":
+      { "/test": { status: 404, body: "the body" } },
+  "don't throw passes through 404 response":
+      { "/test": { status: 404, body: "the body" } },
+  "throws on non-2xx response by default":
+      { "/test": { status: 404, body: "not found" } },
+  "Response can be converted to JSON via as JSON":
+      { "/test": { status: 200, body: '{"name":"Joe"}',
+                   contentType: "application/json" } },
+  "can catch an error that occurs when using fetch":
+      { "/test": { networkError: true } },
+  "triggers an event just before fetching":
+      { "/test": { status: 200, body: "yay",
+                   contentType: "text/html" } },
+};
+```
+
+`_mockFetch(url)` / the `io-fetch` branch of `_driveAsync` reads
+`_fetchScripts[_currentTestName]?.[url]` first, falls back to
+`_fetchRoutes[url]`. Network-error entries set `networkError: true` → the
+io-fetch handler resumes with `{ _network_error: true, message: "aborted" }`
+so `hs-fetch` can raise.
+
+### 4b. `window.addEventListener('hyperscript:beforeFetch', …)` support
+
+The JSDOM-lite `El` in the runner already implements
+`addEventListener`/`dispatchEvent` on elements. `window` in the runner is a
+plain object — we need a minimal `window.addEventListener` that records
+listeners and fires when `hs-fetch` dispatches the event. Either:
+
+- Option A: make `window` an `El` instance (cheap, uniform).
+- Option B: add a tiny event-target shim on `window` with the usual methods.
+
+Option A is smaller. The test-7 listener is registered on `window`, but
+since `bubbles: true`, we can also dispatch on `document.body` if `window`
+routing is awkward; upstream dispatches on the element and bubbles.
+
+### 4c. No real HTTP
+
+All fetches remain synchronous-resume via `_driveAsync`. Zero network.
+
+## 5. Test delta estimate (per-test)
+
+All seven are currently `SKIP` and count zero. Converting them means
+(a) removing from `SKIP_TEST_NAMES`, (b) teaching the generator to emit
+them, (c) runtime + mock changes. Each is mechanically independent:
+
+| # | Test | Risk | Est |
+|---|------|------|-----|
+| 1 | Response → JSON chain | Compiler: chained `as`; runtime: `:_json` re-parse | +1 |
+| 2 | `as response` on 404 | Runtime: don't throw when format=response | +1 |
+| 3 | catch abort | Runtime: raise on network error; mock: networkError flag | +1 |
+| 4 | `do not throw` + 404 | Tokenizer/parser: multi-word modifier; runtime: flag | +1 |
+| 5 | `don't throw` + 404 | Tokenizer: apostrophe contraction accepted | +1 |
+| 6 | non-2xx throws default | Runtime: throw when !ok && !format=response && !do-not-throw | +1 |
+| 7 | beforeFetch event | Runtime: dispatch event on target; runner: `window` listener | +1 |
+
+Total: +7. Plus possible knock-on fixes in neighbouring non-skipped
+tests — the existing "can do a simple fetch with a response object" test
+(line 7177) currently passes with a thin mock; the restructure must keep
+it green.
+
+## 6. Risks
+
+- **Cluster-1 JSON unwrap collision.** `hs-fetch` today unwraps JSON via
+  `hs-host-to-sx raw` when `fmt = "json"`. The new wrapper keeps `:_json`
+  as a raw string; the existing `as json` path must now parse `:_json`
+  through host JSON before calling `hs-host-to-sx`. Failure mode: doubled
+  parse or raw-string leak into tests `can do a simple fetch w/ json*`
+  (currently green — regression risk).
+- **Generator coverage.** Five of the seven upstream bodies use
+  `page.route`/`route.fulfill`/`route.abort` — the generator currently
+  gives up on these. Need a new translator step that recognises
+  `page.route('**/test', …)` and emits a `(set-fetch-script! "...")`
+  preamble so `_driveAsync` picks up the right response. This is a small
+  generator change, not a deep rewrite.
+- **Request interception** for the `beforeFetch` test requires
+  `window.addEventListener` on a JSDOM-lite host — new surface; keep the
+  shim minimal (addEventListener/removeEventListener/dispatchEvent only).
+- **Tokenizer for `don't`.** Apostrophe handling in the HS tokenizer is
+  fragile (see `hs-tokenizer.sx`). May require a targeted fix so `don't`
+  tokenises as one word in the fetch-modifier position.
+- **Catchable errors.** Hyperscript `catch` flow in our runtime currently
+  catches host exceptions; confirm raising a dict via `(raise {...})`
+  propagates into the catch handler and binds as `e`.
+- **`hyperscript:beforeFetch` bubble path.** Upstream dispatches on the
+  element; the test listener is on `window`. Element→document→window
+  bubbling in our JSDOM-lite is partial; simplest fix is to dispatch on
+  the element *and* invoke any `window` listeners for that event type.
+
+## 7. Implementation checklist (commit-sized)
+
+1. **Design merge** (this doc). One commit, no code.
+2. **Runner: `_fetchScripts` + `_currentTestName` plumbing.** Extend
+   `tests/hs-run-filtered.js` only; add a self-test row in the mock
+   scripts. No generator / runtime changes yet.
+3. **Runner: `window.addEventListener` shim** + dispatch relay from
+   element events. Add a smoke assertion.
+4. **Generator: recognise `page.route` / `route.fulfill` / `route.abort`.**
+   Emit `(set-fetch-script! …)` SX form; remove matched tests from
+   `SKIP_TEST_NAMES` incrementally (one per commit below).
+5. **Runtime: response wrapper + reshape `hs-fetch`.** Keep existing
+   `as json`/`as text`/naked paths green; un-skip **test 2** (`as response`
+   on 404) first — smallest surface. Commit: `HS: fetch response wrapper
+   preserves status on 404 (+1)`.
+6. **Runtime: chained `as JSON` on Response.** Un-skip **test 1**.
+   Commit: `HS: as JSON on Response wrapper (+1)`.
+7. **Tokenizer / parser: `do not throw` modifier.** Un-skip **test 4**.
+   Commit: `HS: fetch do-not-throw modifier (+1)`.
+8. **Tokenizer: `don't throw` contraction.** Un-skip **test 5**.
+   Commit: `HS: fetch dont-throw contraction (+1)`.
+9. **Runtime: throw on non-2xx default.** Un-skip **test 6**. Risk: the
+   default-behaviour change could break unrelated `fetch /test` tests if
+   any current routes are non-2xx. Audit `_fetchRoutes`. Commit:
+   `HS: fetch throws on non-2xx default (+1)`.
+10. **Runtime: raise on network error + runner networkError flag.**
+    Un-skip **test 3**. Commit: `HS: fetch catch network error (+1)`.
+11. **Runtime: dispatch `hyperscript:beforeFetch`.** Un-skip **test 7**.
+    Commit: `HS: hyperscript:beforeFetch event (+1)`.
+12. **Sync WASM staging** after each runtime change:
+    `cp lib/hyperscript/runtime.sx shared/static/wasm/sx/hs-runtime.sx`
+    (per plan rule 4).
+13. **Scoreboard + plan log** in the same commit as each code commit
+    (plan rule 11).
+
+Total commits: 1 design + 2 infrastructure + 7 test-unlocks + 1 wrap-up
+scoreboard summary = ~10-11 commits. Each unlock is independently
+committable and safely revertible.