rose-ash/plans/hs-conformance-to-100.md

# Hyperscript conformance → 100%

Goal: take the hyperscript upstream conformance suite from **1213/1496 (81%)** to a clean 100%. Queue-driven — single-agent loop on `architecture` branch, one cluster per commit.

## North star

```
Baseline: 1213/1496 (81.1%)
Target:   1496/1496
Gap:      283 tests  (130 real fails + 153 SKIPs)
```

Track after each iteration via `mcp__hs-test__hs_test_run` on the relevant suite, not the whole thing (full runs take 10+min and include hanging tests — 196/199/200/615/1197/1198 hang under the 200k step limit).

## How to run tests

```
mcp__hs-test__hs_test_run(suite="hs-upstream-<cluster>")        # fastest, one suite
mcp__hs-test__hs_test_run(start=0, end=195)                     # early range
mcp__hs-test__hs_test_run(start=201, end=614)                   # mid range (skip hypertrace hangs)
mcp__hs-test__hs_test_run(start=616, end=1196)                  # late-1, skip repeat-forever hangs
mcp__hs-test__hs_test_run(start=1199)                           # late-2 after hangs
```

## File layout

Runtime/compiler/parser live in `lib/hyperscript/*.sx`. The test runner at `tests/hs-run-filtered.js` loads `shared/static/wasm/sx/hs-*.sx` — **after every `.sx` edit you must `cp lib/hyperscript/<file>.sx shared/static/wasm/sx/hs-<file>.sx`**.

The test fixtures live in `spec/tests/test-hyperscript-behavioral.sx`, generated from `tests/playwright/generate-sx-tests.py`. **Never edit the behavioral.sx fixture directly** — fix the generator or the runtime.

## Cluster queue

Each cluster below is one commit. Order is rough — a loop agent may skip ahead if a predecessor is blocked. **Status:** `pending` / `in-progress` / `done (+N)` / `blocked (<reason>)`.

### Bucket A: runtime fixes, single-file (low risk, high yield)

1. **[done (+4)] fetch JSON unwrap** — `hs-upstream-fetch` 4 tests (`can do a simple fetch w/ json` + 3 variants) got `{:__host_handle N}`. Root: `hs-fetch` in `runtime.sx` returns raw host Response object instead of parsing JSON body. Fix: when format is `"json"`, unwrap via `host-get "_json"` and `json-parse`. Expected: +4.

2. **[done (+1)] element → HTML via outerHTML** — `asExpression / converts an element into HTML` (1 test) + unlocks response fetches. Mock DOM `El` class in `tests/hs-run-filtered.js` has no `outerHTML` getter. Add a getter computed from `tagName` + `attributes` + `children` (recurse). Expected: +1 direct, + knock-on in fetch.

3. **[done (+2)] Values dict insertion order** — `asExpression / Values | FormEncoded` + `| JSONString` (2 tests) — form fields come out `lastName, phone, firstName, areaCode`. Root: `hs-values-absorb` in `runtime.sx` uses `dict-set!` but keys iterate in non-insertion order. Investigate `hs-gather-form-nodes` walk — the recursive `kids` traversal silently fails when `children` is a JS Array (not sx-list), so nested inputs arrive via a different path. Fix: either coerce children to sx-list at the gather boundary OR rewrite gather to explicitly use sx-level iteration helpers. Expected: +2.

4. **[done (+3)] `not` precedence over `or`** — `expressions/not` 3 tests (`not has higher precedence than or`, `not with numeric truthy/falsy`, `not with string truthy/falsy`). Check parser precedence — `not` should bind tighter than `or`. Fix in `parser.sx` expression-level precedence. Expected: +3.

5. **[done (+1)] `some` selector for nonempty match** — `expressions/some / some returns true for nonempty selector` (1 test). `some .class` probably returns the list, not a boolean. Runtime fix. Expected: +1.

6. **[done (+2)] string template `${x}`** — `expressions/strings / string templates work w/ props` + `w/ braces` (2 tests). Template interpolation isn't substituting property accesses. Check `hs-template` runtime. Expected: +2.

7. **[in-progress] `put` hyperscript reprocessing** — `put / properly processes hyperscript at end/start/content/symbol` (4 tests, all `Expected 42, got 40`). After a put operation, newly inserted HS scripts aren't being activated. Fix: `hs-put-at!` should `hs-boot-subtree!` on the target after DOM insertion. Expected: +4.

8. **[pending] `select returns selected text`** (1 test, `hs-upstream-select`). Likely `select` command needs to return `window.getSelection().toString()` equivalent. Add host-call to selection API in mock. Expected: +1.

9. **[pending] `wait on event` basics** — `wait / can wait on event`, `on another element`, `waiting ... sets it to the event`, `destructure properties in a wait` (4 tests). Event-waiter suspension issue. Expected: +3-4.

10. **[pending] `swap` variable ↔ property** — `swap / can swap a variable with a property` (1 test). Swap command doesn't handle mixed var/prop targets. Expected: +1.

11. **[pending] `hide` strategy** — `hide / can configure hidden as default`, `can hide with custom strategy`, `can set default to custom strategy`, `hide element then show element retains original display` (4 tests). Strategy config plumbing. Expected: +3-4.

12. **[pending] `show` multi-element + display retention** — `show / can show multiple elements with inline-block`, `can filter over a set of elements using the its symbol` (2 tests). Expected: +2.

13. **[pending] `toggle` multi-class + timed + until-event** — `toggle` (3 assertion-fail tests). Expected: +3.

14. **[pending] `unless` modifier** — `unlessModifier / unless can conditionally execute` (1 test). Parser/compiler addition. Expected: +1.

15. **[pending] `transition` query-ref + multi-prop + initial** — `transition` 3 tests. Expected: +2-3.

16. **[pending] `send can reference sender`** — 1 assertion fail. Expected: +1.

17. **[pending] `tell` semantics** — `tell / attributes refer to the thing being told`, `does not overwrite me symbol`, `your symbol represents thing being told` (3 tests). Expected: +3.

18. **[pending] `throw respond async/sync`** — `throw / can respond to async/sync exceptions in event handler` (2 tests). Expected: +2.

### Bucket B: parser/compiler additions (medium risk, shared files)

19. **[pending] `pick` regex + indices** — `pick` 13 tests. Regex match, flags, `of` syntax, start/end, negative indices. Big enough that a single commit might fail — break into pick-regex and pick-indices if needed. Expected: +10-13.

20. **[pending] `repeat` property for-loops + where** — `repeat / basic property for loop`, `can nest loops`, `where clause can use the for loop variable name` (3 tests). Expected: +3.

21. **[pending] `possessiveExpression` property access via its** — `possessive / can access its properties` (1 test, Expected `foo` got ``). Expected: +1.

22. **[pending] window global fn fallback** — `regressions / can invoke functions w/ numbers in name` + unlocks several others. When calling `foo()` where `foo` isn't SX-defined, fall back to `(host-global "foo")`. Design decision: either compile-time emit `(or foo (host-global "foo"))` via a helper, or add runtime lookup in the dispatch path. Expected: +2-4.

23. **[pending] `me symbol works in from expressions`** — `regressions` (1 test, Expected `Foo`). Check `from` expression compilation. Expected: +1.

24. **[pending] `properly interpolates values 2`** — URL interpolation regression (1 test). Likely template string + property access. Expected: +1.

25. **[pending] `can support parenthesized commands and features`** — `parser` (1 test, Expected `clicked`). Parser needs to accept `(cmd...)` grouping in more contexts. Expected: +1.

### Bucket C: feature stubs (DOM observer mocks)

26. **[pending] resize observer mock + `on resize`** — 3 tests. Add a minimal `ResizeObserver` mock to `hs-run-filtered.js`, plus parse/compile `on resize`. Expected: +3.

27. **[pending] intersection observer mock + `on intersection`** — 3 tests. Mock `IntersectionObserver`; compile `on intersection` with margin/threshold modifiers. Expected: +3.

28. **[pending] `ask`/`answer` + prompt/confirm mock** — `askAnswer` 4 tests. **Requires test-name-keyed mock**: first test wants `confirm → true`, second `confirm → false`, third `prompt → "Alice"`, fourth `prompt → null`. Keyed via `_current-test-name` in the runner. Expected: +4.

29. **[pending] `hyperscript:before:init` / `:after:init` / `:parse-error` events** — 6 tests in `bootstrap` + `parser`. Fire DOM events at activation boundaries. Expected: +4-6.

30. **[pending] `logAll` config** — 1 test. Global config that console.log's each command. Expected: +1.

### Bucket D: medium features (bigger commits, plan-first)

31. **[pending] runtime null-safety error reporting** — 18 tests in `runtimeErrors`. When accessing `.foo` on nil, emit a structured error with position info. One coordinated fix in the compiler emit paths for property access, function calls, set/put. Expected: +15-18.

32. **[pending] MutationObserver mock + `on mutation` dispatch** — 15 tests in `on`. Add MO mock to runner. Compile `on mutation [of attribute/childList/attribute-specific]`. Expected: +10-15.

33. **[pending] cookie API** — 5 tests in `expressions/cookies`. `document.cookie` mock in runner + `the cookies` + `set the xxx cookie` keywords. Expected: +5.

34. **[pending] event modifier DSL** — 8 tests in `on`. `elsewhere`, `every`, `first click`, count filters (`once / twice / 3 times`, ranges), `from elsewhere`. Expected: +6-8.

35. **[pending] namespaced `def`** — 3 tests. `def ns.foo() ...` creates `ns.foo`. Expected: +3.

### Bucket E: subsystems (DO NOT LOOP — human-driven)

36. **[blocked: needs design] WebSocket + `socket` + rpc proxy** — 16 tests. Ship only with intentional design review.

37. **[blocked: needs design] Tokenizer-as-API** — 17 tests. Expose tokens as inspectable SX data.

38. **[blocked: needs design] SourceInfo API** — 4 tests. `(get line N)` / `(get source N)` metadata on compiled AST.

39. **[blocked: needs design] WebWorker plugin** — 1 test.

40. **[blocked: needs design] Fetch non-2xx / before-fetch event / real response object** — 7 tests. Sinon-level route mocks or real fetch interception.

### Bucket F: generator translation gaps (after bucket A-D)

Many tests are `SKIP (untranslated)` because `tests/playwright/generate-sx-tests.py` bailed with `return None`. These need patches to the generator to recognize more JS test patterns. Estimated ~25 recoverable tests. Defer to a dedicated generator-repair cluster once the queue above drains.

---

## Ground rules for the loop agent

1. **One cluster per commit.** Don't batch. Short commit message: `HS: <cluster name> (+N tests)`.
2. **Baseline first, verify at the end.** Before starting: record the current pass count for the target suite AND for one smoke range (0-195). After fixing: rerun both. Abort and mark blocked if:
   - Target suite didn't improve by at least +1.
   - Smoke range regressed (any test flipped pass → fail).
3. **Never edit `.sx` files with `Edit`/`Read`/`Write`.** Use sx-tree MCP (`sx_read_subtree`, `sx_replace_node`, `sx_insert_child`, `sx_insert_near`, `sx_replace_by_pattern`, `sx_rename_symbol`, `sx_validate`, `sx_write_file`).
4. **Sync WASM staging.** After every edit to `lib/hyperscript/<f>.sx`, run `cp lib/hyperscript/<f>.sx shared/static/wasm/sx/hs-<f>.sx`.
5. **Never edit `spec/tests/test-hyperscript-behavioral.sx` directly.** Fix the generator or the runtime.
6. **Scope:** `lib/hyperscript/**`, `shared/static/wasm/sx/hs-*`, `tests/hs-run-filtered.js`, `tests/playwright/generate-sx-tests.py`, `plans/hs-conformance-to-100.md`. Do not touch `spec/evaluator.sx`, the broader SX kernel, or unrelated files.
7. **Commit even partial fixes.** If you get +N where N is less than expected, commit what you have and mark the cluster `done (+N) — partial, <what's left>`.
8. **If stuck >30min on a cluster**, mark it `blocked (<reason>)` in the plan and move to the next pending cluster.
9. **Branch: `architecture`.** Commit locally. Never push. Never touch `main`.
10. **Log every iteration** in the Progress log below: one paragraph, what you touched, delta, commit SHA.

## Known gotchas

- `env-bind!` creates bindings; `env-set!` mutates existing ones.
- SX `do` is R7RS iteration — use `begin` for multi-expr sequences.
- `cond` / `when` / `let` clause bodies evaluate only the last expr — wrap in `begin`.
- `list?` in SX checks for `{_type:'list'}` — it returns **false** on raw JS Arrays. `host-get node "children"` returns a JS Array in the mock, so recursion via `(list? kids)` silently drops nested elements.
- `append!` on a list-valued scoped var (`:s`) requires `emit-set` in the compiler — done, see commit 1613f551.
- When symbol target is `the-result`, also sync `it` (done, see emit-set).
- Hypertrace tests (196, 199, 200) and query-template test (615) hang under 200k step limit — always filter around them.
- `repeat forever` tests (1197, 1198) also hang.

## Progress log

(Reverse chronological — newest at top.)

### 2026-04-23 — cluster 6 string template ${x}
- **108e25d4** — `HS: string template ${x} (+2 tests)`. Two-part fix: (a) `compiler.sx` now emits `(host-global "window")` (plus other well-known globals) for bare dot-chain base identifiers that would otherwise be unbound symbols. (b) `generate-sx-tests.py` now has `eval-hs-locals` ALSO call `host-set!` on `window.<name>` for each binding, so tests whose `window.X = Y` setup was translated as a local pair can still see `window.X`. Suite hs-upstream-expressions/strings: 5/8 → 7/8. Smoke 0-195: 162/195 unchanged.

### 2026-04-23 — cluster 5 some selector for nonempty match
- **e7b86264** — `HS: some selector for nonempty match (+1 test)`. `some <html/>` → `(not (hs-falsy? (hs-query-first "html")))` → `document.querySelector('html')`. Mock's querySelector searched from `_body`, missing the `_html` element. Fixed the mock to short-circuit for `html`/`body` and walk `documentElement`. Suite hs-upstream-expressions/some: 5/6 → 6/6. Smoke 0-195: 162/195 unchanged.

### 2026-04-23 — cluster 4 not precedence over or
- **4fe0b649** — `HS: not precedence over or + truthy/falsy coercion (+3 tests)`. `parse-atom`'s `not` branch emitted `(not (parse-expr))`, which let or/and capture the whole RHS, and also used SX's `not` which treats only nil/false as falsy. Fixed to emit `(hs-falsy? (parse-atom))` — tight binding + hyperscript truthiness (0, "", nil, false, []). Suite hs-upstream-expressions/not: 6/9 → 9/9. Smoke 0-195: 162/195 unchanged.

### 2026-04-23 — cluster 3 Values dict insertion order
- **e59c0b8e** — `HS: Values dict insertion order (+2 tests)`. Root cause was the OCaml kernel's dict implementation iterating keys in scrambled (non-insertion) order. Added `_order` hidden list tracked by `hs-values-absorb`, and taught `hs-coerce` FormEncoded/JSONString branches to iterate via `_order` when present (filtering the `_order` marker out). Suite hs-upstream-expressions/asExpression: 28/42 → 30/42. Smoke 0-195: 162/195 unchanged.

### 2026-04-23 — cluster 2 element→HTML via outerHTML
- **e195b5bd** — `HS: element → HTML via outerHTML (+1 test)`. Added an `outerHTML` getter on the mock `El` class in `tests/hs-run-filtered.js`. Merges `.id`/`.className` (host-set! targets) with `.attributes`, falls back to `innerText`/`textContent`. Suite hs-upstream-expressions/asExpression: 27/42 → 28/42. Smoke 0-195: 162/195 unchanged.

### 2026-04-23 — cluster 1 fetch JSON unwrap
- **39a597e9** — `HS: fetch JSON unwrap (+4 tests)`. Added `hs-host-to-sx` helper in `runtime.sx` that converts raw host-handle JS objects/arrays to proper SX dicts/lists via Object.keys/Array walks. `hs-fetch` now calls it on the result when format is `"json"`. Detects host-handle dicts by checking `(host-get v "_type") == "dict"` — genuine SX dicts have the marker, host handles don't. Suite hs-upstream-fetch: 11/23 → 15/23. Smoke 0-195: 162/195 unchanged.

### 2026-04-23 — cluster fixes session baseline
- **6b0334af** — `HS: remove bare @attr, set X @attr, JSON clean, FormEncoded, HTML join` (+3)
- **1613f551** — `HS add/append: Set dedup, @attr support, when-clause result tracking` (+6)
- Pre-loop baseline: 1213/1496 (81.1%).