rose-ash/plans/hs-conformance-to-100.md

Hyperscript conformance → 100%

Goal: take the hyperscript upstream conformance suite from 1213/1496 (81%) to a clean 100%. Queue-driven — single-agent loop on architecture branch, one cluster per commit.

North star

Baseline: 1213/1496 (81.1%)
Target:   1496/1496
Gap:      283 tests  (130 real fails + 153 SKIPs)

Track after each iteration via mcp__hs-test__hs_test_run on the relevant suite, not the whole thing (full runs take 10+min and include hanging tests — 196/199/200/615/1197/1198 hang under the 200k step limit).

How to run tests

mcp__hs-test__hs_test_run(suite="hs-upstream-<cluster>")        # fastest, one suite
mcp__hs-test__hs_test_run(start=0, end=195)                     # early range
mcp__hs-test__hs_test_run(start=201, end=614)                   # mid range (skip hypertrace hangs)
mcp__hs-test__hs_test_run(start=616, end=1196)                  # late-1, skip repeat-forever hangs
mcp__hs-test__hs_test_run(start=1199)                           # late-2 after hangs

File layout

Runtime/compiler/parser live in lib/hyperscript/*.sx. The test runner at tests/hs-run-filtered.js loads shared/static/wasm/sx/hs-*.sx — after every .sx edit you must cp lib/hyperscript/<file>.sx shared/static/wasm/sx/hs-<file>.sx.

The test fixtures live in spec/tests/test-hyperscript-behavioral.sx, generated from tests/playwright/generate-sx-tests.py. Never edit the behavioral.sx fixture directly — fix the generator or the runtime.

Cluster queue

Each cluster below is one commit. Order is rough — a loop agent may skip ahead if a predecessor is blocked. Status: pending / in-progress / done (+N) / blocked (<reason>).

Bucket A: runtime fixes, single-file (low risk, high yield)

1. [done (+4)] fetch JSON unwrap — hs-upstream-fetch 4 tests (can do a simple fetch w/ json + 3 variants) got {:__host_handle N}. Root: hs-fetch in runtime.sx returns raw host Response object instead of parsing JSON body. Fix: when format is "json", unwrap via host-get "_json" and json-parse. Expected: +4.

2. [done (+1)] element → HTML via outerHTML — asExpression / converts an element into HTML (1 test) + unlocks response fetches. Mock DOM El class in tests/hs-run-filtered.js has no outerHTML getter. Add a getter computed from tagName + attributes + children (recurse). Expected: +1 direct, + knock-on in fetch.

3. [done (+2)] Values dict insertion order — asExpression / Values | FormEncoded + | JSONString (2 tests) — form fields come out lastName, phone, firstName, areaCode. Root: hs-values-absorb in runtime.sx uses dict-set! but keys iterate in non-insertion order. Investigate hs-gather-form-nodes walk — the recursive kids traversal silently fails when children is a JS Array (not sx-list), so nested inputs arrive via a different path. Fix: either coerce children to sx-list at the gather boundary OR rewrite gather to explicitly use sx-level iteration helpers. Expected: +2.

4. [done (+3)] not precedence over or — expressions/not 3 tests (not has higher precedence than or, not with numeric truthy/falsy, not with string truthy/falsy). Check parser precedence — not should bind tighter than or. Fix in parser.sx expression-level precedence. Expected: +3.

5. [done (+1)] some selector for nonempty match — expressions/some / some returns true for nonempty selector (1 test). some .class probably returns the list, not a boolean. Runtime fix. Expected: +1.

6. [done (+2)] string template ${x} — expressions/strings / string templates work w/ props + w/ braces (2 tests). Template interpolation isn't substituting property accesses. Check hs-template runtime. Expected: +2.

7. [done (+1) — partial, 3 tests remain: inserted-button handler doesn't fire for afterbegin/innerHTML paths; might need targeted trace of hs-boot-subtree! or _setInnerHTML timing] put hyperscript reprocessing — put / properly processes hyperscript at end/start/content/symbol (4 tests, all Expected 42, got 40). After a put operation, newly inserted HS scripts aren't being activated. Fix: hs-put-at! should hs-boot-subtree! on the target after DOM insertion. Expected: +4.

8. [blocked: mock selection state needs setup the JS test does via createRange+setStart+setEnd+addRange; attempted generator pattern + window.getSelection()/createRange mock but result still empty — host-get textContent doesn't propagate through range.setStart stored arg. Would need deeper mock of DOM text nodes with propagation.] select returns selected text (1 test, hs-upstream-select). Likely select command needs to return window.getSelection().toString() equivalent. Add host-call to selection API in mock. Expected: +1.

9. [done (+4)] wait on event basics — wait / can wait on event, on another element, waiting ... sets it to the event, destructure properties in a wait (4 tests). Event-waiter suspension issue. Expected: +3-4.

10. [done (+1)] swap variable ↔ property — swap / can swap a variable with a property (1 test). Swap command doesn't handle mixed var/prop targets. Expected: +1.

11. [done (+3) — partial, hide element then show element retains original display remains; needs on click N count-filtered event handlers, out of scope for this cluster] hide strategy — hide / can configure hidden as default, can hide with custom strategy, can set default to custom strategy, hide element then show element retains original display (4 tests). Strategy config plumbing. Expected: +3-4.

12. [done (+2)] show multi-element + display retention — show / can show multiple elements with inline-block, can filter over a set of elements using the its symbol (2 tests). Expected: +2.

13. [done (+2) — partial, can toggle for a fixed amount of time needs an async mock scheduler (sync io-sleep collapses the toggle/un-toggle into one click frame)] toggle multi-class + timed + until-event — toggle (3 assertion-fail tests). Expected: +3.

14. [blocked: parser unless + compiler hs-unless-wrap produces correct compile output but test still fails with Undefined symbol: _test-result — some interaction with the test harness's _run-test-thunk + _test-result define. Also needs generator classList.add pattern (which was bundled in). Deferred until harness interaction root-caused.] unless modifier — unlessModifier / unless can conditionally execute (1 test). Parser/compiler addition. Expected: +1.

15. [done (+2) — partial, can use initial to transition to original value needs on click N count-filtered events (same sync-mock block as clusters 11/13)] transition query-ref + multi-prop + initial — transition 3 tests. Expected: +2-3.

16. [done (+1)] send can reference sender — 1 assertion fail. Expected: +1.

17. [blocked: tell semantics are subtle — me should stay as the original element for explicit to me writes but the implicit default for bare add .bar inside tell X should be X. Attempted just leaving you/yourself scoped (dropping the me shadow) regressed 4 passing tests (restores proper implicit me, works with an array, etc.) which rely on bare commands using me as told-target. Proper fix requires a beingTold symbol distinct from me, with bare commands compiling to beingTold-or-me and explicit me always the original — more than a 30-min cluster budget.] tell semantics — tell / attributes refer to the thing being told, does not overwrite me symbol, your symbol represents thing being told (3 tests). Expected: +3.

18. [done (+2)] throw respond async/sync — throw / can respond to async/sync exceptions in event handler (2 tests). Expected: +2.

Bucket B: parser/compiler additions (medium risk, shared files)

19. [pending] pick regex + indices — pick 13 tests. Regex match, flags, of syntax, start/end, negative indices. Big enough that a single commit might fail — break into pick-regex and pick-indices if needed. Expected: +10-13.

20. [pending] repeat property for-loops + where — repeat / basic property for loop, can nest loops, where clause can use the for loop variable name (3 tests). Expected: +3.

21. [pending] possessiveExpression property access via its — possessive / can access its properties (1 test, Expected foo got ``). Expected: +1.

22. [pending] window global fn fallback — regressions / can invoke functions w/ numbers in name + unlocks several others. When calling foo() where foo isn't SX-defined, fall back to (host-global "foo"). Design decision: either compile-time emit (or foo (host-global "foo")) via a helper, or add runtime lookup in the dispatch path. Expected: +2-4.

23. [pending] me symbol works in from expressions — regressions (1 test, Expected Foo). Check from expression compilation. Expected: +1.

24. [pending] properly interpolates values 2 — URL interpolation regression (1 test). Likely template string + property access. Expected: +1.

25. [pending] can support parenthesized commands and features — parser (1 test, Expected clicked). Parser needs to accept (cmd...) grouping in more contexts. Expected: +1.

Bucket C: feature stubs (DOM observer mocks)

26. [pending] resize observer mock + on resize — 3 tests. Add a minimal ResizeObserver mock to hs-run-filtered.js, plus parse/compile on resize. Expected: +3.

27. [pending] intersection observer mock + on intersection — 3 tests. Mock IntersectionObserver; compile on intersection with margin/threshold modifiers. Expected: +3.

28. [pending] ask/answer + prompt/confirm mock — askAnswer 4 tests. Requires test-name-keyed mock: first test wants confirm → true, second confirm → false, third prompt → "Alice", fourth prompt → null. Keyed via _current-test-name in the runner. Expected: +4.

29. [pending] hyperscript:before:init / :after:init / :parse-error events — 6 tests in bootstrap + parser. Fire DOM events at activation boundaries. Expected: +4-6.

30. [pending] logAll config — 1 test. Global config that console.log's each command. Expected: +1.

Bucket D: medium features (bigger commits, plan-first)

31. [pending] runtime null-safety error reporting — 18 tests in runtimeErrors. When accessing .foo on nil, emit a structured error with position info. One coordinated fix in the compiler emit paths for property access, function calls, set/put. Expected: +15-18.

32. [pending] MutationObserver mock + on mutation dispatch — 15 tests in on. Add MO mock to runner. Compile on mutation [of attribute/childList/attribute-specific]. Expected: +10-15.

33. [pending] cookie API — 5 tests in expressions/cookies. document.cookie mock in runner + the cookies + set the xxx cookie keywords. Expected: +5.

34. [pending] event modifier DSL — 8 tests in on. elsewhere, every, first click, count filters (once / twice / 3 times, ranges), from elsewhere. Expected: +6-8.

35. [pending] namespaced def — 3 tests. def ns.foo() ... creates ns.foo. Expected: +3.

Bucket E: subsystems (DO NOT LOOP — human-driven)

36. [blocked: needs design] WebSocket + socket + rpc proxy — 16 tests. Ship only with intentional design review.

37. [blocked: needs design] Tokenizer-as-API — 17 tests. Expose tokens as inspectable SX data.

38. [blocked: needs design] SourceInfo API — 4 tests. (get line N) / (get source N) metadata on compiled AST.

39. [blocked: needs design] WebWorker plugin — 1 test.

40. [blocked: needs design] Fetch non-2xx / before-fetch event / real response object — 7 tests. Sinon-level route mocks or real fetch interception.

Bucket F: generator translation gaps (after bucket A-D)

Many tests are SKIP (untranslated) because tests/playwright/generate-sx-tests.py bailed with return None. These need patches to the generator to recognize more JS test patterns. Estimated ~25 recoverable tests. Defer to a dedicated generator-repair cluster once the queue above drains.

---

Ground rules for the loop agent

1. One cluster per commit. Don't batch. Short commit message: HS: <cluster name> (+N tests).
2. Baseline first, verify at the end. Before starting: record the current pass count for the target suite AND for one smoke range (0-195). After fixing: rerun both. Abort and mark blocked if:
   • Target suite didn't improve by at least +1.
   • Smoke range regressed (any test flipped pass → fail).
3. Never edit .sx files with Edit/Read/Write. Use sx-tree MCP (sx_read_subtree, sx_replace_node, sx_insert_child, sx_insert_near, sx_replace_by_pattern, sx_rename_symbol, sx_validate, sx_write_file).
4. Sync WASM staging. After every edit to lib/hyperscript/<f>.sx, run cp lib/hyperscript/<f>.sx shared/static/wasm/sx/hs-<f>.sx.
5. Never edit spec/tests/test-hyperscript-behavioral.sx directly. Fix the generator or the runtime.
6. Scope: lib/hyperscript/**, shared/static/wasm/sx/hs-*, tests/hs-run-filtered.js, tests/playwright/generate-sx-tests.py, plans/hs-conformance-to-100.md. Do not touch spec/evaluator.sx, the broader SX kernel, or unrelated files.
7. Commit even partial fixes. If you get +N where N is less than expected, commit what you have and mark the cluster done (+N) — partial, <what's left>.
8. If stuck >30min on a cluster, mark it blocked (<reason>) in the plan and move to the next pending cluster.
9. Branch: architecture. Commit locally. Never push. Never touch main.
10. Log every iteration in the Progress log below: one paragraph, what you touched, delta, commit SHA.
11. Update the scoreboard at plans/hs-conformance-scoreboard.md in the SAME plan-update commit: bump the Merged: line, update the row's Status / Δ / Commit, and adjust the buckets roll-up counts.
12. Also expand scope to include plans/hs-conformance-scoreboard.md (for rule 6 purposes).

Known gotchas

• env-bind! creates bindings; env-set! mutates existing ones.
• SX do is R7RS iteration — use begin for multi-expr sequences.
• cond / when / let clause bodies evaluate only the last expr — wrap in begin.
• list? in SX checks for {_type:'list'} — it returns false on raw JS Arrays. host-get node "children" returns a JS Array in the mock, so recursion via (list? kids) silently drops nested elements.
• append! on a list-valued scoped var (:s) requires emit-set in the compiler — done, see commit 1613f551.
• When symbol target is the-result, also sync it (done, see emit-set).
• Hypertrace tests (196, 199, 200) and query-template test (615) hang under 200k step limit — always filter around them.
• repeat forever tests (1197, 1198) also hang.

Progress log

(Reverse chronological — newest at top.)

2026-04-24 — cluster 18 throw respond via exception event

• dda3becb — HS: throw respond via exception event (+2 tests). hs-on wraps each event handler in a guard that catches thrown exceptions and re-dispatches them as an exception DOM event on the same target with {error: e} as detail. on exception(error) handlers (also registered via hs-on) receive the event and destructure error from detail. Wrapping skips exception/error event handlers to avoid infinite loops. Suite hs-upstream-throw: 5/7 → 7/7. Smoke 0-195: 162/195 unchanged.

2026-04-24 — cluster 17 tell semantics (blocked, reverted)

• Attempted: drop the me shadow from tell X compile so add .bar to me preserves original and put your innerText into me writes to original. Fixed tests 2 and 3 but regressed 4 others (restores a proper implicit me, works with an array, establishes a proper beingTold symbol, ignores null) which require bare commands like add .bar (no explicit target) to use the told as default. Reverted per abort rule. Proper fix needs a distinct beingTold symbol with compiler rewriting bare commands to target beingTold-or-me while leaving explicit me alone — >30min cluster budget.

2026-04-24 — cluster 15 transition query-ref + multi-prop (partial +2)

• 3d352055 — HS: transition query-ref + multi-prop (+2 tests). Three parts: (a) parser collect-transitions recognises style tokens (*prop) as a continuation, so transition *width from A to B *height from A to B chains both transitions instead of dropping the second. (b) Mock El class gets nextSibling/previousSibling (+ *ElementSibling aliases) so transition *W of the next <span/> can resolve the next-sibling target via host-get. (c) Generator pattern for const X = await evaluate(() => { const el = document.querySelector(SEL); el.dispatchEvent(new Event(NAME, ...)); return ... }); optionally prefixed by destructuring and allowing trailing expect(...).toBe(...) junk because _body_statements only splits on ; at depth 0. Remaining initial test needs on click N count-filtered events. Suite hs-upstream-transition: 13/17 → 15/17. Smoke 0-195: 162/195 unchanged.

2026-04-24 — cluster 13 toggle multi-class + until (partial +2)

• bd821c04 — HS: toggle multi-class + until event (+2 tests). Parser parse-toggle-cmd: after the leading class ref collect any additional class refs and treat toggle .foo .bar as toggle-between (pair-only). Recognise until EVENT [from SOURCE] modifier and emit a new toggle-class-until AST. Compiler handles the new node by emitting (begin (hs-toggle-class! tgt cls) (hs-wait-for src ev) (hs-toggle-class! tgt cls)) — reuses the cluster-9 event waiter so the class flips back when the event fires. can toggle for a fixed amount of time remains — sync mock io-sleep collapses the two toggles into one click frame; needs async scheduler. Suite hs-upstream-toggle: 22/25 → 24/25. Smoke 0-195: 162/195 unchanged.

2026-04-24 — cluster 12 show multi-element + display retention

• 98c957b3 — HS: show multi-element + display retention (+2 tests). Two fixes in tests/hs-run-filtered.js: (a) mt (matches-selector) now splits comma-separated selector lists and matches if any clause matches, so qsa("#d1, #d2") returns both elements. (b) host-get on an El for innerText returns textContent (DOM-level alias) so when its innerText contains "foo" predicates can see the mock's stored text. Suite hs-upstream-show: 16/18 → 18/18. Smoke 0-195: 162/195 unchanged.

2026-04-24 — cluster 11 hide strategy (partial +3)

• beb120ba — HS: hide strategy config (+3 tests). Three parts: (a) runtime.sx hs-hide-one!/hs-show-one! consult a new _hs-hide-strategies dict (and _hs-default-hide-strategy override) before falling through to the built-in display/opacity/hidden cases. Strategy fn is called directly with (op, el, arg). New setters hs-set-hide-strategies! and hs-set-default-hide-strategy!. (b) Generator _hs_config_setup_ops recognises _hyperscript.config.defaultHideShowStrategy = "X", delete …default…, and hideShowStrategies = { NAME: function (op, el, arg) { if … classList.add/remove } } with brace-matched function body extraction. (c) Pre-setup emitter handles __hs_config__ pseudo-name by emitting the SX expression as-is. Suite hs-upstream-hide: 12/16 → 15/16. Remaining test (hide element then show element retains original display) needs on click N count-filtered event handlers — separate feature. Smoke 0-195: 162/195 unchanged.

2026-04-24 — cluster 10 swap variable with property

• 30f33341 — HS: swap variable with property (+1 test). Mock El class in tests/hs-run-filtered.js: dataset is now a Proxy that forwards property writes to attributes["data-*"], and setAttribute("data-*",...) populates the backing dataset with camelCase key. That way #target.dataset.val = "new" updates the data-val attribute so the swap command can read+write the property correctly. Suite hs-upstream-swap: 3/4 → 4/4. Smoke 0-195: 162/195 unchanged.

2026-04-24 — cluster 9 wait on event basics

• f79f96c1 — HS: wait on event basics (+4 tests). Five parts: (a) tests/hs-run-filtered.js io-wait-event mock now registers a one-shot listener on the target element and resumes with the actual event (was unconditionally doResume(null)). (b) New hs-wait-for-or target event-name timeout-ms runtime form carrying a timeout; mock resumes immediately when timeout is present (covers 0ms tests). (c) parser.sx parse-wait-cmd recognises wait for EV(v1, v2) destructure syntax, emits :destructure (names) on the wait-for AST. (d) compiler.sx emit-wait-for handles :from/:or combos; new __bind-from-detail__ form compiles to (define v (host-get (host-get it "detail") v)); the do-sequence handler pre-expands wait-for with destructure into the plain wait-for plus synthetic bind forms. (e) generator extracts detail: ... from CustomEvent option blocks. Suite hs-upstream-wait: 3/7 → 7/7. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 16 send can reference sender

• ed8d71c9 — HS: send can reference sender (+1 test). Three parts: (a) emit-send builds {:sender me} detail instead of nil for send NAME target and send NAME. (b) Parser parse-atom recognises sender keyword (previously swallowed as noise) and emits (sender). (c) Compiler translates bare sender symbol and (sender) list head to (hs-sender event), a new runtime helper that reads detail.sender. Suite hs-upstream-send: 7/8 → 8/8. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 14 unless modifier (blocked, reverted)

• Attempted: parser cl-collect handles cmd unless cond by emitting (hs-unless-wrap cond cmd), compiler adds a hs-unless-wrap case that translates to (if (hs-falsy? cond) cmd nil). Compile output correct. But test fails with Undefined symbol: _test-result suggesting the test-harness thunk eval throws somehow. Also added a generator pattern for classList.add/remove/toggle but that alone didn't fix the test. Reverted per abort rule; compile-shape fix looks sound and should be revisited after clusters that don't depend on harness internals.

2026-04-23 — cluster 8 select returns selected text (blocked, reverted)

• Attempted: added hs-get-selection runtime, compiler branch to rewrite bare selection to (hs-get-selection), generator pattern to translate evaluate(() => { var range = document.createRange(); ...; window.getSelection().addRange(range); }), and mock support in hs-run-filtered.js for document.createRange / window.getSelection / firstChild text node. Tests still returned empty — range.toString() wasn't picking up the text. Reverted per the abort rule. Would need a more faithful mock of DOM text nodes with data propagation.

2026-04-23 — cluster 7 put hyperscript reprocessing (partial)

• f21eb008 — HS: put hyperscript reprocessing — generator fix (+1 test). Generator was swallowing non-window-setup evaluate(() => { ... }) blocks. Fixed to only continue when a window-setup actually parsed, else fall through. Added a new pattern for evaluate(() => { const e = new Event(...); SEL.dispatchEvent(e); }). Suite hs-upstream-put: 33/38 → 34/38. "at end of" now passes; "at start of" / "in a element target" / "in a symbol write" still fail because the inserted-button handler doesn't activate on the afterbegin/innerHTML code paths. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 6 string template ${x}

• 108e25d4 — HS: string template ${x} (+2 tests). Two-part fix: (a) compiler.sx now emits (host-global "window") (plus other well-known globals) for bare dot-chain base identifiers that would otherwise be unbound symbols. (b) generate-sx-tests.py now has eval-hs-locals ALSO call host-set! on window.<name> for each binding, so tests whose window.X = Y setup was translated as a local pair can still see window.X. Suite hs-upstream-expressions/strings: 5/8 → 7/8. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 5 some selector for nonempty match

• e7b86264 — HS: some selector for nonempty match (+1 test). some <html/> → (not (hs-falsy? (hs-query-first "html"))) → document.querySelector('html'). Mock's querySelector searched from _body, missing the _html element. Fixed the mock to short-circuit for html/body and walk documentElement. Suite hs-upstream-expressions/some: 5/6 → 6/6. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 4 not precedence over or

• 4fe0b649 — HS: not precedence over or + truthy/falsy coercion (+3 tests). parse-atom's not branch emitted (not (parse-expr)), which let or/and capture the whole RHS, and also used SX's not which treats only nil/false as falsy. Fixed to emit (hs-falsy? (parse-atom)) — tight binding + hyperscript truthiness (0, "", nil, false, []). Suite hs-upstream-expressions/not: 6/9 → 9/9. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 3 Values dict insertion order

• e59c0b8e — HS: Values dict insertion order (+2 tests). Root cause was the OCaml kernel's dict implementation iterating keys in scrambled (non-insertion) order. Added _order hidden list tracked by hs-values-absorb, and taught hs-coerce FormEncoded/JSONString branches to iterate via _order when present (filtering the _order marker out). Suite hs-upstream-expressions/asExpression: 28/42 → 30/42. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 2 element→HTML via outerHTML

• e195b5bd — HS: element → HTML via outerHTML (+1 test). Added an outerHTML getter on the mock El class in tests/hs-run-filtered.js. Merges .id/.className (host-set! targets) with .attributes, falls back to innerText/textContent. Suite hs-upstream-expressions/asExpression: 27/42 → 28/42. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster 1 fetch JSON unwrap

• 39a597e9 — HS: fetch JSON unwrap (+4 tests). Added hs-host-to-sx helper in runtime.sx that converts raw host-handle JS objects/arrays to proper SX dicts/lists via Object.keys/Array walks. hs-fetch now calls it on the result when format is "json". Detects host-handle dicts by checking (host-get v "_type") == "dict" — genuine SX dicts have the marker, host handles don't. Suite hs-upstream-fetch: 11/23 → 15/23. Smoke 0-195: 162/195 unchanged.

2026-04-23 — cluster fixes session baseline

• 6b0334af — HS: remove bare @attr, set X @attr, JSON clean, FormEncoded, HTML join (+3)
• 1613f551 — HS add/append: Set dedup, @attr support, when-clause result tracking (+6)
• Pre-loop baseline: 1213/1496 (81.1%).