Files
rose-ash/plans/hs-conformance-scoreboard.md
giles 4d92eafb36
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 39s
HS scoreboard: dict-eq fix entry + post-JIT-Phase-2 regression note
Records that the 1514/1514 claim was relative to the kernel as of
92619301; the value-handle ABI + numeric tower + JIT Phase 2 commits
introduced three regressions (1 dict-eq, now fixed in 4db1f85f, and 2
event-or-timeout wait tests still pending).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 21:22:00 +00:00

8.2 KiB
Raw Blame History

HS Conformance Scoreboard

Live tally for plans/hs-conformance-to-100.md. Update after every cluster commit.

Baseline:    1213/1496 (81.1%)  initial scrape
Snapshot:    1514/1514          upstream sync 2026-05-08 (+18 new upstream tests)
Conformance: 1514/1514 (100.0%) — zero skips, full upstream coverage
Wall:        23m33s sequential (8 batches × 200) via tests/hs-run-batched.js
Note: full-suite single-process is unreliable due to JIT cache saturation;
      use hs-run-batched.js (fresh kernel per batch) for deterministic numbers.

Cleared this session (18 → 0 skips):
  - Toggle parser ambiguity (1)       → 2-token lookahead in parse-toggle
  - Throttled-at modifier (1)         → parser + emit-on wrap + hs-throttle!/hs-debounce!
  - Tokenizer-stream API (13)         → hs-stream wrapper + 15 stream primitives
  - Template-component scope (2)      → manual bodies for enclosing-scope-via-$varname semantics
  - Async event dispatch (1)          → manual body covers parse+compile+dispatch path
  - Compiler perf (cross-cutting)     → hoist _strip-throttle-debounce to module level
                                         (was JIT-recompiling per emit-on call)

Status: 1514/1514 ✓ — no remaining work in upstream conformance.

2026-05-12 — kernel-eq regression after JIT Phase 2 + numeric tower

The 100% claim held against the kernel as it was at 92619301. Subsequent commits (Phase 1+2+3 JIT, value-handle ABI, numeric tower) regressed three tests; one is now fixed:

  • arrayLiteral / arrays containing objects work — fixed in 4db1f85f (deep_equal in sx_browser.ml had no Integer branch; safe_eq for Dict/Dict only handled DOM handles, never structural). Suite back to 8/8.
  • hs-upstream-wait / can wait on event or timeout 1 — still failing (assertion-fail). wait for foo or 0ms likely a parser/runtime path.
  • hs-upstream-wait / can wait on event or timeout 2 — same shape.

Batched run (75-test batches, parallel=2) currently: 1436+1 confirmed pass, 2 fails, 75 unverified (batch 150-225 hits >15min on slow reactivity tests). Worst-case score: 1437/1514 (94.9%); best-case (75 unverified pass): 1512/1514 (99.87%).

Future architectural items NOT required for conformance, tracked for roadmap:

  • True <script type="text/hyperscript-template" component="..."> custom-element registrar
  • True async kernel suspension for repeat until event (yielding to JS event loop)
  • Parser fix for from #<id-ref> after event NAME in until-expressions
  • wait for foo or 0ms (event-or-timeout) — parser support needed

Cluster ledger

Bucket A — runtime fixes

# Cluster Status Δ Commit
1 fetch JSON unwrap done +4 39a597e9
2 element → HTML via outerHTML done +1 e195b5bd
3 Values dict insertion order done +2 e59c0b8e
4 not precedence over or done +3 4fe0b649
5 some selector for nonempty match done +1 e7b86264
6 string template ${x} done +2 108e25d4
7 put hyperscript reprocessing done +5 247bd85c
8 select returns selected text done +1 d862efe8
9 wait on event basics done +4 f79f96c1
10 swap variable ↔ property done +1 30f33341
11 hide strategy partial +3 beb120ba
12 show multi-element + display retention done +2 98c957b3
13 toggle multi-class + timed + until-event partial +2 bd821c04
14 unless modifier done +1 c4da0698
15 transition query-ref + multi-prop + initial partial +3 3d352055
16 send can reference sender done +1 ed8d71c9
17 tell semantics blocked
18 throw respond async/sync done +2 dda3becb

Bucket B — parser/compiler additions

# Cluster Status Δ Commit
19 pick regex + indices done +13 4be90bf2
20 repeat property for-loops + where done +3 c932ad59
21 possessiveExpression property access via its done +1 f0c41278
22 window global fn fallback done +1 d31565d5
23 me symbol works in from expressions done +1 0d38a75b
24 properly interpolates values 2 done +1 cb37259d
25 parenthesized commands and features done +1 d7a88d85

Bucket C — feature stubs (observer mocks)

# Cluster Status Δ Commit
26 resize observer mock + on resize done +3 304a52d2
27 intersection observer mock + on intersection done +3 0c31dd27
28 ask/answer + prompt/confirm mock done +4 6c1da921
29 hyperscript:before:init / :after:init / :parse-error partial +2 e01a3baa
30 logAll config done +1 64bcefff

Bucket D — medium features

# Cluster Status Δ
31 runtime null-safety error reporting done +13
32 MutationObserver mock + on mutation done +7
33 cookie API partial +4
34 event modifier DSL partial +7
35 namespaced def done +3
36b call result binds to it done +1

Bucket E — subsystems (design docs landed, pending review + implementation)

# Cluster Status Design doc
36 WebSocket + socket + RPC proxy done +16
37 Tokenizer-as-API done +17
38 SourceInfo API done +2
39 WebWorker plugin done +1
40 Fetch non-2xx / before-fetch / real response done +7

Step-limit fix

# Cluster Status Δ Commit
SL raise default step limit 200k→1M done +70 225fa2e8
17 tell semantics done (included in SL)
33 cookie API (remaining 1) done (included in SL)

Bucket F — generator translation gaps

Defer until AD drain. Estimated ~25 recoverable tests.

# Cluster Status Δ Commit
F1 add CSS template interpolation done +1 5a76a040
F2 empty multi-element (query→for-each) done +1 875e9ba3
F3 hs-make-object _order + assert= for dicts done +1 daea2808
F4 array literal arg to JS fn (sxToJs + reduce→SX) done +1 da2e6b1b
F5 bind feature parser stub done +32 846650da
F6 asyncError rejected promise catch done +1
F7 hs-on nil-target guard (skip-list rescue) done +1 1751cd05
F8 on EVENT from SRC or EVENT from SRC multi-source done +1 f1428009
F9 obj.method() via host-call (T9 from plan) done +1 hs-f
F10 obj.method(promiseArg) resolved sync (F2) done +1 hs-f
F11 obj.asyncMethod(promiseArg) resolved sync (F3) done +1 hs-f
F12 fetch /url as html → DocumentFragment via io-parse-html done +1 hs-f
F13 hs-null-error! self-contained guard (avoid slow host_error path) done +3 hs-f
F14 when @attr changes parser+compiler+runtime — MutationObserver wiring done +1 hs-f
F15 def/default/empty suites: NO_STEP_LIMIT for legitimate scoped-var cascades done +N hs-f

Buckets roll-up

Bucket Done Partial In-prog Pending Blocked Design-done Total
A 12 4 0 0 1 17
B 7 0 0 0 0 7
C 4 1 0 0 0 5
D 2 2 0 0 1 5
E 5 0 0 0 0 0 5
F ~10 ~10

Maintenance

After each cluster commit, update:

  • Merged: pass count + delta line at top
  • The row's Status / Δ / Commit in the relevant bucket
  • The buckets roll-up table counts

Use mcp__hs-test__hs_test_run(start=0, end=195) + targeted-suite runs to get the real number; don't run the full suite every iteration (hangs on 196/199/200/615/1197/1198).