giles
197c073308
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 47s
Investigation of the long-standing 'why does the runner say 1494/1494 not
1496/1496?' question. The answer is in tests/hs-run-filtered.js:969 — two
tests are skipped via _SKIP_TESTS for documented architectural reasons:
1. 'until event keyword works' — uses 'repeat until event click from #x',
which suspends the OCaml kernel waiting for a click that is never
dispatched from outside K.eval. The sync test runner has no way to
fire the click while the kernel is suspended.
2. 'throttled at <time> drops events within the window' — the HS parser
does not implement the 'throttled at <ms>' modifier. The compiled SX
for the handler is malformed: handler body is the literal symbol
'throttled', the time expression dangles outside the closure as
stray (do 200 ...). Genuinely needs parser+compiler+runtime work,
not just a deadline bump.
Both are documented at the skip site with a comment explaining why they
can't run synchronously. The conformance number is 1494/1494 = 100% on
counted tests, with 2 explicit, justified skips out of 1496 total.
This was the source of the cumulative-vs-isolated test-count discrepancy.
Suite filter runs see them as 'not in this suite,' batched runs see them
as 'continued past'. Either way: not failures.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
139 lines
6.7 KiB
Markdown
139 lines
6.7 KiB
Markdown
# HS Conformance Scoreboard
|
||
|
||
Live tally for `plans/hs-conformance-to-100.md`. Update after every cluster commit.
|
||
|
||
```
|
||
Baseline: 1213/1496 (81.1%)
|
||
Merged: 1494/1494 (100.0%) on counted tests; 2 documented skips
|
||
Worktree: all landed
|
||
Skipped: 2 — 'until event keyword works' (async event dispatch needs the
|
||
kernel suspended outside K.eval), 'throttled at <time> drops events
|
||
within the window' (parser doesn't implement the throttled modifier;
|
||
emits malformed SX). Both documented in tests/hs-run-filtered.js.
|
||
Note: step limit raised 200k→1M in 225fa2e8 revealed 70 previously-masked passes
|
||
Note: full-suite run via tests/hs-run-batched.js — fresh-kernel-per-batch
|
||
bypasses the JIT cache saturation that hits a single-process run after
|
||
~500 tests. Sequential at batch=200: 10m47s, 1494/1494.
|
||
Note: hs-f loop totals — T9, F2, F3, F9, hs-null-error! self-guard, T6 @attr
|
||
observer (parser+compiler+runtime), batched runner, def/default/empty
|
||
suites no-step-limit, deadline tuning.
|
||
```
|
||
|
||
## Cluster ledger
|
||
|
||
### Bucket A — runtime fixes
|
||
|
||
| # | Cluster | Status | Δ | Commit |
|
||
|---|---------|--------|---|--------|
|
||
| 1 | fetch JSON unwrap | done | +4 | 39a597e9 |
|
||
| 2 | element → HTML via outerHTML | done | +1 | e195b5bd |
|
||
| 3 | Values dict insertion order | done | +2 | e59c0b8e |
|
||
| 4 | `not` precedence over `or` | done | +3 | 4fe0b649 |
|
||
| 5 | `some` selector for nonempty match | done | +1 | e7b86264 |
|
||
| 6 | string template `${x}` | done | +2 | 108e25d4 |
|
||
| 7 | `put` hyperscript reprocessing | done | +5 | 247bd85c |
|
||
| 8 | `select` returns selected text | done | +1 | d862efe8 |
|
||
| 9 | `wait on event` basics | done | +4 | f79f96c1 |
|
||
| 10 | `swap` variable ↔ property | done | +1 | 30f33341 |
|
||
| 11 | `hide` strategy | partial | +3 | beb120ba |
|
||
| 12 | `show` multi-element + display retention | done | +2 | 98c957b3 |
|
||
| 13 | `toggle` multi-class + timed + until-event | partial | +2 | bd821c04 |
|
||
| 14 | `unless` modifier | done | +1 | c4da0698 |
|
||
| 15 | `transition` query-ref + multi-prop + initial | partial | +3 | 3d352055 |
|
||
| 16 | `send can reference sender` | done | +1 | ed8d71c9 |
|
||
| 17 | `tell` semantics | blocked | — | — |
|
||
| 18 | `throw` respond async/sync | done | +2 | dda3becb |
|
||
|
||
### Bucket B — parser/compiler additions
|
||
|
||
| # | Cluster | Status | Δ | Commit |
|
||
|---|---------|--------|---|--------|
|
||
| 19 | `pick` regex + indices | done | +13 | 4be90bf2 |
|
||
| 20 | `repeat` property for-loops + where | done | +3 | c932ad59 |
|
||
| 21 | `possessiveExpression` property access via its | done | +1 | f0c41278 |
|
||
| 22 | window global fn fallback | done | +1 | d31565d5 |
|
||
| 23 | `me symbol works in from expressions` | done | +1 | 0d38a75b |
|
||
| 24 | `properly interpolates values 2` | done | +1 | cb37259d |
|
||
| 25 | parenthesized commands and features | done | +1 | d7a88d85 |
|
||
|
||
### Bucket C — feature stubs (observer mocks)
|
||
|
||
| # | Cluster | Status | Δ | Commit |
|
||
|---|---------|--------|---|--------|
|
||
| 26 | resize observer mock + `on resize` | done | +3 | 304a52d2 |
|
||
| 27 | intersection observer mock + `on intersection` | done | +3 | 0c31dd27 |
|
||
| 28 | `ask`/`answer` + prompt/confirm mock | done | +4 | 6c1da921 |
|
||
| 29 | `hyperscript:before:init` / `:after:init` / `:parse-error` | partial | +2 | e01a3baa |
|
||
| 30 | `logAll` config | done | +1 | 64bcefff |
|
||
|
||
### Bucket D — medium features
|
||
|
||
| # | Cluster | Status | Δ |
|
||
|---|---------|--------|---|
|
||
| 31 | runtime null-safety error reporting | done | +13 |
|
||
| 32 | MutationObserver mock + `on mutation` | done | +7 |
|
||
| 33 | cookie API | partial | +4 |
|
||
| 34 | event modifier DSL | partial | +7 |
|
||
| 35 | namespaced `def` | done | +3 |
|
||
| 36b | `call` result binds to `it` | done | +1 | 35f498ec |
|
||
|
||
### Bucket E — subsystems (design docs landed, pending review + implementation)
|
||
|
||
| # | Cluster | Status | Design doc |
|
||
|---|---------|--------|------------|
|
||
| 36 | WebSocket + `socket` + RPC proxy | done | +16 | 623529d3 |
|
||
| 37 | Tokenizer-as-API | done | +17 | 54b54f4e |
|
||
| 38 | SourceInfo API | done | +2 | 48eaeb04 |
|
||
| 39 | WebWorker plugin | done | +1 | 8e8c2a73 |
|
||
| 40 | Fetch non-2xx / before-fetch / real response | done | +7 | d7244d1d |
|
||
|
||
### Step-limit fix
|
||
|
||
| # | Cluster | Status | Δ | Commit |
|
||
|---|---------|--------|---|--------|
|
||
| SL | raise default step limit 200k→1M | done | +70 | 225fa2e8 |
|
||
| 17 | `tell` semantics | done | (included in SL) | — |
|
||
| 33 | cookie API (remaining 1) | done | (included in SL) | — |
|
||
|
||
### Bucket F — generator translation gaps
|
||
|
||
Defer until A–D drain. Estimated ~25 recoverable tests.
|
||
|
||
| # | Cluster | Status | Δ | Commit |
|
||
|---|---------|--------|---|--------|
|
||
| F1 | add CSS template interpolation | done | +1 | 5a76a040 |
|
||
| F2 | empty multi-element (query→for-each) | done | +1 | 875e9ba3 |
|
||
| F3 | hs-make-object _order + assert= for dicts | done | +1 | daea2808 |
|
||
| F4 | array literal arg to JS fn (sxToJs + reduce→SX) | done | +1 | da2e6b1b |
|
||
| F5 | `bind` feature parser stub | done | +32 | 846650da |
|
||
| F6 | `asyncError` rejected promise catch | done | +1 | — |
|
||
| F7 | `hs-on` nil-target guard (skip-list rescue) | done | +1 | 1751cd05 |
|
||
| F8 | `on EVENT from SRC or EVENT from SRC` multi-source | done | +1 | f1428009 |
|
||
| F9 | `obj.method()` via host-call (T9 from plan) | done | +1 | hs-f |
|
||
| F10 | `obj.method(promiseArg)` resolved sync (F2) | done | +1 | hs-f |
|
||
| F11 | `obj.asyncMethod(promiseArg)` resolved sync (F3) | done | +1 | hs-f |
|
||
| F12 | `fetch /url as html` → DocumentFragment via io-parse-html | done | +1 | hs-f |
|
||
| F13 | `hs-null-error!` self-contained guard (avoid slow host_error path) | done | +3 | hs-f |
|
||
| F14 | `when @attr changes` parser+compiler+runtime — MutationObserver wiring | done | +1 | hs-f |
|
||
| F15 | def/default/empty suites: NO_STEP_LIMIT for legitimate scoped-var cascades | done | +N | hs-f |
|
||
|
||
## Buckets roll-up
|
||
|
||
| Bucket | Done | Partial | In-prog | Pending | Blocked | Design-done | Total |
|
||
|--------|-----:|--------:|--------:|--------:|--------:|------------:|------:|
|
||
| A | 12 | 4 | 0 | 0 | 1 | — | 17 |
|
||
| B | 7 | 0 | 0 | 0 | 0 | — | 7 |
|
||
| C | 4 | 1 | 0 | 0 | 0 | — | 5 |
|
||
| D | 2 | 2 | 0 | 0 | 1 | — | 5 |
|
||
| E | 5 | 0 | 0 | 0 | 0 | 0 | 5 |
|
||
| F | — | — | — | ~10 | — | — | ~10 |
|
||
|
||
## Maintenance
|
||
|
||
After each cluster commit, update:
|
||
- `Merged:` pass count + `delta` line at top
|
||
- The row's `Status` / `Δ` / `Commit` in the relevant bucket
|
||
- The buckets roll-up table counts
|
||
|
||
Use `mcp__hs-test__hs_test_run(start=0, end=195)` + targeted-suite runs to get the real number; don't run the full suite every iteration (hangs on 196/199/200/615/1197/1198).
|