- scrape-hs-upstream.py: new scraper walks /tmp/hs-upstream/test/**/*.js
and emits body-style records for all 1,496 v0.9.90 tests (up from 831).
Widens coverage into 66 previously-missing categories — templates,
reactivity, behavior, worker, classRef, make, throw, htmx, tailwind,
viewTransition, and more.
- build-hs-manifest.py + hyperscript-upstream-manifest.{json,md}:
coverage manifest tagging each upstream test with a status
(runnable / skip-listed / untranslated / missing) and block reason.
- generate-sx-tests.py: emit (error "SKIP (...)") instead of silent
(hs-cleanup!) no-op for both skip-listed tests and generator-
untranslatable bodies. Stub counter now reports both buckets.
- hyperscript-feature-audit-0.9.90.md: gap audit against the 0.9.90
spec; pre-0.9.90.json backs up prior 831-test snapshot.
New honest baseline (ocaml runner, test-hyperscript-behavioral):
831 -> 1,496 tests; 645 -> 1,013 passing (67.7% conformance).
483 failures split: 45 skip-list, 151 untranslated, 287 real.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7.7 KiB
7.7 KiB
_hyperscript v0.9.90 upstream test coverage manifest
- Upstream tag:
v0.9.90(commita13de2ca, 2026-04-13) - Our snapshot JSON:
spec/tests/hyperscript-upstream-tests.json(831 tests, scraped 2026-04-09) - Total upstream tests: 1496
- Runnable (present + not skip-listed): 764 (51.1%)
- Skip-listed (present but guarded in generator): 44 (2.9%)
- Missing from our snapshot: 688 (46.0%)
Current conformance: runnable / total = 764/1496 = 51.1%.
Per-category
| Category | Upstream | Runnable | Skip-listed | Missing |
|---|---|---|---|---|
| add | 19 | 19 | 0 | 0 |
| api | 1 | 0 | 0 | 1 |
| append | 13 | 13 | 0 | 0 |
| arrayIndex | 14 | 0 | 0 | 14 |
| arrayLiteral | 8 | 0 | 0 | 8 |
| asExpression | 42 | 17 | 0 | 25 |
| askAnswer | 5 | 5 | 0 | 0 |
| assignableElements | 8 | 8 | 0 | 0 |
| asyncError | 2 | 2 | 0 | 0 |
| attributeRef | 22 | 1 | 0 | 21 |
| beep! | 6 | 0 | 0 | 6 |
| behavior | 10 | 0 | 0 | 10 |
| bind | 44 | 38 | 0 | 6 |
| blockLiteral | 4 | 0 | 0 | 4 |
| boolean | 2 | 0 | 0 | 2 |
| bootstrap | 26 | 14 | 0 | 12 |
| breakpoint | 2 | 0 | 0 | 2 |
| call | 6 | 6 | 0 | 0 |
| classRef | 9 | 0 | 0 | 9 |
| closest | 10 | 3 | 0 | 7 |
| collectionExpressions | 28 | 22 | 0 | 6 |
| comparisonOperator | 83 | 40 | 0 | 43 |
| component | 20 | 18 | 0 | 2 |
| cookies | 5 | 1 | 0 | 4 |
| def | 27 | 24 | 3 | 0 |
| default | 15 | 9 | 0 | 6 |
| dialog | 12 | 9 | 0 | 3 |
| dom-scope | 25 | 25 | 0 | 0 |
| empty | 13 | 13 | 0 | 0 |
| evalStatically | 8 | 8 | 0 | 0 |
| eventsource | 13 | 0 | 0 | 13 |
| fetch | 23 | 15 | 8 | 0 |
| focus | 3 | 3 | 0 | 0 |
| functionCalls | 12 | 0 | 0 | 12 |
| go | 5 | 5 | 0 | 0 |
| halt | 7 | 7 | 0 | 0 |
| hide | 16 | 14 | 0 | 2 |
| hs-include | 10 | 0 | 0 | 10 |
| htmx | 9 | 0 | 0 | 9 |
| idRef | 4 | 0 | 0 | 4 |
| if | 19 | 19 | 0 | 0 |
| in | 10 | 1 | 0 | 9 |
| increment | 20 | 20 | 0 | 0 |
| init | 3 | 3 | 0 | 0 |
| js | 11 | 1 | 0 | 10 |
| live | 23 | 23 | 0 | 0 |
| liveTemplate | 16 | 10 | 0 | 6 |
| log | 4 | 4 | 0 | 0 |
| logicalOperator | 10 | 3 | 0 | 7 |
| make | 8 | 0 | 0 | 8 |
| mathOperator | 15 | 5 | 0 | 10 |
| measure | 6 | 2 | 0 | 4 |
| morph | 10 | 10 | 0 | 0 |
| no | 9 | 5 | 0 | 4 |
| not | 9 | 0 | 0 | 9 |
| null | 1 | 0 | 0 | 1 |
| numbers | 1 | 0 | 0 | 1 |
| objectLiteral | 12 | 1 | 0 | 11 |
| on | 70 | 27 | 33 | 10 |
| parser | 14 | 7 | 0 | 7 |
| pick | 24 | 7 | 0 | 17 |
| positionalExpression | 7 | 0 | 0 | 7 |
| possessiveExpression | 23 | 0 | 0 | 23 |
| propertyAccess | 12 | 0 | 0 | 12 |
| pseudoCommand | 11 | 0 | 0 | 11 |
| put | 38 | 38 | 0 | 0 |
| queryRef | 13 | 1 | 0 | 12 |
| reactive-properties | 4 | 4 | 0 | 0 |
| reactivity | 8 | 0 | 0 | 8 |
| regressions | 16 | 0 | 0 | 16 |
| relativePositionalExpression | 23 | 4 | 0 | 19 |
| remove | 19 | 14 | 0 | 5 |
| repeat | 30 | 29 | 0 | 1 |
| reset | 8 | 8 | 0 | 0 |
| resize | 3 | 3 | 0 | 0 |
| runtime | 7 | 0 | 0 | 7 |
| runtimeErrors | 18 | 0 | 0 | 18 |
| scoping | 20 | 1 | 0 | 19 |
| scroll | 8 | 8 | 0 | 0 |
| security | 1 | 0 | 0 | 1 |
| select | 4 | 4 | 0 | 0 |
| send | 8 | 8 | 0 | 0 |
| set | 31 | 25 | 0 | 6 |
| settle | 3 | 1 | 0 | 2 |
| show | 18 | 2 | 0 | 16 |
| socket | 16 | 4 | 0 | 12 |
| some | 6 | 0 | 0 | 6 |
| sourceInfo | 4 | 0 | 0 | 4 |
| splitJoin | 7 | 7 | 0 | 0 |
| stringPostfix | 3 | 0 | 0 | 3 |
| strings | 8 | 0 | 0 | 8 |
| styleRef | 6 | 0 | 0 | 6 |
| swap | 4 | 4 | 0 | 0 |
| symbol | 2 | 0 | 0 | 2 |
| tailwind | 12 | 0 | 0 | 12 |
| take | 15 | 12 | 0 | 3 |
| tell | 10 | 10 | 0 | 0 |
| templates | 48 | 0 | 0 | 48 |
| throw | 7 | 0 | 0 | 7 |
| toggle | 25 | 25 | 0 | 0 |
| tokenizer | 17 | 0 | 0 | 17 |
| transition | 17 | 17 | 0 | 0 |
| trigger | 6 | 0 | 0 | 6 |
| typecheck | 5 | 0 | 0 | 5 |
| unlessModifier | 1 | 0 | 0 | 1 |
| viewTransition | 9 | 0 | 0 | 9 |
| wait | 7 | 7 | 0 | 0 |
| when | 41 | 41 | 0 | 0 |
| worker | 1 | 0 | 0 | 1 |
| TOTAL | 1496 | 764 | 44 | 688 |
What unlocks how many — MISSING tests by block_reason
| Block reason | Missing | Example | Est. effort |
|---|---|---|---|
unscraped-category |
282 | breakpoint / parses as a top-level command | low — extend scraper to cover these upstream files |
unscraped-in-known-category |
170 | default / can default variables | low — re-scrape; file was walked but these cases missed |
added-post-snapshot |
122 | hide / can hide via the hidden attribute strategy | low — re-scrape upstream, bump JSON snapshot |
needs-pattern:${} |
51 | liveTemplate / script type="text/hyperscript-template" works as a l... | low — template string interpolation in HS parser |
needs-script-tag |
26 | throw / can throw a basic exception | medium — emit <script type="text/hyperscript"> wrapper in generator |
needs-websocket |
12 | socket / with timeout parses and uses the configured timeout | high — WebSocket mock server |
needs-css-transitions |
8 | viewTransition / runs the body when view transitions API is unavail... | medium — transitionend event dispatch in fixtures |
needs-pattern:<sel/> |
8 | comparisonOperator / exists works | low — parser rule for positional expr |
needs-pattern:[@attr] |
5 | attributeRef / attributeRef with no value works | low — attribute-ref parser rule |
needs-dialog-api |
3 | dialog / show opens a dialog (non-modal) | low — stub showModal/close on HTMLDialogElement in fixture DOM |
needs-intersection-observer |
1 | on / on intersection fires when the element is in the viewport | medium — IntersectionObserver mock |
Skip-listed tests by block_reason
| Block reason | Skipped | Example | Est. effort |
|---|---|---|---|
translation-TBD |
39 | fetch / can do a simple fetch w/ html | unknown — needs case-by-case generator work |
needs-script-tag |
5 | def / functions can be namespaced | medium — emit <script type="text/hyperscript"> wrapper in generator |
How this was built
git clone --depth 1 --branch v0.9.90 https://github.com/bigskysoftware/_hyperscript /tmp/hs-upstream;git fetch --unshallowfor dated history.- Walked
test/(excludingvendor/,manual/,fixtures.js,global-*.js,entry.js,htmx-fixtures.js,playwright.config.js). - For each
.jsfile, a small Python parser finds everytest.describe(...)block, then everytest(...)within it — balanced-paren scan that ignores strings, regex literals, line/block comments. - Category = filename stem (e.g.
add.js→add). Test name = the first string literal argument oftest(...). - Matched each upstream test against
spec/tests/hyperscript-upstream-tests.jsonusing(category, name-normalized)keys (whitespace-collapsed lowercase). Copiedcomplexitywhen found; inferred it otherwise from body content (sinon./script-tag/dialog/Promise/evaluate). status = runnableif present and name not in generator'sSKIP_TEST_NAMES;skip-listedif present and in that set;missingotherwise.block_reasonclassified from body content — sinon./script tag/dialog/worker/eventsource/WebSocket/MutationObserver/transition/focus/ResizeObserver patterns or<sel/>/${}/[@attr]HS syntax. Missing tests in files touched between 2026-04-09 and 2026-04-14 (git log --after --before -- test/) are taggedadded-post-snapshot.- Regenerate:
python3 tests/playwright/build-hs-manifest.py(expects/tmp/hs-upstreamclone atv0.9.90).
Untranslated caveat
The markdown reports runnable = present + not skip-listed. Empirical baseline is 645 pass / 109 fail / 77 skip on 831 present tests. The ~109 failures are in-scope but reveal implementation gaps; they are not statically identifiable from upstream source without running the generator. This manifest therefore does not surface an untranslated bucket — treat the 109 empirical fails as the lower bound of that bucket inside the runnable count.