Files
rose-ash/spec/tests/hyperscript-upstream-manifest.md
giles fd1dfea9b3 HS tests: scrape v0.9.90 upstream in full, flip silent stubs to loud SKIPs
- scrape-hs-upstream.py: new scraper walks /tmp/hs-upstream/test/**/*.js
  and emits body-style records for all 1,496 v0.9.90 tests (up from 831).
  Widens coverage into 66 previously-missing categories — templates,
  reactivity, behavior, worker, classRef, make, throw, htmx, tailwind,
  viewTransition, and more.

- build-hs-manifest.py + hyperscript-upstream-manifest.{json,md}:
  coverage manifest tagging each upstream test with a status
  (runnable / skip-listed / untranslated / missing) and block reason.

- generate-sx-tests.py: emit (error "SKIP (...)") instead of silent
  (hs-cleanup!) no-op for both skip-listed tests and generator-
  untranslatable bodies. Stub counter now reports both buckets.

- hyperscript-feature-audit-0.9.90.md: gap audit against the 0.9.90
  spec; pre-0.9.90.json backs up prior 831-test snapshot.

New honest baseline (ocaml runner, test-hyperscript-behavioral):
  831 -> 1,496 tests; 645 -> 1,013 passing (67.7% conformance).
  483 failures split: 45 skip-list, 151 untranslated, 287 real.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 20:27:22 +00:00

7.7 KiB

_hyperscript v0.9.90 upstream test coverage manifest

  • Upstream tag: v0.9.90 (commit a13de2ca, 2026-04-13)
  • Our snapshot JSON: spec/tests/hyperscript-upstream-tests.json (831 tests, scraped 2026-04-09)
  • Total upstream tests: 1496
  • Runnable (present + not skip-listed): 764 (51.1%)
  • Skip-listed (present but guarded in generator): 44 (2.9%)
  • Missing from our snapshot: 688 (46.0%)

Current conformance: runnable / total = 764/1496 = 51.1%.

Per-category

Category Upstream Runnable Skip-listed Missing
add 19 19 0 0
api 1 0 0 1
append 13 13 0 0
arrayIndex 14 0 0 14
arrayLiteral 8 0 0 8
asExpression 42 17 0 25
askAnswer 5 5 0 0
assignableElements 8 8 0 0
asyncError 2 2 0 0
attributeRef 22 1 0 21
beep! 6 0 0 6
behavior 10 0 0 10
bind 44 38 0 6
blockLiteral 4 0 0 4
boolean 2 0 0 2
bootstrap 26 14 0 12
breakpoint 2 0 0 2
call 6 6 0 0
classRef 9 0 0 9
closest 10 3 0 7
collectionExpressions 28 22 0 6
comparisonOperator 83 40 0 43
component 20 18 0 2
cookies 5 1 0 4
def 27 24 3 0
default 15 9 0 6
dialog 12 9 0 3
dom-scope 25 25 0 0
empty 13 13 0 0
evalStatically 8 8 0 0
eventsource 13 0 0 13
fetch 23 15 8 0
focus 3 3 0 0
functionCalls 12 0 0 12
go 5 5 0 0
halt 7 7 0 0
hide 16 14 0 2
hs-include 10 0 0 10
htmx 9 0 0 9
idRef 4 0 0 4
if 19 19 0 0
in 10 1 0 9
increment 20 20 0 0
init 3 3 0 0
js 11 1 0 10
live 23 23 0 0
liveTemplate 16 10 0 6
log 4 4 0 0
logicalOperator 10 3 0 7
make 8 0 0 8
mathOperator 15 5 0 10
measure 6 2 0 4
morph 10 10 0 0
no 9 5 0 4
not 9 0 0 9
null 1 0 0 1
numbers 1 0 0 1
objectLiteral 12 1 0 11
on 70 27 33 10
parser 14 7 0 7
pick 24 7 0 17
positionalExpression 7 0 0 7
possessiveExpression 23 0 0 23
propertyAccess 12 0 0 12
pseudoCommand 11 0 0 11
put 38 38 0 0
queryRef 13 1 0 12
reactive-properties 4 4 0 0
reactivity 8 0 0 8
regressions 16 0 0 16
relativePositionalExpression 23 4 0 19
remove 19 14 0 5
repeat 30 29 0 1
reset 8 8 0 0
resize 3 3 0 0
runtime 7 0 0 7
runtimeErrors 18 0 0 18
scoping 20 1 0 19
scroll 8 8 0 0
security 1 0 0 1
select 4 4 0 0
send 8 8 0 0
set 31 25 0 6
settle 3 1 0 2
show 18 2 0 16
socket 16 4 0 12
some 6 0 0 6
sourceInfo 4 0 0 4
splitJoin 7 7 0 0
stringPostfix 3 0 0 3
strings 8 0 0 8
styleRef 6 0 0 6
swap 4 4 0 0
symbol 2 0 0 2
tailwind 12 0 0 12
take 15 12 0 3
tell 10 10 0 0
templates 48 0 0 48
throw 7 0 0 7
toggle 25 25 0 0
tokenizer 17 0 0 17
transition 17 17 0 0
trigger 6 0 0 6
typecheck 5 0 0 5
unlessModifier 1 0 0 1
viewTransition 9 0 0 9
wait 7 7 0 0
when 41 41 0 0
worker 1 0 0 1
TOTAL 1496 764 44 688

What unlocks how many — MISSING tests by block_reason

Block reason Missing Example Est. effort
unscraped-category 282 breakpoint / parses as a top-level command low — extend scraper to cover these upstream files
unscraped-in-known-category 170 default / can default variables low — re-scrape; file was walked but these cases missed
added-post-snapshot 122 hide / can hide via the hidden attribute strategy low — re-scrape upstream, bump JSON snapshot
needs-pattern:${} 51 liveTemplate / script type="text/hyperscript-template" works as a l... low — template string interpolation in HS parser
needs-script-tag 26 throw / can throw a basic exception medium — emit <script type="text/hyperscript"> wrapper in generator
needs-websocket 12 socket / with timeout parses and uses the configured timeout high — WebSocket mock server
needs-css-transitions 8 viewTransition / runs the body when view transitions API is unavail... medium — transitionend event dispatch in fixtures
needs-pattern:<sel/> 8 comparisonOperator / exists works low — parser rule for positional expr
needs-pattern:[@attr] 5 attributeRef / attributeRef with no value works low — attribute-ref parser rule
needs-dialog-api 3 dialog / show opens a dialog (non-modal) low — stub showModal/close on HTMLDialogElement in fixture DOM
needs-intersection-observer 1 on / on intersection fires when the element is in the viewport medium — IntersectionObserver mock

Skip-listed tests by block_reason

Block reason Skipped Example Est. effort
translation-TBD 39 fetch / can do a simple fetch w/ html unknown — needs case-by-case generator work
needs-script-tag 5 def / functions can be namespaced medium — emit <script type="text/hyperscript"> wrapper in generator

How this was built

  1. git clone --depth 1 --branch v0.9.90 https://github.com/bigskysoftware/_hyperscript /tmp/hs-upstream; git fetch --unshallow for dated history.
  2. Walked test/ (excluding vendor/, manual/, fixtures.js, global-*.js, entry.js, htmx-fixtures.js, playwright.config.js).
  3. For each .js file, a small Python parser finds every test.describe(...) block, then every test(...) within it — balanced-paren scan that ignores strings, regex literals, line/block comments.
  4. Category = filename stem (e.g. add.jsadd). Test name = the first string literal argument of test(...).
  5. Matched each upstream test against spec/tests/hyperscript-upstream-tests.json using (category, name-normalized) keys (whitespace-collapsed lowercase). Copied complexity when found; inferred it otherwise from body content (sinon./script-tag/dialog/Promise/evaluate).
  6. status = runnable if present and name not in generator's SKIP_TEST_NAMES; skip-listed if present and in that set; missing otherwise.
  7. block_reason classified from body content — sinon./script tag/dialog/worker/eventsource/WebSocket/MutationObserver/transition/focus/ResizeObserver patterns or <sel/>/${}/[@attr] HS syntax. Missing tests in files touched between 2026-04-09 and 2026-04-14 (git log --after --before -- test/) are tagged added-post-snapshot.
  8. Regenerate: python3 tests/playwright/build-hs-manifest.py (expects /tmp/hs-upstream clone at v0.9.90).

Untranslated caveat

The markdown reports runnable = present + not skip-listed. Empirical baseline is 645 pass / 109 fail / 77 skip on 831 present tests. The ~109 failures are in-scope but reveal implementation gaps; they are not statically identifiable from upstream source without running the generator. This manifest therefore does not surface an untranslated bucket — treat the 109 empirical fails as the lower bound of that bucket inside the runnable count.