HS tests: scrape v0.9.90 upstream in full, flip silent stubs to loud SKIPs

- scrape-hs-upstream.py: new scraper walks /tmp/hs-upstream/test/**/*.js
  and emits body-style records for all 1,496 v0.9.90 tests (up from 831).
  Widens coverage into 66 previously-missing categories — templates,
  reactivity, behavior, worker, classRef, make, throw, htmx, tailwind,
  viewTransition, and more.

- build-hs-manifest.py + hyperscript-upstream-manifest.{json,md}:
  coverage manifest tagging each upstream test with a status
  (runnable / skip-listed / untranslated / missing) and block reason.

- generate-sx-tests.py: emit (error "SKIP (...)") instead of silent
  (hs-cleanup!) no-op for both skip-listed tests and generator-
  untranslatable bodies. Stub counter now reports both buckets.

- hyperscript-feature-audit-0.9.90.md: gap audit against the 0.9.90
  spec; pre-0.9.90.json backs up prior 831-test snapshot.

New honest baseline (ocaml runner, test-hyperscript-behavioral):
  831 -> 1,496 tests; 645 -> 1,013 passing (67.7% conformance).
  483 failures split: 45 skip-list, 151 untranslated, 287 real.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-04-22 20:27:22 +00:00
parent 802ccd23e8
commit fd1dfea9b3
8 changed files with 35232 additions and 8311 deletions

View File

@@ -299,9 +299,22 @@ def parse_action(action, ref):
exprs.append(f'(dom-dispatch {ref(m.group(1))} "click" nil)')
continue
m = re.match(r'(\w+)\.dispatchEvent\(new CustomEvent\("([\w:.-]+)"', part)
m = re.match(r'(\w+)\.dispatchEvent\(new CustomEvent\("([\w:.-]+)"\s*(?:,\s*\{(.*)\})?', part)
if m:
exprs.append(f'(dom-dispatch {ref(m.group(1))} "{m.group(2)}" nil)')
detail_expr = 'nil'
body = m.group(3)
if body:
dm = re.search(r'detail:\s*"([^"]*)"', body)
if dm:
detail_expr = f'"{dm.group(1)}"'
else:
dm = re.search(r'detail:\s*\{([^}]*)\}', body)
if dm:
pairs = re.findall(r'(\w+):\s*"([^"]*)"', dm.group(1))
if pairs:
items = ' '.join(f':{k} "{v}"' for k, v in pairs)
detail_expr = '{' + items + '}'
exprs.append(f'(dom-dispatch {ref(m.group(1))} "{m.group(2)}" {detail_expr})')
continue
m = re.match(r'(\w+)\.setAttribute\("([\w-]+)",\s*"([^"]*)"\)', part)
@@ -844,13 +857,27 @@ def emit_element_setup(lines, elements, var_names, root='(dom-body)', indent='
def emit_skip_test(test):
"""Emit a trivial passing deftest for tests that depend on unimplemented
hyperscript features. Keeps coverage in the source JSON but lets the run
move on."""
"""Emit a deftest that raises a SKIP error for tests depending on
unimplemented hyperscript features. The test runner records these as
failures so the pass rate reflects real coverage — grep the run output
for 'SKIP:' to enumerate them."""
name = sx_name(test['name'])
raw = test['name'].replace('"', "'")
return (
f' (deftest "{name}"\n'
f' (hs-cleanup!))'
f' (error "SKIP (skip-list): {raw}"))'
)
def emit_untranslatable_test(test):
"""Emit a deftest that raises a SKIP error for tests whose upstream body
our generator could not translate to SX. Same loud-fail semantics as
emit_skip_test; different tag so we can tell the two buckets apart."""
name = sx_name(test['name'])
raw = test['name'].replace('"', "'")
return (
f' (deftest "{name}"\n'
f' (error "SKIP (untranslated): {raw}"))'
)
@@ -1486,10 +1513,13 @@ for cat, tests in categories.items():
output.append(sx)
total += 1
cat_gen += 1
# SKIP emissions still go through generate_test() → emit_skip_test;
# detect them here so the counter reports real coverage.
if 'SKIP (' in sx:
cat_stub += 1
cat_gen -= 1
else:
safe_name = t['name'].replace('"', "'")
output.append(f' (deftest "{safe_name}"')
output.append(f' (hs-cleanup!))')
output.append(emit_untranslatable_test(t))
total += 1
cat_stub += 1