HS tests: scrape v0.9.90 upstream in full, flip silent stubs to loud SKIPs
- scrape-hs-upstream.py: new scraper walks /tmp/hs-upstream/test/**/*.js
and emits body-style records for all 1,496 v0.9.90 tests (up from 831).
Widens coverage into 66 previously-missing categories — templates,
reactivity, behavior, worker, classRef, make, throw, htmx, tailwind,
viewTransition, and more.
- build-hs-manifest.py + hyperscript-upstream-manifest.{json,md}:
coverage manifest tagging each upstream test with a status
(runnable / skip-listed / untranslated / missing) and block reason.
- generate-sx-tests.py: emit (error "SKIP (...)") instead of silent
(hs-cleanup!) no-op for both skip-listed tests and generator-
untranslatable bodies. Stub counter now reports both buckets.
- hyperscript-feature-audit-0.9.90.md: gap audit against the 0.9.90
spec; pre-0.9.90.json backs up prior 831-test snapshot.
New honest baseline (ocaml runner, test-hyperscript-behavioral):
831 -> 1,496 tests; 645 -> 1,013 passing (67.7% conformance).
483 failures split: 45 skip-list, 151 untranslated, 287 real.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
446
spec/tests/hyperscript-feature-audit-0.9.90.md
Normal file
446
spec/tests/hyperscript-feature-audit-0.9.90.md
Normal file
@@ -0,0 +1,446 @@
|
||||
# _hyperscript 0.9.90 Gap Audit
|
||||
|
||||
**Date:** 2026-04-22
|
||||
**Target release:** upstream 0.9.90 (2026-04-13), with awareness of 0.9.91 bugfixes (2026-04-14)
|
||||
**Supersedes:** `hyperscript-feature-audit.md` (2026-04-09)
|
||||
|
||||
Our implementation lives in `/root/rose-ash/lib/hyperscript/`:
|
||||
- `tokenizer.sx` (618 lines), `parser.sx` (2414 lines), `compiler.sx` (1739 lines), `runtime.sx` (1609 lines), `integration.sx` (109 lines).
|
||||
|
||||
Tests in `spec/tests/test-hyperscript-*.sx` + extracted fixture `hyperscript-upstream-tests.json` (831 entries).
|
||||
|
||||
---
|
||||
|
||||
## 1. Version alignment
|
||||
|
||||
### What release is our JSON snapshot from?
|
||||
|
||||
The fixture `spec/tests/hyperscript-upstream-tests.json` and the prior audit were both written on **2026-04-09** (four days before 0.9.90 tagged on 2026-04-13). File timestamps confirm this; there is no earlier git history for the JSON (it first appears in commit `7492ceac`).
|
||||
|
||||
The contents are clearly **0.9.90 (pre-tag) or equivalent**:
|
||||
|
||||
- 63 `on` tests, 44 `bind`, 41 `when`, 23 `live`, 10 `liveTemplate`, 19 `component`, 4 `reactive-properties` — all new-in-0.9.90 reactivity features
|
||||
- 10 `morph`, 8 `reset`, 13 `empty`, 10 `dialog`, 4 `swap`, 7 `halt`, 5 `askAnswer`, 3 `focus`, 8 `scroll`, 23 `fetch` including `fetch:beforeRequest`/`fetch:error` lifecycle — all 0.9.90 features or overhauls
|
||||
- Keywords `between`, `ignoring` (case), `precedes`/`follows`, `where`, `sorted`/`mapped`/`split`/`joined by` — 0.9.90 collection expressions
|
||||
- Total 831 tests vs upstream's "~1332 tests" claim for 0.9.90 — we have the subset that was non-sinon, non-script-tag, non-dialog-API when extracted
|
||||
|
||||
**Conclusion:** the JSON is a valid 0.9.90-era snapshot. No re-extraction is strictly required, but upstream added tests post-tag (changelog mentions 1332 tests at 0.9.90 release) — we may be missing late additions.
|
||||
|
||||
### What 0.9.90 added over 0.9.8 / 0.9.14
|
||||
|
||||
Full changelog consulted at `github.com/bigskysoftware/_hyperscript/blob/master/CHANGELOG.md`.
|
||||
|
||||
**Breaking changes in 0.9.90:**
|
||||
- `processNode()` → `process()`
|
||||
- `as JSON` now parses a string; old stringify behavior → `as JSONString`; `as Values | JSONString` / `as Values | FormEncoded` replace `Values:JSON` / `Values:Form`
|
||||
- `default` uses **nullish** check (no longer overwrites `0`/`false`)
|
||||
- `go to url ...` deprecated → `go to /path` or `go to "url"`
|
||||
- `go to the top of ...` deprecated → `scroll to the top of`
|
||||
- `async` keyword removed (was confusing)
|
||||
- `/* */` block comments removed
|
||||
- `transition width` → `transition *width` (explicit `*` style ref)
|
||||
- `fetch` throws on non-2xx by default; `do not throw` or `as Response` restores old behavior
|
||||
- `[@attr]` bracket-style attribute access **deprecated** (still parsed, use `@attr`)
|
||||
|
||||
**New features in 0.9.90:**
|
||||
- **Reactivity** (`live`, `when ... changes`, `bind`) — dependency tracking + UI updates
|
||||
- **Templates in core** — `render` command + `<template>` elements + `${}` interpolation + `#for`/`#if`/`#else`/`#end`
|
||||
- **`morph`** command (idiomorph-based DOM morph)
|
||||
- **Components system** — `<template component="name">` custom elements with slots
|
||||
- **DOM-scoped variables** (`^name`)
|
||||
- `open` / `close`, `focus` / `blur`, `empty` / `clear`, `reset`, `swap`, `select`, `ask` / `answer`, `speak`, `breakpoint`
|
||||
- **`toggle between`** attributes: `toggle between [@data-state='active'] and [@data-state='inactive']`
|
||||
- **`toggle x between a, b and c`** N-way cycle
|
||||
- **Collection expressions**: `where`, `sorted by`, `mapped to`, `split by`, `joined by`
|
||||
- **Magic symbols**: `clipboard`, `selection`
|
||||
- `on resize` (ResizeObserver), `on first click` (one-shot)
|
||||
- `view transition` command (View Transitions API)
|
||||
- `intercept` feature (service-worker DSL)
|
||||
- **Pipe operator** `|` for chained conversions (`as Values | JSONString`)
|
||||
- `repeat ... until x end` / `repeat ... while x end` (bottom-tested)
|
||||
- `ignoring case` modifier on comparisons
|
||||
- `between`, `starts with`, `ends with` comparisons
|
||||
- `put null into @attr` removes attribute
|
||||
- DOM expressions (`#id`, `.class`, `<sel/>`) are writable via `.replaceWith()`
|
||||
- `set arr to arr + [1, 2]` array concatenation
|
||||
- `cleanup()` API + `hyperscript:before/after:init` / `:cleanup` lifecycle events + `data-hyperscript-powered`
|
||||
|
||||
**0.9.91 bugfixes** (worth tracking):
|
||||
- `on resize from window/document` falls through to native resize (not ResizeObserver)
|
||||
- `toggle ... for <duration>` no longer consumes a following `for-in`
|
||||
- `${}` template hang fix (debug log leftover)
|
||||
|
||||
---
|
||||
|
||||
## 2. Category coverage gap
|
||||
|
||||
Our behavioral test file `test-hyperscript-behavioral.sx` contains 67 `defsuite` blocks covering 52 unique JSON categories. The file has **831 `eval-hs` calls matching the 831 JSON entries** — every upstream test **has** a behavioral entry, but **89 of them are replaced with safe no-op / `NOT-IMPLEMENTED` stubs** (commit `71cf5b84`).
|
||||
|
||||
Per-category upstream size vs. stub count:
|
||||
|
||||
| Category | Upstream tests | Stubs in behavioral | Simple-clean upstream (estimate) | Status |
|
||||
|----------|---------------:|-------------------:|--------:|--------|
|
||||
| on | 63 | 32 | ~30 | many non-simple (event queue / debounced / throttled / intersection / mutation) |
|
||||
| bind | 44 | ~44 | 1 | almost entirely new-0.9.90, not implemented |
|
||||
| when | 41 | ~36 | 5 | `when ... changes` reactivity mostly missing |
|
||||
| comparisonOperator | 40 | ~36 | 4 | `starts with` / `ends with` / `between` / `ignoring case` mostly missing |
|
||||
| put | 38 | 8 | 37 | implemented; stubs are complex positional forms |
|
||||
| toggle | 30 | 2 | 28 | strong coverage |
|
||||
| repeat | 30 | 4 | 23 | `break` / `continue` / `until` now work (recent commits) |
|
||||
| def | 27 | 7 | 3 | basic only; async/catch variants partial |
|
||||
| set | 25 | 7 | 24 | strong; stubs are array-concat and rare forms |
|
||||
| dom-scope | 25 | — | 23 | 23/25 upstream simple — `^var` works |
|
||||
| transition | 23 | 1 | 22 | strong |
|
||||
| live | 23 | — | 0 | 0/23 clean-simple — **reactivity runtime missing** |
|
||||
| fetch | 23 | 3 | 6 | 11/23 passing per recent commit `5c66095b`; sinon-required tests stubbed |
|
||||
| collectionExpressions | 22 | — | 5 | `where` / `sorted by` / `mapped to` / `split by` / `joined by` — **new 0.9.90, mostly missing** |
|
||||
| increment | 20 | — | 20 | full |
|
||||
| if | 19 | — | 18 | full |
|
||||
| component | 19 | — | 14 | 14/19 simple — `<template component>` custom-element bridge missing on JS host |
|
||||
| add | 19 | 4 | 18 | full minus `[@attr]` / `{css}` forms |
|
||||
| asExpression | 17 | — | 0 | 0/17 simple — `as JSON/JSONString/FormEncoded` + pipe operator missing |
|
||||
| remove | 14 | 4 | 12 | strong |
|
||||
| hide | 14 | — | 14 | full |
|
||||
| bootstrap | 14 | — | 2 | 2/14 simple — lifecycle events / `cleanup()` / `data-hyperscript-powered` not covered |
|
||||
| empty | 13 | — | 12 | full (recent commit `802ccd23`) |
|
||||
| append | 13 | 5 | 13 | strong |
|
||||
| take | 12 | — | 12 | full |
|
||||
| tell | 10 | 6 | 10 | parses; stubs are the multi-statement tell variants |
|
||||
| morph | 10 | — | 4 | 4/10 simple — string target variant works, element-target variants stubbed |
|
||||
| liveTemplate | 10 | — | 7 | 7/10 simple — **no implementation of `<template>` + `#for`/`#if`** |
|
||||
| dialog | 10 | 1 | 5 | 5/10 simple — dialog `open`/`close` partial |
|
||||
| default | 9 | — | 9 | full |
|
||||
| send | 8 | — | 8 | full |
|
||||
| scroll | 8 | — | 0 | 0/8 simple — **`scroll to` command not implemented** |
|
||||
| reset | 8 | — | 8 | full (recent commit `802ccd23`) |
|
||||
| evalStatically | 8 | — | 0 | 0/8 simple — **static eval API not exposed** |
|
||||
| assignableElements | 8 | — | 3 | partial |
|
||||
| wait | 7 | — | 7 | full |
|
||||
| splitJoin | 7 | — | 0 | 0/7 simple — `split by` / `joined by` absent |
|
||||
| pick | 7 | — | 0 | 0/7 simple — regex/char/item pick missing |
|
||||
| parser | 7 | — | 3 | partial |
|
||||
| halt | 7 | — | 6 | full (recent commit `802ccd23`) |
|
||||
| call | 6 | 7 | 5 | partial — `call functions that return promises` stubbed |
|
||||
| no | 5 | — | 1 | `no <expr>` predicate mostly missing |
|
||||
| mathOperator | 5 | — | 0 | 0/5 simple — `mod`/`abs` + async math promise-aware ops missing |
|
||||
| go | 5 | — | 3 | basic `go to` works; new `scroll to` form missing |
|
||||
| askAnswer | 5 | — | 0 | 0/5 simple — `ask` / `answer` not parsed |
|
||||
| swap | 4 | — | 4 | full |
|
||||
| socket | 4 | — | 0 | extension, out of scope |
|
||||
| select | 4 | — | 0 | 0/4 — `select` command not parsed (only compiler stub) |
|
||||
| relativePositionalExpression | 4 | — | 0 | 0/4 — `first in`, `last in`, `random in` collection-relative expressions missing |
|
||||
| reactive-properties | 4 | — | 0 | 0/4 — property-reactivity runtime not present |
|
||||
| log | 4 | — | 4 | full |
|
||||
| resize | 3 | — | 0 | 0/3 — `on resize` synthetic event missing |
|
||||
| logicalOperator | 3 | — | 0 | 0/3 — async-safe `and`/`or` promise propagation absent |
|
||||
| init | 3 | — | 1 | partial |
|
||||
| focus | 3 | — | 0 | 0/3 — `focus` / `blur` commands not wired |
|
||||
| closest | 3 | — | 1 | partial |
|
||||
| show | 2 | — | 2 | full |
|
||||
| measure | 2 | — | 0 | 0/2 — `measure` command stubbed |
|
||||
| asyncError | 2 | — | 0 | 0/2 — async error propagation |
|
||||
| settle | 1 | — | 1 | full |
|
||||
| scoping | 1 | — | 1 | — |
|
||||
| queryRef | 1 | — | 0 | `<sel/>` selector refs — tokenizer skips |
|
||||
| objectLiteral | 1 | — | 0 | JS-object literal in hyperscript expression missing |
|
||||
| js | 1 | — | 1 | `js ... end` inline JS — we parse but don't emit real JS (no JS host) |
|
||||
| in | 1 | — | 0 | `in` collection op missing |
|
||||
| cookies | 1 | — | 0 | `cookies.name` magic symbol missing |
|
||||
| attributeRef | 1 | — | 0 | `@attr` attribute-ref expressions |
|
||||
|
||||
**Categories with 0 simple-clean coverage** (highest priority): `asExpression` (17), `splitJoin` (7), `pick` (7), `mathOperator` (5), `askAnswer` (5), `scroll` (8), `evalStatically` (8), `socket` (4), `select` (4), `relativePositionalExpression` (4), `reactive-properties` (4), `resize` (3), `logicalOperator` (3), `focus` (3), `measure` (2), `asyncError` (2), `queryRef` (1), `objectLiteral` (1), `in` (1), `cookies` (1), `attributeRef` (1), **`live` (23)**, **`bind` (43)**, **`collectionExpressions` (17)**.
|
||||
|
||||
---
|
||||
|
||||
## 3. Feature gaps (parser / compiler / runtime)
|
||||
|
||||
### Commands
|
||||
|
||||
| Command | tokenizer | parser | compiler | runtime | Notes |
|
||||
|---------|:-:|:-:|:-:|:-:|-------|
|
||||
| add | yes | yes | yes | yes | complete minus `[@attr]` / `{css-block}` forms (new 0.9.90 block-CSS added in `b23da319` ✓) |
|
||||
| remove | yes | yes | yes | yes | as above |
|
||||
| toggle | yes | yes | yes | yes | inc. `between` and N-way `cycle` (commit `00bf13a2`) |
|
||||
| set | yes | yes | yes | yes | array concat form (`set a to a + [1]`) untested |
|
||||
| put | yes | yes | yes | yes | most positional forms |
|
||||
| append | yes | yes | yes | yes | landed `c8aab54d` |
|
||||
| prepend | **no** | **no** | **no** | **no** | not a 0.9.90 core command — only `insert before` via `put`, ignore |
|
||||
| transition | yes | yes | yes | yes | inc. possessives |
|
||||
| show / hide | yes | yes | yes | yes | `when` clause on `hide` (new 0.9.90) untested |
|
||||
| wait / settle | yes | yes | yes | yes | via `perform` IO suspension |
|
||||
| send / trigger | yes | yes | — | partial | `send` lands in compiler dispatch; `trigger` alias |
|
||||
| call / get | yes | yes | yes | partial | `call functions that return promises` stubbed |
|
||||
| log | yes | yes | — | partial | behavior OK |
|
||||
| fetch | yes | yes | yes | yes | 11/23 passing; non-2xx `throw` + `as Response` + `do not throw` missing |
|
||||
| throw | yes | yes | yes | yes | `raise`-based |
|
||||
| continue / break | yes | yes | yes | yes | landed `f200418d` |
|
||||
| return / exit | yes | yes | yes | yes | guard-based (commit `97818c6d`) |
|
||||
| halt | yes | yes | yes | yes | modes landed `922e7a78` |
|
||||
| repeat | yes | yes | yes | yes | `forever`, `while`, `until`, `times`, `for-in`, `index` |
|
||||
| for | yes | yes | yes | yes | alias of `repeat for` |
|
||||
| if / unless / else / end / then / with | yes | yes | yes | yes | `unless` recognized but not parsed as separate — unchanged from prior audit |
|
||||
| js | yes | (no) | (no) | (no) | embedded JS — we can't execute JS without a JS host |
|
||||
| go | yes | yes | yes | yes | basic `go to <path>`; `go to the top/bottom of` **deprecated** in 0.9.90 — replaced by `scroll to` |
|
||||
| scroll | yes | yes | yes | partial | `scroll to the top/bottom of` — parser has 2 hits, but **0/8 simple tests pass** — runtime likely does not dispatch all forms |
|
||||
| push/pull state | no | no | no | no | history API — not in 0.9.90 core docs either |
|
||||
| pick | yes | yes | yes | yes | landed `41cfa562`; 0/7 simple — pick command needs item/char/regex runtime |
|
||||
| render | yes | yes | — | — | parser has 8 hits but NO compiler emit — **`render` + `<template>` interpolation not implemented** |
|
||||
| make | yes | yes | yes | yes | `hs-make` + `hs-make-object` |
|
||||
| increment / decrement | yes | yes | yes | yes | full |
|
||||
| measure | yes | yes | yes | partial | 0/2 — runtime stub incomplete |
|
||||
| tell | yes | yes | — | partial | parses, compiler handles single-cmd tell |
|
||||
| take | yes | yes | yes | yes | full |
|
||||
| focus | yes | yes | — | — | parser has 2 hits; **no compiler emit, 0/3 simple pass** |
|
||||
| blur | **no** | **no** | **no** | **no** | 0.9.90 addition, absent |
|
||||
| hide the / show the | — | — | — | — | alt grammar variant; not tracked separately |
|
||||
| beep! | yes | yes | yes | yes | (runtime no-op) |
|
||||
| trim / cut / copy / paste | **no** | **no** | **no** | **no** | clipboard magic symbols + commands absent |
|
||||
| media | **no** | **no** | **no** | **no** | not in 0.9.90 core |
|
||||
| dialog | **no keyword** | yes | yes | partial | parser parses dialog-open/close forms, 5/10 simple pass; commit `802ccd23` recent |
|
||||
| morph | yes | yes | yes | yes | landed `5b0c8569`; 4/10 simple |
|
||||
| empty / clear | yes | yes | yes | yes | landed `802ccd23` |
|
||||
| reset | yes | yes | yes | yes | landed (form reset `9d246f5c`) |
|
||||
| swap | yes | yes | — | partial | tokenizer has it; compiler dispatch missing — 4/4 simple currently pass? (verify) |
|
||||
| open / close | yes | yes | yes | partial | dialog/details/popover/fullscreen variants |
|
||||
| select | yes | yes | partial | partial | parser + compiler dispatch; `select` command 0/4 simple — runtime not complete |
|
||||
| ask / answer | **no** | **no** | **no** | **no** | 0/5 simple — new 0.9.90, absent |
|
||||
| speak | **no** | **no** | **no** | **no** | 0.9.90 addition, absent (Web Speech API) |
|
||||
| breakpoint | **no** | **no** | **no** | **no** | 0.9.90 addition, absent |
|
||||
| view transition | no | no | no | no | 0.9.90 addition, absent |
|
||||
| intercept | no | no | no | no | 0.9.90 service-worker DSL, out of scope |
|
||||
|
||||
### Expressions / operators
|
||||
|
||||
| Expression | Status | Notes |
|
||||
|------------|--------|-------|
|
||||
| `me`, `my`, `it`, `its`, `result`, `event`, `detail`, `target` | **yes** | tokenizer & parser |
|
||||
| `that` | **no** | not in tokenizer — uncommon, low priority |
|
||||
| `element` | partial | parser has 1 hit |
|
||||
| `body`, `document`, `window` | **no** | not tokenized — **high-value gap** for e.g. `on resize from window` |
|
||||
| `you`, `yourself`, `sender` | partial | `you`/`sender` tokenized, not parsed |
|
||||
| `response` | **no** | fetch response magic — used in error handlers |
|
||||
| `clipboard`, `selection` | **no** | 0.9.90 additions |
|
||||
| `first/last/random in` | partial | tokens; `random` has 1 parser hit — 0/4 relativePositionalExpression tests pass |
|
||||
| `closest` | yes | 1/3 simple |
|
||||
| `next`, `previous` | yes | tokenized |
|
||||
| `parent of`, `children of` | **no** | **not tokenized — parent/children expressions missing** |
|
||||
| class ref (`.foo`), id ref (`#foo`) | yes | tokenizer handles |
|
||||
| selector ref (`<sel/>`) | partial | tokenizer has `<` but not `/>` closer — **88 JSON tests use this** |
|
||||
| attribute ref (`@attr`) | yes | but `[@attr]` bracket form deprecated — we likely have mixed support |
|
||||
| style ref (`*prop`) | yes | parser has 7 hits |
|
||||
| CSS block (`{color: red}`) | **no** | — **19 JSON tests use this** |
|
||||
| string interpolation `${...}` | **no** | — **21 JSON tests use this** |
|
||||
| time refs (`1s`, `1ms`) | yes | parse-duration |
|
||||
| math `+ - * /` | yes | via evaluator |
|
||||
| `mod` | yes tok | 0/5 simple — runtime doesn't propagate promises |
|
||||
| `abs` | **no tok** | absent |
|
||||
| floor/ceil/round | **no** | absent as hyperscript keywords (only via JS interop) |
|
||||
| `is` / `is not` / `==` | yes | |
|
||||
| `matches` / `contains` | yes | inc. ignore-case (`ef5faa6b`) |
|
||||
| `starts with` / `ends with` | yes | parser has both with ignore-case variants |
|
||||
| `between ... and ...` | yes | 9 parser hits; may not cover comparison form |
|
||||
| `precedes` / `follows` | yes | 3 parser hits each |
|
||||
| `ignoring case` | yes | 1 tokenizer hit; needs coverage per-primitive |
|
||||
| `in` / `not in` | yes-partial | 3 tests |
|
||||
| `empty` (predicate) | yes | |
|
||||
| `exists` | partial | dropped short-circuit (`ef5faa6b`); 1 parser hit |
|
||||
| `no <expr>` | partial | 1/5 simple |
|
||||
| `not`, `and`, `or` | yes | short-circuit async missing → 0/3 logicalOperator simple |
|
||||
| async `and`/`or` | **no** | **0/3 logicalOperator simple; needs promise propagation** |
|
||||
| `some` | partial | tokenized |
|
||||
| collection: `where` | yes tok | 1 parser hit — 0 working simple tests |
|
||||
| collection: `sorted by` / `sorted by desc` | yes tok | runtime `hs-sorted-by` / `-desc` present, not wired from parser for all forms |
|
||||
| collection: `mapped to` | yes tok | — |
|
||||
| collection: `split by` / `joined by` | yes tok | runtime `hs-split-by` / `hs-joined-by` present — 0/7 splitJoin simple |
|
||||
| type conversion `as Foo` | partial | 4/17 asExpression simple — `JSONString` / `FormEncoded` / **pipe `|`** absent |
|
||||
| pipe `|` for conversions | **no** | 0.9.90 addition |
|
||||
| ternary / `if ... else ...` expression | partial | command-form only |
|
||||
| object literal `{key: val}` | partial | 0/1 — special parser |
|
||||
|
||||
### Event syntax / feature forms
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| `on <event>` | yes | core |
|
||||
| `on every <event>` | yes | `hs-on-every` |
|
||||
| `on <event> from <target>` | yes | inc. `window` / `document` textual — but **tokenizer lacks `window`/`document`** |
|
||||
| event queueing (`queue all/none/first/last`) | **no** | 0 parser hits for `queue` — new in 0.9.90-era polish |
|
||||
| `debounced at <duration>` | **no** | 0 parser hits |
|
||||
| `throttled at <duration>` | **no** | 0 parser hits |
|
||||
| `halt the event` / `halt bubbling` / `halt default` | yes | `mode` form in parser line 1953 |
|
||||
| `on <event> when <cond>` | yes | when-cond parser |
|
||||
| `on intersection` / `on mutation` observers | **no** | needs `IntersectionObserver`/`MutationObserver` wiring |
|
||||
| `on resize` (synthetic) | **no** | 0/3 resize tests; `ResizeObserver` adapter missing |
|
||||
| `on first <event>` | **no** | one-shot modifier |
|
||||
| `on load` / `init` | yes | in feat parser |
|
||||
| `on <event> in <selector>` | partial | delegation |
|
||||
| `behavior Name ... install <Name>` | yes (parsed) | behavior parsing at line 2390+ |
|
||||
| `def name(args) ... end` | yes | (commit `9d246f5c`) |
|
||||
| `worker` | **no** | extension, out of scope |
|
||||
| `live` feature | partial | tokenizer has `live`, parser 3 hits; **no reactivity runtime** → 0/23 live simple |
|
||||
| `bind` feature | **no** | tokenizer + parser both 0 hits — **biggest single feature gap** (44 upstream tests) |
|
||||
| `when <expr> changes` | partial | `changes` is tokenized; reactive wiring missing |
|
||||
|
||||
### Scope
|
||||
|
||||
| Feature | Status | Notes |
|
||||
|---------|--------|-------|
|
||||
| locals (`set x ...`) | yes | full |
|
||||
| element-scope (`me.x`, `my x`) | yes | `my`/`your` alias |
|
||||
| globals (`$x`) | yes | |
|
||||
| dotted names | yes | |
|
||||
| `local x` | partial | tokenized |
|
||||
| DOM-scope `^var` | yes | 23/25 simple — strong coverage |
|
||||
|
||||
---
|
||||
|
||||
## 4. Test complexity buckets we're NOT running
|
||||
|
||||
| Bucket | Upstream count | Our coverage | What's missing |
|
||||
|--------|---------------:|-------------:|----------------|
|
||||
| `simple` | 469 | ~454 via behavioral (real) + ~89 stubs | 15 skipped for pattern reasons |
|
||||
| `evaluate` | 125 | **0** as evaluate-typed tests; some merged into behavioral | DOM-mutation-after-eval tests — need real browser, not sandbox |
|
||||
| `run-eval` | 83 | **0 distinct** | Mixed run+eval — possible via sandbox with state reads |
|
||||
| `promise` | 57 | partial via `perform` | Async promise chains — we have IO suspension but no sinon stub |
|
||||
| `eval-only` | 39 | partial | Pure expression tests — easy to add (just run `eval-hs` and assert result) |
|
||||
| `script-tag` | 36 | **0** | `<script type="text/hyperscript">` bootstrap path — we don't simulate the script-tag loader |
|
||||
| `sinon` | 17 | **0** | Fetch stubs — our fetch tests mock via `host-*` server stubs, not sinon. 11/23 fetch pass, but the 17 sinon-specific tests aren't run |
|
||||
| `dialog` | 5 | partial | `<dialog>` element's `showModal()` / `close()` — needs dialog API mock |
|
||||
|
||||
**What each bucket would need:**
|
||||
|
||||
- **`eval-only` and `run-eval`** — cheapest wins. These just need `eval-hs` with result assertion. Missing infrastructure: none (we already have this in `test-framework.sx`). Gap is that the extractor tagged them `eval-only` so they weren't included in the "simple" migration.
|
||||
- **`evaluate`** — needs DOM mutation → DOM query. Our sandbox has mock DOM so most should be portable.
|
||||
- **`script-tag`** — needs script-tag loader: scan HTML for `<script type="text/hyperscript">`, compile and run. Separate from `_=` attribute bootstrap. Not hard — just a hook in integration.sx.
|
||||
- **`sinon`** — needs fetch mocking per-test. We have `host-*` server stubs globally. Fix: extend `hs-fetch` mock to support per-test route registration (some already done in `673be857`).
|
||||
- **`dialog`** — needs `HTMLDialogElement.showModal()` / `.close()` + `returnValue` attribute. Minimal mock.
|
||||
- **`promise`** — most already work via `perform`. The ones that don't involve `Promise.resolve(x)` wrapping in JS (`call functions that return promises`). Needs awaiter mock.
|
||||
|
||||
---
|
||||
|
||||
## 5. Pattern classes the tokenizer skips
|
||||
|
||||
Global JSON counts (from `html + check + action` concatenation):
|
||||
|
||||
| Pattern | Upstream tests using it | Upstream feature | Our parser |
|
||||
|---------|----------------------:|------------------|------------|
|
||||
| `[@attr="val"]` bracket attribute | 6 | **Deprecated in 0.9.90** — attribute-access/mutation syntax | tokenizer doesn't emit `[@` pair — 4 add, 4 toggle, 1 remove tests skipped |
|
||||
| `${...}` template interpolation | 21 | String interpolation in expressions and templates | Not tokenized — blocks all `live`/`liveTemplate`/`render` tests (~35 tests) |
|
||||
| `<sel/>` selector ref | 88 | Inline CSS-selector reference as a value | Tokenizer has `<` but no closing-slash detection |
|
||||
| `{prop:val;...}` CSS block | 19 | CSS-property block in `add`/`remove`/`toggle` | Tokenizer has `{`/`}` but no CSS-block parsing (partially landed `b23da319` for `add`; not for `remove`/`toggle`) |
|
||||
|
||||
**Parser surface:**
|
||||
- Attribute ref `@foo` — yes (as expression), 1/1 simple attributeRef test still 0 pass → compiler/runtime gap
|
||||
- Selector ref `<sel/>` — tokenizer returns `"op"` for `/`, parser has no rule to assemble as a single value
|
||||
- String interp `${x}` — tokenizer returns `"str"` but doesn't split on `${...}`; no parser for expression embedding
|
||||
- CSS block — tokenizer consumes `{`/`}` as punctuation; no recognition of CSS-property list inside
|
||||
|
||||
**Recommendation:**
|
||||
- `[@attr]` — since 0.9.90 deprecated it, don't invest
|
||||
- `${...}` — required for `live`, `liveTemplate`, `render`, `bind`. High priority
|
||||
- `<sel/>` — 88 tests blocked; large ROI. Tokenizer change + parser rule
|
||||
- `{css}` — already partially done; extend to `remove` / `toggle`
|
||||
|
||||
---
|
||||
|
||||
## 6. Summary table
|
||||
|
||||
| Category | Upstream | Impl (parse+compile+run) | Simple tests passing | Gap class |
|
||||
|----------|---------:|:-----------------------:|--------------------:|-----------|
|
||||
| add | 19 | yes (minus `[@`/`{css}` forms) | 18 | skip patterns |
|
||||
| remove | 14 | yes | 12 | skip patterns |
|
||||
| toggle | 30 | yes (inc. between/cycle) | 28 | skip patterns |
|
||||
| set | 25 | yes | 24 | array-concat |
|
||||
| put | 38 | yes | 37 | complex positionals |
|
||||
| append | 13 | yes | 13 | — |
|
||||
| transition | 23 | yes | 22 | — |
|
||||
| show | 2 | yes | 2 | — |
|
||||
| hide | 14 | yes | 14 | — |
|
||||
| wait | 7 | yes | 7 | — |
|
||||
| send/trigger | 8 | yes | 8 | — |
|
||||
| call | 6 | partial | 5 | promise await |
|
||||
| fetch | 23 | partial | 11 | non-2xx throw, `do not throw`, pipe `|`, sinon |
|
||||
| throw/catch/finally | incl. def | yes | — | — |
|
||||
| if/unless/else | 19 | yes | 18 | — |
|
||||
| repeat | 30 | yes | 23 | `index`, `by` step |
|
||||
| for | — | yes | — | — |
|
||||
| def | 27 | partial | 3 | async/catch variants |
|
||||
| behavior/install | — | parses | — | — |
|
||||
| init | 3 | partial | 1 | immediate init variants |
|
||||
| increment/decrement | 20 | yes | 20 | — |
|
||||
| take | 12 | yes | 12 | — |
|
||||
| tell | 10 | partial | 10 | multi-stmt tell |
|
||||
| halt | 7 | yes | 6 | — |
|
||||
| empty/clear | 13 | yes | 12 | — |
|
||||
| reset | 8 | yes | 8 | — |
|
||||
| swap | 4 | partial-runtime | 4 | — |
|
||||
| open/close | — | partial | — | dialog/popover |
|
||||
| dialog | 10 | partial | 5 | dialog API mock |
|
||||
| morph | 10 | yes | 4 | element-target forms |
|
||||
| focus | 3 | **no** | 0 | not wired through compile |
|
||||
| **blur** | — | **no** | — | absent |
|
||||
| scroll | 8 | partial | 0 | runtime dispatch incomplete |
|
||||
| measure | 2 | partial | 0 | runtime stub |
|
||||
| pick | 7 | partial | 0 | runtime item/regex modes |
|
||||
| select | 4 | partial | 0 | select-command runtime |
|
||||
| **bind** | 44 | **no** | 1 | **reactivity runtime** |
|
||||
| **live** | 23 | **no** | 0 | **reactivity runtime** |
|
||||
| **when-changes** | 41 | partial | 5 | reactivity deps |
|
||||
| **reactive-properties** | 4 | **no** | 0 | property reactivity |
|
||||
| **liveTemplate** | 10 | **no** | 7 | `${}` + `<template>` |
|
||||
| **component** | 19 | partial | 14 | custom-element register |
|
||||
| dom-scope (`^var`) | 25 | yes | 23 | — |
|
||||
| **collectionExpressions** | 22 | partial | 5 | where/sorted/mapped/split/joined |
|
||||
| **comparisonOperator** | 40 | partial | 4 | starts/ends/between/ic async |
|
||||
| logicalOperator | 3 | partial | 0 | async and/or |
|
||||
| mathOperator | 5 | partial | 0 | mod/abs + async |
|
||||
| **asExpression** | 17 | partial | 0 | JSONString/FormEncoded + pipe |
|
||||
| **askAnswer** | 5 | **no** | 0 | `ask`/`answer` keywords |
|
||||
| **speak** | — | **no** | — | Web Speech API |
|
||||
| **breakpoint** | — | **no** | — | devtools breakpoint |
|
||||
| **intercept** | — | no | — | service-worker (out of scope) |
|
||||
| **view transition** | — | **no** | — | View Transitions API |
|
||||
| **resize** | 3 | **no** | 0 | ResizeObserver synthetic event |
|
||||
| bootstrap | 14 | partial | 2 | lifecycle events + cleanup() |
|
||||
| evalStatically | 8 | **no** | 0 | `_hyperscript.evaluate(...)` static API |
|
||||
| relativePositionalExpression | 4 | partial | 0 | `first in xs`/`last in xs` |
|
||||
| assignableElements | 8 | partial | 3 | — |
|
||||
| go | 5 | partial | 3 | `go to the top of` deprecated → `scroll to` |
|
||||
| in (operator) | 1 | partial | 0 | — |
|
||||
| no (predicate) | 5 | partial | 1 | — |
|
||||
| closest | 3 | partial | 1 | — |
|
||||
| cookies | 1 | **no** | 0 | `cookies.name` magic |
|
||||
| queryRef | 1 | **no** | 0 | `<sel/>` as value |
|
||||
| objectLiteral | 1 | partial | 0 | JS-object literal expr |
|
||||
| attributeRef | 1 | partial | 0 | `@attr` as expression |
|
||||
| js (inline) | 1 | parsed | 1 | — |
|
||||
| splitJoin | 7 | runtime-only | 0 | parser wiring |
|
||||
| asyncError | 2 | partial | 0 | async catch paths |
|
||||
| parser | 7 | partial | 3 | parse-error events |
|
||||
| scoping | 1 | yes | 1 | — |
|
||||
| settle | 1 | yes | 1 | — |
|
||||
|
||||
### Top 5 priority gaps (by upstream test count × foundational weight)
|
||||
|
||||
1. **Reactivity runtime (`bind` + `live` + `when...changes` + `reactive-properties`)** — blocks 112 upstream tests (44 + 23 + 41 + 4). Requires: dependency-tracked evaluation, signal graph per-element, change propagation, two-way binding with dedup. Foundational: unlocks templates and components. Needs `${}` string interp in parser.
|
||||
2. **`${...}` interpolation + `<template>` render + `#for`/`#if`/`#else` control flow (`render` + `liveTemplate`)** — blocks ~35 tests but is foundational for templating. Tokenizer change required (`${...}` inside strings); new `render-command` runtime; template AST with `#` tags.
|
||||
3. **Comparison + collection + conversion operator completion (`comparisonOperator` + `collectionExpressions` + `asExpression` + `splitJoin`)** — 86 tests. Already have primitives (`hs-split-by`, `hs-joined-by`, `hs-sorted-by`, `hs-starts-with-ic`, etc.) but parser doesn't wire them. Lower-risk plumbing work.
|
||||
4. **Missing simple commands: `focus`/`blur`, `scroll to`, `ask`/`answer`, `speak`, `breakpoint`, `select` (command), `measure`** — ~25 tests, and small per-command scope. `focus`/`blur` + `scroll to` alone unblocks 11 tests. `ask`/`answer` need `host-prompt`/`host-alert`/`host-confirm` IO effects.
|
||||
5. **Event syntax modifiers: `queue`, `debounced at`, `throttled at`, `on first`, `on resize`, `on intersection`, `on mutation`** — 0 parser hits for any of these. Each is small in scope but together they affect test realism. `on resize` with ResizeObserver synth is the most valuable (0.9.91 bug in this area).
|
||||
|
||||
### Secondary gaps (foundational but lower-count)
|
||||
|
||||
- **`body`/`document`/`window` magic symbols** — blocks `on resize from window` and many `scroll` tests. Add to tokenizer.
|
||||
- **`parent of` / `children of` expressions** — not tokenized. Easy win.
|
||||
- **Pipe operator `|`** — required for 0.9.90 chained conversions.
|
||||
- **`fetch` non-2xx throws + `do not throw` + `as Response`** — 0.9.90 breaking change; 12 fetch tests blocked.
|
||||
- **`cleanup()` API + lifecycle events** — needed for `bootstrap` suite (14 tests) and htmx 4 interop.
|
||||
- **Script-tag loader** — 36 tests in `script-tag` bucket blocked for want of a `<script type="text/hyperscript">` parser entry.
|
||||
|
||||
### Non-priorities
|
||||
|
||||
- `worker`, `socket`, `eventsource` — extensions, out of scope
|
||||
- `intercept` — service-worker DSL, out of scope
|
||||
- `view transition` — browser API, nice-to-have
|
||||
- `[@attr]` bracket form — deprecated upstream, don't invest
|
||||
- `js ... end` — can't execute real JS without a JS host; we parse but can't semantically emit
|
||||
13466
spec/tests/hyperscript-upstream-manifest.json
Normal file
13466
spec/tests/hyperscript-upstream-manifest.json
Normal file
File diff suppressed because it is too large
Load Diff
163
spec/tests/hyperscript-upstream-manifest.md
Normal file
163
spec/tests/hyperscript-upstream-manifest.md
Normal file
@@ -0,0 +1,163 @@
|
||||
# _hyperscript v0.9.90 upstream test coverage manifest
|
||||
|
||||
- Upstream tag: `v0.9.90` (commit `a13de2ca`, 2026-04-13)
|
||||
- Our snapshot JSON: `spec/tests/hyperscript-upstream-tests.json` (831 tests, scraped 2026-04-09)
|
||||
- Total upstream tests: **1496**
|
||||
- Runnable (present + not skip-listed): **764** (51.1%)
|
||||
- Skip-listed (present but guarded in generator): **44** (2.9%)
|
||||
- Missing from our snapshot: **688** (46.0%)
|
||||
|
||||
Current conformance: runnable / total = **764/1496 = 51.1%**.
|
||||
|
||||
## Per-category
|
||||
|
||||
| Category | Upstream | Runnable | Skip-listed | Missing |
|
||||
|---|---:|---:|---:|---:|
|
||||
| add | 19 | 19 | 0 | 0 |
|
||||
| api | 1 | 0 | 0 | 1 |
|
||||
| append | 13 | 13 | 0 | 0 |
|
||||
| arrayIndex | 14 | 0 | 0 | 14 |
|
||||
| arrayLiteral | 8 | 0 | 0 | 8 |
|
||||
| asExpression | 42 | 17 | 0 | 25 |
|
||||
| askAnswer | 5 | 5 | 0 | 0 |
|
||||
| assignableElements | 8 | 8 | 0 | 0 |
|
||||
| asyncError | 2 | 2 | 0 | 0 |
|
||||
| attributeRef | 22 | 1 | 0 | 21 |
|
||||
| beep! | 6 | 0 | 0 | 6 |
|
||||
| behavior | 10 | 0 | 0 | 10 |
|
||||
| bind | 44 | 38 | 0 | 6 |
|
||||
| blockLiteral | 4 | 0 | 0 | 4 |
|
||||
| boolean | 2 | 0 | 0 | 2 |
|
||||
| bootstrap | 26 | 14 | 0 | 12 |
|
||||
| breakpoint | 2 | 0 | 0 | 2 |
|
||||
| call | 6 | 6 | 0 | 0 |
|
||||
| classRef | 9 | 0 | 0 | 9 |
|
||||
| closest | 10 | 3 | 0 | 7 |
|
||||
| collectionExpressions | 28 | 22 | 0 | 6 |
|
||||
| comparisonOperator | 83 | 40 | 0 | 43 |
|
||||
| component | 20 | 18 | 0 | 2 |
|
||||
| cookies | 5 | 1 | 0 | 4 |
|
||||
| def | 27 | 24 | 3 | 0 |
|
||||
| default | 15 | 9 | 0 | 6 |
|
||||
| dialog | 12 | 9 | 0 | 3 |
|
||||
| dom-scope | 25 | 25 | 0 | 0 |
|
||||
| empty | 13 | 13 | 0 | 0 |
|
||||
| evalStatically | 8 | 8 | 0 | 0 |
|
||||
| eventsource | 13 | 0 | 0 | 13 |
|
||||
| fetch | 23 | 15 | 8 | 0 |
|
||||
| focus | 3 | 3 | 0 | 0 |
|
||||
| functionCalls | 12 | 0 | 0 | 12 |
|
||||
| go | 5 | 5 | 0 | 0 |
|
||||
| halt | 7 | 7 | 0 | 0 |
|
||||
| hide | 16 | 14 | 0 | 2 |
|
||||
| hs-include | 10 | 0 | 0 | 10 |
|
||||
| htmx | 9 | 0 | 0 | 9 |
|
||||
| idRef | 4 | 0 | 0 | 4 |
|
||||
| if | 19 | 19 | 0 | 0 |
|
||||
| in | 10 | 1 | 0 | 9 |
|
||||
| increment | 20 | 20 | 0 | 0 |
|
||||
| init | 3 | 3 | 0 | 0 |
|
||||
| js | 11 | 1 | 0 | 10 |
|
||||
| live | 23 | 23 | 0 | 0 |
|
||||
| liveTemplate | 16 | 10 | 0 | 6 |
|
||||
| log | 4 | 4 | 0 | 0 |
|
||||
| logicalOperator | 10 | 3 | 0 | 7 |
|
||||
| make | 8 | 0 | 0 | 8 |
|
||||
| mathOperator | 15 | 5 | 0 | 10 |
|
||||
| measure | 6 | 2 | 0 | 4 |
|
||||
| morph | 10 | 10 | 0 | 0 |
|
||||
| no | 9 | 5 | 0 | 4 |
|
||||
| not | 9 | 0 | 0 | 9 |
|
||||
| null | 1 | 0 | 0 | 1 |
|
||||
| numbers | 1 | 0 | 0 | 1 |
|
||||
| objectLiteral | 12 | 1 | 0 | 11 |
|
||||
| on | 70 | 27 | 33 | 10 |
|
||||
| parser | 14 | 7 | 0 | 7 |
|
||||
| pick | 24 | 7 | 0 | 17 |
|
||||
| positionalExpression | 7 | 0 | 0 | 7 |
|
||||
| possessiveExpression | 23 | 0 | 0 | 23 |
|
||||
| propertyAccess | 12 | 0 | 0 | 12 |
|
||||
| pseudoCommand | 11 | 0 | 0 | 11 |
|
||||
| put | 38 | 38 | 0 | 0 |
|
||||
| queryRef | 13 | 1 | 0 | 12 |
|
||||
| reactive-properties | 4 | 4 | 0 | 0 |
|
||||
| reactivity | 8 | 0 | 0 | 8 |
|
||||
| regressions | 16 | 0 | 0 | 16 |
|
||||
| relativePositionalExpression | 23 | 4 | 0 | 19 |
|
||||
| remove | 19 | 14 | 0 | 5 |
|
||||
| repeat | 30 | 29 | 0 | 1 |
|
||||
| reset | 8 | 8 | 0 | 0 |
|
||||
| resize | 3 | 3 | 0 | 0 |
|
||||
| runtime | 7 | 0 | 0 | 7 |
|
||||
| runtimeErrors | 18 | 0 | 0 | 18 |
|
||||
| scoping | 20 | 1 | 0 | 19 |
|
||||
| scroll | 8 | 8 | 0 | 0 |
|
||||
| security | 1 | 0 | 0 | 1 |
|
||||
| select | 4 | 4 | 0 | 0 |
|
||||
| send | 8 | 8 | 0 | 0 |
|
||||
| set | 31 | 25 | 0 | 6 |
|
||||
| settle | 3 | 1 | 0 | 2 |
|
||||
| show | 18 | 2 | 0 | 16 |
|
||||
| socket | 16 | 4 | 0 | 12 |
|
||||
| some | 6 | 0 | 0 | 6 |
|
||||
| sourceInfo | 4 | 0 | 0 | 4 |
|
||||
| splitJoin | 7 | 7 | 0 | 0 |
|
||||
| stringPostfix | 3 | 0 | 0 | 3 |
|
||||
| strings | 8 | 0 | 0 | 8 |
|
||||
| styleRef | 6 | 0 | 0 | 6 |
|
||||
| swap | 4 | 4 | 0 | 0 |
|
||||
| symbol | 2 | 0 | 0 | 2 |
|
||||
| tailwind | 12 | 0 | 0 | 12 |
|
||||
| take | 15 | 12 | 0 | 3 |
|
||||
| tell | 10 | 10 | 0 | 0 |
|
||||
| templates | 48 | 0 | 0 | 48 |
|
||||
| throw | 7 | 0 | 0 | 7 |
|
||||
| toggle | 25 | 25 | 0 | 0 |
|
||||
| tokenizer | 17 | 0 | 0 | 17 |
|
||||
| transition | 17 | 17 | 0 | 0 |
|
||||
| trigger | 6 | 0 | 0 | 6 |
|
||||
| typecheck | 5 | 0 | 0 | 5 |
|
||||
| unlessModifier | 1 | 0 | 0 | 1 |
|
||||
| viewTransition | 9 | 0 | 0 | 9 |
|
||||
| wait | 7 | 7 | 0 | 0 |
|
||||
| when | 41 | 41 | 0 | 0 |
|
||||
| worker | 1 | 0 | 0 | 1 |
|
||||
| **TOTAL** | **1496** | **764** | **44** | **688** |
|
||||
|
||||
## What unlocks how many — MISSING tests by block_reason
|
||||
|
||||
| Block reason | Missing | Example | Est. effort |
|
||||
|---|---:|---|---|
|
||||
| `unscraped-category` | 282 | breakpoint / parses as a top-level command | low — extend scraper to cover these upstream files |
|
||||
| `unscraped-in-known-category` | 170 | default / can default variables | low — re-scrape; file was walked but these cases missed |
|
||||
| `added-post-snapshot` | 122 | hide / can hide via the hidden attribute strategy | low — re-scrape upstream, bump JSON snapshot |
|
||||
| `needs-pattern:${}` | 51 | liveTemplate / script type="text/hyperscript-template" works as a l... | low — template string interpolation in HS parser |
|
||||
| `needs-script-tag` | 26 | throw / can throw a basic exception | medium — emit <script type="text/hyperscript"> wrapper in generator |
|
||||
| `needs-websocket` | 12 | socket / with timeout parses and uses the configured timeout | high — WebSocket mock server |
|
||||
| `needs-css-transitions` | 8 | viewTransition / runs the body when view transitions API is unavail... | medium — transitionend event dispatch in fixtures |
|
||||
| `needs-pattern:<sel/>` | 8 | comparisonOperator / exists works | low — parser rule for <sel/> positional expr |
|
||||
| `needs-pattern:[@attr]` | 5 | attributeRef / attributeRef with no value works | low — attribute-ref parser rule |
|
||||
| `needs-dialog-api` | 3 | dialog / show opens a dialog (non-modal) | low — stub showModal/close on HTMLDialogElement in fixture DOM |
|
||||
| `needs-intersection-observer` | 1 | on / on intersection fires when the element is in the viewport | medium — IntersectionObserver mock |
|
||||
|
||||
## Skip-listed tests by block_reason
|
||||
|
||||
| Block reason | Skipped | Example | Est. effort |
|
||||
|---|---:|---|---|
|
||||
| `translation-TBD` | 39 | fetch / can do a simple fetch w/ html | unknown — needs case-by-case generator work |
|
||||
| `needs-script-tag` | 5 | def / functions can be namespaced | medium — emit <script type="text/hyperscript"> wrapper in generator |
|
||||
|
||||
## How this was built
|
||||
|
||||
1. `git clone --depth 1 --branch v0.9.90 https://github.com/bigskysoftware/_hyperscript /tmp/hs-upstream`; `git fetch --unshallow` for dated history.
|
||||
2. Walked `test/` (excluding `vendor/`, `manual/`, `fixtures.js`, `global-*.js`, `entry.js`, `htmx-fixtures.js`, `playwright.config.js`).
|
||||
3. For each `.js` file, a small Python parser finds every `test.describe(...)` block, then every `test(...)` within it — balanced-paren scan that ignores strings, regex literals, line/block comments.
|
||||
4. Category = filename stem (e.g. `add.js` → `add`). Test name = the first string literal argument of `test(...)`.
|
||||
5. Matched each upstream test against `spec/tests/hyperscript-upstream-tests.json` using `(category, name-normalized)` keys (whitespace-collapsed lowercase). Copied `complexity` when found; inferred it otherwise from body content (sinon./script-tag/dialog/Promise/evaluate).
|
||||
6. `status = runnable` if present and name not in generator's `SKIP_TEST_NAMES`; `skip-listed` if present and in that set; `missing` otherwise.
|
||||
7. `block_reason` classified from body content — sinon./script tag/dialog/worker/eventsource/WebSocket/MutationObserver/transition/focus/ResizeObserver patterns or `<sel/>`/`${}`/`[@attr]` HS syntax. Missing tests in files touched between 2026-04-09 and 2026-04-14 (`git log --after --before -- test/`) are tagged `added-post-snapshot`.
|
||||
8. Regenerate: `python3 tests/playwright/build-hs-manifest.py` (expects `/tmp/hs-upstream` clone at `v0.9.90`).
|
||||
|
||||
## Untranslated caveat
|
||||
|
||||
The markdown reports `runnable = present + not skip-listed`. Empirical baseline is 645 pass / 109 fail / 77 skip on 831 present tests. The ~109 failures are in-scope but reveal implementation gaps; they are not statically identifiable from upstream source without running the generator. This manifest therefore does not surface an `untranslated` bucket — treat the 109 empirical fails as the lower bound of that bucket inside the `runnable` count.
|
||||
File diff suppressed because it is too large
Load Diff
8936
spec/tests/hyperscript-upstream-tests.pre-0.9.90.json
Normal file
8936
spec/tests/hyperscript-upstream-tests.pre-0.9.90.json
Normal file
File diff suppressed because it is too large
Load Diff
549
tests/playwright/build-hs-manifest.py
Normal file
549
tests/playwright/build-hs-manifest.py
Normal file
@@ -0,0 +1,549 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Build coverage manifest for _hyperscript v0.9.90 upstream tests.
|
||||
|
||||
Strategy:
|
||||
1. Parse every test file under /tmp/hs-upstream/test/ for `test(...)` and
|
||||
`test.describe(...)` calls (Playwright style).
|
||||
2. Stack nested describes into name prefixes (the describe is the category
|
||||
marker — but our JSON keys by filename basename, so we use that).
|
||||
3. Match against /root/rose-ash/spec/tests/hyperscript-upstream-tests.json.
|
||||
4. Classify complexity, status, block_reason.
|
||||
5. Emit manifest JSON + markdown summary.
|
||||
"""
|
||||
import json, os, re, sys
|
||||
from collections import Counter, defaultdict
|
||||
from pathlib import Path
|
||||
|
||||
HS_ROOT = Path('/tmp/hs-upstream')
|
||||
TEST_ROOT = HS_ROOT / 'test'
|
||||
OUR_JSON = Path('/root/rose-ash/spec/tests/hyperscript-upstream-tests.json')
|
||||
GEN_PY = Path('/root/rose-ash/tests/playwright/generate-sx-tests.py')
|
||||
OUT_JSON = Path('/root/rose-ash/spec/tests/hyperscript-upstream-manifest.json')
|
||||
OUT_MD = Path('/root/rose-ash/spec/tests/hyperscript-upstream-manifest.md')
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Load SKIP_TEST_NAMES from generator
|
||||
# ---------------------------------------------------------------------------
|
||||
gen_src = GEN_PY.read_text()
|
||||
m = re.search(r'SKIP_TEST_NAMES\s*=\s*\{(.*?)\n\}', gen_src, re.DOTALL)
|
||||
assert m, "could not find SKIP_TEST_NAMES"
|
||||
skip_block = m.group(1)
|
||||
SKIP_NAMES = set()
|
||||
for line in skip_block.splitlines():
|
||||
line = line.strip().rstrip(',')
|
||||
if not line or line.startswith('#'):
|
||||
continue
|
||||
# line is a quoted string literal
|
||||
try:
|
||||
# carefully parse the first string literal on the line
|
||||
# use a tiny regex because names contain both single and double quotes-escaped
|
||||
mm = re.match(r'^(["\'])(.*)\1\s*,?\s*(#.*)?$', line)
|
||||
if mm:
|
||||
raw = mm.group(2)
|
||||
# python-style unescape: mainly \\' and \\"
|
||||
raw = raw.encode('utf-8').decode('unicode_escape')
|
||||
SKIP_NAMES.add(raw)
|
||||
except Exception:
|
||||
pass
|
||||
print(f"Loaded {len(SKIP_NAMES)} skip names", file=sys.stderr)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Load our snapshot JSON
|
||||
# ---------------------------------------------------------------------------
|
||||
our_tests = json.load(OUR_JSON.open())
|
||||
our_map = {}
|
||||
def norm(s):
|
||||
return re.sub(r'\s+', ' ', s.strip()).lower()
|
||||
for t in our_tests:
|
||||
key = (t['category'], norm(t['name']))
|
||||
our_map[key] = t
|
||||
print(f"Our JSON: {len(our_tests)} tests, {len(our_map)} unique (cat,name)", file=sys.stderr)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Extract upstream tests
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pattern: test(<quoted-string>, ... or test.describe(<quoted-string>, ...
|
||||
# We need to parse balanced parens to find the body snippet for classification.
|
||||
|
||||
SKIP_FILES = {'fixtures.js', 'global-setup.js', 'global-teardown.js',
|
||||
'entry.js', 'htmx-fixtures.js', 'playwright.config.js'}
|
||||
|
||||
def parse_string_literal(src, i):
|
||||
"""src[i] must be quote; return (value, next_i)."""
|
||||
q = src[i]
|
||||
assert q in ('"', "'", '`'), f"not a quote at {i}: {src[i-5:i+5]!r}"
|
||||
i += 1
|
||||
out = []
|
||||
while i < len(src):
|
||||
c = src[i]
|
||||
if c == '\\':
|
||||
nxt = src[i+1] if i+1 < len(src) else ''
|
||||
# preserve common escapes
|
||||
if nxt == 'n':
|
||||
out.append('\n'); i += 2
|
||||
elif nxt == 't':
|
||||
out.append('\t'); i += 2
|
||||
elif nxt == '\\':
|
||||
out.append('\\'); i += 2
|
||||
elif nxt == q:
|
||||
out.append(q); i += 2
|
||||
else:
|
||||
out.append(nxt); i += 2
|
||||
elif c == q:
|
||||
return ''.join(out), i + 1
|
||||
else:
|
||||
out.append(c); i += 1
|
||||
raise ValueError(f"unterminated string starting at {i}")
|
||||
|
||||
def find_matching_paren(src, start):
|
||||
"""start is index of '('; return index of matching ')'."""
|
||||
depth = 0
|
||||
i = start
|
||||
while i < len(src):
|
||||
c = src[i]
|
||||
if c in ('"', "'", '`'):
|
||||
_, i = parse_string_literal(src, i)
|
||||
continue
|
||||
if c == '/' and i+1 < len(src):
|
||||
# regex or comment
|
||||
if src[i+1] == '/':
|
||||
# line comment
|
||||
j = src.find('\n', i)
|
||||
i = len(src) if j == -1 else j + 1
|
||||
continue
|
||||
if src[i+1] == '*':
|
||||
j = src.find('*/', i)
|
||||
i = len(src) if j == -1 else j + 2
|
||||
continue
|
||||
# regex literal: heuristic — only treat as regex if preceded by an
|
||||
# operator/paren/comma/etc
|
||||
prev = ''
|
||||
k = i - 1
|
||||
while k >= 0 and src[k].isspace(): k -= 1
|
||||
prev = src[k] if k >= 0 else ''
|
||||
if prev in '(,;=!?&|:+-*/<>%^~{[' or prev == '' or (k >= 0 and src[k-5:k+1] == 'return' if k>=5 else False):
|
||||
# scan regex
|
||||
j = i + 1
|
||||
while j < len(src):
|
||||
cc = src[j]
|
||||
if cc == '\\':
|
||||
j += 2; continue
|
||||
if cc == '[':
|
||||
while j < len(src) and src[j] != ']':
|
||||
if src[j] == '\\': j += 2
|
||||
else: j += 1
|
||||
j += 1; continue
|
||||
if cc == '/':
|
||||
j += 1
|
||||
while j < len(src) and src[j].isalpha(): j += 1
|
||||
break
|
||||
if cc == '\n':
|
||||
break
|
||||
j += 1
|
||||
i = j
|
||||
continue
|
||||
if c == '(':
|
||||
depth += 1; i += 1
|
||||
elif c == ')':
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return i
|
||||
i += 1
|
||||
else:
|
||||
i += 1
|
||||
return -1
|
||||
|
||||
def extract_tests_from_file(path):
|
||||
src = path.read_text()
|
||||
tests = [] # list of dicts: name, body, describe_name
|
||||
# Find all test.describe( and test( calls, tracking nesting.
|
||||
# We do this by regex+scan approach: iterate over the file and at each match
|
||||
# track current describe stack (via start/end of the describe's body).
|
||||
|
||||
# Simpler: find test.describe(...) blocks first, then within each find test(...)
|
||||
describes = [] # (name, start_idx_of_body_lbrace, end_idx)
|
||||
i = 0
|
||||
while i < len(src):
|
||||
m = re.search(r'test\.describe\s*\(\s*', src[i:])
|
||||
if not m:
|
||||
break
|
||||
start = i + m.start()
|
||||
paren = i + m.end() - 1
|
||||
# paren points at '(' after describe
|
||||
# parse quoted string right after
|
||||
j = paren + 1
|
||||
# skip whitespace
|
||||
while j < len(src) and src[j].isspace(): j += 1
|
||||
if j >= len(src) or src[j] not in ('"', "'", '`'):
|
||||
i = start + len(m.group(0))
|
||||
continue
|
||||
try:
|
||||
name, j = parse_string_literal(src, j)
|
||||
except Exception:
|
||||
i = start + len(m.group(0))
|
||||
continue
|
||||
# find matching close paren
|
||||
endp = find_matching_paren(src, paren)
|
||||
if endp == -1:
|
||||
i = start + len(m.group(0))
|
||||
continue
|
||||
describes.append((name, paren, endp))
|
||||
i = paren + 1
|
||||
|
||||
# Now extract tests; each test is `test(` or `test(<tag>,` not `test.describe`
|
||||
for dname, dstart, dend in describes:
|
||||
region = src[dstart:dend]
|
||||
# Iterate matches of test( inside region that are not test.describe
|
||||
k = 0
|
||||
while k < len(region):
|
||||
m = re.search(r'(?<![a-zA-Z0-9_.])test\s*\(', region[k:])
|
||||
if not m:
|
||||
break
|
||||
abs_paren = dstart + k + m.end() - 1
|
||||
# Skip if this is actually test.describe (shouldn't happen due to negative lookbehind)
|
||||
# Also skip test.skip, test.only etc
|
||||
# Skip if it's test.fixme / test.describe inside nested
|
||||
check_before = region[k + m.start(): k + m.end()]
|
||||
if 'describe' in check_before:
|
||||
k += m.end(); continue
|
||||
# parse name
|
||||
j = abs_paren + 1
|
||||
while j < len(src) and src[j].isspace(): j += 1
|
||||
if j >= len(src) or src[j] not in ('"', "'", '`'):
|
||||
k += m.end(); continue
|
||||
try:
|
||||
tname, j = parse_string_literal(src, j)
|
||||
except Exception:
|
||||
k += m.end(); continue
|
||||
endp = find_matching_paren(src, abs_paren)
|
||||
if endp == -1:
|
||||
k += m.end(); continue
|
||||
body = src[abs_paren:endp+1]
|
||||
tests.append({'name': tname, 'body': body, 'describe': dname})
|
||||
# advance past this test
|
||||
k = (endp + 1) - dstart
|
||||
return tests
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Extract from all files
|
||||
# ---------------------------------------------------------------------------
|
||||
upstream_tests = []
|
||||
for path in sorted(TEST_ROOT.rglob('*.js')):
|
||||
if path.name in SKIP_FILES:
|
||||
continue
|
||||
if 'vendor' in path.parts or 'node_modules' in path.parts or 'manual' in path.parts:
|
||||
continue
|
||||
rel = path.relative_to(HS_ROOT)
|
||||
category = path.stem # filename without .js
|
||||
# Special cases for organization
|
||||
tests = extract_tests_from_file(path)
|
||||
for t in tests:
|
||||
upstream_tests.append({
|
||||
'category': category,
|
||||
'name': t['name'],
|
||||
'body': t['body'],
|
||||
'describe': t['describe'],
|
||||
'upstream_file': str(rel),
|
||||
})
|
||||
|
||||
print(f"Extracted {len(upstream_tests)} upstream tests", file=sys.stderr)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Identify post-snapshot additions
|
||||
# ---------------------------------------------------------------------------
|
||||
# A test is "post-snapshot" if it appears in a file modified in the window
|
||||
# AND not in our JSON. We'll compute based on the in_our_json check instead.
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Classify each test
|
||||
# ---------------------------------------------------------------------------
|
||||
def classify_complexity(body, name, existing):
|
||||
if existing:
|
||||
return existing.get('complexity', 'simple')
|
||||
b = body
|
||||
if 'sinon.' in b:
|
||||
return 'sinon'
|
||||
if '<script type="text/hyperscript"' in b or "<script type='text/hyperscript'" in b:
|
||||
return 'script-tag'
|
||||
if 'showModal' in b or 'HTMLDialogElement' in b or '.close()' in b and '<dialog' in b.lower():
|
||||
return 'dialog'
|
||||
if 'new Promise' in b or 'async function' in b or ' await ' in b and ('setTimeout' in b or 'resolve' in b):
|
||||
# many tests use async functions trivially; require an actual Promise / delay
|
||||
if 'new Promise' in b or 'setTimeout' in b:
|
||||
return 'promise'
|
||||
if 'evaluate' in b and 'html' not in b and 'find' not in b:
|
||||
return 'eval-only'
|
||||
if '_hyperscript.evaluate' in b or 'evaluate(' in b:
|
||||
return 'evaluate'
|
||||
if 'runCmd' in b or "_hyperscript('" in b or '_hyperscript("' in b:
|
||||
return 'run-eval'
|
||||
return 'simple'
|
||||
|
||||
def classify_block_reason(body, name, category):
|
||||
"""Return (block_reason, unlocks_hint) or (None, None)."""
|
||||
# sinon
|
||||
if 'sinon.' in body:
|
||||
return 'needs-sinon'
|
||||
# script tag
|
||||
if '<script type="text/hyperscript"' in body or "<script type='text/hyperscript'" in body:
|
||||
return 'needs-script-tag'
|
||||
if '<script type="text/hypertemplate"' in body or "<script type='text/hypertemplate'" in body:
|
||||
return 'needs-script-tag'
|
||||
# dialog
|
||||
if 'showModal' in body or '<dialog' in body.lower():
|
||||
return 'needs-dialog-api'
|
||||
# worker
|
||||
if 'Worker(' in body or 'worker.js' in body:
|
||||
return 'needs-worker'
|
||||
# eventsource
|
||||
if 'EventSource' in body or 'eventsource' in body.lower():
|
||||
return 'needs-eventsource'
|
||||
# Server-sent events
|
||||
if 'WebSocket' in body or category == 'socket':
|
||||
return 'needs-websocket'
|
||||
# mutation observers / focus/blur bubbling / transition events
|
||||
if 'MutationObserver' in body or 'mutationobserver' in body.lower():
|
||||
return 'needs-dom-mutation-observer'
|
||||
if 'transitionend' in body or 'transition-' in body or 'viewTransition' in body or category == 'transition' or category == 'viewTransition':
|
||||
return 'needs-css-transitions'
|
||||
if '.focus()' in body or '.blur()' in body or 'focusin' in body or 'focusout' in body:
|
||||
if category == 'focus':
|
||||
return 'needs-dom-focus'
|
||||
if 'resize' in body and 'ResizeObserver' in body:
|
||||
return 'needs-resize-observer'
|
||||
if 'IntersectionObserver' in body:
|
||||
return 'needs-intersection-observer'
|
||||
# patterns in hs strings
|
||||
hs_strings = re.findall(r"_=(['\"])([^'\"]*)\1", body) + re.findall(r"['\"]on\s+[^'\"]+['\"]", body)
|
||||
# Grab any quoted hs fragment
|
||||
all_quotes = re.findall(r"['\"`]([^'\"`]{3,})['\"`]", body)
|
||||
combined = ' '.join(all_quotes)
|
||||
# positional expr `<sel/>`
|
||||
if re.search(r'<[^<>]*\/>', combined) and not '/>' in body.split('expect')[0][:1000]:
|
||||
# detect <sel/>
|
||||
if re.search(r"<[a-zA-Z#.][^<>]*\/>", combined):
|
||||
return 'needs-pattern:<sel/>'
|
||||
# template interpolation ${}
|
||||
if '${' in combined:
|
||||
return 'needs-pattern:${}'
|
||||
# {css}
|
||||
if re.search(r'\{[^{}]*[:\s][^{}]+\}', combined):
|
||||
# css-like block literal inside HS string
|
||||
if re.search(r"'\s*\{", combined) or re.search(r'"\s*\{', combined):
|
||||
pass # too noisy
|
||||
if re.search(r'\[@[\w-]+', combined):
|
||||
return 'needs-pattern:[@attr]'
|
||||
# DOM required
|
||||
if 'html(' in body and 'find(' in body:
|
||||
return 'translation-TBD'
|
||||
return 'translation-TBD'
|
||||
|
||||
# Files touched in post-snapshot window
|
||||
POST_SNAPSHOT_FILES = set()
|
||||
import subprocess
|
||||
out = subprocess.check_output(
|
||||
['git', 'log', '--after=2026-04-09', '--before=2026-04-14',
|
||||
'--name-only', '--pretty=format:', '--', 'test/'],
|
||||
cwd=str(HS_ROOT)
|
||||
).decode()
|
||||
for line in out.splitlines():
|
||||
line = line.strip()
|
||||
if line.startswith('test/') and line.endswith('.js'):
|
||||
POST_SNAPSHOT_FILES.add(line)
|
||||
print(f"Post-snapshot touched files: {len(POST_SNAPSHOT_FILES)}", file=sys.stderr)
|
||||
|
||||
OUR_CATEGORIES = set(t['category'] for t in our_tests)
|
||||
|
||||
# Build manifest
|
||||
manifest = []
|
||||
for t in upstream_tests:
|
||||
key = (t['category'], norm(t['name']))
|
||||
existing = our_map.get(key)
|
||||
in_our_json = existing is not None
|
||||
complexity = classify_complexity(t['body'], t['name'], existing)
|
||||
# Status
|
||||
if not in_our_json:
|
||||
status = 'missing'
|
||||
elif t['name'] in SKIP_NAMES:
|
||||
status = 'skip-listed'
|
||||
else:
|
||||
status = 'runnable'
|
||||
# Block reason
|
||||
if status == 'runnable':
|
||||
block_reason = None
|
||||
else:
|
||||
# First check for concrete infra gaps (sinon, worker, patterns, etc.)
|
||||
br = classify_block_reason(t['body'], t['name'], t['category'])
|
||||
if br != 'translation-TBD':
|
||||
block_reason = br
|
||||
elif status == 'missing':
|
||||
# No concrete infra gap — categorise by why it's missing
|
||||
if t['category'] not in OUR_CATEGORIES:
|
||||
block_reason = 'unscraped-category'
|
||||
elif t['upstream_file'] in POST_SNAPSHOT_FILES:
|
||||
block_reason = 'added-post-snapshot'
|
||||
else:
|
||||
block_reason = 'unscraped-in-known-category'
|
||||
else:
|
||||
block_reason = br
|
||||
manifest.append({
|
||||
'category': t['category'],
|
||||
'name': t['name'],
|
||||
'complexity': complexity,
|
||||
'status': status,
|
||||
'block_reason': block_reason,
|
||||
'upstream_file': t['upstream_file'],
|
||||
'in_our_json': in_our_json,
|
||||
})
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# De-duplicate (a test could be collected twice if nested describes overlap)
|
||||
# ---------------------------------------------------------------------------
|
||||
seen = set()
|
||||
dedup = []
|
||||
for m in manifest:
|
||||
k = (m['upstream_file'], m['category'], m['name'])
|
||||
if k in seen:
|
||||
continue
|
||||
seen.add(k)
|
||||
dedup.append(m)
|
||||
manifest = dedup
|
||||
print(f"Final manifest: {len(manifest)} tests", file=sys.stderr)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Write JSON
|
||||
# ---------------------------------------------------------------------------
|
||||
with OUT_JSON.open('w') as f:
|
||||
json.dump(manifest, f, indent=2, ensure_ascii=False)
|
||||
f.write('\n')
|
||||
print(f"Wrote {OUT_JSON}", file=sys.stderr)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Stats + markdown
|
||||
# ---------------------------------------------------------------------------
|
||||
total = len(manifest)
|
||||
runnable = sum(1 for m in manifest if m['status'] == 'runnable')
|
||||
skip_listed = sum(1 for m in manifest if m['status'] == 'skip-listed')
|
||||
missing = sum(1 for m in manifest if m['status'] == 'missing')
|
||||
|
||||
cat_stats = defaultdict(lambda: {'total': 0, 'runnable': 0, 'skip-listed': 0, 'missing': 0})
|
||||
for m in manifest:
|
||||
cat_stats[m['category']]['total'] += 1
|
||||
cat_stats[m['category']][m['status']] += 1
|
||||
|
||||
# Unlock tables
|
||||
missing_by_reason = Counter()
|
||||
example_by_reason = {}
|
||||
for m in manifest:
|
||||
if m['status'] == 'missing':
|
||||
r = m['block_reason'] or 'translation-TBD'
|
||||
missing_by_reason[r] += 1
|
||||
if r not in example_by_reason:
|
||||
example_by_reason[r] = f"{m['category']} / {m['name']}"
|
||||
|
||||
skip_by_reason = Counter()
|
||||
skip_example = {}
|
||||
for m in manifest:
|
||||
if m['status'] == 'skip-listed':
|
||||
r = m['block_reason'] or 'translation-TBD'
|
||||
skip_by_reason[r] += 1
|
||||
if r not in skip_example:
|
||||
skip_example[r] = f"{m['category']} / {m['name']}"
|
||||
|
||||
EFFORT = {
|
||||
'needs-sinon': 'medium — build fetch-mock shim keyed by URL/response',
|
||||
'needs-script-tag': 'medium — emit <script type="text/hyperscript"> wrapper in generator',
|
||||
'needs-dialog-api': 'low — stub showModal/close on HTMLDialogElement in fixture DOM',
|
||||
'needs-worker': 'high — Web Worker host adapter',
|
||||
'needs-eventsource': 'high — EventSource mock + streaming',
|
||||
'needs-websocket': 'high — WebSocket mock server',
|
||||
'needs-dom-mutation-observer': 'medium — hook MutationObserver into event queue',
|
||||
'needs-css-transitions': 'medium — transitionend event dispatch in fixtures',
|
||||
'needs-dom-focus': 'low — focus/blur dispatch in fixture browser',
|
||||
'needs-resize-observer': 'medium — ResizeObserver mock',
|
||||
'needs-intersection-observer': 'medium — IntersectionObserver mock',
|
||||
'needs-pattern:<sel/>': 'low — parser rule for <sel/> positional expr',
|
||||
'needs-pattern:${}': 'low — template string interpolation in HS parser',
|
||||
'needs-pattern:[@attr]': 'low — attribute-ref parser rule',
|
||||
'needs-dom': 'medium — fixture DOM extensions',
|
||||
'added-post-snapshot': 'low — re-scrape upstream, bump JSON snapshot',
|
||||
'unscraped-category': 'low — extend scraper to cover these upstream files',
|
||||
'unscraped-in-known-category': 'low — re-scrape; file was walked but these cases missed',
|
||||
'translation-TBD': 'unknown — needs case-by-case generator work',
|
||||
}
|
||||
|
||||
md = []
|
||||
md.append("# _hyperscript v0.9.90 upstream test coverage manifest")
|
||||
md.append("")
|
||||
md.append(f"- Upstream tag: `v0.9.90` (commit `a13de2ca`, 2026-04-13)")
|
||||
md.append(f"- Our snapshot JSON: `spec/tests/hyperscript-upstream-tests.json` (831 tests, scraped 2026-04-09)")
|
||||
md.append(f"- Total upstream tests: **{total}**")
|
||||
md.append(f"- Runnable (present + not skip-listed): **{runnable}** ({runnable*100/total:.1f}%)")
|
||||
md.append(f"- Skip-listed (present but guarded in generator): **{skip_listed}** ({skip_listed*100/total:.1f}%)")
|
||||
md.append(f"- Missing from our snapshot: **{missing}** ({missing*100/total:.1f}%)")
|
||||
md.append("")
|
||||
md.append(f"Current conformance: runnable / total = **{runnable}/{total} = {runnable*100/total:.1f}%**.")
|
||||
md.append("")
|
||||
md.append("## Per-category")
|
||||
md.append("")
|
||||
md.append("| Category | Upstream | Runnable | Skip-listed | Missing |")
|
||||
md.append("|---|---:|---:|---:|---:|")
|
||||
for cat in sorted(cat_stats.keys()):
|
||||
s = cat_stats[cat]
|
||||
md.append(f"| {cat} | {s['total']} | {s['runnable']} | {s['skip-listed']} | {s['missing']} |")
|
||||
md.append(f"| **TOTAL** | **{total}** | **{runnable}** | **{skip_listed}** | **{missing}** |")
|
||||
md.append("")
|
||||
md.append("## What unlocks how many — MISSING tests by block_reason")
|
||||
md.append("")
|
||||
md.append("| Block reason | Missing | Example | Est. effort |")
|
||||
md.append("|---|---:|---|---|")
|
||||
for r, n in missing_by_reason.most_common():
|
||||
ex = example_by_reason.get(r, '')
|
||||
if len(ex) > 70: ex = ex[:67] + '...'
|
||||
md.append(f"| `{r}` | {n} | {ex} | {EFFORT.get(r, 'unknown')} |")
|
||||
md.append("")
|
||||
md.append("## Skip-listed tests by block_reason")
|
||||
md.append("")
|
||||
md.append("| Block reason | Skipped | Example | Est. effort |")
|
||||
md.append("|---|---:|---|---|")
|
||||
for r, n in skip_by_reason.most_common():
|
||||
ex = skip_example.get(r, '')
|
||||
if len(ex) > 70: ex = ex[:67] + '...'
|
||||
md.append(f"| `{r}` | {n} | {ex} | {EFFORT.get(r, 'unknown')} |")
|
||||
md.append("")
|
||||
md.append("## How this was built")
|
||||
md.append("")
|
||||
md.append("1. `git clone --depth 1 --branch v0.9.90 https://github.com/bigskysoftware/_hyperscript /tmp/hs-upstream`; `git fetch --unshallow` for dated history.")
|
||||
md.append("2. Walked `test/` (excluding `vendor/`, `manual/`, `fixtures.js`, `global-*.js`, `entry.js`, `htmx-fixtures.js`, `playwright.config.js`).")
|
||||
md.append("3. For each `.js` file, a small Python parser finds every `test.describe(...)` block, then every `test(...)` within it — balanced-paren scan that ignores strings, regex literals, line/block comments.")
|
||||
md.append("4. Category = filename stem (e.g. `add.js` → `add`). Test name = the first string literal argument of `test(...)`.")
|
||||
md.append("5. Matched each upstream test against `spec/tests/hyperscript-upstream-tests.json` using `(category, name-normalized)` keys (whitespace-collapsed lowercase). Copied `complexity` when found; inferred it otherwise from body content (sinon./script-tag/dialog/Promise/evaluate).")
|
||||
md.append("6. `status = runnable` if present and name not in generator's `SKIP_TEST_NAMES`; `skip-listed` if present and in that set; `missing` otherwise.")
|
||||
md.append("7. `block_reason` classified from body content — sinon./script tag/dialog/worker/eventsource/WebSocket/MutationObserver/transition/focus/ResizeObserver patterns or `<sel/>`/`${}`/`[@attr]` HS syntax. Missing tests in files touched between 2026-04-09 and 2026-04-14 (`git log --after --before -- test/`) are tagged `added-post-snapshot`.")
|
||||
md.append("8. Regenerate: `python3 /tmp/build_manifest.py`.")
|
||||
md.append("")
|
||||
md.append("## Untranslated caveat")
|
||||
md.append("")
|
||||
md.append("The markdown reports `runnable = present + not skip-listed`. Empirical baseline is 645 pass / 109 fail / 77 skip on 831 present tests. The ~109 failures are in-scope but reveal implementation gaps; they are not statically identifiable from upstream source without running the generator. This manifest therefore does not surface an `untranslated` bucket — treat the 109 empirical fails as the lower bound of that bucket inside the `runnable` count.")
|
||||
md.append("")
|
||||
|
||||
OUT_MD.write_text('\n'.join(md))
|
||||
print(f"Wrote {OUT_MD}", file=sys.stderr)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Print summary
|
||||
# ---------------------------------------------------------------------------
|
||||
print()
|
||||
print(f"TOTAL upstream: {total}")
|
||||
print(f" runnable: {runnable}")
|
||||
print(f" skip-listed: {skip_listed}")
|
||||
print(f" missing: {missing}")
|
||||
print(f" conformance: {runnable*100/total:.1f}%")
|
||||
print()
|
||||
print("Top missing by block_reason:")
|
||||
for r, n in missing_by_reason.most_common(8):
|
||||
print(f" {r:35s} {n:4d} ({example_by_reason.get(r,'')[:50]})")
|
||||
print()
|
||||
print("Skip-listed by block_reason:")
|
||||
for r, n in skip_by_reason.most_common(8):
|
||||
print(f" {r:35s} {n:4d}")
|
||||
@@ -299,9 +299,22 @@ def parse_action(action, ref):
|
||||
exprs.append(f'(dom-dispatch {ref(m.group(1))} "click" nil)')
|
||||
continue
|
||||
|
||||
m = re.match(r'(\w+)\.dispatchEvent\(new CustomEvent\("([\w:.-]+)"', part)
|
||||
m = re.match(r'(\w+)\.dispatchEvent\(new CustomEvent\("([\w:.-]+)"\s*(?:,\s*\{(.*)\})?', part)
|
||||
if m:
|
||||
exprs.append(f'(dom-dispatch {ref(m.group(1))} "{m.group(2)}" nil)')
|
||||
detail_expr = 'nil'
|
||||
body = m.group(3)
|
||||
if body:
|
||||
dm = re.search(r'detail:\s*"([^"]*)"', body)
|
||||
if dm:
|
||||
detail_expr = f'"{dm.group(1)}"'
|
||||
else:
|
||||
dm = re.search(r'detail:\s*\{([^}]*)\}', body)
|
||||
if dm:
|
||||
pairs = re.findall(r'(\w+):\s*"([^"]*)"', dm.group(1))
|
||||
if pairs:
|
||||
items = ' '.join(f':{k} "{v}"' for k, v in pairs)
|
||||
detail_expr = '{' + items + '}'
|
||||
exprs.append(f'(dom-dispatch {ref(m.group(1))} "{m.group(2)}" {detail_expr})')
|
||||
continue
|
||||
|
||||
m = re.match(r'(\w+)\.setAttribute\("([\w-]+)",\s*"([^"]*)"\)', part)
|
||||
@@ -844,13 +857,27 @@ def emit_element_setup(lines, elements, var_names, root='(dom-body)', indent='
|
||||
|
||||
|
||||
def emit_skip_test(test):
|
||||
"""Emit a trivial passing deftest for tests that depend on unimplemented
|
||||
hyperscript features. Keeps coverage in the source JSON but lets the run
|
||||
move on."""
|
||||
"""Emit a deftest that raises a SKIP error for tests depending on
|
||||
unimplemented hyperscript features. The test runner records these as
|
||||
failures so the pass rate reflects real coverage — grep the run output
|
||||
for 'SKIP:' to enumerate them."""
|
||||
name = sx_name(test['name'])
|
||||
raw = test['name'].replace('"', "'")
|
||||
return (
|
||||
f' (deftest "{name}"\n'
|
||||
f' (hs-cleanup!))'
|
||||
f' (error "SKIP (skip-list): {raw}"))'
|
||||
)
|
||||
|
||||
|
||||
def emit_untranslatable_test(test):
|
||||
"""Emit a deftest that raises a SKIP error for tests whose upstream body
|
||||
our generator could not translate to SX. Same loud-fail semantics as
|
||||
emit_skip_test; different tag so we can tell the two buckets apart."""
|
||||
name = sx_name(test['name'])
|
||||
raw = test['name'].replace('"', "'")
|
||||
return (
|
||||
f' (deftest "{name}"\n'
|
||||
f' (error "SKIP (untranslated): {raw}"))'
|
||||
)
|
||||
|
||||
|
||||
@@ -1486,10 +1513,13 @@ for cat, tests in categories.items():
|
||||
output.append(sx)
|
||||
total += 1
|
||||
cat_gen += 1
|
||||
# SKIP emissions still go through generate_test() → emit_skip_test;
|
||||
# detect them here so the counter reports real coverage.
|
||||
if 'SKIP (' in sx:
|
||||
cat_stub += 1
|
||||
cat_gen -= 1
|
||||
else:
|
||||
safe_name = t['name'].replace('"', "'")
|
||||
output.append(f' (deftest "{safe_name}"')
|
||||
output.append(f' (hs-cleanup!))')
|
||||
output.append(emit_untranslatable_test(t))
|
||||
total += 1
|
||||
cat_stub += 1
|
||||
|
||||
|
||||
297
tests/playwright/scrape-hs-upstream.py
Normal file
297
tests/playwright/scrape-hs-upstream.py
Normal file
@@ -0,0 +1,297 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Scrape every test from _hyperscript v0.9.90 upstream into our JSON format.
|
||||
|
||||
Walks /tmp/hs-upstream/test/**/*.js, parses `test.describe(...)` and `test(...)`
|
||||
calls with balanced-paren scanning, extracts the arrow function body, and the
|
||||
first html(...) argument. Emits /root/rose-ash/spec/tests/hyperscript-upstream-tests.json
|
||||
in body-style Playwright format (matching existing body entries).
|
||||
"""
|
||||
import json, os, re, sys
|
||||
from collections import Counter
|
||||
from pathlib import Path
|
||||
|
||||
HS_ROOT = Path('/tmp/hs-upstream')
|
||||
TEST_ROOT = HS_ROOT / 'test'
|
||||
OUT_JSON = Path('/root/rose-ash/spec/tests/hyperscript-upstream-tests.json')
|
||||
BACKUP = Path('/root/rose-ash/spec/tests/hyperscript-upstream-tests.pre-0.9.90.json')
|
||||
|
||||
SKIP_FILES = {'fixtures.js', 'global-setup.js', 'global-teardown.js',
|
||||
'entry.js', 'htmx-fixtures.js', 'playwright.config.js'}
|
||||
|
||||
# --- tokeniser-ish balanced-paren scanner -----------------------------------
|
||||
|
||||
def parse_string_literal(src, i):
|
||||
"""src[i] must be quote; return (value, next_i). Handles template literals with ${...}."""
|
||||
q = src[i]
|
||||
i += 1
|
||||
out = []
|
||||
while i < len(src):
|
||||
c = src[i]
|
||||
if c == '\\':
|
||||
nxt = src[i+1] if i+1 < len(src) else ''
|
||||
if nxt == 'n': out.append('\n'); i += 2
|
||||
elif nxt == 't': out.append('\t'); i += 2
|
||||
elif nxt == 'r': out.append('\r'); i += 2
|
||||
elif nxt == '\\': out.append('\\'); i += 2
|
||||
elif nxt == q: out.append(q); i += 2
|
||||
else:
|
||||
out.append(nxt); i += 2
|
||||
elif c == q:
|
||||
return ''.join(out), i + 1
|
||||
elif q == '`' and c == '$' and i+1 < len(src) and src[i+1] == '{':
|
||||
# template interpolation — skip balanced braces
|
||||
out.append('${'); i += 2
|
||||
depth = 1
|
||||
while i < len(src) and depth > 0:
|
||||
cc = src[i]
|
||||
if cc in ('"', "'", '`'):
|
||||
_, i = parse_string_literal(src, i)
|
||||
continue
|
||||
if cc == '{': depth += 1
|
||||
elif cc == '}': depth -= 1
|
||||
out.append(cc); i += 1
|
||||
else:
|
||||
out.append(c); i += 1
|
||||
raise ValueError("unterminated string")
|
||||
|
||||
def skip_comment_or_regex(src, i):
|
||||
"""If src[i:] starts a // comment, /* block */, or regex literal, return next index. Else None."""
|
||||
if src[i] != '/' or i+1 >= len(src):
|
||||
return None
|
||||
nxt = src[i+1]
|
||||
if nxt == '/':
|
||||
j = src.find('\n', i)
|
||||
return len(src) if j == -1 else j + 1
|
||||
if nxt == '*':
|
||||
j = src.find('*/', i)
|
||||
return len(src) if j == -1 else j + 2
|
||||
# regex heuristic: preceding non-space char is operator-ish
|
||||
k = i - 1
|
||||
while k >= 0 and src[k].isspace(): k -= 1
|
||||
prev = src[k] if k >= 0 else ''
|
||||
if prev and prev not in '(,;=!?&|:+-*/<>%^~{[\n' and prev not in '' :
|
||||
# not regex context — looks like division
|
||||
return None
|
||||
j = i + 1
|
||||
while j < len(src):
|
||||
cc = src[j]
|
||||
if cc == '\\':
|
||||
j += 2; continue
|
||||
if cc == '[':
|
||||
j += 1
|
||||
while j < len(src) and src[j] != ']':
|
||||
if src[j] == '\\': j += 2
|
||||
else: j += 1
|
||||
if j < len(src): j += 1
|
||||
continue
|
||||
if cc == '/':
|
||||
j += 1
|
||||
while j < len(src) and src[j].isalpha(): j += 1
|
||||
return j
|
||||
if cc == '\n':
|
||||
return None
|
||||
j += 1
|
||||
return None
|
||||
|
||||
def find_matching(src, start, open_c='(', close_c=')'):
|
||||
"""start is index of open_c; return index of matching close_c."""
|
||||
depth = 0
|
||||
i = start
|
||||
while i < len(src):
|
||||
c = src[i]
|
||||
if c in ('"', "'", '`'):
|
||||
try:
|
||||
_, i = parse_string_literal(src, i)
|
||||
except ValueError:
|
||||
return -1
|
||||
continue
|
||||
j = skip_comment_or_regex(src, i)
|
||||
if j is not None:
|
||||
i = j
|
||||
continue
|
||||
if c == open_c:
|
||||
depth += 1; i += 1
|
||||
elif c == close_c:
|
||||
depth -= 1
|
||||
if depth == 0:
|
||||
return i
|
||||
i += 1
|
||||
else:
|
||||
i += 1
|
||||
return -1
|
||||
|
||||
# --- test extraction --------------------------------------------------------
|
||||
|
||||
def extract_arrow_body(call_src):
|
||||
"""Given the full `(...args...)` source of test(name, fn), extract the fn body.
|
||||
Returns the content between { and } of the arrow function body, or None."""
|
||||
# Find the arrow
|
||||
arrow = call_src.find('=>')
|
||||
if arrow == -1:
|
||||
return None
|
||||
# Find the first { after =>
|
||||
j = arrow + 2
|
||||
while j < len(call_src) and call_src[j].isspace(): j += 1
|
||||
if j >= len(call_src) or call_src[j] != '{':
|
||||
return None
|
||||
end = find_matching(call_src, j, '{', '}')
|
||||
if end == -1:
|
||||
return None
|
||||
body = call_src[j+1:end]
|
||||
# Strip leading newline + common indentation (for readability)
|
||||
return body
|
||||
|
||||
def extract_first_html(body):
|
||||
"""Find the first html(...) call in body and extract its literal string argument.
|
||||
Supports html("x" + "y"), html(`x`), html("x"). Returns '' if not findable."""
|
||||
m = re.search(r'\bhtml\s*\(', body)
|
||||
if not m:
|
||||
return ''
|
||||
lp = m.end() - 1
|
||||
rp = find_matching(body, lp, '(', ')')
|
||||
if rp == -1:
|
||||
return ''
|
||||
args = body[lp+1:rp].strip()
|
||||
# Args should be a string or concatenation of strings.
|
||||
parts = []
|
||||
i = 0
|
||||
while i < len(args):
|
||||
c = args[i]
|
||||
if c.isspace() or c == '+':
|
||||
i += 1; continue
|
||||
if c in ('"', "'", '`'):
|
||||
try:
|
||||
val, i = parse_string_literal(args, i)
|
||||
parts.append(val)
|
||||
except ValueError:
|
||||
return ''
|
||||
else:
|
||||
# not a pure string concatenation — bail
|
||||
return ''
|
||||
return ''.join(parts)
|
||||
|
||||
def extract_tests_from_file(path, rel_category):
|
||||
src = path.read_text()
|
||||
# Find every test( call (not test.describe, not test.skip.)
|
||||
tests = []
|
||||
i = 0
|
||||
while i < len(src):
|
||||
m = re.search(r'(?<![a-zA-Z0-9_$.])test\s*\(', src[i:])
|
||||
if not m:
|
||||
break
|
||||
abs_start = i + m.start()
|
||||
abs_paren = i + m.end() - 1
|
||||
# Ensure this is not test.describe / test.only / test.skip
|
||||
# The lookbehind prevents .describe case. But test( is fine.
|
||||
# parse name arg
|
||||
j = abs_paren + 1
|
||||
while j < len(src) and src[j].isspace(): j += 1
|
||||
if j >= len(src) or src[j] not in ('"', "'", '`'):
|
||||
i = abs_paren + 1
|
||||
continue
|
||||
try:
|
||||
tname, j2 = parse_string_literal(src, j)
|
||||
except ValueError:
|
||||
i = abs_paren + 1
|
||||
continue
|
||||
endp = find_matching(src, abs_paren, '(', ')')
|
||||
if endp == -1:
|
||||
i = abs_paren + 1
|
||||
continue
|
||||
call_src = src[abs_paren:endp+1]
|
||||
body = extract_arrow_body(call_src)
|
||||
if body is None:
|
||||
i = endp + 1
|
||||
continue
|
||||
html = extract_first_html(body)
|
||||
tests.append({
|
||||
'category': rel_category,
|
||||
'name': tname,
|
||||
'html': html,
|
||||
'body': body,
|
||||
'async': True,
|
||||
'complexity': classify_complexity(body),
|
||||
})
|
||||
i = endp + 1
|
||||
return tests
|
||||
|
||||
def classify_complexity(body):
|
||||
if 'sinon.' in body:
|
||||
return 'sinon'
|
||||
if '<script type="text/hyperscript"' in body or "<script type='text/hyperscript'" in body:
|
||||
return 'script-tag'
|
||||
if '<script type="text/hypertemplate"' in body or "<script type='text/hypertemplate'" in body:
|
||||
return 'script-tag'
|
||||
if 'showModal' in body or '<dialog' in body.lower():
|
||||
return 'dialog'
|
||||
if 'new Promise' in body or '.resolves' in body or 'Promise.' in body:
|
||||
return 'promise'
|
||||
if 'html(' not in body:
|
||||
if '_hyperscript.evaluate' in body or re.search(r'\bevaluate\s*\(', body):
|
||||
return 'eval-only'
|
||||
if re.search(r'\brun\s*\(', body):
|
||||
return 'run-eval'
|
||||
return 'simple'
|
||||
|
||||
# --- main -------------------------------------------------------------------
|
||||
|
||||
def rel_category(path):
|
||||
"""For test/commands/foo.js, test/features/foo.js → 'foo'.
|
||||
For test/core/foo.js → 'core/foo'. test/templates/foo.js → 'templates/foo' etc."""
|
||||
rel = path.relative_to(TEST_ROOT)
|
||||
parts = rel.parts
|
||||
stem = path.stem
|
||||
if len(parts) == 1:
|
||||
# Top-level — shouldn't happen since all tests are in subdirs
|
||||
return stem
|
||||
top = parts[0]
|
||||
if top in ('commands', 'features'):
|
||||
return stem
|
||||
# Single subdir like core/api.js → 'core/api'
|
||||
if len(parts) == 2:
|
||||
return f'{top}/{stem}'
|
||||
# Deeper nesting — join all parts except final extension
|
||||
return '/'.join(parts[:-1] + (stem,))
|
||||
|
||||
def main():
|
||||
# Back up existing JSON
|
||||
if OUT_JSON.exists() and not BACKUP.exists():
|
||||
import shutil
|
||||
shutil.copy2(OUT_JSON, BACKUP)
|
||||
print(f'Backed up existing JSON to {BACKUP}', file=sys.stderr)
|
||||
|
||||
all_tests = []
|
||||
file_count = 0
|
||||
for path in sorted(TEST_ROOT.rglob('*.js')):
|
||||
if path.name in SKIP_FILES:
|
||||
continue
|
||||
if any(p in ('vendor', 'node_modules', 'manual') for p in path.parts):
|
||||
continue
|
||||
cat = rel_category(path)
|
||||
tests = extract_tests_from_file(path, cat)
|
||||
all_tests.extend(tests)
|
||||
file_count += 1
|
||||
|
||||
# Dedup by (category, name) — stable
|
||||
seen = {}
|
||||
for t in all_tests:
|
||||
key = (t['category'], t['name'])
|
||||
if key not in seen:
|
||||
seen[key] = t
|
||||
deduped = list(seen.values())
|
||||
deduped.sort(key=lambda t: (t['category'], t['name']))
|
||||
|
||||
# Stats
|
||||
cat_counts = Counter(t['category'] for t in deduped)
|
||||
print(f'Scanned {file_count} files, extracted {len(all_tests)} tests ({len(deduped)} unique)')
|
||||
print(f'Categories: {len(cat_counts)}')
|
||||
for cat, n in cat_counts.most_common():
|
||||
print(f' {cat:40s} {n:4d}')
|
||||
|
||||
with OUT_JSON.open('w') as f:
|
||||
json.dump(deduped, f, indent=2, ensure_ascii=False)
|
||||
f.write('\n')
|
||||
print(f'\nWrote {OUT_JSON} ({len(deduped)} tests)')
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user