giles
4a277941b6
Root cause: every sx_server worker session used js-eval on the 3.6KB HARNESS_STUB, paying ~15s for tokenize+parse+transpile even though every session does the same thing. Over a full scoreboard with periodic worker restarts that's minutes of wasted work. Fix: transpile once per Python process. Spin up a throwaway sx_server, run (inspect (js-transpile (js-parse (js-tokenize HARNESS_STUB)))), write the resulting SX source to lib/js/.harness-cache/stub.<fingerprint>.sx and a stable-name symlink-ish copy stub.sx. Every worker session then does a single (load .harness-cache/stub.sx) instead of re-running js-eval. Fingerprint: sha256(HARNESS_STUB + lexer.sx + parser.sx + transpile.sx). Transpiler edits invalidate the cache automatically. Runs back-to-back reuse the cache — only the first run after a transpiler change pays the ~15s precompute. Transpile had to gain a $-to-_js_dollar_ name-mangler: the SX tokenizer rejects $ in identifiers, which broke round-tripping via inspect. JS $DONOTEVALUATE → SX _js_dollar_DONOTEVALUATE. Internal JS-on-SX names are unaffected (none contain $). Measured: 300-test wide (Math+Number+String @ 100/cat, --per-test-timeout 5): 593.7s → 288.0s, 2.06x speedup. Scoreboard 114→115/300 (38.3%, noise band). Math 40%, Number 44%, String 30% — same shape as prior. Baselines: 520/522 unit, 148/148 slice — unchanged.
48 lines
1.5 KiB
Markdown
48 lines
1.5 KiB
Markdown
# test262 scoreboard
|
|
|
|
Pinned commit: `d5e73fc8d2c663554fb72e2380a8c2bc1a318a33`
|
|
Wall time: 288.0s
|
|
|
|
**Total:** 115/300 runnable passed (38.3%). Raw: pass=115 fail=174 skip=1597 timeout=11 total=1897.
|
|
|
|
## Top failure modes
|
|
|
|
- **128x** Test262Error (assertion failed)
|
|
- **37x** TypeError: not a function
|
|
- **11x** Timeout
|
|
- **2x** ReferenceError (undefined symbol)
|
|
- **2x** Unhandled: Not callable: \\\
|
|
- **2x** Unhandled: Not callable: {:__proto__ {}} (kont=6 frames)\
|
|
- **1x** SyntaxError (parse/unsupported syntax)
|
|
- **1x** Unhandled: Not callable: {:__proto__ {}} (kont=5 frames)\
|
|
- **1x** Unhandled: js-transpile-binop: unsupported op: >>>\
|
|
|
|
## Categories (worst pass-rate first, min 10 runnable)
|
|
|
|
| Category | Pass | Fail | Skip | Timeout | Total | Pass % |
|
|
|---|---:|---:|---:|---:|---:|---:|
|
|
| built-ins/String | 31 | 63 | 1123 | 6 | 1223 | 31.0% |
|
|
| built-ins/Math | 40 | 59 | 227 | 1 | 327 | 40.0% |
|
|
| built-ins/Number | 44 | 52 | 240 | 4 | 340 | 44.0% |
|
|
|
|
## Per-category top failures (min 10 runnable, worst first)
|
|
|
|
### built-ins/String (31/100 — 31.0%)
|
|
|
|
- **53x** Test262Error (assertion failed)
|
|
- **6x** Timeout
|
|
- **2x** ReferenceError (undefined symbol)
|
|
- **2x** Unhandled: Not callable: \\\
|
|
- **2x** Unhandled: Not callable: {:__proto__ {}} (kont=6 frames)\
|
|
|
|
### built-ins/Math (40/100 — 40.0%)
|
|
|
|
- **36x** TypeError: not a function
|
|
- **23x** Test262Error (assertion failed)
|
|
- **1x** Timeout
|
|
|
|
### built-ins/Number (44/100 — 44.0%)
|
|
|
|
- **52x** Test262Error (assertion failed)
|
|
- **4x** Timeout
|