js-on-sx: bump test262 runner per-test timeout 5s→15s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 59s
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 59s
With 4 parallel workers contending, the 5s default timed out 85/99 built-ins/String tests. Bumping to 15s yields 65/99 (65.7%) with real failure modes now visible instead of "85x Timeout".
This commit is contained in:
@@ -52,7 +52,7 @@ UPSTREAM = REPO / "lib" / "js" / "test262-upstream"
|
||||
TEST_ROOT = UPSTREAM / "test"
|
||||
HARNESS_DIR = UPSTREAM / "harness"
|
||||
|
||||
DEFAULT_PER_TEST_TIMEOUT_S = 5.0
|
||||
DEFAULT_PER_TEST_TIMEOUT_S = 15.0
|
||||
DEFAULT_BATCH_TIMEOUT_S = 120
|
||||
|
||||
# Cache dir for precomputed SX source of harness JS (one file per Python run).
|
||||
|
||||
@@ -1,76 +1,66 @@
|
||||
{
|
||||
"totals": {
|
||||
"pass": 66,
|
||||
"fail": 25,
|
||||
"skip": 1130,
|
||||
"timeout": 9,
|
||||
"total": 1230,
|
||||
"runnable": 100,
|
||||
"pass_rate": 66.0
|
||||
"pass": 65,
|
||||
"fail": 26,
|
||||
"skip": 1,
|
||||
"timeout": 8,
|
||||
"total": 100,
|
||||
"runnable": 99,
|
||||
"pass_rate": 65.7
|
||||
},
|
||||
"categories": [
|
||||
{
|
||||
"category": "built-ins/String",
|
||||
"total": 1223,
|
||||
"pass": 66,
|
||||
"fail": 25,
|
||||
"skip": 1123,
|
||||
"timeout": 9,
|
||||
"pass_rate": 66.0,
|
||||
"total": 100,
|
||||
"pass": 65,
|
||||
"fail": 26,
|
||||
"skip": 1,
|
||||
"timeout": 8,
|
||||
"pass_rate": 65.7,
|
||||
"top_failures": [
|
||||
[
|
||||
"Test262Error (assertion failed)",
|
||||
14
|
||||
16
|
||||
],
|
||||
[
|
||||
"Timeout",
|
||||
9
|
||||
8
|
||||
],
|
||||
[
|
||||
"TypeError: not a function",
|
||||
6
|
||||
],
|
||||
[
|
||||
"ReferenceError (undefined symbol)",
|
||||
"Unhandled: Not callable: \\\\\\",
|
||||
2
|
||||
],
|
||||
[
|
||||
"Unhandled: Not callable: \\\\\\",
|
||||
2
|
||||
"ReferenceError (undefined symbol)",
|
||||
1
|
||||
]
|
||||
]
|
||||
},
|
||||
{
|
||||
"category": "built-ins/StringIteratorPrototype",
|
||||
"total": 7,
|
||||
"pass": 0,
|
||||
"fail": 0,
|
||||
"skip": 7,
|
||||
"timeout": 0,
|
||||
"pass_rate": 0.0,
|
||||
"top_failures": []
|
||||
}
|
||||
],
|
||||
"top_failure_modes": [
|
||||
[
|
||||
"Test262Error (assertion failed)",
|
||||
14
|
||||
16
|
||||
],
|
||||
[
|
||||
"Timeout",
|
||||
9
|
||||
8
|
||||
],
|
||||
[
|
||||
"TypeError: not a function",
|
||||
6
|
||||
],
|
||||
[
|
||||
"ReferenceError (undefined symbol)",
|
||||
"Unhandled: Not callable: \\\\\\",
|
||||
2
|
||||
],
|
||||
[
|
||||
"Unhandled: Not callable: \\\\\\",
|
||||
2
|
||||
"ReferenceError (undefined symbol)",
|
||||
1
|
||||
],
|
||||
[
|
||||
"SyntaxError (parse/unsupported syntax)",
|
||||
@@ -78,6 +68,6 @@
|
||||
]
|
||||
],
|
||||
"pinned_commit": "d5e73fc8d2c663554fb72e2380a8c2bc1a318a33",
|
||||
"elapsed_seconds": 157.9,
|
||||
"elapsed_seconds": 420.4,
|
||||
"workers": 1
|
||||
}
|
||||
@@ -1,31 +1,31 @@
|
||||
# test262 scoreboard
|
||||
|
||||
Pinned commit: `d5e73fc8d2c663554fb72e2380a8c2bc1a318a33`
|
||||
Wall time: 157.9s
|
||||
Wall time: 420.4s
|
||||
|
||||
**Total:** 66/100 runnable passed (66.0%). Raw: pass=66 fail=25 skip=1130 timeout=9 total=1230.
|
||||
**Total:** 65/99 runnable passed (65.7%). Raw: pass=65 fail=26 skip=1 timeout=8 total=100.
|
||||
|
||||
## Top failure modes
|
||||
|
||||
- **14x** Test262Error (assertion failed)
|
||||
- **9x** Timeout
|
||||
- **16x** Test262Error (assertion failed)
|
||||
- **8x** Timeout
|
||||
- **6x** TypeError: not a function
|
||||
- **2x** ReferenceError (undefined symbol)
|
||||
- **2x** Unhandled: Not callable: \\\
|
||||
- **1x** ReferenceError (undefined symbol)
|
||||
- **1x** SyntaxError (parse/unsupported syntax)
|
||||
|
||||
## Categories (worst pass-rate first, min 10 runnable)
|
||||
|
||||
| Category | Pass | Fail | Skip | Timeout | Total | Pass % |
|
||||
|---|---:|---:|---:|---:|---:|---:|
|
||||
| built-ins/String | 66 | 25 | 1123 | 9 | 1223 | 66.0% |
|
||||
| built-ins/String | 65 | 26 | 1 | 8 | 100 | 65.7% |
|
||||
|
||||
## Per-category top failures (min 10 runnable, worst first)
|
||||
|
||||
### built-ins/String (66/100 — 66.0%)
|
||||
### built-ins/String (65/99 — 65.7%)
|
||||
|
||||
- **14x** Test262Error (assertion failed)
|
||||
- **9x** Timeout
|
||||
- **16x** Test262Error (assertion failed)
|
||||
- **8x** Timeout
|
||||
- **6x** TypeError: not a function
|
||||
- **2x** ReferenceError (undefined symbol)
|
||||
- **2x** Unhandled: Not callable: \\\
|
||||
- **1x** ReferenceError (undefined symbol)
|
||||
|
||||
Reference in New Issue
Block a user