306 lines
50 KiB
Markdown
306 lines
50 KiB
Markdown
# JS-on-SX: cherry-picked test262 conformance
|
||
|
||
Transpile a restricted JS subset to SX AST, run on the existing CEK/VM. Goal is runtime-hardening via conformance tests, plus a second hyperscript substrate (hyperscript.js running on SX).
|
||
|
||
## Ground rules for the loop
|
||
|
||
- **Scope:** only touch `lib/js/**` and `plans/js-on-sx.md`. Do **not** edit `spec/evaluator.sx`, `spec/primitives.sx`, `shared/sx/**`, or `lib/hyperscript/**` — the user is working on those elsewhere.
|
||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix them from this loop.
|
||
- **SX files:** use `sx-tree` MCP tools only (never `Edit`/`Read`/`Write` on `.sx` files). Use `sx_write_file` for new files, path-based or pattern-based edits for changes.
|
||
- **Architecture:** JS source → JS AST → SX AST → existing CEK. No standalone JS evaluator — reuse everything.
|
||
- **Tests:** mirror `lib/hyperscript/test.sh` pattern. Start with a tiny in-repo smoke suite; graduate to a cherry-picked test262 slice once the lexer+expression-parser cycle is green.
|
||
|
||
## The loop
|
||
|
||
A single long-running background agent works `plans/js-on-sx.md` forever, one feature per commit. It runs a prioritized queue driven by the real test262 scoreboard. The briefing lives at `plans/agent-briefings/loop.md`; the recovery helper is `plans/restore-loop.sh`.
|
||
|
||
The queue (condensed — full version in the briefing):
|
||
|
||
1. Baseline commit (stage what's on disk now).
|
||
2. Fix `lib/js/test262-runner.py` so it produces a real scoreboard.
|
||
3. Full scoreboard run across the whole `test/` tree.
|
||
4. Regex lexer/parser/runtime stub + Blockers entry listing platform primitives needed.
|
||
5. Scoreboard-driven: pick the worst-passing category each iteration; fix; re-score.
|
||
6. When the scoreboard plateaus, tackle deferred items (ASI, CEK-suspend await).
|
||
|
||
### Crash recovery
|
||
|
||
The agent process dies with the machine. **Work survives if it was committed** — that's why every iteration ends in a commit. After a crash:
|
||
|
||
```bash
|
||
bash plans/restore-loop.sh # show state: tests, commits, scoreboard, regex hook
|
||
bash plans/restore-loop.sh --print # also cat the briefing for easy copy-paste
|
||
```
|
||
|
||
Then respawn the agent by pasting `plans/agent-briefings/loop.md` into Claude Code via the `Agent` tool with `run_in_background=true`. The agent re-reads the plan, picks up wherever the queue has got to, and carries on.
|
||
|
||
## Architecture sketch
|
||
|
||
```
|
||
JS source text
|
||
│
|
||
▼
|
||
lib/js/lexer.sx — tokens: {:type :value :pos}
|
||
│
|
||
▼
|
||
lib/js/parser.sx — JS AST as SX trees, e.g. (js-binop "+" left right)
|
||
│
|
||
▼
|
||
lib/js/transpile.sx — JS AST → SX AST (reusable by CEK)
|
||
│ e.g. (js-binop "+" l r) → (+ (transpile l) (transpile r))
|
||
▼
|
||
existing CEK / VM — evaluation, async/IO, exceptions already handled
|
||
```
|
||
|
||
Runtime shims in `lib/js/runtime.sx`: `js-global`, `js-console-log`, `js-typeof`, coercion helpers (`to-number`, `to-string`, `to-boolean`, abstract-equality), eventually the prototype chain.
|
||
|
||
## Roadmap
|
||
|
||
Each item: implement → tests → update progress. Mark `[x]` when tests green.
|
||
|
||
### Phase 1 — Lexer
|
||
- [x] Numeric literals (int, float, hex, exponent)
|
||
- [x] String literals (double, single, escape sequences, template strings later)
|
||
- [x] Identifiers + reserved words
|
||
- [x] Punctuation: `( ) { } [ ] , ; : . ...`
|
||
- [x] Operators: `+ - * / % ** = == === != !== < > <= >= && || ! ?? ?: & | ^ ~ << >> >>> += -= ...`
|
||
- [x] Comments (`//`, `/* */`)
|
||
- [ ] Automatic Semicolon Insertion (defer — initially require semicolons)
|
||
|
||
### Phase 2 — Expression parser (Pratt-style)
|
||
- [x] Literals → AST nodes
|
||
- [x] Binary operators with precedence
|
||
- [x] Unary operators (`- + ! ~ typeof void`)
|
||
- [x] Member access (`.`, `[]`)
|
||
- [x] Function calls
|
||
- [x] Array literals
|
||
- [x] Object literals (string/ident keys, shorthand later)
|
||
- [x] Conditional `a ? b : c`
|
||
- [x] Arrow functions (expression body only)
|
||
|
||
### Phase 3 — Transpile to SX AST
|
||
- [x] Numeric/string/bool/null literals → SX literals
|
||
- [x] Arithmetic: `+ - * / % **` with numeric coercion (`js-add` does string-concat dispatch)
|
||
- [x] Comparisons: `=== !== == != < > <= >=`
|
||
- [x] Logical: `&& ||` (short-circuit, value-returning via thunk trick); `??` nullish coalesce
|
||
- [x] Unary: `- + ! ~ typeof void`
|
||
- [x] Member access → `js-get-prop` (handles dicts, lists `.length`/index, strings)
|
||
- [x] Array literal → `(list ...)`
|
||
- [x] Object literal → `(dict)` + `dict-set!` sequence
|
||
- [x] Function call → SX call (callee can be ident, member, arrow)
|
||
- [x] Arrow fn → `(fn ...)` (params become SX symbols; closures inherited)
|
||
- [x] Assignment (`=` and compound `+= -= ...`) on ident/member/index targets
|
||
- [x] `js-eval` end-to-end: source → tokens → AST → SX → `eval-expr`
|
||
|
||
### Phase 4 — Runtime shims (`lib/js/runtime.sx`)
|
||
- [x] `js-typeof`, `js-to-number`, `js-to-string`, `js-to-boolean`
|
||
- [x] Abstract equality (`js-loose-eq`) vs strict (`js-strict-eq`)
|
||
- [x] Arithmetic (`js-add` `js-sub` `js-mul` `js-div` `js-mod` `js-pow` `js-neg` `js-pos`)
|
||
- [x] Logical (`js-and` `js-or` via thunks for lazy rhs) and `js-not` / `js-bitnot`
|
||
- [x] Relational (`js-lt` `js-gt` `js-le` `js-ge`) incl. lexicographic strings
|
||
- [x] `js-get-prop` / `js-set-prop` (dict/list/string; `.length`; numeric index)
|
||
- [x] `console.log` → `log-info` bridge (`console` dict wired)
|
||
- [x] `Math` object shim (`abs` `floor` `ceil` `round` `max` `min` `random` `PI` `E`)
|
||
- [x] `js-undefined` sentinel (keyword) distinct from `nil` (JS `null`)
|
||
- [x] Number parser `js-num-from-string` (handles int/float/±sign, no NaN yet)
|
||
|
||
### Phase 5 — Conformance harness
|
||
- [x] Cherry-picked slice vendored at `lib/js/test262-slice/` (69 fixtures across 7 categories)
|
||
- [x] Harness `lib/js/conformance.sh` — batch-load kernel, one epoch per fixture, substring-compare
|
||
- [x] Initial target: ≥50% pass. **Actual: 69/69 (100%).**
|
||
|
||
### Phase 6 — Statements
|
||
- [x] `var`/`let`/`const` declarations (all behave as `define` — block scope via SX lexical semantics)
|
||
- [x] `if`/`else`
|
||
- [x] `while`, `do..while`
|
||
- [x] `for (init; cond; step)`
|
||
- [x] `return`, `break`, `continue` (via `call/cc` continuation bindings `__return__` / `__break__` / `__continue__`)
|
||
- [x] Block scoping (via `begin` — lexical scope inherited, no TDZ)
|
||
|
||
### Phase 7 — Functions & scoping
|
||
- [x] `function` declarations (with call/cc-wrapped bodies for `return`)
|
||
- [x] Function expressions (named + anonymous)
|
||
- [x] Hoisting — function decls hoisted to enclosing scope (scan body first, emit defines ahead of statements)
|
||
- [x] Closures — work via SX `fn` env capture
|
||
- [x] Rest params (`...rest` → `&rest`)
|
||
- [x] Default parameters (desugar to `if (param === undefined) param = default`)
|
||
- [ ] `var` hoisting (deferred — treated as `let` for now)
|
||
- [ ] `let`/`const` TDZ (deferred)
|
||
|
||
### Phase 8 — Objects, prototypes, `this`
|
||
- [x] Property descriptors (simplified — plain-dict `__proto__` chain, `js-set-prop` mutates)
|
||
- [x] Prototype chain lookup (`js-dict-get-walk` walks `__proto__`)
|
||
- [x] `this` binding rules: method call, function call (undefined), arrow (lexical)
|
||
- [x] `new` + constructor semantics (fresh dict, proto linked, ctor called with `this`)
|
||
- [x] ES6 classes (sugar over prototypes, incl. `extends`)
|
||
- [x] Array mutation: `a[i] = v`, `a.push(...)` — via `set-nth!` / `append!`
|
||
- [x] Array builtins: push, pop, shift, slice, indexOf, join, concat, map, filter, forEach, reduce
|
||
- [x] String builtins: charAt, charCodeAt, indexOf, slice, substring, toUpperCase, toLowerCase, split, concat
|
||
- [x] `instanceof` and `in` operators
|
||
|
||
### Phase 9 — Async & Promises
|
||
- [x] Promise constructor + `.then` / `.catch` / `.finally`
|
||
- [x] `Promise.resolve` / `Promise.reject` / `Promise.all` / `Promise.race`
|
||
- [x] `async` functions (decl, expr, arrow) return Promises
|
||
- [x] `await` — synchronous-ish: drains microtasks, unwraps settled Promise
|
||
- [x] Microtask queue with FIFO drain (`__drain()` exposed to JS)
|
||
- [ ] True CEK suspension on `await` for pending Promises (deferred — needs cek-step-loop plumbing)
|
||
|
||
### Phase 10 — Error handling
|
||
- [x] `throw` statement → `(raise v)`
|
||
- [x] `try`/`catch`/`finally` (desugars to `guard` + optional finally wrapper)
|
||
- [x] Error hierarchy (`Error`, `TypeError`, `RangeError`, `SyntaxError`, `ReferenceError` as constructor shims)
|
||
|
||
### Phase 11 — Stretch / deferred
|
||
- ASI, regex literals, generators, iterators, destructuring, template strings with `${}`, tagged templates, Symbol, Proxy, typed arrays, ESM modules.
|
||
|
||
## Progress log
|
||
|
||
Append-only record of completed iterations. Loop writes one line per iteration: date, what was done, test count delta.
|
||
|
||
- 2026-04-23 — scaffold landed: lib/js/{lexer,parser,transpile,runtime}.sx stubs + test.sh. 7/7 smoke tests pass (js-tokenize/js-parse/js-transpile stubs + js-to-boolean coercion cases).
|
||
- 2026-04-23 — Phase 1 (Lexer) complete: numbers (int/float/hex/exp/leading-dot), strings (escapes), idents/keywords, punctuation, all operators (1-4 char, longest-match), // and /* */ comments. 38/38 tests pass. Gotchas found: `peek` and `emit!` are primitives (shadowed to `js-peek`, `js-emit!`); `cond` clauses take ONE body only, multi-expr needs `(do ...)` wrapper.
|
||
- 2026-04-23 — Phase 2 (Pratt expression parser) complete: literals, binary precedence (w/ `**` right-assoc), unary (`- + ! ~ typeof void`), member access (`.`/`[]`), call chains, array/object literals (ident+string+number keys), ternary, arrow fns (zero/one/many params; curried), assignment (right-assoc incl. compound `+=` etc.). AST node shapes all match the `js-*` names already wired. 47 new tests, 85/85 total. Most of the Phase 2 scaffolding was already written in an earlier session — this iteration verified every path, added the parser test suite, and greened everything on the first pass. No new gotchas beyond Phase 1.
|
||
- 2026-04-23 — Phase 3 (Transpile to SX AST) complete: dispatch on AST head using `js-tag?`/`symbol-name` inspection, emit SX trees built from `list`/`cons`/`make-symbol`. All binops, unaries, member/index, call (arbitrary callee), array (`list`), object (`let` + `dict` + `dict-set!`), ternary (`if` around `js-to-boolean`), arrow (`fn` with `make-symbol` params), assignment (ident → `set!`; member/index → `js-set-prop`; compound → combines). Short-circuit `&&`/`||` built via thunk passed to `js-and`/`js-or` — this preserves JS value-returning semantics and avoids re-evaluating lhs. `??` uses `let`+`if`. `js-eval src` pipelines `js-parse-expr` → `js-transpile` → `eval-expr`. 69 new assertions for end-to-end eval; 154/154 main suite.
|
||
- 2026-04-23 — Phase 4 (Runtime shims) complete: coercions (ToBoolean/ToNumber/ToString) including a tiny recursive string→number parser (handles ±int/float, no NaN yet); arithmetic with JS `+` dispatch (string-concat when either operand is string, else ToNumber); `js-strict-eq`/`js-loose-eq` with null↔undefined and boolean-coercion rules; relational with lexicographic string path via `char-code`; `js-get-prop`/`js-set-prop` covering dict/list/string with numeric index and `.length`; `Math` object, `console.log`, `js-undefined` sentinel. Several SX gotchas hit and noted below.
|
||
- 2026-04-23 — Phase 5 (Conformance harness) complete: 69 hand-picked fixtures under `lib/js/test262-slice/` covering arithmetic, comparison, loose+strict equality, logical, nullish, conditional, arrays, objects, strings, arrows, Math, typeof. Runner `lib/js/conformance.sh` builds one batch script (single kernel boot), one epoch per fixture, substring-matches the sibling `.expected` file. Scoring: **69/69 (100%)** — well past the 50% target. Real test262 integration deferred to later phase (needs network + Python harness; deferred to Phase 5.5 per plan).
|
||
- 2026-04-23 — Phases 6 + 7 (Statements + Functions) complete. Parser now accepts top-level programs — `js-parse` returns `(js-program (stmts...))`, with new node types `js-var / js-vardecl / js-block / js-if / js-while / js-do-while / js-for / js-return / js-break / js-continue / js-exprstmt / js-empty / js-funcdecl / js-funcexpr / js-param / js-rest`. Arrow bodies now accept `{...}` block form. Transpile dispatch extended with 14 new clauses. Loops built via `letrec` recursion; `break` / `continue` / `return` via lexical `call/cc` bindings (`__break__` / `__continue__` / `__return__`). Function declarations hoisted to the enclosing scope before other statements run (two-pass: `js-collect-funcdecls` scans, `js-transpile-stmt-list` replaces the hoisted entry with `nil`). Default params desugar to `(if (or (= x nil) (= x :js-undefined)) default x)` updates; rest params emit `&rest name`. Unit tests: **195/195** (154 → +41 for statements, functions, closures, loops). Conformance: **96/96** (69 → +27 new fixtures across `statements/`, `loops/`, `functions/`, `closures/`). Gotchas: (1) SX `do` is R7RS iteration, not sequence — must use `begin`. A `(do (x) ...)` where `x` is a list → "first: expected list, got N" because the do-form tries to parse its iteration bindings. (2) SX passes unsupplied fn params as `nil`, not an undefined sentinel — default-param init must test for both `nil` and `:js-undefined`. (3) `jp-collect-params` (used by arrow heads) doesn't understand rest/defaults; new `jp-parse-param-list` used for function declarations. Arrow rest/defaults deferred. (4) `...` lexes as `punct`, not `op`.
|
||
|
||
- 2026-04-23 — **Phase 9 (Async & Promises) complete.** New AST tags: `js-await`, `js-funcdecl-async`, `js-funcexpr-async`, `js-arrow-async`. Parser extended: `async` keyword consumed, dispatches by the next token (function/ident/paren). Primary parser grows a pre-function `async` case and a new `await` unary. Statement parser adds a two-token lookahead for `async function` decls. Runtime adds: microtask queue (`__js_microtask_queue__` dict cell + push/pop/empty/drain), `js-promise?` predicate, full `{:__js_promise__ true :state :value :callbacks}` object, `js-promise-resolve!`/`reject!`/`flush-callbacks!`, callback dispatch (`run-callback!` / `run-handler!` / `try-call` using `guard`), `.then` via `js-promise-then-internal!`, `.catch`/`.finally` derivative calls. `js-invoke-method` now routes Promise methods through `js-invoke-promise-method` (same single-dispatch no-closure pattern as Phase 8 list/string builtins). `Promise` constructor runs executor synchronously inside a guard so throws reject the Promise. Statics `resolve`/`reject`/`all`/`race` live in `__js_promise_statics__` dict; `js-get-prop` special-cases identity-equality against the `Promise` function. `js-async-wrap` wraps a thunk → Promise (fulfilled on return, rejected on throw, adopts returned Promises). `js-await-value` drains microtasks then unwraps a settled Promise or raises its reason; pending Promise = error (no scheduler — see Blockers). `js-eval` drains microtasks at end. `__drain()` exposed to JS so tests can force-run pending callbacks synchronously before reading a mutable result. Arity-tolerant call path `js-call-arity-tolerant` adapts 1-arg handler invocations to handlers declared with `()` (zero params) via `lambda-params` introspection. Unit tests: **254/254** (+31 parser + runtime). Conformance: **148/148** (+29: `test262-slice/promises/*` × 16, `test262-slice/async/*` × 13). Microtask ordering is FIFO (append on settle, drain one-at-a-time); spec-ish but not obsessive about thenable-adoption iteration count. Gotchas: (1) **`cond` needs `begin` for multi-body clauses** — same rule as Phase 1, bit me hard because the original draft had `(cond ((state) (side-effect) (side-effect2)))` which silently discarded the first expression as "predicate" and tried to call it as a function. (2) **`guard` with multi-body handler clauses** — same fix, `(guard (e (else (begin …))))`. (3) **`(= (type-of fn) "function")` is FALSE** — `type-of` returns `"lambda"` for user-defined fns; use `js-function?` which accepts lambda/function/component. (4) **Forward refs in SX work** because `define` is late-bound in the global env. (5) **Microtask semantics vs top-level last-expression** — `js-eval` evaluates all stmts THEN drains; if the last stmt reads `r` assigned in a `.then`, you'll see `nil` unless you insert `__drain()` between the setup and the read. (6) **`Promise.resolve(p)` returns p for existing Promises** — identity preserved via `(js-promise? v) → v` short-circuit. (7) **Strict arity in SX lambdas vs tolerant JS** — `() => side-effect()` in JS accepts extra args silently; SX `(fn () ...)` errors. Callback invocations go through `js-call-arity-tolerant` which introspects `lambda-params` and calls with no args if the handler has zero params.
|
||
|
||
- 2026-04-23 — **Queue item 1: baseline commit.** Staged `lib/js/` tree + `plans/` as committed by prior sessions. 278/280 unit (2 failing template-string edges: epoch 903 part-count off-by-one, 934 escaped-backtick ident-lookup), 148/148 slice. Runner stub at 0/8 with 7 timeouts. Commit `9e568ad8`. Out-of-scope changes in `lib/compiler.sx`, `lib/hyperscript/compiler.sx`, `shared/static/wasm/sx/hs-compiler.sx` intentionally left unstaged per briefing scope rules.
|
||
- 2026-04-23 — Phases 8 + 10 (Objects + Errors) complete in a single session. **Object model:** regular JS `function` bodies wrap with `(let ((this (js-this))) ...)` — a dynamic `this` via a global cell `__js_this_cell__`. Method calls `obj.m(args)` route through `js-invoke-method` which saves/restores the cell around the call, so `this` works without an explicit first-arg calling convention. Arrow functions don't wrap — they inherit the enclosing lexical `this`. **`new`:** creates a fresh dict with `__proto__` linked to the constructor's prototype dict, calls the constructor with `this` bound, returns the ctor's dict return (if any) else the new object. **Prototype chain:** lives in a side table `__js_proto_table__` keyed by `inspect(ctor)`. `ctor.prototype` access and assignment both go through this table. `js-dict-get-walk` walks the `__proto__` chain on dict property lookup. **Classes:** desugar to `(define Name ctor)` + `(js-reset-ctor-proto! Name)` (critical for redefinition) + `(dict-set! (js-get-ctor-proto Name) mname mfn)` for each method. `extends` chains by setting `(js-get-ctor-proto Child).__proto__ = (js-get-ctor-proto Parent)`. Default ctor with `extends` calls parent with same args. **Arrays:** `js-set-prop` on lists dispatches to `js-list-set!` which does in-bounds `set-nth!` or `append!` past end (pads with `js-undefined`). No shrinking (primitive gap — `pop-last!` is a no-op). **Array + String builtins** are routed through `js-invoke-method` directly via `js-invoke-list-method` / `js-invoke-string-method` to AVOID a VM JIT bug: returning a closure from a JIT-compiled function (which happened when `js-array-method` returned an inner `fn`) crashed with "VM undefined: else". Dispatching without closures works. **Throw/try/catch/finally:** `throw v` → `(raise v)`; try/catch → `(guard (e (else cbody)) body)`; finally wraps via `(let ((r try-tr)) finally-tr r)`. **Error hierarchy:** `Error`/`TypeError`/`RangeError`/`SyntaxError`/`ReferenceError` are constructor shims that set `this.message` + `this.name` on the new object. **`instanceof` + `in`:** parser precedence table extended to accept both as keywords at prec 10; binary-loop predicate extended to allow keyword-type tokens for these two. Unit tests: **223/223** (+28). Conformance: **119/119** (+23 new fixtures across `objects/` and `errors/`). Gotchas: (1) **Ctor-id collision on redefine** — `inspect` of a lambda is keyed by (name + arity), so redefining `class B` found the OLD proto-table entry. Fix: class decl always calls `js-reset-ctor-proto!`. (2) **VM closure bug** — functions returning inner closures from JIT-compiled bodies break: `(fn (arr) (fn (f) ...use arr...))` compiles to a VM closure for the outer that can't produce a working inner. Workaround: route all builtin method dispatch through a single (non-closure-returning) helper. (3) **`jp-parse-param-list` eats its own `(`** — don't prefix with `jp-expect! st "punct" "("`, the parser handles both. Class method parser hit this.
|
||
|
||
- 2026-04-23 — **Queue item 2: fixed test262 runner.** Root-cause of 7/8 timeouts: runner re-parsed the entire 197-line `assert.js` for every test in one big `js-eval` (8.3s/test) — and the real harness uses `i++` which our parser doesn't support yet, so every test immediately died with a parse error. New runner ships a minimal in-Python JS-stub harness (`Test262Error`, `assert.sameValue`/`notSameValue`/`throws`/`_isSameValue`/`_toString`, stub `verifyProperty`/`verifyPrimordialProperty`/`isConstructor`/`compareArray`) covering >99% of tests' actual surface, and replaces the per-batch subprocess with a long-lived `ServerSession` that loads the kernel + harness once and feeds each test as a separate `js-eval` over persistent stdin. Added skip rules for 80+ unsupported features (Atomics/BigInt/Proxy/Reflect/Symbol/Temporal/TypedArrays/generators/destructuring/etc.) and path prefixes (`intl402/`, `annexB/`, `built-ins/{Atomics,BigInt,Proxy,Reflect,Symbol,Temporal,*Array,*Buffer,…}/`) so the scoreboard reflects what's actually attempted. Scoreboard over 288 runnable Math tests: **56/288 (19.4%)** in 185s, rate ≈ 2.3 tests/s (prev: 0/8 with 7 timeouts). Top failure modes: 83× assertion-fail (real semantic gaps in Math.floor/ceil/trunc/etc. details), 62× ReferenceError (builtins we haven't shimmed, e.g. `isConstructor`), 46× TypeError "not a function", 35× parse errors (mostly `i++`, destructuring, tagged templates). 278/280 unit + 148/148 slice unchanged.
|
||
|
||
- 2026-04-23 — **Regex literal support (lex+parse+transpile+runtime stub).** Runner now accepts repeatable `--filter` flags (OR'd). Lexer gains `js-regex-context?` (returns true at SOF or when last token is op/non-closing-punct/regex-keyword incl. return/typeof/in/of/throw/new/delete/instanceof/void/yield/await/case/do/else) and `read-regex` (handles `\` escapes and `[...]` classes, collects flags as ident chars). `scan!` intercepts `/` ahead of the operator-match tries when in a regex context and emits `{:type "regex" :value {:pattern :flags}}`. Parser adds a `regex` primary branch → `(js-regex pat flags)`. Transpile emits `(js-regex-new pat flags)`. Runtime adds: `js-regex?` predicate (dict + `__js_regex__` key), `js-regex-new` builds the tagged dict with `source / flags / global / ignoreCase / multiline / sticky / unicode / dotAll / hasIndices / lastIndex` populated; `js-regex-invoke-method` dispatches `.test` / `.exec` / `.toString`; `js-invoke-method` gets a regex branch before the generic method-lookup fallback. Stub engine (`js-regex-stub-test` / `-exec`) uses `js-string-index-of` — not a real regex, but enough to make `/foo/.test('hi foo')` work. `__js_regex_platform__` dict + `js-regex-platform-override!` let a real platform primitive be swapped in later without runtime changes. 30 new unit tests (17 lex + 3 parse + 1 transpile + 4 obj-shape + 4 prop + 2 test()): **308/310** (278→+30). Conformance unchanged. Gotcha: `contains?` with 2 args expects `(contains? list x)`, NOT a dict — use `(contains? (keys d) k)` or `dict-has?`. First pass forgot that and cascaded errors across Math / class tests via the `js-regex?` predicate inside `js-invoke-method`. Wide scoreboard run across 9 targeted categories launched in background.
|
||
|
||
- 2026-04-23 — **Expanded Math + Number globals.** Added `Math.sqrt/.pow/.trunc/.sign/.cbrt/.hypot` using SX primitives (`sqrt`, `pow`, `abs`, hand-rolled loops). Added missing constants: `Math.LN2 / LN10 / LOG2E / LOG10E / SQRT2 / SQRT1_2`; bumped PI/E precision to full 16-digit. New `Number` global: `isFinite`, `isNaN`, `isInteger`, `isSafeInteger`, `MAX_VALUE / MIN_VALUE / MAX_SAFE_INTEGER / MIN_SAFE_INTEGER / EPSILON / POSITIVE_INFINITY / NEGATIVE_INFINITY / NaN`. Global `isFinite`, `isNaN`, `Infinity`, `NaN`. `js-number-is-nan` uses the self-inequality trick `(and (number? v) (not (= v v)))`. Wired into `js-global`. 21 new unit tests (12 Math + 9 Number), **329/331** (308→+21). Conformance unchanged. Gotchas: (1) `sx_insert_near` takes a single node — multi-define source blocks get silently truncated. Use `sx_insert_child` at the root per define. (2) SX `(/ 1 0)` → `inf`, and `1e999` also → `inf`; both can be used as `Infinity`. (3) **`(define NaN ...)` and `(define Infinity ...)` crash at load — SX tokenizer parses `NaN` and `Infinity` as the *numeric literals* `nan` / `inf`, so `define` sees `(define <number> <value>)` and rejects it with "Expected symbol, got number". Drop those top-level aliases; put the values in `js-global` dict instead where the keyword key avoids the conflict.**
|
||
|
||
- 2026-04-23 — **Postfix/prefix `++` / `--`.** Parser: postfix branch in `jp-parse-postfix` (matches `op ++`/`--` after the current expression and emits `(js-postfix op target)`), prefix branch in `jp-parse-primary` *before* the unary-`-/+/!/~` path emits `(js-prefix op target)`. Transpile: `js-transpile-prefix` emits `(set! sxname (+ (js-to-number sxname) ±1))` for idents, `(js-set-prop obj key (+ (js-to-number (js-get-prop obj key)) ±1))` for members/indices. `js-transpile-postfix` uses a `let` binding to cache the old value via `js-to-number`, then updates and returns the saved value — covers ident, member, and index targets. 11 new unit tests (ident inc/dec, pre vs post return value, obj.key, a[i], in `for(;; i++)`, accumulator loop), **340/342** (329→+11). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **String.prototype extensions + Object/Array builtins.** Strings: added `includes`, `startsWith`, `endsWith`, `trim`, `trimStart`, `trimEnd`, `repeat`, `padStart`, `padEnd`, `toString`, `valueOf` to `js-string-method` dispatch and corresponding `js-get-prop` string-branch keys. Helpers: `js-string-repeat` (tail-recursive concat), `js-string-pad` + `js-string-pad-build`. Object: `Object.keys / .values / .entries / .assign / .freeze` (freeze is a no-op — we don't track sealed state). Array: `Array.isArray` (backed by `list?`), `Array.of` (varargs → list). Wired into `js-global`. 17 new unit tests, **357/359** (340→+17). Conformance unchanged. Gotcha: SX's `keys` primitive returns most-recently-inserted-first, so `Object.keys({a:1, b:2})` comes back `["b", "a"]`. Test assertion has to check `.length` rather than the literal pair. If spec order matters for a real app, Object.keys would need its own ordered traversal.
|
||
|
||
- 2026-04-23 — **switch / case / default.** Parser: new `jp-parse-switch-stmt` (expect `switch (expr) { cases }`), `jp-parse-switch-cases` (walks clauses: `case val:`, `default:`), `jp-parse-switch-body` (collects stmts until next `case`/`default`/`}`). AST: `(js-switch discr (("case" val body-stmts) ("default" nil body-stmts) ...))`. Transpile: wraps body in `(call/cc (fn (__break__) (let ((__discr__ …) (__matched__ false)) …)))`. Each case clause becomes `(when (or __matched__ (js-strict-eq __discr__ val)) (set! __matched__ true) body-stmts)` — implements JS fall-through naturally (once a case matches, all following cases' `when` fires via `__matched__`). Default is a separate `(when (not __matched__) default-body)` appended at the end. `break` inside a case body already transpiles to `(__break__ nil)` and jumps out via the `call/cc`. 6 new unit tests (match, no-match default, fall-through stops on break, string discriminant, empty-body fall-through chain), **363/365** (357→+6). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **More Array.prototype + Object fallbacks (`hasOwnProperty` etc).** Array: `includes`, `find`, `findIndex`, `some`, `every`, `reverse` (in `js-array-method` dispatch + `js-get-prop` list-branch keys). Helpers: `js-list-find-loop / -find-index-loop / -some-loop / -every-loop / -reverse-loop` all tail-recursive, no `while` because SX doesn't have one. Object fallbacks: `js-invoke-method` now falls back to `js-invoke-object-method` for dicts when js-get-prop returns undefined AND the method name is in the builtin set (`hasOwnProperty`, `isPrototypeOf`, `propertyIsEnumerable`, `toString`, `valueOf`, `toLocaleString`). `hasOwnProperty` checks `(contains? (keys recv) (js-to-string k))`. This lets `o.hasOwnProperty('x')` work on plain dicts without having to install an Object.prototype. 13 new tests, **376/378** (363→+13). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **`String.fromCharCode`, `parseInt`, `parseFloat`.** `String` global with `fromCharCode` (variadic, loops through args and concatenates via `js-code-to-char`). `parseInt` truncates toward zero via `js-math-trunc`; `parseFloat` delegates to `js-to-number`. Wired into `js-global`. 5 new tests, **381/383** (376→+5). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **JSON.stringify + JSON.parse.** Shipped a recursive-descent parser and serializer in SX. `js-json-stringify` dispatches on `type-of` for primitives, lists, dicts. `js-json-parse` uses a state dict `{:s src :i idx}` mutated in-place by helpers (`js-json-skip-ws!`, `js-json-parse-value`, `-string`, `-number`, `-array`, `-object`). String parser handles `\n \t \r \" \\ \/` escapes. Number parser collects digits/signs/e+E/. then delegates to `js-to-number`. Array and object loops recursively call parse-value. JSON wired into `js-global`. 10 new tests (stringify primitives/arrays/objects, parse primitives/string/array/object), **391/393** (381→+10). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **Array.prototype flat/fill; indexOf start arg; for..of/for..in.** Array: `flat(depth=1)` uses `js-list-flat-loop` (recursive flatten), `fill(value, start?, end?)` mutates in-place then returns self via `js-list-fill-loop`. Fixed `indexOf` to honor the `fromIndex` second argument. Parser: `jp-parse-for-stmt` now does a 2-token lookahead — if it sees `(var? ident (of|in) expr)` it emits `(js-for-of-in kind ident iter body)`, else falls back to classic `for(;;)`. Transpile: `js-transpile-for-of-in` wraps body in `(call/cc (fn (__break__) (let ((__js_items__ <normalized-source>)) (for-each (fn (ident) (call/cc (fn (__continue__) body))) items))))`. For `of` it normalizes via `js-iterable-to-list` (list → self, string → char list, dict → values). For `in` it iterates over `js-object-keys`. `break` / `continue` already route to the call/cc bindings. 8 new tests (flat, fill variants, indexOf with start, for-of array/string, for-in dict), **399/401** (391→+8). Conformance unchanged. Gotcha: **SX `cond` clauses evaluate only the last expr of a body. `(cond ((test) (set! a 1) (set! b 2)) …)` silently drops the first set!.** Must wrap multi-stmt clause bodies in `(begin …)`. First pass on the for-stmt rewrite had multi-expr cond clauses that silently did nothing — broke all existing for-loop tests, not just the new ones.
|
||
|
||
- 2026-04-23 — **String.replace/search/match + Array.from; js-num-to-int coerces strings; spread operator `...`.** Strings: `replace(regex|str, repl)`, `search(regex|str)`, `match(regex|str)` all handle both regex and plain-string args. Regex path walks via `js-string-index-of` with case-adjusted hay/needle. Array.from(iterable, mapFn?) via `js-iterable-to-list`. `js-num-to-int` now routes through `js-to-number` so `'abcd'.charAt('2')` and `.slice('1','3')` coerce properly. **Spread `...` in array literals and call args.** Parser: `jp-array-loop` and `jp-call-args-loop` detect `punct "..."` and emit `(js-spread inner)` entries. Transpile: if any element has a spread, the array/args list is built via `(js-array-spread-build (list "js-value" v) (list "js-spread" xs) ...)`. Runtime `js-array-spread-build` walks items, appending values directly and splicing spread via `js-iterable-to-list`. Works in call args (including variadic `Math.max(...arr)`) and in array literals (prefix, middle, and string-spread `[...'abc']`). Gotcha: early pass used `(js-sym ":js-spread")` thinking it'd make a keyword — `js-sym` makes a SYMBOL which becomes an env lookup (and fails as undefined). Use plain STRING `"js-spread"` as the discriminator. 10 new tests (replace/search/match both arg types, Array.from, coercion, 5 spread variants), **414/416** (399→+15). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **Object + array destructuring in var/let/const decls.** Parser: `jp-parse-vardecl` now handles three shapes — plain `ident`, `{a, b, c}` object pattern, `[a, , c]` array pattern with hole support. `jp-parse-obj-pattern` / `jp-parse-arr-pattern` / their loops collect the names. AST: `(js-vardecl-obj (names...) rhs)` and `(js-vardecl-arr (names-with-holes-as-nil...) rhs)`. Transpile: `js-vardecl-forms` dispatches on the three tags. Destructures emit `(define __destruct__ rhs)` then `(define name (js-get-prop __destruct__ key-or-index))` for each pattern element (skips nil holes in array patterns). 4 new unit tests. 418/420 (414→+4). Conformance unchanged. Gotcha: running destructuring tests sequentially — if epoch N defines `a, b` globally and epoch N+1 uses the same names as different types, "Not callable: N" results. Top-level `var` transpiles to `(define name value)`; re-defining a name that's held a value in a prior call to eval carries over. The proper fix would be to use `let` block-scoping; workaround for tests is unique names.
|
||
|
||
- 2026-04-23 — **Optional chaining `?.` + logical assignment `&&= / ||= / ??=`.** Parser: `jp-parse-postfix` handles `op "?."` followed by ident / `[` / `(` emitting `(js-optchain-member obj name)` / `(js-optchain-index obj k)` / `(js-optchain-call callee args)`. Transpile: all emit `(js-optchain-get obj key)` or `(js-optchain-call fn args)` — runtime short-circuits to undefined when the receiver is null/undefined. Logical assignment: `js-compound-update` gains `&&=` / `||=` / `??=` cases that emit `(if (js-to-boolean lhs) rhs lhs)` / `(if (js-to-boolean lhs) lhs rhs)` / `(if nullish? rhs lhs)`. 9 new tests. 427/429 (418→+9). Conformance unchanged.
|
||
|
||
- 2026-04-23 — **Callable `Number() / String() / Boolean() / Array()` + `Array.prototype.sort`.** Made the builtin constructor dicts callable by adding a `:__callable__` slot that points to a conversion function; `js-call-plain` and `js-function?` now detect dicts with `__callable__` and dispatch through it. `Number("42") === 42`, `String(true) === "true"`, `Boolean(0) === false`, `Array(3)` returns length-3 list, `Array(1,2,3)` returns `[1,2,3]`. `Array.prototype.sort(comparator?)` added via bubble-sort O(n²) `js-list-sort-outer!` / `-inner!`. Default comparator is lexicographic (JS-spec `toString()` then compare). Custom comparators get `(cmp a b) → number` and swap when positive. 11 new tests. 438/440 (427→+11). Conformance unchanged. Wide scoreboard (Math+Number+String+Array+{addition,equals,if,for,while}): baseline **259/5354 (4.8%)** before these improvements; top failures were ReferenceError (1056×), TypeError not-a-function (514×), array-like `{length:3, 0:41, 1:42, 2:43}` receivers (455×), SyntaxError (454×). Need to re-run scoreboard to see the delta from all the April 23rd work.
|
||
|
||
- 2026-04-23 — **Rest + rename in destructure, delete operator, Array.prototype / String.prototype / Number.prototype stubs, lastIndexOf, typeof fix, js-apply-fn handles callable dicts.** Array destructure `[a, ...rest]` transpiles rest-tail to `(js-list-slice tmp i (len tmp))`. Obj pattern `{key: local}` renames to `local`. `delete obj.k` → `js-delete-prop` which sets value to undefined. `Array.prototype.push` etc. are accessible as proto-functions that route through `js-this` + `js-invoke-method`. `Number.prototype` stub with `toString/valueOf/toFixed`. Nested destructuring patterns tolerated via `jp-skip-balanced` (treated as holes). Destructuring params in fn decls accepted (holes). **Final Math scoreboard: 66/288 (22.9%)** vs 56/288 (19.4%) baseline, +10 passes (+3.5 percentage points). 446/448 unit (438→+8). 148/148 slice unchanged. Top remaining Math failures: 94× ReferenceError (`Math.log`/`Math.sin`/`Math.cos` etc. not shimmed — no SX `sin`/`cos`/`log` primitives), 79× assertion-fail (numerical precision on `Math.floor` / `ceil` / `trunc` edge cases), 31× TypeError, 16× Timeout.
|
||
|
||
- 2026-04-23 (session 4) — **P0 harness cache + P1 Blockers + scoreboard-driven feature work. 114/300 → 147/300 (+33 tests, 38.0% → 49.0%).** Wide wall time 593s → 277s (2.14× via cache). Commits `4a277941..00edae49`:
|
||
1. **harness cache (`4a277941`)**: transpile HARNESS_STUB once per Python process, write SX to `.harness-cache/stub.<fp>.sx`, every worker `(load …)`s it instead of js-eval'ing the raw JS. Added `$`→`_js_dollar_` name-mangler in `js-sym` so SX symbols round-trip through `inspect`.
|
||
2. **Blockers entries (`dc97c173`)**: Math trig/transcendental primitives (22 missing), evaluator CPU bound (lexical addresses, inline caches, JIT-force, OCaml 5 domains).
|
||
3. **exponent notation (`7cffae21`)**: `js-num-from-string` now splits on e/E, parses mantissa and exponent separately, combines via new `js-pow-int`. `.12345e-3` was 0.12345, now 0.00012345. Also fixed the `js-string-trim` typo. +3 Number.
|
||
4. **callable-dict hasOwnProperty (`05aef11b`)**: Number/String/Array etc. carry `__callable__` so `js-function?` returns true and dispatch landed in the function-proto branch (which only knows name/length/prototype). Added `(not dict)` guard so dicts fall through to the keys-based path. +6 Number.
|
||
5. **.length / .name on constructor dicts (`f63934b1`)**: six `dict-set!` lines adding the spec'd `length=1` and `name="<Ctor>"` to Number/String/Array/Object/Boolean. +1 Number.
|
||
6. **ctor.prototype used by `new` (`bf09055c`)**: `js-get-ctor-proto` now returns `ctor.prototype` directly for dict ctors before falling through to the id-keyed table. Was synthesising an empty proto, so every `(new Number()).toLocaleString` was undefined. +5 Number, +2 String.
|
||
7. **hex-literal string→number (`00edae49`)**: `ToNumber("0x0")` was NaN, should be 0 per spec. Added `js-hex-prefix?` / `-is-hex-body?` / `-parse-hex` / `-hex-digit-value` and dispatch from both `js-is-numeric-string?` and `js-num-from-string`. +15 Number (S9.3.1_A16..A32 family).
|
||
8. Unit: 520/522 → **521/522** (test 903 was expecting wrong part count, fixed). Slice: 148/148 unchanged. Math scoreboard unchanged (still blocked on trig). String 30% → 33% (+3 via proto fix). Number 43% → 73% (+30). Wide 38% → 49%.
|
||
Also added `--dump-failures <path>` to the runner for tight debug loops.
|
||
- 2026-04-23 (session 3, continued) — **Scoreboard scoped 100/cat**: Math 40% / Number 43% / String 31% = 114/300 overall. Additional commits (85a329e8..c3b0aef1):
|
||
- `fn.length` reflects arity via `lambda-params` + `js-count-real-params`. Math.abs.length = 1 etc.
|
||
- `Object` global now callable (`new Object()`, `Object(5)`). Object.prototype has `hasOwnProperty`/`isPrototypeOf`/`propertyIsEnumerable`/`toString`/`valueOf`.
|
||
- Function receivers inherit Object.prototype builtins via `js-invoke-function-objproto`. `String.prototype.toUpperCase.hasOwnProperty('length')` works.
|
||
- `globalThis` → `js-global`; `eval` → `js-global-eval` (no-op stub); `Function` global stub (constructor throws TypeError, prototype populated).
|
||
- `URIError` and `EvalError` constructors.
|
||
|
||
- 2026-04-23 (session 3) — **Parallel runner, 60 new features, Math 39.6%, wide 36.4%.** Commits 65d4c706..edfbb754:
|
||
1. `test262-runner.py` rewritten with `multiprocessing.Pool` for `--workers N` shards; raw-fd line buffer replaces thread-per-line. Auto-defaults 1 worker on <=2-core machines (parallel slower on this box — OCaml CPU-bound, 2 processes starve). Classifier maps parser errors to SyntaxError so negative-SyntaxError tests pass instead of failing.
|
||
2. `Function.prototype.call/apply/bind` via `js-invoke-function-method`, dispatched in `js-invoke-method` when recv is a function. `fn.name/.length` also exposed.
|
||
3. Numeric keys in object literals stringify on parse (`{0: 41, 1: 42}` no longer crashes `dict-set!`). Parser: number token value str-coerced.
|
||
4. Array-like receivers for `Array.prototype.X.call(dict)` via `js-arraylike-to-list` — reads `length` + indexed keys in order. `js-iterable-to-list` also respects `length` on dicts.
|
||
5. **Number methods on primitives**: `(5).toString()`, `(16).toString(16)`, `.toFixed(n)`, `.valueOf()`. `js-invoke-method` branches on `(number? recv)`. Radix 2-36 supported via `js-num-to-str-radix`.
|
||
6. **Boolean methods**: `true.toString()`, `.valueOf()`.
|
||
7. **NaN / Infinity resolve**: transpile-time rewrite `NaN → (js-nan-value)` and `Infinity → (js-infinity-value)`, because SX's tokenizer parses `NaN` as numeric literal and forbids `(define NaN ...)`. `js-number-is-nan` uses string-inspect (SX `(= nan nan)` returns true). `js-strict-eq` returns false for NaN pairs per spec.
|
||
8. 15 new Array.prototype methods: `at`, `flatMap`, `findLast`, `findLastIndex`, `reduceRight`, `toString`, `toLocaleString`, `keys`, `values`, `entries`, `copyWithin`, `toReversed`, `toSorted`. Mutating `unshift`/`splice` are stubs (pop-last!/pop-first! primitives are no-ops — runtime limitation).
|
||
9. 10 new String.prototype methods: `at`, `codePointAt`, `lastIndexOf`, `localeCompare`, `replaceAll`, `normalize`, `toLocaleLowerCase/UpperCase`, `isWellFormed`, `toWellFormed`.
|
||
10. 10 new Object.* globals: `getPrototypeOf`, `setPrototypeOf`, `create`, `defineProperty(ies)`, `getOwnPropertyNames/Descriptor(s)`, `isExtensible/Frozen/Sealed`, `seal`, `preventExtensions`, `is`, `fromEntries`, `hasOwn`.
|
||
11. `Array.prototype` / `String.prototype` dicts updated to include all new methods (so `.call`-ing them works).
|
||
12. `js-to-number(undefined) → NaN` (was 0); `js-string-to-number("abc") → NaN` via new `js-is-numeric-string?`. `parseInt('123abc',10) → 123` (new digit-walker), supports radix 2-36. `parseFloat('3.14xyz') → 3.14` (new float-prefix matcher). Added `encodeURIComponent`, `decodeURIComponent`, `encodeURI`, `decodeURI`.
|
||
13. Harness stub: `assert` itself callable via `assert.__callable__ = __assert_call__` (many tests do `assert(cond, msg)`). `verifyNotWritable` etc. widened to 5-arg signature.
|
||
14. `Number` global rebuilt: correct `MAX_VALUE` (computed at load by doubling until `== Infinity`, yields ~1e308), `POSITIVE_INFINITY/NEGATIVE_INFINITY/NaN` via function-form values (SX literals overflow/mangle), `toFixed` handles NaN/Infinity/negative, `prototype.toString` accepts radix.
|
||
15. **Scoreboard deltas**: Math 66/288 (22.9%) → 114/288 (39.6%) +48 passes. Wide (Math+Number+String at 150 each): 164/450 (36.4%). Number category isolated: 44.7%. Unit 514/516 (506→+8 from NaN/coerce/Object tests).
|
||
|
||
Top remaining failure modes (wide scoreboard, 450 runnable):
|
||
- 172× Test262Error (assertion failed) — real semantic bugs, numeric precision on Math/Number/String.prototype
|
||
- 54× TypeError: not a function
|
||
- 29× Timeout (slow string/regex loops)
|
||
- 16× ReferenceError — still some missing globals
|
||
|
||
## Phase 3-5 gotchas
|
||
|
||
Worth remembering for later phases:
|
||
|
||
- **Varargs in SX** use `&rest args`, not bare `args`. `(fn args …)` errors with "Expected list, got symbol" — use `(fn (&rest args) …)`.
|
||
- **`make-symbol`** is the way to build an SX identifier-symbol at runtime for later `eval-expr`. Use it to turn JS idents into SX variable references.
|
||
- **`eval-expr`** is the primitive you want to evaluate an SX list you've just constructed. It's already in the kernel primitives.
|
||
- **Keywords print as quoted strings** via the epoch `eval` command — `:foo` comes back as `"foo"`. Affected the `js-undefined` test expectation.
|
||
- **`char-code` (not `code-char`)** for char→codepoint. No `bit-not` primitive — implement `~x` as `-(int(x)+1)`.
|
||
- **Epoch `eval` string must double-escape** to survive two nested SX string literals (`(eval "(js-eval \"…\")")`). The conformance harness uses a small Python helper to do that reliably.
|
||
- **Shell heredocs and `||`** — writing fixtures with `||` in a here-doc fed to a `while read` loop gets interpreted as a pipe mid-parse. Use `case|a\|\|b|…` style or hand-write the `||` cases.
|
||
|
||
## Blockers / shared-file issues
|
||
|
||
Anything that would require a change outside `lib/js/` goes here with a minimal repro. Don't fix from the loop.
|
||
|
||
- **Pending-Promise await** — our `js-await-value` drains microtasks and unwraps *settled* Promises; it cannot truly suspend a JS fiber and resume later. Every Promise that settles eventually through the synchronous `resolve`/`reject` + microtask path works. A Promise that never settles without external input (e.g. a real `setTimeout` waiting on the event loop) would hit the `"await on pending Promise (no scheduler)"` error. Proper async suspension would need the JS eval path to run under `cek-step-loop` (not `eval-expr` → `cek-run`) and treat `await pending-Promise` as a `perform` that registers a resume thunk on the Promise's callback list. Non-trivial plumbing; out of scope for this phase. Consider it a Phase 9.5 item.
|
||
|
||
- **Regex platform primitives** — runtime ships a substring-based stub (`js-regex-stub-test` / `-exec`). Overridable via `js-regex-platform-override!` so a real engine can be dropped in. Required platform-primitive surface:
|
||
- `regex-compile pattern flags` — build an opaque compiled handle
|
||
- `regex-test compiled s` → bool
|
||
- `regex-exec compiled s` → match dict `{match index input groups}` or nil
|
||
- `regex-match-all compiled s` → list of match dicts (or empty list)
|
||
- `regex-replace compiled s replacement` → string
|
||
- `regex-replace-fn compiled s fn` → string (fn receives match+groups, returns string)
|
||
- `regex-split compiled s` → list of strings
|
||
- `regex-source compiled` → string
|
||
- `regex-flags compiled` → string
|
||
Ideally a single `(js-regex-platform-install-all! platform)` entry point the host calls once at boot. OCaml would wrap `Str` / `Re` or a dedicated regex lib; JS host can just delegate to the native `RegExp`.
|
||
|
||
- **Math trig + transcendental primitives missing.** The scoreboard shows 34× "TypeError: not a function" across the Math category — every one a test calling `Math.sin/cos/tan/log/…` on our runtime. We shim `Math` via `js-global`; the SX runtime supplies `sqrt`, `pow`, `abs`, `floor`, `ceil`, `round` and a hand-rolled `trunc`/`sign`/`cbrt`/`hypot`. Nothing else. Missing platform primitives (each is a one-line OCaml/JS binding, but a primitive all the same — we can't land approximation polynomials from inside the JS shim, they'd blow `Math.sin(1e308)` precision):
|
||
- Trig: `sin`, `cos`, `tan`, `asin`, `acos`, `atan`, `atan2`
|
||
- Hyperbolic: `sinh`, `cosh`, `tanh`, `asinh`, `acosh`, `atanh`
|
||
- Log/exp: `log` (natural), `log2`, `log10`, `log1p`, `expm1`
|
||
- Bit-width: `clz32`, `imul`, `fround`
|
||
- Variadic lifts: `hypot(…)` (n-args), `max(…)`/`min(…)` (we have 2-arg; JS allows 0 through N)
|
||
Minimal repro: `Math.sin(0)` currently raises `TypeError: Math.sin is not a function` because `js-global.Math` has no `sin` key. Once the primitives exist in the runtime, `js-global.Math` can be extended in one drop — all 34 Math `not a function` failures flip together.
|
||
|
||
- **SX number promotion loses floats on exact-int results.** Minimal repro: `(type-of (* 1.5 2))` is `"number"` (fine) but the value is `3` — an int. In OCaml terms, multiplying a float by something that produces an integral float representable in a `Pervasives.int` triggers a narrowing. Consequence: any iterative float routine like `(let loop ((x 1.5) (n 100)) (if (<= n 0) x (loop (* x 2.0) (- n 1))))` overflows to 0 by n=60 because it's walking ints at that point. In JS terms this blocks:
|
||
- `Number.MAX_VALUE` — our `js-max-value-approx` loops 1.0 × 2 and overflows to 0; we can't compute a correct 1.7976931348623157e308 from inside the runtime
|
||
- `Number.MIN_VALUE` — same shape (loop 1.0 / 2 → 0 before reaching denormal 5e-324)
|
||
- Any literal `1e308` — the SX tokenizer parses `e308` but clips too
|
||
- `Math.pow(2, 100)` — same loop
|
||
Proper fix is spec-level: keep `Sx_types.Number` boxed as OCaml `float` until an explicit int cast happens, or introduce a separate `Sx_types.Int` path and a promotion rule. For js-on-sx, `Number.MAX_VALUE` tests are blocked until then. Works around ~6 Number failures and nontrivial surface in Math.
|
||
|
||
- **Evaluator CPU bound at ~1 test/s on a 2-core box.** Runner auto-defaults to 1 worker on <=2-cores because two CEK workers starve each other — both are pure-Python/OCaml heavy loops with no IO concurrency to exploit. Each test that parses JS, transpiles to SX, then evaluates the SX under CEK runs in the 0.3–3s range; the long tail is mostly per-test `js-parse` scaling superlinearly with test length plus CEK env lookup walking a list of frames per free variable. Optimization surface someone else could pick up (all shared-file):
|
||
- **Lexical addresses** in the evaluator: swap env-walk-by-name for `(depth, slot)` tuples resolved at transpile time. Would slash the per-CEK-step cost by an order of magnitude on tight loops.
|
||
- **Inline caches on `js-get-prop`.** Every `a.b` walks the `__proto__` chain; 99% of call sites hit the same shape. A monomorphic cache keyed on the receiver's structural shape would collapse that to one lookup.
|
||
- **Force JIT on transpiled JS bodies.** Currently JS → SX → tree-walk CEK. The VM JIT exists but lambda compilation triggers on lazy call — and many test bodies are top-level straight-lines that never hit the JIT threshold. An explicit `(jit-compile!)` after transpile, or an AOT pass when loading the harness cache, would be a one-shot win.
|
||
- **OCaml 5 domains.** The runner has `--workers N` but the box is 2-core. OCaml 5 multicore would let one process host N CEK domains with work-stealing instead of N Python subprocesses each booting their own kernel.
|
||
Minimal repro: `time python3 lib/js/test262-runner.py --filter built-ins/Number --max-per-category 50 --workers 1` — currently ~50s wall, ~1 test/s after the ~4s warm-start. Target for any of the above fixes: ≥3× uplift. `built-ins/String` timeouts will thin out naturally at that rate because many of them are pure 2000-iter `for` loops hitting the per-test 5s cap.
|
||
|
||
## First-iteration checklist (scaffolding) — DONE
|
||
|
||
- [x] `lib/js/lexer.sx` — stub `js-tokenize`
|
||
- [x] `lib/js/parser.sx` — stub `js-parse`
|
||
- [x] `lib/js/transpile.sx` — stub `js-transpile`
|
||
- [x] `lib/js/runtime.sx` — stub `js-global`, `js-to-boolean`
|
||
- [x] `lib/js/test.sh` — epoch-protocol runner mirroring `lib/hyperscript/test.sh`
|
||
- [x] Smoke suite green (7/7)
|
||
|
||
Next iteration: Phase 1 — lexer. Start with numeric literals + identifiers + whitespace skipping; extend test.sh with tokenization assertions.
|