Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 27s
542 lines
133 KiB
Markdown
542 lines
133 KiB
Markdown
# JS-on-SX: cherry-picked test262 conformance
|
||
|
||
Transpile a restricted JS subset to SX AST, run on the existing CEK/VM. Goal is runtime-hardening via conformance tests, plus a second hyperscript substrate (hyperscript.js running on SX).
|
||
|
||
## Ground rules for the loop
|
||
|
||
- **Scope:** only touch `lib/js/**` and `plans/js-on-sx.md`. Do **not** edit `spec/evaluator.sx`, `spec/primitives.sx`, `shared/sx/**`, or `lib/hyperscript/**` — the user is working on those elsewhere.
|
||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix them from this loop.
|
||
- **SX files:** use `sx-tree` MCP tools only (never `Edit`/`Read`/`Write` on `.sx` files). Use `sx_write_file` for new files, path-based or pattern-based edits for changes.
|
||
- **Architecture:** JS source → JS AST → SX AST → existing CEK. No standalone JS evaluator — reuse everything.
|
||
- **Tests:** mirror `lib/hyperscript/test.sh` pattern. Start with a tiny in-repo smoke suite; graduate to a cherry-picked test262 slice once the lexer+expression-parser cycle is green.
|
||
|
||
## The loop
|
||
|
||
A single long-running background agent works `plans/js-on-sx.md` forever, one feature per commit. It runs a prioritized queue driven by the real test262 scoreboard. The briefing lives at `plans/agent-briefings/loop.md`; the recovery helper is `plans/restore-loop.sh`.
|
||
|
||
The queue (condensed — full version in the briefing):
|
||
|
||
1. Baseline commit (stage what's on disk now).
|
||
2. Fix `lib/js/test262-runner.py` so it produces a real scoreboard.
|
||
3. Full scoreboard run across the whole `test/` tree.
|
||
4. Regex lexer/parser/runtime stub + Blockers entry listing platform primitives needed.
|
||
5. Scoreboard-driven: pick the worst-passing category each iteration; fix; re-score.
|
||
6. When the scoreboard plateaus, tackle deferred items (ASI, CEK-suspend await).
|
||
|
||
### Crash recovery
|
||
|
||
The agent process dies with the machine. **Work survives if it was committed** — that's why every iteration ends in a commit. After a crash:
|
||
|
||
```bash
|
||
bash plans/restore-loop.sh # show state: tests, commits, scoreboard, regex hook
|
||
bash plans/restore-loop.sh --print # also cat the briefing for easy copy-paste
|
||
```
|
||
|
||
Then respawn the agent by pasting `plans/agent-briefings/loop.md` into Claude Code via the `Agent` tool with `run_in_background=true`. The agent re-reads the plan, picks up wherever the queue has got to, and carries on.
|
||
|
||
## Architecture sketch
|
||
|
||
```
|
||
JS source text
|
||
│
|
||
▼
|
||
lib/js/lexer.sx — tokens: {:type :value :pos}
|
||
│
|
||
▼
|
||
lib/js/parser.sx — JS AST as SX trees, e.g. (js-binop "+" left right)
|
||
│
|
||
▼
|
||
lib/js/transpile.sx — JS AST → SX AST (reusable by CEK)
|
||
│ e.g. (js-binop "+" l r) → (+ (transpile l) (transpile r))
|
||
▼
|
||
existing CEK / VM — evaluation, async/IO, exceptions already handled
|
||
```
|
||
|
||
Runtime shims in `lib/js/runtime.sx`: `js-global`, `js-console-log`, `js-typeof`, coercion helpers (`to-number`, `to-string`, `to-boolean`, abstract-equality), eventually the prototype chain.
|
||
|
||
## Roadmap
|
||
|
||
Each item: implement → tests → update progress. Mark `[x]` when tests green.
|
||
|
||
### Phase 1 — Lexer
|
||
- [x] Numeric literals (int, float, hex, exponent)
|
||
- [x] String literals (double, single, escape sequences, template strings later)
|
||
- [x] Identifiers + reserved words
|
||
- [x] Punctuation: `( ) { } [ ] , ; : . ...`
|
||
- [x] Operators: `+ - * / % ** = == === != !== < > <= >= && || ! ?? ?: & | ^ ~ << >> >>> += -= ...`
|
||
- [x] Comments (`//`, `/* */`)
|
||
- [x] Automatic Semicolon Insertion (defer — initially require semicolons)
|
||
|
||
### Phase 2 — Expression parser (Pratt-style)
|
||
- [x] Literals → AST nodes
|
||
- [x] Binary operators with precedence
|
||
- [x] Unary operators (`- + ! ~ typeof void`)
|
||
- [x] Member access (`.`, `[]`)
|
||
- [x] Function calls
|
||
- [x] Array literals
|
||
- [x] Object literals (string/ident keys, shorthand later)
|
||
- [x] Conditional `a ? b : c`
|
||
- [x] Arrow functions (expression body only)
|
||
|
||
### Phase 3 — Transpile to SX AST
|
||
- [x] Numeric/string/bool/null literals → SX literals
|
||
- [x] Arithmetic: `+ - * / % **` with numeric coercion (`js-add` does string-concat dispatch)
|
||
- [x] Comparisons: `=== !== == != < > <= >=`
|
||
- [x] Logical: `&& ||` (short-circuit, value-returning via thunk trick); `??` nullish coalesce
|
||
- [x] Unary: `- + ! ~ typeof void`
|
||
- [x] Member access → `js-get-prop` (handles dicts, lists `.length`/index, strings)
|
||
- [x] Array literal → `(list ...)`
|
||
- [x] Object literal → `(dict)` + `dict-set!` sequence
|
||
- [x] Function call → SX call (callee can be ident, member, arrow)
|
||
- [x] Arrow fn → `(fn ...)` (params become SX symbols; closures inherited)
|
||
- [x] Assignment (`=` and compound `+= -= ...`) on ident/member/index targets
|
||
- [x] `js-eval` end-to-end: source → tokens → AST → SX → `eval-expr`
|
||
|
||
### Phase 4 — Runtime shims (`lib/js/runtime.sx`)
|
||
- [x] `js-typeof`, `js-to-number`, `js-to-string`, `js-to-boolean`
|
||
- [x] Abstract equality (`js-loose-eq`) vs strict (`js-strict-eq`)
|
||
- [x] Arithmetic (`js-add` `js-sub` `js-mul` `js-div` `js-mod` `js-pow` `js-neg` `js-pos`)
|
||
- [x] Logical (`js-and` `js-or` via thunks for lazy rhs) and `js-not` / `js-bitnot`
|
||
- [x] Relational (`js-lt` `js-gt` `js-le` `js-ge`) incl. lexicographic strings
|
||
- [x] `js-get-prop` / `js-set-prop` (dict/list/string; `.length`; numeric index)
|
||
- [x] `console.log` → `log-info` bridge (`console` dict wired)
|
||
- [x] `Math` object shim (`abs` `floor` `ceil` `round` `max` `min` `random` `PI` `E`)
|
||
- [x] `js-undefined` sentinel (keyword) distinct from `nil` (JS `null`)
|
||
- [x] Number parser `js-num-from-string` (handles int/float/±sign, no NaN yet)
|
||
|
||
### Phase 5 — Conformance harness
|
||
- [x] Cherry-picked slice vendored at `lib/js/test262-slice/` (69 fixtures across 7 categories)
|
||
- [x] Harness `lib/js/conformance.sh` — batch-load kernel, one epoch per fixture, substring-compare
|
||
- [x] Initial target: ≥50% pass. **Actual: 69/69 (100%).**
|
||
|
||
### Phase 6 — Statements
|
||
- [x] `var`/`let`/`const` declarations (all behave as `define` — block scope via SX lexical semantics)
|
||
- [x] `if`/`else`
|
||
- [x] `while`, `do..while`
|
||
- [x] `for (init; cond; step)`
|
||
- [x] `return`, `break`, `continue` (via `call/cc` continuation bindings `__return__` / `__break__` / `__continue__`)
|
||
- [x] Block scoping (via `begin` — lexical scope inherited, no TDZ)
|
||
|
||
### Phase 7 — Functions & scoping
|
||
- [x] `function` declarations (with call/cc-wrapped bodies for `return`)
|
||
- [x] Function expressions (named + anonymous)
|
||
- [x] Hoisting — function decls hoisted to enclosing scope (scan body first, emit defines ahead of statements)
|
||
- [x] Closures — work via SX `fn` env capture
|
||
- [x] Rest params (`...rest` → `&rest`)
|
||
- [x] Default parameters (desugar to `if (param === undefined) param = default`)
|
||
- [x] `var` hoisting (shallow — collects direct `var` decls, emits `(define name :js-undefined)` before funcdecls)
|
||
- [ ] `let`/`const` TDZ (deferred)
|
||
|
||
### Phase 8 — Objects, prototypes, `this`
|
||
- [x] Property descriptors (simplified — plain-dict `__proto__` chain, `js-set-prop` mutates)
|
||
- [x] Prototype chain lookup (`js-dict-get-walk` walks `__proto__`)
|
||
- [x] `this` binding rules: method call, function call (undefined), arrow (lexical)
|
||
- [x] `new` + constructor semantics (fresh dict, proto linked, ctor called with `this`)
|
||
- [x] ES6 classes (sugar over prototypes, incl. `extends`)
|
||
- [x] Array mutation: `a[i] = v`, `a.push(...)` — via `set-nth!` / `append!`
|
||
- [x] Array builtins: push, pop, shift, slice, indexOf, join, concat, map, filter, forEach, reduce
|
||
- [x] String builtins: charAt, charCodeAt, indexOf, slice, substring, toUpperCase, toLowerCase, split, concat
|
||
- [x] `instanceof` and `in` operators
|
||
|
||
### Phase 9 — Async & Promises
|
||
- [x] Promise constructor + `.then` / `.catch` / `.finally`
|
||
- [x] `Promise.resolve` / `Promise.reject` / `Promise.all` / `Promise.race`
|
||
- [x] `async` functions (decl, expr, arrow) return Promises
|
||
- [x] `await` — synchronous-ish: drains microtasks, unwraps settled Promise
|
||
- [x] Microtask queue with FIFO drain (`__drain()` exposed to JS)
|
||
- [ ] True CEK suspension on `await` for pending Promises (deferred — needs cek-step-loop plumbing)
|
||
|
||
### Phase 10 — Error handling
|
||
- [x] `throw` statement → `(raise v)`
|
||
- [x] `try`/`catch`/`finally` (desugars to `guard` + optional finally wrapper)
|
||
- [x] Error hierarchy (`Error`, `TypeError`, `RangeError`, `SyntaxError`, `ReferenceError` as constructor shims)
|
||
|
||
### Phase 11 — Stretch / deferred
|
||
- ASI, regex literals, generators, iterators, destructuring, template strings with `${}`, tagged templates, Symbol, Proxy, typed arrays, ESM modules.
|
||
|
||
## Progress log
|
||
|
||
Append-only record of completed iterations. Loop writes one line per iteration: date, what was done, test count delta.
|
||
|
||
- 2026-05-10 — **Real `Date` construction + getters via Howard-Hinnant civil-day arithmetic.** `js-date-from-parts` now computes a true ms-since-epoch from `(year, month, day, hour, min, sec, ms)` via `js-date-civil-to-days` (the inverse of last iteration's `days-to-ymd`), with the legacy 2-digit-year coercion (0..99 → 1900+y). `getFullYear/Month/Date/Day/Hours/Minutes/Seconds/Milliseconds` (UTC + non-UTC) all share a new `js-date-getter`: TypeErrors on non-Date this, returns NaN on invalid time, otherwise decomposes ms into y/m/d/h/m/s/ms/dow. Plus added `Date.prototype.constructor = Date` (was missing). Result: each of the 8 Date getter categories went 2/6 → 5/6 (+3 each, +24 total). Date toISOString 11/16 → 13/16. Some Date construction-loop tests now exceed the 15s per-test timeout — the new civil math is heavier than the old (year-1970)*ms-per-year approximation, but correctness wins. conformance.sh: 148/148.
|
||
|
||
- 2026-05-10 — **`Date.prototype.toISOString` produces real `YYYY-MM-DDTHH:mm:ss.sssZ` and validates input.** Old `js-date-iso` only computed the year and hardcoded the rest as `01-01T00:00:00.000Z`. Added: (1) TypeError when this isn't a Date (no `__js_is_date__` slot); (2) RangeError when ms is NaN, undefined, or |ms| > 8.64e15; (3) full date breakdown via Howard-Hinnant `days_to_civil` algorithm (`js-date-days-to-ymd`) → year/month/day, plus modular hours/min/sec/ms; (4) extended-year format `±YYYYYY` for years outside 0..9999. Result: built-ins/Date/prototype/toISOString 7/16 → 11/16 (+4). Date 21/30. conformance.sh: 148/148.
|
||
|
||
- 2026-05-10 — **`JSON.stringify` honours `replacer` (function + array forms), `space`, and `toJSON`.** Previous impl ignored the second/third arguments entirely and never called `toJSON`. Rewrote around a `js-json-serialize-property(key, holder, rep-fn, rep-keys, gap, indent)` core: walks `toJSON` first, then replacer-fn (with `holder` as `this`); arrays-as-replacer become a property-name allowlist; numeric `space` clamped to 0..10 spaces, string `space` truncated to 10 chars, non-empty gap activates indented output with `:` → `: ` separator. Number wrapper / String wrapper / Boolean wrapper unwrap before serialization; non-finite numbers serialize as `"null"`; functions serialize as `undefined`. Result: built-ins/JSON/stringify 6/30 → 14/30 (+8). conformance.sh: 148/148.
|
||
|
||
- 2026-05-10 — **`JSON.parse` raises spec-correct `SyntaxError` instances and rejects malformed input.** Previously `JSON.parse("12 34")` silently returned `12` (no trailing-content check), `JSON.parse('" |