js-on-sx: baseline commit (278/280 unit, 148/148 slice, runner stub)
Initial commit of the lib/js/ tree and plans/ directory. A previous session left template-string work in progress — 278/280 unit tests pass (2 failing: tpl part-count off-by-one, escaped-backtick ident lookup). test262-runner.py and scoreboard are placeholders (0/8 with 7 timeouts); fixing the runner is the next queue item.
This commit is contained in:
88
plans/agent-briefings/loop.md
Normal file
88
plans/agent-briefings/loop.md
Normal file
@@ -0,0 +1,88 @@
|
||||
# js-on-sx loop agent (single agent, queue-driven)
|
||||
|
||||
Role: iterates `plans/js-on-sx.md` forever. Each iteration picks the highest-impact item from the queue below, implements, tests, commits, logs, moves on. Scoreboard-driven: the real test262 pass rate is the north star.
|
||||
|
||||
```
|
||||
description: js-on-sx queue loop
|
||||
subagent_type: general-purpose
|
||||
run_in_background: true
|
||||
```
|
||||
|
||||
## Prompt
|
||||
|
||||
You are the sole background agent working `/root/rose-ash/plans/js-on-sx.md`. A previous three-agent setup was consolidated — there is only you now. You work a prioritized queue, forever, one item per commit.
|
||||
|
||||
## Current state (restart baseline — verify before iterating)
|
||||
|
||||
- Branch: `architecture`. HEAD: `14b6586e` (HS-related, not js-on-sx).
|
||||
- `lib/js/` is **untracked** — nothing is committed yet. First commit should stage everything current on disk.
|
||||
- `lib/js/test262-upstream/` is a clone of tc39/test262 pinned at `d5e73fc8d2c663554fb72e2380a8c2bc1a318a33`. **Gitignore it** (`lib/js/.gitignore` → `test262-upstream/`). Do not commit the 50k test files.
|
||||
- `lib/js/test262-runner.py` exists but is buggy — current scoreboard is `0/8 (7 timeouts, 1 fail)`. The runner needs real work: harness script loading, batching, per-test timeout tuning, strict-mode skipping.
|
||||
- `lib/js/test262-scoreboard.{json,md}` exist as placeholders. Regenerate after fixing the runner.
|
||||
- `bash lib/js/test.sh` → **254/254** pass (timeout 180s already applied).
|
||||
- `bash lib/js/conformance.sh` → **148/148** pass.
|
||||
|
||||
## The queue
|
||||
|
||||
Work in this order. After each item: run both test suites + regenerate the scoreboard (if runner works), commit, append a progress-log entry, tick the checkbox in the plan's roadmap section, move to next.
|
||||
|
||||
1. **Baseline commit.** Stage the whole `lib/js/` tree + `plans/` directory as-is and commit. Message: `js-on-sx: baseline commit (254/254 unit, 148/148 slice, runner stub)`. After this, no more giant commits — one feature per commit.
|
||||
|
||||
2. **Fix the test262 runner** (`lib/js/test262-runner.py`). Goal: get a real pass-rate number against at least a few thousand tests. Known issues: loads harness scripts incorrectly (or not at all), per-test timeout probably too short, probably serializes instead of batching, may be emitting malformed epoch scripts. Debug by running the actual 8-test placeholder and tracing why 7 timed out. Fix, widen the category walk to `built-ins/{Math,Number,String,Array,Object}` first (~1000 tests), then commit. Re-run — expect somewhere in 5-30% pass rate.
|
||||
|
||||
3. **Full scoreboard run.** Widen to the whole `test/` tree. Record wall-time, pass rate, top-10 worst categories, top-10 failure modes. Save to `test262-scoreboard.{json,md}`. Commit. This is the dial that drives everything after.
|
||||
|
||||
4. **Regex lexer+parser+runtime stub.** Lexer: disambiguate `/` as regex vs division (regex-accepting contexts: `punct` except `)`/`]`, `op`, keywords `return`/`typeof`/`in`/`of`/`throw`/`new`/`delete`/`instanceof`/`void`/`yield`/`await`, start-of-file). Emit `{:type "regex" :value {:pattern :flags}}`. Parser: `(js-regex pat flags)` AST. Transpile: `(js-regex-new pat flags)`. Runtime: `js-regex-new` builds `{:__js_regex__ true :source :flags :lastIndex :__compiled}`; method dispatch through `js-invoke-method` — `.test`, `.exec`, `.source`, `.flags`, booleans for flags. Route `String.prototype.{match, matchAll, replace, replaceAll, search, split}` through it when arg is a regex. **Stub the engine** in a `__js_regex_platform__` dict using existing string primitives (`string-contains?`, `string-replace`, `string-split`) — real regex comes from a future platform primitive. Expose `js-regex-platform-override!` so the primitive can be swapped in later. Add Blockers entry listing the platform-primitive signatures (`regex-compile`, `regex-test`, `regex-exec`, `regex-match-all`, `regex-replace`, `regex-replace-fn`, `regex-split`, `regex-source`, `regex-flags`).
|
||||
|
||||
5. **Scoreboard-driven from here.** Each iteration: re-read `lib/js/test262-scoreboard.md`, pick the worst-passing category where the failure mode is something you can fix in <a day's work, fix it, re-run the scoreboard, log the delta. Typical candidates, in likely order of impact:
|
||||
- Template strings with `${}` (lexer `read-template` in `lib/js/lexer.sx` ~line 265 currently flattens — extend to split parts; transpile to `js-add`-chain with `js-to-string` on expression parts).
|
||||
- `for...of` + iterator protocol.
|
||||
- Array/object destructuring (declarations and function params).
|
||||
- Spread `...` in call args / array / object literals.
|
||||
- UTF-16 `JsString` value type (big one — see "JsString shape" below).
|
||||
- Optional chaining `?.`, nullish assignment `??=`, logical assignment `&&=` / `||=`.
|
||||
- Private class fields `#x`.
|
||||
- Proper `var` hoisting + TDZ.
|
||||
- Strict mode detection and honoring.
|
||||
- Property descriptors + getters/setters (big — full meta-object protocol).
|
||||
- Symbol, Proxy, Reflect.
|
||||
- Generators (`function*` / `yield`) — needs call/cc machinery.
|
||||
- Async iterators / `for await`.
|
||||
- Tagged templates.
|
||||
|
||||
6. **When scoreboard plateaus**, drop back to the deferred items from earlier phases: ASI (Phase 1), true CEK suspension on pending await (Phase 9).
|
||||
|
||||
## JsString shape (for when you pick it up)
|
||||
|
||||
Tagged dict: `{:__js_string__ true :utf16 <list-of-uint16> :str <lazy-utf8-cache-or-nil>}`. `:utf16` is canonical. Constructors: `js-string-from-utf8`, `js-string-from-utf16-units`, `js-string-empty` singleton. Observers: `js-string-length` (code-unit count, what `.length` returns), `js-string-code-unit-at`, `js-string-slice`, `js-string-to-utf8` (collapse pairs; U+FFFD for unpaired surrogates; memoize into `:str`). Every JS "string" entry point lifts raw SX strings via `js-as-js-string`. `js-eval` result: convert JsString → SX string for terminal output. Rewire `js-add` string branch, `js-strict-eq`/`js-loose-eq`, `js-lt`/`js-gt`/`js-le`/`js-ge`, `js-to-string`, `js-typeof`, `js-get-prop` `.length` and indexing, and every `String.prototype` method. Non-goals: `.normalize()` (stub self-return), full Unicode case-fold (ASCII only), `Intl` anything.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **Scope:** only `lib/js/**` and `plans/js-on-sx.md`. Do NOT touch `spec/`, `shared/`, `lib/hyperscript/`. Shared-file issues go under the plan's "Blockers" section.
|
||||
- **SX files:** `sx-tree` MCP tools ONLY. `sx_summarise` / `sx_read_subtree` / `sx_find_all` / `sx_get_context` before edits. `sx_replace_node` / `sx_insert_child` / `sx_insert_near` / `sx_replace_by_pattern` / `sx_rename_symbol` for edits. `sx_validate` after. `sx_write_file` for new files. Never `Edit`/`Read`/`Write` on `.sx`.
|
||||
- **Shell, Python, Markdown, JSON:** edit normally.
|
||||
- **Branch:** `architecture`. Commit locally. Never push. Never touch `main`.
|
||||
- **Commit granularity:** one feature per commit. Short, factual commit messages. Commit even if a partial fix — don't hoard changes.
|
||||
- **Tests:** `bash lib/js/test.sh` (254/254 baseline) and `bash lib/js/conformance.sh` (148/148 baseline). Never regress. If a feature requires larger refactor, split into multiple commits each green.
|
||||
- **Plan file:** append one paragraph per iteration to "Progress log". Tick `[x]` boxes. Don't rewrite history.
|
||||
|
||||
## Gotchas already learned
|
||||
|
||||
- SX `do` is R7RS iteration — **use `begin`** for multi-expr sequences.
|
||||
- `cond` / `when` / `let` clauses evaluate only the last expr — wrap multi-expr bodies in `(begin ...)`.
|
||||
- `guard` handler clauses same rule: `(guard (e (else (begin ...))))`.
|
||||
- `type-of` on user fn returns `"lambda"`, not `"function"`. Use `js-function?`.
|
||||
- **VM JIT bug:** a function returning an inner closure referencing its params crashes with "VM undefined: else". Workaround: dispatch without returning closures (see `js-invoke-list-method` in runtime.sx).
|
||||
- `&rest args` is SX varargs. `make-symbol` builds identifier symbols.
|
||||
- Keywords render as quoted strings through the `eval` epoch command.
|
||||
- Shell heredoc `||` gets eaten by bash — escape or use `case`.
|
||||
- `...` lexes as `punct`, not `op`.
|
||||
|
||||
## Style
|
||||
|
||||
- No comments in `.sx` unless non-obvious.
|
||||
- No new planning docs — update the plan inline.
|
||||
- One feature per iteration. Commit. Log. Next.
|
||||
- If blocked for two iterations on the same issue, add to Blockers and move on.
|
||||
|
||||
Go. Start with (1) baseline commit, then (2) fix the runner. Keep iterating indefinitely until stopped.
|
||||
200
plans/js-on-sx.md
Normal file
200
plans/js-on-sx.md
Normal file
@@ -0,0 +1,200 @@
|
||||
# JS-on-SX: cherry-picked test262 conformance
|
||||
|
||||
Transpile a restricted JS subset to SX AST, run on the existing CEK/VM. Goal is runtime-hardening via conformance tests, plus a second hyperscript substrate (hyperscript.js running on SX).
|
||||
|
||||
## Ground rules for the loop
|
||||
|
||||
- **Scope:** only touch `lib/js/**` and `plans/js-on-sx.md`. Do **not** edit `spec/evaluator.sx`, `spec/primitives.sx`, `shared/sx/**`, or `lib/hyperscript/**` — the user is working on those elsewhere.
|
||||
- **Shared-file issues** go under "Blockers" below with a minimal repro; do not fix them from this loop.
|
||||
- **SX files:** use `sx-tree` MCP tools only (never `Edit`/`Read`/`Write` on `.sx` files). Use `sx_write_file` for new files, path-based or pattern-based edits for changes.
|
||||
- **Architecture:** JS source → JS AST → SX AST → existing CEK. No standalone JS evaluator — reuse everything.
|
||||
- **Tests:** mirror `lib/hyperscript/test.sh` pattern. Start with a tiny in-repo smoke suite; graduate to a cherry-picked test262 slice once the lexer+expression-parser cycle is green.
|
||||
|
||||
## The loop
|
||||
|
||||
A single long-running background agent works `plans/js-on-sx.md` forever, one feature per commit. It runs a prioritized queue driven by the real test262 scoreboard. The briefing lives at `plans/agent-briefings/loop.md`; the recovery helper is `plans/restore-loop.sh`.
|
||||
|
||||
The queue (condensed — full version in the briefing):
|
||||
|
||||
1. Baseline commit (stage what's on disk now).
|
||||
2. Fix `lib/js/test262-runner.py` so it produces a real scoreboard.
|
||||
3. Full scoreboard run across the whole `test/` tree.
|
||||
4. Regex lexer/parser/runtime stub + Blockers entry listing platform primitives needed.
|
||||
5. Scoreboard-driven: pick the worst-passing category each iteration; fix; re-score.
|
||||
6. When the scoreboard plateaus, tackle deferred items (ASI, CEK-suspend await).
|
||||
|
||||
### Crash recovery
|
||||
|
||||
The agent process dies with the machine. **Work survives if it was committed** — that's why every iteration ends in a commit. After a crash:
|
||||
|
||||
```bash
|
||||
bash plans/restore-loop.sh # show state: tests, commits, scoreboard, regex hook
|
||||
bash plans/restore-loop.sh --print # also cat the briefing for easy copy-paste
|
||||
```
|
||||
|
||||
Then respawn the agent by pasting `plans/agent-briefings/loop.md` into Claude Code via the `Agent` tool with `run_in_background=true`. The agent re-reads the plan, picks up wherever the queue has got to, and carries on.
|
||||
|
||||
## Architecture sketch
|
||||
|
||||
```
|
||||
JS source text
|
||||
│
|
||||
▼
|
||||
lib/js/lexer.sx — tokens: {:type :value :pos}
|
||||
│
|
||||
▼
|
||||
lib/js/parser.sx — JS AST as SX trees, e.g. (js-binop "+" left right)
|
||||
│
|
||||
▼
|
||||
lib/js/transpile.sx — JS AST → SX AST (reusable by CEK)
|
||||
│ e.g. (js-binop "+" l r) → (+ (transpile l) (transpile r))
|
||||
▼
|
||||
existing CEK / VM — evaluation, async/IO, exceptions already handled
|
||||
```
|
||||
|
||||
Runtime shims in `lib/js/runtime.sx`: `js-global`, `js-console-log`, `js-typeof`, coercion helpers (`to-number`, `to-string`, `to-boolean`, abstract-equality), eventually the prototype chain.
|
||||
|
||||
## Roadmap
|
||||
|
||||
Each item: implement → tests → update progress. Mark `[x]` when tests green.
|
||||
|
||||
### Phase 1 — Lexer
|
||||
- [x] Numeric literals (int, float, hex, exponent)
|
||||
- [x] String literals (double, single, escape sequences, template strings later)
|
||||
- [x] Identifiers + reserved words
|
||||
- [x] Punctuation: `( ) { } [ ] , ; : . ...`
|
||||
- [x] Operators: `+ - * / % ** = == === != !== < > <= >= && || ! ?? ?: & | ^ ~ << >> >>> += -= ...`
|
||||
- [x] Comments (`//`, `/* */`)
|
||||
- [ ] Automatic Semicolon Insertion (defer — initially require semicolons)
|
||||
|
||||
### Phase 2 — Expression parser (Pratt-style)
|
||||
- [x] Literals → AST nodes
|
||||
- [x] Binary operators with precedence
|
||||
- [x] Unary operators (`- + ! ~ typeof void`)
|
||||
- [x] Member access (`.`, `[]`)
|
||||
- [x] Function calls
|
||||
- [x] Array literals
|
||||
- [x] Object literals (string/ident keys, shorthand later)
|
||||
- [x] Conditional `a ? b : c`
|
||||
- [x] Arrow functions (expression body only)
|
||||
|
||||
### Phase 3 — Transpile to SX AST
|
||||
- [x] Numeric/string/bool/null literals → SX literals
|
||||
- [x] Arithmetic: `+ - * / % **` with numeric coercion (`js-add` does string-concat dispatch)
|
||||
- [x] Comparisons: `=== !== == != < > <= >=`
|
||||
- [x] Logical: `&& ||` (short-circuit, value-returning via thunk trick); `??` nullish coalesce
|
||||
- [x] Unary: `- + ! ~ typeof void`
|
||||
- [x] Member access → `js-get-prop` (handles dicts, lists `.length`/index, strings)
|
||||
- [x] Array literal → `(list ...)`
|
||||
- [x] Object literal → `(dict)` + `dict-set!` sequence
|
||||
- [x] Function call → SX call (callee can be ident, member, arrow)
|
||||
- [x] Arrow fn → `(fn ...)` (params become SX symbols; closures inherited)
|
||||
- [x] Assignment (`=` and compound `+= -= ...`) on ident/member/index targets
|
||||
- [x] `js-eval` end-to-end: source → tokens → AST → SX → `eval-expr`
|
||||
|
||||
### Phase 4 — Runtime shims (`lib/js/runtime.sx`)
|
||||
- [x] `js-typeof`, `js-to-number`, `js-to-string`, `js-to-boolean`
|
||||
- [x] Abstract equality (`js-loose-eq`) vs strict (`js-strict-eq`)
|
||||
- [x] Arithmetic (`js-add` `js-sub` `js-mul` `js-div` `js-mod` `js-pow` `js-neg` `js-pos`)
|
||||
- [x] Logical (`js-and` `js-or` via thunks for lazy rhs) and `js-not` / `js-bitnot`
|
||||
- [x] Relational (`js-lt` `js-gt` `js-le` `js-ge`) incl. lexicographic strings
|
||||
- [x] `js-get-prop` / `js-set-prop` (dict/list/string; `.length`; numeric index)
|
||||
- [x] `console.log` → `log-info` bridge (`console` dict wired)
|
||||
- [x] `Math` object shim (`abs` `floor` `ceil` `round` `max` `min` `random` `PI` `E`)
|
||||
- [x] `js-undefined` sentinel (keyword) distinct from `nil` (JS `null`)
|
||||
- [x] Number parser `js-num-from-string` (handles int/float/±sign, no NaN yet)
|
||||
|
||||
### Phase 5 — Conformance harness
|
||||
- [x] Cherry-picked slice vendored at `lib/js/test262-slice/` (69 fixtures across 7 categories)
|
||||
- [x] Harness `lib/js/conformance.sh` — batch-load kernel, one epoch per fixture, substring-compare
|
||||
- [x] Initial target: ≥50% pass. **Actual: 69/69 (100%).**
|
||||
|
||||
### Phase 6 — Statements
|
||||
- [x] `var`/`let`/`const` declarations (all behave as `define` — block scope via SX lexical semantics)
|
||||
- [x] `if`/`else`
|
||||
- [x] `while`, `do..while`
|
||||
- [x] `for (init; cond; step)`
|
||||
- [x] `return`, `break`, `continue` (via `call/cc` continuation bindings `__return__` / `__break__` / `__continue__`)
|
||||
- [x] Block scoping (via `begin` — lexical scope inherited, no TDZ)
|
||||
|
||||
### Phase 7 — Functions & scoping
|
||||
- [x] `function` declarations (with call/cc-wrapped bodies for `return`)
|
||||
- [x] Function expressions (named + anonymous)
|
||||
- [x] Hoisting — function decls hoisted to enclosing scope (scan body first, emit defines ahead of statements)
|
||||
- [x] Closures — work via SX `fn` env capture
|
||||
- [x] Rest params (`...rest` → `&rest`)
|
||||
- [x] Default parameters (desugar to `if (param === undefined) param = default`)
|
||||
- [ ] `var` hoisting (deferred — treated as `let` for now)
|
||||
- [ ] `let`/`const` TDZ (deferred)
|
||||
|
||||
### Phase 8 — Objects, prototypes, `this`
|
||||
- [x] Property descriptors (simplified — plain-dict `__proto__` chain, `js-set-prop` mutates)
|
||||
- [x] Prototype chain lookup (`js-dict-get-walk` walks `__proto__`)
|
||||
- [x] `this` binding rules: method call, function call (undefined), arrow (lexical)
|
||||
- [x] `new` + constructor semantics (fresh dict, proto linked, ctor called with `this`)
|
||||
- [x] ES6 classes (sugar over prototypes, incl. `extends`)
|
||||
- [x] Array mutation: `a[i] = v`, `a.push(...)` — via `set-nth!` / `append!`
|
||||
- [x] Array builtins: push, pop, shift, slice, indexOf, join, concat, map, filter, forEach, reduce
|
||||
- [x] String builtins: charAt, charCodeAt, indexOf, slice, substring, toUpperCase, toLowerCase, split, concat
|
||||
- [x] `instanceof` and `in` operators
|
||||
|
||||
### Phase 9 — Async & Promises
|
||||
- [x] Promise constructor + `.then` / `.catch` / `.finally`
|
||||
- [x] `Promise.resolve` / `Promise.reject` / `Promise.all` / `Promise.race`
|
||||
- [x] `async` functions (decl, expr, arrow) return Promises
|
||||
- [x] `await` — synchronous-ish: drains microtasks, unwraps settled Promise
|
||||
- [x] Microtask queue with FIFO drain (`__drain()` exposed to JS)
|
||||
- [ ] True CEK suspension on `await` for pending Promises (deferred — needs cek-step-loop plumbing)
|
||||
|
||||
### Phase 10 — Error handling
|
||||
- [x] `throw` statement → `(raise v)`
|
||||
- [x] `try`/`catch`/`finally` (desugars to `guard` + optional finally wrapper)
|
||||
- [x] Error hierarchy (`Error`, `TypeError`, `RangeError`, `SyntaxError`, `ReferenceError` as constructor shims)
|
||||
|
||||
### Phase 11 — Stretch / deferred
|
||||
- ASI, regex literals, generators, iterators, destructuring, template strings with `${}`, tagged templates, Symbol, Proxy, typed arrays, ESM modules.
|
||||
|
||||
## Progress log
|
||||
|
||||
Append-only record of completed iterations. Loop writes one line per iteration: date, what was done, test count delta.
|
||||
|
||||
- 2026-04-23 — scaffold landed: lib/js/{lexer,parser,transpile,runtime}.sx stubs + test.sh. 7/7 smoke tests pass (js-tokenize/js-parse/js-transpile stubs + js-to-boolean coercion cases).
|
||||
- 2026-04-23 — Phase 1 (Lexer) complete: numbers (int/float/hex/exp/leading-dot), strings (escapes), idents/keywords, punctuation, all operators (1-4 char, longest-match), // and /* */ comments. 38/38 tests pass. Gotchas found: `peek` and `emit!` are primitives (shadowed to `js-peek`, `js-emit!`); `cond` clauses take ONE body only, multi-expr needs `(do ...)` wrapper.
|
||||
- 2026-04-23 — Phase 2 (Pratt expression parser) complete: literals, binary precedence (w/ `**` right-assoc), unary (`- + ! ~ typeof void`), member access (`.`/`[]`), call chains, array/object literals (ident+string+number keys), ternary, arrow fns (zero/one/many params; curried), assignment (right-assoc incl. compound `+=` etc.). AST node shapes all match the `js-*` names already wired. 47 new tests, 85/85 total. Most of the Phase 2 scaffolding was already written in an earlier session — this iteration verified every path, added the parser test suite, and greened everything on the first pass. No new gotchas beyond Phase 1.
|
||||
- 2026-04-23 — Phase 3 (Transpile to SX AST) complete: dispatch on AST head using `js-tag?`/`symbol-name` inspection, emit SX trees built from `list`/`cons`/`make-symbol`. All binops, unaries, member/index, call (arbitrary callee), array (`list`), object (`let` + `dict` + `dict-set!`), ternary (`if` around `js-to-boolean`), arrow (`fn` with `make-symbol` params), assignment (ident → `set!`; member/index → `js-set-prop`; compound → combines). Short-circuit `&&`/`||` built via thunk passed to `js-and`/`js-or` — this preserves JS value-returning semantics and avoids re-evaluating lhs. `??` uses `let`+`if`. `js-eval src` pipelines `js-parse-expr` → `js-transpile` → `eval-expr`. 69 new assertions for end-to-end eval; 154/154 main suite.
|
||||
- 2026-04-23 — Phase 4 (Runtime shims) complete: coercions (ToBoolean/ToNumber/ToString) including a tiny recursive string→number parser (handles ±int/float, no NaN yet); arithmetic with JS `+` dispatch (string-concat when either operand is string, else ToNumber); `js-strict-eq`/`js-loose-eq` with null↔undefined and boolean-coercion rules; relational with lexicographic string path via `char-code`; `js-get-prop`/`js-set-prop` covering dict/list/string with numeric index and `.length`; `Math` object, `console.log`, `js-undefined` sentinel. Several SX gotchas hit and noted below.
|
||||
- 2026-04-23 — Phase 5 (Conformance harness) complete: 69 hand-picked fixtures under `lib/js/test262-slice/` covering arithmetic, comparison, loose+strict equality, logical, nullish, conditional, arrays, objects, strings, arrows, Math, typeof. Runner `lib/js/conformance.sh` builds one batch script (single kernel boot), one epoch per fixture, substring-matches the sibling `.expected` file. Scoring: **69/69 (100%)** — well past the 50% target. Real test262 integration deferred to later phase (needs network + Python harness; deferred to Phase 5.5 per plan).
|
||||
- 2026-04-23 — Phases 6 + 7 (Statements + Functions) complete. Parser now accepts top-level programs — `js-parse` returns `(js-program (stmts...))`, with new node types `js-var / js-vardecl / js-block / js-if / js-while / js-do-while / js-for / js-return / js-break / js-continue / js-exprstmt / js-empty / js-funcdecl / js-funcexpr / js-param / js-rest`. Arrow bodies now accept `{...}` block form. Transpile dispatch extended with 14 new clauses. Loops built via `letrec` recursion; `break` / `continue` / `return` via lexical `call/cc` bindings (`__break__` / `__continue__` / `__return__`). Function declarations hoisted to the enclosing scope before other statements run (two-pass: `js-collect-funcdecls` scans, `js-transpile-stmt-list` replaces the hoisted entry with `nil`). Default params desugar to `(if (or (= x nil) (= x :js-undefined)) default x)` updates; rest params emit `&rest name`. Unit tests: **195/195** (154 → +41 for statements, functions, closures, loops). Conformance: **96/96** (69 → +27 new fixtures across `statements/`, `loops/`, `functions/`, `closures/`). Gotchas: (1) SX `do` is R7RS iteration, not sequence — must use `begin`. A `(do (x) ...)` where `x` is a list → "first: expected list, got N" because the do-form tries to parse its iteration bindings. (2) SX passes unsupplied fn params as `nil`, not an undefined sentinel — default-param init must test for both `nil` and `:js-undefined`. (3) `jp-collect-params` (used by arrow heads) doesn't understand rest/defaults; new `jp-parse-param-list` used for function declarations. Arrow rest/defaults deferred. (4) `...` lexes as `punct`, not `op`.
|
||||
|
||||
- 2026-04-23 — **Phase 9 (Async & Promises) complete.** New AST tags: `js-await`, `js-funcdecl-async`, `js-funcexpr-async`, `js-arrow-async`. Parser extended: `async` keyword consumed, dispatches by the next token (function/ident/paren). Primary parser grows a pre-function `async` case and a new `await` unary. Statement parser adds a two-token lookahead for `async function` decls. Runtime adds: microtask queue (`__js_microtask_queue__` dict cell + push/pop/empty/drain), `js-promise?` predicate, full `{:__js_promise__ true :state :value :callbacks}` object, `js-promise-resolve!`/`reject!`/`flush-callbacks!`, callback dispatch (`run-callback!` / `run-handler!` / `try-call` using `guard`), `.then` via `js-promise-then-internal!`, `.catch`/`.finally` derivative calls. `js-invoke-method` now routes Promise methods through `js-invoke-promise-method` (same single-dispatch no-closure pattern as Phase 8 list/string builtins). `Promise` constructor runs executor synchronously inside a guard so throws reject the Promise. Statics `resolve`/`reject`/`all`/`race` live in `__js_promise_statics__` dict; `js-get-prop` special-cases identity-equality against the `Promise` function. `js-async-wrap` wraps a thunk → Promise (fulfilled on return, rejected on throw, adopts returned Promises). `js-await-value` drains microtasks then unwraps a settled Promise or raises its reason; pending Promise = error (no scheduler — see Blockers). `js-eval` drains microtasks at end. `__drain()` exposed to JS so tests can force-run pending callbacks synchronously before reading a mutable result. Arity-tolerant call path `js-call-arity-tolerant` adapts 1-arg handler invocations to handlers declared with `()` (zero params) via `lambda-params` introspection. Unit tests: **254/254** (+31 parser + runtime). Conformance: **148/148** (+29: `test262-slice/promises/*` × 16, `test262-slice/async/*` × 13). Microtask ordering is FIFO (append on settle, drain one-at-a-time); spec-ish but not obsessive about thenable-adoption iteration count. Gotchas: (1) **`cond` needs `begin` for multi-body clauses** — same rule as Phase 1, bit me hard because the original draft had `(cond ((state) (side-effect) (side-effect2)))` which silently discarded the first expression as "predicate" and tried to call it as a function. (2) **`guard` with multi-body handler clauses** — same fix, `(guard (e (else (begin …))))`. (3) **`(= (type-of fn) "function")` is FALSE** — `type-of` returns `"lambda"` for user-defined fns; use `js-function?` which accepts lambda/function/component. (4) **Forward refs in SX work** because `define` is late-bound in the global env. (5) **Microtask semantics vs top-level last-expression** — `js-eval` evaluates all stmts THEN drains; if the last stmt reads `r` assigned in a `.then`, you'll see `nil` unless you insert `__drain()` between the setup and the read. (6) **`Promise.resolve(p)` returns p for existing Promises** — identity preserved via `(js-promise? v) → v` short-circuit. (7) **Strict arity in SX lambdas vs tolerant JS** — `() => side-effect()` in JS accepts extra args silently; SX `(fn () ...)` errors. Callback invocations go through `js-call-arity-tolerant` which introspects `lambda-params` and calls with no args if the handler has zero params.
|
||||
|
||||
- 2026-04-23 — Phases 8 + 10 (Objects + Errors) complete in a single session. **Object model:** regular JS `function` bodies wrap with `(let ((this (js-this))) ...)` — a dynamic `this` via a global cell `__js_this_cell__`. Method calls `obj.m(args)` route through `js-invoke-method` which saves/restores the cell around the call, so `this` works without an explicit first-arg calling convention. Arrow functions don't wrap — they inherit the enclosing lexical `this`. **`new`:** creates a fresh dict with `__proto__` linked to the constructor's prototype dict, calls the constructor with `this` bound, returns the ctor's dict return (if any) else the new object. **Prototype chain:** lives in a side table `__js_proto_table__` keyed by `inspect(ctor)`. `ctor.prototype` access and assignment both go through this table. `js-dict-get-walk` walks the `__proto__` chain on dict property lookup. **Classes:** desugar to `(define Name ctor)` + `(js-reset-ctor-proto! Name)` (critical for redefinition) + `(dict-set! (js-get-ctor-proto Name) mname mfn)` for each method. `extends` chains by setting `(js-get-ctor-proto Child).__proto__ = (js-get-ctor-proto Parent)`. Default ctor with `extends` calls parent with same args. **Arrays:** `js-set-prop` on lists dispatches to `js-list-set!` which does in-bounds `set-nth!` or `append!` past end (pads with `js-undefined`). No shrinking (primitive gap — `pop-last!` is a no-op). **Array + String builtins** are routed through `js-invoke-method` directly via `js-invoke-list-method` / `js-invoke-string-method` to AVOID a VM JIT bug: returning a closure from a JIT-compiled function (which happened when `js-array-method` returned an inner `fn`) crashed with "VM undefined: else". Dispatching without closures works. **Throw/try/catch/finally:** `throw v` → `(raise v)`; try/catch → `(guard (e (else cbody)) body)`; finally wraps via `(let ((r try-tr)) finally-tr r)`. **Error hierarchy:** `Error`/`TypeError`/`RangeError`/`SyntaxError`/`ReferenceError` are constructor shims that set `this.message` + `this.name` on the new object. **`instanceof` + `in`:** parser precedence table extended to accept both as keywords at prec 10; binary-loop predicate extended to allow keyword-type tokens for these two. Unit tests: **223/223** (+28). Conformance: **119/119** (+23 new fixtures across `objects/` and `errors/`). Gotchas: (1) **Ctor-id collision on redefine** — `inspect` of a lambda is keyed by (name + arity), so redefining `class B` found the OLD proto-table entry. Fix: class decl always calls `js-reset-ctor-proto!`. (2) **VM closure bug** — functions returning inner closures from JIT-compiled bodies break: `(fn (arr) (fn (f) ...use arr...))` compiles to a VM closure for the outer that can't produce a working inner. Workaround: route all builtin method dispatch through a single (non-closure-returning) helper. (3) **`jp-parse-param-list` eats its own `(`** — don't prefix with `jp-expect! st "punct" "("`, the parser handles both. Class method parser hit this.
|
||||
|
||||
## Phase 3-5 gotchas
|
||||
|
||||
Worth remembering for later phases:
|
||||
|
||||
- **Varargs in SX** use `&rest args`, not bare `args`. `(fn args …)` errors with "Expected list, got symbol" — use `(fn (&rest args) …)`.
|
||||
- **`make-symbol`** is the way to build an SX identifier-symbol at runtime for later `eval-expr`. Use it to turn JS idents into SX variable references.
|
||||
- **`eval-expr`** is the primitive you want to evaluate an SX list you've just constructed. It's already in the kernel primitives.
|
||||
- **Keywords print as quoted strings** via the epoch `eval` command — `:foo` comes back as `"foo"`. Affected the `js-undefined` test expectation.
|
||||
- **`char-code` (not `code-char`)** for char→codepoint. No `bit-not` primitive — implement `~x` as `-(int(x)+1)`.
|
||||
- **Epoch `eval` string must double-escape** to survive two nested SX string literals (`(eval "(js-eval \"…\")")`). The conformance harness uses a small Python helper to do that reliably.
|
||||
- **Shell heredocs and `||`** — writing fixtures with `||` in a here-doc fed to a `while read` loop gets interpreted as a pipe mid-parse. Use `case|a\|\|b|…` style or hand-write the `||` cases.
|
||||
|
||||
## Blockers / shared-file issues
|
||||
|
||||
Anything that would require a change outside `lib/js/` goes here with a minimal repro. Don't fix from the loop.
|
||||
|
||||
- **Pending-Promise await** — our `js-await-value` drains microtasks and unwraps *settled* Promises; it cannot truly suspend a JS fiber and resume later. Every Promise that settles eventually through the synchronous `resolve`/`reject` + microtask path works. A Promise that never settles without external input (e.g. a real `setTimeout` waiting on the event loop) would hit the `"await on pending Promise (no scheduler)"` error. Proper async suspension would need the JS eval path to run under `cek-step-loop` (not `eval-expr` → `cek-run`) and treat `await pending-Promise` as a `perform` that registers a resume thunk on the Promise's callback list. Non-trivial plumbing; out of scope for this phase. Consider it a Phase 9.5 item.
|
||||
|
||||
## First-iteration checklist (scaffolding) — DONE
|
||||
|
||||
- [x] `lib/js/lexer.sx` — stub `js-tokenize`
|
||||
- [x] `lib/js/parser.sx` — stub `js-parse`
|
||||
- [x] `lib/js/transpile.sx` — stub `js-transpile`
|
||||
- [x] `lib/js/runtime.sx` — stub `js-global`, `js-to-boolean`
|
||||
- [x] `lib/js/test.sh` — epoch-protocol runner mirroring `lib/hyperscript/test.sh`
|
||||
- [x] Smoke suite green (7/7)
|
||||
|
||||
Next iteration: Phase 1 — lexer. Start with numeric literals + identifiers + whitespace skipping; extend test.sh with tokenization assertions.
|
||||
65
plans/restore-loop.sh
Executable file
65
plans/restore-loop.sh
Executable file
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bash
|
||||
# restore-loop.sh — print recovery state for the js-on-sx loop.
|
||||
#
|
||||
# Claude Code can't be driven from shell, so this script shows you where things
|
||||
# stand and points at the briefing file. To respawn the agent, feed
|
||||
# plans/agent-briefings/loop.md into Claude via the Agent tool with
|
||||
# run_in_background=true.
|
||||
#
|
||||
# Usage:
|
||||
# bash plans/restore-loop.sh # status snapshot
|
||||
# bash plans/restore-loop.sh --print # also cat the briefing
|
||||
#
|
||||
set -uo pipefail
|
||||
|
||||
cd "$(dirname "$0")/.."
|
||||
|
||||
echo "=== js-on-sx loop state ==="
|
||||
echo
|
||||
echo "Branch: $(git rev-parse --abbrev-ref HEAD)"
|
||||
echo "HEAD: $(git log -1 --oneline)"
|
||||
echo
|
||||
echo "Recent commits on lib/js/ and plans/:"
|
||||
git log -10 --oneline -- lib/js/ plans/js-on-sx.md 2>/dev/null || echo " (none yet)"
|
||||
echo
|
||||
echo "=== Test baseline ==="
|
||||
if [ -x hosts/ocaml/_build/default/bin/sx_server.exe ]; then
|
||||
echo -n "Unit tests: "; bash lib/js/test.sh 2>&1 | tail -1 || true
|
||||
echo -n "Conformance: "; bash lib/js/conformance.sh 2>&1 | tail -1 || true
|
||||
else
|
||||
echo "sx_server.exe not built — run ./scripts/sx-build-all.sh first."
|
||||
fi
|
||||
echo
|
||||
echo "=== test262 scoreboard ==="
|
||||
if [ -f lib/js/test262-scoreboard.json ]; then
|
||||
echo " ✓ lib/js/test262-scoreboard.json exists"
|
||||
python3 -c "import json;d=json.load(open('lib/js/test262-scoreboard.json'));t=d.get('totals', d.get('overall', {}));print(f\" totals: {t}\")" 2>/dev/null || true
|
||||
else
|
||||
echo " ✗ lib/js/test262-scoreboard.json NOT found"
|
||||
fi
|
||||
echo
|
||||
echo "=== regex platform hook ==="
|
||||
if grep -q js-regex-platform-override lib/js/runtime.sx 2>/dev/null; then
|
||||
echo " ✓ js-regex-platform-override! wired in runtime.sx"
|
||||
else
|
||||
echo " ✗ js-regex-platform-override! not yet wired"
|
||||
fi
|
||||
echo
|
||||
echo "=== Briefing ==="
|
||||
for f in plans/agent-briefings/*.md; do
|
||||
[ -f "$f" ] && echo " $f"
|
||||
done
|
||||
echo
|
||||
echo "To respawn: paste plans/agent-briefings/loop.md into Claude via the Agent tool"
|
||||
echo " with run_in_background=true. The agent reads the plan's progress"
|
||||
echo " log and picks up wherever the queue left off."
|
||||
|
||||
if [ "${1:-}" = "--print" ]; then
|
||||
echo
|
||||
echo "=== Briefing contents ==="
|
||||
for f in plans/agent-briefings/*.md; do
|
||||
echo
|
||||
echo "---- $f ----"
|
||||
cat "$f"
|
||||
done
|
||||
fi
|
||||
Reference in New Issue
Block a user