js-on-sx: regex literal lex+parse+transpile+runtime stub
Lexer: js-regex-context? disambiguates / based on prior token;
read-regex handles [...] classes and \ escapes. Emits
{:type "regex" :value {:pattern :flags}}.
Parser: new primary branch → (js-regex pat flags).
Transpile: (js-regex-new pat flags).
Runtime: js-regex? predicate, js-regex-new builds tagged dict with
source/flags/global/ignoreCase/multiline/sticky/unicode/dotAll/
hasIndices/lastIndex. js-regex-invoke-method dispatches .test/.exec/
.toString. js-invoke-method detects regex receivers. Stub engine
uses js-string-index-of; __js_regex_platform__ + override! let a
real engine plug in later.
Runner: repeatable --filter flags (OR'd).
308/310 unit (+30 regex tests), 148/148 slice unchanged.
This commit is contained in:
@@ -173,6 +173,8 @@ Append-only record of completed iterations. Loop writes one line per iteration:
|
||||
|
||||
- 2026-04-23 — **Queue item 2: fixed test262 runner.** Root-cause of 7/8 timeouts: runner re-parsed the entire 197-line `assert.js` for every test in one big `js-eval` (8.3s/test) — and the real harness uses `i++` which our parser doesn't support yet, so every test immediately died with a parse error. New runner ships a minimal in-Python JS-stub harness (`Test262Error`, `assert.sameValue`/`notSameValue`/`throws`/`_isSameValue`/`_toString`, stub `verifyProperty`/`verifyPrimordialProperty`/`isConstructor`/`compareArray`) covering >99% of tests' actual surface, and replaces the per-batch subprocess with a long-lived `ServerSession` that loads the kernel + harness once and feeds each test as a separate `js-eval` over persistent stdin. Added skip rules for 80+ unsupported features (Atomics/BigInt/Proxy/Reflect/Symbol/Temporal/TypedArrays/generators/destructuring/etc.) and path prefixes (`intl402/`, `annexB/`, `built-ins/{Atomics,BigInt,Proxy,Reflect,Symbol,Temporal,*Array,*Buffer,…}/`) so the scoreboard reflects what's actually attempted. Scoreboard over 288 runnable Math tests: **56/288 (19.4%)** in 185s, rate ≈ 2.3 tests/s (prev: 0/8 with 7 timeouts). Top failure modes: 83× assertion-fail (real semantic gaps in Math.floor/ceil/trunc/etc. details), 62× ReferenceError (builtins we haven't shimmed, e.g. `isConstructor`), 46× TypeError "not a function", 35× parse errors (mostly `i++`, destructuring, tagged templates). 278/280 unit + 148/148 slice unchanged.
|
||||
|
||||
- 2026-04-23 — **Regex literal support (lex+parse+transpile+runtime stub).** Runner now accepts repeatable `--filter` flags (OR'd). Lexer gains `js-regex-context?` (returns true at SOF or when last token is op/non-closing-punct/regex-keyword incl. return/typeof/in/of/throw/new/delete/instanceof/void/yield/await/case/do/else) and `read-regex` (handles `\` escapes and `[...]` classes, collects flags as ident chars). `scan!` intercepts `/` ahead of the operator-match tries when in a regex context and emits `{:type "regex" :value {:pattern :flags}}`. Parser adds a `regex` primary branch → `(js-regex pat flags)`. Transpile emits `(js-regex-new pat flags)`. Runtime adds: `js-regex?` predicate (dict + `__js_regex__` key), `js-regex-new` builds the tagged dict with `source / flags / global / ignoreCase / multiline / sticky / unicode / dotAll / hasIndices / lastIndex` populated; `js-regex-invoke-method` dispatches `.test` / `.exec` / `.toString`; `js-invoke-method` gets a regex branch before the generic method-lookup fallback. Stub engine (`js-regex-stub-test` / `-exec`) uses `js-string-index-of` — not a real regex, but enough to make `/foo/.test('hi foo')` work. `__js_regex_platform__` dict + `js-regex-platform-override!` let a real platform primitive be swapped in later without runtime changes. 30 new unit tests (17 lex + 3 parse + 1 transpile + 4 obj-shape + 4 prop + 2 test()): **308/310** (278→+30). Conformance unchanged. Gotcha: `contains?` with 2 args expects `(contains? list x)`, NOT a dict — use `(contains? (keys d) k)` or `dict-has?`. First pass forgot that and cascaded errors across Math / class tests via the `js-regex?` predicate inside `js-invoke-method`. Wide scoreboard run across 9 targeted categories launched in background.
|
||||
|
||||
## Phase 3-5 gotchas
|
||||
|
||||
Worth remembering for later phases:
|
||||
@@ -191,6 +193,18 @@ Anything that would require a change outside `lib/js/` goes here with a minimal
|
||||
|
||||
- **Pending-Promise await** — our `js-await-value` drains microtasks and unwraps *settled* Promises; it cannot truly suspend a JS fiber and resume later. Every Promise that settles eventually through the synchronous `resolve`/`reject` + microtask path works. A Promise that never settles without external input (e.g. a real `setTimeout` waiting on the event loop) would hit the `"await on pending Promise (no scheduler)"` error. Proper async suspension would need the JS eval path to run under `cek-step-loop` (not `eval-expr` → `cek-run`) and treat `await pending-Promise` as a `perform` that registers a resume thunk on the Promise's callback list. Non-trivial plumbing; out of scope for this phase. Consider it a Phase 9.5 item.
|
||||
|
||||
- **Regex platform primitives** — runtime ships a substring-based stub (`js-regex-stub-test` / `-exec`). Overridable via `js-regex-platform-override!` so a real engine can be dropped in. Required platform-primitive surface:
|
||||
- `regex-compile pattern flags` — build an opaque compiled handle
|
||||
- `regex-test compiled s` → bool
|
||||
- `regex-exec compiled s` → match dict `{match index input groups}` or nil
|
||||
- `regex-match-all compiled s` → list of match dicts (or empty list)
|
||||
- `regex-replace compiled s replacement` → string
|
||||
- `regex-replace-fn compiled s fn` → string (fn receives match+groups, returns string)
|
||||
- `regex-split compiled s` → list of strings
|
||||
- `regex-source compiled` → string
|
||||
- `regex-flags compiled` → string
|
||||
Ideally a single `(js-regex-platform-install-all! platform)` entry point the host calls once at boot. OCaml would wrap `Str` / `Re` or a dedicated regex lib; JS host can just delegate to the native `RegExp`.
|
||||
|
||||
## First-iteration checklist (scaffolding) — DONE
|
||||
|
||||
- [x] `lib/js/lexer.sx` — stub `js-tokenize`
|
||||
|
||||
Reference in New Issue
Block a user