Adds Sx_vm.bytecode_uses_extension_opcodes — an operand-aware
bytecode scanner that walks past CONST u16, CALL_PRIM u16+u8, and
CLOSURE u16+dynamic upvalue descriptors so operand bytes that happen
to be ≥200 don't false-positive as extension opcodes.
jit_compile_lambda calls the scanner on the inner closure's bytecode.
On hit it returns None — the lambda then runs through CEK
interpretation. The VM's dispatch fallthrough still routes the
extension opcodes themselves through the registry; this change just
prevents the JIT from claiming code it has no plan for.
Tests: 7 new foundation cases — pure core eligible, head/middle/
post-CLOSURE detection, CONST + CALL_PRIM + CLOSURE-descriptor false-
positive avoidance. +7 pass vs Phase D baseline, no regressions
across 11 conformance suites.
Loop complete: acceptance criteria 1-4 met. Hand-off to the Erlang
loop — lib/erlang/vm/dispatcher.sx's Phase 9b stub can now be
replaced with a real hosts/ocaml/lib/extensions/erlang.ml consumer.
lib/extensions/ becomes the new home for VM extensions, wired in via
(include_subdirs unqualified). README documents the registration
pattern, opcode-ID range conventions (200-209 guest_vm, 210-219
inline test, 220-229 test_ext, 230-247 ports), and naming rules.
extensions/test_ext.ml is the canonical worked example — two
operand-less opcodes (220 push 42, 221 double TOS) carrying a per-
extension state slot (TestExtState invocation counter). Test_ext.register
called from run_tests.ml at the start of the Phase D suite, on top of
the inline test_reg from earlier suites (disjoint opcode IDs).
Sx_vm.opcode_name now consults extension_opcode_name_ref (forward ref
in the same style as extension_dispatch_ref), so disassemble shows
extension opcodes by name instead of UNKNOWN_n. Registry maintains
name_of_id_table and installs the lookup at module init.
Tests: 5 new foundation cases — primitive resolves test_ext name,
end-to-end bytecode (push + double + return → 84), disassemble shows
"test_ext.OP_TEST_PUSH_42" / "test_ext.OP_TEST_DOUBLE_TOS",
unregistered ext opcodes still fall back to UNKNOWN_n, invocation
counter records the two dispatches. +5 pass vs Phase C baseline, no
regressions across 11 conformance suites.
Registers extension-opcode-id from sx_vm_extensions.ml module init.
Lives downstream of both sx_primitives and sx_vm to avoid a build
cycle. Accepts a string or symbol; returns Integer id when the opcode
is registered, Nil otherwise.
Compilers (lib/compiler.sx) call this to emit extension opcodes by
name. Returning Nil rather than failing on unknown names lets a port's
optimization opt in per-build — missing extensions degrade to slower
correct execution.
Tests: 5 new foundation cases — registered lookup, unknown → nil,
symbol arg, zero-arg + integer-arg rejection. +5 pass vs Phase B
baseline, no regressions across 11 conformance suites.
sx_vm_extension.ml: handler type, extensible extension_state variant,
EXTENSION first-class module signature.
sx_vm_extensions.ml: register / dispatch / id_of_name /
state_of_extension. install_dispatch () runs at module init,
swapping Phase A's stub for the real registry. Rejects out-of-range
opcode IDs (must be 200-247), duplicate IDs, duplicate names, and
duplicate extension names.
Tests: 9 new foundation cases — lookup hits/misses, end-to-end VM
dispatch including opcode composition, all four rejection paths.
+9 pass vs Phase A baseline, no regressions across 11 conformance
suites.
Adds Invalid_opcode of int exception and extension_dispatch_ref forward
ref (default raises Invalid_opcode op), plus the |op when op >= 200 arm
before the catch-all in the bytecode dispatch loop. Partition comment
documents 1-199 core / 200-247 extensions / 248-255 reserved.
Phase B will install the real registry's dispatch into the ref at module
init, replacing this stub.
Tests: 4 new foundation cases (Invalid_opcode for 200/224/247, Eval_error
for 199 to pin the threshold). +4 pass vs baseline, no regressions.
5 phases (A-E) per plans/sx-vm-opcode-extension.md:
- A: Sx_vm dispatch fallthrough for opcodes ≥200 + Invalid_opcode + extension_dispatch_ref
- B: Sx_vm_extension interface + Sx_vm_extensions registry (register / dispatch /
id_of_name / state_of_extension), installs into the dispatch_ref at module init
- C: extension-opcode-id SX primitive for compiler-side lookup
- D: lib/extensions/ subtree wired via include_subdirs, test_ext.ml as the canonical
worked example, opcode_name forward-ref so disassemble shows ext opcodes by name
- E: bytecode_uses_extension_opcodes scanner + JIT skip path so lambdas containing
extension opcodes run interpreted via CEK
26 new foundation tests across 5 suites, all green. Zero regressions across 11
language-port conformance suites (erlang 530, haskell 285, datalog 276, prolog 590,
smalltalk 847, common-lisp 487, apl 562, js 148, forth 632, tcl 3, ocaml-on-sx unit 607).
Hand-off: lib/erlang/vm/dispatcher.sx (Phase 9b stub) can now be replaced with a real
hosts/ocaml/lib/extensions/erlang.ml consumer.
Adds Sx_vm.bytecode_uses_extension_opcodes — an operand-aware
bytecode scanner that walks past CONST u16, CALL_PRIM u16+u8, and
CLOSURE u16+dynamic upvalue descriptors so operand bytes that happen
to be ≥200 don't false-positive as extension opcodes.
jit_compile_lambda calls the scanner on the inner closure's bytecode.
On hit it returns None — the lambda then runs through CEK
interpretation. The VM's dispatch fallthrough still routes the
extension opcodes themselves through the registry; this change just
prevents the JIT from claiming code it has no plan for.
Tests: 7 new foundation cases — pure core eligible, head/middle/
post-CLOSURE detection, CONST + CALL_PRIM + CLOSURE-descriptor false-
positive avoidance. +7 pass vs Phase D baseline, no regressions
across 11 conformance suites.
Loop complete: acceptance criteria 1-4 met. Hand-off to the Erlang
loop — lib/erlang/vm/dispatcher.sx's Phase 9b stub can now be
replaced with a real hosts/ocaml/lib/extensions/erlang.ml consumer.
lib/extensions/ becomes the new home for VM extensions, wired in via
(include_subdirs unqualified). README documents the registration
pattern, opcode-ID range conventions (200-209 guest_vm, 210-219
inline test, 220-229 test_ext, 230-247 ports), and naming rules.
extensions/test_ext.ml is the canonical worked example — two
operand-less opcodes (220 push 42, 221 double TOS) carrying a per-
extension state slot (TestExtState invocation counter). Test_ext.register
called from run_tests.ml at the start of the Phase D suite, on top of
the inline test_reg from earlier suites (disjoint opcode IDs).
Sx_vm.opcode_name now consults extension_opcode_name_ref (forward ref
in the same style as extension_dispatch_ref), so disassemble shows
extension opcodes by name instead of UNKNOWN_n. Registry maintains
name_of_id_table and installs the lookup at module init.
Tests: 5 new foundation cases — primitive resolves test_ext name,
end-to-end bytecode (push + double + return → 84), disassemble shows
"test_ext.OP_TEST_PUSH_42" / "test_ext.OP_TEST_DOUBLE_TOS",
unregistered ext opcodes still fall back to UNKNOWN_n, invocation
counter records the two dispatches. +5 pass vs Phase C baseline, no
regressions across 11 conformance suites.
Registers extension-opcode-id from sx_vm_extensions.ml module init.
Lives downstream of both sx_primitives and sx_vm to avoid a build
cycle. Accepts a string or symbol; returns Integer id when the opcode
is registered, Nil otherwise.
Compilers (lib/compiler.sx) call this to emit extension opcodes by
name. Returning Nil rather than failing on unknown names lets a port's
optimization opt in per-build — missing extensions degrade to slower
correct execution.
Tests: 5 new foundation cases — registered lookup, unknown → nil,
symbol arg, zero-arg + integer-arg rejection. +5 pass vs Phase B
baseline, no regressions across 11 conformance suites.
sx_vm_extension.ml: handler type, extensible extension_state variant,
EXTENSION first-class module signature.
sx_vm_extensions.ml: register / dispatch / id_of_name /
state_of_extension. install_dispatch () runs at module init,
swapping Phase A's stub for the real registry. Rejects out-of-range
opcode IDs (must be 200-247), duplicate IDs, duplicate names, and
duplicate extension names.
Tests: 9 new foundation cases — lookup hits/misses, end-to-end VM
dispatch including opcode composition, all four rejection paths.
+9 pass vs Phase A baseline, no regressions across 11 conformance
suites.
Adds Invalid_opcode of int exception and extension_dispatch_ref forward
ref (default raises Invalid_opcode op), plus the |op when op >= 200 arm
before the catch-all in the bytecode dispatch loop. Partition comment
documents 1-199 core / 200-247 extensions / 248-255 reserved.
Phase B will install the real registry's dispatch into the ref at module
init, replacing this stub.
Tests: 4 new foundation cases (Invalid_opcode for 200/224/247, Eval_error
for 199 to pin the threshold). +4 pass vs baseline, no regressions.
plans/sx-vm-opcode-extension.md ports over from loops/erlang (f6a68656)
with the opcode partition adjusted to match real VM usage: 1-199 core
(current ceiling 175 = OP_DEC), 200-247 extensions, 248-255 reserved.
plans/agent-briefings/sx-vm-extensions-loop.md captures the per-fire
workflow and ground rules.
Lua now joins tcl/ocaml/kernel/common-lisp in consuming lib/guest/lex.sx via
prefix-rename. Removes 28 lines of duplicated character-class helpers
(lua-make-token, lua-digit?, lua-hex-digit?, lua-letter?, lua-ident-start?,
lua-ident-char?, lua-ws?) and replaces with the 8-line prefix-rename block.
The byte-table additions from loops/lua (__ascii-tok, __lua-127-255-tok,
lua-byte-to-char) are preserved at the top of tokenizer.sx — those provide
Lua's 8-bit-clean string semantics on top of the shared lex layer.
test.sh updated to preload lib/guest/lex.sx + lib/guest/prefix.sx before
lua sources, matching the load order arch's pre-merge test.sh used.
393/395 maintained. The 2 pre-existing failures are unrelated:
- math.random(n) primitive arity issue
- os.clock returns rational instead of number (SX division semantics)
Skipped from the planned follow-up: delay/force port. Arch's lua-force was
defined but never referenced anywhere — dead code, not worth porting.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Updates Phase 7 status:
- env.sx ✓ extracted (three live consumers: Kernel, Tcl, Smalltalk,
with Scheme also using it directly)
- class-chain.sx ✓ extracted (bonus — not on the original six-file
list but surfaced by the same chiselling discipline; Smalltalk +
CLOS consumers)
- quoting.sx ✓ extracted (Kernel + Scheme consumers)
- evaluator.sx DECLINED — too thin to be its own kit; the shared
content is protocol/API surface, not algorithm. Documented
in-plan, no file created.
- combiner.sx, short-circuit.sx — still need fexpr-having
second consumers
- hygiene.sx — still awaits Scheme Phase 6c (research-grade
scope-set work)
Three kits live, one declined, three still gated.
lib/guest/reflective/quoting.sx — quasiquote walker with adapter cfg.
Three forms:
- refl-quasi-walk-with CFG FORM ENV (top-level)
- refl-quasi-walk-list-with CFG FORMS ENV (list walker, splice-aware)
- refl-quasi-list-concat XS YS (pure-SX helper)
Adapter cfg keys:
- :unquote-name — string keyword ("$unquote" or "unquote")
- :unquote-splicing-name — string keyword
- :eval — fn (form env) → value
The shared algorithm is identical in Kernel and Scheme; the only
divergences are the keyword names (`$unquote` vs `unquote`) and
which host evaluator runs at unquote points (`kernel-eval` vs
`scheme-eval`). Both surface through the cfg.
Migrations:
- lib/kernel/runtime.sx: knl-quasi-walk reduces to a 3-line wrapper
that builds knl-quasi-cfg and delegates. Removed knl-quasi-walk-
list + knl-list-concat (~40 LoC) — now provided by the kit.
- lib/scheme/eval.sx: scm-quasi-walk reduces to a 3-line wrapper
around scm-quasi-cfg. Removed scm-quasi-walk-list + scm-list-
concat. scm-collect-exports (module impl) was a hidden consumer
of scm-list-concat — rewired to refl-quasi-list-concat.
lib/scheme/test.sh — loads lib/guest/reflective/quoting.sx before
lib/scheme/parser.sx so the kit is available when eval.sx loads.
Both consumers' tests green:
- Kernel: 322 tests across 7 suites
- Scheme: 296 tests across 9 suites
**Second reflective-kit extraction landed.** The kit-extraction
playbook from env.sx and class-chain.sx — adapter-cfg pattern from
lib/guest/match.sx, same algorithm bridges different keyword names —
works again on a third structurally different problem (quasiquote
walking). The cumulative extraction story: env.sx → class-chain.sx
→ quoting.sx, three independent kits, all using the same pattern.
`evaluator.sx` (the other deferred candidate the Scheme port
unlocked) is NOT extracted — the genuinely shared content is too
thin (one helper for closure-capturing interaction-environment).
The eval-protocol is more about API surface than algorithm.
Documented as a non-extraction.
Two additions from loops/hs needed for the new WebSocket socket tests:
- unhandledRejection suppressor — synchronous test harness doesn't await RPC promises
- Fake setTimeout/clearTimeout + __hsFlushTimers — drain RPC timeout tests synchronously
Plan update: mark E36 WebSocket as DONE (previously "design-done, pending review").
Skipped: loops/hs's tests/playwright/generate-sx-tests.py — architecture's version
is 1468 lines vs loops/hs's 290; arch's is the further-evolved version.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Loop closer documenting what 10 feature commits landed across the
session. Phase-by-phase outcomes captured, including the SX cond
multi-expression bug found and fixed during Phase 4.
Chisel ledger:
- env.sx already EXTRACTED with Scheme as third consumer
- evaluator.sx + quoting.sx second-consumer-ready for follow-on
kit-extraction commits
- hygiene.sx still awaits the deferred Phase 6c (scope-set work)
- combiner.sx and short-circuit.sx don't apply (Scheme has no
fexprs and uses syntactic and/or)
Deferred phases listed: full hygiene, nested quasi-depth, R7RS
module rich features, dotted-pair syntax, full call/cc-wind
interaction.
Loop's defining feature: lib/guest CHISELLING discipline — every
commit had a chisel note, and the cumulative work satisfies the
two-consumer rule for three new kit extractions.
lib/scheme/test.sh — single-process test runner. Loads parser/eval/
runtime + lib/guest/reflective/env.sx once, then for each test
suite loads its file and calls its (*-tests-run!) function. Parses
the {:passed N :failed N ...} dict output and aggregates.
Usage:
bash lib/scheme/test.sh # summary
bash lib/scheme/test.sh -v # per-suite breakdown
Output: "ok 296/296 scheme-on-sx tests passed (9 suites)"
lib/scheme/scoreboard.md — per-suite passing counts, phase status,
deferred items, reflective-kit consumption ledger.
The scoreboard documents the chisel value of the Scheme port:
three reflective kits unlocked (env.sx — already extracted with
Scheme as third consumer; evaluator.sx + quoting.sx — second-
consumer-ready for extraction whenever a follow-up commit is run).
Loop status: 11 phases done (1, 2, 3, 3.5, 4, 5abc, 6ab, 7, 8, 9,
10, 11). Two deferred (6c hygiene, full call/cc-wind interaction).
296 tests, 1830 LoC of Scheme implementation. Zero substrate fixes
required across the loop.
eval.sx adds module support:
(define-library NAME EXPR...)
Where EXPR is one of:
(export NAME ...)
(import LIB-NAME ...)
(begin BODY ...)
(import LIB-NAME ...)
Looks up each library by key, copies its exported names
into the current env.
Library values: {:scm-tag :library :name :exports :env}
Stored in scheme-library-registry keyed by joined library-name
(`(my math)` → `"my/math"`).
Library body runs in a FRESH standard env (each library is its
own namespace). Only :exports are visible after import; private
internal definitions stay in the library's env. Internal calls
between library functions use the library's env, so public-facing
exports can rely on private helpers.
Multiple imports work — each library is independent.
NOT yet supported: cond-expand, include, include-library-
declarations, renaming (`(only ...)`, `(except ...)`, `(prefix ...)`,
`(rename ...)`). Standard R7RS modules use these but the core
two-operation flow (define-library / import) covers most everyday
module use.
7 tests: single export, multi-export, private-not-visible,
internal-calls-private, two-libs-both-imported, unknown-lib-error,
single-symbol library name.
296 total Scheme tests (62+23+49+78+25+20+13+10+9+7).
Phases done: 1, 2, 3, 3.5, 4, 5abc, 6ab, 7, 8, 9, 10.
Deferred: 6c (hygiene/scope-set — research-grade), 11 (conformance).
eval.sx adds the define-record-type syntactic operator:
(define-record-type NAME
(CONSTRUCTOR ARG...)
PREDICATE
(FIELD ACCESSOR [MUTATOR])...)
Records are tagged dicts:
{:scm-record TYPE-NAME :fields {FIELD VALUE ...}}
For each record type, the operator binds:
- Constructor: takes the listed ARGs, populates :fields, returns
the record. Fields not in CONSTRUCTOR ARGs default to nil.
- Predicate: returns true iff its arg is a record of THIS type
(tag-match via :scm-record).
- Accessor per field: extracts the field value; errors if not
a record of the right type.
- Mutator per field (optional): sets the field via dict-set!;
same type-check.
Distinct types are isolated via their tag — point? returns false
on a circle, even if both have the same shape.
9 tests cover: constructor + predicate + accessors, mutator,
distinct-types-via-tag, records as first-class values (in lists,
passed to map/filter), constructor arity errors.
289 total Scheme tests (62+23+49+78+25+20+13+10+9).
