Added vm-compile command: iterates env, compiles lambdas to bytecode,
replaces with NativeFn VM wrappers (with CEK fallback on error).
Tested: 3/109 compile, reduces CEK steps 23%.
Disabled auto-compile in production — the compiler doesn't handle
closures with upvalues yet, and compiled functions that reference
dynamic env vars crash. Infrastructure stays for when compiler
handles all SX features.
Also: added set-nth! and mutable-list primitives (needed by
compiler.sx for bytecode patching). Fixed compiler.sx to use
mutable lists on OCaml (ListRef for append!/set-nth! mutation).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
End-to-end pipeline working:
Python compiler.sx → bytecode → OCaml VM → result
Verified: (+ (* 3 4) 2) → 14 ✓
(+ 0 1 2 ... 49) → 1225 ✓
Benchmark (500 iterations, 50 additions each):
CEK machine: 327ms
Bytecode VM: 145ms
Speedup: 2.2x
VM handles: constants, local variables, global variables,
primitive calls, jumps, conditionals, closures (via NativeFn
wrapper), define, return.
Protocol: (vm-exec {:bytecode (...) :constants (...)})
- Compiler outputs clean format (no internal index dict)
- VM converts bytecode list to int array, constants to value array
- Stack-based execution with direct opcode dispatch
The 2.2x speedup is for pure arithmetic. For aser (the real
target), the speedup will be larger because aser involves:
- String building (no CEK frame allocation in VM)
- Map/filter iterations (no frame-per-iteration in VM)
- Closure calls (no thunk/trampoline in VM)
Next: compile and run the aser adapter on the VM.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three new files forming the bytecode compilation pipeline:
spec/bytecode.sx — opcode definitions (~65 ops):
- Stack/constant ops (CONST, NIL, TRUE, POP, DUP)
- Lexical variable access (LOCAL_GET/SET, UPVALUE_GET/SET, GLOBAL_GET/SET)
- Jump-based control flow (JUMP, JUMP_IF_FALSE/TRUE)
- Function ops (CALL, TAIL_CALL, RETURN, CLOSURE, CALL_PRIM)
- HO form ops (ITER_INIT/NEXT, MAP_OPEN/APPEND/CLOSE)
- Scope/continuation ops (SCOPE_PUSH/POP, RESET, SHIFT)
- Aser specialization (ASER_TAG, ASER_FRAG)
spec/compiler.sx — SX-to-bytecode compiler (SX code, portable):
- Scope analysis: resolve variables to local/upvalue/global at compile time
- Tail position detection for TCO
- Code generation for: if, when, and, or, let, begin, lambda,
define, set!, quote, function calls, primitive calls
- Constant pool with deduplication
- Jump patching for forward references
hosts/ocaml/lib/sx_vm.ml — bytecode interpreter (OCaml):
- Stack-based VM with array-backed operand stack
- Call frames with base pointer for locals
- Direct opcode dispatch via pattern match
- Zero allocation per step (unlike CEK machine's dict-per-step)
- Handles: constants, variables, jumps, calls, primitives,
collections, string concat, define
Architecture: compiler.sx is spec (SX, portable). VM is platform
(OCaml-native). Same bytecode runs on JS/WASM VMs.
Also includes: CekFrame record optimization in transpiler.sx
(29 frame types as records instead of Hashtbl).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
