Commit Graph

10 Commits

Author SHA1 Message Date
c0665ba58e Adopt Step 7 language features across SX codebase
112 conversions across 19 .sx files using match, let-match, and pipe operators:

match (17): type/value dispatch replacing cond/if chains
  - lib/vm.sx: HO form dispatch (for-each/map/filter/reduce/some/every?)
  - lib/tree-tools.sx: node-display, node-matches?, rename, count, replace, free-symbols
  - lib/types.sx: narrow-type, substitute-in-type, infer-type, resolve-type
  - web/engine.sx: default-trigger, resolve-target, classify-trigger
  - web/deps.sx: scan-refs-walk, scan-io-refs-walk

let-match (89): dict destructuring replacing (get d "key") patterns
  - shared/page-functions.sx (20), blog/admin.sx (17), pub-api.sx (13)
  - events/ layouts/page/tickets/entries/forms (27 total)
  - specs-explorer.sx (7), federation/social.sx (3), lib/ small files (3)

-> pipes (6): replacing triple-chained gets in lib/vm.sx
  - frame-closure → closure-code → code-bytecode chains

Also: lib/vm.sx accessor upgrades (get vm "sp" → vm-sp vm throughout)

2650/2650 tests pass, zero regressions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 20:49:02 +00:00
2727577702 VM import suspension for browser lazy loading
Bytecode compiler now emits OP_PERFORM for (import ...) and compiles
(define-library ...) bodies. The VM stores the import request in
globals["__io_request"] and stops the run loop — no exceptions needed.
vm-execute-module returns a suspension dict, vm-resume-module continues.

Browser: sx_browser.ml detects suspension dicts from execute_module and
returns JS {suspended, op, request, resume} objects. The sx-platform.js
while loop handles cascading suspensions via handleImportSuspension.

13 modules load via .sxbc bytecode in 226ms (manifest-driven), both
islands hydrate, all handlers wired. 2650/2650 tests pass including
6 new vm-import-suspension tests.

Also: consolidated sx-platform-2.js → sx-platform.js, fixed
vm-execute-module missing code-from-value call, fixed bootstrap.py
protocol registry transpiler issues.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 17:11:12 +00:00
cd61c049e3 vm.sx feature parity: JIT dispatch, active VM tracking, CEK fallback
Add missing features to lib/vm.sx that sx_vm.ml has:
- *active-vm* mutable global for HO primitive callback VM reuse
- *jit-compile-fn* platform-settable JIT compilation hook
- try-jit-call: check lambda-compiled, attempt JIT, fallback to CEK
- vm-call: VmClosure→push-frame, Lambda→try-jit, Component→CEK
- vm-call-closure: save/restore *active-vm* around execution
- vm-push-frame: refactored to use accessor functions
- cek-call-or-suspend: preamble-provided CEK interop

Transpiled output (sx_vm_ref.ml) now has 12 functions (was 9):
  *active-vm*, *jit-compile-fn*, try-jit-call, vm-call,
  vm-resolve-ho-form, vm-call-external, env-walk, env-walk-set!,
  vm-run, vm-step, vm-call-closure, vm-execute-module

48 preamble functions (native OCaml type access).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 13:00:07 +00:00
fc2b5e502f Step 5p6 lazy loading + Step 6b VM transpilation prep
Lazy module loading (Step 5 piece 6 completion):
- Add define-library wrappers + import declarations to 13 source .sx files
- compile-modules.js generates module-manifest.json with dependency graph
- compile-modules.js strips define-library/import before bytecode compilation
  (VM doesn't handle these as special forms)
- sx-platform.js replaces hardcoded 24-file loadWebStack() with manifest-driven
  recursive loader — only downloads modules the page needs
- Result: 12 modules loaded (was 24), zero errors, zero warnings
- Fallback to full load if manifest missing

VM transpilation prep (Step 6b):
- Refactor lib/vm.sx: 20 accessor functions replace raw dict access
- Factor out collect-n-from-stack, collect-n-pairs, pad-n-nils helpers
- bootstrap_vm.py: transpiles 9 VM logic functions to OCaml
- sx_vm_ref.ml: proof that vm.sx transpiles (preamble has stubs)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-04 12:18:41 +00:00
2d7dd7d582 Step 5 piece 6: migrate 23 .sx files to define-library/import
Wraps all core .sx files in R7RS define-library with explicit export
lists, plus (import ...) at end for backward-compatible global re-export.

Libraries registered:
  (sx bytecode)      — 83 opcode constants
  (sx render)        — 15 tag registries + render helpers
  (sx signals)       — 23 reactive signal primitives
  (sx r7rs)          — 21 R7RS aliases
  (sx compiler)      — 42 compiler functions
  (sx vm)            — 32 VM functions
  (sx freeze)        — 9 freeze/thaw functions
  (sx content)       — 6 content store functions
  (sx callcc)        — 1 call/cc wrapper
  (sx highlight)     — 13 syntax highlighting functions
  (sx stdlib)        — 47 stdlib functions
  (sx swap)          — 13 swap algebra functions
  (sx render-trace)  — 8 render trace functions
  (sx harness)       — 21 test harness functions
  (sx canonical)     — 12 canonical serialization functions
  (web adapter-html) — 13 HTML renderer functions
  (web adapter-sx)   — 13 SX wire format functions
  (web engine)       — 33 hypermedia engine functions
  (web request-handler) — 4 request handling functions
  (web page-helpers) — 12 page helper functions
  (web router)       — 36 routing functions
  (web deps)         — 19 dependency analysis functions
  (web orchestration) — 59 page orchestration functions

Key changes:
- define-library now inherits parent env (env-extend env instead of
  env-extend make-env) so library bodies can access platform primitives
- sx_server.ml: added resolve_library_path + load_library_file for
  import resolution (maps library specs to file paths)
- cek_run_with_io: handles "import" locally instead of sending to
  Python bridge

2608/2608 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 21:48:54 +00:00
b6f304e91a Step 5.5 phase 4 (partial): add OP_PERFORM to vm.sx spec
Adds opcode 112 (OP_PERFORM / IO suspension) to lib/vm.sx's vm-step
dispatch. The native sx_vm.ml already has this opcode — this brings
the SX spec into alignment.

Full VM transpilation deferred to Step 6 (define-record-type): the
dict-based state in vm.sx maps to get_val/sx_dict_set_b calls which
are ~5x slower than native record/array access in the hot loop.
define-record-type will let vm.sx use typed records that the
transpiler maps to OCaml records natively.

2598/2598 tests passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-03 20:29:27 +00:00
a2348e5281 Fix VM HO forms: for-each/map/filter resolve in SX-level VM
The SX-level VM couldn't find for-each/map/etc. because they're CEK
special forms, not primitives in the JS host. Added vm-resolve-ho-form
which creates wrapper functions that dispatch callbacks through
vm-call-external (handling VmClosure callbacks correctly).

Also fixed vm-call dispatch order: Lambda/Component checked before
generic callable? to avoid calling SX Lambdas as JS functions.

JS full: 1585/1585 (was 1582/3 failed). All 3 pre-existing failures fixed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 17:55:43 +00:00
d9aa19cfe9 Revert VM HOF primitives — broke OCaml JIT CALL_PRIM dispatch
Registering map/for-each/reduce as PRIMITIVES caused the compiler to
emit CALL_PRIM for them everywhere. The OCaml VM's call-primitive
can't invoke VM closures, causing "Undefined symbol: resource" crashes.

Revert vm.sx to original CALL_PRIM handler. Remove map/for-each/reduce
from JS PRIMITIVES so compiler emits OP_CALL instead (handled by
vm-call which dispatches correctly).

3 JS VM tests remain failing (VM closure interop) but production is stable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 10:18:07 +00:00
141a351acc Fix last 3 VM tests: HOF primitives dispatch through vm-call-closure
Handle for-each/map/reduce in VM's CALL_PRIM handler instead of JS
primitives. VM closures are called via vm-call-closure (fresh VM,
shared upvalue cells). Native callables dispatch directly.

Named let doesn't work inside vm-step context — use explicit define
+ recursion for reduce loop.

JS tests: 1585/1585 (0 failures)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 09:57:24 +00:00
f3f70cc00b Move stdlib out of spec — clean spec/library boundary
spec/ now contains only the language definition (5 files):
  evaluator.sx, parser.sx, primitives.sx, render.sx, special-forms.sx

lib/ contains code written IN the language (8 files):
  stdlib.sx, types.sx, freeze.sx, content.sx,
  bytecode.sx, compiler.sx, vm.sx, callcc.sx

Test files follow source: spec/tests/ for core language tests,
lib/tests/ for library tests (continuations, freeze, types, vm).

Updated all consumers:
- JS/Python/OCaml bootstrappers: added lib/ to source search paths
- OCaml bridge: spec_dir for parser/render, lib_dir for compiler/freeze
- JS test runner: scans spec/tests/ (always) + lib/tests/ (--full)
- OCaml test runner: scans spec/tests/, lib tests via explicit request
- Docker dev mounts: added ./lib:/app/lib:ro

Tests: 1041 JS standard, 1322 JS full, 1101 OCaml — all pass

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 23:18:30 +00:00