Commit Graph

98 Commits

Author SHA1 Message Date
c794e33dda sx-http: load sxc/ components (~docs/page), auto-detect Docker paths
Load sx/sxc/ directory for core layout components like ~docs/page,
~docs/section, ~docs/code. Auto-detect Docker (/app/sxc) vs dev
(/project/sx/sxc) paths with SX_SXC_DIR env override.

Fixes pages that use ~docs/page — all content pages now render
correctly with full component expansion.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:26:25 +00:00
2ae42d3898 sx-http: Docker compose overlay + auto-detect component dir
docker-compose.dev-sx-native.yml overrides entrypoint to run OCaml
HTTP server inside Docker container (Caddy on externalnet can reach it).

Auto-detect sx component directory: checks /app/sx (Docker) first,
falls back to /project/sx/sx (dev). SX_COMPONENTS_DIR env override.

Live on sx.rose-ash.com via Caddy → Docker → OCaml HTTP server.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:20:27 +00:00
7108b01e37 sx-http: response cache — 323 req/s, 3ms TTFB, 47x throughput vs Quart
In-memory response cache populated during startup pre-warm. Cache misses
render on-demand and cache the result. All cached pages serve in 2-14ms.

Performance (cached, 2 worker domains, 2MB RSS):
  Homepage:  3-10ms TTFB (was 202ms Quart) — 20-60x faster
  Geography: 3-14ms TTFB (was 144ms Quart) — 10-48x faster
  Reactive:  2-5ms TTFB (was 187ms Quart) — 37-94x faster
  Throughput: 323 req/s at c=10 (was 6.8 Quart) — 47x higher
  Memory: 2MB (was 570MB Quart) — 285x less

Cache invalidation: not yet implemented (restart to refresh).
Future: file watcher or content-hash based invalidation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:11:51 +00:00
abca040a5d sx-http: pre-warm pages on startup for JIT hot-start
Render 5 key pages during startup to trigger JIT compilation of page
functions, aser, render-to-html, and all component paths. Steady-state
performance after warmup:

  Homepage:  212-280ms (aser=136-173ms, ssr=44-90ms)
  Geography: 282-354ms (aser=215-273ms, ssr=44-61ms)
  Reactive:  356-522ms (aser=306-390ms, ssr=38-109ms)

SSR phase is fast (streaming renderer). Aser is the bottleneck —
response caching eliminates it for static pages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:09:16 +00:00
1412648f6e sx-http: buffer-based streaming renderer, eliminates string allocation
New render_to_buf / render_to_html_streaming in sx_render.ml — writes
HTML directly to a Buffer.t instead of building intermediate strings.
Eliminates hundreds of string concatenations per page render.

Full parallel renderer: render_to_buf, render_element_buf,
render_component_buf, render_cond_buf, render_let_buf, render_map_buf,
render_for_each_buf — all buffer-native.

HTTP server SSR + shell now use streaming renderer.

Performance (warm, 2 worker domains, 2MB RSS):
  Homepage:  138-195ms TTFB (Quart: 202ms) — faster
  Geography: 218-286ms TTFB (Quart: 144ms)
  Throughput: 6.85 req/s at c=5 (Quart: 6.8 req/s) — matched

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:06:27 +00:00
3620a433c1 sx-http: log JIT fallbacks once per function, not silently or flooding
JIT runtime errors now log once per function name via _jit_warned
hashtable, then stay quiet for that function. No more silent swallowing
(which hid real errors) or per-call flooding (which spammed thousands
of lines and blocked the server).

VM-level fallbacks (inside JIT-compiled code calling other JIT code)
are silent — dedup happens at the hook level.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:02:27 +00:00
8e1870246d sx-http: OCaml 5 domain pool + per-request env safety
Domain pool with N workers (Domain.recommended_domain_count), mutex+
condition queue for request dispatch. Each domain has its own minor
heap — GC pauses don't block other requests.

expand-components? bound once at startup (always true in HTTP mode)
instead of per-request mutation. Shell rendering uses native
Sx_render.render_to_html for domain safety.

Performance (warm, 2 worker domains, 2MB RSS):
  Homepage:  107-194ms TTFB (Quart: 202ms) — faster
  Geography: 199-306ms TTFB (Quart: 144ms) — close
  Reactive:  351-382ms TTFB (Quart: 187ms) — 2x slower
  Concurrent: 5.88 req/s at c=5 (Quart: 6.8 req/s)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 17:00:13 +00:00
8105064e82 sx-http: native SSR + island placeholders — homepage 158ms, 2MB RSS
Switch SSR from SX adapter to native Sx_render.render_to_html.
Islands that fail SSR emit <span data-sx-island="name"> placeholders
instead of recursing into client-only bodies. Eliminates the <home
symbol error cascade that made the homepage render 23s.

Performance (warm, single process):
  Homepage:  131-191ms TTFB (was 202ms on Quart) — faster
  Geography: 203-229ms TTFB (was 144ms on Quart) — close
  Reactive:  155-290ms TTFB (was 187ms on Quart) — similar
  Memory:    2MB RSS (was 570MB on Quart) — 285x reduction

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 16:52:32 +00:00
1d064a1914 sx-http: silent JIT fallback, load signals+engine, fix render-to-html
JIT runtime errors now silently fall back to CEK without disabling the
compiled bytecode or logging. This prevents render-to-html from being
permanently disabled when one page has an unresolved symbol (e.g. the
homepage stepper's <home).

Load spec/signals.sx and web/engine.sx for reactive primitives.
Skip test files and non-rendering directories (tests/, plans/, essays/).

Performance with JIT active (warm, single process, 2MB RSS):
- Aser (component expansion): 21-87ms — faster than Quart baseline
- SSR + shell: variable due to homepage <home fallback issue
- Geography total: ~500ms when JIT stays active

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 16:49:15 +00:00
9fc13efa1a sx-http: JIT enabled, signals loaded, recursive dir loading, assoc fix
- Enable lazy JIT in HTTP mode — pre-compile 24 compiler functions at startup
- Load spec/signals.sx + web/engine.sx for reactive primitives
- Recursive directory loading for subdirectory components (geography/, etc.)
- Re-bind native variadic assoc after stdlib.sx overwrites it
- Skip test files, plans/, essays/ directories during HTTP load
- Homepage aser: 21-38ms warm, Geography aser: 39-87ms warm

Remaining: render-to-html JIT gets disabled by <home symbol error on
first request (falls back to CEK). ~docs/page component missing for
some pages. Fix those for full parity with Quart.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 16:32:41 +00:00
0d5770729f sx-host step 3: HTTP server mode + define shorthand + SX highlighter
HTTP server (--http PORT): OCaml serves sx-docs directly, no Python.
Loads components at startup, routes /sx/ URLs, renders full pages with
shell. Geography page: 124ms TTFB (vs 144ms Quart). Single process.

define shorthand: (define (name args) body) desugars to
(define name (fn (args) body)) in the CEK step function.

SX highlighter (lib/highlight.sx): pure SX syntax highlighting with
Tailwind spans. Tokenizes SX/Lisp code — comments, strings, keywords,
components, specials, numbers, booleans. Replaces Python highlight.py.

Platform constructors: make-lambda, make-component, make-island,
make-macro, make-thunk, make-env + accessor functions bound for
evaluator.sx compatibility in HTTP mode.

Tests: 1116/1117 OCaml, 7/7 Playwright (main tree).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 16:15:58 +00:00
153f02c672 sx-host plan steps 1-2: defhelper + SX config + SXTP spec + nav tools
Step 1 — defhelper: SX-defined page data helpers replace Python helpers.
(defhelper name (params) body) in .sx files, using existing IO primitives
(query, action, service). Loaded into OCaml kernel as pure SX defines.

Step 2 — SX config: app-config.sx replaces app-config.yaml with (defconfig)
form. (env-get "VAR") resolves secrets from environment. Kebab-to-underscore
aliasing ensures backward compatibility with all 174 config consumers.

Also: SXTP protocol spec (applications/sxtp/spec.sx), docs article,
sx_nav move/delete modes, reactive-runtime moved to geography.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 15:18:45 +00:00
0cae1fbb6b Fix event-bridge + add client? primitive + header store foundation
- Event-bridge: rewrite island to use document-level addEventListener
  via effect + host-callback, bypassing broken container-ref + schedule-idle.
  Also use host-get for event-detail (WASM host handles).

- Add client? primitive: false on server (sx_primitives._is_client ref),
  true in browser (sx_browser.ml sets ref). Enables SSR-safe conditional
  logic for client-only features like def-store.

- Header island: use def-store for idx/shade signals when client? is true,
  falling back to plain signals on server. Foundation for SPA nav state
  preservation (store registry persistence still needs work).

- Remove unused client? K.eval override from sx-platform.js.

100 passed, 1 skipped (isomorphic nav — store registry resets on SPA nav), 0 failed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 10:05:32 +00:00
07fabeb4ed Fix apply primitive for Lambda/VmClosure + Playwright test fixes
The OCaml `apply` primitive only handled NativeFn, causing swap! to
fail in the WASM browser when called with lambda arguments. Extended
to handle all callable types via _sx_call_fn/_sx_trampoline_fn.

Also fixes:
- Pre-existing build errors from int-interned env.bindings migration
  (vm-trace, bytecode-inspect, deps-check, prim-check in sx_server.ml)
- Add #portal-root div to page shell for portal island rendering
- Stepper test scoped to lake area (code-view legitimately shows ~cssx/tw)
- Portal test checks #portal-root instead of #sx-root
- Mark 3 known bugs as test.fixme (event-bridge, resource, isomorphic-nav)

98 passed, 3 skipped, 0 failed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-28 08:58:48 +00:00
77a80e0640 Merge semantics: eval rules, capabilities, modules, geography pages
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:14:14 +00:00
91185ff520 Fix sx_nav tool: scan nav-tree.sx + evaluate (dict ...) calls
The tool only scanned nav-data.sx with raw AST walking, missing entries
that use (dict :key val) call syntax instead of {:key val} literals.
Now scans both nav-data.sx and nav-tree.sx, evaluating expressions
through the SX evaluator so dict calls produce proper Dict values.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 16:10:16 +00:00
70d03eca18 Add sx_nav MCP tool for docs navigation management
Three modes:
- list: show all nav items, filterable by section
- check: validate consistency (duplicate hrefs, missing page functions,
  components without nav entries)
- add: scaffold new article (component file + page function + nav entry)

Scans nav-data.sx, page-functions.sx, and all .sx component files.
Prevents the class of bugs where nav entries, page functions, and
component definitions get out of sync.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 15:37:01 +00:00
31af9a5ca3 Merge ocaml-vm: 6 VM debugging and build tools
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 01:24:45 +00:00
dab81fc571 Add 6 VM/bytecode debugging and build tools
OCaml server commands:
- vm-trace: step-by-step bytecode execution trace (opcode, stack, depth)
- bytecode-inspect: disassemble compiled function (opcodes, constants, arity)
- deps-check: strict symbol resolution (resolved vs unresolved symbols)
- prim-check: verify CALL_PRIM opcodes match real primitives

Scripts:
- hosts/ocaml/browser/test_boot.sh: WASM boot test in Node.js
- scripts/sx-build-all.sh: full pipeline (OCaml + JS + tests)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 01:23:55 +00:00
c923a34fa8 Fix WASM browser click handlers: 8 bugs, 50 new VM tests
The sx-get links were doing full page refreshes because click handlers
never attached. Root causes: VM frame management bug, missing primitives,
CEK/VM type dispatch mismatch, and silent error swallowing.

Fixes:
- VM frame exhaustion: frames <- [] now properly pops to rest_frames
- length primitive: add alias for len in OCaml primitives
- call_sx_fn: use sx_call directly instead of eval_expr (CEK checks
  for type "lambda" but VmClosure reports "function")
- Boot error surfacing: Sx.init() now has try/catch + failure summary
- Callback error surfacing: catch-all handler for non-Eval_error exceptions
- Silent JIT failures: log before CEK fallback instead of swallowing
- vm→env sync: loadModule now calls sync_vm_to_env()
- sx_build_bytecode MCP tool added for bytecode compilation

Tests: 50 new tests across test-vm.sx and test-vm-primitives.sx covering
nested VM calls, frame integrity, CEK bridge, primitive availability,
cross-module symbol resolution, and callback dispatch.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 00:37:21 +00:00
10576f86d1 WASM browser build: interned env keys, async kernel boot, bundled .sx platform
- Symbol interning in sx_types.ml: env lookups use int keys (intern/unintern)
  to avoid repeated string hashing in scope chain walks
- sx-platform.js: poll for SxKernel availability (WASM init is async)
- shell.sx: load sx_browser.bc.wasm.js when SX_USE_WASM=1
- bundle.sh: fix .sx file paths (web-signals.sx rename)
- browser/dune: target byte+js+wasm modes
- Bundle 23 .sx platform files for browser (dom, signals, router, boot, etc.)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 16:37:42 +00:00
96d0d29f10 Add capability-based evaluation context primitives
Four new primitives for capability-aware evaluation:
- with-capabilities: restrict capabilities for a body (sets global cap stack)
- current-capabilities: query current capability set
- has-capability?: check if a specific capability is available
- require-capability!: assert a capability, error if missing

Uses a global OCaml ref (cap_stack) for cross-CEK-boundary visibility.
Note: with-capabilities error propagation from CEK sub-evaluations
needs deeper integration — the primitives themselves work correctly.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 14:35:15 +00:00
5a5521f21f Add (use) module declarations
- (use module-name) is a no-op at eval time — purely declarative
- find-use-declarations in tree-tools.sx scans files for use forms
- sx_deps now reports declared modules alongside dependency analysis
- Native 'use' binding in MCP server so files with use don't error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 14:28:41 +00:00
36ba3eb298 Add eval-rules.sx and sx_explain tool
Machine-readable SX semantics reference with 35 evaluation rules
covering literals, lookup, special forms, definitions, higher-order
forms, scopes, continuations, and reactive primitives.

New sx_explain MCP tool: query by form name ("let", "map") or
category ("special-form", "higher-order") to get pattern, rule,
effects, and examples.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 14:26:31 +00:00
5ed984e7e3 Add 5 MCP tools, refactor nav-data, fix deep path bug, fix Playwright failures
Nav refactoring:
- Split nav-data.sx (32 forms) into 6 files: nav-geography, nav-language,
  nav-applications, nav-etc, nav-tools, nav-tree
- Add Tools top-level nav category with SX Tools and Services pages
- New services-tools.sx page documenting the rose-ash-services MCP server

JS build fixes (fixes 5 Playwright failures):
- Wire web/web-signals.sx into JS build (stores, events, resources)
- Add cek-try primitive to JS platform (island hydration error handling)
- Merge PRIMITIVES into getRenderEnv (island env was missing primitives)
- Rename web/signals.sx → web/web-signals.sx to avoid spec/ collision

New MCP tools:
- sx_trace: step-through CEK evaluation showing lookups, calls, returns
- sx_deps: dependency analysis — free symbols + cross-file resolution
- sx_build_manifest: show build contents for JS and OCaml targets
- sx_harness_eval extended: multi-file loading + setup expressions

Deep path bug fix:
- Native OCaml list-replace and navigate bypass CEK callback chain
- Fixes sx_replace_node and sx_read_subtree corruption on paths 6+ deep

Tests: 1478/1478 JS full suite, 91/91 Playwright

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 12:09:22 +00:00
4e88b8a9dd Merge ocaml-vm: reactive runtime demos, sx-web plan, build fixes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 09:22:22 +00:00
b53a0fabea Inline test runner: 5/5 temperature converter tests passing
Fixed three fundamental issues:
1. cek-try arg passing: handler was called with raw string instead of
   (List [String msg]), causing "lambda expects 1 args, got N" errors
2. Silent island hydration failures: hydrate-island now wraps body
   render in cek-try, displaying red error box with stack trace instead
   of empty div. No more silent failures.
3. swap! thunk leak: apply result wasn't trampolined, storing thunks
   as signal values instead of evaluated results

Also fixed: assert= uses = instead of equal? for value comparison,
assert-signal-value uses deref instead of signal-value, HTML entity
decoding in script tag test source via host-call replaceAll.

Temperature converter demo page now shows live test results:
  ✓ initial celsius is 20
  ✓ computed fahrenheit = celsius * 1.8 + 32
  ✓ +5 increments celsius
  ✓ fahrenheit updates on celsius change
  ✓ multiple clicks accumulate

1116/1116 OCaml tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 01:51:30 +00:00
5754a9ff9f Add inline test runner for temperature converter demo
Temperature converter tests (6 tests): initial value, computed
fahrenheit derivation, +5/-5 click handlers, reactive propagation,
multiple click accumulation.

New components:
- sx/sx/reactive-islands/test-runner.sx — reusable defisland that
  parses test source, runs defsuite/deftest forms via cek-eval, and
  displays pass/fail results with re-run button
- sx/sx/reactive-islands/test-temperature.sx — standalone test file

Added cek-try primitive to both browser (sx_browser.ml) and server
(sx_server.ml) for safe test execution with error catching.

Browser bundle now includes harness files (harness.sx,
harness-reactive.sx, harness-web.sx) for inline test execution.

Known: SSR renders test runner body instead of placeholder, causing
arity error on complex str expressions. Needs island SSR handling fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 01:00:07 +00:00
fea44f9fcc Add reactive runtime demos + sx-web federation plan
7 interactive island demos for the reactive runtime layers:
- L0 Ref (timer + DOM handle), L1 Foreign (canvas via host-call),
  L2 Machine (traffic light), L3 Commands (undo/redo),
  L4 Loop (bouncing ball), L5 Keyed Lists, L6 App Shell

Fix OCaml build: add (wrapped false) to lib/dune, remove Sx. qualifiers.
Fix JS build: include dom-lib + browser-lib in adapter compilation.

New plan: sx-web federated component web — browser nodes via WebTransport,
server nodes via IPFS, in-browser authoring, AI composition layer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 00:54:23 +00:00
b104663481 Split signals: core spec (spec/signals.sx) + web extensions (web/signals.sx)
Core reactive primitives (signal, deref, reset!, swap!, computed, effect,
batch, notify-subscribers, dispose-computed, with-island-scope,
register-in-scope) moved to spec/signals.sx — pure SX, zero platform
dependencies. Usable by any host: web, CLI, embedded, server, harness.

Web extensions (marsh scopes, stores, event bridge, resource) remain in
web/signals.sx, which now depends on spec/signals.sx.

Updated all load paths:
- hosts/ocaml/bin/sx_server.ml — loads spec/signals.sx before web/signals.sx
- hosts/ocaml/bin/run_tests.ml — loads both in order
- hosts/ocaml/browser/bundle.sh + sx-platform.js — loads core-signals.sx first
- shared/sx/ocaml_bridge.py — loads spec/signals.sx before web extensions

1116/1116 OCaml tests pass. Browser reactive island preview works.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 00:23:35 +00:00
b1690a92c4 Add SX test harness: mock IO platform for testing components
spec/harness.sx — spec-level test harness with:
- Mock platform (30+ default IO mocks: fetch, query, DOM, storage, etc.)
- Session management (make-harness, harness-reset!, harness-set!/get)
- IO interception (make-interceptor, install-interceptors)
- IO log queries (io-calls, io-call-count, io-call-nth, io-call-args)
- IO assertions (assert-io-called, assert-no-io, assert-io-count, etc.)

15 harness tests passing on both OCaml (1116/1116) and JS (15/15).
Loaded automatically by both test runners.

MCP tool: sx_harness_eval — evaluate SX with mock IO, returns result + IO trace.

The harness is extensible: new platforms just add entries to the platform dict.
Components can ship with deftest forms that verify IO behavior against mocks.
Tests are independent objects that can be published separately (by CID).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 00:00:19 +00:00
7047a5d7f3 Add git-aware SX tools: changed, diff_branch, blame, doc_gen, playwright
- sx_changed: list .sx files changed since a ref with structural summaries
- sx_diff_branch: structural diff of all .sx changes vs base ref
- sx_blame: git blame for .sx files, optionally focused on a tree path
- sx_doc_gen: generate component docs from defcomp/defisland signatures
- sx_playwright: run Playwright browser tests with structured results

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:25:00 +00:00
76ce0c3ecb Add 15 new MCP tools to sx-tree: project-wide search, smart editing, dev workflow
New comprehension tools:
- sx_find_across: search pattern across all .sx files in a directory
- sx_comp_list: list all definitions (defcomp/defisland/defmacro/defpage/define)
- sx_comp_usage: find all uses of a component across files
- sx_diff: structural diff between two .sx files (ADDED/REMOVED/CHANGED)
- sx_eval: REPL — evaluate SX expressions in the MCP server env

Smart read_tree enhancements:
- Auto-summarise large files (>200 lines)
- focus param: expand only matching subtrees, collapse rest
- max_depth/max_lines/offset for depth limiting and pagination

Smart editing tools:
- sx_rename_symbol: rename all occurrences of a symbol in a file
- sx_replace_by_pattern: find+replace first/all pattern matches
- sx_insert_near: insert before/after a pattern match (top-level)
- sx_rename_across: rename symbol across all .sx files (with dry_run)
- sx_write_file: create .sx files with parse validation

Development tools:
- sx_pretty_print: reformat .sx files with indentation (also used by all edit tools)
- sx_build: build JS bundle or OCaml binary
- sx_test: run test suites with structured pass/fail results
- sx_format_check: lint for empty bindings, missing bodies, duplicate params
- sx_macroexpand: evaluate expressions with a file's macro definitions loaded

Also: updated hook to block Write on .sx files, added custom explore agent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:19:10 +00:00
3559ce44f2 Add named path navigation: "defisland > let > letrec" resolves to index path
Named paths let you navigate by structure name instead of opaque indices.
Both formats work in all MCP tools:
  - Index: "(0 3 2)"
  - Named: "defisland > let > letrec"

The server detects ">" in the path string and calls resolve-named-path
(SX function) which walks the tree matching child names at each level.

New SX functions: resolve-named-path, split-path-string, find-child-by-name.
MCP server: added trim/split primitives, resolve_path dispatcher.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:39:12 +00:00
63babc0d2d Add render tab to tree editor, switch MCP paths to SX format
Tree editor island now has 4 tabs: tree, context, validate, render.
The render tab evaluates SX source as live HTML — type a (div (h2 "Hello"))
and see it rendered immediately.

MCP server paths changed from JSON arrays [0,2,1] to SX strings "(0 2 1)".
Fixes serialization issues and is more natural for an SX tool. The
json_to_path function now parses SX via sx-parse.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:35:40 +00:00
6d0e512f19 Add interactive tree editor island (Phase 4) + MCP server fixes
Phase 4: defisland ~sx-tools/tree-editor — interactive tree viewer
embedded in the SX Tools page. Features:
- Textarea with :bind for SX source input
- Parse button to re-parse on demand
- Tree view: annotated tree with path labels, clickable nodes
- Context view: enclosing chain from root to selected node
- Validate view: structural integrity checks (catches missing body etc.)

MCP server fixes: added ident-start?, ident-char?, make-keyword,
escape-string, sx-expr-source — needed by parser.sx when loaded
into the MCP evaluator.

Also: .mcp.json for Claude Code MCP server config, CLAUDE.md protocol
for structural .sx file editing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:11:27 +00:00
934604c2bd Add SX tree tools: comprehension, editing, and MCP server
Phase 1-3 of the SX Tools plan — structural reading, editing, and
MCP server for .sx files.

lib/tree-tools.sx — Pure SX functions for tree comprehension and editing:
  Comprehension: annotate-tree, summarise, read-subtree, get-context,
    find-all, get-siblings, validate, navigate
  Editing: replace-node, insert-child, delete-node, wrap-node, tree-set
  Helpers: list-replace, list-insert, list-remove, replace-placeholder

lib/tests/test-tree-tools.sx — 107 tests covering all functions.

hosts/ocaml/bin/mcp_tree.ml — MCP server (stdio JSON-RPC) exposing
  11 tools. Loads tree-tools.sx into the OCaml evaluator, parses .sx
  files with the native parser, calls SX functions for tree operations.

The MCP server can be configured in Claude Code's settings.json as:
  "mcpServers": { "sx-tree": { "command": "path/to/mcp_tree.exe" } }

1429 tests passing (1322 existing + 107 new tree-tools).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 19:16:41 +00:00
43cf590541 Expand JIT closure tests: nested, mutual recursion, set!, island, deep
22 JIT closure scoping tests covering:
- Basic closure var in map callback + context switch
- Signal + letrec + map (stepper pattern)
- Nested closures (inner lambda sees outer let var)
- Mutual recursion in letrec (is-even/is-odd)
- set! mutation of closure var after JIT compilation
- defisland with signal + letrec + map
- Deep nesting (for-each inside map inside letrec inside let)

All test the critical invariant: JIT-compiled lambdas must use
their closure's vm_env_ref, not the caller's globals.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:16:00 +00:00
57e7ce9fe4 Add JIT closure scoping tests
Tests the exact pattern that broke the home stepper: a component
with letrec bindings referenced inside a map callback. The JIT
compiles the callback with closure vars merged into vm_env_ref.
Subsequent renders must use that env, not the caller's globals.

7 tests covering:
- letrec closure var in map callback (fmt function)
- Render, unrelated render, re-render (env not polluted)
- Signal + letrec + map (the stepper pattern)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:13:28 +00:00
759309c5c4 Fix create-text-node on server: return string instead of nil
Server-side create-text-node was returning Nil, causing imperative
text nodes (stopwatch "Start"/"0.0s", imperative counter "0") to
render as empty in SSR HTML. Now returns the text as a String value,
which render-to-html handles via escape-html.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:29:06 +00:00
f3f70cc00b Move stdlib out of spec — clean spec/library boundary
spec/ now contains only the language definition (5 files):
  evaluator.sx, parser.sx, primitives.sx, render.sx, special-forms.sx

lib/ contains code written IN the language (8 files):
  stdlib.sx, types.sx, freeze.sx, content.sx,
  bytecode.sx, compiler.sx, vm.sx, callcc.sx

Test files follow source: spec/tests/ for core language tests,
lib/tests/ for library tests (continuations, freeze, types, vm).

Updated all consumers:
- JS/Python/OCaml bootstrappers: added lib/ to source search paths
- OCaml bridge: spec_dir for parser/render, lib_dir for compiler/freeze
- JS test runner: scans spec/tests/ (always) + lib/tests/ (--full)
- OCaml test runner: scans spec/tests/, lib tests via explicit request
- Docker dev mounts: added ./lib:/app/lib:ro

Tests: 1041 JS standard, 1322 JS full, 1101 OCaml — all pass

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 23:18:30 +00:00
57cffb8bcc Fix isomorphic SSR: revert inline opcodes, add named let compilation, fix cookie decode
Three bugs broke island SSR rendering of the home stepper widget:

1. Inline VM opcodes (OP_ADD..OP_DEC) broke JIT-compiled functions.
   The compiler emitted single-byte opcodes for first/rest/len/= etc.
   that produced wrong results in complex recursive code (sx-parse
   returned nil, split-tag produced 1 step instead of 16). Reverted
   compiler to use CALL_PRIM for all primitives. VM opcode handlers
   kept for future use.

2. Named let (let loop ((x init)) body) had no compiler support —
   silently produced broken bytecode. Added desugaring to letrec.

3. URL-encoded cookie values not decoded server-side. Client set-cookie
   uses encodeURIComponent but Werkzeug doesn't decode cookie values.
   Added unquote() in bridge cookie injection.

Also: call-lambda used eval_expr which copies Dict values (signals),
breaking mutations through aser lambda calls. Switched to cek_call.

Also: stepper preview now includes ~cssx/tw spreads for SSR styling.

Tests: 1317 JS, 1114 OCaml, 26 integration (2 pre-existing failures)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 22:32:51 +00:00
91cf39153b Add promise-delayed SSR stub; fix startup JIT DISABLED noise
promise-delayed is a browser-only primitive used by the resource island
demo. The SSR renderer needs it as a stub to avoid "Undefined symbol"
errors during render-to-html JIT compilation.

The stub returns the value argument (skipping the delay), so SSR renders
the resolved state immediately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:39:48 +00:00
13ba5ee423 Unify JIT to lazy-only: remove allowlist, all lambdas compile on first call
Replace the manual jit_allowlist (StringSet of ~40 function names) with
universal lazy compilation. Every named lambda gets one compile attempt
on first call; failures are sentineled and never retried.

Compiler internals are still pre-compiled at startup (bootstrapping the
JIT itself), but everything else compiles lazily — no manual curation.

Remove jit-allow command (no longer needed). Remove StringSet module.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:34:27 +00:00
a6e0e84521 Split setup_type_operations into 6 focused functions
125-line monolith split into:
- setup_core_operations (assert, append!, apply, equal?, primitive?)
- setup_type_constructors (make-keyword, make-symbol, escape-string, etc.)
- setup_character_classification (ident-start?, ident-char?, char-numeric?)
- setup_env_operations (env-get, env-has?, env-bind!, env-set!, etc.)
- setup_strict_mode (gradual type system support)
- setup_io_bridges (json-encode, into, sleep, response headers)

make_server_env now calls 12 focused setup functions total.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:31:37 +00:00
3ae49b69f5 Fix env-shadowing: rebind host extension points after .sx file load
evaluator.sx defines *custom-special-forms* and register-special-form!
which shadow the host's native bindings when loaded at runtime. The
native bindings route to Sx_ref.custom_special_forms (the dict the CEK
evaluator checks), but the SX-level defines create a separate dict.

Fix: rebind_host_extensions runs after every load command, re-asserting
the native register-special-form! and *custom-special-forms* bindings.

Add regression test: custom form registered before evaluator.sx load
survives and remains callable via CEK dispatch afterward.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:29:29 +00:00
33af6b9266 Fix serialize_value for SxExpr/Spread; handle List-of-SxExpr in aser output
serialize_value was falling through to "nil" for SxExpr and Spread values.
Now SxExpr passes through as raw SX text, Spread serializes as make-spread.

The aser command's result handler now joins a List of SxExprs as a
space-separated fragment (from map/filter producing multiple SxExprs).

Investigation ongoing: handler aser responses still have "class" strings
where :class keywords should be — the component expansion path in aser
loses keyword types during CEK evaluation of component bodies.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 18:16:33 +00:00
3df8c41ca1 Split make_server_env, eliminate all runtime sx_ref imports, fix auth-menu tests
make_server_env split into 7 focused setup functions:
- setup_browser_stubs (22 DOM no-ops)
- setup_scope_env (18 scope primitives from sx_scope.ml)
- setup_evaluator_bridge (CEK eval-expr, trampoline, expand-macro, etc.)
- setup_introspection (type predicates, component/lambda accessors)
- setup_type_operations (string/env/dict/equality/parser helpers)
- setup_html_tags (~100 HTML tag functions)
- setup_io_env (query, action, helper IO bridge)

Eliminate ALL runtime sx_ref.py imports:
- sx/sxc/pages/helpers.py: 24 imports → _ocaml_helpers.py bridge
- sx/sxc/pages/sx_router.py: remove SX_USE_REF fallback
- shared/sx/query_registry.py: use register_components instead of eval

Unify JIT compilation: pre-compile list derived from allowlist
(no manual duplication), only compiler internals pre-compiled.

Fix test_components auth-menu: ~auth-menu → ~shared:fragments/auth-menu

Tests: 1114 OCaml, 29/29 components, 35/35 regression, 6/6 Playwright

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 17:23:09 +00:00
6ef9688bd2 Fix primitive? lookup + replace coercion; remove debug output
primitive? in make_server_env was checking env bindings only (NativeFn),
missing all 132 primitives in the Sx_primitives hashtable. Now checks
both primitives table and env. get-primitive similarly fixed.

replace primitive now coerces SxExpr/Thunk/RawHTML/etc to strings instead
of crashing with "replace: 3 string args" — fixes aser JIT DISABLED.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:29:52 +00:00
f9f810ffd7 Complete Python eval removal: epoch protocol, scope consolidation, JIT fixes
Route all rendering through OCaml bridge — render_to_html no longer uses
Python async_eval. Fix register_components to parse &key params and &rest
children from defcomp forms. Remove all dead sx_ref.py imports.

Epoch protocol (prevents pipe desync):
- Every command prefixed with (epoch N), all responses tagged with epoch
- Both sides discard stale-epoch messages — desync structurally impossible
- OCaml main loop discards stale io-responses between commands

Consolidate scope primitives into sx_scope.ml:
- Single source of truth for scope-push!/pop!/peek, collect!/collected,
  emit!/emitted, context, and 12 other scope operations
- Removes duplicate registrations from sx_server.ml (including bugs where
  scope-emit! and clear-collected! were registered twice with different impls)
- Bind scope prims into env so JIT VM finds them via OP_GLOBAL_GET

JIT VM fixes:
- Trampoline thunks before passing args to CALL_PRIM
- as_list resolves thunks via _sx_trampoline_fn
- len handles all value types (Bool, Number, RawHTML, SxExpr, Spread, etc.)

Other fixes:
- ~cssx/tw signature: (tokens) → (&key tokens) to match callers
- Minimal Python evaluator in html.py for sync sx() Jinja function
- Python scope primitive stubs (thread-local) for non-OCaml paths
- Reader macro resolution via OcamlSync instead of sx_ref.py

Tests: 1114 OCaml, 1078 JS, 35 Python regression, 6/6 Playwright SSR

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 16:14:40 +00:00