Commit Graph

361 Commits

Author SHA1 Message Date
7a8a166326 Test runner: SSR fix, cek-try primitive, HTML entity handling
- render-html-island wraps body SSR in cek-try (graceful fallback to
  empty container when island body has DOM/signal code)
- defcomp placeholder pattern: server renders safe placeholder div
  with data-sx-island, browser hydrates the actual island
- cek-try primitive added to both server and browser OCaml kernels
- assert/assert= added to spec/harness.sx for standalone use
- Test source stored in <script type="text/sx-test" data-for="...">
  with HTML entity decoding via host-call replaceAll

Temperature converter: 5 tests embedded in demo page. Test runner
hydrates and finds tests but body render is empty — needs debugging
of the specific construct that silently fails in render-to-dom.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 01:38:10 +00:00
5754a9ff9f Add inline test runner for temperature converter demo
Temperature converter tests (6 tests): initial value, computed
fahrenheit derivation, +5/-5 click handlers, reactive propagation,
multiple click accumulation.

New components:
- sx/sx/reactive-islands/test-runner.sx — reusable defisland that
  parses test source, runs defsuite/deftest forms via cek-eval, and
  displays pass/fail results with re-run button
- sx/sx/reactive-islands/test-temperature.sx — standalone test file

Added cek-try primitive to both browser (sx_browser.ml) and server
(sx_server.ml) for safe test execution with error catching.

Browser bundle now includes harness files (harness.sx,
harness-reactive.sx, harness-web.sx) for inline test execution.

Known: SSR renders test runner body instead of placeholder, causing
arity error on complex str expressions. Needs island SSR handling fix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 01:00:07 +00:00
f7cf5dbd47 Live reactive island preview in render tab
defisland expressions are now rendered as fully reactive live previews.
The component is registered via sx-load-components, then called through
the native render-dom-island adapter path — same code path as real
islands on the page.

Bare (deref sig) expressions are fully reactive — clicking buttons
updates signal values and DOM updates in real time. Compound expressions
like (str (deref c) "°C") render statically (cek-reactive-text only
tracks bare deref). Next: auto-wrap deref-containing expressions in
computed signals for full reactive expression support.

Also cleaned up stale effect and preview-el signal from earlier attempts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:51:59 +00:00
d8161050a8 Render tab: live island preview via effect + render-to-dom
defisland expressions in the render tab are now evaluated through
render-to-dom inside with-island-scope, producing real DOM with signal
bindings. The rendered output persists in a stable container managed by
an effect, outside the reactive render tree.

Known limitation: signal subscriptions don't fire on button clicks yet —
the reactive wiring needs investigation into how deferred (setTimeout)
execution contexts interact with CEK signal tracking. Static rendering
(initial values) works correctly.

Also: removed stale effect that attempted hydrate-island approach.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:37:36 +00:00
7047a5d7f3 Add git-aware SX tools: changed, diff_branch, blame, doc_gen, playwright
- sx_changed: list .sx files changed since a ref with structural summaries
- sx_diff_branch: structural diff of all .sx changes vs base ref
- sx_blame: git blame for .sx files, optionally focused on a tree path
- sx_doc_gen: generate component docs from defcomp/defisland signatures
- sx_playwright: run Playwright browser tests with structured results

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:25:00 +00:00
76ce0c3ecb Add 15 new MCP tools to sx-tree: project-wide search, smart editing, dev workflow
New comprehension tools:
- sx_find_across: search pattern across all .sx files in a directory
- sx_comp_list: list all definitions (defcomp/defisland/defmacro/defpage/define)
- sx_comp_usage: find all uses of a component across files
- sx_diff: structural diff between two .sx files (ADDED/REMOVED/CHANGED)
- sx_eval: REPL — evaluate SX expressions in the MCP server env

Smart read_tree enhancements:
- Auto-summarise large files (>200 lines)
- focus param: expand only matching subtrees, collapse rest
- max_depth/max_lines/offset for depth limiting and pagination

Smart editing tools:
- sx_rename_symbol: rename all occurrences of a symbol in a file
- sx_replace_by_pattern: find+replace first/all pattern matches
- sx_insert_near: insert before/after a pattern match (top-level)
- sx_rename_across: rename symbol across all .sx files (with dry_run)
- sx_write_file: create .sx files with parse validation

Development tools:
- sx_pretty_print: reformat .sx files with indentation (also used by all edit tools)
- sx_build: build JS bundle or OCaml binary
- sx_test: run test suites with structured pass/fail results
- sx_format_check: lint for empty bindings, missing bodies, duplicate params
- sx_macroexpand: evaluate expressions with a file's macro definitions loaded

Also: updated hook to block Write on .sx files, added custom explore agent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:19:10 +00:00
68c05dcd28 Add HyperSX view tab: indentation-based alternative syntax for SX
sx->hypersx transform converts parsed SX to a readable indentation-based
format: CSS selector shorthand (div.card#main), signal sugar (@count,
signal(), :=, <-), string interpolation ("Count: {@count}"), and
structural keywords (when, if, let, map, for).

Implemented as pure SX in web/lib/hypersx.sx, loaded in browser via
js_of_ocaml platform. Added as "hypersx" tab in the tree editor.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:01:43 +00:00
85702e92c9 Add cond-scheme? primitive and parameter binding to tree editor render tab
Register cond-scheme? as OCaml primitive — was defined in spec/evaluator.sx
but never exposed to the browser runtime, causing render.sx to crash with
"Undefined symbol: cond-scheme?" on every SX response. This broke URL
updates on navigation (handle-history never ran after the rendering error).

Tree editor render tab now extracts &key params from defcomp/defisland
definitions and shows input fields. Values substitute into the rendered
preview live as you type. Inputs live outside the reactive cond branch
so signal updates don't steal focus.

sx-tools page function accepts &key params (title, etc.) forwarded to
the overview component.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 22:16:08 +00:00
8c99ec4fac Fix render tab: render HTML expressions directly, serialize non-HTML
The render tab now correctly renders HTML tag expressions as live DOM
(div, h2, p, ul, etc. with Tailwind classes). Non-HTML expressions
(defcomp, define, etc.) are shown as serialized SX text.

Previous version tried eval-expr which errored silently in the browser.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:46:00 +00:00
63babc0d2d Add render tab to tree editor, switch MCP paths to SX format
Tree editor island now has 4 tabs: tree, context, validate, render.
The render tab evaluates SX source as live HTML — type a (div (h2 "Hello"))
and see it rendered immediately.

MCP server paths changed from JSON arrays [0,2,1] to SX strings "(0 2 1)".
Fixes serialization issues and is more natural for an SX tool. The
json_to_path function now parses SX via sx-parse.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:35:40 +00:00
6d0e512f19 Add interactive tree editor island (Phase 4) + MCP server fixes
Phase 4: defisland ~sx-tools/tree-editor — interactive tree viewer
embedded in the SX Tools page. Features:
- Textarea with :bind for SX source input
- Parse button to re-parse on demand
- Tree view: annotated tree with path labels, clickable nodes
- Context view: enclosing chain from root to selected node
- Validate view: structural integrity checks (catches missing body etc.)

MCP server fixes: added ident-start?, ident-char?, make-keyword,
escape-string, sx-expr-source — needed by parser.sx when loaded
into the MCP evaluator.

Also: .mcp.json for Claude Code MCP server config, CLAUDE.md protocol
for structural .sx file editing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 20:11:27 +00:00
6f96452f70 Fix empty code blocks: rename ~docs/code param, fix batched IO dispatch
Two bugs caused code blocks to render empty across the site:

1. ~docs/code component had parameter named `code` which collided with
   the HTML <code> tag name. Renamed to `src` and updated all 57
   callers. Added font-mono class for explicit monospace.

2. Batched IO dispatch in ocaml_bridge.py only skipped one leading
   number (batch ID) but the format has two (epoch + ID):
   (io-request EPOCH ID "name" args...). Changed to skip all leading
   numbers so the string name is correctly found. This fixes highlight
   and other batchable helpers returning empty results.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 18:08:40 +00:00
739d04672b Merge branch 'full-react' into ocaml-vm
# Conflicts:
#	sx/sx/nav-data.sx
#	sx/sx/page-functions.sx
2026-03-25 17:53:28 +00:00
c18f46278f Fix home-stepper paren bugs, harden defisland multi-body, add SX Tools essay
Two paren bugs in home-stepper.sx caused the home page to render blank:

1. Line 222 had one extra ) that prematurely closed the letrec bindings
   list — rebuild-preview and do-back became body expressions instead
   of bindings, making them undefined in scope.

2. Lines 241-308 were outside the let/letrec scope entirely — the outer
   let closed at line 240, so freeze-scope, cookie restore, source
   parsing, and the entire div rendering tree had no access to signals
   or letrec functions.

Also hardens defisland to wrap multi-expression bodies in (begin ...),
matching the Python-side fix from 9f0c541. Both spec/evaluator.sx and
the OCaml transpiled sx_ref.ml are updated.

Adds SX Tools essay under Applications — the revised plan for structural
tree reading/editing tools for .sx files, motivated by this exact bug.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 17:52:16 +00:00
1777f631a5 Merge sx-pub branch into ocaml-vm
Brings in sx-pub federated publishing protocol (Phases 1-4):
- DB models, IPFS publishing, federation, anchoring
- Live dashboard UI on the plan page
- Dev infrastructure (docker-compose, dev-pub.sh)

Conflicts: sx-browser.js (kept ocaml-vm rebuild), sx-pub.sx (kept sx-pub version with dashboard)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:37:04 +00:00
c62e7319cf Optimize stepper: rebuild-preview replaces step replay
- do-back: uses steps-to-preview + render-to-dom (O(1) render)
  instead of replaying every do-step from 0 (O(n) DOM ops + cssx)
- Hydration effect: same rebuild-preview instead of step replay
- Cookie save moved to button on-click only (not per-step)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:20:13 +00:00
effa767b09 Move reactive runtime from plans to applications section
Promotes reactive-runtime from a plan page to a full applications section
with 7 nav items (ref, foreign FFI, state machines, commands, render loop,
keyed lists, app shell) and its own page function.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:20:12 +00:00
2775ce935b Fix JIT closure scoping: use vm_env_ref not caller's globals
When the VM called a JIT-compiled lambda, it passed vm.globals
(the caller's global env) instead of cl.vm_env_ref (the closure's
captured env that was merged at compile time). Closure-captured
variables like code-tokens from island let/letrec scopes were
invisible at runtime, causing "Undefined symbol" errors that
cascaded to disable render-to-html globally.

Fix: call_closure uses cl.vm_env_ref at both call sites (cached
bytecode and fresh compilation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 16:11:41 +00:00
296729049e Fix home-stepper paren balance in rebuild-preview
rebuild-preview had one extra close paren that closed the outer
(when container) prematurely, pushing do-back and build-code-dom
out of the letrec scope. Result: "Undefined symbol: build-code-dom".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 15:54:38 +00:00
5ca6952217 Fix home-stepper.sx paren balance (extra close parens from edits)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 15:37:49 +00:00
0c8c0b6426 Cache-bust .sx files, optimize stepper back/hydration
Platform:
- sx-platform.js: extract ?v= query from script tag URL, append to
  all .sx file XHR requests. Prevents stale cached .sx files.

Stepper performance:
- do-back: use rebuild-preview (pure SX→DOM render) instead of
  replaying every do-step from 0. O(1) instead of O(n).
- Hydration effect: same rebuild-preview instead of step replay.
- Cookie save moved from do-step to button on-click handlers only.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 14:54:43 +00:00
07bbcaf1bb OCaml VM browser: safe equality, thunk trampolining, platform functions, nav pipeline
Core runtime fixes:
- Safe equality (=, !=): physical equality for dicts/lambdas/signals,
  structural only for acyclic types. Prevents infinite loops on circular
  signal subscriber chains.
- contains?: same safe comparison (physical first, structural for simple types)
- Thunk trampolining in as_number and to_string: leaked thunks auto-resolve
  instead of showing <thunk> or erroring "Expected number, got thunk"
- Diagnostic first error: shows actual type received

Island hydration fixes:
- adapter-dom.sx: skip scope-emit for spreads inside islands (was tripling classes)
- schedule-idle: wrap callback to absorb requestIdleCallback deadline arg
- home-stepper: remove spread-specific highlighting (all tokens same style per step)

Platform functions (boot-helpers.sx):
- fetch-request: 3-arg interface (config, success-fn, error-fn) with promise chain
- build-request-body: form serialization for GET/POST
- strip-component-scripts / extract-response-css: SX text processing
- Navigation: bind-boost-link, bind-client-route-click via execute-request
- Loading state: show-indicator, disable-elements, clear-loading-state
- DOM extras: dom-remove, dom-attr-list (name/value pairs), dom-child-list (SX list),
  dom-is-active-element?, dom-is-input-element?, dom-is-child-of?, dom-on,
  dom-parse-html-document, dom-body-inner-html, create-script-clone
- All remaining stubs: csrf-token, loaded-component-names, observe-intersection,
  event-source-connect/listen, with-transition, cross-origin?, etc.

Navigation pipeline:
- browser-push-state/replace-state: accept 1-arg (URL only) or 3-arg
- boot.sx: wire popstate listener to handle-popstate
- URL updates working via handle-history + pushState fix

Morph debugging (WIP):
- dom-child-list returns proper SX list (was JS Array)
- dom-query accepts optional root element for scoped queries
- Navigation fetches and renders SX responses, URL updates, but morph
  doesn't replace content div (investigating dom-child-list on new elements)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 12:57:24 +00:00
68fcdd6cc0 sx-pub: live dashboard UI on the plan page
Server-rendered dashboard showing live data from sx-pub API:
- Server status (DB, IPFS, actor, domain)
- Actor identity card with public key
- Collections grid with paths
- Published documents with CIDs and sizes
- Recent outbox activity feed
- Followers list
- API endpoint links for direct access

All phases marked as complete.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 02:18:45 +00:00
d12f38a9d5 sx-pub Phase 4: anchoring — Merkle trees, OpenTimestamps, verification
New endpoints:
- POST /pub/anchor — batch unanchored Publish activities into Merkle tree,
  pin tree to IPFS, submit root to OpenTimestamps, store OTS proof on IPFS
- GET /pub/verify/<cid> — verify a CID's Merkle proof against anchored tree

Uses existing shared/utils/anchoring.py infrastructure:
- build_merkle_tree (SHA256, deterministic sort)
- get_merkle_proof / verify_merkle_proof (inclusion proofs)
- submit_to_opentimestamps (3 calendar servers with fallback)

Tested: anchored 1 activity, Merkle tree + OTS proof pinned to IPFS,
verification returns :verified true with full proof chain.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 02:12:40 +00:00
aa1d4d7a67 sx-pub Phase 3: federation — outbox, inbox, follow, delivery, HTTP signatures
New endpoints:
- GET /pub/outbox — paginated activity feed
- POST /pub/inbox — receive Follow/Accept/Publish from remote servers
- POST /pub/follow — follow a remote sx-pub server
- GET /pub/followers — list accepted followers
- GET /pub/following — list who we follow

Federation mechanics:
- HTTP Signature generation (RSA-SHA256) for signed outgoing requests
- HTTP Signature verification for incoming requests
- Auto-accept Follow → store follower → send Accept back
- Accept handling → update following state
- Publish mirroring → pin remote CID to local IPFS
- deliver_to_followers → fan out signed activities to all follower inboxes
- Publish now records activity in outbox for federation delivery

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 01:38:36 +00:00
5aea9d2678 Fix sx_docs Dockerfile: copy integration_tests.ml for dune build
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 01:34:39 +00:00
cf130c4174 sx-pub Phase 2: publish to IPFS, browse collections, resolve by path + CID
New endpoints:
- POST /pub/publish — pin SX content to IPFS, store path→CID in DB
- GET /pub/browse/<collection> — list published documents
- GET /pub/doc/<collection>/<slug> — resolve path to CID, fetch from IPFS
- GET /pub/cid/<cid> — direct CID fetch (immutable, cache forever)

New helpers: pub-publish, pub-collection-items, pub-resolve-document, pub-fetch-cid

Tested: published stdlib.sx (6.9KB) → QmQQyR3Ltqi5sFiwZh5dutPbAM4QsEBnw419RyNnTj4fFM

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 01:30:05 +00:00
99e2009c2b Fix sx_docs Dockerfile: install dune + set PATH for OCaml build
The opam base image has dune in the switch but not on PATH.
RUN eval $(opam env) doesn't persist across layers. Install dune
explicitly and set PATH so dune is available in build steps.

Also fix run-tests.sh to respect QUICK env var from caller
(was being overwritten to false).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 01:20:48 +00:00
7b3d763291 sx-pub Phase 1: DB schema, IPFS wiring, actor + webfinger + collections + status endpoints
Foundation for the sx-pub federated publishing protocol:

- 6 SQLAlchemy models: actor, collections, documents, activities, followers, following
- Conditional DB enablement in sx_docs (DATABASE_URL present → enable DB)
- Python IO helpers: get_or_create_actor (auto-generates RSA keypair),
  list_collections, check_status, seed_default_collections
- 4 defhandler endpoints returning text/sx (no JSON):
  /pub/actor, /pub/webfinger, /pub/collections, /pub/status
- Alembic migration infrastructure for sx service
- Docker compose: DB + pgbouncer + IPFS + env vars

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 01:17:27 +00:00
7d7de86034 Fix stepper client-side [object Object] flash and missing CSSX styles
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 5m54s
Three issues in the stepper island's client-side rendering:

1. do-step used eval-expr with empty env for ~cssx/tw spreads — component
   not found, result leaked as [object Object]. Fixed: call ~cssx/tw
   directly (in scope from island env) with trampoline.

2. steps-to-preview excluded spreads — SSR preview had no styling.
   Fixed: include spreads in the tree so both SSR and client render
   with CSSX classes.

3. build-children used named let (let loop ...) which produces
   unresolved Thunks in render mode due to the named-let compiler
   desugaring interacting with the render/eval boundary. Fixed:
   rewrote as plain recursive function bc-loop avoiding named let.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 00:11:06 +00:00
858275dff9 sx-pub: plan page, dev infrastructure, JS bundle rebuild
- Add sx-pub plan page under Applications with full protocol spec:
  wire format, endpoints, storage, flows, AP critique, phases
- Add nav entry and page function for /sx/(applications.(sx-pub))
- Add docker-compose.dev-pub.yml (sx_docs image + DB + IPFS + Redis)
- Add dev-pub.sh launch script (pub.sx-web.org via Caddy)
- Rebuild sx-browser.js with var fixes for SES compatibility

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 23:41:28 +00:00
57cffb8bcc Fix isomorphic SSR: revert inline opcodes, add named let compilation, fix cookie decode
Three bugs broke island SSR rendering of the home stepper widget:

1. Inline VM opcodes (OP_ADD..OP_DEC) broke JIT-compiled functions.
   The compiler emitted single-byte opcodes for first/rest/len/= etc.
   that produced wrong results in complex recursive code (sx-parse
   returned nil, split-tag produced 1 step instead of 16). Reverted
   compiler to use CALL_PRIM for all primitives. VM opcode handlers
   kept for future use.

2. Named let (let loop ((x init)) body) had no compiler support —
   silently produced broken bytecode. Added desugaring to letrec.

3. URL-encoded cookie values not decoded server-side. Client set-cookie
   uses encodeURIComponent but Werkzeug doesn't decode cookie values.
   Added unquote() in bridge cookie injection.

Also: call-lambda used eval_expr which copies Dict values (signals),
breaking mutations through aser lambda calls. Switched to cek_call.

Also: stepper preview now includes ~cssx/tw spreads for SSR styling.

Tests: 1317 JS, 1114 OCaml, 26 integration (2 pre-existing failures)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 22:32:51 +00:00
3df8c41ca1 Split make_server_env, eliminate all runtime sx_ref imports, fix auth-menu tests
make_server_env split into 7 focused setup functions:
- setup_browser_stubs (22 DOM no-ops)
- setup_scope_env (18 scope primitives from sx_scope.ml)
- setup_evaluator_bridge (CEK eval-expr, trampoline, expand-macro, etc.)
- setup_introspection (type predicates, component/lambda accessors)
- setup_type_operations (string/env/dict/equality/parser helpers)
- setup_html_tags (~100 HTML tag functions)
- setup_io_env (query, action, helper IO bridge)

Eliminate ALL runtime sx_ref.py imports:
- sx/sxc/pages/helpers.py: 24 imports → _ocaml_helpers.py bridge
- sx/sxc/pages/sx_router.py: remove SX_USE_REF fallback
- shared/sx/query_registry.py: use register_components instead of eval

Unify JIT compilation: pre-compile list derived from allowlist
(no manual duplication), only compiler internals pre-compiled.

Fix test_components auth-menu: ~auth-menu → ~shared:fragments/auth-menu

Tests: 1114 OCaml, 29/29 components, 35/35 regression, 6/6 Playwright

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 17:23:09 +00:00
d735e28b39 Delete sx_ref.py — OCaml is the sole SX evaluator
Removes the 5993-line bootstrapped Python evaluator (sx_ref.py) and all
code that depended on it exclusively. Both bootstrappers (JS + OCaml)
now use a new synchronous OCaml bridge (ocaml_sync.py) to run the
transpiler. JS build produces identical output; OCaml bootstrap produces
byte-identical sx_ref.ml.

Key changes:
- New shared/sx/ocaml_sync.py: sync subprocess bridge to sx_server.exe
- hosts/javascript/bootstrap.py: serialize defines → temp file → OCaml eval
- hosts/ocaml/bootstrap.py: same pattern for OCaml transpiler
- shared/sx/{html,async_eval,resolver,jinja_bridge,handlers,pages,deps,helpers}:
  stub or remove sx_ref imports; runtime uses OCaml bridge (SX_USE_OCAML=1)
- sx/sxc/pages: parse defpage/defhandler from AST instead of Python eval
- hosts/ocaml/lib/sx_primitives.ml: append handles non-list 2nd arg per spec
- Deleted: sx_ref.py, async_eval_ref.py, 6 Python test runners, misc ref/ files

Test results: JS 1078/1078, OCaml 1114/1114.
sx_docs SSR has pre-existing rendering issues to investigate separately.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 14:32:55 +00:00
ee868f686b Migrate 6 reactive demo handlers from Python f-strings to SX defhandlers
Moved flash-sale, settle-data, search-products/events/posts, and catalog
endpoints from bp/pages/routes.py into sx/sx/handlers/reactive-api.sx.
routes.py now contains only the SSE endpoint (async generators need Python).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 13:26:25 +00:00
9caf8b6e94 Fix runtime PRIMITIVES for dom/browser library functions
dom.sx and browser.sx are library source (not transpiled into the bundle),
so their functions need explicit PRIMITIVES registration for runtime-eval'd
SX code (islands, data-init scripts). Restore registrations for all dom/
browser functions used at runtime. Revert bootstrap.py transpilation of
dom-lib/browser-lib which overrode native platform implementations that
have essential runtime integration (cekCall wrapping, post-render hooks).

Add Playwright regression test for [object Object] nav link issue.
Replace console-log calls with log-info in init-client.sx.txt.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 12:10:54 +00:00
8ccf5f7c1e Stepper: steps-to-preview for isomorphic preview text (WIP)
steps-to-preview is a pure recursive descent function inside the island's
letrec that builds an SX expression tree from steps[0..target-1].
The preview lake uses it to show partial text (e.g. "the joy of " at step 9).

Still WIP: stepper island doesn't SSR because DOM-only code (effect,
dom-query, dom-create-element) runs in the island body and fails on server.
Need to guard client-only code so SSR can render the pure parts
(code view, counter, preview).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 04:24:12 +00:00
bf305deae1 Isomorphic cookie support + stepper cookie persistence
get-cookie / set-cookie primitives on both server and client:
  - JS: reads/writes document.cookie
  - OCaml: get-cookie reads from _request_cookies hashtable,
    set-cookie is no-op (server sets cookies via HTTP headers)
  - Python bridge: _inject_request_cookies_locked() sends
    (set-request-cookies {:name "val"}) to kernel before page render

Stepper island (home-stepper.sx):
  - Persistence switched from localStorage to cookie (sx-home-stepper)
  - freeze-scope/thaw-from-sx mechanism preserved, just different storage
  - Server reads cookie → thaw restores step-idx → SSR renders correct step
  - Code highlighting: removed imperative code-spans/build-code-dom/
    update-code-highlight; replaced with live DOM query that survives morphs
  - Removed code-view lake wrapper (now plain reactive DOM)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 04:13:53 +00:00
e021184935 Stepper: isomorphic code highlighting + steps-to-preview (WIP)
Code view: SSR now uses same highlighting logic as client update-code-highlight
(bg-amber-100 for current step, font-bold for active, opacity-40 for future).

steps-to-preview: pure function that replays step machine as SX expression
tree — intended for isomorphic preview rendering. Currently working for
simple cases but needs fix for partial step counts (close-loop issue).

Close steps now carry open-attrs/open-spreads for steps-to-preview.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 03:37:27 +00:00
ffe849df8e Stepper: render tokenized code with syntax highlighting in SSR
Replace raw source <pre> with styled spans from build-code-tokens.
Each token gets its colour class, and tokens before the current
step get font-bold text-xs, tokens after get opacity-40.

Home page currently blocked by a separate Map key parse error.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 20:20:11 +00:00
6e804bbb5c Stepper island: eager parsing + SSR content in lakes
Moved source parsing (sx-parse, split-tag, build-code-tokens) out
of the effect so it runs eagerly during SSR. Only DOM manipulation
(build-code-dom, schedule-idle) stays in the effect.

Lakes now have SSR content:
- code-view: shows source code as preformatted text
- home-preview: shows "the joy of sx" with styled spans

Client hydrates and replaces with interactive version.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 18:26:51 +00:00
639a6a2a53 Standalone OOB layout: remove :affinity :server, let client render
The OOB layout delegates to ~shared:layout/oob-sx which needs to
produce sx-swap-oob elements. Without server affinity, the aser
serializes the component call for client-side rendering (matching
production behaviour where the client renders the OOB structure).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 17:48:14 +00:00
3cce3df5b0 Fix standalone OOB layout: delegate to shared layout for proper OOB swaps
The standalone OOB layout was returning nil, so SPA navigation
responses had no OOB swap structure. The header island wasn't
included in responses, so:
- Colour state was lost (island not morphed, signals reset)
- Copyright path wasn't updated (lake not in response)

Now delegates to ~shared:layout/oob-sx which wraps content in
proper OOB sections (filter, aside, menu, main-panel). The header
island with updated :path is included in the content, allowing
the morph to preserve island signals and update lakes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 17:15:57 +00:00
318c818728 Lazy JIT compilation: lambdas compile to bytecode on first call
Replace AOT adapter compilation with lazy JIT — each named lambda is
compiled to VM bytecode on first call, cached in l_compiled field for
subsequent calls. Compilation failures fall back to CEK gracefully.

VM types (vm_code, vm_upvalue_cell, vm_closure) moved to sx_types.ml
mutual recursion block. Lambda and Component records gain mutable
l_compiled/c_compiled cache fields. jit_compile_lambda in sx_vm.ml
wraps body as (fn (params) body), invokes spec/compiler.sx via CEK,
extracts inner closure from OP_CLOSURE constant.

JIT hooks in both paths:
- vm_call: Lambda calls from compiled VM code
- continue_with_call: Lambda calls from CEK step loop (injected by
  bootstrap.py post-processing)

Pre-mark sentinel prevents re-entrancy (compile function itself was
hanging when JIT'd mid-compilation). VM execution errors caught and
fall back to CEK with sentinel marking.

Also: add kbd/samp/var to HTML_TAGS, rebuild sx-browser.js, add page
URL to sx-page-full-py timing log.

Performance: first page 28s (JIT compiles 17 functions), subsequent
pages 0.31s home / 0.71s wittgenstein (was 2.3s). All 1945 tests pass.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-23 08:18:44 +00:00
7628659854 Fix geography index: restore default content and add page gutters
The geography page function returned nil instead of the index-content
component, and the index layout was missing the standard doc page wrapper.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 23:34:51 +00:00
df461beec2 SxExpr aser wire format fix + Playwright test infrastructure + blob protocol
Aser serialization: aser-call/fragment now return SxExpr instead of String.
serialize/inspect passes SxExpr through unquoted, preventing the double-
escaping (\" → \\\" ) that broke client-side parsing when aser wire format
was output via raw! into <script> tags. Added make-sx-expr + sx-expr-source
primitives to OCaml and JS hosts.

Binary blob protocol: eval, aser, aser-slot, and sx-page-full now send SX
source as length-prefixed blobs instead of escaped strings. Eliminates pipe
desync from concurrent requests and removes all string-escape round-trips
between Python and OCaml.

Bridge safety: re-entrancy guard (_in_io_handler) raises immediately if an
IO handler tries to call the bridge, preventing silent deadlocks.

Fetch error logging: orchestration.sx error callback now logs method + URL
via log-warn. Platform catches (fetchAndRestore, fetchPreload, bindBoostForm)
also log errors instead of silently swallowing them.

Transpiler fixes: makeEnv, scopePeek, scopeEmit, makeSxExpr added as
platform function definitions + transpiler mappings — were referenced in
transpiled code but never defined as JS functions.

Playwright test infrastructure:
- nav() captures JS errors and fails fast with the actual error message
- Checks for [object Object] rendering artifacts
- New tests: delete-row interaction, full page refresh, back button,
  direct load with fresh context, code block content verification
- Default base URL changed to localhost:8013 (standalone dev server)
- docker-compose.dev-sx.yml: port 8013 exposed for local testing
- test-sx-build.sh: build + unit tests + Playwright smoke tests

Geography content: index page component written (sx/sx/geography/index.sx)
describing OCaml evaluator, wire formats, rendering pipeline, and topic
links. Wiring blocked by aser-expand-component children passing issue.

Tests: 1080/1080 JS, 952/952 OCaml, 66/66 Playwright

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 22:17:43 +00:00
373a4f0134 Fix pipe desync: send_ok_raw escapes newlines, expand-components? in env
- send_ok_raw: when SX wire format contains newlines (string literals),
  fall back to (ok "...escaped...") instead of (ok-raw ...) to keep
  the pipe single-line. Prevents multi-line responses from desyncing
  subsequent requests.
- expand-components? flag set in kernel env (not just VM adapter globals)
  so aser-list's env-has? check finds it during component expansion.
- SX_STANDALONE: restore no_oauth but generate CSRF via session cookie
  so mutation handlers (DELETE etc.) still work without account service.
- Shell statics injection: only inject small values (hashes, URLs) as
  kernel vars. Large blobs (CSS, component_defs) use placeholder tokens.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 12:32:03 +00:00
ae0e87fbf8 VM aser-slot → sx-page-full: single-call page render, 0.55s warm
Compiler fixes:
- Upvalue re-lookup returns own position (uv-index), not parent slot
- Spec: cek-call uses (make-env) not (dict) — OCaml Dict≠Env
- Bootstrap post-processes transpiler Dict→Env for cek_call

VM runtime fixes:
- compile_adapter evaluates constant defines (SPECIAL_FORM_NAMES etc.)
  via execute_module instead of wrapping as NativeFn closures
- Native primitives: map-indexed, some, every?
- Nil-safe HO forms: map/filter/for-each/some/every? accept nil as empty
- expand-components? set in kernel env (not just VM globals)
- unwrap_env diagnostic: reports actual type received

sx-page-full command:
- Single OCaml call: aser-slot body + render-to-html shell
- Eliminates two pipe round-trips (was: aser-slot→Python→shell render)
- Shell statics (component_defs, CSS, pages_sx) cached in Python,
  injected into kernel once, referenced by symbol in per-request command
- Large blobs use placeholder tokens — Python splices post-render,
  pipe transfers ~51KB instead of 2MB

Performance (warm):
- Server total: 0.55s (was ~2s)
- aser-slot VM: 0.3s, shell render: 0.01s, pipe: 0.06s
- kwargs computation: 0.000s (cached)

SX_STANDALONE mode for sx_docs dev (skips fragment fetches).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-20 11:06:04 +00:00
8707f21ca2 Single-pass aser_slot for HTMX path + kernel eval timing + stable hash
Eliminated double-aser for HTMX requests: build OOB wrapper AST
(~shared:layout/oob-sx :content wrapped_ast) and aser_slot in ONE
pass — same pattern as the full-page path. Halves aser_slot calls.

Added kernel-side timing to stderr:
  [aser-slot] eval=3.6s io_flush=0.0s batched=3 result=22235 chars

Results show batch IO works (io_flush=0.0s for 3 highlight calls)
and the bottleneck is pure CEK evaluation time, not IO.

Performance after single-pass fix:
  Home: 0.7s eval (was 2.2s total)
  Reactive: 3.6s eval (was 6.8s total)
  Language: 1.1s eval (was 18.9s total — double-aser eliminated)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 17:03:47 +00:00
d3b3b4b720 Fix pipe desync: async drain on _send, robust Playwright tests
Root cause: OcamlBridge._send() used write() without drain().
asyncio.StreamWriter buffers writes — without drain(), multiple
commands accumulate and flush as a batch. The kernel processes
them sequentially, sending responses, but Python only reads one
response per command → pipe desync → "unexpected response" errors.

Fix: _send() is now async, calls drain() after every write.
All 14 callers updated to await.

Playwright tests rewritten:
- test_home_has_header: verifies #logo-opacity visible (was only
  checking for "sx" text — never caught missing header)
- test_home_has_nav_children: Geography link must be visible
- test_home_has_main_panel: #main-panel must have child elements
- TestDirectPageLoad: fresh browser.new_context() per test to
  avoid stale component hash in localStorage
- _setup_error_capture + _check_no_fatal_errors helpers

_render_to_sx uses aser_slot (not aser) — layout wrappers contain
re-parsed content that needs full expansion capability.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-19 16:11:55 +00:00