coop/rose-ash
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

OCaml evaluator for page dispatch + handler aser, 83/83 Playwright tests

Browse Source 
Major architectural change: page function dispatch and handler execution
now go through the OCaml kernel instead of the Python bootstrapped evaluator.

OCaml integration:
- Page dispatch: bridge.eval() evaluates SX URL expressions (geography, marshes, etc.)
- Handler aser: bridge.aser() serializes handler responses as SX wire format
- _ensure_components loads all .sx files into OCaml kernel (spec, web adapter, handlers)
- defhandler/defpage registered as no-op special forms so handler files load
- helper IO primitive dispatches to Python page helpers + IO handlers
- ok-raw response format for SX wire format (no double-escaping)
- Natural list serialization in eval (no (list ...) wrapper)
- Clean pipe: _read_until_ok always sends io-response on error

SX adapter (aser):
- scope-emit!/scope-peek aliases to avoid CEK special form conflict
- aser-fragment/aser-call: strings starting with "(" pass through unserialized
- Registered cond-scheme?, is-else-clause?, primitive?, get-primitive in kernel
- random-int, parse-int as kernel primitives; json-encode, into via IO bridge

Handler migration:
- All IO calls converted to (helper "name" args...) pattern
- request-arg, request-form, state-get, state-set!, now, component-source etc.
- Fixed bare (effect ...) in island bodies leaking disposer functions as text
- Fixed lower-case → lower, ~search-results → ~examples/search-results

Reactive islands:
- sx-hydrate-islands called after client-side navigation swap
- force-dispose-islands-in for outerHTML swaps (clears hydration markers)
- clear-processed! platform primitive for re-hydration

Content restructuring:
- Design, event bridge, named stores, phase 2 consolidated into reactive overview
- Marshes split into overview + 5 example sub-pages
- Nav links use sx-get/sx-target for client-side navigation

Playwright test suite (sx/tests/test_demos.py):
- 83 tests covering hypermedia demos, reactive islands, marshes, spec explorer
- Server-side rendering, handler interactions, island hydration, navigation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
giles
2026-03-18 17:22:51 +00:00
parent 5b6e883e6d
commit 71c2003a60
33 changed files with 1848 additions and 852 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
shared/sx/handlers.py
View File

@@ -137,36 +137,55 @@ async def execute_handler(
1. Build env from component env + handler closure
2. Bind handler params from args (typically request.args)
3. Evaluate via ``async_eval_to_sx`` (I/O inline, components serialized)
3. Evaluate via OCaml kernel (or Python fallback)
4. Return ``SxExpr`` wire format
"""
from .jinja_bridge import get_component_env, _get_request_context
from .pages import get_page_helpers
from .parser import serialize
from .types import NIL, SxExpr
import os
if os.environ.get("SX_USE_REF") == "1":
from .ref.async_eval_ref import async_eval_to_sx
else:
from .async_eval import async_eval_to_sx
from .types import NIL
if args is None:
args = {}
# Build environment
env = dict(get_component_env())
env.update(get_page_helpers(service_name))
env.update(handler_def.closure)
use_ocaml = os.environ.get("SX_USE_OCAML") == "1"
# Bind handler params from request args
for param in handler_def.params:
env[param] = args.get(param, args.get(param.replace("-", "_"), NIL))
if use_ocaml:
from .ocaml_bridge import get_bridge
# Get request context for I/O primitives
ctx = _get_request_context()
# Serialize handler body with bound params as a let expression
param_bindings = []
for param in handler_def.params:
val = args.get(param, args.get(param.replace("-", "_"), NIL))
param_bindings.append(f"({param} {serialize(val)})")
# Async eval → sx source — I/O primitives are awaited inline,
# but component/tag calls serialize to sx wire format (not HTML).
return await async_eval_to_sx(handler_def.body, env, ctx)
body_sx = serialize(handler_def.body)
if param_bindings:
sx_text = f"(let ({' '.join(param_bindings)}) {body_sx})"
else:
sx_text = body_sx
bridge = await get_bridge()
ocaml_ctx = {"_helper_service": service_name}
result_sx = await bridge.aser(sx_text, ctx=ocaml_ctx)
return SxExpr(result_sx or "")
else:
# Python fallback
if os.environ.get("SX_USE_REF") == "1":
from .ref.async_eval_ref import async_eval_to_sx
else:
from .async_eval import async_eval_to_sx
env = dict(get_component_env())
env.update(get_page_helpers(service_name))
env.update(handler_def.closure)
for param in handler_def.params:
env[param] = args.get(param, args.get(param.replace("-", "_"), NIL))
ctx = _get_request_context()
return await async_eval_to_sx(handler_def.body, env, ctx)
# ---------------------------------------------------------------------------

189
shared/sx/ocaml_bridge.py
View File

@@ -101,28 +101,26 @@ class OcamlBridge:
"""Load an .sx file for side effects (defcomp, define, defmacro)."""
async with self._lock:
self._send(f'(load "{_escape(path)}")')
kind, value = await self._read_response()
if kind == "error":
raise OcamlBridgeError(f"load {path}: {value}")
value = await self._read_until_ok(ctx=None)
return int(float(value)) if value else 0
async def load_source(self, source: str) -> int:
"""Evaluate SX source for side effects."""
async with self._lock:
self._send(f'(load-source "{_escape(source)}")')
kind, value = await self._read_response()
if kind == "error":
raise OcamlBridgeError(f"load-source: {value}")
value = await self._read_until_ok(ctx=None)
return int(float(value)) if value else 0
async def eval(self, source: str) -> str:
"""Evaluate SX expression, return serialized result."""
async def eval(self, source: str, ctx: dict[str, Any] | None = None) -> str:
"""Evaluate SX expression, return serialized result.
Supports io-requests (helper calls, query, action, etc.) via the
coroutine bridge, just like render().
"""
await self._ensure_components()
async with self._lock:
self._send(f'(eval "{_escape(source)}")')
kind, value = await self._read_response()
if kind == "error":
raise OcamlBridgeError(f"eval: {value}")
return value or ""
return await self._read_until_ok(ctx)
async def render(
self,
@@ -135,40 +133,84 @@ class OcamlBridge:
self._send(f'(render "{_escape(source)}")')
return await self._read_until_ok(ctx)
async def aser(self, source: str, ctx: dict[str, Any] | None = None) -> str:
"""Evaluate SX and return SX wire format, handling io-requests."""
await self._ensure_components()
async with self._lock:
self._send(f'(aser "{_escape(source)}")')
return await self._read_until_ok(ctx)
async def _ensure_components(self) -> None:
"""Load component definitions into the kernel on first use."""
"""Load all .sx source files into the kernel on first use.
Errors during loading are handled gracefully — IO responses are
always sent back to keep the pipe clean.
"""
if self._components_loaded:
return
self._components_loaded = True
try:
from .jinja_bridge import get_component_env, _CLIENT_LIBRARY_SOURCES
from .parser import serialize
from .types import Component, Island, Macro
from .jinja_bridge import _watched_dirs, _dirs_from_cache
import glob
env = get_component_env()
parts: list[str] = list(_CLIENT_LIBRARY_SOURCES)
for key, val in env.items():
if isinstance(val, Island):
ps = ["&key"] + list(val.params)
if val.has_children:
ps.extend(["&rest", "children"])
parts.append(f"(defisland ~{val.name} ({' '.join(ps)}) {serialize(val.body)})")
elif isinstance(val, Component):
ps = ["&key"] + list(val.params)
if val.has_children:
ps.extend(["&rest", "children"])
parts.append(f"(defcomp ~{val.name} ({' '.join(ps)}) {serialize(val.body)})")
elif isinstance(val, Macro):
ps = list(val.params)
if val.rest_param:
ps.extend(["&rest", val.rest_param])
parts.append(f"(defmacro {val.name} ({' '.join(ps)}) {serialize(val.body)})")
if parts:
source = "\n".join(parts)
await self.load_source(source)
_logger.info("Loaded %d definitions into OCaml kernel", len(parts))
# Skip patterns — files that use constructs not available in the kernel
skip_names = {"boundary.sx", "forms.sx"}
skip_dirs = {"tests"}
# Collect files to load
all_files: list[str] = []
# Spec files needed by aser
spec_dir = os.path.join(os.path.dirname(__file__), "../../spec")
for spec_file in ["parser.sx", "render.sx"]:
path = os.path.normpath(os.path.join(spec_dir, spec_file))
if os.path.isfile(path):
all_files.append(path)
# All directories loaded into the Python env
all_dirs = list(set(_watched_dirs) | _dirs_from_cache)
# Web adapters (aser lives in adapter-sx.sx) — only load specific files
web_dir = os.path.join(os.path.dirname(__file__), "../../web")
if os.path.isdir(web_dir):
for web_file in ["adapter-sx.sx"]:
path = os.path.normpath(os.path.join(web_dir, web_file))
if os.path.isfile(path):
all_files.append(path)
for directory in sorted(all_dirs):
files = sorted(
glob.glob(os.path.join(directory, "**", "*.sx"), recursive=True)
)
for filepath in files:
basename = os.path.basename(filepath)
# Skip known-bad files
if basename in skip_names:
continue
# Skip test and handler directories
parts = filepath.replace("\\", "/").split("/")
if any(d in skip_dirs for d in parts):
continue
all_files.append(filepath)
# Load all files under a single lock
count = 0
skipped = 0
async with self._lock:
for filepath in all_files:
try:
self._send(f'(load "{_escape(filepath)}")')
value = await self._read_until_ok(ctx=None)
# Response may be a number (count) or a value — just count files
count += 1
except OcamlBridgeError as e:
skipped += 1
_logger.warning("OCaml load skipped %s: %s",
filepath, e)
_logger.info("Loaded %d definitions from .sx files into OCaml kernel (%d skipped)",
count, skipped)
except Exception as e:
_logger.error("Failed to load components into OCaml kernel: %s", e)
_logger.error("Failed to load .sx files into OCaml kernel: %s", e)
self._components_loaded = False # retry next time
async def reset(self) -> None:
@@ -217,14 +259,19 @@ class OcamlBridge:
"""Read lines until (ok ...) or (error ...).
Handles (io-request ...) by fulfilling IO and sending (io-response ...).
ALWAYS sends a response to keep the pipe clean, even on error.
"""
while True:
line = await self._readline()
if line.startswith("(io-request "):
result = await self._handle_io_request(line, ctx)
# Send response back to OCaml
self._send(f"(io-response {_serialize_for_ocaml(result)})")
try:
result = await self._handle_io_request(line, ctx)
self._send(f"(io-response {_serialize_for_ocaml(result)})")
except Exception as e:
# MUST send a response or the pipe desyncs
_logger.warning("IO request failed, sending nil: %s", e)
self._send("(io-response nil)")
continue
kind, value = _parse_response(line)
@@ -264,7 +311,15 @@ class OcamlBridge:
return self._io_request_method()
elif req_name == "ctx":
return self._io_ctx(args, ctx)
elif req_name == "helper":
return await self._io_helper(args, ctx)
else:
# Fall back to registered IO handlers (set-response-status, sleep, etc.)
from .primitives_io import _IO_HANDLERS, RequestContext
io_handler = _IO_HANDLERS.get(req_name)
if io_handler is not None:
helper_args = [_to_python(a) for a in args]
return await io_handler(helper_args, {}, ctx or RequestContext())
raise OcamlBridgeError(f"Unknown io-request type: {req_name}")
async def _io_query(self, args: list) -> Any:
@@ -309,6 +364,43 @@ class OcamlBridge:
key = _to_str(args[0]) if args else ""
return ctx.get(key)
async def _io_helper(self, args: list, ctx: dict[str, Any] | None) -> Any:
"""Handle (io-request "helper" name arg1 arg2 ...).
Dispatches to registered page helpers — Python functions like
read-spec-file, bootstrapper-data, etc. The helper service name
is passed via ctx["_helper_service"].
"""
import asyncio
from .pages import get_page_helpers
from .primitives_io import _IO_HANDLERS, RequestContext
name = _to_str(args[0]) if args else ""
helper_args = [_to_python(a) for a in args[1:]]
# Check page helpers first (application-level)
service = (ctx or {}).get("_helper_service", "sx")
helpers = get_page_helpers(service)
fn = helpers.get(name)
if fn is not None:
result = fn(*helper_args)
if asyncio.iscoroutine(result):
result = await result
return result
# Fall back to IO primitives (now, state-get, state-set!, etc.)
io_handler = _IO_HANDLERS.get(name)
if io_handler is not None:
return await io_handler(helper_args, {}, RequestContext())
# Fall back to regular primitives (json-encode, into, etc.)
from .primitives import get_primitive as _get_prim
prim = _get_prim(name)
if prim is not None:
return prim(*helper_args)
raise OcamlBridgeError(f"Unknown helper: {name!r}")
# ------------------------------------------------------------------
# Module-level singleton
@@ -344,6 +436,9 @@ def _parse_response(line: str) -> tuple[str, str | None]:
line = line.strip()
if line == "(ok)":
return ("ok", None)
if line.startswith("(ok-raw "):
# Raw SX wire format — no unescaping needed
return ("ok", line[8:-1])
if line.startswith("(ok "):
value = line[4:-1] # strip (ok and )
# If the value is a quoted string, unquote it
@@ -369,6 +464,16 @@ def _unescape(s: str) -> str:
)
def _to_python(val: Any) -> Any:
"""Convert an SX parsed value to a plain Python value."""
from .types import NIL as _NIL
if val is None or val is _NIL:
return None
if hasattr(val, "name"): # Symbol or Keyword
return val.name
return val
def _to_str(val: Any) -> str:
"""Convert an SX parsed value to a Python string."""
if isinstance(val, str):

39
shared/sx/primitives_io.py
View File

@@ -642,7 +642,8 @@ from . import primitives_ctx # noqa: E402, F401
# Auto-derive IO_PRIMITIVES from registered handlers
# ---------------------------------------------------------------------------
IO_PRIMITIVES: frozenset[str] = frozenset(_IO_HANDLERS.keys())
# Placeholder — rebuilt at end of file after all handlers are registered
IO_PRIMITIVES: frozenset[str] = frozenset()
# ---------------------------------------------------------------------------
@@ -703,9 +704,45 @@ _PRIMITIVES["relations-from"] = _bridge_relations_from
# Validate all IO handlers against boundary.sx
# ---------------------------------------------------------------------------
@register_io_handler("helper")
async def _io_helper(
args: list[Any], kwargs: dict[str, Any], ctx: RequestContext
) -> Any:
"""``(helper "name" args...)`` → dispatch to page helpers or IO handlers.
Universal IO dispatcher — same interface as the OCaml kernel's helper
IO primitive. Checks page helpers first, then IO handlers.
"""
if not args:
raise ValueError("helper requires a name")
name = str(args[0])
helper_args = args[1:]
# Check page helpers first
from .pages import get_page_helpers
helpers = get_page_helpers("sx")
fn = helpers.get(name)
if fn is not None:
import asyncio
result = fn(*helper_args)
if asyncio.iscoroutine(result):
result = await result
return result
# Fall back to IO handlers
io_handler = _IO_HANDLERS.get(name)
if io_handler is not None:
return await io_handler(helper_args, {}, ctx)
raise ValueError(f"Unknown helper: {name!r}")
def _validate_io_handlers() -> None:
from .boundary import validate_io
for name in _IO_HANDLERS:
validate_io(name)
_validate_io_handlers()
# Rebuild IO_PRIMITIVES now that all handlers (including helper) are registered
IO_PRIMITIVES = frozenset(_IO_HANDLERS.keys())