VM aser-slot → sx-page-full: single-call page render, 0.55s warm

Compiler fixes:
- Upvalue re-lookup returns own position (uv-index), not parent slot
- Spec: cek-call uses (make-env) not (dict) — OCaml Dict≠Env
- Bootstrap post-processes transpiler Dict→Env for cek_call

VM runtime fixes:
- compile_adapter evaluates constant defines (SPECIAL_FORM_NAMES etc.)
  via execute_module instead of wrapping as NativeFn closures
- Native primitives: map-indexed, some, every?
- Nil-safe HO forms: map/filter/for-each/some/every? accept nil as empty
- expand-components? set in kernel env (not just VM globals)
- unwrap_env diagnostic: reports actual type received

sx-page-full command:
- Single OCaml call: aser-slot body + render-to-html shell
- Eliminates two pipe round-trips (was: aser-slot→Python→shell render)
- Shell statics (component_defs, CSS, pages_sx) cached in Python,
  injected into kernel once, referenced by symbol in per-request command
- Large blobs use placeholder tokens — Python splices post-render,
  pipe transfers ~51KB instead of 2MB

Performance (warm):
- Server total: 0.55s (was ~2s)
- aser-slot VM: 0.3s, shell render: 0.01s, pipe: 0.06s
- kwargs computation: 0.000s (cached)

SX_STANDALONE mode for sx_docs dev (skips fragment fetches).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-20 11:06:04 +00:00
parent 8dd3eaa1d9
commit ae0e87fbf8
13 changed files with 477 additions and 149 deletions

View File

@@ -12,6 +12,8 @@ x-dev-env: &dev-env
WORKERS: "1"
SX_USE_REF: "1"
SX_BOUNDARY_STRICT: "1"
SX_USE_OCAML: "1"
SX_OCAML_BIN: "/app/bin/sx_server"
x-sibling-models: &sibling-models
# Every app needs all sibling __init__.py + models/ for cross-domain SQLAlchemy imports
@@ -44,6 +46,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./blog/alembic.ini:/app/blog/alembic.ini:ro
- ./blog/alembic:/app/blog/alembic:ro
- ./blog/app.py:/app/app.py
@@ -83,6 +88,9 @@ services:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- /root/rose-ash/_snapshot:/app/_snapshot
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./market/alembic.ini:/app/market/alembic.ini:ro
- ./market/alembic:/app/market/alembic:ro
- ./market/app.py:/app/app.py
@@ -121,6 +129,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./cart/alembic.ini:/app/cart/alembic.ini:ro
- ./cart/alembic:/app/cart/alembic:ro
- ./cart/app.py:/app/app.py
@@ -159,6 +170,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./events/alembic.ini:/app/events/alembic.ini:ro
- ./events/alembic:/app/events/alembic:ro
- ./events/app.py:/app/app.py
@@ -197,6 +211,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./federation/alembic.ini:/app/federation/alembic.ini:ro
- ./federation/alembic:/app/federation/alembic:ro
- ./federation/app.py:/app/app.py
@@ -235,6 +252,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./account/alembic.ini:/app/account/alembic.ini:ro
- ./account/alembic:/app/account/alembic:ro
- ./account/app.py:/app/app.py
@@ -273,6 +293,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./relations/alembic.ini:/app/relations/alembic.ini:ro
- ./relations/alembic:/app/relations/alembic:ro
- ./relations/app.py:/app/app.py
@@ -304,6 +327,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./likes/alembic.ini:/app/likes/alembic.ini:ro
- ./likes/alembic:/app/likes/alembic:ro
- ./likes/app.py:/app/app.py
@@ -335,6 +361,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./orders/alembic.ini:/app/orders/alembic.ini:ro
- ./orders/alembic:/app/orders/alembic:ro
- ./orders/app.py:/app/app.py
@@ -369,6 +398,9 @@ services:
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./test/app.py:/app/app.py
- ./test/sx:/app/sx
- ./test/bp:/app/bp
@@ -393,9 +425,13 @@ services:
- "8012:8000"
environment:
<<: *dev-env
SX_STANDALONE: "true"
volumes:
- /root/rose-ash/_config/app-config.yaml:/app/config/app-config.yaml:ro
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./sx/app.py:/app/app.py
- ./sx/sxc:/app/sxc
- ./sx/bp:/app/bp
@@ -431,6 +467,9 @@ services:
dockerfile: test/Dockerfile.unit
volumes:
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./artdag/core:/app/artdag/core
- ./artdag/l1/tests:/app/artdag/l1/tests
- ./artdag/l1/sexp_effects:/app/artdag/l1/sexp_effects
@@ -456,6 +495,9 @@ services:
dockerfile: test/Dockerfile.integration
volumes:
- ./shared:/app/shared
- ./hosts/ocaml/_build/default/bin/sx_server.exe:/app/bin/sx_server:ro
- ./spec:/app/spec:ro
- ./web:/app/web:ro
- ./artdag:/app/artdag
profiles:
- test

View File

@@ -697,38 +697,36 @@ let compile_adapter env =
(Array.length outer_code.Sx_vm.constants)
(if Array.length outer_code.Sx_vm.constants > 0 then
type_of outer_code.Sx_vm.constants.(0) else "empty");
(* The compiled define body is (fn ...) which compiles to
OP_CLOSURE + [upvalue descriptors] + OP_RETURN.
Extract the inner code object from constants[idx]. *)
let code =
let bc = outer_code.Sx_vm.bytecode in
if Array.length bc >= 4 && bc.(0) = 51 then begin
let idx = bc.(1) lor (bc.(2) lsl 8) in
let bc = outer_code.Sx_vm.bytecode in
if Array.length bc >= 4 && bc.(0) = 51 then begin
(* The compiled define body is (fn ...) which compiles to
OP_CLOSURE + [upvalue descriptors] + OP_RETURN.
Extract the inner code object from constants[idx]. *)
let idx = bc.(1) lor (bc.(2) lsl 8) in
let code =
if idx < Array.length outer_code.Sx_vm.constants then begin
let inner_val = outer_code.Sx_vm.constants.(idx) in
try Sx_vm.code_from_value inner_val
with e ->
Printf.eprintf "[vm] inner code_from_value failed for %s: %s\n%!"
name (Printexc.to_string e);
Printf.eprintf "[vm] inner val type: %s\n%!" (type_of inner_val);
(match inner_val with
| Dict d ->
Printf.eprintf "[vm] inner keys: %s\n%!"
(String.concat ", " (Hashtbl.fold (fun k _ acc -> k::acc) d []));
(match Hashtbl.find_opt d "bytecode" with
| Some v -> Printf.eprintf "[vm] bytecode type: %s\n%!" (type_of v)
| None -> Printf.eprintf "[vm] NO bytecode key\n%!")
| _ -> ());
raise e
end else outer_code
end else outer_code
in
let cl = { Sx_vm.code; upvalues = [||]; name = Some name;
env_ref = globals } in
Hashtbl.replace globals name
(NativeFn ("vm:" ^ name, fun args ->
Sx_vm.call_closure cl args globals));
incr compiled
in
let cl = { Sx_vm.code; upvalues = [||]; name = Some name;
env_ref = globals } in
Hashtbl.replace globals name
(NativeFn ("vm:" ^ name, fun args ->
Sx_vm.call_closure cl args globals));
incr compiled
end else begin
(* Not a lambda — constant expression (e.g. (list ...)).
Execute once and store the resulting value directly. *)
let value = Sx_vm.execute_module outer_code globals in
Hashtbl.replace globals name value;
Printf.eprintf "[vm] %s: constant (type=%s)\n%!" name (type_of value);
incr compiled
end
| _ -> () (* non-dict result — skip *)
with e ->
Printf.eprintf "[vm] FAIL adapter %s: %s\n%!" name (Printexc.to_string e))
@@ -861,11 +859,11 @@ let dispatch env cmd =
io_queue := [];
io_counter := 0;
let t0 = Unix.gettimeofday () in
let expand_fn = NativeFn ("expand-components?", fun _args -> Bool true) in
ignore (env_bind env "expand-components?" expand_fn);
let result = match !vm_adapter_globals with
| Some globals ->
(* VM path: call compiled aser directly *)
Hashtbl.replace globals "expand-components?"
(NativeFn ("expand-components?", fun _args -> Bool true));
Hashtbl.replace globals "expand-components?" expand_fn;
let aser_fn = try Hashtbl.find globals "aser"
with Not_found -> raise (Eval_error "VM: aser not compiled") in
let r = match aser_fn with
@@ -875,15 +873,10 @@ let dispatch env cmd =
Hashtbl.remove globals "expand-components?";
r
| None ->
(* CEK fallback *)
ignore (env_bind env "expand-components?"
(NativeFn ("expand-components?", fun _args -> Bool true)));
let call = List [Symbol "aser";
List [Symbol "quote"; expr];
Env env] in
let r = Sx_ref.eval_expr call (Env env) in
Hashtbl.remove env.bindings "expand-components?";
r
Sx_ref.eval_expr call (Env env)
in
let t1 = Unix.gettimeofday () in
io_batch_mode := false;
@@ -911,6 +904,78 @@ let dispatch env cmd =
Hashtbl.remove env.bindings "expand-components?";
send_error (Printexc.to_string exn))
| List (Symbol "sx-page-full" :: String page_src :: shell_kwargs) ->
(* Full page render: aser-slot body + render-to-html shell in ONE call.
shell_kwargs are keyword pairs: :title "..." :csrf "..." etc.
These are passed directly to ~shared:shell/sx-page-shell. *)
(try
(* Phase 1: aser-slot the page body *)
let exprs = Sx_parser.parse_all page_src in
let expr = match exprs with
| [e] -> e | [] -> Nil | _ -> List (Symbol "<>" :: exprs)
in
io_batch_mode := true;
io_queue := [];
io_counter := 0;
let t0 = Unix.gettimeofday () in
let expand_fn = NativeFn ("expand-components?", fun _args -> Bool true) in
ignore (env_bind env "expand-components?" expand_fn);
let body_result = match !vm_adapter_globals with
| Some globals ->
Hashtbl.replace globals "expand-components?" expand_fn;
let aser_fn = try Hashtbl.find globals "aser"
with Not_found -> raise (Eval_error "VM: aser not compiled") in
let r = match aser_fn with
| NativeFn (_, fn) -> fn [expr; Env env]
| _ -> raise (Eval_error "VM: aser not a function")
in
Hashtbl.remove globals "expand-components?";
r
| None ->
let call = List [Symbol "aser";
List [Symbol "quote"; expr];
Env env] in
Sx_ref.eval_expr call (Env env)
in
let t1 = Unix.gettimeofday () in
io_batch_mode := false;
Hashtbl.remove env.bindings "expand-components?";
let body_str = match body_result with
| String s | SxExpr s -> s
| _ -> serialize_value body_result
in
let body_final = flush_batched_io body_str in
let t2 = Unix.gettimeofday () in
(* Phase 2: render shell with body + all kwargs.
Resolve symbol references (e.g. __shell-component-defs) to their
values from the env — these were pre-injected by the bridge. *)
let resolved_kwargs = List.map (fun v ->
match v with
| Symbol s ->
(try env_get env s
with _ -> try Sx_primitives.get_primitive s with _ -> v)
| _ -> v
) shell_kwargs in
let shell_args = Keyword "page-sx" :: String body_final :: resolved_kwargs in
let shell_call = List (Symbol "~shared:shell/sx-page-shell" :: shell_args) in
let html = Sx_render.render_to_html shell_call env in
let t3 = Unix.gettimeofday () in
Printf.eprintf "[sx-page-full] aser=%.3fs io=%.3fs shell=%.3fs total=%.3fs body=%d html=%d\n%!"
(t1 -. t0) (t2 -. t1) (t3 -. t2) (t3 -. t0)
(String.length body_final) (String.length html);
send_ok_string html
with
| Eval_error msg ->
io_batch_mode := false;
io_queue := [];
Hashtbl.remove env.bindings "expand-components?";
send_error msg
| exn ->
io_batch_mode := false;
io_queue := [];
Hashtbl.remove env.bindings "expand-components?";
send_error (Printexc.to_string exn))
| List [Symbol "render"; String src] ->
(try
let exprs = Sx_parser.parse_all src in

View File

@@ -200,6 +200,17 @@ def compile_spec_to_ml(spec_dir: str | None = None) -> str:
return '\n'.join(fixed)
output = fix_mutable_reads(output)
# Fix cek_call: the spec passes (make-env) as the env arg to
# continue_with_call, but the transpiler evaluates it at transpile
# time (it's a primitive), producing Dict instead of Env.
# Fix cek_call: the spec passes (make-env) as the env arg to
# continue_with_call, but the transpiler evaluates make-env at
# transpile time (it's a primitive), producing Dict instead of Env.
output = output.replace(
"((Dict (Hashtbl.create 0))) (a) ((List []))",
"(Env (Sx_types.make_env ())) (a) ((List []))",
)
return output

View File

@@ -618,20 +618,44 @@ let () =
match args with
| [f; (List items | ListRef { contents = items })] ->
List (List.map (fun x -> call_any f [x]) items)
| [_; Nil] -> List []
| _ -> raise (Eval_error "map: expected (fn list)"));
register "map-indexed" (fun args ->
match args with
| [f; (List items | ListRef { contents = items })] ->
List (List.mapi (fun i x -> call_any f [Number (float_of_int i); x]) items)
| [_; Nil] -> List []
| _ -> raise (Eval_error "map-indexed: expected (fn list)"));
register "filter" (fun args ->
match args with
| [f; (List items | ListRef { contents = items })] ->
List (List.filter (fun x -> sx_truthy (call_any f [x])) items)
| [_; Nil] -> List []
| _ -> raise (Eval_error "filter: expected (fn list)"));
register "for-each" (fun args ->
match args with
| [f; (List items | ListRef { contents = items })] ->
List.iter (fun x -> ignore (call_any f [x])) items; Nil
| _ -> raise (Eval_error "for-each: expected (fn list)"));
| [_; Nil] -> Nil (* nil collection = no-op *)
| _ ->
let types = String.concat ", " (List.map (fun v -> type_of v) args) in
raise (Eval_error (Printf.sprintf "for-each: expected (fn list), got (%s) %d args" types (List.length args))));
register "reduce" (fun args ->
match args with
| [f; init; (List items | ListRef { contents = items })] ->
List.fold_left (fun acc x -> call_any f [acc; x]) init items
| _ -> raise (Eval_error "reduce: expected (fn init list)"));
register "some" (fun args ->
match args with
| [f; (List items | ListRef { contents = items })] ->
(try List.find (fun x -> sx_truthy (call_any f [x])) items
with Not_found -> Bool false)
| [_; Nil] -> Bool false
| _ -> raise (Eval_error "some: expected (fn list)"));
register "every?" (fun args ->
match args with
| [f; (List items | ListRef { contents = items })] ->
Bool (List.for_all (fun x -> sx_truthy (call_any f [x])) items)
| [_; Nil] -> Bool true
| _ -> raise (Eval_error "every?: expected (fn list)"));
()

View File

@@ -422,7 +422,7 @@ and step_sf_deref args env kont =
(* cek-call *)
and cek_call f args =
(let a = (if sx_truthy ((is_nil (args))) then (List []) else args) in (if sx_truthy ((is_nil (f))) then Nil else (if sx_truthy ((let _or = (is_lambda (f)) in if sx_truthy _or then _or else (is_callable (f)))) then (cek_run ((continue_with_call (f) (a) ((Dict (Hashtbl.create 0))) (a) ((List []))))) else Nil)))
(let a = (if sx_truthy ((is_nil (args))) then (List []) else args) in (if sx_truthy ((is_nil (f))) then Nil else (if sx_truthy ((let _or = (is_lambda (f)) in if sx_truthy _or then _or else (is_callable (f)))) then (cek_run ((continue_with_call (f) (a) ((make_env ())) (a) ((List []))))) else Nil)))
(* reactive-shift-deref *)
and reactive_shift_deref sig' env kont =

View File

@@ -240,7 +240,7 @@ let type_of v = String (Sx_types.type_of v)
The transpiled CEK machine stores envs in dicts as Env values. *)
let unwrap_env = function
| Env e -> e
| _ -> raise (Eval_error "Expected env")
| v -> raise (Eval_error ("Expected env, got " ^ Sx_types.type_of v))
let env_has e name = Bool (Sx_types.env_has (unwrap_env e) (value_to_str name))
let env_get e name = Sx_types.env_get (unwrap_env e) (value_to_str name)

View File

@@ -792,6 +792,162 @@ def _sx_literal(v: object) -> str:
_cached_shell_static: dict[str, Any] | None = None
_cached_shell_comp_hash: str | None = None
def invalidate_shell_cache():
"""Call on component hot-reload to recompute shell statics."""
global _cached_shell_static, _cached_shell_comp_hash
_cached_shell_static = None
_cached_shell_comp_hash = None
def _get_shell_static() -> dict[str, Any]:
"""Compute and cache all shell kwargs that don't change per-request.
This is the expensive part: component dep scanning, serialization,
CSS class scanning, rule lookup, pages registry. All stable until
components are hot-reloaded.
"""
global _cached_shell_static, _cached_shell_comp_hash
from .jinja_bridge import components_for_page, css_classes_for_page, _component_env_hash
from .css_registry import lookup_rules, get_preamble, registry_loaded, store_css_hash
current_hash = _component_env_hash()
if _cached_shell_static is not None and _cached_shell_comp_hash == current_hash:
return _cached_shell_static
import time
t0 = time.monotonic()
from quart import current_app as _ca
from .jinja_bridge import client_components_tag, _COMPONENT_ENV, _CLIENT_LIBRARY_SOURCES
from .jinja_bridge import _component_env_hash
from .parser import serialize as _serialize
# Send ALL component definitions — the hash is stable per env so the
# browser caches them across all pages. Server-side expansion handles
# the per-page subset; the client needs the full set for client-side
# routing to any page.
parts = []
for key, val in _COMPONENT_ENV.items():
from .types import Island, Component, Macro
if isinstance(val, Island):
ps = ["&key"] + list(val.params)
if val.has_children: ps.extend(["&rest", "children"])
parts.append(f"(defisland ~{val.name} ({' '.join(ps)}) {_serialize(val.body, pretty=True)})")
elif isinstance(val, Component):
ps = ["&key"] + list(val.params)
if val.has_children: ps.extend(["&rest", "children"])
parts.append(f"(defcomp ~{val.name} ({' '.join(ps)}) {_serialize(val.body, pretty=True)})")
elif isinstance(val, Macro):
ps = list(val.params)
if val.rest_param: ps.extend(["&rest", val.rest_param])
parts.append(f"(defmacro {val.name} ({' '.join(ps)}) {_serialize(val.body, pretty=True)})")
all_parts = list(_CLIENT_LIBRARY_SOURCES) + parts
component_defs = "\n".join(all_parts)
component_hash = _component_env_hash()
# CSS: scan ALL components (not per-page) for the static cache
sx_css = ""
sx_css_classes = ""
if registry_loaded():
classes: set[str] = set()
from .types import Island as _I, Component as _C
for val in _COMPONENT_ENV.values():
if isinstance(val, (_I, _C)) and val.css_classes:
classes.update(val.css_classes)
classes.update(["bg-stone-50", "text-stone-900"])
rules = lookup_rules(classes)
sx_css = get_preamble() + rules
sx_css_classes = store_css_hash(classes)
pages_sx = _build_pages_sx(_ca.name)
_shell_cfg = _ca.config.get("SX_SHELL", {})
static = dict(
component_hash=component_hash,
component_defs=component_defs,
pages_sx=pages_sx,
sx_css=sx_css,
sx_css_classes=sx_css_classes,
sx_js_hash=_script_hash("sx-browser.js"),
body_js_hash=_script_hash("body.js"),
asset_url=_ca.config.get("ASSET_URL", "/static"),
head_scripts=_shell_cfg.get("head_scripts"),
inline_css=_shell_cfg.get("inline_css"),
inline_head_js=_shell_cfg.get("inline_head_js"),
init_sx=_shell_cfg.get("init_sx"),
body_scripts=_shell_cfg.get("body_scripts"),
)
t1 = time.monotonic()
import logging
logging.getLogger("sx.pages").info(
"[shell-static] computed in %.3fs, comp_defs=%d css=%d pages=%d",
t1 - t0, len(component_defs), len(sx_css), len(pages_sx))
_cached_shell_static = static
_cached_shell_comp_hash = current_hash
return static
async def _build_shell_kwargs(ctx: dict, page_sx: str, *,
meta_html: str = "",
head_scripts: list[str] | None = None,
inline_css: str | None = None,
inline_head_js: str | None = None,
init_sx: str | None = None,
body_scripts: list[str] | None = None) -> dict[str, Any]:
"""Compute all shell kwargs for sx-page-shell.
Static parts (components, CSS, pages) are cached. Only per-request
values (title, csrf) are computed fresh.
"""
static = _get_shell_static()
asset_url = get_asset_url(ctx) or static["asset_url"]
title = ctx.get("base_title", "Rose Ash")
csrf = _get_csrf_token()
kwargs: dict[str, Any] = dict(static)
kwargs.update(
title=_html_escape(title),
asset_url=asset_url,
meta_html=meta_html,
csrf=_html_escape(csrf),
)
# Per-page CSS: scan THIS page's classes and add to cached CSS
from .css_registry import scan_classes_from_sx, lookup_rules, registry_loaded
if registry_loaded() and page_sx:
page_classes = scan_classes_from_sx(page_sx)
if page_classes:
extra_rules = lookup_rules(page_classes)
if extra_rules:
kwargs["sx_css"] = static["sx_css"] + extra_rules
# Cookie-based component caching
client_hash = _get_sx_comp_cookie()
if not _is_dev_mode() and client_hash and client_hash == static["component_hash"]:
kwargs["component_defs"] = ""
# Per-call overrides
if head_scripts is not None:
kwargs["head_scripts"] = head_scripts
if inline_css is not None:
kwargs["inline_css"] = inline_css
if inline_head_js is not None:
kwargs["inline_head_js"] = inline_head_js
if init_sx is not None:
kwargs["init_sx"] = init_sx
if body_scripts is not None:
kwargs["body_scripts"] = body_scripts
return kwargs
async def sx_page(ctx: dict, page_sx: str, *,
meta_html: str = "",
head_scripts: list[str] | None = None,
@@ -799,109 +955,18 @@ async def sx_page(ctx: dict, page_sx: str, *,
inline_head_js: str | None = None,
init_sx: str | None = None,
body_scripts: list[str] | None = None) -> str:
"""Return a minimal HTML shell that boots the page from sx source.
The browser loads component definitions and page sx, then sx.js
renders everything client-side. CSS rules are scanned from the sx
source and component defs, then injected as a <style> block.
The shell is rendered from the ~shared:shell/sx-page-shell SX component
(shared/sx/templates/shell.sx).
"""
from .jinja_bridge import components_for_page, css_classes_for_page
from .css_registry import lookup_rules, get_preamble, registry_loaded, store_css_hash
# Per-page component bundle: this page's deps + all :data page deps
from quart import current_app as _ca
component_defs, component_hash = components_for_page(page_sx, service=_ca.name)
# Check if client already has this version cached (via cookie)
# In dev mode, always send full source so edits are visible immediately
client_hash = _get_sx_comp_cookie()
if not _is_dev_mode() and client_hash and client_hash == component_hash:
# Client has current components cached — send empty source
component_defs = ""
# Scan for CSS classes — only from components this page uses + page source
sx_css = ""
sx_css_classes = ""
sx_css_hash = ""
if registry_loaded():
classes = css_classes_for_page(page_sx, service=_ca.name)
# Always include body classes
classes.update(["bg-stone-50", "text-stone-900"])
rules = lookup_rules(classes)
sx_css = get_preamble() + rules
sx_css_hash = store_css_hash(classes)
sx_css_classes = sx_css_hash
asset_url = get_asset_url(ctx)
title = ctx.get("base_title", "Rose Ash")
csrf = _get_csrf_token()
# Dev mode: pretty-print page sx for readable View Source
if _is_dev_mode() and page_sx and page_sx.startswith("("):
from .parser import parse as _parse, serialize as _serialize
try:
page_sx = _serialize(_parse(page_sx), pretty=True)
except Exception as e:
import logging
logging.getLogger("sx").warning("Pretty-print page_sx failed: %s", e)
# Page registry for client-side routing
import logging
_plog = logging.getLogger("sx.pages")
from quart import current_app
pages_sx = _build_pages_sx(current_app.name)
_plog.debug("sx_page: pages_sx %d bytes for service %s", len(pages_sx), current_app.name)
if pages_sx:
_plog.debug("sx_page: pages_sx first 200 chars: %s", pages_sx[:200])
# Ensure page_sx is a plain str, not SxExpr — _build_component_ast
# parses SxExpr back into AST, which _arender then evaluates as HTML
# instead of passing through as raw content for the script tag.
"""Return a minimal HTML shell that boots the page from sx source."""
# Ensure page_sx is a plain str
if isinstance(page_sx, SxExpr):
page_sx = "".join([page_sx])
# Per-app shell config: check explicit args, then app config, then defaults
from quart import current_app as _app
_shell_cfg = _app.config.get("SX_SHELL", {})
if head_scripts is None:
head_scripts = _shell_cfg.get("head_scripts")
if inline_css is None:
inline_css = _shell_cfg.get("inline_css")
if inline_head_js is None:
inline_head_js = _shell_cfg.get("inline_head_js")
if init_sx is None:
init_sx = _shell_cfg.get("init_sx")
if body_scripts is None:
body_scripts = _shell_cfg.get("body_scripts")
shell_kwargs: dict[str, Any] = dict(
title=_html_escape(title),
asset_url=asset_url,
meta_html=meta_html,
csrf=_html_escape(csrf),
component_hash=component_hash,
component_defs=component_defs,
pages_sx=pages_sx,
page_sx=page_sx,
sx_css=sx_css,
sx_css_classes=sx_css_classes,
sx_js_hash=_script_hash("sx-browser.js"),
body_js_hash=_script_hash("body.js"),
)
if head_scripts is not None:
shell_kwargs["head_scripts"] = head_scripts
if inline_css is not None:
shell_kwargs["inline_css"] = inline_css
if inline_head_js is not None:
shell_kwargs["inline_head_js"] = inline_head_js
if init_sx is not None:
shell_kwargs["init_sx"] = init_sx
if body_scripts is not None:
shell_kwargs["body_scripts"] = body_scripts
return await render_to_html("shared:shell/sx-page-shell", **shell_kwargs)
kwargs = await _build_shell_kwargs(
ctx, page_sx, meta_html=meta_html,
head_scripts=head_scripts, inline_css=inline_css,
inline_head_js=inline_head_js, init_sx=init_sx,
body_scripts=body_scripts)
kwargs["page_sx"] = page_sx
return await render_to_html("shared:shell/sx-page-shell", **kwargs)
_SX_STREAMING_RESOLVE = """\

View File

@@ -342,6 +342,8 @@ def reload_if_changed() -> None:
_CLIENT_LIBRARY_SOURCES.clear()
_dirs_from_cache.clear()
invalidate_component_hash()
from .helpers import invalidate_shell_cache
invalidate_shell_cache()
# Reload SX libraries first (e.g. z3.sx) so reader macros resolve
for cb in _reload_callbacks:
cb()
@@ -360,6 +362,8 @@ def reload_if_changed() -> None:
from .ocaml_bridge import _bridge
if _bridge is not None:
_bridge._components_loaded = False
_bridge._shell_statics_injected = False
_bridge._helpers_injected = False
# Recompute render plans for all services that have pages
from .pages import _PAGE_REGISTRY, compute_page_render_plans

View File

@@ -159,6 +159,102 @@ class OcamlBridge:
await self._send(f'(aser-slot "{_escape(source)}")')
return await self._read_until_ok(ctx)
_shell_statics_injected: bool = False
async def _inject_shell_statics_locked(self) -> None:
"""Inject cached shell static data into kernel. MUST hold lock."""
if self._shell_statics_injected:
return
from .helpers import _get_shell_static
try:
static = _get_shell_static()
except Exception:
return # not ready yet (no app context)
# Define small values as kernel variables.
# Large blobs (component_defs, pages_sx, init_sx) use placeholders
# at render time — NOT injected here.
for key in ("sx_css", "component_hash", "sx_css_classes", "asset_url",
"sx_js_hash", "body_js_hash"):
val = static.get(key, "")
if val is None:
val = ""
var = f"__shell-{key.replace('_', '-')}"
defn = f'(define {var} "{_escape(str(val))}")'
try:
await self._send(f'(load-source "{_escape(defn)}")')
await self._read_until_ok(ctx=None)
except OcamlBridgeError:
pass
# Also inject list/nil values
for key in ("head_scripts", "inline_css", "inline_head_js", "body_scripts"):
val = static.get(key)
var = f"__shell-{key.replace('_', '-')}"
if val is None:
defn = f'(define {var} nil)'
elif isinstance(val, list):
items = " ".join(f'"{_escape(str(v))}"' for v in val)
defn = f'(define {var} (list {items}))'
else:
defn = f'(define {var} "{_escape(str(val))}")'
try:
await self._send(f'(load-source "{_escape(defn)}")')
await self._read_until_ok(ctx=None)
except OcamlBridgeError:
pass
self._shell_statics_injected = True
_logger.info("Injected shell statics into OCaml kernel")
async def sx_page_full(
self,
page_source: str,
shell_kwargs: dict[str, Any],
ctx: dict[str, Any] | None = None,
) -> str:
"""Render full page HTML in one OCaml call: aser-slot + shell render.
Static data (component_defs, CSS, pages_sx) is pre-injected as
kernel vars on first call. Per-request command sends only small
values (title, csrf) + references to the kernel vars.
"""
await self._ensure_components()
async with self._lock:
await self._inject_helpers_locked()
await self._inject_shell_statics_locked()
# Large blobs (component_defs, pages_sx, init_sx) use placeholders.
# OCaml renders the shell with short tokens; Python splices in
# the real values. This avoids piping ~1MB through stdin/stdout.
PLACEHOLDER_KEYS = {"component_defs", "pages_sx", "init_sx"}
placeholders = {}
static_keys = {"component_hash", "sx_css_classes", "asset_url",
"sx_js_hash", "body_js_hash", "sx_css",
"head_scripts", "inline_css", "inline_head_js", "body_scripts"}
parts = [f'(sx-page-full "{_escape(page_source)}"']
for key, val in shell_kwargs.items():
k = key.replace("_", "-")
if key in PLACEHOLDER_KEYS:
token = f"__SLOT_{key.upper()}__"
placeholders[token] = str(val) if val else ""
parts.append(f' :{k} "{token}"')
elif key in static_keys:
parts.append(f' :{k} __shell-{k}')
elif val is None:
parts.append(f' :{k} nil')
elif isinstance(val, bool):
parts.append(f' :{k} {"true" if val else "false"}')
elif isinstance(val, list):
items = " ".join(f'"{_escape(str(v))}"' for v in val)
parts.append(f' :{k} ({items})')
else:
parts.append(f' :{k} "{_escape(str(val))}"')
parts.append(")")
cmd = "".join(parts)
await self._send(cmd)
html = await self._read_until_ok(ctx)
# Splice in large blobs
for token, blob in placeholders.items():
html = html.replace(token, blob)
return html
async def _inject_helpers_locked(self) -> None:
"""Inject page helpers into the kernel. MUST be called with lock held."""
if self._helpers_injected:

View File

@@ -76,7 +76,7 @@
(uv-found (some (fn (u) (= (get u "name") name)) upvals)))
(if uv-found
(let ((uv (first (filter (fn (u) (= (get u "name") name)) upvals))))
{:type "upvalue" :index (get uv "index")})
{:type "upvalue" :index (get uv "uv-index")})
;; Look in parent
(let ((parent (get scope "parent")))
(if (nil? parent)
@@ -91,7 +91,8 @@
(append! (get scope "upvalues")
{:name name
:is-local (= (get parent-result "type") "local")
:index (get parent-result "index")})
:index (get parent-result "index")
:uv-index uv-idx})
{:type "upvalue" :index uv-idx})
;; Let scope — pass through (same frame)
parent-result))))))))))))

View File

@@ -1132,6 +1132,9 @@
(= name "false") false
(= name "nil") nil
:else (error (str "Undefined symbol: " name)))))
;; Warn when a ~component symbol resolves to nil (likely missing)
(when (and (nil? val) (starts-with? name "~"))
(debug-log "Component not found:" name))
(make-cek-value val env kont)))
;; --- Keyword → string ---
@@ -1551,7 +1554,7 @@
(cond
(nil? f) nil
(or (lambda? f) (callable? f))
(cek-run (continue-with-call f a (dict) a (list)))
(cek-run (continue-with-call f a (make-env) a (list)))
:else nil))))
;; reactive-shift-deref: the heart of deref-as-shift
@@ -2257,9 +2260,7 @@
(env-bind! local "children" children))
(make-cek-state (component-body f) local kont))
:else (error (str "Not callable: " (inspect f)
(when raw-args
(str " in (" (inspect (first raw-args)) " ...)")))))))
:else (error (str "Not callable: " (inspect f))))))

View File

@@ -182,11 +182,11 @@ def create_app() -> "Quart":
from quart import request, make_response
from shared.browser.app.utils.htmx import is_htmx_request
from shared.sx.jinja_bridge import get_component_env, _get_request_context
from shared.sx.async_eval import async_eval_slot_to_sx
from shared.sx.types import Symbol, Keyword
from shared.sx.helpers import full_page_sx, oob_page_sx, sx_response
from shared.sx.pages import get_page_helpers
from shared.sx.page import get_template_context
import os
path = request.path
content_ast = [
@@ -199,7 +199,15 @@ def create_app() -> "Quart":
ctx = _get_request_context()
try:
content_sx = await async_eval_slot_to_sx(content_ast, env, ctx)
if os.environ.get("SX_USE_OCAML") == "1":
from shared.sx.ocaml_bridge import get_bridge
from shared.sx.parser import serialize
bridge = await get_bridge()
sx_text = serialize(content_ast)
content_sx = await bridge.aser_slot(sx_text, ctx={"_helper_service": "sx"})
else:
from shared.sx.async_eval import async_eval_slot_to_sx
content_sx = await async_eval_slot_to_sx(content_ast, env, ctx)
except Exception:
from shared.browser.app.errors import _sx_error_page
html = _sx_error_page("404", "NOT FOUND",

View File

@@ -212,16 +212,27 @@ async def eval_sx_url(raw_path: str) -> Any:
serialize(oob_ast), ctx=ocaml_ctx))
return sx_response(content_sx)
else:
# Full page: single-pass — layout + content in ONE aser_slot
# Full page: single OCaml call — aser-slot + shell render
full_ast = [
Symbol("~shared:layout/app-body"),
Keyword("content"), wrapped_ast,
]
body_sx = SxExpr(await bridge.aser_slot(
serialize(full_ast), ctx=ocaml_ctx))
page_source = serialize(full_ast)
# Pre-compute shell kwargs in Python
import time as _time
_t0 = _time.monotonic()
from shared.sx.helpers import _build_shell_kwargs
tctx = await get_template_context()
return await make_response(
await sx_page(tctx, body_sx), 200)
_t1 = _time.monotonic()
shell_kwargs = await _build_shell_kwargs(tctx, page_source)
_t2 = _time.monotonic()
html = await bridge.sx_page_full(
page_source, shell_kwargs, ctx=ocaml_ctx)
_t3 = _time.monotonic()
logger.info("[sx-page-full-py] ctx=%.3fs kwargs=%.3fs ocaml=%.3fs total=%.3fs",
_t1-_t0, _t2-_t1, _t3-_t2, _t3-_t0)
return await make_response(html, 200)
else:
content_sx = await _eval_slot(wrapped_ast, env, ctx)
except Exception as e: