Isolated the dict-set!/reduce error to complex island body parsing,
not the reactive system or library functions. Proven working:
- break-lines inside effect ✓
- canvas.measureText inside effect ✓
- pretext-layout-lines inside effect ✓
- signal + slider + reactive update ✓
The error triggers only with large island bodies (many ~tw spreads,
nested controls). This is a component definition parser bug in the
WASM kernel, not a Pretext or reactive system issue.
Current island: minimal working version with effect-based layout,
slider control, and innerHTML rendering. Ready for incremental
expansion once the parser size limit is identified.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- 7 new tests in computed-ho-forms suite: computed with map, reduce,
for-each, nested map, dict creation, signal updates. All pass on
OCaml and WASM sandbox.
- Removed standalone pretext-position-line and pretext-layout-lines
from pretext-demo.sx — now in text-layout library only
- Root cause of island error: pretext-demo.sx had old define with
(reduce + 0 lwid) that the server serialized into component defs,
overriding the library's sum-loop version
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Added sublist helper (portable list extraction, avoids 3-arg slice
which fails in browser WASM kernel)
- Replaced reduce + 0 lwid with manual sum loop (reduce has browser
compat issues with dict-set! error in call stack)
- Imperative DOM update via effect for clean paragraph re-rendering
on signal changes (clear container, create new spans)
- String slice in hyphenate-word kept (works on strings)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The (let ((lines (deref layout))) ...) pattern captured the layout value
once at island initialization. Replacing with (deref layout) inline in the
DOM expressions makes the reactive system track the dependency and
re-render when signals change.
Sliders and algorithm toggle now trigger layout recomputation and DOM
update. Remaining: reactive DOM patching for absolutely-positioned spans
creates visual artifacts (old spans persist). Needs keyed list or full
container replacement.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: sx_insert_near placed break-lines-greedy, pretext-position-line,
pretext-layout-lines OUTSIDE the define-library begin block. The bytecode
compiler only compiles forms inside begin as STORE_GLOBAL — forms outside
are invisible to the browser VM.
Fix: moved all function definitions inside (begin ...) of (define-library).
Bytecode now includes all 17 functions (11K compiled, was 9K).
Browser load-sxbc: simplified VmSuspended handling — just catch and
continue, since STORE_GLOBAL ops already ran before the import OP_PERFORM.
sync_vm_to_env copies them to global_env.
Island now calls break-lines and pretext-layout-lines from bytecode-compiled
library — runs on VM, not CEK interpreter.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- text-layout.sx added to WASM bytecode pipeline (9K compiled)
- Fix multi-list map calls (map-indexed + nth instead of map fn list1 list2)
- pretext-layout-lines and pretext-position-line moved to library exports
- Browser load-sxbc: handle VmSuspended for import, copy library exports
to global_env after module load (define-library export fix)
- compile-modules.js: text-layout in SOURCE_MAP, FILES, and entry deps
- Island uses library functions (break-lines, pretext-layout-lines)
instead of inlining — runs on bytecode VM when exports resolve
Known issue: define-library exports don't propagate to browser global env
yet. The load-sxbc import suspension handler resumes correctly but
bind_import_set doesn't fire. Needs deeper investigation into how the
WASM kernel's define-library registers exports vs how other libraries
(adapter-html, tw) make their exports available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move all layout functions inside defisland body (browser can't access
top-level defines from component defs bundle)
- Use div placeholder with data-sx-island attr (matches island root tag)
- Rename pretext-island.sx → pretext-client.sx for alphabetical load order
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
defisland ~pretext-demo/live — same Knuth-Plass algorithm running in
the browser with canvas.measureText for pixel-perfect font metrics.
- Width slider (200-700px), font size slider (10-24px)
- Greedy vs Knuth-Plass toggle button
- Reactive re-layout on every control change
- All layout functions inlined in the island (no library deps)
- Perfectly straight right edges — browser measures AND renders
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parser/compiler/runtime for focus command. Tokenizer: focus, blur,
precedes, follows, ignoring, case keywords. Test spec: per-test
failure output for diagnosis.
374/831 (45%)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
stream-colors dict had green/blue keys but data used emerald/violet — all three
slots now render with correct Tailwind color classes. Platform: resolveSuspense
must not exist on Sx until boot completes, otherwise bootstrap __sxResolve calls
it before web stack loads and resolves silently fail. Moved to post-boot setup
so all pre-boot resolves queue in __sxPending and drain correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Streaming chunked transfer with shell-first suspense and resolve scripts.
Hyperscript parser/compiler/runtime expanded for conformance. WASM static
assets added to OCaml host. Playwright streaming and page-level test suites.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server (sx_server.ml):
- eval_with_io: CEK evaluator with IO suspension handling (io-sleep, import)
- io-sleep platform primitive: raises CekPerformRequest, resolved by eval_with_io
- Streaming render uses eval_with_io for data + content evaluation
- Data items with "delay" field sleep before resolving (async streaming)
- Removed hardcoded streaming-demo-data — application logic belongs in .sx
Application (streaming-demo.sx):
- streaming-demo-data defined in SX: 3 items with 1s/3s/5s delays
- Each item has delay, stream-id, and display data fields
- Shell renders instantly, slots fill progressively as IO completes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The lake preview ("the joy of sx") was flashing because:
1. SSR renders preview in lake (server-only guard)
2. replaceChildren swaps island DOM (lake now empty)
3. rebuild-preview effect was either skipped or deferred (rAF/setTimeout)
4. Browser paints empty lake → visible flash
Fix: first-run effect uses queueMicrotask instead of schedule-idle.
Microtasks fire after the current synchronous code (including
replaceChildren) but BEFORE the browser paints. The lake is filled
before any frame renders with empty content.
Also restored the (when (not (client?))) lake guard — the client
can't render steps-to-preview (returns raw SX expressions that
render-to-dom shows as source text, not rendered HTML).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: the lake had (when (not (client?)) ...) guard — SSR rendered
"the joy of sx" preview but client skipped it. replaceChildren swapped
in an empty lake. The rebuild-preview effect was skipped (first-run
optimization), so the preview stayed blank for ~500ms.
Fix: remove the client? guard so the lake renders on both server and
client. The template's steps-to-preview produces the initial preview.
The effect only fires on subsequent step changes (not first run).
Test: replaced MutationObserver approach with screenshot comparison.
Loads page with JS blocked (pure SSR), takes screenshot. Loads with JS
(hydration), takes screenshot. Compares pixels. Any visual difference
fails the test.
Result: "No visual flash: screenshots identical" — passes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: the stepper's rebuild effect (update-code-highlight,
rebuild-preview) fired immediately on hydration via schedule-idle,
modifying the DOM after replaceChildren swapped in identical content.
This caused a visible text change after the initial frame.
Fix: track initial step-idx value and first-run flag. Skip the
effect on first run if the current step matches the SSR state
(from cookie). The effect only fires on actual user interaction.
Result: SSR and hydrated text content are identical. replaceChildren
swaps DOM nodes but the visual content doesn't change. Zero flash.
Test: "No clobber: clean" — 0 text changes during hydration.
All 8 home features pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three changes to eliminate the stepper flash:
1. home-stepper.sx: server path reads cookie via (get-cookie) for
step-idx initial value. Client path reads document.cookie via
def-store. Both default to 0 when no cookie exists.
2. sx_server.ml: bypass response cache when sx-home-stepper cookie
is present. Render on main thread (not worker) so get-cookie
sees the parsed request cookies.
3. site-full.spec.js: flash detection test sets cookie=7 via
Playwright context, checks SSR HTML matches hydrated state.
Test: "No flash: SSR=7 hydrated=7 (cookie=7)" — passes.
Tested on fresh stack=site server subprocess.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously SSR rendered at step 16 (hardcoded) but client initialized
from cookie, causing a flash from 16 to the cookie value on return visits.
Fix: Both SSR and client default to step 0. The def-store initializer
reads the cookie for the client's initial value. Return visits show
a progressive fill (0 → cookie value) instead of a jarring state jump.
- step-idx default: (signal 0) in both SSR and client paths
- def-store: reads sx-home-stepper cookie for initial value, defaults to 0
- Removed redundant post-hydration cookie reset block
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reset to last known-good state (908f4f80) where links, stepper, and
islands all work, then recovered all hyperscript implementation,
conformance tests, behavioral tests, Playwright specs, site sandbox,
IO-aware server loading, and upstream test suite from f271c88a.
Excludes runtime changes (VM resolve hook, VmSuspended browser handler,
sx_ref.ml guard recovery) that need careful re-integration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ~hyperscript/example component: shows "Try it" button with _= attr
for all on-click examples, source pre wraps long lines
- Added CSS for .active/.light/.dark demo classes with !important
to override Tailwind hover states
- Added #target div for the "put into" example
- Replaced broken examples (items, ~card, js-date-now) with
self-contained ones that use available primitives
- Repeat example left in with note: continuation after loop pending
- New test suite io-suspension-continuation documenting the stub VM
bug: outer do continuation lost after suspension/resume completes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Break up the 1735-line handle_tool match into 45 individual handler functions
with hashtable-based dispatch. Add mtime-based file parse caching (AST + CST),
consolidated run_command helper replacing 9 bare open_process_in patterns,
require_file/require_dir input validation, and pagination (limit/offset) for
sx_find_across, sx_comp_list, sx_comp_usage. Also includes pending VM changes:
rest-arity support, hyperscript parser, compiler/transpiler updates.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sx/sx/hyperscript.sx — _hyperscript playground page at
/sx/(applications.(hyperscript))
- compile-result defcomp (SSR compilation display)
- pipeline documentation (tokenize → parse → compile)
- example showcases with pre-compiled output
- sx-post form → handler for interactive compilation
sx/sx/handlers/hyperscript-api.sx — POST handler:
/sx/(applications.(hyperscript.(api.compile)))
Accepts source param, returns compiled SX + parse tree HTML
NOTE: hs-parse returns (do) in server context — JIT/CEK runtime
issue where parser closures don't evaluate correctly. Works in
test runner (3127/3127). Investigating separately.
sx_server.ml — url_decode fix: decode + as space in form data
Standard application/x-www-form-urlencoded uses + for spaces.
Nav: _hyperscript added to Applications section.
Config: handler:hs- prefix added for handler dispatch.
3127/3127 tests, zero regressions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the VM or CEK hits an undefined symbol, it checks a symbol→library
index (built from manifest exports at boot), loads the library that
exports it, and returns the value. Execution continues as if the module
was always loaded. No import statements, no load-library! calls, no
Suspense boundaries — just call the function.
This is the same mechanism as IO suspension for data fetching. The
programmer doesn't distinguish between calling a local function and
calling one that needs its module fetched first. The runtime treats
code as just another resource.
Implementation:
- _symbol_resolve_hook in sx_types.ml — called by env_get_id (CEK path)
and vm_global_get (VM path) when a symbol isn't found
- Symbol→library index built from manifest exports in sx-platform.js
- __resolve-symbol native calls __sxLoadLibrary, module loads, symbol
appears in globals, execution resumes
- compile-modules.js extracts export lists into module-manifest.json
- Playground page demonstrates: (freeze-scope) triggers freeze.sxbc
download transparently on first use
2650/2650 tests pass.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- load-library! native: islands can declare module dependencies at
hydration time, triggering on-demand .sxbc loading
- JIT compiler lazy-load: compiler.sxbc loads via setTimeout after boot,
eliminating "JIT: compiler not loaded" errors
- _import_hook on sx_types: infrastructure for hosts to resolve import
suspensions inside eval_expr (server wiring deferred to Step 8)
- Playground page (/sx/(tools.(playground))): REPL island that lazy-loads
the compiler module when navigated to — demonstrates the full
lazy loading pipeline
Known remaining issues:
- SPA navigation broken for pages using let-match (orchestration.sx,
router.sx) — bytecode compiler doesn't handle let-match special form
- Server-side "IO suspension in non-IO context" during http_load_files —
needs cek_run import handling (deferred to Step 8)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 2 of architecture roadmap. The OCaml HTTP server is now
generic — all sx_docs-specific values (layout components, path prefix,
title, warmup paths, handler prefixes, CSS/JS, client libs) move into
sx/sx/app-config.sx as a __app-config dict. Server reads config at
startup with hardcoded defaults as fallback, so it works with no config,
partial config, or full config.
Removed: 9 demo data stubs, stepper cookie cache logic, page-functions.sx
directory heuristic. Added: 29-test server config test suite covering
standard, custom, no-config, and minimal-config scenarios.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 Step 1 of the architecture roadmap. The old cssx.sx
(cssx-resolve, cssx-process-token, cssx-template, old tw function)
is superseded by the ~tw component system in tw.sx.
- Delete shared/sx/templates/cssx.sx
- Remove cssx.sx from all load lists (sx_server.ml, run_tests.ml,
mcp_tree.ml, compile-modules.js, bundle.sh, sx-build-all.sh)
- Replace (tw "tokens") inline style calls with (~tw :tokens "tokens")
in layouts.sx and not-found.sx
- Remove _css-hash / init-css-tracking / SX-Css header plumbing
(dead code — ~tw/flush + flush-collected-styles handle CSS now)
- Remove sx-css-classes param and meta tag from shell template
- Update stale data-cssx references to data-sx-css in tests
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Reverted render-dom-lake SSR reuse — it broke OOB swaps (claimed
old lake elements during morph, stale content in copyright). The
framework's morphing handles lake updates correctly already.
Stepper: lake passes nil on client (prevents raw SX flash), effect
always calls rebuild-preview (no initial-render flag needed). Server
renders the expression for SSR; client rebuilds via render-to-dom
after boot when ~tw is available.
Removed initial-render dict flag — unnecessary complexity.
Copyright route not updating is a pre-existing issue: render-dom-island
renders the header island inline during OOB content rendering (sets
island-hydrated mark), but the copyright lake content doesn't reflect
the new path. Separate investigation needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three fixes:
1. Framework: render-dom-lake preserves SSR elements during hydration.
When client-side render-to-dom encounters a lake with an existing
DOM element (from SSR), it reuses that element instead of creating
a new one. This prevents the SSR HTML from being replaced with
unresolvable raw SX expressions (~tw calls).
2. Stepper: skip rebuild-preview on initial hydration. Uses a non-
reactive dict flag (not a signal) to avoid triggering the effect
twice. On first run, just initializes the DOM stack from the
existing SSR content by computing open-element depth from step
types and walking lastElementChild.
3. Stepper: rebuild-preview computes correct DOM stack after re-render.
Same depth computation + DOM walk approach. This fixes the bug where
do-step after do-back would append elements to the wrong parent
(e.g. "sx" span outside h1).
Also: increased code view font-size from 0.5rem to 0.85rem.
Playwright tests:
- lake never shows raw SX during hydration (mutation observer)
- back 6 + forward 6 keeps all 4 spans inside h1
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two bugs fixed:
1. Hydration flash: The effect's schedule-idle called rebuild-preview
on initial hydration, which cleared the SSR HTML and re-rendered
(flashing raw SX source or blank). Fix: skip rebuild-preview on
initial render — the SSR content is already correct. Only rebuild
when stepping.
2. Stepping structure: After do-back calls rebuild-preview, the DOM
stack was reset to just [container]. Subsequent do-step calls
appended elements to the wrong parent (e.g. "sx" span outside h1).
Fix: compute the correct stack depth by replaying open/close step
counts, then walk the rendered DOM tree that many levels deep via
lastElementChild to find the actual DOM nodes.
Proven by harness test: compute-depth returns 3 at step 10 (inside
div > h1 > span), 2 at step 8 (inside div > h1), 0 at step 16 (all
closed).
Playwright test: lake never shows raw SX (~tw, :tokens) during
hydration — now passes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: default step-idx was 9, but the expression has 16 steps.
At step 9, only "the joy" + empty emerald span renders. Changed default
to 16 so all four words display after hydration.
Reverted mutable-list changes — (list) already creates ListRef in the
OCaml kernel, so append! works correctly with plain (list).
Added spec/tests/test-stepper.sx (7 tests) proving the split-tag +
steps-to-preview pipeline works correctly at each step boundary.
Updated Playwright stepper.spec.js with four tests:
- no raw SX visible after hydration
- default view shows all four words
- all spans inside h1
- stepping forward renders styled text
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The stepper's split-tag and steps-to-preview used (list) with append!,
but in the WASM kernel (list) creates immutable List values — append!
returns a new list without mutating, so children accumulate nowhere.
Changed all accumulator initializations to (mutable-list):
- split-tag: cch, cat, spreads
- steps-to-preview: bc-loop inner children, initial call
- result and tokens lists in the parsing setup
Also includes WASM rebuild with append! primitive and &rest fixes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
{(~tw ...)} is invalid — dict keys must be keywords/strings/symbols.
Changed to (make-spread (~tw ...)) which evaluates the component first.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Parse Cookie header in OCaml HTTP server for get-cookie primitive
- Stepper saves step-idx to cookie via host-set! FFI on click
- Stepper restores from cookie: get-cookie on server, host-get FFI on client
- Cache key includes stepper cookie value to avoid stale SSR
- registerNative: also update Sx_primitives table for CALL_PRIM dispatch
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- spec-explore-define uses serialize (full body) instead of signature
- _spec-search-dirs expanded: spec/, web/, lib/, shared/sx/ref/, sx/, sxc/, shared/sx/templates/
- explore() works with any filename, not just nav spec items
- Playwright tests use flexible regex for line/define counts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fix 3 OCaml bugs that caused spec explorer to hang:
- sx_types: inspect outputs quoted string for SxExpr (not bare symbol)
- sx_primitives: serialize/to_string extract SxExpr/RawHTML content
- sx_render: handle SxExpr in both render-to-html paths
Restructure spec explorer for performance:
- Lightweight overview: name + kind only (was full source for 141 defs)
- Drill-in detail: click definition → params, effects, signature
- explore() page function accepts optional second arg for drill-in
- spec() passes through non-string slugs from nested routing
Fix aser map result wrapping:
- aser-special map now wraps results in fragment (<> ...) via aser-fragment
- Prevents ((div ...) (div ...)) nested lists that caused client "Not callable"
5 Playwright tests: overview load, no errors, SPA nav, drill-in detail+params
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The ~geography/scopes-demo-example component had (&key demo code), and
its body used (code code) — the parameter shadowed the HTML tag, causing
"Undefined symbol: code" at render time. Renamed to (&key demo src).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Handler source is now SX (from defhandler forms), not Python.
Default handler-lang changed from "python" to "sx".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
highlight returns (code (span ...)) elements which render-to-html
handles but aser doesn't. OOB responses go through aser. Plain text
for now — highlighting needs to happen client-side or aser needs
HTML tag support.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Now calls (highlight text) to produce syntax-colored SX output.
OCaml-side aser has full env so code/span tags resolve correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
