rose-ash/plans/dream-on-sx.md

Dream-on-SX: OCaml's Dream web framework on the SX CEK

[activated — target user confirmed; gated only on ocaml-on-sx]

Carved out of plans/ocaml-on-sx.md. The OCaml-on-SX plan was scoped down to substrate validation + HM + reference oracle (Phases 1–5 + minimal stdlib slice). Dream is the practical alternative-stack story — the opposite framing — and is now the chosen framework layer for the rose-ash host: the decision is to move off Quart and adopt Dream (not Quart) as the ergonomic HTTP front door over the native SX server. plans/host-on-sx.md Phase 4 is the concrete consumer that pulls Dream.

Target user — CONFIRMED. The earlier "needs a concrete target user" condition is met: rose-ash itself is the user — the subsystems (feed/acl/mod/commerce/identity/…) need an ergonomic HTTP front door, and the project owner has chosen Dream over Quart for it. This plan is no longer cold; it is gated, not deferred.

Do not start without (the one remaining gate):

1. OCaml-on-SX Phases 1–5 + Phase 6 minimal stdlib green. (As of writing ocaml-on-sx is at 480/480 and advancing — verify its scoreboard covers Phases 1–5 + the stdlib slice before starting.)

Until that gate is green, the native server (host-on-sx Phases 1–3) carries the host; do not block host migration on Dream.

Why this might be worth doing (when the time comes)

Dream is the cleanest middleware-shaped HTTP framework in any language:

• handler = request -> response promise
• middleware = handler -> handler
• m1 @@ m2 @@ handler — left-fold composition

It maps onto SX with almost no impedance — @@ is function composition, request → response promise is (perform (:http-respond ...)), middleware chain is plain SX function composition. So the integration cost is low if the OCaml-on-SX foundation is in place.

The user-facing story: rose-ash users who'd never touch s-expressions might write Dream/OCaml apps that integrate with the same federation, auth, and storage primitives. Demo: a Dream app serving sx.rose-ash.com — the framework that describes the runtime it runs on.

Dream semantic mappings

Dream construct	SX mapping
handler = request -> response promise	(fn (req) (perform (:http-respond ...)))
middleware = handler -> handler	(fn (next) (fn (req) ...))
Dream.router [routes]	(ocaml-dream-router routes) — dispatch on method+path
Dream.get "/path" h	route record {:method "GET" :path "/path" :handler h}
Dream.scope "/p" [ms] [rs]	prefix mount with middleware chain
Dream.param req "name"	path param extracted during routing
m1 @@ m2 @@ handler	(m1 (m2 handler)) — left-fold composition
Dream.session_field req "k"	(perform (:session-get req "k"))
Dream.set_session_field req "k" v	(perform (:session-set req "k" v))
Dream.flash req	(perform (:flash-get req))
Dream.form req	(perform (:form-parse req)) — returns Ok/Error ADT
Dream.websocket handler	(perform (:websocket handler))
Dream.run handler	starts SX HTTP server with handler as root

Roadmap

The five types: request, response, handler = request -> response, middleware = handler -> handler, route. Everything else is a function over these.

• Core types in lib/dream/types.sx: request/response records, route record.
• Router in lib/dream/router.sx: - dream-get path handler, dream-post path handler, etc. for all HTTP methods. - dream-scope prefix middlewares routes — prefix mount with middleware chain. - dream-router routes — dispatch tree, returns handler; no match → 404. - Path param extraction: :name segments, ** wildcard. - dream-param req name — retrieve matched path param.
• Middleware in lib/dream/middleware.sx: - dream-pipeline middlewares handler — compose middleware left-to-right. - dream-no-middleware — identity. - Logger: (dream-logger next req) — logs method, path, status, timing. - Content-type sniffer.
• Sessions in lib/dream/session.sx: - Cookie-backed session middleware. - dream-session-field req key, dream-set-session-field req key val. - dream-invalidate-session req.
• Flash messages in lib/dream/flash.sx: - dream-flash-middleware — single-request cookie store. - dream-add-flash-message req category msg. - dream-flash-messages req — returns list of (category, msg).
• Forms + CSRF in lib/dream/form.sx: - [x] dream-form req — returns (Ok fields) or (Err :csrf-token-invalid). - [x] dream-multipart req — multipart form data (in-memory, not yet streaming). - [x] CSRF middleware: stateless signed tokens, session-scoped. - [x] dream-csrf-tag req — returns hidden input fragment for SX templates.
• WebSockets in lib/dream/websocket.sx: - dream-websocket handler — upgrades request; handler (fn (ws) ...). - dream-send ws msg, dream-receive ws, dream-close ws.
• Static files: dream-static root-path — serves files, ETags, range requests.
• dream-run: wires root handler into SX's perform (:http-listen ...).
• Demos in lib/dream/demos/: - hello.ml → lib/dream/demos/hello.sx: "Hello, World!" route. - counter.ml → lib/dream/demos/counter.sx: in-memory counter with sessions. - chat.ml → lib/dream/demos/chat.sx: multi-room WebSocket chat. - todo.ml → lib/dream/demos/todo.sx: CRUD list with forms + CSRF.
• Tests in lib/dream/tests/: routing dispatch, middleware composition, session round-trip, CSRF accept/reject, flash read-after-write — 60+ tests.

Stdlib additions Dream will need

Dream pushes beyond OCaml-on-SX's Phase 6 minimal stdlib slice. When this plan activates, OCaml-on-SX gets a follow-on phase that adds at minimum:

• Bytes (binary buffers — request bodies, websocket frames)
• Buffer (mutable string building)
• Format (full pretty-printer, not just Printf.sprintf)
• More String (index_opt, contains, starts_with, ends_with, replace_all)
• Sys (argv, getenv_opt, getcwd)
• Hashtbl extensions (iter, fold, length, remove)
• Map.Make / Set.Make functors

Confirm scope before starting; some of these may be addable as Dream-internal helpers rather than full stdlib modules.

Ground rules

• Scope: only lib/dream/** and plans/dream-on-sx.md. Plus the stdlib additions listed above which land in lib/ocaml/runtime.sx.
• Hard prerequisite: OCaml-on-SX Phases 1–5 + Phase 6 minimal stdlib. Verify scoreboard before starting.
• SX files: sx-tree MCP tools only.
• Don't reinvent the SX HTTP server. Dream wraps the existing perform (:http-listen ...) — it does not implement its own listener loop.

Progress log

• 2026-06-07 — Core types (lib/dream/types.sx, 41 tests). OCaml gate verified green (scoreboard 480/480, Phases 1–5 + Phase 6 stdlib). Dream is implemented in plain SX over the CEK — keywords are strings, so headers are dicts with lowercased string keys (:content-type == "content-type"). request (method/target/path/ query/headers/body/params), response (status/headers/body), route records with constructors + accessors; smart response constructors (html/text/json/empty/ not-found/redirect); dream-coerce-response wraps bare strings; query-string parsing. Conformance runner lib/dream/conformance.sh modelled on flow's.
• 2026-06-07 — Router (lib/dream/router.sx, 27 tests). dream-get/post/put/ delete/patch/head/options/any route constructors; dream-router flattens routes (incl. nested scopes) and dispatches by method+path, first-match-wins, 404 on no match. Path matching is recursive over /-split segments: literal, :name binds a param, ** catch-all binds remaining path under key "**". Trailing slashes and query strings are ignored for routing. dream-scope prefix mws routes prepends the prefix and folds the middleware chain (m1 @@ m2 @@ h, first = outermost) onto each route's handler; nests correctly (inner mw innermost). Shared dr/apply-middlewares fold will back dream-pipeline.
• 2026-06-07 — Middleware (lib/dream/middleware.sx, 20 tests). dream-pipeline (reuses dr/apply-middlewares), dream-no-middleware identity. dream-logger-with clock sink is the testable core (records {:method :path :status :elapsed}); dream-logger wires it to (perform (:dream-clock)) / (perform (:dream-log …)); dream-log-line formats one line. dream-content-type sniffs body (<→html, {/[→json, else text) only when the handler left Content-Type unset. Bonus dream-set-header and dream-tap-request combinators.
• 2026-06-07 — Sessions (lib/dream/session.sx, 30 tests). Solved the request→response mutation-visibility problem the way Dream does: the cookie carries only a session id; fields live in an injectable back-end store (the mapping table's (perform (:session-get …))). dream-memory-sessions is an in-memory store built on a set!-mutated captured let binding (no ref/atom in base env); dream-perform-sessions is the production back-end. dream-sessions backend middleware reads/creates the id, attaches {:sid :io} to the request, and emits a Set-Cookie (HttpOnly, SameSite=Lax) only for new sessions. Handler API: dream-session-field / dream-set-session-field / dream-session-all / dream-invalidate-session / dream-session-id. Also added shared cookie infra (dr/parse-cookies, dream-cookie(s), dr/build-cookie, dream-set-cookie, dream-resp-cookies, dream-drop-cookie) — outgoing cookies accumulate in a :set-cookies list on the response so multiple Set-Cookie headers don't collide; reused by flash + CSRF. Full counter round-trip verified across three requests.
• 2026-06-07 — Flash (lib/dream/flash.sx, 14 tests). dream-flash middleware: decodes the incoming dream.flash cookie into the request, gives the handler a mutable outbox cell (dr/flash-box, the same set!-captured-let trick), then on response writes the outbox as a fresh flash cookie, or drops the cookie (Max-Age=0) when there were incoming messages but no new ones — so messages show exactly once. Handler API: dream-add-flash-message / dream-flash-messages (returns the PREVIOUS request's messages) / dream-flash-of (by category) / accessors. Cookie codec percent-escapes the |/~/% separators so categories/messages round-trip. Read-after-write verified across request boundaries incl. multi-category.
• 2026-06-07 — Forms + CSRF (urlencoded) (lib/dream/form.sx, 26 tests). Ok/Err result values (dream-ok/dream-err + predicates/accessors). dream-form-fields parses application/x-www-form-urlencoded with a full percent-decoder (%XX via char-from-code, +→space). CSRF is stateless + signed + session- scoped: token = sid.signature, verified by recomputing the signature and checking the session id — no server storage. Signing is injectable (dream-csrf-with); the default dream-csrf-sign-default is a pure-SX dual-base polynomial keyed hash (NOT cryptographic — production should inject a host HMAC). dream-csrf attaches context (needs the session middleware upstream for the sid); dream-csrf-token / dream-csrf-tag (hidden input for templates); dream-form returns Ok fields or Err :csrf-token-invalid; dream-csrf-protect auto-rejects unsafe methods (403) lacking a valid token. Full session→csrf→form stack verified accept + reject. Multipart deferred to the next commit.
• 2026-06-07 — Multipart (lib/dream/form.sx +9 tests, 35 total). dream-multipart req parses multipart/form-data into parts {:name :filename :content-type :content}, returns Ok parts | Err :not-multipart. Needed a substring splitter dr/split-on because the split primitive is character-class based (multi-char separators split on every char) — important gotcha. Boundary from the Content-Type (handles quoted form); segments filtered to those starting with CRLF; each split on the first \r\n\r\n into headers/content with one edge CRLF stripped (inner CRLFs in file content preserved). dream-multipart-field / dream-multipart-file accessors. In-memory, not streaming (noted for future). \r/\n string escapes work in SX literals.
• 2026-06-07 — WebSockets (lib/dream/websocket.sx, 16 tests). dream-websocket handler wraps a (fn (ws) …) into an ordinary handler returning a 101 upgrade response carrying the ws handler (dream-websocket? / dream-ws-handler for the host to detect + dispatch). dream-send / dream-receive / dream-close / dream-ws-open? / dream-ws-broadcast operate over an injectable io; production io is (perform op), tests use dream-mock-ws (in-memory inbox/outbox/closed via the cell pattern) with dream-ws-sent / dream-ws-closed? introspection and dream-ws-run to drive a handler. Echo loop + room broadcast verified.
• 2026-06-07 — Static files (lib/dream/static.sx, 28 tests). dream-static root mounts at a ** route and serves files: content-type by extension (mime map), weak ETag ("hash-length") with If-None-Match → 304 (incl. *), and Range: bytes= requests → 206 with Content-Range (open-ended bytes=N- supported, unsatisfiable → 416). ../absolute path traversal → 403; missing → 404; full responses advertise Accept-Ranges. Filesystem is injectable — dream-static-perform-fs (host) vs dream-memory-fs (in-memory map for tests).

Blockers

(none — gate green, loop active)