# Dream-on-SX: OCaml's Dream web framework on the SX CEK

`[activated — target user confirmed; gated only on ocaml-on-sx]`

Carved out of `plans/ocaml-on-sx.md`. The OCaml-on-SX plan was scoped down to **substrate validation + HM + reference oracle** (Phases 1–5 + minimal stdlib slice). Dream is the practical alternative-stack story — the opposite framing — and is now the **chosen framework layer for the rose-ash host**: the decision is to move off Quart and adopt Dream (not Quart) as the ergonomic HTTP front door over the native SX server. `plans/host-on-sx.md` Phase 4 is the concrete consumer that pulls Dream.

**Target user — CONFIRMED.** The earlier "needs a concrete target user" condition is met: rose-ash itself is the user — the subsystems (feed/acl/mod/commerce/identity/…) need an ergonomic HTTP front door, and the project owner has chosen Dream over Quart for it. This plan is no longer cold; it is *gated*, not deferred.

**Do not start without (the one remaining gate):**
1. OCaml-on-SX Phases 1–5 + Phase 6 minimal stdlib green. (As of writing ocaml-on-sx is at 480/480 and advancing — verify its scoreboard covers Phases 1–5 + the stdlib slice before starting.)

Until that gate is green, the native server (host-on-sx Phases 1–3) carries the host; do not block host migration on Dream.

## Why this might be worth doing (when the time comes)

Dream is the cleanest middleware-shaped HTTP framework in any language:
- `handler = request -> response promise`
- `middleware = handler -> handler`
- `m1 @@ m2 @@ handler` — left-fold composition

It maps onto SX with almost no impedance — `@@` is function composition, `request → response promise` is `(perform (:http-respond ...))`, middleware chain is plain SX function composition. So the integration cost is low *if* the OCaml-on-SX foundation is in place.

The user-facing story: rose-ash users who'd never touch s-expressions might write Dream/OCaml apps that integrate with the same federation, auth, and storage primitives. Demo: a Dream app serving sx.rose-ash.com — the framework that describes the runtime it runs on.

## Dream semantic mappings

| Dream construct | SX mapping |
|----------------|-----------|
| `handler = request -> response promise` | `(fn (req) (perform (:http-respond ...)))` |
| `middleware = handler -> handler` | `(fn (next) (fn (req) ...))` |
| `Dream.router [routes]` | `(ocaml-dream-router routes)` — dispatch on method+path |
| `Dream.get "/path" h` | route record `{:method "GET" :path "/path" :handler h}` |
| `Dream.scope "/p" [ms] [rs]` | prefix mount with middleware chain |
| `Dream.param req "name"` | path param extracted during routing |
| `m1 @@ m2 @@ handler` | `(m1 (m2 handler))` — left-fold composition |
| `Dream.session_field req "k"` | `(perform (:session-get req "k"))` |
| `Dream.set_session_field req "k" v` | `(perform (:session-set req "k" v))` |
| `Dream.flash req` | `(perform (:flash-get req))` |
| `Dream.form req` | `(perform (:form-parse req))` — returns Ok/Error ADT |
| `Dream.websocket handler` | `(perform (:websocket handler))` |
| `Dream.run handler` | starts SX HTTP server with handler as root |

## Roadmap

The five types: `request`, `response`, `handler = request -> response`, `middleware = handler -> handler`, `route`. Everything else is a function over these.

- [x] **Core types** in `lib/dream/types.sx`: request/response records, route record.
- [x] **Router** in `lib/dream/router.sx`:
      - `dream-get path handler`, `dream-post path handler`, etc. for all HTTP methods.
      - `dream-scope prefix middlewares routes` — prefix mount with middleware chain.
      - `dream-router routes` — dispatch tree, returns handler; no match → 404.
      - Path param extraction: `:name` segments, `**` wildcard.
      - `dream-param req name` — retrieve matched path param.
- [x] **Middleware** in `lib/dream/middleware.sx`:
      - `dream-pipeline middlewares handler` — compose middleware left-to-right.
      - `dream-no-middleware` — identity.
      - Logger: `(dream-logger next req)` — logs method, path, status, timing.
      - Content-type sniffer.
- [x] **Sessions** in `lib/dream/session.sx`:
      - Cookie-backed session middleware.
      - `dream-session-field req key`, `dream-set-session-field req key val`.
      - `dream-invalidate-session req`.
- [x] **Flash messages** in `lib/dream/flash.sx`:
      - `dream-flash-middleware` — single-request cookie store.
      - `dream-add-flash-message req category msg`.
      - `dream-flash-messages req` — returns list of `(category, msg)`.
- [x] **Forms + CSRF** in `lib/dream/form.sx`:
      - [x] `dream-form req` — returns `(Ok fields)` or `(Err :csrf-token-invalid)`.
      - [x] `dream-multipart req` — multipart form data (in-memory, not yet streaming).
      - [x] CSRF middleware: stateless signed tokens, session-scoped.
      - [x] `dream-csrf-tag req` — returns hidden input fragment for SX templates.
- [x] **WebSockets** in `lib/dream/websocket.sx`:
      - `dream-websocket handler` — upgrades request; handler `(fn (ws) ...)`.
      - `dream-send ws msg`, `dream-receive ws`, `dream-close ws`.
- [x] **Static files:** `dream-static root-path` — serves files, ETags, range requests.
- [ ] **`dream-run`**: wires root handler into SX's `perform (:http-listen ...)`.
- [ ] **Demos** in `lib/dream/demos/`:
      - `hello.ml` → `lib/dream/demos/hello.sx`: "Hello, World!" route.
      - `counter.ml` → `lib/dream/demos/counter.sx`: in-memory counter with sessions.
      - `chat.ml` → `lib/dream/demos/chat.sx`: multi-room WebSocket chat.
      - `todo.ml` → `lib/dream/demos/todo.sx`: CRUD list with forms + CSRF.
- [ ] Tests in `lib/dream/tests/`: routing dispatch, middleware composition, session round-trip, CSRF accept/reject, flash read-after-write — 60+ tests.

## Stdlib additions Dream will need

Dream pushes beyond OCaml-on-SX's Phase 6 minimal stdlib slice. When this plan activates, OCaml-on-SX gets a follow-on phase that adds at minimum:

- `Bytes` (binary buffers — request bodies, websocket frames)
- `Buffer` (mutable string building)
- `Format` (full pretty-printer, not just `Printf.sprintf`)
- More `String` (`index_opt`, `contains`, `starts_with`, `ends_with`, `replace_all`)
- `Sys` (`argv`, `getenv_opt`, `getcwd`)
- `Hashtbl` extensions (`iter`, `fold`, `length`, `remove`)
- `Map.Make` / `Set.Make` functors

Confirm scope before starting; some of these may be addable as Dream-internal helpers rather than full stdlib modules.

## Ground rules

- **Scope:** only `lib/dream/**` and `plans/dream-on-sx.md`. Plus the stdlib additions listed above which land in `lib/ocaml/runtime.sx`.
- **Hard prerequisite:** OCaml-on-SX Phases 1–5 + Phase 6 minimal stdlib. Verify scoreboard before starting.
- **SX files:** `sx-tree` MCP tools only.
- **Don't reinvent the SX HTTP server.** Dream wraps the existing `perform (:http-listen ...)` — it does not implement its own listener loop.

## Progress log

- **2026-06-07 — Core types** (`lib/dream/types.sx`, 41 tests). OCaml gate verified
  green (scoreboard 480/480, Phases 1–5 + Phase 6 stdlib). Dream is implemented in
  plain SX over the CEK — keywords are strings, so headers are dicts with lowercased
  string keys (`:content-type` == `"content-type"`). request (method/target/path/
  query/headers/body/params), response (status/headers/body), route records with
  constructors + accessors; smart response constructors (html/text/json/empty/
  not-found/redirect); `dream-coerce-response` wraps bare strings; query-string
  parsing. Conformance runner `lib/dream/conformance.sh` modelled on flow's.
- **2026-06-07 — Router** (`lib/dream/router.sx`, 27 tests). `dream-get/post/put/
  delete/patch/head/options/any` route constructors; `dream-router` flattens routes
  (incl. nested scopes) and dispatches by method+path, first-match-wins, 404 on no
  match. Path matching is recursive over `/`-split segments: literal, `:name` binds
  a param, `**` catch-all binds remaining path under key `"**"`. Trailing slashes and
  query strings are ignored for routing. `dream-scope prefix mws routes` prepends the
  prefix and folds the middleware chain (`m1 @@ m2 @@ h`, first = outermost) onto each
  route's handler; nests correctly (inner mw innermost). Shared `dr/apply-middlewares`
  fold will back `dream-pipeline`.
- **2026-06-07 — Middleware** (`lib/dream/middleware.sx`, 20 tests). `dream-pipeline`
  (reuses `dr/apply-middlewares`), `dream-no-middleware` identity. `dream-logger-with
  clock sink` is the testable core (records `{:method :path :status :elapsed}`);
  `dream-logger` wires it to `(perform (:dream-clock))` / `(perform (:dream-log …))`;
  `dream-log-line` formats one line. `dream-content-type` sniffs body (`<`→html,
  `{`/`[`→json, else text) only when the handler left Content-Type unset. Bonus
  `dream-set-header` and `dream-tap-request` combinators.
- **2026-06-07 — Sessions** (`lib/dream/session.sx`, 30 tests). Solved the
  request→response mutation-visibility problem the way Dream does: the cookie carries
  only a session id; fields live in an injectable back-end store (the mapping table's
  `(perform (:session-get …))`). `dream-memory-sessions` is an in-memory store built
  on a `set!`-mutated captured `let` binding (no `ref`/`atom` in base env);
  `dream-perform-sessions` is the production back-end. `dream-sessions backend`
  middleware reads/creates the id, attaches `{:sid :io}` to the request, and emits a
  `Set-Cookie` (HttpOnly, SameSite=Lax) only for new sessions. Handler API:
  `dream-session-field` / `dream-set-session-field` / `dream-session-all` /
  `dream-invalidate-session` / `dream-session-id`. Also added shared cookie infra
  (`dr/parse-cookies`, `dream-cookie(s)`, `dr/build-cookie`, `dream-set-cookie`,
  `dream-resp-cookies`, `dream-drop-cookie`) — outgoing cookies accumulate in a
  `:set-cookies` list on the response so multiple Set-Cookie headers don't collide;
  reused by flash + CSRF. Full counter round-trip verified across three requests.
- **2026-06-07 — Flash** (`lib/dream/flash.sx`, 14 tests). `dream-flash` middleware:
  decodes the incoming `dream.flash` cookie into the request, gives the handler a
  mutable outbox cell (`dr/flash-box`, the same `set!`-captured-`let` trick), then on
  response writes the outbox as a fresh flash cookie, or drops the cookie (Max-Age=0)
  when there were incoming messages but no new ones — so messages show exactly once.
  Handler API: `dream-add-flash-message` / `dream-flash-messages` (returns the
  PREVIOUS request's messages) / `dream-flash-of` (by category) / accessors. Cookie
  codec percent-escapes the `|`/`~`/`%` separators so categories/messages round-trip.
  Read-after-write verified across request boundaries incl. multi-category.
- **2026-06-07 — Forms + CSRF (urlencoded)** (`lib/dream/form.sx`, 26 tests). Ok/Err
  result values (`dream-ok`/`dream-err` + predicates/accessors). `dream-form-fields`
  parses `application/x-www-form-urlencoded` with a full percent-decoder
  (`%XX` via `char-from-code`, `+`→space). CSRF is stateless + signed + session-
  scoped: token = `sid.signature`, verified by recomputing the signature and checking
  the session id — no server storage. Signing is **injectable** (`dream-csrf-with`);
  the default `dream-csrf-sign-default` is a pure-SX dual-base polynomial keyed hash
  (NOT cryptographic — production should inject a host HMAC). `dream-csrf` attaches
  context (needs the session middleware upstream for the sid); `dream-csrf-token` /
  `dream-csrf-tag` (hidden input for templates); `dream-form` returns `Ok fields` or
  `Err :csrf-token-invalid`; `dream-csrf-protect` auto-rejects unsafe methods (403)
  lacking a valid token. Full session→csrf→form stack verified accept + reject.
  Multipart deferred to the next commit.
- **2026-06-07 — Multipart** (`lib/dream/form.sx` +9 tests, 35 total). `dream-multipart
  req` parses `multipart/form-data` into parts `{:name :filename :content-type
  :content}`, returns `Ok parts | Err :not-multipart`. Needed a substring splitter
  `dr/split-on` because the `split` primitive is **character-class** based (multi-char
  separators split on every char) — important gotcha. Boundary from the Content-Type
  (handles quoted form); segments filtered to those starting with CRLF; each split on
  the first `\r\n\r\n` into headers/content with one edge CRLF stripped (inner CRLFs
  in file content preserved). `dream-multipart-field` / `dream-multipart-file`
  accessors. In-memory, not streaming (noted for future). `\r`/`\n` string escapes
  work in SX literals.
- **2026-06-07 — WebSockets** (`lib/dream/websocket.sx`, 16 tests). `dream-websocket
  handler` wraps a `(fn (ws) …)` into an ordinary handler returning a 101 upgrade
  response carrying the ws handler (`dream-websocket?` / `dream-ws-handler` for the
  host to detect + dispatch). `dream-send` / `dream-receive` / `dream-close` /
  `dream-ws-open?` / `dream-ws-broadcast` operate over an injectable io; production io
  is `(perform op)`, tests use `dream-mock-ws` (in-memory inbox/outbox/closed via the
  cell pattern) with `dream-ws-sent` / `dream-ws-closed?` introspection and
  `dream-ws-run` to drive a handler. Echo loop + room broadcast verified.
- **2026-06-07 — Static files** (`lib/dream/static.sx`, 28 tests). `dream-static root`
  mounts at a `**` route and serves files: content-type by extension (mime map),
  weak ETag (`"hash-length"`) with `If-None-Match` → 304 (incl. `*`), and `Range:
  bytes=` requests → 206 with `Content-Range` (open-ended `bytes=N-` supported,
  unsatisfiable → 416). `..`/absolute path traversal → 403; missing → 404; full
  responses advertise `Accept-Ranges`. Filesystem is injectable —
  `dream-static-perform-fs` (host) vs `dream-memory-fs` (in-memory map for tests).

## Blockers

_(none — gate green, loop active)_