Merge loops/dream into architecture: dream-on-sx — OCaml Dream web framework reimplemented in plain SX

Full roadmap + 10 extensions, 413/413 tests across 17 suites (lib/dream/).
Five types (request/response/route + handler/middleware fns); router (params,
scopes, 405/HEAD), middleware, sessions (signed), flash, forms+CSRF+multipart,
websockets, static files, error handling, CORS, JSON, auth (base64/basic/bearer),
HTML escaping, security headers, dream-run + api facade, 4 demos.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

plans/dream-on-sx.md

@@ -44,42 +44,127 @@ The user-facing story: rose-ash users who'd never touch s-expressions might writ
 
 The five types: `request`, `response`, `handler = request -> response`, `middleware = handler -> handler`, `route`. Everything else is a function over these.
 
-- [ ] **Core types** in `lib/dream/types.sx`: request/response records, route record.
-- [ ] **Router** in `lib/dream/router.sx`:
+- [x] **Core types** in `lib/dream/types.sx`: request/response records, route record.
+- [x] **Router** in `lib/dream/router.sx`:
       - `dream-get path handler`, `dream-post path handler`, etc. for all HTTP methods.
       - `dream-scope prefix middlewares routes` — prefix mount with middleware chain.
       - `dream-router routes` — dispatch tree, returns handler; no match → 404.
       - Path param extraction: `:name` segments, `**` wildcard.
       - `dream-param req name` — retrieve matched path param.
-- [ ] **Middleware** in `lib/dream/middleware.sx`:
+- [x] **Middleware** in `lib/dream/middleware.sx`:
       - `dream-pipeline middlewares handler` — compose middleware left-to-right.
       - `dream-no-middleware` — identity.
       - Logger: `(dream-logger next req)` — logs method, path, status, timing.
       - Content-type sniffer.
-- [ ] **Sessions** in `lib/dream/session.sx`:
+- [x] **Sessions** in `lib/dream/session.sx`:
       - Cookie-backed session middleware.
       - `dream-session-field req key`, `dream-set-session-field req key val`.
       - `dream-invalidate-session req`.
-- [ ] **Flash messages** in `lib/dream/flash.sx`:
+- [x] **Flash messages** in `lib/dream/flash.sx`:
       - `dream-flash-middleware` — single-request cookie store.
       - `dream-add-flash-message req category msg`.
       - `dream-flash-messages req` — returns list of `(category, msg)`.
-- [ ] **Forms + CSRF** in `lib/dream/form.sx`:
-      - `dream-form req` — returns `(Ok fields)` or `(Err :csrf-token-invalid)`.
-      - `dream-multipart req` — streaming multipart form data.
-      - CSRF middleware: stateless signed tokens, session-scoped.
-      - `dream-csrf-tag req` — returns hidden input fragment for SX templates.
-- [ ] **WebSockets** in `lib/dream/websocket.sx`:
+- [x] **Forms + CSRF** in `lib/dream/form.sx`:
+      - [x] `dream-form req` — returns `(Ok fields)` or `(Err :csrf-token-invalid)`.
+      - [x] `dream-multipart req` — multipart form data (in-memory, not yet streaming).
+      - [x] CSRF middleware: stateless signed tokens, session-scoped.
+      - [x] `dream-csrf-tag req` — returns hidden input fragment for SX templates.
+- [x] **WebSockets** in `lib/dream/websocket.sx`:
       - `dream-websocket handler` — upgrades request; handler `(fn (ws) ...)`.
       - `dream-send ws msg`, `dream-receive ws`, `dream-close ws`.
-- [ ] **Static files:** `dream-static root-path` — serves files, ETags, range requests.
-- [ ] **`dream-run`**: wires root handler into SX's `perform (:http-listen ...)`.
-- [ ] **Demos** in `lib/dream/demos/`:
-      - `hello.ml` → `lib/dream/demos/hello.sx`: "Hello, World!" route.
-      - `counter.ml` → `lib/dream/demos/counter.sx`: in-memory counter with sessions.
-      - `chat.ml` → `lib/dream/demos/chat.sx`: multi-room WebSocket chat.
-      - `todo.ml` → `lib/dream/demos/todo.sx`: CRUD list with forms + CSRF.
-- [ ] Tests in `lib/dream/tests/`: routing dispatch, middleware composition, session round-trip, CSRF accept/reject, flash read-after-write — 60+ tests.
+- [x] **Static files:** `dream-static root-path` — serves files, ETags, range requests.
+- [x] **`dream-run`**: wires root handler into SX's `perform (:http-listen ...)`.
+- [x] **Demos** in `lib/dream/demos/`:
+      - [x] `hello.ml` → `lib/dream/demos/hello.sx`: "Hello, World!" route.
+      - [x] `counter.ml` → `lib/dream/demos/counter.sx`: in-memory counter with sessions.
+      - [x] `chat.ml` → `lib/dream/demos/chat.sx`: multi-room WebSocket chat.
+      - [x] `todo.ml` → `lib/dream/demos/todo.sx`: CRUD list with forms + CSRF.
+- [x] Tests in `lib/dream/tests/`: routing dispatch, middleware composition, session round-trip, CSRF accept/reject, flash read-after-write — **258 tests across 10 suites** (well past the 60+ target). Runner: `lib/dream/conformance.sh`.
+
+**Roadmap complete (2026-06-07): all boxes ticked, 258/258 green.** Loop continues
+with extensions + hardening below.
+- **2026-06-07 — Ext: router HTTP correctness** (router suite 27→36, 267 total).
+  Dispatch now tracks which routes' *paths* matched: path matched + method didn't →
+  `405 Method Not Allowed` with an `Allow` header listing the path's methods (was a
+  blanket 404); genuinely-absent paths stay 404. `HEAD` falls back to the matching
+  `GET` handler with the body blanked but headers kept. `dr/route-params` (path-only
+  match) + `dr/method-accepts?` (ANY / HEAD→GET) + `dream-method-not-allowed`. NOTE:
+  in this worktree every `sx-tree` *edit* tool (`sx_replace_node`,
+  `sx_replace_by_pattern`, `sx_insert_near`) raises a yojson `Expected string, got
+  null` error — only `sx_write_file` works, so edits rewrite the whole file.
+- **2026-06-07 — Ext: error handling + status phrases** (`lib/dream/error.sx`, 15
+  tests, 282 total). `dream-status-text` / `dream-status-line` reason-phrase map (string
+  keys); `dream-status-page` renders a status page. `dream-catch` is a `guard`-based
+  middleware that turns a raised error into a 500 (`dream-catch-with on-error` for a
+  custom page receiving `(req e)`); normal responses pass through untouched, composes
+  around a router. (`guard` catches explicit `(error …)` raises; `e` stringifies to the
+  message.)
+- **2026-06-07 — Ext: CORS** (`lib/dream/cors.sx`, 12 tests, 294 total). `dream-cors`
+  decorates responses with `Access-Control-Allow-Origin` (+ credentials), and
+  short-circuits preflight `OPTIONS` with a 204 carrying Allow-Methods/Headers/Max-Age.
+  `dream-cors-origin` for a specific origin, `dream-cors-with opts` for full control
+  (origin/methods/headers/credentials/max-age). Composes around a router.
+- **2026-06-07 — Ext: JSON** (`lib/dream/json.sx`, 35 tests, 329 total). Host JSON
+  primitives live in the ocaml-on-sx runtime (not the base env), so Dream ships its own
+  pure-SX `dream-json-encode` (scalars/list/dict, string escaping) + `dream-json-parse`
+  (recursive-descent over chars, objects/arrays/strings/numbers/true/false/null,
+  whitespace-tolerant). `dream-json-value` (encode → application/json response) and
+  `dream-json-body` (parse request body). GOTCHA: `number?` is unreliable in this env —
+  used `(= (type-of v) "number")`; `parse-float` handles decimals. Multi-key dict
+  encode order follows `keys` (non-deterministic) so tests assert via parse round-trip.
+- **2026-06-07 — Ext: signed session cookies** (`lib/dream/session.sx`, session suite
+  30→41, 340 total). The default store uses guessable sids (`s1`, `s2`), so
+  `dream-sessions-signed backend secret` signs the cookie value (`sid.signature`) and
+  rejects any cookie whose signature doesn't verify — a forged plaintext `s1` or a
+  wrong-secret cookie yields a fresh session instead of a hijack. `dream-cookie-sign` /
+  `dream-cookie-unsign` (keyed hash; same not-cryptographic caveat — inject a host HMAC
+  in production). Plain `dream-sessions` unchanged for the no-secret case.
+- **2026-06-07 — Ext: query/header convenience** (`lib/dream/types.sx`, types suite
+  41→59, 358 total). `dream-queries`, `dream-query-param-or` / `dream-header-or` /
+  `dream-param-or` (defaults), `dream-has-query?` / `-header?` / `-param?`,
+  `dream-content-type-of`, `dream-method-is?`, `dream-accepts?` / `dream-wants-json?`
+  (Accept-header content negotiation).
+- **2026-06-07 — Ext: api.sx facade + README** (`lib/dream/api.sx`, 9 tests, 367 total).
+  `dream-version`, `dream-defaults` (pure stack: error-catch + content-type; logger is
+  opt-in since it performs IO), `dream-make-app routes`, `dream-make-app-with`,
+  `dream-serve`/`dream-serve-port`. `lib/dream/README.md` documents the full public
+  surface, quickstart, the dependency-injection testing story, and caveats. **All
+  planned extensions complete — 367/367 across 14 suites.**
+- **2026-06-07 — Ext: auth** (`lib/dream/auth.sx`, 23 tests, 390 total). Pure-SX base64
+  codec (`dream-base64-encode`/`-decode`, arithmetic via `quotient`/`mod` — no bitwise),
+  verified against RFC vectors (Man/Ma/M padding). `dream-basic-auth realm check` →
+  401 + `WWW-Authenticate: Basic realm=…`, attaches `:dream-user` on success;
+  `dream-basic-credentials` / `dream-authorization` accessors. `dream-require-bearer
+  check` → attaches `:dream-principal` or 401; `dream-bearer-token` accessor.
+- **2026-06-07 — Ext: HTML escaping** (`lib/dream/html.sx`, 11 tests, 401 total).
+  `dream-escape` (&/</>/"/' entities, ampersand first to avoid double-escape),
+  `dream-attr`, `dream-escape-join`. Fixed a real **XSS hole** in the todo demo, which
+  interpolated user text into `<li>` unescaped — now `(dream-escape (get it :text))`;
+  regression test asserts `<script>` renders as `&lt;script&gt;`. 16 suites, 401/401.
+- **2026-06-07 — Ext: security headers + cache-control** (`lib/dream/headers.sx`, 12
+  tests, 413 total). `dream-security-headers` middleware (X-Content-Type-Options
+  nosniff, X-Frame-Options DENY, Referrer-Policy no-referrer; opt-in HSTS via
+  `dream-security-headers-with`). Cache helpers `dream-cache`/`dream-private-cache`/
+  `dream-no-store`/`dream-no-cache` + `dream-cache-for` middleware. **dream-on-sx is
+  feature-complete: roadmap + 10 extensions, 413/413 across 17 suites. SATURATED —
+  remaining work is host-on-sx's job to consume `dream-run` (don't edit hosts/).**
+
+## Extensions (post-roadmap)
+
+The five-types core is complete; these harden it toward a production HTTP front door.
+
+- [x] **Router HTTP correctness**: 405 Method Not Allowed + `Allow` header; automatic
+      HEAD (serve the GET handler with an empty body).
+- [x] **Status reason phrases** + `dream-status-text` (`lib/dream/error.sx`).
+- [x] **CORS middleware** (`dream-cors`).
+- [x] **Error-handling middleware** (`dream-catch` / custom 500 templates; `guard`-based).
+- [x] **Signed session cookies** (`dream-sessions-signed` — tamper-evident sid).
+- [x] **JSON helpers** (encode + recursive-descent parse, pure SX).
+- [x] **Query/header convenience** (`dream-queries`, `*-or` defaults, `dream-accepts?`).
+- [x] **`api.sx` facade + README** — `dream-make-app` / `dream-serve` + `README.md`.
+- [x] **Auth** — base64 (pure SX), HTTP Basic auth + Bearer-token middleware.
+- [x] **HTML escaping** (`dream-escape`/`dream-attr`) — fixed an XSS hole in the todo demo.
+- [x] **Security headers + cache-control** (`dream-security-headers`, `dream-cache`/`-no-store`).
 
 ## Stdlib additions Dream will need
 
@@ -104,8 +189,114 @@ Confirm scope before starting; some of these may be addable as Dream-internal he
 
 ## Progress log
 
-_(awaiting activation conditions)_
+- **2026-06-07 — Core types** (`lib/dream/types.sx`, 41 tests). OCaml gate verified
+  green (scoreboard 480/480, Phases 1–5 + Phase 6 stdlib). Dream is implemented in
+  plain SX over the CEK — keywords are strings, so headers are dicts with lowercased
+  string keys (`:content-type` == `"content-type"`). request (method/target/path/
+  query/headers/body/params), response (status/headers/body), route records with
+  constructors + accessors; smart response constructors (html/text/json/empty/
+  not-found/redirect); `dream-coerce-response` wraps bare strings; query-string
+  parsing. Conformance runner `lib/dream/conformance.sh` modelled on flow's.
+- **2026-06-07 — Router** (`lib/dream/router.sx`, 27 tests). `dream-get/post/put/
+  delete/patch/head/options/any` route constructors; `dream-router` flattens routes
+  (incl. nested scopes) and dispatches by method+path, first-match-wins, 404 on no
+  match. Path matching is recursive over `/`-split segments: literal, `:name` binds
+  a param, `**` catch-all binds remaining path under key `"**"`. Trailing slashes and
+  query strings are ignored for routing. `dream-scope prefix mws routes` prepends the
+  prefix and folds the middleware chain (`m1 @@ m2 @@ h`, first = outermost) onto each
+  route's handler; nests correctly (inner mw innermost). Shared `dr/apply-middlewares`
+  fold will back `dream-pipeline`.
+- **2026-06-07 — Middleware** (`lib/dream/middleware.sx`, 20 tests). `dream-pipeline`
+  (reuses `dr/apply-middlewares`), `dream-no-middleware` identity. `dream-logger-with
+  clock sink` is the testable core (records `{:method :path :status :elapsed}`);
+  `dream-logger` wires it to `(perform (:dream-clock))` / `(perform (:dream-log …))`;
+  `dream-log-line` formats one line. `dream-content-type` sniffs body (`<`→html,
+  `{`/`[`→json, else text) only when the handler left Content-Type unset. Bonus
+  `dream-set-header` and `dream-tap-request` combinators.
+- **2026-06-07 — Sessions** (`lib/dream/session.sx`, 30 tests). Solved the
+  request→response mutation-visibility problem the way Dream does: the cookie carries
+  only a session id; fields live in an injectable back-end store (the mapping table's
+  `(perform (:session-get …))`). `dream-memory-sessions` is an in-memory store built
+  on a `set!`-mutated captured `let` binding (no `ref`/`atom` in base env);
+  `dream-perform-sessions` is the production back-end. `dream-sessions backend`
+  middleware reads/creates the id, attaches `{:sid :io}` to the request, and emits a
+  `Set-Cookie` (HttpOnly, SameSite=Lax) only for new sessions. Handler API:
+  `dream-session-field` / `dream-set-session-field` / `dream-session-all` /
+  `dream-invalidate-session` / `dream-session-id`. Also added shared cookie infra
+  (`dr/parse-cookies`, `dream-cookie(s)`, `dr/build-cookie`, `dream-set-cookie`,
+  `dream-resp-cookies`, `dream-drop-cookie`) — outgoing cookies accumulate in a
+  `:set-cookies` list on the response so multiple Set-Cookie headers don't collide;
+  reused by flash + CSRF. Full counter round-trip verified across three requests.
+- **2026-06-07 — Flash** (`lib/dream/flash.sx`, 14 tests). `dream-flash` middleware:
+  decodes the incoming `dream.flash` cookie into the request, gives the handler a
+  mutable outbox cell (`dr/flash-box`, the same `set!`-captured-`let` trick), then on
+  response writes the outbox as a fresh flash cookie, or drops the cookie (Max-Age=0)
+  when there were incoming messages but no new ones — so messages show exactly once.
+  Handler API: `dream-add-flash-message` / `dream-flash-messages` (returns the
+  PREVIOUS request's messages) / `dream-flash-of` (by category) / accessors. Cookie
+  codec percent-escapes the `|`/`~`/`%` separators so categories/messages round-trip.
+  Read-after-write verified across request boundaries incl. multi-category.
+- **2026-06-07 — Forms + CSRF (urlencoded)** (`lib/dream/form.sx`, 26 tests). Ok/Err
+  result values (`dream-ok`/`dream-err` + predicates/accessors). `dream-form-fields`
+  parses `application/x-www-form-urlencoded` with a full percent-decoder
+  (`%XX` via `char-from-code`, `+`→space). CSRF is stateless + signed + session-
+  scoped: token = `sid.signature`, verified by recomputing the signature and checking
+  the session id — no server storage. Signing is **injectable** (`dream-csrf-with`);
+  the default `dream-csrf-sign-default` is a pure-SX dual-base polynomial keyed hash
+  (NOT cryptographic — production should inject a host HMAC). `dream-csrf` attaches
+  context (needs the session middleware upstream for the sid); `dream-csrf-token` /
+  `dream-csrf-tag` (hidden input for templates); `dream-form` returns `Ok fields` or
+  `Err :csrf-token-invalid`; `dream-csrf-protect` auto-rejects unsafe methods (403)
+  lacking a valid token. Full session→csrf→form stack verified accept + reject.
+  Multipart deferred to the next commit.
+- **2026-06-07 — Multipart** (`lib/dream/form.sx` +9 tests, 35 total). `dream-multipart
+  req` parses `multipart/form-data` into parts `{:name :filename :content-type
+  :content}`, returns `Ok parts | Err :not-multipart`. Needed a substring splitter
+  `dr/split-on` because the `split` primitive is **character-class** based (multi-char
+  separators split on every char) — important gotcha. Boundary from the Content-Type
+  (handles quoted form); segments filtered to those starting with CRLF; each split on
+  the first `\r\n\r\n` into headers/content with one edge CRLF stripped (inner CRLFs
+  in file content preserved). `dream-multipart-field` / `dream-multipart-file`
+  accessors. In-memory, not streaming (noted for future). `\r`/`\n` string escapes
+  work in SX literals.
+- **2026-06-07 — WebSockets** (`lib/dream/websocket.sx`, 16 tests). `dream-websocket
+  handler` wraps a `(fn (ws) …)` into an ordinary handler returning a 101 upgrade
+  response carrying the ws handler (`dream-websocket?` / `dream-ws-handler` for the
+  host to detect + dispatch). `dream-send` / `dream-receive` / `dream-close` /
+  `dream-ws-open?` / `dream-ws-broadcast` operate over an injectable io; production io
+  is `(perform op)`, tests use `dream-mock-ws` (in-memory inbox/outbox/closed via the
+  cell pattern) with `dream-ws-sent` / `dream-ws-closed?` introspection and
+  `dream-ws-run` to drive a handler. Echo loop + room broadcast verified.
+- **2026-06-07 — Static files** (`lib/dream/static.sx`, 28 tests). `dream-static root`
+  mounts at a `**` route and serves files: content-type by extension (mime map),
+  weak ETag (`"hash-length"`) with `If-None-Match` → 304 (incl. `*`), and `Range:
+  bytes=` requests → 206 with `Content-Range` (open-ended `bytes=N-` supported,
+  unsatisfiable → 416). `..`/absolute path traversal → 403; missing → 404; full
+  responses advertise `Accept-Ranges`. Filesystem is injectable —
+  `dream-static-perform-fs` (host) vs `dream-memory-fs` (in-memory map for tests).
+- **2026-06-07 — dream-run** (`lib/dream/run.sx`, 20 tests). `dream-run handler`
+  installs the root handler via `(perform (:http/listen {:port :host :app …}))` — no
+  socket code, it wraps the existing server. `dream-app handler` is the adapter the
+  host invokes per request: raw `{:method :target :headers :body}` → `dream-request`
+  → handler → serialised `{:status :headers :body :set-cookies}`, or a `{:status 101
+  :websocket …}` upgrade. Bare-string handlers coerced; method defaults to GET;
+  set-cookies (from session/flash) flow through. Listen transport injectable
+  (`dream-run-with`) so the full wiring is tested with a mock that captures the op and
+  re-runs the captured app. `dream-run-port` / `dream-run-opts` variants.
+- **2026-06-07 — Demos: hello + counter** (`lib/dream/demos/`, 10 tests). `hello.sx`
+  is the canonical router with a `:name` param route. `counter.sx` is a per-session
+  visit counter on the session middleware (+ a `/reset` POST that redirects),
+  demonstrating session isolation across browsers. End-to-end tests drive both apps
+  as the host would. chat (ws) + todo (forms+CSRF) next.
+- **2026-06-07 — Demos: chat + todo** (`lib/dream/demos/`, demos suite now 27 tests).
+  `chat.sx` is a multi-room WebSocket chat over a room registry (join/leave/members/
+  broadcast on the cell pattern); verified three clients see each other's broadcasts
+  and a disconnect leaves the room. `todo.sx` is a CRUD list wiring session→csrf→
+  router: add/toggle/delete go through `dream-form` (CSRF-guarded), an in-memory store
+  holds items, pages render the list + `dream-csrf-tag`; verified the full
+  add→render→toggle→delete cycle plus a 403 on a token-less POST. ws object equality
+  is by reference, so the `:leave` filter removes exactly the right connection.
 
 ## Blockers
 
-_(none yet — plan is cold)_
+_(none — gate green, loop active)_