diff --git a/plans/commerce-on-sx.md b/plans/commerce-on-sx.md new file mode 100644 index 00000000..fe7b7641 --- /dev/null +++ b/plans/commerce-on-sx.md @@ -0,0 +1,82 @@ +# commerce-on-sx: Catalog, cart, pricing & orders on miniKanren + +> **DRAFT outline.** The revenue vertical. Depends on `store-on-sx` (durable +> orders) and `flow-on-sx` (checkout as a durable flow). Don't start before +> store-on-sx Phase 1 is green. + +rose-ash's revenue engine — market (catalog), cart (checkout), orders (SumUp +payment, reconciliation) — has no SX subsystem. The hard part of commerce isn't +CRUD; it's **pricing**: discounts, bundles, tax, membership rates, promotions that +stack (or don't). These are relations, and a relational engine can run them in +multiple directions — forward ("what's the total?") and backward ("what promo code +yields this total?", "which line item triggered the discount?"). + +That's a miniKanren fit. Pricing/promotion rules are relational; cart and order +*lifecycle* (reserve → pay → fulfil → reconcile) is a durable `flow`; the order +ledger is a `store` stream. Commerce is the first real **composition** subsystem. + +End-state: a catalog model, a relational pricing/promotion engine, a cart with +deterministic totals, and an order lifecycle flow with payment-webhook +reconciliation — all auditable via the event log. + +## Status (rolling) + +`bash lib/commerce/conformance.sh` → **0/0** (not yet started) + +## Ground rules + +- **Scope:** only `lib/commerce/**` and `plans/commerce-on-sx.md`. May **import** + from `lib/minikanren/`, and (once they exist) `lib/store/` + `lib/flow/`. Do not + edit substrates. +- **Architecture:** prices/promotions are miniKanren relations over catalog facts; + a cart total is a *deterministic* query result (first solution under a fixed rule + order). Order lifecycle is a `flow` that suspends at the payment IO boundary. + Money is integer minor units — never floats. +- **Determinism:** promotion stacking must have explicit, tested precedence; + totals must be reproducible from the cart + catalog snapshot. +- **Commits:** one feature per commit. Progress log + tick boxes. + +## Architecture sketch + +``` +Catalog + cart Total / order + product(id,price,tags) {:subtotal :discounts :tax :total} + │ ▲ + ▼ │ +lib/commerce/catalog.sx lib/commerce/price.sx + — product / variant / stock facts — miniKanren pricing relations + │ — promo stacking, membership rates + ▼ ▲ +lib/commerce/cart.sx lib/commerce/order.sx (flow + store) + — line items, quantities — reserve→pay→fulfil→reconcile + │ — SumUp webhook = flow resume + ▼ │ +lib/commerce/api.sx ── (commerce/add) (commerce/total) (commerce/checkout) ──┘ +``` + +## Phase 1 — Catalog + cart + deterministic totals +- [ ] `catalog.sx` — product/variant/stock as facts +- [ ] `cart.sx` — line items, add/remove/qty +- [ ] `price.sx` — base pricing relation, subtotal; tax +- [ ] `api.sx` + tests + scoreboard + conformance.sh + +## Phase 2 — Promotions (relational) +- [ ] promo rules: percentage, fixed, bundle, member rate +- [ ] explicit stacking precedence; "best price" backward query +- [ ] tests: stacking order, mutually-exclusive promos, member vs guest + +## Phase 3 — Order lifecycle (flow + store) +- [ ] order flow: reserve stock → await payment → fulfil +- [ ] payment webhook resumes the suspended flow +- [ ] order ledger as a `store` stream; idempotent reconciliation + +## Phase 4 — Reconciliation + federation +- [ ] mismatch detection (paid≠ordered) as queries over the ledger +- [ ] cross-instance catalog (federated marketplace) — out-of-scope stub +- [ ] tests: webhook replay, partial refund, double-charge guard + +## Progress log +(loop fills this in) + +## Blockers +(loop fills this in) diff --git a/plans/content-on-sx.md b/plans/content-on-sx.md new file mode 100644 index 00000000..6096454c --- /dev/null +++ b/plans/content-on-sx.md @@ -0,0 +1,82 @@ +# content-on-sx: Documents, blocks & collaborative editing on Smalltalk + +> **DRAFT outline.** The CMS vertical — blog, WYSIWYG editor, Ghost sync. Depends +> on `store-on-sx` (document history as an event log). Ghost/CMS sync stays a thin +> external adapter (Python/FFI) until a native replacement exists. + +rose-ash's `blog` domain is content management: a block-based WYSIWYG editor, +navigation, Ghost CMS sync. A document is a tree of live blocks; editing is a +stream of operations; collaboration needs conflict-free merge. That is an object +model — blocks are objects, edits are messages, and a document is the object graph +responding to them. Smalltalk's "everything is an object responding to messages" +maps directly to a block/WYSIWYG model, and a semilattice (CRDT) merge keeps +concurrent edits conflict-free. + +End-state: a Smalltalk-on-SX document model (typed blocks, structural ops), +operation log + CRDT merge for collaborative editing, versioning/history via the +event store, and a render boundary to HTML/SX. External CMS (Ghost) sync is an +injected adapter, not core. + +## Status (rolling) + +`bash lib/content/conformance.sh` → **0/0** (not yet started) + +## Ground rules + +- **Scope:** only `lib/content/**` and `plans/content-on-sx.md`. May **import** + from `lib/smalltalk/`, and (once it exists) `lib/store/`. Do not edit substrates. +- **Architecture:** a document is an ordered tree of blocks (objects); an edit is a + message (`insert`/`update`/`move`/`delete`); concurrent edits merge via a + commutative (CRDT/semilattice) operation so order doesn't matter. History is the + `store` event stream; any version is a replay. +- **Determinism:** merge must be commutative + idempotent (test: apply ops in any + order / twice → same document). +- **Commits:** one feature per commit. Progress log + tick boxes. + +## Architecture sketch + +``` +Edit op Rendered document + (insert block after id) ... HTML / SX tree + │ ▲ + ▼ │ +lib/content/block.sx lib/content/render.sx + — typed blocks as objects — block tree → HTML/SX + — heading/text/image/embed — (reuses SX render boundary) + │ ▲ + ▼ │ +lib/content/doc.sx lib/content/merge.sx + — ordered block tree — CRDT/semilattice op merge + — apply op, structural moves — concurrent-edit reconciliation + │ ▲ + ▼ │ +lib/content/api.sx ── (content/edit) (content/render) (content/history) ──┐ + │ │ + ├── op log + versions → store │ + └── Ghost/CMS sync → injected external adapter (thin, non-core) ──┘ +``` + +## Phase 1 — Block document model +- [ ] `block.sx` — typed block objects +- [ ] `doc.sx` — ordered tree, apply edit op, structural moves +- [ ] `render.sx` — block tree → HTML/SX +- [ ] `api.sx` + tests + scoreboard + conformance.sh + +## Phase 2 — Op log + versioning +- [ ] edit ops as `store` events; replay to any version +- [ ] `(content/history doc)`, diff between versions + +## Phase 3 — Collaborative merge (CRDT) +- [ ] commutative/idempotent op merge +- [ ] concurrent-edit tests (any order, double-apply → identical) + +## Phase 4 — External sync + federation +- [ ] Ghost/CMS sync via injected adapter (import/export) +- [ ] federated documents (peer-authored blocks) — trust-gated stub +- [ ] tests: round-trip import/export, conflict on concurrent external edit + +## Progress log +(loop fills this in) + +## Blockers +(loop fills this in) diff --git a/plans/events-on-sx.md b/plans/events-on-sx.md new file mode 100644 index 00000000..f5d196a4 --- /dev/null +++ b/plans/events-on-sx.md @@ -0,0 +1,81 @@ +# events-on-sx: Calendar, ticketing & notification delivery on Datalog + +> **DRAFT outline.** The events vertical + the shared notification-delivery edge. +> Depends on `store-on-sx` (bookings ledger) and `flow-on-sx` (reminders, retrying +> delivery). Pairs with `commerce-on-sx` for paid tickets. + +rose-ash's `events` domain is calendar + ticketing: recurring events, availability, +capacity, bookings. Scheduling is constraint reasoning — "is this slot free given +recurrence, capacity, and the attendee's other bookings?" — which is rule +evaluation over facts. Datalog expresses availability, recurrence expansion, and +capacity as rules; a booking is a transaction; reminders and digests are durable +`flow`s. Notification *delivery* (email/push) — needed here and by `feed/notify` — +is folded in as an injected transport, extractable later. + +End-state: a Datalog-on-SX events layer with recurrence expansion, availability + +capacity rules, transactional booking, and a flow-driven notification dispatcher +(reminders, digests, retries) over an injected transport. + +## Status (rolling) + +`bash lib/events/conformance.sh` → **0/0** (not yet started) + +## Ground rules + +- **Scope:** only `lib/events/**` and `plans/events-on-sx.md`. May **import** from + `lib/datalog/`, and (once they exist) `lib/store/` + `lib/flow/`. Do not edit + substrates. +- **Architecture:** events/availability/capacity are Datalog facts + rules; + recurrence expands to occurrence facts within a window; a booking checks rules + then appends a `store` event (idempotent, capacity-safe). Notifications are flows + that suspend on transport IO and retry on failure. +- **Determinism:** recurrence expansion + availability must be reproducible for a + fixed window + ruleset; capacity checks must be race-safe (no overbooking). +- **Commits:** one feature per commit. Progress log + tick boxes. + +## Architecture sketch + +``` +Event + booking Result + event(id,start,rrule,capacity) {:booked | :full | :conflict} + reminders + │ ▲ + ▼ │ +lib/events/calendar.sx lib/events/availability.sx + — event facts, recurrence (RRULE) — free/busy + capacity rules (Datalog) + — expand occurrences in window │ + │ ▲ + ▼ │ +lib/events/booking.sx lib/events/notify.sx (flow) + — transactional, capacity-safe — reminders / digests, retry on fail + — bookings → store ledger — injected transport (email/push) + │ │ + ▼ ▼ +lib/events/api.sx ── (events/schedule) (events/book) (events/agenda) ──────┘ +``` + +## Phase 1 — Calendar + recurrence +- [ ] `calendar.sx` — event facts, RRULE expansion in a window +- [ ] `availability.sx` — free/busy rules +- [ ] `api.sx` + tests + scoreboard + conformance.sh + +## Phase 2 — Ticketing + booking +- [ ] capacity rules; transactional booking → `store` (no overbooking) +- [ ] paid tickets compose with `commerce` order flow +- [ ] tests: capacity edge, double-book guard, conflict detection + +## Phase 3 — Notification delivery (flow) +- [ ] `notify.sx` — reminder/digest flows over injected transport +- [ ] retry/backoff on transport failure (flow suspend/resume) +- [ ] tests: delivery success, retry path, idempotent re-send +- [ ] NOTE: shared with `feed/notify` — candidate for later extraction to a + `delivery-on-sx` once a second consumer is real + +## Phase 4 — Federation +- [ ] cross-instance events (peer calendar) — trust-gated stub +- [ ] tests: federated agenda merge + +## Progress log +(loop fills this in) + +## Blockers +(loop fills this in) diff --git a/plans/host-on-sx.md b/plans/host-on-sx.md new file mode 100644 index 00000000..7179545e --- /dev/null +++ b/plans/host-on-sx.md @@ -0,0 +1,100 @@ +# host-on-sx: The SX web host — off Quart, onto the kernel (Dream-bound) + +> **DRAFT outline.** The integration boundary that turns the subsystem libraries +> into running services, and the strangler path off Python/Quart. This is the +> dependency hub — it imports every subsystem. Decision recorded below: native +> server + SXTP **now**, `dream-on-sx` framework layer **next**, Python only at the +> external-integration edges. + +The subsystems (`feed`, `search`, `acl`, `mod`, `flow`, `commerce`, `identity`, +`content`, `events`) are libraries. Something has to receive an HTTP request, route +it, call the right subsystem, and serialize the response. Today that's Python/Quart +— the one large non-SX component in the stack: separate runtime, deploy, and +failure mode. The goal is to move the web/host/domain layer onto the SX substrate +and retire Quart, **incrementally (strangler-fig), never big-bang.** + +This is already underway: a native OCaml HTTP server is live in prod on +`sx.rose-ash.com` (~3ms cached, ~323 req/s, ~2MB RSS), `defhandler`/`defpage` +exist, and a partial **SXTP** protocol is specced. That is the unblocked near-term +host — no `ocaml-on-sx` dependency. + +## Two layers, two timelines + +1. **Now (unblocked): native server + SXTP adapter + SX handlers.** Route rose-ash + endpoints onto the SX host one at a time. Each migrated endpoint is an SX + handler dispatching to a subsystem; Quart proxies the rest until cut over. +2. **Next: `dream-on-sx` as the framework layer.** Dream gives Quart-grade + ergonomics — typed routing, middleware stacks, sessions, CSRF. It is gated on + `ocaml-on-sx` Phases 1–5 + minimal stdlib. **This plan is the concrete target + user that un-parks `dream-on-sx`** (see `plans/dream-on-sx.md`): "the subsystems + need an HTTP front door" is the real feature pulling Dream. Until then, do not + block migration on Dream — the native server is sufficient. +3. **Always: Python only at the edges.** External integrations — SumUp payments, + Ghost CMS, ActivityPub crypto, IPFS/Kubo — ride Python libraries today. They + stay as thin injected adapters (Python/FFI) behind subsystem interfaces until + native replacements exist. "Drop Quart" ≠ "drop every line of Python." + +## Status (rolling) + +`bash lib/host/conformance.sh` → **0/0** (not yet started) + +## Ground rules + +- **Scope:** `lib/host/**` and `plans/host-on-sx.md`. May **import** every subsystem + + the kernel's server/SXTP surface. Do **not** edit `spec/`, `hosts/`, `shared/`, + or subsystem internals — wire to their public APIs only. Host-primitive / server + changes belong in `hosts/` (out of scope) → Blockers. +- **Architecture:** a route maps (method, path) → handler; a handler is an SX fn + `request -> response` that calls subsystem APIs; middleware is composed handlers + (auth via `identity`, permission via `acl`, mute via subsystem prefs). SXTP is the + wire format between host and subsystem-as-service. +- **Migration discipline:** each endpoint moved must be behavior-equivalent to its + Quart original (golden-response test before flip). Keep a migration ledger. +- **Commits:** one feature per commit. Progress log + tick boxes. + +## Architecture sketch + +``` +HTTP request HTTP response + │ ▲ + ▼ │ +native OCaml http server (prod) ──────► lib/host/router.sx + (hosts/ — out of scope) — (method,path) → handler + │ ▲ + ▼ │ +lib/host/middleware.sx lib/host/handler.sx + — auth(identity) ∘ acl ∘ mute ∘ ... — request → subsystem call → response + │ ▲ + ▼ │ +lib/host/sxtp.sx subsystem APIs (feed/search/commerce/…) + — wire format, host↔service — called via public interfaces + │ + └── external edges: SumUp / Ghost / AP / IPFS → injected Python/FFI adapters +``` + +## Phase 1 — Router + handler + one real endpoint +- [ ] `router.sx` — route table, (method,path) match +- [ ] `handler.sx` — request/response model, subsystem dispatch +- [ ] migrate ONE read endpoint (e.g. a feed timeline) end-to-end, golden test +- [ ] `conformance.sh` + scoreboard + +## Phase 2 — Middleware + SXTP +- [ ] `middleware.sx` — composable auth/acl/mute/error layers +- [ ] `sxtp.sx` — host↔subsystem wire format (align with existing spec) +- [ ] migrate a write endpoint (auth + permission + action) + +## Phase 3 — Strangler migration ledger +- [ ] enumerate Quart endpoints; track migrated vs proxied +- [ ] golden-response harness vs the live Quart responses +- [ ] cut over a whole domain (smallest: `likes` or `relations`) as proof + +## Phase 4 — Dream framework layer (gated) +- [ ] gate: `ocaml-on-sx` Phases 1–5 + minimal stdlib green +- [ ] adopt `dream-on-sx` routing/middleware/session ergonomics over the same handlers +- [ ] re-home external adapters as native where replacements land + +## Progress log +(loop fills this in) + +## Blockers +(loop fills this in) diff --git a/plans/identity-on-sx.md b/plans/identity-on-sx.md new file mode 100644 index 00000000..3dd86dcb --- /dev/null +++ b/plans/identity-on-sx.md @@ -0,0 +1,84 @@ +# identity-on-sx: OAuth2, sessions & membership on Erlang + +> **DRAFT outline.** The identity core `acl-on-sx` assumes already exists. `acl` +> answers "may X do Y"; identity answers "who is X, and how did they prove it." +> Depends on `store-on-sx` (grant/audit ledger). Pairs with `acl-on-sx`. + +rose-ash's `account` domain is the OAuth2 authorization server every other app is +a client of: silent SSO, per-app first-party cookies, grant verification, +membership. Sessions and grants are **long-lived, concurrent, individually +addressable, and expire on their own** — that is the actor model. Erlang's +processes + mailboxes map cleanly: a session is a process, token issue/refresh/ +revoke are messages, expiry is a process timeout, and SSO is one process answering +many apps. + +End-state: an Erlang-on-SX layer with the OAuth2 authorization-code + silent +(`prompt=none`) flows as message protocols, a session/grant registry, token +lifecycle (issue/refresh/revoke/introspect), and membership state — all auditable +through the event log, all authorization questions delegated to `acl-on-sx`. + +## Status (rolling) + +`bash lib/identity/conformance.sh` → **0/0** (not yet started) + +## Ground rules + +- **Scope:** only `lib/identity/**` and `plans/identity-on-sx.md`. May **import** + from `lib/erlang/`, and (once they exist) `lib/store/` + `lib/acl/`. Do not edit + substrates. +- **Architecture:** a session/grant is a process holding its own state; the + registry routes messages by subject/client id. Tokens are opaque + introspected, + not self-validating (revocation must be real). Authorization decisions are NOT + made here — `identity` proves identity, `acl` decides permission. +- **Security:** revocation is immediate (kill the process / tombstone the grant); + no decision relies on a token that outlived its grant. Negative answers are + explicit, never "absence of a yes." +- **Commits:** one feature per commit. Progress log + tick boxes. + +## Architecture sketch + +``` +Auth request Token / session + (authorize client scope subject) {:access :refresh :expires :grant} + │ ▲ + ▼ │ +lib/identity/oauth.sx lib/identity/token.sx + — authz-code + prompt=none flows — issue / refresh / revoke / introspect + — as Erlang message protocols — opaque tokens, grant-backed + │ ▲ + ▼ │ +lib/identity/session.sx lib/identity/registry.sx + — session = process, expiry=timeout — route by subject/client; SSO fan-out + │ │ + ▼ ▼ +lib/identity/api.sx ── (identity/login) (identity/grant?) (identity/revoke) ──┐ + │ │ + └──────── grant + audit events → store ; permission? → acl ──────────┘ +``` + +## Phase 1 — Sessions + tokens +- [ ] `session.sx` — session process, create/lookup/expire +- [ ] `token.sx` — issue/introspect/revoke (opaque, grant-backed) +- [ ] `registry.sx` — route by subject/client +- [ ] `api.sx` + tests + scoreboard + conformance.sh + +## Phase 2 — OAuth2 flows +- [ ] authorization-code flow as a message protocol +- [ ] refresh + rotation; revocation cascades to issued tokens +- [ ] tests: full code exchange, refresh, revoke-then-use (must fail) + +## Phase 3 — Silent SSO + membership +- [ ] `prompt=none` cross-app login (one session, many clients) +- [ ] membership state + per-app grant projection +- [ ] grant verification delegated cache (mirror Redis-cache pattern) + +## Phase 4 — Audit + federation +- [ ] every issue/refresh/revoke is a `store` event; `(identity/audit subject)` +- [ ] federated identity (peer-asserted subject) — advisory, trust-gated stub +- [ ] tests: audit completeness, cross-instance subject mapping + +## Progress log +(loop fills this in) + +## Blockers +(loop fills this in) diff --git a/plans/store-on-sx.md b/plans/store-on-sx.md new file mode 100644 index 00000000..2795b722 --- /dev/null +++ b/plans/store-on-sx.md @@ -0,0 +1,92 @@ +# store-on-sx: Event Sourcing on the SX kernel + +> **DRAFT outline.** Foundation subsystem — the durable substrate the other five +> currently fake with in-memory mutable lists. Build this first. + +rose-ash needs durable state: every subsystem (feed log, flow store, mod audit, +search index, acl grants) today hand-rolls an in-memory list that vanishes on +restart. They all secretly want the same thing — an append-only event log with +pure projections over it. Event sourcing makes that one substrate: the log is the +source of truth, state is a fold, durability is an IO boundary. + +This is **substrate-level**, not a guest language. It lives directly on the SX +kernel's IO-suspension primitives (`perform`/`cek-resume` — the third CEK phase) +so a projection can `perform` a read/append and the kernel persists at the +boundary. Storage backends (in-memory, file, later Postgres/IPFS) are injected. + +End-state: an `append`/`fold`/`project`/`snapshot` API with an injectable backend, +optimistic-concurrency on streams, replay + snapshotting, and a subscription hook +so projections (feeds, indices, audit logs) update incrementally. The other +subsystems swap their mutable list for a `store/stream`. + +## Status (rolling) + +`bash lib/store/conformance.sh` → **0/0** (not yet started) + +## Ground rules + +- **Scope:** only `lib/store/**` and `plans/store-on-sx.md`. Do **not** edit + `spec/`, `hosts/`, `shared/`, or `lib//`. May **import** the kernel's + IO-suspension surface (`perform`, the platform IO ops) — verify what's exported + before relying on it. Do not add host primitives; if a needed durable IO op is + missing, file it under Blockers (it belongs in `hosts/` / fed-prims, out of scope). +- **Architecture:** an event is `{:stream :seq :type :at :data}`. A log is an + ordered, append-only vector of events. A projection is `(fold step seed events)`. + Persistence is an injected backend `{:append :read :snapshot-read :snapshot-write}`; + the in-memory backend is the test default, real backends wire in unchanged. +- **Determinism:** replay must be pure — same log → same state, always. No clocks + or randomness inside projections; timestamps live on the event, not the fold. +- **Commits:** one feature per commit. Keep Progress log + tick boxes. + +## Architecture sketch + +``` +Command Read model + (append stream type data) (project stream step seed) + │ ▲ + ▼ │ +lib/store/event.sx lib/store/project.sx + — {:stream :seq :type :at :data} — fold step seed over a stream + — stream ids, seq ordering — incremental: resume from snapshot + │ ▲ + ▼ │ +lib/store/log.sx lib/store/snapshot.sx + — append-only, optimistic concurrency — periodic fold checkpoint + — read range / read-from-seq — replay = snapshot + tail + │ ▲ + ▼ (perform → backend) │ +lib/store/backend.sx lib/store/api.sx + — injected {:append :read ...} — (store/append ...) (store/project ...) + — mem backend (tests) | file | pg — (store/subscribe stream fn) +``` + +## Phase 1 — Log + in-memory backend +- [ ] `lib/store/event.sx` — event record, stream/seq helpers +- [ ] `lib/store/backend.sx` — injectable backend protocol + in-memory impl +- [ ] `lib/store/log.sx` — `append` (optimistic seq), `read`, `read-from` +- [ ] `lib/store/api.sx` — `(store/append ...)`, `(store/events stream)` +- [ ] `lib/store/tests/log.sx` + scoreboard + conformance.sh + +## Phase 2 — Projections + subscriptions +- [ ] `lib/store/project.sx` — `(project stream step seed)`, incremental fold +- [ ] subscription hook — projection re-runs on append +- [ ] concurrency conflict surfaced as a real result, not a crash + +## Phase 3 — Snapshots + replay +- [ ] `lib/store/snapshot.sx` — checkpoint a projection, replay = snapshot + tail +- [ ] compaction policy; replay determinism tests + +## Phase 4 — Durable backend via kernel IO +- [ ] file/log backend driven through `perform` (IO-suspension boundary) +- [ ] crash/restart replay test (mock IO platform) +- [ ] migration notes for swapping mem → durable under a live subsystem + +## Consumers (post-foundation, not in scope here) +feed/-log, flow store, mod/audit, search index, acl grant set all become +`store/stream`s. Track the migration in each subsystem's plan, not this one. + +## Progress log +(loop fills this in) + +## Blockers +(loop fills this in)