mono/docs/masterplan-sprint.md

# Rose Ash: Fortnightly Sprints

**Fit the task to the timescale, not vice versa.**

The golden rule of project management. Don't take a 20-week plan and cram it into 2 weeks — that's denial. Ask instead: what's the highest-value thing you can *finish* — not start, finish — in two weeks?

Each sprint ships one or two deliverables. Both complete, both deployed, both making everything after easier. The master plan is still the map. You walk it two steps at a time.

---

## Sprint 1: Sexp Content + Sexp Wire (Weeks 1-2)

### Week 1: Posts become sexp

Ghost still runs for membership and newsletters. Don't touch that. Just the content path.

**Deliverable: every post is sexp in the database, rendered through the sexp evaluator, editable through the existing editor.**

- [ ] Lexical → sexp converter script (one-time migration)
- [ ] Add `body_sexp` column to Post model (Alembic migration)
- [ ] Run converter against all posts
- [ ] Diff HTML output: Lexical render vs sexp render for every post
- [ ] Fix conversion gaps (embeds, cards, special blocks)
- [ ] Switch rendering pipeline: `post_data()` reads `body_sexp`
- [ ] Editor save endpoint writes sexp directly to DB
- [ ] Native media upload endpoints (image, audio, files) — retarget from Ghost proxy
- [ ] Native OEmbed lookup endpoint
- [ ] Deploy, verify every page renders correctly

### Week 2: Fragment endpoints return sexp

HTTP+HMAC stays as transport. The change is what travels over it.

**Deliverable: `fetch_fragment` returns sexp trees instead of HTML strings. Callers render at the boundary.**

- [ ] Relations service returns raw sexp (already renders from sexp — just stop calling `to_html()`)
- [ ] Blog renders relations sexp to HTML at the route level
- [ ] Events renders relations sexp to HTML at the route level
- [ ] Market renders relations sexp to HTML at the route level
- [ ] Callers can now filter, reorder, transform fragments before rendering
- [ ] Measure: is this faster, slower, or same as HTML fragments?
- [ ] Unit tests for sexp fragment round-trip (parse → filter → render)

### Sprint 1 outcome

Posts are sexp. Internal fragments are sexp. Ghost still runs but only for membership/newsletters — the content path is clean. Everything that follows builds on this.

---

## Sprint 2: Kill Ghost + New Relations (Weeks 3-4)

### Week 3: Membership decoupling

**Deliverable: all Ghost membership/newsletter infrastructure replaced with native equivalents.**

- [ ] Rename `ghost_labels` → `labels`, drop `ghost_id`
- [ ] Rename `ghost_tiers` → `membership_tiers`, drop `ghost_id`
- [ ] Clean User model (drop `ghost_id`, `ghost_status`, `ghost_subscribed`, `ghost_note`, `ghost_raw`)
- [ ] Add `membership_tier_id`, `membership_status` to User
- [ ] Native `subscriptions` table replacing `ghost_subscriptions`
- [ ] Wire Stripe directly on orders service (Checkout + Webhooks)
- [ ] Native newsletter model + `user_newsletters`
- [ ] Email sending via SMTP/SES
- [ ] Newsletter templates as sexp → email-safe HTML
- [ ] Post → email campaign workflow

### Week 4: Delete Ghost + add entity relations

**Deliverable: Ghost is gone. New entity relations are live.**

- [ ] Delete `blog/bp/blog/ghost/` directory
- [ ] Delete `ghost_db.py`, `web_hooks/`, `ghost_admin_token.py`, `ghost_membership.py`
- [ ] Remove Ghost Docker service, all `GHOST_*` env vars
- [ ] Alembic migration: drop all `ghost_id` columns
- [ ] Rename model files: `ghost_content.py` → `content.py`, `ghost_membership_entities.py` → `membership.py`
- [ ] Add new `defrelation`s: `post->post`, `market->product`, `calendar->calendar_entry`, `page->marketplace`
- [ ] Migrate `CalendarEntryPost` junction → `ContainerRelation`
- [ ] Verify: entire platform runs with zero Ghost code

### Sprint 2 outcome

Ghost is dead. Membership is native. New entity relations work. The platform is self-contained.

---

## Sprint 3: Cart Split + Sexp Pages (Weeks 5-6)

### Week 5: Cart microservices split

**Deliverable: cart is thin CRUD. Likes, orders, and PageConfig are in their proper homes.**

- [ ] Scaffold likes service (port 8009)
- [ ] Move PageConfig to blog service
- [ ] Orders service owns Order/OrderItem, checkout, Stripe/SumUp webhooks
- [ ] Cart becomes CartItem CRUD + checkout delegation
- [ ] Fragment cleanup: move remaining domain templates out of `shared/templates/`

### Week 6: Sexp page layouts

**Deliverable: page layouts are sexp components, not Jinja template inheritance.**

- [ ] Base page layout as sexp (head, nav, main, footer)
- [ ] Blog post page layout
- [ ] Market product page layout
- [ ] Event calendar page layout
- [ ] Component composition replaces `{% extends "base.html" %}`
- [ ] Content negotiation: same route returns HTML or sexp based on `Accept` header

### Sprint 3 outcome

Services are properly bounded. Pages are sexp top to bottom. The `Accept: application/sexp` header works — the door to client-side sexp is open.

---

## Sprint 4: Internal Protocol (Weeks 7-8)

### Week 7: Python sexp client/server library

**Deliverable: working `SexpConnection` class, Quart handler, drop-in `fetch_sexp()`.**

- [ ] Wire format: length-prefixed sexp over Unix sockets
- [ ] `SexpConnection`: connect, send sexp, receive sexp, handle streams
- [ ] Quart integration: handler accepts sexp requests, returns sexp responses
- [ ] `fetch_sexp()` — unified replacement for `fetch_data()` + `fetch_fragment()`
- [ ] Error model: `(err :code 404 :message "not found")`
- [ ] Unit tests for protocol library

### Week 8: Internal mesh migration

**Deliverable: all inter-service communication runs on native sexp protocol.**

- [ ] Blog ↔ relations on sexp
- [ ] Events ↔ relations on sexp
- [ ] Market ↔ relations on sexp
- [ ] Cart ↔ orders on sexp
- [ ] Account ↔ all services on sexp
- [ ] Benchmark: latency/throughput vs HTTP+HMAC
- [ ] HTTP kept as fallback for external/third-party calls

### Sprint 4 outcome

The protocol is real. Running in production. Carrying every inter-service call. Battle-tested on real traffic.

---

## Sprint 5: sexpr.js (Weeks 9-10)

### Week 9: Core runtime

**Deliverable: sexpr.js parses, renders, and mutates the DOM from sexp.**

- [ ] Sexp parser in JS (<5KB gzipped)
- [ ] DOM renderer: sexp tree → DOM nodes
- [ ] `swap!`, `batch!`, `class!` mutation primitives
- [ ] `request!` — fetch sexp from server, apply mutations
- [ ] Component system: `defcomp`, slots

### Week 10: Integration + caching

**Deliverable: sexpr.js on every rose-ash page, with content-addressed caching.**

- [ ] `<script src="sexpr.js">` on every page — progressive enhancement
- [ ] Partial page updates via sexp mutations (replaces full reload for nav, cart, likes)
- [ ] Content-addressed caching: hash component/post body, cache in localStorage
- [ ] Server sends hash manifest — unchanged content served from cache
- [ ] DevTools: sexp inspector in browser console

### Sprint 5 outcome

The client speaks sexp. Pages load faster. Navigation doesn't reload. Components cache locally. Tier 1 is live.

---

## Sprint 6: Federation + Stability (Weeks 11-12)

### Week 11: AP over sexp

**Deliverable: two rose-ash instances federate using sexp activities.**

- [ ] Sexp-formatted activities (Create, Follow, Like, Accept)
- [ ] Federation test: two instances in Docker, full activity flow
- [ ] Identity verification over sexp (RSA signatures in sexp envelope)
- [ ] Cross-instance content rendering (sexp posts display correctly on remote instance)

### Week 12: Harden + deploy

**Deliverable: everything stable, tested, and deployed.**

- [ ] Unit test coverage for all new code (protocol, converter, newsletter, likes)
- [ ] Deploy Tier 0 scalability (DB split, PgBouncer, Hypercorn workers)
- [ ] Bug sweep: end-to-end test every user flow
- [ ] Performance baseline: measure page load, inter-service latency, cache hit rates
- [ ] Documentation: update CLAUDE.md with new architecture

### Sprint 6 outcome

Platform is stable, federated, performant, and documented. The protocol has been tested across network boundaries.

---

## After Sprint 6: The 10%

These are real projects, not sprint tasks. Build them when they're needed:

- **Rust client library + native client** — when you want Tier 2 performance
- **Browser extension** — when sexpr.js proves the concept and you want Tier 1.5
- **Client-as-node / IPFS / GPU mesh** — when the cooperative network has multiple members with hardware
- **Multi-cooperative commerce** — when a second co-op actually exists and wants to federate
- **Scalability Tiers 1-3** — when traffic hits Tier 0 limits

---

## The Rhythm

```
Each sprint:
  Monday-Thursday:  Build. AI generates, you steer and test.
  Friday:           Deploy to dev, verify, fix what broke.
  Weekend:          Rest or think about next sprint.

Each week:
  One deliverable. Finished. Deployed. Working.
```

**12 weeks, 12 deliverables, each one complete.** Not 20 weeks of partial progress. Not 2 weeks of rushed everything. The right amount of work for the time available.

---

## Sprint Schedule

```
Sprint 1 (W1-2):   Sexp content + sexp wire          ← content path is clean
Sprint 2 (W3-4):   Kill Ghost + new relations         ← platform is self-contained
Sprint 3 (W5-6):   Cart split + sexp pages            ← services bounded, pages are sexp
Sprint 4 (W7-8):   Internal sexp protocol             ← protocol is real and running
Sprint 5 (W9-10):  sexpr.js client runtime             ← client speaks sexp
Sprint 6 (W11-12): Federation + stability              ← federated and stable
```

Each sprint builds on the last. Each is valuable on its own. If you stop after any sprint, you've shipped something real.