# Rose Ash Decoupling Plan

## Context

The four Rose Ash apps (blog, market, cart, events) are tightly coupled through:
- A shared model layer (`blog/shared_lib/models/`) containing ALL models for ALL apps
- Cross-app foreign keys (calendars→posts, cart_items→market_places, calendar_entries→orders, etc.)
- `Post` as the universal parent — calendars, markets, page_configs all hang off `post_id`
- Internal HTTP calls for menu items, cart summaries, and login adoption

This makes it impossible to attach services to anything other than a Post, and means apps can't have independent databases. The goal is to decouple so apps are independently deployable, new services can be added easily, and the composition of "what's attached to what" is defined in a separate glue layer.

---

## Phase 1: Extract `shared_lib` out of `blog/`

**What:** Move shared infrastructure into a top-level `shared/` package. Split models by ownership.

### New structure

```
/root/rose-ash/
  shared/                              # Extracted from blog/shared_lib/
    db/base.py, session.py             # Unchanged
    models/                            # ONLY shared models:
      user.py                          #   User (used by all apps)
      kv.py                            #   KV (settings)
      magic_link.py                    #   MagicLink (auth)
      ghost_membership_entities.py     #   Ghost labels/newsletters/tiers/subscriptions
      menu_item.py                     #   MenuItem (temporary, moves to glue in Phase 4)
    infrastructure/                    # Renamed from shared/
      factory.py                       #   create_base_app()
      internal_api.py                  #   HTTP client for inter-app calls
      context.py                       #   base_context()
      user_loader.py, jinja_setup.py, cart_identity.py, cart_loader.py, urls.py, http_utils.py
    browser/                           # Renamed from suma_browser/
      (middleware, templates, csrf, errors, filters, redis, payments, authz)
    config.py, config/
    alembic/, static/, editor/

  blog/models/                         # Blog-owned models
    ghost_content.py                   #   Post, Author, Tag, PostAuthor, PostTag, PostLike
    snippet.py                         #   Snippet
    tag_group.py                       #   TagGroup, TagGroupTag

  market/models/                       # Market-owned models
    market.py                          #   Product, CartItem, NavTop, NavSub, Listing, etc.
    market_place.py                    #   MarketPlace

  cart/models/                         # Cart-owned models
    order.py                           #   Order, OrderItem
    page_config.py                     #   PageConfig

  events/models/                       # Events-owned models
    calendars.py                       #   Calendar, CalendarEntry, CalendarSlot, Ticket, TicketType
```

### Key changes
- Update `path_setup.py` in each app to add project root to `sys.path`
- Update all `from models import X` → `from blog.models import X` / `from shared.models import X` etc.
- Update `from db.base import Base` → `from shared.db.base import Base` in every model file
- Update `from shared.factory import` → `from shared.infrastructure.factory import` in each `app.py`
- Alembic `env.py` imports from all locations so `Base.metadata` sees every table
- Add a transitional compat layer in old location that re-exports everything (remove later)

### Critical files to modify
- `blog/app.py` (line 9: `from shared.factory`), `market/app.py`, `cart/app.py`, `events/app.py`
- `blog/shared_lib/shared/factory.py` → `shared/infrastructure/factory.py`
- Every model file (Base import)
- `blog/shared_lib/alembic/env.py` → `shared/alembic/env.py`
- Each app's `path_setup.py`

### Verify
- All four apps start without import errors
- `alembic check` produces no diff (schema unchanged)
- All routes return correct responses
- Internal API calls between apps still work

---

## Phase 2: Event Infrastructure + Logging

**What:** Add the durable event system (transactional outbox) and shared structured logging.

### 2a. DomainEvent model (the outbox)

New file: `shared/models/domain_event.py`

```python
class DomainEvent(Base):
    __tablename__ = "domain_events"

    id            = Integer, primary_key
    event_type    = String(128), indexed     # "calendar.created", "order.completed"
    aggregate_type = String(64)              # "calendar", "order"
    aggregate_id  = Integer                  # ID of the thing that changed
    payload       = JSONB                    # Event-specific data
    state         = String(20), default "pending"  # pending → processing → completed | failed
    attempts      = Integer, default 0
    max_attempts  = Integer, default 5
    last_error    = Text, nullable
    created_at    = DateTime, server_default now()
    processed_at  = DateTime, nullable
```

The critical property: `emit_event()` writes to this table **in the same DB transaction** as the domain change. If the app crashes after commit, the event is already persisted. If it crashes before commit, neither the domain change nor the event exists. This is atomic.

### 2b. Event bus

New directory: `shared/events/`

```
shared/events/
    __init__.py        # exports emit_event, register_handler, EventProcessor
    bus.py             # emit_event(session, event_type, aggregate_type, aggregate_id, payload)
                       # register_handler(event_type, async_handler_fn)
    processor.py       # EventProcessor: polls domain_events table, dispatches to handlers
```

**`emit_event(session, ...)`** — called within service functions, writes to outbox in current transaction
**`register_handler(event_type, fn)`** — called at app startup (by glue layer) to register handlers
**`EventProcessor`** — background polling loop:
  1. `SELECT ... FROM domain_events WHERE state='pending' FOR UPDATE SKIP LOCKED`
  2. Run all registered handlers for that event_type
  3. Mark completed or retry on failure
  4. Runs as an `asyncio.create_task` within each app process (started in `factory.py`)

### 2c. Structured logging

New directory: `shared/logging/`

```
shared/logging/
    __init__.py
    setup.py           # configure_logging(app_name), get_logger(name)
```

- JSON-structured output to stdout (timestamp, level, app, message, plus optional fields: event_type, user_id, request_id, duration_ms)
- `configure_logging(app_name)` called in `create_base_app()`
- All apps get consistent log format; in production these go to a log aggregator

### 2d. Integration

Update `shared/infrastructure/factory.py`:
- Call `configure_logging(name)` at app creation
- Start `EventProcessor` as background task in `@app.before_serving`
- Stop it in `@app.after_serving`

### Verify
- `domain_events` table exists after migration
- Call `emit_event()` in a test, verify row appears in table
- `EventProcessor` picks up pending events and marks them completed
- JSON logs appear on stdout with correct structure
- No behavioral changes — this is purely additive infrastructure

---

## Phase 3: Generic Container Concept

**What:** Replace cross-app `post_id` FKs with `container_type` + `container_id` soft references.

### Models to change

**Calendar** (`events/models/calendars.py`):
```python
# REMOVE: post_id = Column(Integer, ForeignKey("posts.id"), ...)
# REMOVE: post = relationship("Post", ...)
# ADD:
container_type = Column(String(32), nullable=False)  # "page", "market", etc.
container_id = Column(Integer, nullable=False)
```

**MarketPlace** (`market/models/market_place.py`):
```python
# Same pattern: remove post_id FK, add container_type + container_id
```

**PageConfig** (`cart/models/page_config.py`):
```python
# Same pattern
```

**CalendarEntryPost** → rename to **CalendarEntryContent**:
```python
# REMOVE: post_id FK
# ADD: content_type + content_id (generic reference)
```

### From Post model (`blog/models/ghost_content.py`), remove:
- `calendars` relationship
- `markets` relationship
- `page_config` relationship
- `calendar_entries` relationship (via CalendarEntryPost)
- `menu_items` relationship (moves to glue in Phase 4)

### Helper in `shared/containers.py`:
```python
class ContainerType:
    PAGE = "page"
    # Future: MARKET = "market", GROUP = "group", etc.

def container_filter(model, container_type, container_id):
    """Return SQLAlchemy filter clauses."""
    return [model.container_type == container_type, model.container_id == container_id]
```

### Three-step migration (non-breaking)
1. **Add columns** (nullable) — keeps old post_id FK intact
2. **Backfill** — `UPDATE calendars SET container_type='page', container_id=post_id`; make NOT NULL
3. **Drop old FK** — remove post_id column and FK constraint

### Update all queries
Key files that reference `Calendar.post_id`, `MarketPlace.post_id`, `PageConfig.post_id`:
- `events/app.py` (~line 108)
- `market/app.py` (~line 119)
- `cart/app.py` (~line 131)
- `cart/bp/cart/services/checkout.py` (lines 77-85, 160-163) — `resolve_page_config()` and `create_order_from_cart()`
- `cart/bp/cart/services/page_cart.py`
- `cart/bp/cart/api.py`

All change from `X.post_id == post.id` to `X.container_type == "page", X.container_id == post.id`.

### Verify
- Creating a calendar/market/page_config uses container_type + container_id
- Cart checkout still resolves correct page config via container references
- No cross-app FKs remain for these three models
- Alembic migration is clean

---

## Phase 4: Glue Layer

**What:** New top-level `glue/` package that owns container relationships, navigation, and event handlers.

### Structure

```
/root/rose-ash/glue/
    __init__.py
    models/
        container_relation.py      # Parent-child container relationships
        menu_node.py               # Navigation tree (replaces MenuItem)
    services/
        navigation.py              # Build menu from relationship tree
        relationships.py           # attach_child(), get_children(), detach_child()
    handlers/
        calendar_handlers.py       # on calendar attached → rebuild nav
        market_handlers.py         # on market attached → rebuild nav
        order_handlers.py          # on order completed → confirm calendar entries
        login_handlers.py          # on login → adopt anonymous cart/calendar items
    setup.py                       # Registers all handlers at app startup
```

### ContainerRelation model
```python
class ContainerRelation(Base):
    __tablename__ = "container_relations"
    id, parent_type, parent_id, child_type, child_id, sort_order, label, created_at, deleted_at
    # Unique constraint: (parent_type, parent_id, child_type, child_id)
```

This is the central truth about "what's attached to what." A page has calendars and markets attached to it — defined here, not by FKs on the calendar/market tables.

### MenuNode model (replaces MenuItem)
```python
class MenuNode(Base):
    __tablename__ = "menu_nodes"
    id, container_type, container_id,
    parent_id (self-referential tree), sort_order, depth,
    label, slug, href, icon, feature_image,
    created_at, updated_at, deleted_at
```

This is a **cached navigation tree** built FROM ContainerRelations. A page doesn't know it has markets — but its MenuNode has child MenuNodes for the market because the glue layer put them there.

### Navigation service (`glue/services/navigation.py`)
- `get_navigation_tree(session)` → nested dict for templates (replaces `/internal/menu-items` API)
- `rebuild_navigation(session)` → reads ContainerRelations, creates/updates MenuNodes
- Called by event handlers when relationships change

### Relationship service (`glue/services/relationships.py`)
- `attach_child(session, parent_type, parent_id, child_type, child_id)` → creates ContainerRelation + emits `container.child_attached` event
- `get_children(session, parent_type, parent_id, child_type=None)` → query children
- `detach_child(...)` → soft delete + emit `container.child_detached` event

### Event handlers (the "real code" in the glue layer)
```python
# glue/handlers/calendar_handlers.py
@handler("container.child_attached")
async def on_child_attached(payload, session):
    if payload["child_type"] in ("calendar", "market"):
        await rebuild_navigation(session)

# glue/handlers/order_handlers.py  (Phase 5 but registered here)
@handler("order.created")
async def on_order_created(payload, session):
    # Confirm calendar entries for this order
    ...

# glue/handlers/login_handlers.py  (Phase 5 but registered here)
@handler("user.logged_in")
async def on_user_logged_in(payload, session):
    # Adopt anonymous cart items and calendar entries
    ...
```

### Replace menu_items flow
**Old:** Each app calls `GET /internal/menu-items` → coop queries MenuItem → returns JSON
**New:** Each app calls `glue.services.navigation.get_navigation_tree(g.s)` → direct DB query of MenuNode

Update context functions in all four `app.py` files:
```python
# REMOVE: menu_data = await api_get("coop", "/internal/menu-items")
# ADD:    from glue.services.navigation import get_navigation_tree
#         ctx["menu_items"] = await get_navigation_tree(g.s)
```

### Data migration
- Backfill `menu_nodes` from existing `menu_items` + posts
- Backfill `container_relations` from existing calendar/market/page_config container references
- Deprecate (then remove) old MenuItem model and `/internal/menu-items` endpoint
- Update menu admin UI (`blog/bp/menu_items/`) to manage ContainerRelations + MenuNodes

### Verify
- Navigation renders correctly in all four apps without HTTP calls
- Adding a market to a page (via ContainerRelation) triggers nav rebuild and market appears in menu
- Adding a calendar to a page does the same
- Menu admin UI works with new models

---

## Phase 5: Event-Driven Cross-App Workflows

**What:** Replace remaining cross-app FKs and HTTP calls with event-driven flows.

### 5a. Replace `CalendarEntry.order_id` FK with soft reference

```python
# REMOVE: order_id = Column(Integer, ForeignKey("orders.id"), ...)
# ADD:    order_ref_id = Column(Integer, nullable=True, index=True)
# (No FK constraint — just stores the order ID as an integer)
```

Same for `Ticket.order_id`. Three-step migration (add, backfill, drop FK).

### 5b. Replace `CartItem.market_place_id` FK with soft reference
```python
# REMOVE: market_place_id = ForeignKey("market_places.id")
# ADD:    market_ref_id = Column(Integer, nullable=True, index=True)
```

### 5c. Event-driven order completion
Currently `cart/bp/cart/services/checkout.py` line 166 directly writes `CalendarEntry` rows (cross-domain). Replace:

```python
# In create_order_from_cart(), instead of direct UPDATE on CalendarEntry:
await emit_event(session, "order.created", "order", order.id, {
    "order_id": order.id,
    "user_id": user_id,
    "calendar_entry_ids": [...],
})
```

Glue handler picks it up and updates calendar entries via events-domain code.

### 5d. Event-driven login adoption
Currently `blog/bp/auth/routes.py` line 265 calls `POST /internal/cart/adopt`. Replace:

```python
# In magic() route, instead of api_post("cart", "/internal/cart/adopt"):
await emit_event(session, "user.logged_in", "user", user_id, {
    "user_id": user_id,
    "anonymous_session_id": anon_session_id,
})
```

Glue handler adopts cart items and calendar entries.

### 5e. Remove cross-domain ORM relationships
From models, remove:
- `Order.calendar_entries` (relationship to CalendarEntry)
- `CalendarEntry.order` (relationship to Order)
- `Ticket.order` (relationship to Order)
- `CartItem.market_place` (relationship to MarketPlace)

### 5f. Move cross-domain queries to glue services
`cart/bp/cart/services/checkout.py` currently imports `CalendarEntry`, `Calendar`, `MarketPlace` directly. Move these queries to glue service functions that bridge the domains:
- `glue/services/cart_calendar.py` — query calendar entries for a cart identity
- `glue/services/page_resolution.py` — determine which page/container a cart belongs to using ContainerRelation

### Final FK audit after Phase 5
All remaining FKs are either:
- **Within the same app domain** (Order→OrderItem, Calendar→CalendarSlot, etc.)
- **To shared models** (anything→User)
- **One pragmatic exception:** `OrderItem.product_id → products.id` (cross cart→market, but OrderItem already snapshots title/price, so this FK is just for reporting)

### Verify
- Login triggers `user.logged_in` event → cart/calendar adoption happens via glue handler
- Order creation triggers `order.created` event → calendar entries confirmed via glue handler
- No cross-app FKs remain (except the pragmatic OrderItem→Product)
- All apps could theoretically point at separate databases
- Event processor reliably processes and retries all events

---

## Execution Order

Each phase leaves the system fully working. No big-bang migration.

| Phase | Risk | Size | Depends On |
|-------|------|------|------------|
| 1. Extract shared_lib | Low (mechanical refactor) | Medium | Nothing |
| 2. Event infra + logging | Low (purely additive) | Small | Phase 1 |
| 3. Generic containers | Medium (schema + query changes) | Medium | Phase 1 |
| 4. Glue layer | Medium (new subsystem, menu migration) | Large | Phases 2 + 3 |
| 5. Event-driven workflows | Medium (behavioral change in checkout/login) | Medium | Phase 4 |

Phases 2 and 3 can run in parallel after Phase 1. Phase 4 needs both. Phase 5 needs Phase 4.