Files
mono/.claude/plans/glittery-zooming-hummingbird.md
giles 094b6c55cd Fix AP blueprint cross-DB queries + harden Ghost sync init
AP blueprints (activitypub.py, ap_social.py) were querying federation
tables (ap_actor_profiles etc.) on g.s which points to the app's own DB
after the per-app split. Now uses g._ap_s backed by get_federation_session()
for non-federation apps.

Also hardens Ghost sync before_app_serving to catch/rollback on failure
instead of crashing the Hypercorn worker.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-25 14:06:42 +00:00

426 lines
22 KiB
Markdown

# Phase 6: Full Cross-App Decoupling via Glue Services
## Context
Phases 1-5 are complete. All cross-domain FK constraints have been dropped (except `OrderItem.product_id` and `CartItem.product_id`/`market_place_id`/`user_id`, kept as pragmatic exceptions). Cross-domain **writes** go through glue services.
However, **25+ cross-app model imports** remain — apps still `from blog.models.ghost_content import Post`, `from market.models.market import CartItem`, etc. This means every app needs every other app's code on disk to start, making separate databases or independent deployment impossible.
**Goal:** Eliminate all cross-app model imports. Every app only imports from its own `models/`, from `shared/`, and from `glue/`. Cross-domain access goes through glue services. After this phase, each app could theoretically run against its own database.
---
## Inventory of Cross-App Imports to Eliminate
### Cart app imports (9 files, 4 foreign models):
| File | Import | Usage |
|------|--------|-------|
| `cart/bp/cart/api.py` | `market.models.market.CartItem` | Query cart items |
| `cart/bp/cart/api.py` | `market.models.market_place.MarketPlace` | Filter by container |
| `cart/bp/cart/api.py` | `events.models.calendars.CalendarEntry, Calendar` | Query pending entries |
| `cart/bp/cart/api.py` | `blog.models.ghost_content.Post` | Resolve page slug |
| `cart/bp/cart/services/checkout.py` | `market.models.market.Product, CartItem` | Find cart items, validate products |
| `cart/bp/cart/services/checkout.py` | `events.models.calendars.CalendarEntry, Calendar` | Resolve page containers |
| `cart/bp/cart/services/checkout.py` | `market.models.market_place.MarketPlace` | Get container_id |
| `cart/bp/cart/services/page_cart.py` | `market.models.market.CartItem` | Query page cart |
| `cart/bp/cart/services/page_cart.py` | `market.models.market_place.MarketPlace` | Join for container |
| `cart/bp/cart/services/page_cart.py` | `events.models.calendars.CalendarEntry, Calendar` | Query page entries |
| `cart/bp/cart/services/page_cart.py` | `blog.models.ghost_content.Post` | Batch-load posts |
| `cart/bp/cart/services/get_cart.py` | `market.models.market.CartItem` | Query cart items |
| `cart/bp/cart/services/calendar_cart.py` | `events.models.calendars.CalendarEntry` | Query pending entries |
| `cart/bp/cart/services/clear_cart_for_order.py` | `market.models.market.CartItem` | Soft-delete items |
| `cart/bp/cart/services/clear_cart_for_order.py` | `market.models.market_place.MarketPlace` | Filter by page |
| `cart/bp/orders/routes.py` | `market.models.market.Product` | Join for search |
| `cart/bp/order/routes.py` | `market.models.market.Product` | Load product details |
| `cart/app.py` | `blog.models.ghost_content.Post` | Page slug hydration |
### Blog app imports (8 files, 3 foreign models):
| File | Import | Usage |
|------|--------|-------|
| `blog/bp/post/admin/routes.py` | `cart.models.page_config.PageConfig` (3 places) | Load/update page config |
| `blog/bp/post/admin/routes.py` | `events.models.calendars.Calendar` (3 places) | Query calendars |
| `blog/bp/post/admin/routes.py` | `market.models.market_place.MarketPlace` (3 places) | Query/create/delete markets |
| `blog/bp/post/services/markets.py` | `market.models.market_place.MarketPlace` | Create/delete markets |
| `blog/bp/post/services/markets.py` | `cart.models.page_config.PageConfig` | Check feature flag |
| `blog/bp/post/services/entry_associations.py` | `events.models.calendars.CalendarEntry, CalendarEntryPost, Calendar` | Post-entry associations |
| `blog/bp/post/routes.py` | `events.models.calendars.Calendar` | Page context |
| `blog/bp/post/routes.py` | `market.models.market_place.MarketPlace` | Page context |
| `blog/bp/blog/ghost_db.py` | `cart.models.page_config.PageConfig` | Query page configs |
| `blog/bp/blog/ghost/ghost_sync.py` | `cart.models.page_config.PageConfig` | Sync page config |
| `blog/bp/blog/services/posts_data.py` | `events.models.calendars.CalendarEntry, CalendarEntryPost` | Fetch associated entries |
### Events app imports (5 files, 3 foreign models):
| File | Import | Usage |
|------|--------|-------|
| `events/app.py` | `blog.models.ghost_content.Post` | Page slug hydration |
| `events/app.py` | `market.models.market_place.MarketPlace` | Context processor |
| `events/bp/markets/services/markets.py` | `market.models.market_place.MarketPlace` | Create/delete markets |
| `events/bp/markets/services/markets.py` | `blog.models.ghost_content.Post` | Validate post exists |
| `events/bp/markets/routes.py` | `market.models.market_place.MarketPlace` | Query/delete markets |
| `events/bp/calendars/services/calendars.py` | `blog.models.ghost_content.Post` | Validate post exists |
| `events/bp/calendar_entry/services/post_associations.py` | `blog.models.ghost_content.Post` | Manage post-entry assocs |
| `events/bp/payments/routes.py` | `cart.models.page_config.PageConfig` | Load/update SumUp config |
### Market app imports (1 file):
| File | Import | Usage |
|------|--------|-------|
| `market/app.py` | `blog.models.ghost_content.Post` | Page slug hydration |
### Glue layer imports (2 files):
| File | Import | Usage |
|------|--------|-------|
| `glue/services/cart_adoption.py` | `market.models.market.CartItem` | Adopt cart items |
| `glue/services/cart_adoption.py` | `events.models.calendars.CalendarEntry` | Adopt entries |
| `glue/services/order_lifecycle.py` | `events.models.calendars.CalendarEntry, Calendar` | Claim/confirm entries |
---
## Design Decisions
1. **Glue services return ORM objects** (not dicts) when the model is standalone — PageConfig, MarketPlace, Calendar, CalendarEntry. This avoids template changes and keeps SQLAlchemy lazy-load working.
2. **Glue services for Post return dicts** — other apps only need `{id, slug, title, is_page, feature_image}`. Returning the full ORM object would couple them to the blog schema.
3. **CartItem stays in `market/models/market.py`** — it has FKs to `products.id`, `market_places.id`, and `users.id`, plus relationships to `Product`, `MarketPlace`, and `User`. Moving it to cart/ would just reverse the cross-app import direction. Instead, cart reads CartItem through glue.
4. **OrderItem.product relationship uses string forward-ref** — already works via SQLAlchemy string resolution as long as Product is registered in the mapper. Glue setup handles this.
5. **Glue services are allowed to import from any app's models** — that's the glue layer's job. Apps call glue; glue touches models.
6. **blog/bp/post/services/markets.py and entry_associations.py move to glue** — these are pure cross-domain CRUD (blog writes to MarketPlace, blog reads CalendarEntry). They belong in glue.
---
## Step 1: Glue service for pages (Post access)
New file: `glue/services/pages.py`
Provides dict-based Post access for non-blog apps:
```python
async def get_page_by_slug(session, slug) -> dict | None:
"""Return {id, slug, title, is_page, feature_image, ...} or None."""
async def get_page_by_id(session, post_id) -> dict | None:
"""Return page dict by id."""
async def get_pages_by_ids(session, post_ids) -> dict[int, dict]:
"""Batch-load pages. Returns {id: page_dict}."""
async def page_exists(session, post_id) -> bool:
"""Check if post exists (for validation before creating calendars/markets)."""
async def is_page(session, post_id) -> bool:
"""Check if post exists and is_page=True."""
async def search_posts(session, query, page=1, per_page=10) -> tuple[list[dict], int]:
"""Search posts by title (for events post_associations)."""
```
All functions import `from blog.models.ghost_content import Post` internally.
**Files changed:**
- `market/app.py` — replace `from blog.models.ghost_content import Post` with `from glue.services.pages import get_page_by_slug`
- `events/app.py` — same
- `cart/app.py` — same
- `cart/bp/cart/api.py` — replace Post import with `from glue.services.pages import get_page_by_slug`
- `cart/bp/cart/services/page_cart.py` — replace Post import with `from glue.services.pages import get_pages_by_ids`
- `events/bp/calendars/services/calendars.py` — replace `from blog.models.ghost_content import Post` with `from glue.services.pages import page_exists, is_page`
- `events/bp/markets/services/markets.py` — replace `from blog.models.ghost_content import Post` with `from glue.services.pages import page_exists, is_page`
---
## Step 2: Glue service for page config
New file: `glue/services/page_config.py`
```python
async def get_page_config(session, post_id) -> PageConfig | None:
"""Load PageConfig for a page."""
async def get_or_create_page_config(session, post_id) -> PageConfig:
"""Load or create PageConfig. Emits container.child_attached if created."""
async def get_page_configs_by_ids(session, post_ids) -> dict[int, PageConfig]:
"""Batch-load PageConfigs by container_id."""
```
Imports `from cart.models.page_config import PageConfig` internally.
**Files changed:**
- `blog/bp/post/admin/routes.py` — replace `from cart.models.page_config import PageConfig` with glue service calls
- `blog/bp/post/services/markets.py` — replace PageConfig import
- `blog/bp/blog/ghost_db.py` — replace PageConfig import
- `blog/bp/blog/ghost/ghost_sync.py` — replace PageConfig import
- `events/bp/payments/routes.py` — replace PageConfig import
- `cart/bp/cart/services/checkout.py` — replace `from models.page_config import PageConfig` stays (same app)
---
## Step 3: Glue service for calendars (events access from blog)
New file: `glue/services/calendars.py`
```python
async def get_calendars_for_page(session, post_id) -> list[Calendar]:
"""Return active calendars for a page."""
async def get_calendar_entries_for_posts(session, post_ids) -> dict[int, list]:
"""Fetch confirmed CalendarEntries associated with posts (via CalendarEntryPost).
Returns {post_id: [entry, ...]}."""
```
Move and adapt from `blog/bp/post/services/entry_associations.py`:
```python
async def toggle_entry_association(session, post_id, entry_id) -> tuple[bool, str | None]:
async def get_post_entry_ids(session, post_id) -> set[int]:
async def get_associated_entries(session, post_id, page=1, per_page=10) -> dict:
```
These functions import from `events.models.calendars` internally.
**Files changed:**
- `blog/bp/post/routes.py` — replace `from events.models.calendars import Calendar` + `from market.models.market_place import MarketPlace` with glue service calls
- `blog/bp/post/admin/routes.py` — replace Calendar imports with glue service calls
- `blog/bp/post/services/entry_associations.py`**delete file**, moved to glue
- `blog/bp/blog/services/posts_data.py` — replace `from events.models.calendars import CalendarEntry, CalendarEntryPost` with glue service call
---
## Step 4: Glue service for marketplaces
New file: `glue/services/marketplaces.py`
```python
async def get_marketplaces_for_page(session, post_id) -> list[MarketPlace]:
"""Return active marketplaces for a page."""
async def create_marketplace(session, post_id, name) -> MarketPlace:
"""Create marketplace (validates page exists via pages service)."""
async def soft_delete_marketplace(session, post_slug, market_slug) -> bool:
"""Soft-delete a marketplace."""
```
Move the logic from `blog/bp/post/services/markets.py` and `events/bp/markets/services/markets.py` (they're nearly identical).
**Files changed:**
- `blog/bp/post/services/markets.py`**delete file**, moved to glue
- `blog/bp/post/admin/routes.py` — replace MarketPlace imports + service calls with glue
- `blog/bp/post/routes.py` — replace MarketPlace import with glue service
- `events/bp/markets/services/markets.py`**delete file**, moved to glue
- `events/bp/markets/routes.py` — replace MarketPlace import, use glue
- `events/app.py` — replace MarketPlace import with glue service
---
## Step 5: Glue service for cart items (market model access from cart)
New file: `glue/services/cart_items.py`
```python
async def get_cart_items(session, user_id=None, session_id=None, *, page_post_id=None) -> list[CartItem]:
"""Get cart items for identity, optionally scoped to page."""
async def find_or_create_cart_item(session, product_id, user_id, session_id) -> CartItem | None:
"""Find existing or create new cart item. Returns None if product missing."""
async def clear_cart_for_order(session, order, *, page_post_id=None) -> None:
"""Soft-delete cart items for order identity."""
async def get_calendar_cart_entries(session, user_id=None, session_id=None, *, page_post_id=None) -> list[CalendarEntry]:
"""Get pending calendar entries for identity, optionally scoped to page."""
```
Imports `CartItem`, `Product`, `MarketPlace` from market, `CalendarEntry`, `Calendar` from events internally.
**Files changed:**
- `cart/bp/cart/services/get_cart.py` — replace CartItem import with glue call
- `cart/bp/cart/services/calendar_cart.py` — replace CalendarEntry import with glue call
- `cart/bp/cart/services/clear_cart_for_order.py` — replace CartItem/MarketPlace imports with glue call
- `cart/bp/cart/services/checkout.py` — replace CartItem/Product/MarketPlace/CalendarEntry/Calendar imports with glue calls
- `cart/bp/cart/api.py` — replace CartItem/MarketPlace/CalendarEntry/Calendar imports with glue calls
- `cart/bp/cart/services/page_cart.py` — replace CartItem/MarketPlace/CalendarEntry/Calendar imports with glue calls
---
## Step 6: Glue service for products (market access from cart orders)
New file: `glue/services/products.py`
```python
async def get_product(session, product_id) -> Product | None:
"""Get product by ID."""
```
This is minimal — only needed by `cart/bp/order/routes.py` and `cart/bp/orders/routes.py` for search/display. However, `OrderItem.product` relationship already resolves via string forward-ref. We only need Product for the join-based search in orders listing.
**Files changed:**
- `cart/bp/orders/routes.py` — replace `from market.models.market import Product` with glue import or use `OrderItem.product` relationship
- `cart/bp/order/routes.py` — replace `from market.models.market import Product` (already uses OrderItem.product relationship for display)
---
## Step 7: Glue service for post associations (events-side)
Move `events/bp/calendar_entry/services/post_associations.py` into glue:
New additions to `glue/services/pages.py` (or separate file `glue/services/post_associations.py`):
```python
async def add_post_to_entry(session, entry_id, post_id) -> tuple[bool, str | None]:
async def remove_post_from_entry(session, entry_id, post_id) -> tuple[bool, str | None]:
async def get_entry_posts(session, entry_id) -> list[dict]:
async def search_posts_for_entry(session, query, page=1, per_page=10) -> tuple[list[dict], int]:
```
**Files changed:**
- `events/bp/calendar_entry/services/post_associations.py`**delete file**, moved to glue
- Update any routes in events that call this service to use glue instead
---
## Step 8: Update glue model registration
`glue/setup.py` needs to ensure all models from all apps are registered in SQLAlchemy's mapper when starting any app. This is because string-based relationship references (like `OrderItem.product → "Product"`) need the target model class registered.
```python
def register_models():
"""Import all model modules to register them with SQLAlchemy mapper."""
# These are already imported by each app, but ensure completeness:
try:
import blog.models.ghost_content # noqa
except ImportError:
pass
try:
import market.models.market # noqa
import market.models.market_place # noqa
except ImportError:
pass
try:
import cart.models.order # noqa
import cart.models.page_config # noqa
except ImportError:
pass
try:
import events.models.calendars # noqa
except ImportError:
pass
```
Each app's `app.py` calls `register_models()` at startup. The try/except guards handle Docker where only one app's code is present — but since all apps share `glue/` and the DB, all model files need to be importable.
**Note:** In Docker, each container only has its own app + shared + glue. For glue services that import from other apps' models, those models must be available. This means either:
- (a) Include all model files in each container (symlinks or copies), or
- (b) Have glue services that import other apps' models use try/except at import time
Since all apps already share one DB and all model files are available in development, option (a) is cleaner for production. Alternatively, the current Docker setup could be extended to include cross-app model files in each image.
---
## Step 9: Update existing glue services
**`glue/services/cart_adoption.py`** — already imports from market and events (correct — this is glue's job). No change needed.
**`glue/services/order_lifecycle.py`** — already imports from events. No change needed.
---
## Step 10: Clean up dead imports and update app.py files
After all glue services are wired:
- `cart/app.py` — remove `from blog.models.ghost_content import Post`, use `from glue.services.pages import get_page_by_slug`
- `market/app.py` — remove `from blog.models.ghost_content import Post`, use `from glue.services.pages import get_page_by_slug`
- `events/app.py` — remove `from blog.models.ghost_content import Post` and `from market.models.market_place import MarketPlace`
- Remove any now-empty cross-app model directories if they exist
---
## Files Summary
| Repo | File | Change |
|------|------|--------|
| **glue** | `services/pages.py` | **NEW** — Post access (slug, id, exists, search) |
| **glue** | `services/page_config.py` | **NEW** — PageConfig CRUD |
| **glue** | `services/calendars.py` | **NEW** — Calendar queries + entry associations (from blog) |
| **glue** | `services/marketplaces.py` | **NEW** — MarketPlace CRUD (from blog+events) |
| **glue** | `services/cart_items.py` | **NEW** — CartItem/CalendarEntry queries for cart |
| **glue** | `services/products.py` | **NEW** — Product access for cart orders |
| **glue** | `services/post_associations.py` | **NEW** — Post-CalendarEntry associations (from events) |
| **glue** | `setup.py` | Add `register_models()` |
| **cart** | `app.py` | Replace Post import with glue |
| **cart** | `bp/cart/api.py` | Replace all 4 cross-app imports with glue |
| **cart** | `bp/cart/services/checkout.py` | Replace cross-app imports with glue |
| **cart** | `bp/cart/services/page_cart.py` | Replace all cross-app imports with glue |
| **cart** | `bp/cart/services/get_cart.py` | Replace CartItem import with glue |
| **cart** | `bp/cart/services/calendar_cart.py` | Replace CalendarEntry import with glue |
| **cart** | `bp/cart/services/clear_cart_for_order.py` | Replace CartItem/MarketPlace with glue |
| **cart** | `bp/orders/routes.py` | Replace Product import with glue |
| **cart** | `bp/order/routes.py` | Replace Product import with glue |
| **blog** | `bp/post/admin/routes.py` | Replace PageConfig/Calendar/MarketPlace with glue |
| **blog** | `bp/post/routes.py` | Replace Calendar/MarketPlace with glue |
| **blog** | `bp/post/services/entry_associations.py` | **DELETE** — moved to `glue/services/calendars.py` |
| **blog** | `bp/post/services/markets.py` | **DELETE** — moved to `glue/services/marketplaces.py` |
| **blog** | `bp/blog/ghost_db.py` | Replace PageConfig import with glue |
| **blog** | `bp/blog/ghost/ghost_sync.py` | Replace PageConfig import with glue |
| **blog** | `bp/blog/services/posts_data.py` | Replace CalendarEntry/CalendarEntryPost with glue |
| **events** | `app.py` | Replace Post + MarketPlace imports with glue |
| **events** | `bp/markets/services/markets.py` | **DELETE** — moved to `glue/services/marketplaces.py` |
| **events** | `bp/markets/routes.py` | Replace MarketPlace import, use glue |
| **events** | `bp/calendars/services/calendars.py` | Replace Post import with glue |
| **events** | `bp/calendar_entry/services/post_associations.py` | **DELETE** — moved to `glue/services/post_associations.py` |
| **events** | `bp/payments/routes.py` | Replace PageConfig import with glue |
| **market** | `app.py` | Replace Post import with glue |
---
## Implementation Order
1. **Step 1** (pages.py) — unlocks Steps 2-4 which depend on page validation
2. **Step 2** (page_config.py) — independent after Step 1
3. **Steps 3-4** (calendars.py, marketplaces.py) — can be done in parallel, both use pages.py
4. **Step 5** (cart_items.py) — depends on steps 1, 3 for calendar queries
5. **Step 6** (products.py) — independent
6. **Step 7** (post_associations.py) — independent, uses pages.py
7. **Steps 8-10** (registration, cleanup) — after all services exist
---
## What's NOT changing
- **CartItem stays in `market/models/market.py`** — moving it creates equal or worse coupling
- **OrderItem stays in `cart/models/order.py`** with `product_id` FK — pragmatic exception
- **OrderItem.product_id FK** — kept, denormalized `product_title` makes it non-critical
- **CartItem.product_id FK** — kept, same DB
- **CartItem.market_place_id FK** — kept, same DB
- **CartItem.user_id FK** — kept, shared model
- **Internal HTTP APIs** (cart/summary, coop/*, events/*) — not changing
- **`shared/` models** (User, MagicLink, etc.) — shared across all apps by design
---
## Docker Consideration
For glue services to work in Docker (single app per container), model files from other apps must be importable. Options:
1. **Copy model files** into each Docker image during build (just the `models/` dirs)
2. **Use try/except** in glue services at import time (degrade gracefully)
3. **Mount shared volume** with all model files
Recommend option 2 for now — glue services that can't import a model simply raise ImportError at call time, which only happens if the service is called from the wrong app (shouldn't happen in practice).
---
## Verification
1. `grep -r "from blog\.models" cart/ market/ events/ glue/` — should return zero results (only in blog/ itself)
2. `grep -r "from market\.models" blog/ cart/ events/` — should return zero results (only in market/ and glue/)
3. `grep -r "from cart\.models" blog/ market/ events/` — should return zero results (only in cart/ and glue/)
4. `grep -r "from events\.models" blog/ cart/ market/` — should return zero results (only in events/ and glue/)
5. All 4 apps start without import errors
6. Checkout flow works end-to-end
7. Blog admin: can toggle features, create/delete markets, manage calendar entries
8. Events admin: can create calendars, manage markets, configure payments
9. Market app: markets listing page loads correctly