Add fragment composition plan to docs

Includes batch fragments, consumer-side styling, viral video distribution (AP attachments + oEmbed + Open Graph), and link-card fragment design. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 20:57:51 +00:00
parent 567888c9e0
commit 4d7f8cfea2
1 changed files with 502 additions and 0 deletions
--- a/docs/fragment-composition-plan.md
+++ b/docs/fragment-composition-plan.md
@@ -0,0 +1,502 @@
+# Eliminate Shared Templates: Server-Side Micro-Frontend Composition
+
+## Context
+
+All 6 coop apps share ~291 templates via `shared/browser/templates/`. This couples every app's UI to a shared submodule. The widget registry (`NavWidget`, `CardWidget`, `AccountPageWidget`, `AccountNavLink`) adds cross-app UI coupling.
+
+**Goal:** Each app owns ALL its templates. Apps expose HTML **fragments** via internal HTTP endpoints. Consuming apps fetch and compose these fragments. No shared templates, no shared widget system.
+
+**Pattern:** [Server-side template composition](https://martinfowler.com/articles/micro-frontends.html) — each app is both a **container** (assembling fragments from others into its pages) and a **micro-frontend** (providing fragments to others). Authentication centralized in account app, injected via fragment. Fowler's "harvest" approach for shared UI: allow duplication first, extract proven patterns later.
+
+---
+
+## Architecture
+
+### Fragment API Contract
+
+Each app exposes `GET /internal/fragments/{type}` (internal Docker network only).
+
+- **Response**: Raw HTML (`text/html`), ready for `{{ html | safe }}`
+- **Errors**: Return `200` with empty body (graceful degradation)
+- **Header**: `X-Fragment-Request: 1` on requests
+- **Context**: Query params for per-user/per-page data (e.g. `?user_id=5`)
+- **Variant**: Optional `?variant=compact|full|inline|card` query param — provider renders a layout-appropriate version (see Consumer-Side Styling below)
+
+### Fragment Catalog
+
+| Fragment | Provider | Consumers | TTL |
+|----------|----------|-----------|-----|
+| `nav-item` | each app | all apps | 300s |
+| `nav-tree` | blog | all apps | 60s |
+| `auth-menu` | account | all apps | 60s |
+| `cart-mini` | cart | all apps | 10s |
+| `container-nav` | events, market | blog, events | 30s |
+| `container-cards` | events | blog | 30s |
+| `account-nav-item` | events, cart, federation | account | 300s |
+| `account-page/{slug}` | events, cart | account | 30s |
+| `link-card` | blog, market, events, federation | any app | 300s |
+| `/oembed` (public) | blog, market, events | Mastodon, Slack, Discord, etc. | 300s |
+| `/{slug}/embed` (public) | blog, events | Slack, Discord, WordPress (oEmbed iframe) | — |
+| OG `<meta>` tags | all apps | Twitter/X, Facebook, LinkedIn | — |
+| AP `Video` attachment | federation | Mastodon, Pleroma, Misskey | — |
+
+### Fragment Client
+
+New file: `shared/infrastructure/fragments.py`
+
+```python
+async def fetch_fragment(app_name, fragment_type, *, params=None, timeout=2.0) -> str:
+    """Fetch HTML fragment from another app. Returns '' on error."""
+
+async def fetch_fragments(requests: list[tuple[str, str, dict|None]]) -> list[str]:
+    """Fetch multiple fragments concurrently via asyncio.gather."""
+
+async def fetch_fragment_cached(app_name, fragment_type, *, params=None, ttl=30) -> str:
+    """fetch_fragment + Redis cache layer."""
+
+async def fetch_fragment_batch(
+    app_name, fragment_type, *, keys: list[str], params=None, ttl=30,
+) -> dict[str, str]:
+    """Fetch a batch of keyed fragments in a single HTTP call.
+
+    Sends keys as comma-separated query param. Provider returns HTML with
+    <!-- fragment:{key} --> comment markers. Client splits and returns
+    a dict mapping each key to its HTML (missing keys → '').
+
+    Example: fetch_fragment_batch("cart", "cart-mini", keys=["flour", "bread"],
+             params={"user_id": 5})
+    → GET /internal/fragments/cart-mini?keys=flour,bread&user_id=5
+    → {"flour": "<div ...>...</div>", "bread": "<div ...>...</div>"}
+    """
+```
+
+Uses `INTERNAL_URL_{APP}` env vars (e.g. `http://blog:8000`). Falls back to `http://{app}:8000`.
+
+### Batch Fragments (list pages)
+
+Fragments that appear per-item on list pages (product cards, timeline posts, etc.)
+support batch fetching to avoid N+1 HTTP calls. One request returns all items,
+delimited by `<!-- fragment:{key} -->` comment markers.
+
+**Pattern (already proven by `container-cards`):**
+
+```
+GET /internal/fragments/cart-mini?keys=flour,bread,milk&user_id=5
+```
+
+```html
+<!-- fragment:flour -->
+<div class="cart-mini" data-product="flour">...</div>
+<!-- fragment:bread -->
+<div class="cart-mini" data-product="bread">...</div>
+<!-- fragment:milk -->
+<div class="cart-mini" data-product="milk">...</div>
+```
+
+The client helper `fetch_fragment_batch()` splits by markers and returns `dict[key, html]`.
+Templates access per-item HTML via the dict:
+
+```jinja2
+{% for product in products %}
+  {{ batch_cart_minis[product.slug] | safe }}
+{% endfor %}
+```
+
+**Batch-eligible fragments:**
+
+| Fragment | Key field | Use case |
+|----------|-----------|----------|
+| `cart-mini` | `product_slug` | Add-to-cart buttons on browse/listing pages |
+| `like-button` | `slug` | Like buttons on product cards, blog listings |
+| `link-card` | `slug` | Multiple rich link previews in a blog post |
+| `container-cards` | `post_id` | Calendar entry cards on blog listing (already implemented) |
+
+**Provider-side:** batch handlers receive `keys` query param, query all items in one
+DB call (e.g. `WHERE slug IN (...)`), render each to HTML, join with comment markers.
+This is a single SQL query + template render loop, not N separate handler calls.
+
+### Jinja Integration
+
+Register `fragment` as Jinja async global in `jinja_setup.py`:
+```jinja2
+{{ fragment("cart", "cart-mini", user_id=uid) | safe }}
+```
+
+**Preferred pattern**: Pre-fetch all fragments in context processor with `asyncio.gather`, pass as template vars.
+
+### Consumer-Side Styling
+
+Fragments need to look different depending on where they appear. A cart-mini button
+on a product card looks different from one in a sidebar or a mobile nav. The consumer
+controls the presentation context; the provider supplies the content.
+
+**Three complementary mechanisms:**
+
+**1. Provider variants (`?variant=`)** — the provider renders a structurally different
+version. Use when layout differs significantly (not just CSS tweaks):
+
+```python
+# Context processor
+cart_mini_html = await fetch_fragment("cart", "cart-mini",
+    params={"product_slug": "flour", "variant": "compact"})
+```
+
+Standard variant names:
+- `full` — default, complete rendering (product pages, dedicated sections)
+- `compact` — minimal version (listing cards, sidebars, tight spaces)
+- `inline` — single-line/inline-flow version (within text, table cells)
+- `icon` — icon-only, no text (toolbars, mobile nav)
+
+Provider templates: `fragments/cart_mini.html` checks `variant` param and renders
+accordingly. One template with conditionals, or separate includes per variant.
+
+**2. CSS wrapper classes** — the consumer wraps the fragment in a styled container.
+Use for spacing, sizing, color scheme adaptation:
+
+```jinja2
+{# Same fragment, different contexts #}
+<div class="product-card__actions">
+  {{ batch_cart_minis[product.slug] | safe }}
+</div>
+
+<aside class="sidebar-cart compact text-sm">
+  {{ cart_mini_html | safe }}
+</aside>
+```
+
+Fragments use BEM-style or data-attribute selectors (`[data-fragment="cart-mini"]`)
+so consumers can target them with CSS without relying on internal class names.
+
+**3. Data attributes for JS hooks** — fragments include `data-fragment="type"`
+and `data-app="provider"` attributes on their root element. Consumers can attach
+behavior without coupling to internal markup:
+
+```html
+<!-- Provider renders: -->
+<div data-fragment="cart-mini" data-app="cart" data-product="flour">
+  ...
+</div>
+
+<!-- Consumer CSS can target: -->
+<style>
+  .sidebar [data-fragment="cart-mini"] { font-size: 0.875rem; }
+  .product-card [data-fragment="cart-mini"] button { width: 100%; }
+</style>
+```
+
+**Convention:** Every fragment's root element MUST include:
+- `data-fragment="{type}"` — the fragment type name
+- `data-app="{provider}"` — the app that rendered it
+- A single root element (no bare text nodes) so wrapping/targeting works reliably
+
+---
+
+## Implementation Phases
+
+### Phase 1: Fragment Infrastructure (foundation, no breaking changes)
+
+**New files:**
+- `shared/infrastructure/fragments.py` — HTTP client + Redis cache
+- `{blog,market,cart,events,federation,account}/bp/fragments/__init__.py` + `routes.py` — fragment blueprints
+- `art-dag/celery/app/routers/fragments.py` — artdag fragment endpoints
+
+**Modified files:**
+- `shared/infrastructure/jinja_setup.py` — add `fragment` Jinja global
+- Each app's `app.py` — register fragments blueprint
+
+### Phase 2: Cart + Account Fragments (highest cross-app value)
+
+Cart implements: `cart-mini`, `nav-item`, `account-nav-item`
+Account implements: `auth-menu`, `sign-in`
+Blog updates context processor to fetch these fragments.
+
+### Phase 3: Navigation Fragments
+
+Blog implements: `nav-tree` (renders `menu_items` to HTML)
+All apps update context processors to fetch `nav-tree` from blog.
+Copy `_types/root/` templates to each app, replace nav includes with fragment slots.
+
+### Phase 4: Container Widgets → Fragments
+
+Events implements: `container-nav`, `container-cards`
+Market implements: `container-nav`
+Blog/events replace `widgets.container_nav`/`widgets.container_cards` with fragment fetches.
+
+### Phase 5: Link Card Fragments (rich internal links)
+
+Any app can request a rich link preview from any other app:
+
+```python
+html = await fetch_fragment("blog", "link-card", params={"slug": "about-us"})
+html = await fetch_fragment("market", "link-card", params={"slug": "organic-flour"})
+html = await fetch_fragment("events", "link-card", params={"slug": "summer-fair", "calendar_id": "42"})
+```
+
+Each app's `link-card` handler returns a self-contained HTML card with domain-appropriate data:
+
+| Provider | Card contents |
+|----------|--------------|
+| **blog** | title, feature image, excerpt |
+| **market** | product name, image, price, availability |
+| **events** | event name, date/time, image, ticket status |
+| **federation** | actor display name, avatar, bio |
+
+**Key design principles:**
+
+1. **Provider owns the base markup** — the card HTML includes sensible default styling (image, title, description), but is wrapped in a simple container (`<div class="link-card" data-app="blog" ...>`) that consumers can restyle via CSS or wrapper markup.
+
+2. **Consumer controls context** — the calling app can wrap the card in its own styling, layout, or container. A link card in a blog sidebar looks different from one in a social timeline or an admin panel. The consumer adds the contextual presentation; the provider supplies the content.
+
+3. **Media-aware** — cards can include video embeds, audio players, or image galleries depending on the content type. The provider knows what media the content has and renders appropriately.
+
+4. **Cacheable** — Redis-cached with TTL like other fragments. Content changes propagate on next cache miss.
+
+5. **Graceful degradation** — if the fragment fetch fails, the consumer falls back to a plain `<a>` tag.
+
+**Implementation per app:**
+- New `link-card` handler in each app's `bp/fragments/routes.py`
+- Template: `{app}/templates/fragments/link_card.html`
+- Accepts `slug` (and optionally `id`, `type`) as query params
+- Returns HTML card or empty string if not found
+
+**Use cases:**
+- Rich links inside Ghost blog post HTML (post-process `<a>` tags pointing to internal domains)
+- Cross-app "related content" sections
+- Social timeline embedded previews
+- Admin UIs referencing cross-app content
+- Email templates with content previews
+
+**oEmbed endpoint (external):**
+
+The same data that powers internal link-cards also powers a public oEmbed endpoint for
+external consumers (Mastodon, Slack, Discord, WordPress, etc.).
+
+Each app exposes `GET /oembed?url=...&maxwidth=...&maxheight=...&format=json`:
+
+```json
+{
+  "version": "1.0",
+  "type": "photo",
+  "title": "Summer Fair 2026",
+  "author_name": "Rose Ash Cooperative",
+  "author_url": "https://blog.rose-ash.com",
+  "provider_name": "Rose Ash",
+  "provider_url": "https://blog.rose-ash.com",
+  "thumbnail_url": "https://blog.rose-ash.com/static/uploads/summer-fair.jpg",
+  "thumbnail_width": 1200,
+  "thumbnail_height": 630,
+  "url": "https://blog.rose-ash.com/static/uploads/summer-fair.jpg",
+  "width": 1200,
+  "height": 630
+}
+```
+
+Discovery via `<link>` tag in page `<head>` (added to shared base template):
+```html
+<link rel="alternate" type="application/json+oembed"
+      href="https://blog.rose-ash.com/oembed?url=https://blog.rose-ash.com/summer-fair&format=json"
+      title="Summer Fair 2026">
+```
+
+Supported oEmbed types (Mastodon rejects `rich` — no iframes):
+- `link` — title + thumbnail (default for posts/pages)
+- `photo` — direct image embed (for image-heavy posts)
+- `video` — video embed with `html` field (for posts with video, events with recordings)
+
+Implementation:
+- `shared/infrastructure/oembed.py` — shared oEmbed response builder + `<link>` tag helper
+- Each app's `app.py` registers `/oembed` route using shared builder
+- Base template includes oEmbed discovery `<link>` tag via context var
+- Redis-cached (same data as link-card, different serialization)
+- Handles the Mastodon "thundering herd" gracefully — every instance that receives
+  a post with your URL will independently call your oEmbed endpoint; Redis cache
+  absorbs the burst
+
+This means:
+- When your AP activities include a link, every receiving Mastodon instance gets a rich preview card
+- Slack/Discord unfurl your links with images, titles, descriptions
+- WordPress embeds your content with proper attribution
+- All powered by the same slug lookup that serves internal link-cards
+
+**Viral video distribution (three channels, one content lookup):**
+
+The same content that powers link-cards and oEmbed also drives video distribution
+across three channels, each optimized for its platform:
+
+| Channel | Platform | Mechanism | Video playback |
+|---------|----------|-----------|----------------|
+| **AP media attachments** | Mastodon, Pleroma, Misskey | `Video` attachment on `Create`/`Note` activity | Native player in Mastodon UI |
+| **oEmbed `type: "video"`** | Slack, Discord, WordPress | `html` field with `<iframe>` pointing to embed player | Inline iframe player |
+| **Open Graph `og:video`** | Twitter/X, Facebook, LinkedIn | `<meta>` tags in page `<head>` | Platform-native player |
+
+All three link back to the content page on Rose Ash.
+
+**The viral loop:**
+
+1. User creates video content on Rose Ash (Art-DAG processes/transcodes)
+2. Content is published as AP activity with `Video` attachment → plays natively in Mastodon
+3. Link shared on Slack/Discord → oEmbed `type: "video"` renders inline player
+4. Link shared on Twitter/LinkedIn → Open Graph `og:video` renders inline player
+5. Every embed links back to Rose Ash landing page
+6. Landing page serves dual purpose: **content playback + creation CTA** ("make your own")
+7. New user creates content → shares → repeat
+
+**Key: the landing page does double duty.** It's not just a viewer — it's the entry point
+for content creation. The video plays, and below/beside it is the tool to create your own.
+Every view is a potential conversion.
+
+**AP video attachments (Mastodon path):**
+
+Mastodon rejects oEmbed `rich`/`video` iframes but plays AP media natively. When
+federation publishes a `Create` activity for video content:
+
+```json
+{
+  "type": "Create",
+  "object": {
+    "type": "Note",
+    "content": "<p>Check out our summer fair highlights!</p>",
+    "url": "https://events.rose-ash.com/summer-fair",
+    "attachment": [
+      {
+        "type": "Document",
+        "mediaType": "video/mp4",
+        "url": "https://cdn.rose-ash.com/videos/summer-fair-highlights.mp4",
+        "name": "Summer Fair 2026 Highlights",
+        "width": 1920,
+        "height": 1080,
+        "blurhash": "LEHV6nWB2yk8pyoJadR*.7kCMdnj"
+      }
+    ]
+  }
+}
+```
+
+- Mastodon renders native video player with blurhash placeholder
+- `url` field links to the Rose Ash page (the viral landing page)
+- Art-DAG provides transcoded video + blurhash + dimensions
+- `federation_publish.py` attaches media metadata from Art-DAG when content has video
+
+**Open Graph video tags (Twitter/Facebook/LinkedIn path):**
+
+Added to page `<head>` alongside the oEmbed discovery `<link>`:
+
+```html
+<meta property="og:type" content="video.other">
+<meta property="og:video" content="https://cdn.rose-ash.com/videos/summer-fair.mp4">
+<meta property="og:video:type" content="video/mp4">
+<meta property="og:video:width" content="1920">
+<meta property="og:video:height" content="1080">
+<meta property="og:image" content="https://events.rose-ash.com/static/uploads/summer-fair.jpg">
+<meta property="og:title" content="Summer Fair 2026 Highlights">
+<meta property="og:description" content="Watch and create your own event videos">
+```
+
+Implementation:
+- `shared/infrastructure/oembed.py` also generates OG meta tags (same data source)
+- Base template renders OG tags from context var `og_meta`
+- Art-DAG webhook notifies coop apps when video processing completes (transcode ready)
+
+**Implementation files (video-specific additions):**
+- `shared/infrastructure/oembed.py` — add `build_og_meta()` helper + video oEmbed type
+- `shared/infrastructure/activitypub.py` — add media attachment builder for `Create` activities
+- Base template `<head>` — render `og_meta` context var + oEmbed `<link>` tag
+- Each app's embed player route: `GET /{slug}/embed` — lightweight player page for oEmbed iframes
+- `federation_publish.py` — attach Art-DAG media metadata to outgoing activities
+
+### Phase 6: Account Widgets → Fragments
+
+Events implements: `account-page/tickets`, `account-page/bookings`
+Cart implements: `account-page/orders`
+Account replaces `widgets.account_nav` + `widget_page()` with fragment fetches.
+
+### Phase 7: Template Migration
+
+Move all domain templates from `shared/browser/templates/` to owning apps:
+- `_types/blog/`, `_types/post/`, `_types/home/` → blog
+- `_types/browse/`, `_types/market/`, `_types/product/` → market
+- `_types/cart/`, `_types/order/` → cart
+- `_types/calendar/`, `_types/day/`, `_types/entry/`, `_types/slot/`, `_types/ticket*/` → events
+- `_types/auth/` → account
+- `_types/root/` → copied to ALL apps (each owns its base layout, ~84 lines)
+- `macros/` → copied to all apps that use them
+
+### Phase 8: Cleanup
+
+Delete from shared/:
+- `browser/templates/` (all 291+ files)
+- `contracts/widgets.py`
+- `services/widget_registry.py`
+- `services/widgets/` (entire directory)
+- `services/navigation.py`
+
+Remove from `factory.py`: `register_all_widgets()` call
+Remove from `jinja_setup.py`: widget registry Jinja global
+
+---
+
+## What Stays in shared/
+
+After migration, `shared/` contains only infrastructure (no templates, no UI):
+- `models/`, `db/` — ORM + session (still shared DB)
+- `config.py`, `log_config.py`
+- `infrastructure/` — factory, jinja_setup, context, fragments (new), oembed (new), oauth, activitypub, urls, user_loader, cart_identity
+- `contracts/protocols.py`, `contracts/dtos.py` — service protocols + DTOs
+- `services/registry.py`, `services/*_impl.py` — SQL implementations
+- `events/` — event processor + handlers
+- `browser/app/` — middleware, redis, csrf, errors
+- `static/` — shared CSS/JS (CDN migration is separate)
+
+---
+
+## Key Decisions
+
+1. **HTML fragments, not JSON** — each app owns its markup, consumers just `{{ html | safe }}`. This is the "server-side template composition" pattern from Fowler's micro-frontends taxonomy.
+2. **Context processor pre-fetch** — `asyncio.gather` all fragments before rendering (faster than Jinja async)
+3. **Each app owns its base layout** — 6 copies of ~84-line shell, fragment slots for cross-app content. Each app is a "container" that assembles fragments from other "micro-frontends."
+4. **Macros copied per-app** — pure Jinja, no domain logic, can diverge independently. Per Fowler: shared component libraries must "contain only UI logic, no business or domain logic."
+5. **Graceful degradation** — failed fragment = empty string = section absent from page
+6. **Harvest approach** — allow duplication initially across apps, extract proven patterns into shared UI library only after they stabilize. Don't premature-abstract.
+7. **Consumer-driven contracts** — fragment providers define a contract (endpoint + response format). Consumers test against the contract, not the provider's internals. URL/routing is the primary cross-app communication channel.
+8. **Provider supplies content, consumer controls context** — three styling mechanisms: provider variants (`?variant=compact`) for structural differences, CSS wrapper classes for layout adaptation, `data-fragment`/`data-app` attributes for targeted styling. Fragments always have a single root element with data attributes.
+
+---
+
+## Artdag Participation
+
+Artdag already has its own templates. It participates as:
+- **Provider**: `/internal/fragments/nav-item` (Art-DAG link)
+- **Consumer**: Can fetch `nav-tree`, `auth-menu` from coop apps over Docker network
+
+New file: `art-dag/celery/app/routers/fragments.py`
+
+---
+
+## Files Summary
+
+| Action | Files |
+|--------|-------|
+| **New** | `shared/infrastructure/fragments.py` |
+| **New** | `{6 apps}/bp/fragments/__init__.py` + `routes.py` |
+| **New** | `art-dag/celery/app/routers/fragments.py` |
+| **New** | `shared/infrastructure/oembed.py` — oEmbed response builder + OG meta + `<link>` tag helper |
+| **New** | `{blog,market,events}/bp/embed/routes.py` — lightweight embed player pages for oEmbed iframes |
+| **Copy** | 291+ templates from `shared/browser/templates/` → owning apps |
+| **Modify** | `shared/infrastructure/jinja_setup.py` — add fragment global |
+| **Modify** | `shared/infrastructure/factory.py` — remove widget registration |
+| **Modify** | `shared/infrastructure/activitypub.py` — media attachment builder for AP activities |
+| **Modify** | `shared/services/federation_publish.py` — attach Art-DAG media to outgoing activities |
+| **Modify** | Base template `<head>` — OG meta tags + oEmbed discovery `<link>` |
+| **Modify** | Each app's context processor — fetch fragments |
+| **Modify** | Each app's `app.py` — register fragments + `/oembed` route |
+| **Delete** | `shared/browser/templates/` (all), widget system files, `navigation.py` |
+
+---
+
+## Verification
+
+1. Start with cart-mini fragment: cart app serves it, blog fetches it → cart icon renders on blog
+2. Auth menu fragment: account serves it → all apps show login/user info
+3. Nav tree fragment: blog serves it → all apps show navigation
+4. Deploy one app at a time — existing shared templates remain as fallback until removed
+5. Final: remove shared templates, verify all 6 apps + artdag render correctly