rose-ash/docs/relation-system-plan.md

Flexible Entity Relationship System

Context

The rose-ash platform has 9 microservices with decoupled entities (posts, pages, markets, calendars, entries, products, etc.) connected via a generic ContainerRelation table. The current system is limited:

• Parent-child only — no cardinality, no relation types, no many-to-many
• Feature flags gate everything — PageConfig.features["market"]=true must be set before creating a market on a page
• Navigation is disconnected — rebuild_navigation() is a no-op; MenuNode is manually managed
• Per-domain boilerplate — each service has its own container-nav fragment handler, its own create-and-attach flow
• No relation semantics — you can't distinguish how two entities are connected, only that they are

The s-expression engine (phases 1-7 complete) gives us a declarative language to define relations and auto-generate UI. Building this now means the remaining sexp work gets designed around the relation model.

Goal: A registry-driven system where relation types are declared in s-expressions, cardinality is enforced, navigation is auto-generated, and attach/detach is generic.

---

Key Decisions

1. Relations is a graph store, not an orchestrator. Domain services create their own entities, then call relate on the relations service. Relations validates cardinality and stores the link. It never calls out to domain services to create entities.

2. All entity connections live in ContainerRelation. Including many-to-many (e.g. post↔calendar_entry). Single source of truth. Domain-specific junction tables (like calendar_entry_posts) migrate here.

Architecture: Three Layers

┌─────────────────────────────────────────────────────────┐
│  RELATION REGISTRY  (shared/sexp/relations.py)          │
│  defrelation declarations — loaded at startup by all    │
│  services from shared code. Pure data, no DB.           │
├─────────────────────────────────────────────────────────┤
│  RELATION STORAGE  (relations service, db_relations)    │
│  container_relations table + relation_type column.      │
│  Graph store. Validates against registry.               │
│  Domain services create entities, then call relate.     │
├─────────────────────────────────────────────────────────┤
│  RELATION CONSUMERS  (all services)                     │
│  Nav auto-generation, UI components, fragment rendering │
│  Read registry to know how to display/manage relations  │
└─────────────────────────────────────────────────────────┘

---

Phase A: Relation Registry

A.1 — defrelation s-expression form

New special form for the evaluator, alongside defcomp:

(defrelation :page->market
  :from "page"
  :to "market"
  :cardinality :one-to-many
  :inverse :market->page
  :nav :submenu
  :nav-icon "fa fa-shopping-bag"
  :nav-label "markets")

(defrelation :page->calendar
  :from "page"
  :to "calendar"
  :cardinality :one-to-many
  :inverse :calendar->page
  :nav :submenu
  :nav-icon "fa fa-calendar"
  :nav-label "calendars")

(defrelation :post->calendar_entry
  :from "post"
  :to "calendar_entry"
  :cardinality :many-to-many
  :inverse :calendar_entry->post
  :nav :inline
  :nav-icon "fa fa-file-alt"
  :nav-label "events")

(defrelation :page->menu_node
  :from "page"
  :to "menu_node"
  :cardinality :one-to-one
  :nav :hidden)

A.2 — RelationDef type

File: shared/sexp/types.py — add alongside Component, Lambda:

@dataclass(frozen=True)
class RelationDef:
    name: str               # "page->market"
    from_type: str          # "page"
    to_type: str            # "market"
    cardinality: str        # "one-to-one" | "one-to-many" | "many-to-many"
    inverse: str | None     # "market->page"
    nav: str                # "submenu" | "tab" | "badge" | "inline" | "hidden"
    nav_icon: str | None    # "fa fa-shopping-bag"
    nav_label: str | None   # "markets" — display label for nav sections

A.3 — Registry module

New file: shared/sexp/relations.py

_RELATION_REGISTRY: dict[str, RelationDef] = {}

def load_relation_registry() -> None:
    """Parse defrelation s-expressions, populate registry."""
    for source in [_PAGE_MARKET, _PAGE_CALENDAR, _POST_ENTRY, _PAGE_MENU_NODE]:
        register_relations(source)

def get_relation(name: str) -> RelationDef | None: ...
def relations_from(entity_type: str) -> list[RelationDef]: ...
def relations_to(entity_type: str) -> list[RelationDef]: ...

Called from create_base_app() at startup alongside load_shared_components().

A.4 — defrelation in the evaluator

File: shared/sexp/evaluator.py — add to _SPECIAL_FORMS:

Parses keyword args from the s-expression, creates a RelationDef, stores it in _RELATION_REGISTRY and the current env.

A.5 — Replaces PageConfig feature flags

• The existence of defrelation :page->market means pages can have markets. No feature flag needed.
• Admin UI queries relations_from("page") to show create buttons.
• PageConfig survives only for SumUp payment credentials. The features JSON column is deprecated.

A.6 — Tests

New file: shared/sexp/tests/test_relations.py

• Parse defrelation, verify RelationDef fields
• Registry lookup: get_relation(), relations_from(), relations_to()
• Cardinality values validated
• Inverse relation lookup

Files touched

• shared/sexp/types.py — add RelationDef
• shared/sexp/evaluator.py — add defrelation special form
• shared/sexp/relations.py — NEW: registry + definitions
• shared/sexp/tests/test_relations.py — NEW: registry tests
• shared/infrastructure/factory.py — call load_relation_registry() at startup

---

Phase B: Schema Evolution

B.1 — Add relation_type column

File: shared/models/container_relation.py

relation_type: Mapped[Optional[str]] = mapped_column(String(64), nullable=True, index=True)

New composite index: (relation_type, parent_type, parent_id) for filtered child queries.

B.2 — Alembic migration

File: relations/alembic/versions/xxx_add_relation_type.py

1. Add nullable relation_type column
2. Add indexes
3. Backfill existing rows:
   • (page, *, market, *) → relation_type = "page->market"
   • (page, *, calendar, *) → relation_type = "page->calendar"
   • (page, *, menu_node, *) → relation_type = "page->menu_node"

B.3 — Add get_parents() query

File: shared/services/relationships.py

Inverse of get_children() — query by (child_type, child_id) with optional relation_type filter. Uses existing ix_container_relations_child index.

B.4 — Update attach_child() / detach_child()

Add optional relation_type parameter. When provided:

• Stored in the relation row
• Included in emitted activity object_data
• Used for cardinality enforcement (Phase C)

Existing callers without relation_type continue to work (backward compatible).

Files touched

• shared/models/container_relation.py — add column + index
• shared/services/relationships.py — add relation_type param, add get_parents()
• relations/alembic/versions/ — NEW migration

---

Phase C: Generic Relate/Unrelate API

C.1 — New action endpoints on relations service

File: relations/bp/actions/routes.py

relate — validates against registry, enforces cardinality, delegates to attach_child():

POST /internal/actions/relate
{
  "relation_type": "page->market",
  "from_id": 42,
  "to_id": 7,
  "label": "Farm Shop",
  "metadata": {"slug": "farm-shop"}
}

unrelate — validates, delegates to detach_child():

POST /internal/actions/unrelate
{
  "relation_type": "page->market",
  "from_id": 42,
  "to_id": 7
}

can-relate — pre-flight check (cardinality, registry validation) without creating anything:

POST /internal/actions/can-relate
{
  "relation_type": "page->market",
  "from_id": 42
}
→ {"allowed": true} or {"allowed": false, "reason": "one-to-one already exists"}

Domain services call can-relate before creating the entity to avoid orphans on cardinality failure.

C.2 — Typical domain flow

1. Domain service receives "create market on page 42" request
2. call_action("relations", "can-relate", {relation_type: "page->market", from_id: 42})
3. If not allowed → return error to user (no entity created)
4. Create the entity locally (market service creates MarketPlace row)
5. call_action("relations", "relate", {relation_type: "page->market", from_id: 42, to_id: 7, ...})

C.3 — Cardinality enforcement

In the relate and can-relate handlers:

• one-to-one: Check no active relation of this type exists for from_id. Reject if found.
• one-to-many: No limit (current behavior).
• many-to-many: No limit in either direction.

C.4 — Enhanced activity emission

Activities now include relation_type in object_data:

object_data={
    "relation_type": "page->market",
    "parent_type": "page",
    "parent_id": 42,
    "child_type": "market",
    "child_id": 7,
}

C.5 — Existing endpoints stay as aliases

attach-child and detach-child remain during transition. They infer relation_type from (parent_type, child_type) when not provided.

Files touched

• relations/bp/actions/routes.py — add relate, unrelate, can-relate
• shared/services/relationships.py — cardinality check in attach_child()

---

Phase D: Navigation Auto-Generation

D.1 — Generic container-nav fragment

New handler: relations/bp/fragments/routes.py

The relations service becomes a fragment provider. A single generic handler replaces per-service container-nav handlers in market and events:

async def _container_nav_handler():
    container_type = request.args["container_type"]
    container_id = int(request.args["container_id"])
    post_slug = request.args.get("post_slug", "")

    nav_defs = [d for d in relations_from(container_type) if d.nav != "hidden"]
    parts = []

    for defn in nav_defs:
        children = await get_children(
            g.s, container_type, container_id,
            child_type=defn.to_type,
            relation_type=defn.name,
        )
        for child in children:
            parts.append(render_sexp(
                '(~relation-nav :href href :name name :icon icon :nav-class nc)',
                href=_build_href(defn, child, post_slug, ctx),
                name=child.label or "",
                icon=defn.nav_icon or "",
                nc=nav_class,
            ))

    return "\n".join(parts)

D.2 — Nav hints

nav value	Behavior
submenu	Clickable item in container-nav header/sidebar
tab	Tab in a tabbed interface
badge	Count badge on parent
inline	Inline within parent content
hidden	Not shown in navigation

D.3 — rebuild_navigation() becomes real

File: shared/services/navigation.py

For now: invalidate nav caches when relations change. The top-level MenuNode entries (depth=0, main nav bar) remain manually managed. Child navigation (markets, calendars under a page) is generated dynamically from the relation registry via the generic container-nav fragment.

Future: rebuild_navigation() could materialize the full nav tree from the relation graph for performance.

D.4 — Href computation

The ContainerRelation.label column stores the display name. Add a metadata JSON column for additional denormalized data (slug, icon) needed to compute hrefs without fetching from the owner service:

metadata: Mapped[Optional[dict]] = mapped_column(JSON, nullable=True)
# e.g., {"slug": "farm-shop", "icon": "..."}

The relate action populates this from the caller's payload.

Files touched

• relations/bp/fragments/routes.py — NEW: generic container-nav
• shared/services/navigation.py — cache invalidation
• shared/models/container_relation.py — add metadata JSON column
• relations/alembic/versions/ — migration for metadata column
• Blog, market, events fragment routes — deprecate per-service container-nav

---

Phase E: S-Expression UI Components

E.1 — ~relation-nav component

File: shared/sexp/components.py — replaces ~market-link-nav, ~calendar-link-nav:

(defcomp ~relation-nav (&key href name icon nav-class relation-type)
  (a :href href :class (or nav-class "...")
    (when icon (i :class icon :aria-hidden "true"))
    (div name)))

E.2 — ~relation-attach component

Generic "add child" button driven by registry:

(defcomp ~relation-attach (&key create-url label icon)
  (a :href create-url
     :hx-get create-url
     :hx-target "#main-panel"
     :hx-swap "outerHTML"
     :hx-push-url "true"
     :class "flex items-center gap-2 text-sm p-2 rounded hover:bg-stone-100"
    (when icon (i :class icon))
    (span (or label "Add"))))

E.3 — ~relation-detach component

Confirm-and-remove button:

(defcomp ~relation-detach (&key detach-url name)
  (button :hx-delete detach-url
          :hx-confirm (str "Remove " name "?")
          :class "text-red-500 hover:text-red-700 text-sm"
    (i :class "fa fa-times")))

E.4 — Keep specializations where needed

~calendar-entry-nav has date display logic — keep it as a specialization. The generic handler can delegate to specific components based on to_type when richer rendering is needed.

Files touched

• shared/sexp/components.py — add ~relation-nav, ~relation-attach, ~relation-detach

---

Phase F: Migration of Existing Relations

F.1 — Migration order

1. Add relation_type column + backfill (Phase B)
2. Deploy registry + new endpoints (Phases A, C)
3. Switch callers one-by-one to use can-relate + relate:
   • shared/services/market_impl.py — market creation: check can-relate, create, relate
   • events/bp/calendars/services/calendars.py — calendar creation
   • blog/bp/menu_items/services/menu_items.py — menu node creation
4. Deploy generic container-nav fragment (Phase D)
5. Switch fragment consumers from per-service to generic
6. Deprecate per-service container-nav handlers
7. Deprecate PageConfig.features column

F.2 — CalendarEntryPost migration (many-to-many)

The calendar_entry_posts junction table in db_events is a domain-specific many-to-many. Strategy:

• Add defrelation :post->calendar_entry with :cardinality :many-to-many
• Migrate existing rows to ContainerRelation with relation_type="post->calendar_entry"
• Update shared/services/entry_associations.py to use relate/unrelate
• Eventually drop calendar_entry_posts table

F.3 — PageConfig simplification

• Remove features JSON column (after all feature-flag checks are replaced by registry lookups)
• Keep PageConfig for SumUp credentials only
• Consider renaming to PaymentConfig

Files touched (incremental)

• shared/services/market_impl.py — use relate
• events/bp/calendars/services/calendars.py — use relate
• blog/bp/menu_items/services/menu_items.py — use relate
• shared/services/entry_associations.py — use relate/unrelate
• blog/bp/post/services/markets.py — simplify (no feature flag check)
• market/bp/fragments/routes.py — deprecate container-nav
• events/bp/fragments/routes.py — deprecate container-nav

---

Verification

Unit tests

• shared/sexp/tests/test_relations.py — registry parsing, lookup, cardinality rules
• shared/sexp/tests/test_evaluator.py — defrelation form evaluation
• shared/sexp/tests/test_components.py — ~relation-nav, ~relation-attach rendering

Integration tests

• Relations service: relate with valid/invalid types, cardinality enforcement
• can-relate pre-flight: returns allowed/denied correctly
• unrelate: soft-deletes, emits Remove activity
• Generic container-nav fragment: returns correct HTML for various relation types

Manual testing

• Browse site with Playwright after each phase
• Verify nav renders correctly
• Test create/delete flows for markets and calendars
• Check activity bus still fires on relation changes

---

Phase Order & Dependencies

Phase A (Registry)  ──────┐
                          ├──▶ Phase C (API)  ──▶ Phase D (Nav)  ──▶ Phase F (Migration)
Phase B (Schema)   ───────┘                        ▲
                                                   │
                                    Phase E (UI Components)

A and B can run in parallel. C needs both. D and E need C. F is incremental, runs alongside D and E.