giles
76bc293faa
- CLAUDE.md: add SX rendering pipeline overview, service sx/ vs sxc/ convention, dev container mount convention - docker-compose.dev.yml: add missing ./sx/sx:/app/sx bind mount for sx_docs (root cause of "Unknown component: ~sx-layout-full") - async_eval.py: add evaluation modes table to module docstring; log error when async_eval_slot_to_sx can't find a component instead of silently falling through to client-side serialization - helpers.py: remove debug logging from render_to_sx_with_env Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
169 lines
8.4 KiB
Markdown
169 lines
8.4 KiB
Markdown
# Rose Ash Monorepo
|
|
|
|
Cooperative web platform: federated content, commerce, events, and media processing. Each domain runs as an independent Quart microservice with its own database, communicating via HMAC-signed internal HTTP and ActivityPub events.
|
|
|
|
## Deployment
|
|
|
|
- **Do NOT push** until explicitly told to. Pushes reload code to dev automatically.
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
blog/ # Content management, Ghost CMS sync, navigation, WYSIWYG editor
|
|
market/ # Product catalog, marketplace pages, web scraping
|
|
cart/ # Shopping cart CRUD, checkout (delegates order creation to orders)
|
|
events/ # Calendar & event management, ticketing
|
|
federation/ # ActivityPub social hub, user profiles
|
|
account/ # OAuth2 authorization server, user dashboard, membership
|
|
orders/ # Order history, SumUp payment/webhook handling, reconciliation
|
|
relations/ # (internal) Cross-domain parent/child relationship tracking
|
|
likes/ # (internal) Unified like/favourite tracking across domains
|
|
shared/ # Shared library: models, infrastructure, templates, static assets
|
|
artdag/ # Art DAG — media processing engine (separate codebase, see below)
|
|
```
|
|
|
|
### Shared Library (`shared/`)
|
|
|
|
```
|
|
shared/
|
|
models/ # Canonical SQLAlchemy ORM models for all domains
|
|
db/ # Async session management, per-domain DB support, alembic helpers
|
|
infrastructure/ # App factory, OAuth, ActivityPub, fragments, internal auth, Jinja
|
|
services/ # Domain service implementations + DI registry
|
|
contracts/ # DTOs and service protocols
|
|
browser/ # Middleware, Redis caching, CSRF, error handlers
|
|
events/ # Activity bus + background processor (AP-shaped events)
|
|
config/ # YAML config loading (frozen/readonly)
|
|
static/ # Shared CSS, JS, images
|
|
templates/ # Base HTML layouts, partials (inherited by all apps)
|
|
```
|
|
|
|
### Art DAG (`artdag/`)
|
|
|
|
Federated content-addressed DAG execution engine for distributed media processing.
|
|
|
|
```
|
|
artdag/
|
|
core/ # DAG engine (artdag package) — nodes, effects, analysis, planning
|
|
l1/ # L1 Celery rendering server (FastAPI + Celery + Redis + PostgreSQL)
|
|
l2/ # L2 ActivityPub registry (FastAPI + PostgreSQL)
|
|
common/ # Shared templates, middleware, models (artdag_common package)
|
|
client/ # CLI client
|
|
test/ # Integration & e2e tests
|
|
```
|
|
|
|
## Tech Stack
|
|
|
|
**Web platform:** Python 3.11+, Quart (async Flask), SQLAlchemy (asyncpg), Jinja2, HTMX, PostgreSQL, Redis, Docker Swarm, Hypercorn.
|
|
|
|
**Art DAG:** FastAPI, Celery, JAX (CPU/GPU), IPFS/Kubo, Pydantic.
|
|
|
|
## Key Commands
|
|
|
|
### Development
|
|
```bash
|
|
./dev.sh # Start all services + infra (db, redis, pgbouncer)
|
|
./dev.sh blog market # Start specific services + infra
|
|
./dev.sh --build blog # Rebuild image then start
|
|
./dev.sh down # Stop everything
|
|
./dev.sh logs blog # Tail service logs
|
|
```
|
|
|
|
### Deployment
|
|
```bash
|
|
./deploy.sh # Auto-detect changed apps, build + push + restart
|
|
./deploy.sh blog market # Deploy specific apps
|
|
./deploy.sh --all # Deploy everything
|
|
```
|
|
|
|
### Art DAG
|
|
```bash
|
|
cd artdag/l1 && pytest tests/ # L1 unit tests
|
|
cd artdag/core && pytest tests/ # Core unit tests
|
|
cd artdag/test && python run.py # Full integration pipeline
|
|
cd artdag/l1 && ruff check . # Lint
|
|
cd artdag/l1 && mypy app/types.py app/routers/recipes.py tests/
|
|
```
|
|
|
|
## Architecture Patterns
|
|
|
|
### Web Platform
|
|
|
|
- **App factory:** `create_base_app(name, context_fn, before_request_fns, domain_services_fn)` in `shared/infrastructure/factory.py` — creates Quart app with DB, Redis, CSRF, OAuth, AP, session management
|
|
- **Blueprint pattern:** Each blueprint exposes `register() -> Blueprint`, handlers stored in `_handlers` dict
|
|
- **Per-service database:** Each service has own PostgreSQL DB via PgBouncer; cross-domain data fetched via HTTP
|
|
- **Alembic per-service:** Each service declares `MODELS` and `TABLES` in `alembic/env.py`, delegates to `shared.db.alembic_env.run_alembic()`
|
|
- **Inter-service reads:** `fetch_data(service, query, params)` → GET `/internal/data/{query}` (HMAC-signed, 3s timeout)
|
|
- **Inter-service writes:** `call_action(service, action, payload)` → POST `/internal/actions/{action}` (HMAC-signed, 5s timeout)
|
|
- **Inter-service AP inbox:** `send_internal_activity()` → POST `/internal/inbox` (HMAC-signed, AP-shaped activities for cross-service writes)
|
|
- **Fragments:** HTML fragments fetched cross-service via `fetch_fragments()` for composing shared UI (nav, cart mini, auth menu)
|
|
- **Soft deletes:** Models use `deleted_at` column pattern
|
|
- **Context processors:** Each app provides its own `context_fn` that assembles template context from local DB + cross-service fragments
|
|
|
|
### Auth
|
|
|
|
- **Account** is the OAuth2 authorization server; all other apps are OAuth clients
|
|
- Per-app first-party session cookies (Safari ITP compatible), synchronized via device ID
|
|
- Grant verification: apps check grant validity against account DB (cached in Redis)
|
|
- Silent SSO: `prompt=none` OAuth flow for automatic cross-app login
|
|
- ActivityPub: RSA signatures, per-app virtual actor projections sharing same keypair
|
|
|
|
### SX Rendering Pipeline
|
|
|
|
The SX system renders component trees defined in s-expressions. The same AST can be evaluated in different modes depending on where the server/client rendering boundary is drawn:
|
|
|
|
- `render_to_html(name, **kw)` — server-side, produces HTML. Used by route handlers returning full HTML.
|
|
- `render_to_sx(name, **kw)` — server-side, produces SX wire format. Component calls stay **unexpanded** (serialized for client-side rendering by sx.js).
|
|
- `render_to_sx_with_env(name, env, **kw)` — server-side, **expands the top-level component** then serializes children as SX wire format. Used by layout components that need Python context (auth state, fragments, URLs) resolved server-side.
|
|
- `sx_page(ctx, page_sx)` — produces the full HTML shell (`<!doctype html>...`) with component definitions, CSS, and page SX inlined for client-side boot.
|
|
|
|
See the docstring in `shared/sx/async_eval.py` for the full evaluation modes table.
|
|
|
|
### Service SX Directory Convention
|
|
|
|
Each service has two SX-related directories:
|
|
|
|
- **`{service}/sx/`** — service-specific component definitions (`.sx` files with `defcomp`). Loaded at startup by `load_service_components()`. These define layout components, reusable UI fragments, etc.
|
|
- **`{service}/sxc/`** — page definitions and Python rendering logic. Contains `defpage` definitions (client-routed pages) and the Python functions that compose headers, layouts, and page content.
|
|
|
|
Shared components live in `shared/sx/templates/` and are loaded by `load_shared_components()` in the app factory.
|
|
|
|
### Art DAG
|
|
|
|
- **3-Phase Execution:** Analyze → Plan → Execute (tasks in `artdag/l1/tasks/`)
|
|
- **Content-Addressed:** All data identified by SHA3-256 hashes or IPFS CIDs
|
|
- **S-Expression Effects:** Composable effect language in `artdag/l1/sexp_effects/`
|
|
- **Storage:** Local filesystem, S3, or IPFS backends
|
|
- L1 ↔ L2: scoped JWT tokens; L2: password + OAuth SSO
|
|
|
|
## Domains
|
|
|
|
| Service | Public URL | Dev Port |
|
|
|---------|-----------|----------|
|
|
| blog | blog.rose-ash.com | 8001 |
|
|
| market | market.rose-ash.com | 8002 |
|
|
| cart | cart.rose-ash.com | 8003 |
|
|
| events | events.rose-ash.com | 8004 |
|
|
| federation | federation.rose-ash.com | 8005 |
|
|
| account | account.rose-ash.com | 8006 |
|
|
| relations | (internal only) | 8008 |
|
|
| likes | (internal only) | 8009 |
|
|
| orders | orders.rose-ash.com | 8010 |
|
|
|
|
## Dev Container Mounts
|
|
|
|
Dev bind mounts in `docker-compose.dev.yml` must mirror the Docker image's COPY paths. When adding a new directory to a service (e.g. `{service}/sx/`), add a corresponding volume mount (`./service/sx:/app/sx`) or the directory won't be visible inside the dev container. Hypercorn `--reload` watches for Python file changes; `.sx` file hot-reload is handled by `reload_if_changed()` in `shared/sx/jinja_bridge.py`.
|
|
|
|
## Key Config Files
|
|
|
|
- `docker-compose.yml` / `docker-compose.dev.yml` — service definitions, env vars, volumes
|
|
- `deploy.sh` / `dev.sh` — deployment and development scripts
|
|
- `shared/infrastructure/factory.py` — app factory (all services use this)
|
|
- `{service}/alembic/env.py` — per-service migration config
|
|
- `_config/app-config.yaml` — runtime YAML config (mounted into containers)
|
|
|
|
## Tools
|
|
|
|
- Use Context7 MCP for up-to-date library documentation
|
|
- Playwright MCP is available for browser automation/testing
|