New next/kernel/delivery_state.erl folds delivery events into a
per-peer worker-shaped snapshot so the outbound queue survives
kernel restart.
Event proplist shapes:
[{type, enqueued}, {peer, _}, {activity, _}]
[{type, delivered}, {peer, _}, {cid, _}]
[{type, failed}, {peer, _}, {cid, _}, {now, _}]
[{type, dead_lettered}, {peer, _}, {cid, _}]
Projection state shape:
[{PeerId, [{peer, _}, {pending, _}, {attempts, _},
{next_retry, _}, {dead_letter, _}]}, ...]
Mirrors delivery_worker:new/1 (minus :dispatch_fn — that's the
live worker's concern) so a fresh gen_server can be hydrated
from the projection on restart.
Public API:
new/0
fold/2, fold_fn/0
peer_state/2, peers/1
pending/2, attempts/2, next_retry/2, dead_letter/2
The failed branch calls delivery_worker:backoff_for/1 directly,
so the projection and the live worker compute identical retry
slots and dead-letter thresholds. 6th failure -> dead-letter,
matching the worker.
14/14 in next/tests/delivery_state.sh covering:
- new/0 -> []
- enqueued appends to pending (FIFO)
- two peers maintain independent queues
- delivered clears matching pending entry
- failed bumps :attempts and sets :next_retry
- 6th failed -> dead-lettered (activity out of pending)
- explicit dead_lettered event moves activity to dead_letter
- peers/1 lists touched peers
- peer_state {ok, _} | not_found
- fold_fn/0 is fun/2 for projection:start_link
- unknown event type passes through
- delivered after failed clears retry state
delivery_worker.sh 17/17 unchanged, delivery_retry.sh 11/11
unchanged. Conformance preserved at 761/761.
The restart hydration helper (delivery_worker:state_from_proj/2
or similar) lands once 8b-timer can wire the live retry loop
(Blockers #3 — erlang:send_after substrate gap still open).
Rose Ash
Monorepo for the Rose Ash cooperative platform — six Quart microservices sharing a common infrastructure layer, a single PostgreSQL database, and an ActivityPub federation layer.
Services
| Service | URL | Description |
|---|---|---|
| blog | blog.rose-ash.com | Content management, Ghost sync, navigation, editor |
| market | market.rose-ash.com | Product listings, scraping, market pages |
| cart | cart.rose-ash.com | Shopping cart, checkout, orders, SumUp payments |
| events | events.rose-ash.com | Calendar, event entries, container widgets |
| federation | federation.rose-ash.com | OAuth2 authorization server, ActivityPub hub, social features |
| account | account.rose-ash.com | User dashboard, newsletters, tickets, bookings |
All services are Python 3.11 / Quart apps served by Hypercorn, deployed as a Docker Swarm stack.
Repository structure
rose-ash/
├── shared/ # Common code: models, services, infrastructure, templates
│ ├── models/ # Canonical SQLAlchemy ORM models (all domains)
│ ├── services/ # Domain service implementations + registry
│ ├── contracts/ # DTOs, protocols, widget contracts
│ ├── infrastructure/ # App factory, OAuth, ActivityPub, fragments, Jinja setup
│ ├── templates/ # Shared base templates and partials
│ ├── static/ # Shared CSS, JS, images
│ ├── editor/ # Prose editor (Node build, blog only)
│ └── alembic/ # Database migrations
├── blog/ # Blog app
├── market/ # Market app
├── cart/ # Cart app
├── events/ # Events app
├── federation/ # Federation app
├── account/ # Account app
├── docker-compose.yml # Swarm stack definition
├── deploy.sh # Local build + restart script
├── .gitea/workflows/ # CI: build changed apps + deploy
├── _config/ # Runtime config (app-config.yaml)
├── schema.sql # Reference schema snapshot
└── .env # Environment variables (not committed)
Each app follows the same layout:
{app}/
├── app.py # App entry point (creates Quart app)
├── path_setup.py # Adds project root + app dir to sys.path
├── entrypoint.sh # Container entrypoint (wait for DB, run migrations, start)
├── Dockerfile # Build instructions (monorepo context)
├── bp/ # Blueprints (routes, handlers)
│ └── fragments/ # Fragment endpoints for cross-app composition
├── models/ # Re-export stubs pointing to shared/models/
├── services/ # App-specific service wiring
├── templates/ # App-specific templates (override shared/)
└── config/ # App-specific config
Key architecture patterns
Shared models — All ORM models live in shared/models/. Each app's models/ directory contains thin re-export stubs. factory.py imports all six apps' models at startup so SQLAlchemy relationship references resolve across domains.
Service contracts — Apps communicate through typed protocols (shared/contracts/protocols.py) and frozen dataclass DTOs (shared/contracts/dtos.py), wired via a singleton registry (shared/services/registry.py). No direct HTTP calls between apps for domain logic.
Fragment composition — Apps expose HTML fragments at /internal/fragments/<type> for cross-app UI composition. The blog fetches cart, account, navigation, and event fragments to compose its pages. Fragments are cached in Redis with short TTLs.
OAuth SSO — Federation is the OAuth2 authorization server. All other apps are OAuth clients with per-app first-party session cookies (Safari ITP compatible). Login/callback/logout routes are auto-registered via shared/infrastructure/oauth.py.
ActivityPub — Each app has its own AP actor (virtual projection of the same keypair). The federation app is the social hub (timeline, compose, follow, notifications). Activities are emitted to ap_activities table and processed by EventProcessor.
Development
Quick deploy (skip CI)
# Rebuild + restart one app
./deploy.sh blog
# Rebuild + restart multiple apps
./deploy.sh blog market
# Rebuild all
./deploy.sh --all
# Auto-detect changes from git
./deploy.sh
Full stack deploy
source .env
docker stack deploy -c docker-compose.yml coop
Build a single app image
docker build -f blog/Dockerfile -t registry.rose-ash.com:5000/blog:latest .
Run migrations
Migrations run automatically on the blog service startup when RUN_MIGRATIONS=true is set (only blog runs migrations; all other apps skip them).
# Manual migration
docker exec -it $(docker ps -qf name=coop_blog) bash -c "cd shared && alembic upgrade head"
CI/CD
A single Gitea Actions workflow (.gitea/workflows/ci.yml) handles all six apps:
- Detects which files changed since the last deploy
- If
shared/ordocker-compose.ymlchanged, rebuilds all apps - Otherwise rebuilds only apps with changes (or missing images)
- Pushes images to the private registry
- Runs
docker stack deployto update the swarm
Required secrets
| Secret | Value |
|---|---|
DEPLOY_SSH_KEY |
Private SSH key for root access to the deploy host |
DEPLOY_HOST |
Hostname or IP of the deploy server |
Infrastructure
- Runtime: Python 3.11, Quart (async Flask), Hypercorn
- Database: PostgreSQL 16 (shared by all apps)
- Cache: Redis 7 (page cache, fragment cache, sessions)
- Orchestration: Docker Swarm
- Registry:
registry.rose-ash.com:5000 - CI: Gitea Actions
- Reverse proxy: Caddy (external, not in this repo)