delivery_worker state shape gains :next_retry proplist alongside
the existing :attempts:
[{peer, _}, {pending, _}, {attempts, [{Cid, N}]},
{next_retry, [{Cid, NextRetryAt}]}, {dead_letter, _},
{dispatch_fn, _}]
New pure-functional exports:
record_failure_pure/3(Cid, Now, State)
Bumps :attempts for Cid. On the 6th failure
(backoff_for returns dead_letter) moves the matching
activity from :pending to :dead_letter and clears the
:next_retry entry. Otherwise sets next_retry to
Now + backoff_for(NewAttempts).
record_success_pure/2(Cid, State)
Clears both :attempts and :next_retry for Cid.
next_due_pure/2(Now, State)
Returns cids whose retry time has passed (insertion
order preserved so the worker drains in FIFO retry
order).
attempts_for/2, next_retry_at/2, dead_letter_list/1
Read-side accessors.
Internal helper move_to_dead_letter/2 + take_by_cid/4 walks
:pending to find the matching activity by cid.
11/11 in next/tests/delivery_retry.sh covering:
- fresh state: 0 attempts / undefined retry / [] dead_letter
- record_failure bumps to 1
- record_failure sets next_retry_at = Now + 30 (slot 1)
- second failure: attempts=2, NextRetryAt = Now + 300 (slot 2)
- record_success clears both
- next_due returns due cids
- next_due empty before due
- 6th failure -> dead-letter; activity out of :pending
- dead-lettered cid removed from :next_retry
- per-cid isolation: success on one doesn't disturb another
delivery_worker.sh 17/17 unchanged (new exports are additive).
Blockers added:
#2 — Native http-request primitive missing in bin/sx_server.ml
(briefing assumed it existed; only http-listen exists).
Belongs to loops/fed-prims. Step 8e wrapper waits for
the native.
#3 — erlang:send_after-style timer primitive missing. Needed
for the real retry loop. Belongs to loops/erlang. 8b-pure
captures the semantics so 8b-timer is a 1-shot wiring
when the primitive lands.
Conformance preserved at 761/761.
Rose Ash
Monorepo for the Rose Ash cooperative platform — six Quart microservices sharing a common infrastructure layer, a single PostgreSQL database, and an ActivityPub federation layer.
Services
| Service | URL | Description |
|---|---|---|
| blog | blog.rose-ash.com | Content management, Ghost sync, navigation, editor |
| market | market.rose-ash.com | Product listings, scraping, market pages |
| cart | cart.rose-ash.com | Shopping cart, checkout, orders, SumUp payments |
| events | events.rose-ash.com | Calendar, event entries, container widgets |
| federation | federation.rose-ash.com | OAuth2 authorization server, ActivityPub hub, social features |
| account | account.rose-ash.com | User dashboard, newsletters, tickets, bookings |
All services are Python 3.11 / Quart apps served by Hypercorn, deployed as a Docker Swarm stack.
Repository structure
rose-ash/
├── shared/ # Common code: models, services, infrastructure, templates
│ ├── models/ # Canonical SQLAlchemy ORM models (all domains)
│ ├── services/ # Domain service implementations + registry
│ ├── contracts/ # DTOs, protocols, widget contracts
│ ├── infrastructure/ # App factory, OAuth, ActivityPub, fragments, Jinja setup
│ ├── templates/ # Shared base templates and partials
│ ├── static/ # Shared CSS, JS, images
│ ├── editor/ # Prose editor (Node build, blog only)
│ └── alembic/ # Database migrations
├── blog/ # Blog app
├── market/ # Market app
├── cart/ # Cart app
├── events/ # Events app
├── federation/ # Federation app
├── account/ # Account app
├── docker-compose.yml # Swarm stack definition
├── deploy.sh # Local build + restart script
├── .gitea/workflows/ # CI: build changed apps + deploy
├── _config/ # Runtime config (app-config.yaml)
├── schema.sql # Reference schema snapshot
└── .env # Environment variables (not committed)
Each app follows the same layout:
{app}/
├── app.py # App entry point (creates Quart app)
├── path_setup.py # Adds project root + app dir to sys.path
├── entrypoint.sh # Container entrypoint (wait for DB, run migrations, start)
├── Dockerfile # Build instructions (monorepo context)
├── bp/ # Blueprints (routes, handlers)
│ └── fragments/ # Fragment endpoints for cross-app composition
├── models/ # Re-export stubs pointing to shared/models/
├── services/ # App-specific service wiring
├── templates/ # App-specific templates (override shared/)
└── config/ # App-specific config
Key architecture patterns
Shared models — All ORM models live in shared/models/. Each app's models/ directory contains thin re-export stubs. factory.py imports all six apps' models at startup so SQLAlchemy relationship references resolve across domains.
Service contracts — Apps communicate through typed protocols (shared/contracts/protocols.py) and frozen dataclass DTOs (shared/contracts/dtos.py), wired via a singleton registry (shared/services/registry.py). No direct HTTP calls between apps for domain logic.
Fragment composition — Apps expose HTML fragments at /internal/fragments/<type> for cross-app UI composition. The blog fetches cart, account, navigation, and event fragments to compose its pages. Fragments are cached in Redis with short TTLs.
OAuth SSO — Federation is the OAuth2 authorization server. All other apps are OAuth clients with per-app first-party session cookies (Safari ITP compatible). Login/callback/logout routes are auto-registered via shared/infrastructure/oauth.py.
ActivityPub — Each app has its own AP actor (virtual projection of the same keypair). The federation app is the social hub (timeline, compose, follow, notifications). Activities are emitted to ap_activities table and processed by EventProcessor.
Development
Quick deploy (skip CI)
# Rebuild + restart one app
./deploy.sh blog
# Rebuild + restart multiple apps
./deploy.sh blog market
# Rebuild all
./deploy.sh --all
# Auto-detect changes from git
./deploy.sh
Full stack deploy
source .env
docker stack deploy -c docker-compose.yml coop
Build a single app image
docker build -f blog/Dockerfile -t registry.rose-ash.com:5000/blog:latest .
Run migrations
Migrations run automatically on the blog service startup when RUN_MIGRATIONS=true is set (only blog runs migrations; all other apps skip them).
# Manual migration
docker exec -it $(docker ps -qf name=coop_blog) bash -c "cd shared && alembic upgrade head"
CI/CD
A single Gitea Actions workflow (.gitea/workflows/ci.yml) handles all six apps:
- Detects which files changed since the last deploy
- If
shared/ordocker-compose.ymlchanged, rebuilds all apps - Otherwise rebuilds only apps with changes (or missing images)
- Pushes images to the private registry
- Runs
docker stack deployto update the swarm
Required secrets
| Secret | Value |
|---|---|
DEPLOY_SSH_KEY |
Private SSH key for root access to the deploy host |
DEPLOY_HOST |
Hostname or IP of the deploy server |
Infrastructure
- Runtime: Python 3.11, Quart (async Flask), Hypercorn
- Database: PostgreSQL 16 (shared by all apps)
- Cache: Redis 7 (page cache, fragment cache, sessions)
- Orchestration: Docker Swarm
- Registry:
registry.rose-ash.com:5000 - CI: Gitea Actions
- Reverse proxy: Caddy (external, not in this repo)