giles 0f85bd963a
Some checks failed
Test, Build, and Deploy / test-build-deploy (push) Failing after 25s
fed-sx-m1: Step 8b-start — http_server:start/1 + dict↔proplist marshaling; live TCP smoke 5/5
`next/kernel/http_server.erl` gains `start/1(Port)` + `start/2(Port, Cfg)`. Both spawn an Erlang process that hosts
the native `http:listen/2` accept loop with the Cfg-aware `route/2` as the handler.

The blocker — the BIF wrapper in `lib/erlang/runtime.sx` had no dict↔proplist marshaling, so Erlang handler funs
couldn't pattern-match on an opaque SX request dict — is resolved by a new family of helpers added next to `er-of-sx`
(which is left untouched so non-HTTP callers see no behavioural drift):

  er-request-dict-to-proplist   request dict -> [{method,<<>>},{path,<<>>},...] (atom keys)
  er-of-sx-deep                 recursive marshal: dicts -> binary-keyed proplist
  er-dict-to-header-proplist    headers: [{<<"content-type">>,<<"text/plain">>},...]
                                 (binary keys keep arbitrary user input out of the atom table)
  er-proplist-to-dict           response proplist -> SX dict for native serialiser
  er-proplist-fill!             dict-set! walker over a cons-of-2-tuples
  er-to-sx-deep                 recursive marshal: cons-of-2-tuples -> nested dict
  er-proplist-2tuple?           predicate distinguishing a header proplist from a binary body

`er-bif-http-listen`'s body is updated to route through the new pair instead of `er-of-sx` / `er-to-sx`. Existing
`http_listen_bif.sh` (Step 8a) still passes — the BIF's external contract (port + handler validation, registration)
hasn't changed, only the request/response shape the handler sees.

This commit also lands a small pre-existing unstaged refactor that was sitting in the same file (er-binary->string
helper above er-bif-http-listen, a "Register everything at load time." comment move, and the binary_to_list /
list_to_binary / er-iolist-walk! defines reshuffled into the er-register-builtin-bifs! body). The refactor was
agreed-out-of-scope earlier in the loop but was unblocked this iteration when the user OK'd progress on 8b-start.
Bundling it here keeps the lib/erlang/runtime.sx diff coherent.

Tests:
- `next/tests/http_marshal.sh` (10 cases) — marshaling unit tests: request dict → cons proplist; method as
  <<"GET">> via SX-side proplist walker; path-as-string roundtrip; nested headers reach through binary keys;
  response status/body field marshaling; nested headers reconstruct dict; full round-trip preserves status.
- `next/tests/http_server_start.sh` (6 cases) — structural verification: http_server module loaded, start bound
  in module env, marshalers defined as lambdas, http:listen BIF registered. Can't invoke spawn in an Erlang test
  because the cooperative scheduler (`er-sched-run-all!`) drains every runnable process before returning to the
  caller, and the listener's accept loop never exits.
- `next/tests/http_server_tcp.sh` (5 cases) — **first live end-to-end transport test in the milestone**: boots
  sx_server in background with FIFO-held stdin (~10s boot for all lib/erlang/*.sx loads + module compile +
  Unix.bind), then drives the listener via shell-side curl over real TCP. Verifies GET / → 200, GET
  /.well-known/sx-capabilities → 200, GET unknown → 404, POST /activity → 401 with no/bad bearer. Doubles as the
  smoke surface for 9a-tcp / 9b-tcp.

Erlang conformance **761/761** unchanged. All standing suites stay green (http_listen_bif 5/5, log_disk 12/12,
log_rotate 10/10, term_codec 18/18).

Step 8b-start ticked in plans/fed-sx-milestone-1.md. Remaining in the milestone: 9a-tcp / 9b-tcp — partly covered
by http_server_tcp.sh's smoke probes; the full curl-driven publish flows are the next iteration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-06-05 20:30:15 +00:00
2026-03-25 00:36:57 +00:00
2026-02-24 20:10:23 +00:00

Rose Ash

Monorepo for the Rose Ash cooperative platform — six Quart microservices sharing a common infrastructure layer, a single PostgreSQL database, and an ActivityPub federation layer.

Services

Service URL Description
blog blog.rose-ash.com Content management, Ghost sync, navigation, editor
market market.rose-ash.com Product listings, scraping, market pages
cart cart.rose-ash.com Shopping cart, checkout, orders, SumUp payments
events events.rose-ash.com Calendar, event entries, container widgets
federation federation.rose-ash.com OAuth2 authorization server, ActivityPub hub, social features
account account.rose-ash.com User dashboard, newsletters, tickets, bookings

All services are Python 3.11 / Quart apps served by Hypercorn, deployed as a Docker Swarm stack.

Repository structure

rose-ash/
├── shared/              # Common code: models, services, infrastructure, templates
│   ├── models/          # Canonical SQLAlchemy ORM models (all domains)
│   ├── services/        # Domain service implementations + registry
│   ├── contracts/       # DTOs, protocols, widget contracts
│   ├── infrastructure/  # App factory, OAuth, ActivityPub, fragments, Jinja setup
│   ├── templates/       # Shared base templates and partials
│   ├── static/          # Shared CSS, JS, images
│   ├── editor/          # Prose editor (Node build, blog only)
│   └── alembic/         # Database migrations
├── blog/                # Blog app
├── market/              # Market app
├── cart/                # Cart app
├── events/              # Events app
├── federation/          # Federation app
├── account/             # Account app
├── docker-compose.yml   # Swarm stack definition
├── deploy.sh            # Local build + restart script
├── .gitea/workflows/    # CI: build changed apps + deploy
├── _config/             # Runtime config (app-config.yaml)
├── schema.sql           # Reference schema snapshot
└── .env                 # Environment variables (not committed)

Each app follows the same layout:

{app}/
├── app.py               # App entry point (creates Quart app)
├── path_setup.py        # Adds project root + app dir to sys.path
├── entrypoint.sh        # Container entrypoint (wait for DB, run migrations, start)
├── Dockerfile           # Build instructions (monorepo context)
├── bp/                  # Blueprints (routes, handlers)
│   └── fragments/       # Fragment endpoints for cross-app composition
├── models/              # Re-export stubs pointing to shared/models/
├── services/            # App-specific service wiring
├── templates/           # App-specific templates (override shared/)
└── config/              # App-specific config

Key architecture patterns

Shared models — All ORM models live in shared/models/. Each app's models/ directory contains thin re-export stubs. factory.py imports all six apps' models at startup so SQLAlchemy relationship references resolve across domains.

Service contracts — Apps communicate through typed protocols (shared/contracts/protocols.py) and frozen dataclass DTOs (shared/contracts/dtos.py), wired via a singleton registry (shared/services/registry.py). No direct HTTP calls between apps for domain logic.

Fragment composition — Apps expose HTML fragments at /internal/fragments/<type> for cross-app UI composition. The blog fetches cart, account, navigation, and event fragments to compose its pages. Fragments are cached in Redis with short TTLs.

OAuth SSO — Federation is the OAuth2 authorization server. All other apps are OAuth clients with per-app first-party session cookies (Safari ITP compatible). Login/callback/logout routes are auto-registered via shared/infrastructure/oauth.py.

ActivityPub — Each app has its own AP actor (virtual projection of the same keypair). The federation app is the social hub (timeline, compose, follow, notifications). Activities are emitted to ap_activities table and processed by EventProcessor.

Development

Quick deploy (skip CI)

# Rebuild + restart one app
./deploy.sh blog

# Rebuild + restart multiple apps
./deploy.sh blog market

# Rebuild all
./deploy.sh --all

# Auto-detect changes from git
./deploy.sh

Full stack deploy

source .env
docker stack deploy -c docker-compose.yml coop

Build a single app image

docker build -f blog/Dockerfile -t registry.rose-ash.com:5000/blog:latest .

Run migrations

Migrations run automatically on the blog service startup when RUN_MIGRATIONS=true is set (only blog runs migrations; all other apps skip them).

# Manual migration
docker exec -it $(docker ps -qf name=coop_blog) bash -c "cd shared && alembic upgrade head"

CI/CD

A single Gitea Actions workflow (.gitea/workflows/ci.yml) handles all six apps:

  1. Detects which files changed since the last deploy
  2. If shared/ or docker-compose.yml changed, rebuilds all apps
  3. Otherwise rebuilds only apps with changes (or missing images)
  4. Pushes images to the private registry
  5. Runs docker stack deploy to update the swarm

Required secrets

Secret Value
DEPLOY_SSH_KEY Private SSH key for root access to the deploy host
DEPLOY_HOST Hostname or IP of the deploy server

Infrastructure

  • Runtime: Python 3.11, Quart (async Flask), Hypercorn
  • Database: PostgreSQL 16 (shared by all apps)
  • Cache: Redis 7 (page cache, fragment cache, sessions)
  • Orchestration: Docker Swarm
  • Registry: registry.rose-ash.com:5000
  • CI: Gitea Actions
  • Reverse proxy: Caddy (external, not in this repo)
Description
No description provided
Readme 58 MiB
Languages
Python 58.8%
JavaScript 34%
OCaml 2.8%
HTML 1.7%
Common Lisp 1.5%
Other 1.2%