Merges full history from art-dag/mono.git into the monorepo
under the artdag/ directory. Contains: core (DAG engine),
l1 (Celery rendering server), l2 (ActivityPub registry),
common (shared templates/middleware), client (CLI), test (e2e).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
git-subtree-dir: artdag
git-subtree-mainline: 1a179de547
git-subtree-split: 4c2e716558
Vision document for migrating rose-ash to an s-expression-based
architecture where pages, media renders, and LLM-generated content
share a unified DAG execution model with content-addressed caching
on IPFS/IPNS.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 1 - Relations service (internal): owns ContainerRelation, exposes
get-children data + attach/detach-child actions. Retargeted events, blog,
market callers from cart to relations.
Phase 2 - Likes service (internal): unified Like model replaces ProductLike
and PostLike with generic target_type/target_slug/target_id. Exposes
is-liked, liked-slugs, liked-ids data + toggle action.
Phase 3 - PageConfig → blog: moved ownership to blog with direct DB queries,
removed proxy endpoints from cart.
Phase 4 - Orders service (public): owns Order/OrderItem + SumUp checkout
flow. Cart checkout now delegates to orders via create-order action.
Webhook/return routes and reconciliation moved to orders.
Phase 5 - Infrastructure: docker-compose, deploy.sh, Dockerfiles updated
for all 3 new services. Added orders_url helper and factory model imports.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The global add_to_cart route was calling find_or_create_cart_item without
denormalized product data, leaving NULL columns. Now fetches product info
via fetch_data("market", "products-by-ids") before creating the cart item.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
entry_associations only uses HTTP fetch_data/call_action, no direct DB.
Events app imported it via ..post.services which doesn't exist in events.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
container_relations is a generic parent/child graph used by blog
(menu_nodes), market (marketplaces), and events (calendars). Move it
to cart as shared infrastructure. All services now call cart actions
(attach-child/detach-child) instead of querying the table directly.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
page_configs table lives in db_cart but blog was querying it directly,
causing UndefinedTableError. Move all PageConfig read/write endpoints to
cart service and have blog proxy via fetch_data/call_action.
Also fix OAuth callback to use code_hash lookup (codes are now stored
hashed) and pass grant_token in redirect URL to prevent auth loops.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Restores the missing entry point script for API-mode scraping.
Calls get_access_token() before starting to trigger device flow
login if no token is saved.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove cross-DB relationships (CartItem.product, CartItem.market_place,
OrderItem.product) that break with per-service databases. Denormalize
product and marketplace fields onto cart_items/order_items at write time.
- Add AP internal inbox infrastructure (shared/infrastructure/internal_inbox*)
for synchronous inter-service writes via HMAC-authenticated POST
- Cart inbox blueprint handles Add/Remove/Update rose:CartItem activities
- Market app sends AP activities to cart inbox instead of writing CartItem directly
- Cart services use denormalized columns instead of cross-DB hydration/joins
- Add marketplaces-by-ids data endpoint to market service
- Alembic migration adds denormalized columns to cart_items and order_items
- Add OAuth device flow auth to market scraper persist_api (artdag client pattern)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
market_context() was querying CartItem directly via g.s (db_market),
but cart_items lives in db_cart. Replace with fetch_data("cart",
"cart-items") and add the corresponding data endpoint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
nh3 manages the rel attribute internally — setting it in
tag_attributes triggers an assertion. Use link_rel parameter instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Critical: Add ownership checks to all order routes (IDOR fix).
High: Redis rate limiting on auth endpoints, HMAC-signed internal
service calls replacing header-presence-only checks, nh3 HTML
sanitization on ghost_sync and product import, internal auth on
market API endpoints, SHA-256 hashed OAuth grant/code tokens.
Medium: SECRET_KEY production guard, AP signature enforcement,
is_admin param removal, cart_sid validation, SSRF protection on
remote actor fetch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
_load_user ran before _check_auth_state, so g.user was set to the wrong
user before the grant check could clear the stale session. Now grant
verification runs first, ensuring stale sessions are cleared before
the user is loaded.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two issues fixed:
- Sessions with uid but no grant_token (legacy or corrupt) were not
validated at all, allowing a user to be logged in as whoever got
their old numeric user ID after a DB rebuild
- DB errors during grant verification silently kept stale sessions
alive; now treated as invalid to fail-safe
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
In Docker images, alembic.ini is at /app/alembic.ini (no service subdir).
In dev, it's at /app/{service}/alembic.ini (bind-mounted). Now tries the
service dir first, falls back to current dir.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each service (blog, market, cart, events, federation, account) now owns
its own database schema with independent Alembic migrations. Removes the
monolithic shared/alembic/ that ran all migrations against a single DB.
- Add per-service alembic.ini, env.py, and 0001_initial.py migrations
- Add shared/db/alembic_env.py helper with table-name filtering
- Fix cross-DB FK in blog/models/snippet.py (users lives in db_account)
- Fix cart_impl.py cross-DB queries: fetch products and market_places
via internal data endpoints instead of direct SQL joins
- Fix blog ghost_sync to fetch page_configs from cart via data endpoint
- Add products-by-ids and page-config-ensure data endpoints
- Update all entrypoint.sh to create own DB and run own migrations
- Cart now uses db_cart instead of db_market
- Add docker-compose.dev.yml, dev.sh for local development
- CI deploys both rose-ash swarm stack and rose-ash-dev compose stack
- Fix Quart namespace package crash (root_path in factory.py)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Each app was running 2 workers (~100MB each). On a 3.8GB system with
6 apps, the 12 workers consumed ~1.2GB and caused swap pressure.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the device code grant flow so artdag CLI can authenticate
via browser approval. Includes device/authorize, device/token endpoints,
user code verification page, and approval confirmation template.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
JAX via XLA produces identical output on CPU and GPU. Previously
CUDA hand-written kernels were preferred on GPU, causing visual
differences vs the JAX CPU fallback. Now JAX is always used first,
with legacy CuPy/GPUFrame as fallback only when JAX is unavailable.
Also adds comprehensive CLAUDE.md for the monorepo.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add -pix_fmt yuv420p to multi_res_output.py libx264 path so browsers
can decode CPU-encoded segments (was producing yuv444p / High 4:4:4).
- Switch silent auth check and coop fragment middlewares from opt-out
blocklists to opt-in: only run for GET requests with Accept: text/html.
Prevents unnecessary nav-tree/auth-menu HTTP calls on every HLS segment,
IPFS proxy, and API request.
- Add opaque grant token verification to L1/L2 dependencies.
- Migrate client CLI to device authorization flow.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The StreamInterpreter was created without use_jax=True, so the JAX
compiler was never activated for production rendering. Desktop testing
had this enabled but the celery task path did not.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When CUDA fused kernels aren't available, the fused-pipeline primitive
now uses JAX ops (jax_rotate, jax_scale, jax_shift_hue, etc.) instead
of falling back to one-by-one CuPy/GPUFrame operations. Legacy GPUFrame
path retained as last resort when JAX is also unavailable.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The hardcoded h264_nvenc encoder fails on CPU-only workers. Now uses
check_nvenc_available() to auto-detect and falls back to libx264.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
CPU workers can now run GPU-queue rendering tasks via JAX on CPU.
GPU image overrides with jax[cuda12] for full CUDA acceleration.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The page_config relationship was removed during DB split; use the
page_config_id column and fetch page config via HTTP instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Error handlers for FragmentError and generic Exception now return
self-contained HTML (no render_template) to avoid the infinite loop
where context processor → fetch_fragments → error → render_template
→ context processor → fetch_fragments → error ...
- Account Ghost membership sync moved to background task so it doesn't
block Hypercorn's startup timeout (was causing crash-loop).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
FragmentError now renders a 503 page naming which service is down
instead of a generic 500 error. Helps debug during deploys.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
menu_nodes table lives in db_blog. Non-blog apps (cart, market, events,
account, federation) were querying it on their own DB session, causing
UndefinedTableError. The nav-tree fragment from blog provides the real
navigation; menu_items is now an empty fallback list.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AP blueprints (activitypub.py, ap_social.py) were querying federation
tables (ap_actor_profiles etc.) on g.s which points to the app's own DB
after the per-app split. Now uses g._ap_s backed by get_federation_session()
for non-federation apps.
Also hardens Ghost sync before_app_serving to catch/rollback on failure
instead of crashing the Hypercorn worker.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Prevents PgBouncer connection pool from inheriting dirty
transaction state when the non-syncing worker returns.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Two Hypercorn workers both run sync_all_content_from_ghost on startup,
racing on PostAuthor/PostTag rows. Use pg_try_advisory_lock so only
one worker runs the sync.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AsyncSession.no_autoflush is a sync context manager, can't use
with 'async with'. Toggle autoflush manually instead.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PageConfig (db_blog) decoupling:
- Blog: add page-config, page-config-by-id, page-configs-batch data endpoints
- Blog: add update-page-config action endpoint for events payment admin
- Cart: hydrate_page, resolve_page_config, get_cart_grouped_by_page all
fetch PageConfig from blog via HTTP instead of direct DB query
- Cart: check_sumup_status auto-fetches page_config from blog when needed
- Events: payment routes read/write PageConfig via blog HTTP endpoints
- Order model: remove cross-domain page_config ORM relationship (keep column)
Cart + Market DB merge:
- Cart tables (cart_items, orders, order_items) moved into db_market
- Cart app DATABASE_URL now points to db_market (same bounded context)
- CartItem.product / CartItem.market_place relationships work again
(same database, no cross-domain join issues)
- Updated split-databases.sh, init-databases.sql, docker-compose.yml
Ghost sync fix:
- Wrap PostAuthor/PostTag delete+re-add in no_autoflush block
- Use synchronize_session="fetch" to keep identity map consistent
- Prevents query-invoked autoflush IntegrityError on composite PK
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add explicit flush after DELETE and dedup authors/tags to prevent
autoflush-triggered IntegrityError on composite PK.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move Ghost membership sync from blog to account service so blog no
longer queries account tables (users, ghost_labels, etc.). Account
runs membership sync at startup and exposes HTTP action/data endpoints
for webhook-triggered syncs and user lookups.
Key changes:
- account/services/ghost_membership.py: all membership sync functions
- account/bp/actions + data: ghost-sync-member, user-by-email, newsletters
- blog ghost_sync.py: stripped to content-only (posts, authors, tags)
- blog webhook member: delegates to account via call_action()
- try_publish: opens federation session when DBs differ
- oauth.py callback: uses get_account_session() for OAuthCode
- page_configs moved from db_events to db_blog in split script
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
edoburu/pgbouncer listens on 5432, not 6432. Update all DATABASE_URL
entries to use pgbouncer:5432. Activate per-app DATABASE_URL split
(no longer commented out).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
T0.1: Separate redis-auth service (64mb, noeviction) for auth state
T0.2: Bump data Redis from 256mb to 1gb
T0.3: Per-app DATABASE_URL via PgBouncer to per-domain databases
T0.4: PgBouncer service (transaction mode, pool=20, max_conn=300);
session.py pools reduced to 3+5 with timeout and recycle
T0.5: Hypercorn --workers 2 --keep-alive 75 on all 6 entrypoints
Deploy requires running split-databases.sh first to create per-domain
databases from the existing appdb.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
send_follow now emits a Follow activity via emit_activity() instead of
inline HTTP POST. New ap_follow_handler delivers to the remote inbox;
EventProcessor retries on failure. Wildcard delivery handler skips
Follow type to avoid duplicate broadcast.
Also add /social/ index page to per-app social blueprint.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add app_domain to APNotification model and NotificationDTO so follow
notifications display "followed you on blog" instead of just "followed
you" when the follow targets a per-app actor.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Lightweight social pages (search, follow/unfollow, followers, following,
actor timeline) auto-registered for AP-enabled apps via shared blueprint.
Federation keeps the full social hub. Followers scoped per app_domain;
post cards show "View on Hub" link instead of interaction buttons.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Templates still used the old container_nav_widgets variable from the
widget system. Updated to render container_nav_html (fragment output)
which the context processor already provides.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Internal service-to-service POSTs (call_action) were blocked by CSRF
middleware since they have no session cookie. These requests are already
gated by X-Internal-Action/X-Internal-Data headers and only reachable
on the Docker overlay network.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Unhandled exceptions in action handlers were returned as opaque
400/500 by Quart's default error handler. Now we catch, log the
full traceback, and return a JSON error body with 500 status so
the caller gets useful diagnostics.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Missed cross-app call in blog/bp/blog/routes.py:127 caused
CartService not registered error on blog.rose-ash.com homepage.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
