Monorepo: consolidate 7 repos into one
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 1m5s
All checks were successful
Build and Deploy / build-and-deploy (push) Successful in 1m5s
Combines shared, blog, market, cart, events, federation, and account into a single repository. Eliminates submodule sync, sibling model copying at build time, and per-app CI orchestration. Changes: - Remove per-app .git, .gitmodules, .gitea, submodule shared/ dirs - Remove stale sibling model copies from each app - Update all 6 Dockerfiles for monorepo build context (root = .) - Add build directives to docker-compose.yml - Add single .gitea/workflows/ci.yml with change detection - Add .dockerignore for monorepo build context - Create __init__.py for federation and account (cross-app imports)
This commit is contained in:
22
market/scrape/build_snapshot/tools/capture_sub.py
Normal file
22
market/scrape/build_snapshot/tools/capture_sub.py
Normal file
@@ -0,0 +1,22 @@
|
||||
from urllib.parse import urljoin
|
||||
from urllib.parse import urljoin
|
||||
from shared.config import config
|
||||
from shared.utils import log
|
||||
from ...listings import scrape_products
|
||||
|
||||
async def capture_sub(
|
||||
sub,
|
||||
slug,
|
||||
):
|
||||
sub_slug = sub.get("slug")
|
||||
if not sub_slug:
|
||||
return
|
||||
sub_url = urljoin(config()["base_url"], f"/{slug}/{sub_slug}")
|
||||
log(f"[{slug}/{sub_slug}] page 1…")
|
||||
items_s, total_pages_s = await scrape_products(sub_url, page=1)
|
||||
spmax = int(total_pages_s or 1)
|
||||
for p in range(2, spmax + 1):
|
||||
log(f"[{slug}/{sub_slug}] page {p}…")
|
||||
items_ps, _ = await scrape_products(sub_url, page=p)
|
||||
items_s.extend(items_ps)
|
||||
return (sub_url, items_s, total_pages_s)
|
||||
Reference in New Issue
Block a user