This repository has been archived on 2026-02-24. You can view files and clone it. You cannot open issues or pull requests or push a commit.
Files
market/scrape/product/helpers/text.py
giles 478636f799 feat: decouple market from shared_lib, add app-owned models
Phase 1-3 of decoupling:
- path_setup.py adds project root to sys.path
- Market-owned models in market/models/ (market, market_place)
- All imports updated: shared.infrastructure, shared.db, shared.browser, etc.
- MarketPlace uses container_type/container_id instead of post_id FK

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:46:32 +00:00

17 lines
523 B
Python

from __future__ import annotations
import re
from shared.utils import normalize_text
from shared.config import config
def clean_title(t: str) -> str:
t = normalize_text(t)
t = re.sub(r":\s*$", "", t)
return t
def is_blacklisted_heading(title: str) -> bool:
"""Return True if heading should be skipped based on config blacklist."""
bl = (config().get("blacklist") or {}).get("product-details") or []
low = (title or "").strip().lower()
return any(low == (s or "").strip().lower() for s in bl)