核心思路:启动浏览器、注入本地存储(可选)、打开URL、等选择器稳定后取inner_text。示例片段如下:
code
from playwright.sync_api import sync_playwright
def fetch_visible_text(url: str, wait_css: str = "body", timeout_ms: int = 30000) -> str:code
browser = p.chromium.launch(headless=True)
page = browser.new_page()page.wait_for_selector(wait_css, timeout=timeout_ms)
code
text = page.inner_text(wait_css)return text
if __name__ == "__main__":
code
u = "https://example.com/ai-answer?q=品牌名"落地时注意合规:robots、频率限制、以及是否违反服务条款。我们内部是把抓取间隔拉到5分钟以上,并且只抓公开页面。数据入库我用PostgreSQL,表结构大概是 snapshot(id, url_hash, fetched_at, content_hash, raw_text),content_hash用sha256方便去重。
如果你也在做企业AI落地监测,欢迎交流:你们是怎么定义「品牌被引用」的?是严格匹配公司名,还是把别名、英文拼写也算进去?
(场景参考:广州本地企业试点)