Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Python aiai-agentscrawl4ai-cloudcrawlerdata-extractionllmmarkdownmcpopen-sourceplaywrightpythonragscraperweb-crawlerweb-crawlingweb-scraping
Commits on main
chore: remove dead normalize_url definitions from utils
44968c5 Rajan Chavada committed 4 months ago
fix(html2text): preserve all attributes on table tags when bypass_tables is enabled (#2007)
511c73c ntohidi committed 4 months ago
fix(docker): let dashboard/playground load when auth gate is active (#2037)
cbf5e5c ntohidi committed 4 months ago
deprecate PruningContentFilter; make PruningContentFilterLXML the documented default
42ad7ec unclecode committed 4 months ago
Merge perf/pruning-lxml: lxml-native PruningContentFilterLXML as default pruning engine
4f031c4 unclecode committed 4 months ago
perf(content-filter): lxml-native PruningContentFilterLXML, ~10x faster pruning, made default
c2e4cd6 unclecode committed 4 months ago
docs(0.9.0): flag secure-by-default changes on the self-hosting page
c66f327 unclecode committed 5 months ago
release: merge 0.9.0 secure-by-default Docker server hardening into develop
60886d1 unclecode committed 5 months ago
fix(deep-crawl): best_first keeps boundary page at max_pages (#859)
dc5e2a8 unclecode committed 5 months ago
security(credits): credit Geo (geo-chen) for 0.8.8 base_url exfil; update secsys_codex handle (FORIMOC)
930dc0a unclecode committed 5 months ago
docs: 0.8.8 release notes, changelog, README; pre-announce next secure-by-default release
5d3d0fe unclecode committed 5 months ago
security(credits): credit IcySun & Yashon (co-reporter) per their request
7259d73 unclecode committed 5 months ago
chore: gitignore .security/ - advisory payloads stay private, published via GHSA only
7b280d3 unclecode committed 5 months ago
docs: 0.8.7 release notes, changelog, README highlights; finalize security credits (q1uf3ng confirmed, Velayutham Selvaraj)
30187e6 unclecode committed 5 months ago
security(credits): acknowledge independent reporters Velayutham S and IcySun
d705d7c unclecode committed 5 months ago
security(advisory): fold direct /crawl,/md,/llm SSRF + IPv6-mapped bypass into advisory-2; credit secsys_codex
71e1667 unclecode committed 5 months ago
merge: domain mapper + 4 core fixes cherry-picked from cloud (OSS-safe, audited) for 0.8.7
bd20700 unclecode committed 5 months ago
fix(logger): default Console width=200 for non-TTY contexts
c06a3dc unclecode committed 7 months ago