Official Crawl4AI Cloud: a hosted web scraping, extraction, and search API for AI agents. Free API key, 1,000 credits a month, no card. Fully open source: this repo is the Apache-2.0 crawler that powers it, and you can self-host it.
Python aiai-agentscrawl4ai-cloudcrawlerdata-extractionllmmarkdownmcpopen-sourceplaywrightpythonragscraperweb-crawlerweb-crawlingweb-scraping
Commits on main
docs(pdf): document PDFContentScrapingStrategy pairing requirement
caa34f1 ntohidi committed 3 months ago
Merge pull request #2138 from unclecode/fix/pdf-antibot-false-positive
a259693 Nasrin committed 3 months ago
Merge pull request #2139 from unclecode/fix/csp-sandbox-overlay-hang
2164657 Nasrin committed 3 months ago
fix(docker): chain egress proxy through upstream HTTP(S)_PROXY
890af3d Soham Kukreti committed 3 months ago
fix: remove unconditional setTimeout waits from overlay/consent removal scripts
b536bf5 Soham Kukreti committed 3 months ago
fix: Don't veto PDFCrawlerStrategy placeholder responses (#2135)
2e30e54 Soham Kukreti committed 3 months ago
test(docker): exercise crawl failures through endpoints
4fc16ca nightcityblade committed 3 months ago
fix(config): make body visibility timeout configurable
0a78f07 nightcityblade committed 3 months ago
fix(docker): preserve single-url crawl failure details
588de04 nightcityblade committed 3 months ago
docs(docker): update self-hosting & migration guides for 0.9.x
a659914 Soham Kukreti committed 4 months ago
fix(CONTRIBUTORS.md): update section title from 'Core Team' to 'Maintainers'
9e9e826 ntohidi committed 4 months ago
Merge pull request #2072 from reallav0/agent/cancel-streaming-dispatch-tasks
935f591 Nasrin committed 4 months ago
fix(test): update monitor auth test for WS-safe router config
9ee84cc ntohidi committed 4 months ago
Merge branch 'develop' of https://github.com/unclecode/crawl4ai into develop
f1fdf27 ntohidi committed 4 months ago
Merge pull request #2067 from reallav0/agent/include-playwright-headless-shell
6357f85 Nasrin committed 4 months ago