Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix: skip URL rewriting for published Google Docs (#2500)
2be7a87 Himanshu Gupta committed 11 months ago
feat(api): concurrency limit info in scrape metadata (#2497)
f40d19d delong3 committed 11 months ago
feat(api/sentry): make sampling rates configurable via environment variables (#2495)
36a5702 tom committed 11 months ago
fix: allow partial results when searching multiple sources (#2490)
73a25ef devin-ai-integration[bot] committed 11 months ago
fix: make auto-recharge email show actual credit amount instead of hardcoded 1000 (#2485)
5ba418d devin-ai-integration[bot] committed 11 months ago
fix: off-by-one bug in batch scrape pagination (#2492)
76f777a Himanshu Gupta committed 11 months ago
fix(api/changeTracking): don't need DB row for getting the job content
536211f Gergő Móricz committed 11 months ago
Merge pull request #2193 from Chadha93/chadha93/self-host-concurrency
d880ed7 Gaurav Chadha committed 11 months ago
fix(api): don't bill where stealth proxy was unsupported (#2484)
db1a4e2 tom committed 11 months ago
Merge pull request #2480 from firecrawl/dl3/fix/sitemap-max-age
5a288e6 delong3 committed 11 months ago
fix(api): sitemap / robots 404 handling + skip markdown check for robots
5625f48 delong3 committed 11 months ago