Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix(crawl-status): avoid race conditions where crawl may be deemed failed
b696bfc Gergő Móricz committed 2 years ago
fix(queue-worker): remove concurrency entries when done in sentry-less branch
d288192 Gergő Móricz committed 2 years ago
fix(v1/scrape): add total wait specified in request to timeout
f00c0b8 Gergő Móricz committed 2 years ago
feat(WebScraper): always report error of last scraper in order
43730b5 Gergő Móricz committed 2 years ago