Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix(scrape-worker): cap sitemaps per crawl at 20 to avoid queue flooding
3229963 Gergő Móricz committed 1 year ago
Merge pull request #2143 from firecrawl/feat/adding-viasocket-integration
1740e42 Ademílson Tonato committed 1 year ago
chore: remove unused dependencies + various CI fixes (#2128)
1b47d43 Gergő Móricz committed 1 year ago