Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix(crawl-status, extract-status): reduce the use of BullMQ getState (#1968)
c0c4c7b Gergő Móricz committed 1 year ago
fix(queue-service): reduce redis connections to BullMQ (#1966)
308e4f4 Gergő Móricz committed 1 year ago
(sdks): added summary, fixed usage tests and v2 client for python
55e8d44 rafaelmmiller committed 1 year ago
fix(concurrency-limit): give getNextConcurrentJob some room to breathe
ce6e164 Gergő Móricz committed 1 year ago
fix(types): update maxAge default value to 2 days in baseScrapeOptions
e57f165 Gergő Móricz committed 1 year ago