Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
Merge pull request #727 from mendableai/nsc/error-js-sdk-improv
85e9f7b Nicolas committed 2 years ago
Nick: shouldnt fallback on a 400 + error code should be correct on page status code
a150aa8 Nicolas committed 2 years ago
Merge pull request #721 from mendableai/feat/concurrency-limit
d1b8383 Nicolas committed 2 years ago
Revert "feat(queue-worker): always crawl links from content even if sitemapped"
258c67c Nicolas committed 2 years ago
feat(queue-worker): always crawl links from content even if sitemapped
3c045c4 Gergő Móricz committed 2 years ago