Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix(WebScraper/sitemap): await urlsHandler to fix race condition
71a8f74 Gergő Móricz committed 1 year ago
fix(queue-worker): do not follow redirect URLs if they are not allowed by the crawl options
9005757 Gergő Móricz committed 1 year ago
Merge pull request #991 from RutamBhagat/rust-sdk-conditionally-enforce-api-key
a431caf Nicolas committed 1 year ago
Merge pull request #1018 from mendableai/feat/add-favicon-metadata
05d5f84 Nicolas committed 1 year ago
Merge pull request #955 from mendableai/rafa/fix-default-on-schema-llm-extract
eba5fda Nicolas committed 1 year ago
Merge pull request #990 from RutamBhagat/python-sdk-conditionally-enforce-api-key
cd08be7 Nicolas committed 1 year ago
Merge pull request #1015 from mendableai/nsc/improves-sitemap-fetching
c5b6495 Nicolas committed 1 year ago
Merge pull request #1003 from RutamBhagat/credit-usage-api-docs
2ea0e9a Nicolas committed 1 year ago