Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
Merge pull request #987 from mendableai/default-to-pdf-parse
4ad6665 Eric Ciarla committed 1 year ago
Merge pull request #984 from mendableai/default-to-pdf-parse
db8e9c3 Eric Ciarla committed 1 year ago
Merge branch 'main' into rafa/fix-default-on-schema-llm-extract
8192d75 Rafael Miller committed 1 year ago
fix(WebScraper/tryGetSitemap): deduplicate sitemap links list
e97ee4a Gergő Móricz committed 1 year ago
fix(crawl-redis/lockURL): only add to visited_unique if lock succeeds
37f58ef Gergő Móricz committed 1 year ago