Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥
TypeScript aiai-agentsai-crawlerai-scrapingai-searchcrawlerdata-extractionhtml-to-markdownllmmarkdownscraperscrapingweb-crawlerweb-dataweb-data-extractionweb-scraperweb-scrapingweb-searchwebscraping
Commits on main
fix(api): enhance crawl warnings for low result scenarios
628c22c AdemÃlson Tonato committed 1 year ago
fix(api): refine warning logic for low results in map controller to account for user-specified limit
fc0ca3b AdemÃlson Tonato committed 1 year ago
chore: save work in progress before updating from main
29a125b AdemÃlson Tonato committed 1 year ago
fix(api): update package.json and pnpm-lock.yaml to allow specific scripts and upgrade debug dependency to version 4.4.3; refactor URL schema export in batch-scrape controller
3065838 AdemÃlson Tonato committed 1 year ago
feat(api): add warnings for low results and robots.txt restrictions in map and crawl controllers
75debd6 AdemÃlson Tonato committed 1 year ago
feat(crawl): implement URL modification handling in crawl controller
8136e7f AdemÃlson Tonato committed 1 year ago
Merge pull request #2220 from firecrawl/nsc/improve-crawl-preview
6d2b9f4 Nicolas committed 1 year ago
python-sdk: include "cancelled" in CrawlJob.status and exit wait loop on cancel (fixes #2190) (#2240)
57babba Jeel Rupareliya committed 1 year ago
Revert "feat(api/crawl-status): use crawl_status_2 RPC (#2239)"
e4c9921 Gergő Móricz committed 1 year ago
Allow self-hosted webhook delivery to private IP addresses (#2232)
5756b83 Abimael Martell committed 1 year ago