Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
test: fetch test key value store correctly on all test platforms (#1493)
ee3a8ae Vlad Frangu committed 4 years ago
test: correctly set entry name for the automatic value we test (#1492)
50e0a45 Vlad Frangu committed 4 years ago
fix: wait for persist state listeners to run when event manager closes (#1481)
aa550ed Vlad Frangu committed 4 years ago
docs: say "a X" instead of "the X" in README.md (#1480)
a02db88 Sebastian Carlos committed 4 years ago
chore: bump the patch version when computing canary versions automatically
84aa0c5 Martin Adámek committed 4 years ago
build: trigger docker images build from the CI publish step
f9b25d0 Martin Adámek committed 4 years ago
docs: update 3.0 API docs snapshot + fix type for `Dataset.getData`
d2421e3 Martin Adámek committed 4 years ago