Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(release): update internal dependencies [skip ci]
a93da6c Apify Release Bot committed 2 years ago
fix: ES2022 build compatibility and move to NodeNext for module (#2258)
7fe1e68 Vlad Frangu committed 2 years ago
chore(release): update internal dependencies [skip ci]
22fb053 Apify Release Bot committed 2 years ago
fix: `retryOnBlocked` doesn't override the blocked HTTP codes (#2243)
81672c3 Jindřich Bär committed 2 years ago
fix: make CLI work on Windows too with `--no-purge` (#2244)
83f3179 Vlad Frangu committed 2 years ago
docs(browser-pool): explain retire session in retireBrowserAfterPageC… (#2246)
2a5b94b Lukáš Křivka committed 2 years ago