Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
feat(playwright): add `handleCloudflareChallenge` helper (#2865)
9a1725f Martin Adámek committed 1 year ago
feat: improved cross platform metric collection (#2834)
e41b2f7 NathanSavageKaimai committed 1 year ago
fix: install browser in the `camoufox` template correctly (#2864)
a9d008c Jindřich Bär committed 1 year ago
chore: add `postinstall` script to `camoufox` template (#2863)
3424d34 Jindřich Bär committed 1 year ago
fix: Make log message in RequestQueue.isFinished more accurate (#2848)
3d124ae Jan Buchar committed 1 year ago
fix(cheerio): don't decode HTML entities in `context.body` (#2838)
32d6d0e Jindřich Bär committed 1 year ago
chore(deps): update dependency vitest to v2.1.9 [security]
9faca42 renovate[bot] committed 1 year ago