Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #539 from apifytech/apienx-feature/AP-1523-fix-most-sdk-links
9ddf65a petrpatek committed 6 years ago
Merge branch 'feature/AP-1523-fix-most-sdk-links' of https://github.com/apienx/apify-js into apienx-feature/AP-1523-fix-most-sdk-links
c677596 Petr Patek committed 6 years ago
Merge pull request #523 from apifytech/feature/petr-session-pool-puppeteer
81996cc Ondra Urban committed 6 years ago
Merge branch 'master' into feature/petr-session-pool-puppeteer
3e1908b Petr Patek committed 6 years ago
Merge pull request #519 from apifytech/feature/cheerio-ignore-ssl-by-default
92bd840 Ondra Urban committed 6 years ago
Merge branch 'master' into feature/cheerio-ignore-ssl-by-default
6a1a63e Petr Patek committed 6 years ago
Merge pull request #533 from apifytech/fix/local-storage-dir-emulator
9b57788 Ondra Urban committed 6 years ago
Merge branch 'fix/local-storage-dir-emulator' into feature/petr-session-pool-puppeteer
b084243 Petr Patek committed 6 years ago
Local storage dir emulator introduced to tests, removed settings_rotator.js and settings_rotator.test.js. Refactored code.
abc5eff Petr Patek committed 6 years ago