Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
docs: Add How to build a price tracker with Crawlee and Apify blog post (#2885)
120d79c Percival Villalva committed 1 year ago
chore(release): update internal dependencies [skip ci]
8b882b3 Apify Release Bot committed 1 year ago
chore(deps): bump estree-util-value-to-estree from 3.3.2 to 3.3.3 in /website (#2918)
dc9060a dependabot[bot] committed 1 year ago
chore(release): update internal dependencies [skip ci]
99af95e Apify Release Bot committed 1 year ago
chore(deps): bump image-size from 1.2.0 to 1.2.1 in /website (#2911)
456397e dependabot[bot] committed 1 year ago
fix: don't double increment session usage count in `BrowserCrawler` (#2908)
3107e55 Jindřich Bär committed 1 year ago
fix: treat `406` as other `4xx` status codes in `HttpCrawler` (#2907)
b0e6f6d Jindřich Bär committed 1 year ago
chore(e2e): fix tests after the documentation redesign (#2905)
6b7bb80 Jindřich Bär committed 1 year ago
chore(deps): update dependency camoufox-js to ^0.3.0 (#2895)
8ceadc7 renovate[bot] committed 1 year ago