Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge pull request #200 from apifytech/feature/request_queue_fix
8b7fec7 Marek Trunkát committed 8 years ago
Merge branch 'master' of github.com:Apifier/apify-runtime-js
85977b7 Marek Trunkat committed 8 years ago
Merge branch 'master' of github.com:Apifier/apify-runtime-js into develop
f4e2115 Marek Trunkat committed 8 years ago
Merge pull request #197 from miqh/feat/add-request-overload
cf6cd3e Marek Trunkát committed 8 years ago
Bugfix in PuppeteerPool: Pages were sometimes considered closed even though they weren't
21d292f Jan Curn committed 8 years ago
Fixing bug where duplicate already handled requests added with forefront were returned in fetchNextRequest()
3ec109e Marek Trunkat committed 8 years ago
Merge pull request #199 from apifytech/feature/cache-responses
60564cc Ondra Urban committed 8 years ago
Modify request adding parameter validation for RequestQueue
6c0fc28 Michael Huynh committed 8 years ago
Merge pull request #194 from apifytech/feature/task_to_job
655b637 Ondra Urban committed 8 years ago
Merget branch 'master' of github.com:Apifier/apify-runtime-js into develop
d954fc4 Marek Trunkat committed 8 years ago