Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Merge branch 'master' of https://github.com/apifytech/apify-js into extend-examples
eac1680 Tobey committed 7 years ago
RequestList: Persist downloaded requests from requestsFromUrl
9ad6909 drobnikj committed 7 years ago
Revert "RequestList: Persist downloaded requests from requestsFromUrl"
d88d7ae drobnikj committed 7 years ago
RequestList: Persist downloaded requests from requestsFromUrl
55dda96 drobnikj committed 7 years ago
Merge branch 'master' into feature/session-pool-basic-crawler
8f736be Petr Patek committed 7 years ago
Merge pull request #469 from apifytech/fix/464-requestTimeoutSecs
8dcc232 Marek Trunkát committed 7 years ago
Merge pull request #433 from apifytech/chore/[email protected]
d5aad5b Marek Trunkát committed 7 years ago
Using input param in callTask() instead of depricated body and contentType
e099cd8 Marek Trunkat committed 7 years ago
Merge branch 'fix/464-requestTimeoutSecs' of github.com:apifytech/apify-js into fix/464-requestTimeoutSecs
9a23b42 Petr Patek committed 7 years ago