Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Typescript declarations automatically generated from JSDoc comments (#558)
1b78219 Matej Gagyi committed 6 years ago
fix parameter type on utils.puppeteer.blockRequests (#568)
002c77e Paulo Cesar committed 6 years ago
Merge pull request #553 from apifytech/docs/session-managment-page
6255366 Ondra Urban committed 6 years ago
Added copy buttons to code blocks - new files code-block-buttons.js and code-block-buttons.css
9e4b5e2 Jan Bárta committed 6 years ago
Merge pull request #555 from apifytech/refactoring/session-pool
342c727 Ondra Urban committed 6 years ago
Renamed the checkStatus function to retireOnStatusCode and allow of custom status codes.
384b04c Petr Patek committed 6 years ago
Merge pull request #556 from yin/fix/update-dataset-getdata-docs
311d242 Ondra Urban committed 6 years ago
[docs] Updated Dataset.getData() JSDoc to include the `count` property in the return value
53a8f1f Matej 'Yin' Gagyi committed 6 years ago