Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
Redefine publicUtils exports; fix JSDoc comments to improve Docs
63032d7 Ondra Urban committed 8 years ago
Document regexes, add to utils namespace; Fix error handling
7a7da4f Ondra Urban committed 8 years ago
Merge pull request #135 from apifytech/feature/live-preview-client
9262de1 Marek Trunkát committed 8 years ago
Merge pull request #131 from apifytech/feature/remove-page-close-timeout
dfb42ce Marek Trunkát committed 8 years ago
"Add downloadListOfUrls and extractUrls functions to utils"
cd797ad Ondra Urban committed 8 years ago