Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(deps): update dependency vite to v5.4.21 [security]
51f9700 renovate[bot] committed 1 year ago
feat: export cheerio types in all crawler packages (#3204)
f05790b Matouš Mařík committed 1 year ago
fix: Re-export MemoryStorage from the crawlee package (#3200)
e183ea9 Jan Buchar committed 1 year ago
chore(deps): update actions/setup-node action to v6 (#3195)
369b857 renovate[bot] committed 1 year ago
chore(deps): update dependency eslint-plugin-react-hooks to v7 (#3193)
fb5b774 renovate[bot] committed 1 year ago
docs(basic-crawler): document .teardown and link with .stop (#3192)
df79570 Lukáš Křivka committed 1 year ago
fix: correctly apply `launchOptions` with `useIncognitoPages` (#3181)
84a4b70 Jindřich Bär committed 1 year ago
chore(deps): update dependency eslint-plugin-react-hooks to v6 (#3185)
e1cb54c renovate[bot] committed 1 year ago
chore(deps): update peter-evans/repository-dispatch action to v4 (#3186)
920ee4e renovate[bot] committed 1 year ago
chore(deps): update dependency puppeteer to v24.23.0 (#3174)
103229f renovate[bot] committed 1 year ago
test: Remove --no-prompt flag from Apify CLI push command in E2E tests (#3183)
8c8e1f4 Copilot committed 1 year ago