Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
chore(deps): bump launch-editor from 2.12.0 to 2.14.1 in /website (#3752)
fd43605 dependabot[bot] committed 5 months ago
chore(deps): bump @babel/core from 7.28.6 to 7.29.7 in /website (#3751)
cb5ed47 dependabot[bot] committed 5 months ago
chore(deps): bump markdown-it from 14.1.1 to 14.2.0 in /website (#3743)
468923d dependabot[bot] committed 5 months ago
chore(deps): bump dompurify from 3.4.3 to 3.4.10 in /website (#3744)
53fc945 dependabot[bot] committed 5 months ago
chore(deps): bump joi from 17.13.3 to 17.13.4 in /website (#3738)
df16b4d dependabot[bot] committed 5 months ago
chore(deps): bump shell-quote from 1.8.3 to 1.8.4 in /website (#3731)
46c0862 dependabot[bot] committed 5 months ago
fix(playwright): silence transient errors in `enqueueLinksByClickingElements` (#3732)
7b98cc3 Gil Cohen committed 5 months ago
chore(deps): update dependency camoufox-js to ^0.11.0 (#3720)
e1bcbb3 renovate[bot] committed 5 months ago
fix: update handleCloudflareChallenge for new Cloudflare challenge markup (#3717)
5baa8bd Martin Adámek committed 5 months ago
chore(deps): bump hono from 4.12.18 to 4.12.23 (#3718)
31512a3 dependabot[bot] committed 5 months ago
fix(memory-storage): prevent storage names from escaping the storage directory (#3715)
a04c297 Martin Adámek committed 5 months ago
chore(release): update internal dependencies [skip ci]
8f2663a Apify Release Bot committed 5 months ago
chore: drop Node 18 tests in CI release workflow (#3711)
6b1659b Jindřich Bär committed 5 months ago
fix: BrowserCrawler closes ctx.page before errorHandler runs on navig… (#3655)
ab34886 moaedy committed 5 months ago
chore(deps): update dependency vitest to v4.1.0 [security] (#3705)
158e989 renovate[bot] committed 5 months ago
chore: bump impit to 0.14.1 (drops broken preinstall) (#3695)
184981a Martin Adámek committed 5 months ago
chore(deps): update actions/github-script action to v9 (#3681)
9085843 renovate[bot] committed 5 months ago
chore(deps): update dependency lint-staged to v17 (#3693)
3492c4d renovate[bot] committed 5 months ago
chore(deps): update dependency camoufox-js to ^0.10.0 (#3680)
45159f1 renovate[bot] committed 5 months ago
chore(deps): update actions/upload-pages-artifact action to v5 (#3683)
ceb68de renovate[bot] committed 5 months ago
chore: cache Playwright browsers and add install timeout in CI (#3691)
b4ee1de Martin Adámek committed 5 months ago
chore: stop publishing broken declaration and source maps (#3689)
bb9a386 Martin Adámek committed 5 months ago