Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

8a11ffb
e51b185
chore(deps): update patch/minor dependencies to v6.2.1
renovate[bot] committed 3 years ago
fca7e30
51f8156
docs: add v3.5 docs snapshot
Martin Adámek committed 3 years ago
819b017
chore(release): v3.5.0
Apify Release Bot committed 3 years ago
58cc3d4
feat: add support for `sameDomainDelay` (#2003)
Dinesh Hardasani committed 3 years ago
e796883
c4fb5e2
feat: retire session on proxy error (#2002)
Jindřich Bär committed 3 years ago
8c0928b
chore(deps): update dependency turbo to v1.10.12
renovate[bot] committed 3 years ago
6c79dd0
chore(deps): update dependency playwright to v1.36.2
renovate[bot] committed 3 years ago
5b00ebe
chore(deps): update patch/minor dependencies
renovate[bot] committed 3 years ago
de4b433
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
848c3fc
d0810d3
chore(deps): update dependency puppeteer to v20.9.0
renovate[bot] committed 3 years ago
f193e11
docs: update logger to have new jsdoc example
Martin Adámek committed 3 years ago
4cc77da
chore: fix lint
Martin Adámek committed 3 years ago
782d5eb
chore: fix tests that were mocking RQ improperly 2
Martin Adámek committed 3 years ago
554e51a
chore: fix tests that were mocking RQ improperly
Martin Adámek committed 3 years ago
73023d1
chore(deps): update dependency turbo to v1.10.9
renovate[bot] committed 3 years ago
6c753dd
chore: enforce import order via eslint (#1998)
Martin Adámek committed 3 years ago
15c30a3
37145fa