Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

bbe9045
b3d3c6a
docs: update changelogs
Martin Adámek committed 4 years ago
247c570
chore(release): v3.1.2
Apify Release Bot committed 4 years ago
31913b2
chore: use `userData` for the `Request.state`
Martin Adámek committed 4 years ago
97989df
e7693dc
chore: use `core` package for the version inference
Martin Adámek committed 4 years ago
585a878
chore: update lerna to v6
Martin Adámek committed 4 years ago
47c7a1c
docs: fix RetryRequestError() description (#1676)
Jindřich Bär committed 4 years ago
770b127
chore: update default user agent to chrome 107
Martin Adámek committed 4 years ago
216a1dd
feat(jsdom-crawler): add runScripts option (#1668)
Szymon Marczak committed 4 years ago
8ef90bc
docs: mention `pageOptions` limitation (#1674)
Szymon Marczak committed 4 years ago
0bc147e
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
ca1a098
fix: requestHandlerTimeout timing (#1660)
Jindřich Bär committed 4 years ago
493ea0c
chore(deps): update npm to v9 (#1664)
renovate[bot] committed 4 years ago
49e270c
493a7cf
test: make enqueue links tests more stable
Martin Adámek committed 4 years ago
55182ce
test: fix playwright-enqueue-links e2e test
Martin Adámek committed 4 years ago
b43165c
chore: update lerna to v6
Martin Adámek committed 4 years ago
6db7093
9d3a997
ea2e61b
e6aceb8
docs: update changelog
Martin Adámek committed 4 years ago
ba5f64c
chore(release): v3.1.1
Apify Release Bot committed 4 years ago
f9a11e0