Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

c1fc439
docs: change date of latests blogpost (#2919)
Saurav Jain committed 1 year ago
f56c268
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 1 year ago
8b882b3
chore(release): v3.13.2
Apify Release Bot committed 1 year ago
e293d1c
feat: add `onSkippedRequest` option (#2916)
Martin Adámek committed 1 year ago
764f992
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 1 year ago
99af95e
chore(release): v3.13.1
Apify Release Bot committed 1 year ago
a08646a
fix: rename `RobotsFile` to `RobotsTxtFile` (#2913)
Martin Adámek committed 1 year ago
3160f71
0eabed1
90cd306
6b7bb80
chore(deps): update yarn to v4.8.1
renovate[bot] committed 1 year ago
aca83eb
c3efd07
chore: update to eslint v9 (#2830)
Martin Adámek committed 1 year ago
07cd2c4
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
ff9f256
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
325037a
8ceadc7
docs: homepage fix python command (#2894)
Daniel Wébr committed 1 year ago
572cfef
docs: website design uplift (#2872)
Daniel Wébr committed 1 year ago
2e58992
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
3c2a38a
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
6b30072
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
a77ab9f
docs: blog How to scrape Bluesky with Python (#2784)
Max Bohomolov committed 1 year ago
f7f9728
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
0232ae7
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
e300405