Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: update biome
Martin Adámek committed 2 years ago
02665b0
test: add e2e test for zero concurrency with RQ v2
Martin Adámek committed 2 years ago
1bb7712
chore: run biome as a pre-commit hook (#2493)
Martin Adámek committed 2 years ago
ebe9a8d
chore: fix formatting
Martin Adámek committed 2 years ago
eed657a
chore: remove unused eslint ignore directives
Martin Adámek committed 2 years ago
5e3f341
fa72b4f
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
15fa126
chore: use biome for code formatting (#2301)
Jan Buchar committed 2 years ago
5db533a
docs: add feedback from community. (#2478)
Saurav Jain committed 2 years ago
43381bf
docs: update keywords (#2481)
Saurav Jain committed 2 years ago
4d2d9c3
fix: provide URLs to the error snapshot (#2482)
Martin Adámek committed 2 years ago
7f64145
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
b23858a
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
f0e0de5
docs: improve crawlee seo ranking (#2472)
Saurav Jain committed 2 years ago
f97e125
chore(deps): lock file maintenance
renovate[bot] committed 2 years ago
e6bdff4
docs: fix broken link for a moved content
Martin Adámek committed 2 years ago
fbc9642
docs: add v3.10 snapshot
Martin Adámek committed 2 years ago
26f9226
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
d170bce
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 2 years ago
805fc33
chore(release): v3.10.0
Apify Release Bot committed 2 years ago
5a888d7
feat: add `FileDownload` "crawler" (#2435)
Jindřich Bär committed 2 years ago
d73756b
test: fix e2e tests for error snapshotter
Martin Adámek committed 2 years ago
99e1a94
chore(deps): update dependency puppeteer to v22.9.0
renovate[bot] committed 2 years ago
0a3c518
docs: fixes (#2469)
Saurav Jain committed 2 years ago
3df9236
chore(docker): update docker state [skip ci]
Apify Release Bot committed 2 years ago
f3888b0