Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

fix: using GET for candidate checks - sitemaps (#3464)
nikitachapovskii-dev committed 8 months ago
f82eb04
e976265
chore(deps): bump minimatch from 9.0.5 to 9.0.7 (#3468)
dependabot[bot] committed 8 months ago
1ee8b3b
5d14a96
docs: remove $1M challenge banner (#3455)
Copilot committed 8 months ago
86356c5
chore(deps): batch dependency version bumps (#3451)
Jindřich Bär committed 8 months ago
47f0889
chore(deps): batch merge dependabot version bumps (#3449)
Jindřich Bär committed 8 months ago
adfa13b
chore: use lint-staged in pre-commit hook (#3442)
Richard Solar committed 8 months ago
4ad9948
c3a4b3b
perf: add lazyIterablePromise to kvs iterators (#3413)
Richard Solar committed 9 months ago
2658610
070c258
136d368
98646be
bfb861c
docs: add Crawlee v3.16 release blog post (#3390)
Martin Adámek committed 9 months ago
9c046e9
docs: update docs for minor version
Apify Release Bot committed 9 months ago
0ddf67f
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 9 months ago
e645174
chore(release): v3.16.0
Apify Release Bot committed 9 months ago
4d5fb74
feat: implements async iterators (#3352)
Richard Solar committed 9 months ago
7f7a4ab
chore: replace deprecated stylistic plugin (#3383)
Richard Solar committed 9 months ago
7a6b8da
chore: update .gitignore (#3382)
Richard Solar committed 9 months ago
95c4af4