Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: try to increase timeout for launching playwright
Martin Adámek committed 4 years ago
35257c1
fix: revert to puppeteer 10.x (#1276)
Martin Adámek committed 4 years ago
b5e6220
chore: upgrade `got-scraping to `3.2.7`
Szymon Marczak committed 4 years ago
f8862e5
6f951f1
ed9c80b
docs: fix link on docker-images page
Martin Adámek committed 4 years ago
54403ea
docs: add 2.1 and 2.2 versions
Martin Adámek committed 4 years ago
a8578f6
docs: remove `launch-puppeteer` from v1+ docs
Martin Adámek committed 4 years ago
228bcf6
chore: improve comment
Szymon Marczak committed 4 years ago
dade38c
chore: bump to 2.2.1 for dev builds
Martin Adámek committed 4 years ago
59810ca
chore: update changelog and bump to 2.2
Martin Adámek committed 4 years ago
83530c4
f59d95b
chore: try to change order of puppeteer proxy tests
Martin Adámek committed 4 years ago
0e4e4ea
chore: update changelog before release
Martin Adámek committed 4 years ago
fd4dafb
050aaa3
fix: proxy per page broke session fetching (#1256)
Szymon Marczak committed 4 years ago
b0a5bc0
chore(deps): update dependency typescript to v4.5.4
Renovate Bot committed 4 years ago
f704328
chore: update changelog
Martin Adámek committed 4 years ago
da78296
chore(deps): update dependency typescript to v4.5.4
Renovate Bot committed 4 years ago
5374098
chore(deps): update dependency typescript to v4.5.3
Renovate Bot committed 4 years ago
2f12414
chore: try to increase timeout for launching puppeteer
Martin Adámek committed 4 years ago
cff551c
chore: use stable version of browser-pool
Martin Adámek committed 4 years ago
8236f19
chore: adjust internal timeouts (#1251)
Martin Adámek committed 4 years ago
ffce817
chore: fix timeout error messages having wrong timeouts
Martin Adámek committed 4 years ago
d3f65ff
chore: update changelog
Martin Adámek committed 4 years ago
12a7355
a271862
chore(deps): update patch/minor dependencies
Renovate Bot committed 4 years ago
4542541