Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: disable auto-update on docusaurus
Martin Adámek committed 4 years ago
141c48e
docs: keep docusaurus beta 21 as beta 22 is broken
Martin Adámek committed 4 years ago
3df0982
chore(deps): update patch/minor dependencies
renovate[bot] committed 4 years ago
e310c47
chore: bump version
Martin Adámek committed 4 years ago
dd980e2
chore: use stable crawlee in templates
Martin Adámek committed 4 years ago
e40e78d
docs: update changelog
Martin Adámek committed 4 years ago
f82bca8
chore: change github links to apify/crawlee
Martin Adámek committed 4 years ago
c49f88a
feat: initial crawlee commit (#1384)
Martin Adámek committed 4 years ago
3a59bfc
docs: Fix link to datasetClient.get in docs (#1361)
Matěj Volf committed 4 years ago
6e64ceb
chore: force install deps to allow testing on node 15
Martin Adámek committed 4 years ago
53acb66
test: fix failing tests (#1366)
Andrey Bykov committed 4 years ago
88aa849
docs: fix link to hiring (#1358)
Lukáš Křivka committed 4 years ago
4ffccae
55e0cce
3123e3a
chore(deps): update dependency sinon to v14 (#1353)
renovate[bot] committed 4 years ago
1d99dcf
chore: add note about hiring to readme
Martin Adámek committed 4 years ago
af58bd7
chore: bump version
Martin Adámek committed 4 years ago
3f6877b
d2d5ac7
chore: bump version
Martin Adámek committed 4 years ago
b4676b9
chore: update changelog
Martin Adámek committed 4 years ago
ca3bd71
chore: fix `RequestTransform` return type
Martin Adámek committed 4 years ago
fb14247
898cc43