Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: revert TS to 4.4
Martin Adámek committed 4 years ago
351cc6b
chore(deps): update patch/minor dependencies
Renovate Bot committed 4 years ago
27b5d10
refactor: make the last resort timeout much larger
Martin Adámek committed 4 years ago
9f46835
fix: implement abortable timeouts (#1245)
Martin Adámek committed 4 years ago
6488b94
chore: lock TS to 4.4 to fix build issues
Martin Adámek committed 4 years ago
9727d79
feat: add fingerprinting support (#1243)
petrpatek committed 4 years ago
bd596de
feat: proxy per page (#1228)
Szymon Marczak committed 5 years ago
2d2118e
chore(deps): update dependency typescript to v4.5.2
Renovate Bot committed 5 years ago
e8fdd69
fix: add timeout for task function (#1234)
Szymon Marczak committed 5 years ago
33df439
chore(deps): upgrade @apify/eslint
Martin Adámek committed 5 years ago
d71e1bf
chore(deps): upgrade eslint to v8
Martin Adámek committed 5 years ago
0e05196
chore(deps): upgrade puppeteer to v11 (#1235)
Martin Adámek committed 5 years ago
cc06bdf
591bcdf
chore(deps): update dependency sinon to v12 (#1227)
renovate[bot] committed 5 years ago
e2dbd32
chore: trigger build
Martin Adámek committed 5 years ago
77626db
chore: trigger build
Martin Adámek committed 5 years ago
88301ce
chore: change `apifytech` to `apify` in changelog
Martin Adámek committed 5 years ago
5ca0747
docs: rename Apify app to Apify Console in docs (#1226)
Marek Trunkát committed 5 years ago
a279230
chore(deps): update dependency playwright to v1.16.3
Renovate Bot committed 5 years ago
a8c4c5d
chore(deps): update dependency playwright to v1.16.2
Renovate Bot committed 5 years ago
0bcb0c5
chore(deps): update dependency playwright to v1.16.1
Renovate Bot committed 5 years ago
9fac55a
0b2fd6e
chore(deps): update dependency playwright to v1.16.0
Renovate Bot committed 5 years ago
766fa76
fix: handle extra cases for diffCookie (#1217)
Vlad Frangu committed 5 years ago
32776bc
chore: upgrade `browser-pool` (#1210)
Szymon Marczak committed 5 years ago
b16e0e0
chore(deps): update dependency typescript to v4.4.4
Renovate Bot committed 5 years ago
37e38d5