Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

fix: flaky Snapshotter test
Ondra Urban committed 5 years ago
660a779
chore: update apify-shared packages (#1051)
Martin Adámek committed 5 years ago
a8b1fbc
chore: install dependencies in the lint step
Martin Adámek committed 5 years ago
a1638b0
d44f3b5
docs: Add link to http.IncomingMessage to JS Doc (#1037)
Lukáš Křivka committed 5 years ago
2c7ede6
docs: fix copywriting (#1035)
davidjohnbarton committed 5 years ago
cee6b54
3cedd4d
chore: build docs
Ondra Urban committed 5 years ago
e9c27a9
chore: update changelog
Ondra Urban committed 5 years ago
7d9e419
docs: update docker image guide
Ondra Urban committed 5 years ago
04b801d
chore: update `@apify/ps-tree` to 1.1.4 (#1017)
Jan Čurn committed 5 years ago
935d225
chore: bump storage-local, node 16 (#1012)
petrpatek committed 5 years ago
878c3ce
ab872c3
Bumped got-scraping version.
Petr Patek committed 5 years ago
0732ab5
chore: add perf logging to autoscaledpool (#1001)
Ondra Urban committed 5 years ago
800ea9b
9357205
Feature: got scraping (#980)
petrpatek committed 5 years ago
3a0aa47
chore: bump version, build docs
Ondra Urban committed 5 years ago
7f01029
058af35
chore: update @apify/storage-local to v2 (#988)
Ondra Urban committed 5 years ago
cbb7d89
chore: deprecate utils.waitForRunToFinish
Ondra Urban committed 5 years ago
ff0ff94
Fix waitForFinish parameter (#979)
Lukáš Křivka committed 5 years ago
8607d1f
docs: fix extra property [skip ci]
Ondra Urban committed 5 years ago
90b1881
docs: fix missing await [skip ci]
Ondra Urban committed 5 years ago
9f94909