Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update readme
Ondra Urban committed 5 years ago
362f3df
Created browser launchers (#900)
petrpatek committed 5 years ago
b313fca
Add documentation versioning, clean up bad docs
Ondra Urban committed 5 years ago
de7ab1a
Bump ow to 0.21 because 0.22 is slow
Ondra Urban committed 5 years ago
11ff217
Bump apify-client
Ondra Urban committed 5 years ago
cbce7fc
Change session cookie setting error log to debug
Ondra Urban committed 5 years ago
5f04bc2
Apply 0.22.4 hotfix to master (#897)
Ondra Urban committed 5 years ago
d87e303
Improve doc comments [skip ci]
Ondra Urban committed 5 years ago
7b7118a
Fix lint
Ondra Urban committed 5 years ago
4425441
Remove deprecated code, update docs
Ondra Urban committed 5 years ago
7ac5dea
Add logging to capture weird behavior of Statistics
Ondra Urban committed 5 years ago
9942195
Refactoring playwright crawler (#894)
petrpatek committed 5 years ago
9b92313
c197fbe
Refactored puppeteer crawler (#895)
petrpatek committed 5 years ago
cbf83c4
83aad35
Goto function deprecation (#891)
petrpatek committed 5 years ago
dcce279
Fix typo in release workflow
Ondra Urban committed 5 years ago
27b7d47
Add playwright to release workflow
Ondra Urban committed 5 years ago
f85c344
Update migration guide
Ondra Urban committed 5 years ago
2a8878d
Browser crawler (#858)
petrpatek committed 5 years ago
32bf3ed
a0deb03
Fix flaky test
Ondra Urban committed 5 years ago
087f7dc
Improved ProxyConfiguration.newUrl() docs (#887)
Jan Čurn committed 5 years ago
57b1b9a
Bump version, update changelog
Ondra Urban committed 5 years ago
fec1fe0
Pin cheerio, update changelog
Ondra Urban committed 5 years ago
6782d5a
Bump htmlparser2 from 5.0.1 to 6.0.0 (#872)
dependabot-preview[bot] committed 5 years ago
82020fa
d6935c0
add requestAsBrowser example, fix typos (#877)
Andrey Bykov committed 5 years ago
050a150
Bump apify-client from 1.0.0 to 1.0.1 (#879)
dependabot-preview[bot] committed 5 years ago
8f0e8d0