Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Improve docs and migration guide [skip ci]
Ondra Urban committed 6 years ago
453bd00
Bump @types/jest from 25.2.3 to 26.0.0 (#725)
dependabot-preview[bot] committed 6 years ago
851cec4
Bump iconv-lite from 0.5.2 to 0.6.0 (#722)
dependabot-preview[bot] committed 6 years ago
2bcec5c
Improve Apify docs with authentication [skip ci]
Ondra Urban committed 6 years ago
c4b2c5c
Improve proxy conf docs [skip ci]
Ondra Urban committed 6 years ago
ecb989f
Add sessionId length validation (#720)
Ondra Urban committed 6 years ago
f0d1b01
Bump version, update changelog
Ondra Urban committed 6 years ago
73011dd
Fix and improve proxyConf, update changelog
Ondra Urban committed 6 years ago
063774b
Improve Request property defaults (#719)
Ondra Urban committed 6 years ago
cb0bbf4
Upgrade apify-shared
Ondra Urban committed 6 years ago
3996577
Add proxy guide and fix other stuff (#717)
Ondra Urban committed 6 years ago
ef62220
Bump puppeteer from 3.1.0 to 3.3.0 (#715)
dependabot-preview[bot] committed 6 years ago
3f37fc2
Update gotoFunction docs [skip ci]
Ondra Urban committed 6 years ago
2826c7e
Revert Puppeteer to 3.1
Ondra Urban committed 6 years ago
1d9150a
Feature/proxy configuration custom urls (#706)
Jan Bárta committed 6 years ago
02b8e9e
Bump puppeteer from 3.1.0 to 3.2.0 (#711)
dependabot-preview[bot] committed 6 years ago
15b23a4
dc1f39c
Be more explicit about state in RequestList docs (#708)
Lukáš Křivka committed 6 years ago
15b0c95
Feature/update infinite scroll (#705)
Lukáš Křivka committed 6 years ago
5c009cf
Add option to disable outdated warning (#703)
Ondra Urban committed 6 years ago
62013d3
Bump apify-shared from 0.2.17 to 0.3.0 (#700)
dependabot-preview[bot] committed 6 years ago
b2ebb36
d61204a
Fix lint
Ondra Urban committed 6 years ago
8f1f190
Fix requestAsBrowser max redirects error
Ondra Urban committed 6 years ago
74bcb40
f4b4f35
Improve variable names
Ondra Urban committed 6 years ago
dc14347
Update changelog [skip ci]
Ondra Urban committed 6 years ago
49fe0cb
Fix requests hanging with interception enabled
Ondra Urban committed 6 years ago
73b8654
Create issue template (#685)
Ondra Urban committed 6 years ago
5ff01dc