Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: switch to yarn v3 (#1752)
Martin Adámek committed 3 years ago
a95e667
chore: update dependencies 3
Martin Adámek committed 3 years ago
1f064da
chore: update dependencies 2
Martin Adámek committed 3 years ago
22b5557
chore: update dependencies
Martin Adámek committed 3 years ago
94dc011
2f682f7
test: fix linter
Martin Adámek committed 3 years ago
432904a
test: fix one flaky assertion
Martin Adámek committed 3 years ago
0ed9ee8
chore: update typescript to 4.9
Martin Adámek committed 3 years ago
35664fe
fix(utils): add missing dependency on `ow`
Martin Adámek committed 3 years ago
bf0e03c
chore(deps): update dependency fs-extra to v11 (#1694)
renovate[bot] committed 3 years ago
726d379
chore(deps): update dependency jsdom to v21 (#1733)
renovate[bot] committed 3 years ago
085609f
chore: improve log message (#1727)
Ondra Urban committed 3 years ago
d09aefa
ci: always force publishing of all packages
Martin Adámek committed 3 years ago
6781f12
chore(release): v3.1.4
Apify Release Bot committed 3 years ago
4d36bb7
test: e2e tests fix, better logging (#1713)
Jindřich Bär committed 3 years ago
fd74447
fix: session.markBad() on requestHandler error (#1709)
Jindřich Bär committed 3 years ago
e87eb1f
test: cheerio-default e2e test (#1708)
Jindřich Bär committed 3 years ago
f731144
chore(release): v3.1.3
Apify Release Bot committed 3 years ago
5ff04fa
feat: hideInternalConsole in JSDOMCrawler (#1707)
Jindřich Bär committed 3 years ago
8975f90
11b9565
34aa158
49bba25
docs: fix JSDOM examples (#1693)
Jindřich Bär committed 3 years ago
5ec089d