Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Build docs
Ondra Urban committed 6 years ago
827d29d
eea53e0
Bump version, update changelog
Ondra Urban committed 6 years ago
4fc7077
Fix/save snapshot file format (#651)
Jan Bárta committed 6 years ago
91e0d3e
cddde48
Fix wrong use of htmlparser
Ondra Urban committed 6 years ago
091fcb1
Docs/examples integration (#640)
Jan Bárta committed 6 years ago
d18c6fe
Fix useless encoding in README [skip ci]
Ondra Urban committed 6 years ago
a181afa
Fix docs - waitSecs - string => number (#644)
Lukáš Křivka committed 6 years ago
19613d4
Bump @types/tough-cookie from 2.3.7 to 4.0.0 (#637)
dependabot-preview[bot] committed 6 years ago
8759e86
Bump @types/sinon from 7.5.2 to 9.0.0 (#638)
dependabot-preview[bot] committed 6 years ago
0ce4f66
Bump prettier from 1.19.1 to 2.0.1 (#633)
dependabot-preview[bot] committed 6 years ago
75eb82e
Bump fs-extra from 8.1.0 to 9.0.0 (#631)
dependabot-preview[bot] committed 6 years ago
a4f171d
Bump tough-cookie from 3.0.1 to 4.0.0 (#630)
dependabot-preview[bot] committed 6 years ago
3c16e24
Update changelog [skip ci]
Ondra Urban committed 6 years ago
bbeeb26
84c17ac
c3bd484
Update packages
Ondra Urban committed 6 years ago
4ed09d7
Fix one more flaky test
Ondra Urban committed 6 years ago
35e760c
Fix lint
Ondra Urban committed 6 years ago
9fdf88d
Fix failing CheerioCrawler test
Ondra Urban committed 6 years ago
d58015b
Bump version
Ondra Urban committed 6 years ago
e50507d
Move returns (#628)
Ondra Urban committed 6 years ago
ed4ca99
Docs: fix various links (#625)
František Nesveda committed 6 years ago
1cc63cd
[DOCS] Added error warning to docs. (#626)
petrpatek committed 6 years ago
f0559a3
Improved ApifyEnv docs
Jan Curn committed 6 years ago
99b975e
Improved Apify.addWebhook() docs
Jan Curn committed 6 years ago
986f4d9
Trigger beta release on Travis
Ondra Urban committed 6 years ago
ac53642
Fix/session cookie date calculation (#620)
Jan Bárta committed 6 years ago
423a60f