Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

feat: add `Dataset.getData()` shortcut
Martin Adámek committed 4 years ago
522ed6e
chore(deps): update dependency turbo to v1.4.6
renovate[bot] committed 4 years ago
c01b564
chore: Fix/typo & rephrasing of issue templates (#1530)
Lukáš Křivka committed 4 years ago
ca8f4bd
chore(deps): update dependency playwright to v1.25.2
renovate[bot] committed 4 years ago
714e078
fix: correct some typing exports (#1527)
Vlad Frangu committed 4 years ago
4a136e5
docs: fix `skip-navigation` example (#1529)
Szymon Marczak committed 4 years ago
3fc1951
fix: handle redirect cookies (#1521)
Szymon Marczak committed 4 years ago
2f7fc7c
feat: error tracker (#1467)
Szymon Marczak committed 4 years ago
6bfe1ce
d351b05
8d07108
chore(deps): update patch/minor dependencies
renovate[bot] committed 4 years ago
9918690
chore: fix experimental containers firefox test (#1522)
Szymon Marczak committed 4 years ago
f64f1e2
feat: bump puppeteer support to 17.x (#1519)
Vlad Frangu committed 4 years ago
b97a852
1b4d54c
chore(deps): update dependency turbo to v1.4.5
renovate[bot] committed 4 years ago
20d3e9b
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
36d07b5
c70fdd6
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
77d27e0
chore(deps): update npm to v8.19.1
renovate[bot] committed 4 years ago
5c3a704
chore(deps): update dependency turbo to v1.4.4
renovate[bot] committed 4 years ago
a5eb6f2
chore(deps): update npm to v8.19.0
renovate[bot] committed 4 years ago
381d78b
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
d356a1e
fix: support TypeScript 4.8 (#1507)
Vlad Frangu committed 4 years ago
4c3a504
641da35
docs: remove unused imports
Martin Adámek committed 4 years ago
df4cdb9