Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

feat(core): add `crawler.exportData()` helper (#2166)
Martin Adámek committed 3 years ago
c8c09a5
1f83df5
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
6cfe325
docs: add more links to homepage and improve example
Martin Adámek committed 3 years ago
ab2db91
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
0cea110
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
1f86b8c
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
8c77d59
chore(deps): update dependency puppeteer to v21.5.0
renovate[bot] committed 3 years ago
a11dfb6
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
2db3975
docs: update docusaurus to v3 (#2163)
Martin Adámek committed 3 years ago
9aa8593
430f025
chore: update puppeteer to v21.4.1
Martin Adámek committed 3 years ago
83207a6
e192a0d
abf1841
chore: update yarn to v4 (#2160)
Martin Adámek committed 3 years ago
b20fd74
chore: fix out of sync lockfile
Martin Adámek committed 3 years ago
aa83faf
chore(deps): update node.js to v20 (#2144)
renovate[bot] committed 3 years ago
02a32e2
chore: update linter (#2146)
Martin Adámek committed 3 years ago
f07af75
fix: e2e cheerio-throw-on-ssl-errors (#2154)
Jindřich Bär committed 3 years ago
f2d333d
118515d
52b06fa
chore: make e2e pass again (#2141)
Vlad Frangu committed 3 years ago
7de0d0e
chore(deps): lock file maintenance
renovate[bot] committed 3 years ago
a778b3a