Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

docs: switch from pipx to uvx (#3092)
Vlada Dusek committed 1 year ago
5341487
79f3ba5
docs: remove old documentation snapshots (#3066)
Jindřich Bär committed 1 year ago
d53a94e
b5fcd79
feat: add `maxCrawlDepth` crawler option (#3045)
Jindřich Bär committed 1 year ago
0090df9
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
8c8a817
acc8836
chore(deps): lock file maintenance (#3047)
renovate[bot] committed 1 year ago
8feee63
chore(deps): update dependency puppeteer to v24.12.1
renovate[bot] committed 1 year ago
8a09d62
chore(release): update internal dependencies [skip ci]
Apify Release Bot committed 1 year ago
c81d2b4
chore(release): v3.13.10
Apify Release Bot committed 1 year ago
bef4420
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
8fef386
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
bba92e1
chore(deps): update dependency puppeteer to v24.12.0
renovate[bot] committed 1 year ago
445a49e
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
149a212
7ed3167
chore(deps): update dependency puppeteer to v24.11.2
renovate[bot] committed 1 year ago
a81058e
fix: improve enqueueLinks `limit` checking (#3038)
Jindřich Bär committed 1 year ago
2774124
chore: add repository of the day badge (#3036)
Vlada Dusek committed 1 year ago
d4bed5e
chore(deps): lock file maintenance
renovate[bot] committed 1 year ago
61d4586
38de7af
f1b1f68