Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: build docs and bump patch version [skip ci]
Ondra Urban committed 5 years ago
8676705
fix: pin got-scraping to fix compatibility issues
Ondra Urban committed 5 years ago
9cf09fa
chore: prepare for version 2 release
Ondra Urban committed 5 years ago
062f928
Fix URL parsing logic in `Getting Started` (#1107)
Szymon Marczak committed 5 years ago
fce6728
chore: fix changelog
Martin Adámek committed 5 years ago
7e56474
docs: fix code examples for pre/post navigation hooks
Martin Adámek committed 5 years ago
d0d7d80
Fix/getting started links (#1106)
Sapiens Malti committed 5 years ago
4c709f6
chore: build and publish 1.3.1 docs [skip ci]
Ondra Urban committed 5 years ago
ed570b9
chore: fix peer deps definition
Martin Adámek committed 5 years ago
e43c137
5e00a45
chore: updated changelog [skip ci]
Petr Patek committed 5 years ago
7788078
chore: bump version
Petr Patek committed 5 years ago
21ea570
fix: configuration apiBaseUrl (#1095)
petrpatek committed 5 years ago
60d4086
442a7ac
chore: more debugging of a flaky test
Ondra Urban committed 5 years ago
7e3e35e
chore: disable the `taskTimeoutSecs` by default (#1083)
Martin Adámek committed 5 years ago
94fa888
test: add workflow to test TS support (#1079)
Martin Adámek committed 5 years ago
34f119c
fix: fix support for building TypeScript projects
Martin Adámek committed 5 years ago
b1fb020
584adbb
chore: add logging to a flaky test
Ondra Urban committed 5 years ago
9fbda1f
chore: remove cache from release workflow
Ondra Urban committed 5 years ago
7cf96ac
31a135c