Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore: add more badges to the readme
Martin Adámek committed 5 years ago
44b07b3
test: disable another too flaky assertion
Martin Adámek committed 5 years ago
2e208cd
test: disable too flaky assertion
Martin Adámek committed 5 years ago
751e3ba
chore: add missing `domhandler` types
Martin Adámek committed 5 years ago
e0ccf19
chore: rebuild docs
Martin Adámek committed 5 years ago
8dc52bb
chore: move `@types/*` to dev dependencies
Martin Adámek committed 5 years ago
89d22a8
chore: move fs-extra to dev dependencies
Martin Adámek committed 5 years ago
f72e95c
chore: move express to dev dependencies
Martin Adámek committed 5 years ago
79cc31e
chore: remove unused dependency
Martin Adámek committed 5 years ago
75b6a0d
04dec6b
test: try to fix flaky test
Martin Adámek committed 5 years ago
926be57
build: unify check and release workflows
Martin Adámek committed 5 years ago
641dcf1
chore: update jest to v27
Martin Adámek committed 5 years ago
9be034f
build: skip publishing to npm when not master branch
Martin Adámek committed 5 years ago
87dca90
5e02510
chore(deps): update dependency sinon to v11 (#1167)
renovate[bot] committed 5 years ago
6d8bc50
3f09fa6
chore(deps): update dependency fs-extra to v10 (#1164)
renovate[bot] committed 5 years ago
7c4fa40
chore: remove unused dependency
Martin Adámek committed 5 years ago
6d5042e
chore(deps): update dependency typescript to v4.4.2
Renovate Bot committed 5 years ago
65d60d2
build: run PR checks on renovate branches too
Martin Adámek committed 5 years ago
67d568b
build: add PR title check workflow
Martin Adámek committed 5 years ago
a8c46bf
chore: bump version
Martin Adámek committed 5 years ago
89c40db
b3d66b0
build: remove dependabot in favour of renovate
Martin Adámek committed 5 years ago
02b209f
build: Configure Renovate (#1155)
renovate[bot] committed 5 years ago
7188375
chore: require newer got-scraping and browser-pool
Martin Adámek committed 5 years ago
71150be
chore: update changelog
Martin Adámek committed 5 years ago
137c4f6