Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Build documentation changes
Ondra Urban committed 6 years ago
bedc6d2
Better docs for additionalMimeTypes
Jan Curn committed 6 years ago
cc66418
Fix npm audit, migrate stealth tests to Chrome 80
Ondra Urban committed 6 years ago
336db28
Update Puppeteer
Ondra Urban committed 6 years ago
dbb03a3
002c77e
Update default user agent
Ondra Urban committed 6 years ago
f4918ea
Fix puppeteerModule tests on Travis
Ondra Urban committed 6 years ago
4d4b8c3
Add tests for puppeteerModule, update changelog
Ondra Urban committed 6 years ago
8759062
Remove useless opts from launch log
Ondra Urban committed 6 years ago
db5c387
Fix #566
Ondra Urban committed 6 years ago
31dd49c
Bump version, update changelog
Ondra Urban committed 6 years ago
ab11f69
Migrate Travis configuration to v2
Ondra Urban committed 6 years ago
e65f98f
Merge pull request #562 from apifytech/feature/docs
Ondra Urban committed 6 years ago
9495bd6
7ea84ba
builded docs
Petr Patek committed 6 years ago
ccf36b2
65991d1
Removed Session experimental text. Builded docs.
Petr Patek committed 6 years ago
e58113d
Added changes after text correction.
Petr Patek committed 6 years ago
5d45bba
9863916
cb390b5
builded docs.
Petr Patek committed 6 years ago
bc1a039
finalized the API.
Petr Patek committed 6 years ago
d713ca5
fixed tests
Petr Patek committed 6 years ago
fd49fdf