Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Adding request into handlePageFunction args
Marek Trunkat committed 8 years ago
f20e12e
954b28c
be02359
Fixing documentation and bug in BasicCrawler
Marek Trunkat committed 8 years ago
10f115a
Adding kvstores datasets autoscaledpool etc... (#24)
Marek Trunkát committed 8 years ago
eb4bbfc
Removed squid from .travis.yml
Jan Curn committed 8 years ago
21d32ea
82cc9c4
Renamed old Apifier links and texts to Apify
Jan Curn committed 8 years ago
890012d
Changed links to Travis CI in README.md
Jan Curn committed 8 years ago
c7906f1
Changed links to Travis CI in README.md
Jan Curn committed 8 years ago
2194020
Changed links to Apify GitHub repo
Jan Curn committed 8 years ago
6ccf4d8
Updating changelog + version bumb
Marek Trunkat committed 8 years ago
45bff8e
Publishing pushData documentation
Marek Trunkat committed 8 years ago
597753f
f9b2278
Merge branch 'develop'
Jan Curn committed 8 years ago
f542a14
Merge branch 'develop'
Marek Trunkat committed 8 years ago
ea55e15
Updated version of client
Jaroslav Hejlek committed 8 years ago
f49ba1a
Bumped version
Jaroslav Hejlek committed 8 years ago
f25eb30
Renamed pushItem to pushData and allowed arrays
Jaroslav Hejlek committed 8 years ago
61851a5
Bumped version to 4.40
Jaroslav Hejlek committed 8 years ago
d878672
Fixed tests and increased client version
Jaroslav Hejlek committed 8 years ago
c978f65
Merge pull request #23 from Apifier/feature/fix-call
Marek Trunkát committed 8 years ago
21b9fdc
Updated docs
Jan Curn committed 8 years ago
775754b