Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Add test for relative path
Ondra Urban committed 7 years ago
afd59da
Update index.js
Marek Trunkát committed 7 years ago
f9515da
Update utils.js
Marek Trunkát committed 7 years ago
2941c8b
Update changelog
Ondra Urban committed 7 years ago
0e84a2f
Add options.baseUrl to enqueueLinks
Ondra Urban committed 7 years ago
ca112ed
Updating changelog
Marek Trunkat committed 7 years ago
1f00948
Update changelog
Ondra Urban committed 7 years ago
1fb6b0e
7623f70
4f4da8b
Implement, add tests
Ondra Urban committed 7 years ago
414bca2
Update changelog, bump version, build docs
Ondra Urban committed 7 years ago
c1778a1
Update and build docs
Ondra Urban committed 7 years ago
7e75b2d
Update with PR comments
Ondra Urban committed 7 years ago
d44d566
Update docs
Ondra Urban committed 7 years ago
f23f0cf
Add enqueueLinks to Apify.utils
Ondra Urban committed 7 years ago
dbabd56
Add cheerio to enqueueLinks, add tests
Ondra Urban committed 7 years ago
5efe45d
Move enqueueLinks() to its own file
Ondra Urban committed 7 years ago
0b112a8
Update and build docs
Ondra Urban committed 7 years ago
f736fef
Updated TODOs
Jan Curn committed 7 years ago
65e4802
Improved puppeteer_crawler.js example
Jan Curn committed 7 years ago
6bf89fd
Improve docs and examples
Jan Curn committed 7 years ago
64c25c3
App - adding some basic system info
Marek Trunkat committed 7 years ago
2077cf0
Merge branch 'develop' into fix/launch-pup-error
Ondra Urban committed 7 years ago
0a750de
Build docs
Ondra Urban committed 7 years ago
1f48e1f
Bump version, update Changelog, turn off reusePages
Ondra Urban committed 7 years ago
762d1c1
Updated CHANGELOG
Jan Curn committed 7 years ago
7df8429
Add forEachKey function to key value store (#283)
Ondra Urban committed 7 years ago
8c30b17