Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Bug fix in docs
Lukáš Křivka committed 8 years ago
952aa77
Remove duplicit key from eslint config
Ondra Urban committed 8 years ago
ee8007c
Fix repeating requests on failed page.close()
Ondra Urban committed 8 years ago
4210cb8
50cf26f
Fix failing test and move new test
Ondra Urban committed 8 years ago
142a43b
Allow await in loops in ESLint configuration
Ondra Urban committed 8 years ago
e971997
Fix startup code examples ( _ is not url safe )
Ondra Urban committed 8 years ago
09cbde4
Merge pull request #102 from mnmkng/feature/live-view
Marek Trunkát committed 8 years ago
f99ca09
Merge branch 'develop' into feature/live-view
Ondra Urban committed 8 years ago
2d83dd7
Refactor PuppeteerLiveView to improve readability
Ondra Urban committed 8 years ago
81cde48
Merge branch 'master' into feature/live-view
Ondra Urban committed 8 years ago
c3e6d10
Use CONTAINER_URL/PORT env vars in LiveViewServer
Ondra Urban committed 8 years ago
114df3a
Use promisifyServerListen with LiveServer
Ondra Urban committed 8 years ago
0962288
Fix semver modifier of major v. 0 packages
Ondra Urban committed 8 years ago
b52b419
9651b3f
6ffe535
Bumped package.json version, updated CHANGELOG
Jan Curn committed 8 years ago
ed84b8c
Merge branch 'develop'
Jan Curn committed 8 years ago
a260e5f
Bumped package.json version
Jan Curn committed 8 years ago
99662ac
144d0e8
Merge branch 'develop'
Marek Trunkat committed 8 years ago
89b773e
Refactor code based on commit comments
Ondra Urban committed 8 years ago
0f918da
Fixed broken unit test
Jan Curn committed 8 years ago
b34fd59
f845542
Version bump
Marek Trunkat committed 8 years ago
bfe66ca
d40fda2