Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update question.md
Andrey Bykov committed 5 years ago
78e883b
Add migration guide to SDK website [skip ci]
Ondra Urban committed 5 years ago
057af7b
Build docs with new examples
Ondra Urban committed 5 years ago
be93367
Build docs with new examples
Ondra Urban committed 5 years ago
fe3490d
Update changelog, bump version, build docs
Ondra Urban committed 5 years ago
6ac7a7c
Fix missing type definition in PuppeteerCrawler
Ondra Urban committed 5 years ago
c08cbbb
4f98af2
7aee31d
Increase test timeout
Ondra Urban committed 5 years ago
08792da
Remove deleted docs from v1.0.0
Ondra Urban committed 5 years ago
d7db549
Update front page example [skip ci]
Ondra Urban committed 5 years ago
0540808
Update og image, title and docs index [skip ci]
Ondra Urban committed 5 years ago
ceac9b7
Update docusaurus, changelog and version
Ondra Urban committed 5 years ago
d022dec
Build docs v1.0.0
Ondra Urban committed 5 years ago
e8c70cf
Do not log stack trace of internal error
Ondra Urban committed 5 years ago
d6798df
Do not add charset in setValue
Ondra Urban committed 5 years ago
1ae246b
Remove Apify.main error message prefix
Ondra Urban committed 5 years ago
eb593e4
5bd7d3a
Remove Playwright from 0.22.4 docs
Ondra Urban committed 5 years ago
edccff0
Remove misleading comment
Ondra Urban committed 5 years ago
3b88a0f
Update docker image guide [skip ci]
Ondra Urban committed 5 years ago
7f401de
fixed default persistCookiesPerSession (#910)
petrpatek committed 5 years ago
1266d90
hotfixed webdriver issue. (#909)
petrpatek committed 5 years ago
8951ea8
Bump ow from 0.21.0 to 0.23.0 (#907)
dependabot-preview[bot] committed 5 years ago
908846a
42d3279
1ae56da
Add docker image guide
Ondra Urban committed 5 years ago
1c98fe5
Remove obsolete test
Ondra Urban committed 5 years ago
05aa800
Improve docs
Ondra Urban committed 5 years ago
c59d3d6