Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

766dece
Update docs [skip ci]
Ondra Urban committed 7 years ago
8d989ff
save
Petr Patek committed 7 years ago
a1d0878
fix log message
Tobey committed 7 years ago
299ddb0
6cf404a
Updated changelog
drobnikj committed 7 years ago
0241329
Update changelog
Ondra Urban committed 7 years ago
60b9343
46ae405
Fix RequestQueueLocal race condition
Ondra Urban committed 7 years ago
7c7a79f
Adding missing webhooks param to call() and callTask()
Marek Trunkat committed 7 years ago
be54c26
save
Petr Patek committed 7 years ago
2a345ce
updated test for utils.puppeteer.saveSnapshot
metalwarrior665 committed 7 years ago
39068a4
Documenting regex parameter of RequestList
Marek Trunkat committed 7 years ago
52244f3
add example for filling a form
Tobey committed 7 years ago
4fc96a1
Adding vscode config to gitignore
Marek Trunkat committed 7 years ago
dae971d
8dcc232
d5aad5b
fixed broken test.
Petr Patek committed 7 years ago
e04f6a9
Upgrading PPTR to 2.0.0 and fixing clicking
Marek Trunkat committed 7 years ago
368352f
added utils.puppeteer.saveSnapshot
metalwarrior665 committed 7 years ago
1fbd838
improved message.
Petr Patek committed 7 years ago
15e39b8
improved message.
Petr Patek committed 7 years ago
ec85535
improved message.
Petr Patek committed 7 years ago
2d9ce62