Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Delete TimeoutError.md
Ondra Urban committed 7 years ago
19cb385
Ignore timeout error from docs
Ondra Urban committed 7 years ago
20233d1
Remove invalid char from default user agent
Ondra Urban committed 7 years ago
d7462c8
Updated DEFAULT_USER_AGENT
drobnikj committed 7 years ago
78f9486
Updated changelog [skip ci]
drobnikj committed 7 years ago
db39907
Merging develop
Marek Trunkat committed 7 years ago
77fcdf5
increased chrome version in UA.
Petr Patek committed 7 years ago
61a3444
removed unnecessary console.logs
Petr Patek committed 7 years ago
39a841a
f0b89bf
Implementation improved. Docs Added.
Petr Patek committed 7 years ago
37d2369
aad245d
fixed missing import
Petr Patek committed 7 years ago
45f3d7a
refactored code and builded docs
Petr Patek committed 7 years ago
5a28fd3
builded docs
Petr Patek committed 7 years ago
7ae33ae
basic crawler usage.
Petr Patek committed 7 years ago
a288886
save
Petr Patek committed 7 years ago
66b70cb
5a8fc46
fixed the tests.
Petr Patek committed 7 years ago
1cdbaba
Merge branch 'master' into feature/session-pool
Petr Patek committed 7 years ago
f841ec1
Merge pull request #475 from apifytech/fix/stealth
Ondra Urban committed 7 years ago
7839f7b
renamed function and changed docs.
Petr Patek committed 7 years ago
c29b793
builded docs
Petr Patek committed 7 years ago
6b82a3b
Merge branch 'master' into fix/stealth
Petr Patek committed 7 years ago
8e428fa
builded docs
Petr Patek committed 7 years ago
ea98dfa
Build docs [skip ci]
Ondra Urban committed 7 years ago
f9c2fd8