Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

chore(deps): update dependency globals to v17 (#3320)
renovate[bot] committed 10 months ago
8638d24
9c0580b
docs: `Request.userData` must be serializable (#3327)
Jindřich Bär committed 10 months ago
2037ea9
test: fix flaky tests (#3323)
Martin Adámek committed 10 months ago
a2fa3ee
chore: update playwright to v1.57.0 (#3321)
Martin Adámek committed 10 months ago
d2a1eb7
chore(ci): fix assertions in e2e tests (#3316)
Jindřich Bär committed 10 months ago
ee73379
37b6602
docs: fix the overlapping `LLMButtons` dropdown (#3307)
Richard Biroš committed 11 months ago
2229ab6
docs: port the Python HTTP clients guide to JS (#3301)
Jindřich Bär committed 11 months ago
ace4b36
efac644
chore(deps): update actions/cache action to v5 (#3303)
renovate[bot] committed 11 months ago
c647a8b
chore(ci): remove old ZenHub CI integration (#3297)
Jindřich Bär committed 11 months ago
649e2a4
629daf8
a8177d3
bf503fb
chore: invalidate CloudFront cache when web changes (#3281)
Marko Kubrachenko committed 11 months ago
cd5f43b
docs: add new LLM dropdown (#3267)
Richard Biroš committed 11 months ago
d7cb5a0
e20b4bf
chore: add health check endpoint (#3273)
Marko Kubrachenko committed 11 months ago
df822c2
chore: add nginx conf and deploy workflow (#3268)
Marko Kubrachenko committed 11 months ago
45ec415
6c76d32
chore(e2e): fix flaky e2e test orchestration (#3255)
Jindřich Bär committed 12 months ago
59f2074
chore(deps): update actions/checkout action to v6 (#3260)
renovate[bot] committed 12 months ago
5b143b2
465c246
3ed0568
3450f27