Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

1990b26
fix: more permissive accept (#3373)
nikitachapovskii-dev committed 9 months ago
d03af1b
555a5f8
5313797
chore(deps): update dependency puppeteer to v24.36.1
renovate[bot] committed 9 months ago
90b2042
docs: Fix the broken link (#3366)
Max Bohomolov committed 9 months ago
9fb9a0c
3318147
chore(deps): update dependency puppeteer to v24.36.0
renovate[bot] committed 10 months ago
21f59ba
docs: add Impit HTTP client guide to v3.15 docs (#3360)
Martin Adámek committed 10 months ago
585bf17
docs: add guide for Impit HTTP client (#3359)
Martin Adámek committed 10 months ago
d70fc8f
7a087ae
chore: bump biome to 2.3.11 and run format (#3354)
Martin Adámek committed 10 months ago
63d753d
chore(deps): update patch/minor dependencies (#3235)
renovate[bot] committed 10 months ago
254d0c1
feat(utils): add `discoverValidSitemaps` utility (#3339)
Sviatozar Petrenko committed 10 months ago
29f52ed
16a6967
6cd9456
72aacb4
049413b
fix: remove deprecation from `RequestQueueV1` (#3341)
Martin Adámek committed 10 months ago
89309bc
test: fix flaky requestHandlerTimeoutSecs test (#3340)
Martin Adámek committed 10 months ago
5f33f65
chore(deps): update dependency impit to ^0.8.0 (#3332)
renovate[bot] committed 10 months ago
c292d3e