Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

docs: update typedoc plugin + fix some broken links
Martin Adámek committed 4 years ago
f18ad91
docs: improve onboarding (#1464)
Ondra Urban committed 4 years ago
b2270a0
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
6438a59
docs: try to fix active class on navbar api docs links
Martin Adámek committed 4 years ago
4d598d2
docs: remove debugging code to fix build
Martin Adámek committed 4 years ago
485d83d
docs: rework navbar links to support versioning
Martin Adámek committed 4 years ago
9fa4830
docs: change GTM id
Martin Adámek committed 4 years ago
a1b395b
docs: use strict matching in giscus
Martin Adámek committed 4 years ago
d23235a
chore(deps): update npm to v8.18.0
renovate[bot] committed 4 years ago
293b73d
chore(deps): lock file maintenance
renovate[bot] committed 4 years ago
9c3e2e8
61087a2
chore: copy tsconfig in crawlee templates
Martin Adámek committed 4 years ago
19d6a4b
chore: bump version to 3.0.4
Martin Adámek committed 4 years ago
99d125e
chore: bump version to 3.0.4
Martin Adámek committed 4 years ago
65fb5ca
chore: update fingerprint suite
Martin Adámek committed 4 years ago
42e1419
feat: enable tab-as-a-container for Firefox (#1456)
Szymon Marczak committed 4 years ago
ae5ba4f
docs: fix docs build
Martin Adámek committed 4 years ago
793eb60
docs: tag 3.0 snapshot
Martin Adámek committed 4 years ago
229aed1
chore: rename DOMCrawler to JSDOMCrawler (#1458)
Szymon Marczak committed 4 years ago
b70740d
docs: fix links in project template readmes
Martin Adámek committed 4 years ago
24623fd
docs: fix link to introduction guide
Martin Adámek committed 4 years ago
4b30655
46f6e4c
docs: remove old outdated readme snippets
Martin Adámek committed 4 years ago
058d28d
chore: simplify project templates
Martin Adámek committed 4 years ago
520fe63
fix: make `CheerioCrawlerOptions` type more loose
Martin Adámek committed 4 years ago
d871d8c
89afbbb
docs: change order of packages in API docs
Martin Adámek committed 4 years ago
83e9666
fix: improve dockerfiles and project templates
Martin Adámek committed 4 years ago
7c21a64