Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Trigger travis build
Ondra Urban committed 6 years ago
98e90f2
Feature/typings class exports (#618)
Paulo Cesar committed 6 years ago
a34b7d6
Update required Node version info [skip ci]
Ondra Urban committed 6 years ago
f5b9dfc
Improve docs build script
Ondra Urban committed 6 years ago
35a5a3c
Update gitignore
Ondra Urban committed 6 years ago
7d02d0c
Bump version, update CI, fix engines [skip ci]
Ondra Urban committed 6 years ago
905fffc
Fixed typo in docs
Jan Curn committed 6 years ago
115586c
Pre-release
Ondra Urban committed 6 years ago
53ff5ee
c0507c0
Remove remnants of request package (#608)
Ondra Urban committed 6 years ago
a5e360d
Fix RequestList memory (#606)
Ondra Urban committed 6 years ago
7897bca
Docs/typescript (#614) [skip ci]
Paulo Cesar committed 6 years ago
49b98e4
fixed link (#612) [skip ci]
petrpatek committed 6 years ago
94ebeb2
53c96e6
Bump apify-client from 0.5.26 to 0.6.0 (#597) [skip ci]
dependabot-preview[bot] committed 6 years ago
08fc0a2
Bump proxy-chain from 0.3.3 to 0.4.1 (#604) [skip ci]
dependabot-preview[bot] committed 6 years ago
65d5d1e
Bump semver from 6.3.0 to 7.1.3 (#595) [skip ci]
dependabot-preview[bot] committed 6 years ago
e1ff00e
Bump @types/fs-extra from 7.0.0 to 8.1.0 (#596) [skip ci]
dependabot-preview[bot] committed 6 years ago
e01bbd0
Bump rimraf from 2.7.1 to 3.0.2 (#592) [skip ci]
dependabot-preview[bot] committed 6 years ago
6e0f5ec
Bump htmlparser2 from 3.10.1 to 4.1.0 (#601) [skip ci]
dependabot-preview[bot] committed 6 years ago
669a57e
Bump ws from 6.2.1 to 7.2.1 (#599) [skip ci]
dependabot-preview[bot] committed 6 years ago
c90bd85
Bump eslint from 5.16.0 to 6.8.0 (#593) [skip ci]
dependabot-preview[bot] committed 6 years ago
134dbe8
Bump sinon from 7.5.0 to 9.0.0 (#591) [skip ci]
dependabot-preview[bot] committed 6 years ago
d5aa876
Bump @types/jest from 24.9.1 to 25.1.3 (#590) [skip ci]
dependabot-preview[bot] committed 6 years ago
2605ca0
Pocesar feature/tsc compile (#605)
Ondra Urban committed 6 years ago
9a38f8c
Rebuild types [skip ci]
Ondra Urban committed 6 years ago
294b7ae
Fix docs footer [skip ci]
Ondra Urban committed 6 years ago
dc43187
Feature/new logger (#570)
Marek Trunkát committed 6 years ago
758cff1