Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

5e3ff31
Renamed sequential store to dataset
Jaroslav Hejlek committed 8 years ago
765c584
Updated docs
Jan Curn committed 8 years ago
c5a4618
Updated docs
Jan Curn committed 8 years ago
54fdb58
Updated default user agent
Jan Curn committed 8 years ago
33dd746
Merge pull request #21 from Apifier/develop
Marek Trunkát committed 8 years ago
c9cc12f
Updated docs
Jan Curn committed 8 years ago
0912bf4
Updated comments
Jan Curn committed 8 years ago
00850e1
Fixed comments in Apify.getValue()
Jan Curn committed 8 years ago
d8b8f5a
Fixed test
Jaroslav Hejlek committed 8 years ago
de3b972
Version bump
Marek Trunkat committed 8 years ago
9799c90
Apify client expects object in data parameter
Jaroslav Hejlek committed 8 years ago
feba7b7
Version bump
Marek Trunkat committed 8 years ago
fb00000
45fb615
Ignore documentation for pushRecord
Jaroslav Hejlek committed 8 years ago
2e116c3
Reworked how pushRecord is called
Jaroslav Hejlek committed 8 years ago
7ec3ed0
Tests and improvements
Jaroslav Hejlek committed 8 years ago
0d5938e
pushRecord function
Jaroslav Hejlek committed 8 years ago
6813985
Bumped package.json version
Jan Curn committed 8 years ago
4eb81f6
0571eae
Small fixies according to PR
Marek Trunkat committed 8 years ago
347339a
Adding getMemoryInfo()
Marek Trunkat committed 8 years ago
6a47f6f
abad252
Merge pull request #14 from Apifier/feature/proxy-chain
Marek Trunkát committed 9 years ago
521e209
b023ed0
Small fix in unit tests
Jan Curn committed 9 years ago
235258f
ecec146