Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Commits on master

Update changelog, bump version
Ondra Urban committed 6 years ago
85ee62a
74ac6a2
Fix null response in PuppeteerCrawler
Ondra Urban committed 6 years ago
ec6d66c
Update changelog, bump apify-shared
Ondra Urban committed 6 years ago
6527b30
Fix session management in PuppeteerCrawler (#791)
Ondra Urban committed 6 years ago
4c1ebef
Update links to docs (#790)
Misha committed 6 years ago
7d077c3
Update changelog, bump version
Ondra Urban committed 6 years ago
d50730f
Fix Node 14 (#789)
Ondra Urban committed 6 years ago
3e46bec
Actually bump the version, fix some doc examples
Ondra Urban committed 6 years ago
64a8c10
Update changelog, bump version, fix doctypes
Ondra Urban committed 6 years ago
c777a0e
Update default user-agent
Ondra Urban committed 6 years ago
76c202d
Bump puppeteer from 5.2.0 to 5.2.1 (#753)
dependabot-preview[bot] committed 6 years ago
d1d4790
Fix proxy check crashes (#762)
Ondra Urban committed 6 years ago
42aa993
dd9d546
Fix error message being printed needlessly
Ondra Urban committed 6 years ago
8333d21
Update changelog, bump version
Ondra Urban committed 6 years ago
5011f18
Fix leading dot puppeteer cookies in Session
Ondra Urban committed 6 years ago
91dfc4c
Update changelog, bump version
Ondra Urban committed 6 years ago
ede02ab
Bump puppeteer from 3.3.0 to 5.2.0 (#746)
dependabot-preview[bot] committed 6 years ago
0017b47
Fix waitForRunToFinish docs [skip ci]
Ondra Urban committed 6 years ago
6fcaff1
RequestQueueLocal race condition (#733)
Charlie Orford committed 6 years ago
8966527
Improve docs [skip ci]
Ondra Urban committed 6 years ago
6ad7373
Fix travis link
Ondra Urban committed 6 years ago
ae6bf1f
Update apifytech org to apify
Ondra Urban committed 6 years ago
fb0754e
Fix session debug logging
Ondra Urban committed 6 years ago
06a7338
Improve proxy error message [skip ci]
Ondra Urban committed 6 years ago
c238692
233a006