Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
fix: use merged cookies correctly in `GotScrapingHttpClient` (#3000)
a2985f2 Jindřich Bär committed 1 year ago
chore(release): update internal dependencies [skip ci]
dcb9938 Apify Release Bot committed 1 year ago
chore(deps): update dependency camoufox-js to ^0.4.0 (#2999)
59ee732 renovate[bot] committed 1 year ago
fix: enable full cookie support for `ImpitHttpClient` (#2991)
120f0a7 Jindřich Bär committed 1 year ago
fix: ensure `PlaywrightGotoOptions` won't result in `unknown` when playwright is not installed (#2995)
93eba38 Martin Adámek committed 1 year ago
chore(release): update internal dependencies [skip ci]
9fa523a Apify Release Bot committed 1 year ago
feat: Allow the AdaptivePlaywrightCrawler result comparator to signal an inconclusive result (#2975)
7ba8906 Jan Buchar committed 1 year ago
feat: add `MinimumSpeedStream` and `ByteCounterStream` helpers (#2970)
921c4ee Jindřich Bär committed 1 year ago
chore(deps): update dependency lint-staged to v16 (#2971)
0cebfcc renovate[bot] committed 1 year ago
chore: use self-hosted httpbin instance in e2e tests (#2967)
52531cb Jindřich Bär committed 1 year ago
chore(release): update internal dependencies [skip ci]
d9ec02e Apify Release Bot committed 1 year ago