Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
docs: treat local absolute links from readme as relative (#2287)
009756f Martin Adámek committed 2 years ago
chore(release): update internal dependencies [skip ci]
602aab6 Apify Release Bot committed 2 years ago
chore(deps): update actions/deploy-pages action to v4 (#2270)
d19d4fe renovate[bot] committed 2 years ago
chore(deps): update actions/upload-pages-artifact action to v3 (#2271)
fa8bf34 renovate[bot] committed 2 years ago
fix(RequestQueue): always clear locks when a request is reclaimed (#2263)
0fafe29 Vlad Frangu committed 2 years ago