Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
fix: use correct apify-client instance for snapshotting (#1308)
91d152a Martin Adámek committed 4 years ago
chore(deps): update actions/setup-node action to v3 (#1303)
181ca2b renovate[bot] committed 4 years ago
chore(deps): update actions/checkout action to v3 (#1304)
72a9855 renovate[bot] committed 4 years ago
docs: suggest fnm instead of nvm as its the same but much faster alternative
27b73c0 Martin Adámek committed 4 years ago
fix: improve guessing of chrome executable path on windows (#1294)
82abcdf audi Bookning committed 4 years ago
refactor: improve logging for fetching next request and timeouts (#1292)
8decf42 Martin Adámek committed 4 years ago
fix: expose logger classes on the `utils.log` instance (#1278)
e622e6b Martin Adámek committed 4 years ago