Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Commits on master
test: skip the whole snapshotter timers test as it's too flaky on windows
265f819 Martin Adámek committed 5 years ago
chore(deps): update dependency htmlparser2 to v7 (#1165)
04dec6b renovate[bot] committed 5 years ago
chore(deps): update dependency @types/sinon to v10 (#1160)
5e02510 renovate[bot] committed 5 years ago
chore(deps): update dependency @types/socket.io to v3 (#1161)
3f09fa6 renovate[bot] committed 5 years ago
chore(deps): update actions/setup-node action to v2 (#1157)
b3d66b0 renovate[bot] committed 5 years ago
chore(deps): update styfle/cancel-workflow-action action to v0.9.1 (#1156)
5f67b6b renovate[bot] committed 5 years ago