Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
Updated .gitignore, added new dev dependencies to package.json to fix NPM warning
Jan Curne4e35e42ca12
diff --git a/.gitignore b/.gitignoreindex 58bd17fb0..d7638ddc7 100644--- a/.gitignore+++ b/.gitignore@@ -7,3 +7,4 @@ libcoveragelogspids+.ideadiff --git a/package.json b/package.jsonindex 063ac9dad..3756cb0d9 100644--- a/package.json+++ b/package.json@@ -42,6 +42,10 @@"eslint": "^3.19.0","eslint-config-airbnb": "^14.1.0","eslint-plugin-react": "^6.10.3",+ "eslint-config-airbnb-base": "^11.1.0",+ "eslint-plugin-import": "^2.2.0",+ "eslint-plugin-jsx-a11y": "^4.0.0",+ "eslint-plugin-promise": "^3.4.2","isparta": "^4.0.0","mocha": "^3.2.0"}