Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Updated .gitignore, added new dev dependencies to package.json to fix NPM warning

Jan Curne4e35e42ca12
diff --git a/.gitignore b/.gitignore
index 58bd17fb0..d7638ddc7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@ lib
coverage
logs
pids
+.idea
diff --git a/package.json b/package.json
index 063ac9dad..3756cb0d9 100644
--- a/package.json
+++ b/package.json
@@ -42,6 +42,10 @@
"eslint": "^3.19.0",
"eslint-config-airbnb": "^14.1.0",
"eslint-plugin-react": "^6.10.3",
+ "eslint-config-airbnb-base": "^11.1.0",
+ "eslint-plugin-import": "^2.2.0",
+ "eslint-plugin-jsx-a11y": "^4.0.0",
+ "eslint-plugin-promise": "^3.4.2",
"isparta": "^4.0.0",
"mocha": "^3.2.0"
}