Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

Improving README, consolidated docs directories

Jan Curn5c7832a8d9c1
diff --git a/.gitignore b/.gitignore
index 7d6c628b3..613458a84 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,5 +12,4 @@ pids
.idea
yarn.lock
package-lock.json
-docs
-tmp
\ No newline at end of file
+tmp
diff --git a/README.md b/README.md
index 85f01f8fc..34b92a06d 100644
--- a/README.md
+++ b/README.md
@@ -200,6 +200,10 @@ Apify.main(async () => {
});
```
+When you run the example, you should see Chrome browser automated.
+
+![Image of automated Chrome browsers](https://raw.githubusercontent.com/apifytech/apify-js/feature/readme/docs/hello-world-browsers.gif)
+
By default, Apify SDK stores data to
`./apify_storage` in the current working directory.
You can override this behavior by setting either the
diff --git a/docs/hello-world-browsers.gif b/docs/hello-world-browsers.gif
new file mode 100644
index 000000000..9aceffa50
Binary files /dev/null and b/docs/hello-world-browsers.gif differ
diff --git a/jsdoc/conf.json b/docs/jsdoc-conf.json
similarity index 100%
rename from jsdoc/conf.json
rename to docs/jsdoc-conf.json
diff --git a/package.json b/package.json
index 7ad0f6193..52d39f5fb 100644
--- a/package.json
+++ b/package.json
@@ -38,7 +38,7 @@
],
"scripts": {
"build": "rm -rf ./build && babel src --out-dir build",
- "build-doc": "npm run clean && npm run build && node ./node_modules/jsdoc/jsdoc.js --package ./package.json -c ./jsdoc/conf.json -d docs",
+ "build-doc": "npm run clean && npm run build && node ./node_modules/jsdoc/jsdoc.js --package ./package.json -c ./docs/jsdoc-conf.json -d ./build/docs",
"build-toc": "./node_modules/.bin/markdown-toc README.md -i",
"test": "npm run build && nyc --reporter=html --reporter=text mocha --timeout 60000 --compilers js:babel-core/register --recursive",
"prepare": "npm run build-toc && npm run build",
diff --git a/publish.sh b/publish.sh
index 2a7c930e5..c183c319f 100755
--- a/publish.sh
+++ b/publish.sh
@@ -12,7 +12,7 @@ BRANCH_UP_TO_DATE=`git status | grep 'nothing to commit' | tr -s \n ' '`;
GIT_TAG="v${PACKAGE_VERSION}"
# Upload doc to S3 configuration
-DOC_DIR=${PWD}"/docs"
+DOC_DIR=${PWD}"/build/docs"
AWS_BUCKET="apify-runtime-js-doc"
if [ -z "${BRANCH_UP_TO_DATE}" ]; then