Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

chore: check the documentation builds on every PR (#1426)

Martin Adámek1810103fdacf
diff --git a/.github/workflows/test-and-release.yml b/.github/workflows/test-and-release.yml
index c7593e30f..3f574ec71 100644
--- a/.github/workflows/test-and-release.yml
+++ b/.github/workflows/test-and-release.yml
@@ -62,6 +62,39 @@ jobs:
- name: Tests
run: npm test
+ docs:
+ name: Docs build
+ if: ${{ !contains(github.event.head_commit.message, '[skip ci]') }}
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Source code
+ uses: actions/checkout@v3
+
+ - name: Use Node.js 16
+ uses: actions/setup-node@v3
+ with:
+ node-version: 16
+ cache: 'npm'
+ cache-dependency-path: 'package-lock.json'
+
+ - name: Turbo cache
+ id: turbo-cache
+ uses: actions/cache@v3
+ with:
+ path: .turbo
+ key: turbo-${{ github.job }}-${{ github.ref_name }}-${{ github.sha }}
+ restore-keys: |
+ turbo-${{ github.job }}-${{ github.ref_name }}-
+
+ - name: Install Dependencies
+ run: npm ci
+
+ - name: Build & deploy docs
+ run: |
+ cd website
+ npm ci
+ npm run build
+
lint:
name: Lint
runs-on: ubuntu-latest
diff --git a/docs/guides/apify_platform.mdx b/docs/guides/apify_platform.mdx
index 51694d812..12d461dc7 100644
--- a/docs/guides/apify_platform.mdx
+++ b/docs/guides/apify_platform.mdx
@@ -153,7 +153,7 @@ You can also develop your actor in an online code editor directly on the platfor
There are several things worth mentioning here.
1. Compared to Crawlee, in order to simplify access to the default <ApiLink to="core/class/KeyValueStore">`Key-Value Store`</ApiLink> and <ApiLink to="core/class/Dataset">`Dataset`</ApiLink>, you could also use [`Actor.getValue()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#getValue), [`Actor.setValue()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#setValue) for the default `Key-Value Store` and [`Actor.pushData()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#pushData) for the default `Dataset` directly, instead of using the helper functions of respective storage classes.
-2. When you plan to use the platform storage while developing and running your actor locally, you should use [`Actor.openKeyValueStore()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#openKeyValueStore), [`Actor.openDataset()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#openDataset) and [`Actor.openRequestQueue()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#openRequestQueue) to open the <ApiLink to="core/class/KeyValueStore">`KeyValueStore`</ApiLink>, <ApiLink to="core/class/Datase">`Dataset`</ApiLink> and <ApiLink to="core/class/RequestQueue">`RequestQueue`</ApiLink> respectively. Using each of these methods allows to pass the [`OpenStorageOptions`](https://apify.github.io/apify-sdk-js/api/apify/interface/OpenStorageOptions) as a second argument, which has only one optional property: [`forceCloud`](https://apify.github.io/apify-sdk-js/api/apify/interface/OpenStorageOptions#forceCloud). If set to `true` - cloud storage will be used instead of the folder on the local disk.
+2. When you plan to use the platform storage while developing and running your actor locally, you should use [`Actor.openKeyValueStore()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#openKeyValueStore), [`Actor.openDataset()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#openDataset) and [`Actor.openRequestQueue()`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor#openRequestQueue) to open the <ApiLink to="core/class/KeyValueStore">`KeyValueStore`</ApiLink>, <ApiLink to="core/class/Dataset">`Dataset`</ApiLink> and <ApiLink to="core/class/RequestQueue">`RequestQueue`</ApiLink> respectively. Using each of these methods allows to pass the [`OpenStorageOptions`](https://apify.github.io/apify-sdk-js/api/apify/interface/OpenStorageOptions) as a second argument, which has only one optional property: [`forceCloud`](https://apify.github.io/apify-sdk-js/api/apify/interface/OpenStorageOptions#forceCloud). If set to `true` - cloud storage will be used instead of the folder on the local disk.
:::note
If you don't plan to use the platform storage for a local actor - you don't need to use the [`Actor`](https://apify.github.io/apify-sdk-js/api/apify/class/Actor) class, meaning you could use <ApiLink to="core/class/KeyValueStore#open">`KeyValueStore.open()`</ApiLink>, <ApiLink to="core/class/Dataset#open">`Dataset.open()`</ApiLink> and <ApiLink to="core/class/RequestQueue#open">`RequestQueue.open()`</ApiLink> to open the respective storage both on the platform and locally.
:::