Skip to content

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

chore: add health check endpoint (#3273)

Marko Kubrachenkodf822c2054ff
diff --git a/.github/workflows/deploy-nginx.yml b/.github/workflows/deploy-nginx.yml
index d4d9c61c3..ecf999eb1 100644
--- a/.github/workflows/deploy-nginx.yml
+++ b/.github/workflows/deploy-nginx.yml
@@ -18,7 +18,7 @@ jobs:
run: |
gh workflow run deploy-nginx.yml \
--repo apify/apify-docs-private \
- --field deployment=crawlee-docs
+ --field deployment=crawlee-web
echo "✅ Deployment workflow triggered successfully"
env:
GITHUB_TOKEN: ${{ secrets.APIFY_SERVICE_ACCOUNT_GITHUB_TOKEN }}
diff --git a/website/nginx.conf b/website/nginx.conf
index 3ebe457f8..1739070e5 100644
--- a/website/nginx.conf
+++ b/website/nginx.conf
@@ -3,6 +3,14 @@
server {
listen 0.0.0.0:8080;
server_name 'crawlee.dev';
+
+ # Health check endpoint
+ location /health {
+ access_log off;
+ return 200 '{"status":"UP"}';
+ add_header Content-Type application/json;
+ }
+
location / {
proxy_pass https://apify.github.io/crawlee/;
}