Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript apifyautomationcrawlercrawlingheadlessheadless-chromejavascriptnodejsnpmplaywrightpuppeteerscraperscrapingtypescriptweb-crawlerweb-crawlingweb-scraping
chore: add health check endpoint (#3273)
Marko Kubrachenkodf822c2054ff
diff --git a/.github/workflows/deploy-nginx.yml b/.github/workflows/deploy-nginx.ymlindex d4d9c61c3..ecf999eb1 100644--- a/.github/workflows/deploy-nginx.yml+++ b/.github/workflows/deploy-nginx.yml@@ -18,7 +18,7 @@ jobs:run: |gh workflow run deploy-nginx.yml \--repo apify/apify-docs-private \- --field deployment=crawlee-docs+ --field deployment=crawlee-webecho "✅ Deployment workflow triggered successfully"env:GITHUB_TOKEN: ${{ secrets.APIFY_SERVICE_ACCOUNT_GITHUB_TOKEN }}diff --git a/website/nginx.conf b/website/nginx.confindex 3ebe457f8..1739070e5 100644--- a/website/nginx.conf+++ b/website/nginx.conf@@ -3,6 +3,14 @@server {listen 0.0.0.0:8080;server_name 'crawlee.dev';++ # Health check endpoint+ location /health {+ access_log off;+ return 200 '{"status":"UP"}';+ add_header Content-Type application/json;+ }+location / {proxy_pass https://apify.github.io/crawlee/;}