Skip to content

Supercharge your AI agents with data from the web and beyond. Building the library for superintelligence. 🔥

docs: kubernetes simple update (#1772)

Gergő Móriczac875b8ad9a7
diff --git a/examples/kubernetes/cluster-install/README.md b/examples/kubernetes/cluster-install/README.md
index 126c638d6..d820d7c92 100644
--- a/examples/kubernetes/cluster-install/README.md
+++ b/examples/kubernetes/cluster-install/README.md
@@ -8,20 +8,6 @@
```
Replace `password`, `host`, and `port` with the appropriate values.
-
-2. Build Docker images, and host it in your Docker Registry (replace the target registry with your own)
- 1. API (which is also used as a worker image)
- 1. ```bash
- docker build --no-cache --platform linux/amd64 -t ghcr.io/winkk-dev/firecrawl:latest ../../../apps/api
- docker push ghcr.io/winkk-dev/firecrawl:latest
- ```
- 2. Playwright
- 1. ```bash
- docker build --no-cache --platform linux/amd64 -t ghcr.io/winkk-dev/firecrawl-playwright:latest ../../../apps/playwright-service
- docker push ghcr.io/winkk-dev/firecrawl-playwright:latest
- ```
-3. Replace the image in [worker.yaml](worker.yaml), [api.yaml](api.yaml) and [playwright-service.yaml](playwright-service.yaml)
-
## Install
```bash
kubectl apply -f configmap.yaml
diff --git a/examples/kubernetes/cluster-install/api.yaml b/examples/kubernetes/cluster-install/api.yaml
index c860e4b6c..1c9c897cf 100644
--- a/examples/kubernetes/cluster-install/api.yaml
+++ b/examples/kubernetes/cluster-install/api.yaml
@@ -12,23 +12,34 @@ spec:
labels:
app: api
spec:
+ terminationGracePeriodSeconds: 180
imagePullSecrets:
- name: docker-registry-secret
containers:
- name: api
- image: ghcr.io/winkk-dev/firecrawl:latest
+ image: ghcr.io/mendableai/firecrawl:latest
imagePullPolicy: Always
- args: [ "pnpm", "run", "start:production" ]
+ command: [ "node" ]
+ args: [ "--max-old-space-size=6144", "dist/src/index.js" ]
ports:
- containerPort: 3002
env:
- name: FLY_PROCESS_GROUP
value: "app"
+ - name: PORT
+ value: "3002"
envFrom:
- configMapRef:
name: firecrawl-config
- secretRef:
name: firecrawl-secret
+ resources:
+ requests:
+ memory: "4G"
+ cpu: "2000m"
+ limits:
+ memory: "6G"
+ cpu: "2000m"
livenessProbe:
httpGet:
path: /v0/health/liveness
diff --git a/examples/kubernetes/cluster-install/configmap.yaml b/examples/kubernetes/cluster-install/configmap.yaml
index 5796fe82a..b60dbacfc 100644
--- a/examples/kubernetes/cluster-install/configmap.yaml
+++ b/examples/kubernetes/cluster-install/configmap.yaml
@@ -3,11 +3,12 @@ kind: ConfigMap
metadata:
name: firecrawl-config
data:
- NUM_WORKERS_PER_QUEUE: "8"
- PORT: "3002"
HOST: "0.0.0.0"
REDIS_URL: "redis://redis:6379"
REDIS_RATE_LIMIT_URL: "redis://redis:6379"
PLAYWRIGHT_MICROSERVICE_URL: "http://playwright-service:3000"
USE_DB_AUTHENTICATION: "false"
- HDX_NODE_BETA_MODE: "1"
+ SENTRY_ENVIRONMENT: "production"
+ ENV: "production"
+ LOGGING_LEVEL: "DEBUG"
+ IS_KUBERNETES: "true"
diff --git a/examples/kubernetes/cluster-install/playwright-service.yaml b/examples/kubernetes/cluster-install/playwright-service.yaml
index 43cf15f07..626f6d2c8 100644
--- a/examples/kubernetes/cluster-install/playwright-service.yaml
+++ b/examples/kubernetes/cluster-install/playwright-service.yaml
@@ -23,7 +23,7 @@ spec:
- name: docker-registry-secret
containers:
- name: playwright-service
- image: ghcr.io/winkk-dev/firecrawl-playwright:latest
+ image: ghcr.io/mendableai/playwright-service:latest
imagePullPolicy: Always
ports:
- containerPort: 3000
@@ -32,16 +32,7 @@ spec:
name: playwright-service-config
livenessProbe:
httpGet:
- path: /health/liveness
- port: 3000
- initialDelaySeconds: 30
- periodSeconds: 30
- timeoutSeconds: 5
- successThreshold: 1
- failureThreshold: 3
- readinessProbe:
- httpGet:
- path: /health/readiness
+ path: /health
port: 3000
initialDelaySeconds: 30
periodSeconds: 30
diff --git a/examples/kubernetes/cluster-install/worker.yaml b/examples/kubernetes/cluster-install/worker.yaml
index e2d31ebc3..d99afc6c4 100644
--- a/examples/kubernetes/cluster-install/worker.yaml
+++ b/examples/kubernetes/cluster-install/worker.yaml
@@ -14,16 +14,36 @@ spec:
spec:
imagePullSecrets:
- name: docker-registry-secret
+ terminationGracePeriodSeconds: 60
containers:
- name: worker
- image: ghcr.io/winkk-dev/firecrawl:latest
+ image: ghcr.io/mendableai/firecrawl:latest
imagePullPolicy: Always
- args: [ "pnpm", "run", "workers" ]
+ command: [ "node" ]
+ args: [ "--max-old-space-size=3072", "dist/src/services/queue-worker.js" ]
env:
- name: FLY_PROCESS_GROUP
value: "worker"
+ - name: PORT
+ value: "3005"
envFrom:
- configMapRef:
name: firecrawl-config
- secretRef:
name: firecrawl-secret
+ resources:
+ requests:
+ memory: "3G"
+ cpu: "1000m"
+ limits:
+ memory: "4G"
+ cpu: "1000m"
+ livenessProbe:
+ httpGet:
+ path: /liveness
+ port: 3005
+ initialDelaySeconds: 5
+ periodSeconds: 5
+ timeoutSeconds: 5
+ successThreshold: 1
+ failureThreshold: 3