Free, Open Source, Self-Hosted WhatsApp API Gateway
13 - Horizontal Scaling Guide
⚠️ PARTIALLY IMPLEMENTED — STILL DEPLOY
replicas: 1Supported topology remains exactly one API instance per session-data volume. Do not run the multi-replica examples below yet.
What now exists. Sessions carry an owner (
nodeId) and a renewed lease (leaseExpiresAt). A process claims a session before starting its engine and refuses when another node holds a live claim, so two replicas can no longer both launch the same session — which is what corrupted the shared LocalAuth directory. A booting process also resets only the sessions it may claim, instead of reporting a live peer's sessions as disconnected. Claims are released on a clean shutdown and expire otherwise, so failover does not depend on one.NODE_IDnames the process; it defaults to the hostname and must be stable across restarts and unique per running process: two processes sharing a hostname (host networking, pm2 cluster mode) must each set it, and a node that sees another process renewing leases under itsNODE_IDlogsduplicate_node_id.A node that loses its claim gives up the engine. A lease can lapse while the process is perfectly healthy — a slow query is enough — after which a peer may legitimately take the session; while the holder can reach the database, renewal detects the loss and tears the local engine down at the next heartbeat, so any two-engine overlap is bounded to roughly one heartbeat interval (and a teardown failure is logged as an error rather than silently retried). A failed renewal is deliberately not read as a loss: the TTL is sized to absorb a database blip, and concluding otherwise would stop every healthy engine on the node. The price is the partition case: a holder that cannot reach the database keeps its engines, a peer may adopt the session once the lease lapses, and both engines run until the holder's first successful renewal after it reconnects, so the overlap lasts as long as the outage plus up to one heartbeat.
Bulk-send batches follow the same rule. A batch is only ever driven by the process holding its session's engine, so a booting replica now reaps only the batches whose sessions it may claim, instead of declaring a peer's in-flight batches FAILED while they are still sending. A data import likewise reports the sessions it could not reconcile because another node is running them.
Failover now completes on its own. A periodic takeover sweep (default every 30s,
SESSION_TAKEOVER_SWEEP_MS) adopts sessions whose holder's lease lapsed — a crashed peer, or a recreated container whose new identity boots before its old lease expires. Only authenticated sessions in a running-or-should-be state are adopted; mid-pairing and operator-failedones are left alone, and a cleanly stopped session releases its claim so it is never "lapsed". Adopting a session fails its stuck in-flight batches (no auto-resume — the dead node's already-sent messages are unknowable), and so does an explicitPOST /startorPOST /stopof a session whose holder's lease lapsed. The sweep's adoption is gated by the sameAUTO_START_SESSIONSflag as boot auto-start; the sweep itself is not, and runs on every node, because it also has a job that starts nothing: a row a vanished node leftready,initializing,authenticatingoraction_requiredis marked disconnected once its lease expired more than two TTLs ago, which is three TTLs after the holder's last renewal plus up to one sweep interval. Aqr_readyrow is marked too while it has no phone, since the sweep never adopts a row without one; aqr_readyrow with a phone keeps its status, so the correction never makes it adoptable. Nothing else revisits such a row, since the boot reset skips a foreign claim that is still live. A session an operator stopped (POST /stopor/force-kill), or whoseDELETEfailed part-way or died with its node, is never adopted, even when its claim did lapse, until an explicitPOST /start. While versions are mixed during an upgrade, a node still on 0.25.0 does not read aDELETE's record: it can adopt such a session, and aPOST /startit serves leaves the record in place. A stop during a delete preserves its claim, including repeated stops after its lease expires or becomes NULL. If another node takes over an expired claim, each later removal transaction checks the new owner and refuses the old delete. No transaction remains open across a hook. Successful adoption cleanup commits with its claim or expired-claim release, so delayed cleanup cannot fail a later owner's new batches. Older nodes do not preserve these delete holds, fence each removal transaction, or serialize adoption cleanup with ownership changes. Upgrade every node before relying on them; older backup importers also treat the combined delete/stop record as unknown. Older nodes also lack the final ownership check on a stop write, so they can still report a stop as successful while a peer starts the session. Upgrade every node before relying on that fence.Known limitation: a holder that is alive but cannot reach the database for more than three lease TTLs minus one heartbeat (160s at defaults) is marked disconnected by a peer too, and does not write its status back when it reconnects. Its renewal still finds its own claim, so it detects no loss, and a steadily connected engine emits no new status. The row reads
disconnecteduntil that engine's status next changes.Request routing now exists, opt-in via
NODE_URL. When every node sets its own reachable URL (e.g.NODE_URL=http://10.0.0.5:2785), a session-scoped request landing on a non-owner is forwarded to the live owner and the owner's response is relayed back (x-openwa-served-bynames it). The forward happens after API-key auth, carries the caller's credentials, is bounded bySESSION_PROXY_TIMEOUT_MS(default 60s), and is one hop only — a forwarded request is never forwarded again; one that still lands on a live non-owner (stale ownership, or a client-forged hop marker) is refused with a retryable 409 rather than executed there. A lapsed owner is deliberately NOT forwarded to: the local node handles the request, which is exactly how a takeover begins. WithoutNODE_URLthe whole path is inert and single-node deployments pay nothing.A path on
NODE_URLis kept (0.23.7 and later) and put in front of the forwarded request's own path, which already starts with/api. Give one only when the node is reached through a reverse proxy under that prefix (NODE_URL=https://gw.example.com/node-a); a node reached directly takes none. ANODE_URLending in/apiforwards to/api/api/..., and every routed request answers404.The owner authenticates a forwarded request again, against its own key store. API keys and the audit log live in each node's main SQLite file (
MAIN_DATABASE_NAME, default./data/main.sqlite), not in the shared data database, and nothing replicates them. A forwarded key must therefore also exist on the owner. The receiving node checks its own copy's role,allowedIps,allowedSessionsandallowedChatsbefore forwarding, and the owner checks its copy again; a filtered list, such as a session's chats, is filtered by the owner's copy alone. A missing or narrower copy on the owner answers401or403(or a shorter list). A broader one is accepted and widens those lists, so keep the copies identical. A key created through the API or the dashboard gets a fresh random value and exists only on the node that created it. Matching keys come only from the sameAPI_MASTER_KEYseeded on each node's first boot (it seeds only while no key exists), or from a copy of one node'smain.sqlitetaken while it is stopped (never one file shared over network storage), which authenticates only on nodes with the sameAPI_KEY_PEPPER, since the copied hashes carry the source node's pepper. Each node's audit log records only the requests it handled.List and stats routes answer from the node that received them.
GET /api/sessionsandGET /api/sessions/stats/overviewname no session, so they are never forwarded.lastError,restriction, the statsactivecount andmemoryUsageare that node's own view, not the owner's: the first two come from what this node's engines recorded,activecounts only this node's engines, andmemoryUsageis this process's.engineLoadedis the exception, since it also counts a live claim by another node. With routing on,GET /api/sessions/:sessionIdis forwarded to the owner, so its answer is the owner's;start,stop,logoutandforce-killare forwarded the same way, so atruemeansstop,logoutandforce-killcan act (andstartanswers400, as it does on the owner). WithoutNODE_URLnothing is forwarded, so a node that does not hold the session still reportstruebut cannot act on it:startandstopanswer409there, andlogoutandforce-killanswer400.A failed forward says whether the owner could have acted. When the owner cannot be reached at all (connection refused, unresolvable or unusable
NODE_URL), the answer is503and the request was not carried out, so it is safe to retry. A timeout answers504(no reply withinSESSION_PROXY_TIMEOUT_MS), and any other failure answers502(the connection broke, possibly after the request was sent; a TLS certificate the forwarding node does not trust also lands here, with the cause code in its warning log): the owner may already have carried it out, so a non-idempotent call such as a message send must not be repeated blindly on either.The lease compares timestamps written by different nodes, so their clocks must agree. Each node writes
leaseExpiresAtfrom its own clock and reads every other node's the same way. Just before each renewal the holder's lease is only one TTL minus one heartbeat ahead, so a node whose clock runs more than aboutSESSION_LEASE_TTL_MS - SESSION_LEASE_HEARTBEAT_MSahead (40s at defaults, 20s once one renewal is missed) sees healthy peers as lapsed and will take their sessions over. Run NTP (or any time sync) on every node — the default on ordinary server images — and treat any skew near that margin as a misconfiguration. The status correction reads the same timestamps: a node whose clock runs more than three TTLs minus one heartbeat ahead (160s at defaults) marks a healthy peer's sessions disconnected, even withAUTO_START_SESSIONSoff, and nothing writes them back. The zone each node runs in is no longer part of this: on PostgreSQL the data connection binds, parses and defaults every timestamp in UTC (05 - Database Design), so two nodes in different zones, or one zone that observes daylight saving, still read each other's leases as the instants they were written at. Only the clocks have to agree. One exception, during an upgrade: a node still on 0.23.5 or earlier writes the lease in its own local wall time, so while versions are mixed the old cross-zone error above is back for as long as the older node keeps renewing. Running every node inTZ=UTC(the image default) removes it.A forwarded request is throttled on both nodes. The receiving node counts it before forwarding, and the owner counts it again on arrival; with
REDIS_ENABLED=true, and each peer node listed in the owner'sTRUSTED_PROXIES(below), both counts land in the same shared bucket. Without that, the owner counts every request forwarded by a peer in one bucket keyed on that peer's address. Size the rate limits with that in mind for a routed deployment.Forwards carry the client address in
x-forwarded-for(inbound chain preserved, the observed peer appended). For anallowedIps-restricted key or the per-IP throttler to see the REAL client on forwarded calls, each node must list its peer nodes' addresses in itsTRUSTED_PROXIES— otherwise the owner (correctly) ignores the chain and every forwarded request appears to come from the peer node itself.WebSocket events now fan out across replicas when Redis is enabled (
REDIS_ENABLED=true, the same flag the throttler and cache already use). The gateway broadcasts to rooms; a Redis pub/sub adapter attached to Socket.IO relays those broadcasts to every replica, so a client connected to node A receives an event raised on node B. Scope honestly: this distributes event fan-out only. The per-key WS rate-limit buckets (counted per replica) and the engine registry are still process-local. WithoutREDIS_ENABLEDthe adapter is inert and delivery is single-node, exactly as before. With it, every event of a one-to-one chat also costs the emitting node one or twolid_mappingslookups (one for a@lidchat, none for a group), even when no chat-restricted key exists: it cannot know whether another replica holds a restricted socket, so it names the chat in every form for the rooms to match.The fan-out to other replicas is best effort and keeps no replay queue. A Redis that pauses for less than 10 s (a snapshot fork, a slow fsync) answers afterwards, and every event raised meanwhile reaches the other replicas late, at up to about 1,200 events a second; above that the cap described below applies first and drops the frames beyond it. A longer silence, a restart, a failover or an outage drops the connection a node publishes on, and what was written to it may be lost with it, or may still reach the other replicas late and after newer events, which their clients then apply out of order. The events that node raises while it is disconnected, plus its reconnect backoff of up to 5 s once Redis is back, are dropped for the clients connected to the other replicas: they are not delivered late and not twice. Clients connected to the node that raised an event are served directly, and webhooks do not use this path, so neither is affected. Memory is bounded in two ways. A Redis that sends nothing for 10 s has the connection destroyed, and the node holds nothing more for it; what it had already held, at most 10 s of events, is freed once the node has reconnected. A Redis that keeps answering, but slower than events are raised, is never silent for 10 s, so a cap bounds it instead: once the node holds 12,000 unanswered events (10 s at 1,000 events a second, with a fifth to spare) and the backlog has outlived 2 s, meaning the count has not been below 6,000 for that long, it drops further frames for the other replicas until the count is back below 6,000, with a warning when dropping starts and another, with the count, when it ends. The node can hold the traffic of those 2 s on top of the 12,000. A burst raised in a single tick is not cut as long as raising it past its 6,000th frame takes under 2 s: a healthy Redis answers it right after, and the node holds it until then. A synchronous burst that takes longer than that to raise past 6,000 frames is cut at 12,000 even against a healthy Redis, and the warning names the unanswered count. A replica whose own Redis connection is reconnecting also misses what the others publish meanwhile. The connection a replica receives on only listens, so it is pinged every 15 s and replaced when an answer takes over 30 s: one that stops answering without being closed (a proxy in the path that keeps its side open, a NAT or a firewall that drops the flow) costs that replica at most about 45 s of the others' events before it subscribes again. A Redis that pauses for up to 30 s keeps that connection, so the replica still receives late what the others had written before their own 10 s deadline; a longer silence while a ping waits replaces it, and what the others had published into that silence is then lost for that replica.
Mid-connection key eviction converges on a timer, not a broadcast, and only on one node. The node that processes a revoke, delete, or narrowing tears down that key's sockets synchronously, in the same request. Nothing is published to peers. Every node re-validates the keys behind its own live sockets against its own key store once a minute (
EventsGateway.sweepApiKeyAuthorization, one batched read of the key ids currently holding sockets) and evicts on a row that is gone, inactive, expired, or whose role,allowedIps,allowedSessions,allowedChatsor expiry no longer matches the snapshot the socket authenticated with. That catches an expiry, and a change written to the node's main database other than through its own API. Before this sweep, a revoke, delete or expiry that reached a node that way waited for the client's next subscribe, and a narrowing was never caught at all: it leaves the key valid, so only the new subscribe is rejected while every room joined earlier stays joined. REST reads the row per request, so the node that processed a change rejects the key's REST calls at once. Every other node keeps accepting the key, on REST and on its sockets, until the same change is made there: repeat a revoke, delete, expiry change or narrowing on every node.What does not exist yet, and is why one replica is still the answer. The cross-replica gap just named (WS rate-limit state) remains process-local. API keys and the audit log are per node: each node keeps them in its own main SQLite file, so a key created, revoked or narrowed on one node is unchanged on the others (see the routing paragraph above). Leases are timed by each node's own clock, not the database's (see the clock paragraph above). Not every lifecycle path is fenced: the liveness watchdog and reconnect timers still act on whatever is in the local registry.
BulkMessageServicekeeps its live batch state in process, so a takeover cannot resume a batch — only fail it. MCP/agent tool invocations execute on the node that received them rather than being forwarded.Everything below (node affinity,
replicas: 3) remains a design sketch until those land.
This guide explains a proposed design for deploying OpenWA in a horizontally scaled environment for high availability and increased capacity.
13.1 Architecture Overview
flowchart TB
subgraph LB["Load Balancer"]
NGINX[Nginx/Traefik]
end
subgraph Nodes["OpenWA Nodes"]
N1[OpenWA Node 1]
N2[OpenWA Node 2]
N3[OpenWA Node 3]
end
subgraph Storage["Shared Storage"]
PG[(PostgreSQL)]
REDIS[(Redis)]
S3[S3/MinIO<br/>Media Storage]
end
LB --> N1
LB --> N2
LB --> N3
N1 --> PG
N2 --> PG
N3 --> PG
N1 --> REDIS
N2 --> REDIS
N3 --> REDIS
N1 --> S3
N2 --> S3
N3 --> S3
Key Principles
| Principle | Description |
|---|---|
| Session Affinity | WhatsApp sessions are stateful and must stay on the same node |
| Shared Database | PostgreSQL stores all persistent data across nodes |
| Redis for State | Shared cache and queue coordination |
| Sticky Sessions | Load balancer routes session requests to the correct node |
13.2 Session Affinity Strategy
Since WhatsApp sessions maintain active connections (a browser instance for whatsapp-web.js, or a WebSocket for baileys — set via ENGINE_TYPE), they cannot be freely moved between nodes.
Strategy 1: Session-to-Node Mapping (Recommended)
This mapping ships: each sessions row records its owner in nodeId, nodeUrl, claimedAt and
leaseExpiresAt (migrations AddSessionOwnership and AddSessionNodeUrl). No load balancer reads it:
a node that receives a request for a session it does not own forwards it to the owner's NODE_URL, as
the routing paragraph at the top of this guide describes.
Strategy 2: Consistent Hashing
Route sessions based on session ID hash. (Not implemented — no such routing helper exists in code; the sketch below is illustrative of the future design.)
function getNodeForSession(sessionId: string, nodes: string[]): string {
const hash = crypto.createHash('md5').update(sessionId).digest('hex');
const index = parseInt(hash.substring(0, 8), 16) % nodes.length;
return nodes[index];
}
Strategy 3: Session Claim
Each node "claims" sessions on startup and releases them on shutdown. This is implemented: see the claim, lease and takeover description at the top of this guide.
13.3 Docker Swarm Deployment
docker-compose.swarm.yml
version: '3.8'
services:
openwa:
image: ghcr.io/rmyndharis/openwa:latest
# Teardown takes up to 63 s with every datastore on; raise this by the amount SHUTDOWN_DELAY_MS or
# WEBHOOK_SHUTDOWN_DRAIN_MS exceeds its default, or the 10 s default kills the task mid-teardown.
stop_grace_period: 75s
deploy:
replicas: 1 # MUST stay 1 until multi-replica is supported — multiple replicas on one session volume corrupt WhatsApp auth
update_config:
parallelism: 1
delay: 30s
restart_policy:
condition: on-failure
max_attempts: 3
resources:
limits:
memory: 2G
reservations:
memory: 512M
environment:
- NODE_ENV=production
- DATABASE_TYPE=postgres
- DATABASE_HOST=postgres
- DATABASE_NAME=openwa
- DATABASE_USERNAME=openwa
- DATABASE_PASSWORD=${DB_PASSWORD}
- REDIS_HOST=redis
- QUEUE_ENABLED=true
# The session-ownership identity: it names which process holds each session's engine, and
# it must be STABLE across restarts (a value that changes makes the restarted process a new
# node, which has to wait out its own previous lease). Defaults to the container hostname.
- NODE_ID={{.Node.Hostname}}-{{.Task.Slot}}
volumes:
# The whole data tree, not only sessions/: main.sqlite (API keys, audit log), baileys/,
# media and the generated secrets live there too, and a replaced task would lose them.
- openwa-data:/app/data
networks:
- openwa-net
depends_on:
- postgres
- redis
postgres:
image: postgres:16-alpine
deploy:
replicas: 1
placement:
constraints:
- node.role == manager
environment:
- POSTGRES_DB=openwa
- POSTGRES_USER=openwa
- POSTGRES_PASSWORD=${DB_PASSWORD}
volumes:
- postgres-data:/var/lib/postgresql/data
networks:
- openwa-net
redis:
image: redis:7-alpine
deploy:
replicas: 1
command: redis-server --appendonly yes --maxmemory-policy noeviction
volumes:
- redis-data:/data
networks:
- openwa-net
# NOTE (v0.4.0): OpenWA no longer ships a bundled Traefik container.
# For TLS / public exposure, bring your own reverse proxy (Traefik, nginx,
# Caddy, a cloud load balancer, etc.) and point it at openwa:2785.
# See section 13.5 for Traefik / nginx config examples.
volumes:
postgres-data:
redis-data:
openwa-data:
networks:
openwa-net:
driver: overlay
Deploy to Swarm
# Initialize swarm (if not already)
docker swarm init
# Deploy stack
docker stack deploy -c docker-compose.swarm.yml openwa
# Check status
docker service ls
docker service ps openwa_openwa
Do not scale the
openwaservice (docker service scale openwa_openwa=N). Theopenwa-datavolume above is declared with the default local driver (notexternal), so Swarm creates one per node: replicas co-located on a single node share that directory and corrupt the WhatsApp auth state, while replicas placed on other nodes each get a fresh empty volume and start an unauthenticated engine instead. Either way the deployment breaks — see the warning at the top of this guide. Scaling only becomes safe once the gaps listed at the top of this guide are closed.
13.4 Kubernetes Deployment
k8s/namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: openwa
k8s/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: openwa-config
namespace: openwa
data:
NODE_ENV: 'production'
DATABASE_TYPE: 'postgres'
DATABASE_HOST: 'postgres-service'
DATABASE_PORT: '5432'
DATABASE_NAME: 'openwa'
REDIS_HOST: 'redis-service'
REDIS_PORT: '6379'
QUEUE_ENABLED: 'true'
PORT: '2785'
k8s/secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: openwa-secrets
namespace: openwa
type: Opaque
stringData:
DATABASE_USERNAME: openwa
DATABASE_PASSWORD: your-secure-password
k8s/deployment.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: openwa
namespace: openwa
spec:
serviceName: openwa-headless # must match the headless Service declared in k8s/service.yaml
replicas: 1 # MUST stay 1 until multi-replica is supported — see the warning at the top of this guide
selector:
matchLabels:
app: openwa
template:
metadata:
labels:
app: openwa
spec:
# The teardown can take up to 63 s on a clustered replica (src/common/services/shutdown-budget.ts), so
# Kubernetes' default grace of 30 s would SIGKILL a slow stop half way. Match the shipped chart.
terminationGracePeriodSeconds: 75
# OS-level containment is defence in depth for the plugin sandbox (see docs/30-plugin-sandboxing.md):
# it limits the privileges the API process holds, not what a plugin inside that process can reach.
# Without it the API process, and every plugin loaded into it, keeps whatever the container runtime
# grants by default, so a plugin that abuses Node built-ins (fs, net) can read host files / open raw
# sockets outside the capability model. The shipped compose file runs the image read-only +
# cap_drop:ALL, and the node process runs as the non-root openwa user; the manifest below mirrors
# that so a k8s deploy is not silently weaker. By default the entrypoint starts as root to chown
# /app/data, then drops to openwa (uid/gid 997) via gosu. To run non-root from the start instead,
# add runAsNonRoot: true, runAsUser: 997 and runAsGroup: 997 here and remove the capabilities `add`
# list below; the entrypoint then skips the chown and the drop (see podSecurityContext in
# charts/openwa/values.yaml). fsGroup makes the volume writable by openwa in either mode;
# OnRootMismatch re-owns it only when its root is wrong, not every file on every mount.
securityContext:
fsGroup: 997
fsGroupChangePolicy: OnRootMismatch
containers:
- name: openwa
image: ghcr.io/rmyndharis/openwa:latest
ports:
- containerPort: 2785
name: http
envFrom:
- configMapRef:
name: openwa-config
- secretRef:
name: openwa-secrets
env:
# The session-ownership identity — see the compose example above. It must be STABLE
# across restarts, which a Deployment's pod name is NOT: use a StatefulSet (whose pod
# names are ordinal and stable) for a routed multi-node deployment, or pin a value per
# replica. A changing NODE_ID still converges (the old lease lapses and the sweep
# adopts), but every restart then costs one lease TTL of downtime for its sessions.
- name: NODE_ID
valueFrom:
fieldRef:
fieldPath: metadata.name
# Container-level hardening. readOnlyRootFilesystem requires every writable path to be an
# explicitly mounted volume — /app/data (SQLite, sessions, media, plugin storage) and /tmp
# (Chromium needs a writable HOME/XDG for whatsapp-web.js).
securityContext:
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop: ['ALL']
# The root entrypoint's chown, the gosu privilege drop, and init forwarding signals after it.
add: ['CHOWN', 'DAC_OVERRIDE', 'FOWNER', 'KILL', 'SETGID', 'SETUID']
resources:
requests:
memory: '512Mi'
cpu: '250m'
limits:
memory: '2Gi'
cpu: '1000m'
volumeMounts:
- name: openwa-data
mountPath: /app/data
- name: tmp
mountPath: /tmp
# Same probes as charts/openwa/templates/statefulset.yaml. The startupProbe holds liveness
# off during boot (migrations, connect retry, plugin load, backfills), and every timeoutSeconds is
# explicit because the 1s kubelet default is shorter than /ready's 3s database bound.
startupProbe:
httpGet:
path: /api/health/live
port: 2785
periodSeconds: 5
timeoutSeconds: 5
failureThreshold: 60
livenessProbe:
httpGet:
path: /api/health/live
port: 2785
initialDelaySeconds: 30
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
readinessProbe:
httpGet:
path: /api/health/ready
port: 2785
initialDelaySeconds: 10
periodSeconds: 5
timeoutSeconds: 5
volumes:
- name: tmp
emptyDir: {}
volumeClaimTemplates:
- metadata:
name: openwa-data
spec:
accessModes: ['ReadWriteOnce']
resources:
requests:
storage: 10Gi
k8s/service.yaml
apiVersion: v1
kind: Service
metadata:
name: openwa-service
namespace: openwa
spec:
type: ClusterIP
selector:
app: openwa
ports:
- port: 80
targetPort: 2785
name: http
---
apiVersion: v1
kind: Service
metadata:
name: openwa-headless
namespace: openwa
spec:
clusterIP: None
selector:
app: openwa
ports:
- port: 2785
name: http
k8s/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: openwa-ingress
namespace: openwa
annotations:
nginx.ingress.kubernetes.io/affinity: 'cookie'
nginx.ingress.kubernetes.io/session-cookie-name: 'openwa-session'
nginx.ingress.kubernetes.io/session-cookie-max-age: '172800'
# ingress-nginx refuses a request body over 1m by default, which base64 media sends exceed.
# Keep this at or above BODY_SIZE_LIMIT (25mb by default).
nginx.ingress.kubernetes.io/proxy-body-size: '25m'
spec:
ingressClassName: nginx
rules:
- host: openwa.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: openwa-service
port:
number: 80
tls:
- hosts:
- openwa.example.com
secretName: openwa-tls
Deploy to Kubernetes
The maintained chart is charts/openwa, a single-instance StatefulSet that
already encodes the replicaCount: 1 constraint below; prefer it over hand-applied manifests. The
raw manifests here stay for operators who do not use Helm.
# Apply the namespace first: a directory is applied in file-name order, and the other files name it
kubectl apply -f k8s/namespace.yaml
# Apply all manifests
kubectl apply -f k8s/
# Check pods
kubectl get pods -n openwa
# Check logs
kubectl logs -f statefulset/openwa -n openwa
Do not raise
replicasabove 1 (kubectl scale statefulset openwa --replicas=N). Each pod gets its own PVC, so extra replicas do not share a session directory — they each start their own unauthenticated engine, and withAUTO_START_SESSIONS=trueevery pod tries to drive the same configured sessions from the shared database. See the warning at the top of this guide.
13.5 Load Balancer Configuration
Traefik Dynamic Config
# traefik/dynamic-scaling.yml
http:
routers:
openwa:
rule: 'Host(`openwa.example.com`)'
service: openwa
middlewares:
- sticky-session
middlewares:
sticky-session:
headers:
customResponseHeaders:
X-OpenWA-Node: '{{.Node}}'
services:
openwa:
loadBalancer:
sticky:
cookie:
name: openwa_node
secure: true
httpOnly: true
servers:
- url: 'http://openwa-1:2785'
- url: 'http://openwa-2:2785'
- url: 'http://openwa-3:2785'
healthCheck:
path: /api/health
interval: 10s
timeout: 3s
Nginx Upstream Config
upstream openwa {
ip_hash; # Sticky sessions based on client IP
server openwa-1:2785 weight=1 max_fails=3 fail_timeout=30s;
server openwa-2:2785 weight=1 max_fails=3 fail_timeout=30s;
server openwa-3:2785 weight=1 max_fails=3 fail_timeout=30s;
}
server {
listen 80;
server_name openwa.example.com;
# nginx refuses a request body over 1m by default, below the 25mb OpenWA itself accepts
# (BODY_SIZE_LIMIT), so base64 media sends would answer 413 before reaching a node.
client_max_body_size 25m;
location / {
proxy_pass http://openwa;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# Session affinity cookie
proxy_cookie_path / "/; SameSite=Strict; HttpOnly";
}
location /api/health {
proxy_pass http://openwa;
proxy_connect_timeout 5s;
proxy_read_timeout 5s;
}
}
13.6 Capacity Planning
Resource Requirements per Node
| Sessions | Memory | CPU | Disk |
|---|---|---|---|
| 1-5 | 1 GB | 0.5 vCPU | 5 GB |
| 5-10 | 2 GB | 1 vCPU | 10 GB |
| 10-25 | 4 GB | 2 vCPU | 25 GB |
| 25-50 | 8 GB | 4 vCPU | 50 GB |
Scaling Guidelines
The replica count stays at 1 (see 13.3 and 13.4), so the only levers available today are
vertical — adjust the resources limits/reservations in the manifests above — or run a second
single-instance deployment with its own session volume and split sessions between them. Horizontal
scale up / scale down becomes an option only once the gaps listed at the top of this guide are closed.
On one Docker daemon, the built-in datastores Dashboard > Infrastructure starts are shared, not per
instance: their container names (openwa-postgres, openwa-redis, openwa-minio), their volumes and
the openwa-network network are fixed, so a second instance that turns on Use Built-in attaches to the
first instance's container and data. Let at most one instance on a daemon use them, and point the
others at datastores of their own with their Use Built-in toggles off.
Each instance finds a bundled container by its com.openwa.service label anywhere on the daemon. Its
status reads a running one as the datastore it uses whenever its own host for that datastore is spelled
like the bundled one: postgres, redis, minio or the container name, in any letter case, with or
without a trailing dot. A dashboard restart that then switches that datastore off or to another host
stops the first container on the daemon with that label, which can be the other instance's. So give the
other instances' datastores host names of their own and no com.openwa.* labels; a second copy of the
shipped compose file needs its own project name and container names, and its app must reach its
datastores under names other than postgres, redis and minio. An instance set up that way whose
*_BUILTIN flags are not true leaves the other instance's containers running: its dashboard counts
a container OpenWA created as its own only beside its own flag, and asks to stop none of them.
| Metric | Threshold | Action |
|---|---|---|
| CPU > 80% | 5 minutes | Raise limits.cpu (StatefulSet); the Swarm block above declares no CPU constraint, so add one |
| Memory > 85% | 5 minutes | Raise the memory limit |
| CPU < 30% | 15 minutes | Lower requests.cpu — that is what the scheduler reserves; lowering limits.cpu only tightens throttling |
| Active sessions per instance > 20 | - | Move sessions to a second instance with its own volume |
Throughput Projections
Design targets for 2 vCPU / 4GB RAM nodes, not measurements — no benchmark artifact in this repository backs any row, including the 1-node one, and multi-node operation is not implemented (see the warning at the top of this guide), so the 3- and 5-node rows could not have been run at all:
| Nodes | Sessions | Messages/sec | p95 Latency |
|---|---|---|---|
| 1 | 10 | 50 | 150ms |
| 3 | 30 | 150 | 180ms |
| 5 | 50 | 250 | 200ms |
13.7 Monitoring
Prometheus Metrics
OpenWA exports Prometheus text exposition at GET /api/metrics (openwa_* gauges and counters).
The endpoint returns 404 until METRICS_TOKEN is set, and then requires that token as a Bearer:
# prometheus/prometheus.yml
scrape_configs:
- job_name: 'openwa'
static_configs:
# Swarm service name (13.3). On Kubernetes there is no Service called `openwa` — scrape the
# pod through the headless Service instead, e.g.
# openwa-0.openwa-headless.openwa.svc.cluster.local:2785
- targets: ['openwa:2785']
metrics_path: '/api/metrics'
authorization:
type: Bearer
credentials_file: /etc/prometheus/metrics_token
# prometheus/openwa-rules.yaml
groups:
- name: openwa
rules:
- alert: HighMemoryUsage
expr: container_memory_usage_bytes{container="openwa"} > 1.8e9
for: 5m
labels:
severity: warning
annotations:
summary: 'OpenWA node high memory usage'
- alert: NodeDown
expr: up{job="openwa"} == 0
for: 1m
labels:
severity: critical
annotations:
summary: 'OpenWA node is down'
Health Check Endpoints
| Endpoint | Purpose |
|---|---|
/api/health |
Basic health check — returns status and timestamp; version only for an authenticated caller |
/api/health/live |
Liveness probe (static ok; reflects process liveness only) |
/api/health/ready |
Readiness probe — verifies the main + data databases respond (returns 503 while draining or if a DB is down) |