Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

feat(helm): carry the database CA bundle so verify-full is reachable

Petr Bauch800e996754f1
diff --git a/deploy/remote/helm/switch/templates/_helpers.tpl b/deploy/remote/helm/switch/templates/_helpers.tpl
index 308a22f3f..3baa3d9ab 100644
--- a/deploy/remote/helm/switch/templates/_helpers.tpl
+++ b/deploy/remote/helm/switch/templates/_helpers.tpl
@@ -201,6 +201,62 @@ imagePullSecrets:
{{- end }}
{{- end }}
+{{/*
+Name of the ConfigMap holding the database CA bundle — the user's own, or the
+chart-managed one rendered from postgresql.caBundle.contents. Empty when no
+bundle is configured, which is what every caller tests to decide whether to
+mount anything.
+*/}}
+{{- define "switch.dbCaBundleConfigMap" -}}
+{{- if .Values.postgresql.caBundle.existingConfigMap -}}
+{{- .Values.postgresql.caBundle.existingConfigMap -}}
+{{- else if .Values.postgresql.caBundle.contents -}}
+{{- include "switch.fullname" . }}-db-ca
+{{- end -}}
+{{- end }}
+
+{{/*
+Path the CA bundle is mounted at inside every container that connects to the
+database.
+*/}}
+{{- define "switch.dbCaBundlePath" -}}
+{{- printf "%s/%s" (trimSuffix "/" .Values.postgresql.caBundle.mountPath) .Values.postgresql.caBundle.key -}}
+{{- end }}
+
+{{- define "switch.dbCaBundleVolume" -}}
+- name: db-ca-bundle
+ configMap:
+ name: {{ include "switch.dbCaBundleConfigMap" . }}
+ items:
+ - key: {{ .Values.postgresql.caBundle.key }}
+ path: {{ .Values.postgresql.caBundle.key }}
+{{- end }}
+
+{{- define "switch.dbCaBundleVolumeMount" -}}
+- name: db-ca-bundle
+ mountPath: {{ .Values.postgresql.caBundle.mountPath }}
+ readOnly: true
+{{- end }}
+
+{{/*
+Refuse a TLS configuration that cannot work, at template time rather than on
+the first connection of a rolled-out pod.
+*/}}
+{{- define "switch.validateDbTls" -}}
+{{- $ca := .Values.postgresql.caBundle -}}
+{{- if and $ca.contents $ca.existingConfigMap -}}
+{{- fail "postgresql.caBundle: set contents or existingConfigMap, not both." -}}
+{{- end -}}
+{{- $bundle := include "switch.dbCaBundleConfigMap" . -}}
+{{- $verifying := has .Values.postgresql.sslMode (list "verify-ca" "verify-full") -}}
+{{- if and $verifying (not $bundle) -}}
+{{- fail (printf "postgresql.sslMode=%s checks the server certificate against a CA bundle, but postgresql.caBundle is empty. Supply the provider's root CA (for RDS, https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem) or use sslMode=require." .Values.postgresql.sslMode) -}}
+{{- end -}}
+{{- if and $bundle (not $verifying) -}}
+{{- fail (printf "postgresql.caBundle is set but postgresql.sslMode=%s never checks the server certificate, so the bundle would have no effect. Use verify-ca or verify-full." .Values.postgresql.sslMode) -}}
+{{- end -}}
+{{- end }}
+
{{/*
switch-core container env. Shared by the switch-core Deployment and the
pre-upgrade migration Job so they always run against the same configuration
@@ -223,6 +279,10 @@ Include with `nindent 12`.
value: {{ include "switch.postgresDatabase" . | quote }}
- name: DB_SSL_MODE
value: {{ .Values.postgresql.sslMode | quote }}
+{{- if include "switch.dbCaBundleConfigMap" . }}
+- name: DB_SSL_ROOT_CERT
+ value: {{ include "switch.dbCaBundlePath" . | quote }}
+{{- end }}
- name: DB_POOL_SIZE
value: {{ .Values.postgresql.pool.size | quote }}
- name: DB_MAX_OVERFLOW
diff --git a/deploy/remote/helm/switch/templates/common/db-ca-configmap.yaml b/deploy/remote/helm/switch/templates/common/db-ca-configmap.yaml
new file mode 100644
index 000000000..12e0af311
--- /dev/null
+++ b/deploy/remote/helm/switch/templates/common/db-ca-configmap.yaml
@@ -0,0 +1,12 @@
+{{- if and .Values.postgresql.caBundle.contents (not .Values.postgresql.caBundle.existingConfigMap) }}
+# Certificate authorities, not a secret: a CA bundle is public by construction.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+ name: {{ include "switch.fullname" . }}-db-ca
+ labels:
+ {{- include "switch.labels" . | nindent 4 }}
+data:
+ {{ .Values.postgresql.caBundle.key }}: |
+{{ .Values.postgresql.caBundle.contents | indent 4 }}
+{{- end }}
diff --git a/deploy/remote/helm/switch/templates/mattermost/deployment.yaml b/deploy/remote/helm/switch/templates/mattermost/deployment.yaml
index 078058403..f009ff521 100644
--- a/deploy/remote/helm/switch/templates/mattermost/deployment.yaml
+++ b/deploy/remote/helm/switch/templates/mattermost/deployment.yaml
@@ -42,7 +42,7 @@ spec:
name: {{ include "switch.postgresSecretName" . }}
key: {{ include "switch.postgresSecretKey" . }}
- name: MM_SQLSETTINGS_DATASOURCE
- value: "postgres://{{ include "switch.postgresUser" . }}:$(POSTGRES_PASSWORD)@{{ include "switch.postgresHost" . }}:{{ include "switch.postgresPort" . }}/mattermost?sslmode={{ .Values.postgresql.sslMode }}"
+ value: "postgres://{{ include "switch.postgresUser" . }}:$(POSTGRES_PASSWORD)@{{ include "switch.postgresHost" . }}:{{ include "switch.postgresPort" . }}/mattermost?sslmode={{ .Values.postgresql.sslMode }}{{ if include "switch.dbCaBundleConfigMap" . }}&sslrootcert={{ include "switch.dbCaBundlePath" . }}{{ end }}"
- name: MM_SERVICESETTINGS_SITEURL
value: {{ .Values.mattermost.siteUrl | default "" | quote }}
- name: MM_SERVICESETTINGS_ENABLELOCALMODE
@@ -79,10 +79,16 @@ spec:
volumeMounts:
- name: data
mountPath: /mattermost/data
+ {{- if include "switch.dbCaBundleConfigMap" . }}
+ {{- include "switch.dbCaBundleVolumeMount" . | nindent 12 }}
+ {{- end }}
resources:
{{- toYaml .Values.mattermost.resources | nindent 12 }}
volumes:
- name: data
persistentVolumeClaim:
claimName: {{ include "switch.fullname" . }}-mattermost-data
+ {{- if include "switch.dbCaBundleConfigMap" . }}
+ {{- include "switch.dbCaBundleVolume" . | nindent 8 }}
+ {{- end }}
{{- end }}
diff --git a/deploy/remote/helm/switch/templates/switch-core/deployment.yaml b/deploy/remote/helm/switch/templates/switch-core/deployment.yaml
index a84018e04..42cb0a4ae 100644
--- a/deploy/remote/helm/switch/templates/switch-core/deployment.yaml
+++ b/deploy/remote/helm/switch/templates/switch-core/deployment.yaml
@@ -1,6 +1,7 @@
{{- if ne (int .Values.switchCore.replicaCount) 1 }}
{{- fail "switchCore.replicaCount must be 1: switch-core keeps live Matrix sync sessions in memory and cannot be horizontally scaled (a second replica would duplicate every client and split session state)." }}
{{- end }}
+{{- include "switch.validateDbTls" . }}
apiVersion: apps/v1
kind: Deployment
metadata:
@@ -58,3 +59,11 @@ spec:
periodSeconds: 30
resources:
{{- toYaml .Values.switchCore.resources | nindent 12 }}
+ {{- if include "switch.dbCaBundleConfigMap" . }}
+ volumeMounts:
+ {{- include "switch.dbCaBundleVolumeMount" . | nindent 12 }}
+ {{- end }}
+ {{- if include "switch.dbCaBundleConfigMap" . }}
+ volumes:
+ {{- include "switch.dbCaBundleVolume" . | nindent 8 }}
+ {{- end }}
diff --git a/deploy/remote/helm/switch/templates/switch-core/migrate-job.yaml b/deploy/remote/helm/switch/templates/switch-core/migrate-job.yaml
index 778baae25..83ec25c36 100644
--- a/deploy/remote/helm/switch/templates/switch-core/migrate-job.yaml
+++ b/deploy/remote/helm/switch/templates/switch-core/migrate-job.yaml
@@ -36,4 +36,12 @@ spec:
command: ["alembic", "upgrade", "head"]
env:
{{- include "switch.coreEnv" . | nindent 12 }}
+ {{- if include "switch.dbCaBundleConfigMap" . }}
+ volumeMounts:
+ {{- include "switch.dbCaBundleVolumeMount" . | nindent 12 }}
+ {{- end }}
+ {{- if include "switch.dbCaBundleConfigMap" . }}
+ volumes:
+ {{- include "switch.dbCaBundleVolume" . | nindent 8 }}
+ {{- end }}
{{- end }}
diff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yaml
index ef145a04e..4ad6a6650 100644
--- a/deploy/remote/helm/switch/values.yaml
+++ b/deploy/remote/helm/switch/values.yaml
@@ -307,6 +307,27 @@ postgresql:
# managed cloud databases generally require "require" or stricter.
sslMode: disable
+ # Certificate authorities the server certificate is checked against, as a PEM
+ # bundle. The two verifying sslModes require one and the others ignore one, so
+ # the chart refuses a mismatch rather than rendering a release that fails on
+ # first connect. A managed database is signed by the provider's own root
+ # rather than a publicly trusted one, which is why the system trust store is
+ # not enough; AWS publishes a bundle covering every commercial region at
+ # https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem
+ #
+ # Supply it inline as `contents` (rendered into a chart-managed ConfigMap) or
+ # name a ConfigMap you already have — not both. It is mounted into
+ # switch-core, the migration Job and Mattermost, all of which connect.
+ #
+ # verify-full additionally checks the hostname, so external.host must be the
+ # name the certificate was issued for — for RDS the instance endpoint, not a
+ # CNAME of your own. A CNAME means settling for verify-ca.
+ caBundle:
+ contents: ""
+ existingConfigMap: ""
+ key: ca.pem
+ mountPath: /etc/ssl/switch-db
+
# switch-core's SQLAlchemy connection pool. Size this against the number of
# live agent connections, not human traffic: bearer-token auth hits the
# database on every authenticated request and each agent connection beats
diff --git a/docs/old/rds-migration.md b/docs/old/rds-migration.md
index b376d6e96..cd12bec90 100644
--- a/docs/old/rds-migration.md
+++ b/docs/old/rds-migration.md
@@ -69,19 +69,20 @@ The chart can already point at an external database:
Secret (`existingSecret` supports external-secrets). `sslMode` is plumbed
through to both switch-core and Mattermost.
-Two gaps, one closed here and one deliberately left:
-
-- **Closed:** a verifying TLS mode had no way to be told which certificate
- authority to trust, so `verify-ca` and `verify-full` fell back to the system
- trust store and could not validate an RDS certificate at all. `DB_SSL_ROOT_CERT`
- now takes a PEM bundle, and the connection is built with an SSL context
- instead of a bare mode string. A path that is not a readable file, or a
- bundle set alongside a non-verifying mode, is refused at startup rather than
- discovered on first connect.
-- **Left open:** the chart has no `DB_SSL_ROOT_CERT` and no way to mount the
- bundle into the pod. Getting to `verify-full` in a deployment therefore needs
- a chart change — an env entry plus a ConfigMap or Secret carrying the bundle.
- It is small, and it is not in this change.
+What was missing was a certificate authority. A verifying TLS mode had no way
+to be told which one to trust, so `verify-ca` and `verify-full` fell back to
+the system trust store and could not validate an RDS certificate at all. Both
+halves of that are now closed:
+
+- `DB_SSL_ROOT_CERT` takes a PEM bundle, and the connection is built from an
+ SSL context rather than a bare mode string. A path that is not a readable
+ file, or a bundle set alongside a non-verifying mode, is refused at startup
+ rather than discovered on the first connection.
+- `postgresql.caBundle` in the chart carries the bundle — inline, or the name
+ of a ConfigMap you already have — and mounts it into everything that connects:
+ switch-core, the migration Job and Mattermost. A verifying `sslMode` with no
+ bundle, or a bundle with a mode that would ignore it, fails at template time
+ rather than rolling out pods that cannot connect.
## 4. Do not put a transaction pooler in front of it
@@ -103,6 +104,10 @@ this first.
If connection count becomes the problem, the honest fix is the application's
pool size, not a pooler.
+Nothing enforces this today — it is a note, and notes are forgotten. A
+follow-up should make the constraint impossible to miss from the outside, in
+whatever provisions the database.
+
## 5. TLS
RDS publishes a global CA bundle covering every commercial region:
@@ -116,12 +121,9 @@ AWS is explicit that intermediates should not be added to a trust store. Server
certificates are rotated by RDS on their own schedule and do not need the
bundle re-downloaded.
-Then:
-
-```
-DB_SSL_MODE=verify-full
-DB_SSL_ROOT_CERT=/etc/ssl/certs/rds-global-bundle.pem
-```
+In a deployment that is `postgresql.sslMode: verify-full` plus
+`postgresql.caBundle`; outside one, the same two settings are `DB_SSL_MODE` and
+`DB_SSL_ROOT_CERT`.
**`verify-full` checks the hostname**, and the certificate carries the RDS
endpoint DNS name. Connect to `<instance>.<id>.<region>.rds.amazonaws.com`, not
@@ -168,8 +170,8 @@ a shorter outage for a fiddlier cutover.
2. Create the `switch` and `mattermost` databases and the application role. The
RDS master user is not a superuser, and `pg_dumpall` needs privileges it does
not have — so recreate roles by hand rather than restoring a globals dump.
-3. Download the CA bundle, put it where the pods will read it, and make the
- chart change from §3.
+3. Download the CA bundle and put it in `postgresql.caBundle` — as
+ `existingConfigMap` if it is synced in from elsewhere, otherwise inline.
4. **Rehearse the whole thing** against a scratch database, and time it. The
number you get is the downtime estimate; the one you guess is not.
5. Announce the window.