Skip to content

Open-source AI teammates in Slack, Teams, Discord, Telegram and Mattermost. Self-host it, with any agent harness and any model.

Deploy the decision service with one setting

agent-runner353b69ab9644f15
diff --git a/.env.example b/.env.example
index 48e43ce90..2f833405b 100644
--- a/.env.example
+++ b/.env.example
@@ -209,6 +209,15 @@ FEATURE_FLAGS_ENABLED=
# profile stops the server. Empty: full scopes.
# SERVICE_ACCESS_PROFILES='{"google-workspace": "drive-file"}'
+# ── Decision routing (auto-route) ────────────────────────────────────────────
+# The decision service rooms with auto-route on ask who should answer an
+# untagged message; the server default for every workspace that has not set
+# its own in Switch Console. Needs `follow_the_conversation,decision_routing`
+# in FEATURE_FLAGS_ENABLED. `just decision-up` runs one locally on :8230 (on
+# CPU each decision takes several seconds, hence the longer timeout).
+# DECISION_ENDPOINT_URL=http://localhost:8230
+# DECISION_TIMEOUT_SECONDS=30
+
# ── Server keys ──────────────────────────────────────────────────────────────
# Master keys every signing and encryption key is derived from:
# "<id>:<secret>" comma-separated, current first, secrets of 32+ characters.
diff --git a/.github/workflows/pr-ci.yml b/.github/workflows/pr-ci.yml
index f43b49bf9..05cb7379e 100644
--- a/.github/workflows/pr-ci.yml
+++ b/.github/workflows/pr-ci.yml
@@ -148,6 +148,8 @@ jobs:
- 'deploy/hosted/chart/**'
- 'deploy/hosted/render-check-values.yaml'
- 'scripts/pin_chart_images.py'
+ - 'scripts/release_charts.sh'
+ - 'scripts/check_chart_decision.py'
- '.github/workflows/switch-release.yml'
# Spans every tree, so it gets its own filter rather than riding
# any one of them. Includes the files whose versions the registry is
@@ -427,6 +429,11 @@ jobs:
run: |
scripts/release_charts.sh build ghcr.io/render-check 0.0.0-check "$RUNNER_TEMP/chart" --placeholder-digests
+ - name: The decision service renders as configured
+ # Off renders nothing; on runs it, points switch-core at it and admits
+ # only switch-core; an endpoint elsewhere is wired without it.
+ run: python3 scripts/check_chart_decision.py
+
gateway:
name: Gateway UI
needs: changes
diff --git a/.github/workflows/switch-release.yml b/.github/workflows/switch-release.yml
index 8461ec168..abb3597a0 100644
--- a/.github/workflows/switch-release.yml
+++ b/.github/workflows/switch-release.yml
@@ -1,7 +1,7 @@
name: switch release
-# Builds and publishes the Switch core stack — the switch-core, gateway, setup
-# and switch-hosted-controller container images, the switch and
+# Builds and publishes the Switch core stack — the switch-core, gateway, setup,
+# switch-decision and switch-hosted-controller container images, the switch and
# switch-hosted-controller Helm charts, and the standalone Docker Compose file
# (as an OCI artifact) — so adopters can deploy without building from source.
# See RELEASING.md for the full procedure.
diff --git a/RELEASING.md b/RELEASING.md
index 9a79f327c..f8b449517 100644
--- a/RELEASING.md
+++ b/RELEASING.md
@@ -1,8 +1,8 @@
# Releasing Switch
This describes how to cut a release of the **Switch core stack** — the
-`switch-core`, `gateway`, `setup` and `switch-hosted-controller` container
-images, the Helm charts, and the standalone Docker Compose file. The Switch Console desktop app releases separately
+`switch-core`, `gateway`, `setup`, `switch-decision` and
+`switch-hosted-controller` container images, the Helm charts, and the standalone Docker Compose file. The Switch Console desktop app releases separately
(see `.github/workflows/switch-console-release.yml` and `console/docs/INSTALL.md`).
## Versioning
@@ -83,7 +83,8 @@ peer still on it, and it can never be raised past what is running in the field.
5. The **`switch-release`** workflow (`.github/workflows/switch-release.yml`)
then, on that tag:
- builds multi-arch (`linux/amd64`, `linux/arm64`) images for `switch-core`,
- `gateway`, and `setup` and pushes them to the container registry tagged
+ `gateway`, `setup` and `switch-decision` (and the hosted controller) and
+ pushes them to the container registry tagged
`<version>` and `latest`;
- builds, tests and packs the agent controller at the same version and
publishes it as the `switch-agent-controller-v<version>` GitHub release
@@ -133,7 +134,7 @@ environment runs main from published, digest-pinned artifacts instead of
building from source at deploy time.
- **Separate packages.** Dev builds go to `ghcr.io/<owner>/dev/` (`dev/switch-core`,
- `dev/gateway`, `dev/setup`, `dev/switch-hosted-controller`, `dev/charts/switch`,
+ `dev/gateway`, `dev/setup`, `dev/switch-decision`, `dev/switch-hosted-controller`, `dev/charts/switch`,
`dev/charts/switch-hosted-controller`), so the release packages' tag lists
hold releases and RCs only. Nothing is ever tagged `latest`.
- **The dev charts carry a moving `main` tag.** Once both charts are pushed,
@@ -177,7 +178,9 @@ switchCore:
imagePullPolicy: IfNotPresent
```
-and the same for `gateway` and `setup`. The hosted controller chart
+and the same for `gateway`, `setup` and `decision` (the `switch-decision`
+image, rendered only with `decision.enabled`; the CI render with the optional
+workloads on checks its pin). The hosted controller chart
(`charts/switch-hosted-controller`) is pinned through `image.repository` and
`image.digest` instead, so its image renders as
`ghcr.io/<owner>/switch-hosted-controller@sha256:…`, by digest with no tag.
diff --git a/core/tests/test_pin_chart_images.py b/core/tests/test_pin_chart_images.py
index 2406addb7..e87b156cc 100644
--- a/core/tests/test_pin_chart_images.py
+++ b/core/tests/test_pin_chart_images.py
@@ -26,9 +26,10 @@ DIGESTS = {
"switch-core": "sha256:" + "a" * 64,
"gateway": "sha256:" + "b" * 64,
"setup": "sha256:" + "c" * 64,
+ "switch-decision": "sha256:" + "e" * 64,
"switch-hosted-controller": "sha256:" + "f" * 64,
}
-SWITCH_IMAGES = ("switch-core", "gateway", "setup")
+SWITCH_IMAGES = ("switch-core", "gateway", "setup", "switch-decision")
def _load(name: str) -> ModuleType:
@@ -108,6 +109,10 @@ def test_pins_every_first_party_image_in_the_real_values_file() -> None:
)
assert pinned["gateway"]["image"] == f"gateway:1.2.3@{DIGESTS['gateway']}"
assert pinned["setup"]["image"] == f"setup:1.2.3@{DIGESTS['setup']}"
+ assert (
+ pinned["decision"]["image"]
+ == f"switch-decision:1.2.3@{DIGESTS['switch-decision']}"
+ )
def test_changes_only_the_pinned_lines() -> None:
@@ -119,7 +124,7 @@ def test_changes_only_the_pinned_lines() -> None:
for a, b in zip(original.splitlines(), pinned.splitlines(), strict=True)
if a != b
]
- assert len(changed) == 4 # the registry, then one image line per image
+ assert len(changed) == 5 # the registry, then one image line per image
# Pull policy is left to the chart, which derives it from the digest.
assert yaml.safe_load(pinned)["switchCore"]["imagePullPolicy"] == ""
@@ -237,6 +242,19 @@ def test_verify_rejects_a_render_missing_an_image() -> None:
pin_chart_images.verify("switch", rendered, "ghcr.io/acme")
+def test_verify_accepts_a_render_without_an_optional_image_but_covered_does_not() -> (
+ None
+):
+ refs = pin_chart_images.verify("switch", _render(), "ghcr.io/acme")
+ with pytest.raises(pin_chart_images.PinError, match="switch-decision"):
+ pin_chart_images.covered("switch", "\n".join(refs))
+
+ with_decision = pin_chart_images.verify(
+ "switch", _render(_pinned("switch-decision")), "ghcr.io/acme"
+ )
+ pin_chart_images.covered("switch", "\n".join([*refs, *with_decision]))
+
+
def test_record_lists_every_image_and_chart() -> None:
charts = {
"switch": "sha256:" + "e" * 64,
diff --git a/deploy/local/docker-compose.yml b/deploy/local/docker-compose.yml
index 2fe8cbb2d..42b007b4b 100644
--- a/deploy/local/docker-compose.yml
+++ b/deploy/local/docker-compose.yml
@@ -134,6 +134,134 @@ services:
- "${SWITCH_BIND_ADDR:-127.0.0.1}:1025:1025"
- "${SWITCH_BIND_ADDR:-127.0.0.1}:8025:8025"
+ # ── Decision service (auto-route) ───────────────────────────────────────────
+ # `just decision-up`, or `--profile decision` (CPU, any machine) or
+ # `--profile decision-gpu` (Linux with an NVIDIA GPU and the NVIDIA Container
+ # Toolkit). Either starts the decision service on :8230 and a llama.cpp
+ # server for its model, with the weights downloaded once into the
+ # `decision-models` volume. switch-core runs on the host, so point it at the
+ # service in .env (see .env.example, "Decision routing"):
+ # DECISION_ENDPOINT_URL=http://localhost:8230
+ # DECISION_TIMEOUT_SECONDS=30
+ # On CPU each decision takes several seconds (far over the one-second target
+ # a GPU meets), hence the longer timeout; fine for trying auto-route out.
+ decision-model-download:
+ image: curlimages/curl:8.11.1
+ profiles: ["decision", "decision-gpu"]
+ # The fresh named volume is root's.
+ user: "0:0"
+ environment:
+ MODEL_URL: ${DECISION_MODEL_URL:-https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/resolve/main/Qwen3.5-4B-Q4_K_M.gguf}
+ MODEL_FILE: ${DECISION_MODEL_FILE:-Qwen3.5-4B-Q4_K_M.gguf}
+ # Only for a gated model (one whose licence your Hugging Face account
+ # has accepted).
+ HF_TOKEN: ${HF_TOKEN:-}
+ entrypoint:
+ - sh
+ - -c
+ - |
+ set -eu
+ target="/models/$$MODEL_FILE"
+ if [ -s "$$target" ]; then echo "have $$MODEL_FILE"; exit 0; fi
+ echo "downloading $$MODEL_URL"
+ set -- -fsSL --retry 5 --retry-delay 10 -C - -o "$$target.partial"
+ if [ -n "$$HF_TOKEN" ]; then set -- "$$@" -H "Authorization: Bearer $$HF_TOKEN"; fi
+ curl "$$@" "$$MODEL_URL"
+ mv "$$target.partial" "$$target"
+ volumes:
+ - decision-models:/models
+
+ decision-model:
+ image: ghcr.io/ggml-org/llama.cpp:server
+ profiles: ["decision"]
+ command:
+ - -m
+ - /models/${DECISION_MODEL_FILE:-Qwen3.5-4B-Q4_K_M.gguf}
+ - --host
+ - 0.0.0.0
+ - --port
+ - "8300"
+ - -c
+ - "8192"
+ - --jinja
+ volumes:
+ - decision-models:/models:ro
+ healthcheck:
+ test: ["CMD", "curl", "-fs", "http://localhost:8300/health"]
+ interval: 5s
+ timeout: 3s
+ retries: 60
+ start_period: 10s
+ depends_on:
+ decision-model-download:
+ condition: service_completed_successfully
+
+ decision-model-gpu:
+ image: ghcr.io/ggml-org/llama.cpp:server-cuda
+ profiles: ["decision-gpu"]
+ command:
+ - -m
+ - /models/${DECISION_MODEL_FILE:-Qwen3.5-4B-Q4_K_M.gguf}
+ - --host
+ - 0.0.0.0
+ - --port
+ - "8300"
+ - -ngl
+ - "99"
+ - -c
+ - "8192"
+ - --jinja
+ # The decision service reaches whichever model server runs at one name.
+ networks:
+ default:
+ aliases: [decision-model]
+ volumes:
+ - decision-models:/models:ro
+ deploy:
+ resources:
+ reservations:
+ devices:
+ - driver: nvidia
+ count: 1
+ capabilities: [gpu]
+ healthcheck:
+ test: ["CMD", "curl", "-fs", "http://localhost:8300/health"]
+ interval: 5s
+ timeout: 3s
+ retries: 60
+ start_period: 10s
+ depends_on:
+ decision-model-download:
+ condition: service_completed_successfully
+
+ decision:
+ build:
+ context: decision
+ dockerfile: Dockerfile
+ image: switch-decision:local
+ profiles: ["decision", "decision-gpu"]
+ ports:
+ - "${SWITCH_BIND_ADDR:-127.0.0.1}:${DECISION_PORT:-8230}:8230"
+ environment:
+ DECISION_BACKEND: llamacpp
+ DECISION_MODEL: ${DECISION_MODEL:-qwen3.5-4b-q4_k_m}
+ LLAMACPP_URL: http://decision-model:8300
+ # A CPU decision takes seconds; the service's own model-server timeout.
+ DECISION_TIMEOUT_S: "60"
+ healthcheck:
+ test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8230/health', timeout=3)"]
+ interval: 5s
+ timeout: 5s
+ retries: 30
+ depends_on:
+ decision-model:
+ condition: service_healthy
+ required: false
+ decision-model-gpu:
+ condition: service_healthy
+ required: false
+
volumes:
pgdata:
mmdata:
+ decision-models:
diff --git a/deploy/remote/helm/render-check-optional-values.yaml b/deploy/remote/helm/render-check-optional-values.yaml
index b590806d8..8e303138e 100644
--- a/deploy/remote/helm/render-check-optional-values.yaml
+++ b/deploy/remote/helm/render-check-optional-values.yaml
@@ -4,6 +4,10 @@
mattermost:
enabled: true
switchCore:
+ featureFlags:
+ enabled:
+ - follow_the_conversation
+ - decision_routing
migrations:
preUpgradeJob: true
teamsBridge:
@@ -15,3 +19,11 @@ switchCore:
datadog: "Not offered on this server."
serviceAccessProfiles:
google-workspace: drive-file
+decision:
+ enabled: true
+ bedrock:
+ enabled: true
+ modelServer:
+ model:
+ hfTokenSecret:
+ name: render-check-hf-token
diff --git a/deploy/remote/helm/switch/README.md b/deploy/remote/helm/switch/README.md
index ca6cef188..0939b99d4 100644
--- a/deploy/remote/helm/switch/README.md
+++ b/deploy/remote/helm/switch/README.md
@@ -266,6 +266,60 @@ switchCore:
An unknown service or profile stops switch-core at startup.
+## Auto-route and the decision service
+
+Off by default. With the `decision_routing` feature flag on (it needs
+`follow_the_conversation` too), a room with auto-route on asks a decision
+service who should answer a message that names nobody and that no follow-up
+rule delivered. The chart can run that service itself:
+
+```yaml
+switchCore:
+ featureFlags:
+ enabled: ["follow_the_conversation", "decision_routing"]
+decision:
+ enabled: true
+ nodeSelector:
+ nvidia.com/gpu.present: "true" # whatever labels your GPU nodes carry
+```
+
+With `decision.enabled` the chart renders one Deployment (the `decision`
+container, plus the `model-server` sidecar serving the model), a ClusterIP
+Service on port 8230, a PVC the weights are downloaded into at first start, a
+ServiceAccount, and a NetworkPolicy that admits only this release's switch-core
+pods to port 8230. It sets `DECISION_ENDPOINT_URL` on switch-core to the
+Service, so every workspace's Decision routing settings in Switch Console show
+it as the server default with no further setup.
+
+- **GPU.** The default model (Qwen3.5-4B, 4-bit, on llama.cpp's CUDA server)
+ requests `nvidia.com/gpu: 1`, so the cluster needs the NVIDIA device plugin
+ and a GPU node. The pod tolerates the usual `nvidia.com/gpu` taint; set
+ `decision.nodeSelector` (and `decision.tolerations`) for your nodes. One
+ NVIDIA L4 answers in well under a second.
+- **The model.** `decision.modelServer.image`, `.args` and `.model` (a download
+ URL, the file name, an optional SHA-256, and an optional Secret with a
+ Hugging Face token for a gated model) are all values, so another server with
+ the same API can be swapped in. `decision.model` is the name the service
+ reports. The image is pinned like the chart's other first-party images.
+- **Haiku on Bedrock.** `decision.bedrock.enabled` lets workspaces choose Claude
+ Haiku on Amazon Bedrock (`DECISION_BEDROCK_ENABLED` on the service and on
+ switch-core). The service calls Bedrock with the pod's credentials: give it
+ a role through `decision.serviceAccount.annotations` (on EKS,
+ `eks.amazonaws.com/role-arn` for an IAM role allowing `bedrock:InvokeModel`
+ on Haiku). Without a GPU, set `decision.modelServer.enabled: false` and
+ `decision.backend` to `anthropic` or `bedrock`.
+- **Other settings.** `decision.env` passes anything else in the service's
+ environment (`DECISION_TIMEOUT_S`, `DECISION_HISTORY_LIMIT`, …).
+- **The NetworkPolicy** needs a CNI that enforces it. Turn it off with
+ `decision.networkPolicy.enabled: false` only if something else restricts who
+ reaches the service: it has no key of its own in this setup.
+
+To use a decision service run elsewhere instead, leave `decision.enabled` off
+and set `switchCore.decision.endpointUrl` (and `bedrockEnabled` if that service
+allows Bedrock). Setting both is refused at render time. With neither,
+auto-route is unavailable until a workspace admin sets up a backend in Switch
+Console. `switchCore.decision.timeoutSeconds` sets `DECISION_TIMEOUT_SECONDS`.
+
## Agent management
Managed agents, and the agent controllers that run them on users' machines, are
diff --git a/deploy/remote/helm/switch/templates/_helpers.tpl b/deploy/remote/helm/switch/templates/_helpers.tpl
index 389412560..975bc2b0f 100644
--- a/deploy/remote/helm/switch/templates/_helpers.tpl
+++ b/deploy/remote/helm/switch/templates/_helpers.tpl
@@ -377,6 +377,47 @@ Other service hostnames (used in env vars and init containers).
{{- include "switch.fullname" . }}-mattermost
{{- end }}
+{{- define "switch.decisionHost" -}}
+{{- include "switch.fullname" . }}-decision
+{{- end }}
+
+{{/*
+DECISION_ENDPOINT_URL for switch-core: the chart's own decision service when
+decision.enabled, else switchCore.decision.endpointUrl (empty: none, and
+auto-route is unavailable until a workspace admin sets a backend up).
+*/}}
+{{- define "switch.decisionEndpointUrl" -}}
+{{- if .Values.decision.enabled -}}
+{{- if .Values.switchCore.decision.endpointUrl -}}
+{{- fail (printf "decision.enabled runs the decision service in this release and points switch-core at it, but switchCore.decision.endpointUrl is also set (%q). Clear one of them." .Values.switchCore.decision.endpointUrl) -}}
+{{- end -}}
+{{- printf "http://%s:%v" (include "switch.decisionHost" .) .Values.decision.port -}}
+{{- else -}}
+{{- .Values.switchCore.decision.endpointUrl -}}
+{{- end -}}
+{{- end }}
+
+{{/*
+Whether workspaces may choose Haiku on Bedrock (DECISION_BEDROCK_ENABLED).
+Non-empty for yes, nothing for no.
+*/}}
+{{- define "switch.decisionBedrockEnabled" -}}
+{{- if or .Values.switchCore.decision.bedrockEnabled (and .Values.decision.enabled .Values.decision.bedrock.enabled) -}}
+{{- if not (include "switch.decisionEndpointUrl" .) -}}
+{{- fail "switchCore.decision.bedrockEnabled needs a decision service to run Haiku on Bedrock in: set decision.enabled or switchCore.decision.endpointUrl." -}}
+{{- end -}}
+true
+{{- end -}}
+{{- end }}
+
+{{/*
+Decision pod selector labels, shared by its Deployment, Service and
+NetworkPolicy.
+*/}}
+{{- define "switch.decisionSelectorLabels" -}}
+{{- include "switch.selectorLabels" (dict "Release" .Release "component" "decision") }}
+{{- end }}
+
{{/*
Public origin of the Teams bridge listener: the value the bridge's
public_base_url must be set to.
@@ -853,6 +894,21 @@ this one. Drop it once the oldest supported image reads ID_SERVER_NAME. */}}
- name: FOLLOW_THE_CONVERSATION_WINDOW_SECONDS
value: {{ .Values.switchCore.followTheConversation.windowSeconds | quote }}
{{- end }}
+{{- if and (has "decision_routing" .Values.switchCore.featureFlags.enabled) (not (has "follow_the_conversation" .Values.switchCore.featureFlags.enabled)) }}
+{{- fail "The decision_routing feature flag needs follow_the_conversation in switchCore.featureFlags.enabled as well; switch-core refuses to start without it." }}
+{{- end }}
+{{- with include "switch.decisionEndpointUrl" . }}
+- name: DECISION_ENDPOINT_URL
+ value: {{ . | quote }}
+{{- end }}
+{{- if include "switch.decisionBedrockEnabled" . }}
+- name: DECISION_BEDROCK_ENABLED
+ value: "true"
+{{- end }}
+{{- with .Values.switchCore.decision.timeoutSeconds }}
+- name: DECISION_TIMEOUT_SECONDS
+ value: {{ . | quote }}
+{{- end }}
{{- if not (or .Values.gateway.enabled .Values.switchCore.serveDashboard) }}
{{- fail "gateway.enabled and switchCore.serveDashboard are both off, so nothing serves the operator dashboard. Turn one of them on." }}
{{- end }}
@@ -1137,3 +1193,14 @@ Usage: {{ include "switch.waitForHttp" (dict "name" "gateway" "url" "http://host
image: busybox:1.36
command: ["sh", "-c", "until wget -q --spider {{ .url }}; do echo 'waiting for {{ .name }}...'; sleep 2; done"]
{{- end }}
+
+{{/*
+The weights file name inside /models: decision.modelServer.model.file.
+*/}}
+{{- define "switch.decisionModelFile" -}}
+{{- $file := required "decision.modelServer.model.file is required" .Values.decision.modelServer.model.file -}}
+{{- if contains "/" $file -}}
+{{- fail (printf "decision.modelServer.model.file must be a file name, not a path: %q" $file) -}}
+{{- end -}}
+{{- $file -}}
+{{- end }}
diff --git a/deploy/remote/helm/switch/templates/decision/deployment.yaml b/deploy/remote/helm/switch/templates/decision/deployment.yaml
new file mode 100644
index 000000000..c41b980c7
--- /dev/null
+++ b/deploy/remote/helm/switch/templates/decision/deployment.yaml
@@ -0,0 +1,213 @@
+{{- if .Values.decision.enabled }}
+{{- $d := .Values.decision }}
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: {{ include "switch.decisionHost" . }}
+ labels:
+ {{- include "switch.labels" . | nindent 4 }}
+ {{- include "switch.decisionSelectorLabels" . | nindent 4 }}
+spec:
+ replicas: 1
+ # One GPU and a ReadWriteOnce weights volume: the old pod lets go of both
+ # before the new one starts.
+ strategy:
+ type: Recreate
+ selector:
+ matchLabels:
+ {{- include "switch.decisionSelectorLabels" . | nindent 6 }}
+ template:
+ metadata:
+ labels:
+ {{- include "switch.labels" . | nindent 8 }}
+ {{- include "switch.decisionSelectorLabels" . | nindent 8 }}
+ spec:
+ {{- if .Values.global.imagePullSecrets }}
+ {{- include "switch.imagePullSecrets" . | nindent 6 }}
+ {{- end }}
+ serviceAccountName: {{ include "switch.decisionHost" . }}
+ securityContext:
+ # The decision image's user; the downloader writes the weights as it.
+ fsGroup: 10001
+ fsGroupChangePolicy: OnRootMismatch
+ seccompProfile:
+ type: RuntimeDefault
+ {{- with $d.nodeSelector }}
+ nodeSelector:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- with $d.tolerations }}
+ tolerations:
+ {{- toYaml . | nindent 8 }}
+ {{- end }}
+ {{- if $d.modelServer.enabled }}
+ initContainers:
+ # Downloads the weights into the volume unless they are already there,
+ # resuming a download a restart interrupted.
+ - name: download-model
+ image: {{ $d.modelServer.model.downloaderImage }}
+ command: ["sh", "-c"]
+ args:
+ - |
+ set -eu
+ target="/models/$MODEL_FILE"
+ if [ -s "$target" ]; then
+ echo "have $MODEL_FILE"
+ else
+ echo "downloading $MODEL_URL"
+ set -- -fsSL --retry 5 --retry-delay 10 -C - -o "$target.partial"
+ if [ -n "${HF_TOKEN:-}" ]; then
+ set -- "$@" -H "Authorization: Bearer $HF_TOKEN"
+ fi
+ curl "$@" "$MODEL_URL"
+ mv "$target.partial" "$target"
+ fi
+ if [ -n "$MODEL_SHA256" ]; then
+ echo "$MODEL_SHA256 $target" | sha256sum -c -
+ fi
+ env:
+ - name: MODEL_URL
+ value: {{ required "decision.modelServer.model.url is required" $d.modelServer.model.url | quote }}
+ - name: MODEL_FILE
+ value: {{ include "switch.decisionModelFile" . | quote }}
+ - name: MODEL_SHA256
+ value: {{ $d.modelServer.model.sha256 | quote }}
+ {{- with $d.modelServer.model.hfTokenSecret.name }}
+ - name: HF_TOKEN
+ valueFrom:
+ secretKeyRef:
+ name: {{ . }}
+ key: {{ $d.modelServer.model.hfTokenSecret.key }}
+ {{- end }}
+ securityContext:
+ runAsNonRoot: true
+ runAsUser: 10001
+ runAsGroup: 10001
+ allowPrivilegeEscalation: false
+ readOnlyRootFilesystem: true
+ capabilities:
+ drop: ["ALL"]
+ resources:
+ requests:
+ memory: "64Mi"
+ cpu: "100m"
+ limits:
+ memory: "256Mi"
+ volumeMounts:
+ - name: weights
+ mountPath: /models
+ {{- end }}
+ containers:
+ - name: decision
+ image: {{ include "switch.image" (dict "global" .Values.global "image" $d.image) }}
+ imagePullPolicy: {{ include "switch.imagePullPolicy" (dict "policy" $d.imagePullPolicy "image" $d.image) }}
+ ports:
+ - name: http
+ containerPort: {{ $d.port }}
+ protocol: TCP
+ env:
+ - name: PORT
+ value: {{ $d.port | quote }}
+ - name: DECISION_BACKEND
+ value: {{ $d.backend | quote }}
+ {{- with $d.model }}
+ - name: DECISION_MODEL
+ value: {{ . | quote }}
+ {{- end }}
+ {{- if $d.modelServer.enabled }}
+ - name: LLAMACPP_URL
+ value: "http://127.0.0.1:{{ $d.modelServer.port }}"
+ {{- end }}
+ {{- with $d.confidenceThreshold }}
+ - name: DECISION_CONFIDENCE_THRESHOLD
+ value: {{ . | quote }}
+ {{- end }}
+ - name: DECISION_BEDROCK_ENABLED
+ value: {{ ternary "1" "0" $d.bedrock.enabled | quote }}
+ {{- with $d.bedrock.model }}
+ - name: DECISION_BEDROCK_MODEL
+ value: {{ . | quote }}
+ {{- end }}
+ {{- with $d.bedrock.region }}
+ - name: AWS_REGION
+ value: {{ . | quote }}
+ {{- end }}
+ {{- with $d.env }}
+ {{- toYaml . | nindent 12 }}
+ {{- end }}
+ # /health is 503 while the model server is down or still loading, so
+ # the Service sends nothing to a pod that cannot answer.
+ readinessProbe:
+ httpGet:
+ path: /health
+ port: http
+ periodSeconds: 10
+ failureThreshold: 3
+ # Not /health: a model server still loading is no reason to restart
+ # this container.
+ livenessProbe:
+ tcpSocket:
+ port: http
+ initialDelaySeconds: 10
+ periodSeconds: 20
+ securityContext:
+ runAsNonRoot: true
+ allowPrivilegeEscalation: false
+ readOnlyRootFilesystem: true
+ capabilities:
+ drop: ["ALL"]
+ resources:
+ {{- toYaml $d.resources | nindent 12 }}
+ volumeMounts:
+ - name: tmp
+ mountPath: /tmp
+ {{- if $d.modelServer.enabled }}
+ - name: model-server
+ image: {{ $d.modelServer.image }}
+ imagePullPolicy: {{ $d.modelServer.imagePullPolicy }}
+ {{- with $d.modelServer.args }}
+ args:
+ {{- range . }}
+ - {{ . | quote }}
+ {{- end }}
+ {{- end }}
+ env:
+ - name: MODEL_PATH
+ value: "/models/{{ include "switch.decisionModelFile" . }}"
+ ports:
+ - name: model
+ containerPort: {{ $d.modelServer.port }}
+ protocol: TCP
+ # llama.cpp's /health is 503 until the model is loaded.
+ startupProbe:
+ httpGet:
+ path: /health
+ port: model
+ periodSeconds: 5
+ failureThreshold: 120
+ livenessProbe:
+ httpGet:
+ path: /health
+ port: model
+ periodSeconds: 20
+ failureThreshold: 3
+ securityContext:
+ allowPrivilegeEscalation: false
+ capabilities:
+ drop: ["ALL"]
+ resources:
+ {{- toYaml $d.modelServer.resources | nindent 12 }}
+ volumeMounts:
+ - name: weights
+ mountPath: /models
+ readOnly: true
+ {{- end }}
+ volumes:
+ - name: tmp
+ emptyDir: {}
+ {{- if $d.modelServer.enabled }}
+ - name: weights
+ persistentVolumeClaim:
+ claimName: {{ include "switch.decisionHost" . }}-weights
+ {{- end }}
+{{- end }}
diff --git a/deploy/remote/helm/switch/templates/decision/networkpolicy.yaml b/deploy/remote/helm/switch/templates/decision/networkpolicy.yaml
new file mode 100644
index 000000000..55c330c3d
--- /dev/null
+++ b/deploy/remote/helm/switch/templates/decision/networkpolicy.yaml
@@ -0,0 +1,27 @@
+{{- if and .Values.decision.enabled .Values.decision.networkPolicy.enabled }}
+# Only this release's switch-core reaches the decision service, and only on
+# its port: no other pod, and not the model server's port. Egress is left
+# open: the pod downloads its weights and, for Haiku, calls Anthropic or
+# Bedrock.
+apiVersion: networking.k8s.io/v1
+kind: NetworkPolicy
+metadata:
+ name: {{ include "switch.decisionHost" . }}
+ labels:
+ {{- include "switch.labels" . | nindent 4 }}
+ {{- include "switch.decisionSelectorLabels" . | nindent 4 }}
+spec:
+ podSelector:
+ matchLabels:
+ {{- include "switch.decisionSelectorLabels" . | nindent 6 }}
+ policyTypes:
+ - Ingress
+ ingress:
+ - from:
+ - podSelector:
+ matchLabels:
+ {{- include "switch.selectorLabels" (dict "Release" .Release "component" "switch-core") | nindent 14 }}
+ ports:
+ - protocol: TCP
+ port: {{ .Values.decision.port }}
+{{- end }}
diff --git a/deploy/remote/helm/switch/templates/decision/pvc.yaml b/deploy/remote/helm/switch/templates/decision/pvc.yaml
new file mode 100644
index 000000000..fd2627c17
--- /dev/null
+++ b/deploy/remote/helm/switch/templates/decision/pvc.yaml
@@ -0,0 +1,18 @@
+{{- if and .Values.decision.enabled .Values.decision.modelServer.enabled }}
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+ name: {{ include "switch.decisionHost" . }}-weights
+ labels:
+ {{- include "switch.labels" . | nindent 4 }}
+ {{- include "switch.decisionSelectorLabels" . | nindent 4 }}
+spec:
+ accessModes:
+ - ReadWriteOnce
+ {{- with (.Values.decision.persistence.storageClass | default .Values.global.storageClass) }}
+ storageClassName: {{ . }}
+ {{- end }}
+ resources:
+ requests:
+ storage: {{ .Values.decision.persistence.size }}
+{{- end }}
diff --git a/deploy/remote/helm/switch/templates/decision/service.yaml b/deploy/remote/helm/switch/templates/decision/service.yaml
new file mode 100644
index 000000000..e443d1127
--- /dev/null
+++ b/deploy/remote/helm/switch/templates/decision/service.yaml
@@ -0,0 +1,20 @@
+{{- if .Values.decision.enabled }}
+# Only the decision port; the model server is reached on localhost inside the
+# pod and is not exposed.
+apiVersion: v1
+kind: Service
+metadata:
+ name: {{ include "switch.decisionHost" . }}
+ labels:
+ {{- include "switch.labels" . | nindent 4 }}
+ {{- include "switch.decisionSelectorLabels" . | nindent 4 }}
+spec:
+ type: ClusterIP
+ ports:
+ - port: {{ .Values.decision.port }}
+ targetPort: http
+ protocol: TCP
+ name: http
+ selector:
+ {{- include "switch.decisionSelectorLabels" . | nindent 4 }}
+{{- end }}
diff --git a/deploy/remote/helm/switch/templates/decision/serviceaccount.yaml b/deploy/remote/helm/switch/templates/decision/serviceaccount.yaml
new file mode 100644
index 000000000..c0ecf2ceb
--- /dev/null
+++ b/deploy/remote/helm/switch/templates/decision/serviceaccount.yaml
@@ -0,0 +1,15 @@
+{{- if .Values.decision.enabled }}
+# Its own account, so an IAM role for Bedrock (decision.serviceAccount
+# .annotations) is given to the decision pod and nothing else.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+ name: {{ include "switch.decisionHost" . }}
+ labels:
+ {{- include "switch.labels" . | nindent 4 }}
+ {{- include "switch.decisionSelectorLabels" . | nindent 4 }}
+ {{- with .Values.decision.serviceAccount.annotations }}
+ annotations:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+{{- end }}
diff --git a/deploy/remote/helm/switch/values.yaml b/deploy/remote/helm/switch/values.yaml
index 171c6ab76..1c6bcac24 100644
--- a/deploy/remote/helm/switch/values.yaml
+++ b/deploy/remote/helm/switch/values.yaml
@@ -422,6 +422,22 @@ switchCore:
# people mention the agent again. FOLLOW_THE_CONVERSATION_WINDOW_SECONDS,
# 1 to 3600.
windowSeconds: 300
+ # Auto-route's server default: the decision service switch-core asks who
+ # should answer an untagged message in a room with auto-route on (the
+ # "decision_routing" feature flag, featureFlags.enabled below). Every
+ # workspace that has not chosen its own backend in Switch Console's Decision
+ # routing settings uses it. With decision.enabled (below) the chart runs the
+ # service itself and points switch-core at it; leave endpointUrl empty then.
+ # Set endpointUrl instead to use a decision service run elsewhere.
+ decision:
+ # DECISION_ENDPOINT_URL, e.g. http://switch-decision.decision.svc:8230.
+ endpointUrl: ""
+ # DECISION_BEDROCK_ENABLED: workspaces may choose Claude Haiku on Amazon
+ # Bedrock, which runs inside the decision service on its own AWS
+ # credentials. With decision.enabled, decision.bedrock.enabled sets this.
+ bedrockEnabled: false
+ # DECISION_TIMEOUT_SECONDS; empty keeps switch-core's default.
+ timeoutSeconds: ""
# The agent controller release switch-core tells every controller to run
# (CONTROLLER_RELEASE_VERSION / _URL / _SHA256), except Switch Console's own,
# which updates with Console. A controller that runs
@@ -450,7 +466,7 @@ switchCore:
# them while the server runs. An unknown flag stops switch-core at startup.
# Known flags: ecosystem.show_owners, agent_management, switch_cloud,
# hosted_agents, service_connections, agent_chats, follow_the_conversation,
- # switch_buddy, buddy_slack_handoff.
+ # switch_buddy, buddy_slack_handoff, decision_routing.
featureFlags:
enabled: []
# Catalog services turned off on this server, each with the reason people
@@ -841,6 +857,119 @@ mattermost:
memory: "512Mi"
cpu: "500m"
+# ── Decision service (auto-route) ─────────────────────────────────────────────
+# The service switch-core asks who should answer an untagged message in a room
+# with auto-route on. Off by default. When on, the chart runs one pod with two
+# containers, points switch-core at it (DECISION_ENDPOINT_URL; leave
+# switchCore.decision.endpointUrl empty) and admits only switch-core to it:
+#
+# decision the switch-decision image: POST /decide and GET /health on
+# `port`. CPU only. The Service exposes this port and no other.
+# model-server the self-hosted decision model (llama.cpp's server by
+# default), reached by the decision container on localhost. Its
+# weights are downloaded at first start into a PVC, never baked
+# into an image.
+#
+# The default model needs one NVIDIA GPU (an L4 answers in well under a
+# second): the model server requests nvidia.com/gpu, so the cluster needs the
+# NVIDIA device plugin and a GPU node; point nodeSelector/tolerations at it.
+# Turn on the "decision_routing" and "follow_the_conversation" feature flags
+# (switchCore.featureFlags.enabled) for auto-route to be offered in rooms.
+decision:
+ enabled: false
+ image: switch-decision:latest
+ # Empty: IfNotPresent when the image is pinned by digest (as in a published
+ # chart), Always otherwise. Set it to override.
+ imagePullPolicy: ""
+ port: 8230
+ # DECISION_BACKEND: llamacpp reads the model server's probabilities over the
+ # candidate agents. systemone, anthropic and bedrock are the others; with
+ # anthropic or bedrock, turn modelServer.enabled off and pass their settings
+ # in `env`.
+ backend: llamacpp
+ # DECISION_MODEL, the name /health reports and the decision log records.
+ model: qwen3.5-4b-q4_k_m
+ # DECISION_CONFIDENCE_THRESHOLD; empty keeps the service's default (0.6).
+ confidenceThreshold: ""
+ # Claude Haiku on Amazon Bedrock as a per-workspace choice. Sets
+ # DECISION_BEDROCK_ENABLED on the service and on switch-core. Bedrock is
+ # called with the pod's AWS credentials: on EKS, an IAM role allowing
+ # bedrock:InvokeModel on Haiku, through serviceAccount.annotations.
+ bedrock:
+ enabled: false
+ # DECISION_BEDROCK_MODEL and AWS_REGION; empty keeps the service's default.
+ model: ""
+ region: ""
+ # Extra container env for the decision container (DECISION_TIMEOUT_S,
+ # DECISION_HISTORY_LIMIT, SYSTEMONE_URL, ...), as Kubernetes env entries.
+ env: []
+ resources:
+ requests:
+ memory: "256Mi"
+ cpu: "250m"
+ limits:
+ memory: "1Gi"
+ cpu: "1000m"
+ # The model server, a sidecar in the decision pod. Any server with llama.cpp's
+ # OpenAI-compatible API and GET /health works; swap image and args for
+ # another (a SystemOne server for Strands Decider or Clef, say, with
+ # backend: systemone and SYSTEMONE_URL in env).
+ modelServer:
+ enabled: true
+ image: ghcr.io/ggml-org/llama.cpp:server-cuda
+ imagePullPolicy: IfNotPresent
+ port: 8300
+ # $(MODEL_PATH) is /models/<model.file>, expanded by Kubernetes.
+ args:
+ - -m
+ - $(MODEL_PATH)
+ - --host
+ - 0.0.0.0
+ - --port
+ - "8300"
+ - -ngl
+ - "99"
+ - -c
+ - "8192"
+ - --jinja
+ # The weights, downloaded into the PVC by an init container when the file
+ # is not there yet. Changing them downloads the new file on the next start.
+ model:
+ url: https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/resolve/main/Qwen3.5-4B-Q4_K_M.gguf
+ file: Qwen3.5-4B-Q4_K_M.gguf
+ # Optional hex SHA-256 of the file, checked after the download.
+ sha256: ""
+ # A Secret holding a Hugging Face token, for a gated model (one whose
+ # licence the token's account has accepted). Empty name: no token.
+ hfTokenSecret:
+ name: ""
+ key: HF_TOKEN
+ downloaderImage: curlimages/curl:8.11.1
+ resources:
+ requests:
+ memory: "8Gi"
+ cpu: "2"
+ nvidia.com/gpu: 1
+ limits:
+ memory: "14Gi"
+ nvidia.com/gpu: 1
+ # The weights volume. Empty storageClass uses global.storageClass.
+ persistence:
+ size: 20Gi
+ storageClass: ""
+ nodeSelector: {}
+ tolerations:
+ - key: nvidia.com/gpu
+ operator: Exists
+ effect: NoSchedule
+ serviceAccount:
+ # e.g. eks.amazonaws.com/role-arn for Bedrock through IRSA.
+ annotations: {}
+ # Admit only this release's switch-core pods to the decision port. Needs a
+ # CNI that enforces NetworkPolicy; without one it is accepted and ignored.
+ networkPolicy:
+ enabled: true
+
# ── Setup Job ─────────────────────────────────────────────────────────────────
setup:
image: setup:latest
diff --git a/docs/old/LOCAL_DEVELOPMENT.md b/docs/old/LOCAL_DEVELOPMENT.md
index 3f8f3ca2b..98010ee2c 100644
--- a/docs/old/LOCAL_DEVELOPMENT.md
+++ b/docs/old/LOCAL_DEVELOPMENT.md
@@ -224,10 +224,36 @@ just local-cloud-console # the Console, with Switch Cloud at http://localhost:80
their migrations differently cannot break each other's database this
way. Delete that directory to start the Console from scratch.
+## Auto-route locally
+
+Auto-route (the `decision_routing` feature flag) asks a decision service who
+should answer an untagged message. `just decision-up` starts one beside the
+stack: the decision service on `:8230` (`DECISION_PORT` to change it) and a
+llama.cpp server for its model, Qwen3.5-4B in 4-bit, whose weights (about
+2.7 GB) download once into the `decision-models` volume. It is the compose
+`decision` profile; `just decision-up gpu` is the `decision-gpu` profile, for
+Linux with an NVIDIA GPU and the NVIDIA Container Toolkit.
+
+switch-core runs on the host, so point it at the service in `.env` and
+restart `just run`:
+
+```bash
+FEATURE_FLAGS_ENABLED=follow_the_conversation,decision_routing
+DECISION_ENDPOINT_URL=http://localhost:8230
+DECISION_TIMEOUT_SECONDS=30
+```
+
+On CPU each decision takes several seconds, far over the one-second target a
+GPU meets, so raise the timeout as above. It is fine for trying auto-route
+out, not for judging its latency. Another model is `DECISION_MODEL_URL`,
+`DECISION_MODEL_FILE` and `DECISION_MODEL` (plus `HF_TOKEN` for a gated one).
+`just decision-down` stops it. `decision/README.md` covers the service itself.
+
## Other useful recipes
| Command | What it does |
| --- | --- |
+| `just decision-up [gpu]` / `just decision-down` | Start / stop the decision service for auto-route (see above) |
| `just local-cloud` / `just local-cloud-console` | Run switch-core as a stand-in for Switch Cloud, and the Console pointed at it (see above) |
| `just format` / `just check` | Format with ruff / lint-check in CI mode (no changes) |
| `just typecheck` | mypy over `core/switch_core/` |
diff --git a/justfile b/justfile
index cba829484..7f50e80bd 100644
--- a/justfile
+++ b/justfile
@@ -56,6 +56,32 @@ down:
reset:
docker compose -f deploy/local/docker-compose.yml --project-directory . down -v
+# Start the decision service auto-route asks (with its model server), on CPU or,
+# with `just decision-up gpu`, an NVIDIA GPU on Linux. Weights download once
+# into a volume. docs/old/LOCAL_DEVELOPMENT.md, "Auto-route locally".
+decision-up mode="cpu":
+ #!/usr/bin/env bash
+ set -euo pipefail
+ case "{{mode}}" in
+ cpu) profile=decision ;;
+ gpu) profile=decision-gpu ;;
+ *) echo "usage: just decision-up [cpu|gpu]" >&2; exit 2 ;;
+ esac
+ docker compose -f deploy/local/docker-compose.yml --project-directory . --profile "$profile" up -d --build --wait decision
+ port="${DECISION_PORT:-8230}"
+ echo "Decision service: http://localhost:$port (curl http://localhost:$port/health)"
+ if [ "$profile" = decision ]; then
+ echo "⚠ CPU model server: each decision takes several seconds, well over the"
+ echo " one-second target a GPU meets. Fine for trying auto-route out."
+ fi
+ echo "Point switch-core at it in .env, then restart 'just run':"
+ echo " DECISION_ENDPOINT_URL=http://localhost:$port"
+ echo " DECISION_TIMEOUT_SECONDS=30"
+ echo " FEATURE_FLAGS_ENABLED=follow_the_conversation,decision_routing (plus any you had)"
+
+decision-down:
+ docker compose -f deploy/local/docker-compose.yml --project-directory . --profile decision --profile decision-gpu stop decision decision-model decision-model-gpu
+
# ── Run switch-core locally ────────────────────────────────────────────────────
# The Python project lives in core/; `--project core` selects that environment
# while keeping the repo root as the working directory. Tool configs are passed explicitly since the repo root no longer
diff --git a/scripts/check_chart_decision.py b/scripts/check_chart_decision.py
new file mode 100755
index 000000000..ecce9a79f
--- /dev/null
+++ b/scripts/check_chart_decision.py
@@ -0,0 +1,266 @@
+#!/usr/bin/env python3
+"""Render the switch chart with its decision service off and on, and check it.
+
+The decision service (auto-route) is optional in the chart. Off, nothing of it
+renders and switch-core gets no endpoint. On, the chart runs it, points
+switch-core at it and admits only switch-core to it. An endpoint run elsewhere
+can be named instead, but not as well as the chart's own. These are the
+promises deploy/remote/helm/switch/README.md makes about `decision.*`; a render
+that breaks one fails here, before a cluster finds out.
+
+Needs `helm` on PATH and PyYAML. Run from the repo root:
+
+ python3 scripts/check_chart_decision.py
+"""
+
+from __future__ import annotations
+
+import subprocess
+import sys
+from collections.abc import Callable
+from typing import Any
+
+import yaml
+
+CHART = "deploy/remote/helm/switch"
+BASE = "deploy/remote/helm/render-check-values.yaml"
+RELEASE = "rel"
+FLAGS = [
+ "--set",
+ "switchCore.featureFlags.enabled={follow_the_conversation,decision_routing}",
+]
+
+
+def render(*args: str) -> list[dict[str, Any]]:
+ out = subprocess.run(
+ ["helm", "template", RELEASE, CHART, "-f", BASE, *args],
+ check=True,
+ capture_output=True,
+ text=True,
+ ).stdout
+ return [doc for doc in yaml.safe_load_all(out) if doc]
+
+
+def render_error(*args: str) -> str:
+ result = subprocess.run(
+ ["helm", "template", RELEASE, CHART, "-f", BASE, *args],
+ capture_output=True,
+ text=True,
+ )
+ assert result.returncode != 0, f"expected the render to fail: {args}"
+ return result.stderr
+
+
+def decision_objects(docs: list[dict[str, Any]]) -> dict[str, dict[str, Any]]:
+ return {
+ doc["kind"]: doc
+ for doc in docs
+ if doc["metadata"].get("labels", {}).get("app.kubernetes.io/component")
+ == "decision"
+ }
+
+
+def core_env(docs: list[dict[str, Any]]) -> dict[str, Any]:
+ deployment = next(
+ d
+ for d in docs
+ if d["kind"] == "Deployment"
+ and d["metadata"]["name"] == f"{RELEASE}-switch-core"
+ )
+ container = next(
+ c
+ for c in deployment["spec"]["template"]["spec"]["containers"]
+ if c["name"] == "switch-core"
+ )
+ return {e["name"]: e.get("value", e.get("valueFrom")) for e in container["env"]}
+
+
+def containers(deployment: dict[str, Any]) -> dict[str, dict[str, Any]]:
+ return {c["name"]: c for c in deployment["spec"]["template"]["spec"]["containers"]}
+
+
+def env_of(container: dict[str, Any]) -> dict[str, Any]:
+ return {e["name"]: e.get("value") for e in container.get("env", [])}
+
+
+def test_off_by_default_renders_nothing_and_sets_no_endpoint() -> None:
+ docs = render()
+ assert decision_objects(docs) == {}
+ env = core_env(docs)
+ assert "DECISION_ENDPOINT_URL" not in env
+ assert "DECISION_BEDROCK_ENABLED" not in env
+
+
+def test_on_runs_the_service_and_points_switch_core_at_it() -> None:
+ docs = render("--set", "decision.enabled=true", *FLAGS)
+ objects = decision_objects(docs)
+ assert set(objects) == {
+ "Deployment",
+ "Service",
+ "PersistentVolumeClaim",
+ "ServiceAccount",
+ "NetworkPolicy",
+ }, sorted(objects)
+
+ assert core_env(docs)["DECISION_ENDPOINT_URL"] == f"http://{RELEASE}-decision:8230"
+
+ service = objects["Service"]
+ assert service["metadata"]["name"] == f"{RELEASE}-decision"
+ assert [p["port"] for p in service["spec"]["ports"]] == [8230]
+
+ deployment = objects["Deployment"]
+ pod = deployment["spec"]["template"]["spec"]
+ assert pod["serviceAccountName"] == f"{RELEASE}-decision"
+ assert {
+ "key": "nvidia.com/gpu",
+ "operator": "Exists",
+ "effect": "NoSchedule",
+ } in pod["tolerations"]
+ by_name = containers(deployment)
+ assert set(by_name) == {"decision", "model-server"}
+ assert env_of(by_name["decision"])["LLAMACPP_URL"] == "http://127.0.0.1:8300"
+ assert env_of(by_name["decision"])["DECISION_BACKEND"] == "llamacpp"
+ gpu = by_name["model-server"]["resources"]
+ assert gpu["limits"]["nvidia.com/gpu"] == 1
+ assert "nvidia.com/gpu" not in by_name["decision"]["resources"].get("limits", {})
+ assert [c["name"] for c in pod["initContainers"]] == ["download-model"]
+ assert {
+ v["name"]: v["persistentVolumeClaim"]["claimName"]
+ for v in pod["volumes"]
+ if "persistentVolumeClaim" in v
+ } == {"weights": f"{RELEASE}-decision-weights"}
+
+ policy = objects["NetworkPolicy"]["spec"]
+ assert policy["podSelector"]["matchLabels"] == {
+ "app.kubernetes.io/instance": RELEASE,
+ "app.kubernetes.io/component": "decision",
+ }
+ assert policy["ingress"] == [
+ {
+ "from": [
+ {
+ "podSelector": {
+ "matchLabels": {
+ "app.kubernetes.io/instance": RELEASE,
+ "app.kubernetes.io/component": "switch-core",
+ }
+ }
+ }
+ ],
+ "ports": [{"protocol": "TCP", "port": 8230}],
+ }
+ ]
+
+
+def test_bedrock_and_the_pod_role() -> None:
+ docs = render(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "decision.bedrock.enabled=true",
+ "--set",
+ "decision.serviceAccount.annotations.eks\\.amazonaws\\.com/role-arn=arn:example",
+ *FLAGS,
+ )
+ objects = decision_objects(docs)
+ assert core_env(docs)["DECISION_BEDROCK_ENABLED"] == "true"
+ decision = containers(objects["Deployment"])["decision"]
+ assert env_of(decision)["DECISION_BEDROCK_ENABLED"] == "1"
+ assert objects["ServiceAccount"]["metadata"]["annotations"] == {
+ "eks.amazonaws.com/role-arn": "arn:example"
+ }
+
+
+def test_without_a_model_server_there_is_no_gpu_or_volume() -> None:
+ docs = render(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "decision.modelServer.enabled=false",
+ "--set",
+ "decision.backend=anthropic",
+ *FLAGS,
+ )
+ objects = decision_objects(docs)
+ assert "PersistentVolumeClaim" not in objects
+ pod = objects["Deployment"]["spec"]["template"]["spec"]
+ assert [c["name"] for c in pod["containers"]] == ["decision"]
+ assert "initContainers" not in pod
+
+
+def test_network_policy_can_be_left_out() -> None:
+ docs = render(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "decision.networkPolicy.enabled=false",
+ )
+ assert "NetworkPolicy" not in decision_objects(docs)
+
+
+def test_an_endpoint_elsewhere() -> None:
+ docs = render(
+ "--set",
+ "switchCore.decision.endpointUrl=http://decider.example:8230",
+ "--set",
+ "switchCore.decision.bedrockEnabled=true",
+ "--set",
+ "switchCore.decision.timeoutSeconds=8",
+ )
+ assert decision_objects(docs) == {}
+ env = core_env(docs)
+ assert env["DECISION_ENDPOINT_URL"] == "http://decider.example:8230"
+ assert env["DECISION_BEDROCK_ENABLED"] == "true"
+ assert env["DECISION_TIMEOUT_SECONDS"] == "8"
+
+
+def test_refuses_contradictions() -> None:
+ assert "Clear one of them" in render_error(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "switchCore.decision.endpointUrl=http://decider.example:8230",
+ )
+ assert "needs a decision service" in render_error(
+ "--set", "switchCore.decision.bedrockEnabled=true"
+ )
+ assert "follow_the_conversation" in render_error(
+ "--set", "switchCore.featureFlags.enabled={decision_routing}"
+ )
+ assert "file name, not a path" in render_error(
+ "--set",
+ "decision.enabled=true",
+ "--set",
+ "decision.modelServer.model.file=../x.gguf",
+ )
+
+
+TESTS: list[Callable[[], None]] = [
+ test_off_by_default_renders_nothing_and_sets_no_endpoint,
+ test_on_runs_the_service_and_points_switch_core_at_it,
+ test_bedrock_and_the_pod_role,
+ test_without_a_model_server_there_is_no_gpu_or_volume,
+ test_network_policy_can_be_left_out,
+ test_an_endpoint_elsewhere,
+ test_refuses_contradictions,
+]
+
+
+def main() -> int:
+ failed = 0
+ for test in TESTS:
+ try:
+ test()
+ except (AssertionError, subprocess.CalledProcessError) as exc:
+ failed += 1
+ detail = (
+ exc.stderr if isinstance(exc, subprocess.CalledProcessError) else exc
+ )
+ print(f"FAIL {test.__name__}: {detail}")
+ else:
+ print(f"ok {test.__name__}")
+ return 1 if failed else 0
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/scripts/pin_chart_images.py b/scripts/pin_chart_images.py
index a16f29173..613139207 100644
--- a/scripts/pin_chart_images.py
+++ b/scripts/pin_chart_images.py
@@ -10,7 +10,8 @@ it, and what runs is reproducible from that one pin.
Two charts are published:
-* `switch` (deploy/remote/helm/switch): switch-core, gateway and setup.
+* `switch` (deploy/remote/helm/switch): switch-core, gateway, setup and the
+ optional decision service (switch-decision).
* `switch-hosted-controller` (deploy/hosted/chart): the controller that runs
Switch cloud machines, and its one image.
@@ -37,6 +38,11 @@ Subcommands:
image of that chart — recognised by repository name, whatever registry it
names — renders from the pinning registry and by digest, all of them are
present, and the chart renders nothing else from the pinning registry.
+ An image marked `optional` (rendered only with an optional workload on) may
+ be missing from a render.
+* `covered --chart <name>` reads the image references `verify` printed for all
+ of the chart's renders and fails unless every image of the chart is among
+ them, so an optional image is still checked in some render.
* `record` writes the pins (images and charts) as JSON for whatever promotes the
build next, and as a Markdown table for the run summary.
@@ -48,6 +54,7 @@ Usage:
python scripts/pin_chart_images.py pin ... --placeholder-digests
helm template x <chart> | python scripts/pin_chart_images.py verify --chart switch \\
--registry ghcr.io/<owner>
+ cat <refs of every render> | python scripts/pin_chart_images.py covered --chart switch
python scripts/pin_chart_images.py record --digests-dir <dir> --registry ghcr.io/<owner> \\
--version <version> --channel <channel> --commit <sha> \\
--chart-digest switch=sha256:… --chart-digest switch-hosted-controller=sha256:… \\
@@ -79,6 +86,10 @@ class Image:
values_key: str
dockerfile: str
context: str
+ # Rendered only when an optional workload is switched on (decision.enabled,
+ # say), so a render may leave it out; `covered` still requires it in at
+ # least one of the chart's renders.
+ optional: bool
@dataclass(frozen=True)
@@ -120,19 +131,33 @@ CHARTS: dict[str, Chart] = {
# is built.
IMAGES: dict[str, Image] = {
"switch-core": Image(
- "switch", "switchCore", "deploy/shared_resources/images/Dockerfile.switch", "."
+ "switch",
+ "switchCore",
+ "deploy/shared_resources/images/Dockerfile.switch",
+ ".",
+ False,
),
"gateway": Image(
- "switch", "gateway", "deploy/shared_resources/images/Dockerfile.gateway", "."
+ "switch",
+ "gateway",
+ "deploy/shared_resources/images/Dockerfile.gateway",
+ ".",
+ False,
),
"setup": Image(
- "switch", "setup", "deploy/shared_resources/images/Dockerfile.setup", "."
+ "switch", "setup", "deploy/shared_resources/images/Dockerfile.setup", ".", False
+ ),
+ # The decision service behind auto-route; the chart runs it with
+ # decision.enabled.
+ "switch-decision": Image(
+ "switch", "decision", "decision/Dockerfile", "decision", True
),
"switch-hosted-controller": Image(
"switch-hosted-controller",
"image",
"deploy/hosted/controller/Dockerfile",
"deploy/hosted/controller",
+ False,
),
}
@@ -291,7 +316,11 @@ def verify(chart: str, rendered: str, registry: str) -> list[str]:
problems.append(
f"{ref}: not an image of the {chart} chart; add it to IMAGES if it should be"
)
- absent = sorted(set(images) - seen)
+ absent = sorted(
+ name
+ for name, image in images.items()
+ if name not in seen and not image.optional
+ )
if absent:
problems.append(f"missing from the rendered chart: {absent}")
if problems:
@@ -303,6 +332,22 @@ def verify(chart: str, rendered: str, registry: str) -> list[str]:
return ours
+def covered(chart: str, refs: str) -> None:
+ """Raise unless every image of the chart is among `refs`, one per line.
+
+ `refs` is what `verify` printed for all of the chart's renders together, so
+ an optional image left out of one render must still be in another.
+ """
+ images = images_of(chart)
+ seen = {_repository(ref).rsplit("/", 1)[-1] for ref in refs.split()}
+ absent = sorted(set(images) - seen)
+ if absent:
+ raise PinError(
+ f"no render of the {chart} chart has {absent}; switch on the workload "
+ "that runs it in a render's values (Chart.renders)"
+ )
+
+
def record(
digests: dict[str, str],
registry: str,
@@ -392,6 +437,12 @@ def main(argv: list[str]) -> int:
p_verify.add_argument("--chart", required=True, choices=sorted(CHARTS))
p_verify.add_argument("--registry", required=True)
+ p_covered = sub.add_parser(
+ "covered",
+ help="check the image refs `verify` printed for every render (stdin) cover the chart",
+ )
+ p_covered.add_argument("--chart", required=True, choices=sorted(CHARTS))
+
p_record = sub.add_parser(
"record", help="write the pins as JSON and a Markdown summary"
)
@@ -451,6 +502,8 @@ def main(argv: list[str]) -> int:
elif args.command == "verify":
for ref in verify(args.chart, sys.stdin.read(), args.registry):
print(ref)
+ elif args.command == "covered":
+ covered(args.chart, sys.stdin.read())
else:
pins, summary = record(
read_digests(args.digests_dir),
diff --git a/scripts/release_charts.sh b/scripts/release_charts.sh
index 1f17ecd41..f8893de6b 100755
--- a/scripts/release_charts.sh
+++ b/scripts/release_charts.sh
@@ -11,7 +11,8 @@
# For each chart: pin values.yaml, package it into <out-dir>, lint the
# package (defaults where the chart allows, then each render's values),
# and render the package for each render through `verify`. The image refs
-# the renders produce go to <out-dir>/<chart>.refs.
+# the renders produce go to <out-dir>/<chart>.refs, which must cover
+# every image of the chart (`covered`: an optional one in some render).
# publish <oci-base> <version> <out-dir>
# Push what `build` packaged and print `--chart-digest <chart>=<digest>`
# for each chart. Every chart is checked before any is pushed. A chart
@@ -91,6 +92,7 @@ build)
helm lint "$tgz" "${args[@]}"
done < <(field "$chart" '.renders[] | tojson')
rendered_refs "$chart" "$tgz" "$registry" | tee "$out/$chart.refs"
+ python3 scripts/pin_chart_images.py covered --chart "$chart" <"$out/$chart.refs"
done
;;