The open-source agent harness - the runtime layer that turns an LLM into a working agent.
Accept ${k8s-secret/...} references in chart env values (#982)
Raman Tehlan0676bfa1faa7
diff --git a/charts/trueforge/README.md b/charts/trueforge/README.mdindex 47cbecea1..07ba62e00 100644--- a/charts/trueforge/README.md+++ b/charts/trueforge/README.md@@ -98,10 +98,11 @@ A literal string still works for throwaway clusters (`apiKey: "…"`), but prefe## Extra environment`env` is a map of variable name to value, applied to both the server and the-controller. Values are scalars or `valueFrom` references. A key that matches-something the chart already sets **replaces** it rather than adding a second-entry, so it doubles as the override for computed values like `POSTGRES_HOST`-or `PUBLIC_BASE_URL`.+controller. Values are scalars, `valueFrom` references, or `${k8s-secret/...}`+reference strings. A key that matches something the chart already sets+**replaces** it rather than adding a second entry, so it doubles as the override+for computed values like `POSTGRES_HOST` or `PUBLIC_BASE_URL`. Setting a key to+`null` drops the override and hands the variable back to the chart.This is the extension point for running against a hosting platform. The chartdoes not model any particular platform; those settings live in the caller's@@ -117,6 +118,20 @@ env:secretKeyRef: { name: platform-creds, key: api-key }```+A parent chart that wants its own users to be able to redirect a key at a+different Secret must declare the default as a scalar, because Helm refuses to+replace a map with a scalar. Write such defaults as `${k8s-secret/<secret>/<key>}`+strings instead of `valueFrom` maps. The key must not contain `/`.++```yaml+env:+ REDIS_SENTINEL_PASSWORD: "${k8s-secret/redis-passwords/application_password}"+```++The string renders a `secretKeyRef` with `optional: true`, so a missing key+leaves the variable unset rather than blocking the pod. The same strings are+accepted by every field listed under [Using Secrets](#using-secrets).+When the platform also needs files (an outbound mTLS client certificate, say),mount them with `extraVolumes` / `extraVolumeMounts`, which apply to bothdeployments. `server.extraEnv` and `controller.extraEnv` remain available for@@ -216,7 +231,8 @@ The server always runs peered (`STANDALONE=false`), so Redis is always required.Bundled by default (`redis.enabled=true`, **auth disabled** — fine only whenRedis stays unreachable outside the cluster trust boundary). To use an**external** Redis, set `redis.enabled=false`, `externalRedis.enabled=true`, and exactly one-of `externalRedis.url`, `externalRedis.host`, or Sentinel. Fields accept a string or+of `externalRedis.url`, `externalRedis.host`, or Sentinel. `env.REDIS_HOST` or+`env.REDIS_SENTINEL_NODES` can supply the transport when those are empty. Fields accept a string or`valueFrom.secretKeyRef`. url, host, and sentinel are mutually exclusive:```yaml@@ -313,9 +329,10 @@ OUTBOUND_URL_BLOCKED_HOSTS=["evil.example.com"]Prefer Kubernetes Secrets over literals in values files for production. Thischart does **not** create Secrets for chart-owned fields — supply-`valueFrom.secretKeyRef` (or create Secrets yourself and point at them).+`valueFrom.secretKeyRef` or a `${k8s-secret/<secret>/<key>}` string (or create+Secrets yourself and point at them). The key must not contain `/`.-Fields that accept string | `valueFrom.secretKeyRef`:+Fields that accept string | `valueFrom.secretKeyRef` | `${k8s-secret/...}`:`externalPostgres.host`, `externalPostgres.port`, `externalPostgres.database`,`externalPostgres.user`, `externalPostgres.password`, `externalRedis.url` / `host` / `auth`,`externalRedis.tls` (`caCert`, `cert`, `key`, `keyPassphrase`, `serverName`),diff --git a/charts/trueforge/templates/_helpers.tpl b/charts/trueforge/templates/_helpers.tplindex f8f52cb03..b36bcebc8 100644--- a/charts/trueforge/templates/_helpers.tpl+++ b/charts/trueforge/templates/_helpers.tpl@@ -393,11 +393,51 @@ Replica count is always 1, even when a resourceTier is set.{{- index ($merged.limits | default dict) "ephemeral-storage" -}}{{- end }}+{{/*+True when a value is a `${k8s-secret/...}` reference string.+*/}}+{{- define "trueforge.env.isSecretRef" -}}+{{- $v := index . "value" -}}+{{- if and (kindIs "string" $v) (hasPrefix "${k8s-secret/" $v) -}}true{{- end -}}+{{- end }}++{{/*+JSON `valueFrom` body from a `${k8s-secret/<secret>/<key>}` reference. The key+must not contain `/` (the split is on `/`). Lets a value carry a secretKeyRef+as a plain string, which is the only shape a parent chart can override (Helm+refuses to replace a map with a scalar).+Expects: field (values path for errors), value.+*/}}+{{- define "trueforge.env.k8sSecretRef" -}}+{{- $field := index . "field" -}}+{{- $value := index . "value" -}}+{{- $prefix := "${k8s-secret/" -}}+{{- $secret := "" -}}+{{- $key := "" -}}+{{- if and (kindIs "string" $value) (hasPrefix $prefix $value) (hasSuffix "}" $value) -}}+{{- $body := trimSuffix "}" (trimPrefix $prefix $value) -}}+{{- if and $body (not (contains "}" $body)) (eq (printf "%s%s}" $prefix $body) $value) -}}+{{- $parts := regexSplit "/" $body -1 -}}+{{- if eq (len $parts) 2 -}}+{{- if and (index $parts 0) (index $parts 1) -}}+{{- $secret = index $parts 0 -}}+{{- $key = index $parts 1 -}}+{{- end -}}+{{- end -}}+{{- end -}}+{{- end -}}+{{- if and $secret $key -}}+{{- dict "secretKeyRef" (dict "name" $secret "key" $key "optional" true) | toJson -}}+{{- else -}}+{{- fail (printf "%s has an invalid secret reference %q; use ${k8s-secret/<secret>/<key>}. The key must not contain /" $field $value) -}}+{{- end -}}+{{- end }}+{{/*JSON env entry from a string | { valueFrom: ... } field.Expects: name (env var), field (values path for errors), value.-Literals become env value; valueFrom maps are passed through. The chart does-not create Secrets; callers who need secretKeyRef must supply valueFrom.+Literals become env value; valueFrom maps are passed through; `${k8s-secret/...}`+strings become a secretKeyRef. The chart creates no Secrets either way.*/}}{{- define "trueforge.env.fromStringOrValueFrom" -}}{{- $name := index . "name" -}}@@ -406,14 +446,17 @@ not create Secrets; callers who need secretKeyRef must supply valueFrom.{{- include "trueforge.requireStringOrValueFrom" (dict "name" $field "value" $value) -}}{{- if kindIs "map" $value -}}{{- dict "name" $name "valueFrom" $value.valueFrom | toJson -}}+{{- else if include "trueforge.env.isSecretRef" (dict "value" $value) -}}+{{- dict "name" $name "valueFrom" (include "trueforge.env.k8sSecretRef" (dict "field" $field "value" $value) | fromJson) | toJson -}}{{- else -}}{{- dict "name" $name "value" ($value | toString) | toJson -}}{{- end -}}{{- end }}{{/*-One env entry from the `env` map. Scalars become a literal value; a map must-carry valueFrom and is passed through untouched.+One env entry from the `env` map. Scalars become a literal value, a map must+carry valueFrom and is passed through untouched, and a `${k8s-secret/...}`+string becomes a secretKeyRef.*/}}{{- define "trueforge.env.item" -}}{{- $name := index . "name" -}}@@ -421,6 +464,8 @@ carry valueFrom and is passed through untouched.{{- if kindIs "map" $value -}}{{- if not $value.valueFrom -}}{{- fail (printf "env.%s must set valueFrom when given as a map" $name) -}}{{- end -}}{{- dict "name" $name "valueFrom" $value.valueFrom | toJson -}}+{{- else if include "trueforge.env.isSecretRef" (dict "value" $value) -}}+{{- dict "name" $name "valueFrom" (include "trueforge.env.k8sSecretRef" (dict "field" (printf "env.%s" $name) "value" $value) | fromJson) | toJson -}}{{- else -}}{{- dict "name" $name "value" ($value | toString) | toJson -}}{{- end -}}@@ -450,19 +495,25 @@ fields, wires bundled Postgres/Redis, optional OIDC, then server.extraEnv.{{- if .Values.redis.enabled -}}{{- $env = append $env (dict "name" "REDIS_URL" "value" (include "trueforge.redis.bundledUrl" .)) -}}{{- else if $externalRedis.enabled -}}+{{- $envMap := .Values.env | default dict -}}+{{- $envHost := index $envMap "REDIS_HOST" | default "" -}}+{{- $envSentinelNodes := index $envMap "REDIS_SENTINEL_NODES" | default "" -}}+{{- $envTransport := or (and (kindIs "string" $envHost) $envHost) (and (kindIs "string" $envSentinelNodes) $envSentinelNodes) -}}{{- if and $sentinelEnabled (or $externalRedis.url $externalRedis.host) -}}{{- fail "externalRedis.sentinel cannot be combined with externalRedis.url or externalRedis.host" -}}{{- end -}}{{- if and $externalRedis.url $externalRedis.host -}}{{- fail "externalRedis.url and externalRedis.host are mutually exclusive" -}}{{- end -}}-{{- if and (not $sentinelEnabled) (not $externalRedis.url) (not $externalRedis.host) -}}-{{- fail "externalRedis.url or externalRedis.host is required when using external Redis without sentinel" -}}+{{- if and (not $sentinelEnabled) (not $externalRedis.url) (not $externalRedis.host) (not $envTransport) -}}+{{- fail "externalRedis.url, externalRedis.host, or externalRedis.sentinel is required when env does not set REDIS_HOST or REDIS_SENTINEL_NODES" -}}{{- end -}}{{- if and (not $sentinelEnabled) $externalRedis.url (or $auth.username $auth.password (ne (($externalRedis.db | default 0) | toString) "0")) -}}{{- fail "externalRedis.auth and a non-zero externalRedis.db are ignored when externalRedis.url is set; encode them in the url or use host mode" -}}{{- end -}}-{{- if $sentinelEnabled -}}+{{- if and (not $sentinelEnabled) (not $externalRedis.url) (not $externalRedis.host) -}}+{{- /* Transport comes from env.REDIS_HOST or env.REDIS_SENTINEL_NODES. */ -}}+{{- else if $sentinelEnabled -}}{{- $_ := required "externalRedis.sentinel.nodes is required when externalRedis.sentinel.enabled is true" (join "," $sentinel.nodes) -}}{{- $_ := required "externalRedis.sentinel.masterName is required when externalRedis.sentinel.enabled is true" $sentinel.masterName -}}{{- $env = append $env (dict "name" "REDIS_SENTINEL_ENABLED" "value" "true") -}}@@ -495,6 +546,7 @@ fields, wires bundled Postgres/Redis, optional OIDC, then server.extraEnv.{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_PASSWORD" "field" "externalRedis.auth.password" "value" $auth.password) | fromJson) -}}{{- end -}}{{- end -}}+{{- if or $sentinelEnabled $externalRedis.url $externalRedis.host -}}{{- $env = append $env (dict "name" "REDIS_TLS_ENABLED" "value" (ternary "true" "false" $tlsEnabled)) -}}{{- if $tlsEnabled -}}{{- $rejectUnauthorized := true -}}@@ -518,6 +570,7 @@ fields, wires bundled Postgres/Redis, optional OIDC, then server.extraEnv.{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_TLS_KEY_PASSPHRASE" "field" "externalRedis.tls.keyPassphrase" "value" $tls.keyPassphrase) | fromJson) -}}{{- end -}}{{- end -}}+{{- end -}}{{- else -}}{{- fail "set redis.enabled or externalRedis.enabled" -}}{{- end -}}@@ -596,8 +649,15 @@ fields, wires bundled Postgres/Redis, optional OIDC, then server.extraEnv.{{- end -}}{{- /* env map: replace in place when the chart already emits the name, else- append. Keeps the pod spec free of duplicate env entries. */ -}}-{{- $overrides := .Values.env | default dict -}}+ append. Keeps the pod spec free of duplicate env entries. A null value+ drops the override, so a parent chart can hand a key back to the chart+ (Helm keeps null keys in subchart values rather than removing them). */ -}}+{{- $overrides := dict -}}+{{- range $name, $value := (.Values.env | default dict) -}}+{{- if not (kindIs "invalid" $value) -}}+{{- $_ := set $overrides $name $value -}}+{{- end -}}+{{- end -}}{{- if $overrides -}}{{- $out := list -}}{{- $seen := dict -}}diff --git a/charts/trueforge/values.yaml b/charts/trueforge/values.yamlindex 9328950b3..248a00220 100644--- a/charts/trueforge/values.yaml+++ b/charts/trueforge/values.yaml@@ -144,11 +144,16 @@ controller:# Controller → server API key (TRUEFORGE_API_KEY). Dev default is a known# placeholder — strongly recommended to create a Secret and set:# apiKey: { valueFrom: { secretKeyRef: { name: trueforge-api-key, key: TRUEFORGE_API_KEY } } }+# apiKey: "${k8s-secret/trueforge-api-key/TRUEFORGE_API_KEY}"# The chart never creates that Secret. A plain string also works for local trials.apiKey: "placeholder-value-please-generate-your-own"-# Extra env for BOTH server and controller: name -> scalar or { valueFrom: ... }.+# Extra env for BOTH server and controller: name -> scalar, { valueFrom: ... },+# or a "${k8s-secret/<secret>/<key>}" reference string. The key must not contain "/".# A key that matches a chart-computed variable (POSTGRES_HOST, PUBLIC_BASE_URL,-# ...) replaces it in place. This is the extension point for a parent chart.+# ...) replaces it in place; a null value drops the override and hands the key+# back to the chart. This is the extension point for a parent chart, which can+# only override a key it declares as a scalar - Helm refuses to replace a map+# with a scalar, so a default written as { valueFrom: ... } is final.env: {}# Application feature config (maps to env vars in packages/trueforge/src/config.ts).# clientSecret (and other secret-like fields) accept a string or@@ -279,16 +284,19 @@ externalPostgres:externalRedis:## Use an external Redis instead of the bundled Redis (`redis.enabled` must be false).## Requires `enabled: true`, then exactly one transport: url, host, or sentinel.+ ## env.REDIS_HOST or env.REDIS_SENTINEL_NODES can supply the transport instead.enabled: false## Full connection URL, e.g. redis://:[email protected]:6379.- ## string or valueFrom. Mutually exclusive with host and sentinel.+ ## string, valueFrom, or "${k8s-secret/<secret>/<key>}".+ ## Mutually exclusive with host and sentinel.url: ""## Standalone Redis host. Mutually exclusive with url and sentinel.host: ""port: 6379db: 0auth:- ## string or valueFrom (host/sentinel modes; put credentials in url when using url)+ ## string, valueFrom, or "${k8s-secret/<secret>/<key>}" (host/sentinel modes;+ ## put credentials in url when using url)username: ""password: ""tls:@@ -308,7 +316,7 @@ externalRedis:nodes: []masterName: ""auth:- ## string or valueFrom+ ## string, valueFrom, or "${k8s-secret/<secret>/<key>}"username: ""password: ""# --- Bundled dev sandbox server (tfy-sandbox-server subchart) -------------------