Skip to content

The open-source agent harness - the runtime layer that turns an LLM into a working agent.

feat: add Redis Sentinel and TLS support in TrueForge (#765)

Sujai Kumar Guptacf55de9aed64
diff --git a/.changeset/redis-sentinel.md b/.changeset/redis-sentinel.md
new file mode 100644
index 000000000..72e00471b
--- /dev/null
+++ b/.changeset/redis-sentinel.md
@@ -0,0 +1,6 @@
+---
+'@truefoundry/trueforge': minor
+'@truefoundry/trueforge-core': minor
+---
+
+Add Redis Sentinel + TLS with exclusive transport fail-fast (`REDIS_CONNECTION` DU). Redis is optional at config load for controller/migrate; server still requires it at connect. Sentinel shared-client errors no longer stop the peering heartbeat.
diff --git a/charts/trueforge/Chart.yaml b/charts/trueforge/Chart.yaml
index d1abde1bf..7150ae069 100644
--- a/charts/trueforge/Chart.yaml
+++ b/charts/trueforge/Chart.yaml
@@ -4,7 +4,7 @@ description: TrueForge server (API + UI) served from a single container image.
type: application
# version / appVersion are maintained on main (bot chart-release PR or human).
# Publishing is gated by git tag charts/trueforge@<version> (must match version).
-version: "0.2.2"
+version: "0.2.3"
appVersion: "0.2.1"
kubeVersion: ">=1.25.0-0"
home: https://truefoundry.com
diff --git a/charts/trueforge/README.md b/charts/trueforge/README.md
index eaeb13c22..d80dc2068 100644
--- a/charts/trueforge/README.md
+++ b/charts/trueforge/README.md
@@ -38,7 +38,7 @@ docs.
Postgres and Redis ship as **bundled** dependencies (the Bitnami `postgresql`
and `redis` charts, pulled from the public Bitnami OCI archive and pinned by
`Chart.lock`). They are enabled by default, so a basic install needs **no
-required values**. The chart wires the server's `POSTGRES_*` and `REDIS_URL`
+required values**. The chart wires the server's `POSTGRES_*` and `REDIS_*`
env to the bundled services automatically.
The Bitnami **charts** are still public, but the **container images** they
@@ -213,26 +213,33 @@ externalPostgres:
The server always runs peered (`STANDALONE=false`), so Redis is always required.
Bundled by default (`redis.enabled=true`, **auth disabled** — fine only when
Redis stays unreachable outside the cluster trust boundary). To use an
-**external** Redis, set `redis.enabled=false` and provide `externalRedis.url`
-as a string or `valueFrom.secretKeyRef`:
+**external** Redis, set `redis.enabled=false`, `externalRedis.enabled=true`, and exactly one
+of `externalRedis.url`, `externalRedis.host`, or Sentinel. Fields accept a string or
+`valueFrom.secretKeyRef`. url, host, and sentinel are mutually exclusive:
```yaml
redis:
enabled: false
externalRedis:
- url:
- valueFrom:
- secretKeyRef:
- name: my-redis-secret
- key: redis-url
+ enabled: true
+ url: redis://:[email protected]:6379
+ # or host + auth (not with url):
+ # host: redis-master.databases.svc
+ # port: 6379
+ # auth:
+ # password:
+ # valueFrom:
+ # secretKeyRef:
+ # name: my-redis-secret
+ # key: redis-password
```
`redis.nameOverride` defaults to `trueforge-redis` so bundled Redis objects do
not share names with other Redis chart dependencies when this chart is a
dependency of some other chart.
-For passworded Redis, prefer an external instance and load `REDIS_URL` via
-`valueFrom`.
+For passworded Redis, prefer an external instance and set `externalRedis.url` or
+`externalRedis.auth` (and TLS/Sentinel as needed) via string or `valueFrom`.
## OIDC
@@ -308,8 +315,10 @@ chart does **not** create Secrets for chart-owned fields — supply
Fields that accept string | `valueFrom.secretKeyRef`:
`externalPostgres.host`, `externalPostgres.port`, `externalPostgres.database`,
-`externalPostgres.user`, `externalPostgres.password`, `externalRedis.url`,
-`configs.oidc.clientSecret`.
+`externalPostgres.user`, `externalPostgres.password`, `externalRedis.url` / `host` / `auth`,
+`externalRedis.tls` (`caCert`, `cert`, `key`, `keyPassphrase`, `serverName`),
+`externalRedis.sentinel.auth`, `configs.oidc.clientSecret`.
+`externalRedis.tls.rejectUnauthorized` is a plain boolean (default true).
`configs.oidc.issuerUrl` and `clientId` are plain strings only.
**Bundled Postgres password** still uses Bitnami's API (`postgresql.auth.existingSecret`,
@@ -395,7 +404,7 @@ also sets the `/tmp` `emptyDir.sizeLimit`.
- **Enable `configs.oidc`** — leaving it off grants shared admin to anyone who can reach the server.
- **Replace the `apiKey` placeholder** — create a Secret for `TRUEFORGE_API_KEY` and set `apiKey.valueFrom.secretKeyRef` (do not leave `placeholder-value-please-generate-your-own`).
- **Replace the bundled Postgres password** (`trueforge`) or set `postgresql.auth.existingSecret`.
-- Treat bundled Redis (`redis.auth.enabled: false`) as cluster-internal only, or switch to external passworded Redis via `externalRedis.url`.
+- Treat bundled Redis (`redis.auth.enabled: false`) as cluster-internal only, or switch to external passworded Redis via `externalRedis`.
- Set `server.publicBaseUrl` to the real public application URL before using MCP OAuth or OIDC (include a pathname when the UI is served under a stripped prefix).
- Prefer `valueFrom.secretKeyRef` for Postgres password, Redis URL, and OIDC client secret; do not commit secrets in values files.
- Prefer external managed Postgres/Redis over the bundled subcharts for production HA.
diff --git a/charts/trueforge/templates/NOTES.txt b/charts/trueforge/templates/NOTES.txt
index 13e1114df..038303d86 100644
--- a/charts/trueforge/templates/NOTES.txt
+++ b/charts/trueforge/templates/NOTES.txt
@@ -39,7 +39,7 @@ before any shared / public deploy.
{{- if and .Values.redis.enabled (not .Values.redis.auth.enabled) }}
WARNING: Bundled Redis has auth disabled. Keep it cluster-internal (NetworkPolicy)
-or switch to external passworded Redis via externalRedis.url.
+or switch to external passworded Redis via externalRedis (url/host/auth).
{{- end }}
{{- if and .Values.configs.oidc.enabled (not .Values.server.publicBaseUrl) }}
diff --git a/charts/trueforge/templates/_helpers.tpl b/charts/trueforge/templates/_helpers.tpl
index b6257e67e..060026b7c 100644
--- a/charts/trueforge/templates/_helpers.tpl
+++ b/charts/trueforge/templates/_helpers.tpl
@@ -209,7 +209,7 @@ postgresql subchart (existingSecret override or <release>-postgresql).
{{- end }}
{{/*
-Bitnami redis fullname (mirrors common.names.fullname) so REDIS_URL tracks
+Bitnami redis fullname (mirrors common.names.fullname) so REDIS_HOST tracks
redis.nameOverride / redis.fullnameOverride.
*/}}
{{- define "trueforge.redis.fullname" -}}
@@ -433,20 +433,88 @@ fields, wires bundled Postgres/Redis, optional OIDC, then server.extraEnv.
{{- $env = append $env (dict "name" "STANDALONE" "value" "false") -}}
{{- $env = append $env (dict "name" "GRACEFUL_TIMEOUT_SECONDS" "value" (.Values.server.gracefulTimeoutSeconds | toString)) -}}
+{{- $externalRedis := .Values.externalRedis | default dict -}}
+{{- $sentinel := $externalRedis.sentinel | default dict -}}
+{{- $auth := $externalRedis.auth | default dict -}}
+{{- $tls := $externalRedis.tls | default dict -}}
+{{- $sentinelEnabled := eq (toString ($sentinel.enabled | default false)) "true" -}}
+{{- $tlsEnabled := eq (toString ($tls.enabled | default false)) "true" -}}
+{{- if and .Values.redis.enabled $externalRedis.enabled -}}
+{{- fail "redis.enabled and externalRedis.enabled are mutually exclusive" -}}
+{{- end -}}
{{- if .Values.redis.enabled -}}
{{- $env = append $env (dict "name" "REDIS_URL" "value" (include "trueforge.redis.bundledUrl" .)) -}}
-{{- else -}}
-{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_URL" "field" "externalRedis.url" "value" .Values.externalRedis.url) | fromJson) -}}
-{{- $sentinel := .Values.externalRedis.sentinel | default dict -}}
-{{- if $sentinel.enabled -}}
-{{- $_ := required "externalRedis.sentinel.hosts is required when externalRedis.sentinel.enabled is true" $sentinel.hosts -}}
+{{- else if $externalRedis.enabled -}}
+{{- if and $sentinelEnabled (or $externalRedis.url $externalRedis.host) -}}
+{{- fail "externalRedis.sentinel cannot be combined with externalRedis.url or externalRedis.host" -}}
+{{- end -}}
+{{- if and $externalRedis.url $externalRedis.host -}}
+{{- fail "externalRedis.url and externalRedis.host are mutually exclusive" -}}
+{{- end -}}
+{{- if and (not $sentinelEnabled) (not $externalRedis.url) (not $externalRedis.host) -}}
+{{- fail "externalRedis.url or externalRedis.host is required when using external Redis without sentinel" -}}
+{{- end -}}
+{{- if and (not $sentinelEnabled) $externalRedis.url (or $auth.username $auth.password (ne (($externalRedis.db | default 0) | toString) "0")) -}}
+{{- fail "externalRedis.auth and a non-zero externalRedis.db are ignored when externalRedis.url is set; encode them in the url or use host mode" -}}
+{{- end -}}
+{{- if $sentinelEnabled -}}
+{{- $_ := required "externalRedis.sentinel.nodes is required when externalRedis.sentinel.enabled is true" (join "," $sentinel.nodes) -}}
{{- $_ := required "externalRedis.sentinel.masterName is required when externalRedis.sentinel.enabled is true" $sentinel.masterName -}}
-{{- $env = append $env (dict "name" "REDIS_SENTINEL_HOSTS" "value" $sentinel.hosts) -}}
+{{- $env = append $env (dict "name" "REDIS_SENTINEL_ENABLED" "value" "true") -}}
+{{- $env = append $env (dict "name" "REDIS_SENTINEL_NODES" "value" (join "," $sentinel.nodes)) -}}
{{- $env = append $env (dict "name" "REDIS_SENTINEL_MASTER_NAME" "value" $sentinel.masterName) -}}
-{{- if $sentinel.password -}}
-{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_SENTINEL_PASSWORD" "field" "externalRedis.sentinel.password" "value" $sentinel.password) | fromJson) -}}
+{{- $env = append $env (dict "name" "REDIS_DB" "value" (($externalRedis.db | default 0) | toString)) -}}
+{{- if $auth.username -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_USERNAME" "field" "externalRedis.auth.username" "value" $auth.username) | fromJson) -}}
+{{- end -}}
+{{- if $auth.password -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_PASSWORD" "field" "externalRedis.auth.password" "value" $auth.password) | fromJson) -}}
+{{- end -}}
+{{- $sentinelAuth := $sentinel.auth | default dict -}}
+{{- if $sentinelAuth.username -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_SENTINEL_USERNAME" "field" "externalRedis.sentinel.auth.username" "value" $sentinelAuth.username) | fromJson) -}}
+{{- end -}}
+{{- if $sentinelAuth.password -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_SENTINEL_PASSWORD" "field" "externalRedis.sentinel.auth.password" "value" $sentinelAuth.password) | fromJson) -}}
+{{- end -}}
+{{- else if $externalRedis.url -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_URL" "field" "externalRedis.url" "value" $externalRedis.url) | fromJson) -}}
+{{- else -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_HOST" "field" "externalRedis.host" "value" $externalRedis.host) | fromJson) -}}
+{{- $env = append $env (dict "name" "REDIS_PORT" "value" (($externalRedis.port | default 6379) | toString)) -}}
+{{- $env = append $env (dict "name" "REDIS_DB" "value" (($externalRedis.db | default 0) | toString)) -}}
+{{- if $auth.username -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_USERNAME" "field" "externalRedis.auth.username" "value" $auth.username) | fromJson) -}}
+{{- end -}}
+{{- if $auth.password -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_PASSWORD" "field" "externalRedis.auth.password" "value" $auth.password) | fromJson) -}}
+{{- end -}}
+{{- end -}}
+{{- $env = append $env (dict "name" "REDIS_TLS_ENABLED" "value" (ternary "true" "false" $tlsEnabled)) -}}
+{{- if $tlsEnabled -}}
+{{- $rejectUnauthorized := true -}}
+{{- if hasKey $tls "rejectUnauthorized" -}}
+{{- $rejectUnauthorized = eq (toString $tls.rejectUnauthorized) "true" -}}
+{{- end -}}
+{{- $env = append $env (dict "name" "REDIS_TLS_REJECT_UNAUTHORIZED" "value" (ternary "true" "false" $rejectUnauthorized)) -}}
+{{- if $tls.caCert -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_TLS_CA_CERT" "field" "externalRedis.tls.caCert" "value" $tls.caCert) | fromJson) -}}
{{- end -}}
+{{- if $tls.serverName -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_TLS_SERVERNAME" "field" "externalRedis.tls.serverName" "value" $tls.serverName) | fromJson) -}}
{{- end -}}
+{{- if $tls.cert -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_TLS_CERT" "field" "externalRedis.tls.cert" "value" $tls.cert) | fromJson) -}}
+{{- end -}}
+{{- if $tls.key -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_TLS_KEY" "field" "externalRedis.tls.key" "value" $tls.key) | fromJson) -}}
+{{- end -}}
+{{- if $tls.keyPassphrase -}}
+{{- $env = append $env (include "trueforge.env.fromStringOrValueFrom" (dict "name" "REDIS_TLS_KEY_PASSPHRASE" "field" "externalRedis.tls.keyPassphrase" "value" $tls.keyPassphrase) | fromJson) -}}
+{{- end -}}
+{{- end -}}
+{{- else -}}
+{{- fail "set redis.enabled or externalRedis.enabled" -}}
{{- end -}}
{{- if .Values.postgresql.enabled -}}
diff --git a/charts/trueforge/values.yaml b/charts/trueforge/values.yaml
index ff99c111f..54aa65a58 100644
--- a/charts/trueforge/values.yaml
+++ b/charts/trueforge/values.yaml
@@ -226,9 +226,9 @@ postgresql:
size: 8Gi
# --- Bundled Redis (Bitnami subchart) ------------------------------------------
# Rendered only when redis.enabled is true. Auth is disabled (dev default) so
-# REDIS_URL is a plain redis://host:6379. Safe only while Redis stays inside
-# the cluster trust boundary. For passworded Redis, disable the subchart and
-# use externalRedis with a Secret.
+# Auth is disabled (dev default) so REDIS_HOST points at the bundled master
+# without credentials. Safe only while Redis stays inside the cluster trust
+# boundary. For passworded Redis, disable the subchart and use externalRedis.
redis:
enabled: true
architecture: standalone
@@ -271,23 +271,40 @@ externalPostgres:
sslKeyPath: ""
sslCaPath: ""
externalRedis:
- # Full connection URL, e.g. redis://:[email protected]:6379.
- # string, or { valueFrom: { secretKeyRef: { name, key } } }. Prefer valueFrom in prod.
+ ## Use an external Redis instead of the bundled Redis (`redis.enabled` must be false).
+ ## Requires `enabled: true`, then exactly one transport: url, host, or sentinel.
+ enabled: false
+ ## Full connection URL, e.g. redis://:[email protected]:6379.
+ ## string or valueFrom. Mutually exclusive with host and sentinel.
url: ""
- # Redis Sentinel connection settings, injected as REDIS_SENTINEL_* env.
- # url:
- # valueFrom:
- # secretKeyRef:
- # name: my-redis-secret
- # key: redis-url
+ ## Standalone Redis host. Mutually exclusive with url and sentinel.
+ host: ""
+ port: 6379
+ db: 0
+ auth:
+ ## string or valueFrom (host/sentinel modes; put credentials in url when using url)
+ username: ""
+ password: ""
+ tls:
+ enabled: false
+ ## Path or inline PEM
+ caCert: ""
+ ## Verify the server certificate. Set false only for self-signed certs without a CA.
+ rejectUnauthorized: true
+ serverName: ""
+ cert: ""
+ key: ""
+ keyPassphrase: ""
sentinel:
+ ## Mutually exclusive with url and host. Requires nodes + masterName when enabled.
enabled: false
- # Comma-separated host:port list -> REDIS_SENTINEL_HOSTS.
- hosts: ""
- # Monitored master name -> REDIS_SENTINEL_MASTER_NAME.
+ ## Array of host:port entries
+ nodes: []
masterName: ""
- # string or valueFrom -> REDIS_SENTINEL_PASSWORD.
- password: ""
+ auth:
+ ## string or valueFrom
+ username: ""
+ password: ""
# --- Bundled dev sandbox server (tfy-sandbox-server subchart) -------------------
# Managed by the parent chart; off for standalone installs. When true, the
# subchart is deployed and the app gets TRUEFOUNDRY_SANDBOX_* env pointing at it.
diff --git a/packages/trueforge-core/src/request-reply/client.ts b/packages/trueforge-core/src/request-reply/client.ts
index 99d9fee5b..b580928bf 100644
--- a/packages/trueforge-core/src/request-reply/client.ts
+++ b/packages/trueforge-core/src/request-reply/client.ts
@@ -1,7 +1,7 @@
import { randomUUID } from 'node:crypto';
import { performance } from 'node:perf_hooks';
-import type { RedisClientType } from 'redis';
import { NoResponderError, RequestTimeoutError } from './errors';
+import type { RedisClient } from './redisClient';
import type { JSONReply, JSONValue, PublishedRequest, RequestEnvelope } from './types';
import { jsonReplySchema } from './types';
import { heartbeatKey, replyKey, requestChannel, sleep } from './utils';
@@ -28,7 +28,7 @@ function parseReplyPayload(raw: string) {
return jsonReplySchema.parse(parsed);
}
-async function getDelReply(redisClient: RedisClientType, rKey: string): Promise<JSONReply | null> {
+async function getDelReply(redisClient: RedisClient, rKey: string): Promise<JSONReply | null> {
const raw = await redisClient.getDel(rKey);
if (raw === null) {
return null;
@@ -48,7 +48,7 @@ export async function redisRequest<T extends JSONValue>({
request,
options,
}: {
- redis: RedisClientType;
+ redis: RedisClient;
executorId: string;
path: string;
request: RequestEnvelope<T>;
diff --git a/packages/trueforge-core/src/request-reply/executor.ts b/packages/trueforge-core/src/request-reply/executor.ts
index 6b2e774d5..b1936e8e9 100644
--- a/packages/trueforge-core/src/request-reply/executor.ts
+++ b/packages/trueforge-core/src/request-reply/executor.ts
@@ -1,9 +1,9 @@
import { randomUUID } from 'node:crypto';
-import type { RedisClientType } from 'redis';
import type { Logger } from 'winston';
import z from 'zod';
import { extractErrorLogFields } from '../core/util/errorLogFields';
import { ReplyError } from './errors';
+import type { RedisClient } from './redisClient';
import type { JSONReply, RequestHandler } from './types';
import { publishedRequestSchema } from './types';
import { heartbeatKey, requestChannel } from './utils';
@@ -44,8 +44,8 @@ export class RequestReplyExecutor {
readonly executorId: string;
/** `tfg:rr:req:<executorId>` — the channel this executor subscribes to. */
readonly channel: string;
- private readonly redis: RedisClientType;
- private readonly subscriberClient: RedisClientType;
+ private readonly redis: RedisClient;
+ private readonly subscriberClient: RedisClient;
private readonly logger: Logger;
private readonly heartbeatIntervalMs: number;
private readonly heartbeatTtlMs: number;
@@ -74,9 +74,9 @@ export class RequestReplyExecutor {
}: {
executorId: string;
/** Connected command client, used only for SET (reply + heartbeat). Caller owns its lifecycle. */
- redis: RedisClientType;
+ redis: RedisClient;
/** Connected client to SUBSCRIBE on (duplicate or Sentinel). Caller owns its lifecycle. */
- subscriberClient: RedisClientType;
+ subscriberClient: RedisClient;
requestHandler: RequestHandler;
onError?: RequestReplyErrorHandler | undefined;
logger: Logger;
@@ -108,7 +108,7 @@ export class RequestReplyExecutor {
...extractErrorLogFields(err),
});
this.onError?.(err, { executorId: this.executorId, channel: this.channel });
- this.stopHeartbeat();
+ // error is often transient; the client can reconnect and fire ready again. Do not stop the heartbeat.
});
this.subscriberClient.on('end', () => {
this.logger.warn('[RequestReplyExecutor] Subscriber connection ended', { executorId: this.executorId });
diff --git a/packages/trueforge-core/src/request-reply/index.ts b/packages/trueforge-core/src/request-reply/index.ts
index f8937ebfc..bf5cfd5cf 100644
--- a/packages/trueforge-core/src/request-reply/index.ts
+++ b/packages/trueforge-core/src/request-reply/index.ts
@@ -10,6 +10,7 @@ export type { SendRequestOptions } from './client';
export { NoResponderError, ReplyError, RequestTimeoutError } from './errors';
export { RequestReplyExecutor } from './executor';
export type { RequestReplyErrorHandler, RunExecutorOptions } from './executor';
+export type { RedisClient } from './redisClient';
export { RequestReplyRouter } from './router';
export type { RouteHandler } from './router';
export { jsonReplySchema } from './types';
diff --git a/packages/trueforge-core/src/request-reply/redisClient.ts b/packages/trueforge-core/src/request-reply/redisClient.ts
new file mode 100644
index 000000000..26fcbbee5
--- /dev/null
+++ b/packages/trueforge-core/src/request-reply/redisClient.ts
@@ -0,0 +1,4 @@
+import type { RedisClientType, RedisSentinelType } from 'redis';
+
+/** Standalone or Sentinel client for request-reply / command use. */
+export type RedisClient = RedisClientType | RedisSentinelType;
diff --git a/packages/trueforge/.env.example b/packages/trueforge/.env.example
index daf3e77e6..43161d6ab 100644
--- a/packages/trueforge/.env.example
+++ b/packages/trueforge/.env.example
@@ -58,10 +58,38 @@ PORT=8790
## Windows: %LOCALAPPDATA%\trueforge\Data\db\db.sqlite
# SQLITE_PATH=
-## Used when STANDALONE=false (ignored in standalone). Redis connection URL for
-## executor peering. Defaults to redis://localhost:6379 if unset.
+## Used when STANDALONE=false for the server (ignored in standalone; optional for
+## controller / migrate). Redis transports are mutually exclusive — set exactly
+## one of REDIS_URL, REDIS_HOST, or Sentinel.
## `docker-compose.yml` overrides this to redis://redis:6379 for the server container.
REDIS_URL=redis://localhost:6379
+# Or host-based (mutually exclusive with REDIS_URL / Sentinel):
+# REDIS_HOST=localhost
+# REDIS_PORT=6379
+# REDIS_DB=0
+# REDIS_USERNAME=
+# REDIS_PASSWORD=
+
+## Redis TLS (STANDALONE=false only). Applies to data nodes and Sentinel sockets.
+# REDIS_TLS_ENABLED=false
+# REDIS_TLS_CA_CERT=
+# REDIS_TLS_REJECT_UNAUTHORIZED=true
+# REDIS_TLS_SERVERNAME=
+# REDIS_TLS_CERT=
+# REDIS_TLS_KEY=
+# REDIS_TLS_KEY_PASSPHRASE=
+
+## Redis Sentinel (STANDALONE=false only). Mutually exclusive with REDIS_URL / REDIS_HOST.
+## When enabled, nodes + master name are required; REDIS_USERNAME/PASSWORD apply to data nodes.
+# REDIS_SENTINEL_ENABLED=false
+# REDIS_SENTINEL_NODES=sentinel-0:26379,sentinel-1:26379
+# REDIS_SENTINEL_MASTER_NAME=mymaster
+# REDIS_SENTINEL_USERNAME=
+# REDIS_SENTINEL_PASSWORD=
+
+## Redis client timeouts (STANDALONE=false only). Optional.
+# REDIS_CONNECT_TIMEOUT_MS=20000
+# REDIS_PING_INTERVAL_MS=5000
## Redis request/reply peering knobs (STANDALONE=false only). All optional.
## Max ms to wait for a peer executor's reply before failing with 424. Default 60000.
diff --git a/packages/trueforge/src/apis/sessions.ts b/packages/trueforge/src/apis/sessions.ts
index d238c48e0..cbdf0a944 100644
--- a/packages/trueforge/src/apis/sessions.ts
+++ b/packages/trueforge/src/apis/sessions.ts
@@ -14,11 +14,11 @@ import { extractErrorLogFields } from '@truefoundry/trueforge-core/core';
import {
redisRequest,
RequestTimeoutError,
+ type RedisClient,
type RouteHandler as RequestReplyRouteHandler,
type RequestReplyRouter,
} from '@truefoundry/trueforge-core/request-reply';
import type { Context } from 'hono';
-import type { RedisClientType } from 'redis';
import type { Logger } from 'winston';
import { z } from 'zod';
import type { Authorizer } from '../auth/authorizer';
@@ -81,8 +81,8 @@ export interface SessionsRouterDeps {
resolveSkillStore: ResolveSkillStore;
resolveAgentStore: (c: Context) => IAgentStore;
resolveSandboxProviderStore: (c: Context) => ISandboxProviderStore;
+ redis?: RedisClient | undefined;
resolveWebSearchProviderStore: (c: Context) => IWebSearchProviderStore;
- redis?: RedisClientType | undefined;
requestReplyRouter: RequestReplyRouter;
resolveRequestContext: ResolveRequestContext;
logger: Logger;
@@ -128,7 +128,7 @@ export interface CancelTurnDeps {
activeTurns: ActiveTurnRegistry;
session: Pick<SessionHandle, 'session_id' | 'freezeTurn'>;
sessionStore: Pick<ISessionStore, 'getTurn'>;
- redis?: RedisClientType | undefined;
+ redis?: RedisClient | undefined;
logger: Pick<Logger, 'warn'>;
}
diff --git a/packages/trueforge/src/app.ts b/packages/trueforge/src/app.ts
index cbfecdedb..d893e2f46 100644
--- a/packages/trueforge/src/app.ts
+++ b/packages/trueforge/src/app.ts
@@ -3,12 +3,11 @@ import { swaggerUI } from '@hono/swagger-ui';
import { OpenAPIHono, z } from '@hono/zod-openapi';
import type { ISessionStore, Sessions, TurnStreamingEvent } from '@truefoundry/trueforge-core/agent-session';
import { extractErrorLogFields } from '@truefoundry/trueforge-core/core';
-import type { RequestReplyRouter } from '@truefoundry/trueforge-core/request-reply';
+import type { RedisClient, RequestReplyRouter } from '@truefoundry/trueforge-core/request-reply';
import type { Context, ErrorHandler, MiddlewareHandler } from 'hono';
import { bodyLimit } from 'hono/body-limit';
import { HTTPException } from 'hono/http-exception';
import type { Configuration } from 'openid-client';
-import type { RedisClientType } from 'redis';
import type { Logger } from 'winston';
import { createAgentImportRouter } from './apis/agentImport';
import { createAgentsRouter } from './apis/agents';
@@ -211,7 +210,7 @@ export interface ServerDeps<TTransaction> {
sessions: Sessions;
activeTurns: ActiveTurnRegistry;
/** Primary Redis client (server-owned); undefined in standalone mode. */
- redis?: RedisClientType | undefined;
+ redis?: RedisClient | undefined;
/** Request-reply dispatch table served by this replica's executor. */
requestReplyRouter: RequestReplyRouter;
/** Hands out each turn's resumable event stream to the create and subscribe handlers. */
diff --git a/packages/trueforge/src/config.ts b/packages/trueforge/src/config.ts
index 6426caa2a..6c89a6fea 100644
--- a/packages/trueforge/src/config.ts
+++ b/packages/trueforge/src/config.ts
@@ -8,10 +8,10 @@
*
* `STANDALONE` is a discriminated mode selector:
* - `true` (default): SQLite only; no Redis / executor peering.
- * - `false`: Postgres + Redis (defaults to local trueforge credentials /
- * `redis://localhost:6379`).
+ * - `false`: Postgres; Redis required for the server (optional for controller / migrate).
*/
import { existsSync, readFileSync } from 'node:fs';
+import { isIPv6 } from 'node:net';
import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
@@ -19,6 +19,10 @@ import { fileURLToPath } from 'node:url';
import envPaths from 'env-paths';
import { z } from 'zod';
+import { parseRedisSentinelNodes, type RedisConnection } from './runtime/redis';
+
+export type { RedisConnection };
+
const DEFAULT_PORT = 8790;
/** Loopback default; container images set HOST=0.0.0.0 so probes and Service traffic reach the process. */
const DEFAULT_HOST = 'localhost';
@@ -40,7 +44,6 @@ const DEFAULT_POSTGRES_HOST = 'localhost';
const DEFAULT_POSTGRES_PORT = 5432;
/** Default Postgres schema for app tables + Kysely migration bookkeeping. */
export const DEFAULT_POSTGRES_SCHEMA = 'trueforge';
-const DEFAULT_REDIS_URL = 'redis://localhost:6379';
/** Unquoted Postgres identifier: letter/underscore start, then alnum/underscore, ≤63 chars. */
const POSTGRES_SCHEMA_NAME_RE = /^[a-z_][a-z0-9_]{0,62}$/;
/**
@@ -214,6 +217,18 @@ function parsePositiveInt(options: { envKey: string; raw: string | undefined; de
return value;
}
+function parseNonNegativeInt(options: { envKey: string; raw: string | undefined; defaultValue: number }): number {
+ const { envKey, raw, defaultValue } = options;
+ if (raw === undefined || raw.trim() === '') {
+ return defaultValue;
+ }
+ const value = Number(raw);
+ if (!Number.isInteger(value) || value < 0) {
+ throw new Error(`Environment variable ${envKey} must be a non-negative integer, got "${raw}"`);
+ }
+ return value;
+}
+
/** Parses a boolean env var; anything but `true`/`false` throws instead of reading as `false`. */
function parseBoolean(options: { envKey: string; raw: string | undefined; defaultValue: boolean }): boolean {
const { envKey, raw, defaultValue } = options;
@@ -349,15 +364,122 @@ function resolveCodeModeSocketParent(): string {
return path.join(os.tmpdir(), 'tf_cms');
}
-/** Redis peering URL for distributed mode. Env: `REDIS_URL`. */
-function resolveRedisUrl(): string {
- const raw = getEnv('REDIS_URL', { defaultValue: DEFAULT_REDIS_URL }) ?? DEFAULT_REDIS_URL;
- if (raw.trim() === '') {
- throw new Error('Environment variable REDIS_URL must be non-empty when STANDALONE=false.');
+/** Env string as-is, or undefined when unset/empty (no trimming). */
+function optionalNonEmptyEnv(envKey: string): string | undefined {
+ const raw = getEnv(envKey);
+ if (raw === undefined || raw === '') {
+ return undefined;
}
return raw;
}
+/**
+ * Build a standalone `redis://` URL from discrete host fields.
+ * TLS stays on socket options (`REDIS_TLS_*`), not the URL scheme — CA/cert/key
+ * cannot be expressed in `rediss://`, and mixing scheme + socket TLS is ambiguous.
+ * IPv6 hosts are bracketed so `new URL()` accepts them; username/password are
+ * percent-encoded by the URL setters.
+ */
+export function buildRedisStandaloneUrl(parts: {
+ host: string;
+ port: number;
+ database: number;
+ username: string | undefined;
+ password: string | undefined;
+}): string {
+ const bareHost = parts.host.startsWith('[') && parts.host.endsWith(']') ? parts.host.slice(1, -1) : parts.host;
+ const host = isIPv6(bareHost) ? `[${bareHost}]` : bareHost;
+ const url = new URL(`redis://${host}:${String(parts.port)}/${String(parts.database)}`);
+ if (parts.username !== undefined) {
+ url.username = parts.username;
+ }
+ if (parts.password !== undefined) {
+ url.password = parts.password;
+ }
+ return url.href;
+}
+
+/**
+ * Exactly one Redis transport when configured.
+ * Transports are mutually exclusive: Sentinel, `REDIS_URL`, or `REDIS_HOST`.
+ * Returns undefined when none are set so controller / migrate can boot without Redis;
+ * the server fails at connect time if still unset.
+ */
+function resolveRedisConnection(): RedisConnection | undefined {
+ const sentinelEnabled = parseBoolean({
+ envKey: 'REDIS_SENTINEL_ENABLED',
+ raw: getEnv('REDIS_SENTINEL_ENABLED'),
+ defaultValue: false,
+ });
+ const url = optionalNonEmptyEnv('REDIS_URL');
+ const host = optionalNonEmptyEnv('REDIS_HOST');
+ const database = parseNonNegativeInt({
+ envKey: 'REDIS_DB',
+ raw: getEnv('REDIS_DB'),
+ defaultValue: 0,
+ });
+ const username = optionalNonEmptyEnv('REDIS_USERNAME');
+ const password = getEnv('REDIS_PASSWORD');
+ const passwordOrUndefined = password === undefined || password === '' ? undefined : password;
+
+ if (sentinelEnabled) {
+ const nodes = optionalNonEmptyEnv('REDIS_SENTINEL_NODES');
+ const masterName = optionalNonEmptyEnv('REDIS_SENTINEL_MASTER_NAME');
+ if (!nodes || !masterName || parseRedisSentinelNodes(nodes).length === 0) {
+ throw new Error(
+ 'REDIS_SENTINEL_ENABLED=true requires non-empty REDIS_SENTINEL_NODES and REDIS_SENTINEL_MASTER_NAME.',
+ );
+ }
+ if (url !== undefined || host !== undefined) {
+ throw new Error(
+ 'Redis transports are mutually exclusive: unset REDIS_URL and REDIS_HOST when REDIS_SENTINEL_ENABLED=true.',
+ );
+ }
+ const sentinelUsername = optionalNonEmptyEnv('REDIS_SENTINEL_USERNAME');
+ const sentinelPasswordRaw = getEnv('REDIS_SENTINEL_PASSWORD');
+ const sentinelPassword =
+ sentinelPasswordRaw === undefined || sentinelPasswordRaw === '' ? undefined : sentinelPasswordRaw;
+ return {
+ mode: 'sentinel',
+ nodes,
+ masterName,
+ database,
+ ...(username !== undefined ? { username } : {}),
+ ...(passwordOrUndefined !== undefined ? { password: passwordOrUndefined } : {}),
+ ...(sentinelUsername !== undefined ? { sentinelUsername } : {}),
+ ...(sentinelPassword !== undefined ? { sentinelPassword } : {}),
+ };
+ }
+
+ if (url !== undefined && host !== undefined) {
+ throw new Error(
+ 'Redis transports are mutually exclusive: set only one of REDIS_URL or REDIS_HOST (or enable Sentinel).',
+ );
+ }
+ if (url !== undefined) {
+ return { mode: 'standalone', url };
+ }
+ if (host !== undefined) {
+ const port = parsePositiveInt({
+ envKey: 'REDIS_PORT',
+ raw: getEnv('REDIS_PORT'),
+ defaultValue: 6379,
+ });
+ return {
+ mode: 'standalone',
+ url: buildRedisStandaloneUrl({
+ host,
+ port,
+ database,
+ username,
+ password: passwordOrUndefined,
+ }),
+ };
+ }
+
+ return undefined;
+}
+
/**
* Postgres connection string for distributed mode.
* Prefers `DATABASE_URL` when set (Railway / managed Postgres); otherwise builds from `POSTGRES_*`.
@@ -744,8 +866,30 @@ export type DistributedServerConfiguration = SharedServerConfiguration & {
* Env: `POSTGRES_SCHEMA`. Default `trueforge`.
*/
POSTGRES_SCHEMA: string;
- /** Peering URL shared by all replicas. Env: `REDIS_URL`. Default `redis://localhost:6379`. */
- REDIS_URL: string;
+ /**
+ * Resolved Redis transport (exactly one of url / host / sentinel), or undefined.
+ * Required for the server process; optional for controller / migrate (no peering).
+ * Env: mutually exclusive `REDIS_URL`, `REDIS_HOST`, or `REDIS_SENTINEL_*`.
+ */
+ REDIS_CONNECTION: RedisConnection | undefined;
+ /** Socket connect timeout for Redis clients. Env: `REDIS_CONNECT_TIMEOUT_MS`. Default 20000. */
+ REDIS_CONNECT_TIMEOUT_MS: number;
+ /** Client ping interval for Redis keepalive. Env: `REDIS_PING_INTERVAL_MS`. Default 5000. */
+ REDIS_PING_INTERVAL_MS: number;
+ /** Enable TLS for Redis (and Sentinel when used). Env: `REDIS_TLS_ENABLED`. Default false. */
+ REDIS_TLS_ENABLED: boolean;
+ /** CA cert path or inline PEM. Env: `REDIS_TLS_CA_CERT`. */
+ REDIS_TLS_CA_CERT: string | undefined;
+ /** Verify server cert. Env: `REDIS_TLS_REJECT_UNAUTHORIZED`. Default true. */
+ REDIS_TLS_REJECT_UNAUTHORIZED: boolean;
+ /** TLS SNI server name. Env: `REDIS_TLS_SERVERNAME`. */
+ REDIS_TLS_SERVERNAME: string | undefined;
+ /** Client cert path or inline PEM (mTLS). Env: `REDIS_TLS_CERT`. */
+ REDIS_TLS_CERT: string | undefined;
+ /** Client key path or inline PEM (mTLS). Env: `REDIS_TLS_KEY`. */
+ REDIS_TLS_KEY: string | undefined;
+ /** Client key passphrase. Env: `REDIS_TLS_KEY_PASSPHRASE`. */
+ REDIS_TLS_KEY_PASSPHRASE: string | undefined;
/**
* OIDC configuration for server authentication.
* Undefined means browser login is disabled.
@@ -990,7 +1134,32 @@ const configuration: ServerConfiguration = standalone
defaultValue: 60_000,
}),
POSTGRES_SCHEMA: parsePostgresSchema(getEnv('POSTGRES_SCHEMA')),
- REDIS_URL: resolveRedisUrl(),
+ REDIS_CONNECTION: resolveRedisConnection(),
+ REDIS_CONNECT_TIMEOUT_MS: parsePositiveInt({
+ envKey: 'REDIS_CONNECT_TIMEOUT_MS',
+ raw: getEnv('REDIS_CONNECT_TIMEOUT_MS'),
+ defaultValue: 20_000,
+ }),
+ REDIS_PING_INTERVAL_MS: parsePositiveInt({
+ envKey: 'REDIS_PING_INTERVAL_MS',
+ raw: getEnv('REDIS_PING_INTERVAL_MS'),
+ defaultValue: 5_000,
+ }),
+ REDIS_TLS_ENABLED: parseBoolean({
+ envKey: 'REDIS_TLS_ENABLED',
+ raw: getEnv('REDIS_TLS_ENABLED'),
+ defaultValue: false,
+ }),
+ REDIS_TLS_CA_CERT: getEnv('REDIS_TLS_CA_CERT'),
+ REDIS_TLS_REJECT_UNAUTHORIZED: parseBoolean({
+ envKey: 'REDIS_TLS_REJECT_UNAUTHORIZED',
+ raw: getEnv('REDIS_TLS_REJECT_UNAUTHORIZED'),
+ defaultValue: true,
+ }),
+ REDIS_TLS_SERVERNAME: getEnv('REDIS_TLS_SERVERNAME'),
+ REDIS_TLS_CERT: getEnv('REDIS_TLS_CERT'),
+ REDIS_TLS_KEY: getEnv('REDIS_TLS_KEY'),
+ REDIS_TLS_KEY_PASSPHRASE: getEnv('REDIS_TLS_KEY_PASSPHRASE'),
OIDC: resolveOIDCConfig(),
AUTOMATICALLY_MOVE_TRUEFORGE_TABLES_FROM_PUBLIC_TO_TRUEFORGE_SCHEMA: parseBoolean({
envKey: 'AUTOMATICALLY_MOVE_TRUEFORGE_TABLES_FROM_PUBLIC_TO_TRUEFORGE_SCHEMA',
diff --git a/packages/trueforge/src/main.ts b/packages/trueforge/src/main.ts
index 3308c51e5..50a85f796 100644
--- a/packages/trueforge/src/main.ts
+++ b/packages/trueforge/src/main.ts
@@ -56,9 +56,8 @@ import {
type ISessionStore,
type TurnStreamingEvent,
} from '@truefoundry/trueforge-core/agent-session';
-import { RequestReplyExecutor, RequestReplyRouter } from '@truefoundry/trueforge-core/request-reply';
+import { RequestReplyExecutor, RequestReplyRouter, type RedisClient } from '@truefoundry/trueforge-core/request-reply';
import type { Kysely, Transaction } from 'kysely';
-import type { RedisClientType } from 'redis';
import type { Logger } from 'winston';
import { createServerApp } from './app';
@@ -93,6 +92,7 @@ import type { IOAuthTokenStore } from './mcp/auth/types';
import { PACKAGE_VERSION } from './packageVersion';
import { ActiveTurnRegistry } from './runtime/activeTurns';
import { EventSubscriptionRegistry } from './runtime/event-subscription';
+import type { ConnectedRedis } from './runtime/redis';
import { printStandaloneStartupBanner } from './startupBanner';
import { InlineMcpServerStore } from './truefoundry/InlineMcpServerStore';
import { parseInlineMcpServers, parseInlineSkills, X_TFG_MCP, X_TFG_SKILLS } from './truefoundry/inlineResources';
@@ -136,7 +136,8 @@ interface ServerPersistence<TTransaction> {
agentStore: IAgentStore<TTransaction>;
turnSkillsResolverStore: Pick<ISkillStore<TTransaction>, 'resolveTurnSkills'>;
destroyDb: () => Promise<void>;
- redis: RedisClientType | undefined;
+ /** Connected Redis (client + mode) for distributed peering; undefined in standalone. */
+ redis: ConnectedRedis | undefined;
/** One shared client for TrueFoundry store resolvers + auth; undefined when TrueFoundry mode is off. */
serviceFoundryClient: TrueFoundryServiceFoundryServerClient | undefined;
}
@@ -379,7 +380,16 @@ async function createDistributedPersistence(options: {
DATABASE_POOL_MAX: databasePoolMax,
POSTGRES_STATEMENT_TIMEOUT_MS: statementTimeoutMs,
POSTGRES_IDLE_IN_TRANSACTION_SESSION_TIMEOUT_MS: idleInTransactionSessionTimeoutMs,
- REDIS_URL: redisUrl,
+ REDIS_CONNECTION: redisConnection,
+ REDIS_CONNECT_TIMEOUT_MS: redisConnectTimeoutMs,
+ REDIS_PING_INTERVAL_MS: redisPingIntervalMs,
+ REDIS_TLS_ENABLED: redisTlsEnabled,
+ REDIS_TLS_CA_CERT: redisTlsCaCert,
+ REDIS_TLS_REJECT_UNAUTHORIZED: redisTlsRejectUnauthorized,
+ REDIS_TLS_SERVERNAME: redisTlsServerName,
+ REDIS_TLS_CERT: redisTlsCert,
+ REDIS_TLS_KEY: redisTlsKey,
+ REDIS_TLS_KEY_PASSPHRASE: redisTlsKeyPassphrase,
EXECUTOR_ID: executorId,
} = configuration;
@@ -423,6 +433,12 @@ async function createDistributedPersistence(options: {
});
await migrateToLatest(db);
logger.info(`Executor id: ${executorId}`);
+ if (redisConnection === undefined) {
+ throw new Error(
+ 'Set exactly one Redis transport for the server when STANDALONE=false: REDIS_URL, REDIS_HOST, or ' +
+ 'REDIS_SENTINEL_ENABLED with REDIS_SENTINEL_NODES and REDIS_SENTINEL_MASTER_NAME.',
+ );
+ }
const serviceFoundryClient = createServiceFoundryServerClient(logger);
const tokenStore = new PostgresOAuthTokenStore(db);
const modelProviderStore = new PostgresModelProviderStore(db);
@@ -502,7 +518,21 @@ async function createDistributedPersistence(options: {
agentStore,
turnSkillsResolverStore,
destroyDb: () => db.destroy(),
- redis: await connectRedis({ url: redisUrl, logger }),
+ redis: await connectRedis({
+ connection: redisConnection,
+ connectTimeoutMs: redisConnectTimeoutMs,
+ pingIntervalMs: redisPingIntervalMs,
+ logger,
+ tls: {
+ enabled: redisTlsEnabled,
+ caCert: redisTlsCaCert,
+ rejectUnauthorized: redisTlsRejectUnauthorized,
+ serverName: redisTlsServerName,
+ cert: redisTlsCert,
+ key: redisTlsKey,
+ keyPassphrase: redisTlsKeyPassphrase,
+ },
+ }),
serviceFoundryClient,
};
}
@@ -526,7 +556,7 @@ async function createServerRuntime<TTransaction>(persistence: ServerPersistence<
const activeTurns = new ActiveTurnRegistry();
const requestReplyRouter = new RequestReplyRouter();
- const eventSubscriptions = new EventSubscriptionRegistry<TurnStreamingEvent>(redis);
+ const eventSubscriptions = new EventSubscriptionRegistry<TurnStreamingEvent>(redis?.client);
const sessions = new Sessions({ sessionStore });
const oidc = isOidcConfigured(configuration) ? configuration.OIDC : undefined;
@@ -626,7 +656,7 @@ async function createServerRuntime<TTransaction>(persistence: ServerPersistence<
sessionMetricsStore,
sessions,
activeTurns,
- redis,
+ redis: redis?.client,
requestReplyRouter,
eventSubscriptions,
logger,
@@ -693,22 +723,26 @@ try {
}
// After createServerApp so every request-reply route is registered before
- // the executor starts consuming messages. The executor needs a dedicated
+ // the executor starts consuming messages. Standalone Redis needs a dedicated
// subscriber connection (a subscribed client cannot issue normal commands);
- // this process owns its lifecycle. Connect before init() so init() awaits
- // the initial subscribe + heartbeat — the replica is reachable for peering
- // before the HTTP server starts.
- let requestReplySubscriber: RedisClientType | undefined;
+ // Sentinel owns pub/sub on the shared client. Connect before init() so init()
+ // awaits the initial subscribe + heartbeat — the replica is reachable for
+ // peering before the HTTP server starts.
+ let requestReplySubscriber: RedisClient | undefined;
let requestReplyExecutor: RequestReplyExecutor | undefined;
if (redis) {
- requestReplySubscriber = redis.duplicate();
- requestReplySubscriber.on('error', (error: Error) => {
- logger.error('[RedisSubscriber] Client error', extractErrorLogFields(error));
- });
- await requestReplySubscriber.connect();
+ if (redis.mode !== 'sentinel') {
+ requestReplySubscriber = redis.client.duplicate();
+ requestReplySubscriber.on('error', (error: Error) => {
+ logger.error('[RedisSubscriber] Client error', extractErrorLogFields(error));
+ });
+ await requestReplySubscriber.connect();
+ } else {
+ requestReplySubscriber = redis.client;
+ }
requestReplyExecutor = new RequestReplyExecutor({
executorId: configuration.EXECUTOR_ID,
- redis,
+ redis: redis.client,
subscriberClient: requestReplySubscriber,
requestHandler: requestReplyRouter.createRequestHandler(),
logger,
@@ -779,14 +813,20 @@ try {
await activeTurns.shutdownAndWait(CancellationReason.Abandoned);
await closed;
// Stop serving peer requests (waits for in-flight replies), then close
- // the clients this process owns: the subscriber duplicate and the primary.
+ // Redis clients (Set dedupes when Sentinel shares one client for pub/sub).
await requestReplyExecutor?.drain();
- await requestReplySubscriber?.close().catch((error: unknown) => {
- logger.warn('[Redis] Error closing subscriber client during shutdown', extractErrorLogFields(error));
- });
- await redis?.close().catch((error: unknown) => {
- logger.warn('[Redis] Error closing client during shutdown', extractErrorLogFields(error));
- });
+ const redisClients = new Set<RedisClient>();
+ if (requestReplySubscriber !== undefined) {
+ redisClients.add(requestReplySubscriber);
+ }
+ if (redis !== undefined) {
+ redisClients.add(redis.client);
+ }
+ for (const client of redisClients) {
+ await client.close().catch((error: unknown) => {
+ logger.warn('[Redis] Error closing client during shutdown', extractErrorLogFields(error));
+ });
+ }
if (configuration.STANDALONE) {
await removeCodeModeSocketParent(configuration.CODE_MODE_SOCKET_PARENT).catch((error: unknown) => {
logger.warn('Error removing Code Mode socket parent during shutdown', extractErrorLogFields(error));
diff --git a/packages/trueforge/src/runtime/event-subscription/index.ts b/packages/trueforge/src/runtime/event-subscription/index.ts
index 0485b4730..1fe701545 100644
--- a/packages/trueforge/src/runtime/event-subscription/index.ts
+++ b/packages/trueforge/src/runtime/event-subscription/index.ts
@@ -1,4 +1,4 @@
-import type { RedisClientType } from 'redis';
+import type { RedisClient } from '@truefoundry/trueforge-core/request-reply';
import { InMemoryEventStreamStore, InMemoryEventSubscription } from './inMemory';
import { RedisEventSubscription } from './redis';
@@ -47,7 +47,7 @@ export class EventSubscriptionRegistry<T extends object> {
/** One store for the whole process so producers and subscribers share streams. */
private readonly memoryStore = new InMemoryEventStreamStore<T>();
- constructor(private readonly redis: RedisClientType | undefined) {}
+ constructor(private readonly redis: RedisClient | undefined) {}
get(streamId: string): EventSubscription<T> {
if (this.redis) {
diff --git a/packages/trueforge/src/runtime/event-subscription/redis.ts b/packages/trueforge/src/runtime/event-subscription/redis.ts
index 54b2bb8ff..f0ee5e8fb 100644
--- a/packages/trueforge/src/runtime/event-subscription/redis.ts
+++ b/packages/trueforge/src/runtime/event-subscription/redis.ts
@@ -1,5 +1,5 @@
+import type { RedisClient } from '@truefoundry/trueforge-core/request-reply';
import { setTimeout as sleep } from 'node:timers/promises';
-import type { RedisClientType } from 'redis';
import {
StreamGoneError,
SUBSCRIBE_STREAM_THRESHOLD_MS,
@@ -31,7 +31,7 @@ export class RedisEventSubscription<T extends object> implements EventSubscripti
private nextSequenceNumber = 1;
constructor(
- private readonly redis: RedisClientType,
+ private readonly redis: RedisClient,
private readonly streamId: string,
) {}
diff --git a/packages/trueforge/src/runtime/redis.ts b/packages/trueforge/src/runtime/redis.ts
index 3710c4db7..9563ba04b 100644
--- a/packages/trueforge/src/runtime/redis.ts
+++ b/packages/trueforge/src/runtime/redis.ts
@@ -1,21 +1,227 @@
/**
* Primary Redis connection, owned by the server: created and connected at
* boot, closed last during shutdown. Injected into the request-reply
- * transport (which duplicates it only for its subscriber).
+ * transport (which duplicates it only for its subscriber in standalone mode;
+ * Sentinel shares the same client for pub/sub).
*/
+import { existsSync, readFileSync } from 'node:fs';
+import { setTimeout as sleep } from 'node:timers/promises';
+
import { extractErrorLogFields } from '@truefoundry/trueforge-core/core';
-import { createClient, type RedisClientType } from 'redis';
+import { createClient, createSentinel, type RedisClientType, type RedisSentinelType } from 'redis';
import type { Logger } from 'winston';
-export async function connectRedis(input: { url: string; logger: Logger }): Promise<RedisClientType> {
+const DEFAULT_SENTINEL_PORT = 26379;
+const SENTINEL_RETRY_BASE_MS = 200;
+const SENTINEL_RETRY_MAX_MS = 3_000;
+
+/** Exactly one Redis transport, resolved at config load. */
+export type RedisConnection =
+ | { mode: 'standalone'; url: string }
+ | {
+ mode: 'sentinel';
+ nodes: string;
+ masterName: string;
+ database: number;
+ username?: string;
+ password?: string;
+ sentinelUsername?: string;
+ sentinelPassword?: string;
+ };
+
+export type ConnectedRedis =
+ { mode: 'standalone'; client: RedisClientType } | { mode: 'sentinel'; client: RedisSentinelType };
+
+/** Parse comma-separated `host:port` list into Sentinel root nodes. */
+export function parseRedisSentinelNodes(raw: string): { host: string; port: number }[] {
+ return raw
+ .split(',')
+ .map(entry => entry.trim())
+ .filter(Boolean)
+ .map(entry => {
+ const lastColon = entry.lastIndexOf(':');
+ if (lastColon === -1) {
+ return { host: entry, port: DEFAULT_SENTINEL_PORT };
+ }
+ const host = entry.slice(0, lastColon);
+ const parsedPort = Number.parseInt(entry.slice(lastColon + 1), 10);
+ return { host, port: Number.isNaN(parsedPort) ? DEFAULT_SENTINEL_PORT : parsedPort };
+ });
+}
+
+export interface RedisTlsInput {
+ enabled: boolean | undefined;
+ caCert: string | undefined;
+ rejectUnauthorized: boolean | undefined;
+ serverName: string | undefined;
+ cert: string | undefined;
+ key: string | undefined;
+ keyPassphrase: string | undefined;
+}
+
+/** node-redis socket TLS fields shared by data-node and Sentinel clients. */
+export interface RedisTlsSocketOptions {
+ tls: true;
+ rejectUnauthorized: boolean;
+ ca?: string;
+ cert?: string;
+ key?: string;
+ passphrase?: string;
+ servername?: string;
+}
+
+const PEM_MARKER = '-----BEGIN';
+
+function resolvePemMaterial({ value, label }: { value: string; label: string }): string {
+ if (value.includes(PEM_MARKER)) {
+ return value;
+ }
+ if (existsSync(value)) {
+ try {
+ return readFileSync(value, 'utf8');
+ } catch (error) {
+ throw new Error(
+ `[Redis] Failed to read ${label} from path "${value}": ${error instanceof Error ? error.message : String(error)}`,
+ { cause: error },
+ );
+ }
+ }
+ throw new Error(`[Redis] ${label} is neither a readable file path nor inline PEM (no "${PEM_MARKER}" marker found).`);
+}
+
+/** Shared TLS socket options for data nodes and Sentinel clients. */
+function buildTlsSocketOptions(tls: RedisTlsInput | undefined): RedisTlsSocketOptions | undefined {
+ if (!tls?.enabled) {
+ return undefined;
+ }
+ if ((tls.cert && !tls.key) || (tls.key && !tls.cert)) {
+ throw new Error(
+ '[Redis] TLS misconfigured: REDIS_TLS_CERT and REDIS_TLS_KEY must be set together ' +
+ '(provide both for mutual TLS, or neither).',
+ );
+ }
+
+ return {
+ tls: true,
+ rejectUnauthorized: tls.rejectUnauthorized ?? true,
+ ...(tls.caCert ? { ca: resolvePemMaterial({ value: tls.caCert, label: 'REDIS_TLS_CA_CERT' }) } : {}),
+ ...(tls.cert ? { cert: resolvePemMaterial({ value: tls.cert, label: 'REDIS_TLS_CERT' }) } : {}),
+ ...(tls.key ? { key: resolvePemMaterial({ value: tls.key, label: 'REDIS_TLS_KEY' }) } : {}),
+ ...(tls.keyPassphrase ? { passphrase: tls.keyPassphrase } : {}),
+ ...(tls.serverName ? { servername: tls.serverName } : {}),
+ };
+}
+
+/**
+ * Connect via Redis Sentinel with rebuild-and-retry.
+ *
+ * A standalone `createClient()` can recover from later socket drops via the
+ * library's automatic reconnect. A Sentinel client cannot: if the first
+ * `connect()` rejects, the instance is wedged (retrying `connect()` throws
+ * "already attempting to open"). Discard and rebuild on each failed attempt;
+ * only auth failures (`WRONGPASS` / `NOAUTH`) abort permanently.
+ */
+async function connectSentinelWithRetry(input: {
+ connection: Extract<RedisConnection, { mode: 'sentinel' }>;
+ clientDefaults: { disableOfflineQueue: true; pingInterval: number };
+ connectTimeoutMs: number;
+ socketTls: RedisTlsSocketOptions | undefined;
+ logger: Logger;
+}): Promise<ReturnType<typeof createSentinel>> {
+ const { connection } = input;
+ const auth = {
+ database: connection.database,
+ ...(connection.username ? { username: connection.username } : {}),
+ ...(connection.password ? { password: connection.password } : {}),
+ };
+ let attempt = 0;
+ for (;;) {
+ const client = createSentinel({
+ name: connection.masterName,
+ sentinelRootNodes: parseRedisSentinelNodes(connection.nodes),
+ nodeClientOptions: {
+ ...input.clientDefaults,
+ ...auth,
+ socket: { connectTimeout: input.connectTimeoutMs, ...(input.socketTls ?? {}) },
+ },
+ sentinelClientOptions: {
+ socket: { connectTimeout: input.connectTimeoutMs, ...(input.socketTls ?? {}) },
+ ...(connection.sentinelUsername ? { username: connection.sentinelUsername } : {}),
+ ...(connection.sentinelPassword ? { password: connection.sentinelPassword } : {}),
+ },
+ });
+ // Without an 'error' listener node-redis crashes the process on emit.
+ client.on('error', (error: Error) => {
+ input.logger.error('[Redis] Client error', extractErrorLogFields(error));
+ });
+ try {
+ await client.connect();
+ return client;
+ } catch (error) {
+ attempt += 1;
+ try {
+ await client.close();
+ } catch {
+ // ignore — may never have opened
+ }
+ const message = error instanceof Error ? error.message : String(error);
+ if (message.includes('WRONGPASS') || message.includes('NOAUTH')) {
+ throw new Error(`[Redis] Sentinel connect failed: ${message}`, { cause: error });
+ }
+ input.logger.error('[Redis] Sentinel connect failed, rebuilding and retrying', {
+ attempt,
+ ...extractErrorLogFields(error),
+ });
+ await sleep(Math.min(SENTINEL_RETRY_BASE_MS * 2 ** attempt, SENTINEL_RETRY_MAX_MS));
+ }
+ }
+}
+
+export interface ConnectRedisInput {
+ connection: RedisConnection;
+ tls: RedisTlsInput | undefined;
+ logger: Logger;
+ connectTimeoutMs: number;
+ pingIntervalMs: number;
+}
+
+export async function connectRedis(input: ConnectRedisInput): Promise<ConnectedRedis> {
input.logger.info('Connecting to Redis');
- const client: RedisClientType = createClient({ url: input.url });
- // Without an 'error' listener node-redis crashes the process on emit;
- // reconnects are automatic, so log and keep running.
- client.on('error', (error: Error) => {
- input.logger.error('[Redis] Client error', extractErrorLogFields(error));
- });
- await client.connect();
- input.logger.info('Connected to Redis');
- return client;
+
+ const socketTls = buildTlsSocketOptions(input.tls);
+ const clientDefaults = {
+ disableOfflineQueue: true,
+ pingInterval: input.pingIntervalMs,
+ } as const;
+
+ switch (input.connection.mode) {
+ case 'sentinel': {
+ // @ts-expect-error TS2375 createSentinel return not assignable under EOPT
+ const client: RedisSentinelType = await connectSentinelWithRetry({
+ connection: input.connection,
+ clientDefaults,
+ connectTimeoutMs: input.connectTimeoutMs,
+ socketTls,
+ logger: input.logger,
+ });
+ input.logger.info('Connected to Redis');
+ return { mode: 'sentinel', client };
+ }
+ case 'standalone': {
+ const created = createClient({
+ url: input.connection.url,
+ ...clientDefaults,
+ socket: { connectTimeout: input.connectTimeoutMs, ...(socketTls ?? {}) },
+ });
+ // Without an 'error' listener node-redis crashes the process on emit.
+ created.on('error', (error: Error) => {
+ input.logger.error('[Redis] Client error', extractErrorLogFields(error));
+ });
+ await created.connect();
+ input.logger.info('Connected to Redis');
+ // @ts-expect-error TS2375 createClient return not assignable under EOPT
+ const client: RedisClientType = created;
+ return { mode: 'standalone', client };
+ }
+ }
}
diff --git a/packages/trueforge/tests/unit/config/config.test.ts b/packages/trueforge/tests/unit/config/config.test.ts
index 9b0dbe016..deb701b62 100644
--- a/packages/trueforge/tests/unit/config/config.test.ts
+++ b/packages/trueforge/tests/unit/config/config.test.ts
@@ -1,4 +1,5 @@
-import { getPublicUiBasePath, type ServerConfiguration } from '../../../src/config';
+import type { ServerConfiguration } from '../../../src/config';
+import { buildRedisStandaloneUrl, getPublicUiBasePath } from '../../../src/config';
import { resolveTrueFoundrySandboxProviderConfig } from '../../../src/truefoundry/resolveTrueFoundrySandboxProviderConfig';
/** Minimal distributed config slice for resolve tests (unused fields are irrelevant). */
@@ -129,3 +130,32 @@ describe('getPublicUiBasePath', () => {
).toBe('/custom/proxy/path/');
});
});
+
+describe('buildRedisStandaloneUrl', () => {
+ const base = {
+ port: 6379,
+ database: 0,
+ username: undefined,
+ password: undefined,
+ };
+
+ it('builds a hostname URL', () => {
+ expect(buildRedisStandaloneUrl({ ...base, host: 'redis.internal' })).toBe('redis://redis.internal:6379/0');
+ });
+
+ it('brackets bare IPv6 hosts so the URL is parseable', () => {
+ const url = buildRedisStandaloneUrl({ ...base, host: '::1' });
+ expect(url).toBe('redis://[::1]:6379/0');
+ expect(() => new URL(url)).not.toThrow();
+ });
+
+ it('keeps already-bracketed IPv6 hosts', () => {
+ expect(buildRedisStandaloneUrl({ ...base, host: '[2001:db8::1]', port: 6380, database: 2 })).toBe(
+ 'redis://[2001:db8::1]:6380/2',
+ );
+ });
+
+ it('leaves IPv4 hosts unbracketed', () => {
+ expect(buildRedisStandaloneUrl({ ...base, host: '127.0.0.1' })).toBe('redis://127.0.0.1:6379/0');
+ });
+});