Skip to content

Enhanced ChatGPT Clone: Features Agents, MCP, Skills, DeepSeek, Anthropic, AWS, OpenAI, Responses API, Azure, Groq, o1, GPT-5, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, Code Interpreter, langchain, DALL-E-3, OpenAPI Actions, Functions, Secure Multi-User Auth, Presets, open-source for self-hosting. Active

🛡️ fix: Keep Azure Keys Off Unserved Groups, Let Hides Take Their Specs, and Report Inert Azure Groups (#17411)

Danny Avila03eb7be12e83
diff --git a/docs/endpoint-overrides.md b/docs/endpoint-overrides.md
index 77f7f0af26..4f9fe809a3 100644
--- a/docs/endpoint-overrides.md
+++ b/docs/endpoint-overrides.md
@@ -62,6 +62,11 @@ applies, as for a `librechat.yaml` with an empty spec list.
A tombstone in the `__base__` profile applies to every principal. Every other profile merges after
`__base__`.
+Hiding or removing an endpoint is never refused because the profile's own model specs select it
+(`/changes`, PATCH and the single-field routes alike): those specs go with the endpoint. A write of
+a spec for an endpoint that is not configured answers 400 `unknown_spec_endpoint`, a spec in the
+same batch as the hide included.
+
Hiding is undone by:
- deleting the tombstone (`DELETE …/fields?fieldPath=endpoints.custom%5B%22Name%22%5D`);
@@ -170,7 +175,11 @@ preview are written with the group, never by its key. The Azure clients read wha
resolve to, so a hidden group or field takes effect only where the deployment resolves the groups
a profile leaves (`CONFIG_REDERIVE_ENDPOINTS=true`, `features.endpointRederive`): the groups left
are resolved again, and a group that no longer resolves once a field inside it is hidden is not
-served. Without it, the tombstone is stored and inert, as the profile's group overrides are.
+served; GET of the document and the resolved view list that tombstone in `ignored` with reason
+`azure_groups_unresolved`. A higher profile that writes such a group again gets no key from below
+it: enter the key with the group. Without the switch, the tombstone is stored and inert, as the
+profile's group overrides are, and GET of the document, the resolved view and the `/changes`
+response list each of them in `ignored` with reason `endpoint_rederive_off`.
## After upgrading
diff --git a/packages/api/src/admin/config.changes.integration.spec.ts b/packages/api/src/admin/config.changes.integration.spec.ts
index ebae1e89cb..749d948604 100644
--- a/packages/api/src/admin/config.changes.integration.spec.ts
+++ b/packages/api/src/admin/config.changes.integration.spec.ts
@@ -1514,6 +1514,79 @@ describe('POST …/changes: entries librechat.yaml owns', () => {
});
});
+describe("POST …/changes: hiding or removing the endpoint a profile's model spec selects", () => {
+ const spec = (name: string, endpointName: string) => ({
+ name,
+ label: name,
+ preset: { endpoint: endpointName, model: 'm' },
+ });
+
+ it.each([
+ ['a keyed tombstone', { op: 'tombstone', fieldPath: 'endpoints.custom["yaml-endpoint"]' }],
+ ['a tombstone of the list', { op: 'tombstone', fieldPath: 'endpoints.custom' }],
+ ])("takes the spec with %s of librechat.yaml's endpoint", async (_case, op) => {
+ const params = nextParams();
+ const specs = [spec('yaml', 'yaml-endpoint'), spec('openai', 'openAI')];
+ expect(
+ (await patchFields([{ fieldPath: 'modelSpecs.list', value: specs }], params)).status,
+ ).toBe(200);
+
+ const { status, body } = await send({ ops: [op] }, params);
+
+ expect(status).toBe(200);
+ expect(body.applied.map(({ opIndex }) => opIndex)).toEqual([0]);
+ expect((await storedDocument(params.principalId))!.tombstones).toEqual([op.fieldPath]);
+ });
+
+ it.each([
+ ['removeEntry', { op: 'removeEntry', fieldPath: 'endpoints.custom', key: 'Router' }],
+ ['a keyed unset', { op: 'unset', fieldPath: 'endpoints.custom["Router"]' }],
+ ['a keyed tombstone', { op: 'tombstone', fieldPath: 'endpoints.custom["Router"]' }],
+ ['an unset of the list', { op: 'unset', fieldPath: 'endpoints.custom' }],
+ ])("takes the spec with %s of the profile's own endpoint", async (_case, op) => {
+ const params = nextParams();
+ const written = await patchFields(
+ [
+ { fieldPath: 'endpoints.custom', value: [endpoint('Router')] },
+ { fieldPath: 'modelSpecs.list', value: [spec('router', 'Router')] },
+ ],
+ params,
+ );
+ expect(written.status).toBe(200);
+
+ const { status, body } = await send({ ops: [op] }, params);
+
+ expect(status).toBe(200);
+ expect(body.applied.map(({ opIndex }) => opIndex)).toEqual([0]);
+ const { overrides } = (await storedDocument(params.principalId))!;
+ expect(valueAt(overrides, ['endpoints', 'custom']) ?? []).toEqual([]);
+ expect(valueAt(overrides, ['modelSpecs', 'list'])).toEqual([spec('router', 'Router')]);
+ });
+
+ it('still refuses a spec the batch writes for the endpoint it hides', async () => {
+ const params = nextParams();
+
+ const { status, body } = await send(
+ {
+ ops: [
+ { op: 'tombstone', fieldPath: 'endpoints.custom["yaml-endpoint"]' },
+ { op: 'set', fieldPath: 'modelSpecs.list', value: [spec('yaml', 'yaml-endpoint')] },
+ ],
+ },
+ params,
+ );
+
+ expect(status).toBe(400);
+ expect(body.issues).toEqual([
+ expect.objectContaining({
+ path: 'modelSpecs.list.0.preset.endpoint',
+ code: 'unknown_spec_endpoint',
+ }),
+ ]);
+ expect(await storedDocument(params.principalId)).toBeNull();
+ });
+});
+
describe('POST …/changes: entry names the runtime folds together', () => {
const CUSTOM = 'endpoints.custom';
const ollama = (name: string, extra: Record<string, unknown> = {}) =>
diff --git a/packages/api/src/admin/config.report.spec.ts b/packages/api/src/admin/config.report.spec.ts
index 71bd73476e..2df759c5a8 100644
--- a/packages/api/src/admin/config.report.spec.ts
+++ b/packages/api/src/admin/config.report.spec.ts
@@ -595,6 +595,114 @@ describe('GET reports what the merge leaves out of a stored document', () => {
});
});
+describe('Azure groups a deployment without CONFIG_REDERIVE_ENDPOINTS stores but never serves', () => {
+ const azureGroup = (name: string, model: string) => ({
+ group: name,
+ apiKey: `${name}-key`,
+ instanceName: name,
+ version: '2024-10-21',
+ deploymentName: model,
+ models: { [model]: true },
+ });
+ const AZURE_YAML = {
+ endpoints: {
+ azureOpenAI: { groups: [azureGroup('east', 'gpt-4o'), azureGroup('north', 'o3')] },
+ },
+ };
+ const GROUPS = ['endpoints', 'azureOpenAI', 'groups'];
+ const handlersFor = (rederiveEndpoints: boolean) =>
+ createHandlers({
+ getAppConfig: async () => ({ config: structuredClone(AZURE_YAML) }) as never,
+ rederiveEndpoints,
+ });
+ const ops = [
+ {
+ op: 'upsertEntry',
+ fieldPath: 'endpoints.azureOpenAI.groups',
+ entry: azureGroup('west', 'gpt-5'),
+ },
+ { op: 'tombstone', fieldPath: 'endpoints.azureOpenAI.groups["east"]' },
+ { op: 'tombstone', fieldPath: 'endpoints.azureOpenAI.groups["north"].models.o3' },
+ { op: 'set', fieldPath: 'endpoints.azureOpenAI.titleModel', value: 'gpt-4o' },
+ ];
+
+ it('reports each group write and tombstone of them as inert, on the batch and on a read', async () => {
+ const params = nextParams();
+ const handlers = handlersFor(false);
+
+ const write = await call(handlers.applyConfigChanges, { params, body: { ops } });
+
+ expect(write.statusCode).toBe(200);
+ expect(
+ (write.body?.ignored as Array<{ opIndex: number; segments: string[]; reason: string }>).map(
+ ({ opIndex, segments, reason }) => [opIndex, segments, reason],
+ ),
+ ).toEqual([
+ [0, GROUPS, 'endpoint_rederive_off'],
+ [1, GROUPS, 'endpoint_rederive_off'],
+ [2, GROUPS, 'endpoint_rederive_off'],
+ ]);
+ const read = await call(handlers.getConfig, { params });
+ expect(describeAll(read.body?.ignored)).toEqual([
+ 'override:endpoints.azureOpenAI.groups:endpoint_rederive_off',
+ 'tombstone:endpoints.azureOpenAI.groups["east"]:endpoint_rederive_off',
+ 'tombstone:endpoints.azureOpenAI.groups["north"].models.o3:endpoint_rederive_off',
+ ]);
+
+ const whole = nextParams();
+ const replaced = await call(handlers.applyConfigChanges, {
+ params: whole,
+ body: {
+ ops: [
+ { op: 'tombstone', fieldPath: 'endpoints.azureOpenAI.groups' },
+ {
+ op: 'set',
+ fieldPath: 'endpoints.azureOpenAI',
+ value: { groups: [azureGroup('west', 'gpt-5')] },
+ },
+ ],
+ },
+ });
+ expect(replaced.statusCode).toBe(200);
+ expect(replaced.body?.ignored).toEqual([
+ { opIndex: 0, op: 'tombstone', segments: GROUPS, reason: 'endpoint_rederive_off' },
+ { opIndex: 1, op: 'set', segments: GROUPS, reason: 'endpoint_rederive_off' },
+ ]);
+ });
+
+ it('reports none of them where the deployment resolves the groups', async () => {
+ const params = nextParams();
+ const handlers = handlersFor(true);
+
+ const write = await call(handlers.applyConfigChanges, { params, body: { ops } });
+
+ expect(write.statusCode).toBe(200);
+ expect(write.body?.ignored).toEqual([]);
+ expect((await call(handlers.getConfig, { params })).body?.ignored).toEqual([]);
+ });
+
+ it('reports a tombstone that leaves its group unable to resolve where the groups are resolved', async () => {
+ const params = nextParams();
+ const handlers = handlersFor(true);
+ const tombstone = 'endpoints.azureOpenAI.groups["north"].instanceName';
+
+ const write = await call(handlers.applyConfigChanges, {
+ params,
+ body: { ops: [{ op: 'tombstone', fieldPath: tombstone }] },
+ });
+
+ expect(write.statusCode).toBe(200);
+ expect((await call(handlers.getConfig, { params })).body?.ignored).toEqual([
+ {
+ path: tombstone,
+ segments: [...GROUPS, 'north', 'instanceName'],
+ reason: 'azure_groups_unresolved',
+ kind: 'tombstone',
+ },
+ ]);
+ });
+});
+
describe('every 4xx a config handler answers carries a stable code', () => {
const params = { principalType: 'role', principalId: 'codes' };
const cases: Array<[string, keyof AdminConfigHandlers, Record<string, unknown>, string]> = [
diff --git a/packages/api/src/admin/config.ts b/packages/api/src/admin/config.ts
index 3db4dbe047..22b2cc4dc1 100644
--- a/packages/api/src/admin/config.ts
+++ b/packages/api/src/admin/config.ts
@@ -2401,6 +2401,40 @@ function getIgnoredParts(change: PreparedChange, held: YamlHeld): IgnoredPart[]
return [...parts, ...kept];
}
+const AZURE_GROUPS_SEGMENTS = ['endpoints', EModelEndpoint.azureOpenAI, 'groups'];
+
+function isAzureGroupsTarget(segments: readonly string[]): boolean {
+ return (
+ segments.length === AZURE_GROUPS_SEGMENTS.length &&
+ isSegmentPrefix(AZURE_GROUPS_SEGMENTS, segments)
+ );
+}
+
+/**
+ * The parts of an applied change the merge leaves inert where the deployment does not resolve
+ * Azure groups (`CONFIG_REDERIVE_ENDPOINTS` off; `endpoint_rederive_off`): a `set` of the groups,
+ * or of an object whose value holds them, an `upsertEntry` or `rename` of a group, and a tombstone
+ * of the groups or of one of them (keyed). An `unset` or `removeEntry` only clears what is stored.
+ */
+function getInertGroupParts(change: PreparedChange): IgnoredPart[] {
+ const { segments } = change.target;
+ const whole: IgnoredPart[] = [{ segments: [...segments], reason: 'endpoint_rederive_off' }];
+ if (change.op === 'tombstone' || change.op === 'upsertEntry' || change.op === 'rename') {
+ return isAzureGroupsTarget(segments) ? whole : [];
+ }
+ if (change.op !== 'set' || change.paths) {
+ return [];
+ }
+ if (isSegmentPrefix(AZURE_GROUPS_SEGMENTS, segments)) {
+ return whole;
+ }
+ const inside = AZURE_GROUPS_SEGMENTS.slice(segments.length);
+ return isSegmentPrefix(segments, AZURE_GROUPS_SEGMENTS) &&
+ Array.isArray(getAtPath(change.value, inside))
+ ? [{ segments: [...AZURE_GROUPS_SEGMENTS], reason: 'endpoint_rederive_off' }]
+ : [];
+}
+
/**
* A `set` of a whole `mcpServers` value without the servers librechat.yaml runs as local
* processes, re-encrypted, and the parts it left out; with nothing left, it resets the path. Any
@@ -4571,6 +4605,11 @@ export function createAdminConfigHandlers(deps: AdminConfigDeps): {
for (const part of getIgnoredParts(change, yamlHeld)) {
report.ignored.push({ ...ref, ...part });
}
+ if (!rederiveEndpoints) {
+ for (const part of getInertGroupParts(change)) {
+ report.ignored.push({ ...ref, ...part });
+ }
+ }
state = next;
}
diff --git a/packages/api/src/app/rebinding.spec.ts b/packages/api/src/app/rebinding.spec.ts
index 8d9e20422e..913476f786 100644
--- a/packages/api/src/app/rebinding.spec.ts
+++ b/packages/api/src/app/rebinding.spec.ts
@@ -1584,6 +1584,30 @@ describe('createSecretRebindingGuard', () => {
]);
});
+ it.each([
+ ['hides a field it needs', 'endpoints.azureOpenAI.groups["east"].instanceName'],
+ ['tombstones its groupMap entry', 'endpoints.azureOpenAI.groupMap.east'],
+ ])(
+ "sends no librechat.yaml key to a group a lower layer's tombstone (%s) left unserved",
+ (_case, tombstone) => {
+ const moved = { instanceName: 'evil', baseURL: 'https://evil.example/openai' };
+ const { merged } = mergeGroups(
+ [
+ { ...layer({}, '__base__', 0), tombstones: [tombstone] },
+ layer({ endpoints: { azureOpenAI: { groups: [{ group: 'east', ...moved }] } } }),
+ ],
+ azureBase(),
+ );
+
+ const east = merged.endpoints?.azureOpenAI?.groupMap?.east;
+ expect(east).toMatchObject(moved);
+ expect(east).not.toHaveProperty('apiKey');
+ expect(merged.endpoints?.azureOpenAI?.modelGroupMap).toEqual({
+ 'gpt-4o': { group: 'east' },
+ });
+ },
+ );
+
it("drops a layer's own key for a group whose destination a lower layer moved", () => {
const { merged, rebindings } = mergeGroups(
[
diff --git a/packages/api/src/app/report.spec.ts b/packages/api/src/app/report.spec.ts
index 8ca56c7985..d1a1b6da65 100644
--- a/packages/api/src/app/report.spec.ts
+++ b/packages/api/src/app/report.spec.ts
@@ -73,6 +73,26 @@ describe('getEndpointOverrideReport', () => {
});
});
+ it('lists a document whose only Azure groups override is a keyed tombstone of one', () => {
+ const report = getEndpointOverrideReport(
+ [
+ config('hide', {}, { tombstones: ['endpoints.azureOpenAI.groups["east"]'] }),
+ config(
+ 'inside',
+ {},
+ { tombstones: ['endpoints.azureOpenAI.groups["east"].models.gpt-4o'] },
+ ),
+ config('custom', {}, { tombstones: ['endpoints.custom["east"]'] }),
+ config('endpoint', {}, { tombstones: ['endpoints.azureOpenAI.titleModel'] }),
+ ],
+ yaml,
+ );
+
+ expect(report.azureGroups).toEqual(['role/hide', 'role/inside']);
+ expect(report.rederived).toEqual([]);
+ expect(report.unresolvedAzureGroups).toEqual([]);
+ });
+
it('lists Vertex AI overrides as applied only where librechat.yaml configures Vertex AI', () => {
const configs = [
config('vertex', { endpoints: { anthropic: { vertex: { models: ['m'] } } } }),
diff --git a/packages/api/src/app/report.ts b/packages/api/src/app/report.ts
index 89e0c3b90f..dfc7d43193 100644
--- a/packages/api/src/app/report.ts
+++ b/packages/api/src/app/report.ts
@@ -2,6 +2,8 @@ import { EModelEndpoint } from 'librechat-data-provider';
import {
logger,
getAtPath,
+ parseConfigPath,
+ isSegmentPrefix,
resolveRawConfig,
isVertexConfigured,
mergeConfigEntries,
@@ -49,7 +51,8 @@ export interface EndpointOverrideReport {
ignoredVertex: string[];
/**
* Documents whose Azure `groups`, merged by `group` over the groups below them (librechat.yaml's,
- * and the active Base profile's for any other document), resolve, or that tombstone them.
+ * and the active Base profile's for any other document), resolve, or that tombstone them: the
+ * whole list, a group, or a path inside one.
*/
azureGroups: string[];
/**
@@ -81,6 +84,22 @@ function getTombstones(config: ReportedConfig): string[] {
return (config.tombstones ?? []).filter((path): path is string => typeof path === 'string');
}
+/**
+ * Whether a tombstone removes Azure `groups` and nothing above them: the whole list, or a keyed
+ * tombstone of a group or of a path inside one (`endpoints.azureOpenAI.groups["east"].version`).
+ */
+function isAzureGroupsTombstone(tombstone: string): boolean {
+ if (tombstone === AZURE_GROUPS_PATH) {
+ return true;
+ }
+ const location = parseConfigPath(tombstone);
+ return (
+ location?.entry != null &&
+ location.segments.length === AZURE_GROUPS.length &&
+ isSegmentPrefix(AZURE_GROUPS, location.segments)
+ );
+}
+
/** Whether a document writes `segments`, or tombstones it or anything below it. */
function holds(config: ReportedConfig, tombstones: string[], segments: string[]): boolean {
const path = segments.join('.');
@@ -146,7 +165,7 @@ export function getEndpointOverrideReport(
}
const groups = getAtPath(config.overrides, AZURE_GROUPS);
if (!Array.isArray(groups)) {
- if (tombstones.includes(AZURE_GROUPS_PATH)) {
+ if (tombstones.some(isAzureGroupsTombstone)) {
report.azureGroups.push(description);
}
continue;
diff --git a/packages/data-schemas/src/app/resolution.layers.spec.ts b/packages/data-schemas/src/app/resolution.layers.spec.ts
index c4fd1b3323..f5b391d5c0 100644
--- a/packages/data-schemas/src/app/resolution.layers.spec.ts
+++ b/packages/data-schemas/src/app/resolution.layers.spec.ts
@@ -2,7 +2,12 @@ import { PrincipalType } from 'librechat-data-provider';
import type { DeepPartial, TCustomConfig } from 'librechat-data-provider';
import type { AppConfig, AdminConfigLayerIgnored } from '~/types';
import type { ConfigOverrideLayer } from './resolution';
-import { mergeConfigOverrides, resolveConfigLayers, resolveRawConfig } from './resolution';
+import {
+ resolveRawConfig,
+ resolveConfigLayers,
+ mergeConfigOverrides,
+ getIgnoredConfigOverrides,
+} from './resolution';
import { BASE_CONFIG_PRINCIPAL_ID } from '~/admin/capabilities';
import { AppService } from './service';
import logger from '~/config/winston';
@@ -425,6 +430,27 @@ describe('resolveConfigLayers', () => {
]);
});
+ it("reports a layer's groups as inert where the deployment does not resolve them", () => {
+ const configs = PARITY_CASES.find(([name]) => name.startsWith('Azure groups'))![1]();
+ const resolution = resolveConfigLayers(appConfig, configs);
+ const served = resolution.merged.endpoints?.azureOpenAI;
+ expect(served?.groupMap).toEqual({});
+ expect(served?.modelNames).toEqual([]);
+ expect(resolution.resolved.endpoints?.azureOpenAI?.groups?.map((entry) => entry.group)).toEqual(
+ ['east'],
+ );
+ expect(ignoredOf(resolution, 0)).toEqual([
+ {
+ path: 'endpoints.azureOpenAI.groups',
+ segments: ['endpoints', 'azureOpenAI', 'groups'],
+ reason: 'endpoint_rederive_off',
+ },
+ ]);
+ expect(reasons(ignoredOf(resolution, 1))).toEqual([
+ ['endpoints.azureOpenAI.groups', 'invalid_override'],
+ ]);
+ });
+
it('matches the validation floor where the merge applies a layer as written', () => {
const configs = [
base({ registration: { socialLogins: ['google'] } }),
@@ -449,3 +475,116 @@ describe('resolveConfigLayers', () => {
expect(configs).toEqual(before);
});
});
+
+describe('Azure groups the merge does not resolve, as each layer and a write report them', () => {
+ const azureGroup = (name: string, model: string) => ({
+ group: name,
+ apiKey: `${name}-key`,
+ instanceName: name,
+ version: '2024-10-21',
+ deploymentName: model,
+ models: { [model]: true },
+ });
+ const yaml = {
+ endpoints: {
+ azureOpenAI: { groups: [azureGroup('east', 'gpt-4o'), azureGroup('north', 'o3')] },
+ },
+ };
+ const groupsPath = 'endpoints.azureOpenAI.groups';
+ const groupsSegments = groupsPath.split('.');
+ let azureConfig: AppConfig;
+
+ beforeAll(async () => {
+ azureConfig = await AppService({ config: structuredClone(yaml) as DeepPartial<TCustomConfig> });
+ });
+
+ const servedGroups = (merged: AppConfig) =>
+ Object.keys(merged.endpoints?.azureOpenAI?.groupMap ?? {});
+
+ it('lists the groups and the tombstones of them a layer stores, without CONFIG_REDERIVE_ENDPOINTS', () => {
+ const writer = role({ endpoints: { azureOpenAI: { groups: [azureGroup('west', 'gpt-5')] } } }, [
+ 'endpoints.azureOpenAI.groups["east"]',
+ 'endpoints.azureOpenAI.groups["north"].models.o3',
+ ]);
+ const replacer = group({}, [groupsPath]);
+
+ const resolution = resolveConfigLayers(azureConfig, [writer, replacer]);
+
+ expect(servedGroups(resolution.merged)).toEqual(['east', 'north']);
+ expect(ignoredOf(resolution, 0)).toEqual([
+ {
+ path: 'endpoints.azureOpenAI.groups["east"]',
+ segments: [...groupsSegments, 'east'],
+ reason: 'endpoint_rederive_off',
+ tombstone: true,
+ },
+ {
+ path: 'endpoints.azureOpenAI.groups["north"].models.o3',
+ segments: [...groupsSegments, 'north', 'models', 'o3'],
+ reason: 'endpoint_rederive_off',
+ tombstone: true,
+ },
+ { path: groupsPath, segments: groupsSegments, reason: 'endpoint_rederive_off' },
+ ]);
+ expect(ignoredOf(resolution, 1)).toEqual([
+ {
+ path: groupsPath,
+ segments: groupsSegments,
+ reason: 'endpoint_rederive_off',
+ tombstone: true,
+ },
+ ]);
+ expect(getIgnoredConfigOverrides(yaml as Partial<TCustomConfig>, writer)).toEqual([
+ {
+ path: 'endpoints.azureOpenAI.groups["east"]',
+ segments: [...groupsSegments, 'east'],
+ reason: 'endpoint_rederive_off',
+ kind: 'tombstone',
+ },
+ {
+ path: 'endpoints.azureOpenAI.groups["north"].models.o3',
+ segments: [...groupsSegments, 'north', 'models', 'o3'],
+ reason: 'endpoint_rederive_off',
+ kind: 'tombstone',
+ },
+ {
+ path: groupsPath,
+ segments: groupsSegments,
+ reason: 'endpoint_rederive_off',
+ kind: 'override',
+ },
+ ]);
+
+ const applied = resolveConfigLayers(azureConfig, [writer, replacer], {
+ rederiveEndpoints: true,
+ });
+ expect(servedGroups(applied.merged)).toEqual([]);
+ expect(applied.layers.map(({ ignored }) => ignored)).toEqual([[], []]);
+ });
+
+ it('lists a tombstone that leaves its group unable to resolve, which is no longer served', () => {
+ const tombstone = 'endpoints.azureOpenAI.groups["north"].instanceName';
+ const hider = role({}, [tombstone]);
+ const unresolved = {
+ path: tombstone,
+ segments: [...groupsSegments, 'north', 'instanceName'],
+ reason: 'azure_groups_unresolved' as const,
+ };
+
+ const resolution = resolveConfigLayers(azureConfig, [hider], { rederiveEndpoints: true });
+
+ expect(servedGroups(resolution.merged)).toEqual(['east']);
+ expect(resolution.merged.endpoints?.azureOpenAI?.modelNames).toEqual(['gpt-4o']);
+ expect(ignoredOf(resolution, 0)).toEqual([{ ...unresolved, tombstone: true }]);
+ expect(
+ getIgnoredConfigOverrides(yaml as Partial<TCustomConfig>, hider, [], undefined, {
+ rederiveEndpoints: true,
+ }),
+ ).toEqual([{ ...unresolved, kind: 'tombstone' }]);
+
+ const kept = role({}, ['endpoints.azureOpenAI.groups["north"].models.o3']);
+ expect(
+ ignoredOf(resolveConfigLayers(azureConfig, [kept], { rederiveEndpoints: true }), 0),
+ ).toEqual([]);
+ });
+});
diff --git a/packages/data-schemas/src/app/resolution.spec.ts b/packages/data-schemas/src/app/resolution.spec.ts
index 10e1e801b0..47581d9490 100644
--- a/packages/data-schemas/src/app/resolution.spec.ts
+++ b/packages/data-schemas/src/app/resolution.spec.ts
@@ -4356,6 +4356,35 @@ describe('model spec endpoints', () => {
).toBeUndefined();
},
);
+
+ it('lets a change hide or remove the endpoint a stored spec selects, and refuses a spec it writes there', () => {
+ const router = { ...gateway, name: 'Router' };
+ const stored = {
+ overrides: {
+ endpoints: { custom: [router] },
+ modelSpecs: { list: [spec('Gateway'), spec('Router')] },
+ },
+ tombstones: [],
+ };
+ const change = (changes: Parameters<typeof applyConfigChanges>[1]) => {
+ const { state } = applyConfigChanges(stored, changes);
+ return getConfigChangeIssues(
+ state,
+ yaml,
+ stored,
+ changes.map((step) => [...step.segments]),
+ );
+ };
+
+ expect(change([{ op: 'tombstone', segments: ['endpoints', 'custom'] }])).toEqual([]);
+ expect(change([{ op: 'unset', segments: ['endpoints', 'custom'] }])).toEqual([]);
+ expect(
+ change([
+ { op: 'tombstone', segments: ['endpoints', 'custom'] },
+ { op: 'set', segments: ['modelSpecs', 'list'], value: [spec('Router')] },
+ ]).map(({ path, code }) => [path, code]),
+ ).toEqual([['modelSpecs.list.0.preset.endpoint', 'unknown_spec_endpoint']]);
+ });
});
describe('issue params', () => {
diff --git a/packages/data-schemas/src/app/resolution.ts b/packages/data-schemas/src/app/resolution.ts
index e77d5620d3..f26a6608ad 100644
--- a/packages/data-schemas/src/app/resolution.ts
+++ b/packages/data-schemas/src/app/resolution.ts
@@ -1320,6 +1320,11 @@ function getIssueLocation(segments: readonly string[], overrides: unknown): stri
* (at the same location, entries of merged-by-key arrays matched by key) are left to merge
* time so they do not block an unrelated change.
*
+ * A derived issue the merge copes with on its own (`WRITE_ONLY_DERIVED_ISSUES`: a model spec
+ * whose endpoint is not configured, which the merge leaves out) is reported only where the change
+ * writes, never as one it introduced elsewhere: hiding or removing the endpoint a stored spec
+ * selects takes the spec with it, as the single-field routes let it.
+ *
* The stored document is judged on the base its own tombstones leave, so an issue that a
* removed tombstone brings back counts as new. `lifted` names stored tombstones to treat as
* already removed there: tombstones the change replaced with values of its own.
@@ -1345,11 +1350,13 @@ export function getConfigChangeIssues(
candidateState.overrides,
applyConfigTombstones(base, candidateState.tombstones),
derivation,
- ).filter(
- (issue) =>
- !existing.has(getIssueLocation(issue.segments, candidateState.overrides)) ||
- written.some((segments) => isRelatedPath(issue.segments, [...segments])),
- );
+ ).filter((issue) => {
+ const touched = written.some((segments) => isRelatedPath(issue.segments, [...segments]));
+ if (touched || WRITE_ONLY_DERIVED_ISSUES.has(issue.code)) {
+ return touched;
+ }
+ return !existing.has(getIssueLocation(issue.segments, candidateState.overrides));
+ });
}
/**
@@ -1791,6 +1798,18 @@ function suppliesGroupKey(item: AnyObject | undefined): boolean {
return item != null && hasOwn(item, 'apiKey') && item.apiKey !== undefined;
}
+/** The key fields of a `groupMap` entry, which travel together. */
+const AZURE_GROUP_KEY_FIELDS = ['apiKey', 'apiKeyPreview', 'apiKeyBinding'] as const;
+
+/** A `groupMap` entry without its key fields. */
+function withoutGroupKey(entry: AnyObject): AnyObject {
+ const next: AnyObject = { ...entry };
+ for (const field of AZURE_GROUP_KEY_FIELDS) {
+ delete next[field];
+ }
+ return next;
+}
+
/**
* A layer's Azure `groups` resolved as AppService resolves librechat.yaml's
* (`resolveServedAzureGroups`), merged by `group` over the groups below (the YAML-shaped config
@@ -1799,7 +1818,9 @@ function suppliesGroupKey(item: AnyObject | undefined): boolean {
* read `groupMap`, `modelGroupMap` and the model lists, never `groups`, so those are what change:
* - `groupMap`: each group the layer names is resolved again; every other group keeps the entry
* below, as the merge left it. A named group keeps the key below unless the layer supplies one,
- * so a key a lower layer's merge dropped stays dropped.
+ * so a key a lower layer's merge dropped stays dropped, and a group the config below does not
+ * serve (one a lower layer's tombstone left unable to resolve) has none: its YAML-shaped item
+ * still holds the key below it, which no layer below sends anywhere.
* - `modelGroupMap` and the model lists: resolved from all the merged groups.
* A resolved field the layer also writes directly merges over them (objects) or replaces them.
* `null` when the merged groups do not resolve: the layer's groups then stay unread, with a
@@ -1842,9 +1863,13 @@ function resolveLayerAzureGroups(params: {
if (entry == null) {
continue;
}
+ if (suppliesGroupKey(item)) {
+ groupMap[name] = entry;
+ continue;
+ }
const inherited = isPlainObject(belowMap[name]) ? belowMap[name].apiKey : undefined;
groupMap[name] =
- suppliesGroupKey(item) || inherited === undefined ? entry : { ...entry, apiKey: inherited };
+ inherited === undefined ? withoutGroupKey(entry) : { ...entry, apiKey: inherited };
}
const alone: AnyObject = {};
for (const item of mergeOver(yamlGroups)) {
@@ -1854,12 +1879,7 @@ function resolveLayerAzureGroups(params: {
continue;
}
const entry = toAzureGroupMapEntry(item);
- if (!suppliesGroupKey(own)) {
- delete entry.apiKey;
- delete entry.apiKeyPreview;
- delete entry.apiKeyBinding;
- }
- alone[name] = entry;
+ alone[name] = suppliesGroupKey(own) ? entry : withoutGroupKey(entry);
}
const ownMap = layerAzure.groupMap;
const derived: Record<AzureResolvedGroupField, unknown> = {
@@ -1938,6 +1958,17 @@ function withServedAzureModels<T extends AppConfig>(
};
}
+/** A keyed tombstone of an Azure group (`endpoints.azureOpenAI.groups["g"]`, or a path inside it). */
+function readAzureGroupTombstone(path: string): ConfigEntryLocation | null {
+ const entry = isKeyedConfigPath(path) ? readConfigEntryTombstone(path) : null;
+ return entry != null && entry.segments.join('.') === AZURE_GROUPS_PATH ? entry : null;
+}
+
+/** Whether a tombstone removes Azure groups and nothing else: the whole list, or a keyed one. */
+function isAzureGroupsTombstone(path: string): boolean {
+ return path === AZURE_GROUPS_PATH || readAzureGroupTombstone(path) != null;
+}
+
/** The Azure groups a layer's keyed tombstones name (`endpoints.azureOpenAI.groups["g"]`), whole or not. */
function getAzureGroupTombstones(tombstones: readonly string[]): {
hidden: Set<string>;
@@ -1946,17 +1977,20 @@ function getAzureGroupTombstones(tombstones: readonly string[]): {
const hidden = new Set<string>();
const touched = new Set<string>();
for (const path of tombstones) {
- const entry = isKeyedConfigPath(path) ? readConfigEntryTombstone(path) : null;
- if (entry == null || entry.segments.join('.') !== AZURE_GROUPS_PATH) {
- continue;
+ const entry = readAzureGroupTombstone(path);
+ if (entry != null) {
+ (entry.entry.rest.length === 0 ? hidden : touched).add(entry.identity);
}
- (entry.entry.rest.length === 0 ? hidden : touched).add(entry.identity);
}
return { hidden, touched };
}
-/** The key fields of a `groupMap` entry, which travel together. */
-const AZURE_GROUP_KEY_FIELDS = ['apiKey', 'apiKeyPreview', 'apiKeyBinding'] as const;
+/** {@link withHiddenAzureGroups}' result: the served config, and the groups it no longer serves. */
+interface HiddenAzureGroups<T> {
+ merged: T;
+ /** The groups with a path hidden inside them that the groups left do not let resolve. */
+ unresolved: ReadonlySet<string>;
+}
/**
* `merged` once a layer's keyed tombstones of Azure groups (`tombstones`) applied, where the merge
@@ -1965,33 +1999,41 @@ const AZURE_GROUP_KEY_FIELDS = ['apiKey', 'apiKeyPreview', 'apiKeyBinding'] as c
* `modelGroupMap` and the model lists are resolved from the groups left. A group with a path
* hidden inside it is resolved again from what it keeps, with the key below it, which a tombstone
* never supplies; every other group keeps the entry below. Where the groups left do not resolve,
- * the groups with a path hidden inside them are served no more either, with a warning; where none
- * is left, no group is served (`withoutAzureGroups`).
+ * the groups with a path hidden inside them are served no more either (`unresolved`, judged on
+ * `raw` alone, with a warning unless `quiet`); where none is left, no group is served
+ * (`withoutAzureGroups`).
*/
function withHiddenAzureGroups<T extends AnyObject>(
merged: T,
raw: Partial<TCustomConfig>,
tombstones: readonly string[],
config: ConfigOverrideLayer,
-): T {
+ quiet = false,
+): HiddenAzureGroups<T> {
const { hidden, touched } = getAzureGroupTombstones(tombstones);
+ if (hidden.size === 0 && touched.size === 0) {
+ return { merged, unresolved: new Set() };
+ }
const azureSegments = AZURE_CONFIG_PATH.split('.');
const azure = getPath(merged, azureSegments);
- if ((hidden.size === 0 && touched.size === 0) || !isPlainObject(azure)) {
- return merged;
- }
const kept = getPath(raw, AZURE_GROUPS_PATH.split('.'));
let groups = Array.isArray(kept) ? kept.filter(isPlainObject) : [];
let resolved = resolveServedAzureGroups(groups, touched);
+ const unresolved = resolved.isValid ? new Set<string>() : touched;
if (!resolved.isValid) {
- logger.warn(
- `[mergeConfigOverrides] Serving none of the ${AZURE_GROUPS_PATH} ${[...touched].join(', ')} for ${describePrincipal(config)}: they do not resolve once its tombstones apply: ${resolved.errors.join('; ')}`,
- );
+ if (!quiet && isPlainObject(azure)) {
+ logger.warn(
+ `[mergeConfigOverrides] Serving none of the ${AZURE_GROUPS_PATH} ${[...touched].join(', ')} for ${describePrincipal(config)}: they do not resolve once its tombstones apply: ${resolved.errors.join('; ')}`,
+ );
+ }
groups = groups.filter((item) => typeof item.group !== 'string' || !touched.has(item.group));
resolved = resolveServedAzureGroups(groups, new Set());
}
+ if (!isPlainObject(azure)) {
+ return { merged, unresolved };
+ }
if (groups.length === 0 || !resolved.isValid) {
- return withoutAzureGroups(merged);
+ return { merged: withoutAzureGroups(merged), unresolved };
}
const belowMap = isPlainObject(azure.groupMap) ? azure.groupMap : {};
const groupMap: AnyObject = {};
@@ -2011,7 +2053,7 @@ function withHiddenAzureGroups<T extends AnyObject>(
}
groupMap[name] = next;
}
- return setPath(merged, azureSegments, {
+ const next = setPath(merged, azureSegments, {
...azure,
groupMap,
modelGroupMap: resolved.modelGroupMap,
@@ -2019,6 +2061,7 @@ function withHiddenAzureGroups<T extends AnyObject>(
assistantModels: resolved.assistantModels,
assistantGroups: resolved.assistantGroups,
}) as T;
+ return { merged: next, unresolved };
}
/** What {@link foldConfigLayers} leaves: the served config, the YAML-shaped one, and what layers wrote. */
@@ -2075,6 +2118,22 @@ function toLayerIgnored(
return { path: segments.join('.'), segments: [...segments], reason, ...extra };
}
+/** A reason both a write's `ignored` and a resolution's layer list give. */
+type SharedIgnoreReason = AdminConfigIgnoreReason & AdminConfigLayerIgnoreReason;
+
+/**
+ * Where an ignored tombstone sits: its path as stored, which DELETE `/fields` takes, and its
+ * segments, with a keyed tombstone's entry named by its key, as write responses name a
+ * credential in one.
+ */
+function getTombstoneLocation(path: string): { path: string; segments: string[] } {
+ const entry = isKeyedConfigPath(path) ? readConfigEntryTombstone(path) : null;
+ return {
+ path,
+ segments: entry ? [...entry.segments, entry.entry.key, ...entry.entry.rest] : path.split('.'),
+ };
+}
+
/** `tree` with its top-level keys in AppConfig names (`interface` → `interfaceConfig`, ...). */
function toAppConfigKeys(tree: Partial<TCustomConfig>): AnyObject {
const result: AnyObject = {};
@@ -2199,8 +2258,10 @@ interface FoldObservers {
* (`resolveLayerAzureGroups`), a tombstone of them removes those fields, and a layer that writes
* or removes `endpoints.anthropic.vertex`, or removes `endpoints.anthropic.models`, has Vertex AI
* derived again (`deriveVertexEndpoint`), keeping the models a layer set only while no later
- * layer has removed them;
- * without it, those overrides stay inert in `merged`, as before the merge derived them. Layers
+ * layer has removed them, and a tombstone inside a group that leaves it unable to resolve is
+ * reported (`azure_groups_unresolved`);
+ * without it, those overrides stay inert in `merged`, as before the merge derived them, and a
+ * layer's Azure groups and its tombstones of them are reported (`endpoint_rederive_off`). Layers
* are judged with `vertexConfigured` taken from `initialRaw`, librechat.yaml.
* `webSearchWriters` names the layer behind each paired web search field, by its position in
* that order, and `touches` what the layers wrote and removed. `afterLayer` runs after each layer
@@ -2287,8 +2348,28 @@ function foldConfigLayers<T extends AnyObject>(
track?.tombstoned.push(path);
recordWebSearchTombstone(webSearchWriters, path, removal);
}
+ const reportTombstone = (path: string, reason: SharedIgnoreReason) => {
+ const location = getTombstoneLocation(path);
+ layerSink?.({ ...location, reason, kind: 'tombstone' });
+ track?.ignored.push({ ...location, reason, tombstone: true });
+ };
if (resolvesAzureGroups) {
- merged = withHiddenAzureGroups(merged, raw, tombstoned, config);
+ const hidden = withHiddenAzureGroups(merged, raw, tombstoned, config, layerSink != null);
+ merged = hidden.merged;
+ for (const path of tombstoned) {
+ const entry = readAzureGroupTombstone(path);
+ if (
+ entry != null &&
+ entry.entry.rest.length > 0 &&
+ hidden.unresolved.has(entry.identity)
+ ) {
+ reportTombstone(path, 'azure_groups_unresolved');
+ }
+ }
+ } else {
+ for (const path of tombstoned.filter(isAzureGroupsTombstone)) {
+ reportTombstone(path, 'endpoint_rederive_off');
+ }
}
}
@@ -2415,9 +2496,13 @@ function foldConfigLayers<T extends AnyObject>(
tracker.resolved = deepMerge(tracker.resolved, track.contributed);
}
const layerAzure = getPath(applied, AZURE_CONFIG_PATH.split('.'));
+ const groupsSegments = AZURE_GROUPS_PATH.split('.');
+ if (!resolvesAzureGroups && isPlainObject(layerAzure) && Array.isArray(layerAzure.groups)) {
+ layerSink?.(toIgnored(groupsSegments, 'endpoint_rederive_off', 'override'));
+ track?.ignored.push(toLayerIgnored(groupsSegments, 'endpoint_rederive_off'));
+ }
if (resolvesAzureGroups && isPlainObject(layerAzure) && Array.isArray(layerAzure.groups)) {
const azureSegments = AZURE_CONFIG_PATH.split('.');
- const groupsSegments = AZURE_GROUPS_PATH.split('.');
const resolution = resolveLayerAzureGroups({
layerAzure,
groups: layerAzure.groups.filter(isPlainObject),
@@ -2434,9 +2519,7 @@ function foldConfigLayers<T extends AnyObject>(
merged = setPath(merged, azureSegments, resolution.served) as T;
contributed = setPath(remapped, azureSegments, resolution.contributed) as AnyObject;
} else {
- track?.ignored.push(
- toLayerIgnored(AZURE_GROUPS_PATH.split('.'), 'azure_groups_unresolved'),
- );
+ track?.ignored.push(toLayerIgnored(groupsSegments, 'azure_groups_unresolved'));
}
}
}
diff --git a/packages/data-schemas/src/types/admin.ts b/packages/data-schemas/src/types/admin.ts
index 5d7ffb5dec..79e1680b99 100644
--- a/packages/data-schemas/src/types/admin.ts
+++ b/packages/data-schemas/src/types/admin.ts
@@ -396,9 +396,12 @@ export type AdminConfigSkipReason =
/**
* Why a stored override, tombstone, change or part of one (at its own `segments`) has no effect
- * at runtime. `POST …/changes` reports `process_mcp_server`, `base_only_path` and
- * `deployment_only_path` for the changes it applies; GET `/:principalType/:principalId` reports
- * every reason for what the document stores.
+ * at runtime. `POST …/changes` reports `process_mcp_server`, `base_only_path`,
+ * `deployment_only_path` and `endpoint_rederive_off` for the changes it applies; GET
+ * `/:principalType/:principalId` reports every reason for what the document stores. A keyed
+ * tombstone keeps its keyed `path` (`endpoints.azureOpenAI.groups["east"].version`, which DELETE
+ * `/fields` takes) and names its entry by key in `segments` (`endpoints`, `azureOpenAI`,
+ * `groups`, `east`, `version`), as a credential in an entry is named.
* - `process_mcp_server`: an override or tombstone of an MCP server `librechat.yaml` runs as a
* local process, whose configuration overrides never change, or each such server a
* whole-section `mcpServers` tombstone leaves in place.
@@ -410,6 +413,13 @@ export type AdminConfigSkipReason =
* - `unknown_section`: a top-level key `configSchema` does not define (`sectionAllowlist`).
* - `base_only_section`, `dedicated_section`, `interface_permission`, `derived_path`: as in
* {@link AdminConfigSkipReason}, for a value or tombstone already stored.
+ * - `endpoint_rederive_off`: Azure `groups` a document writes, or a tombstone of them (the whole
+ * list, a group, or a path inside one), where the deployment does not resolve them
+ * (`CONFIG_REDERIVE_ENDPOINTS` off, `features.endpointRederive`): the Azure clients read the
+ * groups librechat.yaml resolves to. They are stored, and apply once the switch is on.
+ * `POST …/changes` reports it for the changes that write or tombstone them.
+ * - `azure_groups_unresolved`: a tombstone of a path inside an Azure group that leaves the group
+ * unable to resolve, so the Azure clients are served no such group.
*/
export type AdminConfigIgnoreReason =
| 'process_mcp_server'
@@ -420,7 +430,9 @@ export type AdminConfigIgnoreReason =
| 'base_only_section'
| 'dedicated_section'
| 'interface_permission'
- | 'derived_path';
+ | 'derived_path'
+ | 'endpoint_rederive_off'
+ | 'azure_groups_unresolved';
/**
* A credential the runtime merge leaves out of a document's layer, or a destination it sets back,
@@ -518,7 +530,12 @@ export type AdminConfigChangeErrorResponse = {
* - `interface_permission`: an interface field that is a role permission.
* - `runtime_stop`: an interface feature `librechat.yaml` turns off, which stays off.
* - `azure_groups_unresolved`: Azure `groups` that do not resolve (a duplicate group or model, a
- * model without a deployment or version), which the Azure clients never read.
+ * model without a deployment or version), which the Azure clients never read, or a tombstone of
+ * a path inside a group that leaves the group unable to resolve, so it is not served.
+ * - `endpoint_rederive_off`: Azure `groups` a layer writes, or its tombstone of them (the whole
+ * list, a group, or a path inside one), where the deployment does not resolve them
+ * (`CONFIG_REDERIVE_ENDPOINTS` off): `resolved` shows them merged, while the Azure clients read
+ * the groups librechat.yaml resolves to.
* - `azure_unserved`: an Azure model list, or `assistants`, the merged groups do not serve.
* - `secret_rebinding`: a credential the merge drops, or (`destination_restored`) a destination
* it sets back, because the credential would be sent somewhere it was not entered for
@@ -542,6 +559,7 @@ export type AdminConfigLayerIgnoreReason =
| 'interface_permission'
| 'runtime_stop'
| 'azure_groups_unresolved'
+ | 'endpoint_rederive_off'
| 'azure_unserved'
| 'secret_rebinding'
| 'destination_restored'
@@ -553,7 +571,8 @@ export type AdminConfigLayerIgnoreReason =
/**
* Part of a layer the merge does not apply as written. `segments` locate it in librechat.yaml
* keys: in the document's overrides (array items by index), or for a credential or destination
- * the merge drops or sets back, as write responses name it (a custom endpoint by its name). A
+ * the merge drops or sets back, or a keyed tombstone, as write responses name it (a custom
+ * endpoint or Azure group by its key, with `path` the tombstone as stored). A
* tombstone above a `base_only_path` or `deployment_only_path` field or a `runtime_stop` feature
* names that field, which the merge keeps while it removes the rest (write responses name such a
* field the same way).