Skip to content

Infisical is the open-source platform for securing secrets, certificates, access, and AI Agents.

feat(secrets): emit secret, folder and import events through the outbox

Calvind489d8be47a8
diff --git a/backend/CLAUDE.md b/backend/CLAUDE.md
index 808f6fbc93..cf0fd9bef4 100644
--- a/backend/CLAUDE.md
+++ b/backend/CLAUDE.md
@@ -802,9 +802,15 @@ await someDAL.transaction(async (tx) => {
anything for this resource is decided in the worker; a row nobody wanted is marked `Delivered` and
pruned within a day. If that ever matters for a hot event type, cache inside the consumer rather than
adding a read to the emit path.
-- Nothing is caught. A swallowed insert failure hands the caller a poisoned transaction (`25P02` on its
- next statement), and a payload that fails the consumer's `payloadSchema` is a bug at the emit site.
- Both checks run before any DB access.
+- A malformed event is dropped, not thrown: a bad key, a payload over 16KB, or a payload a consumer's
+ `payloadSchema` rejects (that consumer's row only) is logged and counted on
+ `infisical.event_outbox.dropped.count`, and the caller's write carries on. It is a bug at the emit
+ site, and failing a secret write over it is worse than the lost event. All three checks run before any
+ DB access, so the transaction is untouched. The insert itself is not caught: a swallowed insert failure
+ hands the caller a poisoned transaction (`25P02` on its next statement).
+- `emitMany(events, tx)` writes a batch in one go (eg one event per secret in a bulk write), in chunks
+ of 1000 rows to stay under Postgres's bind parameter limit. Every event gets one `eventId` (UUID),
+ shared by all its consumers' rows; `TEvent.id` is the consumer's row, `TEvent.eventId` the occurrence.
**How delivery works:**
@@ -866,6 +872,23 @@ emit site shows up in the logs on its first event.
Approval lifecycle events reach the outbox through `TApprovalResource.buildEvent`, which the approval
service emits inside each transition's transaction (PAM folder alerts use it).
+**New events follow the Event Standards and Versioning ADR (CloudEvents 1.0).** Define them with
+`defineEvent` (`event-outbox/event-definition.ts`), which names the event
+`com.infisical.<domain>.<entity>.<action>.v<version>` and throws at module load on a name that breaks the
+convention. A consumer uses the definition's `data` schema as its `payloadSchema`. `toCloudEvent` builds
+the CloudEvents envelope from a delivered `TEvent` for anything that leaves the backend. The events
+emitted before the ADR (eg `identity.authentication.auth-method-changed`) keep their names.
+
+**Secret, folder and import changes emit `com.infisical.secrets.{secret,folder,secret-import}.modified.v1`**
+(definitions in `secret-v2-bridge/secret-events.ts`, `secret-folder/secret-folder-events.ts`,
+`secret-import/secret-import-events.ts`). Secrets emit from inside `fnSecretBulkInsert`,
+`fnSecretBulkUpdate` and `fnSecretBulkDelete`, whose required `eventEmitter` and `eventScope` parameters
+make the type checker list every caller, so a new caller cannot skip the emit. A write to `secrets_v2`
+that bypasses those functions (eg a raw DAL `delete`) must emit with `emitSecretsModified` itself, in the
+same transaction. Only shared secrets emit. The e2e suite registers a recording consumer for these events
+(`e2e-test/fakes/event-outbox-registry.ts`), so `e2e-test/routes/v4/secret-events.spec.ts` can read what a
+write emitted.
+
**The event contract belongs to the domain that emits it, not to a consumer.** The identity auth method
event (key, resource type, change enum, payload schema, and the `emitIdentityAuthMethodChanged` helper)
lives in `src/services/identity/identity-auth-method-events.ts`. The 13 auth method services import the
diff --git a/backend/e2e-test/event-outbox.spec.ts b/backend/e2e-test/event-outbox.spec.ts
index 5da67441fb..20bcaaf556 100644
--- a/backend/e2e-test/event-outbox.spec.ts
+++ b/backend/e2e-test/event-outbox.spec.ts
@@ -21,6 +21,7 @@ const KEY = { consumer: CONSUMER };
const makeRow = (overrides?: Partial<TOutboxInsertRow>): TOutboxInsertRow => ({
consumer: CONSUMER,
+ eventId: randomUUID(),
eventType: "e2e.resource.happened",
payload: { orgId: ORG_ID, resourceType: "e2e.resource", resourceId: "res-1", targetIds: ["t-1"] },
occurredAt: new Date(),
diff --git a/backend/e2e-test/fakes/event-outbox-registry.ts b/backend/e2e-test/fakes/event-outbox-registry.ts
new file mode 100644
index 0000000000..b8cbdb4082
--- /dev/null
+++ b/backend/e2e-test/fakes/event-outbox-registry.ts
@@ -0,0 +1,34 @@
+import { z } from "zod";
+
+import { FolderModifiedEventV1 } from "@app/services/secret-folder/secret-folder-events";
+import { SecretImportModifiedEventV1 } from "@app/services/secret-import/secret-import-events";
+import { SecretModifiedEventV1 } from "@app/services/secret-v2-bridge/secret-events";
+
+// Reached through a path the alias does not match, so these resolve to the real modules.
+import * as RealEventOutboxRegistry from "../../src/services/event-outbox/event-outbox-registry";
+import { EventResultStatus, IEventConsumer } from "../../src/services/event-outbox/event-outbox-types";
+
+// Nothing in src/ subscribes to the secret events yet, and emit writes no row for an event nobody
+// subscribes to, so without a consumer a spec could not see what a write emitted. This one records
+// every secret event and accepts it, so the row stays in event_outbox for a spec to read. Wired up by
+// test.alias in vitest.e2e.config.mts; nothing under src/ references this file.
+export const SECRET_EVENTS_RECORDER = "e2e-secret-events";
+
+const RECORDED_EVENTS = [SecretModifiedEventV1, FolderModifiedEventV1, SecretImportModifiedEventV1] as const;
+
+const secretEventsRecorder: IEventConsumer = {
+ name: SECRET_EVENTS_RECORDER,
+ // The definitions' own schemas, so an emit site that builds a bad payload is dropped here exactly as
+ // it would be for a real consumer, and the spec waiting for its row fails.
+ payloadSchema: z.union([RECORDED_EVENTS[0].data, RECORDED_EVENTS[1].data, RECORDED_EVENTS[2].data]),
+ subscribesTo: (eventType) => RECORDED_EVENTS.some((definition) => definition.type === eventType),
+ handle: async (events) => events.map((event) => ({ id: String(event.id), status: EventResultStatus.Delivered }))
+};
+
+export const eventOutboxRegistryFactory: typeof RealEventOutboxRegistry.eventOutboxRegistryFactory = () => {
+ const registry = RealEventOutboxRegistry.eventOutboxRegistryFactory();
+ registry.register(secretEventsRecorder);
+ return registry;
+};
+
+export type { TEventOutboxRegistry } from "../../src/services/event-outbox/event-outbox-registry";
diff --git a/backend/e2e-test/routes/v4/secret-events.spec.ts b/backend/e2e-test/routes/v4/secret-events.spec.ts
new file mode 100644
index 0000000000..65cc30443d
--- /dev/null
+++ b/backend/e2e-test/routes/v4/secret-events.spec.ts
@@ -0,0 +1,284 @@
+import { SecretType, TableName } from "@app/db/schemas";
+import { FolderModifiedEventV1 } from "@app/services/secret-folder/secret-folder-events";
+import { SecretImportModifiedEventV1 } from "@app/services/secret-import/secret-import-events";
+import { SecretModifiedEventV1 } from "@app/services/secret-v2-bridge/secret-events";
+
+import { SECRET_EVENTS_RECORDER } from "../../fakes/event-outbox-registry";
+import { createIsolatedOrgAndProject } from "../../testUtils/fixtures";
+import { createFolder, deleteFolder } from "../../testUtils/folders";
+import { createSecretImport, deleteSecretImport } from "../../testUtils/secret-imports";
+import { createSecretV2, deleteSecretV2, updateSecretV2 } from "../../testUtils/secrets";
+
+type TRecordedEvent = {
+ eventType: string;
+ payload: { projectId: string; environment: string; secretPath: string } & Record<string, string>;
+};
+
+// No API shows outbox rows, so the events a write emitted are read from the table. The recorder
+// consumer (e2e-test/fakes/event-outbox-registry.ts) is what makes emit write them at all.
+const recordedEvents = async (projectId: string, eventType?: string): Promise<TRecordedEvent[]> => {
+ const query = testDb(TableName.EventOutbox)
+ .where({ consumer: SECRET_EVENTS_RECORDER })
+ .whereRaw(`payload->>'projectId' = ?`, [projectId])
+ .orderBy("id");
+ if (eventType) void query.where({ eventType });
+ const rows = await query.select("eventType", "payload");
+ return rows as TRecordedEvent[];
+};
+
+const locationsOf = (events: TRecordedEvent[]) =>
+ events.map(({ payload }) => `${payload.environment}:${payload.secretPath}`);
+
+describe("Secret, folder and import events", () => {
+ let projectId: string;
+ let authToken: string;
+ let cleanup: () => Promise<void>;
+
+ beforeEach(async () => {
+ ({ projectId, authToken, cleanup } = await createIsolatedOrgAndProject("secret-events"));
+ });
+
+ afterEach(async () => {
+ await cleanup();
+ });
+
+ const createInFolder = (secretPath: string, key: string, value = "value") =>
+ createSecretV2({ workspaceId: projectId, environmentSlug: "dev", secretPath, key, value, authToken });
+
+ test("a single create, update and delete each emit secret.modified for the secret", async () => {
+ const secret = await createInFolder("/", "SINGLE");
+ await updateSecretV2({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/",
+ key: "SINGLE",
+ value: "changed",
+ authToken
+ });
+ await deleteSecretV2({ workspaceId: projectId, environmentSlug: "dev", secretPath: "/", key: "SINGLE", authToken });
+
+ const events = await recordedEvents(projectId, SecretModifiedEventV1.type);
+ expect(events).toHaveLength(3);
+ events.forEach(({ payload }) => {
+ expect(payload).toEqual({ secretId: secret.id, projectId, environment: "dev", secretPath: "/" });
+ });
+ });
+
+ test("a personal override emits nothing", async () => {
+ await createInFolder("/", "SHARED");
+ const before = await recordedEvents(projectId);
+
+ await createSecretV2({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/",
+ key: "SHARED",
+ value: "mine",
+ type: SecretType.Personal,
+ authToken
+ });
+
+ expect(await recordedEvents(projectId)).toHaveLength(before.length);
+ });
+
+ test("a bulk create emits one event per secret", async () => {
+ const res = await testServer.inject({
+ method: "POST",
+ url: "/api/v4/secrets/batch",
+ headers: { authorization: `Bearer ${authToken}` },
+ body: {
+ projectId,
+ environment: "dev",
+ secretPath: "/",
+ secrets: ["BULK_A", "BULK_B", "BULK_C"].map((secretKey) => ({ secretKey, secretValue: "value" }))
+ }
+ });
+ expect(res.statusCode).toBe(200);
+
+ const events = await recordedEvents(projectId, SecretModifiedEventV1.type);
+ expect(events).toHaveLength(3);
+ expect(new Set(events.map(({ payload }) => payload.secretId)).size).toBe(3);
+ });
+
+ test("a move emits at the source and at the destination", async () => {
+ await createFolder({ workspaceId: projectId, environmentSlug: "dev", secretPath: "/", name: "target", authToken });
+ const secret = await createInFolder("/", "MOVED");
+ const createdCount = (await recordedEvents(projectId, SecretModifiedEventV1.type)).length;
+
+ const res = await testServer.inject({
+ method: "POST",
+ url: "/api/v4/secrets/move",
+ headers: { authorization: `Bearer ${authToken}` },
+ body: {
+ projectId,
+ sourceEnvironment: "dev",
+ sourceSecretPath: "/",
+ destinationEnvironment: "dev",
+ destinationSecretPath: "/target",
+ secretIds: [secret.id]
+ }
+ });
+ expect(res.statusCode).toBe(200);
+
+ const moveEvents = (await recordedEvents(projectId, SecretModifiedEventV1.type)).slice(createdCount);
+ expect(locationsOf(moveEvents).sort()).toEqual(["dev:/", "dev:/target"]);
+ });
+
+ test("renaming a referenced secret emits for the secret in another folder that references it", async () => {
+ await createFolder({ workspaceId: projectId, environmentSlug: "dev", secretPath: "/", name: "app", authToken });
+ await createInFolder("/", "DB_HOST", "db.internal");
+ const referencing = await createInFolder("/app", "DB_URL", `postgres://\${dev.DB_HOST}`);
+ const before = (await recordedEvents(projectId, SecretModifiedEventV1.type)).length;
+
+ await updateSecretV2({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/",
+ key: "DB_HOST",
+ newKey: "DATABASE_HOST",
+ authToken
+ });
+
+ const renameEvents = (await recordedEvents(projectId, SecretModifiedEventV1.type)).slice(before);
+ expect(renameEvents.map(({ payload }) => payload)).toContainEqual({
+ secretId: referencing.id,
+ projectId,
+ environment: "dev",
+ secretPath: "/app"
+ });
+ });
+
+ test("a point-in-time rollback emits for the secrets it removes", async () => {
+ await createInFolder("/", "KEPT");
+ const commitsRes = await testServer.inject({
+ method: "GET",
+ url: `/api/v1/pit/commits?projectId=${projectId}&environment=dev&path=/&sort=desc`,
+ headers: { authorization: `Bearer ${authToken}` }
+ });
+ expect(commitsRes.statusCode).toBe(200);
+ const [targetCommit] = commitsRes.json<{ commits: { id: string; folderId: string }[] }>().commits;
+
+ const rolledBack = await createInFolder("/", "ROLLED_BACK");
+ const before = (await recordedEvents(projectId, SecretModifiedEventV1.type)).length;
+
+ const rollbackRes = await testServer.inject({
+ method: "POST",
+ url: `/api/v1/pit/commits/${targetCommit.id}/rollback`,
+ headers: { authorization: `Bearer ${authToken}` },
+ body: { projectId, environment: "dev", folderId: targetCommit.folderId }
+ });
+ expect(rollbackRes.statusCode).toBe(200);
+
+ const rollbackEvents = (await recordedEvents(projectId, SecretModifiedEventV1.type)).slice(before);
+ expect(rollbackEvents.map(({ payload }) => payload)).toContainEqual({
+ secretId: rolledBack.id,
+ projectId,
+ environment: "dev",
+ secretPath: "/"
+ });
+ });
+
+ test("folder create, rename and delete emit folder.modified at the paths they touch", async () => {
+ // The intermediate "/parent" is created by the same request, and emits too.
+ const folder = await createFolder({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/parent",
+ name: "child",
+ authToken
+ });
+ expect(locationsOf(await recordedEvents(projectId, FolderModifiedEventV1.type))).toEqual([
+ "dev:/parent",
+ "dev:/parent/child"
+ ]);
+
+ const renameRes = await testServer.inject({
+ method: "PATCH",
+ url: `/api/v2/folders/${folder.id}`,
+ headers: { authorization: `Bearer ${authToken}` },
+ body: { projectId, environment: "dev", path: "/parent", name: "renamed" }
+ });
+ expect(renameRes.statusCode).toBe(200);
+ expect(locationsOf(await recordedEvents(projectId, FolderModifiedEventV1.type)).slice(2)).toEqual([
+ "dev:/parent/child",
+ "dev:/parent/renamed"
+ ]);
+
+ await deleteFolder({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/parent",
+ id: folder.id,
+ authToken
+ });
+ const events = await recordedEvents(projectId, FolderModifiedEventV1.type);
+ expect(events[events.length - 1].payload).toEqual({
+ folderId: folder.id,
+ projectId,
+ environment: "dev",
+ secretPath: "/parent/renamed"
+ });
+ });
+
+ test("a folder move emits for the new folders and the deleted source", async () => {
+ await createFolder({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/",
+ name: "destination",
+ authToken
+ });
+ const folder = await createFolder({
+ workspaceId: projectId,
+ environmentSlug: "dev",
+ secretPath: "/",
+ name: "source",
+ authToken
+ });
+ const before = (await recordedEvents(projectId, FolderModifiedEventV1.type)).length;
+
+ const res = await testServer.inject({
+ method: "POST",
+ url: "/api/v2/folders/move",
+ headers: { authorization: `Bearer ${authToken}` },
+ body: { projectId, folderId: folder.id, destinationEnvironment: "dev", destinationPath: "/destination" }
+ });
+ expect(res.statusCode).toBe(200);
+
+ const moveEvents = (await recordedEvents(projectId, FolderModifiedEventV1.type)).slice(before);
+ expect(locationsOf(moveEvents).sort()).toEqual(["dev:/destination/source", "dev:/source"]);
+ });
+
+ test("import create, update and delete emit secret-import.modified for the folder holding the import", async () => {
+ await createFolder({ workspaceId: projectId, environmentSlug: "dev", secretPath: "/", name: "shared", authToken });
+ const secretImport = await createSecretImport({
+ workspaceId: projectId,
+ environmentSlug: "staging",
+ secretPath: "/",
+ importEnv: "dev",
+ importPath: "/shared",
+ authToken
+ });
+
+ const updateRes = await testServer.inject({
+ method: "PATCH",
+ url: `/api/v2/secret-imports/${secretImport.id}`,
+ headers: { authorization: `Bearer ${authToken}` },
+ body: { projectId, environment: "staging", path: "/", import: { path: "/" } }
+ });
+ expect(updateRes.statusCode).toBe(200);
+
+ await deleteSecretImport({
+ workspaceId: projectId,
+ environmentSlug: "staging",
+ secretPath: "/",
+ id: secretImport.id,
+ authToken
+ });
+
+ const events = await recordedEvents(projectId, SecretImportModifiedEventV1.type);
+ expect(events.map(({ payload }) => payload)).toEqual(
+ Array(3).fill({ secretImportId: secretImport.id, projectId, environment: "staging", secretPath: "/" })
+ );
+ });
+});
diff --git a/backend/src/db/migrations/20261019120000_event-outbox-event-id.ts b/backend/src/db/migrations/20261019120000_event-outbox-event-id.ts
new file mode 100644
index 0000000000..11f05eb91e
--- /dev/null
+++ b/backend/src/db/migrations/20261019120000_event-outbox-event-id.ts
@@ -0,0 +1,23 @@
+import { Knex } from "knex";
+
+import { TableName } from "../schemas";
+
+// One id per event, shared by every consumer's row, so the same occurrence can be recognised across
+// consumers and handed out as the CloudEvents `id`. The row `id` is per consumer and can't serve.
+// The default only fills rows written before this column existed and rows from a pod that doesn't set
+// it yet; the outbox is pruned within a day, so the rewrite the volatile default causes is small.
+export async function up(knex: Knex): Promise<void> {
+ if (await knex.schema.hasColumn(TableName.EventOutbox, "eventId")) return;
+
+ await knex.schema.alterTable(TableName.EventOutbox, (t) => {
+ t.uuid("eventId").notNullable().defaultTo(knex.fn.uuid());
+ });
+}
+
+export async function down(knex: Knex): Promise<void> {
+ if (!(await knex.schema.hasColumn(TableName.EventOutbox, "eventId"))) return;
+
+ await knex.schema.alterTable(TableName.EventOutbox, (t) => {
+ t.dropColumn("eventId");
+ });
+}
diff --git a/backend/src/db/schemas/event-outbox.ts b/backend/src/db/schemas/event-outbox.ts
index 8fac8bfc82..c4b718f551 100644
--- a/backend/src/db/schemas/event-outbox.ts
+++ b/backend/src/db/schemas/event-outbox.ts
@@ -21,7 +21,8 @@ export const EventOutboxSchema = z.object({
lockToken: z.string().uuid().nullable().optional(),
lastError: z.string().nullable().optional(),
createdAt: z.date(),
- updatedAt: z.date()
+ updatedAt: z.date(),
+ eventId: z.string().uuid()
});
export type TEventOutbox = z.infer<typeof EventOutboxSchema>;
diff --git a/backend/src/ee/services/audit-log-stream-outbox/audit-log-stream-outbox-service.test.ts b/backend/src/ee/services/audit-log-stream-outbox/audit-log-stream-outbox-service.test.ts
index 777f58aea1..90439eb2f2 100644
--- a/backend/src/ee/services/audit-log-stream-outbox/audit-log-stream-outbox-service.test.ts
+++ b/backend/src/ee/services/audit-log-stream-outbox/audit-log-stream-outbox-service.test.ts
@@ -119,7 +119,10 @@ const createService = () => {
)
};
- const eventEmitter = { emit: vi.fn<(event: unknown, tx?: unknown) => Promise<void>>(async () => undefined) };
+ const eventEmitter = {
+ emit: vi.fn<(event: unknown, tx?: unknown) => Promise<void>>(async () => undefined),
+ emitMany: vi.fn(async () => undefined)
+ };
const projectDAL = {
findProjectTypesByIds: vi.fn(async () => [])
@@ -405,7 +408,7 @@ describe("audit-log-stream-outbox-service enqueueForLogs batch fanout", () => {
kmsService: {} as never,
keyStore: keyStore as never,
queueService: queueService as never,
- eventEmitter: { emit: vi.fn(async () => undefined) }
+ eventEmitter: { emit: vi.fn(async () => undefined), emitMany: vi.fn(async () => undefined) }
});
return { service, auditLogStreamOutboxDAL, auditLogStreamDAL, projectDAL, keyStore, queueService };
diff --git a/backend/src/ee/services/honey-token/honey-token-service.ts b/backend/src/ee/services/honey-token/honey-token-service.ts
index bb32d7d144..d2a8dd4bbe 100644
--- a/backend/src/ee/services/honey-token/honey-token-service.ts
+++ b/backend/src/ee/services/honey-token/honey-token-service.ts
@@ -15,6 +15,7 @@ import { OrderByDirection, OrgServiceActor } from "@app/lib/types";
import { TAppConnectionDALFactory } from "@app/services/app-connection/app-connection-dal";
import { TAppConnectionServiceFactory } from "@app/services/app-connection/app-connection-service";
import { ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "@app/services/folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { KmsDataKey } from "@app/services/kms/kms-types";
@@ -107,6 +108,7 @@ export type THoneyTokenServiceFactoryDep = {
secretVersionTagDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
secretTagDAL: TSecretTagDALFactory;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany">;
secretQueueService: Pick<TSecretQueueFactory, "syncSecrets" | "removeSecretReminder">;
webhookDAL: Pick<TWebhookDALFactory, "findAllWebhooks" | "transaction" | "update" | "bulkUpdate">;
@@ -152,6 +154,7 @@ export const honeyTokenServiceFactory = ({
secretVersionTagDAL,
secretTagDAL,
folderCommitService,
+ eventEmitter,
resourceMetadataDAL,
secretQueueService,
webhookDAL,
@@ -178,6 +181,7 @@ export const honeyTokenServiceFactory = ({
secretVersionTagDAL,
secretTagDAL,
folderCommitService,
+ eventEmitter,
resourceMetadataDAL,
secretQueueService,
webhookDAL,
@@ -387,6 +391,8 @@ export const honeyTokenServiceFactory = ({
const createdSecrets = await fnSecretBulkInsert({
folderId: folder.id,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
orgId: actor.orgId,
inputSecrets: inputSecretsWithBlindIndex,
secretDAL,
@@ -438,6 +444,10 @@ export const honeyTokenServiceFactory = ({
const { projectId } = honeyToken;
const [folderWithEnv] = await folderDAL.findSecretPathByFolderIds(projectId, [honeyToken.folderId]);
+ if (!folderWithEnv) {
+ throw new NotFoundError({ message: `Folder of honey token with ID "${honeyTokenId}" not found` });
+ }
+ const eventScope = { projectId, environment: folderWithEnv.environmentSlug, secretPath: folderWithEnv.path };
const { permission: updatePermission } = await permissionService.getProjectPermission({
actor: actor.type,
actorId: actor.id,
@@ -547,6 +557,8 @@ export const honeyTokenServiceFactory = ({
return honeyTokenDAL.transaction(async (tx) => {
await fnSecretBulkDelete({
folderId: honeyToken.folderId,
+ eventEmitter,
+ eventScope,
projectId,
inputSecrets: oldSecretKeys.map((key) => ({ type: SecretType.Shared, secretKey: key })),
actorId: actor.id,
@@ -571,6 +583,8 @@ export const honeyTokenServiceFactory = ({
const createdSecrets = await fnSecretBulkInsert({
folderId: honeyToken.folderId,
+ eventEmitter,
+ eventScope,
orgId: actor.orgId,
inputSecrets: inputSecretsWithBlindIndex,
secretDAL,
@@ -629,6 +643,10 @@ export const honeyTokenServiceFactory = ({
const { projectId } = honeyToken;
const [folderWithEnv] = await folderDAL.findSecretPathByFolderIds(projectId, [honeyToken.folderId]);
+ if (!folderWithEnv) {
+ throw new NotFoundError({ message: `Folder of honey token with ID "${honeyTokenId}" not found` });
+ }
+ const eventScope = { projectId, environment: folderWithEnv.environmentSlug, secretPath: folderWithEnv.path };
const { permission: revokePermission } = await permissionService.getProjectPermission({
actor: actor.type,
actorId: actor.id,
@@ -706,6 +724,8 @@ export const honeyTokenServiceFactory = ({
await fnSecretBulkDelete({
folderId: honeyToken.folderId,
+ eventEmitter,
+ eventScope,
projectId,
inputSecrets: secretKeys.map((key) => ({ type: SecretType.Shared, secretKey: key })),
actorId: actor.id,
diff --git a/backend/src/ee/services/secret-approval-request/secret-approval-request-merge-fns.ts b/backend/src/ee/services/secret-approval-request/secret-approval-request-merge-fns.ts
index 663bd71301..39e7d8f914 100644
--- a/backend/src/ee/services/secret-approval-request/secret-approval-request-merge-fns.ts
+++ b/backend/src/ee/services/secret-approval-request/secret-approval-request-merge-fns.ts
@@ -8,6 +8,7 @@ import { getConfig } from "@app/lib/config/env";
import { NotFoundError } from "@app/lib/errors";
import { groupBy } from "@app/lib/fn";
import { ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "@app/services/folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TNotificationServiceFactory } from "@app/services/notification/notification-service";
@@ -60,6 +61,7 @@ type TSecretApprovalRequestMergeFnsFactoryDep = {
secretTagDAL: Pick<TSecretTagDALFactory, "saveTagsToSecretV2" | "deleteTagsToSecretV2" | "find">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
secretQueueService: Pick<TSecretQueueFactory, "syncSecrets" | "removeSecretReminder">;
secretApprovalRequestSecretDAL: Pick<TSecretApprovalRequestSecretDALFactory, "updateV2ById">;
validateSecrets: TSecretApprovalRequestCommitFnsFactory["validateSecrets"];
@@ -253,6 +255,7 @@ export const secretApprovalRequestMergeFnsFactory = ({
secretTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretQueueService,
secretApprovalRequestSecretDAL,
validateSecrets,
@@ -375,6 +378,8 @@ export const secretApprovalRequestMergeFnsFactory = ({
? await fnSecretBulkInsert({
tx,
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath },
actor: { actorId, type: actor },
orgId: actorOrgId,
inputSecrets: creates.map((el, idx) => ({
@@ -437,6 +442,7 @@ export const secretApprovalRequestMergeFnsFactory = ({
secretVersionDAL: secretVersionV2BridgeDAL,
folderCommitService,
secretQueueService,
+ eventEmitter,
folderDAL,
encryptor: ({ plainText }) => secretManagerEncryptor({ plainText }),
decryptor: ({ cipherTextBlob }) => secretManagerDecryptor({ cipherTextBlob }),
@@ -473,6 +479,8 @@ export const secretApprovalRequestMergeFnsFactory = ({
const updated = updates.length
? await fnSecretBulkUpdate({
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath },
orgId: actorOrgId,
actor: { actorId, type: actor },
tx,
@@ -531,6 +539,7 @@ export const secretApprovalRequestMergeFnsFactory = ({
secretVersionDAL: secretVersionV2BridgeDAL,
folderCommitService,
secretQueueService,
+ eventEmitter,
folderDAL,
encryptor: ({ plainText }) => secretManagerEncryptor({ plainText }),
decryptor: ({ cipherTextBlob }) => secretManagerDecryptor({ cipherTextBlob }),
@@ -543,6 +552,8 @@ export const secretApprovalRequestMergeFnsFactory = ({
? await fnSecretBulkDelete({
projectId,
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath },
tx,
actorId,
actorType: actor,
diff --git a/backend/src/ee/services/secret-approval-request/secret-approval-request-service.ts b/backend/src/ee/services/secret-approval-request/secret-approval-request-service.ts
index 62e37d6103..d5403820ac 100644
--- a/backend/src/ee/services/secret-approval-request/secret-approval-request-service.ts
+++ b/backend/src/ee/services/secret-approval-request/secret-approval-request-service.ts
@@ -24,6 +24,7 @@ import { triggerWorkflowIntegrationNotification } from "@app/lib/workflow-integr
import { TriggerFeature } from "@app/lib/workflow-integrations/types";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "@app/services/folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { KmsDataKey } from "@app/services/kms/kms-types";
@@ -154,6 +155,7 @@ type TSecretApprovalRequestServiceFactoryDep = {
projectMicrosoftTeamsConfigDAL: Pick<TProjectMicrosoftTeamsConfigDALFactory, "getIntegrationDetailsByProject">;
microsoftTeamsService: Pick<TMicrosoftTeamsServiceFactory, "sendNotification">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
notificationService: Pick<TNotificationServiceFactory, "createUserNotifications">;
telemetryService: Pick<TTelemetryServiceFactory, "sendPostHogEvents">;
queueService: Pick<TQueueServiceFactory, "queue">;
@@ -204,6 +206,7 @@ export const secretApprovalRequestServiceFactory = ({
projectMicrosoftTeamsConfigDAL,
microsoftTeamsService,
folderCommitService,
+ eventEmitter,
notificationService,
telemetryService,
queueService,
@@ -231,6 +234,7 @@ export const secretApprovalRequestServiceFactory = ({
secretTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretQueueService,
secretApprovalRequestSecretDAL,
validateSecrets,
diff --git a/backend/src/ee/services/secret-change-global-request-bridge/secret-change-global-request-bridge-service.ts b/backend/src/ee/services/secret-change-global-request-bridge/secret-change-global-request-bridge-service.ts
index b0baa2e428..c114419a6a 100644
--- a/backend/src/ee/services/secret-change-global-request-bridge/secret-change-global-request-bridge-service.ts
+++ b/backend/src/ee/services/secret-change-global-request-bridge/secret-change-global-request-bridge-service.ts
@@ -26,6 +26,7 @@ import {
upsertApprovalRequestStepDecision
} from "@app/services/approval-policy/approval-request-fns";
import { ActorAuthMethod, ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "@app/services/folder-commit/folder-commit-service";
import { TIdentityDALFactory } from "@app/services/identity/identity-dal";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
@@ -156,6 +157,7 @@ type TSecretChangeGlobalRequestBridgeServiceFactoryDep = {
secretTagDAL: Pick<TSecretTagDALFactory, "findManyTagsById" | "saveTagsToSecretV2" | "deleteTagsToSecretV2" | "find">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
secretQueueService: Pick<TSecretQueueFactory, "syncSecrets" | "removeSecretReminder">;
secretValidationRuleService: Pick<TSecretValidationRuleServiceFactory, "validateSecrets">;
userDAL: Pick<TUserDALFactory, "findById" | "find" | "findOne">;
@@ -200,6 +202,7 @@ export const secretChangeGlobalRequestBridgeServiceFactory = ({
secretTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretQueueService,
secretValidationRuleService,
userDAL,
@@ -237,6 +240,7 @@ export const secretChangeGlobalRequestBridgeServiceFactory = ({
secretTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretQueueService,
secretApprovalRequestSecretDAL,
validateSecrets,
diff --git a/backend/src/ee/services/secret-replication/secret-replication-service.ts b/backend/src/ee/services/secret-replication/secret-replication-service.ts
index e756d7dc4d..2a883b5186 100644
--- a/backend/src/ee/services/secret-replication/secret-replication-service.ts
+++ b/backend/src/ee/services/secret-replication/secret-replication-service.ts
@@ -1,3 +1,5 @@
+import path from "node:path";
+
import { SecretType, TSecrets, TSecretsV2 } from "@app/db/schemas";
import { shouldApplyPolicy } from "@app/ee/services/secret-approval-policy/secret-approval-policy-fns";
import { TSecretApprovalPolicyServiceFactory } from "@app/ee/services/secret-approval-policy/secret-approval-policy-service";
@@ -8,6 +10,7 @@ import { NotFoundError } from "@app/lib/errors";
import { groupBy, unique } from "@app/lib/fn";
import { logger } from "@app/lib/logger";
import { QueueName, TQueueServiceFactory } from "@app/queue";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "@app/services/folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { KmsDataKey } from "@app/services/kms/kms-types";
@@ -23,11 +26,13 @@ import { SecretOperations } from "@app/services/secret/secret-types";
import { TSecretVersionDALFactory } from "@app/services/secret/secret-version-dal";
import { TSecretVersionTagDALFactory } from "@app/services/secret/secret-version-tag-dal";
import { TSecretFolderDALFactory } from "@app/services/secret-folder/secret-folder-dal";
+import { FolderModifiedEventV1 } from "@app/services/secret-folder/secret-folder-events";
import { ReservedFolders } from "@app/services/secret-folder/secret-folder-types";
import { TSecretImportDALFactory } from "@app/services/secret-import/secret-import-dal";
import { fnSecretsFromImports, fnSecretsV2FromImports } from "@app/services/secret-import/secret-import-fns";
import { TSecretTagDALFactory } from "@app/services/secret-tag/secret-tag-dal";
import { createSecretBlindIndexer } from "@app/services/secret-v2-bridge/secret-blind-index-fns";
+import { emitSecretsModified } from "@app/services/secret-v2-bridge/secret-events";
import { getAllSecretReferences } from "@app/services/secret-v2-bridge/secret-reference-fns";
import { TSecretV2BridgeDALFactory } from "@app/services/secret-v2-bridge/secret-v2-bridge-dal";
import {
@@ -90,6 +95,7 @@ type TSecretReplicationServiceFactoryDep = {
projectBotService: Pick<TProjectBotServiceFactory, "getBotKey">;
kmsService: Pick<TKmsServiceFactory, "createCipherPairWithDataKey">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
projectFolderGrantDAL: Pick<TProjectFolderGrantDALFactory, "find">;
orgDAL: Pick<TOrgDALFactory, "findOrgById">;
};
@@ -138,6 +144,7 @@ export const secretReplicationServiceFactory = ({
secretV2BridgeDAL,
kmsService,
folderCommitService,
+ eventEmitter,
projectFolderGrantDAL,
orgDAL,
resourceMetadataDAL
@@ -362,20 +369,38 @@ export const secretReplicationServiceFactory = ({
message: `Imported folder with ID '${destinationSecretImport.folderId}' not found in project with ID ${projectId}`
});
+ const destinationReplicationFolderName = getReplicationFolderName(destinationSecretImport.id);
+ const destinationReplicationFolderPath = path.posix.join(
+ destinationFolder.path,
+ destinationReplicationFolderName
+ );
let destinationReplicationFolder = await folderDAL.findOne({
parentId: destinationFolder.id,
- name: getReplicationFolderName(destinationSecretImport.id),
+ name: destinationReplicationFolderName,
isReserved: true
});
if (!destinationReplicationFolder) {
destinationReplicationFolder = await folderDAL.create({
parentId: destinationFolder.id,
- name: getReplicationFolderName(destinationSecretImport.id),
+ name: destinationReplicationFolderName,
envId: destinationFolder.envId,
isReserved: true
});
+ await eventEmitter.emit(
+ FolderModifiedEventV1.build({
+ folderId: destinationReplicationFolder.id,
+ projectId,
+ environment: destinationFolder.environmentSlug,
+ secretPath: destinationReplicationFolderPath
+ })
+ );
}
const destinationReplicationFolderId = destinationReplicationFolder.id;
+ const replicationEventScope = {
+ projectId,
+ environment: destinationFolder.environmentSlug,
+ secretPath: destinationReplicationFolderPath
+ };
const destinationLocalSecretsFromDB = await secretV2BridgeDAL.find({
folderId: destinationReplicationFolderId
@@ -491,6 +516,8 @@ export const secretReplicationServiceFactory = ({
if (locallyCreatedSecrets.length) {
await fnSecretV2BridgeBulkInsert({
folderId: destinationReplicationFolderId,
+ eventEmitter,
+ eventScope: replicationEventScope,
orgId,
secretVersionDAL: secretVersionV2BridgeDAL,
secretDAL: secretV2BridgeDAL,
@@ -523,6 +550,8 @@ export const secretReplicationServiceFactory = ({
await fnSecretV2BridgeBulkUpdate({
orgId,
folderId: destinationReplicationFolderId,
+ eventEmitter,
+ eventScope: replicationEventScope,
secretVersionDAL: secretVersionV2BridgeDAL,
folderCommitService,
secretDAL: secretV2BridgeDAL,
@@ -566,6 +595,7 @@ export const secretReplicationServiceFactory = ({
},
tx
);
+ await emitSecretsModified(eventEmitter, replicationEventScope, locallyDeletedSecrets, tx);
}
await secretV2BridgeDAL.invalidateSecretCacheByProjectId(projectId, tx);
});
diff --git a/backend/src/ee/services/secret-rotation-v2/secret-rotation-v2-service.ts b/backend/src/ee/services/secret-rotation-v2/secret-rotation-v2-service.ts
index 7487fc05ee..d75d8aca7c 100644
--- a/backend/src/ee/services/secret-rotation-v2/secret-rotation-v2-service.ts
+++ b/backend/src/ee/services/secret-rotation-v2/secret-rotation-v2-service.ts
@@ -78,6 +78,7 @@ import { AppConnection } from "@app/services/app-connection/app-connection-enums
import { decryptAppConnection } from "@app/services/app-connection/app-connection-fns";
import { TAppConnectionServiceFactory } from "@app/services/app-connection/app-connection-service";
import { ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "@app/services/folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { KmsDataKey } from "@app/services/kms/kms-types";
@@ -88,6 +89,7 @@ import { SecretsOrderBy } from "@app/services/secret/secret-types";
import { TSecretFolderDALFactory } from "@app/services/secret-folder/secret-folder-dal";
import { TSecretTagDALFactory } from "@app/services/secret-tag/secret-tag-dal";
import { createSecretBlindIndexer } from "@app/services/secret-v2-bridge/secret-blind-index-fns";
+import { emitSecretsModified, secretModifiedEvents } from "@app/services/secret-v2-bridge/secret-events";
import { TSecretV2BridgeDALFactory } from "@app/services/secret-v2-bridge/secret-v2-bridge-dal";
import {
fnSecretBulkDelete,
@@ -174,6 +176,7 @@ export type TSecretRotationV2ServiceFactoryDep = {
queueService: Pick<TQueueServiceFactory, "queue">;
appConnectionDAL: Pick<TAppConnectionDALFactory, "findById" | "update" | "updateById" | "transaction">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
gatewayV2Service: Pick<TGatewayV2ServiceFactory, "getPlatformConnectionDetailsByGatewayId">;
gatewayPoolService: Pick<TGatewayPoolServiceFactory, "resolveEffectiveGatewayId">;
telemetryService: Pick<TTelemetryServiceFactory, "sendPostHogEvents">;
@@ -243,6 +246,7 @@ export const secretRotationV2ServiceFactory = ({
keyStore,
queueService,
folderCommitService,
+ eventEmitter,
appConnectionDAL,
gatewayV2Service,
gatewayPoolService,
@@ -713,6 +717,8 @@ export const secretRotationV2ServiceFactory = ({
);
const mappedSecrets = await fnSecretBulkInsert({
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
folderId: folder.id,
orgId: connection.orgId,
tx,
@@ -851,6 +857,8 @@ export const secretRotationV2ServiceFactory = ({
// update mapped secrets names
await fnSecretBulkUpdate({
+ eventEmitter,
+ eventScope: { projectId, environment: environment.slug, secretPath: folder.path },
folderId,
orgId: connection.orgId,
tx,
@@ -973,6 +981,8 @@ export const secretRotationV2ServiceFactory = ({
secretRotationV2DAL.transaction(async (tx) => {
if (deleteSecrets) {
await fnSecretBulkDelete({
+ eventEmitter,
+ eventScope: { projectId, environment: environment.slug, secretPath: folder.path },
secretDAL: secretV2BridgeDAL,
secretQueueService,
inputSecrets: Object.values(secretsMapping as TSecretRotationV2["secretsMapping"]).map((secretKey) => ({
@@ -1158,12 +1168,18 @@ export const secretRotationV2ServiceFactory = ({
);
});
- await secretV2BridgeDAL.deleteMany(
+ const overwrittenSecrets = await secretV2BridgeDAL.deleteMany(
conflictingDestinationSecrets.map((s) => ({ key: s.key, type: SecretType.Shared })),
destinationFolder.id,
actor.id,
tx
);
+ await emitSecretsModified(
+ eventEmitter,
+ { projectId, environment: destinationEnvironment, secretPath: destinationFolder.path },
+ overwrittenSecrets,
+ tx
+ );
}
const sourceSecrets = await secretV2BridgeDAL.find(
@@ -1195,6 +1211,21 @@ export const secretRotationV2ServiceFactory = ({
tx
);
+ // Moved in place, so each secret keeps its id and is modified at both locations.
+ await eventEmitter.emitMany(
+ [
+ ...secretModifiedEvents(
+ { projectId, environment: environment.slug, secretPath: folder.path },
+ sourceSecrets
+ ),
+ ...secretModifiedEvents(
+ { projectId, environment: destinationEnvironment, secretPath: destinationFolder.path },
+ sourceSecrets
+ )
+ ],
+ tx
+ );
+
for await (const secret of sourceSecrets) {
await fnUpdateMovedSecretReferences({
orgId: actor.orgId,
@@ -1212,6 +1243,7 @@ export const secretRotationV2ServiceFactory = ({
folderCommitService,
folderDAL,
secretQueueService,
+ eventEmitter,
encryptor: ({ plainText }) => secretManagerEncryptor({ plainText }),
decryptor: ({ cipherTextBlob }) => secretManagerDecryptor({ cipherTextBlob }),
blindIndexer,
@@ -1399,6 +1431,8 @@ export const secretRotationV2ServiceFactory = ({
);
await fnSecretBulkUpdate({
+ eventEmitter,
+ eventScope: { projectId, environment: environment.slug, secretPath: folder.path },
folderId,
orgId: connection.orgId,
tx,
@@ -2113,6 +2147,8 @@ export const secretRotationV2ServiceFactory = ({
const passwordBuffer = Buffer.from(localAccountCredentials.password);
await fnSecretBulkUpdate({
+ eventEmitter,
+ eventScope: { projectId, environment: environment.slug, secretPath: folder.path },
folderId,
orgId: connection.orgId,
tx,
diff --git a/backend/src/lib/telemetry/metrics.ts b/backend/src/lib/telemetry/metrics.ts
index 1a1d1c2e91..7fd705a840 100644
--- a/backend/src/lib/telemetry/metrics.ts
+++ b/backend/src/lib/telemetry/metrics.ts
@@ -831,6 +831,32 @@ export const recordEventOutboxExhaustedMetric = (params: { consumer: string; cou
eventOutboxExhaustedCounter.add(params.count, { "event_outbox.consumer": params.consumer });
});
+export enum EventOutboxDropReason {
+ InvalidEvent = "invalid_event",
+ PayloadTooLarge = "payload_too_large",
+ PayloadRejected = "payload_rejected"
+}
+
+export const eventOutboxDroppedCounter = infisicalCoreMeter.createCounter("infisical.event_outbox.dropped.count", {
+ description:
+ "Events never written to the outbox because the emit site built a bad one, by event type and reason. The business write still committed, so every one is a change no consumer will hear about. Alarm on any occurrence: it is a bug at the emit site.",
+ unit: "{event}"
+});
+
+export const recordEventOutboxDroppedMetric = (params: {
+ eventType: string;
+ reason: EventOutboxDropReason;
+ consumer?: string;
+}) =>
+ safely(() => {
+ if (!isTelemetryEnabled()) return;
+ eventOutboxDroppedCounter.add(1, {
+ "event_outbox.event_type": params.eventType,
+ "event_outbox.drop_reason": params.reason,
+ ...(params.consumer ? { "event_outbox.consumer": params.consumer } : {})
+ });
+ });
+
export enum ProductAnalyticsDropReason {
Retention = "retention",
Unparseable = "unparseable"
diff --git a/backend/src/lib/telemetry/telemetry-attributes.ts b/backend/src/lib/telemetry/telemetry-attributes.ts
index d522c0e476..d97965ebec 100644
--- a/backend/src/lib/telemetry/telemetry-attributes.ts
+++ b/backend/src/lib/telemetry/telemetry-attributes.ts
@@ -25,6 +25,8 @@ export const INFISICAL_CORE_METER_ATTRIBUTES = [
"audit_log_stream.id",
"event_outbox.consumer",
"event_outbox.status",
+ "event_outbox.event_type",
+ "event_outbox.drop_reason",
"scim.operation",
"sso.provider",
"sso.action",
diff --git a/backend/src/server/routes/index.ts b/backend/src/server/routes/index.ts
index 6b4adffc49..578ffa3004 100644
--- a/backend/src/server/routes/index.ts
+++ b/backend/src/server/routes/index.ts
@@ -1418,6 +1418,7 @@ export const registerRoutes = async (
folderDAL
});
const folderCommitService = folderCommitServiceFactory({
+ eventEmitter: eventOutboxService,
folderCommitDAL,
folderCommitChangesDAL,
folderCheckpointDAL,
@@ -2270,6 +2271,7 @@ export const registerRoutes = async (
});
const secretSyncQueue = secretSyncQueueFactory({
+ eventEmitter: eventOutboxService,
queueService,
cronJob,
secretSyncDAL,
@@ -2308,6 +2310,7 @@ export const registerRoutes = async (
});
const secretQueueService = secretQueueFactory({
+ eventEmitter: eventOutboxService,
keyStore,
queueService,
secretDAL,
@@ -2436,6 +2439,7 @@ export const registerRoutes = async (
projectDAL
});
const secretImportService = secretImportServiceFactory({
+ eventEmitter: eventOutboxService,
licenseService,
projectBotService,
projectFolderGrantDAL,
@@ -2466,6 +2470,7 @@ export const registerRoutes = async (
});
const secretChangeGlobalRequestBridgeService = secretChangeGlobalRequestBridgeServiceFactory({
+ eventEmitter: eventOutboxService,
approvalRequestDAL,
membershipUserDAL,
approvalRequestStepsDAL,
@@ -2509,6 +2514,7 @@ export const registerRoutes = async (
});
const secretV2BridgeService = secretV2BridgeServiceFactory({
+ eventEmitter: eventOutboxService,
folderDAL,
projectDAL,
secretVersionDAL: secretVersionV2BridgeDAL,
@@ -2532,6 +2538,7 @@ export const registerRoutes = async (
});
const secretApprovalRequestService = secretApprovalRequestServiceFactory({
+ eventEmitter: eventOutboxService,
permissionService,
projectBotService,
folderDAL,
@@ -2596,6 +2603,7 @@ export const registerRoutes = async (
});
const folderService = secretFolderServiceFactory({
+ eventEmitter: eventOutboxService,
permissionService,
additionalPrivilegeDAL,
folderDAL,
@@ -2727,6 +2735,7 @@ export const registerRoutes = async (
});
const secretReplicationService = secretReplicationServiceFactory({
+ eventEmitter: eventOutboxService,
secretTagDAL,
secretVersionTagDAL,
secretDAL,
@@ -3261,6 +3270,7 @@ export const registerRoutes = async (
});
const externalMigrationQueue = externalMigrationQueueFactory({
+ eventEmitter: eventOutboxService,
projectEnvService,
projectDAL,
projectService,
@@ -3397,6 +3407,7 @@ export const registerRoutes = async (
});
const honeyTokenService = honeyTokenServiceFactory({
+ eventEmitter: eventOutboxService,
honeyTokenDAL,
honeyTokenConfigDAL,
honeyTokenEventDAL,
@@ -3487,6 +3498,7 @@ export const registerRoutes = async (
});
const secretRotationV2Service = secretRotationV2ServiceFactory({
+ eventEmitter: eventOutboxService,
secretRotationV2DAL,
permissionService,
appConnectionService,
diff --git a/backend/src/services/alert/providers/identity-credential-alert-provider.test.ts b/backend/src/services/alert/providers/identity-credential-alert-provider.test.ts
index c1ddb9407d..80bc32a415 100644
--- a/backend/src/services/alert/providers/identity-credential-alert-provider.test.ts
+++ b/backend/src/services/alert/providers/identity-credential-alert-provider.test.ts
@@ -199,7 +199,10 @@ describe("identity credential alert provider", () => {
// place the two get checked against each other.
test("the emitted payload is accepted by the delivery schema", async () => {
const emitted: unknown[] = [];
- const emitter = { emit: async (event: { payload: unknown }) => void emitted.push(event.payload) };
+ const emitter = {
+ emit: async (event: { payload: unknown }) => void emitted.push(event.payload),
+ emitMany: async () => {}
+ };
const membership = { identity: { id: "ident-1", projectId: null }, scopeOrgId: "org-1" };
await emitIdentityAuthMethodChanged(
diff --git a/backend/src/services/alert/providers/secret-reminder-alert-provider.test.ts b/backend/src/services/alert/providers/secret-reminder-alert-provider.test.ts
index 824dfb79f6..464d1a0d12 100644
--- a/backend/src/services/alert/providers/secret-reminder-alert-provider.test.ts
+++ b/backend/src/services/alert/providers/secret-reminder-alert-provider.test.ts
@@ -106,7 +106,8 @@ describe("secret reminder alert provider", () => {
{
emit: async (event) => {
emitted = event.payload;
- }
+ },
+ emitMany: async () => {}
},
{
orgId: "org-1",
diff --git a/backend/src/services/event-outbox/event-definition.test.ts b/backend/src/services/event-outbox/event-definition.test.ts
new file mode 100644
index 0000000000..d995c25bb6
--- /dev/null
+++ b/backend/src/services/event-outbox/event-definition.test.ts
@@ -0,0 +1,81 @@
+import { z } from "zod";
+
+import { defineEvent, toCloudEvent } from "./event-definition";
+import { EventInputSchema } from "./event-outbox-types";
+
+const ExampleEvent = defineEvent({
+ domain: "secrets",
+ entity: "secret-import",
+ action: "modified",
+ version: 2,
+ data: z.object({ secretImportId: z.string() })
+});
+
+describe("defineEvent", () => {
+ test("names the event and its source by the ADR convention", () => {
+ expect(ExampleEvent.type).toBe("com.infisical.secrets.secret-import.modified.v2");
+ expect(ExampleEvent.source).toBe("urn:infisical:secrets");
+ });
+
+ // The outbox validates every emitted key; a definition must never produce one it drops.
+ test("builds an input the outbox accepts", () => {
+ const input = ExampleEvent.build({ secretImportId: "import-1" });
+
+ expect(input).toEqual({
+ eventType: "com.infisical.secrets.secret-import.modified.v2",
+ payload: { secretImportId: "import-1" }
+ });
+ expect(EventInputSchema.safeParse(input).success).toBe(true);
+ });
+
+ test.each([
+ ["domain", { domain: "Secrets" }],
+ ["entity", { entity: "secret_import" }],
+ ["action", { action: "modified.again" }],
+ ["entity", { entity: "" }]
+ ])("rejects a %s that is not lower-kebab-case", (_, override) => {
+ expect(() =>
+ defineEvent({
+ domain: "secrets",
+ entity: "secret",
+ action: "modified",
+ version: 1,
+ data: z.object({}),
+ ...override
+ })
+ ).toThrow("must be lower-kebab-case");
+ });
+
+ test.each([0, -1, 1.5])("rejects version %s", (version) => {
+ expect(() =>
+ defineEvent({ domain: "secrets", entity: "secret", action: "modified", version, data: z.object({}) })
+ ).toThrow("must be a positive integer");
+ });
+});
+
+describe("toCloudEvent", () => {
+ const event = {
+ id: 42,
+ eventId: "550e8400-e29b-41d4-a716-446655440000",
+ eventType: ExampleEvent.type,
+ payload: { secretImportId: "import-1" },
+ occurredAt: new Date("2026-08-20T17:42:31.000Z")
+ };
+
+ test("maps the stored event onto the CloudEvents envelope", () => {
+ expect(toCloudEvent(ExampleEvent, event)).toEqual({
+ specversion: "1.0",
+ id: "550e8400-e29b-41d4-a716-446655440000",
+ source: "urn:infisical:secrets",
+ type: "com.infisical.secrets.secret-import.modified.v2",
+ time: "2026-08-20T17:42:31.000Z",
+ data: { secretImportId: "import-1" }
+ });
+ });
+
+ test("refuses an event of another type", () => {
+ expect(() =>
+ toCloudEvent(ExampleEvent, { ...event, eventType: "com.infisical.secrets.secret.modified.v1" })
+ ).toThrow("cannot be read as");
+ });
+});
diff --git a/backend/src/services/event-outbox/event-definition.ts b/backend/src/services/event-outbox/event-definition.ts
new file mode 100644
index 0000000000..be503cecc6
--- /dev/null
+++ b/backend/src/services/event-outbox/event-definition.ts
@@ -0,0 +1,75 @@
+import RE2 from "re2";
+import { z } from "zod";
+
+import { TEvent, TEventInput } from "./event-outbox-types";
+
+// The naming and envelope rules from the Event Standards and Versioning ADR, which adopts CloudEvents
+// 1.0: an event is named com.infisical.<domain>.<entity>.<action>.v<version>, every segment
+// lower-kebab, the entity singular and the action past tense. The version moves only on a breaking
+// change to `data`, so adding an optional field is not a new version.
+const SEGMENT_PATTERN = new RE2("^[a-z][a-z0-9]*(?:-[a-z0-9]+)*$");
+
+// Throws at module load, so a name that breaks the convention fails the process on boot rather than
+// reaching a request.
+export const defineEvent = <TData extends z.AnyZodObject>({
+ domain,
+ entity,
+ action,
+ version,
+ data
+}: {
+ domain: string;
+ entity: string;
+ action: string;
+ version: number;
+ data: TData;
+}) => {
+ Object.entries({ domain, entity, action }).forEach(([segment, value]) => {
+ if (!SEGMENT_PATTERN.test(value)) {
+ throw new Error(`Event ${segment} '${value}' must be lower-kebab-case, e.g. 'secret-import'`);
+ }
+ });
+ if (!Number.isInteger(version) || version < 1) {
+ throw new Error(`Event version '${version}' must be a positive integer`);
+ }
+
+ const type = `com.infisical.${domain}.${entity}.${action}.v${version}`;
+
+ return {
+ type,
+ source: `urn:infisical:${domain}`,
+ data,
+ build: (payload: z.input<TData>): TEventInput => ({ eventType: type, payload })
+ };
+};
+
+export type TEventDefinition<TData extends z.AnyZodObject> = ReturnType<typeof defineEvent<TData>>;
+
+// The CloudEvents 1.0 envelope with the ADR's constraints. Consumers inside the backend read `TEvent`;
+// this is the shape an event takes when it leaves the backend (eg a customer webhook).
+export type TCloudEvent<TData> = {
+ specversion: "1.0";
+ id: string;
+ source: string;
+ type: string;
+ time: string;
+ data: TData;
+};
+
+export const toCloudEvent = <TData extends z.AnyZodObject>(
+ definition: TEventDefinition<TData>,
+ event: TEvent
+): TCloudEvent<z.infer<TData>> => {
+ if (event.eventType !== definition.type) {
+ throw new Error(`Event '${event.eventType}' cannot be read as '${definition.type}'`);
+ }
+
+ return {
+ specversion: "1.0",
+ id: event.eventId,
+ source: definition.source,
+ type: definition.type,
+ time: new Date(event.occurredAt).toISOString(),
+ data: definition.data.parse(event.payload)
+ };
+};
diff --git a/backend/src/services/event-outbox/event-outbox-dal.test.ts b/backend/src/services/event-outbox/event-outbox-dal.test.ts
index 102c417f64..7c0606855f 100644
--- a/backend/src/services/event-outbox/event-outbox-dal.test.ts
+++ b/backend/src/services/event-outbox/event-outbox-dal.test.ts
@@ -118,6 +118,7 @@ describe("event outbox dal", () => {
[
{
consumer: "alert",
+ eventId: "550e8400-e29b-41d4-a716-446655440000",
eventType: "approval.workflow.request_opened",
payload: { orgId: "org-1", resourceType: "approval.workflow", resourceId: "policy-1", targetIds: ["req-1"] },
occurredAt: new Date()
diff --git a/backend/src/services/event-outbox/event-outbox-dal.ts b/backend/src/services/event-outbox/event-outbox-dal.ts
index 40ac53508e..7a0d28f021 100644
--- a/backend/src/services/event-outbox/event-outbox-dal.ts
+++ b/backend/src/services/event-outbox/event-outbox-dal.ts
@@ -5,6 +5,7 @@ import { Knex } from "knex";
import { TDbClient } from "@app/db";
import { TableName, TEventOutbox } from "@app/db/schemas";
import { DatabaseError } from "@app/lib/errors";
+import { chunkArray } from "@app/lib/fn";
import { computeBackoffMs, EventOutboxStatus, TOutboxFlushKey } from "./event-outbox-types";
@@ -17,26 +18,34 @@ export type TOutboxClaim = {
export type TOutboxInsertRow = {
consumer: string;
+ eventId: string;
eventType: string;
payload: unknown;
idempotencyKey?: string | null;
occurredAt: Date;
};
+// Postgres caps a statement at 65535 bind parameters, and a row takes seven, so a bulk secret write
+// emitting one event per secret would otherwise fail the whole write past ~9k secrets.
+const INSERT_BATCH_SIZE = 1000;
+
export const eventOutboxDALFactory = (db: TDbClient) => {
const insertEvents = async (rows: TOutboxInsertRow[], tx?: Knex): Promise<void> => {
if (rows.length === 0) return;
try {
- await (tx || db)(TableName.EventOutbox)
- .insert(
- rows.map((row) => ({
- ...row,
- payload: JSON.stringify(row.payload),
- idempotencyKey: row.idempotencyKey ?? null
- }))
- )
- .onConflict(db.raw('("consumer", "idempotencyKey") WHERE "idempotencyKey" IS NOT NULL'))
- .ignore();
+ for (const chunk of chunkArray(rows, INSERT_BATCH_SIZE)) {
+ // eslint-disable-next-line no-await-in-loop -- one statement at a time on the caller's connection
+ await (tx || db)(TableName.EventOutbox)
+ .insert(
+ chunk.map((row) => ({
+ ...row,
+ payload: JSON.stringify(row.payload),
+ idempotencyKey: row.idempotencyKey ?? null
+ }))
+ )
+ .onConflict(db.raw('("consumer", "idempotencyKey") WHERE "idempotencyKey" IS NOT NULL'))
+ .ignore();
+ }
} catch (error) {
throw new DatabaseError({ error, name: "EventOutbox: insertEvents" });
}
diff --git a/backend/src/services/event-outbox/event-outbox-service.test.ts b/backend/src/services/event-outbox/event-outbox-service.test.ts
index 2cce8ceb34..5270e1582c 100644
--- a/backend/src/services/event-outbox/event-outbox-service.test.ts
+++ b/backend/src/services/event-outbox/event-outbox-service.test.ts
@@ -19,11 +19,22 @@ vi.mock("@app/lib/logger", () => ({
initLogger: () => {}
}));
-const { exhaustedMetrics } = vi.hoisted(() => ({ exhaustedMetrics: [] as { consumer: string; count: number }[] }));
+const { exhaustedMetrics, droppedMetrics } = vi.hoisted(() => ({
+ exhaustedMetrics: [] as { consumer: string; count: number }[],
+ droppedMetrics: [] as { eventType: string; reason: string; consumer?: string }[]
+}));
vi.mock("@app/lib/telemetry/metrics", () => ({
+ EventOutboxDropReason: {
+ InvalidEvent: "invalid_event",
+ PayloadTooLarge: "payload_too_large",
+ PayloadRejected: "payload_rejected"
+ },
recordEventOutboxLagMetric: () => {},
recordEventOutboxExhaustedMetric: (params: { consumer: string; count: number }) => {
exhaustedMetrics.push(params);
+ },
+ recordEventOutboxDroppedMetric: (params: { eventType: string; reason: string; consumer?: string }) => {
+ droppedMetrics.push(params);
}
}));
@@ -105,32 +116,90 @@ describe("event outbox emit", () => {
expect(inserted[0].tx).toBe(TX);
});
- // An unreadable payload means a wrong emit site. Failing loudly beats dropping a notification over a
- // misspelled field.
- test("throws when the payload does not match the consumer's schema", async () => {
- const { service, inserted } = buildService([makeConsumer()]);
+ // A bad event is a bug at the emit site, but it must not fail the business write it describes.
+ describe("drops a malformed event without failing the write", () => {
+ beforeEach(() => {
+ droppedMetrics.length = 0;
+ });
- await expect(service.emit(makeEvent({ payload: makePayload({ targetIds: [] }) }), TX)).rejects.toThrow(
- "the 'alert' consumer cannot accept"
- );
- expect(inserted).toHaveLength(0);
+ test("when the payload does not match the consumer's schema", async () => {
+ const { service, inserted } = buildService([makeConsumer()]);
+
+ await service.emit(makeEvent({ payload: makePayload({ targetIds: [] }) }), TX);
+
+ expect(inserted).toHaveLength(0);
+ expect(droppedMetrics).toEqual([
+ { eventType: "approval.workflow.request_opened", reason: "payload_rejected", consumer: "alert" }
+ ]);
+ });
+
+ test("only for the consumer that rejects it", async () => {
+ const { service, insertedRows } = buildService([
+ makeConsumer({ name: "alert" }),
+ makeConsumer({ name: "lenient", payloadSchema: z.object({}) as never })
+ ]);
+
+ await service.emit(makeEvent({ payload: makePayload({ targetIds: [] }) }), TX);
+
+ expect(insertedRows().map((row) => (row as { consumer: string }).consumer)).toEqual(["lenient"]);
+ });
+
+ test("when the payload is oversized", async () => {
+ const { service, inserted } = buildService([
+ makeConsumer({ payloadSchema: z.object({ blob: z.string() }) as never })
+ ]);
+
+ await service.emit(makeEvent({ payload: { blob: "x".repeat(MAX_OUTBOX_PAYLOAD_BYTES + 1) } }), TX);
+
+ expect(inserted).toHaveLength(0);
+ expect(droppedMetrics).toEqual([{ eventType: "approval.workflow.request_opened", reason: "payload_too_large" }]);
+ });
+
+ test("when the envelope is malformed", async () => {
+ const { service, inserted } = buildService([makeConsumer()]);
+
+ await service.emit(makeEvent({ eventType: "Not A Key" }), TX);
+
+ expect(inserted).toHaveLength(0);
+ expect(droppedMetrics).toEqual([{ eventType: "Not A Key", reason: "invalid_event" }]);
+ });
+
+ test("and still writes the well-formed events emitted with it", async () => {
+ const { service, insertedRows } = buildService([makeConsumer()]);
+
+ await service.emitMany([makeEvent({ eventType: "Not A Key" }), makeEvent()], TX);
+
+ expect(insertedRows()).toHaveLength(1);
+ });
});
- test("throws on an oversized payload rather than storing a document", async () => {
- const { service, inserted } = buildService([
- makeConsumer({ payloadSchema: z.object({ blob: z.string() }) as never })
- ]);
+ test("gives every consumer's row for one event the same eventId", async () => {
+ const { service, insertedRows } = buildService([makeConsumer({ name: "alert" }), makeConsumer({ name: "other" })]);
- await expect(service.emit(makeEvent({ payload: { blob: "x".repeat(20_000) } }), TX)).rejects.toThrow(
- `over the ${MAX_OUTBOX_PAYLOAD_BYTES} byte limit`
- );
- expect(inserted).toHaveLength(0);
+ await service.emit(makeEvent(), TX);
+
+ const [first, second] = insertedRows() as { eventId: string }[];
+ expect(first.eventId).toMatch(/^[0-9a-f-]{36}$/);
+ expect(second.eventId).toBe(first.eventId);
});
- test("throws on a malformed event envelope", async () => {
- const { service } = buildService([makeConsumer()]);
+ test("writes many events in one insert, each with its own eventId, inside the caller's tx", async () => {
+ const { service, inserted } = buildService([makeConsumer()]);
- await expect(service.emit(makeEvent({ eventType: "Not A Key" }), TX)).rejects.toThrow("Invalid outbox event");
+ await service.emitMany([makeEvent(), makeEvent(), makeEvent()], TX);
+
+ expect(inserted).toHaveLength(1);
+ expect(inserted[0].tx).toBe(TX);
+ const eventIds = (inserted[0].rows as { eventId: string }[]).map((row) => row.eventId);
+ expect(new Set(eventIds).size).toBe(3);
+ });
+
+ test("does not touch the database when every event is dropped", async () => {
+ const { service, inserted } = buildService([makeConsumer()]);
+
+ await service.emitMany([makeEvent({ eventType: "Not A Key" })], TX);
+
+ expect(inserted).toHaveLength(0);
});
// Swallowing it would hand the caller a poisoned transaction instead of the real error.
diff --git a/backend/src/services/event-outbox/event-outbox-service.ts b/backend/src/services/event-outbox/event-outbox-service.ts
index 595fd4cbcc..13c04b9b26 100644
--- a/backend/src/services/event-outbox/event-outbox-service.ts
+++ b/backend/src/services/event-outbox/event-outbox-service.ts
@@ -1,9 +1,15 @@
+import crypto from "node:crypto";
+
import { Knex } from "knex";
import { TEventOutbox } from "@app/db/schemas";
-import { InternalServerError } from "@app/lib/errors";
import { logger } from "@app/lib/logger";
-import { recordEventOutboxExhaustedMetric, recordEventOutboxLagMetric } from "@app/lib/telemetry/metrics";
+import {
+ EventOutboxDropReason,
+ recordEventOutboxDroppedMetric,
+ recordEventOutboxExhaustedMetric,
+ recordEventOutboxLagMetric
+} from "@app/lib/telemetry/metrics";
import { TEventOutboxDALFactory, TOutboxInsertRow } from "./event-outbox-dal";
import { TEventOutboxRegistry } from "./event-outbox-registry";
@@ -97,54 +103,71 @@ export type TEventOutboxServiceFactoryDep = {
export type TEventOutboxServiceFactory = ReturnType<typeof eventOutboxServiceFactory>;
export const eventOutboxServiceFactory = ({ eventOutboxDAL, eventOutboxRegistry }: TEventOutboxServiceFactoryDep) => {
- // Pass the caller's `tx` so the row commits with the business write; omit it only when there is no
- // business write to be atomic with. Nothing is caught on purpose: a bad event is a bug at the emit
- // site (500, not 400), and swallowing a failed insert would leave the caller with a poisoned
- // transaction.
- const emit = async (event: TEventInput, tx?: Knex): Promise<void> => {
+ // A malformed event is dropped, logged and counted, never thrown: it is a bug at the emit site, and
+ // failing the caller's write over it would turn a missed notification into a failed secret update.
+ // Every check runs before any DB access, so a drop leaves the caller's transaction untouched.
+ const $drop = (eventType: string, reason: EventOutboxDropReason, detail: string, consumer?: string) => {
+ logger.error(
+ `event-outbox: dropped event [eventType=${eventType}] [reason=${reason}]${consumer ? ` [consumer=${consumer}]` : ""}: ${detail}`
+ );
+ recordEventOutboxDroppedMetric({ eventType, reason, consumer });
+ return [];
+ };
+
+ const $buildRows = (event: TEventInput, eventId: string): TOutboxInsertRow[] => {
const parsed = EventInputSchema.safeParse(event);
if (!parsed.success) {
- throw new InternalServerError({
- name: "EventOutboxInvalidEvent",
- message: `Invalid outbox event: ${formatIssues(parsed.error.issues)}`
- });
+ return $drop(event.eventType, EventOutboxDropReason.InvalidEvent, formatIssues(parsed.error.issues));
}
const validated = parsed.data;
const payloadBytes = Buffer.byteLength(JSON.stringify(validated.payload), "utf8");
if (payloadBytes > MAX_OUTBOX_PAYLOAD_BYTES) {
- throw new InternalServerError({
- name: "EventOutboxPayloadTooLarge",
- message: `Outbox event payload for '${validated.eventType}' is ${payloadBytes} bytes, over the ${MAX_OUTBOX_PAYLOAD_BYTES} byte limit. The outbox carries identifiers, not documents.`
- });
+ return $drop(
+ validated.eventType,
+ EventOutboxDropReason.PayloadTooLarge,
+ `payload is ${payloadBytes} bytes, over the ${MAX_OUTBOX_PAYLOAD_BYTES} byte limit; the outbox carries identifiers, not documents`
+ );
}
- const subscribers = eventOutboxRegistry.subscribersOf(validated.eventType);
- if (subscribers.length === 0) return;
-
const occurredAt = validated.occurredAt ?? new Date();
- const rows: TOutboxInsertRow[] = subscribers.map((consumer) => {
+ return eventOutboxRegistry.subscribersOf(validated.eventType).flatMap((consumer) => {
const payload = consumer.payloadSchema.safeParse(validated.payload);
if (!payload.success) {
- throw new InternalServerError({
- name: "EventOutboxPayloadRejected",
- message: `Outbox event '${validated.eventType}' has a payload the '${consumer.name}' consumer cannot accept: ${formatIssues(payload.error.issues)}`
- });
+ return $drop(
+ validated.eventType,
+ EventOutboxDropReason.PayloadRejected,
+ formatIssues(payload.error.issues),
+ consumer.name
+ );
}
- return {
- consumer: consumer.name,
- eventType: validated.eventType,
- payload: validated.payload,
- idempotencyKey: validated.idempotencyKey ?? null,
- occurredAt
- };
+ return [
+ {
+ consumer: consumer.name,
+ eventId,
+ eventType: validated.eventType,
+ payload: validated.payload,
+ idempotencyKey: validated.idempotencyKey ?? null,
+ occurredAt
+ }
+ ];
});
+ };
+
+ // Pass the caller's `tx` so the rows commit with the business write; omit it only when there is no
+ // business write to be atomic with. A failed insert is not caught: swallowing it would leave the
+ // caller with a poisoned transaction.
+ const emitMany = async (events: TEventInput[], tx?: Knex): Promise<void> => {
+ const rows = events.flatMap((event) => $buildRows(event, crypto.randomUUID()));
+ if (rows.length === 0) return;
await eventOutboxDAL.insertEvents(rows, tx);
};
+ const emit = (event: TEventInput, tx?: Knex): Promise<void> => emitMany([event], tx);
+
const $commitWithRetry = async (
{ consumer }: TOutboxFlushKey,
input: Parameters<TEventOutboxDALFactory["commitResults"]>[0]
@@ -348,5 +371,5 @@ export const eventOutboxServiceFactory = ({ eventOutboxDAL, eventOutboxRegistry
}
};
- return { emit, drain, sweepStaleClaims, pruneTerminalRows };
+ return { emit, emitMany, drain, sweepStaleClaims, pruneTerminalRows };
};
diff --git a/backend/src/services/event-outbox/event-outbox-types.ts b/backend/src/services/event-outbox/event-outbox-types.ts
index efd1a58a5a..ad25b6335e 100644
--- a/backend/src/services/event-outbox/event-outbox-types.ts
+++ b/backend/src/services/event-outbox/event-outbox-types.ts
@@ -59,10 +59,13 @@ export type TEventInput = z.input<typeof EventInputSchema>;
export type TEventEmitter = {
emit: (event: TEventInput, tx?: Knex) => Promise<void>;
+ emitMany: (events: TEventInput[], tx?: Knex) => Promise<void>;
};
-// What a consumer sees. Lock, attempt and status columns are the outbox's business.
-export type TEvent = Pick<TEventOutbox, "id" | "eventType" | "payload" | "occurredAt">;
+// What a consumer sees. Lock, attempt and status columns are the outbox's business. `id` is this
+// consumer's row, which is what a result reports on; `eventId` is the occurrence, shared by every
+// consumer's row for it.
+export type TEvent = Pick<TEventOutbox, "id" | "eventId" | "eventType" | "payload" | "occurredAt">;
export enum EventResultStatus {
Delivered = "delivered",
diff --git a/backend/src/services/external-migration/external-migration-fns/envkey.ts b/backend/src/services/external-migration/external-migration-fns/envkey.ts
index ffdf80e9c0..5a69e3ef8d 100644
--- a/backend/src/services/external-migration/external-migration-fns/envkey.ts
+++ b/backend/src/services/external-migration/external-migration-fns/envkey.ts
@@ -5,6 +5,7 @@ import tweetnaclUtil from "tweetnacl-util";
import { crypto } from "@app/lib/crypto/cryptography";
import { BadRequestError } from "@app/lib/errors";
import { logger } from "@app/lib/logger";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "../../folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "../../kms/kms-service";
@@ -39,6 +40,7 @@ export type TImportDataIntoInfisicalDTO = {
projectEnvService: Pick<TProjectEnvServiceFactory, "createEnvironment">;
secretV2BridgeService: Pick<TSecretV2BridgeServiceFactory, "createManySecret">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
folderVersionDAL: Pick<TSecretFolderVersionDALFactory, "create">;
input: TImportInfisicalDataCreate;
diff --git a/backend/src/services/external-migration/external-migration-fns/import.ts b/backend/src/services/external-migration/external-migration-fns/import.ts
index 7307721d7b..9a4fef9c6c 100644
--- a/backend/src/services/external-migration/external-migration-fns/import.ts
+++ b/backend/src/services/external-migration/external-migration-fns/import.ts
@@ -1,3 +1,5 @@
+import path from "node:path";
+
import slugify from "@sindresorhus/slugify";
import { SecretType, TSecretFolders } from "@app/db/schemas";
@@ -8,6 +10,7 @@ import { alphaNumericNanoId } from "@app/lib/nanoid";
import { CommitType } from "@app/services/folder-commit/folder-commit-service";
import { KmsDataKey } from "@app/services/kms/kms-types";
import { createSecretBlindIndexer } from "@app/services/secret-v2-bridge/secret-blind-index-fns";
+import { TSecretEventScope } from "@app/services/secret-v2-bridge/secret-events";
import { getAllSecretReferences } from "@app/services/secret-v2-bridge/secret-reference-fns";
import { fnSecretBulkInsert } from "@app/services/secret-v2-bridge/secret-v2-bridge-fns";
@@ -26,6 +29,7 @@ export const importDataIntoInfisicalFn = async ({
resourceMetadataDAL,
folderVersionDAL,
folderCommitService,
+ eventEmitter,
input: { data, actor, actorId, actorOrgId, actorAuthMethod }
}: TImportDataIntoInfisicalDTO) => {
// Import data to infisical
@@ -45,6 +49,7 @@ export const importDataIntoInfisicalFn = async ({
envSlug: string;
folderId: string;
projectId: string;
+ secretPath: string;
}
>();
const projectsNotImported: string[] = [];
@@ -167,7 +172,8 @@ export const importDataIntoInfisicalFn = async ({
folderId: newFolder.id,
envId: parentEnv?.envId || parentFolder?.envId || "",
envSlug: parentEnv?.envSlug || parentFolder?.envSlug || "",
- projectId: parentEnv?.projectId || parentFolder?.projectId || ""
+ projectId: parentEnv?.projectId || parentFolder?.projectId || "",
+ secretPath: path.posix.join(parentFolder?.secretPath ?? "/", newFolder.name)
});
}
}
@@ -226,14 +232,18 @@ export const importDataIntoInfisicalFn = async ({
logger.info("[importDataIntoInfisicalFn]: Processing secrets for targetId", targetId);
let selectedFolder: TSecretFolders | undefined;
- let selectedProjectId: string | undefined;
+ let selectedScope: TSecretEventScope | undefined;
// Case 1: Secret belongs to a folder / branch / branch of a block
const foundFolder = originalToNewFolderId.get(targetId);
if (foundFolder) {
logger.info("[importDataIntoInfisicalFn]: Processing secrets for folder");
selectedFolder = await folderDAL.findById(foundFolder.folderId, tx);
- selectedProjectId = foundFolder.projectId;
+ selectedScope = {
+ projectId: foundFolder.projectId,
+ environment: foundFolder.envSlug,
+ secretPath: foundFolder.secretPath
+ };
} else {
logger.info("[importDataIntoInfisicalFn]: Processing secrets for normal environment");
const environment = data.environments.find((env) => env.id === targetId);
@@ -277,7 +287,7 @@ export const importDataIntoInfisicalFn = async ({
}
selectedFolder = folder;
- selectedProjectId = projectId;
+ selectedScope = { projectId, environment: env.envSlug, secretPath: "/" };
}
if (!selectedFolder) {
@@ -287,7 +297,7 @@ export const importDataIntoInfisicalFn = async ({
});
}
- if (!selectedProjectId) {
+ if (!selectedScope) {
throw new NotFoundError({
message: `Project not found for the given environment slug & secret path`,
name: "CreateSecret"
@@ -297,12 +307,12 @@ export const importDataIntoInfisicalFn = async ({
const { encryptor: secretManagerEncrypt } = await kmsService.createCipherPairWithDataKey(
{
type: KmsDataKey.SecretManager,
- projectId: selectedProjectId
+ projectId: selectedScope.projectId
},
tx
);
const blindIndexer = await createSecretBlindIndexer({
- projectId: selectedProjectId,
+ projectId: selectedScope.projectId,
orgId: actorOrgId,
kmsService,
tx
@@ -346,6 +356,8 @@ export const importDataIntoInfisicalFn = async ({
await fnSecretBulkInsert({
inputSecrets: inputSecretsWithBlindIndex,
folderId: selectedFolder.id,
+ eventEmitter,
+ eventScope: selectedScope,
orgId: actorOrgId,
resourceMetadataDAL,
secretDAL,
diff --git a/backend/src/services/external-migration/external-migration-queue.ts b/backend/src/services/external-migration/external-migration-queue.ts
index a5aa5870a6..1b101e787a 100644
--- a/backend/src/services/external-migration/external-migration-queue.ts
+++ b/backend/src/services/external-migration/external-migration-queue.ts
@@ -8,6 +8,7 @@ import { crypto } from "@app/lib/crypto/cryptography";
import { logger } from "@app/lib/logger";
import { recordLegacyRootKeyUsageMetric } from "@app/lib/telemetry/metrics";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TFolderCommitServiceFactory } from "../folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "../kms/kms-service";
@@ -53,6 +54,7 @@ export type TExternalMigrationQueueFactoryDep = {
projectEnvService: Pick<TProjectEnvServiceFactory, "createEnvironment">;
secretV2BridgeService: Pick<TSecretV2BridgeServiceFactory, "createManySecret" | "dispatchSecretCreateSideEffects">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
folderVersionDAL: Pick<TSecretFolderVersionDALFactory, "create">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
@@ -78,6 +80,7 @@ export const externalMigrationQueueFactory = ({
secretVersionTagDAL,
folderDAL,
folderCommitService,
+ eventEmitter,
folderVersionDAL,
resourceMetadataDAL,
notificationService,
@@ -156,6 +159,7 @@ export const externalMigrationQueueFactory = ({
projectEnvService,
secretV2BridgeService,
folderCommitService,
+ eventEmitter,
folderVersionDAL,
resourceMetadataDAL
});
diff --git a/backend/src/services/folder-commit/folder-commit-service.test.ts b/backend/src/services/folder-commit/folder-commit-service.test.ts
index 9dabc3a5bd..75e351e9c7 100644
--- a/backend/src/services/folder-commit/folder-commit-service.test.ts
+++ b/backend/src/services/folder-commit/folder-commit-service.test.ts
@@ -96,7 +96,13 @@ describe("folderCommitServiceFactory", () => {
findById: vi.fn().mockResolvedValue({})
};
+ const mockEventEmitter = {
+ emit: vi.fn().mockResolvedValue(undefined),
+ emitMany: vi.fn().mockResolvedValue(undefined)
+ };
+
const mockFolderDAL = {
+ findSecretPathByFolderIds: vi.fn().mockResolvedValue([]),
findByParentId: vi.fn().mockResolvedValue([]),
findByProjectId: vi.fn().mockResolvedValue([]),
deleteById: vi.fn().mockResolvedValue({}),
@@ -203,7 +209,8 @@ describe("folderCommitServiceFactory", () => {
permissionService: mockPermissionService,
kmsService: mockKmsService,
secretTagDAL: mockSecretTagDAL,
- resourceMetadataDAL: mockResourceMetadataDAL
+ resourceMetadataDAL: mockResourceMetadataDAL,
+ eventEmitter: mockEventEmitter
});
});
diff --git a/backend/src/services/folder-commit/folder-commit-service.ts b/backend/src/services/folder-commit/folder-commit-service.ts
index 96607497e9..898ec4fe1a 100644
--- a/backend/src/services/folder-commit/folder-commit-service.ts
+++ b/backend/src/services/folder-commit/folder-commit-service.ts
@@ -1,9 +1,12 @@
/* eslint-disable no-await-in-loop */
+import { posix as posixPath } from "node:path";
+
import { ForbiddenError, subject } from "@casl/ability";
import { Knex } from "knex";
import {
ActionProjectType,
+ SecretType,
TSecretFolders,
TSecretFolderVersions,
TSecretV2TagJunctionInsert,
@@ -13,10 +16,11 @@ import { TPermissionServiceFactory } from "@app/ee/services/permission/permissio
import { ProjectPermissionCommitsActions, ProjectPermissionSub } from "@app/ee/services/permission/project-permission";
import { getConfig } from "@app/lib/config/env";
import { BadRequestError, DatabaseError, NotFoundError } from "@app/lib/errors";
-import { chunkArray } from "@app/lib/fn";
+import { chunkArray, unique } from "@app/lib/fn";
import { logger } from "@app/lib/logger";
import { ActorAuthMethod, ActorType } from "../auth/auth-type";
+import { TEventEmitter } from "../event-outbox/event-outbox-types";
import { TFolderCheckpointDALFactory } from "../folder-checkpoint/folder-checkpoint-dal";
import { TFolderCheckpointResourcesDALFactory } from "../folder-checkpoint-resources/folder-checkpoint-resources-dal";
import { TFolderCommitChangesDALFactory } from "../folder-commit-changes/folder-commit-changes-dal";
@@ -28,9 +32,11 @@ import { KmsDataKey } from "../kms/kms-types";
import { TProjectDALFactory } from "../project/project-dal";
import { TResourceMetadataDALFactory } from "../resource-metadata/resource-metadata-dal";
import { TSecretFolderDALFactory } from "../secret-folder/secret-folder-dal";
+import { FolderModifiedEventV1 } from "../secret-folder/secret-folder-events";
import { TSecretFolderVersionDALFactory } from "../secret-folder/secret-folder-version-dal";
import { TSecretTagDALFactory } from "../secret-tag/secret-tag-dal";
import { createSecretBlindIndexer } from "../secret-v2-bridge/secret-blind-index-fns";
+import { emitSecretsModified } from "../secret-v2-bridge/secret-events";
import { TSecretV2BridgeDALFactory } from "../secret-v2-bridge/secret-v2-bridge-dal";
import { TSecretVersionV2DALFactory } from "../secret-v2-bridge/secret-version-dal";
import { TUserDALFactory } from "../user/user-dal";
@@ -165,6 +171,7 @@ type TFolderCommitServiceFactoryDep = {
userDAL: TUserDALFactory;
identityDAL: TIdentityDALFactory;
folderDAL: TSecretFolderDALFactory;
+ eventEmitter: TEventEmitter;
folderVersionDAL: TSecretFolderVersionDALFactory;
secretVersionV2BridgeDAL: TSecretVersionV2DALFactory;
secretV2BridgeDAL: TSecretV2BridgeDALFactory;
@@ -240,7 +247,8 @@ export const folderCommitServiceFactory = ({
permissionService,
kmsService,
secretTagDAL,
- resourceMetadataDAL
+ resourceMetadataDAL,
+ eventEmitter
}: TFolderCommitServiceFactoryDep) => {
const appCfg = getConfig();
@@ -1045,6 +1053,7 @@ export const folderCommitServiceFactory = ({
tx?: Knex
) => {
const commitChanges = [];
+ const modifiedSecretIds: string[] = [];
const folder = await folderDAL.findById(folderId, tx);
if (!folder) {
return [];
@@ -1094,6 +1103,7 @@ export const folderCommitServiceFactory = ({
}
];
await secretV2BridgeDAL.insertMany(newSecret, tx);
+ modifiedSecretIds.push(change.id);
const metadata: { key: string; value?: string; encryptedValue?: string }[] = Array.isArray(
secretVersion.metadata
@@ -1158,6 +1168,7 @@ export const folderCommitServiceFactory = ({
case "update":
{
+ modifiedSecretIds.push(change.id);
await secretV2BridgeDAL.updateById(
change.id,
{
@@ -1254,6 +1265,7 @@ export const folderCommitServiceFactory = ({
// Delete case remains unchanged
case "delete":
await secretV2BridgeDAL.deleteById(change.id, tx);
+ modifiedSecretIds.push(change.id);
commitChanges.push({
type: ChangeType.DELETE,
secretVersionId: change.versionId
@@ -1265,6 +1277,19 @@ export const folderCommitServiceFactory = ({
}
}
+ if (modifiedSecretIds.length) {
+ const [folderLocation] = await folderDAL.findSecretPathByFolderIds(folder.projectId, [folderId], tx);
+ if (folderLocation) {
+ // Folder commits only record shared secrets, so everything restored here is shared.
+ await emitSecretsModified(
+ eventEmitter,
+ { projectId: folder.projectId, environment: folderLocation.environmentSlug, secretPath: folderLocation.path },
+ modifiedSecretIds.map((id) => ({ id, type: SecretType.Shared })),
+ tx
+ );
+ }
+ }
+
return commitChanges;
};
@@ -1306,6 +1331,15 @@ export const folderCommitServiceFactory = ({
(change): change is ResourceChange & FolderChange => change.type === ResourceType.FOLDER
);
+ // An update can rename and re-parent, so the old paths are read before any write rather than derived.
+ const existingFolderIds = folderChanges.filter(({ changeType }) => changeType !== "create").map(({ id }) => id);
+ const [parentLocation, ...existingLocations] = folderChanges.length
+ ? await folderDAL.findSecretPathByFolderIds(projectId, [folderId, ...existingFolderIds], tx)
+ : [];
+ const modifiedFolders = existingLocations.flatMap((location, index) =>
+ location ? [{ folderId: existingFolderIds[index], secretPath: location.path }] : []
+ );
+
for (const change of folderChanges) {
const folderVersion = folderVersions[change.id];
@@ -1321,6 +1355,12 @@ export const folderCommitServiceFactory = ({
description: folderVersion.description
};
await folderDAL.create(newFolder, tx);
+ if (parentLocation) {
+ modifiedFolders.push({
+ folderId: change.id,
+ secretPath: posixPath.join(parentLocation.path, folderVersion.name)
+ });
+ }
const newFolderVersion = await folderVersionDAL.create(
{
@@ -1376,6 +1416,12 @@ export const folderCommitServiceFactory = ({
);
if (latestVersionDetails && Object.keys(latestVersionDetails).length > 0) {
const versionDetails = Object.values(latestVersionDetails)[0];
+ if (parentLocation) {
+ modifiedFolders.push({
+ folderId: change.id,
+ secretPath: posixPath.join(parentLocation.path, versionDetails.name)
+ });
+ }
await folderDAL.updateById(
change.id,
{
@@ -1422,6 +1468,15 @@ export const folderCommitServiceFactory = ({
throw new BadRequestError({ message: `Unknown change type: ${change.changeType}` });
}
}
+
+ if (parentLocation) {
+ await eventEmitter.emitMany(
+ unique(modifiedFolders, (folder) => `${folder.folderId}:${folder.secretPath}`).map((folder) =>
+ FolderModifiedEventV1.build({ ...folder, projectId, environment: parentLocation.environmentSlug })
+ ),
+ tx
+ );
+ }
return commitChanges;
};
diff --git a/backend/src/services/secret-folder/secret-folder-events.ts b/backend/src/services/secret-folder/secret-folder-events.ts
new file mode 100644
index 0000000000..4fa1974579
--- /dev/null
+++ b/backend/src/services/secret-folder/secret-folder-events.ts
@@ -0,0 +1,21 @@
+import { z } from "zod";
+
+import { defineEvent } from "@app/services/event-outbox/event-definition";
+
+// Covers create, rename, move and delete. `secretPath` is the folder's own path at the time of the
+// change. A rename or move emits once for the old path and once for the new one, because whatever
+// covered the folder at either location (eg a sync on a parent folder) is affected. A deleted folder
+// takes its subfolders with it, and those are not emitted separately: the event for the deleted folder
+// covers everything beneath its path.
+export const FolderModifiedEventV1 = defineEvent({
+ domain: "secrets",
+ entity: "folder",
+ action: "modified",
+ version: 1,
+ data: z.object({
+ folderId: z.string().uuid(),
+ projectId: z.string().min(1),
+ environment: z.string().min(1),
+ secretPath: z.string().min(1)
+ })
+});
diff --git a/backend/src/services/secret-folder/secret-folder-service.ts b/backend/src/services/secret-folder/secret-folder-service.ts
index 98df639c02..e2044470da 100644
--- a/backend/src/services/secret-folder/secret-folder-service.ts
+++ b/backend/src/services/secret-folder/secret-folder-service.ts
@@ -20,6 +20,7 @@ import { BadRequestError, NotFoundError } from "@app/lib/errors";
import { OrderByDirection, OrgServiceActor } from "@app/lib/types";
import { TAdditionalPrivilegeDALFactory } from "@app/services/additional-privilege/additional-privilege-dal";
import { ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { TSecretQueueFactory } from "@app/services/secret/secret-queue";
import { SecretsOrderBy } from "@app/services/secret/secret-types";
@@ -54,6 +55,7 @@ import { TSecretVersionV2DALFactory } from "../secret-v2-bridge/secret-version-d
import { TSecretVersionV2TagDALFactory } from "../secret-v2-bridge/secret-version-tag-dal";
import { TSecretValidationRuleServiceFactory } from "../secret-validation-rule/secret-validation-rule-service";
import { TSecretFolderDALFactory } from "./secret-folder-dal";
+import { FolderModifiedEventV1 } from "./secret-folder-events";
import {
TCreateFolderDTO,
TCreateManyFoldersDTO,
@@ -79,6 +81,7 @@ type TSecretFolderServiceFactoryDep = {
projectEnvDAL: Pick<TProjectEnvDALFactory, "findOne" | "findBySlugs" | "find">;
folderVersionDAL: Pick<TSecretFolderVersionDALFactory, "findLatestFolderVersions" | "create" | "insertMany" | "find">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
projectDAL: Pick<TProjectDALFactory, "findProjectBySlug">;
secretApprovalPolicyService: Pick<
TSecretApprovalPolicyServiceFactory,
@@ -130,6 +133,7 @@ export const secretFolderServiceFactory = ({
projectEnvDAL,
folderVersionDAL,
folderCommitService,
+ eventEmitter,
projectDAL,
secretApprovalPolicyService,
secretV2BridgeDAL,
@@ -209,15 +213,18 @@ export const secretFolderServiceFactory = ({
}
let currentParentId = parentFolder.id;
+ const createdFolders: { folderId: string; secretPath: string }[] = [];
// build the full path we need by processing each segment
if (parentFolder.path !== secretPath) {
const missingSegments = secretPath.substring(parentFolder.path.length).split("/").filter(Boolean);
const newFolders: TSecretFoldersInsert[] = [];
+ let segmentPath = parentFolder.path;
// process each segment sequentially
for await (const segment of missingSegments) {
+ segmentPath = path.posix.join(segmentPath, segment);
const existingSegment = await folderDAL.findOne(
{
name: segment,
@@ -242,6 +249,7 @@ export const secretFolderServiceFactory = ({
currentParentId = newFolder.id;
newFolders.push(newFolder);
+ createdFolders.push({ folderId: newFolder.id, secretPath: segmentPath });
}
}
@@ -321,6 +329,14 @@ export const secretFolderServiceFactory = ({
});
}
+ createdFolders.push({ folderId: doc.id, secretPath: folderWithFullPath.path });
+ await eventEmitter.emitMany(
+ createdFolders.map((createdFolder) =>
+ FolderModifiedEventV1.build({ ...createdFolder, projectId, environment })
+ ),
+ tx
+ );
+
return { ...doc, path: folderWithFullPath.path };
});
@@ -467,6 +483,20 @@ export const secretFolderServiceFactory = ({
});
}
+ if (doc.name !== folder.name) {
+ await eventEmitter.emitMany(
+ [folder.name, doc.name].map((folderName) =>
+ FolderModifiedEventV1.build({
+ folderId: doc.id,
+ projectId: projectId as string,
+ environment,
+ secretPath: path.posix.join(parentFolder.path, folderName)
+ })
+ ),
+ tx
+ );
+ }
+
return { oldFolder: folder, newFolder: doc };
})
);
@@ -600,6 +630,15 @@ export const secretFolderServiceFactory = ({
});
}
+ if (newFolderWithPath.path !== oldFolderWithPath.path) {
+ await eventEmitter.emitMany(
+ [oldFolderWithPath.path, newFolderWithPath.path].map((folderPath) =>
+ FolderModifiedEventV1.build({ folderId: doc.id, projectId, environment: env.slug, secretPath: folderPath })
+ ),
+ tx
+ );
+ }
+
return { newFolder: doc, newFolderPath: newFolderWithPath.path, oldFolderPath: oldFolderWithPath.path };
});
@@ -853,6 +892,16 @@ export const secretFolderServiceFactory = ({
},
tx
);
+
+ await eventEmitter.emit(
+ FolderModifiedEventV1.build({
+ folderId: doc.id,
+ projectId,
+ environment,
+ secretPath: path.posix.join(parentFolder.path, doc.name)
+ }),
+ tx
+ );
return doc;
});
@@ -1283,13 +1332,16 @@ export const secretFolderServiceFactory = ({
}
let currentParentId = parentFolder.id;
+ const createdFolderPaths: { folderId: string; secretPath: string }[] = [];
// Build the full path we need by processing each segment
if (parentFolder.path !== secretPath) {
const missingSegments = secretPath.substring(parentFolder.path.length).split("/").filter(Boolean);
const newFolders: TSecretFoldersInsert[] = [];
+ let segmentPath = parentFolder.path;
for (const segment of missingSegments) {
+ segmentPath = path.posix.join(segmentPath, segment);
const existingSegment = await folderDAL.findOne(
{
name: segment,
@@ -1313,6 +1365,7 @@ export const secretFolderServiceFactory = ({
currentParentId = newFolder.id;
newFolders.push(newFolder);
+ createdFolderPaths.push({ folderId: newFolder.id, secretPath: segmentPath });
}
}
@@ -1353,6 +1406,13 @@ export const secretFolderServiceFactory = ({
{ name, envId: env.id, version: 1, parentId: currentParentId, description },
tx
);
+ createdFolderPaths.push({ folderId: doc.id, secretPath: pathWithFolder });
+ await eventEmitter.emitMany(
+ createdFolderPaths.map((createdFolder) =>
+ FolderModifiedEventV1.build({ ...createdFolder, projectId, environment })
+ ),
+ tx
+ );
const folderVersion = await folderVersionDAL.create(
{
@@ -1539,6 +1599,16 @@ export const secretFolderServiceFactory = ({
);
}
+ await eventEmitter.emit(
+ FolderModifiedEventV1.build({
+ folderId: doc.id,
+ projectId,
+ environment,
+ secretPath: path.posix.join(parentFolder.path, doc.name)
+ }),
+ tx
+ );
+
deletedFolders.push(doc);
}
}
@@ -1967,6 +2037,17 @@ export const secretFolderServiceFactory = ({
description: entry.description
}));
const createdFolders = await folderDAL.insertMany(newFolderRows, tx);
+ await eventEmitter.emitMany(
+ plan.map((entry) =>
+ FolderModifiedEventV1.build({
+ folderId: entry.newFolderId,
+ projectId,
+ environment: destinationEnvironment,
+ secretPath: entry.destinationAbsPath
+ })
+ ),
+ tx
+ );
// folder-scoped additional privileges must follow the move: the source subtree is deleted below
// and additional_privileges.folderId cascades on delete, so repoint them at the recreated folders
@@ -2035,6 +2116,7 @@ export const secretFolderServiceFactory = ({
secretVersionTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretApprovalPolicyService,
secretApprovalRequestCreationFns,
secretQueueService,
@@ -2048,6 +2130,15 @@ export const secretFolderServiceFactory = ({
// removing only the source root removes the entire subtree. the source is always fully emptied here: a
// subtree governed by a source approval policy was rejected up front, so no secret is left pending.
const [deletedRoot] = await folderDAL.delete({ id: sourceFolder.id }, tx);
+ await eventEmitter.emit(
+ FolderModifiedEventV1.build({
+ folderId: deletedRoot.id,
+ projectId,
+ environment: sourceEnvironment,
+ secretPath: sourceFolderPath
+ }),
+ tx
+ );
const latestVersions = await folderVersionDAL.findLatestFolderVersions([deletedRoot.id], tx);
await folderCommitService.createCommit(
{
diff --git a/backend/src/services/secret-import/secret-import-events.ts b/backend/src/services/secret-import/secret-import-events.ts
new file mode 100644
index 0000000000..f3ad170e28
--- /dev/null
+++ b/backend/src/services/secret-import/secret-import-events.ts
@@ -0,0 +1,18 @@
+import { z } from "zod";
+
+import { defineEvent } from "@app/services/event-outbox/event-definition";
+
+// Covers create, update and delete of an import. `environment` and `secretPath` locate the folder the
+// import lives in, which is the folder whose effective secrets change, not the folder it imports from.
+export const SecretImportModifiedEventV1 = defineEvent({
+ domain: "secrets",
+ entity: "secret-import",
+ action: "modified",
+ version: 1,
+ data: z.object({
+ secretImportId: z.string().uuid(),
+ projectId: z.string().min(1),
+ environment: z.string().min(1),
+ secretPath: z.string().min(1)
+ })
+});
diff --git a/backend/src/services/secret-import/secret-import-service.ts b/backend/src/services/secret-import/secret-import-service.ts
index 7b5015af15..ec80885e79 100644
--- a/backend/src/services/secret-import/secret-import-service.ts
+++ b/backend/src/services/secret-import/secret-import-service.ts
@@ -20,6 +20,7 @@ import { BadRequestError, ForbiddenRequestError, NotFoundError } from "@app/lib/
import { requestMemoKeys } from "@app/lib/request-context/memo-keys";
import { requestMemoize } from "@app/lib/request-context/request-memoizer";
+import { TEventEmitter } from "../event-outbox/event-outbox-types";
import { TKmsServiceFactory } from "../kms/kms-service";
import { KmsDataKey } from "../kms/kms-types";
import { TOrgDALFactory } from "../org/org-dal";
@@ -31,9 +32,11 @@ import { TSecretDALFactory } from "../secret/secret-dal";
import { decryptSecretRaw } from "../secret/secret-fns";
import { TSecretQueueFactory } from "../secret/secret-queue";
import { TSecretFolderDALFactory } from "../secret-folder/secret-folder-dal";
+import { FolderModifiedEventV1 } from "../secret-folder/secret-folder-events";
import { TSecretV2BridgeDALFactory } from "../secret-v2-bridge/secret-v2-bridge-dal";
import { recursivelyGetSecretPaths } from "../secret-v2-bridge/secret-v2-bridge-fns";
import { TSecretImportDALFactory } from "./secret-import-dal";
+import { SecretImportModifiedEventV1 } from "./secret-import-events";
import { fnSecretsFromImports, fnSecretsV2FromImports } from "./secret-import-fns";
import {
TCreateSecretImportDTO,
@@ -48,6 +51,7 @@ import {
type TSecretImportServiceFactoryDep = {
secretImportDAL: TSecretImportDALFactory;
folderDAL: TSecretFolderDALFactory;
+ eventEmitter: TEventEmitter;
secretDAL: Pick<TSecretDALFactory, "find">;
secretV2BridgeDAL: Pick<TSecretV2BridgeDALFactory, "find" | "findByFolderIds" | "invalidateSecretCacheByProjectId">;
projectBotService: Pick<TProjectBotServiceFactory, "getBotKey">;
@@ -67,6 +71,7 @@ export type TSecretImportServiceFactory = ReturnType<typeof secretImportServiceF
export const secretImportServiceFactory = ({
secretImportDAL,
+ eventEmitter,
projectEnvDAL,
projectFolderGrantDAL,
orgDAL,
@@ -258,6 +263,11 @@ export const secretImportServiceFactory = ({
tx
);
}
+
+ await eventEmitter.emit(
+ SecretImportModifiedEventV1.build({ secretImportId: doc.id, projectId, environment, secretPath: folder.path }),
+ tx
+ );
return doc;
});
@@ -446,6 +456,11 @@ export const secretImportServiceFactory = ({
},
tx
);
+
+ await eventEmitter.emit(
+ SecretImportModifiedEventV1.build({ secretImportId: doc.id, projectId, environment, secretPath: folder.path }),
+ tx
+ );
return doc;
});
@@ -500,12 +515,26 @@ export const secretImportServiceFactory = ({
tx
);
await folderDAL.deleteById(replicatedFolder.id, tx);
+ await eventEmitter.emit(
+ FolderModifiedEventV1.build({
+ folderId: replicatedFolder.id,
+ projectId,
+ environment,
+ secretPath: replicatedFolder.path
+ }),
+ tx
+ );
}
await secretImportDAL.updateAllPosition(folder.id, doc.position, -1, 2, tx);
} else {
await secretImportDAL.updateAllPosition(folder.id, doc.position, -1, 1, tx);
}
+ await eventEmitter.emit(
+ SecretImportModifiedEventV1.build({ secretImportId: doc.id, projectId, environment, secretPath: folder.path }),
+ tx
+ );
+
const importEnv = await projectEnvDAL.findById(doc.importEnv);
if (!importEnv) {
throw new NotFoundError({
diff --git a/backend/src/services/secret-sync/secret-sync-queue.ts b/backend/src/services/secret-sync/secret-sync-queue.ts
index a09cca8754..17f5529a43 100644
--- a/backend/src/services/secret-sync/secret-sync-queue.ts
+++ b/backend/src/services/secret-sync/secret-sync-queue.ts
@@ -18,6 +18,7 @@ import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
import { SecretNameSchema } from "@app/server/lib/schemas";
import { decryptAppConnectionCredentials } from "@app/services/app-connection/app-connection-fns";
import { ActorType } from "@app/services/auth/auth-type";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TKmsServiceFactory } from "@app/services/kms/kms-service";
import { KmsDataKey } from "@app/services/kms/kms-types";
import { TOrgDALFactory } from "@app/services/org/org-dal";
@@ -128,6 +129,7 @@ type TSecretSyncQueueFactoryDep = {
secretVersionTagV2BridgeDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
licenseService: Pick<TLicenseServiceFactory, "getPlan">;
gatewayV2Service: Pick<TGatewayV2ServiceFactory, "getPlatformConnectionDetailsByGatewayId">;
gatewayPoolService: Pick<TGatewayPoolServiceFactory, "resolveEffectiveGatewayId">;
@@ -182,6 +184,7 @@ export const secretSyncQueueFactory = ({
secretVersionTagV2BridgeDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
licenseService,
gatewayV2Service,
gatewayPoolService,
@@ -223,7 +226,8 @@ export const secretSyncQueueFactory = ({
secretV2BridgeDAL,
secretVersionTagV2BridgeDAL,
resourceMetadataDAL,
- folderCommitService
+ folderCommitService,
+ eventEmitter
});
const $updateManySecretsRawFn = updateManySecretsRawFnFactory({
@@ -240,7 +244,8 @@ export const secretSyncQueueFactory = ({
secretV2BridgeDAL,
secretVersionTagV2BridgeDAL,
resourceMetadataDAL,
- folderCommitService
+ folderCommitService,
+ eventEmitter
});
// INCR + cap check + EXPIRE happen atomically inside a Lua script. TTL is refreshed
diff --git a/backend/src/services/secret-v2-bridge/secret-events.ts b/backend/src/services/secret-v2-bridge/secret-events.ts
new file mode 100644
index 0000000000..1ff5e5b1dc
--- /dev/null
+++ b/backend/src/services/secret-v2-bridge/secret-events.ts
@@ -0,0 +1,43 @@
+import { Knex } from "knex";
+import { z } from "zod";
+
+import { SecretType } from "@app/db/schemas";
+import { defineEvent } from "@app/services/event-outbox/event-definition";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
+
+// One event per shared secret per write, for create, update and delete alike: a consumer reacts to the
+// secret having changed, and re-reads its current state (or finds it gone) to learn how. A move emits
+// at both locations, since it deletes the secret at the source and writes it at the destination.
+// Personal overrides belong to one user, not the project's shared state, so nothing is emitted for them.
+export const SecretModifiedEventV1 = defineEvent({
+ domain: "secrets",
+ entity: "secret",
+ action: "modified",
+ version: 1,
+ data: z.object({
+ secretId: z.string().uuid(),
+ projectId: z.string().min(1),
+ environment: z.string().min(1),
+ secretPath: z.string().min(1)
+ })
+});
+
+// Where the written secrets live. Every low-level secret write takes one, so a caller cannot write
+// secrets without saying where, and the emit cannot be skipped.
+export type TSecretEventScope = {
+ projectId: string;
+ environment: string;
+ secretPath: string;
+};
+
+export const secretModifiedEvents = (scope: TSecretEventScope, secrets: { id: string; type: string }[]) =>
+ secrets
+ .filter(({ type }) => type === SecretType.Shared)
+ .map(({ id }) => SecretModifiedEventV1.build({ secretId: id, ...scope }));
+
+export const emitSecretsModified = (
+ eventEmitter: TEventEmitter,
+ scope: TSecretEventScope,
+ secrets: { id: string; type: string }[],
+ tx?: Knex
+): Promise<void> => eventEmitter.emitMany(secretModifiedEvents(scope, secrets), tx);
diff --git a/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.test.ts b/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.test.ts
index 04f071bf1d..d83cab73aa 100644
--- a/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.test.ts
+++ b/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.test.ts
@@ -322,6 +322,8 @@ describe("expandSecretReferencesGroupedByPath", () => {
});
describe("fnSecretBulkDelete", () => {
+ const emitMany = vi.fn().mockResolvedValue(undefined);
+
test("removes the reminder of every deleted shared secret, whatever the legacy repeat-days column holds", async () => {
const removeSecretReminder = vi.fn().mockResolvedValue(undefined);
const tx = { tx: "bulk-delete" } as never;
@@ -341,10 +343,47 @@ describe("fnSecretBulkDelete", () => {
},
secretQueueService: { removeSecretReminder },
folderCommitService: { createCommit: vi.fn() },
- secretVersionDAL: { findLatestVersionMany: vi.fn().mockResolvedValue({}) }
+ secretVersionDAL: { findLatestVersionMany: vi.fn().mockResolvedValue({}) },
+ eventEmitter: { emitMany },
+ eventScope: { projectId: "proj-1", environment: "dev", secretPath: "/" }
} as never);
expect(removeSecretReminder).toHaveBeenCalledTimes(1);
expect(removeSecretReminder).toHaveBeenCalledWith({ secretId: "secret-1", projectId: "proj-1" }, tx);
});
+
+ // The personal override deleted alongside a shared key is one user's, not the project's.
+ test("emits secret.modified for the deleted shared secret only, in the caller's transaction", async () => {
+ const tx = { tx: "bulk-delete" } as never;
+
+ await fnSecretBulkDelete({
+ folderId: "folder-1",
+ projectId: "proj-1",
+ inputSecrets: [{ type: SecretType.Shared, secretKey: "DB_PASSWORD" }],
+ actorId: "user-1",
+ tx,
+ commitChanges: [],
+ secretDAL: {
+ deleteMany: vi.fn().mockResolvedValue([
+ { id: "secret-1", type: SecretType.Shared },
+ { id: "secret-2", type: SecretType.Personal }
+ ])
+ },
+ secretQueueService: { removeSecretReminder: vi.fn() },
+ folderCommitService: { createCommit: vi.fn() },
+ secretVersionDAL: { findLatestVersionMany: vi.fn().mockResolvedValue({}) },
+ eventEmitter: { emitMany },
+ eventScope: { projectId: "proj-1", environment: "dev", secretPath: "/app" }
+ } as never);
+
+ expect(emitMany).toHaveBeenLastCalledWith(
+ [
+ {
+ eventType: "com.infisical.secrets.secret.modified.v1",
+ payload: { secretId: "secret-1", projectId: "proj-1", environment: "dev", secretPath: "/app" }
+ }
+ ],
+ tx
+ );
+ });
});
diff --git a/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.ts b/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.ts
index 7aea0a997e..1d08ed3bed 100644
--- a/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.ts
+++ b/backend/src/services/secret-v2-bridge/secret-v2-bridge-fns.ts
@@ -25,6 +25,7 @@ import { groupBy } from "@app/lib/fn";
import { logger } from "@app/lib/logger";
import { ActorType } from "../auth/auth-type";
+import { TEventEmitter } from "../event-outbox/event-outbox-types";
import { CommitType, TFolderCommitServiceFactory } from "../folder-commit/folder-commit-service";
import { KmsDataKey } from "../kms/kms-types";
import { TProjectEnvDALFactory } from "../project-env/project-env-dal";
@@ -39,6 +40,7 @@ import {
SecretValidationError
} from "../secret-validation-rule/secret-validation-rule-errors";
import { createSecretBlindIndexer, TSecretBlindIndexer, TSecretValueBlindIndexes } from "./secret-blind-index-fns";
+import { emitSecretsModified, secretModifiedEvents } from "./secret-events";
import { expandSecretReferencesFactory, getAllSecretReferences } from "./secret-reference-fns";
import { TSecretV2BridgeDALFactory } from "./secret-v2-bridge-dal";
import {
@@ -72,6 +74,8 @@ export const fnSecretBulkInsert = async ({
secretTagDAL,
secretVersionTagDAL,
folderCommitService,
+ eventEmitter,
+ eventScope,
actor,
tx
}: TFnSecretBulkInsert) => {
@@ -222,6 +226,8 @@ export const fnSecretBulkInsert = async ({
{ tx }
);
+ await emitSecretsModified(eventEmitter, eventScope, newSecrets, tx);
+
return secretsWithTags.map((secret) => ({ ...secret, _id: secret.id }));
};
@@ -237,6 +243,8 @@ export const fnSecretBulkUpdate = async ({
secretTagDAL,
secretVersionTagDAL,
resourceMetadataDAL,
+ eventEmitter,
+ eventScope,
actor
}: TFnSecretBulkUpdate) => {
const userActorId = actor && actor?.type === ActorType.USER ? actor?.actorId : undefined;
@@ -420,6 +428,8 @@ export const fnSecretBulkUpdate = async ({
}
}
+ await emitSecretsModified(eventEmitter, eventScope, newSecrets, tx);
+
return secretsWithTags.map((secret) => ({ ...secret, _id: secret.id }));
};
@@ -434,7 +444,9 @@ export const fnSecretBulkDelete = async ({
folderCommitService,
secretVersionDAL,
projectId,
- commitChanges
+ commitChanges,
+ eventEmitter,
+ eventScope
}: TFnSecretBulkDelete) => {
const deletedSecrets = await secretDAL.deleteMany(
inputSecrets.map(({ type, secretKey }) => ({
@@ -486,6 +498,8 @@ export const fnSecretBulkDelete = async ({
}
}
+ await emitSecretsModified(eventEmitter, eventScope, deletedSecrets, tx);
+
return deletedSecrets;
};
@@ -795,6 +809,7 @@ type TFnUpdateSecretLinkedReferences = {
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
folderDAL: Pick<TSecretFolderDALFactory, "findSecretPathByFolderIds">;
secretQueueService: Pick<TSecretQueueFactory, "syncSecrets">;
+ eventEmitter: TEventEmitter;
encryptor: (data: { plainText: Buffer }) => { cipherTextBlob: Buffer };
decryptor: (data: { cipherTextBlob: Buffer }) => Buffer;
blindIndexer: TSecretBlindIndexer;
@@ -819,6 +834,7 @@ export const fnUpdateSecretLinkedReferences = async ({
folderCommitService,
folderDAL,
secretQueueService,
+ eventEmitter,
encryptor,
decryptor,
blindIndexer,
@@ -980,6 +996,18 @@ export const fnUpdateSecretLinkedReferences = async ({
if (updatedFolderIds.length > 0) {
const folderPaths = await folderDAL.findSecretPathByFolderIds(projectId, updatedFolderIds, tx);
+ await eventEmitter.emitMany(
+ folderPaths.flatMap((folder) =>
+ folder
+ ? secretModifiedEvents(
+ { projectId, environment: folder.environmentSlug, secretPath: folder.path },
+ (updatedSecretsByFolder.get(folder.id) ?? []).map(({ secret }) => secret)
+ )
+ : []
+ ),
+ tx
+ );
+
await Promise.all(
folderPaths.map(async (folder) => {
if (folder) {
@@ -1022,6 +1050,7 @@ type TFnUpdateMovedSecretReferences = {
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
folderDAL: Pick<TSecretFolderDALFactory, "findSecretPathByFolderIds">;
secretQueueService: Pick<TSecretQueueFactory, "syncSecrets">;
+ eventEmitter: TEventEmitter;
encryptor: (data: { plainText: Buffer }) => { cipherTextBlob: Buffer };
decryptor: (data: { cipherTextBlob: Buffer }) => Buffer;
blindIndexer: TSecretBlindIndexer;
@@ -1051,6 +1080,7 @@ export const fnUpdateMovedSecretReferences = async ({
folderCommitService,
folderDAL,
secretQueueService,
+ eventEmitter,
encryptor,
decryptor,
blindIndexer,
@@ -1385,6 +1415,18 @@ export const fnUpdateMovedSecretReferences = async ({
if (updatedFolderIds.length > 0) {
const folderPaths = await folderDAL.findSecretPathByFolderIds(projectId, updatedFolderIds, tx);
+ await eventEmitter.emitMany(
+ folderPaths.flatMap((folder) =>
+ folder
+ ? secretModifiedEvents(
+ { projectId, environment: folder.environmentSlug, secretPath: folder.path },
+ (updatedSecretsByFolder.get(folder.id) ?? []).map(({ secret }) => secret)
+ )
+ : []
+ ),
+ tx
+ );
+
await Promise.all(
folderPaths.map(async (folder) => {
if (folder) {
@@ -1620,6 +1662,7 @@ export const fnSecretMove = async (dto: TFnSecretMove): Promise<TFnSecretMoveRes
secretVersionTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretApprovalPolicyService,
secretApprovalRequestCreationFns,
secretQueueService,
@@ -1928,6 +1971,8 @@ export const fnSecretMove = async (dto: TFnSecretMove): Promise<TFnSecretMoveRes
createdSecrets = await fnSecretBulkInsert({
folderId: destinationFolder.id,
+ eventEmitter,
+ eventScope: { projectId, environment: destinationEnvironment, secretPath: destinationFolder.path },
orgId: actorOrgId,
secretVersionDAL,
secretDAL,
@@ -1977,6 +2022,8 @@ export const fnSecretMove = async (dto: TFnSecretMove): Promise<TFnSecretMoveRes
await fnSecretBulkUpdate({
folderId: destinationFolder.id,
+ eventEmitter,
+ eventScope: { projectId, environment: destinationEnvironment, secretPath: destinationFolder.path },
orgId: actorOrgId,
resourceMetadataDAL,
folderCommitService,
@@ -2058,6 +2105,12 @@ export const fnSecretMove = async (dto: TFnSecretMove): Promise<TFnSecretMoveRes
},
tx
);
+ await emitSecretsModified(
+ eventEmitter,
+ { projectId, environment: sourceEnvironment, secretPath: sourceFolder.path },
+ locallyDeletedSecrets,
+ tx
+ );
isSourceUpdated = true;
}
@@ -2089,6 +2142,7 @@ export const fnSecretMove = async (dto: TFnSecretMove): Promise<TFnSecretMoveRes
folderCommitService,
folderDAL,
secretQueueService,
+ eventEmitter,
encryptor: ({ plainText }) => secretManagerEncryptor({ plainText }),
decryptor: ({ cipherTextBlob }) => secretManagerDecryptor({ cipherTextBlob }),
blindIndexer,
diff --git a/backend/src/services/secret-v2-bridge/secret-v2-bridge-service.ts b/backend/src/services/secret-v2-bridge/secret-v2-bridge-service.ts
index d68babdf37..b26ec0b3be 100644
--- a/backend/src/services/secret-v2-bridge/secret-v2-bridge-service.ts
+++ b/backend/src/services/secret-v2-bridge/secret-v2-bridge-service.ts
@@ -47,6 +47,7 @@ import {
} from "@app/lib/telemetry/metrics";
import { ActorType } from "../auth/auth-type";
+import { TEventEmitter } from "../event-outbox/event-outbox-types";
import { TCommitResourceChangeDTO, TFolderCommitServiceFactory } from "../folder-commit/folder-commit-service";
import { TKmsServiceFactory } from "../kms/kms-service";
import { KmsDataKey } from "../kms/kms-types";
@@ -153,6 +154,7 @@ type TSecretV2BridgeServiceFactoryDep = {
secretTagDAL: TSecretTagDALFactory;
permissionService: Pick<TPermissionServiceFactory, "getProjectPermission" | "getProjectPermissionFingerprint">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
projectEnvDAL: Pick<TProjectEnvDALFactory, "findOne" | "findBySlugs">;
folderDAL: Pick<
TSecretFolderDALFactory,
@@ -205,6 +207,7 @@ export const secretV2BridgeServiceFactory = ({
secretTagDAL,
secretVersionDAL,
folderCommitService,
+ eventEmitter,
folderDAL,
permissionService,
secretQueueService,
@@ -481,6 +484,8 @@ export const secretV2BridgeServiceFactory = ({
const secret = await secretDAL.transaction(async (tx) => {
const [createdSecret] = await fnSecretBulkInsert({
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
orgId: actorOrgId,
inputSecrets: [
{
@@ -807,6 +812,8 @@ export const secretV2BridgeServiceFactory = ({
const updatedSecret = await secretDAL.transaction(async (tx) => {
const modifiedSecretsInDB = await fnSecretBulkUpdate({
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
orgId: actorOrgId,
resourceMetadataDAL,
folderCommitService,
@@ -858,6 +865,7 @@ export const secretV2BridgeServiceFactory = ({
folderCommitService,
folderDAL,
secretQueueService,
+ eventEmitter,
encryptor: ({ plainText }) => secretManagerEncryptor({ plainText }),
decryptor: ({ cipherTextBlob }) => secretManagerDecryptor({ cipherTextBlob }),
blindIndexer,
@@ -996,6 +1004,8 @@ export const secretV2BridgeServiceFactory = ({
const modifiedSecretsInDB = await fnSecretBulkDelete({
projectId,
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
actorId,
actorType: actor,
folderCommitService,
@@ -2332,6 +2342,9 @@ export const secretV2BridgeServiceFactory = ({
const modifiedSecretsInDB = await fnSecretBulkInsert({
inputSecrets: inputSecretsWithBlindIndex,
folderId,
+ eventEmitter,
+ // A caller-supplied folder carries no path, so this is the path the folder was resolved from.
+ eventScope: { projectId, environment, secretPath },
commitChanges,
orgId: actorOrgId,
secretDAL,
@@ -2730,6 +2743,8 @@ export const secretV2BridgeServiceFactory = ({
const bulkUpdatedSecrets = await fnSecretBulkUpdate({
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath },
orgId: actorOrgId,
folderCommitService,
tx,
@@ -2762,6 +2777,7 @@ export const secretV2BridgeServiceFactory = ({
folderCommitService,
folderDAL,
secretQueueService,
+ eventEmitter,
encryptor: ({ plainText }) => secretManagerEncryptor({ plainText }),
decryptor: ({ cipherTextBlob }) => secretManagerDecryptor({ cipherTextBlob }),
blindIndexer,
@@ -2814,6 +2830,8 @@ export const secretV2BridgeServiceFactory = ({
const bulkInsertedSecrets = await fnSecretBulkInsert({
inputSecrets: inputSecretsForCreate,
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
orgId: actorOrgId,
secretDAL,
resourceMetadataDAL,
@@ -2983,6 +3001,8 @@ export const secretV2BridgeServiceFactory = ({
})),
projectId,
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
actorId,
actorType: actor,
commitChanges,
@@ -3276,6 +3296,7 @@ export const secretV2BridgeServiceFactory = ({
secretVersionTagDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
secretApprovalPolicyService,
secretApprovalRequestCreationFns,
secretQueueService,
diff --git a/backend/src/services/secret-v2-bridge/secret-v2-bridge-types.ts b/backend/src/services/secret-v2-bridge/secret-v2-bridge-types.ts
index 14606fbf49..948680f66f 100644
--- a/backend/src/services/secret-v2-bridge/secret-v2-bridge-types.ts
+++ b/backend/src/services/secret-v2-bridge/secret-v2-bridge-types.ts
@@ -20,12 +20,14 @@ import {
import { TSecretFolderDALFactory } from "@app/services/secret-folder/secret-folder-dal";
import { TSecretTagDALFactory } from "@app/services/secret-tag/secret-tag-dal";
+import { TEventEmitter } from "../event-outbox/event-outbox-types";
import { TCommitResourceChangeDTO, TFolderCommitServiceFactory } from "../folder-commit/folder-commit-service";
import { TReminderServiceFactory } from "../reminder/reminder-types";
import { TResourceMetadataDALFactory } from "../resource-metadata/resource-metadata-dal";
import { ResourceMetadataWithEncryptionDTO } from "../resource-metadata/resource-metadata-schema";
import { TSecretValidationRuleServiceFactory } from "../secret-validation-rule/secret-validation-rule-service";
import { TSecretValueBlindIndexes } from "./secret-blind-index-fns";
+import { TSecretEventScope } from "./secret-events";
import { TSecretV2BridgeDALFactory } from "./secret-v2-bridge-dal";
import { TSecretVersionV2DALFactory } from "./secret-version-dal";
import { TSecretVersionV2TagDALFactory } from "./secret-version-tag-dal";
@@ -207,6 +209,8 @@ export type TFnSecretBulkInsert = {
secretTagDAL: Pick<TSecretTagDALFactory, "saveTagsToSecretV2" | "find">;
secretVersionTagDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
+ eventScope: TSecretEventScope;
actor?: {
type: string;
actorId?: string;
@@ -243,6 +247,8 @@ export type TFnSecretBulkUpdate = {
secretTagDAL: Pick<TSecretTagDALFactory, "saveTagsToSecretV2" | "deleteTagsToSecretV2" | "find">;
secretVersionTagDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
+ eventScope: TSecretEventScope;
actor?: {
type: string;
actorId?: string;
@@ -264,6 +270,8 @@ export type TFnSecretBulkDelete = {
removeSecretReminder: (data: TRemoveSecretReminderDTO, tx?: Knex) => Promise<void>;
};
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
+ eventScope: TSecretEventScope;
secretVersionDAL: Pick<TSecretVersionV2DALFactory, "findLatestVersionMany">;
};
@@ -305,6 +313,7 @@ export type TFnSecretMove = {
secretVersionTagDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
secretApprovalPolicyService: Pick<TSecretApprovalPolicyServiceFactory, "getSecretApprovalPolicy">;
secretApprovalRequestCreationFns: Pick<
TSecretApprovalRequestCreationFnsFactory,
diff --git a/backend/src/services/secret/secret-fns.ts b/backend/src/services/secret/secret-fns.ts
index b6613f499c..f66d8f022f 100644
--- a/backend/src/services/secret/secret-fns.ts
+++ b/backend/src/services/secret/secret-fns.ts
@@ -734,6 +734,7 @@ export const createManySecretsRawFnFactory = ({
secretV2BridgeDAL,
secretVersionTagV2BridgeDAL,
folderCommitService,
+ eventEmitter,
kmsService,
resourceMetadataDAL
}: TCreateManySecretsRawFnFactory) => {
@@ -806,6 +807,8 @@ export const createManySecretsRawFnFactory = ({
tagIds: el.tags
})),
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
orgId: project.orgId,
secretDAL: secretV2BridgeDAL,
resourceMetadataDAL,
@@ -925,6 +928,7 @@ export const updateManySecretsRawFnFactory = ({
secretV2BridgeDAL,
resourceMetadataDAL,
folderCommitService,
+ eventEmitter,
kmsService
}: TUpdateManySecretsRawFnFactory) => {
const getBotKeyFn = getBotKeyFnFactory(projectBotDAL, projectDAL);
@@ -1012,6 +1016,8 @@ export const updateManySecretsRawFnFactory = ({
const updatedSecrets = await secretDAL.transaction(async (tx) =>
fnSecretV2BridgeBulkUpdate({
folderId,
+ eventEmitter,
+ eventScope: { projectId, environment, secretPath: folder.path },
orgId: project.orgId,
tx,
inputSecrets: inputSecrets.map((el) => ({
diff --git a/backend/src/services/secret/secret-queue.ts b/backend/src/services/secret/secret-queue.ts
index 2b78014b42..70250c9a6e 100644
--- a/backend/src/services/secret/secret-queue.ts
+++ b/backend/src/services/secret/secret-queue.ts
@@ -26,6 +26,7 @@ import { requestMemoKeys } from "@app/lib/request-context/memo-keys";
import { requestMemoize } from "@app/lib/request-context/request-memoizer";
import { highCardinalityMeter, recordLegacyRootKeyUsageMetric } from "@app/lib/telemetry/metrics";
import { QueueJobs, QueueName, TQueueServiceFactory } from "@app/queue";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TProjectBotDALFactory } from "@app/services/project-bot/project-bot-dal";
import { createManySecretsRawFnFactory, updateManySecretsRawFnFactory } from "@app/services/secret/secret-fns";
import { TSecretVersionDALFactory } from "@app/services/secret/secret-version-dal";
@@ -122,6 +123,7 @@ type TSecretQueueFactoryDep = {
orgService: Pick<TOrgServiceFactory, "addGhostUser">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
secretSyncQueue: Pick<TSecretSyncQueueFactory, "queueSecretSyncsSyncSecretsByPath">;
reminderService: Pick<TReminderServiceFactory, "createReminder" | "deleteReminderBySecretId">;
projectEventsService: TProjectEventsService;
@@ -189,6 +191,7 @@ export const secretQueueFactory = ({
resourceMetadataDAL,
secretSyncQueue,
folderCommitService,
+ eventEmitter,
reminderService,
projectEventsService,
licenseService,
@@ -316,7 +319,8 @@ export const secretQueueFactory = ({
secretV2BridgeDAL,
secretVersionTagV2BridgeDAL,
resourceMetadataDAL,
- folderCommitService
+ folderCommitService,
+ eventEmitter
});
const updateManySecretsRawFn = updateManySecretsRawFnFactory({
@@ -333,7 +337,8 @@ export const secretQueueFactory = ({
secretV2BridgeDAL,
secretVersionTagV2BridgeDAL,
resourceMetadataDAL,
- folderCommitService
+ folderCommitService,
+ eventEmitter
});
/**
diff --git a/backend/src/services/secret/secret-types.ts b/backend/src/services/secret/secret-types.ts
index 84547a42a4..950ea84f14 100644
--- a/backend/src/services/secret/secret-types.ts
+++ b/backend/src/services/secret/secret-types.ts
@@ -11,6 +11,7 @@ import {
} from "@app/db/schemas";
import { ProjectPermissionSecretActions } from "@app/ee/services/permission/project-permission";
import { OrderByDirection, TGenericPermission, TProjectPermission } from "@app/lib/types";
+import { TEventEmitter } from "@app/services/event-outbox/event-outbox-types";
import { TProjectDALFactory } from "@app/services/project/project-dal";
import { TProjectBotDALFactory } from "@app/services/project-bot/project-bot-dal";
import { TSecretDALFactory } from "@app/services/secret/secret-dal";
@@ -483,6 +484,7 @@ export type TCreateManySecretsRawFnFactory = {
secretVersionTagV2BridgeDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
};
export type TCreateManySecretsRawFn = {
@@ -521,6 +523,7 @@ export type TUpdateManySecretsRawFnFactory = {
secretVersionTagV2BridgeDAL: Pick<TSecretVersionV2TagDALFactory, "insertMany">;
resourceMetadataDAL: Pick<TResourceMetadataDALFactory, "insertMany" | "delete">;
folderCommitService: Pick<TFolderCommitServiceFactory, "createCommit">;
+ eventEmitter: TEventEmitter;
};
export type TUpdateManySecretsRawFn = {
diff --git a/backend/vitest.e2e.config.mts b/backend/vitest.e2e.config.mts
index b8cb8ffa3f..ab8a8d258e 100644
--- a/backend/vitest.e2e.config.mts
+++ b/backend/vitest.e2e.config.mts
@@ -99,6 +99,12 @@ export default defineConfig({
find: /^@app\/services\/event-outbox\/event-outbox-queue$/,
replacement: path.resolve(__dirname, "./e2e-test/fakes/event-outbox-queue")
},
+ // Adds a consumer that records the secret events. server/routes/index.ts and
+ // audit-log-stream-outbox.integration.spec.ts import this specifier; the latter emits none of them.
+ {
+ find: /^@app\/services\/event-outbox\/event-outbox-registry$/,
+ replacement: path.resolve(__dirname, "./e2e-test/fakes/event-outbox-registry")
+ },
// Only audit-log-stream-outbox-service.ts imports this specifier.
{
find: "./audit-log-stream-outbox-constants",