Skip to content

Infisical is the open-source platform for securing secrets, certificates, access, and AI Agents.

partition table

Dimitris Staratzis10961bc208d4
diff --git a/backend/CLAUDE.md b/backend/CLAUDE.md
index 985179c64a..19ff97e6d3 100644
--- a/backend/CLAUDE.md
+++ b/backend/CLAUDE.md
@@ -618,7 +618,7 @@ Built-in roles: `Admin`, `Member`, `Viewer`, `NoAccess`. For PAM and Agent Vault
- **Membership and role changes need no explicit invalidation call.** The fingerprint self-corrects within the marker TTL (10s eventual consistency for access granting). The old `invalidateProjectPermissionCache` / DAL version counter pattern has been removed. Folder grants are the one exception: they are covered by a version counter that writers must bump (see Folder RBAC below).
- Cache helpers (`cacheGet`, `cacheSet`, `applyReviver`) in `src/lib/cache/with-cache.ts` are shared between the simple `withCache` and the fingerprint-based `withCacheFingerprint`.
-**Folder RBAC (folder-scoped privileges)**: `additional_privileges` rows with a `folderId` + `role` (`SecretFolderRole` tiers in `folder-roles.ts`) are fetched by `$fetchProjectPermissionData` and ride in the **same** cached blob as the membership permissions, behind the same fingerprint — there is no separate folder cache. `getProjectPermissionFingerprint` (`permission-fns.ts`) is that fingerprint: `permissionDAL.getPermissionFingerprint` with the project's Postgres folder-permission version counter appended, and it is the only fingerprint source (the secret-read ETag path in `secret-v2-bridge-service.ts` uses it too). The counter is what covers folder grants, because `getPermissionFingerprint` excludes `folderId`-scoped rows and a grant's effective `secretPath` is derived from the folder tree rather than stored on the grant, so a rename changes what a grant allows without touching any row. **Only writes the membership fingerprint cannot see bump it**, via `permissionService.invalidateProjectFolderPermissionCache`: the folder-access grant create/update/delete endpoints and every folder rename/move/delete path. Membership teardowns (user, identity or group leaving a project or org, actor leaving a group) must **not** bump, even though they reap `additional_privileges` rows: they delete a `Membership` or group-membership row that the actor's `getPermissionFingerprint` already hashes, so the blob is invalidated on the same marker expiry and a bump would only be a second invalidation of the same entry. That argument holds only while a teardown reaps grants in exactly the projects the actor is losing (`filterProjectsByUserMembership` / `filterProjectsByIdentityMembership`); a reap that reached a project whose membership is untouched would need the counter again. The grants are stored raw in the blob and filtered through `isActiveRole` per request, so a temporary grant lapsing takes effect immediately; admins skip the fetch entirely, and the fingerprint's `rExp` column makes a temporary admin role lapsing re-evaluate that gate. When an actor holds folder grants, `buildProjectPermissionRules` appends per-grant CASL rules built by `buildFolderScopedPrivilegeRules` (`permission-fns.ts`): a deny of every secretPath-capable subject conditioned on `{ environment, secretPath }` (exact path, folder-only, subfolders fall back to base roles), followed by the folder role's allows with the same conditions. CASL gives the last matching rule precedence, so the folder role fully replaces base permissions at the granted path and nowhere else. The deny list `FOLDER_SCOPED_DENY_RULES` (`folder-roles.ts`) is deliberately literal: `folder-scoped-privilege-rules.test.ts` enumerates `ProjectPermissionV2Schema` and fails when a new secretPath-scoped subject or action is added, forcing a human to extend the deny list and reconsider the role tiers. Only `getProjectPermission` evaluates folder grants; audits and `getProjectPermissions` do not yet. The folder access sheet endpoints (`folderPermissionService.listFolderAccessUsers` / `listFolderAccessIdentities`) answer "who has access on this folder" without building an ability per actor: `folder-access-roles-fns.ts` builds one ability per distinct project role, probes it with every `(action, subject)` pair from `FOLDER_SCOPED_DENY_RULES` plus `SecretFolders.Read`, and caches the evaluated roster per `(projectId, folderId, actorType)` with `withCacheFingerprint` (10m data / 20s marker, fingerprint = membership counts and max timestamps plus the folder's env and path). Folder grants are merged live so a grant or revoke shows on the next request.
+**Folder RBAC (folder-scoped privileges)**: `additional_privileges` rows with a `folderId` + `role` (`SecretFolderRole` tiers in `folder-roles.ts`) are fetched by `$fetchProjectPermissionData` and ride in the **same** cached blob as the membership permissions, behind the same fingerprint — there is no separate folder cache. `getProjectPermissionFingerprint` (`permission-fns.ts`) is that fingerprint: `permissionDAL.getPermissionFingerprint` with the project's Postgres folder-permission version counter appended, and it is the only fingerprint source (the secret-read ETag path in `secret-v2-bridge-service.ts` uses it too). The counter is what covers folder grants, because `getPermissionFingerprint` excludes `folderId`-scoped rows and a grant's effective `secretPath` is derived from the folder tree rather than stored on the grant, so a rename changes what a grant allows without touching any row. **Only writes the membership fingerprint cannot see bump it**, via `permissionService.invalidateProjectFolderPermissionCache`: the folder-access grant create/update/delete endpoints and every folder rename/move/delete path. Membership teardowns (user, identity or group leaving a project or org, actor leaving a group) must **not** bump, even though they reap `additional_privileges` rows: they delete a `Membership` or group-membership row that the actor's `getPermissionFingerprint` already hashes, so the blob is invalidated on the same marker expiry and a bump would only be a second invalidation of the same entry. That argument holds only while a teardown reaps grants in exactly the projects the actor is losing (`filterProjectsByUserMembership` / `filterProjectsByIdentityMembership`); a reap that reached a project whose membership is untouched would need the counter again. The grants are stored raw in the blob and filtered through `isActiveRole` per request, so a temporary grant lapsing takes effect immediately; admins skip the fetch entirely, and the fingerprint's `rExp` column makes a temporary admin role lapsing re-evaluate that gate. When an actor holds folder grants, `buildProjectPermissionRules` appends per-grant CASL rules built by `buildFolderScopedPrivilegeRules` (`permission-fns.ts`): a deny of every secretPath-capable subject conditioned on `{ environment, secretPath }` (exact path, folder-only, subfolders fall back to base roles), followed by the folder role's allows with the same conditions. CASL gives the last matching rule precedence, so the folder role fully replaces base permissions at the granted path and nowhere else. The deny list `FOLDER_SCOPED_DENY_RULES` (`folder-roles.ts`) is deliberately literal: `folder-scoped-privilege-rules.test.ts` enumerates `ProjectPermissionV2Schema` and fails when a new secretPath-scoped subject or action is added, forcing a human to extend the deny list and reconsider the role tiers. `getProjectPermission` and the project-wide `getProjectPermissions` evaluate folder grants (the latter loads every grant in the project in one query and skips admins the same way); audits do not yet. The folder access sheet endpoints (`folderPermissionService.listFolderAccessUsers` / `listFolderAccessIdentities`) answer "who has access on this folder" without building an ability per actor: `folder-access-roles-fns.ts` builds one ability per distinct project role, probes it with every `(action, subject)` pair from `FOLDER_SCOPED_DENY_RULES` plus `SecretFolders.Read`, and caches the evaluated roster per `(projectId, folderId, actorType)` with `withCacheFingerprint` (10m data / 20s marker, fingerprint = membership counts and max timestamps plus the folder's env and path). Folder grants are merged live so a grant or revoke shows on the next request.
### Request-Scoped Memoization
@@ -1095,14 +1095,24 @@ and the events cannot say which folders a recursive read returned. Instead it ke
per day through. It is keyed on the request, not the ETag, since older clients never send one. Gated reads go to a
Redis stream drained the way the audit log ingest stream is (collect, write, then trim, under a
non-blocking lock), by the `SecretReadActivityFlush` cron every minute into an upsert.
-- **No foreign keys and no index on `lastReadAt`.** Rows of deleted objects stop updating and the daily
- `SecretReadActivityPrune` cron removes anything older than 90 days, walking the primary key. Keeping
- `lastReadAt` out of every index keeps the daily upsert a HOT update. A move creates new ids, so the window
+- **Partitioned by month, no foreign keys, no index on `lastReadAt`.** The table is range-partitioned on
+ `readMonth` (the UTC month of `lastReadAt`, part of the primary key), so a reader has one row per month and
+ `findLastReads` folds them into the latest. The hourly `SecretReadActivityPrune` cron creates the current and
+ next three months' partitions and drops a month once all of it is older than the 30-day window, since nothing
+ reads before it. Never prune with `DELETE`: at this table's size that is a full scan plus vacuum. Partition DDL
+ takes an exclusive lock on the whole table, so it runs under a short `lock_timeout` and retries next hour.
+ Partitions use `fillfactor = 90` and `lastReadAt` is in no index, which together keep the upsert a HOT update.
+ Rows of deleted objects age out with their month. A move creates new ids, so the window
starts no earlier than the secret's and its folder's creation (`trackedSince`), nor before the table existed.
- **Read path.** A read counts when it reaches the secret through its folder, an import of it, a secret
referencing it, or the source of a replication import (imports and references check the reader's own access
to the source, so revoking it would break them). The card computes the whole project once and caches it for
5 minutes, grouping principals by identical rules so each rule set is checked once per secret.
+- **Unread is only "No reads" after a full window.** A principal who has not read a secret is `tracking`, not
+ `no-reads`, until both the secret (`trackedSince`) and their access have existed for the whole window. Access
+ age is `findAccessChangedAt`: the latest membership creation, role or privilege (folder grants included)
+ creation or edit. Which secrets a change opened up is not recorded, so any change restarts the window, and a
+ custom role's rules being edited is not seen at all.
### Audit Log Event Classes and Settings
diff --git a/backend/scripts/generate-schema-types.ts b/backend/scripts/generate-schema-types.ts
index fb6d9ad9c0..db9737d139 100644
--- a/backend/scripts/generate-schema-types.ts
+++ b/backend/scripts/generate-schema-types.ts
@@ -101,6 +101,7 @@ const main = async () => {
!el.tableName.includes("certificate_requests_") &&
!el.tableName.includes("user_notifications_") &&
!el.tableName.includes("scim_events_") &&
+ !el.tableName.includes("secret_read_activity_") &&
!el.tableName.includes("active_locks") &&
el.tableName !== "intermediate_audit_logs" &&
el.tableName !== "intermediate_certificate_requests"
diff --git a/backend/src/db/migrations/20261015120000_secret-read-activity.ts b/backend/src/db/migrations/20261015120000_secret-read-activity.ts
index 4032d35073..4cb481c594 100644
--- a/backend/src/db/migrations/20261015120000_secret-read-activity.ts
+++ b/backend/src/db/migrations/20261015120000_secret-read-activity.ts
@@ -2,21 +2,50 @@ import { Knex } from "knex";
import { TableName } from "../schemas";
+// The partition cron keeps the same number of months ahead, so a read always has a partition to land in.
+const PARTITION_MONTHS_AHEAD = 3;
+
+const getUtcMonthStart = (date: Date, monthOffset: number) =>
+ new Date(Date.UTC(date.getUTCFullYear(), date.getUTCMonth() + monthOffset, 1));
+
+const formatPartitionDate = (date: Date) => date.toISOString().slice(0, 10).replaceAll("-", "");
+
export async function up(knex: Knex): Promise<void> {
if (!(await knex.schema.hasTable(TableName.SecretReadActivity))) {
- await knex.schema.createTable(TableName.SecretReadActivity, (t) => {
- // A folder or secret id. No foreign keys: each would need its own index, and rows of deleted
- // resources stop being updated, so the daily prune removes them anyway.
- t.uuid("objectId").notNullable();
- t.string("objectType", 32).notNullable();
- t.uuid("actorId").notNullable();
- t.string("actorType", 16).notNullable();
- // Left out of every index so that the daily update is a HOT update and never touches an index.
- t.datetime("lastReadAt", { useTz: true }).notNullable();
- // objectType is part of the key because one secret can be read both with and without its references
- // expanded, and the two credit different secrets.
- t.primary(["objectId", "actorId", "objectType"]);
- });
+ const createTableSql = knex.schema
+ .createTable(TableName.SecretReadActivity, (t) => {
+ // A folder or secret id. No foreign keys: each would need its own index, and rows of deleted
+ // resources stop being updated, so their month's partition is dropped with them.
+ t.uuid("objectId").notNullable();
+ t.string("objectType", 32).notNullable();
+ t.uuid("actorId").notNullable();
+ t.string("actorType", 16).notNullable();
+ // Left out of every index so that the daily update is a HOT update and never touches an index.
+ t.datetime("lastReadAt", { useTz: true }).notNullable();
+ // The UTC month of lastReadAt and the partition key, so old reads are dropped a month at a time instead
+ // of deleted row by row. A reader gets one row per month.
+ t.datetime("readMonth", { useTz: true }).notNullable();
+ // objectType is part of the key because one secret can be read both with and without its references
+ // expanded, and the two credit different secrets.
+ t.primary(["objectId", "actorId", "objectType", "readMonth"]);
+ })
+ .toString();
+ await knex.schema.raw(`${createTableSql} PARTITION BY RANGE ("readMonth")`);
+
+ const now = new Date();
+ for (let offset = 0; offset <= PARTITION_MONTHS_AHEAD; offset += 1) {
+ const from = getUtcMonthStart(now, offset);
+ const to = getUtcMonthStart(now, offset + 1);
+ // fillfactor leaves room on each page for the updated row, which a HOT update needs.
+ // eslint-disable-next-line no-await-in-loop
+ await knex.raw(
+ `CREATE TABLE ?? PARTITION OF ?? FOR VALUES FROM ('${from.toISOString()}') TO ('${to.toISOString()}') WITH (fillfactor = 90)`,
+ [
+ `${TableName.SecretReadActivity}_${formatPartitionDate(from)}_${formatPartitionDate(to)}`,
+ TableName.SecretReadActivity
+ ]
+ );
+ }
}
}
diff --git a/backend/src/db/schemas/secret-read-activity.ts b/backend/src/db/schemas/secret-read-activity.ts
index 3095237dd7..f05e0d6141 100644
--- a/backend/src/db/schemas/secret-read-activity.ts
+++ b/backend/src/db/schemas/secret-read-activity.ts
@@ -12,7 +12,8 @@ export const SecretReadActivitySchema = z.object({
objectType: z.string(),
actorId: z.string().uuid(),
actorType: z.string(),
- lastReadAt: z.date()
+ lastReadAt: z.date(),
+ readMonth: z.date()
});
export type TSecretReadActivity = z.infer<typeof SecretReadActivitySchema>;
diff --git a/backend/src/ee/routes/v1/secret-router.ts b/backend/src/ee/routes/v1/secret-router.ts
index 7fe8c4ab4d..0b904bf386 100644
--- a/backend/src/ee/routes/v1/secret-router.ts
+++ b/backend/src/ee/routes/v1/secret-router.ts
@@ -109,6 +109,7 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
200: z.object({
readActivity: z.object({
trackedSince: z.date(),
+ windowStart: z.date(),
reads: z
.object({
actorType: z.enum([ActorType.USER, ActorType.IDENTITY]),
@@ -116,6 +117,12 @@ export const registerSecretRouter = async (server: FastifyZodProvider) => {
status: z.nativeEnum(SecretReadStatus),
lastReadAt: z.date()
})
+ .array(),
+ accessChanges: z
+ .object({
+ actorId: z.string(),
+ accessChangedAt: z.date()
+ })
.array()
})
})
diff --git a/backend/src/ee/services/permission/permission-service.ts b/backend/src/ee/services/permission/permission-service.ts
index 1e9c6dce2a..1e931c24de 100644
--- a/backend/src/ee/services/permission/permission-service.ts
+++ b/backend/src/ee/services/permission/permission-service.ts
@@ -941,7 +941,43 @@ export const permissionServiceFactory = ({
});
};
+ // The project-wide permission listings leave folder grants out, so they are loaded once for the whole project
+ // and applied per actor, the way getProjectPermission applies them.
+ const $findProjectFolderScopedPrivileges = async (projectId: string) => {
+ const grants = (await additionalPrivilegeDAL.find({ projectId, $notNull: ["folderId"] })).filter(isActiveRole);
+ const privilegesByActorId = new Map<string, TProjectFolderScopedPrivilege[]>();
+ if (!grants.length) return privilegesByActorId;
+
+ const folders = await secretFolderDAL.findSecretPathByFolderIds(
+ projectId,
+ grants.map((grant) => grant.folderId as string)
+ );
+ grants.forEach((grant, index) => {
+ const folder = folders[index];
+ const actorId = grant.actorUserId ?? grant.actorIdentityId;
+ if (!folder || !grant.role || !actorId) return;
+ privilegesByActorId.set(actorId, [
+ ...(privilegesByActorId.get(actorId) ?? []),
+ {
+ id: grant.id,
+ folderId: grant.folderId as string,
+ role: grant.role,
+ environmentSlug: folder.environmentSlug,
+ secretPath: folder.path
+ }
+ ]);
+ });
+ return privilegesByActorId;
+ };
+
const getProjectPermissions: TPermissionServiceFactory["getProjectPermissions"] = async (projectId, orgId) => {
+ const folderScopedPrivilegesByActorId = await $findProjectFolderScopedPrivileges(projectId);
+ // Admins are not narrowed by folder grants, as in getProjectPermission.
+ const folderScopedPrivilegesOf = (actorId: string, roles: { role: string }[] = []) =>
+ roles.some(({ role }) => role === ProjectMembershipRole.Admin)
+ ? undefined
+ : folderScopedPrivilegesByActorId.get(actorId);
+
// fetch user permissions
const rawUserProjectPermissions = await permissionDAL.getProjectUserPermissions(projectId, orgId);
const userPermissions = rawUserProjectPermissions.map((userProjectPermission) => {
@@ -953,7 +989,11 @@ export const permissionServiceFactory = ({
permissions
})) || [];
- const rules = buildProjectPermissionRules(rolePermissions.concat(additionalPrivileges));
+ const rules = buildProjectPermissionRules(
+ rolePermissions.concat(additionalPrivileges),
+ undefined,
+ folderScopedPrivilegesOf(userProjectPermission.userId, userProjectPermission.roles)
+ );
const metadataKeyValuePair = escapeHandlebarsMissingDict(
objectify(
userProjectPermission.metadata,
@@ -992,7 +1032,11 @@ export const permissionServiceFactory = ({
permissions
})) || [];
- const rules = buildProjectPermissionRules(rolePermissions.concat(additionalPrivileges));
+ const rules = buildProjectPermissionRules(
+ rolePermissions.concat(additionalPrivileges),
+ undefined,
+ folderScopedPrivilegesOf(identityProjectPermission.identityId, identityProjectPermission.roles)
+ );
const metadataKeyValuePair = escapeHandlebarsMissingDict(
objectify(
identityProjectPermission.metadata,
diff --git a/backend/src/ee/services/secret-read-activity/secret-read-activity-dal.ts b/backend/src/ee/services/secret-read-activity/secret-read-activity-dal.ts
index f9354ac133..b6dc849458 100644
--- a/backend/src/ee/services/secret-read-activity/secret-read-activity-dal.ts
+++ b/backend/src/ee/services/secret-read-activity/secret-read-activity-dal.ts
@@ -1,18 +1,28 @@
import { TDbClient } from "@app/db";
-import { TableName, TSecretReadActivity, TSecretReadActivityInsert } from "@app/db/schemas";
+import { AccessScope, TableName } from "@app/db/schemas";
import { DatabaseError } from "@app/lib/errors";
import { chunkArray } from "@app/lib/fn";
+import { getUtcMonthStart } from "./secret-read-activity-fns";
+import { TSecretLastRead } from "./secret-read-activity-types";
+
const UPSERT_CHUNK_SIZE = 1000;
const LOOKUP_CHUNK_SIZE = 10_000;
-const PRUNE_BATCH_SIZE = 10_000;
+const PARTITION_LOCK_TIMEOUT_MS = 10_000;
+// Matches the migration, which creates the first partitions.
+const PARTITION_FILLFACTOR = 90;
+const PARTITION_NAME_PATTERN = new RegExp(`^${TableName.SecretReadActivity}_(\\d{8})_(\\d{8})$`);
+
+const formatPartitionDate = (date: Date) => date.toISOString().slice(0, 10).replaceAll("-", "");
+const parsePartitionDate = (value: string) =>
+ new Date(`${value.slice(0, 4)}-${value.slice(4, 6)}-${value.slice(6, 8)}T00:00:00.000Z`);
export type TSecretReadActivityDALFactory = ReturnType<typeof secretReadActivityDALFactory>;
export const secretReadActivityDALFactory = (db: TDbClient) => {
- const upsertLastReads = async (rows: TSecretReadActivityInsert[]) => {
+ const upsertLastReads = async (rows: TSecretLastRead[]) => {
// Postgres refuses an upsert that touches the same row twice in one statement, so keep the latest per key.
- const latestByKey = new Map<string, TSecretReadActivityInsert>();
+ const latestByKey = new Map<string, TSecretLastRead>();
rows.forEach((row) => {
const key = `${row.objectId}:${row.actorId}:${row.objectType}`;
const current = latestByKey.get(key);
@@ -23,8 +33,8 @@ export const secretReadActivityDALFactory = (db: TDbClient) => {
for (const chunk of chunkArray([...latestByKey.values()], UPSERT_CHUNK_SIZE)) {
// eslint-disable-next-line no-await-in-loop
await db(TableName.SecretReadActivity)
- .insert(chunk)
- .onConflict(["objectId", "actorId", "objectType"])
+ .insert(chunk.map((row) => ({ ...row, readMonth: getUtcMonthStart(row.lastReadAt) })))
+ .onConflict(["objectId", "actorId", "objectType", "readMonth"])
.merge(["lastReadAt"])
.where(`${TableName.SecretReadActivity}.lastReadAt`, "<", db.raw("excluded.??", ["lastReadAt"]));
}
@@ -33,15 +43,20 @@ export const secretReadActivityDALFactory = (db: TDbClient) => {
}
};
+ // A reader has one row per month, so the months are folded into the latest read.
const findLastReads = async (objectIds: string[], since: Date) => {
try {
- const rows: TSecretReadActivity[] = [];
+ const rows: TSecretLastRead[] = [];
for (const chunk of chunkArray(objectIds, LOOKUP_CHUNK_SIZE)) {
// eslint-disable-next-line no-await-in-loop
- const batch = await db
+ const batch = (await db
.replicaNode()(TableName.SecretReadActivity)
.whereIn("objectId", chunk)
- .where("lastReadAt", ">=", since);
+ .where("readMonth", ">=", getUtcMonthStart(since))
+ .where("lastReadAt", ">=", since)
+ .groupBy("objectId", "objectType", "actorId", "actorType")
+ .select("objectId", "objectType", "actorId", "actorType")
+ .max("lastReadAt as lastReadAt")) as TSecretLastRead[];
rows.push(...batch);
}
return rows;
@@ -50,46 +65,59 @@ export const secretReadActivityDALFactory = (db: TDbClient) => {
}
};
- // Walks the primary key in batches: lastReadAt has no index, and searching by it would scan the table
- // once per batch.
- const pruneOlderThan = async (cutoff: Date) => {
- let deleted = 0;
- let cursor: { objectId: string; actorId: string; objectType: string } | undefined;
- try {
- for (;;) {
- const after = cursor;
- // eslint-disable-next-line no-await-in-loop
- const batch = await db(TableName.SecretReadActivity)
- .where((qb) => {
- if (after) {
- void qb.whereRaw(`("objectId", "actorId", "objectType") > (?, ?, ?)`, [
- after.objectId,
- after.actorId,
- after.objectType
- ]);
- }
- })
- .orderBy([{ column: "objectId" }, { column: "actorId" }, { column: "objectType" }])
- .limit(PRUNE_BATCH_SIZE)
- .select("objectId", "actorId", "objectType", "lastReadAt");
+ const $listPartitions = async () => {
+ const { rows } = (await db.raw(
+ `SELECT c.relname AS name FROM pg_inherits i JOIN pg_class c ON c.oid = i.inhrelid WHERE i.inhparent = ?::regclass`,
+ [TableName.SecretReadActivity]
+ )) as { rows: { name: string }[] };
+ return rows.flatMap(({ name }) => {
+ const match = PARTITION_NAME_PATTERN.exec(name);
+ return match ? [{ name, endsAt: parsePartitionDate(match[2]) }] : [];
+ });
+ };
- const expired = batch.filter((row) => row.lastReadAt < cutoff);
- if (expired.length) {
+ // Creating or dropping a partition takes an exclusive lock on the whole table. Give up rather than wait
+ // behind a slow read, which would stall every query queued after it; the next run tries again.
+ const $runPartitionDdl = (sql: string, identifiers: string[]) =>
+ db.transaction(async (tx) => {
+ await tx.raw(`SET LOCAL lock_timeout = ${PARTITION_LOCK_TIMEOUT_MS}`);
+ await tx.raw(sql, identifiers);
+ });
+
+ const createMonthlyPartitions = async (firstMonth: Date, count: number) => {
+ try {
+ const existing = new Set((await $listPartitions()).map(({ name }) => name));
+ let created = 0;
+ for (let offset = 0; offset < count; offset += 1) {
+ const from = getUtcMonthStart(firstMonth, offset);
+ const to = getUtcMonthStart(firstMonth, offset + 1);
+ const name = `${TableName.SecretReadActivity}_${formatPartitionDate(from)}_${formatPartitionDate(to)}`;
+ if (!existing.has(name)) {
+ // The bounds are inlined because DDL takes no bind parameters. They come from date math, not input.
// eslint-disable-next-line no-await-in-loop
- deleted += await db(TableName.SecretReadActivity)
- .whereIn(
- ["objectId", "actorId", "objectType"],
- expired.map((row) => [row.objectId, row.actorId, row.objectType])
- )
- .del();
+ await $runPartitionDdl(
+ `CREATE TABLE IF NOT EXISTS ?? PARTITION OF ?? FOR VALUES FROM ('${from.toISOString()}') TO ('${to.toISOString()}') WITH (fillfactor = ${PARTITION_FILLFACTOR})`,
+ [name, TableName.SecretReadActivity]
+ );
+ created += 1;
}
+ }
+ return created;
+ } catch (error) {
+ throw new DatabaseError({ error, name: "Create secret read activity partitions" });
+ }
+ };
- if (batch.length < PRUNE_BATCH_SIZE) break;
- cursor = batch[batch.length - 1];
+ const dropPartitionsEndingBefore = async (cutoff: Date) => {
+ try {
+ const expired = (await $listPartitions()).filter(({ endsAt }) => endsAt <= cutoff);
+ for (const { name } of expired) {
+ // eslint-disable-next-line no-await-in-loop
+ await $runPartitionDdl(`DROP TABLE IF EXISTS ??`, [name]);
}
- return deleted;
+ return expired.length;
} catch (error) {
- throw new DatabaseError({ error, name: "Prune secret read activity" });
+ throw new DatabaseError({ error, name: "Drop secret read activity partitions" });
}
};
@@ -105,5 +133,52 @@ export const secretReadActivityDALFactory = (db: TDbClient) => {
}
};
- return { upsertLastReads, findLastReads, pruneOlderThan, findTrackingStartedAt };
+ // The last time each user or identity joined the project or had a role, privilege or folder grant created or
+ // changed. Which secrets a change opened up is not recorded, so any of them restarts the window. Membership
+ // updatedAt is left out because logins bump it.
+ const findAccessChangedAt = async (projectId: string) => {
+ try {
+ const [roleChanges, privilegeChanges] = await Promise.all([
+ db
+ .replicaNode()(TableName.Membership)
+ .join(TableName.MembershipRole, `${TableName.Membership}.id`, `${TableName.MembershipRole}.membershipId`)
+ .where(`${TableName.Membership}.scope`, AccessScope.Project)
+ .where(`${TableName.Membership}.scopeProjectId`, projectId)
+ .select(
+ db.ref("actorUserId").withSchema(TableName.Membership),
+ db.ref("actorIdentityId").withSchema(TableName.Membership),
+ db.ref("createdAt").withSchema(TableName.Membership).as("membershipCreatedAt"),
+ db.ref("createdAt").withSchema(TableName.MembershipRole),
+ db.ref("updatedAt").withSchema(TableName.MembershipRole)
+ ),
+ db
+ .replicaNode()(TableName.AdditionalPrivilege)
+ .where({ projectId })
+ .select("actorUserId", "actorIdentityId", "createdAt", "updatedAt")
+ ]);
+
+ const changedAtByActorId = new Map<string, Date>();
+ [...roleChanges, ...privilegeChanges].forEach((change) => {
+ const actorId = change.actorUserId ?? change.actorIdentityId;
+ if (!actorId) return;
+ const dates = [change.createdAt, change.updatedAt];
+ if ("membershipCreatedAt" in change) dates.push(change.membershipCreatedAt as Date);
+ const latest = new Date(Math.max(...dates.map((date) => new Date(date).getTime())));
+ const current = changedAtByActorId.get(actorId);
+ if (!current || latest > current) changedAtByActorId.set(actorId, latest);
+ });
+ return changedAtByActorId;
+ } catch (error) {
+ throw new DatabaseError({ error, name: "Find project access changes" });
+ }
+ };
+
+ return {
+ upsertLastReads,
+ findLastReads,
+ createMonthlyPartitions,
+ dropPartitionsEndingBefore,
+ findTrackingStartedAt,
+ findAccessChangedAt
+ };
};
diff --git a/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.test.ts b/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.test.ts
index 4907fe0ea2..cbaff9bc14 100644
--- a/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.test.ts
+++ b/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.test.ts
@@ -1,10 +1,21 @@
import { describe, expect, test } from "vitest";
-import { TSecretReadActivity } from "@app/db/schemas";
import { ActorType } from "@app/services/auth/auth-type";
-import { classifySecretReads, computeSecretReach, indexSecretReads } from "./secret-read-activity-fns";
-import { SecretReadObjectType, SecretReadStatus, TReachSecret, TSecretReachGraph } from "./secret-read-activity-types";
+import {
+ classifySecretReads,
+ computeSecretReach,
+ getUnreadStatus,
+ getUtcMonthStart,
+ indexSecretReads
+} from "./secret-read-activity-fns";
+import {
+ SecretReadObjectType,
+ SecretReadStatus,
+ TReachSecret,
+ TSecretLastRead,
+ TSecretReachGraph
+} from "./secret-read-activity-types";
const dbPass: TReachSecret = { id: "db-pass", folderId: "shared", key: "DB_PASS" };
const dbUrl: TReachSecret = { id: "db-url", folderId: "shared", key: "DB_URL" };
@@ -18,7 +29,7 @@ const graph: TSecretReachGraph = {
getImporterFolderIds: (folderId) => ({ nested: ["consumer"], consumer: ["edge"] })[folderId] ?? []
};
-const read = (overrides: Partial<TSecretReadActivity>): TSecretReadActivity => ({
+const read = (overrides: Partial<TSecretLastRead>): TSecretLastRead => ({
objectId: "shared",
objectType: SecretReadObjectType.Folder,
actorId: "identity-1",
@@ -29,6 +40,18 @@ const read = (overrides: Partial<TSecretReadActivity>): TSecretReadActivity => (
const trackedSince = new Date("2026-10-01T00:00:00.000Z");
+describe("getUtcMonthStart", () => {
+ test("uses the UTC month, not the local one", () => {
+ expect(getUtcMonthStart(new Date("2026-10-31T23:59:59.999Z"))).toEqual(new Date("2026-10-01T00:00:00.000Z"));
+ expect(getUtcMonthStart(new Date("2026-11-01T00:00:00.000Z"))).toEqual(new Date("2026-11-01T00:00:00.000Z"));
+ });
+
+ test("rolls over into the next and previous year", () => {
+ expect(getUtcMonthStart(new Date("2026-12-15T00:00:00.000Z"), 1)).toEqual(new Date("2027-01-01T00:00:00.000Z"));
+ expect(getUtcMonthStart(new Date("2026-01-15T00:00:00.000Z"), -1)).toEqual(new Date("2025-12-01T00:00:00.000Z"));
+ });
+});
+
describe("computeSecretReach", () => {
const reach = computeSecretReach(dbPass, graph);
@@ -43,7 +66,7 @@ describe("computeSecretReach", () => {
describe("classifySecretReads", () => {
const reach = computeSecretReach(dbPass, graph);
- const classify = (reads: TSecretReadActivity[]) =>
+ const classify = (reads: TSecretLastRead[]) =>
classifySecretReads(indexSecretReads(reads), reach, trackedSince).get(`${ActorType.IDENTITY}:identity-1`);
test("counts a read of a folder that imports the secret through a chain", () => {
@@ -90,3 +113,27 @@ describe("classifySecretReads", () => {
});
});
});
+
+describe("getUnreadStatus", () => {
+ const windowStart = new Date("2026-09-15T00:00:00.000Z");
+
+ test("flags no reads once the secret and the access have both existed for the whole window", () => {
+ expect(getUnreadStatus(windowStart, new Date("2026-08-01T00:00:00.000Z"), windowStart)).toEqual({
+ trackedSince: windowStart,
+ readStatus: SecretReadStatus.NoReads
+ });
+ });
+
+ test("keeps tracking a secret newer than the window", () => {
+ const createdAt = new Date("2026-10-01T00:00:00.000Z");
+ expect(getUnreadStatus(createdAt, undefined, windowStart).readStatus).toBe(SecretReadStatus.Tracking);
+ });
+
+ test("keeps tracking, from the change, a reader whose access changed inside the window", () => {
+ const accessChangedAt = new Date("2026-10-10T00:00:00.000Z");
+ expect(getUnreadStatus(windowStart, accessChangedAt, windowStart)).toEqual({
+ trackedSince: accessChangedAt,
+ readStatus: SecretReadStatus.Tracking
+ });
+ });
+});
diff --git a/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.ts b/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.ts
index 694012e500..3ef13307fc 100644
--- a/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.ts
+++ b/backend/src/ee/services/secret-read-activity/secret-read-activity-fns.ts
@@ -1,10 +1,10 @@
-import { TSecretReadActivity } from "@app/db/schemas";
import { groupBy, prefixWithSlash, removeTrailingSlash } from "@app/lib/fn";
import {
SecretReadObjectType,
SecretReadStatus,
TReachSecret,
+ TSecretLastRead,
TSecretLocation,
TSecretReach,
TSecretReachGraph,
@@ -12,6 +12,9 @@ import {
TSecretReadResult
} from "./secret-read-activity-types";
+export const getUtcMonthStart = (date: Date, monthOffset = 0) =>
+ new Date(Date.UTC(date.getUTCFullYear(), date.getUTCMonth() + monthOffset, 1));
+
export const normalizeSecretPath = (secretPath: string) => removeTrailingSlash(prefixWithSlash(secretPath));
export const toLocationKey = ({ environment, secretPath }: TSecretLocation) =>
@@ -47,7 +50,17 @@ export const computeSecretReach = (secret: TReachSecret, graph: TSecretReachGrap
return { secretId: secret.id, secretIds, folderIds };
};
-export const indexSecretReads = (reads: TSecretReadActivity[]) => groupBy(reads, (read) => read.objectId);
+export const indexSecretReads = (reads: TSecretLastRead[]) => groupBy(reads, (read) => read.objectId);
+
+// An unread secret only counts as unused once both it and the reader's current access have existed for the
+// whole window.
+export const getUnreadStatus = (trackedSince: Date, accessChangedAt: Date | undefined, windowStart: Date) => {
+ const judgedSince = accessChangedAt && accessChangedAt > trackedSince ? accessChangedAt : trackedSince;
+ return {
+ trackedSince: judgedSince,
+ readStatus: judgedSince > windowStart ? SecretReadStatus.Tracking : SecretReadStatus.NoReads
+ };
+};
// Reads before trackedSince are ignored: the secret or its folder did not exist yet.
export const classifySecretReads = (
diff --git a/backend/src/ee/services/secret-read-activity/secret-read-activity-service.ts b/backend/src/ee/services/secret-read-activity/secret-read-activity-service.ts
index ddf3fec792..c7643289b8 100644
--- a/backend/src/ee/services/secret-read-activity/secret-read-activity-service.ts
+++ b/backend/src/ee/services/secret-read-activity/secret-read-activity-service.ts
@@ -1,4 +1,4 @@
-import { SecretType, TableName, TSecretReadActivityInsert } from "@app/db/schemas";
+import { SecretType, TableName } from "@app/db/schemas";
import { DAY_IN_MS } from "@app/ee/services/audit-report/audit-report-fns";
import { reviveDates } from "@app/ee/services/insights/insights-fns";
import { checkInsightsPermission } from "@app/ee/services/insights/insights-service";
@@ -9,7 +9,7 @@ import {
} from "@app/ee/services/permission/permission-fns";
import { TPermissionServiceFactory } from "@app/ee/services/permission/permission-service-types";
import { ProjectPermissionSecretActions } from "@app/ee/services/permission/project-permission";
-import { KeyStorePrefixes, KeyStoreTtls, TKeyStoreFactory } from "@app/keystore/keystore";
+import { KeyStorePrefixes, TKeyStoreFactory } from "@app/keystore/keystore";
import { getCacheTtl, withCache } from "@app/lib/cache/with-cache";
import { getConfig } from "@app/lib/config/env";
import { CronJobName, TCronJobFactory } from "@app/lib/cron/cron-job";
@@ -33,6 +33,8 @@ import { TSecretReadActivityDALFactory } from "./secret-read-activity-dal";
import {
classifySecretReads,
computeSecretReach,
+ getUnreadStatus,
+ getUtcMonthStart,
indexSecretReads,
normalizeSecretPath,
toLocationKey
@@ -45,6 +47,7 @@ import {
TReadActivityScope,
TRecordBulkSecretReadDTO,
TRecordSecretReadDTO,
+ TSecretLastRead,
TSecretReachGraph,
TSecretReaderType,
TSecretReadStreamEntry,
@@ -52,13 +55,15 @@ import {
} from "./secret-read-activity-types";
const SECRET_READ_ACTIVITY_WINDOW_DAYS = 30;
-const SECRET_READ_RETENTION_DAYS = 90;
+// Matches the migration, which creates the first partitions.
+const SECRET_READ_PARTITION_MONTHS_AHEAD = 3;
const SECRET_READ_GATE_TTL_SECONDS = 25 * 60 * 60;
const SECRET_READ_STREAM_MAX_LENGTH = 100_000;
const SECRET_READ_FLUSH_BATCH_SIZE = 1000;
const SECRET_READ_FLUSH_MAX_ENTRIES = 50_000;
const SECRET_READ_CONSUMER_LOCK_TTL_MS = 60_000;
const SECRET_READ_STREAM_ENTRY_MAX_SECRETS = 1000;
+const UNUSED_SECRET_ACCESS_CACHE_TTL_SECONDS = 30;
type TSecretReadActivityServiceFactoryDep = {
permissionService: Pick<TPermissionServiceFactory, "getProjectPermission" | "getProjectPermissions">;
@@ -298,7 +303,7 @@ export const secretReadActivityServiceFactory = ({
return subtrees.get(folderId) as string[];
};
- const rows: TSecretReadActivityInsert[] = [];
+ const rows: TSecretLastRead[] = [];
const toRow = (read: TSecretReadStreamEntry, objectType: SecretReadObjectType, objectId: string) => ({
objectId,
objectType,
@@ -361,7 +366,7 @@ export const secretReadActivityServiceFactory = ({
}
});
- const rows: TSecretReadActivityInsert[] = [];
+ const rows: TSecretLastRead[] = [];
for (const [projectId, projectReads] of Object.entries(groupBy(reads, (read) => read.projectId))) {
// eslint-disable-next-line no-await-in-loop
rows.push(...(await $resolveProjectReads(projectId, projectReads)));
@@ -391,6 +396,8 @@ export const secretReadActivityServiceFactory = ({
}
};
+ const $getWindowStart = () => new Date(Date.now() - SECRET_READ_ACTIVITY_WINDOW_DAYS * DAY_IN_MS);
+
const init = () => {
const appCfg = getConfig();
cronJob.register({
@@ -400,22 +407,28 @@ export const secretReadActivityServiceFactory = ({
enabled: !appCfg.isSecondaryInstance,
handler: $consumeReadStream
});
+ // Hourly although a month's partition is needed once, so that an instance back from a long outage gets
+ // partitions for its reads within the hour.
cronJob.register({
name: CronJobName.SecretReadActivityPrune,
- pattern: "15 1 * * *",
- runHashTtlS: 3 * 24 * 60 * 60,
+ pattern: "15 * * * *",
+ runHashTtlS: 24 * 60 * 60,
enabled: !appCfg.isSecondaryInstance,
handler: async () => {
- const deleted = await secretReadActivityDAL.pruneOlderThan(
- new Date(Date.now() - SECRET_READ_RETENTION_DAYS * DAY_IN_MS)
+ const created = await secretReadActivityDAL.createMonthlyPartitions(
+ getUtcMonthStart(new Date()),
+ SECRET_READ_PARTITION_MONTHS_AHEAD + 1
+ );
+ // Nothing reads before the window, so a month is dropped once all of it has left the window.
+ const dropped = await secretReadActivityDAL.dropPartitionsEndingBefore($getWindowStart());
+ logger.info(
+ `cron[${CronJobName.SecretReadActivityPrune}]: Maintained partitions [created=${created}] [dropped=${dropped}]`
);
- logger.info(`cron[${CronJobName.SecretReadActivityPrune}]: pruned ${deleted} rows`);
}
});
};
- const $getReadActivity = async ({ projectId, windowDays, secret: requestedSecret }: TReadActivityScope) => {
- const windowStart = new Date(Date.now() - windowDays * DAY_IN_MS);
+ const $getReadActivity = async ({ projectId, windowStart, secret: requestedSecret }: TReadActivityScope) => {
const tree = await $loadProjectTree(projectId);
const { folders, folderById, locationOf, pathOf } = tree;
@@ -547,11 +560,11 @@ export const secretReadActivityServiceFactory = ({
) => {
const { permission } = await checkInsightsPermission(permissionService, licenseService, projectId, actor);
- const [activity] = await $getReadActivity({
- projectId,
- windowDays: SECRET_READ_ACTIVITY_WINDOW_DAYS,
- secret: { environment, secretPath, secretName }
- });
+ const windowStart = $getWindowStart();
+ const [[activity], accessChangedAtByActorId] = await Promise.all([
+ $getReadActivity({ projectId, windowStart, secret: { environment, secretPath, secretName } }),
+ secretReadActivityDAL.findAccessChangedAt(projectId)
+ ]);
if (!activity) {
throw new NotFoundError({
message: `Secret with name '${secretName}' not found at path '${secretPath}' in environment '${environment}'`
@@ -564,13 +577,23 @@ export const secretReadActivityServiceFactory = ({
secretTags: activity.secret.tags?.map((tag) => tag.slug)
});
- return { trackedSince: activity.trackedSince, reads: [...activity.reads.values()] };
+ return {
+ trackedSince: activity.trackedSince,
+ windowStart,
+ reads: [...activity.reads.values()],
+ accessChanges: [...accessChangedAtByActorId].map(([actorId, accessChangedAt]) => ({ actorId, accessChangedAt }))
+ };
};
- // Everyone who can read a secret's value but has not read it, ranked by the secrets with the most of them.
+ // Everyone who can read a secret's value but has not read it, ranked by the secrets with the most of them. Those
+ // whose secret or access is newer than the window come last, marked as still being tracked.
const $computeUnusedSecretAccess = async (projectId: string, orgId: string) => {
- const activities = await $getReadActivity({ projectId, windowDays: SECRET_READ_ACTIVITY_WINDOW_DAYS });
- const { userPermissions, identityPermissions } = await permissionService.getProjectPermissions(projectId, orgId);
+ const windowStart = $getWindowStart();
+ const [activities, { userPermissions, identityPermissions }, accessChangedAtByActorId] = await Promise.all([
+ $getReadActivity({ projectId, windowStart }),
+ permissionService.getProjectPermissions(projectId, orgId),
+ secretReadActivityDAL.findAccessChangedAt(projectId)
+ ]);
// Principals with identical rules always get the same answer, so each distinct rule set is checked once.
const principalGroups = Object.values(
groupBy(
@@ -610,14 +633,19 @@ export const secretReadActivityServiceFactory = ({
principalType: principal.actorType,
principalId: principal.id,
principalName: principal.name,
- readStatus: SecretReadStatus.NoReads,
lastReadAt: null,
- trackedSince
+ ...getUnreadStatus(trackedSince, accessChangedAtByActorId.get(principal.id), windowStart)
};
})
);
}
- return rowsBySecret.sort((a, b) => b.length - a.length).flat();
+ const countNoReads = (rows: TUnusedSecretAccess[]) =>
+ rows.filter((row) => row.readStatus === SecretReadStatus.NoReads).length;
+ const ranked = rowsBySecret.sort((a, b) => countNoReads(b) - countNoReads(a)).flat();
+ return [
+ ...ranked.filter((row) => row.readStatus === SecretReadStatus.NoReads),
+ ...ranked.filter((row) => row.readStatus === SecretReadStatus.Tracking)
+ ];
};
// The whole project is computed once and cached, so paging through it is cheap.
@@ -631,7 +659,7 @@ export const secretReadActivityServiceFactory = ({
const unusedAccess = await withCache({
keyStore,
key: cacheKey,
- ttlSeconds: KeyStoreTtls.InsightsCacheInSeconds,
+ ttlSeconds: UNUSED_SECRET_ACCESS_CACHE_TTL_SECONDS,
fetcher: () => $computeUnusedSecretAccess(projectId, actor.orgId),
reviver: (parsed) => {
reviveDates(parsed, "lastReadAt", "trackedSince");
diff --git a/backend/src/ee/services/secret-read-activity/secret-read-activity-types.ts b/backend/src/ee/services/secret-read-activity/secret-read-activity-types.ts
index 7f7704f66d..e48ae14e14 100644
--- a/backend/src/ee/services/secret-read-activity/secret-read-activity-types.ts
+++ b/backend/src/ee/services/secret-read-activity/secret-read-activity-types.ts
@@ -1,9 +1,17 @@
+import { TSecretReadActivity } from "@app/db/schemas";
import { OrderByDirection } from "@app/lib/types";
import { ActorType } from "@app/services/auth/auth-type";
+// A reader's last read of an object. The table also keys it by month (its partition key), which the DAL adds
+// on write and folds away on read.
+export type TSecretLastRead = Omit<TSecretReadActivity, "readMonth">;
+
+// Tracking: not read yet, but the secret or the reader's access to it is newer than the window, so it is too
+// early to call the access unused.
export enum SecretReadStatus {
Read = "read",
- NoReads = "no-reads"
+ NoReads = "no-reads",
+ Tracking = "tracking"
}
// A bulk read that returned whole folders is recorded against them. Anything narrower, and every single read,
@@ -89,7 +97,7 @@ export type TGetSecretReadActivityDTO = {
// Without a secret, the whole project is in scope.
export type TReadActivityScope = {
projectId: string;
- windowDays: number;
+ windowStart: Date;
secret?: Omit<TGetSecretReadActivityDTO, "projectId">;
};
diff --git a/frontend/src/components/secrets/SecretReadStatusBadge.tsx b/frontend/src/components/secrets/SecretReadStatusBadge.tsx
index 5d38d02489..91720b4c38 100644
--- a/frontend/src/components/secrets/SecretReadStatusBadge.tsx
+++ b/frontend/src/components/secrets/SecretReadStatusBadge.tsx
@@ -1,11 +1,14 @@
-import { differenceInCalendarDays } from "date-fns";
+import { addDays, differenceInCalendarDays, format } from "date-fns";
-import { Badge } from "@app/components/v3";
+import { Badge, Tooltip, TooltipContent, TooltipTrigger } from "@app/components/v3";
import { SecretReadStatus } from "@app/hooks/api/secrets/types";
+const SECRET_READ_WINDOW_DAYS = 30;
+
type Props = {
status: SecretReadStatus;
lastReadAt?: string | null;
+ trackedSince?: string;
};
// Only a reader's first read of the day is recorded, so the time of day would understate later reads.
@@ -16,9 +19,23 @@ const formatReadDay = (lastReadAt: string) => {
return `${days} days ago`;
};
-export const SecretReadStatusBadge = ({ status, lastReadAt }: Props) => {
+export const SecretReadStatusBadge = ({ status, lastReadAt, trackedSince }: Props) => {
if (status === SecretReadStatus.Read && lastReadAt) {
return <span>{formatReadDay(lastReadAt)}</span>;
}
+ if (status === SecretReadStatus.Tracking && trackedSince) {
+ return (
+ <Tooltip>
+ <TooltipTrigger asChild>
+ <Badge variant="neutral">Tracking</Badge>
+ </TooltipTrigger>
+ <TooltipContent>
+ Not read since {format(new Date(trackedSince), "PP")}, when this secret or access was
+ created or last changed. Shown as unused if still unread on{" "}
+ {format(addDays(new Date(trackedSince), SECRET_READ_WINDOW_DAYS), "PP")}.
+ </TooltipContent>
+ </Tooltip>
+ );
+ }
return <Badge variant="danger">No reads</Badge>;
};
diff --git a/frontend/src/hooks/api/secretInsights/queries.tsx b/frontend/src/hooks/api/secretInsights/queries.tsx
index f5319bf34d..7d74c9827d 100644
--- a/frontend/src/hooks/api/secretInsights/queries.tsx
+++ b/frontend/src/hooks/api/secretInsights/queries.tsx
@@ -70,6 +70,7 @@ export const secretInsightsKeys = {
};
const INSIGHTS_STALE_TIME = 5 * 60 * 1000; // 5 minutes
+const UNUSED_SECRET_ACCESS_REFRESH_INTERVAL = 30 * 1000;
export const useGetCalendarInsights = (
params: TGetCalendarInsightsDTO,
@@ -146,7 +147,8 @@ export const useGetUnusedSecretAccess = (
const remainingTtl = Number(res.headers["x-cache-ttl"] ?? -1);
return { ...res.data, remainingTtl };
},
- staleTime: INSIGHTS_STALE_TIME,
+ staleTime: UNUSED_SECRET_ACCESS_REFRESH_INTERVAL,
+ refetchInterval: UNUSED_SECRET_ACCESS_REFRESH_INTERVAL,
...options
});
};
diff --git a/frontend/src/hooks/api/secrets/types.ts b/frontend/src/hooks/api/secrets/types.ts
index c9561cc3c4..a8b18c525a 100644
--- a/frontend/src/hooks/api/secrets/types.ts
+++ b/frontend/src/hooks/api/secrets/types.ts
@@ -164,17 +164,23 @@ export type TGetSecretReadActivityDTO = Omit<TGetSecretAccessListDTO, "includeAl
export enum SecretReadStatus {
Read = "read",
- NoReads = "no-reads"
+ NoReads = "no-reads",
+ Tracking = "tracking"
}
export type TSecretReadActivity = {
trackedSince: string;
+ windowStart: string;
reads: {
actorType: "user" | "identity";
actorId: string;
status: SecretReadStatus;
lastReadAt: string;
}[];
+ accessChanges: {
+ actorId: string;
+ accessChangedAt: string;
+ }[];
};
export type TCreateSecretsV3DTO = {
diff --git a/frontend/src/pages/secret-manager/InsightsPage/components/UnusedSecretAccessCard.tsx b/frontend/src/pages/secret-manager/InsightsPage/components/UnusedSecretAccessCard.tsx
index 0d7df8f7b9..23fbf858a7 100644
--- a/frontend/src/pages/secret-manager/InsightsPage/components/UnusedSecretAccessCard.tsx
+++ b/frontend/src/pages/secret-manager/InsightsPage/components/UnusedSecretAccessCard.tsx
@@ -30,6 +30,15 @@ import { usePagination, useResetPageHelper } from "@app/hooks";
import { OrderByDirection } from "@app/hooks/api/generic/types";
import { UnusedSecretAccessOrderBy, useGetUnusedSecretAccess } from "@app/hooks/api/secretInsights";
+const formatRemainingTtl = (seconds: number) => {
+ if (seconds < 60) {
+ const remainingSeconds = Math.max(1, seconds);
+ return `${remainingSeconds} ${remainingSeconds === 1 ? "second" : "seconds"}`;
+ }
+ const minutes = Math.ceil(seconds / 60);
+ return `${minutes} ${minutes === 1 ? "minute" : "minutes"}`;
+};
+
export const UnusedSecretAccessCard = () => {
const { currentProject, projectId } = useProject();
const { subscription } = useSubscription();
@@ -54,9 +63,9 @@ export const UnusedSecretAccessCard = () => {
const totalCount = data?.totalCount ?? 0;
useResetPageHelper({ totalCount, offset, setPage });
- const remainingMinutes =
+ const updatesIn =
data?.remainingTtl != null && data.remainingTtl >= 0
- ? Math.max(1, Math.ceil(data.remainingTtl / 60))
+ ? formatRemainingTtl(data.remainingTtl)
: null;
const environmentNames = new Map(currentProject.environments.map((env) => [env.slug, env.name]));
@@ -85,9 +94,9 @@ export const UnusedSecretAccessCard = () => {
<CardHeader>
<CardTitle>
Unused Secret Access
- {remainingMinutes != null && (
+ {updatesIn && (
<Badge variant="neutral" className="ml-2 font-normal">
- Updates in {remainingMinutes} {remainingMinutes === 1 ? "minute" : "minutes"}
+ Updates in {updatesIn}
</Badge>
)}
</CardTitle>
@@ -177,6 +186,7 @@ export const UnusedSecretAccessCard = () => {
<SecretReadStatusBadge
status={row.readStatus}
lastReadAt={row.lastReadAt}
+ trackedSince={row.trackedSince}
/>
</TableCell>
<TableCell>{format(new Date(row.trackedSince), "PP")}</TableCell>
diff --git a/frontend/src/pages/secret-manager/OverviewPage/components/SecretTableRow/SecretAccessInsights.tsx b/frontend/src/pages/secret-manager/OverviewPage/components/SecretTableRow/SecretAccessInsights.tsx
index 5426d98d13..9bdd7336c0 100644
--- a/frontend/src/pages/secret-manager/OverviewPage/components/SecretTableRow/SecretAccessInsights.tsx
+++ b/frontend/src/pages/secret-manager/OverviewPage/components/SecretTableRow/SecretAccessInsights.tsx
@@ -484,6 +484,13 @@ export function SecretAccessInsights({ secretKey, environment, secretPath }: Pro
() => new Map(readActivity?.reads.map((read) => [`${read.actorType}:${read.actorId}`, read])),
[readActivity]
);
+ const accessChangedAt = useMemo(
+ () =>
+ new Map(
+ readActivity?.accessChanges.map((change) => [change.actorId, change.accessChangedAt])
+ ),
+ [readActivity]
+ );
let readActivityDescription = "Loading read activity...";
if (readActivityError) {
@@ -501,10 +508,23 @@ export function SecretAccessInsights({ secretKey, environment, secretPath }: Pro
if (!readActivity) return null;
const read = lastReads.get(`${row.type}:${row.entry.id}`);
+ if (read) return <SecretReadStatusBadge status={read.status} lastReadAt={read.lastReadAt} />;
+
+ // Mirrors the Insights card: unread access is only called unused once both the secret and the access
+ // have existed for the whole window.
+ const changedAt = accessChangedAt.get(row.entry.id);
+ const trackedSince =
+ changedAt && new Date(changedAt) > new Date(readActivity.trackedSince)
+ ? changedAt
+ : readActivity.trackedSince;
return (
<SecretReadStatusBadge
- status={read?.status ?? SecretReadStatus.NoReads}
- lastReadAt={read?.lastReadAt}
+ status={
+ new Date(trackedSince) > new Date(readActivity.windowStart)
+ ? SecretReadStatus.Tracking
+ : SecretReadStatus.NoReads
+ }
+ trackedSince={trackedSince}
/>
);
};