Skip to content

7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

TypeScript

Prune the quota observation log in 5k-row chunks (#1124)

Tashfeen4a8f0950fe5c
diff --git a/server/src/__tests__/services/provider-quota.test.ts b/server/src/__tests__/services/provider-quota.test.ts
index c8be34ed..147e8f71 100644
--- a/server/src/__tests__/services/provider-quota.test.ts
+++ b/server/src/__tests__/services/provider-quota.test.ts
@@ -135,9 +135,9 @@ describe('provider-quota: record + read round-trip', () => {
// A zero budget allows exactly one chunk before the check trips.
const first = pruneQuotaObservations(db, now, 0);
expect(first.done).toBe(false);
- expect(first.deleted).toBe(20_000);
+ expect(first.deleted).toBe(5_000);
const rest = pruneQuotaObservations(db, now, 60_000);
- expect(rest).toEqual({ deleted: 25_000, done: true });
+ expect(rest).toEqual({ deleted: 40_000, done: true });
});
it('records an observation and surfaces it via getQuotaStateForKeys', () => {
diff --git a/server/src/services/request-retention.ts b/server/src/services/request-retention.ts
index 2829fc10..d767bb6e 100644
--- a/server/src/services/request-retention.ts
+++ b/server/src/services/request-retention.ts
@@ -84,8 +84,9 @@ export function getQuotaObservationRetentionConfig(): QuotaObservationRetentionC
// thousands of rows; deleting them in one statement held the loop for ~4.5s on
// a 470k-row log. Work is chunked under a wall-clock budget instead, and a
// sweep that runs out of budget is resumed on the next 60s tick rather than
-// tomorrow.
-const QUOTA_OBSERVATIONS_PRUNE_CHUNK = 20_000;
+// tomorrow. The chunk is the smallest unit of stall: 20k rows took ~870ms on a
+// 2-vCPU box under load, 5k keeps each tick near the budget.
+const QUOTA_OBSERVATIONS_PRUNE_CHUNK = 5_000;
const QUOTA_OBSERVATIONS_PRUNE_BUDGET_MS = 250;
/**