Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

docs: external GFS budgets, agent limits and the real sliding bound

Alfredo Lopezf2b49271d868
diff --git a/control-api/package-lock.json b/control-api/package-lock.json
index 7c2a1ed14..46b5a4117 100644
--- a/control-api/package-lock.json
+++ b/control-api/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "clerum-control-api",
- "version": "0.1.612",
+ "version": "0.1.613",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "clerum-control-api",
- "version": "0.1.612",
+ "version": "0.1.613",
"dependencies": {
"@clerum/display-field": "file:../packages/display-field",
"@clerum/grok-provider-attempt-contract": "file:../packages/grok-provider-attempt-contract",
diff --git a/control-api/package.json b/control-api/package.json
index 7480cba97..65faa571e 100644
--- a/control-api/package.json
+++ b/control-api/package.json
@@ -1,6 +1,6 @@
{
"name": "clerum-control-api",
- "version": "0.1.612",
+ "version": "0.1.613",
"description": "Internal Clerum control plane REST API + UI for managing CRDs and secrets",
"main": "dist/main.js",
"engines": {
diff --git a/control-api/src/config.ts b/control-api/src/config.ts
index 9fca7bbeb..8ed852a22 100644
--- a/control-api/src/config.ts
+++ b/control-api/src/config.ts
@@ -1031,6 +1031,12 @@ export const config: Config = {
// the nominal sum was 1920. No all-read per-actor cap exists; the test
// suite states the sum instead of a boot invariant.
//
+ // A budget L is per calendar minute in Postgres. The per-route express
+ // backstop has its own 60 s window, started by a key's first request, so
+ // the two windows are not aligned: one actor can pass about 2L - 1 requests
+ // of a class in any sliding 60 s, per control-api replica (959 at 480).
+ // The stress suite's E2 case pins that bound.
+ //
// Peaks below are the busiest sliding 60 s of the 30-day funnel log, prod /
// dev.
//
diff --git a/control-api/src/routes/external/gfs.ts b/control-api/src/routes/external/gfs.ts
index 4deabc3e5..dc3a04065 100644
--- a/control-api/src/routes/external/gfs.ts
+++ b/control-api/src/routes/external/gfs.ts
@@ -508,8 +508,9 @@ export function createExternalGfsRouter(): Router {
onBackendUnavailable: 'closed',
})
- // Keep the existing mutation bucket identity and 30/min budget for
- // compatibility with telemetry and abuse controls; only read/list calls
+ // Keep the existing mutation bucket identity for compatibility with
+ // telemetry and abuse controls; its budget is the shared mutation budget
+ // (externalGfsOperationRlPerMin, 90/min by default). Only read/list calls
// move to the separate read bucket above.
const externalGrantsMutationRateLimit = rateLimitMiddleware({
bucketType: 'gfs_grants_external',
diff --git a/docs/architecture/diagrams/gfs-desktop-operator-parity.png b/docs/architecture/diagrams/gfs-desktop-operator-parity.png
index 555ed6fd3..ec5be9254 100644
Binary files a/docs/architecture/diagrams/gfs-desktop-operator-parity.png and b/docs/architecture/diagrams/gfs-desktop-operator-parity.png differ
diff --git a/docs/architecture/diagrams/gfs-desktop-operator-parity.svg b/docs/architecture/diagrams/gfs-desktop-operator-parity.svg
index 3fb80a06a..c9c142ce5 100644
--- a/docs/architecture/diagrams/gfs-desktop-operator-parity.svg
+++ b/docs/architecture/diagrams/gfs-desktop-operator-parity.svg
@@ -62,8 +62,8 @@
<text x="730" y="143" class="node-title">Control API</text>
<text x="730" y="168" class="node-copy">authority resolver + token broker</text>
<text x="730" y="195" class="small">10 / min token per Desktop user</text>
- <text x="730" y="216" class="small">120 / min reads per session/IP/actor</text>
- <text x="730" y="237" class="small">30 / min operations per session/IP/actor</text>
+ <text x="730" y="216" class="small">reads 480 resource/proxy, 120 grants/shares</text>
+ <text x="730" y="237" class="small">90 / min operations; all per session/IP/actor</text>
<text x="730" y="264" class="small">active user + admin + link generation</text>
<text x="730" y="283" class="small">signed principal + request correlation</text>
diff --git a/docs/architecture/gfs-desktop-operator-parity.md b/docs/architecture/gfs-desktop-operator-parity.md
index 8aa7e3f46..824ccc386 100644
--- a/docs/architecture/gfs-desktop-operator-parity.md
+++ b/docs/architecture/gfs-desktop-operator-parity.md
@@ -52,10 +52,45 @@ External REST applies the GFS edge backstop in this order:
Control API then applies distributed, authority-aware limits:
- token mint: 10/min per Desktop user;
-- resource/proxy reads: 120/min per session, trusted client IP, and resolved
- actor class;
-- mutations, grants, and shares: 30/min per session, trusted client IP, and
- resolved actor class.
+- per-class budgets, each applied per session, per (class, trusted client IP)
+ and per resolved actor:
+
+ | Class | Routes | Default | Env var |
+ | ------------- | ---------------------------------------------------------------------- | ------- | --------------------------------------------------- |
+ | `resource` | resolve, capabilities, upload status, resources, affordances, children | 480/min | `CONTROL_API_EXTERNAL_GFS_RESOURCE_READ_RL_PER_MIN` |
+ | `proxy-read` | `GET proxy/:rid` | 480/min | `CONTROL_API_EXTERNAL_GFS_PROXY_READ_RL_PER_MIN` |
+ | `grants-read` | `GET grants` | 120/min | `CONTROL_API_EXTERNAL_GFS_GRANTS_READ_RL_PER_MIN` |
+ | `shares-read` | `GET shares` | 120/min | `CONTROL_API_EXTERNAL_GFS_SHARES_READ_RL_PER_MIN` |
+ | mutations | every write, grant and share mutation | 90/min | `CONTROL_API_EXTERNAL_GFS_OPERATION_RL_PER_MIN` |
+
+- all classes together: 1200/min per trusted client IP. The four read
+ defaults sum to the same 1200.
+
+Each value is overridable up to a compiled ceiling. Control API refuses to
+boot when the mutation budget exceeds the resource, grants or shares read
+budget, or when any read budget exceeds the per-IP bucket. The proxy has no
+mutation counterpart and is checked only against the per-IP bucket.
+
+These Postgres buckets count calendar minutes. Behind them, each route has a
+per-replica in-memory backstop whose 60 s window starts at the first request
+of a key. The two windows are not aligned, so one actor can get up to about
+`2L − 1` requests of one class through in any sliding 60 s, per Control API
+replica (959 at the 480 resource default), not `L`.
+
+When the rate-limit Postgres backend cannot answer, the external GFS routes
+fail closed: `503 { "error": "gfs_rate_limit_unavailable" }` with
+`Retry-After: 2`. The limiter uses its own pool (`RATE_LIMIT_POOL_MAX`,
+default 6), so a saturated core pool does not take it down. The limit is not
+skipped.
+
+GFSC applies a second, per-replica limit to agent (host principal) calls:
+`GFS_AGENT_READ_RL_PER_MIN_PER_REPLICA` (default 300) and
+`GFS_AGENT_WRITE_RL_PER_MIN_PER_REPLICA` (default 120). Host Context
+Controller writes both into every gfsc pod from
+`CONTEXT_MAPPER_GFSC_AGENT_READ_RL_PER_MIN_PER_REPLICA` and
+`CONTEXT_MAPPER_GFSC_AGENT_WRITE_RL_PER_MIN_PER_REPLICA`. Changing either
+value, or the gfsc image, changes the gfsc pod template and restarts the gfsc
+fleet once.
The buckets remain separate. A request rejected by a narrower boundary cannot
consume a broader bucket. IPv6 canonical-key normalization and cross-layer 429