Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
fix(np08): raisable HCC staleness ceiling so HCC rollout doesn't drop MCP fleet (#425)
Alfredo Lopez9497625d8c62
diff --git a/deploy/overlays/minikube/configmaps/mcp-host-config.yaml b/deploy/overlays/minikube/configmaps/mcp-host-config.yamlindex e9f55075b..ebb02bb04 100644--- a/deploy/overlays/minikube/configmaps/mcp-host-config.yaml+++ b/deploy/overlays/minikube/configmaps/mcp-host-config.yaml@@ -11,6 +11,10 @@ data:CLERUM_CONTEXT_MAPPER_URL: 'http://host-context-controller-api-gateway.control-plane.svc.cluster.local:8081'CLERUM_CONTEXT_MAPPER_POLL_INTERVAL: '30000'MCP_PROXY_ENABLED: 'false'+ # e2e lane pin: 60s keeps the revoke-on-staleness scenario fast and+ # deterministic. Production defaults to 180s (bounded 600s) in+ # mcp-host/src/config.ts so a normal HCC rollout (~77s in clerum-dev) does not+ # tear down the MCP fleet. See issue #425.HCC_AUTHORITY_MAX_STALENESS_MS: '60000'MCP_PROXY_URL: 'http://mcp-proxy.mcp-server.svc.cluster.local:8083'CLERUM_AGENT_TASK_DELAY: '3'diff --git a/docs/architecture/overview.md b/docs/architecture/overview.mdindex 00cb1f8ab..7ea628254 100644--- a/docs/architecture/overview.md+++ b/docs/architecture/overview.md@@ -766,10 +766,13 @@ HTTP 410; the remaining global v1 inventory is metadata-only transitionalsurface for the separately reviewed mcp-proxy PR2 and is not used by mcp-host.Polling interval: 30 seconds (configurable via `CLERUM_CONTEXT_MAPPER_POLL_INTERVAL`).-In cluster mode it must remain strictly below the 60-second-`HCC_AUTHORITY_MAX_STALENESS_MS` ceiling; mcp-host rejects an invalid relation at-startup rather than allowing the fleet to flap between successful polls and-authority revocation.+In cluster mode it must remain strictly below `HCC_AUTHORITY_MAX_STALENESS_MS`+(default 180 s, operator-tunable up to a 600 s hard ceiling); mcp-host rejects an+invalid relation at startup rather than allowing the fleet to flap between+successful polls and authority revocation. The staleness window governs only the+`unavailable` failure class (HCC unreachable / 5xx): it is sized to absorb a+normal HCC `Recreate` rollout without tearing down the MCP fleet. Identity/401+revocations are immediate and are not gated by this window (issue #425).On each poll, mcp-host compares new server list with previous state and:@@ -822,7 +825,7 @@ The HTTP response is held open until the agent finishes processing. The response| `CLERUM_SERVER_PORT` | `8080` | HTTP server port || `CLERUM_CONTEXT_MAPPER_URL` | auto | Context-mapper service URL || `CLERUM_CONTEXT_MAPPER_POLL_INTERVAL` | `30000` | Poll interval (ms) |-| `HCC_AUTHORITY_MAX_STALENESS_MS` | `60000` | Maximum retained HCC authority (ms); must be greater than the poll interval in cluster mode |+| `HCC_AUTHORITY_MAX_STALENESS_MS` | `180000` | Max retained HCC authority while HCC is unreachable (ms); operator-tunable, hard-capped at 600000; must be greater than the poll interval in cluster mode (issue #425) || `CLERUM_MODEL_PROVIDER` | auto-detected | `"openai"`, `"claude"`, `"zai"`, or `"bailian"` || `CLERUM_MODEL_NAME` | provider default | Specific model name || `OPENAI_API_KEY` | - | OpenAI key (dev mode) |diff --git a/mcp-host/package-lock.json b/mcp-host/package-lock.jsonindex 908000646..96f47cfe4 100644--- a/mcp-host/package-lock.json+++ b/mcp-host/package-lock.json@@ -1,12 +1,12 @@{"name": "clerum-mcp-host",- "version": "0.1.275",+ "version": "0.1.276","lockfileVersion": 3,"requires": true,"packages": {"": {"name": "clerum-mcp-host",- "version": "0.1.275",+ "version": "0.1.276","license": "SEE LICENSE IN LICENSE","dependencies": {"@anthropic-ai/sdk": "^0.32.0",diff --git a/mcp-host/package.json b/mcp-host/package.jsonindex 9e8a37131..feb6bd6ea 100644--- a/mcp-host/package.json+++ b/mcp-host/package.json@@ -1,6 +1,6 @@{"name": "clerum-mcp-host",- "version": "0.1.275",+ "version": "0.1.276","description": "MCP Host - reads Host CRD and provides LLM access via OpenAI/Claude","main": "dist/main.js","engines": {diff --git a/mcp-host/src/config.hccAuthority.test.ts b/mcp-host/src/config.hccAuthority.test.tsindex 0296814d9..eac421207 100644--- a/mcp-host/src/config.hccAuthority.test.ts+++ b/mcp-host/src/config.hccAuthority.test.ts@@ -23,20 +23,24 @@ afterEach(() => {})describe('HCC_AUTHORITY_MAX_STALENESS_MS', () => {- it('defaults to the 60-second authority ceiling', async () => {- await expect(loadAuthorityMaxStaleness(undefined)).resolves.toBe(60_000)+ it('defaults to the 180-second rollout-tolerant window', async () => {+ await expect(loadAuthorityMaxStaleness(undefined)).resolves.toBe(180_000)})it('allows an operator to fail closed sooner', async () => {await expect(loadAuthorityMaxStaleness('15000', '5000')).resolves.toBe(15_000)})- it('cannot be configured beyond the 60-second authority ceiling', async () => {- await expect(loadAuthorityMaxStaleness('120000')).resolves.toBe(60_000)+ it('allows an operator to raise the window for a slower-starting cluster (issue #425)', async () => {+ await expect(loadAuthorityMaxStaleness('300000')).resolves.toBe(300_000)})- it('uses the safe ceiling for an invalid value', async () => {- await expect(loadAuthorityMaxStaleness('not-a-duration')).resolves.toBe(60_000)+ it('cannot be configured beyond the 600-second hard ceiling', async () => {+ await expect(loadAuthorityMaxStaleness('700000')).resolves.toBe(600_000)+ })++ it('uses the default window for an invalid value', async () => {+ await expect(loadAuthorityMaxStaleness('not-a-duration')).resolves.toBe(180_000)})it('rejects a poll interval that reaches the authority ceiling', async () => {@@ -53,8 +57,8 @@ describe('HCC_AUTHORITY_MAX_STALENESS_MS', () => {)})- it('accepts the default 30-second poll inside the 60-second ceiling', async () => {+ it('accepts the default 30-second poll inside the default staleness window', async () => {const { validateHccAuthorityTiming } = await import('./config')- expect(() => validateHccAuthorityTiming(30_000, 60_000)).not.toThrow()+ expect(() => validateHccAuthorityTiming(30_000, 180_000)).not.toThrow()})})diff --git a/mcp-host/src/config.ts b/mcp-host/src/config.tsindex 71b4db83c..27fe2480d 100644--- a/mcp-host/src/config.ts+++ b/mcp-host/src/config.ts@@ -579,9 +579,27 @@ const contextMapperPollInterval = parseInt(getEnv('CLERUM_CONTEXT_MAPPER_POLL_INTERVAL', '30000')!,10)+// Fail-closed window for an UNREACHABLE HCC (5xx / transport failures only).+// It is bounded so a revoked-but-unconfirmable authority cannot linger forever,+// yet generous enough to survive a normal HCC `Recreate` rollout (measured ~77s+// in clerum-dev, with a startupProbe budget of up to 120s) without tearing down+// the whole MCP fleet and aborting in-flight tool calls.+//+// Crucially, this window ONLY governs the `unavailable` failure class. Identity /+// authorization failures (401/403 — Host UID change, revoked grant) revoke+// immediately on the next reachable call and are NOT gated by this value, and a+// grant change while HCC is healthy is applied by the next successful poll. So+// raising this ceiling does not delay any real authority revocation and does not+// widen the NP-08 credential-disclosure surface (which lives entirely in HCC's+// per-request authorization). See issue #425.+//+// Operators may lower it, or raise it up to HCC_AUTHORITY_MAX_STALENESS_CEILING_MS+// for slower-starting clusters. The minikube e2e lane pins 60000 explicitly so the+// revoke-on-staleness scenario stays fast and deterministic.+const HCC_AUTHORITY_MAX_STALENESS_CEILING_MS = 600_000 // 10 min hard upper boundconst hccAuthorityMaxStalenessMs = Math.min(- getEnvNumber('HCC_AUTHORITY_MAX_STALENESS_MS', 60_000),- 60_000+ getEnvNumber('HCC_AUTHORITY_MAX_STALENESS_MS', 180_000),+ HCC_AUTHORITY_MAX_STALENESS_CEILING_MS)// Dev mode does not retain a Kubernetes-authoritative MCP fleet, so thediff --git a/scripts/tests/test-np08-render-contract.sh b/scripts/tests/test-np08-render-contract.shindex 433da1cc0..a14487509 100755--- a/scripts/tests/test-np08-render-contract.sh+++ b/scripts/tests/test-np08-render-contract.sh@@ -30,7 +30,11 @@ for rendered in "$@"; dohost_config = require_resource.call("ConfigMap", "mcp-host-config", "mcp-host")abort("mcp-host must keep direct MCP transport") unless host_config.dig("data", "MCP_PROXY_ENABLED") == "false"- abort("HCC authority staleness must stay bounded at 60 seconds") unless host_config.dig("data", "HCC_AUTHORITY_MAX_STALENESS_MS") == "60000"+ # The minikube e2e lane pins staleness to 60s so the revoke-on-staleness+ # scenario is fast and deterministic. Production defaults to 180s (bounded at+ # 600s) in mcp-host/src/config.ts so a normal HCC rollout does not tear down+ # the MCP fleet — see issue #425. This assertion guards the e2e-lane pin.+ abort("HCC authority staleness must stay pinned to 60s on the minikube e2e lane") unless host_config.dig("data", "HCC_AUTHORITY_MAX_STALENESS_MS") == "60000"hcc = require_resource.call("Deployment", "host-context-controller", "control-plane")hcc_container = Array(hcc.dig("spec", "template", "spec", "containers")).find do |container|