Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

fix(hcc): cover class D on the resync tick and unblock the census (#478)

Alfredo Lopez6e9db75280bd
diff --git a/host-context-controller/README.md b/host-context-controller/README.md
index e9e958c6a..7bf380e66 100644
--- a/host-context-controller/README.md
+++ b/host-context-controller/README.md
@@ -439,16 +439,16 @@ Read by `src/gfsConfig.ts`. Invalid values **throw at startup** rather than sile
### NetworkPolicy
-| Variable | Default | Description |
-| ------------------------------------------------- | ----------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `CONTEXT_MAPPER_RUNTIME_NAMESPACES` | `mcp-server,mcp-host,sandbox-recipes,rpc-proxy` | Namespaces that get L0 deny-all + L1 infrastructure policies. Manifest overrides to `mcp-server,sandbox-recipes,rpc-proxy,sandbox-ui` — note it **drops `mcp-host`**, whose policies are static (`deploy/base/mcp-host/networkpolicies.yaml`), because an HCC deny-all would block mcp-host from reaching LLM APIs. |
-| `CONTEXT_MAPPER_MINIMAL_INFRA_NAMESPACES` | `` (empty) | Subset of runtime namespaces that get **only** deny-all + DNS egress — no HCC-API or K8s-API egress. Manifest sets `sandbox-ui`. |
-| `CONTEXT_MAPPER_K8S_API_CIDRS` | _(unset → `[]`)_ | K8s API-server CIDRs for `allow-k8s-api-egress-*`. Empty falls back to `KUBERNETES_SERVICE_HOST`. **Validated fail-closed at module load**: a malformed entry, or one broader than `/24` (IPv4) or `/120` (IPv6), **crashes startup** rather than programming a permissive policy. |
-| `CONTEXT_MAPPER_NODELOCAL_DNS_CIDR` | `` (empty) | DNS infrastructure CIDR for NodeLocal DNSCache / kube-dns. Must be exactly one IPv4 `/32`; anything else throws at startup. |
-| `CONTEXT_MAPPER_NETPOL_RESYNC_SEC` | `0` | Periodic NetworkPolicy coordinated convergence (seconds). Re-enters the single-flight pass; `0` disables the interval (merge default). Startup still converges once. Default `0` logs a `console.warn` on every boot (same pattern as Host/SFS/GFS/LlmHook) — expected; it names the self-heal gap. **Enable is a separate ops decision** (issue #478): before any non-zero value, add a freshness gate **or** use a controller-runtime 10h-scale period. Do not treat `300` as the enable value while passes last minutes. |
-| `CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP` | `10` | Absolute orphan-delete cap for a namespace-wide NetworkPolicy sweep. Candidates **strictly above** this count refuse **deletes** and increment `clerum_hcc_netpol_orphan_sweep_capped_total`; the pass **still certifies** (#478: alert and do not delete). Governs every `fullReconcile` (startup, event-driven, timer). Absolute is checked first, so at a fleet ≥ 50 with these defaults the percent rule never binds. Census: `scripts/ops/hcc-netpol-orphan-census.sh` — `live_cap_would_trip` is Deployment env only; `controller_cap_would_trip` applies compiled defaults when unset. |
-| `CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT` | `20` | Percent orphan-delete cap against the listed managed fleet. Strict `>`. Inert on a tiny inventory where `percent * listed < 1`. Same every-`fullReconcile` scope as the absolute cap; only binds when the fleet is small enough that 20% is below the absolute cap of 10. |
-| `KUBERNETES_SERVICE_HOST` | _(injected by Kubernetes)_ | Not set by the operator — the `/32` fallback source when `CONTEXT_MAPPER_K8S_API_CIDRS` is empty. |
+| Variable | Default | Description |
+| ------------------------------------------------- | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `CONTEXT_MAPPER_RUNTIME_NAMESPACES` | `mcp-server,mcp-host,sandbox-recipes,rpc-proxy` | Namespaces that get L0 deny-all + L1 infrastructure policies. Manifest overrides to `mcp-server,sandbox-recipes,rpc-proxy,sandbox-ui` — note it **drops `mcp-host`**, whose policies are static (`deploy/base/mcp-host/networkpolicies.yaml`), because an HCC deny-all would block mcp-host from reaching LLM APIs. |
+| `CONTEXT_MAPPER_MINIMAL_INFRA_NAMESPACES` | `` (empty) | Subset of runtime namespaces that get **only** deny-all + DNS egress — no HCC-API or K8s-API egress. Manifest sets `sandbox-ui`. |
+| `CONTEXT_MAPPER_K8S_API_CIDRS` | _(unset → `[]`)_ | K8s API-server CIDRs for `allow-k8s-api-egress-*`. Empty falls back to `KUBERNETES_SERVICE_HOST`. **Validated fail-closed at module load**: a malformed entry, or one broader than `/24` (IPv4) or `/120` (IPv6), **crashes startup** rather than programming a permissive policy. |
+| `CONTEXT_MAPPER_NODELOCAL_DNS_CIDR` | `` (empty) | DNS infrastructure CIDR for NodeLocal DNSCache / kube-dns. Must be exactly one IPv4 `/32`; anything else throws at startup. |
+| `CONTEXT_MAPPER_NETPOL_RESYNC_SEC` | `0` | Periodic NetworkPolicy coordinated convergence (seconds). Re-enters the single-flight pass with `ensureDefaults: true` so a tick covers orphan sweep (class A) and L0/L1 defaults (class D). `0` disables the interval (merge default). Startup still converges once; `start()` already applied defaults as a bootstrap barrier. Default `0` logs a `console.warn` on every boot (same pattern as Host/SFS/GFS/LlmHook) — expected; it names the self-heal gap. **Enable is a separate ops decision** (issue #478): before any non-zero value, add a freshness gate **or** use a controller-runtime 10h-scale period. Do not treat `300` as the enable value while passes last minutes. |
+| `CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP` | `10` | Absolute orphan-delete cap for a namespace-wide NetworkPolicy sweep. Candidates **strictly above** this count refuse **deletes** and increment `clerum_hcc_netpol_orphan_sweep_capped_total`; the pass **still certifies** (#478: alert and do not delete). Governs every `fullReconcile` (startup, event-driven, timer). Absolute is checked first, so at a fleet ≥ 50 with these defaults the percent rule never binds. `0` refuses every orphan delete; a negative value fails loud at load. Census: `scripts/ops/hcc-netpol-orphan-census.sh` — `live_cap_would_trip` is Deployment env only; `controller_cap_would_trip` applies compiled defaults when unset. When the Deployment omits `CONTEXT_MAPPER_RPC_PROXY_NAMESPACE`, the census assumes the compiled `rpc-proxy` default. |
+| `CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT` | `20` | Percent orphan-delete cap against the listed managed fleet. Strict `>`. Inert on a tiny inventory where `percent * listed < 1`. Same every-`fullReconcile` scope as the absolute cap; only binds when the fleet is small enough that 20% is below the absolute cap of 10. |
+| `KUBERNETES_SERVICE_HOST` | _(injected by Kubernetes)_ | Not set by the operator — the `/32` fallback source when `CONTEXT_MAPPER_K8S_API_CIDRS` is empty. |
### Plugin image allowlist
diff --git a/host-context-controller/package-lock.json b/host-context-controller/package-lock.json
index fd8dcd343..ac6fc1214 100644
--- a/host-context-controller/package-lock.json
+++ b/host-context-controller/package-lock.json
@@ -1,12 +1,12 @@
{
"name": "clerum-host-context-controller",
- "version": "0.1.251",
+ "version": "0.1.252",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "clerum-host-context-controller",
- "version": "0.1.251",
+ "version": "0.1.252",
"license": "SEE LICENSE IN LICENSE",
"dependencies": {
"@clerum/image-policy": "file:../packages/image-policy",
diff --git a/host-context-controller/package.json b/host-context-controller/package.json
index 8e8a38126..4729d0eb1 100644
--- a/host-context-controller/package.json
+++ b/host-context-controller/package.json
@@ -1,6 +1,6 @@
{
"name": "clerum-host-context-controller",
- "version": "0.1.251",
+ "version": "0.1.252",
"description": "Host Context Controller - provides McpServer CRDs to mcp-host via REST API",
"main": "dist/main.js",
"engines": {
diff --git a/host-context-controller/src/config.netPolResync.test.ts b/host-context-controller/src/config.netPolResync.test.ts
index 1c5f041e6..adac25644 100644
--- a/host-context-controller/src/config.netPolResync.test.ts
+++ b/host-context-controller/src/config.netPolResync.test.ts
@@ -44,4 +44,21 @@ describe('NetworkPolicy resync and orphan-sweep config (#478)', () => {
expect(config.netPolOrphanDeleteCap).toBe(8)
expect(config.netPolOrphanDeleteCapPercent).toBe(25)
})
+
+ it('accepts an explicit 0 absolute cap (refuse any orphan delete)', async () => {
+ const { config } = await loadConfig({ CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP: '0' })
+ expect(config.netPolOrphanDeleteCap).toBe(0)
+ })
+
+ it('fails loud when the absolute orphan-delete cap is negative', async () => {
+ await expect(loadConfig({ CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP: '-1' })).rejects.toThrow(
+ /CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP/
+ )
+ })
+
+ it('fails loud when the percent orphan-delete cap is negative', async () => {
+ await expect(
+ loadConfig({ CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT: '-1' })
+ ).rejects.toThrow(/CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT/)
+ })
})
diff --git a/host-context-controller/src/config.ts b/host-context-controller/src/config.ts
index ff96962df..4b756f2ac 100644
--- a/host-context-controller/src/config.ts
+++ b/host-context-controller/src/config.ts
@@ -179,7 +179,8 @@ export interface Config {
// Absolute orphan-delete cap for a NetworkPolicy fullReconcile sweep.
// Candidates strictly above this count refuse deletes and increment the
- // cap metric; the pass still certifies. Set via
+ // cap metric; the pass still certifies. 0 refuses every orphan delete.
+ // Negative values fail loud at load. Set via
// CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP.
netPolOrphanDeleteCap: number
@@ -282,6 +283,14 @@ function getEnvInt(key: string, defaultValue: number): number {
return isNaN(parsed) ? defaultValue : parsed
}
+/** Reject negative orphan-delete caps. 0 is a valid never-delete kill-switch. */
+function requireNonNegativeEnvInt(key: string, value: number): number {
+ if (!Number.isSafeInteger(value) || value < 0) {
+ throw new Error(`${key} must be a non-negative integer, got '${process.env[key] ?? ''}'`)
+ }
+ return value
+}
+
export function parseExternalEgressDnsTimeoutMs(raw: string | undefined): number {
if (raw === undefined || raw.trim() === '') return 5_000
const parsed = Number(raw)
@@ -764,8 +773,14 @@ export const config: Config = {
// Mass-delete guard for the namespace-wide orphan sweep. Thresholds are
// strict `>`: exactly N candidates still delete; N+1 refuses deletes and
// still certifies.
- netPolOrphanDeleteCap: getEnvInt('CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP', 10),
- netPolOrphanDeleteCapPercent: getEnvInt('CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT', 20),
+ netPolOrphanDeleteCap: requireNonNegativeEnvInt(
+ 'CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP',
+ getEnvInt('CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP', 10)
+ ),
+ netPolOrphanDeleteCapPercent: requireNonNegativeEnvInt(
+ 'CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT',
+ getEnvInt('CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT', 20)
+ ),
// Every finite Kubernetes request that can hold Host convergence gets its
// own deadline. Watch streams are intentionally excluded.
diff --git a/host-context-controller/src/k8sClient.test.ts b/host-context-controller/src/k8sClient.test.ts
index d73abbf73..8d654cebe 100644
--- a/host-context-controller/src/k8sClient.test.ts
+++ b/host-context-controller/src/k8sClient.test.ts
@@ -11332,12 +11332,17 @@ describe('McpServerWatcher NetworkPolicy periodic resync (#478)', () => {
const watcher = await startWatcherForNetPolResync()
expect(mocks.netPolFullReconcile).toHaveBeenCalledTimes(1)
+ expect(mocks.netPolFullReconcile.mock.calls[0]?.[2]).toEqual(
+ expect.objectContaining({ ensureDefaults: false })
+ )
+
await vi.advanceTimersByTimeAsync(300_000)
expect(mocks.netPolFullReconcile).toHaveBeenCalledTimes(2)
expect(mocks.netPolFullReconcile).toHaveBeenLastCalledWith(
expect.any(Array),
expect.any(Array),
expect.objectContaining({
+ ensureDefaults: true,
onAuthoritativeRevocationComplete: expect.any(Function),
runContextEffect: expect.any(Function),
contextInventoryAuthoritative: expect.any(Function),
@@ -11368,6 +11373,9 @@ describe('McpServerWatcher NetworkPolicy periodic resync (#478)', () => {
activePass.resolve()
await vi.waitFor(() => expect(mocks.netPolFullReconcile).toHaveBeenCalledTimes(2))
+ expect(mocks.netPolFullReconcile.mock.calls[1]?.[2]).toEqual(
+ expect.objectContaining({ ensureDefaults: true })
+ )
await watcher.stop()
})
diff --git a/host-context-controller/src/k8sClient.ts b/host-context-controller/src/k8sClient.ts
index 64f75dee4..6e0eabd7c 100644
--- a/host-context-controller/src/k8sClient.ts
+++ b/host-context-controller/src/k8sClient.ts
@@ -893,6 +893,10 @@ export class McpServerWatcher implements McpServerProvider {
// single-flight pass. Disabled by default (interval <= 0); startup still
// runs one convergence. Never call netPolReconciler.fullReconcile from here.
private netPolResyncTimer: ReturnType<typeof setInterval> | null = null
+ // Set by a timer tick so the next coordinated NetworkPolicy pass re-applies
+ // L0/L1 defaults (class D). Startup and event-driven entry leave this false
+ // — start() already ran ensureDefaultPolicies as a bootstrap barrier.
+ private netPolConvergenceEnsureDefaults = false
constructor() {
if (!kc) {
@@ -2855,7 +2859,7 @@ export class McpServerWatcher implements McpServerProvider {
const netPolResyncSec = config.netPolResyncIntervalSec
if (netPolResyncSec > 0) {
this.netPolResyncTimer = setInterval(() => {
- void this.runInitialNetworkPolicyConvergence()
+ void this.runInitialNetworkPolicyConvergence({ ensureDefaults: true })
}, netPolResyncSec * 1000)
console.log(`[K8s] NetworkPolicy periodic resync enabled (every ${netPolResyncSec}s)`)
} else {
@@ -3036,7 +3040,12 @@ export class McpServerWatcher implements McpServerProvider {
}
}
- private runInitialNetworkPolicyConvergence(): Promise<void> {
+ private runInitialNetworkPolicyConvergence(options?: {
+ ensureDefaults?: boolean
+ }): Promise<void> {
+ if (options?.ensureDefaults) {
+ this.netPolConvergenceEnsureDefaults = true
+ }
return this.runInitialConvergence('NetworkPolicy')
}
@@ -3123,9 +3132,11 @@ export class McpServerWatcher implements McpServerProvider {
this.mcpServerCacheSynced &&
this.mcpServerDesiredRevision === capturedSafetyCertificate.serverRevision
console.log('[K8s] Running initial NetworkPolicy background reconciliation...')
+ const ensureDefaults = this.netPolConvergenceEnsureDefaults
+ this.netPolConvergenceEnsureDefaults = false
await this.netPolReconciler.fullReconcile(initialContexts, initialServers, {
serverInventoryComplete,
- ensureDefaults: false,
+ ensureDefaults,
contextInventoryAuthoritative,
serverInventoryAuthoritative,
runContextEffect: (contextId, work) =>
@@ -4566,6 +4577,7 @@ export class McpServerWatcher implements McpServerProvider {
clearInterval(this.netPolResyncTimer)
this.netPolResyncTimer = null
}
+ this.netPolConvergenceEnsureDefaults = false
for (const timer of this.initialConvergenceRetryTimers.values()) {
clearTimeout(timer)
}
diff --git a/scripts/ops/hcc-netpol-orphan-census.sh b/scripts/ops/hcc-netpol-orphan-census.sh
index 088cc1eb1..5daa5d9c8 100755
--- a/scripts/ops/hcc-netpol-orphan-census.sh
+++ b/scripts/ops/hcc-netpol-orphan-census.sh
@@ -16,6 +16,15 @@
# treat live_cap_would_trip=none as "the controller will delete".
# A trip refuses deletes only; the pass still certifies.
#
+# Namespaces: mapper and host must be set on the Deployment (they are in
+# deploy/base). rpc-proxy may be omitted — the live line stays UNSET and
+# is labelled `(controller applies compiled default: rpc-proxy)`, same
+# two-tier shape as live_cap vs controller_cap. Sampling then uses that
+# compiled default. Do not declare the key in deploy/ to unblock the job.
+#
+# This census reflects on-cluster orphans by policy-type, not the
+# controller's current pass authority gate.
+#
# Double-samples 90s apart. Adjudicates only when the desired Context +
# McpServer identity set is identical across samples; otherwise
# INCONCLUSIVE_RERUN.
@@ -61,16 +70,29 @@ ORPHAN_CAP_PCT="$(deployment_env CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT
echo "[census] live Deployment env:"
echo " CONTEXT_MAPPER_NAMESPACE=${MAPPER_NS}"
echo " CONTEXT_MAPPER_HOST_NAMESPACE=${HOST_NS}"
-echo " CONTEXT_MAPPER_RPC_PROXY_NAMESPACE=${RPC_NS}"
+COMPILED_RPC_NS_DEFAULT=rpc-proxy
+if [[ "${RPC_NS}" == "UNSET" ]]; then
+ echo " CONTEXT_MAPPER_RPC_PROXY_NAMESPACE=UNSET (controller applies compiled default: ${COMPILED_RPC_NS_DEFAULT})"
+else
+ echo " CONTEXT_MAPPER_RPC_PROXY_NAMESPACE=${RPC_NS}"
+fi
echo " CONTEXT_MAPPER_NETPOL_RESYNC_SEC=${RESYNC_SEC}"
echo " CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP=${ORPHAN_CAP}"
echo " CONTEXT_MAPPER_NETPOL_ORPHAN_DELETE_CAP_PERCENT=${ORPHAN_CAP_PCT}"
-if [[ "${MAPPER_NS}" == "UNSET" || "${HOST_NS}" == "UNSET" || "${RPC_NS}" == "UNSET" ]]; then
- echo "[census] FATAL: Deployment did not declare mapper/host/rpc-proxy namespaces; refusing to invent defaults" >&2
+if [[ "${MAPPER_NS}" == "UNSET" || "${HOST_NS}" == "UNSET" ]]; then
+ echo "[census] FATAL: Deployment did not declare mapper/host namespaces; refusing to invent defaults" >&2
exit 2
fi
+# Base manifests omit CONTEXT_MAPPER_RPC_PROXY_NAMESPACE; the controller
+# compiles rpc-proxy (config.ts). Sample with that default; the live line
+# above already labelled UNSET vs compiled so we do not print a code
+# default as a cluster fact.
+if [[ "${RPC_NS}" == "UNSET" ]]; then
+ RPC_NS="${COMPILED_RPC_NS_DEFAULT}"
+fi
+
# One read-only snapshot of desired identities + orphan candidates.
# Filters on policy-type, not managed-by.
sample_state() {