Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
docs(hcc): add readiness & external-egress design note with flow diagram [R3-M5]
Alfredo Lopeze7f17f1cd2c7
diff --git a/docs/architecture/hcc-readiness-flows.md b/docs/architecture/hcc-readiness-flows.mdnew file mode 100644index 000000000..5618ab592--- /dev/null+++ b/docs/architecture/hcc-readiness-flows.md@@ -0,0 +1,87 @@+# HCC readiness & external-egress — design note (PR #205)++> Design note for the readiness-decoupling and single-writer NetworkPolicy work in+> `host-context-controller` (HCC). Addresses the "no design/decision doc for the+> 500+-line external-egress coordinator and the certification barrier" review gap.++++## 1. Readiness lifecycle — decoupled from initial fleet convergence++Before this work, `McpServerWatcher.start()` awaited full initial fleet+reconciliation, so a large MCP/Context/Host fleet could keep `provider.start()`+and `/ready` unavailable until every individual reconcile finished.++Now readiness is gated on an **authoritative, fail-closed safety boundary**, not on+full convergence:++1. **HCC start** — four watch lanes (Context ingress, mcp-host egress, rpc-proxy+ egress, external egress) start. A transient LIST/WATCH failure on any lane enters+ in-process recovery and **does not kill the process**; HCC simply stays unready.+2. **4-LIST inventory** — bounded inventory across the four lanes.+3. **Certify safety inventory** — the authoritative pass revokes any stale /+ unprovable allow **before** recording a certificate (fail-closed).+4. **`/ready = 200`** — served **only** when the inventory is certified and no stale+ allow survives.+5. **Additive convergence** — the remaining fleet converges **after** ready,+ asynchronously, without blocking it.++## 2. Fail-closed fences++| Fence | Rule | Source |+|-------|------|--------|+| Readiness gate | `/ready = ready && providerAuthoritativeFn()`; constructor default `() => false`; a throwing gate → 503 | `server.ts`, `readinessGate.ts` |+| G1 — unconfigured authority | `currentWhenUnconfigured()` returns `false` + warns once; prod always wires the source, dev never constructs the reconciler | `reconciler.ts` |+| F1 — readiness-poll rebind | rebinds `isCurrent`/`server` on a superseding reconcile so the poll window does not self-destruct on a stale fence | `reconciler.ts` |++The affirmative literal (`() => true`) is reachable only from the dev wiring; every+production path resolves the real authority function.++## 3. DNS external-egress contract (B3) — retain / revoke decision table++A DNS-derived external-egress allow is compared against the desired policy+reconstructed from the **live** resolved CIDRs ("modulo cidr"). The decision:++| Observed change | Classification | Action | Rationale |+|-----------------|----------------|--------|-----------|+| identity unchanged (same protocol, port, selector, ownership; cidrs only) | identity-stable | **RETAIN** while Ready; refresh cidrs in the additive lane | revoking a healthy binding every pass created a guaranteed deny window |+| protocol drift (TCP→UDP) | identity-changed | **REVOKE before readiness** | old allow is unprovable/stale |+| spec-level port drift | identity-changed | **REVOKE before readiness** | " |+| ownership drift | identity-changed | **REVOKE before readiness** | " |+| zero-cidr (deny-in-disguise) | never retained | **REVOKE** | an empty cidr set is never a valid retained allow |++Revocation of an unprovable policy runs immediately (`cleanupExternalEgress`) under+the `isCurrent()` fence, **before** the safety inventory is certified — so `/ready`+never reports 200 with a stale divergent policy present. The unit suite pins both+directions (retain identity-stable; revoke on each drift), and the branch-owned+Minikube gate `e2e-hcc-mcp-context-readiness.sh` exercises both phases live.++## 4. ExternalEgressConvergenceCoordinator++The coordinator is the **single owner** of external-egress reconcile cadence+(replacing an inline timer in `k8sClient`):++- **Per-server FIFO + concurrency limiter** — a slow or never-resolving binding+ cannot head-of-line block the control-plane event loop.+- **Sliding-window CIDR accumulation (#299)** — resolved IPs live for `TTL + overlap`;+ large provider pools are covered by provider CIDRs rather than per-IP `/32`s. A+ failed refresh deletes the allow (fail-closed), never widens it.+- **TTL-aware periodic resync (H2)** — `startPeriodicResync` re-arms after each pass+ with a delay of `min(interval, TTL/2)` bounded below by a floor, so low-TTL hosts+ refresh promptly. The timer is `.unref()`'d and re-programs **only after a pass+ completes** (no overlap).+- **Fence** — startup rejects `interval > overlap/2`, so the accumulated window can+ never lapse regardless of the configured cadence.+- **Observability** — `clerum_hcc_external_egress_retries_at_cap` gauges how many+ servers are stuck retrying at the delay cap (a never-resolving DNS binding is+ otherwise invisible: denied with `/ready` = 200 and no signal).++## 5. Deploy invariant — single writer++The HCC Deployment ships `strategy: Recreate`, `replicas: 1`, and+`progressDeadlineSeconds: 1200`: the stateless single-writer safety model has no+leader election, so two concurrently-running controllers must be impossible during a+rollout. The trade-off (full HCC control-plane downtime per rollout; a rollout whose+new pod never becomes Ready needs a manual `kubectl rollout undo`) is declared in the+manifest and asserted by the bootstrap gate. `validate-postgres-single-writer-strategy.sh`+enforces the invariant on the rendered manifest in CI.diff --git a/docs/architecture/pr205-hcc-flows.png b/docs/architecture/pr205-hcc-flows.pngnew file mode 100644index 000000000..955e1fe2fBinary files /dev/null and b/docs/architecture/pr205-hcc-flows.png differdiff --git a/docs/architecture/pr205-hcc-flows.svg b/docs/architecture/pr205-hcc-flows.svgnew file mode 100644index 000000000..082622f27--- /dev/null+++ b/docs/architecture/pr205-hcc-flows.svg@@ -0,0 +1,126 @@+<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1280 880" font-family="Inter, Segoe UI, Helvetica, Arial, sans-serif">+<defs>+<marker id="aBlue" markerWidth="9" markerHeight="8" refX="7.5" refY="4" orient="auto"><path d="M0,0 L9,4 L0,8 Z" fill="#2563eb"/></marker>+<marker id="aRose" markerWidth="9" markerHeight="8" refX="7.5" refY="4" orient="auto"><path d="M0,0 L9,4 L0,8 Z" fill="#dc2626"/></marker>+<marker id="aViolet" markerWidth="9" markerHeight="8" refX="7.5" refY="4" orient="auto"><path d="M0,0 L9,4 L0,8 Z" fill="#7c3aed"/></marker>+<marker id="aAmber" markerWidth="9" markerHeight="8" refX="7.5" refY="4" orient="auto"><path d="M0,0 L9,4 L0,8 Z" fill="#d97706"/></marker>+<marker id="aGreen" markerWidth="9" markerHeight="8" refX="7.5" refY="4" orient="auto"><path d="M0,0 L9,4 L0,8 Z" fill="#059669"/></marker>+<marker id="aGray" markerWidth="9" markerHeight="8" refX="7.5" refY="4" orient="auto"><path d="M0,0 L9,4 L0,8 Z" fill="#64748b"/></marker>+<filter id="sh" x="-5%" y="-6%" width="110%" height="122%"><feDropShadow dx="0" dy="1.5" stdDeviation="1.8" flood-color="#0f172a" flood-opacity="0.16"/></filter>+</defs>+<rect width="1280" height="880" fill="#ffffff"/>+<text x="40" y="36" font-size="20" font-weight="800" fill="#0f172a">host-context-controller — flujos que resuelve el PR #205</text>+<text x="40" y="57" font-size="12.5" fill="#64748b">decouple readiness from initial fleet convergence · seguridad fail-closed · gobernanza single-writer de NetworkPolicy</text>+<text x="40" y="92" font-size="13" font-weight="700" fill="#1d4ed8">1 · Lifecycle de readiness — DESACOPLADO de la convergencia de la flota</text>+<rect x="40.0" y="104" width="214" height="92" rx="11" fill="#eff6ff" stroke="#2563eb" stroke-width="1.8" filter="url(#sh)"/>+<text x="147.0" y="127" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">HCC start</text>+<text x="147.0" y="146" text-anchor="middle" font-size="11" fill="#475569">4 watch lanes arrancan;</text>+<text x="147.0" y="162" text-anchor="middle" font-size="11" fill="#475569">una lane que falla → recovery</text>+<text x="147.0" y="178" text-anchor="middle" font-size="11" fill="#475569">in-process, NO mata el proceso</text>+<rect x="286.5" y="104" width="214" height="92" rx="11" fill="#eff6ff" stroke="#2563eb" stroke-width="1.8" filter="url(#sh)"/>+<text x="393.5" y="127" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">4-LIST inventory</text>+<text x="393.5" y="146" text-anchor="middle" font-size="11" fill="#475569">Context ingress · mcp-host</text>+<text x="393.5" y="162" text-anchor="middle" font-size="11" fill="#475569">egress · rpc-proxy egress</text>+<text x="393.5" y="178" text-anchor="middle" font-size="11" fill="#475569">· external egress</text>+<rect x="533.0" y="104" width="214" height="92" rx="11" fill="#eff6ff" stroke="#2563eb" stroke-width="1.8" filter="url(#sh)"/>+<text x="640.0" y="127" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">Certificar safety inventory</text>+<text x="640.0" y="146" text-anchor="middle" font-size="11" fill="#475569">revoca allows stale/</text>+<text x="640.0" y="162" text-anchor="middle" font-size="11" fill="#475569">improbables ANTES de</text>+<text x="640.0" y="178" text-anchor="middle" font-size="11" fill="#475569">certificar (fail-closed)</text>+<rect x="779.5" y="104" width="214" height="92" rx="11" fill="#dbeafe" stroke="#1d4ed8" stroke-width="1.8" filter="url(#sh)"/>+<text x="886.5" y="127" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">/ready = 200</text>+<text x="886.5" y="146" text-anchor="middle" font-size="11" fill="#475569">solo si el inventory está</text>+<text x="886.5" y="162" text-anchor="middle" font-size="11" fill="#475569">certificado y ningún allow</text>+<text x="886.5" y="178" text-anchor="middle" font-size="11" fill="#475569">stale sobrevive</text>+<rect x="1026.0" y="104" width="214" height="92" rx="11" fill="#eff6ff" stroke="#2563eb" stroke-width="1.8" filter="url(#sh)"/>+<text x="1133.0" y="127" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">Additive convergence</text>+<text x="1133.0" y="146" text-anchor="middle" font-size="11" fill="#475569">la flota converge DESPUÉS</text>+<text x="1133.0" y="162" text-anchor="middle" font-size="11" fill="#475569">del ready — asíncrono,</text>+<text x="1133.0" y="178" text-anchor="middle" font-size="11" fill="#475569">sin bloquearlo</text>+<line x1="254.0" y1="150.0" x2="286.5" y2="150.0" stroke="#2563eb" stroke-width="2.6" marker-end="url(#aBlue)" stroke-linecap="round"/>+<rect x="229.05" y="130.0" width="82.4" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="270.25" y="142.0" text-anchor="middle" font-size="10" font-weight="700" fill="#1e293b">inventariar</text>+<line x1="500.5" y1="150.0" x2="533.0" y2="150.0" stroke="#2563eb" stroke-width="2.6" marker-end="url(#aBlue)" stroke-linecap="round"/>+<rect x="478.75" y="130.0" width="76.0" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="516.75" y="142.0" text-anchor="middle" font-size="10" font-weight="700" fill="#1e293b">certificar</text>+<line x1="747.0" y1="150.0" x2="779.5" y2="150.0" stroke="#2563eb" stroke-width="2.6" marker-end="url(#aBlue)" stroke-linecap="round"/>+<rect x="738.05" y="130.0" width="50.400000000000006" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="763.25" y="142.0" text-anchor="middle" font-size="10" font-weight="700" fill="#1e293b">/ready</text>+<line x1="993.5" y1="150.0" x2="1026.0" y2="150.0" stroke="#64748b" stroke-width="2.4" stroke-dasharray="6,4" marker-end="url(#aGray)" stroke-linecap="round"/>+<rect x="946.15" y="130.0" width="127.2" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="1009.75" y="142.0" text-anchor="middle" font-size="10" font-weight="700" fill="#475569">async · post-ready</text>+<text x="40" y="275" font-size="13" font-weight="700" fill="#b91c1c">2 · Fences fail-closed (rondas R3 / R4)</text>+<rect x="40.0" y="284" width="380.0" height="76" rx="11" fill="#fef2f2" stroke="#dc2626" stroke-width="1.8" filter="url(#sh)"/>+<text x="230.0" y="307" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">Readiness gate — default () => false</text>+<text x="230.0" y="326" text-anchor="middle" font-size="11" fill="#475569">/ready = ready && providerAuthoritativeFn()</text>+<text x="230.0" y="342" text-anchor="middle" font-size="11" fill="#475569">throw → 503 · nunca un 200 equivocado</text>+<rect x="450.0" y="284" width="380.0" height="76" rx="11" fill="#fef2f2" stroke="#dc2626" stroke-width="1.8" filter="url(#sh)"/>+<text x="640.0" y="307" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">G1 — deny sin inventory-authority</text>+<text x="640.0" y="326" text-anchor="middle" font-size="11" fill="#475569">currentWhenUnconfigured() → false + warn</text>+<text x="640.0" y="342" text-anchor="middle" font-size="11" fill="#475569">dev nunca construye el reconciler</text>+<rect x="860.0" y="284" width="380.0" height="76" rx="11" fill="#fef2f2" stroke="#dc2626" stroke-width="1.8" filter="url(#sh)"/>+<text x="1050.0" y="307" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">F1 — rebind del readiness-poll</text>+<text x="1050.0" y="326" text-anchor="middle" font-size="11" fill="#475569">re-liga isCurrent/server en reconciles</text>+<text x="1050.0" y="342" text-anchor="middle" font-size="11" fill="#475569">superpuestos → la ventana no se autodestruye</text>+<text x="40" y="421" font-size="13" font-weight="700" fill="#6d28d9">3 · Egress externo DNS — contrato B3 (retener/revocar) + sliding-window #299</text>+<rect x="40" y="438" width="224" height="80" rx="11" fill="#f5f3ff" stroke="#7c3aed" stroke-width="1.8" filter="url(#sh)"/>+<text x="152.0" y="461" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">DNS binding observado</text>+<text x="152.0" y="480" text-anchor="middle" font-size="11" fill="#475569">resolveExternalEgressDns</text>+<text x="152.0" y="496" text-anchor="middle" font-size="11" fill="#475569">timeout 5s + fence isCurrent</text>+<polygon points="372,436 472,478 372,520 272,478" fill="#f5f3ff" stroke="#7c3aed" stroke-width="1.8" filter="url(#sh)"/>+<text x="372" y="475" text-anchor="middle" font-size="12" font-weight="700" fill="#0f172a">¿identity</text>+<text x="372" y="491" text-anchor="middle" font-size="12" font-weight="700" fill="#0f172a">unchanged?</text>+<line x1="264" y1="478" x2="272" y2="478" stroke="#7c3aed" stroke-width="2.6" marker-end="url(#aViolet)" stroke-linecap="round"/>+<rect x="540" y="414" width="306" height="60" rx="11" fill="#ecfdf5" stroke="#059669" stroke-width="1.8" filter="url(#sh)"/>+<text x="693.0" y="437" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">RETAIN — allow identity-stable</text>+<text x="693.0" y="456" text-anchor="middle" font-size="11" fill="#475569">mismo protocolo/puerto/ownership/cidrs</text>+<text x="693.0" y="472" text-anchor="middle" font-size="11" fill="#475569">· HCC sigue Ready (módulo cidr)</text>+<rect x="540" y="496" width="306" height="60" rx="11" fill="#fef2f2" stroke="#dc2626" stroke-width="1.8" filter="url(#sh)"/>+<text x="693.0" y="519" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">REVOKE antes de readiness</text>+<text x="693.0" y="538" text-anchor="middle" font-size="11" fill="#475569">drift TCP→UDP · port · ownership · zero-cidr</text>+<text x="693.0" y="554" text-anchor="middle" font-size="11" fill="#475569">→ unprovable → cleanup fail-closed</text>+<path d="M 472,470 H 506.0 V 444 H 540" fill="none" stroke="#059669" stroke-width="2.6" marker-end="url(#aGreen)" stroke-linejoin="round"/>+<rect x="423.6" y="415" width="88.80000000000001" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="468" y="427" text-anchor="middle" font-size="10" font-weight="700" fill="#047857">sí · retiene</text>+<path d="M 472,486 H 506.0 V 526 H 540" fill="none" stroke="#dc2626" stroke-width="2.6" marker-end="url(#aRose)" stroke-linejoin="round"/>+<rect x="417.2" y="523" width="101.60000000000001" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="468" y="535" text-anchor="middle" font-size="10" font-weight="700" fill="#b91c1c">drift · revoca</text>+<rect x="892" y="430" width="348" height="88" rx="11" fill="#fffbeb" stroke="#d97706" stroke-width="1.8" filter="url(#sh)"/>+<text x="1066.0" y="453" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">ExternalEgressConvergenceCoordinator</text>+<text x="1066.0" y="472" text-anchor="middle" font-size="11" fill="#475569">resync periódico TTL-aware (≤ TTL/2, floor)</text>+<text x="1066.0" y="488" text-anchor="middle" font-size="11" fill="#475569">sliding-window accumulate · provider CIDRs</text>+<text x="1066.0" y="504" text-anchor="middle" font-size="11" fill="#475569">fence interval ≤ overlap/2 · limiter</text>+<line x1="846" y1="444" x2="892" y2="460" stroke="#d97706" stroke-width="2.6" marker-end="url(#aAmber)" stroke-linecap="round"/>+<rect x="821.4" y="425" width="95.2" height="16" rx="4" fill="#ffffff" stroke="#e2e8f0" stroke-width="0.8"/>+<text x="869" y="437" text-anchor="middle" font-size="10" font-weight="700" fill="#b45309">add (aditiva)</text>+<text x="40" y="575" font-size="13" font-weight="700" fill="#047857">4 · Seam de rotación #223 (preservado) & invariante de deploy single-writer</text>+<rect x="40" y="584" width="236" height="66" rx="11" fill="#ecfdf5" stroke="#059669" stroke-width="1.8" filter="url(#sh)"/>+<text x="158.0" y="607" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">validateSecret()</text>+<text x="158.0" y="626" text-anchor="middle" font-size="11" fill="#475569">sha256(canonicalJSON(secret.data))</text>+<rect x="306" y="584" width="306" height="66" rx="11" fill="#ecfdf5" stroke="#059669" stroke-width="1.8" filter="url(#sh)"/>+<text x="459.0" y="607" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">ensureDeployment(rev, isCurrent)</text>+<text x="459.0" y="626" text-anchor="middle" font-size="11" fill="#475569">el fence de #205 se propaga aquí</text>+<rect x="642" y="584" width="300" height="66" rx="11" fill="#ecfdf5" stroke="#059669" stroke-width="1.8" filter="url(#sh)"/>+<text x="792.0" y="607" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">pod annotation</text>+<text x="792.0" y="626" text-anchor="middle" font-size="11" fill="#475569">clerum.io/credentials-revision → rueda 1 vez</text>+<line x1="276" y1="617" x2="306" y2="617" stroke="#059669" stroke-width="2.6" marker-end="url(#aGreen)" stroke-linecap="round"/>+<line x1="612" y1="617" x2="642" y2="617" stroke="#059669" stroke-width="2.6" marker-end="url(#aGreen)" stroke-linecap="round"/>+<rect x="972" y="584" width="268" height="66" rx="11" fill="#f8fafc" stroke="#475569" stroke-width="1.8" filter="url(#sh)"/>+<text x="1106.0" y="607" text-anchor="middle" font-size="14" font-weight="700" fill="#0f172a">Deploy single-writer</text>+<text x="1106.0" y="626" text-anchor="middle" font-size="11" fill="#475569">strategy: Recreate · replicas 1</text>+<text x="1106.0" y="642" text-anchor="middle" font-size="11" fill="#475569">· progressDeadlineSeconds 1200</text>+<text x="40" y="700" font-size="11.5" font-weight="700" fill="#334155">Leyenda</text>+<rect x="118" y="688" width="14" height="14" rx="3" fill="#2563eb" opacity="0.9"/>+<text x="137" y="700" font-size="11" fill="#475569">lifecycle readiness</text>+<rect x="282.6" y="688" width="14" height="14" rx="3" fill="#dc2626" opacity="0.9"/>+<text x="301.6" y="700" font-size="11" fill="#475569">fence fail-closed / revoke</text>+<rect x="492.0" y="688" width="14" height="14" rx="3" fill="#7c3aed" opacity="0.9"/>+<text x="511.0" y="700" font-size="11" fill="#475569">egress DNS (B3)</text>+<rect x="631.0" y="688" width="14" height="14" rx="3" fill="#d97706" opacity="0.9"/>+<text x="650.0" y="700" font-size="11" fill="#475569">coordinator #299</text>+<rect x="776.4" y="688" width="14" height="14" rx="3" fill="#059669" opacity="0.9"/>+<text x="795.4" y="700" font-size="11" fill="#475569">retain / rotación #223</text>+<rect x="960.2" y="688" width="14" height="14" rx="3" fill="#475569" opacity="0.9"/>+<text x="979.2" y="700" font-size="11" fill="#475569">invariante deploy</text>+<text x="40" y="858" font-size="10.5" fill="#94a3b8">PR #205 · evenfire-ai/evenfire · el /ready certifica una frontera de seguridad autoritativa ANTES de que la flota converja — la convergencia restante es aditiva y asíncrona</text>+</svg>\ No newline at end of file