Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
test: guard subscription T0 lanes and document Grok rollout
Jose Gabriel Ramirez Encinasd0ee13909921
diff --git a/.github/workflows/ci-public.yml b/.github/workflows/ci-public.ymlindex 88835d502..bc1e52655 100644--- a/.github/workflows/ci-public.yml+++ b/.github/workflows/ci-public.yml@@ -264,7 +264,10 @@ jobs:pluginWorkloadSdkCodexDualLedger.realPostgres.integration \pluginWorkloadSdkGrokDualLedger.realPostgres.integration \services.grokSubscriptionConnection.realPostgres.integration \- services.grokProviderAttemptRedemption.realPostgres.integration+ services.grokProviderAttemptRedemption.realPostgres.integration \+ services.grokProviderAttemptRedemption.refresh.realPostgres.integration \+ services.codexSubscriptionLifecycle.realPostgres.integration \+ db.llmProviderAttemptConnectionIntegrity.realPostgres.integrationdogrep -Fq "$suite" "$normalized_log" || {echo "real-Postgres control-api lane omitted Codex suite ${suite}" >&2diff --git a/docs/architecture/codex-subscription-transport-contract.md b/docs/architecture/codex-subscription-transport-contract.mdindex 5ce49ad09..c22645472 100644--- a/docs/architecture/codex-subscription-transport-contract.md+++ b/docs/architecture/codex-subscription-transport-contract.md@@ -148,6 +148,69 @@ Deployment readiness checks after a combined apply do not enforce the orderingabove. The infrastructure rollout barrier remains a separate release dependency;this document alone does not certify mixed-version deployment safety.+### Codex changes shipped with the Grok broker++The Grok subscription broker (see+`docs/architecture/grok-subscription-transport-contract.md`) shares the+contract, authorizer, proxy robustness and recipe-grant code with Codex. The V1+wire and hash projection are unchanged. Operators should know about these Codex+behavior changes:++- **Revoke and reconnect.** Codex connection keys stay reusable after revoke:+ `deployment-default` and named keys can be revoked and then reconnected with+ a new OAuth flow.+ - Revoke also cancels every pending browser and device OAuth state for that+ key, atomically in the same statement.+ - A flow that started before the revoke cannot persist a grant. It fails+ with `state_cancelled`.+ - When two first grants race on the same key, the losing active-key insert+ maps to a stale-revision conflict instead of a 500.+- **Attempt integrity (migration 0114).** A constraint trigger requires a+ non-null `llm_provider_attempts.connection_id` of a Codex attempt to exist in+ `codex_subscription_connections`. This replaces the FK that 0112 dropped.+ Historic rows are not revalidated, and the migration is forward-only.+- **Catalog bounds.** Catalog sync stores at most 256 discovered models, each+ with an id of at most 128 characters.+ - Extra entries are dropped with a count-only warning.+ - The catalog record, row mutations and union rebuild commit in one+ transaction.+ - `codex-llm-proxy` admin catalog and connection-test calls have a 15 s+ deadline and an 8 MiB streamed body cap.+- **Request identity.** `mcp-host` hashes with `hashCanonicalCodexRequest(raw)`+ (parse, then hash the canonical value), so empty `generation`, `tools` or+ `transportHints` objects hash the same as on the server. Contract trees are+ capped at a nesting depth of 64. The authorizer rejects deeper bodies with+ `invalid_request`.+- **Terminal outcomes.** Only `success` is a completion. `canceled`, `error`,+ and `unknown` with partial text or tool calls are errors. The proxy maps an+ unrecognized finalize outcome to `unknown`.+- **Proxy robustness.**+ - `codex-llm-proxy` fails at startup when its control-api URL or service+ token is empty.+ - It respects SSE write backpressure.+ - It drops queued stream-gate waiters on abort and checks the abort signal+ before redeeming a ticket.+ - It requires `maxStreamDurationMs` greater than 0.+- **Live-target attestation.** Codex authorize attests the live Host or recipe+ target. The allowed providers come only from the spec's model, allowed+ models and fallbacks (Hosts) or agent providers (recipes). A target that+ reached Codex only through a grant annotation or `connectionRef` now gets+ `host_binding_mismatch`.+- **Recipe grant annotations.** control-api now writes both+ `codex-connection-ref` and `subscription-connection-ref` for Codex recipes.+ It rejects explicit disagreeing pairs with 422+ `subscriptionAnnotationsDisagree`. Broker changes that omit the annotations+ return 422 `providerChangeRequiresGrant`.+- **Rollout.** Roll out control-api first, and wait until every pod runs the+ new image. An older control-api rejects the entire workflow-control token+ issue when HCC or WRC request the unknown `llm:grok:execute` scope.+- **Rollback.** Turn the Grok flags off and wait for HCC and WRC to drop+ `llm:grok:execute` before rolling back control-api. While an older+ control-api serves, a reassigned Codex recipe grant updates only+ `codex-connection-ref`. The new WRC then reads the disagreeing pair as+ `unassigned`. After rolling forward again, re-assign those Codex recipe+ grants with explicit annotations.+## ErrorsStable codes: `insufficient_scope`, `no_grant`, `model_not_allowed`,diff --git a/docs/architecture/grok-subscription-transport-contract.md b/docs/architecture/grok-subscription-transport-contract.mdindex 85162162e..986a30c1d 100644--- a/docs/architecture/grok-subscription-transport-contract.md+++ b/docs/architecture/grok-subscription-transport-contract.md@@ -3,9 +3,31 @@Starting freeze: `grok-subscription-transport.v1`.This is the Phase 0 architecture freeze for provider `grok-subscription`.-Catalog origin and CLI identity headers remain probe-gated (see-`docs/superpowers/specs/2026-09-15-llm-coding-plan-subscriptions-design.md`-§9 and R2-M3). Live SuperGrok confirm may tighten, never widen, this allowlist.++## Probe gate++The following facts are a starting freeze taken from Grok Build docs and+third-party probes. No live SuperGrok probe has confirmed them yet:++- the catalog origin,+- the CLI identity headers,+- the numeric limits,+- the public CLI `client_id`,+- the OAuth scope list,+- whether the upstream accepts `temperature`.++Rule: before the production flag flip, commit a redacted probe record beside+this document. Regenerate the sanitized fixture+(`tests/e2e/fixtures/grok-subscription/sanitized-upstream-contract.json`) from+that record. The freeze test must assert that the fixture matches the enforced+origin-policy constants. A recorded `GROK_REAL_UPSTREAM_CONFIRM=1` run must also+exist.++A live probe may tighten this allowlist but must never widen it. If the probe+finds a different catalog endpoint, or no list API at all, the origin may be+replaced once. After that, it cannot be widened. Until the probe record exists,+the fixture's `approvedTestAccountFingerprint` stays+`redacted:grok-subscription-pending-live-confirm`.## Ownership@@ -22,13 +44,61 @@ Exact HTTPS origins (no caller-supplied URL):- Device authorization: `https://auth.x.ai/oauth2/device/code`- Token: `https://auth.x.ai/oauth2/token`+- Revoke: `https://auth.x.ai/oauth2/revoke`- Completions: `https://cli-chat-proxy.grok.com/v1/responses`- Catalog (starting candidate): `https://cli-chat-proxy.grok.com/v1/models`Forbidden: `https://api.x.ai`, HTTP, loopback, link-local, metadata.Redirects: at most one, HTTPS, same origin and same path and same query.-OAuth `device/code` and `token` use `redirect: 'manual'`; any 3xx is an error.+OAuth `device/code`, `token` and `revoke` use `redirect: 'manual'`; any 3xx is+an error.++## OAuth++- `client_id`: `b1a00492-073a-47ea-816f-4c329264a828`. This is the public+ Grok CLI registration, not a secret. `CONTROL_API_GROK_OAUTH_CLIENT_ID` may+ override it, but only to pin a documented registration.+- Grant: `urn:ietf:params:oauth:grant-type:device_code`.+- Scopes (`GROK_OAUTH_SCOPES` in+ `control-api/src/services/grokSubscriptionOAuth.ts`): `openid`, `profile`,+ `email`, `offline_access`, `grok-cli:access`, `api:access`,+ `conversations:read`, `conversations:write`.+ - This list is **pending the live probe trim**. `api:access` and+ `conversations:read/write` are broader than inference needs: a leaked+ access token could read or write Grok conversations. The probe must record+ the minimum set the CLI proxy accepts, and the list must shrink to that+ set before the production flag flip.+- The refresh token rotates on every refresh. The new ciphertext is persisted+ before any fallible post-processing.+- Revoke revokes the refresh token upstream. Access tokens that were already+ issued stay valid until they expire (at most one access-token lifetime).++## Connection lifecycle++- **Grok connection keys are terminal after revoke (migration 0113).**+ - A revoked key cannot be reconnected. Create a new connection key instead.+ - A device flow that started before the revoke cannot revive the key: it+ fails with `revoked grant cannot be reused`.+ - 0113 archives superseded revoked rows for the same key as+ `<key>~revoked~<id>`. It also cancels pending OAuth states for revoked+ keys and replaces the partial active-key index with a unique index on+ `connection_key`.+ - Archived `~revoked~` rows fall outside the key grammar. Admin list and GET+ hide them. The key's own terminal tombstone stays listed as `revoked`.+- **Attempt integrity (migration 0114).** A constraint trigger on+ `llm_provider_attempts` INSERT, and on UPDATE of `provider` or+ `connection_id`, requires a non-null `connection_id` to exist in the+ provider's own connection table:+ - `grok-subscription` → `grok_subscription_connections`+ - `codex-subscription` → `codex_subscription_connections`++ This replaces the FK that 0112 dropped, so it covers Codex too. It does not+ revalidate historic rows: the migration logs only a count of dangling rows.+- Catalog sync stores at most 256 discovered models, each with an id of at most+ 128 characters. Extra entries are dropped with a structured count-only+ warning. The catalog record, row mutations and union rebuild commit in one+ transaction.## Ticket@@ -36,11 +106,37 @@ OAuth `device/code` and `token` use `redirect: 'manual'`; any 3xx is an error.- `aud`: `grok-llm-proxy`- Admin permit `typ`/`aud`: `grok-admin-permit` / `grok-llm-proxy-admin`+## Request identity++- `mcp-host` hashes the request with `hashCanonicalGrokRequest(raw)` from+ `@clerum/grok-provider-attempt-contract`. This function parses the request+ first and then hashes the parsed canonical value, so empty `generation: {}`,+ `tools: []` or `transportHints: {}` hash the same on the client and the+ server. Codex uses the twin `hashCanonicalCodexRequest`.+- Contract trees are capped at a nesting depth of 64 (`fail('limit')`). The+ control-api authorizer rejects deeper bodies with `invalid_request` before it+ calls `JSON.stringify`.+## LimitsOwned by `@clerum/grok-provider-attempt-contract`. Independent of Codex`LIMITS` (`maxToolCalls` is 64, not 32).+Proxy robustness (both proxies):++- Admin catalog and connection-test upstream calls have a 15 s deadline that+ covers headers and body, and a streamed body cap: 1 MiB for Grok, 8 MiB for+ Codex. Normalized models are capped at 256, and each id at 128 characters,+ matching control-api.+- The SSE writer respects `res.write` backpressure: it waits for `drain` and+ stops waiting if the client closes.+- Queued stream-gate waiters stop when the request aborts, and the proxy+ checks the abort signal before redeeming a ticket.+- The proxy fails at startup when `GROK_LLM_PROXY_CONTROL_API_URL` or+ `GROK_LLM_PROXY_CONTROL_API_TOKEN` is empty.+- An unrecognized finalize outcome maps to `unknown`, never `success`.+ `maxStreamDurationMs` must be greater than 0.+## Identity headersStamped in `grok-llm-proxy`, never taken from mcp-host or the hashed request:@@ -50,24 +146,130 @@ Stamped in `grok-llm-proxy`, never taken from mcp-host or the hashed request:CLI impersonation headers (`x-xai-token-auth`, `x-grok-client-identifier`) stayoff until a recorded SuperGrok probe requires them. Do not send `OpenAI-Beta`-or `service_tier`. `max_output_tokens` is bindable on this wire.+or `service_tier`.++`max_output_tokens` is bindable on this wire and is sent.++`temperature` is **withheld upstream** while+`GROK_UPSTREAM_TEMPERATURE_PROBE_CONFIRMED` (in+`grok-llm-proxy/src/grokTransport.ts`) is `false`. The Codex Responses wire+rejects sampling fields, and no live probe has shown that Grok accepts them.+`generation.temperature` still counts toward the authorize hash. Set the+constant to `true` only with recorded live-probe evidence.Tool payloads send `strict: false` so optional MCP fields stay optional on theResponses-shaped Grok wire (sibling of Codex #648). Agent tool presentationuses the same `CODEX_TOOL_PRESENTATION` knob as Codex and defaults to `direct`when Grok is the primary or a fallback (sibling of Codex #644).+## Terminal outcomes++For both providers, a stream counts as a completion only when it ends with+`success`. `mcp-host` reports these outcomes as errors, never as a completion:++- `canceled`,+- `error`,+- `unknown` with partial text or tool calls.++## Feature flags++All flags default to off.++| Flag | Owner | Gates |+| --- | --- | --- |+| `CONTROL_API_GROK_SUBSCRIPTION_ENABLED` | control-api ConfigMap | Admin Grok routes, Host/recipe acceptance of `grok-subscription`, and the `clerum.io/grok-enabled` annotation that control-api publishes on `clerum-llm-allowed-models`. |+| `WRC_GROK_SUBSCRIPTION_ENABLED` | workflow-recipes Deployment | Folded into the recipe verdict. Gates the `llm:grok:execute` scope, the `<recipe>-mcp-host-to-grok-proxy` egress NetworkPolicy, the bootstrap binding, the pod env, and `grantRedeemable`. |+| `GROK_LLM_PROXY_EXECUTION_ENABLED` | grok-llm-proxy ConfigMap | Kill switch: while it is off, runtime and admin requests return 404 `disabled`, even with a valid JWT, ticket or permit. |+| `MCP_HOST_GROK_SUBSCRIPTION_ENABLED` | **injected, not a manual key** | HCC injects it on Host pods whose Grok projection mints `llm:grok:execute`. WRC injects it, with `GROK_LLM_PROXY_RUNTIME_URL`, on recipe pods that declare Grok when `WRC_GROK_SUBSCRIPTION_ENABLED` is on. Do not add it to `mcp-host-config`. |++HCC has no local Grok switch. Its Grok projection follows the+`clerum.io/grok-enabled` annotation together with the Host's own Grok+references.++## Recipe grant annotations++Annotation write shapes:++- Codex: `{codex-connection-ref: k, subscription-connection-ref: k}`+- Grok: `{codex-connection-ref: '', subscription-connection-ref: k}`++The server (`control-api/src/services/recipeGrantTransition.ts`) enforces+these transitions. None of them may return a 500:++| Case | Behavior |+| --- | --- |+| Explicit alias and canonical both non-empty and unequal (any spec) | 422 `subscriptionAnnotationsDisagree` |+| Create, or static → broker | Omitted annotations: 422 (grant required). Explicit: validate, then write the broker's shape. |+| Same broker, annotations omitted | Keep the stored annotations |+| Same broker, annotations explicit | Validate the key, then write that broker's shape |+| Codex → Grok | Omitted annotations: 422 `providerChangeRequiresGrant`. Non-empty alias: 422. Otherwise validate the canonical key and write the Grok shape. |+| Grok → Codex | Omitted annotations: 422 `providerChangeRequiresGrant`. Otherwise validate (canonical-only is allowed) and write the Codex shape. Never clear silently. |+| Broker → static | Without an SDK: clear both annotations. With an SDK: keep them (the SDK route owns them). |+| Static → static | Keep the stored annotations (the SDK route owns static-recipe identity) |+## Allowlist ConfigMap republish-control-api is `replicas: 1`. After rolling a new control-api that writes+control-api is `replicas: 1`. After rolling out a new control-api that writes`clerum.io/grok-*` annotations, republish `clerum-llm-allowed-models` once the-new pod is Ready so a mutation on the old pod cannot strip Grok annotations.+new pod is Ready. Otherwise a mutation served by the old pod could strip the+Grok annotations. A new control-api also republishes on boot.++## Rollout order++1. **evenfire-infra first.** Pin the `grok-llm-proxy` image for `gcp-dev` and+ `gcp-prod`. Provision `grok-llm-proxy-secrets`+ (`GROK_LLM_PROXY_CONTROL_API_TOKEN`) with+ `deploy/scripts/apply-inter-service-tokens.sh`. Both must be in place before+ the merge to `dev` triggers the automatic deploy.+2. **control-api** with migrations 0109–0114. Wait until every control-api pod+ runs the new image. An old control-api rejects the whole workflow-control+ token issue when the request includes the unknown `llm:grok:execute` scope+ (`invalid_workflow_control_scopes`). HCC or WRC deployed first could+ therefore break token issuance for every pod they mint for, not only Grok+ pods (C-RP-016).+3. **HCC, WRC, mcp-host and grok-llm-proxy.** Grok flags stay off.+4. Republish `clerum-llm-allowed-models` (see above).+5. Turn on the flags only after the control-api rollout is complete:+ `CONTROL_API_GROK_SUBSCRIPTION_ENABLED`, `WRC_GROK_SUBSCRIPTION_ENABLED` and+ `GROK_LLM_PROXY_EXECUTION_ENABLED`. `MCP_HOST_GROK_SUBSCRIPTION_ENABLED`+ follows automatically through HCC and WRC.-Rollback: drop `llm:grok:execute` and Grok proxy NetworkPolicies in HCC/WRC-before rolling control-api back. A new control-api republishes on boot.+## Rollback order++1. Turn the Grok flags off. control-api republishes+ `clerum.io/grok-enabled=false`.+2. Wait for HCC and WRC to reconcile, so that no Host or recipe pod still mints+ `llm:grok:execute` and the Grok proxy NetworkPolicies are gone. Only then+ roll back control-api. An old control-api rejects the entire token issue+ for the unknown scope (C-RP-016).+3. After a control-api rollback, the old control-api writes only+ `codex-connection-ref` when a Codex recipe grant is reassigned. The stale+ `subscription-connection-ref` then disagrees with it. The new WRC reads the+ recipe as `unassigned`, and authorize returns a mismatch (B-M5b).+ - Once a control-api that writes both keys is serving again, list the Codex+ recipes whose two annotations are both non-empty and unequal.+ - Re-assign each grant with explicit annotations, which write the Codex+ shape. In the UI, re-pick the connection.+ - A save that omits the annotations keeps the stored disagreeing pair, so+ it does not repair the grant.+4. Migrations 0113 (terminal Grok keys) and 0114 (attempt connection+ integrity) are forward-only. An older control-api still runs against them:+ - The 0114 trigger only rejects connection ids that do not exist.+ - Under 0113, a revoked Grok key can no longer be reconnected by the old+ code path. That attempt fails on the unique key index instead of reviving+ the key.## SDK bootstrap-Grok writers emit `subscriptionBinding` only. Readers accept `subscriptionBinding`-or `codexBinding`. Grok missing-binding reason is `execution_binding_missing`.-Codex keeps `codexBinding` and `codex_execution_binding_missing`.+Binding slots are strict:++- Grok writers emit `subscriptionBinding`, and Grok readers (mcp-host+ `bootstrapIdentity`, WRC provisioner and bootstrap proof parsers) read only+ `subscriptionBinding`.+- Codex writes and reads only `codexBinding`. A proof in the other slot is+ ignored.++Missing-binding reasons:++- Grok: `execution_binding_missing`+- Codex: `codex_execution_binding_missing`diff --git a/scripts/tests/test-codex-subscription-t0.sh b/scripts/tests/test-codex-subscription-t0.shindex 25594c85f..2a2db3224 100755--- a/scripts/tests/test-codex-subscription-t0.sh+++ b/scripts/tests/test-codex-subscription-t0.sh@@ -4,6 +4,9 @@# Fails when a required suite is missing, executes zero tests, reports# skipped/todo cases, or exits non-zero. Exit 0 alone is never enough: the# script parses machine-readable counts from Vitest or node:test.+#+# Lane separation: Grok-only suites run in test-grok-subscription-t0.sh, never+# here. Provider-neutral suites shared by both brokers may appear in both.set -uROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"@@ -221,17 +224,17 @@ run_group "control-api" "control-api" \"test/routes.adminRecipes.test.ts" \"test/routes.adminPluginWorkloadSdk.test.ts" \"test/services.recipeCodexGrantIdentity.test.ts" \+ "test/services.recipeGrantTransition.test.ts" \"test/crd.llmProviderEnums.test.ts" \"test/hostSpecValidation.codexSubscription.test.ts" \- "test/hostSpecValidation.grokSubscription.test.ts" \"test/llmProviders.test.ts" \"test/services.codexSubscriptionConnection.test.ts" \"test/services.llmAllowedModelsConfigMap.test.ts" \"test/services.codexSubscriptionOAuth.test.ts" \- "test/services.grokSubscriptionOAuth.test.ts" \"test/services.codexSubscriptionCatalog.test.ts" \+ "test/services.subscriptionCatalogBounds.test.ts" \"test/services.llmProviderAttemptAuthorizer.test.ts" \- "test/services.llmProviderAttemptAuthorizer.grok.test.ts" \+ "test/services.llmProviderAttemptAuthorizer.depth.test.ts" \"test/services.llmProviderAttemptTicket.test.ts" \"test/services.llmProviderAttemptRedemption.test.ts" \"test/services.llmProviderAttemptFinalization.test.ts" \@@ -244,11 +247,14 @@ run_group "control-api" "control-api" \run_group "codex-llm-proxy" "codex-llm-proxy" \"test/approvedToolsUpstream.test.ts" \+ "test/catalogBounds.test.ts" \"test/codexTransport.conformance.test.ts" \"test/controlApiClient.test.ts" \"test/originPolicy.test.ts" \"test/redaction.test.ts" \- "test/server.security.test.ts"+ "test/requestLimits.test.ts" \+ "test/server.security.test.ts" \+ "test/sseBackpressure.test.ts"run_group "mcp-host" "mcp-host" \"src/capabilities/toolCatalogTools.test.ts" \@@ -263,6 +269,7 @@ run_group "mcp-host" "mcp-host" \"src/agent/__tests__/taskExecutor.test.ts" \"src/llm/__tests__/codexSubscription.test.ts" \"src/llm/__tests__/codexLlmProxyClient.test.ts" \+ "src/llm/__tests__/subscriptionRequestHash.test.ts" \"src/llm/__tests__/providerAttemptAuthorizer.test.ts" \"src/llm/hostLlmBinding.test.ts" \"src/config/configStore.test.ts" \@@ -291,9 +298,7 @@ run_group "workflow-recipes" "workflow-recipes" \"src/workflow/llmAllowedModelsSnapshot.test.ts" \"src/workflow/networkPolicyFactory.codex.test.ts" \"src/workflow/sdkOnlyCodexBinding.test.ts" \- "src/workflow/sdkOnlyGrokBinding.test.ts" \"src/workflow/pluginWorkloadSdkProvisioner.codexPolicy.test.ts" \- "src/workflow/pluginWorkloadSdkProvisioner.grokPolicy.test.ts" \"src/reconciler/pluginWorkloadSdkValidator.test.ts" \"tests/unit/workflow/modelConfigHandler.test.ts" \"tests/unit/workflow/modelConfigHandler.pluginSdkBroker.test.ts"diff --git a/scripts/tests/test-grok-subscription-t0.sh b/scripts/tests/test-grok-subscription-t0.shindex cc2f526fd..cf43e81c4 100755--- a/scripts/tests/test-grok-subscription-t0.sh+++ b/scripts/tests/test-grok-subscription-t0.sh@@ -1,12 +1,23 @@#!/usr/bin/env bash# Wave-2 Grok subscription T0 aggregator.+## Fails when a required suite is missing, executes zero tests, reports-# skipped/todo cases, or exits non-zero.+# skipped/todo cases, runs fewer Vitest files than listed, or exits non-zero.+# Exit 0 alone is never enough: the script parses machine-readable counts from+# Vitest or node:test. Every listed suite is guarded by require_file before its+# group runs, because `vitest run <present> <missing>` still exits 0.+#+# Lane separation: Codex-only suites live in test-codex-subscription-t0.sh.+# Real-Postgres suites need CONTROL_API_REAL_PG_ADMIN_URL and run in the CI+# real-PG lane; this script checks they exist and that the lane lists them.set -uROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"FAIL=0GROUPS_RUN=0+EXPECTED_GROUPS=9+REGISTERED=()+COUNT_SUMMARY=""pass() { echo "PASS: $1"; }fail() { echo "FAIL: $1"; FAIL=1; }@@ -24,20 +35,32 @@ strip_ansi() {sed $'s/\033\\[[0-9;]*m//g'}+# Return 0 when the captured reporter proves at least one passing test, zero+# skipped/todo cases and, for Vitest, exactly the expected number of files.+# Sets COUNT_SUMMARY on success.assert_executed_counts() {local name="$1"local log="$2"+ local expected_files="$3"local normalizednormalized="$(mktemp)"strip_ansi <"$log" >"$normalized"local vitest_passedvitest_passed="$(- grep -Eo 'Tests[[:space:]]+[1-9][0-9]* passed' "$normalized" | head -n 1 || true+ grep -Eo 'Tests[[:space:]]+[1-9][0-9]* passed' "$normalized" | head -n 1 | awk '{print $2}' || true+ )"+ local vitest_files+ vitest_files="$(+ grep -Eo 'Test Files[[:space:]]+[0-9]+ passed' "$normalized" | head -n 1 | awk '{print $3}' || true+ )"+ local node_tests+ node_tests="$(+ grep -E '^(#|ℹ) tests[[:space:]]+[0-9]+$' "$normalized" | awk '{print $NF}' | tail -n 1 || true)"local node_passnode_pass="$(- grep -E 'pass[[:space:]]+[1-9][0-9]*$' "$normalized" | awk '{print $NF}' | tail -n 1 || true+ grep -E '^(#|ℹ) pass[[:space:]]+[1-9][0-9]*$' "$normalized" | awk '{print $NF}' | tail -n 1 || true)"if [[ -z "${vitest_passed}" && -z "${node_pass}" ]]; then@@ -48,6 +71,19 @@ assert_executed_counts() {return 1fi+ if [[ -n "${node_pass}" && "${node_tests}" != "${node_pass}" ]]; then+ fail "${name}: node:test pass count ${node_pass} does not match tests ${node_tests:-missing}"+ rm -f "$normalized"+ return 1+ fi++ if [[ -n "${vitest_passed}" && "${vitest_files}" != "${expected_files}" ]]; then+ fail "${name}: Vitest passed ${vitest_files:-0} test files, expected ${expected_files}"+ grep -E 'Test Files|No test files found' "$normalized" || true+ rm -f "$normalized"+ return 1+ fi+if grep -Eq '(Test Files|Tests).*[1-9][0-9]* skipped' "$normalized"; thenfail "${name}: Vitest reported skipped tests"rm -f "$normalized"@@ -58,6 +94,11 @@ assert_executed_counts() {rm -f "$normalized"return 1fi+ if grep -Eq '^(#|ℹ) skipped[[:space:]]+[1-9]' "$normalized"; then+ fail "${name}: node:test reported skipped tests"+ rm -f "$normalized"+ return 1+ fiif grep -Eq '^(#|ℹ) todo[[:space:]]+[1-9]' "$normalized"; thenfail "${name}: node:test reported todo tests"rm -f "$normalized"@@ -69,123 +110,262 @@ assert_executed_counts() {return 1fi+ if [[ -n "${vitest_passed}" ]]; then+ COUNT_SUMMARY="${vitest_files} files, ${vitest_passed} tests"+ else+ COUNT_SUMMARY="${node_pass}/${node_tests} node:test tests"+ firm -f "$normalized"return 0}+# run_group <name> <package dir> <suite files relative to the package...>+# Runs the package's own `npm test -- <files>`; the grok-llm-proxy test script+# runs its test typecheck before Vitest.run_group() {local name="$1"- shift- GROUPS_RUN=$((GROUPS_RUN + 1))+ local prefix="$2"+ shift 2+ if [[ "$#" -eq 0 ]]; then+ fail "${name}: group listed no suite files"+ return 1+ fi+ local files=("$@")+ local rel+ for rel in "${files[@]}"; do+ REGISTERED+=("${prefix}/${rel}")+ done+ for rel in "${files[@]}"; do+ require_file "${prefix}/${rel}" || return 1+ done+local loglog="$(mktemp)"- if "$@" >"$log" 2>&1; then- if assert_executed_counts "$name" "$log"; then- pass "$name"- fi- else- fail "$name"- echo "----- ${name} -----"+ local npm_args=(--prefix "${ROOT}/${prefix}" test --)+ npm_args+=("${files[@]}")++ echo "── ${name} ──"+ if ! (+ cd "${ROOT}"+ FORCE_COLOR=0 NO_COLOR=1 npm "${npm_args[@]}"+ ) >"$log" 2>&1; then+ fail "${name}: command failed"cat "$log"+ rm -f "$log"+ return 1+ fi++ COUNT_SUMMARY=""+ if ! assert_executed_counts "$name" "$log" "${#files[@]}"; then+ rm -f "$log"+ return 1fi++ GROUPS_RUN=$((GROUPS_RUN + 1))+ pass "${name} (${COUNT_SUMMARY})"rm -f "$log"}-require_file packages/grok-provider-attempt-contract/index.test.cjs-require_file grok-llm-proxy/test/contractFreeze.test.ts-require_file grok-llm-proxy/test/originPolicy.test.ts-require_file control-api/test/services.grokSubscriptionOAuth.test.ts-require_file control-api/test/services.grokProviderAttemptTicket.test.ts-require_file control-api/test/services.grokSubscriptionConnection.realPostgres.integration.test.ts-require_file control-api/test/pluginWorkloadSdkGrokDualLedger.realPostgres.integration.test.ts-require_file control-api/test/services.grokProviderAttemptRedemption.realPostgres.integration.test.ts-require_file control-api/test/services.grokProviderAttemptRedemption.test.ts-require_file control-api/test/db.grokSubscriptionMigration.test.ts-require_file control-api/test/services.llmProviderAttemptAuthorizer.grok.test.ts-require_file control-api/test/subscriptionGrantIdentity.test.ts-require_file control-api/test/routes.adminPluginWorkloadSdk.test.ts-require_file control-api/test/services.recipeCodexGrantIdentity.test.ts-require_file control-api/test/hostSpecValidation.grokSubscription.test.ts-require_file control-api/test/crd.llmProviderEnums.test.ts-require_file control-api/test/routes.mcp-host.plugin-workload-sdk.test.ts-require_file mcp-host/src/pluginWorkloadSdk/promptBridge/controlApiClient.test.ts-require_file mcp-host/src/pluginWorkloadSdk/promptBridge/llmBridge.failover.test.ts-require_file control-ui/components/__tests__/HostWizard.test.tsx-require_file control-ui/components/__tests__/HostDetailsPage.identity.test.tsx-require_file scripts/tests/test-grok-llm-proxy-deploy-contract.sh-require_file tests/e2e/fixtures/grok-subscription/sanitized-upstream-contract.json-require_file workflow-recipes/src/workflow/codexRecipeVerdict.test.ts-require_file workflow-recipes/src/workflow/sdkOnlyGrokBinding.test.ts-require_file workflow-recipes/src/workflow/pluginWorkloadSdkProvisioner.grokPolicy.test.ts-require_file workflow-recipes/tests/unit/workflow/modelConfigHandler.pluginSdkBroker.test.ts-require_file mcp-host/src/pluginWorkloadSdk/bootstrapIdentity.test.ts-require_file packages/codex-catalog-projection/index.test.cjs-require_file control-api/test/services.usageEvents.test.ts-require_file control-api/test/services.llmProviderAttemptFinalization.test.ts-require_file workflow-recipes/tests/unit/workflow/podFactory.test.ts-require_file control-ui/components/__tests__/RecipeEditor.test.tsx-require_file control-ui/lib/__tests__/llm.test.ts-require_file control-ui/components/__tests__/PluginWorkloadSdkPage.test.tsx+require_ci_matrix_entry() {+ local entry="$1"+ if ! grep -Eq "^[[:space:]]+- ${entry}$" "${ROOT}/.github/workflows/ci-public.yml"; then+ fail "ci-public.yml matrix is missing ${entry}"+ return 1+ fi+ pass "ci-public.yml matrix includes ${entry}"+}++# Real-Postgres suites are env-gated (skipped without a real PG), so T0 only+# proves they exist and that the CI real-PG lane asserts each one ran.+require_real_pg_suite() {+ local rel="$1"+ REGISTERED+=("${rel}")+ require_file "${rel}" || return 1+ local suite+ suite="$(basename "${rel}" .test.ts)"+ if ! sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*\\$//' "${ROOT}/.github/workflows/ci-public.yml" |+ grep -Fxq "${suite}"; then+ fail "ci-public.yml real-PG lane does not list ${suite}"+ return 1+ fi+ pass "real-PG suite present and listed in CI: ${suite}"+}++echo "Grok subscription T0 aggregator"+require_ci_matrix_entry "grok-llm-proxy"+require_ci_matrix_entry "packages/grok-provider-attempt-contract"++oauth_source="${ROOT}/control-api/src/services/grokSubscriptionOAuth.ts"expected_device_url='https://auth.x.ai/oauth2/device/code'expected_token_url='https://auth.x.ai/oauth2/token'+expected_revoke_url='https://auth.x.ai/oauth2/revoke'expected_client_id='b1a00492-073a-47ea-816f-4c329264a828'-api_device_url="$(- sed -n "s/^export const GROK_OAUTH_DEVICE_URL = '\\(.*\\)'$/\\1/p" \- "${ROOT}/control-api/src/services/grokSubscriptionOAuth.ts"-)"-api_token_url="$(- sed -n "s/^export const GROK_OAUTH_TOKEN_URL = '\\(.*\\)'$/\\1/p" \- "${ROOT}/control-api/src/services/grokSubscriptionOAuth.ts"-)"+api_device_url="$(sed -n "s/^export const GROK_OAUTH_DEVICE_URL = '\\(.*\\)'$/\\1/p" "${oauth_source}")"+api_token_url="$(sed -n "s/^export const GROK_OAUTH_TOKEN_URL = '\\(.*\\)'$/\\1/p" "${oauth_source}")"+api_revoke_url="$(sed -n "s/^export const GROK_OAUTH_REVOKE_URL = '\\(.*\\)'$/\\1/p" "${oauth_source}")"api_client_id="$(sed -n "s/.*CONTROL_API_GROK_OAUTH_CLIENT_ID || '\\(.*\\)',$/\\1/p" \"${ROOT}/control-api/src/config.ts")"if [[ "${api_device_url}" == "${expected_device_url}" &&"${api_token_url}" == "${expected_token_url}" &&+ "${api_revoke_url}" == "${expected_revoke_url}" &&"${api_client_id}" == "${expected_client_id}" ]]; thenpass "Grok OAuth origin/client lock"else- fail "Grok OAuth origin/client lock (device=${api_device_url:-missing} token=${api_token_url:-missing} client=${api_client_id:-missing})"+ fail "Grok OAuth origin/client lock (device=${api_device_url:-missing} token=${api_token_url:-missing} revoke=${api_revoke_url:-missing} client=${api_client_id:-missing})"fi-run_group "grok-provider-attempt-contract" \- bash -lc "cd '${ROOT}/packages/grok-provider-attempt-contract' && node --test index.test.cjs"+run_group "grok-provider-attempt-contract" "packages/grok-provider-attempt-contract" \+ "index.test.cjs"-run_group "grok-catalog-projection" \- bash -lc "cd '${ROOT}/packages/codex-catalog-projection' && node --test index.test.cjs"+# Shared projection package; its suite covers the Grok ConfigMap annotations.+run_group "grok-catalog-projection" "packages/codex-catalog-projection" \+ "index.test.cjs"-run_group "grok-llm-proxy freeze+origin" \- bash -lc "cd '${ROOT}/grok-llm-proxy' && npx vitest run test/contractFreeze.test.ts test/originPolicy.test.ts test/grokTransport.conformance.test.ts --no-file-parallelism"+# Every proxy suite, including the hermetic runtime e2e. The unlisted-suite+# check below fails when a proxy test file is added without being listed here.+run_group "grok-llm-proxy" "grok-llm-proxy" \+ "test/abortWhenClientDisconnects.test.ts" \+ "test/approvedToolsUpstream.test.ts" \+ "test/catalogBounds.test.ts" \+ "test/contractFreeze.test.ts" \+ "test/controlApiClient.test.ts" \+ "test/grokTransport.conformance.test.ts" \+ "test/grokUpstreamHeaders.test.ts" \+ "test/originPolicy.test.ts" \+ "test/redaction.test.ts" \+ "test/requestLimits.test.ts" \+ "test/runtimePath.hermetic.e2e.test.ts" \+ "test/server.security.test.ts" \+ "test/sseBackpressure.test.ts" \+ "test/toolNameMap.test.ts"-run_group "grok-llm-proxy tsc" \- bash -lc "cd '${ROOT}/grok-llm-proxy' && npx tsc --noEmit && echo 'pass 1'"+run_group "control-api grok" "control-api" \+ "test/db.grokSubscriptionMigration.test.ts" \+ "test/db.llmProviderAttemptMigration.test.ts" \+ "test/services.grokSubscriptionConnection.test.ts" \+ "test/services.grokSubscriptionOAuth.test.ts" \+ "test/services.grokSubscriptionCatalog.test.ts" \+ "test/services.subscriptionCatalogBounds.test.ts" \+ "test/services.grokProviderAttemptTicket.test.ts" \+ "test/services.grokProviderAttemptRedemption.test.ts" \+ "test/services.llmProviderAttemptAuthorizer.grok.test.ts" \+ "test/services.llmProviderAttemptAuthorizer.depth.test.ts" \+ "test/services.llmProviderAttemptFinalization.test.ts" \+ "test/services.usageEvents.test.ts" \+ "test/services.recipeGrantTransition.test.ts" \+ "test/services.recipeCodexGrantIdentity.test.ts" \+ "test/subscriptionGrantIdentity.test.ts" \+ "test/llmProviders.test.ts" \+ "test/routes.admin.grokSubscription.test.ts" \+ "test/routes.internal.llmProviderAttempts.grok.test.ts" \+ "test/routes.adminPluginWorkloadSdk.test.ts" \+ "test/routes.adminRecipes.test.ts" \+ "test/routes.mcp-host.plugin-workload-sdk.test.ts" \+ "test/hostSpecValidation.grokSubscription.test.ts" \+ "test/hostSpecValidation.limits.test.ts" \+ "test/crd.hostCel.test.ts" \+ "test/crd.llmProviderEnums.test.ts"-run_group "control-api grok grant/oauth/redeem/authorize" \- bash -lc "cd '${ROOT}/control-api' && npx vitest run test/db.grokSubscriptionMigration.test.ts test/services.grokSubscriptionConnection.test.ts test/services.grokSubscriptionOAuth.test.ts test/services.grokSubscriptionCatalog.test.ts test/services.grokProviderAttemptRedemption.test.ts test/services.grokProviderAttemptTicket.test.ts test/services.llmProviderAttemptAuthorizer.grok.test.ts test/subscriptionGrantIdentity.test.ts test/routes.adminPluginWorkloadSdk.test.ts test/services.recipeCodexGrantIdentity.test.ts test/hostSpecValidation.grokSubscription.test.ts test/crd.llmProviderEnums.test.ts test/routes.mcp-host.plugin-workload-sdk.test.ts test/services.usageEvents.test.ts test/services.llmProviderAttemptFinalization.test.ts --no-file-parallelism"+require_real_pg_suite "control-api/test/services.grokSubscriptionConnection.realPostgres.integration.test.ts"+require_real_pg_suite "control-api/test/services.grokProviderAttemptRedemption.realPostgres.integration.test.ts"+require_real_pg_suite "control-api/test/services.grokProviderAttemptRedemption.refresh.realPostgres.integration.test.ts"+require_real_pg_suite "control-api/test/pluginWorkloadSdkGrokDualLedger.realPostgres.integration.test.ts"+require_real_pg_suite "control-api/test/db.llmProviderAttemptConnectionIntegrity.realPostgres.integration.test.ts"-run_group "workflow-recipes grok SDK" \- bash -lc "cd '${ROOT}/workflow-recipes' && npx vitest run src/workflow/codexRecipeVerdict.test.ts src/workflow/sdkOnlyGrokBinding.test.ts src/workflow/pluginWorkloadSdkProvisioner.grokPolicy.test.ts src/reconciler/pluginWorkloadSdkValidator.test.ts tests/unit/workflow/modelConfigHandler.pluginSdkBroker.test.ts tests/unit/workflow/podFactory.test.ts --no-file-parallelism"+run_group "mcp-host grok" "mcp-host" \+ "src/llm/__tests__/grokSubscription.test.ts" \+ "src/llm/__tests__/grokLlmProxyClient.test.ts" \+ "src/llm/__tests__/grokPolicyBinding.test.ts" \+ "src/llm/__tests__/subscriptionRequestHash.test.ts" \+ "src/llm/__tests__/registry.test.ts" \+ "src/llm/__tests__/makeProvider.throws.test.ts" \+ "src/config/configStore.test.ts" \+ "src/core/orchestration/__tests__/toolPresentationPolicy.test.ts" \+ "src/pluginWorkloadSdk/bootstrapIdentity.test.ts" \+ "src/pluginWorkloadSdk/bootstrapServer.test.ts" \+ "src/pluginWorkloadSdk/promptBridge/controlApiClient.test.ts" \+ "src/pluginWorkloadSdk/promptBridge/llmBridge.failover.test.ts"-run_group "mcp-host grok bootstrap" \- bash -lc "cd '${ROOT}/mcp-host' && npx vitest run src/pluginWorkloadSdk/bootstrapIdentity.test.ts src/pluginWorkloadSdk/promptBridge/controlApiClient.test.ts src/pluginWorkloadSdk/promptBridge/llmBridge.failover.test.ts --no-file-parallelism"+run_group "workflow-recipes grok" "workflow-recipes" \+ "src/workflow/codexRecipeVerdict.test.ts" \+ "src/workflow/sdkOnlyGrokBinding.test.ts" \+ "src/workflow/pluginWorkloadSdkProvisioner.grokPolicy.test.ts" \+ "src/workflow/workflowReconciler.grokWrcFlag.test.ts" \+ "src/reconciler/workflowRecipeReconciler.grokFlagWiring.test.ts" \+ "src/reconciler/pluginWorkloadSdkValidator.test.ts" \+ "src/mcp/server.pluginSdkBroker.test.ts" \+ "tests/unit/workflow/modelConfigHandler.test.ts" \+ "tests/unit/workflow/modelConfigHandler.pluginSdkBroker.test.ts" \+ "tests/unit/workflow/podFactory.test.ts"-run_group "control-ui grok SDK picker" \- bash -lc "cd '${ROOT}/control-ui' && npx vitest run components/__tests__/PluginWorkloadSdkPage.test.tsx components/__tests__/HostWizard.test.tsx components/__tests__/HostDetailsPage.identity.test.tsx components/__tests__/RecipeEditor.test.tsx lib/__tests__/llm.test.ts --no-file-parallelism"+run_group "host-context-controller grok" "host-context-controller" \+ "src/hostReconciler.grokHostRuntime.test.ts" \+ "test/hostReconciler.test.ts"-if ! bash "${ROOT}/scripts/tests/test-grok-llm-proxy-deploy-contract.sh"; then- fail "grok-llm-proxy deploy contract"-else- GROUPS_RUN=$((GROUPS_RUN + 1))- pass "grok-llm-proxy deploy contract"+run_group "control-ui grok" "control-ui" \+ "components/__tests__/CodexSubscriptionHub.test.tsx" \+ "components/__tests__/LlmCatalogFormsGrokGate.test.tsx" \+ "components/__tests__/RecipeEditor.test.tsx" \+ "components/__tests__/HostWizard.test.tsx" \+ "components/__tests__/HostDetailsPage.identity.test.tsx" \+ "components/__tests__/PluginWorkloadSdkPage.test.tsx" \+ "components/__tests__/LlmProviderConfig.test.tsx" \+ "components/__tests__/LlmPolicyEditor.test.tsx" \+ "components/__tests__/LlmModelForm.test.tsx" \+ "lib/__tests__/grokSubscriptionFeature.test.ts" \+ "lib/__tests__/llm.test.ts" \+ "lib/hooks/__tests__/useGrokSubscriptionEnabled.test.tsx"++echo "── grok-llm-proxy deploy contract ──"+if require_file "scripts/tests/test-grok-llm-proxy-deploy-contract.sh" &&+ require_file "tests/e2e/fixtures/grok-subscription/sanitized-upstream-contract.json"; then+ if bash "${ROOT}/scripts/tests/test-grok-llm-proxy-deploy-contract.sh"; then+ GROUPS_RUN=$((GROUPS_RUN + 1))+ pass "grok-llm-proxy deploy contract"+ else+ fail "grok-llm-proxy deploy contract"+ fi+fi++# A Grok suite that exists but is not listed above is lost coverage. Every+# proxy/contract test file and every *grok* test file in the Grok-touching+# packages must be registered in a group or as a real-PG presence check.+is_registered() {+ local candidate="$1" entry+ for entry in "${REGISTERED[@]}"; do+ [[ "${entry}" == "${candidate}" ]] && return 0+ done+ return 1+}+unlisted=0+while IFS= read -r rel; do+ [[ -n "${rel}" ]] || continue+ if ! is_registered "${rel}"; then+ fail "unlisted Grok suite ${rel}"+ unlisted=1+ fi+done < <(+ cd "${ROOT}" &&+ {+ find grok-llm-proxy/test packages/grok-provider-attempt-contract \+ -name node_modules -prune -o -type f \( -name '*.test.ts' -o -name '*.test.cjs' \) -print+ find control-api mcp-host workflow-recipes host-context-controller control-ui \+ \( -name node_modules -o -name dist -o -name .next -o -name coverage \) -prune -o \+ -type f -iname '*grok*' \( -name '*.test.ts' -o -name '*.test.tsx' \) -print+ } | sort -u+)+if [[ "${unlisted}" -eq 0 ]]; then+ pass "every Grok suite is registered in T0"fi-if [[ "$GROUPS_RUN" -lt 9 ]]; then- fail "T0 ran too few groups (${GROUPS_RUN})"+if [[ "${GROUPS_RUN}" -ne "${EXPECTED_GROUPS}" ]]; then+ fail "expected all ${EXPECTED_GROUPS} T0 groups, ran ${GROUPS_RUN}"fi-if [[ "$FAIL" -ne 0 ]]; then+if [[ "${FAIL}" -ne 0 ]]; thenecho "FAIL: grok subscription T0"exit 1fi