Skip to content

Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure

TypeScript

test(minikube): harden branch-owned gate harness

Alfredo Lopezcaeffae474e4
diff --git a/.cursor/skills/minikube-t0-t1-t2/SKILL.md b/.cursor/skills/minikube-t0-t1-t2/SKILL.md
index a089af573..d8346d6c6 100644
--- a/.cursor/skills/minikube-t0-t1-t2/SKILL.md
+++ b/.cursor/skills/minikube-t0-t1-t2/SKILL.md
@@ -22,17 +22,27 @@ Before running anything, verify ALL of these:
branch).
- [ ] Identify the branch-owned `MINIKUBE_PROFILE` for THIS worktree. Reuse it.
Do NOT create a new profile because HEAD, gate, or command changed.
+ Resolve it with the primary checkout `.local-notes/minikube-profiles/branch.mk`;
+ profile identity is stable for canonical worktree + branch, while the
+ pre-gate marker—not a profile-name SHA—proves exact-HEAD freshness.
- [ ] Confirm the profile is not owned by another active branch/worktree. If
ownership is ambiguous, stop and use a dedicated profile instead.
- [ ] Every `kubectl` you run manually uses `--context=<owned profile>`.
Never change the global kubectl current-context.
+- [ ] Docker resolves to a local Unix socket or loopback TCP endpoint. The
+ harness pins that endpoint into an empty task-local config; do not use a
+ remote Docker context or copy ambient registry credentials into it.
+- [ ] Mutating image builds use the public Make target/orchestrator and inherit
+ its exact profile lease. Do not call `build-images.sh` directly;
+ `--verify-only` is the read-only exception.
- [ ] Never read `~/.cache/clerum/minikube-profiles/` directly (HARD DENY —
it holds private profile state). The harness reads it for you.
## Step 1 — Plan (read-only)
```bash
-make minikube-t2-preflight MINIKUBE_PROFILE=<owned-profile>
+MINIKUBE_PROFILE=<owned-profile> CONTROL_API_REAL_PG_CONTEXT=<owned-profile> \
+ make minikube-t2-preflight
```
This is a planner, NOT a lane. `T2_PREFLIGHT_PASS` is NOT a T2 verdict.
@@ -44,9 +54,11 @@ expected, not an error to work around.
```text
transition = already-synced AND T0+T1 already green on this exact HEAD?
- └── yes → make minikube-t2-runtime MINIKUBE_PROFILE=<owned-profile>
+ └── yes → MINIKUBE_PROFILE=<owned-profile> CONTROL_API_REAL_PG_CONTEXT=<owned-profile>
+ make minikube-t2-runtime
(T2-only close; refused unless marker matches HEAD)
- └── no → make minikube-t2 MINIKUBE_PROFILE=<owned-profile>
+ └── no → MINIKUBE_PROFILE=<owned-profile> CONTROL_API_REAL_PG_CONTEXT=<owned-profile>
+ make minikube-t2
(full: T0 → bootstrap/reconcile/targeted-sync → T1 → T2 verdict)
transition = full-bootstrap (fresh/uninitialized profile)?
@@ -79,12 +91,22 @@ Rules that override any shortcut idea:
## Step 3 — T1 specifics (Real PostgreSQL)
T1 runs inside `make minikube-t2`, or explicitly via
-`make minikube-t2-real-postgres MINIKUBE_PROFILE=<owned-profile>`.
-
+`MINIKUBE_PROFILE=<owned-profile> CONTROL_API_REAL_PG_CONTEXT=<owned-profile>
+make minikube-t2-real-postgres`.
+
+- The orchestrator runs a fast Node/package/Docker preflight before T0. Fix
+ that first instead of paying T0/bootstrap cost for a missing local dependency.
+ Docker probes, pulls, builds, image loads, Minikube metadata reads, and the
+ targeted user-facing health command are bounded and terminate their process
+ groups on timeout or interrupt.
+- T1 is intentionally serial (`VITEST_MAX_WORKERS=1`, no file parallelism).
+ The Real PostgreSQL fixtures and cluster-global roles make wider concurrency
+ unsafe, not an optimization.
- T1 is fail-loud: an unavailable DSN, an isolated server that did not start,
or zero executed tests is FAIL. Never report green from skipped suites. The
- JSON reporter must be complete and green, and the Vitest process must also
- exit zero; a green reporter cannot hide teardown, worker, OOM, or signal
+ JSON reporter must be complete and green, must identify exactly every
+ selected physical file, and the Vitest process must also exit zero; a green
+ reporter cannot hide teardown, worker, OOM, signal, or partial-selection
failure.
- Role-reset suites (cluster-global role drop/rewrite) run against a
throwaway `postgres:16-alpine`, never the shared `control-postgres`.
@@ -112,14 +134,17 @@ MINIKUBE_T2_PASS
T0=PASS|SKIPPED
T1=PASS|SKIPPED
T2=PASS
+NP08_HCC_AUTHORIZATION=PASS
+Health=PASS|NOT_RUN
Playwright=PASS|NOT_RUN
evidence=<path under .local-notes/infra/runs/>
```
Report to the user exactly these lane statuses plus HEAD, profile, and the
evidence path. `SKIPPED` is legitimate only for T0/T1 previously green on the
-same HEAD (say so explicitly). Optional user-facing checks are opt-in:
-`T2_HEALTHCHECK_COMMAND`, `T2_PLAYWRIGHT_COMMAND`
+same HEAD (say so explicitly). `T2_HEALTHCHECK_COMMAND` is mandatory for
+`targeted-sync` and bounded by `T2_HEALTHCHECK_TIMEOUT_SECONDS` (default 120s);
+it is optional for other transitions. `T2_PLAYWRIGHT_COMMAND` remains opt-in
(`T2_REQUIRE_PLAYWRIGHT=true` refuses a missing journey).
Evidence stays under the ignored `.local-notes/infra/runs/`. Never commit it.
@@ -128,6 +153,11 @@ Before committing anything else, run `make minikube-t2-public-boundary`.
## Step 5 — On failure
Failures print a stable code and a next safe command. Repair the FIRST
-reported precondition and re-run the same entry point on the same HEAD.
+reported precondition on the same HEAD. While debugging T1, use the standalone
+Real PostgreSQL target; finish with one full certification run. If T0/T1 lane
+evidence is already green and NP08, health, or Playwright fails, retry with
+`minikube-t2-runtime` on the same profile instead of repeating T0/T1. NP08 may
+observe a newer same-binding access token but must never refresh/reissue or
+consume the Host refresh-token lineage.
Do not widen the command, switch clusters, reset PVCs, or delete locks with a
live owner. Code-by-code guidance is in `reference.md`.
diff --git a/.cursor/skills/minikube-t0-t1-t2/reference.md b/.cursor/skills/minikube-t0-t1-t2/reference.md
index a1f8a9b1d..378eb3086 100644
--- a/.cursor/skills/minikube-t0-t1-t2/reference.md
+++ b/.cursor/skills/minikube-t0-t1-t2/reference.md
@@ -13,6 +13,9 @@ Companion to `SKILL.md`. Source of truth: `scripts/minikube/t2.sh`,
product E2E lane (e.g. issue #391 readiness), a separate evidence lane.
- **Do not re-run full `make minikube-t2` to "close T2"** when T0 and T1 are
already green on the same HEAD — use `make minikube-t2-runtime`.
+- **Do not derive a new profile from the current HEAD.** Resolve the profile
+ through the primary checkout helper. Worktree + branch own profile identity;
+ the exact-head marker owns deployed freshness and `ports.env` is persistent.
- **Do not `ls`/`cat` `~/.cache/clerum/minikube-profiles/`.** Private profile
state (ports, pids, markers). The harness is the only reader.
- **Do not invent `ADMIN_PASSWORD`** or any credential. When a product E2E
@@ -22,34 +25,60 @@ Companion to `SKILL.md`. Source of truth: `scripts/minikube/t2.sh`,
fixed localhost ports** — only the profile-owned random port mapping.
- **Do not touch another worktree's Minikube profile** (e.g. an NP-08 lane
profile) or kill its port-forwards, even if it looks idle.
+- **Do not run a mutating `build-images.sh` directly.** Use the public Make
+ target or T2 orchestrator so the exact worktree/profile/context lease is
+ inherited. Only `--verify-only` is lease-exempt.
+- **Do not use a remote/ambiguous Docker endpoint or inherit ambient registry
+ auth.** The harness accepts local Unix sockets and loopback TCP endpoints,
+ pins the resolved endpoint before isolation, and requires an explicit
+ `MINIKUBE_DOCKER_AUTH_CONFIG` for private pulls.
- **Do not pack `kubectl` + `port-forward` + profile name into one wrapper
argv** when avoidable. `t2_process_check` accepts only real `kubectl`
- processes (`comm=kubectl`) whose PIDs are recorded in the profile cache or
- legacy `/tmp/pf-<profile>-*.pid`; a lookalike wrapper trips
- `PORT_FORWARD_CONFLICT`. The awk pre-filter matches `kubectl` as an argv0
- or path token plus a later standalone `port-forward` token; wrappers that
- only mention those words are rejected by `comm`.
+ processes (`comm=kubectl`) with exactly one complete structured ownership
+ record in the profile cache; a lookalike wrapper or live legacy `/tmp`
+ pidfile trips `PORT_FORWARD_CONFLICT`. The awk pre-filter matches `kubectl`
+ as an argv0 or path token plus a later standalone `port-forward` token;
+ wrappers that only mention those words are rejected by `comm`.
- **Do not commit** `.local-notes/`, lockfiles produced by an incidental
`npm ci`, generated ports/profile metadata, or anything secret-like. Run
`make minikube-t2-public-boundary` first.
- **Do not weaken fail-loud T1**: no green on unavailable DSN, zero executed
tests, or silently skipped suites; the JSON reporter must be complete and
- green, and the Vitest process must also exit zero.
+ green, must name the exact physical file set, and the Vitest process must
+ also exit zero. Do not override its one-worker/no-file-parallelism contract.
Role-reset suites stay on the throwaway `postgres:16-alpine` (#412), never
shared `control-postgres`.
+- **Do not let NP08 refresh or reissue the Host lineage.** It may observe a
+ fresher persisted access token only when its Host/recipe binding exactly
+ matches the mounted token. Refresh tokens remain owned by mcp-host.
## Stable failure codes
| Code | Meaning | Safe next step |
| --- | --- | --- |
-| `BOOTSTRAP_REQUIRED` | Profile missing/uninitialized, or the planner produced no transition. Standalone preflight refuses to bootstrap. | Run `make minikube-t2` — its internal planner (`T2_PLAN_MODE=true`) makes `full-bootstrap` reachable and runs the supported setup with `IMAGE_SOURCE=local`. |
-| `HEAD_MARKER_MISMATCH` | Pre-gate marker does not match this worktree/HEAD; the final T2 preflight selected something other than `already-synced`. | Re-run `make minikube-t2` on this HEAD so `pre-gate-sync` updates the marker. Never hand-edit the marker. |
+| `LOCAL_DEPENDENCY_MISSING` | Node/package-local dependencies, Python, Docker, kubectl, Minikube, or another required local tool is unavailable. | Repair the named local prerequisite first. The T1 preflight runs before expensive T0 work and never auto-installs. |
+| `UNSUPPORTED_T1_CONCURRENCY` | A caller attempted more than one T1 worker. | Remove the override or set `VITEST_MAX_WORKERS=1`; T1 role/fixture mutation is serial by contract. |
+| `PROFILE_OWNERSHIP_MISMATCH` | Profile metadata, context, worktree, or branch does not match this lane. | Resolve through the canonical helper. Never adopt, kill, or mutate a foreign/ambiguous profile. |
+| `PROFILE_LOCK_REQUIRED` | A mutating child did not inherit the exact parent lease. | Invoke the public Make target/orchestrator; do not fabricate or bypass its lock token. |
+| `DOCKER_ENDPOINT_REQUIRED` / `DOCKER_ENDPOINT_UNSAFE` / `DOCKER_ENDPOINT_MISMATCH` | Docker did not resolve exactly one approved local endpoint, or isolation selected a different endpoint. | Select a local Docker Desktop/rootless context or set an explicit local `DOCKER_HOST`; do not weaken the endpoint check. |
+| `DOCKER_CONFIG_REQUIRED` / `REGISTRY_AUTH_REQUIRED` | A Docker operation lacks its empty task-local config, or a private pull lacks an explicit readable auth config. | Use the canonical Make/orchestrator path; for a genuinely private source, pass only the intended config directory through `MINIKUBE_DOCKER_AUTH_CONFIG`. |
+| `DOCKER_DEADLINE_INVALID` | A runtime deadline/retry value is malformed or exceeds its finite maximum. | Remove the override or set it within the documented range; do not disable the deadline. |
+| `BOOTSTRAP_REQUIRED` | Profile missing/uninitialized, or the planner produced no transition. Standalone preflight refuses to bootstrap. | Run `MINIKUBE_PROFILE=<owned> CONTROL_API_REAL_PG_CONTEXT=<owned> make minikube-t2`; its internal planner makes `full-bootstrap` reachable. |
+| `HEAD_MARKER_MISMATCH` | Pre-gate marker does not match this worktree/HEAD; the final T2 preflight selected something other than `already-synced`. | Re-run the full target on this HEAD so `pre-gate-sync` updates the marker. Never hand-edit the marker. |
+| `IMAGE_MANIFEST_MISMATCH` | Deployed image provenance does not match the exact marker/worktree state. | Re-run the full target for the same owned profile; do not relabel or hand-edit the manifest. |
| `PORT_FORWARD_CONFLICT` | A port-forward for this profile is owned by a process not recorded for it (or not a real `kubectl`). | Identify the foreign owner; if it belongs to another lane, stop. Restart forwards via `scripts/minikube/pf-all-stack.sh` for this profile only. |
| `PROFILE_BUSY` | Profile lock held. Live owner PID → genuinely busy. No valid owner PID → orphaned lock. | Live owner: wait or coordinate; never remove. Orphan: verify no T2 process owns the profile, then remove ONLY `$T2_LOCK_ROOT/<profile>.lock` and retry. Never remove the lock root. |
| `DEVELOPMENT_SCOPE_REQUIRED` | Preflight/final preflight failed a precondition, or T2-only mode was attempted without `already-synced`. | Repair the first reported condition; if T2-only was refused, run full `make minikube-t2`. |
+| `CERTIFICATION_REQUIRED` | Runtime-only was requested without valid exact-head T0/T1 evidence for the full ownership tuple. | Run the full target once; do not hand-create or copy evidence. |
+| `SECRET_MISSING` / `CONFIGMAP_MISSING` | A required named runtime input is absent or unreadable. | Reconcile the same owned profile through the full target; never invent values. |
| `ZERO_TESTS_EXECUTED` | A lane was configured off or executed nothing (including a required-but-missing Playwright journey). | Re-run with the lane enabled, or supply the required `T2_PLAYWRIGHT_COMMAND`. |
+| `REAL_PG_REQUIRED_BUT_UNAVAILABLE` | The explicit context, isolated PostgreSQL, DSN route, or exact-head database precondition is unavailable. | Use the standalone T1 target while repairing, then run one full certification. |
+| `REAL_PG_REPORT_INCOMPLETE` | Vitest JSON did not prove the exact expected physical files. | Repair test selection/reporter output; do not accept suite counters as file identity. |
+| `REAL_PG_SUITE_FAILED` | Reporter, test assertions, process exit, or parent result contract failed. | Iterate with standalone T1; once green, run full T2 for final exact-head lane evidence. |
| `POSTGRES_NOT_READY` | PostgreSQL precondition failed, or a PVC reset was requested without the exact expected UID. | Fix DB readiness; never guess a PVC UID. |
-| `PROFILE_UNHEALTHY` | A required deployment is unready in a fail-loud check (`T2_PLAN_MODE=false`: standalone preflight or the final exact-head T2), or the opt-in user-facing health command failed. The orchestrator planner never emits this for an unready deployment — it selects `full-reconcile` instead. | Re-run `make minikube-t2` on the same HEAD; the single run reconciles and restores GFS credentials itself. Do not run manual repair scripts between runs. |
+| `PROFILE_UNHEALTHY` | A required deployment is unready in a fail-loud check (`T2_PLAN_MODE=false`: standalone preflight or the final exact-head T2), the mandatory targeted-sync health command is missing, or a supplied user-facing health command failed or timed out. The orchestrator planner never emits this merely for an unready deployment — it selects `full-reconcile` instead. | For a targeted sync, supply the affected service's profile-owned `T2_HEALTHCHECK_COMMAND`; otherwise repair the reported health failure. Re-run on the same HEAD and profile. Do not run manual repair scripts between runs. |
+| `NP08_HCC_AUTHORIZATION_FAILED` | The deployed Host-to-HCC authorization journey or cleanup failed after T0/T1 passed. | Repair the first NP08 failure, then use `minikube-t2-runtime` on the exact same profile/HEAD; never refresh/reissue in the test. |
+| `PLAYWRIGHT_FAILED` | The opt-in user-visible journey failed after runtime and lane evidence passed. | Repair the journey and retry with `minikube-t2-runtime` on the same exact tuple. |
## GFS restore pointers (high level, no secrets)
diff --git a/AGENTS.md b/AGENTS.md
index a21b3f620..99ac28864 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -7,6 +7,15 @@ iterations. Do not create a new profile merely because the current commit,
gate, or test command changes. Preserve the verified profile by passing its
explicit `MINIKUBE_PROFILE` value into the next operation.
+Resolve that profile through the primary checkout
+`.local-notes/minikube-profiles/branch.mk`; do not derive a profile from the
+current `HEAD` or invent ports. Profile identity is stable for the canonical
+worktree path plus branch, while deployed freshness is owned by the exact
+`gitHead` + `worktreeId` + `clusterFingerprint` marker. A creation SHA in
+legacy profile metadata is historical, not a reason to allocate another
+profile. Reuse the persisted `ports.env` byte-for-byte; missing, corrupt, or
+ambiguous ownership metadata must fail closed instead of regenerating ports.
+
Before reusing a profile, record and compare all of the following:
- active worktree path, branch, `HEAD`, and `origin/dev`;
@@ -23,6 +32,20 @@ also required when the selected profile is missing or unhealthy. Never use
shared fixed localhost ports for branch-owned profiles; use the profile-owned
random port mapping already recorded for that profile.
+Port-forwards are owned by atomic `0600` records bound to the exact profile,
+context, canonical worktree, namespace, Service, local/remote ports, PID,
+process start time, and `kubectl` argv. Never kill or adopt a live legacy
+`/tmp/pf-*.pid` process: it lacks enough provenance and must fail closed.
+
+All image builds and targeted deploys must enter through the documented Make
+targets or the T2 orchestrator so they inherit the exact live profile mutation
+lease. Calling `scripts/minikube/build-images.sh` directly is unsupported for
+mutating work; `--verify-only` is the read-only exception. Docker and Minikube
+operations must use finite deadlines. Docker endpoint discovery must resolve
+to an explicit local Unix socket or loopback TCP endpoint before switching to
+an empty task-local Docker config. Ambient registry credentials are never
+copied; a private pull requires an explicit `MINIKUBE_DOCKER_AUTH_CONFIG`.
+
## Incremental local image updates
For a service-only change in an already healthy profile, rebuild and restart
@@ -31,7 +54,7 @@ only the affected workload. Prefer the existing targeted path:
```bash
MINIKUBE_PROFILE=<verified-profile> \
make minikube-deploy-service SVC=<image-selector> NS=<namespace> \
- MINIKUBE_DEPLOYMENT=<deployment>
+ DEPLOYMENT=<deployment>
```
Examples: `control-api` / `control-plane` / `control-api`,
@@ -68,12 +91,13 @@ runbook.
Use these canonical entry points:
```text
-make minikube-t2-preflight
-make minikube-t2
-make minikube-t2-runtime
+MINIKUBE_PROFILE=<verified-profile> CONTROL_API_REAL_PG_CONTEXT=<verified-context> make minikube-t2-preflight
+MINIKUBE_PROFILE=<verified-profile> CONTROL_API_REAL_PG_CONTEXT=<verified-context> make minikube-t2
+MINIKUBE_PROFILE=<verified-profile> CONTROL_API_REAL_PG_CONTEXT=<verified-context> make minikube-t2-runtime
```
-`make minikube-t2-preflight` is a read-only planner. It is not T0, T1, or T2.
+`make minikube-t2-preflight` is a cluster-read-only planner; it may write its
+ignored local lock/evidence metadata. It is not T0, T1, or T2.
With the default `T2_PLAN_MODE=false` it fails loud on an unbootstrapped
profile and does not call `pre-gate-sync`. `make minikube-t2` is the full
orchestrator: T0, the selected bootstrap/reconcile, T1, then T2. A T2 verdict
@@ -110,17 +134,30 @@ Real PostgreSQL suites are opt-in in the ordinary test
matrix, but a T1 run that requires them must fail when the database/DSN is
unavailable or when zero tests execute; a green run must never be produced by
silently skipping the suites. The JSON reporter must be complete and green,
-and the Vitest process must also exit zero; a green reporter cannot hide a
-teardown, worker, OOM, or signal failure.
+must identify the exact selected physical files, and the Vitest process must
+also exit zero; a green reporter cannot hide a teardown, worker, OOM, signal,
+or partial-selection failure. Run the local Node/package/Docker preflight
+before expensive T0 work. T1 is serial by safety contract
+(`VITEST_MAX_WORKERS=1`, no file parallelism); do not widen it for speed.
Suites that drop or rewrite cluster-global roles must use the harness throwaway
Postgres 16, never the shared `control-postgres`.
+NP08 observes the existing Host access-token lineage and may reread a newer,
+same-binding persisted access token. It must never consume a refresh token,
+call refresh/reissue, or weaken single-use rotation. After T0/T1 have emitted
+an exact-head lane attestation, failures in NP08, a user-facing health check, or
+Playwright retry through `minikube-t2-runtime`; T1 failures may use the
+standalone Real PostgreSQL target while iterating, followed by one full
+certification run once green.
+
T0 (static/unit/contract checks), T1 (real PostgreSQL), T2 (validated runtime),
CI, Control UI/Desktop Playwright, and product E2E scripts such as
`scripts/e2e/e2e-hcc-rollout-readiness.sh` are separate evidence lanes. One
-lane does not stand in for another. User-facing health and Playwright journeys
-are opt-in on T2 via `T2_HEALTHCHECK_COMMAND` and `T2_PLAYWRIGHT_COMMAND`
-(`T2_REQUIRE_PLAYWRIGHT=true` to refuse `NOT_RUN`). Private operational state,
+lane does not stand in for another. User-facing health is mandatory and bounded
+for a `targeted-sync` transition via `T2_HEALTHCHECK_COMMAND`; it remains
+opt-in for bootstrap, full reconcile, and already-synced runs. Playwright is
+opt-in via `T2_PLAYWRIGHT_COMMAND` (`T2_REQUIRE_PLAYWRIGHT=true` refuses
+`NOT_RUN`). Private operational state,
generated ports, profile metadata, logs, and evidence belong under the ignored
`.local-notes/infra/runs/` path and must never be committed.
diff --git a/Makefile b/Makefile
index 4e367b802..cd25bc0c8 100644
--- a/Makefile
+++ b/Makefile
@@ -135,17 +135,29 @@ SKIP_UIS ?= false
E2E_KUBECONTEXT ?= $(MINIKUBE_PROFILE)
KC := kubectl --context=$(MINIKUBE_PROFILE)
LOCAL_KUBE_CONTEXT ?=
-minikube_deployment = $(if $(filter mcp-host,$(1)),chatllm,$(1))
-MINIKUBE_DEPLOYMENT = $(or $(DEPLOYMENT),$(call minikube_deployment,$(SVC)))
+# Exact selectors with an established deployment route in scripts/minikube/dev.sh.
+# Keep this public Make boundary aligned with build-images.sh so typos fail
+# before a lock is acquired or any image/deployment mutation begins.
+MINIKUBE_DEPLOY_SERVICE_SELECTORS := control-api control-ui external-rest-api hcc mcp-host profile-ui rpc-proxy
+MINIKUBE_DEPLOY_SERVICE := $(strip $(SVC))
+MINIKUBE_DEPLOY_SERVICE_SUPPORTED := $(and $(filter 1,$(words $(MINIKUBE_DEPLOY_SERVICE))),$(filter $(MINIKUBE_DEPLOY_SERVICE_SELECTORS),$(MINIKUBE_DEPLOY_SERVICE)))
+MINIKUBE_DEPLOY_NAMESPACE := $(strip $(NS))
+minikube_deployment = $(if $(filter mcp-host,$(strip $(1))),chatllm,$(strip $(1)))
+DEPLOYMENT ?= $(call minikube_deployment,$(MINIKUBE_DEPLOY_SERVICE))
+MINIKUBE_EFFECTIVE_DEPLOYMENT := $(or $(strip $(DEPLOYMENT)),$(call minikube_deployment,$(MINIKUBE_DEPLOY_SERVICE)))
.PHONY: minikube-start
minikube-start: ## Start minikube cluster (starts Docker Desktop if needed)
- @if ! docker info >/dev/null 2>&1; then \
+ @if ! scripts/minikube/docker-cli-env.sh --check-info; then \
echo "Starting Docker Desktop..."; \
open -a "Docker Desktop" 2>/dev/null || open -a Docker 2>/dev/null || true; \
echo "Waiting for Docker daemon..."; \
- for i in $$(seq 1 30); do docker info >/dev/null 2>&1 && break; sleep 2; done; \
- docker info >/dev/null 2>&1 || { echo "ERROR: Docker not available after 60s"; exit 1; }; \
+ docker_start_timeout="$${MINIKUBE_DOCKER_START_TIMEOUT_SECONDS:-60}"; \
+ MINIKUBE_DOCKER_START_TIMEOUT_SECONDS="$$docker_start_timeout" \
+ scripts/minikube/docker-cli-env.sh --wait-for-info || { \
+ echo "ERROR: Docker not available after $${docker_start_timeout}s"; \
+ exit 1; \
+ }; \
echo "Docker ready."; \
fi
MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" MINIKUBE_MULTI_NODE="$(MINIKUBE_MULTI_NODE)" MINIKUBE_NODES="$(MINIKUBE_NODES)" MINIKUBE_MEMORY="$(MINIKUBE_MEMORY)" MINIKUBE_CPUS="$(MINIKUBE_CPUS)" scripts/minikube/start.sh
@@ -182,8 +194,7 @@ minikube-setup-e2e: ## Full setup + E2E fixtures (test user, e2e-* recipes, demo
@$(MAKE) --no-print-directory minikube-setup SEED_PROFILE=e2e
@if [ "$(IMAGE_SOURCE)" = "ghcr" ]; then \
echo "Building the two unpublished E2E coordinator fixtures..."; \
- MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh --only=workflow-custom-sdk-e2e; \
- MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh --only=workflow-plugin-sdk-e2e; \
+ $(MAKE) --no-print-directory minikube-build-e2e-fixtures; \
fi
.PHONY: minikube-teardown
@@ -195,13 +206,45 @@ minikube-pull-images: ## Pull ALL published images into minikube at the pinned r
@MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" MINIKUBE_IMAGE_TAG="$(MINIKUBE_IMAGE_TAG)" \
scripts/minikube/pull-images.sh
-.PHONY: minikube-build-images
+.PHONY: minikube-build-images minikube-build-images-body
minikube-build-images: ## Build and load ALL Docker images into minikube (with SHA verification)
- @scripts/minikube/build-images.sh
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK="$(T2_SKIP_LOCK)" T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/with-t2-mutation-lock.sh -- \
+ $(MAKE) --no-print-directory minikube-build-images-body
+
+minikube-build-images-body:
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK=true T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/require-t2-mutation-lock.sh
+ @MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh
-.PHONY: minikube-build-custom-coordinator-fixture
+.PHONY: minikube-build-custom-coordinator-fixture minikube-build-custom-coordinator-fixture-body
minikube-build-custom-coordinator-fixture: ## Build only the custom coordinator E2E fixture image in minikube
- @scripts/minikube/build-images.sh --only=workflow-custom-sdk-e2e
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK="$(T2_SKIP_LOCK)" T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/with-t2-mutation-lock.sh -- \
+ $(MAKE) --no-print-directory minikube-build-custom-coordinator-fixture-body
+
+minikube-build-custom-coordinator-fixture-body:
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK=true T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/require-t2-mutation-lock.sh
+ @MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh --only=workflow-custom-sdk-e2e
+
+.PHONY: minikube-build-e2e-fixtures minikube-build-e2e-fixtures-body
+minikube-build-e2e-fixtures: ## Build the two unpublished coordinator E2E fixtures under one mutation lease
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK="$(T2_SKIP_LOCK)" T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/with-t2-mutation-lock.sh -- \
+ $(MAKE) --no-print-directory minikube-build-e2e-fixtures-body
+
+minikube-build-e2e-fixtures-body:
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK=true T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/require-t2-mutation-lock.sh
+ @MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh --only=workflow-custom-sdk-e2e
+ @MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh --only=workflow-plugin-sdk-e2e
.PHONY: minikube-verify-images
minikube-verify-images: ## Verify every image the cluster runs is present. The mode comes from deploy/minikube/.image-manifest.json (what was actually built/pulled), not from IMAGE_SOURCE; SEED_PROFILE=e2e also checks the two E2E fixtures.
@@ -376,22 +419,52 @@ minikube-deploy-crds-body:
kubectl --context=$(MINIKUBE_PROFILE) apply -f ./charts/clerum-crds/crds/; \
fi
-.PHONY: minikube-deploy-service
+.PHONY: minikube-deploy-service minikube-deploy-service-body
minikube-deploy-service: ## Rebuild single image + rollout restart deployment (usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm])
- @if [ -z "$(SVC)" ]; then echo "ERROR: SVC required. Usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1; fi
- @if [ -z "$(NS)" ]; then echo "ERROR: NS required. Usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1; fi
- @echo "Deploying image selector $(SVC) to deployment/$(MINIKUBE_DEPLOYMENT) in namespace $(NS)"
- @scripts/minikube/build-images.sh --only=$(SVC)
- kubectl --context=$(MINIKUBE_PROFILE) -n $(NS) rollout restart deployment/$(MINIKUBE_DEPLOYMENT)
- kubectl --context=$(MINIKUBE_PROFILE) -n $(NS) rollout status deployment/$(MINIKUBE_DEPLOYMENT) --timeout=180s
-
-.PHONY: minikube-restart-deploy
+ @$(if $(MINIKUBE_DEPLOY_SERVICE),:,echo "ERROR: SVC required. Usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_SERVICE_SUPPORTED),:,echo "ERROR: unsupported SVC selector. Supported: $(MINIKUBE_DEPLOY_SERVICE_SELECTORS)"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_NAMESPACE),:,echo "ERROR: NS required. Usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_EFFECTIVE_DEPLOYMENT),:,echo "ERROR: effective DEPLOYMENT could not be resolved from SVC"; exit 1)
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK="$(T2_SKIP_LOCK)" T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/with-t2-mutation-lock.sh -- \
+ $(MAKE) --no-print-directory minikube-deploy-service-body
+
+minikube-deploy-service-body:
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK=true T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/require-t2-mutation-lock.sh
+ @$(if $(MINIKUBE_DEPLOY_SERVICE),:,echo "ERROR: SVC required. Usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_SERVICE_SUPPORTED),:,echo "ERROR: unsupported SVC selector. Supported: $(MINIKUBE_DEPLOY_SERVICE_SELECTORS)"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_NAMESPACE),:,echo "ERROR: NS required. Usage: make minikube-deploy-service SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_EFFECTIVE_DEPLOYMENT),:,echo "ERROR: effective DEPLOYMENT could not be resolved from SVC"; exit 1)
+ @echo "Deploying image selector $(MINIKUBE_DEPLOY_SERVICE) to deployment/$(MINIKUBE_EFFECTIVE_DEPLOYMENT) in namespace $(MINIKUBE_DEPLOY_NAMESPACE)"
+ @MINIKUBE_PROFILE="$(MINIKUBE_PROFILE)" scripts/minikube/build-images.sh --only=$(MINIKUBE_DEPLOY_SERVICE)
+ kubectl --context=$(MINIKUBE_PROFILE) -n $(MINIKUBE_DEPLOY_NAMESPACE) rollout restart deployment/$(MINIKUBE_EFFECTIVE_DEPLOYMENT)
+ kubectl --context=$(MINIKUBE_PROFILE) -n $(MINIKUBE_DEPLOY_NAMESPACE) rollout status deployment/$(MINIKUBE_EFFECTIVE_DEPLOYMENT) --timeout=180s
+
+.PHONY: minikube-restart-deploy minikube-restart-deploy-body
minikube-restart-deploy: ## Restart a single deployment without rebuilding (usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm])
- @if [ -z "$(SVC)" ]; then echo "ERROR: SVC required. Usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1; fi
- @if [ -z "$(NS)" ]; then echo "ERROR: NS required. Usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1; fi
- @echo "Restarting deployment/$(MINIKUBE_DEPLOYMENT) in namespace $(NS)"
- kubectl --context=$(MINIKUBE_PROFILE) -n $(NS) rollout restart deployment/$(MINIKUBE_DEPLOYMENT)
- kubectl --context=$(MINIKUBE_PROFILE) -n $(NS) rollout status deployment/$(MINIKUBE_DEPLOYMENT) --timeout=180s
+ @$(if $(MINIKUBE_DEPLOY_SERVICE),:,echo "ERROR: SVC required. Usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_SERVICE_SUPPORTED),:,echo "ERROR: unsupported SVC selector. Supported: $(MINIKUBE_DEPLOY_SERVICE_SELECTORS)"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_NAMESPACE),:,echo "ERROR: NS required. Usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_EFFECTIVE_DEPLOYMENT),:,echo "ERROR: effective DEPLOYMENT could not be resolved from SVC"; exit 1)
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK="$(T2_SKIP_LOCK)" T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/with-t2-mutation-lock.sh -- \
+ $(MAKE) --no-print-directory minikube-restart-deploy-body
+
+minikube-restart-deploy-body:
+ @T2_PROJECT_DIR="$(CURDIR)" T2_PROFILE="$(MINIKUBE_PROFILE)" T2_CONTEXT="$(MINIKUBE_PROFILE)" \
+ T2_SKIP_LOCK=true T2_LOCK_TOKEN="$(T2_LOCK_TOKEN)" \
+ bash scripts/minikube/require-t2-mutation-lock.sh
+ @$(if $(MINIKUBE_DEPLOY_SERVICE),:,echo "ERROR: SVC required. Usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_SERVICE_SUPPORTED),:,echo "ERROR: unsupported SVC selector. Supported: $(MINIKUBE_DEPLOY_SERVICE_SELECTORS)"; exit 1)
+ @$(if $(MINIKUBE_DEPLOY_NAMESPACE),:,echo "ERROR: NS required. Usage: make minikube-restart-deploy SVC=mcp-host NS=mcp-host [DEPLOYMENT=chatllm]"; exit 1)
+ @$(if $(MINIKUBE_EFFECTIVE_DEPLOYMENT),:,echo "ERROR: effective DEPLOYMENT could not be resolved from SVC"; exit 1)
+ @echo "Restarting deployment/$(MINIKUBE_EFFECTIVE_DEPLOYMENT) in namespace $(MINIKUBE_DEPLOY_NAMESPACE)"
+ kubectl --context=$(MINIKUBE_PROFILE) -n $(MINIKUBE_DEPLOY_NAMESPACE) rollout restart deployment/$(MINIKUBE_EFFECTIVE_DEPLOYMENT)
+ kubectl --context=$(MINIKUBE_PROFILE) -n $(MINIKUBE_DEPLOY_NAMESPACE) rollout status deployment/$(MINIKUBE_EFFECTIVE_DEPLOYMENT) --timeout=180s
# ── Minikube Secrets & Keys ─────────────────────────────────────────
#
diff --git a/docs/testing/minikube-t2-runbook.md b/docs/testing/minikube-t2-runbook.md
index e5e291c5c..a9408c675 100644
--- a/docs/testing/minikube-t2-runbook.md
+++ b/docs/testing/minikube-t2-runbook.md
@@ -8,12 +8,16 @@ URLs, DSNs, credentials, customer data, or raw runtime logs.
The contract is local-development-only. It requires a clean development
branch descended from the current `origin/dev`, a generated profile owned by
-that worktree and `HEAD`, and an explicit Kubernetes context for that profile.
+that canonical worktree path plus branch, and an explicit Kubernetes context
+for that profile. Exact-HEAD freshness is proved by the deployed marker and
+gate evidence, not by reallocating a profile after every commit.
It refuses protected branches, production/GKE/Cloudflare contexts, shared
profiles, ambiguous ownership, and Kubernetes contexts whose cluster endpoint
does not resolve to a local Minikube address.
-Run the read-only planner first. It is not T0, T1, or T2. Default
+Run the cluster-read-only planner first. It may write ignored local
+lock/evidence metadata, but it does not mutate the cluster and is not T0, T1,
+or T2. Default
`T2_PLAN_MODE=false` fails loud on `full-bootstrap` and never calls
`pre-gate-sync`:
@@ -42,8 +46,11 @@ matches HEAD. `make minikube-pre-gate-sync` reconciles the profile; it does
not emit a T2 verdict.
The profile helper that generated the profile remains the source of truth for
-the profile metadata and random localhost port mapping. Do not copy ports from
-another worktree or use shared fixed ports.
+the profile metadata and random localhost port mapping. Resolve it from the
+primary checkout `.local-notes/minikube-profiles/branch.mk`. A legacy creation
+SHA is historical metadata; it does not override stable worktree+branch
+ownership. Persisted `ports.env` is allocated once. Missing, corrupt, or
+ambiguous metadata fails closed; never regenerate or copy another lane's ports.
## State transitions
@@ -101,6 +108,13 @@ The pre-gate marker must contain the current worktree identifier, exact `HEAD`,
cluster fingerprint, and image coordinate. A mismatch stops with a stable
error code instead of allowing a mixed-commit run.
+Mutating image builds and targeted deploys are children of that same exact
+profile lease. Public Make targets acquire it; private body targets and
+`build-images.sh` validate the inherited token again before the first Docker,
+Minikube, or Kubernetes operation. `build-images.sh --verify-only` is the
+read-only exception. Empty/unknown selectors fail before the lease or runtime
+is touched.
+
### Orphaned lock recovery
If `PROFILE_BUSY` reports that the lock has no valid owner PID, or that a stale
@@ -123,13 +137,27 @@ the `control-postgres` PVC/deployment, all required deployment readiness, the
image manifest/source, and real `kubectl port-forward` processes for this
profile. The conflict check is a loose argv pre-filter (`kubectl` as argv0 or
path token plus a later standalone `port-forward` token; flags may sit
-between them) plus a `comm=kubectl` hardening and a PID allowlist.
-Allowed port-forward PIDs live in
-`$HOME/.cache/clerum/minikube-profiles/<profile>/pids/*.pid` (and the
-legacy `/tmp/pf-<profile>-*.pid` files written by `pf-all-stack.sh`).
+between them), followed by exact `comm`, argv, PID/start-time, profile,
+context, canonical worktree, Service, and port bindings. Each live process
+must have exactly one atomic `0600` ownership record under
+`$HOME/.cache/clerum/minikube-profiles/<profile>/pids/`. A live legacy
+`/tmp/pf-<profile>-*.pid` record cannot be adopted or killed because it lacks
+the full binding; dead legacy records may be pruned.
`make minikube-t2` invokes `pre-gate-sync` with `--skip-port-forwards` so the
orchestrator does not plant forwards that fail its own T2 check. Secret values
-are never printed. The local Real PostgreSQL lane resolves the
+are never printed.
+
+Docker endpoint discovery runs before isolation and accepts only an explicit
+local Unix socket or loopback TCP endpoint. The endpoint is then pinned while
+Docker uses an empty task-local config; ambient auth, credential helpers,
+custom headers, and context precedence do not cross that boundary. Public
+pulls remain unauthenticated. A private pull must opt in with an explicit
+`MINIKUBE_DOCKER_AUTH_CONFIG`, scoped only to that pull. Docker probes, pulls,
+builds, Minikube image operations, Minikube status/docker-env, Kubernetes node
+inventory, and targeted health commands all have validated finite deadlines
+and process-group cleanup on timeout or interrupt.
+
+The local Real PostgreSQL lane resolves the
`control-postgres` Secret using the explicit context, constructs its admin DSN
only in process memory, and passes it only to the shared-server suites. Suites
that drop or rewrite cluster-global roles (`db.realPostgresMigration`,
@@ -210,17 +238,20 @@ non-T2 scope guard.
`control-postgres`, except the role-reset suites which use an isolated
Postgres 16; the lane reports `PASS`, `FAIL`, `SKIPPED`, and `NOT_RUN`
separately and fails on an unavailable DSN, an isolated server that did not
- start, or zero executed tests. The JSON reporter must be complete and green
- (expected files, executed/passed, zero failures, zero pending), and the
- Vitest process must also exit zero; a green reporter cannot hide teardown,
- worker, OOM, or signal failure.
+ start, or zero executed tests. A fast Node/package/Docker preflight runs
+ before T0. T1 is serial by contract (`VITEST_MAX_WORKERS=1`, no file
+ parallelism). The JSON reporter must be complete and green, its
+ `testResults[].name` set must exactly equal the selected physical files, and
+ the Vitest process must also exit zero; a green reporter cannot hide
+ teardown, worker, OOM, signal, or partial-selection failure.
* **T2** — the final exact-head preflight inside `make minikube-t2` (or
`make minikube-t2-runtime`): the marker matches this worktree/`HEAD`, the
image manifest is current, PostgreSQL and required namespaces/Services are
present, deployments are Ready, and no foreign `kubectl port-forward` owns
- this profile. User-facing health and Control UI/Desktop Playwright journeys
- are opt-in via `T2_HEALTHCHECK_COMMAND` / `T2_PLAYWRIGHT_COMMAND` and are
- recorded as separate evidence statuses (`NOT_RUN` by default;
+ this profile. A targeted sync requires a bounded user-facing journey via
+ `T2_HEALTHCHECK_COMMAND`; other transitions may leave it opt-in. Control
+ UI/Desktop Playwright remains opt-in via `T2_PLAYWRIGHT_COMMAND`. Both are
+ recorded as separate evidence statuses (`NOT_RUN` when optional;
`T2_REQUIRE_PLAYWRIGHT=true` refuses a missing journey). Product E2E scripts
such as `scripts/e2e/e2e-hcc-rollout-readiness.sh` are not T2.
@@ -247,6 +278,14 @@ profile:
The canonical T2 journey remains
`scripts/e2e/e2e-np08-hcc-authorization.sh`; these helpers do not replace it.
+That journey observes the existing Host access-token lineage. It may reread a
+newer persisted access token only when the sole `hostRefs` entry,
+`recipeNamespace`, and `recipeName` exactly match the mounted access-token
+binding. It never reads a
+refresh token or calls refresh/reissue; mcp-host remains the sole writer of the
+single-use lineage. Before fixture mutation it requires the in-pod runtime
+health endpoint, and a 401 triggers a bounded access-state reread rather than a
+second writer.
Their manual status must be recorded as `PASS`, `FAIL`, or `NOT_RUN` in the
sanitized run evidence rather than inferred from unit or CI results.
@@ -263,6 +302,14 @@ refused. An interrupted bootstrap leaves the profile intact and can be safely
retried after the reported prerequisite is repaired. Do not reuse evidence
from a different `HEAD` or profile.
+Retry by phase. During a T1 failure, iterate with
+`minikube-t2-real-postgres`, then run one full `minikube-t2` certification once
+green. After exact-head T0/T1 lane evidence is already `PASS`, a failure in
+NP08, a user-facing health check, or Playwright is repaired and retried with
+`minikube-t2-runtime` on the same profile/context; repeating T0/T1 adds cost
+without new evidence. A bootstrap, marker, infrastructure, or final-preflight
+failure still uses the full target.
+
## Evidence and redaction
Each run writes a sanitized `evidence.json` below the ignored local path
diff --git a/scripts/dev/repo-intake-packet.sh b/scripts/dev/repo-intake-packet.sh
index 8c1b55a1f..6a6c61914 100755
--- a/scripts/dev/repo-intake-packet.sh
+++ b/scripts/dev/repo-intake-packet.sh
@@ -2,6 +2,7 @@
set -euo pipefail
BASE_REF="${REPO_INTAKE_BASE_REF:-origin/dev}"
+SCRIPT_ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
BLOCKERS=()
WARNINGS=()
TOUCHED_AREAS=()
@@ -28,13 +29,6 @@ resolve_ref() {
short_ref() { git rev-parse --short=8 "$1" 2>/dev/null || printf 'unknown'; }
ahead_behind() { git rev-list --left-right --count "$1...$2" 2>/dev/null || printf 'n/a n/a\n'; }
-slugify() {
- printf '%s' "$1" |
- tr '[:upper:]' '[:lower:]' |
- sed -E 's/[^a-z0-9]+/-/g; s/^-+//; s/-+$//; s/-+/-/g' |
- cut -c1-47
-}
-
area_for_path() {
case "$1" in
control-api/*) printf 'control-api' ;;
@@ -143,8 +137,8 @@ resolve_primary_checkout() {
fi
}
-profile_value() {
- awk -F= -v wanted="$2" '$1 == wanted { sub(/^[^=]*=/, ""); print; exit }' "$1" 2>/dev/null || true
+record_value() {
+ awk -F= -v wanted="$2" '$1 == wanted { sub(/^[^=]*=/, ""); print; exit }' <<<"$1"
}
REPO_ROOT="$(git rev-parse --show-toplevel 2>/dev/null || true)"
@@ -163,9 +157,6 @@ fi
HEAD_FULL="$(git rev-parse --verify HEAD 2>/dev/null || true)"
[[ -n "${HEAD_FULL}" ]] || die "unable to resolve HEAD"
HEAD_SHORT="$(short_ref HEAD)"
-BRANCH_SLUG="$(slugify "${PROFILE_BRANCH}")"
-[[ -n "${BRANCH_SLUG}" ]] || BRANCH_SLUG="detached-${HEAD_SHORT}"
-EXPECTED_PROFILE="clerum-${BRANCH_SLUG}-${HEAD_SHORT}"
UPSTREAM="$(git rev-parse --abbrev-ref --symbolic-full-name '@{upstream}' 2>/dev/null || true)"
[[ -n "${UPSTREAM}" ]] || UPSTREAM="none"
@@ -217,6 +208,8 @@ LOCAL_HELPER_EXISTS="no"
PRIMARY_HELPER_EXISTS="no"
PROFILE_HELPER_EXISTS="no"
PROFILE_HELPER_COMMAND="unavailable"
+PROFILE_OWNER_SCRIPT="${SCRIPT_ROOT}/scripts/minikube/profile-owner.sh"
+PROFILE_OWNER_AVAILABLE="no"
[[ -f "${LOCAL_PROFILE_HELPER}" ]] && LOCAL_HELPER_EXISTS="yes"
if [[ -f "${PRIMARY_PROFILE_HELPER}" ]]; then
@@ -230,34 +223,108 @@ else
BLOCKERS+=("missing_branch_profile_helper")
fi
-PROFILE_ENV="${HOME}/.cache/clerum/minikube-profiles/${EXPECTED_PROFILE}/profile.env"
-PROFILE_CACHE_STATE="absent"
-PROFILE_CACHE_REPO_MATCH="n/a"
-PROFILE_CACHE_SHA_MATCH="n/a"
-if [[ -f "${PROFILE_ENV}" ]]; then
- PROFILE_CACHE_STATE="present"
- PROFILE_CACHE_REPO="$(profile_value "${PROFILE_ENV}" REPO_DIR)"
- PROFILE_CACHE_SHA="$(profile_value "${PROFILE_ENV}" SHA_SHORT)"
-
- if [[ -n "${PROFILE_CACHE_REPO}" ]]; then
- if [[ "${PROFILE_CACHE_REPO}" == "${REPO_ROOT}" ]]; then
- PROFILE_CACHE_REPO_MATCH="yes"
- else
- PROFILE_CACHE_REPO_MATCH="no"
- BLOCKERS+=("profile_cache_belongs_to_other_worktree")
- fi
+if [[ -x "${PROFILE_OWNER_SCRIPT}" ]]; then
+ PROFILE_OWNER_AVAILABLE="yes"
+else
+ BLOCKERS+=("missing_profile_owner_resolver")
+fi
+
+PROFILE_ROOT="${REPO_INTAKE_PROFILE_ROOT:-${CLERUM_PROFILE_CACHE_ROOT:-${HOME}/.cache/clerum/minikube-profiles}}"
+EXPLICIT_PROFILE="${REPO_INTAKE_PROFILE:-${MINIKUBE_PROFILE:-}}"
+EXPLICIT_PROFILE_ENV="${REPO_INTAKE_PROFILE_ENV:-${T2_PROFILE_ENV:-}}"
+EXPLICIT_PORTS_ENV="${REPO_INTAKE_PORTS_ENV:-${CLERUM_PROFILE_PORTS_ENV:-${T2_PORTS_ENV:-}}}"
+PROFILE_SELECTION_EXPLICIT="no"
+if [[ -n "${EXPLICIT_PROFILE}" || -n "${EXPLICIT_PROFILE_ENV}" || -n "${EXPLICIT_PORTS_ENV}" ]]; then
+ PROFILE_SELECTION_EXPLICIT="yes"
+fi
+
+EXPECTED_PROFILE="unavailable"
+NEW_PROFILE_CANDIDATE="unavailable"
+PROFILE_OWNER_ID="unavailable"
+PROFILE_WORKTREE_ID="unavailable"
+PROFILE_IDENTITY_OUTPUT=""
+if [[ "${PROFILE_OWNER_AVAILABLE}" == "yes" ]]; then
+ identity_status=0
+ PROFILE_IDENTITY_OUTPUT="$("${PROFILE_OWNER_SCRIPT}" identity \
+ --repo-dir "${REPO_ROOT}" --branch "${PROFILE_BRANCH}" \
+ --created-head "${HEAD_FULL}" 2>&1)" || identity_status=$?
+ if (( identity_status == 0 )); then
+ EXPECTED_PROFILE="$(record_value "${PROFILE_IDENTITY_OUTPUT}" PROFILE)"
+ NEW_PROFILE_CANDIDATE="${EXPECTED_PROFILE}"
+ PROFILE_OWNER_ID="$(record_value "${PROFILE_IDENTITY_OUTPUT}" OWNER_ID)"
+ PROFILE_WORKTREE_ID="$(record_value "${PROFILE_IDENTITY_OUTPUT}" WORKTREE_ID)"
+ else
+ BLOCKERS+=("profile_identity_resolution_failed")
fi
+fi
- if [[ -n "${PROFILE_CACHE_SHA}" ]]; then
- if [[ "${PROFILE_CACHE_SHA}" == "${HEAD_SHORT}" ]]; then
- PROFILE_CACHE_SHA_MATCH="yes"
- else
- PROFILE_CACHE_SHA_MATCH="no"
- BLOCKERS+=("profile_cache_sha_mismatch")
- fi
+PROFILE_CACHE_STATE="absent"
+PROFILE_CACHE_REPO_MATCH="n/a"
+PROFILE_CACHE_BRANCH_MATCH="n/a"
+PROFILE_CACHE_OWNER_MATCH="n/a"
+PROFILE_CACHE_SHA_MATCH="not_applicable"
+PROFILE_CACHE_SCHEMA="n/a"
+PROFILE_CACHE_CREATION_HEAD="n/a"
+RESOLVED_PROFILE="none"
+RESOLVED_PROFILE_ENV="none"
+RESOLVED_PORTS_ENV="none"
+PROFILE_RESOLUTION_CODE="not_attempted"
+
+if [[ "${PROFILE_OWNER_AVAILABLE}" == "yes" ]]; then
+ resolve_args=(resolve --repo-dir "${REPO_ROOT}" --branch "${PROFILE_BRANCH}" --profile-root "${PROFILE_ROOT}")
+ [[ -z "${EXPLICIT_PROFILE}" ]] || resolve_args+=(--profile "${EXPLICIT_PROFILE}")
+ [[ -z "${EXPLICIT_PROFILE_ENV}" ]] || resolve_args+=(--profile-env "${EXPLICIT_PROFILE_ENV}")
+ [[ -z "${EXPLICIT_PORTS_ENV}" ]] || resolve_args+=(--ports-env "${EXPLICIT_PORTS_ENV}")
+ resolution_status=0
+ PROFILE_RESOLUTION_OUTPUT="$("${PROFILE_OWNER_SCRIPT}" "${resolve_args[@]}" 2>&1)" || resolution_status=$?
+ if (( resolution_status == 0 )); then
+ PROFILE_RESOLUTION_CODE="resolved"
+ PROFILE_CACHE_STATE="present"
+ PROFILE_CACHE_REPO_MATCH="yes"
+ PROFILE_CACHE_BRANCH_MATCH="yes"
+ PROFILE_CACHE_OWNER_MATCH="yes"
+ PROFILE_CACHE_SCHEMA="$(record_value "${PROFILE_RESOLUTION_OUTPUT}" PROFILE_SCHEMA_VERSION)"
+ PROFILE_CACHE_CREATION_HEAD="$(record_value "${PROFILE_RESOLUTION_OUTPUT}" CREATED_HEAD)"
+ RESOLVED_PROFILE="$(record_value "${PROFILE_RESOLUTION_OUTPUT}" PROFILE)"
+ EXPECTED_PROFILE="${RESOLVED_PROFILE}"
+ RESOLVED_PROFILE_ENV="$(record_value "${PROFILE_RESOLUTION_OUTPUT}" PROFILE_ENV)"
+ RESOLVED_PORTS_ENV="$(record_value "${PROFILE_RESOLUTION_OUTPUT}" PORTS_ENV)"
+ else
+ PROFILE_RESOLUTION_CODE="${PROFILE_RESOLUTION_OUTPUT%%:*}"
+ case "${PROFILE_RESOLUTION_CODE}" in
+ PROFILE_NOT_FOUND)
+ if [[ "${PROFILE_SELECTION_EXPLICIT}" == "yes" ]]; then
+ PROFILE_CACHE_STATE="invalid"
+ BLOCKERS+=("selected_profile_not_found")
+ else
+ WARNINGS+=("profile_cache_not_initialized")
+ fi ;;
+ PROFILE_SELECTION_AMBIGUOUS)
+ PROFILE_CACHE_STATE="ambiguous"
+ BLOCKERS+=("profile_selection_ambiguous") ;;
+ PROFILE_PORTS_MISSING)
+ PROFILE_CACHE_STATE="invalid"
+ BLOCKERS+=("profile_ports_missing") ;;
+ PROFILE_PORTS_INVALID)
+ PROFILE_CACHE_STATE="invalid"
+ BLOCKERS+=("profile_ports_invalid") ;;
+ PROFILE_METADATA_MISSING)
+ PROFILE_CACHE_STATE="invalid"
+ BLOCKERS+=("profile_metadata_missing") ;;
+ PROFILE_METADATA_INVALID)
+ PROFILE_CACHE_STATE="invalid"
+ BLOCKERS+=("profile_metadata_invalid") ;;
+ PROFILE_OWNERSHIP_MISMATCH)
+ PROFILE_CACHE_STATE="invalid"
+ PROFILE_CACHE_REPO_MATCH="no"
+ PROFILE_CACHE_BRANCH_MATCH="no"
+ PROFILE_CACHE_OWNER_MATCH="no"
+ BLOCKERS+=("profile_cache_ownership_mismatch") ;;
+ *)
+ PROFILE_CACHE_STATE="invalid"
+ BLOCKERS+=("profile_metadata_invalid") ;;
+ esac
fi
-else
- WARNINGS+=("profile_cache_not_initialized")
fi
[[ "${DETACHED}" == "yes" ]] && WARNINGS+=("detached_worktree_confirm_scope")
@@ -301,10 +368,23 @@ for row in \
"profile_helper_primary|${PRIMARY_HELPER_EXISTS}" \
"profile_helper_exists|${PROFILE_HELPER_EXISTS}" \
"profile_helper_command|${PROFILE_HELPER_COMMAND}" \
+ "profile_owner_resolver|${PROFILE_OWNER_AVAILABLE}" \
+ "profile_selection_explicit|${PROFILE_SELECTION_EXPLICIT}" \
"expected_profile|${EXPECTED_PROFILE}" \
+ "new_profile_candidate|${NEW_PROFILE_CANDIDATE}" \
+ "resolved_profile|${RESOLVED_PROFILE}" \
"profile_cache_state|${PROFILE_CACHE_STATE}" \
+ "profile_cache_schema|${PROFILE_CACHE_SCHEMA}" \
+ "profile_cache_creation_head|${PROFILE_CACHE_CREATION_HEAD}" \
"profile_cache_repo_match|${PROFILE_CACHE_REPO_MATCH}" \
- "profile_cache_sha_match|${PROFILE_CACHE_SHA_MATCH}"; do
+ "profile_cache_branch_match|${PROFILE_CACHE_BRANCH_MATCH}" \
+ "profile_cache_owner_match|${PROFILE_CACHE_OWNER_MATCH}" \
+ "profile_cache_sha_match|${PROFILE_CACHE_SHA_MATCH}" \
+ "profile_owner_id|${PROFILE_OWNER_ID}" \
+ "profile_worktree_id|${PROFILE_WORKTREE_ID}" \
+ "profile_env|${RESOLVED_PROFILE_ENV}" \
+ "ports_env|${RESOLVED_PORTS_ENV}" \
+ "profile_resolution|${PROFILE_RESOLUTION_CODE}"; do
kv "${row%%|*}" "${row#*|}"
done
diff --git a/scripts/e2e/_lib/np08-runtime-access.mjs b/scripts/e2e/_lib/np08-runtime-access.mjs
new file mode 100644
index 000000000..45b50f2d4
--- /dev/null
+++ b/scripts/e2e/_lib/np08-runtime-access.mjs
@@ -0,0 +1,562 @@
+import { readFile } from 'node:fs/promises'
+import { request as httpRequest } from 'node:http'
+import { join } from 'node:path'
+
+// NP-08 access-only runtime observer. This file is streamed into the mcp-host pod.
+export const NP08_RUNTIME_ACCESS_ROTATION_WAIT_MS = 35_000
+export const NP08_RUNTIME_ACCESS_POLL_MS = 500
+export const NP08_RUNTIME_ACCESS_MIN_VALIDITY_SECONDS = 30
+
+const DEFAULT_RUNTIME_AUTH_STATE_DIR = '/var/run/clerum/workflow-auth'
+const RUNTIME_AUTH_STATE_FILE = 'approval-auth.json'
+const DEFAULT_RUNTIME_PORT = 8080
+const LOCAL_RUNTIME_HEALTH_PATH = '/v1/runtime/health'
+const MAX_STATE_FILE_BYTES = 128 * 1024
+const MAX_HEALTH_BODY_BYTES = 16 * 1024
+
+function delay(ms) {
+ return new Promise(resolve => setTimeout(resolve, ms))
+}
+
+function strictNonEmptyString(value) {
+ if (typeof value !== 'string' || value.length === 0 || value !== value.trim()) {
+ return null
+ }
+ return value
+}
+
+function finiteNumber(value) {
+ return typeof value === 'number' && Number.isFinite(value) ? value : null
+}
+
+function decodeJwtPayload(value) {
+ if (typeof value !== 'string' || value.length === 0) return null
+ const parts = value.split('.')
+ if (parts.length !== 3 || parts[1].length === 0) return null
+ try {
+ const payload = JSON.parse(Buffer.from(parts[1], 'base64url').toString('utf8'))
+ return payload && typeof payload === 'object' && !Array.isArray(payload) ? payload : null
+ } catch {
+ return null
+ }
+}
+
+export function decodeRuntimeAccessToken(value) {
+ const payload = decodeJwtPayload(value)
+ if (!payload) return null
+
+ const hostRefs = Array.isArray(payload.hostRefs) ? payload.hostRefs : []
+ // HCC runtime authority is a single-Host lineage. Rejecting additional
+ // refs keeps the persisted candidate's complete binding identical to the
+ // mounted identity instead of comparing only a privileged first element.
+ const hostRef = hostRefs.length === 1 ? strictNonEmptyString(hostRefs[0]) : null
+ const recipeNamespace = strictNonEmptyString(payload.recipeNamespace)
+ const recipeName = strictNonEmptyString(payload.recipeName)
+ const exp = finiteNumber(payload.exp)
+ const iat = finiteNumber(payload.iat)
+
+ if (!hostRef || !recipeNamespace || !recipeName) return null
+ return {
+ value,
+ binding: { hostRef, recipeNamespace, recipeName },
+ exp,
+ iat,
+ }
+}
+
+function sameBinding(left, right) {
+ return (
+ left.hostRef === right.hostRef &&
+ left.recipeNamespace === right.recipeNamespace &&
+ left.recipeName === right.recipeName
+ )
+}
+
+function isFresh(candidate, nowMs, minValiditySeconds) {
+ return (
+ candidate?.exp !== null &&
+ candidate.exp > Math.floor(nowMs / 1000) + minValiditySeconds
+ )
+}
+
+function isStrictlyFresher(candidate, baseline) {
+ if (!sameBinding(candidate.binding, baseline.binding)) return false
+
+ if (baseline.iat !== null && (candidate.iat === null || candidate.iat < baseline.iat)) {
+ return false
+ }
+
+ if (baseline.exp === null) return true
+ if (candidate.exp === null) return false
+ if (candidate.exp !== baseline.exp) return candidate.exp > baseline.exp
+
+ if (baseline.iat === null) return candidate.iat !== null
+ return candidate.iat !== null && candidate.iat > baseline.iat
+}
+
+async function readPersistedAccessCandidate(stateFilePath, readFileImpl) {
+ try {
+ const raw = await readFileImpl(stateFilePath, 'utf8')
+ if (typeof raw !== 'string' || raw.length > MAX_STATE_FILE_BYTES) return null
+ const parsed = JSON.parse(raw)
+ if (!parsed || typeof parsed !== 'object' || Array.isArray(parsed)) return null
+ return decodeRuntimeAccessToken(parsed.accessToken)
+ } catch {
+ // Atomic rotation can briefly race a read. Malformed state never becomes
+ // an identity source, so the caller retains the mounted access identity.
+ return null
+ }
+}
+
+export function runtimeAuthStateFilePath(env = process.env) {
+ const configured = strictNonEmptyString(env.MCP_HOST_RUNTIME_AUTH_STATE_DIR)
+ return join(configured ?? DEFAULT_RUNTIME_AUTH_STATE_DIR, RUNTIME_AUTH_STATE_FILE)
+}
+
+export async function observeFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath = runtimeAuthStateFilePath(),
+ nowMs = Date.now(),
+ minValiditySeconds = NP08_RUNTIME_ACCESS_MIN_VALIDITY_SECONDS,
+ readFileImpl = readFile,
+}) {
+ const mounted = decodeRuntimeAccessToken(mountedAccessValue)
+ if (!mounted) throw new Error('runtime_access_binding_unavailable')
+
+ const persisted = await readPersistedAccessCandidate(stateFilePath, readFileImpl)
+ if (
+ persisted &&
+ isFresh(persisted, nowMs, minValiditySeconds) &&
+ isStrictlyFresher(persisted, mounted)
+ ) {
+ return { ...persisted, source: 'persisted' }
+ }
+
+ if (isFresh(mounted, nowMs, minValiditySeconds)) {
+ return { ...mounted, source: 'mounted' }
+ }
+ return null
+}
+
+export async function waitForFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath = runtimeAuthStateFilePath(),
+ timeoutMs = NP08_RUNTIME_ACCESS_ROTATION_WAIT_MS,
+ pollIntervalMs = NP08_RUNTIME_ACCESS_POLL_MS,
+ minValiditySeconds = NP08_RUNTIME_ACCESS_MIN_VALIDITY_SECONDS,
+ readFileImpl = readFile,
+ now = Date.now,
+ sleep = delay,
+}) {
+ const deadline = now() + timeoutMs
+ while (true) {
+ const candidate = await observeFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath,
+ nowMs: now(),
+ minValiditySeconds,
+ readFileImpl,
+ })
+ if (candidate) return candidate
+
+ const remainingMs = deadline - now()
+ if (remainingMs <= 0) throw new Error('runtime_access_fresh_token_timeout')
+ await sleep(Math.min(pollIntervalMs, remainingMs))
+ }
+}
+
+function responseStatus(result) {
+ const status = result?.response?.status ?? result?.status
+ if (!Number.isInteger(status)) throw new Error('runtime_access_request_status_missing')
+ return status
+}
+
+function shouldAdoptObserved(current, observed, nowMs, minValiditySeconds) {
+ if (!current) return true
+ if (current.value === observed.value) return true
+ if (!isFresh(current, nowMs, minValiditySeconds)) return true
+ return isStrictlyFresher(observed, current)
+}
+
+export async function requestWithRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath = runtimeAuthStateFilePath(),
+ currentCandidate = null,
+ request,
+ timeoutMs = NP08_RUNTIME_ACCESS_ROTATION_WAIT_MS,
+ pollIntervalMs = NP08_RUNTIME_ACCESS_POLL_MS,
+ minValiditySeconds = NP08_RUNTIME_ACCESS_MIN_VALIDITY_SECONDS,
+ readFileImpl = readFile,
+ now = Date.now,
+ sleep = delay,
+}) {
+ if (typeof request !== 'function') throw new Error('runtime_access_request_missing')
+
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath,
+ nowMs: now(),
+ minValiditySeconds,
+ readFileImpl,
+ })
+
+ let candidate = currentCandidate
+ if (observed && shouldAdoptObserved(candidate, observed, now(), minValiditySeconds)) {
+ candidate = observed
+ }
+ if (!candidate || !isFresh(candidate, now(), minValiditySeconds)) {
+ candidate = await waitForFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath,
+ timeoutMs,
+ pollIntervalMs,
+ minValiditySeconds,
+ readFileImpl,
+ now,
+ sleep,
+ })
+ }
+
+ let result = await request(candidate.value)
+ if (responseStatus(result) !== 401) {
+ return { candidate, result, rotatedAfterUnauthorized: false }
+ }
+
+ const deadline = now() + timeoutMs
+ const attemptedValues = new Set([candidate.value])
+ while (true) {
+ const remainingMs = deadline - now()
+ if (remainingMs <= 0) throw new Error('runtime_access_rotation_timeout')
+ await sleep(Math.min(pollIntervalMs, remainingMs))
+
+ const next = await observeFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath,
+ nowMs: now(),
+ minValiditySeconds,
+ readFileImpl,
+ })
+ if (
+ !next ||
+ attemptedValues.has(next.value) ||
+ !shouldAdoptObserved(candidate, next, now(), minValiditySeconds)
+ ) {
+ continue
+ }
+
+ candidate = next
+ attemptedValues.add(candidate.value)
+ result = await request(candidate.value)
+ if (responseStatus(result) !== 401) {
+ return { candidate, result, rotatedAfterUnauthorized: true }
+ }
+ }
+}
+
+function timeoutSignal(timeoutMs) {
+ return typeof globalThis.AbortSignal?.timeout === 'function'
+ ? globalThis.AbortSignal.timeout(timeoutMs)
+ : undefined
+}
+
+function runtimePort(env) {
+ const raw = strictNonEmptyString(env.CLERUM_SERVER_PORT) ?? String(DEFAULT_RUNTIME_PORT)
+ if (!/^[1-9][0-9]*$/.test(raw)) throw new Error('mcp_host_runtime_port_invalid')
+ const value = Number(raw)
+ if (!Number.isSafeInteger(value) || value > 65_535) {
+ throw new Error('mcp_host_runtime_port_invalid')
+ }
+ return value
+}
+
+function contextMapperBaseUrl(env) {
+ const raw = strictNonEmptyString(env.CLERUM_CONTEXT_MAPPER_URL)
+ if (!raw) throw new Error('hcc_context_mapper_url_unavailable')
+ let parsed
+ try {
+ parsed = new URL(raw)
+ } catch {
+ throw new Error('hcc_context_mapper_url_invalid')
+ }
+ if (
+ (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') ||
+ parsed.username ||
+ parsed.password ||
+ (parsed.pathname !== '/' && parsed.pathname !== '') ||
+ parsed.search ||
+ parsed.hash
+ ) {
+ throw new Error('hcc_context_mapper_url_invalid')
+ }
+ return parsed.origin
+}
+
+export async function requestLocalRuntimeHealth({
+ port,
+ timeoutMs,
+ requestFactory = httpRequest,
+}) {
+ return new Promise((resolve, reject) => {
+ let settled = false
+ let deadline
+ const finish = (callback, value) => {
+ if (settled) return
+ settled = true
+ clearTimeout(deadline)
+ callback(value)
+ }
+ const request = requestFactory(
+ {
+ hostname: '127.0.0.1',
+ port,
+ path: LOCAL_RUNTIME_HEALTH_PATH,
+ method: 'GET',
+ headers: { accept: 'application/json' },
+ },
+ response => {
+ let bytes = 0
+ const chunks = []
+ response.on('data', chunk => {
+ bytes += chunk.length
+ if (bytes > MAX_HEALTH_BODY_BYTES) {
+ request.destroy(new Error('mcp_host_runtime_health_body_too_large'))
+ return
+ }
+ chunks.push(chunk)
+ })
+ response.once('end', () => {
+ let body = null
+ try {
+ body = JSON.parse(Buffer.concat(chunks).toString('utf8'))
+ } catch {
+ // The caller treats a malformed body as unavailable.
+ }
+ finish(resolve, { status: response.statusCode, body })
+ })
+ response.once('aborted', () => {
+ finish(reject, new Error('mcp_host_runtime_health_aborted'))
+ })
+ response.once('error', error => finish(reject, error))
+ }
+ )
+ // ClientRequest#setTimeout is an inactivity timeout and can be defeated by
+ // a peer that drip-feeds bytes. The gate needs a wall-clock deadline for
+ // the complete local health exchange.
+ deadline = setTimeout(() => {
+ request.destroy(new Error('mcp_host_runtime_health_timeout'))
+ }, timeoutMs)
+ deadline.unref?.()
+ request.once('error', error => finish(reject, error))
+ request.end()
+ })
+}
+
+export async function requireLocalRuntimeHealth({
+ env = process.env,
+ requestImpl = requestLocalRuntimeHealth,
+ timeoutMs = 5_000,
+} = {}) {
+ let result
+ try {
+ result = await requestImpl({ port: runtimePort(env), timeoutMs })
+ } catch {
+ throw new Error('mcp_host_runtime_health_unavailable')
+ }
+
+ if (result?.status !== 200 || result?.body?.status !== 'ok') {
+ throw new Error(`mcp_host_runtime_health_unavailable_http_${result?.status ?? 'unknown'}`)
+ }
+}
+
+async function requestJson(fetchImpl, baseUrl, path, init = {}) {
+ let response
+ try {
+ response = await fetchImpl(`${baseUrl}${path}`, {
+ ...init,
+ redirect: 'error',
+ signal: timeoutSignal(10_000),
+ })
+ } catch {
+ throw new Error('hcc_request_failed')
+ }
+
+ const text = await response.text()
+ let body = null
+ try {
+ body = JSON.parse(text)
+ } catch {
+ body = { nonJson: true }
+ }
+ return { response, body }
+}
+
+async function assertEventually(label, assertion, timeoutMs = 60_000) {
+ const deadline = Date.now() + timeoutMs
+ while (Date.now() < deadline) {
+ if (await assertion()) return
+ await delay(1_000)
+ }
+ throw new Error(`${label} did not converge`)
+}
+
+export async function runDeployedNp08Journey({
+ env = process.env,
+ fetchImpl = globalThis.fetch,
+} = {}) {
+ const mountedAccessValue = env.MCP_HOST_RUNTIME_ACCESS_TOKEN
+ const workflowToken = env.MCP_HOST_WORKFLOW_CONTROL_TOKEN
+ const serverA = env.NP08_SERVER_A
+ const serverB = env.NP08_SERVER_B
+ const serverC = env.NP08_SERVER_C
+ const expectedSyntheticValue = env.NP08_EXPECTED_SYNTHETIC_VALUE
+ const hccBaseUrl = contextMapperBaseUrl(env)
+
+ if (
+ !mountedAccessValue ||
+ !workflowToken ||
+ !serverA ||
+ !serverB ||
+ !serverC ||
+ !expectedSyntheticValue
+ ) {
+ throw new Error('runtime_access_environment_unavailable')
+ }
+
+ const stateFilePath = runtimeAuthStateFilePath(env)
+ let currentCandidate = await waitForFreshRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath,
+ })
+ console.log(
+ currentCandidate.source === 'persisted'
+ ? 'PASS runtime access observed from fresher persisted Host state'
+ : 'PASS runtime access is fresh in the mounted Host identity'
+ )
+
+ let rotationReported = false
+ async function callAsRuntime(path, init = {}) {
+ const outcome = await requestWithRuntimeAccess({
+ mountedAccessValue,
+ stateFilePath,
+ currentCandidate,
+ request: value =>
+ requestJson(fetchImpl, hccBaseUrl, path, {
+ ...init,
+ headers: {
+ ...(init.headers ?? {}),
+ authorization: `Bearer ${value}`,
+ },
+ }),
+ })
+ currentCandidate = outcome.candidate
+ if (outcome.rotatedAfterUnauthorized && !rotationReported) {
+ rotationReported = true
+ console.log('PASS runtime access advanced after an HCC authorization retry')
+ }
+ return outcome.result
+ }
+
+ const credentialPath = '/api/v2/hosts/self/mcpservers/credential'
+ await assertEventually('same-Context inventory', async () => {
+ const { response, body } = await callAsRuntime('/api/v2/hosts/self/mcpservers')
+ if (response.status !== 200 || !Array.isArray(body?.servers)) return false
+ const names = body.servers.map(server => server?.name)
+ if (!names.includes(serverA) || names.includes(serverB)) return false
+ if (
+ JSON.stringify(body).includes('secretRef') ||
+ JSON.stringify(body).includes(expectedSyntheticValue)
+ ) {
+ throw new Error('inventory_exposed_credential_material')
+ }
+ return true
+ })
+ console.log('PASS same-Context inventory is scoped and metadata-only')
+
+ const credentialValueField = 'token'
+ const positive = await callAsRuntime(credentialPath, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ serverName: serverA }),
+ })
+ if (
+ positive.response.status !== 200 ||
+ positive.body?.[credentialValueField] !== expectedSyntheticValue ||
+ typeof positive.body?.credentialRevision !== 'string' ||
+ positive.body.credentialRevision.length === 0
+ ) {
+ throw new Error(`same_context_credential_failed_http_${positive.response.status}`)
+ }
+ console.log('PASS same-Context credential returned the fenced synthetic value in-memory')
+
+ for (const [label, serverName] of [
+ ['cross-Context', serverB],
+ ['unknown', 'np08-e2e-unknown'],
+ ]) {
+ const denied = await callAsRuntime(credentialPath, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ serverName }),
+ })
+ if (
+ denied.response.status !== 404 ||
+ JSON.stringify(denied.body) !== '{"error":"not_found"}'
+ ) {
+ throw new Error(`${label}_credential_not_opaque_404`)
+ }
+ if (Object.hasOwn(denied.body ?? {}, credentialValueField)) {
+ throw new Error(`${label}_credential_response_contained_value`)
+ }
+ console.log(`PASS ${label} credential request denied without a credential response`)
+ }
+
+ const noAuth = await callAsRuntime(credentialPath, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ serverName: serverC }),
+ })
+ if (noAuth.response.status !== 200 || noAuth.body?.[credentialValueField] !== null) {
+ throw new Error(`same_context_no_auth_failed_http_${noAuth.response.status}`)
+ }
+ console.log('PASS same-Context no-auth server returned an explicit null credential')
+
+ const malformedBody = await callAsRuntime(credentialPath, {
+ method: 'POST',
+ headers: { 'content-type': 'application/json' },
+ body: JSON.stringify({ serverName: serverA, contextRef: 'caller-supplied-context' }),
+ })
+ if (malformedBody.response.status !== 400 || malformedBody.body?.error !== 'bad_request') {
+ throw new Error('caller_supplied_context_was_accepted')
+ }
+ console.log('PASS caller-supplied Context field was rejected')
+
+ const bearer = value => ({ authorization: `Bearer ${value}` })
+ for (const [label, headers] of [
+ ['anonymous', {}],
+ ['workflow-only', bearer(workflowToken)],
+ ['malformed-bearer', bearer('not-a-jwt')],
+ ]) {
+ const denied = await requestJson(fetchImpl, hccBaseUrl, '/api/v2/hosts/self/mcpservers', { headers })
+ if (denied.response.status !== 401 || denied.body?.error !== 'unauthorized') {
+ throw new Error(`${label}_inventory_not_generic_401`)
+ }
+ console.log(`PASS ${label} Host inventory request rejected`)
+ }
+
+ for (const path of [
+ '/api/v1/mcpservers/context/np08-e2e-context-b',
+ `/api/v1/mcpservers/${serverB}/auth`,
+ ]) {
+ const retired = await requestJson(fetchImpl, hccBaseUrl, path)
+ if (retired.response.status !== 410 || retired.body?.error !== 'gone') {
+ throw new Error('legacy_route_was_not_tombstoned')
+ }
+ }
+ console.log('PASS legacy caller-selected routes are tombstoned')
+}
+
+if (process.env.NP08_RUNTIME_ACTION === 'health') {
+ await requireLocalRuntimeHealth()
+ console.log('PASS local mcp-host runtime health is ready before fixture mutation')
+} else if (process.env.NP08_RUNTIME_ACTION === 'journey') {
+ await runDeployedNp08Journey()
+}
diff --git a/scripts/e2e/e2e-np08-hcc-authorization.sh b/scripts/e2e/e2e-np08-hcc-authorization.sh
index 5717792fa..b91a67241 100755
--- a/scripts/e2e/e2e-np08-hcc-authorization.sh
+++ b/scripts/e2e/e2e-np08-hcc-authorization.sh
@@ -20,6 +20,7 @@ USAGE
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
PROJECT_DIR="$(cd -- "${SCRIPT_DIR}/../.." && pwd -P)"
+NP08_RUNTIME_MODULE="${SCRIPT_DIR}/_lib/np08-runtime-access.mjs"
context=''
while [[ $# -gt 0 ]]; do
@@ -68,6 +69,11 @@ for command_name in kubectl jq git shasum awk python3 node; do
}
done
+[[ -r "${NP08_RUNTIME_MODULE}" ]] || {
+ echo 'FAIL: NP-08 runtime access module is unavailable' >&2
+ exit 1
+}
+
kctl() {
kubectl --context="${context}" "$@"
}
@@ -246,6 +252,12 @@ if kctl -n "${MCP_NS}" get context "${CONTEXT_A}" -o json | jq -e --arg servera
exit 1
fi
+# Prove the actual mcp-host process is healthy before creating any fixture.
+# The reviewed module is streamed into the pod, so the deployed journey and
+# its deterministic unit tests execute the same access-only implementation.
+kctl -n "${HOST_NS}" exec -i "deploy/${HOST_DEPLOYMENT}" -- \
+ env NP08_RUNTIME_ACTION=health node --input-type=module - < "${NP08_RUNTIME_MODULE}"
+
# The fixtures are deliberately managed:false: HCC must expose their live
# authority state without creating a workload or opening a new transport lane.
kctl -n "${MCP_NS}" apply -f - >/dev/null <<YAML
@@ -362,156 +374,7 @@ echo 'E2E setup: waiting for HCC to observe the synthetic Context/McpServer fixt
# Keep the runtime JWT and the returned synthetic token inside the Host pod.
# The process emits only assertion labels and status/error classes.
kctl -n "${HOST_NS}" exec -i "deploy/${HOST_DEPLOYMENT}" -- \
- env "NP08_SERVER_A=${SERVER_A}" "NP08_SERVER_B=${SERVER_B}" "NP08_SERVER_C=${SERVER_C}" \
- "NP08_EXPECTED_SYNTHETIC_TOKEN=np08-synthetic-${RUN_ID}-a" node - <<'NODE'
-const base = 'http://host-context-controller-api-gateway.control-plane.svc.cluster.local:8081'
-let accessToken = process.env.MCP_HOST_RUNTIME_ACCESS_TOKEN
-const refreshToken = process.env.MCP_HOST_RUNTIME_REFRESH_TOKEN
-const workflowToken = process.env.MCP_HOST_WORKFLOW_CONTROL_TOKEN
-const serverA = process.env.NP08_SERVER_A
-const serverB = process.env.NP08_SERVER_B
-const serverC = process.env.NP08_SERVER_C
-const expectedSyntheticToken = process.env.NP08_EXPECTED_SYNTHETIC_TOKEN
-
-if (!accessToken || !refreshToken || !workflowToken || !serverA || !serverB || !serverC || !expectedSyntheticToken) {
- throw new Error('runtime token environment is unavailable')
-}
-
-async function call(path, init = {}) {
- const response = await fetch(`${base}${path}`, init)
- const text = await response.text()
- let body = null
- try { body = JSON.parse(text) } catch { body = { nonJson: true } }
- return { response, body }
-}
-
-async function assertEventually(label, fn, timeoutMs = 60_000) {
- const deadline = Date.now() + timeoutMs
- let last
- while (Date.now() < deadline) {
- try {
- last = await fn()
- if (last) return
- } catch (error) {
- last = error
- }
- await new Promise(resolve => setTimeout(resolve, 1_000))
- }
- throw new Error(`${label} did not converge`)
-}
-
-const bearer = value => ({ authorization: `Bearer ${value}` })
-const jsonHeaders = value => ({ ...bearer(value), 'content-type': 'application/json' })
-const credentialPath = '/api/v2/hosts/self/mcpservers/credential'
-
-// Reuse a still-valid projected access token when possible. Refresh tokens are
-// single-use in the runtime lineage, so repeatedly refreshing a healthy local
-// profile would make a second E2E run fail for an environmental reason. If the
-// access token is expired or malformed, use the same in-band refresh route as
-// mcp-host and keep every returned token inside this pod.
-function tokenExpiry(token) {
- try {
- const payload = JSON.parse(Buffer.from(token.split('.')[1], 'base64url').toString())
- return typeof payload.exp === 'number' ? payload.exp : 0
- } catch {
- return 0
- }
-}
-
-if (tokenExpiry(accessToken) <= Math.floor(Date.now() / 1000) + 30) {
- const refresh = await fetch(
- `${process.env.MCP_HOST_GATEWAY_URL}/api/v1/workflow-auth/refresh`,
- { method: 'POST', headers: bearer(refreshToken) },
- )
- const refreshBody = await refresh.json().catch(() => ({}))
- if (refresh.status !== 200 || typeof refreshBody?.accessToken !== 'string') {
- throw new Error(`runtime access-token refresh failed (HTTP ${refresh.status})`)
- }
- accessToken = refreshBody.accessToken
- console.log('PASS runtime access token refreshed through the existing Host lineage')
-} else {
- console.log('PASS runtime access token is fresh in the existing Host lineage')
-}
-
-await assertEventually('same-Context inventory', async () => {
- const { response, body } = await call('/api/v2/hosts/self/mcpservers', {
- headers: bearer(accessToken),
- })
- if (response.status !== 200) return false
- if (!Array.isArray(body?.servers)) return false
- const names = body.servers.map(server => server?.name)
- if (!names.includes(serverA) || names.includes(serverB)) return false
- if (JSON.stringify(body).includes('secretRef') || JSON.stringify(body).includes(expectedSyntheticToken)) {
- throw new Error('inventory exposed credential material')
- }
- return true
-})
-console.log('PASS same-Context inventory is scoped and metadata-only')
-
-const positive = await call(credentialPath, {
- method: 'POST',
- headers: jsonHeaders(accessToken),
- body: JSON.stringify({ serverName: serverA }),
-})
-if (positive.response.status !== 200 || positive.body?.token !== expectedSyntheticToken ||
- typeof positive.body?.credentialRevision !== 'string' || positive.body.credentialRevision.length === 0) {
- throw new Error(`same-Context credential did not succeed with the fenced DTO (HTTP ${positive.response.status})`)
-}
-console.log('PASS same-Context credential returned the fenced synthetic token in-memory')
-
-for (const [label, serverName] of [['cross-Context', serverB], ['unknown', 'np08-e2e-unknown']]) {
- const denied = await call(credentialPath, {
- method: 'POST',
- headers: jsonHeaders(accessToken),
- body: JSON.stringify({ serverName }),
- })
- if (denied.response.status !== 404 || JSON.stringify(denied.body) !== '{"error":"not_found"}') {
- throw new Error(`${label} credential request was not an opaque 404`)
- }
- if (Object.hasOwn(denied.body ?? {}, 'token')) throw new Error(`${label} response contained a token`)
- console.log(`PASS ${label} credential request denied without a credential response`)
-}
-
-const noAuth = await call(credentialPath, {
- method: 'POST',
- headers: jsonHeaders(accessToken),
- body: JSON.stringify({ serverName: serverC }),
-})
-if (noAuth.response.status !== 200 || noAuth.body?.token !== null) {
- throw new Error(`same-Context no-auth server did not return token:null (HTTP ${noAuth.response.status})`)
-}
-console.log('PASS same-Context no-auth server returned token:null')
-
-const malformedBody = await call(credentialPath, {
- method: 'POST',
- headers: jsonHeaders(accessToken),
- body: JSON.stringify({ serverName: serverA, contextRef: 'caller-supplied-context' }),
-})
-if (malformedBody.response.status !== 400 || malformedBody.body?.error !== 'bad_request') {
- throw new Error('caller-supplied Context field was accepted')
-}
-console.log('PASS caller-supplied Context field was rejected')
-
-for (const [label, headers] of [
- ['anonymous', {}],
- ['workflow-only', bearer(workflowToken)],
- ['malformed-bearer', bearer('not-a-jwt')],
-]) {
- const denied = await call('/api/v2/hosts/self/mcpservers', { headers })
- if (denied.response.status !== 401 || denied.body?.error !== 'unauthorized') {
- throw new Error(`${label} inventory request was not a generic 401`)
- }
- console.log(`PASS ${label} Host inventory request rejected`)
-}
-
-for (const path of [
- '/api/v1/mcpservers/context/np08-e2e-context-b',
- `/api/v1/mcpservers/${serverB}/auth`,
-]) {
- const retired = await call(path)
- if (retired.response.status !== 410 || retired.body?.error !== 'gone') {
- throw new Error(`legacy route was not tombstoned: ${path}`)
- }
-}
-console.log('PASS legacy caller-selected routes are tombstoned')
-NODE
+ env NP08_RUNTIME_ACTION=journey \
+ "NP08_SERVER_A=${SERVER_A}" "NP08_SERVER_B=${SERVER_B}" "NP08_SERVER_C=${SERVER_C}" \
+ "NP08_EXPECTED_SYNTHETIC_VALUE=np08-synthetic-${RUN_ID}-a" \
+ node --input-type=module - < "${NP08_RUNTIME_MODULE}"
diff --git a/scripts/e2e/gfs-real-pg-minikube-gate.sh b/scripts/e2e/gfs-real-pg-minikube-gate.sh
index 73e147b90..91802d547 100755
--- a/scripts/e2e/gfs-real-pg-minikube-gate.sh
+++ b/scripts/e2e/gfs-real-pg-minikube-gate.sh
@@ -34,6 +34,14 @@ T2_CONTROL_NAMESPACE="${CONTROL_NS}"
export T2_PROJECT_DIR MINIKUBE_PROFILE T2_PROFILE T2_CONTEXT T2_PROFILE_ROOT \
T2_PROFILE_ENV T2_PORTS_ENV T2_MARKER_NAME T2_CONTROL_NAMESPACE
source "${PROJECT_DIR}/scripts/minikube/t2-common.sh"
+# shellcheck source=scripts/minikube/docker-cli-env.sh
+source "${PROJECT_DIR}/scripts/minikube/docker-cli-env.sh"
+# shellcheck source=real-postgres-local-preflight.sh
+source "${SCRIPT_DIR}/real-postgres-local-preflight.sh"
+
+GFS_DOCKER_RUN_TIMEOUT_SECONDS="${GFS_DOCKER_RUN_TIMEOUT_SECONDS:-${MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS}}"
+GFS_DOCKER_EXEC_TIMEOUT_SECONDS="${GFS_DOCKER_EXEC_TIMEOUT_SECONDS:-${MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS}}"
+GFS_DOCKER_REMOVE_TIMEOUT_SECONDS="${GFS_DOCKER_REMOVE_TIMEOUT_SECONDS:-${MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS}}"
die() {
printf '[gfs-real-pg-minikube] ERROR: %s\n' "$*" >&2
@@ -102,38 +110,38 @@ PY
}
PROFILE_PG_FORWARD_PID=""
+PROFILE_PG_FORWARD_RECORD=""
stop_profile_postgres_forward() {
[[ -n "${PROFILE_PG_FORWARD_PID}" ]] || return 0
- if ! kill -0 "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1; then
- PROFILE_PG_FORWARD_PID=''
- return 0
- fi
- local command_line
- command_line="$(ps -p "${PROFILE_PG_FORWARD_PID}" -o command= 2>/dev/null || true)"
- if [[ "${command_line}" != *port-forward* || "${command_line}" != *svc/control-postgres* ]]; then
- printf '[gfs-real-pg-minikube] ERROR: profile PostgreSQL forward PID is not the expected child; refusing cleanup success\n' >&2
+ [[ -n "${PROFILE_PG_FORWARD_RECORD}" ]] || {
+ printf '[gfs-real-pg-minikube] ERROR: profile PostgreSQL forward has no structured ownership record; refusing ambiguous cleanup\n' >&2
return 1
- fi
- if ! kill "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1 && kill -0 "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1; then
- return 1
- fi
- if kill -0 "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1; then
- if wait "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1; then
- :
+ }
+ if [[ ! -e "${PROFILE_PG_FORWARD_RECORD}" && ! -L "${PROFILE_PG_FORWARD_RECORD}" ]]; then
+ if [[ "$(pf_owner_process_state "${PROFILE_PG_FORWARD_PID}")" == dead ]]; then
+ PROFILE_PG_FORWARD_PID=''
+ PROFILE_PG_FORWARD_RECORD=''
+ return 0
fi
+ printf '[gfs-real-pg-minikube] ERROR: profile PostgreSQL forward record disappeared while its process may still be live\n' >&2
+ return 1
fi
- if kill -0 "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1; then
+ if ! pf_owner_cleanup_record "${PROFILE_PG_FORWARD_RECORD}" "${T2_PROFILE}" \
+ "${T2_CONTEXT}" "${PROJECT_DIR}" "${CONTROL_NS}" control-postgres \
+ "${PROFILE_PG_FORWARD_PORT}" 5432; then
+ printf '[gfs-real-pg-minikube] ERROR: profile PostgreSQL forward ownership validation or cleanup failed\n' >&2
return 1
fi
PROFILE_PG_FORWARD_PID=''
+ PROFILE_PG_FORWARD_RECORD=''
}
verify_profile_postgres() {
kc -n "${CONTROL_NS}" rollout status deployment/control-postgres --timeout="${TIMEOUT}s" >/dev/null 2>&1 || \
die 'branch-profile control-postgres did not become Ready'
- local key encoded profile_port
+ local key encoded
for key in POSTGRES_USER POSTGRES_PASSWORD POSTGRES_DB; do
encoded="$(kc -n "${CONTROL_NS}" get secret control-postgres -o "jsonpath={.data.${key}}" 2>/dev/null || true)"
[[ -n "${encoded}" ]] || die "branch-profile control-postgres Secret is missing ${key}"
@@ -141,17 +149,27 @@ verify_profile_postgres() {
die "branch-profile control-postgres Secret key is not valid base64: ${key}"
done
- profile_port="$(choose_local_port)"
+ PROFILE_PG_FORWARD_PORT="$(choose_local_port)"
kubectl --context="${CONTEXT}" -n "${CONTROL_NS}" port-forward \
- --address=127.0.0.1 svc/control-postgres "${profile_port}:5432" \
+ --address=127.0.0.1 svc/control-postgres "${PROFILE_PG_FORWARD_PORT}:5432" \
>"${TMP_DIR}/profile-postgres-port-forward.log" 2>&1 &
PROFILE_PG_FORWARD_PID=$!
- if ! wait_for_tcp "${profile_port}"; then
+ PROFILE_PG_FORWARD_RECORD="${TMP_DIR}/profile-postgres-forward.pid"
+ pf_owner_pause 0.2
+ if ! pf_owner_record_process "${PROFILE_PG_FORWARD_RECORD}" \
+ "${PROFILE_PG_FORWARD_PID}" "${T2_PROFILE}" "${T2_CONTEXT}" \
+ "${PROJECT_DIR}" "${CONTROL_NS}" control-postgres \
+ "${PROFILE_PG_FORWARD_PORT}" 5432; then
+ pf_owner_abort_child "${PROFILE_PG_FORWARD_PID}" "${T2_CONTEXT}" \
+ "${CONTROL_NS}" control-postgres "${PROFILE_PG_FORWARD_PORT}" 5432 || true
+ die 'branch-profile control-postgres port-forward ownership could not be proven'
+ fi
+ if ! wait_for_tcp "${PROFILE_PG_FORWARD_PORT}"; then
sanitize_file "${TMP_DIR}/profile-postgres-port-forward.log"
cat "${TMP_DIR}/profile-postgres-port-forward.log" >&2 || true
die 'branch-profile control-postgres port-forward did not become reachable'
fi
- if ! kill -0 "${PROFILE_PG_FORWARD_PID}" >/dev/null 2>&1; then
+ if [[ "$(pf_owner_process_state "${PROFILE_PG_FORWARD_PID}")" != live ]]; then
sanitize_file "${TMP_DIR}/profile-postgres-port-forward.log"
cat "${TMP_DIR}/profile-postgres-port-forward.log" >&2 || true
die 'branch-profile control-postgres port-forward exited before the GFS lane started'
@@ -179,7 +197,7 @@ real_postgres_suite_files() {
run_suite() {
local package="$1" suite_file relative_suite log_file json_file
- local total_files failed_files pending_files total_tests passed_tests pending_tests success
+ local total_files failed_files pending_files total_tests passed_tests pending_tests success reported_files
local suite_index=0 suite_count=0
local suite_files=()
while IFS= read -r suite_file; do
@@ -200,9 +218,10 @@ run_suite() {
cd "${PROJECT_DIR}/${package}"
env \
CONTROL_API_REAL_PG_ADMIN_URL="${ADMIN_DSN}" \
- CONTROL_API_REAL_PG_REQUIRED=1 \
+ CONTROL_API_REAL_PG_REQUIRED=1 VITEST_MAX_WORKERS=1 \
FORCE_COLOR=0 NO_COLOR=1 \
- npm test -- --run "${relative_suite}" --reporter=json --outputFile="${json_file}"
+ npm test -- --run --no-file-parallelism --maxWorkers=1 \
+ "${relative_suite}" --reporter=json --outputFile="${json_file}"
) >"${log_file}" 2>&1; then
sanitize_file "${log_file}"
sanitize_file "${json_file}"
@@ -214,12 +233,22 @@ run_suite() {
sanitize_file "${log_file}"
sanitize_file "${json_file}"
[[ -s "${json_file}" ]] || die "${package}/${relative_suite} reporter produced no JSON result"
- read -r total_files _ failed_files pending_files total_tests passed_tests pending_tests success < <(python3 - "${json_file}" <<'PY'
+ read -r total_files _ failed_files pending_files total_tests passed_tests pending_tests success reported_files < <(python3 - "${json_file}" "${PROJECT_DIR}/${package}" "${relative_suite}" <<'PY'
import json
import sys
from pathlib import Path
result = json.loads(Path(sys.argv[1]).read_text())
+package_root = Path(sys.argv[2]).resolve()
+expected = (package_root / sys.argv[3]).resolve()
+test_results = result.get("testResults")
+reported = []
+if isinstance(test_results, list):
+ for item in test_results:
+ name = item.get("name") if isinstance(item, dict) else None
+ if isinstance(name, str) and name:
+ candidate = Path(name)
+ reported.append((candidate if candidate.is_absolute() else package_root / candidate).resolve())
print(
result.get("numTotalTestSuites", 0),
@@ -230,12 +259,14 @@ print(
result.get("numPassedTests", 0),
result.get("numPendingTests", 0),
str(bool(result.get("success"))).lower(),
+ len(reported) if reported == [expected] else -1,
)
PY
)
if [[ "${success}" != 'true' || "${total_files}" -le 0 || "${failed_files}" -ne 0 ||
"${pending_files}" -ne 0 || "${total_tests}" -le 0 ||
- "${passed_tests}" -ne "${total_tests}" || "${pending_tests}" -ne 0 ]]; then
+ "${passed_tests}" -ne "${total_tests}" || "${pending_tests}" -ne 0 ||
+ "${reported_files}" -ne 1 ]]; then
cat "${log_file}" >&2 || true
cat "${json_file}" >&2 || true
die "${package}/${relative_suite} reporter did not pass all tests"
@@ -250,6 +281,8 @@ TMP_DIR="$(mktemp -d)"
ISOLATED_CONTAINER=""
ISOLATED_PORT=''
ADMIN_DSN=''
+PROFILE_PG_FORWARD_PORT=''
+GFS_DOCKER_ENV_PREPARED=false
GFS_RESTORE_REQUIRED=false
restore_gfs_runtime_credentials() {
# The reader-role real-Postgres suite exercises the production role names,
@@ -271,18 +304,40 @@ restore_gfs_runtime_credentials() {
}
stop_isolated_postgres() {
[[ -n "${ISOLATED_CONTAINER}" ]] || return 0
- if docker rm -f "${ISOLATED_CONTAINER}" >/dev/null 2>&1; then
+ [[ "${GFS_DOCKER_ENV_PREPARED}" == true ]] || return 1
+ if docker_cli_run_public gfs-t1-postgres-remove \
+ "${GFS_DOCKER_REMOVE_TIMEOUT_SECONDS}" \
+ docker rm -f "${ISOLATED_CONTAINER}" >/dev/null 2>&1; then
ISOLATED_CONTAINER=''
return 0
fi
return 1
}
+prepare_gfs_docker() {
+ docker_cli_env_validate_seconds GFS_DOCKER_RUN_TIMEOUT_SECONDS \
+ "${GFS_DOCKER_RUN_TIMEOUT_SECONDS}" 3600 || return $?
+ docker_cli_env_validate_seconds GFS_DOCKER_EXEC_TIMEOUT_SECONDS \
+ "${GFS_DOCKER_EXEC_TIMEOUT_SECONDS}" 300 || return $?
+ docker_cli_env_validate_seconds GFS_DOCKER_REMOVE_TIMEOUT_SECONDS \
+ "${GFS_DOCKER_REMOVE_TIMEOUT_SECONDS}" 300 || return $?
+ docker_cli_env_prepare false || return $?
+ GFS_DOCKER_ENV_PREPARED=true
+}
+cleanup_gfs_docker_env() {
+ [[ "${GFS_DOCKER_ENV_PREPARED}" == true ]] || return 0
+ if docker_cli_env_cleanup; then
+ GFS_DOCKER_ENV_PREPARED=false
+ return 0
+ fi
+ return 1
+}
cleanup() {
local status=$?
trap - EXIT
trap '' INT TERM
if ! stop_profile_postgres_forward; then status=1; fi
if ! stop_isolated_postgres; then status=1; fi
+ if ! cleanup_gfs_docker_env; then status=1; fi
if [ "${GFS_RESTORE_REQUIRED}" = true ] && ! restore_gfs_runtime_credentials; then status=1; fi
if ! rm -rf "${TMP_DIR}"; then status=1; fi
if ! t2_lock_release "${status}"; then status=1; fi
@@ -301,14 +356,17 @@ sock.close()
PY
}
-require_command docker npm
-docker info >/dev/null 2>&1 || die 'Docker is required for the isolated GFS real-Postgres lane'
+if ! real_pg_local_preflight "${PROJECT_DIR}" true; then
+ die "${REAL_PG_PREFLIGHT_ERROR_CODE}: ${REAL_PG_PREFLIGHT_ERROR_MESSAGE}"
+fi
verify_branch_gate
verify_profile_postgres
ISOLATED_PORT="$(choose_local_port)"
ISOLATED_CONTAINER="evenfire-gfs-t1-pg-$$"
-docker run -d --rm --name "${ISOLATED_CONTAINER}" \
+prepare_gfs_docker || die 'isolated GFS Docker runtime could not be prepared safely'
+docker_cli_run_public gfs-t1-postgres-run "${GFS_DOCKER_RUN_TIMEOUT_SECONDS}" \
+ docker run -d --rm --name "${ISOLATED_CONTAINER}" \
-e POSTGRES_USER=postgres \
-e POSTGRES_DB=postgres \
-e POSTGRES_HOST_AUTH_METHOD=trust \
@@ -317,12 +375,16 @@ docker run -d --rm --name "${ISOLATED_CONTAINER}" \
wait_for_tcp "${ISOLATED_PORT}" || die 'isolated GFS PostgreSQL did not become reachable'
deadline=$((SECONDS + TIMEOUT))
while (( SECONDS < deadline )); do
- if docker exec "${ISOLATED_CONTAINER}" pg_isready -U postgres >/dev/null 2>&1; then
+ if docker_cli_run_public gfs-t1-postgres-ready-probe \
+ "${GFS_DOCKER_EXEC_TIMEOUT_SECONDS}" \
+ docker exec "${ISOLATED_CONTAINER}" pg_isready -U postgres >/dev/null 2>&1; then
break
fi
sleep 1
done
-docker exec "${ISOLATED_CONTAINER}" pg_isready -U postgres >/dev/null 2>&1 ||
+docker_cli_run_public gfs-t1-postgres-ready-final \
+ "${GFS_DOCKER_EXEC_TIMEOUT_SECONDS}" \
+ docker exec "${ISOLATED_CONTAINER}" pg_isready -U postgres >/dev/null 2>&1 ||
die 'isolated GFS PostgreSQL did not become ready'
ADMIN_DSN="$(printf 'postgresql://postgres@%s:%s/postgres' "${ISOLATED_HOST}" "${ISOLATED_PORT}")"
diff --git a/scripts/e2e/minikube-real-postgres.sh b/scripts/e2e/minikube-real-postgres.sh
index 008712ab3..9ccab8510 100755
--- a/scripts/e2e/minikube-real-postgres.sh
+++ b/scripts/e2e/minikube-real-postgres.sh
@@ -7,6 +7,10 @@ set +x
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
PROJECT_DIR="$(cd -- "$SCRIPT_DIR/../.." && pwd -P)"
source "$PROJECT_DIR/scripts/minikube/t2-common.sh"
+# shellcheck source=scripts/minikube/docker-cli-env.sh
+source "$PROJECT_DIR/scripts/minikube/docker-cli-env.sh"
+# shellcheck source=real-postgres-local-preflight.sh
+source "$SCRIPT_DIR/real-postgres-local-preflight.sh"
PG_NAMESPACE="$T2_CONTROL_NAMESPACE"
PG_SERVICE=control-postgres
@@ -16,16 +20,21 @@ PG_SECRET=control-postgres
# throwaway PostgreSQL 16 that matches CI; the shared branch-profile database
# is reserved for the non-control-api lane.
T1_ISOLATED_PG_IMAGE="${T1_ISOLATED_PG_IMAGE:-postgres:16-alpine}"
+T1_DOCKER_RUN_TIMEOUT_SECONDS="${T1_DOCKER_RUN_TIMEOUT_SECONDS:-$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS}"
+T1_DOCKER_EXEC_TIMEOUT_SECONDS="${T1_DOCKER_EXEC_TIMEOUT_SECONDS:-$MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS}"
+T1_DOCKER_REMOVE_TIMEOUT_SECONDS="${T1_DOCKER_REMOVE_TIMEOUT_SECONDS:-$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS}"
T1_TIMEOUT="$T2_TIMEOUT_SECONDS"
T1_TMP_ROOT="$T2_TMP_ROOT"
if [ -z "$T1_TMP_ROOT" ]; then T1_TMP_ROOT=/tmp; fi
T1_TMP_DIR=""
PORT_FORWARD_PID=""
+PORT_FORWARD_RECORD=""
LOCAL_PORT=""
ADMIN_DSN=""
ISOLATED_DSN=""
ISOLATED_CONTAINER=""
ISOLATED_PORT=""
+T1_DOCKER_ENV_PREPARED=false
PG_USER=""
PG_PASSWORD=""
PG_DATABASE=""
@@ -35,6 +44,10 @@ T1_NEXT_COMMAND='repair the first reported Real PostgreSQL precondition, then re
T1_TOTAL_TESTS=0
T1_PASSED_TESTS=0
T1_PENDING_TESTS=0
+T1_EXPECTED_FILES=0
+T1_REPORTED_FILES=0
+T1_STARTED_SECONDS="$SECONDS"
+T1_DURATION_SECONDS=0
# Armed just before the Real PostgreSQL suites run. The exit trap must not
# reconcile cluster credentials for a run that never reached the suites
# (e.g. a failed precondition, or this file being sourced by a harness test).
@@ -55,35 +68,58 @@ die_t1() {
stop_isolated_postgres() {
[ -n "$ISOLATED_CONTAINER" ] || return 0
- if docker rm -f "$ISOLATED_CONTAINER" >/dev/null 2>&1; then
+ [ "$T1_DOCKER_ENV_PREPARED" = true ] || return 1
+ if docker_cli_run_public t1-postgres-remove "$T1_DOCKER_REMOVE_TIMEOUT_SECONDS" \
+ docker rm -f "$ISOLATED_CONTAINER" >/dev/null 2>&1; then
ISOLATED_CONTAINER=""
return 0
fi
return 1
}
-stop_control_postgres_forward() {
- [ -n "$PORT_FORWARD_PID" ] || return 0
- if ! kill -0 "$PORT_FORWARD_PID" >/dev/null 2>&1; then
+cleanup_t1_docker_env() {
+ [ "$T1_DOCKER_ENV_PREPARED" = true ] || return 0
+ if docker_cli_env_cleanup; then
+ T1_DOCKER_ENV_PREPARED=false
return 0
fi
- local command_line
- command_line="$(ps -p "$PORT_FORWARD_PID" -o command= 2>/dev/null || true)"
- if [[ "$command_line" != *port-forward* || "$command_line" != *svc/control-postgres* ]]; then
- printf '[minikube-t1] ERROR: T1 port-forward PID is not the expected control-postgres child; refusing to report cleanup success\n' >&2
- return 1
- fi
- if ! kill "$PORT_FORWARD_PID" >/dev/null 2>&1 && kill -0 "$PORT_FORWARD_PID" >/dev/null 2>&1; then
+ return 1
+}
+
+prepare_t1_docker() {
+ docker_cli_env_validate_seconds T1_DOCKER_RUN_TIMEOUT_SECONDS \
+ "$T1_DOCKER_RUN_TIMEOUT_SECONDS" 3600 || return $?
+ docker_cli_env_validate_seconds T1_DOCKER_EXEC_TIMEOUT_SECONDS \
+ "$T1_DOCKER_EXEC_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds T1_DOCKER_REMOVE_TIMEOUT_SECONDS \
+ "$T1_DOCKER_REMOVE_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_prepare false || return $?
+ T1_DOCKER_ENV_PREPARED=true
+}
+
+stop_control_postgres_forward() {
+ [ -n "$PORT_FORWARD_PID" ] || return 0
+ [ -n "$PORT_FORWARD_RECORD" ] || {
+ printf '[minikube-t1] ERROR: T1 port-forward has no structured ownership record; refusing ambiguous cleanup\n' >&2
return 1
- fi
- if kill -0 "$PORT_FORWARD_PID" >/dev/null 2>&1; then
- if wait "$PORT_FORWARD_PID" >/dev/null 2>&1; then
- :
+ }
+ if [ ! -e "$PORT_FORWARD_RECORD" ] && [ ! -L "$PORT_FORWARD_RECORD" ]; then
+ if [ "$(pf_owner_process_state "$PORT_FORWARD_PID")" = dead ]; then
+ PORT_FORWARD_PID=""
+ PORT_FORWARD_RECORD=""
+ return 0
fi
+ printf '[minikube-t1] ERROR: T1 port-forward ownership record disappeared while its process may still be live\n' >&2
+ return 1
fi
- if kill -0 "$PORT_FORWARD_PID" >/dev/null 2>&1; then
+ if ! pf_owner_cleanup_record "$PORT_FORWARD_RECORD" "$T2_PROFILE" \
+ "$T2_CONTEXT" "$PROJECT_DIR" "$PG_NAMESPACE" "$PG_SERVICE" \
+ "$LOCAL_PORT" 5432; then
+ printf '[minikube-t1] ERROR: T1 port-forward ownership validation or cleanup failed\n' >&2
return 1
fi
+ PORT_FORWARD_PID=""
+ PORT_FORWARD_RECORD=""
return 0
}
@@ -154,13 +190,17 @@ cleanup_t1() {
if ! stop_isolated_postgres; then
status=1
fi
+ if ! cleanup_t1_docker_env; then
+ status=1
+ fi
if ! restore_gfs_runtime_credentials; then
status=1
fi
+ T1_DURATION_SECONDS=$((SECONDS - T1_STARTED_SECONDS))
if [ -n "$T1_TMP_DIR" ] && [ -d "$T1_TMP_DIR" ]; then rm -rf "$T1_TMP_DIR"; fi
if [ "$status" -eq 0 ] && [ "$T1_STATUS" = PASS ]; then
- t2_evidence_write T1 PASS "Real PostgreSQL suites passed; tests=$T1_TOTAL_TESTS passed=$T1_PASSED_TESTS pending=$T1_PENDING_TESTS; cleanup=PASS"
- t2_evidence_write complete PASS "T1=PASS tests=$T1_TOTAL_TESTS passed=$T1_PASSED_TESTS pending=$T1_PENDING_TESTS cleanup=PASS"
+ t2_evidence_write T1 PASS "Real PostgreSQL suites passed; files=$T1_REPORTED_FILES/$T1_EXPECTED_FILES tests=$T1_TOTAL_TESTS passed=$T1_PASSED_TESTS pending=$T1_PENDING_TESTS duration=${T1_DURATION_SECONDS}s; cleanup=PASS"
+ t2_evidence_write complete PASS "T1=PASS files=$T1_REPORTED_FILES/$T1_EXPECTED_FILES tests=$T1_TOTAL_TESTS passed=$T1_PASSED_TESTS pending=$T1_PENDING_TESTS duration=${T1_DURATION_SECONDS}s cleanup=PASS"
elif [ "$T1_STATUS" = PASS ]; then
t2_evidence_write T1 FAIL "Real PostgreSQL suites passed but cleanup failed; cleanup=FAIL"
fi
@@ -172,16 +212,9 @@ trap 'handle_t1_signal INT' INT
trap 'handle_t1_signal TERM' TERM
require_t1_commands() {
- local command_name
- for command_name in npm node docker; do
- if ! command -v "$command_name" >/dev/null 2>&1; then
- T1_NEXT_COMMAND="install or enable $command_name, then re-run the Real PostgreSQL lane"
- die_t1 LOCAL_DEPENDENCY_MISSING "required local dependency is unavailable: $command_name"
- fi
- done
- if ! docker info >/dev/null 2>&1; then
- T1_NEXT_COMMAND='start Docker Desktop or the Docker daemon, then re-run T1'
- die_t1 LOCAL_DEPENDENCY_MISSING 'Docker is required to start the isolated T1 PostgreSQL'
+ if ! real_pg_local_preflight "$PROJECT_DIR" true; then
+ T1_NEXT_COMMAND='repair the reported local prerequisite, then re-run the standalone Real PostgreSQL lane'
+ die_t1 "$REAL_PG_PREFLIGHT_ERROR_CODE" "$REAL_PG_PREFLIGHT_ERROR_MESSAGE"
fi
}
@@ -275,9 +308,15 @@ require_isolated_control_api_files() {
}
start_isolated_postgres() {
+ if ! prepare_t1_docker; then
+ T1_NEXT_COMMAND='repair the isolated Docker runtime configuration, then re-run T1'
+ die_t1 REAL_PG_REQUIRED_BUT_UNAVAILABLE \
+ 'isolated T1 Docker runtime could not be prepared safely'
+ fi
ISOLATED_PORT="$(choose_local_port)"
ISOLATED_CONTAINER="evenfire-t1-isolated-pg-$$"
- if ! docker run -d --rm --name "$ISOLATED_CONTAINER" \
+ if ! docker_cli_run_public t1-postgres-run "$T1_DOCKER_RUN_TIMEOUT_SECONDS" \
+ docker run -d --rm --name "$ISOLATED_CONTAINER" \
-e POSTGRES_USER=postgres \
-e POSTGRES_DB=postgres \
-e POSTGRES_HOST_AUTH_METHOD=trust \
@@ -294,7 +333,8 @@ start_isolated_postgres() {
local deadline ready=false
deadline=$((SECONDS + T1_TIMEOUT))
while [ "$SECONDS" -lt "$deadline" ]; do
- if docker exec "$ISOLATED_CONTAINER" pg_isready -U postgres >/dev/null 2>&1; then
+ if docker_cli_run_public t1-postgres-ready-probe "$T1_DOCKER_EXEC_TIMEOUT_SECONDS" \
+ docker exec "$ISOLATED_CONTAINER" pg_isready -U postgres >/dev/null 2>&1; then
ready=true
break
fi
@@ -314,7 +354,8 @@ print("postgresql://[email protected]:" + port + "/postgres", end="")
run_suite() {
local package="$1" lane="$2" admin_url="$3"
- local expected_files=0 file log_file json_file stats
+ local expected_files=0 reported_files=0 file log_file json_file stats
+ local suite_started_seconds="$SECONDS" suite_duration_seconds=0
local -a files=() vitest_filters=()
while IFS= read -r file; do
@@ -354,8 +395,9 @@ run_suite() {
(
cd "$PROJECT_DIR/$package"
CONTROL_API_REAL_PG_ADMIN_URL="$admin_url" \
- CONTROL_API_REAL_PG_REQUIRED=1 FORCE_COLOR=0 NO_COLOR=1 \
- npm test -- --run "${vitest_filters[@]}" --reporter=json --outputFile="$json_file"
+ CONTROL_API_REAL_PG_REQUIRED=1 FORCE_COLOR=0 NO_COLOR=1 VITEST_MAX_WORKERS=1 \
+ npm test -- --run --no-file-parallelism --maxWorkers=1 \
+ "${vitest_filters[@]}" --reporter=json --outputFile="$json_file"
) >"$log_file" 2>&1 || suite_status=$?
sanitize_file "$log_file"
sanitize_file "$json_file"
@@ -364,14 +406,37 @@ run_suite() {
T1_NEXT_COMMAND='inspect the Vitest reporter output, then re-run T1'
die_t1 ZERO_TESTS_EXECUTED "Real PostgreSQL reporter produced no result for $package ($lane)"
}
- # Vitest's suite counters include nested describe blocks, so they are not
- # comparable to the number of physical *realPostgres*.test.ts files above.
- # Validate the reporter's own total/passed suite counts instead; the file
- # discovery count remains a separate zero-selection guard.
- if ! stats="$(python3 - "$json_file" <<'PY'
+ # Vitest's suite counters include nested describe blocks, so compare the
+ # physical selection with testResults[].name instead of suite counters.
+ # This proves that every discovered file, and no unexpected file, ran.
+ if ! stats="$(python3 - "$json_file" "$PROJECT_DIR/$package" "${vitest_filters[@]}" <<'PY'
import json
import sys
-result = json.loads(open(sys.argv[1]).read())
+from pathlib import Path
+
+result = json.loads(Path(sys.argv[1]).read_text())
+package_root = Path(sys.argv[2]).resolve()
+expected = [(package_root / value).resolve() for value in sys.argv[3:]]
+test_results = result.get("testResults")
+if not isinstance(test_results, list) or not test_results:
+ print("reporter testResults is missing or empty", file=sys.stderr)
+ sys.exit(2)
+reported = []
+for item in test_results:
+ name = item.get("name") if isinstance(item, dict) else None
+ if not isinstance(name, str) or not name:
+ print("reporter contains a test result without a physical file name", file=sys.stderr)
+ sys.exit(2)
+ path = Path(name)
+ reported.append((path if path.is_absolute() else package_root / path).resolve())
+if len(reported) != len(set(reported)):
+ print("reporter contains duplicate physical file names", file=sys.stderr)
+ sys.exit(2)
+if set(reported) != set(expected):
+ missing = sorted(str(path) for path in set(expected) - set(reported))
+ unexpected = sorted(str(path) for path in set(reported) - set(expected))
+ print(f"reporter physical file mismatch missing={missing} unexpected={unexpected}", file=sys.stderr)
+ sys.exit(2)
total_suites = int(result.get("numTotalTestSuites") or 0)
passed_suites = int(result.get("numPassedTestSuites") or 0)
failed_suites = int(result.get("numFailedTestSuites") or 0)
@@ -382,16 +447,19 @@ pending_tests = int(result.get("numPendingTests") or 0)
total_tests = int(result.get("numTotalTests") or 0)
success = bool(result.get("success"))
print(total_suites, passed_suites, failed_suites, passed_tests, failed_tests,
- pending_files, pending_tests, total_tests, int(success))
+ pending_files, pending_tests, total_tests, int(success), len(expected),
+ len(reported))
PY
)"; then
cat "$log_file" >&2 || true
cat "$json_file" >&2 || true
T1_NEXT_COMMAND='inspect the Vitest reporter output, then re-run T1'
- die_t1 ZERO_TESTS_EXECUTED "Real PostgreSQL reporter was unreadable for $package ($lane)"
+ die_t1 REAL_PG_REPORT_INCOMPLETE "Real PostgreSQL reporter did not prove the exact file set for $package ($lane)"
fi
read -r total_suites passed_suites failed_suites passed_tests failed_tests \
- pending_files pending_tests total_tests success <<< "$stats"
+ pending_files pending_tests total_tests success expected_files reported_files <<< "$stats"
+ T1_EXPECTED_FILES=$((T1_EXPECTED_FILES + expected_files))
+ T1_REPORTED_FILES=$((T1_REPORTED_FILES + reported_files))
T1_TOTAL_TESTS=$((T1_TOTAL_TESTS + total_tests))
T1_PASSED_TESTS=$((T1_PASSED_TESTS + passed_tests))
T1_PENDING_TESTS=$((T1_PENDING_TESTS + pending_tests))
@@ -413,8 +481,9 @@ PY
T1_NEXT_COMMAND='repair the failed or incomplete Real PostgreSQL lane, then re-run T1'
die_t1 REAL_PG_SUITE_FAILED "Vitest process exited $suite_status for $package ($lane)"
fi
- printf '[minikube-t1] PASS %s %s files=%s tests=%s skipped=0\n' \
- "$package" "$lane" "$expected_files" "$passed_tests"
+ suite_duration_seconds=$((SECONDS - suite_started_seconds))
+ printf '[minikube-t1] PASS %s %s files=%s/%s tests=%s skipped=0 duration=%ss\n' \
+ "$package" "$lane" "$reported_files" "$expected_files" "$passed_tests" "$suite_duration_seconds"
}
main() {
@@ -466,13 +535,26 @@ main() {
--address=127.0.0.1 svc/"$PG_SERVICE" "$LOCAL_PORT:5432" \
>"$T1_TMP_DIR/port-forward.log" 2>&1 &
PORT_FORWARD_PID=$!
+ PORT_FORWARD_RECORD="$T1_TMP_DIR/control-postgres-forward.pid"
+ # Give the direct child one bounded scheduling turn to exec kubectl before
+ # recording exact argv/process-start identity. If that identity cannot be
+ # proven, signal only the still-owned shell child and fail the lane.
+ pf_owner_pause 0.2
+ if ! pf_owner_record_process "$PORT_FORWARD_RECORD" "$PORT_FORWARD_PID" \
+ "$T2_PROFILE" "$T2_CONTEXT" "$PROJECT_DIR" "$PG_NAMESPACE" \
+ "$PG_SERVICE" "$LOCAL_PORT" 5432; then
+ pf_owner_abort_child "$PORT_FORWARD_PID" "$T2_CONTEXT" "$PG_NAMESPACE" \
+ "$PG_SERVICE" "$LOCAL_PORT" 5432 || true
+ T1_NEXT_COMMAND='repair the profile-owned port-forward and re-run T1'
+ die_t1 REAL_PG_REQUIRED_BUT_UNAVAILABLE 'control-postgres port-forward ownership could not be proven'
+ fi
if ! wait_for_tcp "$LOCAL_PORT"; then
sanitize_file "$T1_TMP_DIR/port-forward.log"
cat "$T1_TMP_DIR/port-forward.log" >&2 || true
T1_NEXT_COMMAND='repair the profile-owned port-forward and re-run T1'
die_t1 REAL_PG_REQUIRED_BUT_UNAVAILABLE 'control-postgres port-forward did not become reachable'
fi
- if ! kill -0 "$PORT_FORWARD_PID" >/dev/null 2>&1; then
+ if [ "$(pf_owner_process_state "$PORT_FORWARD_PID")" != live ]; then
sanitize_file "$T1_TMP_DIR/port-forward.log"
cat "$T1_TMP_DIR/port-forward.log" >&2 || true
T1_NEXT_COMMAND='repair the profile-owned port-forward and re-run T1'
@@ -498,7 +580,16 @@ print("postgresql://" + quote(user.decode(), safe="") + ":" + quote(password.dec
run_suite gfs-controller shared "$ADMIN_DSN"
unset PG_USER PG_PASSWORD PG_DATABASE T1_REDACT_PASSWORD
unset ADMIN_DSN ISOLATED_DSN
- stop_isolated_postgres
+ if ! stop_isolated_postgres; then
+ T1_NEXT_COMMAND='remove the isolated T1 PostgreSQL container, then re-run T1'
+ die_t1 REAL_PG_REQUIRED_BUT_UNAVAILABLE \
+ 'isolated T1 PostgreSQL cleanup failed or exceeded its deadline'
+ fi
+ if ! cleanup_t1_docker_env; then
+ T1_NEXT_COMMAND='repair the task-local Docker configuration cleanup, then re-run T1'
+ die_t1 REAL_PG_REQUIRED_BUT_UNAVAILABLE \
+ 'isolated T1 Docker runtime cleanup failed'
+ fi
T1_STATUS=PASS
# PASS evidence is finalized by cleanup_t1 only after the profile restore and
# lock release path has completed successfully.
@@ -506,6 +597,9 @@ print("postgresql://" + quote(user.decode(), safe="") + ":" + quote(password.dec
printf 'T1_TESTS=%s\n' "$T1_TOTAL_TESTS"
printf 'T1_PASSED_TESTS=%s\n' "$T1_PASSED_TESTS"
printf 'T1_PENDING_TESTS=%s\n' "$T1_PENDING_TESTS"
+ printf 'T1_EXPECTED_FILES=%s\n' "$T1_EXPECTED_FILES"
+ printf 'T1_REPORTED_FILES=%s\n' "$T1_REPORTED_FILES"
+ printf 'T1_DURATION_SECONDS=%s\n' "$((SECONDS - T1_STARTED_SECONDS))"
printf 'T1_EVIDENCE=%s\n' "$T2_EVIDENCE_FILE"
printf '[minikube-t1] T1 PASS: Real PostgreSQL executed against the verified local profile\n'
}
diff --git a/scripts/e2e/real-postgres-local-preflight.sh b/scripts/e2e/real-postgres-local-preflight.sh
new file mode 100755
index 000000000..fa653dae9
--- /dev/null
+++ b/scripts/e2e/real-postgres-local-preflight.sh
@@ -0,0 +1,117 @@
+#!/usr/bin/env bash
+# Fast, secret-safe local prerequisites for the Real PostgreSQL T1 lanes.
+# Source this file and call real_pg_local_preflight before any expensive T0 or
+# Minikube work. The helper never installs dependencies or mutates a cluster.
+
+REAL_PG_PREFLIGHT_ERROR_CODE=""
+REAL_PG_PREFLIGHT_ERROR_MESSAGE=""
+REAL_PG_PREFLIGHT_DURATION_SECONDS=0
+REAL_PG_PREFLIGHT_SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+REAL_PG_DOCKER_HELPER="$REAL_PG_PREFLIGHT_SCRIPT_DIR/../minikube/docker-cli-env.sh"
+
+real_pg_preflight_fail() {
+ REAL_PG_PREFLIGHT_ERROR_CODE="$1"
+ REAL_PG_PREFLIGHT_ERROR_MESSAGE="$2"
+ return 1
+}
+
+real_pg_local_preflight() {
+ local project_dir="${1:-}"
+ local require_docker="${2:-true}"
+ local started_seconds="${SECONDS:-0}"
+ local command_name node_major package package_dir docker_timeout docker_kill_grace
+
+ REAL_PG_PREFLIGHT_ERROR_CODE=""
+ REAL_PG_PREFLIGHT_ERROR_MESSAGE=""
+ REAL_PG_PREFLIGHT_DURATION_SECONDS=0
+
+ if [[ -z "$project_dir" || ! -d "$project_dir" ]]; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ 'Real PostgreSQL preflight requires the repository root'
+ return 1
+ fi
+
+ for command_name in node npm python3; do
+ if ! command -v "$command_name" >/dev/null 2>&1; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ "required local dependency is unavailable: $command_name"
+ return 1
+ fi
+ done
+
+ node_major="$(node -p 'Number(process.versions.node.split(".")[0])' 2>/dev/null || true)"
+ if ! [[ "$node_major" =~ ^[0-9]+$ ]] || (( node_major < 24 )); then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ "Node.js >=24 is required for T1; found ${node_major:-unknown}"
+ return 1
+ fi
+
+ if [[ -n "${VITEST_MAX_WORKERS:-}" && "${VITEST_MAX_WORKERS}" != 1 ]]; then
+ real_pg_preflight_fail UNSUPPORTED_T1_CONCURRENCY \
+ "T1 is serial by contract; VITEST_MAX_WORKERS=${VITEST_MAX_WORKERS} is unsupported"
+ return 1
+ fi
+ export VITEST_MAX_WORKERS=1
+
+ for package in control-api gfs-controller; do
+ package_dir="$project_dir/$package"
+ if [[ ! -f "$package_dir/package.json" || ! -x "$package_dir/node_modules/.bin/vitest" ]]; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ "$package local dependencies are missing; run the repository-approved install before T1"
+ return 1
+ fi
+ if ! (cd "$package_dir" && node -e 'require.resolve("vitest"); require.resolve("pg")') >/dev/null 2>&1; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ "$package cannot resolve its local vitest and pg dependencies"
+ return 1
+ fi
+ done
+
+ if [[ "$require_docker" == true ]]; then
+ if ! command -v docker >/dev/null 2>&1; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ 'required local dependency is unavailable: docker'
+ return 1
+ fi
+ docker_timeout="${REAL_PG_DOCKER_INFO_TIMEOUT_SECONDS:-15}"
+ if ! [[ "$docker_timeout" =~ ^[1-9][0-9]*$ ]] || (( docker_timeout > 120 )); then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ 'REAL_PG_DOCKER_INFO_TIMEOUT_SECONDS must be an integer from 1 to 120'
+ return 1
+ fi
+ docker_kill_grace="${REAL_PG_DOCKER_KILL_GRACE_SECONDS:-1}"
+ if ! [[ "$docker_kill_grace" =~ ^[1-9][0-9]*$ ]] || (( docker_kill_grace > 10 )); then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ 'REAL_PG_DOCKER_KILL_GRACE_SECONDS must be an integer from 1 to 10'
+ return 1
+ fi
+ if [[ ! -f "$REAL_PG_DOCKER_HELPER" ]]; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ 'the repository Docker isolation/deadline helper is missing'
+ return 1
+ fi
+ if ! MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS="$docker_timeout" \
+ MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS="$docker_timeout" \
+ MINIKUBE_DOCKER_KILL_GRACE_SECONDS="$docker_kill_grace" \
+ bash "$REAL_PG_DOCKER_HELPER" --check-info >/dev/null 2>&1; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ "Docker is unavailable or did not answer within ${docker_timeout}s"
+ return 1
+ fi
+ elif [[ "$require_docker" != false ]]; then
+ real_pg_preflight_fail LOCAL_DEPENDENCY_MISSING \
+ 'Real PostgreSQL preflight require_docker must be true or false'
+ return 1
+ fi
+
+ REAL_PG_PREFLIGHT_DURATION_SECONDS=$(( ${SECONDS:-0} - started_seconds ))
+ printf '[real-pg-preflight] PASS node=%s workers=1 packages=2 docker=%s duration=%ss\n' \
+ "$node_major" "$require_docker" "$REAL_PG_PREFLIGHT_DURATION_SECONDS"
+}
+
+if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
+ if ! real_pg_local_preflight "${1:-$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)}" true; then
+ printf '%s: %s\n' "$REAL_PG_PREFLIGHT_ERROR_CODE" "$REAL_PG_PREFLIGHT_ERROR_MESSAGE" >&2
+ exit 1
+ fi
+fi
diff --git a/scripts/minikube/build-images.sh b/scripts/minikube/build-images.sh
index 21c191af1..02d236522 100755
--- a/scripts/minikube/build-images.sh
+++ b/scripts/minikube/build-images.sh
@@ -31,7 +31,8 @@
# needs them: no clerum build runs there to pull them in.
# --only=<svc> Build only the image(s) whose tag matches the substring <svc>
# (e.g. --only=control-api, --only=workflow-recipes). Skips
-# public-image pulls and manifest regeneration.
+# public-image pulls and preserves the recorded manifest
+# acquisition mode/tag while refreshing image IDs.
# --include-e2e-fixtures
# --verify-only: also demand the two E2E-only fixtures
# (workflow-custom-sdk-e2e, workflow-plugin-sdk-e2e) in ghcr
@@ -51,9 +52,18 @@
# MINIKUBE_PRELOAD_BASE_IMAGES=false Skip preloading Dockerfile base images
# from the host Docker cache into minikube.
# MINIKUBE_BASE_IMAGE_PULL_RETRIES Pull attempts for each missing base
-# image before failing (default: 3).
+# image before failing (default: 3,
+# maximum: 10).
# MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS Delay between failed pull attempts
-# (default: 5).
+# (default: 5, range: 0-300).
+# MINIKUBE_{STATUS,DOCKER_ENV,KUBECTL}_TIMEOUT_SECONDS
+# Finite deadlines for Minikube status,
+# docker-env, and node inventory probes
+# (default: 30, maximum: 300).
+# MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS
+# kubectl request timeout nested inside
+# its process deadline (default: 20,
+# maximum: 300).
# MINIKUBE_BUILD_DESKTOP_IMAGE=true Include the heavy mcp-host-desktop
# image. It is excluded from normal
# runtime gates unless explicitly needed.
@@ -70,6 +80,22 @@
# MINIKUBE_PLAYWRIGHT_MCP_SOURCE_IMAGE
# Published image to retag locally when
# Playwright MCP is explicitly enabled.
+# MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY
+# private (default) requires the explicit
+# MINIKUBE_DOCKER_AUTH_CONFIG for that
+# source pull; public uses the isolated
+# unauthenticated Docker config.
+# MINIKUBE_DOCKER_AUTH_CONFIG Explicit Docker config directory for a
+# private-registry pull. Ambient Docker
+# auth is never inherited or copied.
+# MINIKUBE_DOCKER_{INFO,PULL,BUILD}_TIMEOUT_SECONDS
+# Finite operation deadlines (defaults:
+# 30, 600, and 1800 seconds).
+# MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS
+# Deadline for each read-only Minikube
+# image inventory (default: 30 seconds).
+# MINIKUBE_DOCKER_BUILDX_PLUGIN Explicit path to an already-installed
+# buildx plugin. No plugin is downloaded.
# MINIKUBE_BUILD_AIRTABLE_MCP_IMAGE=true
# Include the optional local Airtable MCP
# image. MCP servers are distributed via
@@ -92,22 +118,38 @@ export DOCKER_BUILDKIT="${DOCKER_BUILDKIT:-1}"
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
-PROFILE="${MINIKUBE_PROFILE:-clerum-test}"
+PROFILE="${MINIKUBE_PROFILE:-${T2_PROFILE:-clerum-test}}"
+CONTEXT="${T2_CONTEXT:-${CONTROL_API_REAL_PG_CONTEXT:-${PROFILE}}}"
SKIP_PUBLIC=false
VERIFY_ONLY=false
PUBLIC_ONLY=false
ONLY_SVC=""
+ONLY_REQUESTED=false
FAILED_IMAGES=()
MINIKUBE_PRELOAD_BASE_IMAGES="${MINIKUBE_PRELOAD_BASE_IMAGES:-true}"
MINIKUBE_BASE_IMAGE_PULL_RETRIES="${MINIKUBE_BASE_IMAGE_PULL_RETRIES:-3}"
MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS="${MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS:-5}"
+MINIKUBE_STATUS_TIMEOUT_SECONDS="${MINIKUBE_STATUS_TIMEOUT_SECONDS:-30}"
+MINIKUBE_DOCKER_ENV_TIMEOUT_SECONDS="${MINIKUBE_DOCKER_ENV_TIMEOUT_SECONDS:-30}"
+MINIKUBE_KUBECTL_TIMEOUT_SECONDS="${MINIKUBE_KUBECTL_TIMEOUT_SECONDS:-30}"
+MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS="${MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS:-20}"
MINIKUBE_BUILD_DESKTOP_IMAGE="${MINIKUBE_BUILD_DESKTOP_IMAGE:-false}"
MINIKUBE_BUILD_PLAYWRIGHT_MCP_IMAGE="${MINIKUBE_BUILD_PLAYWRIGHT_MCP_IMAGE:-false}"
MINIKUBE_REUSE_PLAYWRIGHT_MCP_IMAGE="${MINIKUBE_REUSE_PLAYWRIGHT_MCP_IMAGE:-true}"
MINIKUBE_PLAYWRIGHT_MCP_SOURCE_IMAGE="${MINIKUBE_PLAYWRIGHT_MCP_SOURCE_IMAGE:-us-central1-docker.pkg.dev/your-gcp-project/clerum/playwright-server:latest}"
+MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY="${MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY:-private}"
MINIKUBE_BUILD_AIRTABLE_MCP_IMAGE="${MINIKUBE_BUILD_AIRTABLE_MCP_IMAGE:-false}"
+MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS="${MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS:-30}"
SKIP_UIS="${MINIKUBE_SKIP_UIS:-false}"
+case "$MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY" in
+ public|private) ;;
+ *)
+ echo "Unknown MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY: '${MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY}' (expected: public | private)" >&2
+ exit 1
+ ;;
+esac
+
case "${SKIP_UIS}" in
true|1|yes) SKIP_UIS=true ;;
*) SKIP_UIS=false ;;
@@ -147,6 +189,21 @@ MANIFEST_FILE="${PROJECT_DIR}/deploy/minikube/.image-manifest.json"
# exact failure this key exists to prevent.
# shellcheck source=scripts/minikube/image-mode.sh
source "${SCRIPT_DIR}/image-mode.sh"
+# shellcheck source=scripts/minikube/docker-cli-env.sh
+source "${SCRIPT_DIR}/docker-cli-env.sh"
+
+cleanup_docker_cli_env() {
+ local status=$? cleanup_status=0
+ trap - EXIT INT TERM
+ docker_cli_env_cleanup || cleanup_status=$?
+ if [[ "$status" -eq 0 && "$cleanup_status" -ne 0 ]]; then
+ status="$cleanup_status"
+ fi
+ exit "$status"
+}
+trap cleanup_docker_cli_env EXIT
+trap 'exit 130' INT
+trap 'exit 143' TERM
recorded_image_source() {
image_mode_recorded_source "$PROJECT_DIR"
@@ -187,16 +244,32 @@ for arg in "$@"; do
--skip-uis) SKIP_UIS=true ;;
--verify-only) VERIFY_ONLY=true ;;
--public-only) PUBLIC_ONLY=true ;;
- --only=*) ONLY_SVC="${arg#--only=}" ;;
+ --only=*)
+ if [[ "$ONLY_REQUESTED" == true ]]; then
+ printf 'Duplicate --only selector: %s\n' "$arg" >&2
+ exit 2
+ fi
+ ONLY_REQUESTED=true
+ ONLY_SVC="${arg#--only=}"
+ ;;
--include-e2e-fixtures) INCLUDE_E2E_FIXTURES=true ;;
--include-desktop-image) MINIKUBE_BUILD_DESKTOP_IMAGE=true ;;
--include-playwright-mcp-image) MINIKUBE_BUILD_PLAYWRIGHT_MCP_IMAGE=true ;;
--include-airtable-mcp-image) MINIKUBE_BUILD_AIRTABLE_MCP_IMAGE=true ;;
+ *)
+ printf 'Unknown build-images argument: %s\n' "$arg" >&2
+ exit 2
+ ;;
esac
done
-# When --only is set we skip public-image pulls and manifest regen.
-if [ -n "$ONLY_SVC" ]; then
+if [[ "$ONLY_REQUESTED" == true && -z "$ONLY_SVC" ]]; then
+ printf '%s\n' 'Invalid --only selector: the selector must not be empty' >&2
+ exit 2
+fi
+
+# When --only is set we skip public-image pulls and preserve manifest mode/tag.
+if [[ "$ONLY_REQUESTED" == true ]]; then
SKIP_PUBLIC=true
if [[ "$ONLY_SVC" == *"mcp-host-desktop"* ]]; then
MINIKUBE_BUILD_DESKTOP_IMAGE=true
@@ -217,6 +290,107 @@ if [ "$SKIP_UIS" = true ]; then
MINIKUBE_BUILD_DESKTOP_IMAGE=false
fi
+# All images built in this session. Every entry here must have a matching
+# row in deploy/images.json (via localRef()) -- scripts/tests/test-images-
+# manifest.sh enforces it. This declaration intentionally precedes every
+# runtime call so an invalid --only selector cannot acquire a lease or touch a
+# daemon before it is rejected.
+ALL_IMAGES=(
+ "clerum/host-context-controller:test"
+ "clerum/workflow-recipes:test"
+ "clerum/workflow-coordinator:test"
+ "clerum/workflow-snippet-runner:test"
+ "clerum/workflow-custom-sdk-e2e:test"
+ "clerum/workflow-plugin-sdk-e2e:test"
+ "clerum/mcp-host:test"
+ "clerum/mcp-host-slim:test"
+ "clerum/mcp-host-full:test"
+ "clerum/mcp-proxy:test"
+ "clerum/nginx-egress-proxy:test"
+ "clerum/gfs-controller:test"
+ "clerum/control-api:test"
+ "clerum/external-rest-api:test"
+ "clerum/rpc-proxy:test"
+ "clerum/webhook-proxy:test"
+ "clerum/webhook-gateway:test"
+ "clerum/channel-reader:test"
+ "clerum/workflow-approval-request-reader:test"
+ "clerum/stdio-bridge:test"
+ "clerum/control-ui:test"
+ "clerum/profile-ui:test"
+ "clerum/mock-mcp-server:test"
+ "clerum/mock-stdio-mcp-server:test"
+ "clerum/workspace-files-controller:test"
+)
+
+KNOWN_BUILD_NAMES=(
+ hcc
+ wrc
+ workflow-coordinator
+ workflow-snippet-runner
+ workflow-custom-sdk-e2e
+ workflow-plugin-sdk-e2e
+ mcp-host
+ mcp-host-slim
+ mcp-host-full
+ mcp-proxy
+ nginx-egress-proxy
+ gfs-controller
+ control-api
+ external-rest-api
+ rpc-proxy
+ webhook-proxy
+ webhook-gateway
+ channel-reader
+ workflow-approval-request-reader
+ stdio-bridge
+ mock-mcp
+ mock-stdio-mcp
+ workspace-files-controller
+)
+
+if [ "$MINIKUBE_BUILD_DESKTOP_IMAGE" = "true" ]; then
+ ALL_IMAGES+=("clerum/mcp-host-desktop:test")
+ KNOWN_BUILD_NAMES+=(mcp-host-desktop)
+fi
+
+if [ "$MINIKUBE_BUILD_PLAYWRIGHT_MCP_IMAGE" = "true" ]; then
+ ALL_IMAGES+=("clerum/playwright-mcp-server:test")
+ KNOWN_BUILD_NAMES+=(playwright-mcp)
+fi
+
+if [ "$MINIKUBE_BUILD_AIRTABLE_MCP_IMAGE" = "true" ]; then
+ ALL_IMAGES+=("clerum/airtable-mcp-server:test")
+ KNOWN_BUILD_NAMES+=(airtable-mcp)
+fi
+
+if [ "$SKIP_UIS" = true ]; then
+ FILTERED_IMAGES=()
+ for image in "${ALL_IMAGES[@]}"; do
+ case "$image" in
+ clerum/control-ui:*|clerum/profile-ui:*) continue ;;
+ *) FILTERED_IMAGES+=("$image") ;;
+ esac
+ done
+ ALL_IMAGES=("${FILTERED_IMAGES[@]}")
+else
+ KNOWN_BUILD_NAMES+=(control-ui profile-ui)
+fi
+
+if [[ "$ONLY_REQUESTED" == true ]]; then
+ SELECTOR_MATCHED=false
+ for candidate in "${ALL_IMAGES[@]}" "${KNOWN_BUILD_NAMES[@]}"; do
+ if [[ "$candidate" == *"$ONLY_SVC"* ]]; then
+ SELECTOR_MATCHED=true
+ break
+ fi
+ done
+ if [[ "$SELECTOR_MATCHED" != true ]]; then
+ printf 'Unknown --only selector: %s\n' "$ONLY_SVC" >&2
+ exit 2
+ fi
+fi
+
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
@@ -229,17 +403,111 @@ ok() { echo -e "${GREEN} OK${NC} -- $*"; }
warn() { echo -e "${YELLOW} WARN${NC} -- $*"; }
err() { echo -e "${RED} ERROR${NC} -- $*"; }
-# Verify minikube is running
-if ! minikube -p "$PROFILE" status &>/dev/null; then
- err "Minikube profile '${PROFILE}' is not running. Start with: minikube start -p ${PROFILE}"
+validate_nonnegative_integer() {
+ local name="$1" value="$2" maximum="$3"
+ if ! [[ "$value" =~ ^[0-9]+$ ]] \
+ || [[ "${#value}" -gt "${#maximum}" ]] \
+ || (( 10#$value > maximum )); then
+ printf 'DOCKER_DEADLINE_INVALID: %s must be an integer from 0 to %s\n' \
+ "$name" "$maximum" >&2
+ return 2
+ fi
+}
+
+validate_build_configuration() {
+ if [[ ! "$PROFILE" =~ ^[A-Za-z0-9][A-Za-z0-9._-]*$ ]]; then
+ printf 'PROFILE_LOCK_REQUIRED: invalid Minikube profile: %s\n' "$PROFILE" >&2
+ return 2
+ fi
+ if [[ ! "$CONTEXT" =~ ^[A-Za-z0-9][A-Za-z0-9._-]*$ ]]; then
+ printf 'PROFILE_LOCK_REQUIRED: invalid Kubernetes context: %s\n' "$CONTEXT" >&2
+ return 2
+ fi
+ if [[ "$PROFILE" != "$CONTEXT" ]]; then
+ printf '%s\n' 'PROFILE_LOCK_REQUIRED: Minikube profile and Kubernetes context must match' >&2
+ return 1
+ fi
+
+ docker_cli_env_validate_deadlines || return $?
+ docker_cli_env_validate_seconds MINIKUBE_STATUS_TIMEOUT_SECONDS \
+ "$MINIKUBE_STATUS_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_ENV_TIMEOUT_SECONDS \
+ "$MINIKUBE_DOCKER_ENV_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_KUBECTL_TIMEOUT_SECONDS \
+ "$MINIKUBE_KUBECTL_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS \
+ "$MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS \
+ "$MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_BASE_IMAGE_PULL_RETRIES \
+ "$MINIKUBE_BASE_IMAGE_PULL_RETRIES" 10 || return $?
+ validate_nonnegative_integer MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS \
+ "$MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS" 300 || return $?
+}
+
+require_inherited_mutation_lease() {
+ if [[ "$VERIFY_ONLY" == true ]]; then
+ return 0
+ fi
+ if [[ -z "${T2_PROJECT_DIR:-}" || -z "${T2_PROFILE:-}" || -z "${T2_CONTEXT:-}" \
+ || "$T2_PROJECT_DIR" != "$PROJECT_DIR" || "$T2_PROFILE" != "$PROFILE" \
+ || "$T2_CONTEXT" != "$CONTEXT" || "$PROFILE" != "$CONTEXT" ]]; then
+ printf '%s\n' 'PROFILE_LOCK_REQUIRED: build-images requires the inherited mutation lease for its exact worktree, profile, and context' >&2
+ return 1
+ fi
+
+ T2_GATE_ID=build-images \
+ MINIKUBE_PROFILE="$PROFILE" CONTROL_API_REAL_PG_CONTEXT="$CONTEXT" \
+ bash "${SCRIPT_DIR}/require-t2-mutation-lock.sh"
+}
+
+run_with_deadline() {
+ local label="$1" timeout_seconds="$2"
+ shift 2
+ node "$DOCKER_CLI_DEADLINE_RUNNER" \
+ --timeout-seconds "$timeout_seconds" \
+ --heartbeat-seconds "$MINIKUBE_DOCKER_HEARTBEAT_SECONDS" \
+ --kill-grace-seconds "$MINIKUBE_DOCKER_KILL_GRACE_SECONDS" \
+ --label "$label" -- "$@"
+}
+
+validate_build_configuration
+require_inherited_mutation_lease
+
+if ! command -v node >/dev/null 2>&1 || [[ ! -f "$DOCKER_CLI_DEADLINE_RUNNER" ]]; then
+ err "The existing deadline runner and Node.js are required"
exit 1
fi
+# Verify minikube is running. Preserve timeout (124), signal, and ordinary
+# child exit statuses so callers can distinguish a dead process from a stopped
+# profile.
+minikube_status=0
+run_with_deadline minikube-status "$MINIKUBE_STATUS_TIMEOUT_SECONDS" \
+ minikube -p "$PROFILE" status >/dev/null || minikube_status=$?
+if [[ "$minikube_status" -ne 0 ]]; then
+ err "Minikube profile '${PROFILE}' is not running. Start with: minikube start -p ${PROFILE}"
+ exit "$minikube_status"
+fi
+
minikube_node_count() {
- kubectl --context "$PROFILE" get nodes --no-headers 2>/dev/null | wc -l | tr -d '[:space:]'
+ local nodes status=0
+ nodes="$(run_with_deadline minikube-get-nodes "$MINIKUBE_KUBECTL_TIMEOUT_SECONDS" \
+ kubectl --context="$CONTEXT" get nodes \
+ --request-timeout="${MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS}s" \
+ --no-headers)" || status=$?
+ if [[ "$status" -ne 0 ]]; then
+ return "$status"
+ fi
+ awk 'NF { count += 1 } END { print count + 0 }' <<<"$nodes"
}
-MINIKUBE_NODE_COUNT="$(minikube_node_count)"
+node_count_status=0
+MINIKUBE_NODE_COUNT="$(minikube_node_count)" || node_count_status=$?
+if [[ "$node_count_status" -ne 0 ]]; then
+ err "Unable to list nodes for minikube profile '${PROFILE}'"
+ exit "$node_count_status"
+fi
if [[ -z "$MINIKUBE_NODE_COUNT" || "$MINIKUBE_NODE_COUNT" == "0" ]]; then
err "Unable to list nodes for minikube profile '${PROFILE}'"
exit 1
@@ -280,33 +548,81 @@ normalize_minikube_image_tag() {
printf '%s' "$image"
}
-minikube_image_present() {
- local image=$1
- local normalized
- normalized="$(normalize_minikube_image_tag "$image")"
- minikube -p "$PROFILE" image ls 2>/dev/null | grep -Fxq "$normalized"
+MINIKUBE_IMAGE_INVENTORY_JSON=""
+MINIKUBE_IMAGE_INVENTORY_CACHED=false
+
+minikube_image_inventory() {
+ local inventory_status=0
+ run_with_deadline minikube-image-inventory \
+ "$MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS" \
+ minikube -p "$PROFILE" image ls --format=json || inventory_status=$?
+ if [[ "$inventory_status" -ne 0 ]]; then
+ err "Minikube image inventory failed or exceeded its deadline" >&2
+ return "$inventory_status"
+ fi
+}
+
+cache_minikube_image_inventory() {
+ local inventory_status=0
+ MINIKUBE_IMAGE_INVENTORY_JSON="$(minikube_image_inventory)" || inventory_status=$?
+ if [[ "$inventory_status" -ne 0 ]]; then
+ return "$inventory_status"
+ fi
+ MINIKUBE_IMAGE_INVENTORY_CACHED=true
}
minikube_image_id() {
local image=$1
- local normalized
+ local normalized inventory inventory_status=0
normalized="$(normalize_minikube_image_tag "$image")"
+ if [[ "$MINIKUBE_IMAGE_INVENTORY_CACHED" == true ]]; then
+ inventory="$MINIKUBE_IMAGE_INVENTORY_JSON"
+ else
+ inventory="$(minikube_image_inventory)" || inventory_status=$?
+ if [[ "$inventory_status" -ne 0 ]]; then
+ return "$inventory_status"
+ fi
+ fi
# shellcheck disable=SC2016
- minikube -p "$PROFILE" image ls --format=json 2>/dev/null \
- | node -e '
+ printf '%s' "$inventory" | node -e '
const tag = process.argv[1]
+const raw = require("fs").readFileSync(0, "utf8").trim()
let items = []
-try { items = JSON.parse(require("fs").readFileSync(0, "utf8")) } catch {}
+if (raw) {
+ try {
+ items = JSON.parse(raw)
+ } catch {
+ process.stderr.write("MINIKUBE_IMAGE_INVENTORY_INVALID: inventory is not valid JSON\n")
+ process.exit(2)
+ }
+ if (!Array.isArray(items)) {
+ process.stderr.write("MINIKUBE_IMAGE_INVENTORY_INVALID: inventory is not an array\n")
+ process.exit(2)
+ }
+}
const hit = items.find((item) => Array.isArray(item.repoTags) && item.repoTags.includes(tag))
process.stdout.write(hit?.id ? `sha256:${hit.id}` : "NOT_FOUND")
' "$normalized"
}
+minikube_image_present() {
+ local image=$1 image_id inventory_status=0
+ image_id="$(minikube_image_id "$image")" || inventory_status=$?
+ if [[ "$inventory_status" -ne 0 ]]; then
+ return "$inventory_status"
+ fi
+ [[ "$image_id" != "NOT_FOUND" ]]
+}
+
pull_host_image_with_retry() {
local image=$1
local attempt=1
+ local pull_status=1
while [ "$attempt" -le "$MINIKUBE_BASE_IMAGE_PULL_RETRIES" ]; do
- if docker pull "$image"; then
+ pull_status=0
+ docker_cli_run_public pull-base-image "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" \
+ docker pull "$image" || pull_status=$?
+ if [ "$pull_status" -eq 0 ]; then
return 0
fi
if [ "$attempt" -lt "$MINIKUBE_BASE_IMAGE_PULL_RETRIES" ]; then
@@ -315,10 +631,57 @@ pull_host_image_with_retry() {
fi
attempt=$((attempt + 1))
done
- return 1
+ return "$pull_status"
+}
+
+require_docker_info() {
+ local label="$1" status=0
+ docker_cli_run_public "$label" "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ docker info >/dev/null || status=$?
+ if [ "$status" -ne 0 ]; then
+ err "Docker daemon check failed (${label})"
+ return "$status"
+ fi
+}
+
+docker_local_image_query() {
+ local image="$1"
+ docker_cli_run_public docker-images-query "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ docker images -q "$image"
+}
+
+docker_local_image_id() {
+ local image="$1"
+ docker_cli_run_public docker-image-id "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ docker inspect --format='{{.Id}}' "$image"
+}
+
+docker_local_image_tag() {
+ local source="$1" tag="$2"
+ docker_cli_run_public docker-image-tag "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ docker tag "$source" "$tag"
+}
+
+minikube_load_local_image() {
+ local image="$1"
+ # A local image transfer can be as large as the preceding build, so it uses
+ # the build budget rather than the short metadata budget.
+ docker_cli_run_public minikube-image-load "$MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS" \
+ minikube -p "$PROFILE" image load "$image" >/dev/null
+}
+
+load_and_resolve_image_id() {
+ local image="$1"
+ if [[ "$MINIKUBE_MULTI_NODE" == true ]]; then
+ minikube_load_local_image "$image" || return $?
+ minikube_image_id "$image"
+ else
+ docker_local_image_id "$image"
+ fi
}
ensure_base_images_in_minikube() {
+ local presence_status query_status image_ids
if [ "$MINIKUBE_PRELOAD_BASE_IMAGES" = "false" ]; then
warn "Skipping base image preload (MINIKUBE_PRELOAD_BASE_IMAGES=false)"
return 0
@@ -326,90 +689,63 @@ ensure_base_images_in_minikube() {
echo -e "\n${BOLD}=== Ensuring Build Base Images ===${NC}"
for image in "${BUILD_BASE_IMAGES[@]}"; do
- if minikube_image_present "$image"; then
+ presence_status=0
+ minikube_image_present "$image" || presence_status=$?
+ if [[ "$presence_status" -eq 0 ]]; then
ok "${image} already present in minikube"
continue
fi
+ if [[ "$presence_status" -ne 1 ]]; then
+ err "Could not inspect '${image}' in minikube"
+ return "$presence_status"
+ fi
- if ! docker image inspect "$image" >/dev/null 2>&1; then
+ query_status=0
+ image_ids="$(docker_local_image_query "$image")" || query_status=$?
+ if [[ "$query_status" -ne 0 ]]; then
+ err "Could not inventory '${image}' in the host Docker cache"
+ return "$query_status"
+ fi
+ if [[ -z "$image_ids" ]]; then
log "Pulling base image '${image}' into host Docker cache..."
- pull_host_image_with_retry "$image"
+ pull_host_image_with_retry "$image" || return $?
fi
log "Loading base image '${image}' into minikube '${PROFILE}'..."
- minikube -p "$PROFILE" image load "$image" >/dev/null
+ minikube_load_local_image "$image" || return $?
ok "${image} loaded into minikube"
done
}
if [ "$VERIFY_ONLY" = false ]; then
+ docker_cli_env_prepare
+ require_docker_info docker-info-host
ensure_base_images_in_minikube
-fi
-
-# ---- Point Docker CLI at minikube's Docker daemon when safe ----
-if [ "$MINIKUBE_MULTI_NODE" = false ]; then
- # This makes `docker build` execute INSIDE minikube — no image transfer needed.
- log "Configuring Docker CLI to use minikube's Docker daemon..."
- eval "$(minikube -p "$PROFILE" docker-env)"
- # Let Docker negotiate API version automatically (minikube v1.38+ requires >=1.44)
- unset DOCKER_API_VERSION 2>/dev/null || true
- ok "Docker CLI now targets minikube '${PROFILE}'"
-else
- log "Skipping docker-env; multi-node profiles use host builds plus 'minikube image load'"
-fi
-
-# All images built in this session. Every entry here must have a matching
-# row in deploy/images.json (via localRef()) -- scripts/tests/test-images-
-# manifest.sh enforces it.
-ALL_IMAGES=(
- "clerum/host-context-controller:test"
- "clerum/workflow-recipes:test"
- "clerum/workflow-coordinator:test"
- "clerum/workflow-snippet-runner:test"
- "clerum/workflow-custom-sdk-e2e:test"
- "clerum/workflow-plugin-sdk-e2e:test"
- "clerum/mcp-host:test"
- "clerum/mcp-host-slim:test"
- "clerum/mcp-host-full:test"
- "clerum/mcp-proxy:test"
- "clerum/nginx-egress-proxy:test"
- "clerum/gfs-controller:test"
- "clerum/control-api:test"
- "clerum/external-rest-api:test"
- "clerum/rpc-proxy:test"
- "clerum/webhook-proxy:test"
- "clerum/webhook-gateway:test"
- "clerum/channel-reader:test"
- "clerum/workflow-approval-request-reader:test"
- "clerum/stdio-bridge:test"
- "clerum/control-ui:test"
- "clerum/profile-ui:test"
- "clerum/mock-mcp-server:test"
- "clerum/mock-stdio-mcp-server:test"
- "clerum/workspace-files-controller:test"
-)
-
-if [ "$MINIKUBE_BUILD_DESKTOP_IMAGE" = "true" ]; then
- ALL_IMAGES+=("clerum/mcp-host-desktop:test")
-fi
-
-if [ "$MINIKUBE_BUILD_PLAYWRIGHT_MCP_IMAGE" = "true" ]; then
- ALL_IMAGES+=("clerum/playwright-mcp-server:test")
-fi
-if [ "$MINIKUBE_BUILD_AIRTABLE_MCP_IMAGE" = "true" ]; then
- ALL_IMAGES+=("clerum/airtable-mcp-server:test")
-fi
-
-if [ "$SKIP_UIS" = true ]; then
- FILTERED_IMAGES=()
- for image in "${ALL_IMAGES[@]}"; do
- case "$image" in
- clerum/control-ui:*|clerum/profile-ui:*) continue ;;
- *) FILTERED_IMAGES+=("$image") ;;
- esac
- done
- ALL_IMAGES=("${FILTERED_IMAGES[@]}")
+ # ---- Point Docker CLI at minikube's Docker daemon when safe ----
+ if [ "$MINIKUBE_MULTI_NODE" = false ]; then
+ # This makes `docker build` execute INSIDE minikube — no image transfer needed.
+ log "Configuring Docker CLI to use minikube's Docker daemon..."
+ docker_env_status=0
+ docker_env_output="$(run_with_deadline minikube-docker-env \
+ "$MINIKUBE_DOCKER_ENV_TIMEOUT_SECONDS" \
+ minikube -p "$PROFILE" docker-env --shell bash)" || docker_env_status=$?
+ if [[ "$docker_env_status" -ne 0 ]]; then
+ err "Could not resolve Docker environment for minikube '${PROFILE}'"
+ exit "$docker_env_status"
+ fi
+ if [[ -z "$docker_env_output" ]]; then
+ err "Minikube '${PROFILE}' returned an empty Docker environment"
+ exit 1
+ fi
+ eval "$docker_env_output"
+ # Let Docker negotiate API version automatically (minikube v1.38+ requires >=1.44)
+ unset DOCKER_API_VERSION 2>/dev/null || true
+ require_docker_info docker-info-minikube
+ ok "Docker CLI now targets minikube '${PROFILE}'"
+ else
+ log "Skipping docker-env; multi-node profiles use host builds plus 'minikube image load'"
+ fi
fi
# ---- Verify-only mode ----
@@ -513,6 +849,11 @@ if [ "$VERIFY_ONLY" = true ]; then
VERIFY_IMAGES=("${FILTERED_VERIFY[@]}")
fi
+ # Verification is read-only and deliberately does not initialise Docker or
+ # buildx. One bounded Minikube inventory is enough for every tag and avoids
+ # multiplying a daemon/runtime hang by the size of VERIFY_IMAGES.
+ cache_minikube_image_inventory
+
fail_count=0
for img in "${VERIFY_IMAGES[@]}"; do
sha=$(minikube_image_id "$img")
@@ -589,18 +930,23 @@ build_image() {
docker_args+=(-f "$dockerfile")
fi
local build_cmd=(docker build "${docker_args[@]}" "$dir")
- if ! "${build_cmd[@]}" 2>&1 | tail -3; then
+ local build_status=0
+ docker_cli_run_public "build-${name}" "$MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS" \
+ "${build_cmd[@]}" || build_status=$?
+ if [ "$build_status" -ne 0 ]; then
err "Failed to build ${tag}"
FAILED_IMAGES+=("$tag")
- return 1
+ return "$build_status"
fi
- local sha
+ local sha sha_status=0
if [ "$MINIKUBE_MULTI_NODE" = true ]; then
log "Loading ${tag} into all minikube nodes..."
- minikube -p "$PROFILE" image load "$tag" >/dev/null
- sha=$(minikube_image_id "$tag")
- else
- sha=$(docker inspect --format='{{.Id}}' "$tag" 2>/dev/null || echo "NOT_FOUND")
+ fi
+ sha="$(load_and_resolve_image_id "$tag")" || sha_status=$?
+ if [[ "$sha_status" -ne 0 ]]; then
+ err "Could not verify ${tag} after build"
+ FAILED_IMAGES+=("$tag")
+ return "$sha_status"
fi
if [ "$sha" = "NOT_FOUND" ]; then
err "Image ${tag} not found after build!"
@@ -622,18 +968,23 @@ build_control_ui_image() {
)
docker_args+=(-f "${PROJECT_DIR}/control-ui/Dockerfile")
local build_cmd=(docker build "${docker_args[@]}" "${PROJECT_DIR}")
- if ! "${build_cmd[@]}" 2>&1 | tail -3; then
+ local build_status=0
+ docker_cli_run_public build-control-ui "$MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS" \
+ "${build_cmd[@]}" || build_status=$?
+ if [ "$build_status" -ne 0 ]; then
err "Failed to build ${tag}"
FAILED_IMAGES+=("$tag")
- return 1
+ return "$build_status"
fi
- local sha
+ local sha sha_status=0
if [ "$MINIKUBE_MULTI_NODE" = true ]; then
log "Loading ${tag} into all minikube nodes..."
- minikube -p "$PROFILE" image load "$tag" >/dev/null
- sha=$(minikube_image_id "$tag")
- else
- sha=$(docker inspect --format='{{.Id}}' "$tag" 2>/dev/null || echo "NOT_FOUND")
+ fi
+ sha="$(load_and_resolve_image_id "$tag")" || sha_status=$?
+ if [[ "$sha_status" -ne 0 ]]; then
+ err "Could not verify ${tag} after build"
+ FAILED_IMAGES+=("$tag")
+ return "$sha_status"
fi
if [ "$sha" = "NOT_FOUND" ]; then
err "Image ${tag} not found after build!"
@@ -644,23 +995,36 @@ build_control_ui_image() {
}
reuse_image_as_tag() {
- local name=$1 source=$2 tag=$3
+ local name=$1 source=$2 tag=$3 visibility=${4:-private}
if [ -n "$ONLY_SVC" ] && [[ "$tag" != *"$ONLY_SVC"* ]] && [[ "$name" != *"$ONLY_SVC"* ]]; then
return 0
fi
log "Reusing ${source} as ${tag}..."
- if ! docker pull "$source" 2>&1 | tail -3; then
+ local pull_status=0
+ if [ "$visibility" = private ]; then
+ docker_cli_run_private pull-reusable-image "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" \
+ docker pull "$source" || pull_status=$?
+ else
+ docker_cli_run_public pull-reusable-image "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" \
+ docker pull "$source" || pull_status=$?
+ fi
+ if [ "$pull_status" -ne 0 ]; then
err "Failed to pull ${source}"
FAILED_IMAGES+=("$tag")
- return 1
+ return "$pull_status"
fi
- docker tag "$source" "$tag"
- local sha
- if [ "$MINIKUBE_MULTI_NODE" = true ]; then
- minikube -p "$PROFILE" image load "$tag" >/dev/null
- sha=$(minikube_image_id "$tag")
- else
- sha=$(docker inspect --format='{{.Id}}' "$tag" 2>/dev/null || echo "NOT_FOUND")
+ local tag_status=0 sha sha_status=0
+ docker_local_image_tag "$source" "$tag" || tag_status=$?
+ if [[ "$tag_status" -ne 0 ]]; then
+ err "Failed to tag ${source} as ${tag}"
+ FAILED_IMAGES+=("$tag")
+ return "$tag_status"
+ fi
+ sha="$(load_and_resolve_image_id "$tag")" || sha_status=$?
+ if [[ "$sha_status" -ne 0 ]]; then
+ err "Could not verify ${tag} after retag"
+ FAILED_IMAGES+=("$tag")
+ return "$sha_status"
fi
if [ "$sha" = "NOT_FOUND" ]; then
err "Image ${tag} not found after retag!"
@@ -829,7 +1193,8 @@ if [ "$MINIKUBE_BUILD_PLAYWRIGHT_MCP_IMAGE" = "true" ]; then
if [ "$MINIKUBE_REUSE_PLAYWRIGHT_MCP_IMAGE" = "true" ]; then
reuse_image_as_tag "playwright-mcp" \
"$MINIKUBE_PLAYWRIGHT_MCP_SOURCE_IMAGE" \
- "clerum/playwright-mcp-server:test"
+ "clerum/playwright-mcp-server:test" \
+ "$MINIKUBE_PLAYWRIGHT_MCP_SOURCE_VISIBILITY"
else
build_image "playwright-mcp" \
"${PROJECT_DIR}/mcp-servers/playwright" \
@@ -868,16 +1233,33 @@ if [ "$SKIP_PUBLIC" = false ]; then
"ghcr.io/aas-ee/open-web-search:latest"
)
for img in "${PUBLIC_IMAGES[@]}"; do
- if [ "$MINIKUBE_MULTI_NODE" = true ] && minikube_image_present "$img"; then
+ public_presence_status=1
+ public_image_ids=""
+ if [[ "$MINIKUBE_MULTI_NODE" == true ]]; then
+ public_presence_status=0
+ minikube_image_present "$img" || public_presence_status=$?
+ else
+ public_presence_status=0
+ public_image_ids="$(docker_local_image_query "$img")" || public_presence_status=$?
+ fi
+ if { [[ "$MINIKUBE_MULTI_NODE" == true ]] && [[ "$public_presence_status" -gt 1 ]]; } \
+ || { [[ "$MINIKUBE_MULTI_NODE" == false ]] && [[ "$public_presence_status" -ne 0 ]]; }; then
+ err "Could not inventory public image '${img}'"
+ exit "$public_presence_status"
+ fi
+
+ if [ "$MINIKUBE_MULTI_NODE" = true ] && [[ "$public_presence_status" -eq 0 ]]; then
log "Image '$img' already present -- skipping"
- elif [ "$MINIKUBE_MULTI_NODE" = false ] && docker images -q "$img" 2>/dev/null | grep -q .; then
+ elif [ "$MINIKUBE_MULTI_NODE" = false ] && [[ -n "$public_image_ids" ]]; then
log "Image '$img' already present -- skipping"
else
log "Pulling '$img' into minikube..."
if [ "$MINIKUBE_MULTI_NODE" = true ]; then
- minikube -p "$PROFILE" image load --pull "$img" >/dev/null
+ docker_cli_run_public pull-public-image "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" \
+ minikube -p "$PROFILE" image load --pull "$img" >/dev/null
else
- docker pull "$img" 2>/dev/null
+ docker_cli_run_public pull-public-image "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" \
+ docker pull "$img"
fi
ok "$img"
fi
@@ -887,8 +1269,6 @@ fi
# ---- Generate build manifest JSON ----
# MANIFEST_FILE is declared at the top of the script; the verify path needs it
# too and exits long before this point.
-mkdir -p "$(dirname "$MANIFEST_FILE")"
-echo -e "\n${BOLD}=== Generating Image Manifest ===${NC}"
# How this cluster's images were acquired, read back by --verify-only.
# pull-images.sh writes "ghcr"; this script only ever BUILDS, so a full run
@@ -924,6 +1304,49 @@ if [ -n "$ONLY_SVC" ]; then
esac
fi
fi
+
+# Resolve every manifest value before opening MANIFEST_FILE. A daemon query is
+# authoritative only when it exits zero: empty output then means the image is
+# absent, while every nonzero status is an inventory failure. In particular,
+# Docker's generic exit 1 must never trigger a pull or be rewritten as
+# NOT_BUILT. Staging the values also preserves any prior good manifest when an
+# inventory or inspect command fails partway through the list.
+MANIFEST_SHAS=()
+if [[ "$MINIKUBE_MULTI_NODE" == true ]]; then
+ cache_minikube_image_inventory
+fi
+for img in "${ALL_IMAGES[@]}"; do
+ sha=""
+ if [[ "$MINIKUBE_MULTI_NODE" == true ]]; then
+ manifest_inventory_status=0
+ sha="$(minikube_image_id "$img")" || manifest_inventory_status=$?
+ if [[ "$manifest_inventory_status" -ne 0 ]]; then
+ err "Could not inventory ${img} while generating the image manifest" >&2
+ exit "$manifest_inventory_status"
+ fi
+ else
+ manifest_inventory_status=0
+ manifest_image_ids="$(docker_local_image_query "$img")" || manifest_inventory_status=$?
+ if [[ "$manifest_inventory_status" -ne 0 ]]; then
+ err "Could not inventory ${img} while generating the image manifest" >&2
+ exit "$manifest_inventory_status"
+ fi
+ if [[ -z "$manifest_image_ids" ]]; then
+ sha="NOT_BUILT"
+ else
+ manifest_inspect_status=0
+ sha="$(docker_local_image_id "$img")" || manifest_inspect_status=$?
+ if [[ "$manifest_inspect_status" -ne 0 ]]; then
+ err "Could not inspect ${img} while generating the image manifest" >&2
+ exit "$manifest_inspect_status"
+ fi
+ fi
+ fi
+ MANIFEST_SHAS+=("$sha")
+done
+
+mkdir -p "$(dirname "$MANIFEST_FILE")"
+echo -e "\n${BOLD}=== Generating Image Manifest ===${NC}"
{
echo "{"
echo " \"generated\": \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\","
@@ -933,13 +1356,10 @@ fi
echo " \"images\": {"
count=0
total=${#ALL_IMAGES[@]}
- for img in "${ALL_IMAGES[@]}"; do
+ for image_index in "${!ALL_IMAGES[@]}"; do
+ img="${ALL_IMAGES[$image_index]}"
+ sha="${MANIFEST_SHAS[$image_index]}"
count=$((count + 1))
- if [ "$MINIKUBE_MULTI_NODE" = true ]; then
- sha=$(minikube_image_id "$img")
- else
- sha=$(docker inspect --format='{{.Id}}' "$img" 2>/dev/null || echo "NOT_BUILT")
- fi
comma=","
if [ "$count" -eq "$total" ]; then comma=""; fi
echo " \"${img}\": \"${sha}\"${comma}"
diff --git a/scripts/minikube/docker-cli-env.sh b/scripts/minikube/docker-cli-env.sh
new file mode 100755
index 000000000..7365117b6
--- /dev/null
+++ b/scripts/minikube/docker-cli-env.sh
@@ -0,0 +1,480 @@
+#!/usr/bin/env bash
+# Isolate local Minikube image operations from ambient Docker credentials.
+# Source this file, call docker_cli_env_prepare, and install a caller-owned EXIT
+# trap that invokes docker_cli_env_cleanup.
+
+if [[ -n "${DOCKER_CLI_ENV_LOADED:-}" ]]; then
+ return 0
+fi
+DOCKER_CLI_ENV_LOADED=true
+
+DOCKER_CLI_ENV_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+DOCKER_CLI_DEADLINE_RUNNER="${DOCKER_CLI_ENV_DIR}/run-with-deadline.mjs"
+DOCKER_CLI_TASK_CONFIG=""
+DOCKER_CLI_AUTH_CONFIG=""
+DOCKER_CLI_PINNED_HOST=""
+DOCKER_CLI_ENDPOINT_SOURCE=""
+DOCKER_CLI_ORIGINAL_CONFIG_SET="${DOCKER_CONFIG+x}"
+DOCKER_CLI_ORIGINAL_CONFIG="${DOCKER_CONFIG-}"
+DOCKER_CLI_ORIGINAL_AUTH_SET="${DOCKER_AUTH_CONFIG+x}"
+DOCKER_CLI_ORIGINAL_AUTH="${DOCKER_AUTH_CONFIG-}"
+DOCKER_CLI_ORIGINAL_CONTEXT_SET="${DOCKER_CONTEXT+x}"
+DOCKER_CLI_ORIGINAL_CONTEXT="${DOCKER_CONTEXT-}"
+DOCKER_CLI_ORIGINAL_HOST_SET="${DOCKER_HOST+x}"
+DOCKER_CLI_ORIGINAL_HOST="${DOCKER_HOST-}"
+DOCKER_CLI_ORIGINAL_TLS_SET="${DOCKER_TLS+x}"
+DOCKER_CLI_ORIGINAL_TLS="${DOCKER_TLS-}"
+DOCKER_CLI_ORIGINAL_TLS_VERIFY_SET="${DOCKER_TLS_VERIFY+x}"
+DOCKER_CLI_ORIGINAL_TLS_VERIFY="${DOCKER_TLS_VERIFY-}"
+DOCKER_CLI_ORIGINAL_CERT_PATH_SET="${DOCKER_CERT_PATH+x}"
+DOCKER_CLI_ORIGINAL_CERT_PATH="${DOCKER_CERT_PATH-}"
+DOCKER_CLI_ORIGINAL_API_VERSION_SET="${DOCKER_API_VERSION+x}"
+DOCKER_CLI_ORIGINAL_API_VERSION="${DOCKER_API_VERSION-}"
+DOCKER_CLI_ORIGINAL_CUSTOM_HEADERS_SET="${DOCKER_CUSTOM_HEADERS+x}"
+DOCKER_CLI_ORIGINAL_CUSTOM_HEADERS="${DOCKER_CUSTOM_HEADERS-}"
+DOCKER_CLI_ORIGINAL_MINIKUBE_ACTIVE_SET="${MINIKUBE_ACTIVE_DOCKERD+x}"
+DOCKER_CLI_ORIGINAL_MINIKUBE_ACTIVE="${MINIKUBE_ACTIVE_DOCKERD-}"
+
+MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS="${MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS:-30}"
+MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS="${MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS:-600}"
+MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS="${MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS:-1800}"
+MINIKUBE_DOCKER_HEARTBEAT_SECONDS="${MINIKUBE_DOCKER_HEARTBEAT_SECONDS:-20}"
+MINIKUBE_DOCKER_KILL_GRACE_SECONDS="${MINIKUBE_DOCKER_KILL_GRACE_SECONDS:-5}"
+MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS="${MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS:-5}"
+MINIKUBE_DOCKER_START_TIMEOUT_SECONDS="${MINIKUBE_DOCKER_START_TIMEOUT_SECONDS:-60}"
+MINIKUBE_DOCKER_START_POLL_SECONDS="${MINIKUBE_DOCKER_START_POLL_SECONDS:-2}"
+
+docker_cli_env_error() {
+ printf '%s\n' "$*" >&2
+}
+
+docker_cli_env_validate_seconds() {
+ local name="$1" value="$2" maximum="$3"
+ if ! [[ "$value" =~ ^[1-9][0-9]*$ ]] || (( value > maximum )); then
+ docker_cli_env_error "DOCKER_DEADLINE_INVALID: ${name} must be an integer from 1 to ${maximum}"
+ return 2
+ fi
+}
+
+docker_cli_env_validate_deadlines() {
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" 3600 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS "$MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS" 7200 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_HEARTBEAT_SECONDS "$MINIKUBE_DOCKER_HEARTBEAT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_KILL_GRACE_SECONDS "$MINIKUBE_DOCKER_KILL_GRACE_SECONDS" 30 || return $?
+}
+
+docker_cli_env_validate_startup_deadlines() {
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS \
+ "$MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS" 60 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_START_TIMEOUT_SECONDS \
+ "$MINIKUBE_DOCKER_START_TIMEOUT_SECONDS" 600 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_START_POLL_SECONDS \
+ "$MINIKUBE_DOCKER_START_POLL_SECONDS" 30 || return $?
+}
+
+docker_cli_env_validate_local_endpoint() {
+ local endpoint="$1" port=""
+ case "$endpoint" in
+ unix:///*)
+ # Unix sockets always terminate on this machine. Require an absolute,
+ # single-line path so a malformed context cannot be reinterpreted by the
+ # Docker CLI after it is pinned into DOCKER_HOST.
+ [[ "$endpoint" != *$'\n'* && "$endpoint" != *$'\r'* \
+ && "$endpoint" != *'?'* && "$endpoint" != *'#'* \
+ && "$endpoint" != "unix:///" ]] || {
+ docker_cli_env_error "DOCKER_ENDPOINT_UNSAFE: the resolved Unix endpoint is malformed"
+ return 1
+ }
+ ;;
+ tcp://*)
+ if [[ "$endpoint" =~ ^tcp://(localhost|127\.0\.0\.1):([1-9][0-9]*)$ ]]; then
+ port="${BASH_REMATCH[2]}"
+ elif [[ "$endpoint" =~ ^tcp://\[::1\]:([1-9][0-9]*)$ ]]; then
+ port="${BASH_REMATCH[1]}"
+ else
+ docker_cli_env_error "DOCKER_ENDPOINT_UNSAFE: refusing a non-loopback or malformed TCP endpoint"
+ return 1
+ fi
+ if ! [[ "$port" =~ ^[1-9][0-9]*$ ]] || (( port > 65535 )); then
+ docker_cli_env_error "DOCKER_ENDPOINT_UNSAFE: the resolved loopback endpoint has an invalid port"
+ return 1
+ fi
+ ;;
+ npipe:////./pipe/docker_engine)
+ [[ "${OS:-}" == Windows_NT ]] || {
+ docker_cli_env_error "DOCKER_ENDPOINT_UNSAFE: a Windows named pipe is not local on this host"
+ return 1
+ }
+ ;;
+ *)
+ docker_cli_env_error "DOCKER_ENDPOINT_UNSAFE: refusing a remote or ambiguous Docker endpoint"
+ return 1
+ ;;
+ esac
+}
+
+docker_cli_env_resolve_context_endpoint() {
+ local record="" status=0 context_host="" skip_tls="" tls_material=""
+
+ # Context inspection is metadata-only and happens before DOCKER_CONFIG is
+ # replaced. DOCKER_AUTH_CONFIG and custom headers are excluded even here so
+ # endpoint discovery cannot consult registry auth or carry ambient headers.
+ record="$({
+ unset DOCKER_AUTH_CONFIG DOCKER_CUSTOM_HEADERS
+ node "$DOCKER_CLI_DEADLINE_RUNNER" \
+ --timeout-seconds "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ --heartbeat-seconds "$MINIKUBE_DOCKER_HEARTBEAT_SECONDS" \
+ --kill-grace-seconds "$MINIKUBE_DOCKER_KILL_GRACE_SECONDS" \
+ --label docker-endpoint-resolve -- \
+ docker context inspect \
+ --format '{{.Endpoints.docker.Host}}{{"\t"}}{{.Endpoints.docker.SkipTLSVerify}}{{"\t"}}{{json .TLSMaterial}}'
+ })" || status=$?
+ if [[ "$status" -ne 0 ]]; then
+ docker_cli_env_error "DOCKER_ENDPOINT_REQUIRED: could not resolve the active Docker context"
+ return "$status"
+ fi
+ if [[ -z "$record" || "$record" == *$'\n'* ]]; then
+ docker_cli_env_error "DOCKER_ENDPOINT_REQUIRED: the active Docker context did not resolve exactly one endpoint"
+ return 1
+ fi
+
+ IFS=$'\t' read -r context_host skip_tls tls_material <<<"$record"
+ [[ -n "$context_host" && -n "$skip_tls" && -n "$tls_material" ]] || {
+ docker_cli_env_error "DOCKER_ENDPOINT_REQUIRED: the active Docker context metadata is incomplete"
+ return 1
+ }
+ case "$skip_tls" in
+ true|false) ;;
+ *)
+ docker_cli_env_error "DOCKER_ENDPOINT_REQUIRED: the active Docker context TLS metadata is invalid"
+ return 1
+ ;;
+ esac
+
+ docker_cli_env_validate_local_endpoint "$context_host" || return $?
+ if [[ "$context_host" == tcp://* ]] \
+ && { [[ "$skip_tls" == true ]] || [[ "$tls_material" != '{}' && "$tls_material" != null ]]; }; then
+ docker_cli_env_error "DOCKER_ENDPOINT_UNSAFE: a config-only loopback TLS context cannot be isolated safely"
+ return 1
+ fi
+
+ DOCKER_CLI_PINNED_HOST="$context_host"
+ DOCKER_CLI_ENDPOINT_SOURCE=context
+}
+
+docker_cli_env_resolve_endpoint() {
+ if [[ -n "${DOCKER_HOST:-}" ]]; then
+ docker_cli_env_validate_local_endpoint "$DOCKER_HOST" || return $?
+ DOCKER_CLI_PINNED_HOST="$DOCKER_HOST"
+ DOCKER_CLI_ENDPOINT_SOURCE=explicit_host
+ return 0
+ fi
+ docker_cli_env_resolve_context_endpoint
+}
+
+docker_cli_env_apply_pinned_endpoint() {
+ [[ -n "$DOCKER_CLI_PINNED_HOST" ]] || {
+ docker_cli_env_error "DOCKER_ENDPOINT_REQUIRED: no approved local Docker endpoint was resolved"
+ return 1
+ }
+
+ # DOCKER_HOST intentionally wins after this point. Clearing DOCKER_CONTEXT
+ # removes both its documented precedence and any dependency on context files
+ # that are absent from the isolated task/private configs.
+ export DOCKER_HOST="$DOCKER_CLI_PINNED_HOST"
+ unset DOCKER_CONTEXT 2>/dev/null || true
+ if [[ "$DOCKER_CLI_ENDPOINT_SOURCE" == context ]]; then
+ # Context-derived local/rootless Unix endpoints do not use ambient TLS.
+ # Loopback context endpoints with TLS were rejected during resolution.
+ unset DOCKER_TLS DOCKER_TLS_VERIFY DOCKER_CERT_PATH 2>/dev/null || true
+ fi
+}
+
+docker_cli_env_find_buildx() {
+ local explicit="${MINIKUBE_DOCKER_BUILDX_PLUGIN:-}"
+ local candidate
+ if [[ -n "$explicit" ]]; then
+ if [[ ! -f "$explicit" || ! -x "$explicit" ]]; then
+ docker_cli_env_error "DOCKER_BUILDX_REQUIRED: the explicitly configured buildx plugin is not executable"
+ return 1
+ fi
+ printf '%s\n' "$explicit"
+ return 0
+ fi
+
+ # Docker Desktop and the official Linux packages install buildx in one of
+ # these CLI-owned locations. User Docker config directories are deliberately
+ # excluded because they sit beside ambient auth and credential-helper state.
+ for candidate in \
+ /Applications/Docker.app/Contents/Resources/cli-plugins/docker-buildx \
+ /usr/local/lib/docker/cli-plugins/docker-buildx \
+ /opt/homebrew/lib/docker/cli-plugins/docker-buildx \
+ /usr/libexec/docker/cli-plugins/docker-buildx \
+ /usr/lib/docker/cli-plugins/docker-buildx; do
+ if [[ -f "$candidate" && -x "$candidate" ]]; then
+ printf '%s\n' "$candidate"
+ return 0
+ fi
+ done
+ return 1
+}
+
+docker_cli_run_with_config() {
+ local config="$1" label="$2" timeout_seconds="$3"
+ shift 3
+ docker_cli_env_validate_local_endpoint "${DOCKER_HOST:-}" || return $?
+ (
+ # This override intentionally belongs only to the child operation.
+ # shellcheck disable=SC2030
+ export DOCKER_CONFIG="$config"
+ unset DOCKER_AUTH_CONFIG DOCKER_CONTEXT DOCKER_CUSTOM_HEADERS
+ node "$DOCKER_CLI_DEADLINE_RUNNER" \
+ --timeout-seconds "$timeout_seconds" \
+ --heartbeat-seconds "$MINIKUBE_DOCKER_HEARTBEAT_SECONDS" \
+ --kill-grace-seconds "$MINIKUBE_DOCKER_KILL_GRACE_SECONDS" \
+ --label "$label" -- "$@"
+ )
+}
+
+docker_cli_env_verify_pinned_endpoint() {
+ local effective_host="" status=0
+ effective_host="$(
+ docker_cli_run_with_config "$DOCKER_CLI_TASK_CONFIG" docker-endpoint-verify \
+ "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ docker context inspect --format '{{.Endpoints.docker.Host}}' default
+ )" || status=$?
+ if [[ "$status" -ne 0 ]]; then
+ docker_cli_env_error "DOCKER_ENDPOINT_REQUIRED: the isolated Docker config could not verify its endpoint"
+ return "$status"
+ fi
+ if [[ "$effective_host" != "$DOCKER_CLI_PINNED_HOST" ]]; then
+ docker_cli_env_error "DOCKER_ENDPOINT_MISMATCH: the isolated Docker config selected a different endpoint"
+ return 1
+ fi
+}
+
+docker_cli_run_public() {
+ local label="$1" timeout_seconds="$2"
+ shift 2
+ if [[ -z "$DOCKER_CLI_TASK_CONFIG" || ! -d "$DOCKER_CLI_TASK_CONFIG" ]]; then
+ docker_cli_env_error "DOCKER_CONFIG_REQUIRED: docker_cli_env_prepare must run before Docker operations"
+ return 1
+ fi
+ docker_cli_run_with_config "$DOCKER_CLI_TASK_CONFIG" "$label" "$timeout_seconds" "$@"
+}
+
+docker_cli_env_require_auth_config() {
+ local requested="${MINIKUBE_DOCKER_AUTH_CONFIG:-}"
+ if [[ -z "$requested" ]]; then
+ docker_cli_env_error "REGISTRY_AUTH_REQUIRED: set MINIKUBE_DOCKER_AUTH_CONFIG to an explicit Docker config directory"
+ return 1
+ fi
+ if [[ ! -d "$requested" || ! -f "$requested/config.json" || ! -r "$requested/config.json" ]]; then
+ docker_cli_env_error "REGISTRY_AUTH_REQUIRED: the explicit Docker config directory is not readable"
+ return 1
+ fi
+ DOCKER_CLI_AUTH_CONFIG="$(cd -- "$requested" && pwd -P)" || {
+ docker_cli_env_error "REGISTRY_AUTH_REQUIRED: the explicit Docker config directory cannot be resolved"
+ return 1
+ }
+}
+
+docker_cli_run_private() {
+ local label="$1" timeout_seconds="$2"
+ shift 2
+ docker_cli_env_require_auth_config || return $?
+ docker_cli_run_with_config "$DOCKER_CLI_AUTH_CONFIG" "$label" "$timeout_seconds" "$@"
+}
+
+docker_cli_env_prepare() {
+ local validate_buildx="${1:-true}"
+ local temp_root old_umask buildx_path="" buildx_status=0
+ if [[ -n "$DOCKER_CLI_TASK_CONFIG" && -d "$DOCKER_CLI_TASK_CONFIG" ]]; then
+ return 0
+ fi
+ case "$validate_buildx" in
+ true|false) ;;
+ *)
+ docker_cli_env_error "DOCKER_CONFIG_REQUIRED: buildx validation mode must be true or false"
+ return 2
+ ;;
+ esac
+
+ command -v node >/dev/null 2>&1 || {
+ docker_cli_env_error "DOCKER_DEADLINE_REQUIRED: node is required for bounded Docker operations"
+ return 1
+ }
+ command -v docker >/dev/null 2>&1 || {
+ docker_cli_env_error "DOCKER_CLI_REQUIRED: docker is not installed"
+ return 1
+ }
+ [[ -f "$DOCKER_CLI_DEADLINE_RUNNER" ]] || {
+ docker_cli_env_error "DOCKER_DEADLINE_REQUIRED: the deadline runner is missing"
+ return 1
+ }
+ if [[ "$validate_buildx" == true ]]; then
+ docker_cli_env_validate_deadlines || return $?
+ else
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS \
+ "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_HEARTBEAT_SECONDS \
+ "$MINIKUBE_DOCKER_HEARTBEAT_SECONDS" 300 || return $?
+ docker_cli_env_validate_seconds MINIKUBE_DOCKER_KILL_GRACE_SECONDS \
+ "$MINIKUBE_DOCKER_KILL_GRACE_SECONDS" 30 || return $?
+ fi
+
+ # Resolve while the caller's context metadata is still available, then pin
+ # the approved local endpoint before replacing the Docker configuration.
+ docker_cli_env_resolve_endpoint || return $?
+
+ temp_root="${TMPDIR:-/tmp}"
+ [[ -d "$temp_root" ]] || {
+ docker_cli_env_error "DOCKER_CONFIG_REQUIRED: the temporary directory is unavailable"
+ return 1
+ }
+ old_umask="$(umask)"
+ umask 077
+ DOCKER_CLI_TASK_CONFIG="$(mktemp -d "${temp_root%/}/evenfire-minikube-docker.XXXXXX")" || {
+ umask "$old_umask"
+ docker_cli_env_error "DOCKER_CONFIG_REQUIRED: could not create the isolated Docker config"
+ return 1
+ }
+ mkdir -p "$DOCKER_CLI_TASK_CONFIG/cli-plugins"
+ printf '{}\n' >"$DOCKER_CLI_TASK_CONFIG/config.json"
+ chmod 700 "$DOCKER_CLI_TASK_CONFIG" "$DOCKER_CLI_TASK_CONFIG/cli-plugins"
+ chmod 600 "$DOCKER_CLI_TASK_CONFIG/config.json"
+ umask "$old_umask"
+
+ docker_cli_env_apply_pinned_endpoint || return $?
+ # The task config must also become the caller's active config.
+ # shellcheck disable=SC2031
+ export DOCKER_CONFIG="$DOCKER_CLI_TASK_CONFIG"
+ unset DOCKER_AUTH_CONFIG 2>/dev/null || true
+ docker_cli_env_verify_pinned_endpoint || return $?
+
+ if [[ "$validate_buildx" == false ]]; then
+ return 0
+ fi
+
+ if [[ -n "${MINIKUBE_DOCKER_BUILDX_PLUGIN:-}" ]]; then
+ buildx_path="$(docker_cli_env_find_buildx)" || return $?
+ ln -s "$buildx_path" "$DOCKER_CLI_TASK_CONFIG/cli-plugins/docker-buildx"
+ elif buildx_path="$(docker_cli_env_find_buildx)"; then
+ ln -s "$buildx_path" "$DOCKER_CLI_TASK_CONFIG/cli-plugins/docker-buildx"
+ fi
+
+ # A system-integrated plugin may work even when no explicit file candidate
+ # was found. Validate through the isolated config and never download a plugin.
+ docker_cli_run_public buildx-version "$MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS" \
+ docker buildx version >/dev/null 2>&1 || buildx_status=$?
+ if [[ "$buildx_status" -ne 0 ]]; then
+ docker_cli_env_error "DOCKER_BUILDX_REQUIRED: no usable installed buildx plugin was found"
+ return "$buildx_status"
+ fi
+}
+
+docker_cli_wait_for_info() {
+ docker_cli_env_validate_startup_deadlines || return $?
+
+ # The literal script is intentional: poll_seconds is positional data passed
+ # to the child shell, while the outer deadline owns the whole process group.
+ # shellcheck disable=SC2016
+ docker_cli_run_public docker-info-startup-wait "$MINIKUBE_DOCKER_START_TIMEOUT_SECONDS" \
+ bash -c '
+ poll_seconds="$1"
+ while ! docker info >/dev/null 2>&1; do
+ sleep "$poll_seconds"
+ done
+ ' _ "$MINIKUBE_DOCKER_START_POLL_SECONDS"
+}
+
+docker_cli_env_cleanup() {
+ local task_config="$DOCKER_CLI_TASK_CONFIG" cleanup_status=0
+ DOCKER_CLI_TASK_CONFIG=""
+ DOCKER_CLI_AUTH_CONFIG=""
+ DOCKER_CLI_PINNED_HOST=""
+ DOCKER_CLI_ENDPOINT_SOURCE=""
+
+ if [[ -n "$task_config" ]]; then
+ case "${task_config##*/}" in
+ evenfire-minikube-docker.*)
+ if [[ -d "$task_config" && ! -L "$task_config" ]]; then
+ rm -rf -- "$task_config" || cleanup_status=$?
+ fi
+ ;;
+ *)
+ docker_cli_env_error "DOCKER_CONFIG_CLEANUP_REFUSED: unexpected task config path"
+ cleanup_status=1
+ ;;
+ esac
+ fi
+
+ docker_cli_env_restore_variable DOCKER_CONFIG "$DOCKER_CLI_ORIGINAL_CONFIG_SET" "$DOCKER_CLI_ORIGINAL_CONFIG"
+ docker_cli_env_restore_variable DOCKER_AUTH_CONFIG "$DOCKER_CLI_ORIGINAL_AUTH_SET" "$DOCKER_CLI_ORIGINAL_AUTH"
+ docker_cli_env_restore_variable DOCKER_CONTEXT "$DOCKER_CLI_ORIGINAL_CONTEXT_SET" "$DOCKER_CLI_ORIGINAL_CONTEXT"
+ docker_cli_env_restore_variable DOCKER_HOST "$DOCKER_CLI_ORIGINAL_HOST_SET" "$DOCKER_CLI_ORIGINAL_HOST"
+ docker_cli_env_restore_variable DOCKER_TLS "$DOCKER_CLI_ORIGINAL_TLS_SET" "$DOCKER_CLI_ORIGINAL_TLS"
+ docker_cli_env_restore_variable DOCKER_TLS_VERIFY "$DOCKER_CLI_ORIGINAL_TLS_VERIFY_SET" "$DOCKER_CLI_ORIGINAL_TLS_VERIFY"
+ docker_cli_env_restore_variable DOCKER_CERT_PATH "$DOCKER_CLI_ORIGINAL_CERT_PATH_SET" "$DOCKER_CLI_ORIGINAL_CERT_PATH"
+ docker_cli_env_restore_variable DOCKER_API_VERSION "$DOCKER_CLI_ORIGINAL_API_VERSION_SET" "$DOCKER_CLI_ORIGINAL_API_VERSION"
+ docker_cli_env_restore_variable DOCKER_CUSTOM_HEADERS "$DOCKER_CLI_ORIGINAL_CUSTOM_HEADERS_SET" "$DOCKER_CLI_ORIGINAL_CUSTOM_HEADERS"
+ docker_cli_env_restore_variable MINIKUBE_ACTIVE_DOCKERD "$DOCKER_CLI_ORIGINAL_MINIKUBE_ACTIVE_SET" "$DOCKER_CLI_ORIGINAL_MINIKUBE_ACTIVE"
+ return "$cleanup_status"
+}
+
+docker_cli_env_restore_variable() {
+ local name="$1" was_set="$2" value="$3"
+ if [[ "$was_set" == x ]]; then
+ printf -v "$name" '%s' "$value"
+ export "${name?}"
+ else
+ unset "$name" 2>/dev/null || true
+ fi
+}
+
+docker_cli_env_main_cleanup() {
+ local status=$? cleanup_status=0
+ trap - EXIT HUP QUIT INT TERM
+ docker_cli_env_cleanup || cleanup_status=$?
+ if [[ "$status" -eq 0 && "$cleanup_status" -ne 0 ]]; then
+ status="$cleanup_status"
+ fi
+ exit "$status"
+}
+
+docker_cli_env_main() {
+ if [[ "$#" -ne 1 ]]; then
+ docker_cli_env_error "usage: ${0##*/} --check-info | --wait-for-info"
+ return 2
+ fi
+
+ # Startup probes do not build images, so requiring buildx here would delay or
+ # block Docker Desktop startup for an unrelated plugin. They still use the
+ # same empty task-local config and process-group deadline runner.
+ docker_cli_env_prepare false || return $?
+ docker_cli_env_validate_startup_deadlines || return $?
+ case "$1" in
+ --check-info)
+ docker_cli_run_public docker-info-startup-probe \
+ "$MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS" docker info >/dev/null
+ ;;
+ --wait-for-info)
+ docker_cli_wait_for_info
+ ;;
+ *)
+ docker_cli_env_error "usage: ${0##*/} --check-info | --wait-for-info"
+ return 2
+ ;;
+ esac
+}
+
+if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
+ set -euo pipefail
+ trap docker_cli_env_main_cleanup EXIT
+ trap 'exit 129' HUP
+ trap 'exit 131' QUIT
+ trap 'exit 130' INT
+ trap 'exit 143' TERM
+ docker_cli_env_main "$@"
+fi
diff --git a/scripts/minikube/full-setup.sh b/scripts/minikube/full-setup.sh
index 40db9ba24..acf0d7022 100755
--- a/scripts/minikube/full-setup.sh
+++ b/scripts/minikube/full-setup.sh
@@ -348,8 +348,30 @@ T2_PROFILE="${T2_PROFILE:-${PROFILE}}"
T2_CONTEXT="${T2_CONTEXT:-${PROFILE}}"
T2_GATE_ID="${T2_GATE_ID:-minikube-setup}"
T2_SKIP_LOCK="${T2_SKIP_LOCK:-false}"
+T2_SETUP_HANDOFF_REQUIRED="${T2_SETUP_HANDOFF_REQUIRED:-false}"
+T2_SETUP_HANDOFF_TRANSITION="${T2_SETUP_HANDOFF_TRANSITION:-}"
+T2_SETUP_HANDOFF_ROOT="${T2_SETUP_HANDOFF_ROOT:-${PROJECT_DIR}/.local-notes/infra/t2-setup-handoffs}"
+T2_SETUP_HANDOFF_TTL_SECONDS="${T2_SETUP_HANDOFF_TTL_SECONDS:-300}"
# shellcheck source=scripts/minikube/t2-common.sh
source "${SCRIPT_DIR}/t2-common.sh"
+case "${T2_SETUP_HANDOFF_REQUIRED}" in
+ true|false) ;;
+ *) err "T2_SETUP_HANDOFF_REQUIRED must be true or false"; exit 1 ;;
+esac
+if [ "${T2_SETUP_HANDOFF_REQUIRED}" = true ]; then
+ if [ "${T2_SKIP_LOCK}" != true ] || [ -z "${T2_RUN_ID}" ]; then
+ err "T2 setup handoff requires the live inherited T2 lease and run identity"
+ exit 1
+ fi
+ case "${T2_SETUP_HANDOFF_TRANSITION}" in
+ full-bootstrap|full-reconcile) ;;
+ *) err "T2 setup handoff requires a full-bootstrap or full-reconcile transition"; exit 1 ;;
+ esac
+ if [ "${IMAGE_SOURCE}" != local ]; then
+ err "T2 setup handoff requires IMAGE_SOURCE=local"
+ exit 1
+ fi
+fi
TOTAL_STEPS=12
# MINIKUBE_IMAGE_TAG overrides the committed pin AT RENDER TIME ONLY.
#
@@ -468,9 +490,55 @@ if [ "${MINIKUBE_FULL_SETUP_CONFIG_ONLY:-false}" = "true" ]; then
fi
MINIKUBE_START_SCRIPT="${MINIKUBE_START_SCRIPT:-${SCRIPT_DIR}/start.sh}"
+MINIKUBE_SETUP_DEADLINE_RUNNER="${MINIKUBE_SETUP_DEADLINE_RUNNER:-${SCRIPT_DIR}/run-with-deadline.mjs}"
+MINIKUBE_SETUP_STATUS_TIMEOUT_SECONDS="${MINIKUBE_SETUP_STATUS_TIMEOUT_SECONDS:-60}"
+MINIKUBE_SETUP_START_TIMEOUT_SECONDS="${MINIKUBE_SETUP_START_TIMEOUT_SECONDS:-900}"
+MINIKUBE_SETUP_VALIDATE_TIMEOUT_SECONDS="${MINIKUBE_SETUP_VALIDATE_TIMEOUT_SECONDS:-300}"
+MINIKUBE_SETUP_DELETE_TIMEOUT_SECONDS="${MINIKUBE_SETUP_DELETE_TIMEOUT_SECONDS:-300}"
+MINIKUBE_SETUP_DOCKER_TIMEOUT_SECONDS="${MINIKUBE_SETUP_DOCKER_TIMEOUT_SECONDS:-30}"
+
+validate_setup_deadline() {
+ local name="$1" value="$2" maximum="$3"
+ if ! [[ "${value}" =~ ^[1-9][0-9]*$ ]] || (( 10#${value} > maximum )); then
+ err "${name} must be an integer from 1 to ${maximum}"
+ return 1
+ fi
+}
+
+validate_setup_deadline MINIKUBE_SETUP_STATUS_TIMEOUT_SECONDS "${MINIKUBE_SETUP_STATUS_TIMEOUT_SECONDS}" 300
+validate_setup_deadline MINIKUBE_SETUP_START_TIMEOUT_SECONDS "${MINIKUBE_SETUP_START_TIMEOUT_SECONDS}" 1800
+validate_setup_deadline MINIKUBE_SETUP_VALIDATE_TIMEOUT_SECONDS "${MINIKUBE_SETUP_VALIDATE_TIMEOUT_SECONDS}" 900
+validate_setup_deadline MINIKUBE_SETUP_DELETE_TIMEOUT_SECONDS "${MINIKUBE_SETUP_DELETE_TIMEOUT_SECONDS}" 900
+validate_setup_deadline MINIKUBE_SETUP_DOCKER_TIMEOUT_SECONDS "${MINIKUBE_SETUP_DOCKER_TIMEOUT_SECONDS}" 300
+[[ -f "${MINIKUBE_SETUP_DEADLINE_RUNNER}" ]] || {
+ err "bounded runtime helper is missing: ${MINIKUBE_SETUP_DEADLINE_RUNNER}"
+ exit 1
+}
+
+run_setup_with_deadline() {
+ local label="$1" timeout_seconds="$2"
+ shift 2
+ node "${MINIKUBE_SETUP_DEADLINE_RUNNER}" \
+ --timeout-seconds "${timeout_seconds}" \
+ --heartbeat-seconds "${MINIKUBE_DOCKER_HEARTBEAT_SECONDS:-20}" \
+ --kill-grace-seconds "${MINIKUBE_DOCKER_KILL_GRACE_SECONDS:-5}" \
+ --label "${label}" -- "$@"
+}
minikube_status_snapshot() {
- minikube -p "$PROFILE" status 2>/dev/null || true
+ local output status=0
+ output="$(run_setup_with_deadline minikube-setup-status \
+ "${MINIKUBE_SETUP_STATUS_TIMEOUT_SECONDS}" \
+ minikube -p "$PROFILE" status)" || status=$?
+ # `minikube status` legitimately returns a small non-zero code for an absent
+ # or partially started profile while still printing the state we classify
+ # below. Runner timeout/signal/spawn failures are operational failures and
+ # must not be reinterpreted as permission to start or recreate a profile.
+ if (( status >= 124 )); then
+ err "Minikube status probe failed before profile state could be classified (status ${status})"
+ return "$status"
+ fi
+ printf '%s' "$output"
}
minikube_status_is_healthy() {
@@ -496,7 +564,12 @@ recreate_broken_minikube_profile() {
exit 1
fi
warn "Minikube profile '${PROFILE}' is partially started (host up, control plane down). Recreating it from scratch..."
- minikube delete -p "$PROFILE" >/dev/null 2>&1 || true
+ if ! run_setup_with_deadline minikube-setup-delete \
+ "${MINIKUBE_SETUP_DELETE_TIMEOUT_SECONDS}" \
+ minikube delete -p "$PROFILE" >/dev/null; then
+ err "Failed to delete explicitly confirmed broken profile '${PROFILE}' within the deadline"
+ exit 1
+ fi
ok "Removed broken '${PROFILE}' profile"
}
@@ -507,7 +580,9 @@ start_minikube_cluster() {
MINIKUBE_PROFILE="${PROFILE}" \
MINIKUBE_MULTI_NODE="${MINIKUBE_MULTI_NODE:-false}" \
MINIKUBE_NODES="${MINIKUBE_NODES:-}" \
- "${MINIKUBE_START_SCRIPT}"
+ run_setup_with_deadline minikube-setup-start \
+ "${MINIKUBE_SETUP_START_TIMEOUT_SECONDS}" \
+ "${MINIKUBE_START_SCRIPT}"
ok "Minikube cluster '${PROFILE}' started"
}
@@ -516,12 +591,18 @@ validate_minikube_cluster() {
MINIKUBE_PROFILE="${PROFILE}" \
MINIKUBE_MULTI_NODE="${MINIKUBE_MULTI_NODE:-false}" \
MINIKUBE_NODES="${MINIKUBE_NODES:-}" \
- "${MINIKUBE_START_SCRIPT}" --validate-only
+ run_setup_with_deadline minikube-setup-validate \
+ "${MINIKUBE_SETUP_VALIDATE_TIMEOUT_SECONDS}" \
+ "${MINIKUBE_START_SCRIPT}" --validate-only
ok "Minikube cluster '${PROFILE}' validated"
}
maybe_exit_after_cluster_step() {
if [ "${MINIKUBE_SETUP_EXIT_AFTER_CLUSTER:-false}" = "true" ]; then
+ if [ "${T2_SETUP_HANDOFF_REQUIRED}" = true ]; then
+ err "T2 strict setup cannot stop after cluster verification"
+ return 1
+ fi
log "MINIKUBE_SETUP_EXIT_AFTER_CLUSTER=true — stopping after cluster verification"
exit 0
fi
@@ -554,8 +635,12 @@ ensure_control_postgres_ready() {
# ======================================================================
step_header 1 $TOTAL_STEPS "Preconditions"
-# Check Docker is running
-if ! docker info &>/dev/null; then
+# Check Docker through the same isolated, bounded local-endpoint contract used
+# by T1 and image builds. This avoids paying setup cost after an ambient Docker
+# context or credential helper has stalled.
+if ! MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS="${MINIKUBE_SETUP_DOCKER_TIMEOUT_SECONDS}" \
+ MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS="${MINIKUBE_SETUP_DOCKER_TIMEOUT_SECONDS}" \
+ bash "${SCRIPT_DIR}/docker-cli-env.sh" --check-info >/dev/null; then
err "Docker is not running. Please start Docker Desktop first."
exit 1
fi
@@ -1637,6 +1722,36 @@ done
echo ""
echo -e "${BOLD}================================================================${NC}"
+if [ "${T2_SETUP_HANDOFF_REQUIRED}" = true ]; then
+ # The ordinary setup summary historically gates only CORE_DEPLOYS and may
+ # return zero after printing "partially complete". A T2 handoff is stricter:
+ # reuse the final T2 deployment inventory contract (including additional
+ # deployed workloads) and make an unready/missing required deployment block
+ # both publication and setup success.
+ prior_bootstrap_required="${T2_BOOTSTRAP_REQUIRED}"
+ prior_plan_mode="${T2_PLAN_MODE}"
+ T2_BOOTSTRAP_REQUIRED=false
+ T2_PLAN_MODE=false
+ if ! t2_deployment_check; then
+ all_ready=false
+ fi
+ T2_BOOTSTRAP_REQUIRED="${prior_bootstrap_required}"
+ T2_PLAN_MODE="${prior_plan_mode}"
+ if ! T2_PROJECT_DIR="${T2_PROJECT_DIR}" T2_WORKTREE_ID="${T2_WORKTREE_ID}" \
+ T2_RUN_ID="${T2_RUN_ID}" T2_PROFILE="${T2_PROFILE}" T2_CONTEXT="${T2_CONTEXT}" \
+ T2_BRANCH="${T2_BRANCH}" T2_HEAD="${T2_HEAD}" T2_SKIP_LOCK="${T2_SKIP_LOCK}" \
+ T2_LOCK_KEY="${T2_LOCK_KEY}" T2_LOCK_TOKEN="${T2_LOCK_TOKEN}" \
+ T2_IMAGE_MANIFEST="${T2_IMAGE_MANIFEST}" \
+ T2_SETUP_HANDOFF_ROOT="${T2_SETUP_HANDOFF_ROOT}" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="${T2_SETUP_HANDOFF_TTL_SECONDS}" \
+ T2_SETUP_HANDOFF_TRANSITION="${T2_SETUP_HANDOFF_TRANSITION}" \
+ T2_SETUP_HANDOFF_SETUP_COMPLETE="${all_ready}" \
+ bash "${SCRIPT_DIR}/t2-setup-handoff.sh" create; then
+ err "T2 strict setup is incomplete; refusing setup success and handoff"
+ exit 1
+ fi
+ ok "Created the one-shot T2 setup-complete handoff"
+fi
if [ "$all_ready" = true ]; then
echo -e "${GREEN}${BOLD} Minikube setup complete! All core services ready.${NC}"
else
diff --git a/scripts/minikube/pf-all-stack.sh b/scripts/minikube/pf-all-stack.sh
index 907f59c23..74851d923 100755
--- a/scripts/minikube/pf-all-stack.sh
+++ b/scripts/minikube/pf-all-stack.sh
@@ -3,25 +3,72 @@
set -euo pipefail
+SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+WORKTREE_ROOT="$(git -C "${SCRIPT_DIR}/../.." rev-parse --show-toplevel 2>/dev/null)" || {
+ echo 'ERROR: unable to resolve the port-forward worktree root' >&2
+ exit 1
+}
+WORKTREE_ROOT="$(cd -- "${WORKTREE_ROOT}" && pwd -P)"
+# shellcheck source=scripts/minikube/port-forward-owner.sh
+source "${SCRIPT_DIR}/port-forward-owner.sh"
+
PROFILE="${MINIKUBE_PROFILE:-clerum-test}"
-KC=(kubectl --context="${PROFILE}")
+CONTEXT="${KUBECONTEXT:-${PROFILE}}"
+KC=(kubectl "--context=${CONTEXT}")
HOLD=false
-PIDS=()
PIDFILES=()
+PIDFILE_NAMESPACES=()
PIDFILE_SERVICES=()
+PIDFILE_LOCAL_PORTS=()
+PIDFILE_REMOTE_PORTS=()
SAFE_PROFILE="${PROFILE//[^A-Za-z0-9_.-]/_}"
HAS_PROFILE_OWNED_PORTS=false
STARTUP_COMPLETE=false
+PROFILE_PIDS_DIR=''
+HEALTH_ATTEMPTS="${PF_HEALTH_ATTEMPTS:-20}"
+HEALTH_DELAY="${PF_HEALTH_DELAY:-0.5}"
+STARTUP_DELAY="${PF_STARTUP_DELAY:-0.2}"
if [[ ! "$PROFILE" =~ ^[A-Za-z0-9][A-Za-z0-9_.-]*$ ]]; then
echo "ERROR: invalid Minikube profile identifier: $PROFILE" >&2
exit 1
fi
-set +u
-PROFILE_PID_ROOT="$CLERUM_PROFILE_CACHE_ROOT"
-set -u
-if [[ -z "$PROFILE_PID_ROOT" ]]; then
- PROFILE_PID_ROOT="$HOME/.cache/clerum/minikube-profiles"
+if [[ "${CONTEXT}" != "${PROFILE}" ]]; then
+ echo "ERROR: Minikube profile/context mismatch (${PROFILE} != ${CONTEXT}); refusing a foreign port-forward" >&2
+ exit 1
fi
+if [[ ! "${HEALTH_ATTEMPTS}" =~ ^[1-9][0-9]*$ ||
+ ! "${HEALTH_DELAY}" =~ ^[0-9]+([.][0-9]+)?$ ||
+ ! "${STARTUP_DELAY}" =~ ^[0-9]+([.][0-9]+)?$ ]]; then
+ echo 'ERROR: invalid port-forward startup/health wait configuration' >&2
+ exit 1
+fi
+PROFILE_PID_ROOT="${CLERUM_PROFILE_CACHE_ROOT:-${HOME}/.cache/clerum/minikube-profiles}"
+[[ "${PROFILE_PID_ROOT}" == /* ]] || {
+ echo "ERROR: profile cache root must be absolute: ${PROFILE_PID_ROOT}" >&2
+ exit 1
+}
+
+ensure_private_directory() {
+ local directory="$1"
+ [[ ! -L "${directory}" ]] || {
+ echo "ERROR: refusing a symlinked profile cache directory: ${directory}" >&2
+ return 1
+ }
+ mkdir -p -- "${directory}"
+ [[ -d "${directory}" && ! -L "${directory}" ]] || {
+ echo "ERROR: profile cache path is not a private directory: ${directory}" >&2
+ return 1
+ }
+}
+
+prepare_profile_pid_directory() {
+ local profile_dir="${PROFILE_PID_ROOT}/${PROFILE}"
+ ensure_private_directory "${PROFILE_PID_ROOT}"
+ ensure_private_directory "${profile_dir}"
+ PROFILE_PIDS_DIR="${profile_dir}/pids"
+ ensure_private_directory "${PROFILE_PIDS_DIR}"
+ chmod 700 "${PROFILE_PIDS_DIR}"
+}
load_branch_profile_ports_env() {
local file="$1"
@@ -60,7 +107,7 @@ load_branch_profile_ports_env() {
done < "${file}"
}
-BRANCH_PROFILE_PORTS_ENV="${CLERUM_PROFILE_PORTS_ENV:-${HOME}/.cache/clerum/minikube-profiles/${PROFILE}/ports.env}"
+BRANCH_PROFILE_PORTS_ENV="${CLERUM_PROFILE_PORTS_ENV:-${PROFILE_PID_ROOT}/${PROFILE}/ports.env}"
if [[ -f "${BRANCH_PROFILE_PORTS_ENV}" ]]; then
load_branch_profile_ports_env "${BRANCH_PROFILE_PORTS_ENV}"
HAS_PROFILE_OWNED_PORTS=true
@@ -129,58 +176,32 @@ require_random_port_for_branch_profile REGISTRY_API_PORT "${REGISTRY_API_PORT}"
require_random_port_for_branch_profile WORKFLOW_APPROVAL_READER_PORT "${WORKFLOW_APPROVAL_READER_PORT}" 8098
require_random_port_for_branch_profile MCP_HOST_PORT "${MCP_HOST_PORT}" 8080
+prepare_profile_pid_directory
+umask 077
+
cleanup() {
- if [[ "${HOLD}" != "true" && "${STARTUP_COMPLETE}" == "true" ]]; then
- return 0
- fi
- local index
- for index in "${!PIDFILES[@]}"; do
- kill_owned_pidfile "${PIDFILES[$index]}" "${PIDFILE_SERVICES[$index]}" || true
+ local index cleanup_status=0
+ for ((index = ${#PIDFILES[@]} - 1; index >= 0; index -= 1)); do
+ if ! pf_owner_cleanup_record "${PIDFILES[$index]}" "${PROFILE}" "${CONTEXT}" \
+ "${WORKTREE_ROOT}" "${PIDFILE_NAMESPACES[$index]}" \
+ "${PIDFILE_SERVICES[$index]}" "${PIDFILE_LOCAL_PORTS[$index]}" \
+ "${PIDFILE_REMOTE_PORTS[$index]}"; then
+ cleanup_status=1
+ fi
done
+ return "${cleanup_status}"
}
-kill_owned_pidfile() {
- local pidfile="$1"
- local service="$2"
- local pid command_line expected_start actual_start
- [[ -f "$pidfile" ]] || return 0
- pid="$(sed -n '1p' "$pidfile" 2>/dev/null || true)"
- if [[ ! "$pid" =~ ^[1-9][0-9]*$ ]]; then
- rm -f -- "$pidfile"
- return 0
- fi
- if ! kill -0 "$pid" 2>/dev/null; then
- rm -f -- "$pidfile"
- return 0
- fi
- command_line="$(ps -p "$pid" -o command= 2>/dev/null || true)"
- if [[ "$command_line" != *port-forward* ||
- "$command_line" != *"svc/$service"* ||
- "$command_line" != *"$PROFILE"* ]]; then
- echo "ERROR: refusing to kill PID $pid from $pidfile; it is not the $PROFILE $service port-forward" >&2
- return 1
- fi
- expected_start="$(sed -n 's/^PROCESS_START=//p' "$pidfile" 2>/dev/null | head -1 || true)"
- if [[ -z "$expected_start" ]]; then
- echo "ERROR: refusing to kill PID $pid from $pidfile; its process-start signature is missing" >&2
- return 1
+on_exit() {
+ local status=$? cleanup_status=0
+ trap - EXIT INT TERM
+ if [[ "${HOLD}" == true || "${STARTUP_COMPLETE}" != true || "${status}" -ne 0 ]]; then
+ cleanup || cleanup_status=$?
fi
- if [[ "$expected_start" != unavailable ]]; then
- actual_start="$(ps -p "$pid" -o lstart= 2>/dev/null | sed 's/^ *//' || true)"
- if [[ -z "$actual_start" || "$actual_start" != "$expected_start" ]]; then
- echo "ERROR: refusing to kill PID $pid from $pidfile; its process-start signature changed" >&2
- return 1
- fi
+ if [[ "${status}" -eq 0 && "${cleanup_status}" -ne 0 ]]; then
+ status="${cleanup_status}"
fi
- kill "$pid" 2>/dev/null || true
- rm -f -- "$pidfile"
-}
-
-write_pidfile() {
- local pidfile="$1" pid="$2" process_start
- process_start="$(ps -p "$pid" -o lstart= 2>/dev/null | sed 's/^ *//' || true)"
- [ -n "$process_start" ] || process_start=unavailable
- printf '%s\nPROCESS_START=%s\n' "$pid" "$process_start" >"$pidfile"
+ exit "${status}"
}
start_pf() {
@@ -190,37 +211,62 @@ start_pf() {
local ports="$4"
local health_url="${5:-}"
local log="/tmp/pf-${SAFE_PROFILE}-${name}.log"
- local pidfile="/tmp/pf-${SAFE_PROFILE}-${name}.pid"
- local profile_pids_dir="$PROFILE_PID_ROOT/$SAFE_PROFILE/pids"
- local profile_pidfile="${profile_pids_dir}/${name}.pid"
+ local legacy_pidfile="/tmp/pf-${SAFE_PROFILE}-${name}.pid"
+ local profile_pidfile="${PROFILE_PIDS_DIR}/${name}.pid"
+ local local_port remote_port pid attempt
+
+ [[ "${ports}" =~ ^([0-9]{1,5}):([0-9]{1,5})$ ]] || {
+ echo "ERROR: invalid port mapping for ${name}: ${ports}" >&2
+ return 1
+ }
+ local_port="${BASH_REMATCH[1]}"
+ remote_port="${BASH_REMATCH[2]}"
+ [[ -n "${health_url}" ]] || {
+ echo "ERROR: ${name} has no required health endpoint" >&2
+ return 1
+ }
- mkdir -p "$profile_pids_dir"
- kill_owned_pidfile "$pidfile" "$service"
- kill_owned_pidfile "$profile_pidfile" "$service"
+ # Do not migrate or kill a live legacy /tmp record: it has no canonical
+ # worktree binding. A dead legacy record is unambiguous and can be pruned.
+ if [[ -e "${legacy_pidfile}" || -L "${legacy_pidfile}" ]]; then
+ pf_owner_cleanup_record "${legacy_pidfile}" "${PROFILE}" "${CONTEXT}" \
+ "${WORKTREE_ROOT}" "${namespace}" "${service}" "${local_port}" "${remote_port}"
+ fi
+ pf_owner_cleanup_record "${profile_pidfile}" "${PROFILE}" "${CONTEXT}" \
+ "${WORKTREE_ROOT}" "${namespace}" "${service}" "${local_port}" "${remote_port}"
nohup "${KC[@]}" -n "${namespace}" port-forward --address=127.0.0.1 "svc/${service}" "${ports}" >"${log}" 2>&1 </dev/null &
- write_pidfile "${pidfile}" "$!"
- write_pidfile "${profile_pidfile}" "$!"
- PIDS+=("$!")
- PIDFILES+=("$pidfile" "$profile_pidfile")
- PIDFILE_SERVICES+=("$service" "$service")
- echo " ${name}: pid=$(cat "${pidfile}") ns=${namespace} svc=${service} ports=${ports}"
- sleep 0.2
- if ! kill -0 "$!" 2>/dev/null; then
+ pid=$!
+ pf_owner_pause "${STARTUP_DELAY}"
+ if ! pf_owner_record_process "${profile_pidfile}" "${pid}" "${PROFILE}" \
+ "${CONTEXT}" "${WORKTREE_ROOT}" "${namespace}" "${service}" \
+ "${local_port}" "${remote_port}"; then
+ echo " ERROR: ${name} port-forward ownership could not be recorded" >&2
+ pf_owner_abort_child "${pid}" "${CONTEXT}" "${namespace}" "${service}" \
+ "${local_port}" "${remote_port}" || true
+ return 1
+ fi
+
+ PIDFILES+=("${profile_pidfile}")
+ PIDFILE_NAMESPACES+=("${namespace}")
+ PIDFILE_SERVICES+=("${service}")
+ PIDFILE_LOCAL_PORTS+=("${local_port}")
+ PIDFILE_REMOTE_PORTS+=("${remote_port}")
+ echo " ${name}: pid=${pid} ns=${namespace} svc=${service} ports=${ports}"
+ if ! kill -0 "${pid}" 2>/dev/null; then
echo " ERROR: ${name} port-forward failed to stay running" >&2
sed -n '1,80p' "${log}" >&2 || true
- exit 1
+ return 1
fi
- if [[ -n "${health_url}" ]]; then
- for _ in $(seq 1 20); do
- if curl -sf -m 2 "${health_url}" >/dev/null 2>&1; then
- return 0
- fi
- sleep 0.5
- done
- echo " WARN: ${name} did not become healthy yet (${health_url})"
- fi
+ for ((attempt = 0; attempt < HEALTH_ATTEMPTS; attempt += 1)); do
+ if curl -sf -m 2 "${health_url}" >/dev/null 2>&1; then
+ return 0
+ fi
+ pf_owner_pause "${HEALTH_DELAY}"
+ done
+ echo " ERROR: ${name} did not become healthy (${health_url})" >&2
+ return 1
}
start_optional_pf() {
@@ -229,12 +275,21 @@ start_optional_pf() {
local service="$3"
local ports="$4"
local health_url="${5:-}"
+ local namespace_resource service_resource
- if ! "${KC[@]}" get namespace "${namespace}" >/dev/null 2>&1; then
+ if ! namespace_resource="$("${KC[@]}" get namespace "${namespace}" --ignore-not-found -o name)"; then
+ echo " ERROR: ${name} could not inspect namespace ${namespace}" >&2
+ return 1
+ fi
+ if [[ -z "${namespace_resource}" ]]; then
echo " ${name}: skipped (namespace ${namespace} not present)"
return 0
fi
- if ! "${KC[@]}" -n "${namespace}" get svc "${service}" >/dev/null 2>&1; then
+ if ! service_resource="$("${KC[@]}" -n "${namespace}" get svc "${service}" --ignore-not-found -o name)"; then
+ echo " ERROR: ${name} could not inspect service ${namespace}/${service}" >&2
+ return 1
+ fi
+ if [[ -z "${service_resource}" ]]; then
echo " ${name}: skipped (service ${namespace}/${service} not present)"
return 0
fi
@@ -249,29 +304,76 @@ start_optional_deployment_pf() {
local service="$4"
local ports="$5"
local health_url="${6:-}"
+ local namespace_resource deployment_resource desired
- if ! "${KC[@]}" get namespace "${namespace}" >/dev/null 2>&1; then
+ if ! namespace_resource="$("${KC[@]}" get namespace "${namespace}" --ignore-not-found -o name)"; then
+ echo " ERROR: ${name} could not inspect namespace ${namespace}" >&2
+ return 1
+ fi
+ if [[ -z "${namespace_resource}" ]]; then
echo " ${name}: skipped (namespace ${namespace} not present)"
return 0
fi
- if ! "${KC[@]}" -n "${namespace}" get deploy "${deployment}" >/dev/null 2>&1; then
+ if ! deployment_resource="$("${KC[@]}" -n "${namespace}" get deploy "${deployment}" --ignore-not-found -o name)"; then
+ echo " ERROR: ${name} could not inspect deployment ${namespace}/${deployment}" >&2
+ return 1
+ fi
+ if [[ -z "${deployment_resource}" ]]; then
echo " ${name}: skipped (deployment ${namespace}/${deployment} not present)"
return 0
fi
- local desired
- desired="$("${KC[@]}" -n "${namespace}" get deploy "${deployment}" -o jsonpath='{.spec.replicas}' 2>/dev/null || true)"
- if [[ -z "${desired}" || "${desired}" == "0" ]]; then
- echo " ${name}: skipped (deployment ${namespace}/${deployment} scaled to ${desired:-0})"
+ if ! desired="$("${KC[@]}" -n "${namespace}" get deploy "${deployment}" -o jsonpath='{.spec.replicas}')"; then
+ echo " ERROR: ${name} could not read desired replicas for ${namespace}/${deployment}" >&2
+ return 1
+ fi
+ [[ "${desired}" =~ ^[0-9]+$ ]] || {
+ echo " ERROR: ${name} received invalid desired replicas for ${namespace}/${deployment}: ${desired:-<empty>}" >&2
+ return 1
+ }
+ if [[ "${desired}" == "0" ]]; then
+ echo " ${name}: skipped (deployment ${namespace}/${deployment} scaled to 0)"
return 0
fi
start_optional_pf "${name}" "${namespace}" "${service}" "${ports}" "${health_url}"
}
+start_optional_mcp_host_pf() {
+ local namespace_resource service_resource
+ if ! namespace_resource="$("${KC[@]}" get namespace mcp-host --ignore-not-found -o name)"; then
+ echo ' ERROR: mcp-host could not inspect namespace mcp-host' >&2
+ return 1
+ fi
+ if [[ -z "${namespace_resource}" ]]; then
+ echo ' mcp-host: skipped (namespace mcp-host not present)'
+ return 0
+ fi
+ if ! service_resource="$("${KC[@]}" -n mcp-host get svc chatllm --ignore-not-found -o name)"; then
+ echo ' ERROR: mcp-host could not inspect service mcp-host/chatllm' >&2
+ return 1
+ fi
+ if [[ -n "${service_resource}" ]]; then
+ start_pf mcp-host mcp-host chatllm "${MCP_HOST_PORT}:8080" \
+ "http://127.0.0.1:${MCP_HOST_PORT}/v1/runtime/health"
+ return
+ fi
+ if ! service_resource="$("${KC[@]}" -n mcp-host get svc mcp-host --ignore-not-found -o name)"; then
+ echo ' ERROR: mcp-host could not inspect service mcp-host/mcp-host' >&2
+ return 1
+ fi
+ if [[ -n "${service_resource}" ]]; then
+ start_pf mcp-host mcp-host mcp-host "${MCP_HOST_PORT}:8080" \
+ "http://127.0.0.1:${MCP_HOST_PORT}/v1/runtime/health"
+ return
+ fi
+ echo ' mcp-host: skipped (service mcp-host/chatllm or mcp-host/mcp-host not present)'
+}
+
# Install the cleanup boundary before the first background process is started.
# A later start_pf failure must not strand forwards that were already launched.
-trap cleanup EXIT
-trap 'cleanup; exit 0' INT TERM
+trap on_exit EXIT
+trap 'exit 130' INT
+trap 'exit 143' TERM
echo "=== Starting gate port-forwards (${PROFILE}) ==="
start_pf control-ui control-plane control-ui "${CONTROL_UI_PORT}:3000" "http://127.0.0.1:${CONTROL_UI_PORT}"
@@ -281,12 +383,7 @@ start_pf external-rest-api profiles external-rest-api "${EXTERNAL_REST_API_PORT}
start_pf rpc-proxy rpc-proxy rpc-proxy "${RPC_PROXY_PORT}:8094" "http://127.0.0.1:${RPC_PROXY_PORT}/health"
start_optional_pf registry-api registry registry-api "${REGISTRY_API_PORT}:8085" "http://127.0.0.1:${REGISTRY_API_PORT}/health"
start_optional_deployment_pf workflow-approval-request-reader channels clerum-workflow-approval-request-reader workflow-approval-request-reader "${WORKFLOW_APPROVAL_READER_PORT}:8098" "http://127.0.0.1:${WORKFLOW_APPROVAL_READER_PORT}/health"
-
-if "${KC[@]}" get svc chatllm -n mcp-host >/dev/null 2>&1; then
- start_pf mcp-host mcp-host chatllm "${MCP_HOST_PORT}:8080" "http://127.0.0.1:${MCP_HOST_PORT}/v1/runtime/health"
-elif "${KC[@]}" get svc mcp-host -n mcp-host >/dev/null 2>&1; then
- start_pf mcp-host mcp-host mcp-host "${MCP_HOST_PORT}:8080" "http://127.0.0.1:${MCP_HOST_PORT}/v1/runtime/health"
-fi
+start_optional_mcp_host_pf
STARTUP_COMPLETE=true
echo "=== Port-forwards refreshed ==="
diff --git a/scripts/minikube/port-forward-owner.sh b/scripts/minikube/port-forward-owner.sh
new file mode 100755
index 000000000..ddaed49a6
--- /dev/null
+++ b/scripts/minikube/port-forward-owner.sh
@@ -0,0 +1,439 @@
+#!/usr/bin/env bash
+# Exact ownership records and fail-closed cleanup for kubectl port-forwards.
+# This file is a library; callers retain lifecycle and health-check policy.
+
+PF_OWNER_RECORD_VERSION=1
+PF_OWNER_ADDRESS=127.0.0.1
+
+pf_owner_error() {
+ printf 'PORT_FORWARD_OWNERSHIP_ERROR: %s\n' "$*" >&2
+ return 1
+}
+
+pf_owner_validate_pid() {
+ [[ "${1:-}" =~ ^[1-9][0-9]*$ ]]
+}
+
+pf_owner_validate_port() {
+ local port="${1:-}"
+ [[ "${port}" =~ ^[0-9]{1,5}$ ]] || return 1
+ (( 10#${port} >= 1 && 10#${port} <= 65535 ))
+}
+
+pf_owner_validate_binding() {
+ local profile="${1:-}" context="${2:-}" worktree="${3:-}"
+ local namespace="${4:-}" service="${5:-}" local_port="${6:-}"
+ local remote_port="${7:-}" canonical_worktree
+
+ [[ "${profile}" =~ ^[A-Za-z0-9][A-Za-z0-9_.-]*$ ]] ||
+ pf_owner_error "invalid profile binding: ${profile:-<empty>}" || return 1
+ [[ "${context}" =~ ^[A-Za-z0-9][A-Za-z0-9._:@/-]*$ ]] ||
+ pf_owner_error "invalid context binding: ${context:-<empty>}" || return 1
+ [[ "${namespace}" =~ ^[a-z0-9]([a-z0-9.-]*[a-z0-9])?$ ]] ||
+ pf_owner_error "invalid namespace binding: ${namespace:-<empty>}" || return 1
+ [[ "${service}" =~ ^[a-z0-9]([a-z0-9.-]*[a-z0-9])?$ ]] ||
+ pf_owner_error "invalid service binding: ${service:-<empty>}" || return 1
+ pf_owner_validate_port "${local_port}" ||
+ pf_owner_error "invalid local port binding: ${local_port:-<empty>}" || return 1
+ pf_owner_validate_port "${remote_port}" ||
+ pf_owner_error "invalid remote port binding: ${remote_port:-<empty>}" || return 1
+ [[ -n "${worktree}" && -d "${worktree}" ]] ||
+ pf_owner_error "worktree binding is missing: ${worktree:-<empty>}" || return 1
+ canonical_worktree="$(cd -- "${worktree}" && pwd -P)" ||
+ pf_owner_error "cannot canonicalize worktree binding: ${worktree}" || return 1
+ [[ "${worktree}" == "${canonical_worktree}" ]] ||
+ pf_owner_error "worktree binding is not canonical: ${worktree}" || return 1
+ [[ "${worktree}" != *$'\n'* && "${worktree}" != *$'\r'* && "${worktree}" != *$'\t'* ]] ||
+ pf_owner_error 'worktree binding contains a control character' || return 1
+}
+
+pf_owner_validate_process_start() {
+ local process_start="${1:-}"
+ [[ -n "${process_start}" && "${process_start}" != unavailable ]] || return 1
+ [[ "${process_start}" != *$'\n'* && "${process_start}" != *$'\r'* &&
+ "${process_start}" != *$'\t'* ]]
+}
+
+pf_owner_process_state() {
+ local pid="$1" observed process_status status
+ if kill -0 "${pid}" 2>/dev/null; then
+ # A terminated child can remain as a zombie until its owning shell reaps
+ # it. Treat that state as safe-to-reap instead of calling blocking `wait`
+ # while an actually live process may still be ignoring TERM.
+ process_status="$(LC_ALL=C ps -ww -p "${pid}" -o stat= 2>/dev/null || true)"
+ process_status="$(printf '%s' "${process_status}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
+ case "${process_status}" in
+ Z*) printf 'dead\n'; return 0 ;;
+ esac
+ printf 'live\n'
+ return 0
+ fi
+
+ status=0
+ observed="$(LC_ALL=C ps -ww -p "${pid}" -o pid= 2>/dev/null)" || status=$?
+ observed="$(printf '%s' "${observed}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
+ if [[ "${observed}" == "${pid}" ]]; then
+ printf 'live\n'
+ elif [[ "${status}" -eq 1 && -z "${observed}" ]]; then
+ printf 'dead\n'
+ else
+ printf 'unknown\n'
+ fi
+}
+
+pf_owner_process_start() {
+ local pid="$1" process_start
+ process_start="$(LC_ALL=C TZ=UTC0 ps -ww -p "${pid}" -o lstart= 2>/dev/null)" || return 1
+ process_start="$(printf '%s' "${process_start}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
+ pf_owner_validate_process_start "${process_start}" || return 1
+ printf '%s\n' "${process_start}"
+}
+
+pf_owner_process_command() {
+ local pid="$1" command_line
+ command_line="$(LC_ALL=C ps -ww -p "${pid}" -o command= 2>/dev/null)" || return 1
+ command_line="$(printf '%s' "${command_line}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')"
+ [[ -n "${command_line}" && "${command_line}" != *$'\n'* && "${command_line}" != *$'\r'* ]] || return 1
+ printf '%s\n' "${command_line}"
+}
+
+pf_owner_command_matches() {
+ local command_line="$1" context="$2" namespace="$3" service="$4"
+ local local_port="$5" remote_port="$6"
+ local -a argv=()
+
+ read -r -a argv <<<"${command_line}"
+ (( ${#argv[@]} == 8 )) || return 1
+ [[ "${argv[0]##*/}" == kubectl &&
+ "${argv[1]}" == "--context=${context}" &&
+ "${argv[2]}" == -n &&
+ "${argv[3]}" == "${namespace}" &&
+ "${argv[4]}" == port-forward &&
+ "${argv[5]}" == "--address=${PF_OWNER_ADDRESS}" &&
+ "${argv[6]}" == "svc/${service}" &&
+ "${argv[7]}" == "${local_port}:${remote_port}" ]]
+}
+
+pf_owner_signal_process() {
+ local pid="$1" signal="$2"
+ kill "-${signal}" -- "${pid}"
+}
+
+pf_owner_reap_process() {
+ wait "$1" 2>/dev/null || true
+}
+
+pf_owner_pause() {
+ sleep "$1"
+}
+
+pf_owner_validate_termination_policy() {
+ local attempts="$1" delay="$2"
+ [[ "${attempts}" =~ ^[1-9][0-9]*$ ]] && (( 10#${attempts} <= 200 )) ||
+ pf_owner_error 'termination attempts must be an integer from 1 to 200' || return 1
+ [[ "${delay}" =~ ^[0-9]+([.][0-9]+)?$ ]] ||
+ pf_owner_error 'termination delay must be a number from 0 to 1 second' || return 1
+ awk -v delay="${delay}" 'BEGIN { exit !(delay >= 0 && delay <= 1) }' ||
+ pf_owner_error 'termination delay must be a number from 0 to 1 second' || return 1
+}
+
+pf_owner_write_record_atomic() {
+ local pidfile="$1" pid="$2" process_start="$3" profile="$4"
+ local context="$5" worktree="$6" namespace="$7" service="$8"
+ local local_port="$9" remote_port="${10}"
+ local pid_dir tmp_file
+
+ pf_owner_validate_pid "${pid}" || pf_owner_error "invalid PID: ${pid:-<empty>}" || return 1
+ pf_owner_validate_process_start "${process_start}" ||
+ pf_owner_error "process-start identity is unavailable for PID ${pid}" || return 1
+ pf_owner_validate_binding "${profile}" "${context}" "${worktree}" \
+ "${namespace}" "${service}" "${local_port}" "${remote_port}" || return 1
+
+ pid_dir="$(dirname -- "${pidfile}")"
+ [[ -d "${pid_dir}" && ! -L "${pid_dir}" ]] ||
+ pf_owner_error "pid directory is missing or unsafe: ${pid_dir}" || return 1
+ [[ ! -L "${pidfile}" ]] ||
+ pf_owner_error "refusing to replace a symlinked pidfile: ${pidfile}" || return 1
+
+ tmp_file="$(mktemp "${pidfile}.tmp.XXXXXX")" || {
+ pf_owner_error "cannot create an atomic pidfile beside ${pidfile}"
+ return 1
+ }
+ if ! chmod 600 "${tmp_file}" || ! {
+ printf '%s\n' "${pid}"
+ printf 'PORT_FORWARD_OWNER_VERSION=%s\n' "${PF_OWNER_RECORD_VERSION}"
+ printf 'PID=%s\n' "${pid}"
+ printf 'PROCESS_START=%s\n' "${process_start}"
+ printf 'PROFILE=%s\n' "${profile}"
+ printf 'CONTEXT=%s\n' "${context}"
+ printf 'WORKTREE=%s\n' "${worktree}"
+ printf 'NAMESPACE=%s\n' "${namespace}"
+ printf 'SERVICE=%s\n' "${service}"
+ printf 'LOCAL_PORT=%s\n' "${local_port}"
+ printf 'REMOTE_PORT=%s\n' "${remote_port}"
+ printf 'ADDRESS=%s\n' "${PF_OWNER_ADDRESS}"
+ } >"${tmp_file}"; then
+ rm -f -- "${tmp_file}"
+ pf_owner_error "cannot write ownership record: ${pidfile}"
+ return 1
+ fi
+ # Publish with an atomic, no-clobber hard link. A concurrent owner that
+ # appeared after cleanup must make this launch fail; replacing its record
+ # would orphan a live process and destroy the evidence needed to stop safely.
+ if ! ln "${tmp_file}" "${pidfile}" 2>/dev/null; then
+ rm -f -- "${tmp_file}"
+ pf_owner_error "cannot publish ownership record without replacing an existing owner: ${pidfile}"
+ return 1
+ fi
+ if ! rm -f -- "${tmp_file}"; then
+ printf 'PORT_FORWARD_OWNERSHIP_WARN: canonical record is valid, but temporary hard link remains: %s\n' \
+ "${tmp_file}" >&2
+ fi
+}
+
+pf_owner_reset_record() {
+ PF_OWNER_RECORD_FIRST_PID=''
+ PF_OWNER_RECORD_VERSION_VALUE=''
+ PF_OWNER_RECORD_PID=''
+ PF_OWNER_RECORD_START=''
+ PF_OWNER_RECORD_PROFILE=''
+ PF_OWNER_RECORD_CONTEXT=''
+ PF_OWNER_RECORD_WORKTREE=''
+ PF_OWNER_RECORD_NAMESPACE=''
+ PF_OWNER_RECORD_SERVICE=''
+ PF_OWNER_RECORD_LOCAL_PORT=''
+ PF_OWNER_RECORD_REMOTE_PORT=''
+ PF_OWNER_RECORD_ADDRESS=''
+}
+
+pf_owner_read_record() {
+ local pidfile="$1" line key value seen='|'
+ pf_owner_reset_record
+ [[ -f "${pidfile}" && -r "${pidfile}" && ! -L "${pidfile}" ]] ||
+ pf_owner_error "pidfile is missing, unreadable, or a symlink: ${pidfile}" || return 1
+
+ IFS= read -r PF_OWNER_RECORD_FIRST_PID <"${pidfile}" ||
+ pf_owner_error "pidfile is empty: ${pidfile}" || return 1
+ pf_owner_validate_pid "${PF_OWNER_RECORD_FIRST_PID}" ||
+ pf_owner_error "pidfile has no numeric first-line PID: ${pidfile}" || return 1
+
+ while IFS= read -r line || [[ -n "${line}" ]]; do
+ [[ "${line}" == *=* && "${line}" != *$'\r'* && "${line}" != *$'\t'* ]] ||
+ pf_owner_error "pidfile has a malformed structured line: ${pidfile}" || return 1
+ key="${line%%=*}"
+ value="${line#*=}"
+ case "${seen}" in
+ *"|${key}|"*) pf_owner_error "pidfile repeats ${key}: ${pidfile}"; return 1 ;;
+ esac
+ seen="${seen}${key}|"
+ case "${key}" in
+ PORT_FORWARD_OWNER_VERSION) PF_OWNER_RECORD_VERSION_VALUE="${value}" ;;
+ PID) PF_OWNER_RECORD_PID="${value}" ;;
+ PROCESS_START) PF_OWNER_RECORD_START="${value}" ;;
+ PROFILE) PF_OWNER_RECORD_PROFILE="${value}" ;;
+ CONTEXT) PF_OWNER_RECORD_CONTEXT="${value}" ;;
+ WORKTREE) PF_OWNER_RECORD_WORKTREE="${value}" ;;
+ NAMESPACE) PF_OWNER_RECORD_NAMESPACE="${value}" ;;
+ SERVICE) PF_OWNER_RECORD_SERVICE="${value}" ;;
+ LOCAL_PORT) PF_OWNER_RECORD_LOCAL_PORT="${value}" ;;
+ REMOTE_PORT) PF_OWNER_RECORD_REMOTE_PORT="${value}" ;;
+ ADDRESS) PF_OWNER_RECORD_ADDRESS="${value}" ;;
+ *) pf_owner_error "pidfile contains unsupported key ${key}: ${pidfile}"; return 1 ;;
+ esac
+ done < <(sed -n '2,$p' "${pidfile}")
+
+ [[ "${PF_OWNER_RECORD_VERSION_VALUE}" == "${PF_OWNER_RECORD_VERSION}" ]] ||
+ pf_owner_error "pidfile has an unsupported record version: ${pidfile}" || return 1
+ [[ "${PF_OWNER_RECORD_PID}" == "${PF_OWNER_RECORD_FIRST_PID}" ]] ||
+ pf_owner_error "pidfile PID fields disagree: ${pidfile}" || return 1
+ pf_owner_validate_process_start "${PF_OWNER_RECORD_START}" ||
+ pf_owner_error "pidfile has no exact process-start identity: ${pidfile}" || return 1
+ [[ "${PF_OWNER_RECORD_ADDRESS}" == "${PF_OWNER_ADDRESS}" ]] ||
+ pf_owner_error "pidfile address binding is not ${PF_OWNER_ADDRESS}: ${pidfile}" || return 1
+ pf_owner_validate_binding "${PF_OWNER_RECORD_PROFILE}" "${PF_OWNER_RECORD_CONTEXT}" \
+ "${PF_OWNER_RECORD_WORKTREE}" "${PF_OWNER_RECORD_NAMESPACE}" \
+ "${PF_OWNER_RECORD_SERVICE}" "${PF_OWNER_RECORD_LOCAL_PORT}" \
+ "${PF_OWNER_RECORD_REMOTE_PORT}" || return 1
+}
+
+pf_owner_record_matches() {
+ local profile="$1" context="$2" worktree="$3" namespace="$4"
+ local service="$5" local_port="$6" remote_port="$7"
+ [[ "${PF_OWNER_RECORD_PROFILE}" == "${profile}" &&
+ "${PF_OWNER_RECORD_CONTEXT}" == "${context}" &&
+ "${PF_OWNER_RECORD_WORKTREE}" == "${worktree}" &&
+ "${PF_OWNER_RECORD_NAMESPACE}" == "${namespace}" &&
+ "${PF_OWNER_RECORD_SERVICE}" == "${service}" &&
+ "${PF_OWNER_RECORD_LOCAL_PORT}" == "${local_port}" &&
+ "${PF_OWNER_RECORD_REMOTE_PORT}" == "${remote_port}" ]]
+}
+
+pf_owner_remove_dead_record() {
+ local pidfile="$1"
+ [[ ! -L "${pidfile}" ]] || pf_owner_error "refusing to remove symlinked pidfile: ${pidfile}" || return 1
+ rm -f -- "${pidfile}"
+}
+
+pf_owner_record_process() {
+ local pidfile="$1" pid="$2" profile="$3" context="$4" worktree="$5"
+ local namespace="$6" service="$7" local_port="$8" remote_port="$9"
+ local state start_before start_after command_line
+
+ pf_owner_validate_binding "${profile}" "${context}" "${worktree}" \
+ "${namespace}" "${service}" "${local_port}" "${remote_port}" || return 1
+ pf_owner_validate_pid "${pid}" || pf_owner_error "invalid PID: ${pid:-<empty>}" || return 1
+ state="$(pf_owner_process_state "${pid}")"
+ [[ "${state}" == live ]] ||
+ pf_owner_error "PID ${pid} is not live while recording ownership (${state})" || return 1
+ start_before="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot capture process-start identity for PID ${pid}" || return 1
+ command_line="$(pf_owner_process_command "${pid}")" ||
+ pf_owner_error "cannot inspect kubectl argv for PID ${pid}" || return 1
+ pf_owner_command_matches "${command_line}" "${context}" "${namespace}" \
+ "${service}" "${local_port}" "${remote_port}" ||
+ pf_owner_error "PID ${pid} does not have the exact expected kubectl argv" || return 1
+ start_after="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot recheck process-start identity for PID ${pid}" || return 1
+ [[ "${start_before}" == "${start_after}" ]] ||
+ pf_owner_error "PID ${pid} changed identity while ownership was recorded" || return 1
+ pf_owner_write_record_atomic "${pidfile}" "${pid}" "${start_before}" \
+ "${profile}" "${context}" "${worktree}" "${namespace}" "${service}" \
+ "${local_port}" "${remote_port}"
+}
+
+pf_owner_cleanup_record() {
+ local pidfile="$1" profile="$2" context="$3" worktree="$4"
+ local namespace="$5" service="$6" local_port="$7" remote_port="$8"
+ local pid state actual_start command_line attempts delay index
+
+ pf_owner_validate_binding "${profile}" "${context}" "${worktree}" \
+ "${namespace}" "${service}" "${local_port}" "${remote_port}" || return 1
+ [[ -e "${pidfile}" || -L "${pidfile}" ]] || return 0
+ [[ ! -L "${pidfile}" ]] ||
+ pf_owner_error "refusing to inspect or remove a symlinked pidfile: ${pidfile}" || return 1
+ IFS= read -r pid <"${pidfile}" ||
+ pf_owner_error "refusing to remove an empty pidfile: ${pidfile}" || return 1
+ pf_owner_validate_pid "${pid}" ||
+ pf_owner_error "refusing to remove a pidfile without a numeric PID: ${pidfile}" || return 1
+
+ state="$(pf_owner_process_state "${pid}")"
+ case "${state}" in
+ dead)
+ pf_owner_reap_process "${pid}"
+ pf_owner_remove_dead_record "${pidfile}"
+ return
+ ;;
+ live) ;;
+ *) pf_owner_error "cannot establish whether PID ${pid} is live; leaving ${pidfile}"; return 1 ;;
+ esac
+
+ pf_owner_read_record "${pidfile}" || return 1
+ pf_owner_record_matches "${profile}" "${context}" "${worktree}" \
+ "${namespace}" "${service}" "${local_port}" "${remote_port}" ||
+ pf_owner_error "live PID ${pid} belongs to a different profile, context, worktree, service, or port binding" || return 1
+
+ actual_start="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot re-read process-start identity for live PID ${pid}" || return 1
+ [[ "${actual_start}" == "${PF_OWNER_RECORD_START}" ]] ||
+ pf_owner_error "live PID ${pid} has a different process-start identity" || return 1
+ command_line="$(pf_owner_process_command "${pid}")" ||
+ pf_owner_error "cannot inspect kubectl argv for live PID ${pid}" || return 1
+ pf_owner_command_matches "${command_line}" "${context}" "${namespace}" \
+ "${service}" "${local_port}" "${remote_port}" ||
+ pf_owner_error "live PID ${pid} does not have the exact recorded kubectl argv" || return 1
+ actual_start="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot recheck process-start identity for live PID ${pid}" || return 1
+ [[ "${actual_start}" == "${PF_OWNER_RECORD_START}" ]] ||
+ pf_owner_error "live PID ${pid} changed identity before cleanup" || return 1
+
+ attempts="${PF_OWNER_TERMINATE_ATTEMPTS:-20}"
+ delay="${PF_OWNER_TERMINATE_DELAY:-0.05}"
+ pf_owner_validate_termination_policy "${attempts}" "${delay}" || return 1
+
+ if ! pf_owner_signal_process "${pid}" TERM 2>/dev/null; then
+ state="$(pf_owner_process_state "${pid}")"
+ [[ "${state}" == dead ]] ||
+ pf_owner_error "TERM failed for exact owned PID ${pid}; leaving ${pidfile}" || return 1
+ fi
+ for ((index = 0; index < attempts; index += 1)); do
+ state="$(pf_owner_process_state "${pid}")"
+ case "${state}" in
+ dead)
+ pf_owner_reap_process "${pid}"
+ pf_owner_remove_dead_record "${pidfile}"
+ return
+ ;;
+ live)
+ actual_start="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot recheck live PID ${pid} after TERM; leaving ${pidfile}" || return 1
+ [[ "${actual_start}" == "${PF_OWNER_RECORD_START}" ]] ||
+ pf_owner_error "PID ${pid} was reused after TERM; leaving ${pidfile}" || return 1
+ ;;
+ *)
+ pf_owner_error "process state became ambiguous after TERM for PID ${pid}; leaving ${pidfile}"
+ return 1
+ ;;
+ esac
+ pf_owner_pause "${delay}"
+ done
+ pf_owner_error "exact owned PID ${pid} did not exit after TERM; leaving ${pidfile}"
+}
+
+pf_owner_abort_child() {
+ local pid="$1" context="$2" namespace="$3" service="$4"
+ local local_port="$5" remote_port="$6" job_pid command_line state found=false
+ local process_start actual_start attempts delay index signal
+
+ pf_owner_validate_pid "${pid}" || return 1
+ while IFS= read -r job_pid; do
+ if [[ "${job_pid}" == "${pid}" ]]; then
+ found=true
+ break
+ fi
+ done < <(jobs -pr)
+ if [[ "${found}" != true ]]; then
+ state="$(pf_owner_process_state "${pid}")"
+ [[ "${state}" == dead ]] && return 0
+ pf_owner_error "unrecorded PID ${pid} is not a current shell child; refusing to signal it"
+ return 1
+ fi
+
+ command_line="$(pf_owner_process_command "${pid}")" ||
+ pf_owner_error "cannot inspect unrecorded child PID ${pid}; refusing to signal it" || return 1
+ pf_owner_command_matches "${command_line}" "${context}" "${namespace}" \
+ "${service}" "${local_port}" "${remote_port}" ||
+ pf_owner_error "unrecorded child PID ${pid} has unexpected argv; refusing to signal it" || return 1
+ process_start="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot capture unrecorded child PID ${pid} identity; refusing to signal it" || return 1
+
+ attempts="${PF_OWNER_TERMINATE_ATTEMPTS:-20}"
+ delay="${PF_OWNER_TERMINATE_DELAY:-0.05}"
+ pf_owner_validate_termination_policy "${attempts}" "${delay}" || return 1
+
+ pf_owner_signal_process "${pid}" TERM 2>/dev/null || true
+ for signal in TERM KILL; do
+ for ((index = 0; index < attempts; index += 1)); do
+ state="$(pf_owner_process_state "${pid}")"
+ if [[ "${state}" == dead ]]; then
+ pf_owner_reap_process "${pid}"
+ return 0
+ fi
+ [[ "${state}" == live ]] ||
+ pf_owner_error "unrecorded child PID ${pid} state became ambiguous after ${signal}" || return 1
+ actual_start="$(pf_owner_process_start "${pid}")" ||
+ pf_owner_error "cannot recheck unrecorded child PID ${pid} after ${signal}" || return 1
+ [[ "${actual_start}" == "${process_start}" ]] ||
+ pf_owner_error "unrecorded child PID ${pid} was reused after ${signal}" || return 1
+ pf_owner_pause "${delay}"
+ done
+ if [[ "${signal}" == TERM ]]; then
+ command_line="$(pf_owner_process_command "${pid}")" ||
+ pf_owner_error "cannot recheck unrecorded child PID ${pid} before KILL" || return 1
+ pf_owner_command_matches "${command_line}" "${context}" "${namespace}" \
+ "${service}" "${local_port}" "${remote_port}" ||
+ pf_owner_error "unrecorded child PID ${pid} changed argv before KILL" || return 1
+ pf_owner_signal_process "${pid}" KILL 2>/dev/null || true
+ fi
+ done
+ pf_owner_error "unrecorded child PID ${pid} did not exit after bounded TERM/KILL cleanup"
+}
diff --git a/scripts/minikube/pre-gate-incremental.sh b/scripts/minikube/pre-gate-incremental.sh
index a8e20094b..7c2f7c11a 100755
--- a/scripts/minikube/pre-gate-incremental.sh
+++ b/scripts/minikube/pre-gate-incremental.sh
@@ -25,6 +25,36 @@ INCREMENTAL_UNMAPPED=()
INCREMENTAL_REPULL_ALL=false
INCREMENTAL_SHADOWED=()
INCREMENTAL_DOCKER_ENV_APPLIED=false
+INCREMENTAL_DEADLINE_RUNNER="${INCREMENTAL_DEADLINE_RUNNER:-${PROJECT_DIR}/scripts/minikube/run-with-deadline.mjs}"
+INCREMENTAL_RUNTIME_TIMEOUT_SECONDS="${INCREMENTAL_RUNTIME_TIMEOUT_SECONDS:-30}"
+INCREMENTAL_IMAGE_LOAD_TIMEOUT_SECONDS="${INCREMENTAL_IMAGE_LOAD_TIMEOUT_SECONDS:-1800}"
+
+incremental_validate_deadline() {
+ local name="$1" value="$2" maximum="$3"
+ if ! [[ "${value}" =~ ^[1-9][0-9]*$ ]] || (( 10#${value} > maximum )); then
+ log "ERROR: ${name} must be an integer from 1 to ${maximum}" >&2
+ return 1
+ fi
+}
+
+incremental_validate_deadline INCREMENTAL_RUNTIME_TIMEOUT_SECONDS \
+ "${INCREMENTAL_RUNTIME_TIMEOUT_SECONDS}" 300 || exit 1
+incremental_validate_deadline INCREMENTAL_IMAGE_LOAD_TIMEOUT_SECONDS \
+ "${INCREMENTAL_IMAGE_LOAD_TIMEOUT_SECONDS}" 3600 || exit 1
+[[ -f "${INCREMENTAL_DEADLINE_RUNNER}" ]] || {
+ log "ERROR: bounded runtime helper is missing: ${INCREMENTAL_DEADLINE_RUNNER}" >&2
+ exit 1
+}
+
+incremental_run_with_deadline() {
+ local label="$1" timeout_seconds="$2"
+ shift 2
+ node "${INCREMENTAL_DEADLINE_RUNNER}" \
+ --timeout-seconds "${timeout_seconds}" \
+ --heartbeat-seconds "${MINIKUBE_DOCKER_HEARTBEAT_SECONDS:-20}" \
+ --kill-grace-seconds "${MINIKUBE_DOCKER_KILL_GRACE_SECONDS:-5}" \
+ --label "${label}" -- "$@"
+}
incremental_add_target() {
local selector="$1" namespace="$2" deployment="$3" target
@@ -77,7 +107,9 @@ incremental_use_minikube_docker() {
INCREMENTAL_DOCKER_ENV_APPLIED=true
return 0
fi
- if ! env_script="$(minikube -p "${PROFILE}" docker-env)"; then
+ if ! env_script="$(incremental_run_with_deadline incremental-docker-env \
+ "${INCREMENTAL_RUNTIME_TIMEOUT_SECONDS}" \
+ minikube -p "${PROFILE}" docker-env --shell bash)"; then
# >&2 because this runs inside a command substitution on the baseline
# path, where anything on stdout would be captured AS the baseline.
log "ERROR: could not point Docker at minikube's daemon; a shadow build would land in the wrong daemon" >&2
@@ -101,7 +133,9 @@ incremental_release_baseline_commit() {
incremental_use_minikube_docker
for probe in control-api workflow-recipes external-rest-api; do
ref="ghcr.io/evenfire-ai/${probe}:${IMAGE_TAG}"
- revision="$(docker inspect --format='{{index .Config.Labels "org.opencontainers.image.revision"}}' "${ref}" 2>&1)"
+ revision="$(incremental_run_with_deadline incremental-image-inspect \
+ "${INCREMENTAL_RUNTIME_TIMEOUT_SECONDS}" \
+ docker inspect --format='{{index .Config.Labels "org.opencontainers.image.revision"}}' "${ref}")"
rc=$?
if [[ "${rc}" -ne 0 ]]; then
continue
@@ -324,12 +358,16 @@ incremental_shadow_selector() {
log " ${local_ref} has no published counterpart; the local build is what runs"
continue
fi
- if ! docker tag "${local_ref}" "${shadow_ref}"; then
+ if ! incremental_run_with_deadline incremental-image-tag \
+ "${INCREMENTAL_RUNTIME_TIMEOUT_SECONDS}" \
+ docker tag "${local_ref}" "${shadow_ref}"; then
log "ERROR: could not shadow ${shadow_ref} with ${local_ref}; the gate would test undeployed code"
exit 1
fi
if [[ "${MINIKUBE_MULTI_NODE:-false}" == "true" ]]; then
- minikube -p "${PROFILE}" image load "${shadow_ref}" >/dev/null
+ incremental_run_with_deadline incremental-image-load \
+ "${INCREMENTAL_IMAGE_LOAD_TIMEOUT_SECONDS}" \
+ minikube -p "${PROFILE}" image load "${shadow_ref}" >/dev/null
fi
INCREMENTAL_SHADOWED+=("${shadow_ref} <- ${local_ref}")
shadowed=$((shadowed + 1))
diff --git a/scripts/minikube/pre-gate-sync.sh b/scripts/minikube/pre-gate-sync.sh
index 7c5e55cd0..40908fac9 100755
--- a/scripts/minikube/pre-gate-sync.sh
+++ b/scripts/minikube/pre-gate-sync.sh
@@ -2,13 +2,20 @@
# Enforces the "sync to minikube before every gate" rule from the platform E2E plan.
set -euo pipefail
-set +u
-PRE_GATE_SYNC_CONFIG_ONLY="$PRE_GATE_SYNC_CONFIG_ONLY"
-set -u
+PRE_GATE_SYNC_CONFIG_ONLY="${PRE_GATE_SYNC_CONFIG_ONLY:-}"
+T2_SETUP_HANDOFF_EXPECTED="${T2_SETUP_HANDOFF_EXPECTED:-}"
+T2_SETUP_HANDOFF_TRANSITION="${T2_SETUP_HANDOFF_TRANSITION:-}"
+T2_SETUP_HANDOFF_ROOT="${T2_SETUP_HANDOFF_ROOT:-}"
+T2_SETUP_HANDOFF_TTL_SECONDS="${T2_SETUP_HANDOFF_TTL_SECONDS:-}"
if [ -z "$PRE_GATE_SYNC_CONFIG_ONLY" ]; then PRE_GATE_SYNC_CONFIG_ONLY=false; fi
+if [ -z "$T2_SETUP_HANDOFF_EXPECTED" ]; then T2_SETUP_HANDOFF_EXPECTED=false; fi
+if [ -z "$T2_SETUP_HANDOFF_TTL_SECONDS" ]; then T2_SETUP_HANDOFF_TTL_SECONDS=300; fi
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+if [ -z "$T2_SETUP_HANDOFF_ROOT" ]; then
+ T2_SETUP_HANDOFF_ROOT="${PROJECT_DIR}/.local-notes/infra/t2-setup-handoffs"
+fi
PROFILE="${MINIKUBE_PROFILE:-clerum-test}"
KC="kubectl --context=${PROFILE}"
# shellcheck source=t2-worktree-id.sh
@@ -109,6 +116,13 @@ fi
source "${SCRIPT_DIR}/pre-gate-runtime.sh"
# shellcheck source=scripts/minikube/pre-gate-incremental.sh
source "${SCRIPT_DIR}/pre-gate-incremental.sh"
+# shellcheck source=scripts/minikube/t2-setup-handoff.sh
+source "${SCRIPT_DIR}/t2-setup-handoff.sh"
+
+case "${T2_SETUP_HANDOFF_EXPECTED}" in
+ true|false) ;;
+ *) log "ERROR: T2_SETUP_HANDOFF_EXPECTED must be true or false"; exit 1 ;;
+esac
# A database/schema migration must not race the eager workflow reconciler. Keep
# the existing replica count so a failed gate restores the branch-owned
@@ -645,6 +659,32 @@ if [[ "${FORCE_RESTART}" == "true" ]] || has_changed infra "${infra_fingerprint}
infra_changed=true
fi
+setup_handoff_result=not-expected
+if [[ "${T2_SETUP_HANDOFF_EXPECTED}" == "true" ]]; then
+ if [[ "${GATE_NAME}" != "minikube-t2" || "${T2_SKIP_LOCK}" != "true" ||
+ "${IMAGE_SOURCE}" != "local" ]]; then
+ log "ERROR: setup-complete handoff requires the inherited minikube-t2 lease and local image mode"
+ exit 1
+ fi
+ if T2_PROJECT_DIR="${T2_PROJECT_DIR}" T2_WORKTREE_ID="${T2_WORKTREE_ID}" \
+ T2_RUN_ID="${T2_RUN_ID}" T2_PROFILE="${T2_PROFILE}" T2_CONTEXT="${T2_CONTEXT}" \
+ T2_BRANCH="${T2_BRANCH}" T2_HEAD="${T2_HEAD}" T2_SKIP_LOCK="${T2_SKIP_LOCK}" \
+ T2_LOCK_KEY="${T2_LOCK_KEY}" T2_LOCK_TOKEN="${T2_LOCK_TOKEN}" \
+ T2_IMAGE_MANIFEST="${T2_IMAGE_MANIFEST}" \
+ T2_SETUP_HANDOFF_ROOT="${T2_SETUP_HANDOFF_ROOT}" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="${T2_SETUP_HANDOFF_TTL_SECONDS}" \
+ T2_SETUP_HANDOFF_TRANSITION="${T2_SETUP_HANDOFF_TRANSITION}" \
+ bash "${SCRIPT_DIR}/t2-setup-handoff.sh" consume -- \
+ make --no-print-directory minikube-verify-images; then
+ setup_handoff_result=consumed
+ log "Consumed the verified one-shot setup-complete handoff"
+ else
+ setup_handoff_result=rejected
+ cluster_changed=true
+ log "Setup-complete handoff was not accepted; forcing the safe full reconcile"
+ fi
+fi
+
run_if_changed packages/workflow-runtime-core "npm test && npm run build"
ensure_artifact packages/workflow-runtime-core dist/index.js "npm run build"
# @clerum/network-policy-core is a file: dependency of workflow-recipes and
@@ -663,6 +703,15 @@ run_if_changed desktop-app "npm test"
if [[ "${cluster_changed}" == "true" ]]; then
incremental_plan
+ if [[ "${T2_SETUP_HANDOFF_EXPECTED}" == "true" ]]; then
+ if ! t2_setup_handoff_apply_plan_result "${setup_handoff_result}"; then
+ log "ERROR: setup-complete handoff reached an ambiguous plan state"
+ exit 1
+ fi
+ if [[ "${setup_handoff_result}" == consumed ]]; then
+ log "Reusing the attested images/deployment from the completed full setup"
+ fi
+ fi
log "Cluster sync plan before ${GATE_NAME}: images=$(incremental_target_summary), full-image-build=${INCREMENTAL_FULL_IMAGE_BUILD}, full-deployment=${INCREMENTAL_FULL_DEPLOYMENT}"
if [[ "${INCREMENTAL_FULL_IMAGE_BUILD}" == "true" ||
diff --git a/scripts/minikube/profile-owner.sh b/scripts/minikube/profile-owner.sh
new file mode 100755
index 000000000..537d5c444
--- /dev/null
+++ b/scripts/minikube/profile-owner.sh
@@ -0,0 +1,459 @@
+#!/usr/bin/env bash
+# Read-only resolver for branch-owned Minikube profile metadata.
+#
+# This file never creates, renames, adopts, or rewrites a profile. The ignored
+# branch-profile helper owns those mutations. This public boundary only derives
+# stable owner identity and validates/selects persisted metadata fail-closed.
+
+PROFILE_OWNER_SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+# shellcheck source=t2-worktree-id.sh
+. "${PROFILE_OWNER_SCRIPT_DIR}/t2-worktree-id.sh"
+
+PROFILE_OWNER_ERROR_CODE=""
+PROFILE_OWNER_ERROR_MESSAGE=""
+
+profile_owner_error() {
+ PROFILE_OWNER_ERROR_CODE="$1"
+ PROFILE_OWNER_ERROR_MESSAGE="$2"
+ return 1
+}
+
+profile_owner_print_error() {
+ printf '%s: %s\n' "${PROFILE_OWNER_ERROR_CODE:-PROFILE_OWNER_ERROR}" \
+ "${PROFILE_OWNER_ERROR_MESSAGE:-profile ownership validation failed}" >&2
+}
+
+profile_owner_canonical_path() {
+ local canonical_path
+ canonical_path="$(t2_canonical_worktree_path "${1:-}")" || return 1
+ [[ "${canonical_path}" != *$'\n'* && "${canonical_path}" != *$'\r'* && "${canonical_path}" != *$'\t'* ]] || return 1
+ printf '%s\n' "${canonical_path}"
+}
+
+profile_owner_slugify() {
+ printf '%s' "$1" |
+ LC_ALL=C tr '[:upper:]' '[:lower:]' |
+ LC_ALL=C sed -E 's/[^a-z0-9]+/-/g; s/^-+//; s/-+$//; s/-+/-/g' |
+ LC_ALL=C cut -c1-47
+}
+
+profile_owner_stable_profile() {
+ local repo_dir="${1:-}" branch="${2:-}" owner_id branch_slug
+ owner_id="$(t2_profile_owner_id "${repo_dir}" "${branch}")" || return 1
+ branch_slug="$(profile_owner_slugify "${branch}")"
+ [[ -n "${branch_slug}" ]] || branch_slug=branch
+ printf 'clerum-%s-%s\n' "${branch_slug}" "${owner_id:0:8}"
+}
+
+profile_owner_value() {
+ local file="$1" key="$2"
+ awk -F= -v wanted="${key}" '$1 == wanted { sub(/^[^=]*=/, ""); print; exit }' \
+ "${file}" 2>/dev/null || true
+}
+
+profile_owner_validate_record_syntax() {
+ local file="${1:-}" label="${2:-metadata}"
+ if [[ ! -f "${file}" || ! -r "${file}" || -L "${file}" ]]; then
+ profile_owner_error PROFILE_METADATA_MISSING "${label} is missing, unreadable, or a symlink: ${file}"
+ return 1
+ fi
+
+ local validation_error
+ validation_error="$(awk '
+ /^$/ { next }
+ {
+ if ($0 !~ /^[A-Z][A-Z0-9_]*=/) {
+ printf "line %d is not KEY=value", NR
+ exit 1
+ }
+ key = $0
+ sub(/=.*/, "", key)
+ if (seen[key]++) {
+ printf "duplicate key %s", key
+ exit 1
+ }
+ value = substr($0, index($0, "=") + 1)
+ if (value ~ /[\r\t]/) {
+ printf "key %s contains a control character", key
+ exit 1
+ }
+ }
+ ' "${file}" 2>&1)" || {
+ profile_owner_error PROFILE_METADATA_INVALID "${label} is malformed (${validation_error}): ${file}"
+ return 1
+ }
+}
+
+profile_owner_validate_branch() {
+ local branch="${1:-}"
+ if [[ -z "${branch}" || "${branch}" == *$'\n'* || "${branch}" == *$'\r'* || "${branch}" == *$'\t'* ]]; then
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'branch is empty or malformed'
+ return 1
+ fi
+}
+
+profile_owner_validate_profile_name() {
+ local profile="${1:-}"
+ if [[ ${#profile} -gt 63 || ! "${profile}" =~ ^[a-zA-Z0-9][a-zA-Z0-9._-]*$ ]]; then
+ profile_owner_error PROFILE_METADATA_INVALID 'profile name is not a safe local Minikube identifier'
+ return 1
+ fi
+ case "${profile}" in
+ *gke*|*prod*|*staging*|clerum-test|default|minikube)
+ profile_owner_error PROFILE_METADATA_INVALID "profile is shared or protected: ${profile}"
+ return 1 ;;
+ esac
+}
+
+profile_owner_validate_ports() {
+ local ports_env="${1:-}"
+ if [[ ! -f "${ports_env}" || ! -r "${ports_env}" || -L "${ports_env}" ]]; then
+ profile_owner_error PROFILE_PORTS_MISSING "persisted ports are missing, unreadable, or a symlink: ${ports_env}"
+ return 1
+ fi
+ if ! profile_owner_validate_record_syntax "${ports_env}" 'ports.env'; then
+ PROFILE_OWNER_ERROR_CODE=PROFILE_PORTS_INVALID
+ return 1
+ fi
+
+ local validation_error
+ validation_error="$(awk -F= '
+ function fail(message) { print message; exit 1 }
+ {
+ key = $1
+ value = substr($0, index($0, "=") + 1)
+ values[key] = value
+ }
+ END {
+ if (!("PORT_BASE" in values) || values["PORT_BASE"] !~ /^[0-9]+$/)
+ fail("PORT_BASE is missing or non-numeric")
+ if (values["PORT_BASE"] < 20000 || values["PORT_BASE"] > 38999)
+ fail("PORT_BASE is outside the branch-profile allocation range")
+
+ for (key in values) {
+ value = values[key]
+ if (key ~ /_PORT$/) {
+ if (value !~ /^[0-9]+$/ || value < 1024 || value > 65535)
+ fail(key " is not a valid local port")
+ if (port_owner[value] != "" && port_owner[value] != key)
+ fail(key " duplicates " port_owner[value])
+ port_owner[value] = key
+ }
+ if (key ~ /(_URL|_BASE_URL)$/) {
+ if (value !~ /^http:\/\/(127\.0\.0\.1|localhost|\[::1\]):[0-9]+\/?$/)
+ fail(key " is not a loopback HTTP URL")
+ url_port = value
+ sub(/^.*:/, "", url_port)
+ sub(/\/$/, "", url_port)
+ if (url_port < 1024 || url_port > 65535)
+ fail(key " uses an invalid local port")
+ port_key = key
+ sub(/_BASE_URL$/, "_PORT", port_key)
+ sub(/_URL$/, "_PORT", port_key)
+ if (!(port_key in values))
+ fail(key " has no matching " port_key)
+ if (values[port_key] != url_port)
+ fail(key " does not match " port_key)
+ }
+ }
+ }
+ ' "${ports_env}" 2>&1)" || {
+ profile_owner_error PROFILE_PORTS_INVALID "persisted ports are invalid (${validation_error}): ${ports_env}"
+ return 1
+ }
+}
+
+profile_owner_validate_selection() {
+ local profile_env="${1:-}" ports_env="${2:-}" expected_repo="${3:-}"
+ local expected_branch="${4:-}" expected_profile="${5:-}"
+ local canonical_repo metadata_repo metadata_canonical schema profile branch
+ local worktree_id owner_id created_head sha_short dirty actual_worktree actual_owner
+
+ canonical_repo="$(profile_owner_canonical_path "${expected_repo}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH "expected worktree is missing: ${expected_repo}"
+ return 1
+ }
+ profile_owner_validate_branch "${expected_branch}" || return 1
+ profile_owner_validate_record_syntax "${profile_env}" 'profile.env' || return 1
+
+ schema="$(profile_owner_value "${profile_env}" PROFILE_SCHEMA_VERSION)"
+ [[ -n "${schema}" ]] || schema=1
+ profile="$(profile_owner_value "${profile_env}" PROFILE)"
+ metadata_repo="$(profile_owner_value "${profile_env}" REPO_DIR)"
+ branch="$(profile_owner_value "${profile_env}" BRANCH)"
+ dirty="$(profile_owner_value "${profile_env}" DIRTY)"
+
+ [[ "${schema}" == 1 || "${schema}" == 2 ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID "unsupported PROFILE_SCHEMA_VERSION=${schema}"
+ return 1
+ }
+ [[ -n "${profile}" && -n "${metadata_repo}" && -n "${branch}" ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID 'profile metadata omits PROFILE, REPO_DIR, or BRANCH'
+ return 1
+ }
+ profile_owner_validate_profile_name "${profile}" || return 1
+ if [[ -n "${expected_profile}" && "${profile}" != "${expected_profile}" ]]; then
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'profile metadata names a different profile'
+ return 1
+ fi
+ metadata_canonical="$(profile_owner_canonical_path "${metadata_repo}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH "profile worktree is missing: ${metadata_repo}"
+ return 1
+ }
+ [[ "${metadata_canonical}" == "${canonical_repo}" ]] || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'profile metadata belongs to another worktree'
+ return 1
+ }
+ [[ "${branch}" == "${expected_branch}" ]] || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'profile metadata belongs to another branch'
+ return 1
+ }
+ [[ -z "${dirty}" || "${dirty}" == false ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID 'profile metadata records a dirty worktree'
+ return 1
+ }
+
+ actual_worktree="$(t2_worktree_id "${canonical_repo}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'unable to derive worktree identity'
+ return 1
+ }
+ actual_owner="$(t2_profile_owner_id "${canonical_repo}" "${branch}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'unable to derive profile owner identity'
+ return 1
+ }
+
+ if [[ "${schema}" == 1 ]]; then
+ sha_short="$(profile_owner_value "${profile_env}" SHA_SHORT)"
+ [[ "${sha_short}" =~ ^[0-9a-fA-F]{7,40}$ ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID 'schema-v1 metadata requires a hexadecimal SHA_SHORT'
+ return 1
+ }
+ [[ "${dirty}" == false ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID 'schema-v1 metadata requires DIRTY=false'
+ return 1
+ }
+ worktree_id="${actual_worktree}"
+ owner_id="${actual_owner}"
+ created_head="$(printf '%s' "${sha_short}" | LC_ALL=C tr '[:upper:]' '[:lower:]')"
+ else
+ worktree_id="$(profile_owner_value "${profile_env}" WORKTREE_ID)"
+ owner_id="$(profile_owner_value "${profile_env}" OWNER_ID)"
+ created_head="$(profile_owner_value "${profile_env}" CREATED_HEAD)"
+ [[ "${worktree_id}" =~ ^[0-9a-f]{40}$ && "${owner_id}" =~ ^[0-9a-f]{40}$ ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID 'schema-v2 metadata requires full lowercase WORKTREE_ID and OWNER_ID'
+ return 1
+ }
+ [[ "${created_head}" =~ ^[0-9a-f]{40}$ ]] || {
+ profile_owner_error PROFILE_METADATA_INVALID 'schema-v2 metadata requires a full lowercase CREATED_HEAD'
+ return 1
+ }
+ [[ "${worktree_id}" == "${actual_worktree}" && "${owner_id}" == "${actual_owner}" ]] || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'schema-v2 owner identity does not match the current worktree and branch'
+ return 1
+ }
+ sha_short="$(profile_owner_value "${profile_env}" SHA_SHORT)"
+ if [[ -n "${sha_short}" && "${created_head}" != "${sha_short}"* ]]; then
+ profile_owner_error PROFILE_METADATA_INVALID 'legacy SHA_SHORT does not prefix CREATED_HEAD'
+ return 1
+ fi
+ fi
+
+ profile_owner_validate_ports "${ports_env}" || return 1
+
+ PROFILE_OWNER_SCHEMA_VERSION="${schema}"
+ PROFILE_OWNER_WORKTREE_ID="${worktree_id}"
+ PROFILE_OWNER_ID="${owner_id}"
+ PROFILE_OWNER_CREATED_HEAD="${created_head}"
+ PROFILE_OWNER_PROFILE="${profile}"
+ PROFILE_OWNER_REPO_DIR="${canonical_repo}"
+ PROFILE_OWNER_BRANCH="${branch}"
+ PROFILE_OWNER_PROFILE_ENV="${profile_env}"
+ PROFILE_OWNER_PORTS_ENV="${ports_env}"
+}
+
+profile_owner_print_selection() {
+ printf 'PROFILE_SCHEMA_VERSION=%s\n' "${PROFILE_OWNER_SCHEMA_VERSION}"
+ printf 'WORKTREE_ID=%s\n' "${PROFILE_OWNER_WORKTREE_ID}"
+ printf 'OWNER_ID=%s\n' "${PROFILE_OWNER_ID}"
+ printf 'CREATED_HEAD=%s\n' "${PROFILE_OWNER_CREATED_HEAD}"
+ printf 'PROFILE=%s\n' "${PROFILE_OWNER_PROFILE}"
+ printf 'REPO_DIR=%s\n' "${PROFILE_OWNER_REPO_DIR}"
+ printf 'BRANCH=%s\n' "${PROFILE_OWNER_BRANCH}"
+ printf 'PROFILE_ENV=%s\n' "${PROFILE_OWNER_PROFILE_ENV}"
+ printf 'PORTS_ENV=%s\n' "${PROFILE_OWNER_PORTS_ENV}"
+}
+
+profile_owner_candidate_targets() {
+ local profile_env="$1" expected_repo="$2" expected_branch="$3"
+ local candidate_repo candidate_branch candidate_canonical
+ candidate_repo="$(profile_owner_value "${profile_env}" REPO_DIR)"
+ candidate_branch="$(profile_owner_value "${profile_env}" BRANCH)"
+ [[ -n "${candidate_repo}" && "${candidate_branch}" == "${expected_branch}" ]] || return 1
+ candidate_canonical="$(profile_owner_canonical_path "${candidate_repo}")" || return 1
+ [[ "${candidate_canonical}" == "${expected_repo}" ]]
+}
+
+profile_owner_resolve() {
+ local repo_dir="$1" branch="$2" profile_root="$3" explicit_profile="${4:-}"
+ local explicit_profile_env="${5:-}" explicit_ports_env="${6:-}"
+ local canonical_repo profile_env ports_env candidate
+ local stable_profile stable_profile_dir
+ local -a matches=()
+
+ canonical_repo="$(profile_owner_canonical_path "${repo_dir}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH "worktree is missing: ${repo_dir}"
+ return 1
+ }
+ profile_owner_validate_branch "${branch}" || return 1
+
+ if [[ -n "${explicit_ports_env}" && -z "${explicit_profile_env}" && -z "${explicit_profile}" ]]; then
+ explicit_profile_env="$(dirname -- "${explicit_ports_env}")/profile.env"
+ fi
+ if [[ -n "${explicit_profile_env}" || -n "${explicit_profile}" ]]; then
+ if [[ -z "${explicit_profile_env}" ]]; then
+ profile_env="${profile_root}/${explicit_profile}/profile.env"
+ else
+ profile_env="${explicit_profile_env}"
+ fi
+ if [[ -z "${explicit_ports_env}" ]]; then
+ ports_env="$(dirname -- "${profile_env}")/ports.env"
+ else
+ ports_env="${explicit_ports_env}"
+ fi
+ profile_owner_validate_selection "${profile_env}" "${ports_env}" \
+ "${canonical_repo}" "${branch}" "${explicit_profile}" || return 1
+ profile_owner_print_selection
+ return 0
+ fi
+
+ [[ -d "${profile_root}" ]] || {
+ profile_owner_error PROFILE_NOT_FOUND "profile root is not initialized: ${profile_root}"
+ return 3
+ }
+ profile_root="$(profile_owner_canonical_path "${profile_root}")" || {
+ profile_owner_error PROFILE_METADATA_INVALID 'profile root cannot be canonicalized safely'
+ return 1
+ }
+ if ! find "${profile_root}" -mindepth 2 -maxdepth 2 -type f -name profile.env -print >/dev/null 2>&1; then
+ profile_owner_error PROFILE_METADATA_INVALID "profile root cannot be inspected completely: ${profile_root}"
+ return 1
+ fi
+
+ stable_profile="$(profile_owner_stable_profile "${canonical_repo}" "${branch}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH 'unable to derive the stable profile identity'
+ return 1
+ }
+ stable_profile_dir="${profile_root}/${stable_profile}"
+ if [[ -d "${stable_profile_dir}" && ! -f "${stable_profile_dir}/profile.env" ]]; then
+ profile_owner_error PROFILE_METADATA_MISSING \
+ "stable profile directory exists without profile.env: ${stable_profile_dir}"
+ return 1
+ fi
+
+ while IFS= read -r -d '' candidate; do
+ if profile_owner_candidate_targets "${candidate}" "${canonical_repo}" "${branch}"; then
+ matches+=("${candidate}")
+ fi
+ done < <(find "${profile_root}" -mindepth 2 -maxdepth 2 -type f -name profile.env -print0 2>/dev/null)
+
+ if (( ${#matches[@]} == 0 )); then
+ profile_owner_error PROFILE_NOT_FOUND 'no persisted profile matches this worktree and branch'
+ return 3
+ fi
+ if (( ${#matches[@]} > 1 )); then
+ profile_owner_error PROFILE_SELECTION_AMBIGUOUS \
+ "multiple persisted profiles match this worktree and branch (${#matches[@]})"
+ return 4
+ fi
+
+ profile_env="${matches[0]}"
+ ports_env="$(dirname -- "${profile_env}")/ports.env"
+ profile_owner_validate_selection "${profile_env}" "${ports_env}" \
+ "${canonical_repo}" "${branch}" '' || return 1
+ profile_owner_print_selection
+}
+
+profile_owner_usage() {
+ cat >&2 <<'EOF_USAGE'
+usage:
+ profile-owner.sh identity --repo-dir DIR --branch BRANCH [--created-head SHA]
+ profile-owner.sh validate --repo-dir DIR --branch BRANCH --profile-env FILE --ports-env FILE [--profile NAME]
+ profile-owner.sh resolve --repo-dir DIR --branch BRANCH --profile-root DIR [--profile NAME] [--profile-env FILE] [--ports-env FILE]
+EOF_USAGE
+}
+
+profile_owner_main() {
+ local action="${1:-}" repo_dir='' branch='' created_head=''
+ local profile_root='' profile='' profile_env='' ports_env=''
+ [[ -n "${action}" ]] || { profile_owner_usage; return 2; }
+ shift
+ while (( $# > 0 )); do
+ case "$1" in
+ --repo-dir|--branch|--created-head|--profile-root|--profile|--profile-env|--ports-env)
+ (( $# >= 2 )) || { profile_owner_usage; return 2; }
+ case "$1" in
+ --repo-dir) repo_dir="$2" ;;
+ --branch) branch="$2" ;;
+ --created-head) created_head="$2" ;;
+ --profile-root) profile_root="$2" ;;
+ --profile) profile="$2" ;;
+ --profile-env) profile_env="$2" ;;
+ --ports-env) ports_env="$2" ;;
+ esac
+ shift 2 ;;
+ *) profile_owner_usage; return 2 ;;
+ esac
+ done
+
+ [[ -n "${repo_dir}" && -n "${branch}" ]] || { profile_owner_usage; return 2; }
+ case "${action}" in
+ identity)
+ local canonical_repo worktree_id owner_id stable_profile
+ canonical_repo="$(profile_owner_canonical_path "${repo_dir}")" || {
+ profile_owner_error PROFILE_OWNERSHIP_MISMATCH "worktree is missing: ${repo_dir}"
+ profile_owner_print_error
+ return 1
+ }
+ if ! profile_owner_validate_branch "${branch}"; then
+ profile_owner_print_error
+ return 1
+ fi
+ worktree_id="$(t2_worktree_id "${canonical_repo}")"
+ owner_id="$(t2_profile_owner_id "${canonical_repo}" "${branch}")"
+ stable_profile="$(profile_owner_stable_profile "${canonical_repo}" "${branch}")"
+ if [[ -n "${created_head}" && ! "${created_head}" =~ ^[0-9a-f]{40}$ ]]; then
+ profile_owner_error PROFILE_METADATA_INVALID 'CREATED_HEAD must be a full lowercase Git SHA'
+ profile_owner_print_error
+ return 1
+ fi
+ printf 'PROFILE_SCHEMA_VERSION=2\nWORKTREE_ID=%s\nOWNER_ID=%s\n' "${worktree_id}" "${owner_id}"
+ [[ -z "${created_head}" ]] || printf 'CREATED_HEAD=%s\n' "${created_head}"
+ printf 'PROFILE=%s\nREPO_DIR=%s\nBRANCH=%s\n' "${stable_profile}" "${canonical_repo}" "${branch}"
+ ;;
+ validate)
+ [[ -n "${profile_env}" && -n "${ports_env}" ]] || { profile_owner_usage; return 2; }
+ if ! profile_owner_validate_selection "${profile_env}" "${ports_env}" \
+ "${repo_dir}" "${branch}" "${profile}"; then
+ profile_owner_print_error
+ return 1
+ fi
+ profile_owner_print_selection
+ ;;
+ resolve)
+ [[ -n "${profile_root}" || -n "${profile_env}" ]] || { profile_owner_usage; return 2; }
+ local status=0
+ profile_owner_resolve "${repo_dir}" "${branch}" "${profile_root}" \
+ "${profile}" "${profile_env}" "${ports_env}" || status=$?
+ if (( status != 0 )); then
+ profile_owner_print_error
+ return "${status}"
+ fi
+ ;;
+ *) profile_owner_usage; return 2 ;;
+ esac
+}
+
+if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
+ set -euo pipefail
+ profile_owner_main "$@"
+fi
diff --git a/scripts/minikube/run-with-deadline.mjs b/scripts/minikube/run-with-deadline.mjs
new file mode 100755
index 000000000..f121e7024
--- /dev/null
+++ b/scripts/minikube/run-with-deadline.mjs
@@ -0,0 +1,241 @@
+#!/usr/bin/env node
+
+import { spawn } from "node:child_process";
+import { constants as osConstants } from "node:os";
+import process from "node:process";
+
+const EXIT_TIMEOUT = 124;
+const EXIT_COMMAND_CANNOT_EXECUTE = 126;
+const EXIT_COMMAND_NOT_FOUND = 127;
+const HANDLED_SIGNALS = ["SIGHUP", "SIGQUIT", "SIGINT", "SIGTERM"];
+
+function usage(message) {
+ if (message) process.stderr.write(`${message}\n`);
+ process.stderr.write(
+ "usage: run-with-deadline.mjs --timeout-seconds N [--heartbeat-seconds N] " +
+ "[--kill-grace-seconds N] --label LABEL -- command [args...]\n",
+ );
+ process.exit(2);
+}
+
+function positiveInteger(flag, raw, maximum) {
+ if (!/^[1-9][0-9]*$/.test(raw ?? "")) {
+ usage(`${flag} must be a positive integer`);
+ }
+ const value = Number(raw);
+ if (!Number.isSafeInteger(value) || value > maximum) {
+ usage(`${flag} must be no greater than ${maximum}`);
+ }
+ return value;
+}
+
+let timeoutSeconds;
+let heartbeatSeconds = 30;
+let killGraceSeconds = 5;
+let label;
+let commandIndex = -1;
+
+for (let index = 2; index < process.argv.length; index += 1) {
+ const arg = process.argv[index];
+ if (arg === "--") {
+ commandIndex = index + 1;
+ break;
+ }
+ const value = process.argv[index + 1];
+ switch (arg) {
+ case "--timeout-seconds":
+ timeoutSeconds = positiveInteger(arg, value, 86_400);
+ index += 1;
+ break;
+ case "--heartbeat-seconds":
+ heartbeatSeconds = positiveInteger(arg, value, 3_600);
+ index += 1;
+ break;
+ case "--kill-grace-seconds":
+ killGraceSeconds = positiveInteger(arg, value, 300);
+ index += 1;
+ break;
+ case "--label":
+ label = value;
+ index += 1;
+ break;
+ default:
+ usage(`unknown argument: ${arg}`);
+ }
+}
+
+if (timeoutSeconds === undefined) usage("--timeout-seconds is required");
+if (!label || !/^[A-Za-z0-9][A-Za-z0-9._:-]{0,79}$/.test(label)) {
+ usage("--label must use only A-Z, a-z, 0-9, dot, underscore, colon, or dash");
+}
+if (commandIndex < 0 || commandIndex >= process.argv.length) {
+ usage("a command is required after --");
+}
+
+const command = process.argv[commandIndex];
+const args = process.argv.slice(commandIndex + 1);
+const startedAt = process.hrtime.bigint();
+const elapsedMilliseconds = () =>
+ Number((process.hrtime.bigint() - startedAt) / 1_000_000n);
+const report = (event, fields = "") => {
+ const suffix = fields ? ` ${fields}` : "";
+ process.stderr.write(`[HARNESS_DEADLINE] label=${label} event=${event}${suffix}\n`);
+};
+const wait = (milliseconds) =>
+ new Promise((resolve) => setTimeout(resolve, milliseconds));
+
+report("start", `timeoutSeconds=${timeoutSeconds}`);
+
+const child = spawn(command, args, {
+ detached: process.platform !== "win32",
+ env: process.env,
+ stdio: "inherit",
+});
+
+let completionResolved = false;
+const completion = new Promise((resolve) => {
+ const finish = (result) => {
+ if (completionResolved) return;
+ completionResolved = true;
+ resolve(result);
+ };
+ child.once("error", (error) => finish({ kind: "spawn-error", error }));
+ child.once("exit", (code, signal) => finish({ kind: "exit", code, signal }));
+});
+
+function signalExitCode(signal) {
+ const signalNumber = osConstants.signals[signal];
+ return Number.isInteger(signalNumber) ? 128 + signalNumber : 1;
+}
+
+function spawnErrorExitCode(error) {
+ if (error?.code === "ENOENT") return EXIT_COMMAND_NOT_FOUND;
+ if (["EACCES", "EPERM", "ENOEXEC", "EISDIR"].includes(error?.code)) {
+ return EXIT_COMMAND_CANNOT_EXECUTE;
+ }
+ return 1;
+}
+
+function killProcessGroup(signal) {
+ if (!child.pid) return;
+ try {
+ if (process.platform === "win32") child.kill(signal);
+ else process.kill(-child.pid, signal);
+ } catch (error) {
+ if (error?.code !== "ESRCH") throw error;
+ }
+}
+
+function processGroupExists() {
+ if (!child.pid) return false;
+ if (process.platform === "win32") {
+ return child.exitCode === null && child.signalCode === null;
+ }
+ try {
+ process.kill(-child.pid, 0);
+ return true;
+ } catch (error) {
+ if (error?.code === "ESRCH") return false;
+ if (error?.code === "EPERM") return true;
+ throw error;
+ }
+}
+
+async function waitForProcessGroupExit(milliseconds) {
+ const deadline = Date.now() + milliseconds;
+ while (processGroupExists() && Date.now() < deadline) {
+ await wait(Math.min(25, Math.max(1, deadline - Date.now())));
+ }
+ return !processGroupExists();
+}
+
+const heartbeat = setInterval(() => {
+ report("heartbeat", `elapsedSeconds=${Math.floor(elapsedMilliseconds() / 1_000)}`);
+}, heartbeatSeconds * 1_000);
+heartbeat.unref();
+
+let timeoutHandle;
+const timeout = new Promise((resolve) => {
+ timeoutHandle = setTimeout(() => resolve({ kind: "timeout" }), timeoutSeconds * 1_000);
+});
+
+let resolveParentSignal;
+let firstParentSignal = null;
+let teardownStarted = false;
+let signalEscalated = false;
+const parentSignal = new Promise((resolve) => {
+ resolveParentSignal = resolve;
+});
+const signalHandlers = new Map();
+for (const signal of HANDLED_SIGNALS) {
+ const handler = () => {
+ if (firstParentSignal === null && !teardownStarted) {
+ firstParentSignal = signal;
+ resolveParentSignal({ kind: "parent-signal", signal });
+ return;
+ }
+
+ if (!signalEscalated) {
+ report("signal-escalated", `signal=${signal} action=SIGKILL`);
+ signalEscalated = true;
+ }
+ teardownStarted = true;
+ killProcessGroup("SIGKILL");
+ };
+ signalHandlers.set(signal, handler);
+ process.on(signal, handler);
+}
+
+const first = await Promise.race([completion, timeout, parentSignal]);
+clearInterval(heartbeat);
+clearTimeout(timeoutHandle);
+teardownStarted = first.kind === "timeout" || first.kind === "parent-signal";
+
+let finalExitCode;
+
+if (first.kind === "exit") {
+ const durationMs = elapsedMilliseconds();
+ if (first.signal) {
+ finalExitCode = signalExitCode(first.signal);
+ report(
+ "exit",
+ `durationMs=${durationMs} signal=${first.signal} exitCode=${finalExitCode}`,
+ );
+ } else {
+ finalExitCode = first.code ?? 1;
+ report("exit", `durationMs=${durationMs} exitCode=${finalExitCode}`);
+ }
+ // A bounded operation may not daemonize work behind the runner. If the
+ // direct child exited while a helper remained in its process group, reap the
+ // helper before returning the child's exact status.
+ killProcessGroup("SIGKILL");
+ await waitForProcessGroupExit(1_000);
+} else if (first.kind === "spawn-error") {
+ finalExitCode = spawnErrorExitCode(first.error);
+ report("spawn-failed", `durationMs=${elapsedMilliseconds()} exitCode=${finalExitCode}`);
+} else {
+ const terminatingSignal = first.kind === "timeout" ? "SIGTERM" : first.signal;
+ finalExitCode =
+ first.kind === "timeout" ? EXIT_TIMEOUT : signalExitCode(first.signal);
+ report(
+ first.kind === "timeout" ? "timeout" : "interrupted",
+ `durationMs=${elapsedMilliseconds()} signal=${terminatingSignal} exitCode=${finalExitCode}`,
+ );
+
+ if (!signalEscalated) killProcessGroup(terminatingSignal);
+ await waitForProcessGroupExit(killGraceSeconds * 1_000);
+
+ // Always address the original process group after graceful teardown. The
+ // direct child may have exited while a credential-helper descendant ignored
+ // the first signal; killing by negative PGID closes that otherwise invisible
+ // orphan path. A repeated parent signal reaches this same action immediately
+ // through the handlers above.
+ killProcessGroup("SIGKILL");
+ await waitForProcessGroupExit(1_000);
+ report("terminated", `durationMs=${elapsedMilliseconds()} exitCode=${finalExitCode}`);
+}
+
+for (const [signal, handler] of signalHandlers) {
+ process.removeListener(signal, handler);
+}
+process.exitCode = finalExitCode;
diff --git a/scripts/minikube/t2-common.sh b/scripts/minikube/t2-common.sh
index 9ce7a8e25..710a533e8 100755
--- a/scripts/minikube/t2-common.sh
+++ b/scripts/minikube/t2-common.sh
@@ -9,6 +9,10 @@ T2_SCRIPT_DIR="$T2_SCRIPT_DIR"
if [ -z "$T2_SCRIPT_DIR" ]; then T2_SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"; fi
# shellcheck source=t2-worktree-id.sh
. "$T2_SCRIPT_DIR/t2-worktree-id.sh"
+# shellcheck source=profile-owner.sh
+. "$T2_SCRIPT_DIR/profile-owner.sh"
+# shellcheck source=port-forward-owner.sh
+. "$T2_SCRIPT_DIR/port-forward-owner.sh"
# shellcheck source=profile-readiness.sh
. "$T2_SCRIPT_DIR/profile-readiness.sh"
T2_PROJECT_DIR="$T2_PROJECT_DIR"
@@ -16,10 +20,23 @@ if [ -z "$T2_PROJECT_DIR" ]; then T2_PROJECT_DIR="$(cd -- "$T2_SCRIPT_DIR/../.."
T2_PROFILE="$T2_PROFILE"
if [ -z "$T2_PROFILE" ]; then T2_PROFILE="$MINIKUBE_PROFILE"; fi
T2_CONTEXT="$T2_CONTEXT"
-if [ -z "$T2_CONTEXT" ]; then T2_CONTEXT="$CONTROL_API_REAL_PG_CONTEXT"; fi
-if [ -z "$T2_CONTEXT" ]; then T2_CONTEXT="$K8S_CONTEXT"; fi
-if [ -z "$T2_CONTEXT" ]; then T2_CONTEXT="$KUBECONTEXT"; fi
-if [ -z "$T2_CONTEXT" ]; then T2_CONTEXT="$T2_PROFILE"; fi
+T2_CONTEXT_SOURCE=explicit-t2-context
+if [ -z "$T2_CONTEXT" ] && [ -n "$CONTROL_API_REAL_PG_CONTEXT" ]; then
+ T2_CONTEXT="$CONTROL_API_REAL_PG_CONTEXT"
+ T2_CONTEXT_SOURCE=explicit-control-api-real-pg-context
+fi
+if [ -z "$T2_CONTEXT" ] && [ -n "$K8S_CONTEXT" ]; then
+ T2_CONTEXT="$K8S_CONTEXT"
+ T2_CONTEXT_SOURCE=explicit-k8s-context
+fi
+if [ -z "$T2_CONTEXT" ] && [ -n "$KUBECONTEXT" ]; then
+ T2_CONTEXT="$KUBECONTEXT"
+ T2_CONTEXT_SOURCE=explicit-kubecontext
+fi
+if [ -z "$T2_CONTEXT" ]; then
+ T2_CONTEXT="$T2_PROFILE"
+ T2_CONTEXT_SOURCE=implicit-profile-fallback
+fi
T2_TIMEOUT_SECONDS="$T2_TIMEOUT_SECONDS"
if [ -z "$T2_TIMEOUT_SECONDS" ]; then T2_TIMEOUT_SECONDS="$T2_TIMEOUT"; fi
if [ -z "$T2_TIMEOUT_SECONDS" ]; then T2_TIMEOUT_SECONDS=180; fi
@@ -68,6 +85,8 @@ T2_LOCK_DIR=""
T2_LOCK_KEY=""
T2_EVIDENCE_DIR=""
T2_EVIDENCE_FILE=""
+T2_EVIDENCE_KIND="$T2_EVIDENCE_KIND"
+if [ -z "$T2_EVIDENCE_KIND" ]; then T2_EVIDENCE_KIND=certification; fi
T2_LOCK_HELD=false
T2_LOCK_RELEASED=false
T2_LOCK_TOKEN="$T2_LOCK_TOKEN"
@@ -143,6 +162,13 @@ t2_mk() {
minikube -p "$T2_PROFILE" "$@"
}
+t2_require_explicit_context() {
+ if [ "$T2_CONTEXT_SOURCE" = implicit-profile-fallback ]; then
+ T2_NEXT_COMMAND='set CONTROL_API_REAL_PG_CONTEXT (or T2_CONTEXT) to the generated branch-owned context'
+ t2_fail DEVELOPMENT_SCOPE_REQUIRED 'Kubernetes context must be supplied explicitly; profile-name fallback is not accepted'
+ fi
+}
+
t2_canonical_path() {
(cd -- "$1" 2>/dev/null && pwd -P)
}
@@ -197,6 +223,7 @@ t2_repo_metadata() {
}
t2_profile_scope() {
+ t2_require_explicit_context
if [ -z "$T2_PROFILE" ] || [ -z "$T2_CONTEXT" ] || [ "$T2_PROFILE" != "$T2_CONTEXT" ]; then
T2_NEXT_COMMAND='set MINIKUBE_PROFILE and CONTROL_API_REAL_PG_CONTEXT to the same generated profile'
t2_fail DEVELOPMENT_SCOPE_REQUIRED 'an explicit branch-owned profile and matching Kubernetes context are required'
@@ -210,40 +237,22 @@ t2_profile_scope() {
T2_NEXT_COMMAND='use the generated profile name without shell metacharacters'
t2_fail DEVELOPMENT_SCOPE_REQUIRED 'profile name is not a valid local Minikube identifier'
fi
- if [ ! -f "$T2_PROFILE_ENV" ]; then
- T2_NEXT_COMMAND='generate profile.env and ports.env with the branch profile helper, then retry'
- t2_fail DEVELOPMENT_SCOPE_REQUIRED "generated profile metadata is missing: $T2_PROFILE_ENV"
- fi
- local profile_name profile_repo profile_branch profile_sha profile_dirty
- profile_name="$(awk -F= '$1 == "PROFILE" {print substr($0, index($0,"=")+1); exit}' "$T2_PROFILE_ENV" 2>/dev/null || true)"
- profile_repo="$(awk -F= '$1 == "REPO_DIR" {print substr($0, index($0,"=")+1); exit}' "$T2_PROFILE_ENV" 2>/dev/null || true)"
- profile_branch="$(awk -F= '$1 == "BRANCH" {print substr($0, index($0,"=")+1); exit}' "$T2_PROFILE_ENV" 2>/dev/null || true)"
- profile_sha="$(awk -F= '$1 == "SHA_SHORT" {print substr($0, index($0,"=")+1); exit}' "$T2_PROFILE_ENV" 2>/dev/null || true)"
- profile_dirty="$(awk -F= '$1 == "DIRTY" {print substr($0, index($0,"=")+1); exit}' "$T2_PROFILE_ENV" 2>/dev/null || true)"
- if [ "$profile_name" != "$T2_PROFILE" ] || [ -z "$profile_repo" ] || [ -z "$profile_branch" ] || [ -z "$profile_sha" ]; then
- T2_NEXT_COMMAND='regenerate the profile metadata from the current worktree, then retry'
- t2_fail PROFILE_OWNERSHIP_MISMATCH 'profile metadata is incomplete or names a different profile'
- fi
- if [ "$profile_dirty" = true ]; then
- T2_NEXT_COMMAND='commit or restore the worktree, then regenerate the profile metadata'
- t2_fail DEVELOPMENT_SCOPE_REQUIRED 'profile metadata was generated from a dirty worktree'
- fi
- if [ "$(t2_canonical_path "$profile_repo")" != "$T2_PROJECT_DIR" ]; then
- T2_NEXT_COMMAND='use the profile generated by this Evenfire worktree, not another worktree'
- t2_fail PROFILE_OWNERSHIP_MISMATCH 'profile metadata belongs to another worktree'
- fi
- if [ "$profile_branch" != "$T2_BRANCH" ]; then
- T2_NEXT_COMMAND='regenerate the profile for the current branch'
- t2_fail PROFILE_OWNERSHIP_MISMATCH 'profile branch does not match current branch'
- fi
- # A healthy branch-owned profile is reusable across commits. The cache SHA
- # is historical naming metadata; exact runtime identity is enforced below
- # by the pre-gate marker's gitHead/worktreeId pair, not by recreating a
- # Minikube cluster for every commit.
- if [ ! -f "$T2_PORTS_ENV" ]; then
- T2_NEXT_COMMAND='generate the profile-owned random ports before starting a gate'
- t2_fail DEVELOPMENT_SCOPE_REQUIRED "profile-owned ports.env is missing: $T2_PORTS_ENV"
+ if ! profile_owner_validate_selection "$T2_PROFILE_ENV" "$T2_PORTS_ENV" \
+ "$T2_PROJECT_DIR" "$T2_BRANCH" "$T2_PROFILE"; then
+ case "$PROFILE_OWNER_ERROR_CODE" in
+ PROFILE_METADATA_MISSING)
+ T2_NEXT_COMMAND='resolve or generate profile.env with the branch profile helper, then retry' ;;
+ PROFILE_PORTS_MISSING)
+ T2_NEXT_COMMAND='restore the persisted profile-owned ports.env; never regenerate adopted ports' ;;
+ PROFILE_PORTS_INVALID)
+ T2_NEXT_COMMAND='repair the selected profile metadata through the branch profile helper; do not invent ports' ;;
+ *)
+ T2_NEXT_COMMAND='resolve the unique profile owned by this canonical worktree and branch, then retry' ;;
+ esac
+ t2_fail PROFILE_OWNERSHIP_MISMATCH "$PROFILE_OWNER_ERROR_MESSAGE"
fi
+ # CREATED_HEAD/SHA_SHORT is historical creation metadata only. Exact runtime
+ # freshness remains the marker's gitHead + worktreeId + clusterFingerprint.
}
t2_context_check() {
@@ -660,65 +669,82 @@ PY
fi
}
+t2_port_forward_targets_context() {
+ local command_line="$1" index context_matches=false port_forward=false
+ local -a argv=()
+ read -r -a argv <<<"$command_line"
+ [ "${#argv[@]}" -gt 0 ] || return 1
+ [ "${argv[0]##*/}" = kubectl ] || return 1
+ for ((index = 1; index < ${#argv[@]}; index += 1)); do
+ case "${argv[$index]}" in
+ port-forward) port_forward=true ;;
+ "--context=$T2_CONTEXT") context_matches=true ;;
+ --context)
+ if [ $((index + 1)) -lt "${#argv[@]}" ] &&
+ [ "${argv[$((index + 1))]}" = "$T2_CONTEXT" ]; then
+ context_matches=true
+ fi
+ ;;
+ esac
+ done
+ [ "$port_forward" = true ] && [ "$context_matches" = true ]
+}
+
t2_pid_file_matches_process() {
- local pid_file="$1" pid="$2" recorded_pid recorded_start actual_start
- recorded_pid="$(sed -n '1p' "$pid_file" 2>/dev/null || true)"
- recorded_start="$(sed -n 's/^PROCESS_START=//p' "$pid_file" 2>/dev/null | head -1 || true)"
- [ "$recorded_pid" = "$pid" ] || return 1
- [ -n "$recorded_start" ] || return 1
- [ "$recorded_start" = unavailable ] && return 0
- actual_start="$(ps -p "$pid" -o lstart= 2>/dev/null | sed 's/^ *//' || true)"
- [ -n "$actual_start" ] && [ "$actual_start" = "$recorded_start" ]
+ local pid_file="$1" pid="$2" actual_start command_line
+ pf_owner_read_record "$pid_file" || return 1
+ [ "$PF_OWNER_RECORD_FIRST_PID" = "$pid" ] || return 1
+ [ "$PF_OWNER_RECORD_PROFILE" = "$T2_PROFILE" ] || return 1
+ [ "$PF_OWNER_RECORD_CONTEXT" = "$T2_CONTEXT" ] || return 1
+ [ "$PF_OWNER_RECORD_WORKTREE" = "$T2_PROJECT_DIR" ] || return 1
+ actual_start="$(pf_owner_process_start "$pid")" || return 1
+ [ "$actual_start" = "$PF_OWNER_RECORD_START" ] || return 1
+ command_line="$(pf_owner_process_command "$pid")" || return 1
+ pf_owner_command_matches "$command_line" "$PF_OWNER_RECORD_CONTEXT" \
+ "$PF_OWNER_RECORD_NAMESPACE" "$PF_OWNER_RECORD_SERVICE" \
+ "$PF_OWNER_RECORD_LOCAL_PORT" "$PF_OWNER_RECORD_REMOTE_PORT" || return 1
+ actual_start="$(pf_owner_process_start "$pid")" || return 1
+ [ "$actual_start" = "$PF_OWNER_RECORD_START" ]
}
t2_process_check() {
- local uid pid ppid rest command_line comm allowed pid_file recorded_pid
- local safe_profile
- safe_profile="$(printf '%s' "$T2_PROFILE" | tr -c 'A-Za-z0-9_.-' '_')"
+ local uid pid ppid rest command_line comm pid_file recorded_pid
+ local matching_records invalid_record
# Only real kubectl port-forward processes. A wrapper whose argv merely
- # mentions those words is not a port-forward (rejected by comm=kubectl).
+ # mentions those words is not a port-forward (rejected by exact argv0/comm).
# Default IFS so UID/PID/PPID split. `IFS=` left pid empty and skipped every line.
# awk is a loose pre-filter: kubectl as argv0/path token AND a later
# standalone port-forward token. Flags may sit between those tokens.
while read -r uid pid ppid rest; do
[ -n "$pid" ] || continue
- command_line="$uid $pid $ppid $rest"
+ command_line="$(pf_owner_process_command "$pid" 2>/dev/null || true)"
+ [ -n "$command_line" ] || continue
comm="$(ps -p "$pid" -o comm= 2>/dev/null || true)"
- case "$comm" in
- *kubectl*) ;;
+ case "${comm##*/}" in
+ kubectl) ;;
*) continue ;;
esac
- allowed=false
- for pid_file in \
- "$T2_PROFILE_ROOT/$T2_PROFILE"/pids/*.pid \
- /tmp/pf-"$safe_profile"-*.pid; do
- [ -f "$pid_file" ] || continue
- recorded_pid="$(sed -n '1p' "$pid_file" 2>/dev/null || true)"
- if [ "$recorded_pid" = "$pid" ] && t2_pid_file_matches_process "$pid_file" "$pid"; then
- allowed=true
- fi
- done
- # pf-all-stack.sh is the canonical gate forwarder and records its child
- # PIDs in /tmp/pf-<profile>-*.pid. Accept those PIDs as profile-owned too;
- # otherwise T1 rejects the forwards that pre-gate-sync just started.
- for pid_file in "/tmp/pf-${safe_profile}-"*.pid; do
+ # Context-less or differently pinned developer forwards are not
+ # attributable to this profile. Exact-token matching prevents a profile
+ # name that is merely a substring of another context from becoming ours.
+ if ! t2_port_forward_targets_context "$command_line"; then
+ continue
+ fi
+ matching_records=0
+ invalid_record=false
+ for pid_file in "$T2_PROFILE_ROOT/$T2_PROFILE"/pids/*.pid; do
[ -f "$pid_file" ] || continue
recorded_pid="$(sed -n '1p' "$pid_file" 2>/dev/null || true)"
- if [[ -n "$recorded_pid" && "$command_line" == *" $recorded_pid "* ]] &&
- t2_pid_file_matches_process "$pid_file" "$recorded_pid"; then
- allowed=true
+ [ "$recorded_pid" = "$pid" ] || continue
+ if t2_pid_file_matches_process "$pid_file" "$pid"; then
+ matching_records=$((matching_records + 1))
+ else
+ invalid_record=true
fi
done
- # Unrelated developer forwards launched without --context are not
- # attributable to this profile and must not make its gate red. A forward
- # that names this profile/context, or one recorded in this profile's PID
- # registry, is attributable and must be owned by this run.
- if [[ "$command_line" != *"$T2_PROFILE"* && "$command_line" != *"$T2_CONTEXT"* && "$allowed" != true ]]; then
- continue
- fi
- if [ "$allowed" != true ]; then
+ if [ "$invalid_record" = true ] || [ "$matching_records" -ne 1 ]; then
T2_NEXT_COMMAND='stop the unrelated profile port-forward or select the owner worktree; do not share it'
- t2_fail PORT_FORWARD_CONFLICT 'a port-forward for this profile is owned by another process'
+ t2_fail PORT_FORWARD_CONFLICT 'a port-forward for this profile lacks one exact ownership record'
return 1
fi
done < <(ps -ef 2>/dev/null | awk '/([^[:space:]]*\/)?kubectl([[:space:]]|$)/ && /[[:space:]]port-forward([[:space:]]|$)/ {print}' || true)
@@ -990,7 +1016,7 @@ t2_evidence_write() {
T2_WORKTREE_ID="$T2_WORKTREE_ID" T2_RUN_ID="$T2_RUN_ID" T2_GATE_ID="$T2_GATE_ID" \
T2_PROFILE="$T2_PROFILE" T2_CONTEXT="$T2_CONTEXT" T2_CLUSTER_FINGERPRINT="$T2_CLUSTER_FINGERPRINT" \
T2_PROFILE_STATUS="$T2_PROFILE_STATUS" T2_PROFILE_HEALTHY="$T2_PROFILE_HEALTHY" \
- T2_EVIDENCE_DIR="$T2_EVIDENCE_DIR" \
+ T2_EVIDENCE_DIR="$T2_EVIDENCE_DIR" T2_EVIDENCE_KIND="$T2_EVIDENCE_KIND" \
T2_IMAGE_MANIFEST="$T2_IMAGE_MANIFEST" T2_IMAGE_SOURCE="$T2_IMAGE_SOURCE" T2_IMAGE_TAG="$T2_IMAGE_TAG" \
python3 - "$file" "$T2_EVIDENCE_FILE" <<'PY'
import json
@@ -1004,14 +1030,24 @@ if prior_path.exists():
try: prior = json.loads(prior_path.read_text())
except ValueError: prior = {}
prior.setdefault("evidenceVersion", 1)
-prior["certificationVersion"] = 1
+evidence_kind = os.environ.get("T2_EVIDENCE_KIND", "certification")
+if evidence_kind not in {"certification", "planner"}:
+ raise SystemExit(f"unsupported T2 evidence kind: {evidence_kind}")
+prior["evidenceKind"] = evidence_kind
+if evidence_kind == "certification":
+ prior["certificationVersion"] = 1
+else:
+ prior.pop("certificationVersion", None)
prior.setdefault("runId", os.environ.get("T2_RUN_ID", ""))
now = datetime.now(timezone.utc)
-prior.setdefault("attestationStartedAt", now.isoformat().replace("+00:00", "Z"))
-prior.setdefault(
- "attestationExpiresAt",
- (now + timedelta(hours=24)).isoformat().replace("+00:00", "Z"),
-)
+if evidence_kind == "certification":
+ prior.setdefault("attestationStartedAt", now.isoformat().replace("+00:00", "Z"))
+ prior.setdefault(
+ "attestationExpiresAt",
+ (now + timedelta(hours=24)).isoformat().replace("+00:00", "Z"),
+ )
+else:
+ prior.setdefault("plannerStartedAt", now.isoformat().replace("+00:00", "Z"))
def redact(value):
import re
value = str(value or "")
@@ -1021,6 +1057,7 @@ def redact(value):
value = re.sub(r"(?i)(?:password|token|secret|api[_-]?key|private[_-]?key)\s*[:=]\s*[^\s,;]+", "<secret-assignment-redacted>", value)
return value
+detail = redact(os.environ.get("DETAIL", ""))
prior.update({
"repository": os.environ.get("T2_PROJECT_DIR", ""),
"branch": os.environ.get("T2_BRANCH", ""),
@@ -1042,27 +1079,46 @@ prior.update({
"imageTag": os.environ.get("T2_IMAGE_TAG", ""),
"phase": os.environ.get("PHASE", ""),
"status": os.environ.get("STATUS", ""),
- "detail": redact(os.environ.get("DETAIL", "")),
+ "detail": detail,
})
phase = os.environ.get("PHASE", "")
status = os.environ.get("STATUS", "")
-attestation = prior.get("attestationStatus", "IN_PROGRESS")
-if status in {"FAIL", "INVALIDATED"}:
- attestation = "INVALIDATED"
-elif phase == "complete" and status == "PASS":
- attestation = "PASS"
-prior["attestationStatus"] = attestation
-lane_attestation = prior.get("laneAttestationStatus", "IN_PROGRESS")
-if phase == "lanes" and status == "PASS":
- lane_attestation = "PASS"
-elif phase == "lock-cleanup" and status == "INVALIDATED":
- lane_attestation = "INVALIDATED"
-prior["laneAttestationStatus"] = lane_attestation
-prior.setdefault("phases", []).append({
+if evidence_kind == "planner":
+ planner_status = prior.get("plannerStatus", "IN_PROGRESS")
+ if status in {"FAIL", "INVALIDATED"}:
+ planner_status = "FAIL"
+ elif phase == "preflight" and status == "PASS":
+ planner_status = "PASS"
+ prior["plannerStatus"] = planner_status
+ prior["attestationStatus"] = "NOT_APPLICABLE"
+ prior["laneAttestationStatus"] = "NOT_APPLICABLE"
+else:
+ attestation = prior.get("attestationStatus", "IN_PROGRESS")
+ if status in {"FAIL", "INVALIDATED"}:
+ attestation = "INVALIDATED"
+ elif phase == "complete" and status == "PASS":
+ attestation = "PASS"
+ prior["attestationStatus"] = attestation
+ lane_attestation = prior.get("laneAttestationStatus", "IN_PROGRESS")
+ if phase == "lanes" and status == "PASS":
+ lane_attestation = "PASS"
+ elif phase == "lock-cleanup" and status == "INVALIDATED":
+ lane_attestation = "INVALIDATED"
+ prior["laneAttestationStatus"] = lane_attestation
+phase_entry = {
"name": os.environ.get("PHASE", ""),
"status": os.environ.get("STATUS", ""),
+ "detail": detail,
"timestamp": __import__("datetime").datetime.now(__import__("datetime").timezone.utc).isoformat().replace("+00:00", "Z"),
-})
+}
+import re
+duration_match = re.search(r"(?:^|[;\s])duration(?:Seconds)?=(\d+(?:\.\d+)?)(ms|s)?(?:$|[;\s])", detail)
+if duration_match:
+ duration = float(duration_match.group(1))
+ if duration_match.group(2) == "ms":
+ duration /= 1000
+ phase_entry["durationSeconds"] = duration
+prior.setdefault("phases", []).append(phase_entry)
path.write_text(json.dumps(prior, indent=2, sort_keys=True) + "\n")
PY
mv -- "$file" "$T2_EVIDENCE_FILE"
diff --git a/scripts/minikube/t2-preflight.sh b/scripts/minikube/t2-preflight.sh
index 26b73146d..4f146c02e 100755
--- a/scripts/minikube/t2-preflight.sh
+++ b/scripts/minikube/t2-preflight.sh
@@ -8,6 +8,8 @@ set -euo pipefail
set +x
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
+T2_EVIDENCE_KIND=planner
+export T2_EVIDENCE_KIND
# shellcheck source=scripts/minikube/t2-common.sh
source "$SCRIPT_DIR/t2-common.sh"
diff --git a/scripts/minikube/t2-setup-handoff.sh b/scripts/minikube/t2-setup-handoff.sh
new file mode 100755
index 000000000..b580145cf
--- /dev/null
+++ b/scripts/minikube/t2-setup-handoff.sh
@@ -0,0 +1,572 @@
+#!/usr/bin/env bash
+# One-shot, local-only setup-complete handoff for the Minikube T2 harness.
+# The raw inherited lease token is used only to derive an attestation hash; it
+# is never written to disk or included in output.
+# shellcheck disable=SC2034
+
+t2_setup_handoff_apply_plan_result() {
+ local result="${1:-}"
+ case "$result" in
+ consumed)
+ INCREMENTAL_FULL_IMAGE_BUILD=false
+ INCREMENTAL_FULL_DEPLOYMENT=false
+ INCREMENTAL_TARGETS=()
+ ;;
+ rejected)
+ # A requested handoff that cannot be consumed is never an optimization
+ # hint. Restore the canonical safe reconcile regardless of what the
+ # incremental classifier selected from local state.
+ INCREMENTAL_FULL_IMAGE_BUILD=true
+ INCREMENTAL_FULL_DEPLOYMENT=true
+ INCREMENTAL_TARGETS=()
+ ;;
+ *)
+ return 2
+ ;;
+ esac
+}
+
+t2_setup_handoff_cli() {
+ local action="${1:-}"
+ if [[ "$action" != create && "$action" != consume ]]; then
+ printf 'usage: %s create | consume -- <image verification command...>\n' \
+ "$(basename -- "${BASH_SOURCE[0]}")" >&2
+ return 2
+ fi
+ shift
+ if [[ "$action" == consume ]]; then
+ if [[ "${1:-}" != -- ]]; then
+ printf 'T2_SETUP_HANDOFF_REJECTED=VERIFY_COMMAND_REQUIRED\n' >&2
+ return 1
+ fi
+ shift
+ if [[ "$#" -eq 0 ]]; then
+ printf 'T2_SETUP_HANDOFF_REJECTED=VERIFY_COMMAND_REQUIRED\n' >&2
+ return 1
+ fi
+ elif [[ "$#" -ne 0 ]]; then
+ printf 'T2_SETUP_HANDOFF_REJECTED=UNEXPECTED_ARGUMENT\n' >&2
+ return 1
+ fi
+
+ python3 - "$action" "$@" <<'PY'
+import hashlib
+import json
+import os
+import re
+import stat
+import subprocess
+import sys
+import tempfile
+import time
+from datetime import datetime, timezone
+from pathlib import Path
+
+KIND = "evenfire.minikube-t2.setup-complete"
+VERSION = 1
+DEFAULT_TTL_SECONDS = 300
+MAX_TTL_SECONDS = 600
+MAX_HANDOFF_BYTES = 64 * 1024
+SAFE_ID = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$")
+FULL_HEAD = re.compile(r"^[0-9a-f]{40,64}$")
+SHA1 = re.compile(r"^[0-9a-f]{40}$")
+SHA256 = re.compile(r"^[0-9a-f]{64}$")
+
+
+class Rejected(Exception):
+ def __init__(self, code):
+ self.code = code
+
+
+def reject(code):
+ raise Rejected(code)
+
+
+def required(name):
+ value = os.environ.get(name, "")
+ if not value:
+ reject(f"{name}_REQUIRED")
+ if any(character in value for character in ("\0", "\n", "\r")):
+ reject(f"{name}_INVALID")
+ return value
+
+
+def git_value(project, *args):
+ result = subprocess.run(
+ ["git", "-C", str(project), *args],
+ check=False,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.DEVNULL,
+ text=True,
+ )
+ if result.returncode != 0:
+ reject("WORKTREE_GIT_METADATA_INVALID")
+ value = result.stdout.strip()
+ if not value:
+ reject("WORKTREE_GIT_METADATA_INVALID")
+ return value
+
+
+def sha256_file(path):
+ digest = hashlib.sha256()
+ try:
+ with path.open("rb") as handle:
+ for chunk in iter(lambda: handle.read(1024 * 1024), b""):
+ digest.update(chunk)
+ except OSError:
+ reject("IMAGE_MANIFEST_UNREADABLE")
+ return digest.hexdigest()
+
+
+def iso8601(epoch):
+ return (
+ datetime.fromtimestamp(epoch, tz=timezone.utc)
+ .isoformat(timespec="seconds")
+ .replace("+00:00", "Z")
+ )
+
+
+def parse_ttl():
+ raw = os.environ.get("T2_SETUP_HANDOFF_TTL_SECONDS", str(DEFAULT_TTL_SECONDS))
+ try:
+ ttl = int(raw)
+ except ValueError:
+ reject("TTL_INVALID")
+ if ttl < 1 or ttl > MAX_TTL_SECONDS:
+ reject("TTL_INVALID")
+ return ttl
+
+
+def path_is_within(path, parent):
+ try:
+ path.relative_to(parent)
+ return True
+ except ValueError:
+ return False
+
+
+def ensure_private_directory(path):
+ path.mkdir(mode=0o700, parents=True, exist_ok=True)
+ try:
+ metadata = path.lstat()
+ except OSError:
+ reject("STATE_ROOT_INVALID")
+ if not stat.S_ISDIR(metadata.st_mode) or path.is_symlink():
+ reject("STATE_ROOT_INVALID")
+ if metadata.st_uid != os.getuid():
+ reject("STATE_ROOT_INVALID")
+ os.chmod(path, 0o700)
+
+
+def context():
+ project_input = Path(required("T2_PROJECT_DIR"))
+ if not project_input.is_absolute():
+ reject("WORKTREE_PATH_INVALID")
+ try:
+ project = project_input.resolve(strict=True)
+ except OSError:
+ reject("WORKTREE_PATH_INVALID")
+ if not project.is_dir():
+ reject("WORKTREE_PATH_INVALID")
+
+ branch = required("T2_BRANCH")
+ head = required("T2_HEAD")
+ if not FULL_HEAD.fullmatch(head):
+ reject("HEAD_INVALID")
+ if git_value(project, "branch", "--show-current") != branch:
+ reject("BRANCH_MISMATCH")
+ if git_value(project, "rev-parse", "--verify", "HEAD") != head:
+ reject("HEAD_MISMATCH")
+
+ worktree_id = required("T2_WORKTREE_ID")
+ canonical_worktree_id = hashlib.sha1(str(project).encode("utf-8")).hexdigest()
+ if not SHA1.fullmatch(worktree_id) or worktree_id != canonical_worktree_id:
+ reject("WORKTREE_ID_MISMATCH")
+
+ run_id = required("T2_RUN_ID")
+ profile = required("T2_PROFILE")
+ kube_context = required("T2_CONTEXT")
+ if not SAFE_ID.fullmatch(run_id):
+ reject("RUN_ID_INVALID")
+ if not SAFE_ID.fullmatch(profile) or not SAFE_ID.fullmatch(kube_context):
+ reject("PROFILE_CONTEXT_INVALID")
+ if profile != kube_context:
+ reject("PROFILE_CONTEXT_MISMATCH")
+
+ transition = required("T2_SETUP_HANDOFF_TRANSITION")
+ if transition not in ("full-bootstrap", "full-reconcile"):
+ reject("TRANSITION_INVALID")
+ if os.environ.get("T2_SKIP_LOCK", "") != "true":
+ reject("INHERITED_LOCK_REQUIRED")
+
+ lock_identity = required("T2_LOCK_KEY")
+ lock_token = required("T2_LOCK_TOKEN")
+ expected_lock_identity = hashlib.sha1(
+ b"\0".join(
+ value.encode("utf-8")
+ for value in (str(project), branch, head, profile, kube_context)
+ )
+ ).hexdigest()
+ if not SHA1.fullmatch(lock_identity) or lock_identity != expected_lock_identity:
+ reject("LOCK_IDENTITY_MISMATCH")
+ lock_token_sha256 = hashlib.sha256(lock_token.encode("utf-8")).hexdigest()
+
+ manifest_input = Path(required("T2_IMAGE_MANIFEST"))
+ if not manifest_input.is_absolute() or manifest_input.is_symlink():
+ reject("IMAGE_MANIFEST_PATH_INVALID")
+ try:
+ manifest = manifest_input.resolve(strict=True)
+ except OSError:
+ reject("IMAGE_MANIFEST_UNREADABLE")
+ if not manifest.is_file() or not path_is_within(manifest, project):
+ reject("IMAGE_MANIFEST_PATH_INVALID")
+
+ local_state_input = project / ".local-notes"
+ if local_state_input.is_symlink():
+ reject("STATE_ROOT_INVALID")
+ local_state_root = local_state_input.resolve(strict=False)
+ if not path_is_within(local_state_root, project):
+ reject("STATE_ROOT_INVALID")
+ root_input = Path(
+ os.environ.get(
+ "T2_SETUP_HANDOFF_ROOT",
+ str(local_state_root / "infra" / "t2-setup-handoffs"),
+ )
+ )
+ if not root_input.is_absolute():
+ reject("STATE_ROOT_INVALID")
+ state_root = root_input.resolve(strict=False)
+ if not path_is_within(state_root, local_state_root):
+ reject("STATE_ROOT_INVALID")
+ relative_root = os.path.relpath(state_root, project)
+ ignored = subprocess.run(
+ ["git", "-C", str(project), "check-ignore", "-q", "--", relative_root],
+ check=False,
+ stdout=subprocess.DEVNULL,
+ stderr=subprocess.DEVNULL,
+ )
+ if ignored.returncode != 0:
+ reject("STATE_ROOT_NOT_IGNORED")
+
+ run_directory = state_root / worktree_id / run_id
+ handoff = run_directory / "setup-complete.json"
+ consumed = run_directory / "setup-complete.consumed.json"
+ claim = run_directory / "setup-complete.consume.claim"
+
+ return {
+ "project": project,
+ "branch": branch,
+ "head": head,
+ "worktree_id": worktree_id,
+ "run_id": run_id,
+ "profile": profile,
+ "context": kube_context,
+ "transition": transition,
+ "lock_identity": lock_identity,
+ "lock_token_sha256": lock_token_sha256,
+ "manifest": manifest,
+ "state_root": state_root,
+ "run_directory": run_directory,
+ "handoff": handoff,
+ "consumed": consumed,
+ "claim": claim,
+ "ttl": parse_ttl(),
+ }
+
+
+def expected_document(values, issued_at):
+ expires_at = issued_at + values["ttl"]
+ return {
+ "version": VERSION,
+ "kind": KIND,
+ "setupComplete": True,
+ "transition": values["transition"],
+ "runId": values["run_id"],
+ "profile": values["profile"],
+ "context": values["context"],
+ "worktree": {
+ "id": values["worktree_id"],
+ "path": str(values["project"]),
+ },
+ "branch": values["branch"],
+ "head": values["head"],
+ "lock": {
+ "identity": values["lock_identity"],
+ "tokenSha256": values["lock_token_sha256"],
+ },
+ "imageManifest": {
+ "path": str(values["manifest"]),
+ "sha256": sha256_file(values["manifest"]),
+ },
+ "issuedAt": iso8601(issued_at),
+ "expiresAt": iso8601(expires_at),
+ "issuedAtEpoch": issued_at,
+ "expiresAtEpoch": expires_at,
+ "ttlSeconds": values["ttl"],
+ }
+
+
+def reject_duplicate_keys(pairs):
+ result = {}
+ for key, value in pairs:
+ if key in result:
+ reject("HANDOFF_MALFORMED")
+ result[key] = value
+ return result
+
+
+def read_document(path):
+ try:
+ metadata = path.lstat()
+ except FileNotFoundError:
+ reject("HANDOFF_MISSING")
+ except OSError:
+ reject("HANDOFF_UNREADABLE")
+ if path.is_symlink() or not stat.S_ISREG(metadata.st_mode):
+ reject("HANDOFF_UNSAFE_FILE")
+ if metadata.st_uid != os.getuid() or stat.S_IMODE(metadata.st_mode) != 0o600:
+ reject("HANDOFF_UNSAFE_MODE")
+ if metadata.st_size > MAX_HANDOFF_BYTES:
+ reject("HANDOFF_MALFORMED")
+ flags = os.O_RDONLY
+ if hasattr(os, "O_NOFOLLOW"):
+ flags |= os.O_NOFOLLOW
+ try:
+ descriptor = os.open(path, flags)
+ opened_metadata = os.fstat(descriptor)
+ if (
+ opened_metadata.st_dev != metadata.st_dev
+ or opened_metadata.st_ino != metadata.st_ino
+ or not stat.S_ISREG(opened_metadata.st_mode)
+ or opened_metadata.st_uid != os.getuid()
+ or stat.S_IMODE(opened_metadata.st_mode) != 0o600
+ or opened_metadata.st_size > MAX_HANDOFF_BYTES
+ ):
+ os.close(descriptor)
+ reject("HANDOFF_CHANGED_DURING_READ")
+ with os.fdopen(descriptor, "r", encoding="utf-8") as handle:
+ raw = handle.read(MAX_HANDOFF_BYTES + 1)
+ except OSError:
+ reject("HANDOFF_UNREADABLE")
+ if len(raw.encode("utf-8")) > MAX_HANDOFF_BYTES:
+ reject("HANDOFF_MALFORMED")
+ try:
+ return json.loads(raw, object_pairs_hook=reject_duplicate_keys)
+ except (UnicodeError, json.JSONDecodeError, TypeError):
+ reject("HANDOFF_MALFORMED")
+
+
+def validate_document(values, path):
+ document = read_document(path)
+ expected_keys = {
+ "version",
+ "kind",
+ "setupComplete",
+ "transition",
+ "runId",
+ "profile",
+ "context",
+ "worktree",
+ "branch",
+ "head",
+ "lock",
+ "imageManifest",
+ "issuedAt",
+ "expiresAt",
+ "issuedAtEpoch",
+ "expiresAtEpoch",
+ "ttlSeconds",
+ }
+ if not isinstance(document, dict) or set(document) != expected_keys:
+ reject("HANDOFF_MALFORMED")
+ if document.get("version") != VERSION or document.get("kind") != KIND:
+ reject("HANDOFF_MALFORMED")
+ if document.get("setupComplete") is not True:
+ reject("SETUP_INCOMPLETE")
+
+ scalar_expectations = {
+ "transition": values["transition"],
+ "runId": values["run_id"],
+ "profile": values["profile"],
+ "context": values["context"],
+ "branch": values["branch"],
+ "head": values["head"],
+ "ttlSeconds": values["ttl"],
+ }
+ mismatch_codes = {
+ "transition": "TRANSITION_MISMATCH",
+ "runId": "RUN_ID_MISMATCH",
+ "profile": "PROFILE_MISMATCH",
+ "context": "CONTEXT_MISMATCH",
+ "branch": "BRANCH_MISMATCH",
+ "head": "HEAD_MISMATCH",
+ "ttlSeconds": "TTL_MISMATCH",
+ }
+ for field, expected in scalar_expectations.items():
+ if document.get(field) != expected:
+ reject(mismatch_codes[field])
+
+ worktree = document.get("worktree")
+ if not isinstance(worktree, dict) or set(worktree) != {"id", "path"}:
+ reject("HANDOFF_MALFORMED")
+ if worktree.get("id") != values["worktree_id"]:
+ reject("WORKTREE_ID_MISMATCH")
+ if worktree.get("path") != str(values["project"]):
+ reject("WORKTREE_PATH_MISMATCH")
+
+ lock = document.get("lock")
+ if not isinstance(lock, dict) or set(lock) != {"identity", "tokenSha256"}:
+ reject("HANDOFF_MALFORMED")
+ if lock.get("identity") != values["lock_identity"]:
+ reject("LOCK_IDENTITY_MISMATCH")
+ if lock.get("tokenSha256") != values["lock_token_sha256"]:
+ reject("LOCK_TOKEN_MISMATCH")
+
+ manifest = document.get("imageManifest")
+ if not isinstance(manifest, dict) or set(manifest) != {"path", "sha256"}:
+ reject("HANDOFF_MALFORMED")
+ if manifest.get("path") != str(values["manifest"]):
+ reject("IMAGE_MANIFEST_PATH_MISMATCH")
+ recorded_digest = manifest.get("sha256")
+ if not isinstance(recorded_digest, str) or not SHA256.fullmatch(recorded_digest):
+ reject("HANDOFF_MALFORMED")
+ if recorded_digest != sha256_file(values["manifest"]):
+ reject("IMAGE_MANIFEST_DIGEST_MISMATCH")
+
+ issued_at = document.get("issuedAtEpoch")
+ expires_at = document.get("expiresAtEpoch")
+ if type(issued_at) is not int or type(expires_at) is not int:
+ reject("HANDOFF_MALFORMED")
+ if expires_at - issued_at != values["ttl"]:
+ reject("HANDOFF_MALFORMED")
+ if document.get("issuedAt") != iso8601(issued_at):
+ reject("HANDOFF_MALFORMED")
+ if document.get("expiresAt") != iso8601(expires_at):
+ reject("HANDOFF_MALFORMED")
+ now = int(time.time())
+ if issued_at > now + 30:
+ reject("HANDOFF_FROM_FUTURE")
+ if expires_at <= now:
+ reject("HANDOFF_STALE")
+ return document
+
+
+def fsync_directory(path):
+ descriptor = os.open(path, os.O_RDONLY)
+ try:
+ os.fsync(descriptor)
+ finally:
+ os.close(descriptor)
+
+
+def create(values):
+ if os.environ.get("T2_SETUP_HANDOFF_SETUP_COMPLETE", "") != "true":
+ reject("SETUP_INCOMPLETE")
+ ensure_private_directory(values["state_root"])
+ ensure_private_directory(values["state_root"] / values["worktree_id"])
+ ensure_private_directory(values["run_directory"])
+ if values["handoff"].exists() or values["consumed"].exists() or values["claim"].exists():
+ reject("HANDOFF_ALREADY_EXISTS")
+
+ document = expected_document(values, int(time.time()))
+ encoded = (json.dumps(document, sort_keys=True, separators=(",", ":")) + "\n").encode("utf-8")
+ descriptor, temporary_name = tempfile.mkstemp(
+ prefix=".setup-complete.", dir=values["run_directory"]
+ )
+ temporary = Path(temporary_name)
+ published = False
+ try:
+ os.fchmod(descriptor, 0o600)
+ with os.fdopen(descriptor, "wb") as handle:
+ handle.write(encoded)
+ handle.flush()
+ os.fsync(handle.fileno())
+ if values["handoff"].exists() or values["consumed"].exists() or values["claim"].exists():
+ reject("HANDOFF_ALREADY_EXISTS")
+ try:
+ os.link(temporary, values["handoff"], follow_symlinks=False)
+ except FileExistsError:
+ reject("HANDOFF_ALREADY_EXISTS")
+ published = True
+ os.unlink(temporary)
+ fsync_directory(values["run_directory"])
+ finally:
+ if not published and temporary.exists():
+ temporary.unlink()
+ print("T2_SETUP_HANDOFF_CREATED")
+
+
+def consume(values, verification_command):
+ if values["claim"].exists() or values["consumed"].exists():
+ reject("HANDOFF_REPLAYED")
+ validate_document(values, values["handoff"])
+
+ verification_environment = os.environ.copy()
+ verification_environment.pop("T2_LOCK_TOKEN", None)
+ verified = subprocess.run(
+ verification_command,
+ cwd=values["project"],
+ env=verification_environment,
+ check=False,
+ stdin=subprocess.DEVNULL,
+ )
+ if verified.returncode != 0:
+ reject("IMAGE_VERIFICATION_FAILED")
+
+ # Revalidate after the external verifier. Only then acquire the atomic
+ # one-shot claim. A crash after this point leaves a replay marker and can
+ # never authorize the optimization on a retry.
+ validate_document(values, values["handoff"])
+ try:
+ os.mkdir(values["claim"], 0o700)
+ except FileExistsError:
+ reject("HANDOFF_REPLAYED")
+ except OSError:
+ reject("HANDOFF_CONSUME_FAILED")
+
+ try:
+ if values["consumed"].exists():
+ reject("HANDOFF_REPLAYED")
+ os.link(values["handoff"], values["consumed"], follow_symlinks=False)
+ os.unlink(values["handoff"])
+ fsync_directory(values["run_directory"])
+ validate_document(values, values["consumed"])
+ except FileNotFoundError:
+ reject("HANDOFF_REPLAYED")
+ except FileExistsError:
+ reject("HANDOFF_REPLAYED")
+ except OSError:
+ reject("HANDOFF_CONSUME_FAILED")
+ print("T2_SETUP_HANDOFF_CONSUMED")
+
+
+def main():
+ action = sys.argv[1]
+ values = context()
+ if action == "create":
+ create(values)
+ return
+ verification_command = sys.argv[2:]
+ if not verification_command:
+ reject("VERIFY_COMMAND_REQUIRED")
+ consume(values, verification_command)
+
+
+try:
+ main()
+except Rejected as failure:
+ print(f"T2_SETUP_HANDOFF_REJECTED={failure.code}", file=sys.stderr)
+ sys.exit(1)
+except Exception:
+ # Do not echo exception values: environment or filesystem errors may carry
+ # sensitive local material. The caller retains/forces the full reconcile.
+ print("T2_SETUP_HANDOFF_REJECTED=INTERNAL_ERROR", file=sys.stderr)
+ sys.exit(1)
+PY
+}
+
+if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
+ set -euo pipefail
+ set +x
+ t2_setup_handoff_cli "$@"
+fi
diff --git a/scripts/minikube/t2-worktree-id.sh b/scripts/minikube/t2-worktree-id.sh
index e3805fbd2..622d596d5 100755
--- a/scripts/minikube/t2-worktree-id.sh
+++ b/scripts/minikube/t2-worktree-id.sh
@@ -1,6 +1,23 @@
#!/usr/bin/env bash
# Shared deterministic worktree identity for the local T2 mutation contract.
+t2_canonical_worktree_path() {
+ [[ -n "${1:-}" && -d "$1" ]] || return 1
+ (cd -- "$1" && pwd -P)
+}
+
t2_worktree_id() {
- printf '%s' "$1" | shasum | awk '{print $1}'
+ local canonical_path
+ canonical_path="$(t2_canonical_worktree_path "${1:-}")" || return 1
+ printf '%s' "${canonical_path}" | shasum | awk '{print $1}'
+}
+
+# Profile ownership is deliberately branch-scoped and HEAD-independent. The
+# NUL separator prevents ambiguous path/branch concatenations while preserving
+# the historical path-only worktree ID used by exact-head runtime markers.
+t2_profile_owner_id() {
+ local canonical_path branch="${2:-}"
+ canonical_path="$(t2_canonical_worktree_path "${1:-}")" || return 1
+ [[ -n "${branch}" && "${branch}" != *$'\n'* && "${branch}" != *$'\r'* && "${branch}" != *$'\t'* ]] || return 1
+ printf '%s\0%s' "${canonical_path}" "${branch}" | shasum | awk '{print $1}'
}
diff --git a/scripts/minikube/t2.sh b/scripts/minikube/t2.sh
index 21ff18c62..d89cd4b9d 100755
--- a/scripts/minikube/t2.sh
+++ b/scripts/minikube/t2.sh
@@ -6,6 +6,9 @@ set +x
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd -P)"
source "$SCRIPT_DIR/t2-common.sh"
+# shellcheck source=scripts/e2e/real-postgres-local-preflight.sh
+source "$T2_PROJECT_DIR/scripts/e2e/real-postgres-local-preflight.sh"
+T2_RUN_ID="${T2_RUN_ID:-}"
T2_SKIP_LOCK="$T2_SKIP_LOCK"
if [ -z "$T2_SKIP_LOCK" ]; then T2_SKIP_LOCK=false; fi
@@ -18,17 +21,23 @@ if [ -z "$T2_REQUIRE_PLAYWRIGHT" ]; then T2_REQUIRE_PLAYWRIGHT=false; fi
T2_T0_COMMAND="$T2_T0_COMMAND"
T2_PLAYWRIGHT_COMMAND="$T2_PLAYWRIGHT_COMMAND"
T2_HEALTHCHECK_COMMAND="$T2_HEALTHCHECK_COMMAND"
+T2_HEALTHCHECK_TIMEOUT_SECONDS="${T2_HEALTHCHECK_TIMEOUT_SECONDS:-120}"
+T2_DEADLINE_RUNNER="${T2_DEADLINE_RUNNER:-$SCRIPT_DIR/run-with-deadline.mjs}"
T2_PLAN_TMP=""
T2_T0_STATUS=NOT_RUN
T2_T1_STATUS=NOT_RUN
T2_T2_STATUS=NOT_RUN
T2_NP08_HCC_AUTHORIZATION_STATUS=NOT_RUN
+T2_HEALTH_STATUS=NOT_RUN
T2_PLAYWRIGHT_STATUS=NOT_RUN
+T2_HEALTHCHECK_REQUIRED=false
# A bootstrap or full reconcile must build the current worktree. Reusing a
# marker's ghcr coordinate would validate a release image rather than HEAD.
T2_BOOTSTRAP_IMAGE_SOURCE=local
T2_TMP_ROOT="$T2_TMP_ROOT"
if [ -z "$T2_TMP_ROOT" ]; then T2_TMP_ROOT=/tmp; fi
+T2_SETUP_HANDOFF_ROOT="${T2_SETUP_HANDOFF_ROOT:-$T2_PROJECT_DIR/.local-notes/infra/t2-setup-handoffs}"
+T2_SETUP_HANDOFF_TTL_SECONDS="${T2_SETUP_HANDOFF_TTL_SECONDS:-300}"
T2_T1_OUTPUT="$T2_TMP_ROOT/evenfire-t1.$$.out"
T2_FINAL_PREFLIGHT_OUTPUT="$T2_TMP_ROOT/evenfire-t2-final-preflight.$$.out"
export T2_PROJECT_DIR T2_PROFILE_ROOT T2_PROFILE_ENV T2_PORTS_ENV T2_CONTEXT T2_LOCK_TOKEN
@@ -51,14 +60,25 @@ cleanup_plan() {
fi
exit "$status"
}
-trap cleanup_plan EXIT
-
t2_lane_completed() {
local status="$1"
[ "$status" = PASS ]
}
+run_t1_local_preflight() {
+ [ "$T2_RUN_T1" = true ] || return 0
+ if ! real_pg_local_preflight "$T2_PROJECT_DIR" true; then
+ T2_NEXT_COMMAND='repair the reported local prerequisite before retrying T0/T1/T2'
+ t2_evidence_write local-preflight FAIL "$REAL_PG_PREFLIGHT_ERROR_CODE"
+ t2_fail "$REAL_PG_PREFLIGHT_ERROR_CODE" "$REAL_PG_PREFLIGHT_ERROR_MESSAGE"
+ return 1
+ fi
+ t2_evidence_write local-preflight PASS \
+ "Node/package/Docker prerequisites passed in ${REAL_PG_PREFLIGHT_DURATION_SECONDS}s; T1 workers=1"
+}
+
run_t0() {
+ local phase_started_seconds="$SECONDS"
if [ "$T2_RUN_T0" != true ]; then
T2_T0_STATUS=SKIPPED
t2_evidence_write T0 SKIPPED 'T2_RUN_T0=false; an exact-head T0 attestation is required before runtime-only certification'
@@ -66,18 +86,22 @@ run_t0() {
fi
printf '[minikube-t2] T0: shell syntax and contract checks\n'
git -C "$T2_PROJECT_DIR" diff --check "$T2_ORIGIN_DEV...$T2_HEAD"
- bash -n "$SCRIPT_DIR/t2-common.sh" "$SCRIPT_DIR/t2-preflight.sh" "$SCRIPT_DIR/t2.sh"
+ bash -n "$SCRIPT_DIR/t2-common.sh" "$SCRIPT_DIR/t2-preflight.sh" "$SCRIPT_DIR/t2.sh" \
+ "$SCRIPT_DIR/t2-setup-handoff.sh" \
+ "$T2_PROJECT_DIR/scripts/tests/test-minikube-t2-setup-handoff.sh"
T0_PROJECT_DIR="$T2_PROJECT_DIR" T0_ORIGIN_DEV="$T2_ORIGIN_DEV" T0_HEAD="$T2_HEAD" \
bash "$T2_PROJECT_DIR/scripts/minikube/t0.sh"
if [ -n "$T2_T0_COMMAND" ]; then
bash -c "$T2_T0_COMMAND"
fi
bash "$T2_PROJECT_DIR/scripts/tests/test-minikube-t2-contract.sh"
+ bash "$T2_PROJECT_DIR/scripts/tests/test-minikube-t2-setup-handoff.sh"
T2_T0_STATUS=PASS
- t2_evidence_write T0 PASS 'syntax, ShellCheck when available, affected package test/build/typecheck, contract, and diff checks passed'
+ t2_evidence_write T0 PASS "syntax, ShellCheck when available, affected package test/build/typecheck, contract, and diff checks passed; duration=$((SECONDS - phase_started_seconds))s"
}
run_preflight_plan() {
+ local phase_started_seconds="$SECONDS"
T2_PLAN_TMP="$(mktemp "$T2_TMP_ROOT/evenfire-t2-plan.XXXXXX")"
if ! T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" T2_PLAN_MODE=true T2_PLAN_FILE="$T2_PLAN_TMP" \
T2_PROJECT_DIR="$T2_PROJECT_DIR" MINIKUBE_PROFILE="$T2_PROFILE" \
@@ -107,6 +131,8 @@ PY
return 1
fi
printf '[minikube-t2] transition=%s reason=%s\n' "$T2_PLAN_STATE" "$T2_PLAN_REASON"
+ t2_evidence_write planner PASS \
+ "state=$T2_PLAN_STATE duration=$((SECONDS - phase_started_seconds))s"
}
run_pvc_reset_if_authorized() {
@@ -130,16 +156,25 @@ run_pvc_reset_if_authorized() {
}
run_bootstrap_or_reconcile() {
+ local phase_started_seconds="$SECONDS"
run_pvc_reset_if_authorized
case "$T2_PLAN_STATE" in
full-bootstrap)
printf '[minikube-t2] bootstrap: full setup, then PostgreSQL/migrations/roles checks\n'
if [ "$T2_RESET_PVC" = true ]; then
T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" \
+ T2_RUN_ID="$T2_RUN_ID" T2_SETUP_HANDOFF_REQUIRED=true \
+ T2_SETUP_HANDOFF_TRANSITION="$T2_PLAN_STATE" \
+ T2_SETUP_HANDOFF_ROOT="$T2_SETUP_HANDOFF_ROOT" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="$T2_SETUP_HANDOFF_TTL_SECONDS" \
MINIKUBE_PROFILE="$T2_PROFILE" IMAGE_SOURCE="$T2_BOOTSTRAP_IMAGE_SOURCE" \
REUSE_DB=false CONTROL_DB_RESET_PVC_UID="$T2_EXPECTED_PVC_UID" make minikube-setup
else
T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" \
+ T2_RUN_ID="$T2_RUN_ID" T2_SETUP_HANDOFF_REQUIRED=true \
+ T2_SETUP_HANDOFF_TRANSITION="$T2_PLAN_STATE" \
+ T2_SETUP_HANDOFF_ROOT="$T2_SETUP_HANDOFF_ROOT" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="$T2_SETUP_HANDOFF_TTL_SECONDS" \
MINIKUBE_PROFILE="$T2_PROFILE" IMAGE_SOURCE="$T2_BOOTSTRAP_IMAGE_SOURCE" \
REUSE_DB=true make minikube-setup
fi
@@ -148,10 +183,18 @@ run_bootstrap_or_reconcile() {
printf '[minikube-t2] full reconcile: infrastructure input changed\n'
if [ "$T2_RESET_PVC" = true ]; then
T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" \
+ T2_RUN_ID="$T2_RUN_ID" T2_SETUP_HANDOFF_REQUIRED=true \
+ T2_SETUP_HANDOFF_TRANSITION="$T2_PLAN_STATE" \
+ T2_SETUP_HANDOFF_ROOT="$T2_SETUP_HANDOFF_ROOT" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="$T2_SETUP_HANDOFF_TTL_SECONDS" \
MINIKUBE_PROFILE="$T2_PROFILE" IMAGE_SOURCE="$T2_BOOTSTRAP_IMAGE_SOURCE" \
REUSE_DB=false CONTROL_DB_RESET_PVC_UID="$T2_EXPECTED_PVC_UID" make minikube-setup
else
T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" \
+ T2_RUN_ID="$T2_RUN_ID" T2_SETUP_HANDOFF_REQUIRED=true \
+ T2_SETUP_HANDOFF_TRANSITION="$T2_PLAN_STATE" \
+ T2_SETUP_HANDOFF_ROOT="$T2_SETUP_HANDOFF_ROOT" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="$T2_SETUP_HANDOFF_TTL_SECONDS" \
MINIKUBE_PROFILE="$T2_PROFILE" IMAGE_SOURCE="$T2_BOOTSTRAP_IMAGE_SOURCE" \
REUSE_DB=true make minikube-setup
fi
@@ -169,7 +212,8 @@ run_bootstrap_or_reconcile() {
return 1
;;
esac
- t2_evidence_write transition PASS "$T2_PLAN_STATE"
+ t2_evidence_write transition PASS \
+ "$T2_PLAN_STATE duration=$((SECONDS - phase_started_seconds))s"
}
run_targeted_sync() {
@@ -182,17 +226,30 @@ run_targeted_sync() {
}
run_pre_gate() {
+ local phase_started_seconds="$SECONDS"
if [ "$T2_PLAN_STATE" = already-synced ]; then
printf '[minikube-t2] skipping pre-gate-sync; marker already matches HEAD\n'
+ t2_evidence_write pre-gate-sync SKIPPED 'marker already matches HEAD; duration=0s'
return 0
fi
printf '[minikube-t2] pre-gate-sync after bootstrap/reconcile\n'
+ local setup_handoff_expected=false
+ case "$T2_PLAN_STATE" in
+ full-bootstrap|full-reconcile) setup_handoff_expected=true ;;
+ esac
T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" \
+ T2_RUN_ID="$T2_RUN_ID" T2_SETUP_HANDOFF_EXPECTED="$setup_handoff_expected" \
+ T2_SETUP_HANDOFF_TRANSITION="$T2_PLAN_STATE" \
+ T2_SETUP_HANDOFF_ROOT="$T2_SETUP_HANDOFF_ROOT" \
+ T2_SETUP_HANDOFF_TTL_SECONDS="$T2_SETUP_HANDOFF_TTL_SECONDS" \
MINIKUBE_PROFILE="$T2_PROFILE" CONTROL_API_REAL_PG_CONTEXT="$T2_CONTEXT" \
make minikube-pre-gate-sync GATE=minikube-t2 ARGS='--skip-port-forwards'
+ t2_evidence_write pre-gate-sync PASS \
+ "duration=$((SECONDS - phase_started_seconds))s setupHandoffExpected=$setup_handoff_expected"
}
run_t1() {
+ local phase_started_seconds="$SECONDS"
if [ "$T2_RUN_T1" != true ]; then
T2_T1_STATUS=SKIPPED
t2_evidence_write T1 SKIPPED 'T2_RUN_T1=false; an exact-head T1 attestation is required before runtime-only certification'
@@ -202,33 +259,40 @@ run_t1() {
if T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" CONTROL_API_REAL_PG_CONTEXT="$T2_CONTEXT" MINIKUBE_PROFILE="$T2_PROFILE" \
bash "$T2_PROJECT_DIR/scripts/e2e/minikube-real-postgres.sh" >"$T2_T1_OUTPUT" 2>&1; then
cat "$T2_T1_OUTPUT"
- local t1_status t1_tests t1_passed t1_pending t1_evidence
+ local t1_status t1_tests t1_passed t1_pending t1_expected_files t1_reported_files t1_duration t1_evidence
t1_status="$(awk -F= '$1 == "T1_STATUS" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
t1_tests="$(awk -F= '$1 == "T1_TESTS" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
t1_passed="$(awk -F= '$1 == "T1_PASSED_TESTS" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
t1_pending="$(awk -F= '$1 == "T1_PENDING_TESTS" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
+ t1_expected_files="$(awk -F= '$1 == "T1_EXPECTED_FILES" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
+ t1_reported_files="$(awk -F= '$1 == "T1_REPORTED_FILES" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
+ t1_duration="$(awk -F= '$1 == "T1_DURATION_SECONDS" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
t1_evidence="$(awk -F= '$1 == "T1_EVIDENCE" { value=$2 } END { print value }' "$T2_T1_OUTPUT")"
if [ "$t1_status" != PASS ] || ! [[ "$t1_tests" =~ ^[1-9][0-9]*$ ]] || \
! [[ "$t1_passed" =~ ^[1-9][0-9]*$ ]] || [ "$t1_passed" -ne "$t1_tests" ] || \
- [ "$t1_pending" != 0 ] || [ -z "$t1_evidence" ]; then
+ [ "$t1_pending" != 0 ] || ! [[ "$t1_expected_files" =~ ^[1-9][0-9]*$ ]] || \
+ ! [[ "$t1_reported_files" =~ ^[1-9][0-9]*$ ]] || \
+ [ "$t1_reported_files" -ne "$t1_expected_files" ] || \
+ ! [[ "$t1_duration" =~ ^[0-9]+$ ]] || [ -z "$t1_evidence" ]; then
T2_T1_STATUS=FAIL
t2_evidence_write T1 FAIL 'Real PostgreSQL lane exited zero but did not emit a complete green T1 result'
- T2_NEXT_COMMAND='repair the T1 result contract; it must emit PASS, positive total/passed counts, zero pending tests, and an evidence path'
+ T2_NEXT_COMMAND='repair the T1 result contract; it must prove the exact file set, positive test counts, zero pending tests, duration, and an evidence path'
t2_fail REAL_PG_SUITE_FAILED 'T1 result was incomplete despite a zero process exit'
return 1
fi
T2_T1_STATUS=PASS
- T2_T1_COUNTS="$(awk -F= '/^T1_TESTS=|^T1_PASSED_TESTS=|^T1_PENDING_TESTS=/{printf "%s%s", (n++ ? "," : ""), $0}' "$T2_T1_OUTPUT")"
- t2_evidence_write T1 PASS "Real PostgreSQL suites executed with no skips; $T2_T1_COUNTS"
+ T2_T1_COUNTS="$(awk -F= '/^T1_TESTS=|^T1_PASSED_TESTS=|^T1_PENDING_TESTS=|^T1_EXPECTED_FILES=|^T1_REPORTED_FILES=/{printf "%s%s", (n++ ? "," : ""), $0}' "$T2_T1_OUTPUT")"
+ t2_evidence_write T1 PASS "Real PostgreSQL suites executed with exact file identity and no skips; $T2_T1_COUNTS; duration=${t1_duration}s; orchestratorDuration=$((SECONDS - phase_started_seconds))s"
else
cat "$T2_T1_OUTPUT" >&2 || true
T2_T1_STATUS=FAIL
- t2_evidence_write T1 FAIL 'Real PostgreSQL lane failed'
+ t2_evidence_write T1 FAIL "Real PostgreSQL lane failed after $((SECONDS - phase_started_seconds))s"
return 1
fi
}
run_final_preflight() {
+ local phase_started_seconds="$SECONDS"
printf '[minikube-t2] final exact-head readiness preflight\n'
# Fail-loud: a stale marker or missing bootstrap must not count as T2.
if ! T2_SKIP_LOCK=true T2_LOCK_TOKEN="$T2_LOCK_TOKEN" T2_PLAN_MODE=false T2_PLAN_FILE="$T2_PLAN_TMP" \
@@ -255,11 +319,12 @@ PY
fi
t2_marker_check
T2_T2_STATUS=PASS
- t2_evidence_write T2 PASS 'exact marker, image, PostgreSQL, namespace, Service, and deployment readiness passed'
+ t2_evidence_write T2 PASS \
+ "exact marker, image, PostgreSQL, namespace, Service, and deployment readiness passed; duration=$((SECONDS - phase_started_seconds))s"
}
run_np08_hcc_authorization() {
- local log_file
+ local log_file phase_started_seconds="$SECONDS"
log_file="$T2_EVIDENCE_DIR/logs/np08-hcc-authorization.log"
printf '[minikube-t2] NP-08: deployed Host-to-HCC authorization journey\n'
if MINIKUBE_PROFILE="$T2_PROFILE" CLERUM_PROFILE_PORTS_ENV="$T2_PORTS_ENV" \
@@ -268,34 +333,81 @@ run_np08_hcc_authorization() {
cat "$log_file"
T2_NP08_HCC_AUTHORIZATION_STATUS=PASS
t2_evidence_write NP08_HCC_AUTHORIZATION PASS \
- 'deployed same-Context, cross-Context, caller-token, and credential-disclosure checks passed'
+ "deployed same-Context, cross-Context, caller-token, and credential-disclosure checks passed; duration=$((SECONDS - phase_started_seconds))s"
return 0
fi
cat "$log_file" >&2 || true
T2_NP08_HCC_AUTHORIZATION_STATUS=FAIL
t2_evidence_write NP08_HCC_AUTHORIZATION FAIL \
- 'deployed Host-to-HCC authorization journey failed; see the secret-safe local log'
- T2_NEXT_COMMAND='repair the first NP-08 authorization failure, verify cleanup, then re-run T2 on the same HEAD'
+ "deployed Host-to-HCC authorization journey failed after $((SECONDS - phase_started_seconds))s; see the secret-safe local log"
+ T2_NEXT_COMMAND="MINIKUBE_PROFILE=$T2_PROFILE CONTROL_API_REAL_PG_CONTEXT=$T2_CONTEXT make minikube-t2-runtime"
t2_fail NP08_HCC_AUTHORIZATION_FAILED 'deployed Host-to-HCC authorization journey failed'
}
+validate_healthcheck_contract() {
+ T2_HEALTHCHECK_REQUIRED=false
+ if ! [[ "$T2_HEALTHCHECK_TIMEOUT_SECONDS" =~ ^[1-9][0-9]*$ ]] || \
+ [ "$T2_HEALTHCHECK_TIMEOUT_SECONDS" -gt 900 ]; then
+ T2_NEXT_COMMAND='set T2_HEALTHCHECK_TIMEOUT_SECONDS to an integer from 1 to 900, then re-run T2'
+ t2_fail DEVELOPMENT_SCOPE_REQUIRED \
+ 'T2_HEALTHCHECK_TIMEOUT_SECONDS must be an integer from 1 to 900'
+ return 1
+ fi
+
+ if [ "$T2_PLAN_STATE" = targeted-sync ]; then
+ T2_HEALTHCHECK_REQUIRED=true
+ if [ -z "$T2_HEALTHCHECK_COMMAND" ]; then
+ T2_NEXT_COMMAND='set T2_HEALTHCHECK_COMMAND to the affected service user-facing health journey, then re-run T2'
+ t2_fail PROFILE_UNHEALTHY \
+ 'targeted sync requires a profile-owned user-facing health command before mutation'
+ return 1
+ fi
+ fi
+
+ [ -n "$T2_HEALTHCHECK_COMMAND" ] || return 0
+ if ! command -v node >/dev/null 2>&1 || [ ! -f "$T2_DEADLINE_RUNNER" ]; then
+ T2_NEXT_COMMAND='restore Node.js and scripts/minikube/run-with-deadline.mjs, then re-run T2'
+ t2_fail LOCAL_DEPENDENCY_MISSING \
+ 'a bounded user-facing health command requires the repository deadline runner'
+ return 1
+ fi
+}
+
run_healthcheck_if_requested() {
+ local phase_started_seconds="$SECONDS" health_status=0
if [ -z "$T2_HEALTHCHECK_COMMAND" ]; then
+ T2_HEALTH_STATUS=NOT_RUN
t2_evidence_write Health NOT_RUN 'no profile-owned user-facing health command was supplied'
return 0
fi
- if bash -c "$T2_HEALTHCHECK_COMMAND"; then
- t2_evidence_write Health PASS 'profile-owned user-facing health command passed'
+ node "$T2_DEADLINE_RUNNER" \
+ --timeout-seconds "$T2_HEALTHCHECK_TIMEOUT_SECONDS" \
+ --heartbeat-seconds 20 --kill-grace-seconds 5 \
+ --label t2-user-facing-health -- \
+ bash -c "$T2_HEALTHCHECK_COMMAND" || health_status=$?
+ if [ "$health_status" -eq 0 ]; then
+ T2_HEALTH_STATUS=PASS
+ t2_evidence_write Health PASS \
+ "profile-owned user-facing health command passed; required=$T2_HEALTHCHECK_REQUIRED duration=$((SECONDS - phase_started_seconds))s"
else
- t2_evidence_write Health FAIL 'profile-owned user-facing health command failed'
- T2_NEXT_COMMAND='repair the first failing user-facing health check, then re-run T2 on the same HEAD'
- t2_fail PROFILE_UNHEALTHY 'user-facing health check failed'
+ T2_HEALTH_STATUS=FAIL
+ t2_evidence_write Health FAIL \
+ "profile-owned user-facing health command failed; exit=$health_status duration=$((SECONDS - phase_started_seconds))s"
+ T2_NEXT_COMMAND="repair the health failure, then run MINIKUBE_PROFILE=$T2_PROFILE CONTROL_API_REAL_PG_CONTEXT=$T2_CONTEXT make minikube-t2-runtime"
+ if [ "$health_status" -eq 124 ]; then
+ t2_fail PROFILE_UNHEALTHY \
+ "user-facing health check exceeded ${T2_HEALTHCHECK_TIMEOUT_SECONDS}s"
+ else
+ t2_fail PROFILE_UNHEALTHY \
+ "user-facing health check failed with exit $health_status"
+ fi
return 1
fi
}
run_playwright_if_requested() {
+ local phase_started_seconds="$SECONDS"
if [ -z "$T2_PLAYWRIGHT_COMMAND" ]; then
T2_PLAYWRIGHT_STATUS=NOT_RUN
t2_evidence_write Playwright NOT_RUN 'no applicable Control UI/Desktop command was supplied'
@@ -308,15 +420,19 @@ run_playwright_if_requested() {
fi
if bash -c "$T2_PLAYWRIGHT_COMMAND"; then
T2_PLAYWRIGHT_STATUS=PASS
- t2_evidence_write Playwright PASS 'user-visible journey command passed'
+ t2_evidence_write Playwright PASS \
+ "user-visible journey command passed; duration=$((SECONDS - phase_started_seconds))s"
else
T2_PLAYWRIGHT_STATUS=FAIL
t2_evidence_write Playwright FAIL 'user-visible journey command failed'
+ T2_NEXT_COMMAND="repair the Playwright journey, then run MINIKUBE_PROFILE=$T2_PROFILE CONTROL_API_REAL_PG_CONTEXT=$T2_CONTEXT make minikube-t2-runtime"
+ t2_fail PLAYWRIGHT_FAILED 'user-visible journey command failed'
return 1
fi
}
main() {
+ trap cleanup_plan EXIT
t2_require_commands
t2_repo_metadata
t2_profile_scope
@@ -327,6 +443,7 @@ main() {
t2_mutation_lock
t2_evidence_init
+ run_t1_local_preflight
run_t0
if [ "$T2_RUN_T0" = true ] && ! t2_lane_completed "$T2_T0_STATUS"; then
T2_NEXT_COMMAND='re-run make minikube-t2 with T2_RUN_T0=true so the static lane is executed'
@@ -334,6 +451,7 @@ main() {
return 1
fi
run_preflight_plan
+ validate_healthcheck_contract
if { [ "$T2_RUN_T0" != true ] || [ "$T2_RUN_T1" != true ]; } && [ "$T2_PLAN_STATE" != already-synced ]; then
T2_NEXT_COMMAND='run make minikube-t2 so bootstrap/reconcile and T0/T1 execute on this HEAD'
t2_fail DEVELOPMENT_SCOPE_REQUIRED 'T2-only mode requires an already-synced exact-head profile'
@@ -371,14 +489,17 @@ main() {
run_healthcheck_if_requested
run_playwright_if_requested
- t2_evidence_write complete PASS "T0=$T2_T0_STATUS T1=$T2_T1_STATUS T2=$T2_T2_STATUS NP08_HCC_AUTHORIZATION=$T2_NP08_HCC_AUTHORIZATION_STATUS Playwright=$T2_PLAYWRIGHT_STATUS"
+ t2_evidence_write complete PASS "T0=$T2_T0_STATUS T1=$T2_T1_STATUS T2=$T2_T2_STATUS NP08_HCC_AUTHORIZATION=$T2_NP08_HCC_AUTHORIZATION_STATUS Health=$T2_HEALTH_STATUS Playwright=$T2_PLAYWRIGHT_STATUS"
printf 'MINIKUBE_T2_PASS\n'
printf 'T0=%s\n' "$T2_T0_STATUS"
printf 'T1=%s\n' "$T2_T1_STATUS"
printf 'T2=%s\n' "$T2_T2_STATUS"
printf 'NP08_HCC_AUTHORIZATION=%s\n' "$T2_NP08_HCC_AUTHORIZATION_STATUS"
+ printf 'Health=%s\n' "$T2_HEALTH_STATUS"
printf 'Playwright=%s\n' "$T2_PLAYWRIGHT_STATUS"
printf 'evidence=%s\n' "$T2_EVIDENCE_FILE"
}
-main "$@"
+if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
+ main "$@"
+fi
diff --git a/scripts/tests/test-branch-profile-port-guard.sh b/scripts/tests/test-branch-profile-port-guard.sh
index de614b3b8..266bea466 100755
--- a/scripts/tests/test-branch-profile-port-guard.sh
+++ b/scripts/tests/test-branch-profile-port-guard.sh
@@ -3,11 +3,28 @@ set -euo pipefail
ROOT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)"
source "${ROOT_DIR}/scripts/e2e/e2e-lib.sh"
+OWNER_SCRIPT="${ROOT_DIR}/scripts/minikube/profile-owner.sh"
tmp_dir="$(mktemp -d)"
trap 'rm -rf "$tmp_dir"' EXIT
-context='clerum-codex-port-guard-deadbeef'
+repo_dir="${tmp_dir}/worktree"
+mkdir -p "${repo_dir}"
+identity_a="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_dir}" \
+ --branch fix/port-guard --created-head aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa)"
+identity_b="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_dir}" \
+ --branch fix/port-guard --created-head bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb)"
+context="$(awk -F= '$1 == "PROFILE" { print $2; exit }' <<<"${identity_a}")"
+context_after_head="$(awk -F= '$1 == "PROFILE" { print $2; exit }' <<<"${identity_b}")"
+[[ "${context}" == "${context_after_head}" ]] || {
+ echo 'FAIL: profile context changed when HEAD advanced' >&2
+ exit 1
+}
+is_branch_scoped_e2e_context "${context}" || {
+ echo 'FAIL: stable owner-derived profile is not recognized as branch-scoped' >&2
+ exit 1
+}
+
ports_env="${tmp_dir}/ports.env"
cat >"${ports_env}" <<'EOF_PORTS'
CONTROL_UI_URL=http://127.0.0.1:36148
@@ -21,7 +38,14 @@ export CONTROL_API_BASE_URL=http://127.0.0.1:36238
export EXTERNAL_REST_API_BASE_URL=http://127.0.0.1:36239
export RPC_PROXY_BASE_URL=http://127.0.0.1:36242
+ports_hash_before="$(shasum "${ports_env}" | awk '{print $1}')"
require_branch_profile_urls "$context" "$ports_env"
+require_branch_profile_urls "$context_after_head" "$ports_env"
+ports_hash_after="$(shasum "${ports_env}" | awk '{print $1}')"
+[[ "${ports_hash_before}" == "${ports_hash_after}" ]] || {
+ echo 'FAIL: branch profile URL validation rewrote persisted ports.env' >&2
+ exit 1
+}
CONTROL_API_BASE_URL=http://127.0.0.1:36239
if require_branch_profile_urls "$context" "$ports_env" >/dev/null 2>&1; then
diff --git a/scripts/tests/test-gfs-real-pg-minikube-gate.sh b/scripts/tests/test-gfs-real-pg-minikube-gate.sh
index 2105a6e44..283811476 100755
--- a/scripts/tests/test-gfs-real-pg-minikube-gate.sh
+++ b/scripts/tests/test-gfs-real-pg-minikube-gate.sh
@@ -14,7 +14,11 @@ grep -Fq 't2_marker_check' "${SCRIPT}"
grep -Fq 't2_mutation_lock' "${SCRIPT}"
grep -Fq 'rollout status deployment/control-postgres' "${SCRIPT}"
grep -Fq 'get secret control-postgres' "${SCRIPT}"
-grep -Fq 'PROFILE_PG_FORWARD_PID' "${SCRIPT}"
+grep -Fq 'PROFILE_PG_FORWARD_RECORD' "${SCRIPT}"
+grep -Fq 'pf_owner_record_process "${PROFILE_PG_FORWARD_RECORD}"' "${SCRIPT}"
+grep -Fq 'pf_owner_cleanup_record "${PROFILE_PG_FORWARD_RECORD}"' "${SCRIPT}"
+grep -Fq 'docker_cli_env_prepare false' "${SCRIPT}"
+grep -Fq 'docker_cli_run_public gfs-t1-postgres-run' "${SCRIPT}"
grep -Fq 'docker run -d --rm' "${SCRIPT}"
grep -Fq 'ISOLATED_PG_IMAGE' "${SCRIPT}"
grep -Fq 'pending_files' "${SCRIPT}"
@@ -27,6 +31,17 @@ grep -Fq 'required gfs-controller-db Secret is missing or unreadable' "${SCRIPT}
grep -Fq 'GFS_RESTORE_ACTIVE_NOLOGIN=true' "${SCRIPT}"
grep -Fq 'failed to restore branch-profile GFS credentials' "${SCRIPT}"
grep -Fq 'docker rm -f' "${SCRIPT}"
+awk '
+ /^[[:space:]]*docker (run|exec|rm) / {
+ if (previous !~ /docker_cli_run_public/ && before_previous !~ /docker_cli_run_public/) {
+ exit 1
+ }
+ }
+ { before_previous = previous; previous = $0 }
+' "${SCRIPT}" || {
+ echo 'FAIL: standalone GFS lane still runs ambient or unbounded Docker commands' >&2
+ exit 1
+}
if grep -Fq 'docker rm -f "${ISOLATED_CONTAINER}" >/dev/null 2>&1 || true' "${SCRIPT}"; then
echo 'FAIL: standalone GFS cleanup suppresses Docker removal failure' >&2
exit 1
diff --git a/scripts/tests/test-minikube-build-images-hardening.sh b/scripts/tests/test-minikube-build-images-hardening.sh
new file mode 100755
index 000000000..8d10e3134
--- /dev/null
+++ b/scripts/tests/test-minikube-build-images-hardening.sh
@@ -0,0 +1,509 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+TMP_DIR="$(mktemp -d)"
+trap 'rm -rf -- "$TMP_DIR"' EXIT
+
+BIN_DIR="$TMP_DIR/bin"
+FIXTURE="$TMP_DIR/repo"
+RUNTIME_TMP="$TMP_DIR/runtime-tmp"
+LOCK_ROOT="$TMP_DIR/locks"
+MINIKUBE_LOG="$TMP_DIR/minikube.log"
+KUBECTL_LOG="$TMP_DIR/kubectl.log"
+DOCKER_LOG="$TMP_DIR/docker.log"
+DESCENDANT_PID_FILE="$TMP_DIR/descendant.pid"
+PROFILE=hardening-profile
+LEASE_VALUE=fixture-value
+LEASE_ENV_KEY=T2_LOCK_TOKEN
+FAIL=0
+
+pass() { printf 'PASS: %s\n' "$1"; }
+fail() { printf 'FAIL: %s\n' "$1" >&2; FAIL=1; }
+
+mkdir -p "$BIN_DIR" "$RUNTIME_TMP" "$TMP_DIR/home"
+
+cat >"$BIN_DIR/fake-runtime-lib.sh" <<'STUB'
+fake_run_mode() {
+ local mode="$1" operation="$2" descendant
+ case "$mode" in
+ success) return 0 ;;
+ hang)
+ (
+ trap '' TERM INT
+ while true; do sleep 1; done
+ ) &
+ descendant=$!
+ printf '%s:%s\n' "$operation" "$descendant" >"${DESCENDANT_PID_FILE:?}"
+ wait "$descendant"
+ ;;
+ death)
+ kill -TERM "$BASHPID"
+ sleep 5
+ ;;
+ exit-*) exit "${mode#exit-}" ;;
+ *) printf 'unknown fake mode: %s\n' "$mode" >&2; exit 98 ;;
+ esac
+}
+STUB
+
+cat >"$BIN_DIR/minikube" <<'STUB'
+#!/usr/bin/env bash
+set -euo pipefail
+# shellcheck source=/dev/null
+source "${FAKE_RUNTIME_LIB:?}"
+printf '%s\n' "$*" >>"${MINIKUBE_LOG:?}"
+
+case "$*" in
+ *" status")
+ fake_run_mode "${FAKE_MINIKUBE_STATUS_MODE:-success}" minikube-status
+ ;;
+ *" docker-env"*)
+ fake_run_mode "${FAKE_MINIKUBE_DOCKER_ENV_MODE:-success}" minikube-docker-env
+ printf 'export MINIKUBE_ACTIVE_DOCKERD="fixture"\n'
+ ;;
+ *" image ls"*)
+ fake_run_mode "${FAKE_MINIKUBE_INVENTORY_MODE:-success}" minikube-image-inventory
+ printf '%s' "${FAKE_MINIKUBE_INVENTORY_JSON:-[]}"
+ ;;
+ *" image load"*)
+ fake_run_mode "${FAKE_MINIKUBE_LOAD_MODE:-success}" minikube-image-load
+ ;;
+esac
+STUB
+
+cat >"$BIN_DIR/kubectl" <<'STUB'
+#!/usr/bin/env bash
+set -euo pipefail
+# shellcheck source=/dev/null
+source "${FAKE_RUNTIME_LIB:?}"
+printf '%s\n' "$*" >>"${KUBECTL_LOG:?}"
+if [[ "$*" == *"get nodes"* ]]; then
+ fake_run_mode "${FAKE_KUBECTL_NODES_MODE:-success}" kubectl-get-nodes
+ printf 'fixture-node Ready control-plane 1d v1.30.0\n'
+fi
+STUB
+
+cat >"$BIN_DIR/docker" <<'STUB'
+#!/usr/bin/env bash
+set -euo pipefail
+printf '%s\n' "$*" >>"${DOCKER_LOG:?}"
+
+if [[ "${1:-}" == context && "${2:-}" == inspect ]]; then
+ if [[ "$*" == *TLSMaterial* ]]; then
+ printf 'unix:///tmp/evenfire-fake-docker.sock\tfalse\t{}\n'
+ else
+ printf 'unix:///tmp/evenfire-fake-docker.sock\n'
+ fi
+ exit 0
+fi
+if [[ "${1:-}" == buildx && "${2:-}" == version ]]; then
+ printf 'github.com/docker/buildx fake\n'
+ exit 0
+fi
+if [[ "${1:-}" == info ]]; then
+ exit 0
+fi
+if [[ "${1:-}" == images && "${2:-}" == -q ]]; then
+ if [[ -z "${FAKE_DOCKER_QUERY_MATCH:-}" || "$*" == *"$FAKE_DOCKER_QUERY_MATCH"* ]]; then
+ query_status="${FAKE_DOCKER_QUERY_STATUS:-0}"
+ [[ "$query_status" -eq 0 ]] || exit "$query_status"
+ printf '%s' "${FAKE_DOCKER_QUERY_OUTPUT:-}"
+ else
+ printf '%s' "${FAKE_DOCKER_DEFAULT_QUERY_OUTPUT:-}"
+ fi
+ exit 0
+fi
+if [[ "${1:-}" == pull ]]; then
+ exit "${FAKE_DOCKER_PULL_STATUS:-0}"
+fi
+if [[ "${1:-}" == image && "${2:-}" == inspect ]]; then
+ printf 'legacy image inspect must not determine absence\n' >&2
+ exit 88
+fi
+if [[ "${1:-}" == inspect ]]; then
+ printf 'sha256:deadbeef0000cafedeadbeef0000cafedeadbeef\n'
+ exit 0
+fi
+exit 0
+STUB
+
+cat >"$BIN_DIR/docker-buildx" <<'STUB'
+#!/usr/bin/env bash
+printf 'github.com/docker/buildx fake\n'
+STUB
+
+chmod +x "$BIN_DIR/minikube" "$BIN_DIR/kubectl" "$BIN_DIR/docker" \
+ "$BIN_DIR/docker-buildx"
+
+prepare_fixture() {
+ mkdir -p "$FIXTURE/scripts/minikube" "$FIXTURE/scripts/release" \
+ "$FIXTURE/deploy/minikube" "$FIXTURE/control-api"
+ cp \
+ "$ROOT/scripts/minikube/build-images.sh" \
+ "$ROOT/scripts/minikube/docker-cli-env.sh" \
+ "$ROOT/scripts/minikube/image-mode.sh" \
+ "$ROOT/scripts/minikube/port-forward-owner.sh" \
+ "$ROOT/scripts/minikube/profile-owner.sh" \
+ "$ROOT/scripts/minikube/profile-readiness.sh" \
+ "$ROOT/scripts/minikube/require-t2-mutation-lock.sh" \
+ "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ "$ROOT/scripts/minikube/t2-common.sh" \
+ "$ROOT/scripts/minikube/t2-worktree-id.sh" \
+ "$FIXTURE/scripts/minikube/"
+ cp "$ROOT/scripts/release/images-manifest.mjs" "$FIXTURE/scripts/release/"
+ cp "$ROOT/deploy/images.json" "$FIXTURE/deploy/images.json"
+ printf 'FROM scratch\n' >"$FIXTURE/control-api/Dockerfile"
+ chmod +x "$FIXTURE/scripts/minikube/build-images.sh" \
+ "$FIXTURE/scripts/minikube/require-t2-mutation-lock.sh" \
+ "$FIXTURE/scripts/minikube/run-with-deadline.mjs"
+
+ git -C "$FIXTURE" init -q
+ git -C "$FIXTURE" config user.email [email protected]
+ git -C "$FIXTURE" config user.name hardening-test
+ git -C "$FIXTURE" add .
+ git -C "$FIXTURE" commit -qm fixture
+}
+
+prepare_lease() {
+ local branch head worktree_id lock_key lock_dir
+ branch="$(git -C "$FIXTURE" branch --show-current)"
+ head="$(git -C "$FIXTURE" rev-parse --verify HEAD)"
+ worktree_id="$(printf '%s' "$FIXTURE" | shasum | awk '{print $1}')"
+ lock_key="$(printf '%s\0%s\0%s\0%s\0%s' \
+ "$FIXTURE" "$branch" "$head" "$PROFILE" "$PROFILE" | shasum | awk '{print $1}')"
+ lock_dir="$LOCK_ROOT/$PROFILE.lock"
+ mkdir -p "$lock_dir"
+ {
+ printf 'REPOSITORY=%s\n' "$FIXTURE"
+ printf 'BRANCH=%s\n' "$branch"
+ printf 'HEAD=%s\n' "$head"
+ printf 'PROFILE=%s\n' "$PROFILE"
+ printf 'CONTEXT=%s\n' "$PROFILE"
+ printf 'WORKTREE_ID=%s\n' "$worktree_id"
+ printf 'LOCK_KEY=%s\n' "$lock_key"
+ printf '%s=%s\n' TOKEN "$LEASE_VALUE"
+ printf 'PID=%s\n' "$$"
+ printf 'PROCESS_START=unavailable\n'
+ } >"$lock_dir/owner.env"
+}
+
+prepare_fixture
+FIXTURE="$(cd "$FIXTURE" && pwd -P)"
+BUILD_SCRIPT="$FIXTURE/scripts/minikube/build-images.sh"
+prepare_lease
+
+clear_runtime_state() {
+ : >"$MINIKUBE_LOG"
+ : >"$KUBECTL_LOG"
+ : >"$DOCKER_LOG"
+ rm -f -- "$DESCENDANT_PID_FILE"
+}
+
+runtime_logs_are_empty() {
+ [[ ! -s "$MINIKUBE_LOG" && ! -s "$KUBECTL_LOG" && ! -s "$DOCKER_LOG" ]]
+}
+
+run_build() {
+ local output="$1" lease_mode="$2"
+ shift 2
+ local -a lease_env=()
+ case "$lease_mode" in
+ valid)
+ lease_env=(
+ "T2_PROJECT_DIR=$FIXTURE"
+ "T2_PROFILE=$PROFILE"
+ "T2_CONTEXT=$PROFILE"
+ "${LEASE_ENV_KEY}=$LEASE_VALUE"
+ )
+ ;;
+ wrong-value)
+ lease_env=(
+ "T2_PROJECT_DIR=$FIXTURE"
+ "T2_PROFILE=$PROFILE"
+ "T2_CONTEXT=$PROFILE"
+ "${LEASE_ENV_KEY}=wrong-value"
+ )
+ ;;
+ wrong-context)
+ lease_env=(
+ "T2_PROJECT_DIR=$FIXTURE"
+ "T2_PROFILE=$PROFILE"
+ T2_CONTEXT=foreign-context
+ "${LEASE_ENV_KEY}=$LEASE_VALUE"
+ )
+ ;;
+ missing|verify) ;;
+ *) printf 'unknown lease mode: %s\n' "$lease_mode" >&2; return 99 ;;
+ esac
+
+ /usr/bin/env -i \
+ "HOME=$TMP_DIR/home" \
+ "PATH=$BIN_DIR:$PATH" \
+ "TMPDIR=$RUNTIME_TMP" \
+ LC_ALL=C \
+ "MINIKUBE_PROFILE=$PROFILE" \
+ "CONTROL_API_REAL_PG_CONTEXT=$PROFILE" \
+ "T2_LOCK_ROOT=$LOCK_ROOT" \
+ "MINIKUBE_LOG=$MINIKUBE_LOG" \
+ "KUBECTL_LOG=$KUBECTL_LOG" \
+ "DOCKER_LOG=$DOCKER_LOG" \
+ "DESCENDANT_PID_FILE=$DESCENDANT_PID_FILE" \
+ "FAKE_RUNTIME_LIB=$BIN_DIR/fake-runtime-lib.sh" \
+ "MINIKUBE_DOCKER_BUILDX_PLUGIN=$BIN_DIR/docker-buildx" \
+ MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS=1 \
+ MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS=2 \
+ MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS=2 \
+ MINIKUBE_DOCKER_HEARTBEAT_SECONDS=1 \
+ MINIKUBE_DOCKER_KILL_GRACE_SECONDS=1 \
+ MINIKUBE_STATUS_TIMEOUT_SECONDS=1 \
+ MINIKUBE_DOCKER_ENV_TIMEOUT_SECONDS=1 \
+ MINIKUBE_KUBECTL_TIMEOUT_SECONDS=1 \
+ MINIKUBE_KUBECTL_REQUEST_TIMEOUT_SECONDS=1 \
+ MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS=1 \
+ MINIKUBE_BASE_IMAGE_PULL_RETRIES=1 \
+ MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS=0 \
+ MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ IMAGE_SOURCE=local \
+ "${lease_env[@]}" \
+ "$@" >"$output" 2>&1
+}
+
+assert_rejected_before_runtime() {
+ local name="$1" pattern="$2"
+ shift 2
+ local output="$TMP_DIR/${name}.out" status=0
+ clear_runtime_state
+ run_build "$output" missing bash "$BUILD_SCRIPT" "$@" || status=$?
+ if [[ "$status" -eq 2 ]] && grep -Fq -- "$pattern" "$output" \
+ && runtime_logs_are_empty; then
+ pass "$name fails before any Docker, Minikube, or kubectl call"
+ else
+ fail "$name did not fail pre-runtime (status=$status)"
+ fi
+}
+
+assert_rejected_before_runtime empty-selector 'selector must not be empty' --only=
+assert_rejected_before_runtime unknown-selector 'Unknown --only selector' --only=not-a-real-image
+
+assert_invalid_knob() {
+ local name="$1" assignment="$2"
+ local output="$TMP_DIR/${name}.out" status=0
+ clear_runtime_state
+ run_build "$output" valid "$assignment" bash "$BUILD_SCRIPT" --only=control-api || status=$?
+ if [[ "$status" -eq 2 ]] && grep -Fq 'DOCKER_DEADLINE_INVALID' "$output" \
+ && runtime_logs_are_empty; then
+ pass "$name is rejected with a finite range before runtime"
+ else
+ fail "$name was accepted or reached runtime (status=$status)"
+ fi
+}
+
+assert_invalid_knob retry-zero MINIKUBE_BASE_IMAGE_PULL_RETRIES=0
+assert_invalid_knob retry-over-max MINIKUBE_BASE_IMAGE_PULL_RETRIES=11
+assert_invalid_knob delay-over-max MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS=301
+assert_invalid_knob delay-not-numeric MINIKUBE_BASE_IMAGE_PULL_DELAY_SECS=forever
+
+assert_lease_failure() {
+ local mode="$1" name="$2" status=0
+ local output="$TMP_DIR/lease-${mode}.out"
+ clear_runtime_state
+ run_build "$output" "$mode" bash "$BUILD_SCRIPT" --only=control-api || status=$?
+ if [[ "$status" -ne 0 ]] && grep -Fq 'PROFILE_LOCK_REQUIRED' "$output" \
+ && runtime_logs_are_empty; then
+ pass "$name fails before Docker, Minikube, and kubectl invocation"
+ else
+ fail "$name escaped the inherited lease boundary (status=$status)"
+ fi
+}
+
+assert_lease_failure missing 'a missing mutation lease'
+assert_lease_failure wrong-value 'a wrong mutation lease value'
+assert_lease_failure wrong-context 'a wrong mutation target context'
+
+clear_runtime_state
+valid_output="$TMP_DIR/lease-valid.out"
+valid_status=0
+run_build "$valid_output" valid FAKE_MINIKUBE_STATUS_MODE=exit-47 \
+ bash "$BUILD_SCRIPT" --only=control-api || valid_status=$?
+if [[ "$valid_status" -eq 47 ]] \
+ && grep -Fq -- "-p $PROFILE status" "$MINIKUBE_LOG" \
+ && grep -Fq 'label=minikube-status event=exit' "$valid_output"; then
+ pass 'the exact canonical inherited lease reaches the bounded profile status probe'
+else
+ fail "the valid inherited lease was rejected (status=$valid_status)"
+fi
+
+clear_runtime_state
+verify_output="$TMP_DIR/verify-only.out"
+verify_status=0
+run_build "$verify_output" verify bash "$BUILD_SCRIPT" --verify-only || verify_status=$?
+if [[ "$verify_status" -eq 1 ]] \
+ && ! grep -Fq 'PROFILE_LOCK_REQUIRED' "$verify_output" \
+ && grep -Fq -- "-p $PROFILE status" "$MINIKUBE_LOG" \
+ && grep -Fq -- "-p $PROFILE image ls --format=json" "$MINIKUBE_LOG" \
+ && grep -Fq -- "--context=$PROFILE get nodes --request-timeout=1s --no-headers" "$KUBECTL_LOG" \
+ && [[ ! -s "$DOCKER_LOG" ]]; then
+ pass 'verify-only is lease-exempt while retaining bounded explicit runtime reads'
+else
+ fail "verify-only did not preserve its read-only lease exemption (status=$verify_status)"
+fi
+
+descendant_is_dead() {
+ local record descendant
+ [[ -f "$DESCENDANT_PID_FILE" ]] || return 1
+ record="$(<"$DESCENDANT_PID_FILE")"
+ descendant="${record#*:}"
+ [[ "$descendant" =~ ^[1-9][0-9]*$ ]] || return 1
+ for _ in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20; do
+ if ! kill -0 "$descendant" 2>/dev/null; then
+ return 0
+ fi
+ sleep 0.1
+ done
+ return 1
+}
+
+assert_bounded_operation() {
+ local operation="$1" status=0 label
+ local output="$TMP_DIR/timeout-${operation}.out"
+ local -a mode_env=()
+ case "$operation" in
+ status)
+ label=minikube-status
+ mode_env=(FAKE_MINIKUBE_STATUS_MODE=hang)
+ ;;
+ nodes)
+ label=minikube-get-nodes
+ mode_env=(FAKE_KUBECTL_NODES_MODE=hang)
+ ;;
+ docker-env)
+ label=minikube-docker-env
+ mode_env=(FAKE_MINIKUBE_DOCKER_ENV_MODE=hang)
+ ;;
+ esac
+ clear_runtime_state
+ run_build "$output" valid "${mode_env[@]}" \
+ bash "$BUILD_SCRIPT" --only=control-api || status=$?
+ if [[ "$status" -eq 124 ]] \
+ && grep -Fq "label=$label event=timeout" "$output" \
+ && grep -Fq "label=$label event=terminated" "$output" \
+ && descendant_is_dead; then
+ pass "$operation has a finite deadline, returns 124, and kills descendants"
+ else
+ fail "$operation deadline contract failed (status=$status)"
+ tail -20 "$output" >&2 || true
+ fi
+}
+
+assert_bounded_operation status
+assert_bounded_operation nodes
+assert_bounded_operation docker-env
+
+if grep -Fq -- "--context=$PROFILE get nodes --request-timeout=1s --no-headers" "$KUBECTL_LOG" \
+ && grep -Fq -- "-p $PROFILE docker-env --shell bash" "$MINIKUBE_LOG"; then
+ pass 'node and docker-env probes carry the explicit profile/context and request timeout'
+else
+ fail 'bounded probes omitted an explicit profile, context, or kubectl request timeout'
+fi
+
+assert_child_death_status() {
+ local operation="$1" status=0 label
+ local output="$TMP_DIR/death-${operation}.out"
+ local -a mode_env=()
+ case "$operation" in
+ status)
+ label=minikube-status
+ mode_env=(FAKE_MINIKUBE_STATUS_MODE=death)
+ ;;
+ nodes)
+ label=minikube-get-nodes
+ mode_env=(FAKE_KUBECTL_NODES_MODE=death)
+ ;;
+ docker-env)
+ label=minikube-docker-env
+ mode_env=(FAKE_MINIKUBE_DOCKER_ENV_MODE=death)
+ ;;
+ esac
+ clear_runtime_state
+ run_build "$output" valid "${mode_env[@]}" \
+ bash "$BUILD_SCRIPT" --only=control-api || status=$?
+ if [[ "$status" -eq 143 ]] \
+ && grep -Fq "label=$label event=exit" "$output" \
+ && grep -Fq 'signal=SIGTERM exitCode=143' "$output"; then
+ pass "$operation preserves deterministic child-death status 143"
+ else
+ fail "$operation changed or hid child-death status (status=$status)"
+ tail -20 "$output" >&2 || true
+ fi
+}
+
+assert_child_death_status status
+assert_child_death_status nodes
+assert_child_death_status docker-env
+
+manifest="$FIXTURE/deploy/minikube/.image-manifest.json"
+printf 'known-good-manifest\n' >"$manifest"
+clear_runtime_state
+query_failure_output="$TMP_DIR/query-status-1.out"
+query_failure_status=0
+run_build "$query_failure_output" valid \
+ MINIKUBE_PRELOAD_BASE_IMAGES=true \
+ FAKE_DOCKER_QUERY_STATUS=1 \
+ bash "$BUILD_SCRIPT" --only=control-api || query_failure_status=$?
+if [[ "$query_failure_status" -eq 1 ]] \
+ && grep -Fq 'images -q node:24-alpine' "$DOCKER_LOG" \
+ && ! grep -Fq 'pull node:24-alpine' "$DOCKER_LOG" \
+ && ! grep -Fq 'image inspect node:24-alpine' "$DOCKER_LOG" \
+ && ! grep -Fq 'Manifest:' "$query_failure_output" \
+ && [[ "$(<"$manifest")" == known-good-manifest ]]; then
+ pass 'daemon inventory status 1 hard-fails without pull or manifest success'
+else
+ fail "daemon inventory status 1 fell through as absence (status=$query_failure_status)"
+ tail -20 "$query_failure_output" >&2 || true
+fi
+
+printf 'known-good-manifest\n' >"$manifest"
+clear_runtime_state
+manifest_query_output="$TMP_DIR/manifest-query-status-1.out"
+manifest_query_status=0
+run_build "$manifest_query_output" valid \
+ FAKE_DOCKER_QUERY_MATCH=clerum/host-context-controller:test \
+ FAKE_DOCKER_QUERY_STATUS=1 \
+ bash "$BUILD_SCRIPT" --only=control-api || manifest_query_status=$?
+if [[ "$manifest_query_status" -eq 1 ]] \
+ && grep -Fq 'build -t clerum/control-api:test' "$DOCKER_LOG" \
+ && grep -Fq 'images -q clerum/host-context-controller:test' "$DOCKER_LOG" \
+ && ! grep -Fq 'Manifest:' "$manifest_query_output" \
+ && [[ "$(<"$manifest")" == known-good-manifest ]]; then
+ pass 'a manifest inventory status 1 preserves the prior manifest and cannot report success'
+else
+ fail "manifest inventory status 1 produced a false manifest result (status=$manifest_query_status)"
+ tail -20 "$manifest_query_output" >&2 || true
+fi
+
+printf 'known-good-manifest\n' >"$manifest"
+clear_runtime_state
+empty_query_output="$TMP_DIR/query-empty.out"
+empty_query_status=0
+run_build "$empty_query_output" valid \
+ MINIKUBE_PRELOAD_BASE_IMAGES=true \
+ FAKE_DOCKER_QUERY_STATUS=0 \
+ FAKE_DOCKER_QUERY_OUTPUT= \
+ FAKE_DOCKER_PULL_STATUS=55 \
+ bash "$BUILD_SCRIPT" --only=control-api || empty_query_status=$?
+if [[ "$empty_query_status" -eq 55 ]] \
+ && grep -Fq 'images -q node:24-alpine' "$DOCKER_LOG" \
+ && grep -Fq 'pull node:24-alpine' "$DOCKER_LOG" \
+ && ! grep -Fq 'Manifest:' "$empty_query_output" \
+ && [[ "$(<"$manifest")" == known-good-manifest ]]; then
+ pass 'only a successful empty daemon inventory is treated as image absence'
+else
+ fail "successful empty inventory did not take the bounded pull path (status=$empty_query_status)"
+ tail -20 "$empty_query_output" >&2 || true
+fi
+
+if [[ "$FAIL" -eq 0 ]]; then
+ printf 'PASS: build-images hardening fake-binary suite\n'
+fi
+exit "$FAIL"
diff --git a/scripts/tests/test-minikube-docker-cli-env.sh b/scripts/tests/test-minikube-docker-cli-env.sh
new file mode 100755
index 000000000..355b97a98
--- /dev/null
+++ b/scripts/tests/test-minikube-docker-cli-env.sh
@@ -0,0 +1,990 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+TMP_DIR="$(mktemp -d)"
+# Invoked indirectly by the EXIT trap.
+# shellcheck disable=SC2329
+cleanup_test_tmp() {
+ if [[ "${KEEP_MINIKUBE_DOCKER_TEST_TMP:-false}" == true ]]; then
+ printf 'Kept test fixtures at %s\n' "$TMP_DIR" >&2
+ else
+ rm -rf -- "$TMP_DIR"
+ fi
+}
+trap cleanup_test_tmp EXIT
+
+BIN_DIR="$TMP_DIR/bin"
+AMBIENT_CONFIG="$TMP_DIR/ambient-docker"
+EXPLICIT_CONFIG="$TMP_DIR/explicit-docker"
+ROOTLESS_CONFIG="$TMP_DIR/rootless-docker"
+DOCKER_LOG="$TMP_DIR/docker.log"
+MINIKUBE_LOG="$TMP_DIR/minikube.log"
+PLUGIN_LOG="$TMP_DIR/buildx.log"
+HELPER_SENTINEL="$TMP_DIR/ambient-helper-called"
+UNSAFE_SENTINEL="$TMP_DIR/unsafe-config"
+DESCENDANT_PID_FILE="$TMP_DIR/descendant.pid"
+TASK_TMP="$TMP_DIR/task-tmp"
+SECRET_VALUE="AUTH_SHOULD_NOT_LEAK_8d45d9"
+FAIL=0
+
+pass() { printf 'PASS: %s\n' "$1"; }
+fail() { printf 'FAIL: %s\n' "$1" >&2; FAIL=1; }
+
+mkdir -p "$BIN_DIR" "$AMBIENT_CONFIG" "$EXPLICIT_CONFIG" "$ROOTLESS_CONFIG" "$TASK_TMP"
+EXPLICIT_CONFIG="$(cd -- "$EXPLICIT_CONFIG" && pwd -P)"
+ROOTLESS_CONFIG="$(cd -- "$ROOTLESS_CONFIG" && pwd -P)"
+printf '{"auths":{"registry.invalid":{"auth":"%s"}},"credsStore":"poison","credHelpers":{"registry.invalid":"poison"}}\n' \
+ "$SECRET_VALUE" >"$AMBIENT_CONFIG/config.json"
+printf '{"auths":{"private.invalid":{"auth":"EXPLICIT_AUTH_NOT_FOR_OUTPUT"}}}\n' \
+ >"$EXPLICIT_CONFIG/config.json"
+printf '{"currentContext":"rootless-local"}\n' >"$ROOTLESS_CONFIG/config.json"
+
+cat >"$BIN_DIR/docker-credential-poison" <<'STUB'
+#!/usr/bin/env bash
+printf 'called\n' >"${HELPER_SENTINEL:?}"
+exit 97
+STUB
+
+cat >"$BIN_DIR/docker-buildx-fixture" <<'STUB'
+#!/usr/bin/env bash
+printf 'buildx\n' >>"${PLUGIN_LOG:?}"
+printf 'github.com/docker/buildx fixture\n'
+STUB
+
+cat >"$BIN_DIR/docker" <<'STUB'
+#!/usr/bin/env bash
+set -euo pipefail
+
+config="${DOCKER_CONFIG:-}"
+if [[ -z "$config" || ! -f "$config/config.json" ]]; then
+ printf 'missing Docker config\n' >"${UNSAFE_SENTINEL:?}"
+ exit 90
+fi
+
+mode="$(node -e 'const fs=require("node:fs"); process.stdout.write(((fs.statSync(process.argv[1]).mode & 0o777).toString(8)))' "$config")"
+printf '%s|%s|host=%s|context=%s|tls=%s|verify=%s|cert=%s\n' \
+ "$config" "$*" "${DOCKER_HOST-<unset>}" "${DOCKER_CONTEXT-<unset>}" \
+ "${DOCKER_TLS-<unset>}" "${DOCKER_TLS_VERIFY-<unset>}" "${DOCKER_CERT_PATH-<unset>}" \
+ >>"${DOCKER_LOG:?}"
+
+if [[ "${1:-}" == context && "${2:-}" == inspect ]]; then
+ if [[ "$config" != "${AMBIENT_CONFIG:?}" \
+ && "$config" != "${ROOTLESS_CONFIG:?}" \
+ && "$config" != "${FAKE_EXPECT_AUTH_CONFIG:-}" \
+ && "$mode" != 700 ]]; then
+ printf 'unsafe Docker config mode\n' >"${UNSAFE_SENTINEL:?}"
+ exit 91
+ fi
+ if [[ -n "${DOCKER_AUTH_CONFIG+x}" || -n "${DOCKER_CUSTOM_HEADERS+x}" ]]; then
+ printf 'ambient Docker auth/header env reached context inspection\n' >"${UNSAFE_SENTINEL:?}"
+ exit 94
+ fi
+
+ endpoint="${DOCKER_HOST:-}"
+ if [[ -z "$endpoint" ]]; then
+ context_name="${DOCKER_CONTEXT:-}"
+ if [[ -z "$context_name" ]]; then
+ context_name="$(node -e '
+ const fs = require("node:fs")
+ const value = JSON.parse(fs.readFileSync(process.argv[1], "utf8"))
+ process.stdout.write(value.currentContext || "default")
+ ' "$config/config.json")"
+ fi
+ case "$context_name" in
+ default) endpoint="unix:///var/run/docker.sock" ;;
+ rootless-local) endpoint="unix:///run/user/1000/docker.sock" ;;
+ hostile-remote) endpoint="ssh://prod.invalid/var/run/docker.sock" ;;
+ *) exit 96 ;;
+ esac
+ fi
+
+ if [[ "$*" == *'TLSMaterial'* ]]; then
+ printf '%s\tfalse\t{}\n' "$endpoint"
+ else
+ if [[ -n "${FAKE_DOCKER_EFFECTIVE_HOST_OVERRIDE:-}" ]]; then
+ endpoint="$FAKE_DOCKER_EFFECTIVE_HOST_OVERRIDE"
+ fi
+ printf '%s\n' "$endpoint"
+ fi
+ exit 0
+fi
+
+if [[ "$config" != "${FAKE_EXPECT_AUTH_CONFIG:-}" && "$mode" != 700 ]]; then
+ printf 'unsafe Docker config mode\n' >"${UNSAFE_SENTINEL:?}"
+ exit 91
+fi
+
+if [[ -n "${DOCKER_CONTEXT+x}" || -n "${DOCKER_AUTH_CONFIG+x}" \
+ || -n "${DOCKER_CUSTOM_HEADERS+x}" ]]; then
+ printf 'ambient Docker endpoint/auth env reached isolated operation\n' >"${UNSAFE_SENTINEL:?}"
+ exit 94
+fi
+
+if grep -Eq '"(credsStore|credHelpers)"[[:space:]]*:' "$config/config.json"; then
+ docker-credential-poison get </dev/null
+fi
+if [[ "$config" != "${FAKE_EXPECT_AUTH_CONFIG:-}" ]] \
+ && grep -Eq '"auths"[[:space:]]*:' "$config/config.json"; then
+ printf 'ambient auth reached isolated Docker config\n' >"${UNSAFE_SENTINEL:?}"
+ exit 92
+fi
+
+if [[ "${1:-}" == buildx && "${2:-}" == version ]]; then
+ plugin="$config/cli-plugins/docker-buildx"
+ [[ -x "$plugin" ]] || exit 93
+ "$plugin" version
+ exit 0
+fi
+
+operation_mode=success
+if [[ -z "${FAKE_DOCKER_MATCH:-}" || "$*" == *"${FAKE_DOCKER_MATCH}"* ]]; then
+ operation_mode="${FAKE_DOCKER_MODE:-success}"
+fi
+case "$operation_mode" in
+ exit)
+ exit "${FAKE_DOCKER_EXIT_CODE:-37}"
+ ;;
+ hang)
+ (
+ trap '' HUP QUIT TERM INT
+ while true; do sleep 1; done
+ ) &
+ descendant=$!
+ printf '%s\n' "$descendant" >"${DESCENDANT_PID_FILE:?}"
+ wait "$descendant"
+ ;;
+esac
+
+case "${1:-}" in
+ images)
+ if [[ -n "${FAKE_DOCKER_IMAGES_PRESENT_MATCH:-}" \
+ && "$*" == *"${FAKE_DOCKER_IMAGES_PRESENT_MATCH}"* ]]; then
+ printf 'sha256:present\n'
+ fi
+ exit 0
+ ;;
+ image)
+ [[ "${2:-}" == inspect ]] && exit 1
+ ;;
+ inspect)
+ printf 'sha256:deadbeef0000cafedeadbeef0000cafedeadbeef\n'
+ ;;
+esac
+exit 0
+STUB
+
+cat >"$BIN_DIR/minikube" <<'STUB'
+#!/usr/bin/env bash
+printf '%s\n' "$*" >>"${MINIKUBE_LOG:?}"
+if { [[ "${FAKE_MINIKUBE_MODE:-}" == hang-inventory && "$*" == *"image ls"*"--format=json"* ]] \
+ || [[ "${FAKE_MINIKUBE_MODE:-}" == hang-load && "$*" == *"image load"* && "$*" != *"--pull"* ]]; }; then
+ (
+ trap '' HUP QUIT TERM INT
+ while true; do sleep 1; done
+ ) &
+ descendant=$!
+ printf '%s\n' "$descendant" >"${DESCENDANT_PID_FILE:?}"
+ wait "$descendant"
+fi
+case "$*" in
+ *status*) exit 0 ;;
+ *docker-env*) printf 'export DOCKER_HOST="tcp://127.0.0.1:2376"\n'; exit 0 ;;
+ *"image ls"*) exit 0 ;;
+esac
+exit 0
+STUB
+
+cat >"$BIN_DIR/kubectl" <<'STUB'
+#!/usr/bin/env bash
+if [[ "$*" == *"get nodes"* ]]; then
+ printf 'fixture-node Ready control-plane 1d v1.30.0\n'
+ if [[ "${FAKE_NODE_COUNT:-1}" -gt 1 ]]; then
+ printf 'fixture-node-2 Ready worker 1d v1.30.0\n'
+ fi
+fi
+exit 0
+STUB
+
+chmod +x "$BIN_DIR/docker-credential-poison" "$BIN_DIR/docker-buildx-fixture" \
+ "$BIN_DIR/docker" "$BIN_DIR/minikube" "$BIN_DIR/kubectl"
+
+export PATH="$BIN_DIR:$PATH"
+export TMPDIR="$TASK_TMP"
+export AMBIENT_CONFIG ROOTLESS_CONFIG DOCKER_LOG MINIKUBE_LOG PLUGIN_LOG
+export HELPER_SENTINEL UNSAFE_SENTINEL DESCENDANT_PID_FILE
+export MINIKUBE_DOCKER_BUILDX_PLUGIN="$BIN_DIR/docker-buildx-fixture"
+export MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS=3
+export MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS=3
+export MINIKUBE_DOCKER_BUILD_TIMEOUT_SECONDS=3
+export MINIKUBE_DOCKER_HEARTBEAT_SECONDS=1
+export MINIKUBE_DOCKER_KILL_GRACE_SECONDS=1
+export MINIKUBE_DOCKER_START_PROBE_TIMEOUT_SECONDS=2
+export MINIKUBE_DOCKER_START_TIMEOUT_SECONDS=2
+export MINIKUBE_DOCKER_START_POLL_SECONDS=1
+export MINIKUBE_IMAGE_INVENTORY_TIMEOUT_SECONDS=2
+export DOCKER_CONFIG="$AMBIENT_CONFIG"
+unset DOCKER_AUTH_CONFIG DOCKER_CONTEXT DOCKER_HOST DOCKER_TLS DOCKER_TLS_VERIFY
+unset DOCKER_CERT_PATH DOCKER_CUSTOM_HEADERS DOCKER_API_VERSION MINIKUBE_ACTIVE_DOCKERD
+unset MINIKUBE_DOCKER_AUTH_CONFIG FAKE_EXPECT_AUTH_CONFIG
+
+docker_log_has_ambient_runtime_operation() {
+ awk -F'|' -v config="$AMBIENT_CONFIG" '
+ $1 == config && $2 !~ /^context inspect / { found = 1 }
+ END { exit(found ? 0 : 1) }
+ ' "$DOCKER_LOG"
+}
+
+prepare_fixture_repo() {
+ local fixture="$1"
+ mkdir -p "$fixture/scripts/minikube" "$fixture/scripts/release" \
+ "$fixture/control-api" "$fixture/deploy/minikube"
+ cp "$ROOT/scripts/minikube/build-images.sh" \
+ "$ROOT/scripts/minikube/docker-cli-env.sh" \
+ "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ "$ROOT/scripts/minikube/image-mode.sh" \
+ "$fixture/scripts/minikube/"
+ cp "$ROOT/scripts/release/images-manifest.mjs" "$fixture/scripts/release/"
+ cp "$ROOT/deploy/images.json" "$fixture/deploy/images.json"
+ cat >"$fixture/scripts/minikube/require-t2-mutation-lock.sh" <<'STUB'
+#!/usr/bin/env bash
+exit 0
+STUB
+ printf 'FROM scratch\n' >"$fixture/control-api/Dockerfile"
+ chmod +x "$fixture/scripts/minikube/build-images.sh" \
+ "$fixture/scripts/minikube/docker-cli-env.sh" \
+ "$fixture/scripts/minikube/run-with-deadline.mjs" \
+ "$fixture/scripts/minikube/require-t2-mutation-lock.sh"
+ export T2_PROJECT_DIR="$fixture"
+ export T2_PROFILE=fixture-profile
+ export T2_CONTEXT=fixture-profile
+ export MINIKUBE_PROFILE=fixture-profile
+}
+
+assert_endpoint_resolution_and_context_precedence() {
+ local reject_output="$TMP_DIR/hostile-context-reject.out"
+ local explicit_output="$TMP_DIR/explicit-host.out"
+ local mismatch_output="$TMP_DIR/endpoint-mismatch.out"
+ local status=0 cert_dir="$TMP_DIR/client-certs"
+ mkdir -p "$cert_dir"
+
+ : >"$DOCKER_LOG"
+ DOCKER_CONFIG="$AMBIENT_CONFIG" DOCKER_CONTEXT=hostile-remote \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare false
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$reject_output" 2>&1 || status=$?
+ if [[ "$status" -ne 0 ]] \
+ && grep -Fq 'DOCKER_ENDPOINT_UNSAFE' "$reject_output" \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]]; then
+ pass "a hostile remote DOCKER_CONTEXT is rejected before task config selection"
+ else
+ fail "a hostile remote DOCKER_CONTEXT was not rejected (status=$status)"
+ fi
+
+ : >"$DOCKER_LOG"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" DOCKER_CONTEXT=hostile-remote \
+ DOCKER_HOST=tcp://127.0.0.1:2376 DOCKER_TLS=1 DOCKER_TLS_VERIFY=0 \
+ DOCKER_CERT_PATH="$cert_dir" \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare false
+ [[ "$DOCKER_HOST" == tcp://127.0.0.1:2376 ]]
+ [[ "${DOCKER_CONTEXT+x}" != x ]]
+ [[ "$DOCKER_TLS" == 1 && "$DOCKER_TLS_VERIFY" == 0 ]]
+ [[ "$DOCKER_CERT_PATH" == "$2" ]]
+ docker_cli_run_public explicit-host 2 docker info >/dev/null
+ task_config="$DOCKER_CLI_TASK_CONFIG"
+ docker_cli_env_cleanup
+ [[ "$DOCKER_CONFIG" == "$3" && "$DOCKER_CONTEXT" == hostile-remote ]]
+ [[ "$DOCKER_HOST" == tcp://127.0.0.1:2376 ]]
+ [[ "$DOCKER_TLS" == 1 && "$DOCKER_TLS_VERIFY" == 0 ]]
+ [[ "$DOCKER_CERT_PATH" == "$2" && ! -e "$task_config" ]]
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" "$cert_dir" "$AMBIENT_CONFIG" \
+ >"$explicit_output" 2>&1 || status=$?
+ if [[ "$status" -eq 0 ]] \
+ && grep -Fq '|info|host=tcp://127.0.0.1:2376|context=<unset>|tls=1|verify=0' "$DOCKER_LOG" \
+ && ! docker_log_has_ambient_runtime_operation; then
+ pass "an explicit local DOCKER_HOST wins over hostile context and preserves TLS semantics"
+ else
+ fail "explicit local Docker host/context precedence failed (status=$status)"
+ fi
+
+ : >"$DOCKER_LOG"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" DOCKER_HOST=unix:///var/run/docker.sock \
+ FAKE_DOCKER_EFFECTIVE_HOST_OVERRIDE=unix:///tmp/different.sock \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare false
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$mismatch_output" 2>&1 || status=$?
+ if [[ "$status" -ne 0 ]] && grep -Fq 'DOCKER_ENDPOINT_MISMATCH' "$mismatch_output"; then
+ pass "the isolated config verifies the effective pinned endpoint"
+ else
+ fail "effective Docker endpoint mismatch was not detected (status=$status)"
+ fi
+}
+
+assert_config_only_rootless_context() {
+ local output="$TMP_DIR/rootless-context.out" status=0
+ : >"$DOCKER_LOG"
+ DOCKER_CONFIG="$ROOTLESS_CONFIG" bash -c '
+ set -euo pipefail
+ unset DOCKER_HOST DOCKER_CONTEXT DOCKER_TLS DOCKER_TLS_VERIFY DOCKER_CERT_PATH
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare false
+ [[ "$DOCKER_HOST" == unix:///run/user/1000/docker.sock ]]
+ [[ "${DOCKER_CONTEXT+x}" != x && "${DOCKER_TLS+x}" != x ]]
+ docker_cli_run_public rootless-info 2 docker info >/dev/null
+ task_config="$DOCKER_CLI_TASK_CONFIG"
+ docker_cli_env_cleanup
+ [[ "$DOCKER_CONFIG" == "$2" && "${DOCKER_HOST+x}" != x ]]
+ [[ "${DOCKER_CONTEXT+x}" != x && ! -e "$task_config" ]]
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" "$ROOTLESS_CONFIG" >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 0 ]] \
+ && grep -Fq "$ROOTLESS_CONFIG|context inspect" "$DOCKER_LOG" \
+ && grep -Fq '|info|host=unix:///run/user/1000/docker.sock|context=<unset>' "$DOCKER_LOG" \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]]; then
+ pass "a config-only local rootless context is resolved and pinned safely"
+ else
+ fail "config-only rootless context support failed (status=$status)"
+ fi
+}
+
+assert_all_original_docker_env_is_restored() {
+ local output="$TMP_DIR/env-restore.out" status=0 cert_dir="$TMP_DIR/restore-certs"
+ mkdir -p "$cert_dir"
+ DOCKER_CONFIG="$AMBIENT_CONFIG" DOCKER_AUTH_CONFIG='{"auths":{}}' \
+ DOCKER_CONTEXT=hostile-remote DOCKER_HOST=tcp://127.0.0.1:2376 \
+ DOCKER_TLS='' DOCKER_TLS_VERIFY=0 DOCKER_CERT_PATH="$cert_dir" \
+ DOCKER_API_VERSION=1.43 DOCKER_CUSTOM_HEADERS='X-Fixture=original' \
+ MINIKUBE_ACTIVE_DOCKERD=original-profile \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare false
+ task_config="$DOCKER_CLI_TASK_CONFIG"
+ export DOCKER_CONFIG=/changed DOCKER_AUTH_CONFIG=changed DOCKER_CONTEXT=changed
+ export DOCKER_HOST=unix:///changed.sock DOCKER_TLS=changed DOCKER_TLS_VERIFY=changed
+ export DOCKER_CERT_PATH=/changed DOCKER_API_VERSION=changed
+ export DOCKER_CUSTOM_HEADERS=changed MINIKUBE_ACTIVE_DOCKERD=changed
+ docker_cli_env_cleanup
+ [[ "$DOCKER_CONFIG" == "$2" && "$DOCKER_AUTH_CONFIG" == "{\"auths\":{}}" ]]
+ [[ "$DOCKER_CONTEXT" == hostile-remote && "$DOCKER_HOST" == tcp://127.0.0.1:2376 ]]
+ [[ "${DOCKER_TLS+x}" == x && -z "$DOCKER_TLS" && "$DOCKER_TLS_VERIFY" == 0 ]]
+ [[ "$DOCKER_CERT_PATH" == "$3" && "$DOCKER_API_VERSION" == 1.43 ]]
+ [[ "$DOCKER_CUSTOM_HEADERS" == X-Fixture=original ]]
+ [[ "$MINIKUBE_ACTIVE_DOCKERD" == original-profile && ! -e "$task_config" ]]
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" "$AMBIENT_CONFIG" "$cert_dir" \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 0 ]]; then
+ pass "cleanup restores every original Docker endpoint/config env value exactly"
+ else
+ fail "Docker environment restoration failed (status=$status)"
+ fi
+}
+
+assert_real_build_script_isolated() {
+ local fixture="$TMP_DIR/repo" output="$TMP_DIR/build.out"
+ prepare_fixture_repo "$fixture"
+ : >"$DOCKER_LOG"
+ if DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=control-api \
+ >"$output" 2>&1; then
+ if grep -Fq '|build -t clerum/control-api:test' "$DOCKER_LOG" \
+ && ! docker_log_has_ambient_runtime_operation \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]] \
+ && [[ -s "$PLUGIN_LOG" ]]; then
+ pass "targeted build uses the clean task config and discovered buildx plugin"
+ else
+ fail "targeted build did not preserve Docker isolation"
+ fi
+ else
+ fail "targeted fake-Docker build failed: $(tail -20 "$output")"
+ fi
+
+ if ! grep -Fq "$SECRET_VALUE" "$output" \
+ && ! grep -Fq "$AMBIENT_CONFIG" "$output" \
+ && ! grep -Fq 'credsStore' "$output"; then
+ pass "build output contains no ambient config or credential contents"
+ else
+ fail "build output leaked ambient Docker configuration"
+ fi
+}
+
+assert_public_pulls_are_isolated() {
+ local fixture="$TMP_DIR/public-repo" output="$TMP_DIR/public.out"
+ prepare_fixture_repo "$fixture"
+ : >"$DOCKER_LOG"
+ if DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ bash "$fixture/scripts/minikube/build-images.sh" --public-only \
+ >"$output" 2>&1; then
+ if grep -Fq '|pull postgres:16-alpine' "$DOCKER_LOG" \
+ && ! docker_log_has_ambient_runtime_operation \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]]; then
+ pass "public pulls never inherit the ambient credential helper"
+ else
+ fail "public pulls escaped the isolated Docker config"
+ fi
+ else
+ fail "public fake-Docker pull run failed: $(tail -20 "$output")"
+ fi
+}
+
+assert_local_image_operations_preserve_status() {
+ local fixture output status
+
+ fixture="$TMP_DIR/base-inspect-failure-repo"
+ output="$TMP_DIR/base-inspect-failure.out"
+ prepare_fixture_repo "$fixture"
+ : >"$DOCKER_LOG"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" \
+ FAKE_DOCKER_MODE=exit FAKE_DOCKER_MATCH='images -q node:24-alpine' \
+ FAKE_DOCKER_EXIT_CODE=42 \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=control-api \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 42 ]] \
+ && grep -Fq 'label=docker-images-query event=exit' "$output" \
+ && grep -Fq 'exitCode=42' "$output" \
+ && ! grep -Fq '|pull node:24-alpine' "$DOCKER_LOG"; then
+ pass "bounded base-image inventory preserves status and never falls through to pull"
+ else
+ fail "base-image inventory status/fallback contract failed (status=$status)"
+ fi
+
+ fixture="$TMP_DIR/public-query-failure-repo"
+ output="$TMP_DIR/public-query-failure.out"
+ prepare_fixture_repo "$fixture"
+ : >"$DOCKER_LOG"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ FAKE_DOCKER_MODE=exit FAKE_DOCKER_MATCH='images -q postgres:16-alpine' \
+ FAKE_DOCKER_EXIT_CODE=39 \
+ bash "$fixture/scripts/minikube/build-images.sh" --public-only \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 39 ]] \
+ && grep -Fq 'label=docker-images-query event=exit' "$output" \
+ && grep -Fq 'exitCode=39' "$output" \
+ && ! grep -Fq '|pull postgres:16-alpine' "$DOCKER_LOG"; then
+ pass "bounded docker images query preserves status and never falls through to pull"
+ else
+ fail "docker images query status/fallback contract failed (status=$status)"
+ fi
+
+ fixture="$TMP_DIR/post-build-inspect-failure-repo"
+ output="$TMP_DIR/post-build-inspect-failure.out"
+ prepare_fixture_repo "$fixture"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ FAKE_DOCKER_MODE=exit \
+ FAKE_DOCKER_MATCH='inspect --format={{.Id}} clerum/control-api:test' \
+ FAKE_DOCKER_EXIT_CODE=37 \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=control-api \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 37 ]] \
+ && grep -Fq 'label=docker-image-id event=exit' "$output" \
+ && grep -Fq 'exitCode=37' "$output"; then
+ pass "bounded post-build docker inspect preserves its exact status"
+ else
+ fail "post-build docker inspect status contract failed (status=$status)"
+ fi
+
+ fixture="$TMP_DIR/manifest-inspect-failure-repo"
+ output="$TMP_DIR/manifest-inspect-failure.out"
+ prepare_fixture_repo "$fixture"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ FAKE_DOCKER_MODE=exit \
+ FAKE_DOCKER_MATCH='inspect --format={{.Id}} clerum/host-context-controller:test' \
+ FAKE_DOCKER_IMAGES_PRESENT_MATCH='clerum/host-context-controller:test' \
+ FAKE_DOCKER_EXIT_CODE=41 \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=control-api \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 41 ]] \
+ && grep -Fq 'Could not inspect clerum/host-context-controller:test while generating the image manifest' "$output" \
+ && grep -Fq 'exitCode=41' "$output"; then
+ pass "bounded manifest docker inspect preserves non-missing failures"
+ else
+ fail "manifest docker inspect status contract failed (status=$status)"
+ fi
+
+ fixture="$TMP_DIR/tag-failure-repo"
+ output="$TMP_DIR/tag-failure.out"
+ prepare_fixture_repo "$fixture"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ MINIKUBE_DOCKER_AUTH_CONFIG="$EXPLICIT_CONFIG" \
+ FAKE_EXPECT_AUTH_CONFIG="$EXPLICIT_CONFIG" \
+ FAKE_DOCKER_MODE=exit FAKE_DOCKER_MATCH='tag ' FAKE_DOCKER_EXIT_CODE=38 \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=playwright \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -eq 38 ]] \
+ && grep -Fq 'label=docker-image-tag event=exit' "$output" \
+ && grep -Fq 'exitCode=38' "$output"; then
+ pass "bounded docker tag preserves its exact status"
+ else
+ fail "docker tag status contract failed (status=$status)"
+ fi
+}
+
+assert_local_minikube_load_timeout_kills_descendants() {
+ local fixture="$TMP_DIR/load-timeout-repo" output="$TMP_DIR/load-timeout.out"
+ local status=0 descendant=""
+ prepare_fixture_repo "$fixture"
+ : >"$MINIKUBE_LOG"
+ rm -f -- "$DESCENDANT_PID_FILE"
+
+ DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ FAKE_NODE_COUNT=2 FAKE_MINIKUBE_MODE=hang-load \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=control-api \
+ >"$output" 2>&1 || status=$?
+ if [[ -f "$DESCENDANT_PID_FILE" ]]; then
+ descendant="$(cat "$DESCENDANT_PID_FILE")"
+ fi
+ for _ in 1 2 3 4 5 6 7 8 9 10; do
+ if [[ -z "$descendant" ]] || ! kill -0 "$descendant" 2>/dev/null; then break; fi
+ sleep 0.1
+ done
+
+ if [[ "$status" -eq 124 && -n "$descendant" ]] \
+ && ! kill -0 "$descendant" 2>/dev/null \
+ && grep -Fq 'label=minikube-image-load event=timeout' "$output" \
+ && grep -Fq 'image load clerum/control-api:test' "$MINIKUBE_LOG"; then
+ pass "bounded local minikube image load kills its full process group"
+ else
+ fail "local minikube image load timeout failed (status=$status descendant=${descendant:-missing})"
+ fi
+}
+
+assert_startup_probes_are_isolated_and_bounded() {
+ local check_output="$TMP_DIR/startup-check.out" wait_output="$TMP_DIR/startup-wait.out"
+ local status=0 descendant="" plugin_before plugin_after
+
+ : >"$DOCKER_LOG"
+ plugin_before="$(wc -l <"$PLUGIN_LOG" 2>/dev/null || printf 0)"
+ if DOCKER_CONFIG="$AMBIENT_CONFIG" \
+ MINIKUBE_DOCKER_BUILDX_PLUGIN="$TMP_DIR/not-installed-buildx" \
+ "$ROOT/scripts/minikube/docker-cli-env.sh" --check-info \
+ >"$check_output" 2>&1; then
+ plugin_after="$(wc -l <"$PLUGIN_LOG" 2>/dev/null || printf 0)"
+ if grep -Fq '|info|' "$DOCKER_LOG" \
+ && ! grep -Fq '|buildx version' "$DOCKER_LOG" \
+ && ! docker_log_has_ambient_runtime_operation \
+ && [[ "$plugin_before" -eq "$plugin_after" ]] \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]]; then
+ pass "startup probe is isolated and does not require buildx"
+ else
+ fail "startup probe escaped isolation or invoked buildx"
+ fi
+ else
+ fail "bounded startup probe failed: $(tail -20 "$check_output")"
+ fi
+
+ rm -f -- "$DESCENDANT_PID_FILE"
+ status=0
+ DOCKER_CONFIG="$AMBIENT_CONFIG" \
+ MINIKUBE_DOCKER_BUILDX_PLUGIN="$TMP_DIR/not-installed-buildx" \
+ FAKE_DOCKER_MODE=hang \
+ "$ROOT/scripts/minikube/docker-cli-env.sh" --wait-for-info \
+ >"$wait_output" 2>&1 || status=$?
+ if [[ -f "$DESCENDANT_PID_FILE" ]]; then
+ descendant="$(cat "$DESCENDANT_PID_FILE")"
+ fi
+ for _ in 1 2 3 4 5 6 7 8 9 10; do
+ if [[ -z "$descendant" ]] || ! kill -0 "$descendant" 2>/dev/null; then break; fi
+ sleep 0.1
+ done
+
+ if [[ "$status" -eq 124 && -n "$descendant" ]] \
+ && ! kill -0 "$descendant" 2>/dev/null \
+ && grep -Fq 'label=docker-info-startup-wait event=timeout' "$wait_output" \
+ && ! grep -Fq "$SECRET_VALUE" "$wait_output" \
+ && ! grep -Fq "$AMBIENT_CONFIG" "$wait_output"; then
+ pass "startup wait has one finite deadline and kills helper descendants"
+ else
+ fail "startup wait deadline failed (status=$status descendant=${descendant:-missing})"
+ fi
+}
+
+assert_verify_inventory_is_read_only_and_bounded() {
+ local fixture="$TMP_DIR/verify-repo" output="$TMP_DIR/verify-timeout.out"
+ local status=0 descendant=""
+ prepare_fixture_repo "$fixture"
+ rm -f -- "$DESCENDANT_PID_FILE"
+ : >"$DOCKER_LOG"
+
+ DOCKER_CONFIG="$AMBIENT_CONFIG" IMAGE_SOURCE=local \
+ FAKE_MINIKUBE_MODE=hang-inventory \
+ bash "$fixture/scripts/minikube/build-images.sh" --verify-only \
+ >"$output" 2>&1 || status=$?
+ if [[ -f "$DESCENDANT_PID_FILE" ]]; then
+ descendant="$(cat "$DESCENDANT_PID_FILE")"
+ fi
+ for _ in 1 2 3 4 5 6 7 8 9 10; do
+ if [[ -z "$descendant" ]] || ! kill -0 "$descendant" 2>/dev/null; then break; fi
+ sleep 0.1
+ done
+
+ if [[ "$status" -eq 124 && -n "$descendant" ]] \
+ && ! kill -0 "$descendant" 2>/dev/null \
+ && grep -Fq 'label=minikube-image-inventory event=timeout' "$output" \
+ && [[ ! -s "$DOCKER_LOG" ]] \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]]; then
+ pass "verify-only bounds its read-only inventory without invoking Docker"
+ else
+ fail "verify-only inventory deadline failed (status=$status descendant=${descendant:-missing})"
+ fi
+}
+
+assert_invalid_deadline_and_explicit_buildx_fail_closed() {
+ local deadline_output="$TMP_DIR/invalid-deadline.out"
+ local buildx_output="$TMP_DIR/invalid-buildx.out"
+ local deadline_status=0 buildx_status=0
+
+ MINIKUBE_DOCKER_INFO_TIMEOUT_SECONDS=0 bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$deadline_output" 2>&1 || deadline_status=$?
+ if [[ "$deadline_status" -eq 2 ]] && grep -Fq 'DOCKER_DEADLINE_INVALID' "$deadline_output"; then
+ pass "invalid Docker deadlines fail before any operation"
+ else
+ fail "invalid Docker deadline was accepted (status=$deadline_status)"
+ fi
+
+ : >"$DOCKER_LOG"
+ MINIKUBE_DOCKER_BUILDX_PLUGIN="$TMP_DIR/not-installed-buildx" bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$buildx_output" 2>&1 || buildx_status=$?
+ if [[ "$buildx_status" -ne 0 ]] && grep -Fq 'DOCKER_BUILDX_REQUIRED' "$buildx_output" \
+ && ! grep -Fq '|buildx version|' "$DOCKER_LOG" \
+ && ! docker_log_has_ambient_runtime_operation; then
+ pass "an invalid explicit buildx path never falls back or downloads"
+ else
+ fail "invalid explicit buildx path was not rejected (status=$buildx_status)"
+ fi
+}
+
+assert_private_registry_requires_explicit_config() {
+ local fixture="$TMP_DIR/private-repo" output="$TMP_DIR/private-missing.out" status=0
+ prepare_fixture_repo "$fixture"
+ : >"$DOCKER_LOG"
+ unset MINIKUBE_DOCKER_AUTH_CONFIG FAKE_EXPECT_AUTH_CONFIG
+ DOCKER_CONFIG="$AMBIENT_CONFIG" MINIKUBE_PRELOAD_BASE_IMAGES=false \
+ bash "$fixture/scripts/minikube/build-images.sh" --only=playwright \
+ >"$output" 2>&1 || status=$?
+ if [[ "$status" -ne 0 ]] && grep -Fq 'REGISTRY_AUTH_REQUIRED' "$output" &&
+ ! grep -Fq '|pull us-central1-docker.pkg.dev/' "$DOCKER_LOG"; then
+ pass "private build mode fails loud before its Docker pull without explicit auth"
+ else
+ fail "private build mode did not enforce REGISTRY_AUTH_REQUIRED (status=$status)"
+ fi
+}
+
+assert_explicit_private_config_is_used_without_copying_ambient_auth() {
+ local output="$TMP_DIR/private-explicit.out"
+ : >"$DOCKER_LOG"
+ if DOCKER_CONFIG="$AMBIENT_CONFIG" \
+ MINIKUBE_DOCKER_AUTH_CONFIG="$EXPLICIT_CONFIG" \
+ FAKE_EXPECT_AUTH_CONFIG="$EXPLICIT_CONFIG" \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare
+ docker_cli_run_private private-pull "$MINIKUBE_DOCKER_PULL_TIMEOUT_SECONDS" docker pull private.invalid/image:test
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$output" 2>&1; then
+ if grep -Fq "$EXPLICIT_CONFIG|pull private.invalid/image:test" "$DOCKER_LOG" \
+ && ! docker_log_has_ambient_runtime_operation \
+ && [[ ! -e "$HELPER_SENTINEL" && ! -e "$UNSAFE_SENTINEL" ]] \
+ && ! grep -Fq 'EXPLICIT_AUTH_NOT_FOR_OUTPUT' "$output"; then
+ pass "explicit private config is scoped to the private pull without leaking auth"
+ else
+ fail "explicit private Docker config was not scoped correctly"
+ fi
+ else
+ fail "explicit private config was rejected: $(tail -20 "$output")"
+ fi
+}
+
+assert_exit_status_and_process_group_timeout() {
+ local exit_output="$TMP_DIR/exit.out" timeout_output="$TMP_DIR/timeout.out"
+ local status=0 descendant=""
+
+ FAKE_DOCKER_MODE=exit FAKE_DOCKER_EXIT_CODE=37 \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare
+ set +e
+ docker_cli_run_public build-exit 3 docker build fixture
+ status=$?
+ set -e
+ exit "$status"
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$exit_output" 2>&1 || status=$?
+ if [[ "$status" -eq 37 ]] && grep -Fq 'event=exit' "$exit_output" \
+ && grep -Fq 'exitCode=37' "$exit_output" \
+ && grep -Fq '[HARNESS_DEADLINE]' "$exit_output" \
+ && ! grep -Fq '[DOCKER_DEADLINE]' "$exit_output"; then
+ pass "deadline runner preserves the exact Docker exit status"
+ else
+ fail "deadline runner changed Docker exit status (got $status)"
+ fi
+
+ status=0
+ FAKE_DOCKER_MODE=hang \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ trap docker_cli_env_cleanup EXIT
+ docker_cli_env_prepare
+ set +e
+ docker_cli_run_public pull-timeout 2 docker pull public.invalid/image:test
+ status=$?
+ set -e
+ exit "$status"
+ ' _ "$ROOT/scripts/minikube/docker-cli-env.sh" >"$timeout_output" 2>&1 || status=$?
+
+ if [[ -f "$DESCENDANT_PID_FILE" ]]; then
+ descendant="$(cat "$DESCENDANT_PID_FILE")"
+ fi
+ for _ in 1 2 3 4 5 6 7 8 9 10; do
+ if [[ -z "$descendant" ]] || ! kill -0 "$descendant" 2>/dev/null; then break; fi
+ sleep 0.1
+ done
+
+ if [[ "$status" -eq 124 && -n "$descendant" ]] \
+ && ! kill -0 "$descendant" 2>/dev/null \
+ && grep -Fq 'event=heartbeat' "$timeout_output" \
+ && grep -Fq 'event=timeout' "$timeout_output" \
+ && grep -Fq 'event=terminated' "$timeout_output"; then
+ pass "timeout kills the fake Docker process group, including descendants"
+ else
+ fail "timeout contract failed (status=$status descendant=${descendant:-missing})"
+ fi
+
+ if ! grep -Fq "$SECRET_VALUE" "$timeout_output" \
+ && ! grep -Fq "$AMBIENT_CONFIG" "$timeout_output" \
+ && ! grep -Fq 'credsStore' "$timeout_output"; then
+ pass "deadline heartbeat and timing output remain credential-safe"
+ else
+ fail "deadline output leaked Docker configuration"
+ fi
+}
+
+wait_for_file_content() {
+ local file="$1"
+ local attempt=0
+ while (( attempt < 120 )); do
+ [[ -s "$file" ]] && return 0
+ sleep 0.05
+ attempt=$((attempt + 1))
+ done
+ return 1
+}
+
+wait_for_output_pattern() {
+ local file="$1" pattern="$2"
+ local attempt=0
+ while (( attempt < 120 )); do
+ grep -Fq "$pattern" "$file" 2>/dev/null && return 0
+ sleep 0.05
+ attempt=$((attempt + 1))
+ done
+ return 1
+}
+
+wait_for_process_death() {
+ local pid="$1"
+ local attempt=0
+ while (( attempt < 120 )); do
+ ! kill -0 "$pid" 2>/dev/null && return 0
+ sleep 0.05
+ attempt=$((attempt + 1))
+ done
+ return 1
+}
+
+assert_repeated_signal_escalates_group() {
+ local signal="$1" expected_status="$2" label="double-${1}"
+ local output="$TMP_DIR/${label}.out" pid_file="$TMP_DIR/${label}.pid"
+ local runner_pid status=0 descendant="" descendant_dead=false
+
+ rm -f -- "$pid_file"
+ # Positional values and $! belong to the child shell.
+ # shellcheck disable=SC2016
+ node "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ --timeout-seconds 15 --heartbeat-seconds 1 --kill-grace-seconds 5 \
+ --label "$label" -- \
+ bash -c '
+ trap "" HUP QUIT INT TERM
+ (
+ trap "" HUP QUIT INT TERM
+ while true; do sleep 1; done
+ ) &
+ printf "%s\n" "$!" >"$1"
+ wait
+ ' _ "$pid_file" >"$output" 2>&1 &
+ runner_pid=$!
+
+ if wait_for_file_content "$pid_file"; then
+ descendant="$(cat "$pid_file")"
+ fi
+ kill -s "$signal" "$runner_pid" 2>/dev/null || true
+ wait_for_output_pattern "$output" 'event=interrupted' || true
+ kill -s "$signal" "$runner_pid" 2>/dev/null || true
+ wait "$runner_pid" || status=$?
+ if [[ -n "$descendant" ]] && wait_for_process_death "$descendant"; then
+ descendant_dead=true
+ fi
+
+ if [[ "$status" -eq "$expected_status" && "$descendant_dead" == true ]] \
+ && grep -Fq 'event=signal-escalated' "$output" \
+ && grep -Fq 'action=SIGKILL' "$output"; then
+ pass "a repeated ${signal} escalates the entire process group and preserves status ${expected_status}"
+ else
+ fail "repeated ${signal} teardown failed (status=$status descendant=${descendant:-missing})"
+ [[ -n "$descendant" ]] && kill -KILL "$descendant" 2>/dev/null || true
+ fi
+}
+
+assert_bash_main_signal_cleanup() {
+ local signal="$1" expected_status="$2" label="bash-main-${1}"
+ local output="$TMP_DIR/${label}.out" docker_log="$TMP_DIR/${label}-docker.log"
+ local pid_file="$TMP_DIR/${label}-descendant.pid"
+ local runner_pid status=0 descendant="" task_config="" descendant_dead=false
+
+ : >"$docker_log"
+ rm -f -- "$pid_file"
+ DOCKER_LOG="$docker_log" DESCENDANT_PID_FILE="$pid_file" \
+ DOCKER_CONFIG="$AMBIENT_CONFIG" FAKE_DOCKER_MODE=hang \
+ MINIKUBE_DOCKER_START_TIMEOUT_SECONDS=20 \
+ node "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ --timeout-seconds 25 --heartbeat-seconds 1 --kill-grace-seconds 3 \
+ --label "$label" -- \
+ "$ROOT/scripts/minikube/docker-cli-env.sh" --wait-for-info \
+ >"$output" 2>&1 &
+ runner_pid=$!
+
+ if wait_for_output_pattern "$docker_log" '|info|'; then
+ task_config="$(awk -F'|' '$2 == "info" { print $1; exit }' "$docker_log")"
+ fi
+ if wait_for_file_content "$pid_file"; then
+ descendant="$(cat "$pid_file")"
+ fi
+ kill -s "$signal" "$runner_pid" 2>/dev/null || true
+ wait "$runner_pid" || status=$?
+ if [[ -n "$descendant" ]] && wait_for_process_death "$descendant"; then
+ descendant_dead=true
+ fi
+
+ if [[ "$status" -eq "$expected_status" && -n "$task_config" && ! -e "$task_config" \
+ && "$descendant_dead" == true ]] \
+ && grep -Fq "signal=SIG${signal}" "$output" \
+ && grep -Fq "exitCode=${expected_status}" "$output"; then
+ pass "Bash main catches ${signal}, returns ${expected_status}, and removes its task config"
+ else
+ fail "Bash main ${signal} cleanup failed (status=$status config=${task_config:-missing} descendant=${descendant:-missing})"
+ [[ -n "$descendant" ]] && kill -KILL "$descendant" 2>/dev/null || true
+ fi
+}
+
+assert_normal_exit_reaps_background_descendants() {
+ local output="$TMP_DIR/normal-descendant.out" pid_file="$TMP_DIR/normal-descendant.pid"
+ local status=0 descendant="" descendant_dead=false
+ rm -f -- "$pid_file"
+
+ # Positional values and $! belong to the child shell.
+ # shellcheck disable=SC2016
+ node "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ --timeout-seconds 5 --heartbeat-seconds 1 --kill-grace-seconds 1 \
+ --label normal-descendant -- \
+ bash -c '
+ (
+ trap "" HUP QUIT INT TERM
+ while true; do sleep 1; done
+ ) &
+ printf "%s\n" "$!" >"$1"
+ exit 0
+ ' _ "$pid_file" >"$output" 2>&1 || status=$?
+ if wait_for_file_content "$pid_file"; then
+ descendant="$(cat "$pid_file")"
+ fi
+ if [[ -n "$descendant" ]] && wait_for_process_death "$descendant"; then
+ descendant_dead=true
+ fi
+
+ if [[ "$status" -eq 0 && "$descendant_dead" == true ]] \
+ && grep -Fq 'event=exit' "$output" && grep -Fq 'exitCode=0' "$output"; then
+ pass "a normal direct-child exit cannot orphan a background descendant"
+ else
+ fail "normal-exit descendant cleanup failed (status=$status descendant=${descendant:-missing})"
+ [[ -n "$descendant" ]] && kill -KILL "$descendant" 2>/dev/null || true
+ fi
+}
+
+assert_conventional_spawn_exit_codes() {
+ local missing_output="$TMP_DIR/missing-command.out"
+ local denied_output="$TMP_DIR/denied-command.out" denied="$TMP_DIR/not-executable"
+ local missing_status=0 denied_status=0
+ printf '#!/usr/bin/env bash\nexit 0\n' >"$denied"
+ chmod 600 "$denied"
+
+ node "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ --timeout-seconds 2 --label missing-command -- definitely-not-a-real-command-evenfire \
+ >"$missing_output" 2>&1 || missing_status=$?
+ node "$ROOT/scripts/minikube/run-with-deadline.mjs" \
+ --timeout-seconds 2 --label denied-command -- "$denied" \
+ >"$denied_output" 2>&1 || denied_status=$?
+
+ if [[ "$missing_status" -eq 127 && "$denied_status" -eq 126 ]] \
+ && grep -Fq 'exitCode=127' "$missing_output" \
+ && grep -Fq 'exitCode=126' "$denied_output"; then
+ pass "spawn failures use conventional exact 127/126 exit codes"
+ else
+ fail "spawn failure exit codes changed (missing=$missing_status denied=$denied_status)"
+ fi
+}
+
+assert_endpoint_resolution_and_context_precedence
+assert_config_only_rootless_context
+assert_all_original_docker_env_is_restored
+assert_real_build_script_isolated
+assert_public_pulls_are_isolated
+assert_local_image_operations_preserve_status
+assert_local_minikube_load_timeout_kills_descendants
+assert_startup_probes_are_isolated_and_bounded
+assert_verify_inventory_is_read_only_and_bounded
+assert_invalid_deadline_and_explicit_buildx_fail_closed
+assert_private_registry_requires_explicit_config
+assert_explicit_private_config_is_used_without_copying_ambient_auth
+assert_exit_status_and_process_group_timeout
+assert_bash_main_signal_cleanup HUP 129
+assert_bash_main_signal_cleanup QUIT 131
+assert_repeated_signal_escalates_group INT 130
+assert_repeated_signal_escalates_group TERM 143
+assert_normal_exit_reaps_background_descendants
+assert_conventional_spawn_exit_codes
+
+if find "$TASK_TMP" -mindepth 1 -maxdepth 1 -name 'evenfire-minikube-docker.*' -print -quit | grep -q .; then
+ fail "a task-local Docker config survived cleanup"
+else
+ pass "task-local Docker configs are removed after every operation"
+fi
+
+exit "$FAIL"
diff --git a/scripts/tests/test-minikube-explicit-context.sh b/scripts/tests/test-minikube-explicit-context.sh
new file mode 100755
index 000000000..b15006f75
--- /dev/null
+++ b/scripts/tests/test-minikube-explicit-context.sh
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+# Public gates must receive a Kubernetes context explicitly, even when it
+# happens to have the same name as the selected Minikube profile.
+set -euo pipefail
+set +x
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+COMMON="$ROOT/scripts/minikube/t2-common.sh"
+PROFILE=clerum-feature-owner
+
+if output="$(T2_PROJECT_DIR="$ROOT" T2_PROFILE="$PROFILE" MINIKUBE_PROFILE="$PROFILE" \
+ bash -c 'source "$1"; t2_require_explicit_context' _ "$COMMON" 2>&1)"; then
+ printf 'FAIL: implicit profile-to-context fallback was accepted\n' >&2
+ exit 1
+fi
+case "$output" in
+ *DEVELOPMENT_SCOPE_REQUIRED*'Kubernetes context must be supplied explicitly'*) ;;
+ *) printf 'FAIL: unexpected implicit-context diagnostic: %s\n' "$output" >&2; exit 1 ;;
+esac
+
+for variable in T2_CONTEXT CONTROL_API_REAL_PG_CONTEXT K8S_CONTEXT KUBECONTEXT; do
+ result="$(env "$variable=$PROFILE" T2_PROJECT_DIR="$ROOT" T2_PROFILE="$PROFILE" \
+ MINIKUBE_PROFILE="$PROFILE" bash -c '
+ source "$1"
+ t2_require_explicit_context
+ printf "%s|%s\n" "$T2_CONTEXT" "$T2_CONTEXT_SOURCE"
+ ' _ "$COMMON")"
+ case "$result" in
+ "$PROFILE"'|explicit-'*) ;;
+ *) printf 'FAIL: %s did not establish explicit context provenance: %s\n' "$variable" "$result" >&2; exit 1 ;;
+ esac
+done
+
+printf 'PASS: public Minikube context provenance is explicit and fail-closed\n'
diff --git a/scripts/tests/test-minikube-full-setup.sh b/scripts/tests/test-minikube-full-setup.sh
index c9336f17d..0274f6aa6 100755
--- a/scripts/tests/test-minikube-full-setup.sh
+++ b/scripts/tests/test-minikube-full-setup.sh
@@ -15,9 +15,16 @@ assert_broken_profile_is_recreated() {
cat > "$tmp/docker" <<'STUB'
#!/usr/bin/env bash
-if [[ "${1:-}" == "info" ]]; then
- exit 0
-fi
+case "${1:-} ${2:-}" in
+ "context inspect")
+ if [[ "$*" == *'TLSMaterial'* ]]; then
+ printf 'unix:///private/tmp/evenfire-test-docker.sock\tfalse\t{}\n'
+ else
+ printf 'unix:///private/tmp/evenfire-test-docker.sock\n'
+ fi
+ ;;
+ "info ") ;;
+esac
exit 0
STUB
@@ -131,9 +138,16 @@ assert_healthy_profile_skips_recreate() {
cat > "$tmp/docker" <<'STUB'
#!/usr/bin/env bash
-if [[ "${1:-}" == "info" ]]; then
- exit 0
-fi
+case "${1:-} ${2:-}" in
+ "context inspect")
+ if [[ "$*" == *'TLSMaterial'* ]]; then
+ printf 'unix:///private/tmp/evenfire-test-docker.sock\tfalse\t{}\n'
+ else
+ printf 'unix:///private/tmp/evenfire-test-docker.sock\n'
+ fi
+ ;;
+ "info ") ;;
+esac
exit 0
STUB
@@ -358,6 +372,22 @@ assert_pipefail_head_guard_prevents_abort() {
fi
}
+assert_setup_runtime_operations_are_bounded() {
+ local setup="scripts/minikube/full-setup.sh"
+ if grep -Fq 'run_setup_with_deadline minikube-setup-status' "$setup" && \
+ grep -Fq 'run_setup_with_deadline minikube-setup-delete' "$setup" && \
+ grep -Fq 'run_setup_with_deadline minikube-setup-start' "$setup" && \
+ grep -Fq 'run_setup_with_deadline minikube-setup-validate' "$setup" && \
+ grep -Fq 'bash "${SCRIPT_DIR}/docker-cli-env.sh" --check-info' "$setup" && \
+ grep -Fq 'if (( status >= 124 )); then' "$setup" && \
+ ! grep -Fq 'minikube -p "$PROFILE" status 2>/dev/null || true' "$setup" && \
+ ! grep -Eq '(^|[[:space:]])docker info([[:space:]]|$)' "$setup"; then
+ pass "full-setup bounds profile lifecycle and isolates its Docker probe"
+ else
+ fail "full-setup contains an unbounded profile lifecycle or ambient Docker probe"
+ fi
+}
+
# full-setup.sh is a 1284-line orchestrator that needs a real cluster to run,
# so these cases read its resolved configuration rather than executing it: they
# source the top of the script with a guard variable set, which stops it before
@@ -1242,6 +1272,7 @@ assert_reuse_db_normalizer_precedes_flag_loop
assert_reset_db_flag_backcompat
assert_skip_build_staleness_find_is_sigpipe_guarded
assert_pipefail_head_guard_prevents_abort
+assert_setup_runtime_operations_are_bounded
assert_ghcr_is_the_default_image_source
assert_image_source_local_is_honoured
assert_bootstrap_seed_deferral_is_opt_in
diff --git a/scripts/tests/test-minikube-makefile.sh b/scripts/tests/test-minikube-makefile.sh
index 3bd772e6c..bc8252bd6 100755
--- a/scripts/tests/test-minikube-makefile.sh
+++ b/scripts/tests/test-minikube-makefile.sh
@@ -43,6 +43,16 @@ assert_file_contains() {
fi
}
+assert_file_not_contains() {
+ local path="$1" needle="$2"
+ if grep -Fq -- "$needle" "$REPO_ROOT/$path"; then
+ echo "FAIL: $path unexpectedly contains '$needle'"
+ FAIL=1
+ else
+ echo "PASS: $path omits '$needle'"
+ fi
+}
+
assert_not_contains() {
local target="$1" needle="$2"
local out
@@ -74,6 +84,165 @@ assert_make_contains() {
fi
}
+assert_deploy_rejected_without_mutation() {
+ local stub_root="$1" description="$2" target="$3" expected_lock="$4" expected_error="$5"
+ shift 5
+ local mutation_log="$stub_root/mutations.log"
+ local lock_log="$stub_root/lock.log"
+ local output_file="$stub_root/output.log"
+ local status=0 problem=""
+
+ : > "$mutation_log"
+ : > "$lock_log"
+ if PATH="$stub_root/bin:$PATH" FAKE_MUTATION_LOG="$mutation_log" FAKE_LOCK_LOG="$lock_log" \
+ make -C "$stub_root" "$target" "$@" >"$output_file" 2>&1; then
+ problem+="unexpectedly succeeded; "
+ else
+ status=$?
+ fi
+
+ if [ "$status" -eq 0 ]; then
+ problem+="exit status was zero; "
+ fi
+ if ! grep -Fq -- "$expected_error" "$output_file"; then
+ problem+="missing error '$expected_error'; "
+ fi
+ if [ -s "$mutation_log" ]; then
+ problem+="ran mutation commands: $(<"$mutation_log"); "
+ fi
+ if [ "$expected_lock" = true ]; then
+ grep -Fxq 'lock-check' "$lock_log" || problem+="body validation did not follow the lock check; "
+ elif [ -s "$lock_log" ]; then
+ problem+="public validation acquired the mutation lock; "
+ fi
+
+ if [ -z "$problem" ]; then
+ pass "$description fails before image build or deployment"
+ else
+ fail "$description: $problem output=$(<"$output_file")"
+ fi
+}
+
+assert_deploy_runs_expected_fake_commands() {
+ local stub_root="$1" description="$2" selector="$3" namespace="$4" deployment="$5" target="$6"
+ shift 6
+ local mutation_log="$stub_root/mutations.log"
+ local lock_log="$stub_root/lock.log"
+ local output_file="$stub_root/output.log"
+ local problem="" calls expected_calls
+
+ : > "$mutation_log"
+ : > "$lock_log"
+ if ! PATH="$stub_root/bin:$PATH" FAKE_MUTATION_LOG="$mutation_log" FAKE_LOCK_LOG="$lock_log" \
+ make -C "$stub_root" "$target" "$@" >"$output_file" 2>&1; then
+ fail "$description failed: $(<"$output_file")"
+ return
+ fi
+
+ [ "$(<"$lock_log")" = 'lock-check' ] || problem+="unexpected lock sequence; "
+ calls="$(<"$mutation_log")"
+ printf -v expected_calls \
+ 'build --only=%s\nkubectl --context=clerum-test -n %s rollout restart deployment/%s\nkubectl --context=clerum-test -n %s rollout status deployment/%s --timeout=180s' \
+ "$selector" "$namespace" "$deployment" "$namespace" "$deployment"
+ [ "$calls" = "$expected_calls" ] || problem+="unexpected mutation sequence; "
+
+ if [ -z "$problem" ]; then
+ pass "$description preserves its targeted build/deploy mapping"
+ else
+ fail "$description: $problem calls=$(<"$mutation_log")"
+ fi
+}
+
+assert_minikube_deploy_service_fail_fast() {
+ local stub_root
+ stub_root="$(mktemp -d)"
+ mkdir -p "$stub_root/bin" "$stub_root/scripts/minikube"
+ cp "$REPO_ROOT/Makefile" "$stub_root/Makefile"
+
+ cat > "$stub_root/scripts/minikube/with-t2-mutation-lock.sh" <<'STUB'
+#!/usr/bin/env bash
+set -eu
+[ "${1:-}" = "--" ] || exit 2
+shift
+exec "$@"
+STUB
+ cat > "$stub_root/scripts/minikube/require-t2-mutation-lock.sh" <<'STUB'
+#!/usr/bin/env bash
+set -eu
+printf '%s\n' 'lock-check' >> "${FAKE_LOCK_LOG:?}"
+STUB
+ cat > "$stub_root/scripts/minikube/build-images.sh" <<'STUB'
+#!/usr/bin/env bash
+set -eu
+printf 'build %s\n' "$*" >> "${FAKE_MUTATION_LOG:?}"
+STUB
+ cat > "$stub_root/bin/kubectl" <<'STUB'
+#!/usr/bin/env bash
+set -eu
+printf 'kubectl %s\n' "$*" >> "${FAKE_MUTATION_LOG:?}"
+STUB
+ chmod +x "$stub_root/scripts/minikube/build-images.sh" "$stub_root/bin/kubectl"
+
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with missing SVC" \
+ minikube-deploy-service false "ERROR: SVC required" NS=control-plane DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with empty SVC" \
+ minikube-deploy-service false "ERROR: SVC required" SVC= NS=control-plane DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with whitespace SVC" \
+ minikube-deploy-service false "ERROR: SVC required" "SVC= " NS=control-plane DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with unsupported SVC" \
+ minikube-deploy-service false "ERROR: unsupported SVC selector" SVC=unknown NS=control-plane DEPLOYMENT=unknown
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with missing NS" \
+ minikube-deploy-service false "ERROR: NS required" SVC=control-api DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with empty NS" \
+ minikube-deploy-service false "ERROR: NS required" SVC=control-api NS= DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "public target with whitespace NS" \
+ minikube-deploy-service false "ERROR: NS required" SVC=control-api "NS= " DEPLOYMENT=control-api
+
+ # The private body is callable directly, so it repeats every input check
+ # after proving that the caller holds the mutation lease and before either
+ # fake mutator can record a call.
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with missing SVC" \
+ minikube-deploy-service-body true "ERROR: SVC required" NS=control-plane DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with empty SVC" \
+ minikube-deploy-service-body true "ERROR: SVC required" SVC= NS=control-plane DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with whitespace SVC" \
+ minikube-deploy-service-body true "ERROR: SVC required" "SVC= " NS=control-plane DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with unsupported SVC" \
+ minikube-deploy-service-body true "ERROR: unsupported SVC selector" SVC=unknown NS=control-plane DEPLOYMENT=unknown
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with missing NS" \
+ minikube-deploy-service-body true "ERROR: NS required" SVC=control-api DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with empty NS" \
+ minikube-deploy-service-body true "ERROR: NS required" SVC=control-api NS= DEPLOYMENT=control-api
+ assert_deploy_rejected_without_mutation "$stub_root" "body target with whitespace NS" \
+ minikube-deploy-service-body true "ERROR: NS required" SVC=control-api "NS= " DEPLOYMENT=control-api
+
+ assert_deploy_runs_expected_fake_commands "$stub_root" "omitted mcp-host DEPLOYMENT" \
+ mcp-host mcp-host chatllm minikube-deploy-service SVC=mcp-host NS=mcp-host
+ assert_deploy_runs_expected_fake_commands "$stub_root" "empty mcp-host DEPLOYMENT" \
+ mcp-host mcp-host chatllm minikube-deploy-service SVC=mcp-host NS=mcp-host DEPLOYMENT=
+ assert_deploy_runs_expected_fake_commands "$stub_root" "whitespace mcp-host DEPLOYMENT" \
+ mcp-host mcp-host chatllm minikube-deploy-service-body \
+ SVC=mcp-host NS=mcp-host "DEPLOYMENT= "
+ assert_deploy_runs_expected_fake_commands "$stub_root" "control-api identity default" \
+ control-api control-plane control-api minikube-deploy-service-body SVC=control-api NS=control-plane
+ assert_deploy_runs_expected_fake_commands "$stub_root" "control-ui identity default" \
+ control-ui control-plane control-ui minikube-deploy-service-body SVC=control-ui NS=control-plane
+ assert_deploy_runs_expected_fake_commands "$stub_root" "external-rest-api identity default" \
+ external-rest-api profiles external-rest-api minikube-deploy-service-body SVC=external-rest-api NS=profiles
+ assert_deploy_runs_expected_fake_commands "$stub_root" "profile-ui identity default" \
+ profile-ui profiles profile-ui minikube-deploy-service-body SVC=profile-ui NS=profiles
+ assert_deploy_runs_expected_fake_commands "$stub_root" "rpc-proxy identity default" \
+ rpc-proxy rpc-proxy rpc-proxy minikube-deploy-service-body SVC=rpc-proxy NS=rpc-proxy
+ assert_deploy_runs_expected_fake_commands "$stub_root" "hcc explicit deployment" \
+ hcc control-plane host-context-controller minikube-deploy-service-body \
+ SVC=hcc NS=control-plane DEPLOYMENT=host-context-controller
+ assert_deploy_runs_expected_fake_commands "$stub_root" "custom mcp-host override" \
+ mcp-host mcp-host custom-host minikube-deploy-service \
+ SVC=mcp-host NS=mcp-host DEPLOYMENT=custom-host
+
+ rm -rf "$stub_root"
+}
+
assert_succeeds_dry minikube-start
assert_succeeds_dry minikube-deploy-all
assert_succeeds_dry minikube-deploy-all-body
@@ -82,14 +251,43 @@ assert_succeeds_dry minikube-verify-networkpolicies
assert_contains minikube-start "minikube-sync-auth-key-if-present"
assert_contains minikube-start "--context=clerum-test"
assert_contains minikube-start "MINIKUBE_STARTUP_AUTH_SYNC_MODE=shared-profile-mcp"
+assert_contains minikube-start "docker-cli-env.sh --check-info"
+assert_contains minikube-start "docker-cli-env.sh --wait-for-info"
+assert_contains minikube-start 'MINIKUBE_DOCKER_START_TIMEOUT_SECONDS'
+assert_contains minikube-start 'open -a "Docker Desktop"'
+assert_not_contains minikube-start "docker info"
assert_contains minikube-deploy-all "with-t2-mutation-lock.sh"
assert_contains minikube-deploy-all-body "minikube-sync-auth-key"
+assert_contains minikube-build-images "with-t2-mutation-lock.sh"
+assert_contains minikube-build-images-body "require-t2-mutation-lock.sh"
+assert_contains minikube-build-custom-coordinator-fixture "with-t2-mutation-lock.sh"
+assert_contains minikube-build-custom-coordinator-fixture-body "require-t2-mutation-lock.sh"
+assert_contains minikube-build-e2e-fixtures "with-t2-mutation-lock.sh"
+assert_contains minikube-build-e2e-fixtures-body "require-t2-mutation-lock.sh"
+assert_contains minikube-setup-e2e "minikube-build-e2e-fixtures"
+assert_not_contains minikube-verify-images "with-t2-mutation-lock.sh"
+assert_contains minikube-deploy-service "with-t2-mutation-lock.sh"
+assert_contains minikube-deploy-service-body "require-t2-mutation-lock.sh"
+assert_contains minikube-deploy-service "unsupported SVC selector"
+assert_contains minikube-deploy-service-body "unsupported SVC selector"
+assert_contains minikube-deploy-service "effective DEPLOYMENT could not be resolved from SVC"
+assert_contains minikube-deploy-service-body "effective DEPLOYMENT could not be resolved from SVC"
+assert_contains minikube-restart-deploy "with-t2-mutation-lock.sh"
+assert_contains minikube-restart-deploy-body "require-t2-mutation-lock.sh"
+assert_contains minikube-restart-deploy "unsupported SVC selector"
+assert_contains minikube-restart-deploy-body "unsupported SVC selector"
assert_contains minikube-sync-auth-key "with-t2-mutation-lock.sh"
assert_contains minikube-sync-auth-key-body "--context=clerum-test"
assert_contains minikube-sync-auth-key-body "scripts/minikube/sync-auth-key.sh"
assert_contains minikube-verify-networkpolicies "verify-networkpolicies.sh --overlay minikube"
assert_contains minikube-sync-auth-key-if-present "rpc-proxy-secrets"
assert_file_contains Makefile "T2_MUTATION_LOCK_WRAPPED"
+# Literal Make syntax, not shell expansion.
+# shellcheck disable=SC2016
+assert_file_contains Makefile 'DEPLOYMENT ?= $(call minikube_deployment,$(MINIKUBE_DEPLOY_SERVICE))'
+# shellcheck disable=SC2016
+assert_file_contains Makefile 'MINIKUBE_EFFECTIVE_DEPLOYMENT := $(or $(strip $(DEPLOYMENT)),$(call minikube_deployment,$(MINIKUBE_DEPLOY_SERVICE)))'
+assert_file_not_contains Makefile "MINIKUBE_DEPLOYMENT"
assert_contains minikube-sync-auth-key-if-present "minikube-sync-auth-key"
assert_contains minikube-sync-auth-key-if-present "NotFound"
assert_contains minikube-sync-auth-key-if-present "rpc_probe_status"
@@ -102,10 +300,19 @@ assert_contains minikube-deploy-all-body "mktemp"
assert_contains minikube-deploy-all-body "filtered_manifest"
assert_contains minikube-deploy-crds-body '!= "true"'
-assert_make_contains "deployment/chatllm" minikube-deploy-service SVC=mcp-host NS=mcp-host
-assert_make_contains "deployment/chatllm" minikube-restart-deploy SVC=mcp-host NS=mcp-host
-assert_make_contains "deployment/control-api" minikube-deploy-service SVC=control-api NS=control-plane
-assert_make_contains "deployment/custom-host" minikube-deploy-service SVC=mcp-host NS=mcp-host DEPLOYMENT=custom-host
+assert_make_contains "deployment/chatllm" minikube-deploy-service-body SVC=mcp-host NS=mcp-host
+assert_make_contains "deployment/chatllm" minikube-restart-deploy-body SVC=mcp-host NS=mcp-host
+assert_make_contains "deployment/chatllm" minikube-restart-deploy-body SVC=mcp-host NS=mcp-host DEPLOYMENT=
+assert_make_contains "deployment/control-api" minikube-deploy-service-body SVC=control-api NS=control-plane
+assert_make_contains "deployment/control-ui" minikube-deploy-service-body SVC=control-ui NS=control-plane
+assert_make_contains "deployment/external-rest-api" minikube-deploy-service-body SVC=external-rest-api NS=profiles
+assert_make_contains "deployment/profile-ui" minikube-deploy-service-body SVC=profile-ui NS=profiles
+assert_make_contains "deployment/rpc-proxy" minikube-deploy-service-body SVC=rpc-proxy NS=rpc-proxy
+assert_make_contains "deployment/host-context-controller" minikube-deploy-service-body SVC=hcc NS=control-plane DEPLOYMENT=host-context-controller
+assert_make_contains "deployment/custom-host" minikube-deploy-service-body SVC=mcp-host NS=mcp-host DEPLOYMENT=custom-host
+assert_make_contains "deployment/custom-host" minikube-restart-deploy-body SVC=mcp-host NS=mcp-host DEPLOYMENT=custom-host
+
+assert_minikube_deploy_service_fail_fast
# `make minikube-setup` is the first command a new contributor runs. Its
# default has to be the fast path, and the slow path has to be a named target
@@ -153,10 +360,15 @@ assert_minikube_pull_images_mirrors_build_images() {
# supply them. They must be built through the EXISTING --only flag rather than
# a second build entry point.
assert_minikube_setup_e2e_builds_both_fixtures() {
- local out missing=""
- out="$(make -n -C "$REPO_ROOT" minikube-setup-e2e SKIP_PREREQS=true 2>&1)"
- grep -q -- '--only=workflow-custom-sdk-e2e' <<< "$out" || missing+="workflow-custom-sdk-e2e "
- grep -q -- '--only=workflow-plugin-sdk-e2e' <<< "$out" || missing+="workflow-plugin-sdk-e2e "
+ local setup_out body_out missing=""
+ setup_out="$(make -n -C "$REPO_ROOT" minikube-setup-e2e SKIP_PREREQS=true 2>&1)"
+ body_out="$(make -n -C "$REPO_ROOT" minikube-build-e2e-fixtures-body 2>&1)"
+ grep -q -- 'minikube-build-e2e-fixtures' <<< "$setup_out" \
+ || missing+="leased fixture target "
+ grep -q -- '--only=workflow-custom-sdk-e2e' <<< "$body_out" \
+ || missing+="workflow-custom-sdk-e2e "
+ grep -q -- '--only=workflow-plugin-sdk-e2e' <<< "$body_out" \
+ || missing+="workflow-plugin-sdk-e2e "
if [ -z "$missing" ]; then
pass "minikube-setup-e2e builds both unpublished E2E fixtures via build-images.sh --only"
else
@@ -177,10 +389,12 @@ assert_minikube_setup_e2e_builds_both_fixtures() {
# the mode-independent "Building the two unpublished..." echo text, with the
# enclosing `if [`/`\tfi` bounds found structurally rather than by line
# number -- and `eval`s it for real (not `-n`) against a stubbed
-# build-images.sh, once per IMAGE_SOURCE value. That is genuine evaluated
-# behaviour: the fixtures must be built in ghcr mode and must NOT be built in
-# local mode (full-setup.sh's own local build already covers them; a second
-# build here would waste minutes on every `minikube-setup-local`+e2e run).
+# recursive fixture target, once per IMAGE_SOURCE value. That is genuine
+# evaluated behaviour: the fixtures must be built in ghcr mode and must NOT be
+# built in local mode (full-setup.sh's own local build already covers them; a
+# second build here would waste minutes on every `minikube-setup-local`+e2e
+# run). The stub Makefile below models only that recursive target; lease
+# enforcement itself is covered by test-minikube-mutation-boundary.sh.
extract_minikube_setup_e2e_fixture_recipe() {
local mode="$1"
make -n -C "$REPO_ROOT" minikube-setup-e2e SKIP_PREREQS=true IMAGE_SOURCE="$mode" 2>&1 | awk '
@@ -212,6 +426,12 @@ assert_minikube_setup_e2e_fixtures_are_gated_on_evaluated_image_source() {
cat > "$stub_root/scripts/minikube/build-images.sh" <<STUB
#!/usr/bin/env bash
printf '%s\n' "\$*" >> "$call_log"
+STUB
+ cat > "$stub_root/Makefile" <<'STUB'
+.PHONY: minikube-build-e2e-fixtures
+minikube-build-e2e-fixtures:
+ @scripts/minikube/build-images.sh --only=workflow-custom-sdk-e2e
+ @scripts/minikube/build-images.sh --only=workflow-plugin-sdk-e2e
STUB
chmod +x "$stub_root/scripts/minikube/build-images.sh"
diff --git a/scripts/tests/test-minikube-mutation-boundary.sh b/scripts/tests/test-minikube-mutation-boundary.sh
index 5353efc36..8ecf59cea 100755
--- a/scripts/tests/test-minikube-mutation-boundary.sh
+++ b/scripts/tests/test-minikube-mutation-boundary.sh
@@ -64,4 +64,46 @@ fi
exit 1
}
+full_plan="$(make -n -C "${ROOT}" minikube-build-images 2>&1)"
+full_body_plan="$(make -n -C "${ROOT}" minikube-build-images-body 2>&1)"
+targeted_plan="$(make -n -C "${ROOT}" minikube-deploy-service SVC=control-api NS=control-plane 2>&1)"
+targeted_body_plan="$(make -n -C "${ROOT}" minikube-deploy-service-body SVC=control-api NS=control-plane 2>&1)"
+restart_plan="$(make -n -C "${ROOT}" minikube-restart-deploy SVC=control-api NS=control-plane 2>&1)"
+restart_body_plan="$(make -n -C "${ROOT}" minikube-restart-deploy-body SVC=control-api NS=control-plane 2>&1)"
+e2e_fixture_plan="$(make -n -C "${ROOT}" minikube-build-e2e-fixtures 2>&1)"
+e2e_fixture_body_plan="$(make -n -C "${ROOT}" minikube-build-e2e-fixtures-body 2>&1)"
+verify_plan="$(make -n -C "${ROOT}" minikube-verify-images 2>&1)"
+
+[[ "$full_plan" == *"with-t2-mutation-lock.sh"* \
+ && "$full_body_plan" == *"require-t2-mutation-lock.sh"* ]] || {
+ echo 'FAIL: full local image build is not enclosed by the T2 mutation lease' >&2
+ exit 1
+}
+[[ "$targeted_plan" == *"with-t2-mutation-lock.sh"* \
+ && "$targeted_body_plan" == *"require-t2-mutation-lock.sh"* \
+ && "$targeted_body_plan" == *"build-images.sh --only=control-api"* ]] || {
+ echo 'FAIL: targeted local image build is not enclosed by the T2 mutation lease' >&2
+ exit 1
+}
+[[ "$restart_plan" == *"with-t2-mutation-lock.sh"* \
+ && "$restart_body_plan" == *"require-t2-mutation-lock.sh"* \
+ && "$restart_body_plan" == *"rollout restart deployment/control-api"* ]] || {
+ echo 'FAIL: targeted deployment restart is not enclosed by the T2 mutation lease' >&2
+ exit 1
+}
+[[ "$e2e_fixture_plan" == *"with-t2-mutation-lock.sh"* \
+ && "$e2e_fixture_body_plan" == *"require-t2-mutation-lock.sh"* \
+ && "$e2e_fixture_body_plan" == *"build-images.sh --only=workflow-custom-sdk-e2e"* \
+ && "$e2e_fixture_body_plan" == *"build-images.sh --only=workflow-plugin-sdk-e2e"* ]] || {
+ echo 'FAIL: E2E fixture image builds are not enclosed by one T2 mutation lease' >&2
+ exit 1
+}
+[[ "$verify_plan" != *"with-t2-mutation-lock.sh"* \
+ && "$verify_plan" != *"require-t2-mutation-lock.sh"* \
+ && "$verify_plan" == *"build-images.sh --verify-only"* ]] || {
+ echo 'FAIL: read-only image verification was incorrectly placed behind a mutation lease' >&2
+ exit 1
+}
+
printf 'PASS: child mutation boundary requires the exact live profile lease and target context\n'
+printf 'PASS: image builds and targeted deployment mutations require the lease while verify-only stays read-only\n'
diff --git a/scripts/tests/test-minikube-port-forward-owner.sh b/scripts/tests/test-minikube-port-forward-owner.sh
new file mode 100755
index 000000000..11ff9bab2
--- /dev/null
+++ b/scripts/tests/test-minikube-port-forward-owner.sh
@@ -0,0 +1,325 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+OWNER="${ROOT}/scripts/minikube/port-forward-owner.sh"
+PF_ALL="${ROOT}/scripts/minikube/pf-all-stack.sh"
+TMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/evenfire-pf-owner.XXXXXX")"
+trap 'rm -rf -- "${TMP_ROOT}"' EXIT
+
+fail() {
+ printf 'FAIL: %s\n' "$*" >&2
+ exit 1
+}
+
+pass() {
+ printf 'PASS: %s\n' "$1"
+}
+
+[[ -f "${OWNER}" ]] || fail 'port-forward-owner.sh is missing'
+# shellcheck source=scripts/minikube/port-forward-owner.sh
+source "${OWNER}"
+
+WORKTREE="${TMP_ROOT}/worktree"
+OTHER_WORKTREE="${TMP_ROOT}/other-worktree"
+RECORD_DIR="${TMP_ROOT}/cache/profile-a/pids"
+RECORD="${RECORD_DIR}/control-api.pid"
+STATE_FILE="${TMP_ROOT}/process.state"
+KILL_LOG="${TMP_ROOT}/kill.log"
+REAP_LOG="${TMP_ROOT}/reap.log"
+mkdir -p "${WORKTREE}" "${OTHER_WORKTREE}" "${RECORD_DIR}"
+WORKTREE="$(cd -- "${WORKTREE}" && pwd -P)"
+OTHER_WORKTREE="$(cd -- "${OTHER_WORKTREE}" && pwd -P)"
+
+PROFILE='profile-a'
+CONTEXT='profile-a'
+NAMESPACE='control-plane'
+SERVICE='control-api'
+LOCAL_PORT='30190'
+REMOTE_PORT='8090'
+PID='4242'
+RECORDED_START='Sun Aug 23 10:11:12 2026'
+ACTUAL_START="${RECORDED_START}"
+ACTUAL_COMMAND="kubectl --context=${CONTEXT} -n ${NAMESPACE} port-forward --address=127.0.0.1 svc/${SERVICE} ${LOCAL_PORT}:${REMOTE_PORT}"
+
+# Deterministic process boundary. No real process is signalled by the ownership
+# unit cases; state transitions happen only through these fixtures.
+pf_owner_process_state() {
+ sed -n '1p' "${STATE_FILE}"
+}
+
+pf_owner_process_start() {
+ printf '%s\n' "${ACTUAL_START}"
+}
+
+pf_owner_process_command() {
+ printf '%s\n' "${ACTUAL_COMMAND}"
+}
+
+pf_owner_signal_process() {
+ printf '%s %s\n' "$1" "$2" >>"${KILL_LOG}"
+ printf 'dead\n' >"${STATE_FILE}"
+}
+
+pf_owner_reap_process() {
+ printf '%s\n' "$1" >>"${REAP_LOG}"
+}
+
+pf_owner_pause() {
+ :
+}
+
+reset_process_fixture() {
+ printf 'live\n' >"${STATE_FILE}"
+ : >"${KILL_LOG}"
+ : >"${REAP_LOG}"
+ rm -f -- "${RECORD}"
+ ACTUAL_START="${RECORDED_START}"
+ ACTUAL_COMMAND="kubectl --context=${CONTEXT} -n ${NAMESPACE} port-forward --address=127.0.0.1 svc/${SERVICE} ${LOCAL_PORT}:${REMOTE_PORT}"
+}
+
+write_record() {
+ local profile="${1:-${PROFILE}}"
+ local context="${2:-${CONTEXT}}"
+ local worktree="${3:-${WORKTREE}}"
+ local namespace="${4:-${NAMESPACE}}"
+ local service="${5:-${SERVICE}}"
+ local local_port="${6:-${LOCAL_PORT}}"
+ local remote_port="${7:-${REMOTE_PORT}}"
+ pf_owner_write_record_atomic "${RECORD}" "${PID}" "${RECORDED_START}" \
+ "${profile}" "${context}" "${worktree}" "${namespace}" "${service}" \
+ "${local_port}" "${remote_port}"
+}
+
+cleanup_record() {
+ pf_owner_cleanup_record "${RECORD}" "${PROFILE}" "${CONTEXT}" \
+ "${WORKTREE}" "${NAMESPACE}" "${SERVICE}" "${LOCAL_PORT}" "${REMOTE_PORT}"
+}
+
+expect_refusal() {
+ local label="$1"
+ if cleanup_record >"${TMP_ROOT}/refusal.out" 2>&1; then
+ fail "${label} was accepted"
+ fi
+ [[ -f "${RECORD}" ]] || fail "${label} removed a live ambiguous record"
+ [[ ! -s "${KILL_LOG}" ]] || fail "${label} signalled a live ambiguous process"
+ pass "${label} fails closed"
+}
+
+reset_process_fixture
+write_record
+mode="$(stat -f '%Lp' "${RECORD}" 2>/dev/null || stat -c '%a' "${RECORD}")"
+[[ "${mode}" == 600 ]] || fail "structured pid record mode is ${mode}, expected 600"
+grep -Fxq "${PID}" "${RECORD}" || fail 'legacy-compatible PID first line is missing'
+grep -Fxq 'PORT_FORWARD_OWNER_VERSION=1' "${RECORD}" || fail 'record version is missing'
+grep -Fxq "PID=${PID}" "${RECORD}" || fail 'structured PID binding is missing'
+grep -Fxq "PROCESS_START=${RECORDED_START}" "${RECORD}" || fail 'process-start binding is missing'
+grep -Fxq "PROFILE=${PROFILE}" "${RECORD}" || fail 'profile binding is missing'
+grep -Fxq "CONTEXT=${CONTEXT}" "${RECORD}" || fail 'context binding is missing'
+grep -Fxq "WORKTREE=${WORKTREE}" "${RECORD}" || fail 'canonical worktree binding is missing'
+grep -Fxq "NAMESPACE=${NAMESPACE}" "${RECORD}" || fail 'namespace binding is missing'
+grep -Fxq "SERVICE=${SERVICE}" "${RECORD}" || fail 'service binding is missing'
+grep -Fxq "LOCAL_PORT=${LOCAL_PORT}" "${RECORD}" || fail 'local-port binding is missing'
+grep -Fxq "REMOTE_PORT=${REMOTE_PORT}" "${RECORD}" || fail 'remote-port binding is missing'
+if find "${RECORD_DIR}" -maxdepth 1 -name '*.tmp.*' -print -quit | grep -q .; then
+ fail 'atomic record write left a temporary file behind'
+fi
+pass 'structured records are complete, atomic, private, and legacy-readable'
+
+record_hash="$(shasum "${RECORD}" | awk '{print $1}')"
+if pf_owner_write_record_atomic "${RECORD}" 5252 "${RECORDED_START}" \
+ "${PROFILE}" "${CONTEXT}" "${WORKTREE}" "${NAMESPACE}" "${SERVICE}" \
+ "${LOCAL_PORT}" "${REMOTE_PORT}" >"${TMP_ROOT}/no-clobber.out" 2>&1; then
+ fail 'atomic publication replaced an existing ownership record'
+fi
+[[ "$(shasum "${RECORD}" | awk '{print $1}')" == "${record_hash}" ]] ||
+ fail 'failed no-clobber publication changed the existing owner'
+pass 'atomic publication never replaces an existing owner'
+
+reset_process_fixture
+write_record 'profile-b'
+expect_refusal 'wrong-profile record'
+
+reset_process_fixture
+write_record "${PROFILE}" 'profile-b'
+expect_refusal 'wrong-context record'
+
+reset_process_fixture
+write_record "${PROFILE}" "${CONTEXT}" "${OTHER_WORKTREE}"
+expect_refusal 'wrong-worktree record'
+
+reset_process_fixture
+write_record
+ACTUAL_COMMAND="kubectl --context=${CONTEXT} -n ${NAMESPACE} port-forward --address=127.0.0.1 svc/foreign-service ${LOCAL_PORT}:${REMOTE_PORT}"
+expect_refusal 'wrong-service argv'
+
+reset_process_fixture
+write_record
+ACTUAL_COMMAND="kubectl --context=${CONTEXT} -n ${NAMESPACE} port-forward --address=127.0.0.1 svc/${SERVICE} 39999:${REMOTE_PORT}"
+expect_refusal 'wrong-ports argv'
+
+reset_process_fixture
+write_record
+ACTUAL_START='Sun Aug 23 10:11:13 2026'
+expect_refusal 'PID-reuse/start mismatch'
+
+reset_process_fixture
+write_record
+ACTUAL_START=''
+expect_refusal 'unavailable process-start identity'
+
+reset_process_fixture
+printf '%s\nPROCESS_START=%s\n' "${PID}" "${RECORDED_START}" >"${RECORD}"
+chmod 600 "${RECORD}"
+printf 'dead\n' >"${STATE_FILE}"
+cleanup_record || fail 'dead legacy pidfile was not cleaned'
+[[ ! -e "${RECORD}" ]] || fail 'dead legacy pidfile remains'
+[[ ! -s "${KILL_LOG}" ]] || fail 'dead legacy cleanup sent a signal'
+pass 'dead stale pidfiles are removed without signalling'
+
+reset_process_fixture
+write_record
+cleanup_record || fail 'exact owned cleanup failed'
+[[ ! -e "${RECORD}" ]] || fail 'exact owned cleanup kept the pidfile'
+grep -Fxq "${PID} TERM" "${KILL_LOG}" || fail 'exact owned cleanup did not send TERM'
+pass 'exact record, start identity, and kubectl argv permit cleanup'
+
+reset_process_fixture
+write_record
+pf_owner_signal_process() {
+ printf '%s %s\n' "$1" "$2" >>"${KILL_LOG}"
+}
+PF_OWNER_TERMINATE_ATTEMPTS=2 PF_OWNER_TERMINATE_DELAY=0
+if cleanup_record >"${TMP_ROOT}/term-timeout.out" 2>&1; then
+ fail 'cleanup accepted a live child that ignored TERM'
+fi
+[[ -f "${RECORD}" ]] || fail 'bounded TERM timeout removed the live ownership record'
+grep -Fxq "${PID} TERM" "${KILL_LOG}" || fail 'bounded TERM timeout did not signal the exact PID'
+[[ ! -s "${REAP_LOG}" ]] || fail 'bounded TERM timeout called blocking reap on a live process'
+pass 'TERM-resistant cleanup is bounded and never waits on a live process'
+
+pf_owner_signal_process() {
+ printf '%s %s\n' "$1" "$2" >>"${KILL_LOG}"
+ printf 'dead\n' >"${STATE_FILE}"
+}
+
+reset_process_fixture
+ACTUAL_START=''
+if pf_owner_record_process "${RECORD}" "${PID}" "${PROFILE}" "${CONTEXT}" \
+ "${WORKTREE}" "${NAMESPACE}" "${SERVICE}" "${LOCAL_PORT}" "${REMOTE_PORT}" \
+ >"${TMP_ROOT}/missing-start.out" 2>&1; then
+ fail 'record_process accepted an unavailable process-start identity'
+fi
+[[ ! -e "${RECORD}" ]] || fail 'record_process persisted an unavailable start identity'
+pass 'new ownership fails closed when process-start cannot be captured'
+
+# Execute the real forwarding entrypoint against command stubs. The first
+# required health endpoint fails, so the script must exit non-zero and remove
+# the exact owned process/record. The fake kubectl only execs /bin/sleep.
+BIN_DIR="${TMP_ROOT}/bin"
+HEALTH_CACHE="${TMP_ROOT}/health-cache"
+KUBECTL_LOG="${TMP_ROOT}/kubectl.log"
+LAUNCHED_PID="${TMP_ROOT}/launched.pid"
+RECORD_SNAPSHOT="${TMP_ROOT}/record.snapshot"
+HEALTH_OUTPUT="${TMP_ROOT}/health.out"
+HEALTH_PROFILE='pf-owner-health'
+HEALTH_CONTEXT='pf-owner-health'
+HEALTH_PIDFILE="${HEALTH_CACHE}/${HEALTH_PROFILE}/pids/control-ui.pid"
+mkdir -p "${BIN_DIR}"
+
+cat >"${BIN_DIR}/kubectl" <<'EOF_KUBECTL'
+#!/usr/bin/env bash
+set -euo pipefail
+printf '%s\n' "$*" >>"${PF_TEST_KUBECTL_LOG:?}"
+printf '%s\n' "$$" >"${PF_TEST_LAUNCHED_PID:?}"
+exec /bin/sleep 30
+EOF_KUBECTL
+
+cat >"${BIN_DIR}/ps" <<'EOF_PS'
+#!/usr/bin/env bash
+set -euo pipefail
+pid=''
+mode=''
+while (( $# > 0 )); do
+ case "$1" in
+ -p)
+ pid="$2"
+ shift 2
+ ;;
+ -o)
+ mode="$2"
+ shift 2
+ ;;
+ *)
+ shift
+ ;;
+ esac
+done
+[[ "${pid}" =~ ^[1-9][0-9]*$ ]] || exit 2
+if ! /bin/kill -0 "${pid}" 2>/dev/null; then
+ exit 1
+fi
+case "${mode}" in
+ lstart=)
+ printf 'Sun Aug 23 10:11:12 2026\n'
+ ;;
+ command=)
+ printf 'kubectl --context=%s -n control-plane port-forward --address=127.0.0.1 svc/control-ui 3000:3000\n' \
+ "${PF_TEST_CONTEXT:?}"
+ ;;
+ pid=)
+ printf '%s\n' "${pid}"
+ ;;
+ *)
+ exit 2
+ ;;
+esac
+EOF_PS
+
+cat >"${BIN_DIR}/curl" <<'EOF_CURL'
+#!/usr/bin/env bash
+set -euo pipefail
+cp "${PF_TEST_PIDFILE:?}" "${PF_TEST_RECORD_SNAPSHOT:?}"
+exit 22
+EOF_CURL
+
+chmod +x "${BIN_DIR}/kubectl" "${BIN_DIR}/ps" "${BIN_DIR}/curl"
+
+health_status=0
+env PATH="${BIN_DIR}:${PATH}" \
+ MINIKUBE_PROFILE="${HEALTH_PROFILE}" \
+ KUBECONTEXT="${HEALTH_CONTEXT}" \
+ CLERUM_PROFILE_CACHE_ROOT="${HEALTH_CACHE}" \
+ CLERUM_PROFILE_PORTS_ENV="${TMP_ROOT}/no-ports.env" \
+ PF_HEALTH_ATTEMPTS=1 \
+ PF_HEALTH_DELAY=0 \
+ PF_STARTUP_DELAY=0 \
+ PF_OWNER_TERMINATE_DELAY=0 \
+ PF_TEST_CONTEXT="${HEALTH_CONTEXT}" \
+ PF_TEST_KUBECTL_LOG="${KUBECTL_LOG}" \
+ PF_TEST_LAUNCHED_PID="${LAUNCHED_PID}" \
+ PF_TEST_PIDFILE="${HEALTH_PIDFILE}" \
+ PF_TEST_RECORD_SNAPSHOT="${RECORD_SNAPSHOT}" \
+ bash "${PF_ALL}" >"${HEALTH_OUTPUT}" 2>&1 || health_status=$?
+
+[[ "${health_status}" -ne 0 ]] || fail 'required health failure returned success'
+grep -Fq 'ERROR: control-ui did not become healthy' "${HEALTH_OUTPUT}" || \
+ fail 'required health failure was not fail-loud'
+grep -Fxq -- "--context=${HEALTH_CONTEXT} -n control-plane port-forward --address=127.0.0.1 svc/control-ui 3000:3000" \
+ "${KUBECTL_LOG}" || fail 'port-forward did not use the explicit context argv'
+[[ -f "${RECORD_SNAPSHOT}" ]] || fail 'health fixture could not snapshot the structured pid record'
+grep -Fxq "PROFILE=${HEALTH_PROFILE}" "${RECORD_SNAPSHOT}" || fail 'runtime record omitted profile'
+grep -Fxq "CONTEXT=${HEALTH_CONTEXT}" "${RECORD_SNAPSHOT}" || fail 'runtime record omitted context'
+grep -Fxq "WORKTREE=${ROOT}" "${RECORD_SNAPSHOT}" || fail 'runtime record omitted canonical worktree'
+grep -Fxq 'NAMESPACE=control-plane' "${RECORD_SNAPSHOT}" || fail 'runtime record omitted namespace'
+grep -Fxq 'SERVICE=control-ui' "${RECORD_SNAPSHOT}" || fail 'runtime record omitted service'
+grep -Fxq 'LOCAL_PORT=3000' "${RECORD_SNAPSHOT}" || fail 'runtime record omitted local port'
+grep -Fxq 'REMOTE_PORT=3000' "${RECORD_SNAPSHOT}" || fail 'runtime record omitted remote port'
+[[ ! -e "${HEALTH_PIDFILE}" ]] || fail 'required health failure left the canonical pidfile behind'
+launched_pid="$(sed -n '1p' "${LAUNCHED_PID}")"
+if /bin/kill -0 "${launched_pid}" 2>/dev/null; then
+ fail 'required health failure left the fake port-forward alive'
+fi
+pass 'required health failure is loud, cleans up, and uses explicit --context'
+
+printf 'PASS: Minikube port-forward ownership hardening\n'
diff --git a/scripts/tests/test-minikube-pre-gate-shadow.sh b/scripts/tests/test-minikube-pre-gate-shadow.sh
index c662fe945..eafa34e69 100755
--- a/scripts/tests/test-minikube-pre-gate-shadow.sh
+++ b/scripts/tests/test-minikube-pre-gate-shadow.sh
@@ -77,7 +77,14 @@ STUB
#!/usr/bin/env bash
printf 'minikube %s\n' "$*" >>"${TEST_LOG_FILE:?}"
case "$*" in
- *docker-env*) echo 'export DOCKER_HOST="tcp://127.0.0.1:2376"'; exit 0 ;;
+ *docker-env*)
+ if [ "${TEST_MINIKUBE_HANG_DOCKER_ENV:-0}" = "1" ]; then
+ trap '' TERM
+ while :; do sleep 1; done
+ fi
+ echo 'export DOCKER_HOST="tcp://127.0.0.1:2376"'
+ exit 0
+ ;;
esac
exit 0
STUB
@@ -953,6 +960,26 @@ assert_the_touched_scripts_parse() {
fi
}
+assert_incremental_runtime_calls_are_bounded() {
+ local d out rc=0
+ d="$(mktemp -d)"
+ prepare_repo "$d"
+ out="$(
+ PATH="$d/bin:$PATH" TEST_LOG_FILE="$d/ops.log" \
+ TEST_MINIKUBE_HANG_DOCKER_ENV=1 \
+ INCREMENTAL_RUNTIME_TIMEOUT_SECONDS=1 \
+ MINIKUBE_DOCKER_KILL_GRACE_SECONDS=1 \
+ run_incremental "$d" 'incremental_use_minikube_docker'
+ )" || rc=$?
+ if [ "$rc" -ne 0 ] \
+ && grep -Fq '[HARNESS_DEADLINE] label=incremental-docker-env event=timeout' <<< "$out"; then
+ pass "incremental Docker daemon selection terminates at its explicit deadline"
+ else
+ fail "incremental docker-env was not bounded (rc=$rc out=$out)"
+ fi
+ rm -rf "$d"
+}
+
assert_every_defined_case_is_invoked() {
local self defined invoked missing
self="$REPO_ROOT/scripts/tests/test-minikube-pre-gate-shadow.sh"
@@ -997,6 +1024,7 @@ assert_the_pinned_render_dir_is_not_gated_on_the_generated_patch
assert_an_unknown_image_source_is_rejected
assert_a_targeted_build_carries_the_recorded_coordinate_forward
assert_the_touched_scripts_parse
+assert_incremental_runtime_calls_are_bounded
assert_every_defined_case_is_invoked
exit $FAIL
diff --git a/scripts/tests/test-minikube-profile-owner.sh b/scripts/tests/test-minikube-profile-owner.sh
new file mode 100755
index 000000000..2e33de5a9
--- /dev/null
+++ b/scripts/tests/test-minikube-profile-owner.sh
@@ -0,0 +1,210 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+OWNER_SCRIPT="${ROOT}/scripts/minikube/profile-owner.sh"
+TMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/evenfire-profile-owner.XXXXXX")"
+trap 'rm -rf "${TMP_ROOT}"' EXIT
+
+fail() {
+ printf 'FAIL: %s\n' "$*" >&2
+ exit 1
+}
+
+value_from() {
+ local payload="$1" key="$2"
+ awk -F= -v wanted="${key}" '$1 == wanted { sub(/^[^=]*=/, ""); print; exit }' <<<"${payload}"
+}
+
+expect_failure() {
+ local expected_code="$1" label="$2"
+ shift 2
+ local output status=0
+ output="$("$@" 2>&1)" || status=$?
+ [[ "${status}" -ne 0 ]] || fail "${label} unexpectedly passed"
+ grep -Fq "${expected_code}" <<<"${output}" || {
+ printf '%s\n' "${output}" >&2
+ fail "${label} did not report ${expected_code}"
+ }
+}
+
+write_ports() {
+ local destination="$1" base="$2"
+ cat >"${destination}" <<EOF_PORTS
+PORT_BASE=${base}
+CONTROL_UI_PORT=$((base + 0))
+PROFILE_UI_PORT=$((base + 1))
+CONTROL_API_PORT=$((base + 90))
+EXTERNAL_REST_API_PORT=$((base + 91))
+MEMBER_REGISTRATION_SERVICE_PORT=$((base + 92))
+RPC_PROXY_PORT=$((base + 94))
+REGISTRY_API_PORT=$((base + 85))
+WORKFLOW_APPROVAL_READER_PORT=$((base + 98))
+MCP_HOST_PORT=$((base + 80))
+CONTROL_UI_URL=http://127.0.0.1:$((base + 0))
+PROFILE_UI_URL=http://127.0.0.1:$((base + 1))
+PROFILE_UI_BASE_URL=http://127.0.0.1:$((base + 1))
+CONTROL_API_URL=http://127.0.0.1:$((base + 90))
+EXTERNAL_REST_API_URL=http://127.0.0.1:$((base + 91))
+MEMBER_REGISTRATION_SERVICE_URL=http://127.0.0.1:$((base + 92))
+RPC_PROXY_URL=http://127.0.0.1:$((base + 94))
+REGISTRY_API_URL=http://127.0.0.1:$((base + 85))
+WORKFLOW_APPROVAL_READER_URL=http://127.0.0.1:$((base + 98))
+MCP_HOST_URL=http://127.0.0.1:$((base + 80))
+EOF_PORTS
+}
+
+write_v1_profile() {
+ local destination="$1" profile="$2" repo="$3" branch="$4" created_short="$5"
+ cat >"${destination}" <<EOF_PROFILE
+PROFILE=${profile}
+BRANCH=${branch}
+SHA_SHORT=${created_short}
+DIRTY=false
+REPO_DIR=${repo}
+EOF_PROFILE
+}
+
+write_v2_profile() {
+ local destination="$1" profile="$2" repo="$3" branch="$4" created_head="$5"
+ local identity worktree_id owner_id
+ identity="$("${OWNER_SCRIPT}" identity --repo-dir "${repo}" --branch "${branch}" --created-head "${created_head}")"
+ worktree_id="$(value_from "${identity}" WORKTREE_ID)"
+ owner_id="$(value_from "${identity}" OWNER_ID)"
+ cat >"${destination}" <<EOF_PROFILE
+PROFILE_SCHEMA_VERSION=2
+WORKTREE_ID=${worktree_id}
+OWNER_ID=${owner_id}
+CREATED_HEAD=${created_head}
+PROFILE=${profile}
+REPO_DIR=${repo}
+BRANCH=${branch}
+EOF_PROFILE
+}
+
+[[ -x "${OWNER_SCRIPT}" ]] || fail "profile-owner.sh is not executable"
+bash -n "${OWNER_SCRIPT}"
+
+repo_a="${TMP_ROOT}/worktree-a"
+repo_b="${TMP_ROOT}/worktree-b"
+mkdir -p "${repo_a}" "${repo_b}"
+repo_a="$(cd -- "${repo_a}" && pwd -P)"
+repo_b="$(cd -- "${repo_b}" && pwd -P)"
+branch='fix/stable-profile-owner'
+head_a='1111111111111111111111111111111111111111'
+head_b='2222222222222222222222222222222222222222'
+
+identity_a="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_a}" --branch "${branch}" --created-head "${head_a}")"
+identity_b="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_a}" --branch "${branch}" --created-head "${head_b}")"
+profile_a="$(value_from "${identity_a}" PROFILE)"
+profile_b="$(value_from "${identity_b}" PROFILE)"
+owner_a="$(value_from "${identity_a}" OWNER_ID)"
+owner_b="$(value_from "${identity_b}" OWNER_ID)"
+[[ "${profile_a}" == "${profile_b}" ]] || fail 'HEAD A -> B changed the stable profile name'
+[[ "${owner_a}" == "${owner_b}" ]] || fail 'HEAD A -> B changed the owner ID'
+[[ "$(value_from "${identity_a}" CREATED_HEAD)" == "${head_a}" ]] || fail 'identity omitted CREATED_HEAD A'
+[[ "$(value_from "${identity_b}" CREATED_HEAD)" == "${head_b}" ]] || fail 'identity omitted CREATED_HEAD B'
+
+repo_alias="${TMP_ROOT}/worktree-a-alias"
+ln -s "${repo_a}" "${repo_alias}"
+identity_alias="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_alias}" --branch "${branch}" --created-head "${head_a}")"
+[[ "$(value_from "${identity_alias}" OWNER_ID)" == "${owner_a}" ]] || fail 'canonical-path alias changed the owner ID'
+
+identity_other_worktree="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_b}" --branch "${branch}" --created-head "${head_a}")"
+[[ "$(value_from "${identity_other_worktree}" OWNER_ID)" != "${owner_a}" ]] || fail 'same branch in another worktree reused the owner ID'
+[[ "$(value_from "${identity_other_worktree}" PROFILE)" != "${profile_a}" ]] || fail 'same branch in another worktree reused the profile name'
+
+identity_other_branch="$("${OWNER_SCRIPT}" identity --repo-dir "${repo_a}" --branch 'fix/other-branch' --created-head "${head_a}")"
+[[ "$(value_from "${identity_other_branch}" OWNER_ID)" != "${owner_a}" ]] || fail 'branch switch reused the owner ID'
+[[ "$(value_from "${identity_other_branch}" PROFILE)" != "${profile_a}" ]] || fail 'branch switch reused the profile name'
+
+v2_root="${TMP_ROOT}/v2-cache"
+v2_dir="${v2_root}/${profile_a}"
+mkdir -p "${v2_dir}"
+write_v2_profile "${v2_dir}/profile.env" "${profile_a}" "${repo_a}" "${branch}" "${head_a}"
+write_ports "${v2_dir}/ports.env" 24000
+ports_before="$(shasum "${v2_dir}/ports.env" | awk '{print $1}')"
+v2_resolved="$("${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${v2_root}")"
+[[ "$(value_from "${v2_resolved}" PROFILE_SCHEMA_VERSION)" == 2 ]] || fail 'schema-v2 profile did not resolve as v2'
+[[ "$(value_from "${v2_resolved}" PROFILE)" == "${profile_a}" ]] || fail 'schema-v2 profile resolved the wrong profile'
+[[ "$(value_from "${v2_resolved}" OWNER_ID)" == "${owner_a}" ]] || fail 'schema-v2 profile resolved the wrong owner'
+[[ "$(shasum "${v2_dir}/ports.env" | awk '{print $1}')" == "${ports_before}" ]] || fail 'read-only resolution rewrote ports.env'
+
+corrupt_metadata_root="${TMP_ROOT}/corrupt-metadata"
+corrupt_metadata_dir="${corrupt_metadata_root}/${profile_a}"
+mkdir -p "${corrupt_metadata_dir}"
+cp "${v2_dir}/profile.env" "${corrupt_metadata_dir}/profile.env"
+printf 'BRANCH=%s\n' "${branch}" >>"${corrupt_metadata_dir}/profile.env"
+write_ports "${corrupt_metadata_dir}/ports.env" 24500
+expect_failure PROFILE_METADATA_INVALID 'auto-selected corrupt metadata' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${corrupt_metadata_root}"
+
+bad_owner_env="${TMP_ROOT}/bad-owner.env"
+cp "${v2_dir}/profile.env" "${bad_owner_env}"
+sed -i.bak 's/^OWNER_ID=.*/OWNER_ID=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/' "${bad_owner_env}"
+expect_failure PROFILE_OWNERSHIP_MISMATCH 'mismatched schema-v2 owner' \
+ "${OWNER_SCRIPT}" validate --repo-dir "${repo_a}" --branch "${branch}" \
+ --profile "${profile_a}" --profile-env "${bad_owner_env}" --ports-env "${v2_dir}/ports.env"
+
+v1_root="${TMP_ROOT}/v1-cache"
+legacy_profile='clerum-fix-stable-profile-owner-deadbeef'
+legacy_dir="${v1_root}/${legacy_profile}"
+mkdir -p "${legacy_dir}"
+write_v1_profile "${legacy_dir}/profile.env" "${legacy_profile}" "${repo_a}" "${branch}" deadbeef
+write_ports "${legacy_dir}/ports.env" 25000
+v1_resolved="$("${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${v1_root}")"
+[[ "$(value_from "${v1_resolved}" PROFILE_SCHEMA_VERSION)" == 1 ]] || fail 'legacy profile did not resolve as schema v1'
+[[ "$(value_from "${v1_resolved}" PROFILE)" == "${legacy_profile}" ]] || fail 'legacy profile was renamed during adoption'
+[[ "$(value_from "${v1_resolved}" CREATED_HEAD)" == deadbeef ]] || fail 'legacy SHA_SHORT was not retained as creation metadata'
+[[ "$(value_from "${v1_resolved}" OWNER_ID)" == "${owner_a}" ]] || fail 'legacy adoption did not derive the stable owner ID'
+
+second_legacy='clerum-fix-stable-profile-owner-cafebabe'
+second_dir="${v1_root}/${second_legacy}"
+mkdir -p "${second_dir}"
+write_v1_profile "${second_dir}/profile.env" "${second_legacy}" "${repo_a}" "${branch}" cafebabe
+write_ports "${second_dir}/ports.env" 26000
+expect_failure PROFILE_SELECTION_AMBIGUOUS 'ambiguous legacy adoption' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${v1_root}"
+
+explicit_legacy="$("${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" \
+ --profile-root "${v1_root}" --profile "${legacy_profile}")"
+[[ "$(value_from "${explicit_legacy}" PROFILE)" == "${legacy_profile}" ]] || fail 'explicit profile did not disambiguate selection'
+
+ports_only_legacy="$("${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" \
+ --profile-root "${v1_root}" --ports-env "${legacy_dir}/ports.env")"
+[[ "$(value_from "${ports_only_legacy}" PROFILE)" == "${legacy_profile}" ]] || fail 'explicit ports.env did not bind to its sibling profile metadata'
+
+missing_ports_root="${TMP_ROOT}/missing-ports"
+missing_ports_dir="${missing_ports_root}/${legacy_profile}"
+mkdir -p "${missing_ports_dir}"
+write_v1_profile "${missing_ports_dir}/profile.env" "${legacy_profile}" "${repo_a}" "${branch}" deadbeef
+expect_failure PROFILE_PORTS_MISSING 'missing persisted ports' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${missing_ports_root}"
+
+printf 'PORT_BASE=not-a-port\n' >"${missing_ports_dir}/ports.env"
+expect_failure PROFILE_PORTS_INVALID 'corrupt persisted ports' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${missing_ports_root}"
+
+symlink_ports_root="${TMP_ROOT}/symlink-ports"
+symlink_ports_dir="${symlink_ports_root}/${legacy_profile}"
+mkdir -p "${symlink_ports_dir}"
+write_v1_profile "${symlink_ports_dir}/profile.env" "${legacy_profile}" "${repo_a}" "${branch}" deadbeef
+ln -s "${legacy_dir}/ports.env" "${symlink_ports_dir}/ports.env"
+expect_failure PROFILE_PORTS_MISSING 'symlinked persisted ports' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" --profile-root "${symlink_ports_root}"
+
+missing_metadata_root="${TMP_ROOT}/missing-metadata"
+mkdir -p "${missing_metadata_root}/${legacy_profile}"
+write_ports "${missing_metadata_root}/${legacy_profile}/ports.env" 27000
+expect_failure PROFILE_METADATA_MISSING 'explicit profile without metadata' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" \
+ --profile-root "${missing_metadata_root}" --profile "${legacy_profile}"
+
+missing_stable_metadata_root="${TMP_ROOT}/missing-stable-metadata"
+mkdir -p "${missing_stable_metadata_root}/${profile_a}"
+write_ports "${missing_stable_metadata_root}/${profile_a}/ports.env" 27500
+expect_failure PROFILE_METADATA_MISSING 'stable profile directory without metadata' \
+ "${OWNER_SCRIPT}" resolve --repo-dir "${repo_a}" --branch "${branch}" \
+ --profile-root "${missing_stable_metadata_root}"
+
+printf 'PASS: stable Minikube profile ownership and v1/v2 resolution\n'
diff --git a/scripts/tests/test-minikube-t1-docker-boundary.sh b/scripts/tests/test-minikube-t1-docker-boundary.sh
new file mode 100755
index 000000000..a714977cb
--- /dev/null
+++ b/scripts/tests/test-minikube-t1-docker-boundary.sh
@@ -0,0 +1,64 @@
+#!/usr/bin/env bash
+set -euo pipefail
+set +x
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+T1="$ROOT/scripts/e2e/minikube-real-postgres.sh"
+PREFLIGHT="$ROOT/scripts/e2e/real-postgres-local-preflight.sh"
+TMP_ROOT="${TMPDIR:-/tmp}"
+TEST_DIR="$(mktemp -d "$TMP_ROOT/evenfire-t1-docker.XXXXXX")"
+trap 'rm -rf -- "$TEST_DIR"' EXIT
+
+# shellcheck source=scripts/e2e/minikube-real-postgres.sh
+source "$T1"
+trap 'rm -rf -- "$TEST_DIR"' EXIT
+
+fail() {
+ printf 'FAIL: %s\n' "$*" >&2
+ exit 1
+}
+
+CALLS="$TEST_DIR/calls"
+docker_cli_run_public() {
+ printf '%s\n' "$@" >"$CALLS"
+}
+
+# Consumed by the sourced production helper; ShellCheck cannot follow that
+# cross-file function body.
+# shellcheck disable=SC2034
+T1_DOCKER_ENV_PREPARED=true
+# shellcheck disable=SC2034
+T1_DOCKER_REMOVE_TIMEOUT_SECONDS=17
+ISOLATED_CONTAINER=evenfire-test-postgres
+stop_isolated_postgres || fail 'bounded isolated-container cleanup failed'
+[[ -z "$ISOLATED_CONTAINER" ]] || fail 'successful cleanup retained the container identity'
+python3 - "$CALLS" <<'PY'
+from pathlib import Path
+import sys
+
+assert Path(sys.argv[1]).read_text().splitlines() == [
+ "t1-postgres-remove",
+ "17",
+ "docker",
+ "rm",
+ "-f",
+ "evenfire-test-postgres",
+]
+PY
+
+grep -Fq 'bash "$REAL_PG_DOCKER_HELPER" --check-info' "$PREFLIGHT"
+grep -Fq 'docker_cli_env_prepare false' "$T1"
+grep -Fq 'docker_cli_run_public t1-postgres-run' "$T1"
+grep -Fq 'docker_cli_run_public t1-postgres-ready-probe' "$T1"
+grep -Fq 'docker_cli_run_public t1-postgres-remove' "$T1"
+
+awk '
+ /^[[:space:]]*docker (run|exec|rm) / {
+ if (previous !~ /docker_cli_run_public/ && before_previous !~ /docker_cli_run_public/) {
+ exit 1
+ }
+ }
+ { before_previous = previous; previous = $0 }
+' "$T1" || fail 'T1 still runs an ambient or unbounded Docker operation'
+
+printf 'PASS: T1 Docker operations are isolated, bounded, and fail-loud\n'
diff --git a/scripts/tests/test-minikube-t1-port-forward-owner.sh b/scripts/tests/test-minikube-t1-port-forward-owner.sh
new file mode 100755
index 000000000..a76e7d63a
--- /dev/null
+++ b/scripts/tests/test-minikube-t1-port-forward-owner.sh
@@ -0,0 +1,80 @@
+#!/usr/bin/env bash
+# T1 must delegate cleanup to the exact structured ownership contract.
+set -euo pipefail
+set +x
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+T1="$ROOT/scripts/e2e/minikube-real-postgres.sh"
+TMP_ROOT="${TMPDIR:-/tmp}"
+TEST_DIR="$(mktemp -d "$TMP_ROOT/evenfire-t1-pf-owner.XXXXXX")"
+trap 'rm -rf -- "$TEST_DIR"' EXIT
+
+# shellcheck source=scripts/e2e/minikube-real-postgres.sh
+source "$T1"
+
+T2_PROFILE=clerum-feature-owner
+T2_CONTEXT=clerum-feature-owner
+PG_NAMESPACE=control-plane
+PG_SERVICE=control-postgres
+LOCAL_PORT=32123
+PORT_FORWARD_PID=4242
+PORT_FORWARD_RECORD="$TEST_DIR/control-postgres.pid"
+CALLS="$TEST_DIR/calls"
+: >"$PORT_FORWARD_RECORD"
+
+pf_owner_cleanup_record() {
+ printf '%s\n' "$@" >"$CALLS"
+ rm -f -- "$1"
+}
+
+stop_control_postgres_forward
+python3 - "$CALLS" "$ROOT" <<'PY'
+import sys
+from pathlib import Path
+
+values = Path(sys.argv[1]).read_text().splitlines()
+assert values[1:] == [
+ "clerum-feature-owner",
+ "clerum-feature-owner",
+ sys.argv[2],
+ "control-plane",
+ "control-postgres",
+ "32123",
+ "5432",
+]
+PY
+[ -z "$PORT_FORWARD_PID" ] && [ -z "$PORT_FORWARD_RECORD" ] || {
+ printf 'FAIL: successful exact cleanup did not clear T1 ownership state\n' >&2
+ exit 1
+}
+
+PORT_FORWARD_PID=4343
+PORT_FORWARD_RECORD="$TEST_DIR/missing-live.pid"
+pf_owner_process_state() { printf 'live\n'; }
+if stop_control_postgres_forward >"$TEST_DIR/missing-live.out" 2>&1; then
+ printf 'FAIL: missing ownership record was accepted for a possibly live process\n' >&2
+ exit 1
+fi
+[ "$PORT_FORWARD_PID" = 4343 ] || {
+ printf 'FAIL: ambiguous live process state was discarded\n' >&2
+ exit 1
+}
+
+PORT_FORWARD_PID=4444
+PORT_FORWARD_RECORD="$TEST_DIR/missing-dead.pid"
+pf_owner_process_state() { printf 'dead\n'; }
+stop_control_postgres_forward
+[ -z "$PORT_FORWARD_PID" ] && [ -z "$PORT_FORWARD_RECORD" ] || {
+ printf 'FAIL: a proven-dead child did not clear stale T1 state\n' >&2
+ exit 1
+}
+
+grep -Fq 'pf_owner_record_process "$PORT_FORWARD_RECORD" "$PORT_FORWARD_PID"' "$T1"
+grep -Fq 'pf_owner_abort_child "$PORT_FORWARD_PID"' "$T1"
+grep -Fq 'pf_owner_cleanup_record "$PORT_FORWARD_RECORD"' "$T1"
+if grep -Eq 'command_line=.*port-forward|command_line.*svc/control-postgres' "$T1"; then
+ printf 'FAIL: T1 still uses fuzzy process-command ownership matching\n' >&2
+ exit 1
+fi
+
+printf 'PASS: T1 port-forward lifecycle uses exact structured ownership\n'
diff --git a/scripts/tests/test-minikube-t2-contract.sh b/scripts/tests/test-minikube-t2-contract.sh
index a3d4a5f0a..96838d91e 100755
--- a/scripts/tests/test-minikube-t2-contract.sh
+++ b/scripts/tests/test-minikube-t2-contract.sh
@@ -7,6 +7,7 @@ set +u
ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
MINIKUBE_DIR="$ROOT/scripts/minikube"
T1="$ROOT/scripts/e2e/minikube-real-postgres.sh"
+T1_LOCAL_PREFLIGHT="$ROOT/scripts/e2e/real-postgres-local-preflight.sh"
PREFLIGHT="$MINIKUBE_DIR/t2-preflight.sh"
T2="$MINIKUBE_DIR/t2.sh"
T0="$MINIKUBE_DIR/t0.sh"
@@ -17,6 +18,7 @@ if [ -z "$TMP_ROOT" ]; then TMP_ROOT=/tmp; fi
set -u
for file in "$MINIKUBE_DIR/profile-readiness.sh" "$ROOT/scripts/tests/test-minikube-profile-readiness.sh" "$COMMON" "$PREFLIGHT" "$T2" "$T1" "$T0" \
+ "$ROOT/scripts/e2e/real-postgres-local-preflight.sh" \
"$ROOT/scripts/e2e/e2e-np08-hcc-authorization.sh" \
"$ROOT/scripts/minikube/settle-gfs-reader-rollout.sh" \
"$ROOT/scripts/minikube/wait-gfs-reader-ready.sh" \
@@ -33,10 +35,23 @@ for file in "$MINIKUBE_DIR/profile-readiness.sh" "$ROOT/scripts/tests/test-minik
"$ROOT/scripts/tests/test-minikube-mutation-boundary.sh" \
"$ROOT/scripts/tests/test-minikube-targeted-gfs-sync.sh" \
"$ROOT/scripts/tests/test-minikube-t2-lock-race.sh" \
- "$ROOT/scripts/tests/test-minikube-t1-gfs-restore.sh"; do
+ "$ROOT/scripts/tests/test-minikube-t1-gfs-restore.sh" \
+ "$ROOT/scripts/tests/test-real-postgres-local-preflight.sh" \
+ "$ROOT/scripts/tests/test-minikube-t1-docker-boundary.sh" \
+ "$ROOT/scripts/tests/test-minikube-t1-port-forward-owner.sh" \
+ "$ROOT/scripts/tests/test-minikube-t2-process-owner.sh" \
+ "$ROOT/scripts/tests/test-minikube-explicit-context.sh" \
+ "$ROOT/scripts/tests/test-minikube-t2-evidence.sh" \
+ "$ROOT/scripts/tests/test-minikube-targeted-health.sh"; do
bash -n "$file"
done
+"$ROOT/scripts/tests/test-minikube-t1-port-forward-owner.sh"
+"$ROOT/scripts/tests/test-minikube-t2-process-owner.sh"
+"$ROOT/scripts/tests/test-minikube-explicit-context.sh"
+"$ROOT/scripts/tests/test-minikube-t2-evidence.sh"
+"$ROOT/scripts/tests/test-minikube-targeted-health.sh"
"$ROOT/scripts/tests/test-minikube-filter-gfs-resources.sh"
+"$ROOT/scripts/tests/test-minikube-t1-docker-boundary.sh"
python3 - "$GFS_FILTER" "$TMP_ROOT" <<'PY'
import os
import py_compile
@@ -67,27 +82,16 @@ for identity_file in "$COMMON" "$ROOT/scripts/minikube/sync-auth-key.sh" "$ROOT/
done
grep -Fq 'bash "$T2_PROJECT_DIR/scripts/tests/test-minikube-t2-contract.sh"' "$T2"
-required_codes="DEVELOPMENT_SCOPE_REQUIRED PROFILE_OWNERSHIP_MISMATCH PROFILE_BUSY PROFILE_LOCK_REQUIRED HEAD_MARKER_MISMATCH IMAGE_MANIFEST_MISMATCH BOOTSTRAP_REQUIRED CERTIFICATION_REQUIRED SECRET_MISSING CONFIGMAP_MISSING POSTGRES_NOT_READY REAL_PG_REQUIRED_BUT_UNAVAILABLE REAL_PG_SUITE_FAILED ZERO_TESTS_EXECUTED PORT_FORWARD_CONFLICT NP08_HCC_AUTHORIZATION_FAILED"
+required_codes="DEVELOPMENT_SCOPE_REQUIRED PROFILE_OWNERSHIP_MISMATCH PROFILE_BUSY PROFILE_LOCK_REQUIRED HEAD_MARKER_MISMATCH IMAGE_MANIFEST_MISMATCH BOOTSTRAP_REQUIRED CERTIFICATION_REQUIRED SECRET_MISSING CONFIGMAP_MISSING POSTGRES_NOT_READY REAL_PG_REQUIRED_BUT_UNAVAILABLE REAL_PG_SUITE_FAILED REAL_PG_REPORT_INCOMPLETE UNSUPPORTED_T1_CONCURRENCY ZERO_TESTS_EXECUTED PORT_FORWARD_CONFLICT NP08_HCC_AUTHORIZATION_FAILED PLAYWRIGHT_FAILED"
for code in $required_codes; do
- grep -Fq "$code" "$COMMON" "$PREFLIGHT" "$T2" "$T1"
+ grep -Fq "$code" "$COMMON" "$PREFLIGHT" "$T2" "$T1" "$T1_LOCAL_PREFLIGHT"
done
grep -Fq 'kubectl --context=' "$COMMON"
-if grep -Eq 'while IFS= read -r uid pid ppid rest' "$COMMON"; then
- echo 'FAIL: t2_process_check IFS= prevents UID/PID split' >&2
- exit 1
-fi
-grep -Fq 'while read -r uid pid ppid rest' "$COMMON"
-if grep -Fq '[0-9]+:[0-9]+(:[0-9]+)?(\.[0-9]+)?' "$COMMON"; then
- echo 'FAIL: t2_process_check awk still anchors on a TIME-column regex' >&2
- exit 1
-fi
-if grep -Fq 'kubectl[[:space:]]+port-forward' "$COMMON"; then
- echo 'FAIL: t2_process_check awk still requires kubectl/port-forward adjacency' >&2
- exit 1
-fi
-grep -Fq '([^[:space:]]*\/)?kubectl([[:space:]]|$)' "$COMMON"
-grep -Fq '[[:space:]]port-forward([[:space:]]|$)' "$COMMON"
+grep -Fq 'port-forward-owner.sh' "$COMMON"
+grep -Fq 't2_port_forward_targets_context' "$COMMON"
+grep -Fq 'matching_records' "$COMMON"
+grep -Fq 'pf_owner_command_matches' "$COMMON"
grep -Fq 'CONTROL_API_REAL_PG_CONTEXT' "$T1"
grep -Fq 'T2_REQUIRED_DEPLOYMENTS' "$COMMON"
grep -Fq 'CONTROL_API_REAL_PG_ADMIN_URL=' "$T1"
@@ -119,6 +123,20 @@ grep -Fq 'pending_tests' "$T1"
grep -Fq 'numTotalTests' "$T1"
grep -Fq 'numTotalTestSuites' "$T1"
grep -Fq 'passed_suites' "$T1"
+grep -Fq 'testResults' "$T1"
+grep -Fq 'T1_EXPECTED_FILES' "$T1"
+grep -Fq 'T1_REPORTED_FILES' "$T1"
+grep -Fq -- '--maxWorkers=1' "$T1"
+grep -Fq 'run_t1_local_preflight' "$T2"
+grep -Fq 'T2_SETUP_HANDOFF_REQUIRED=true' "$T2"
+grep -Fq 't2-setup-handoff.sh" consume --' "$ROOT/scripts/minikube/pre-gate-sync.sh"
+grep -Fq 'make minikube-verify-images' "$ROOT/scripts/minikube/pre-gate-sync.sh"
+for timed_phase in 't2_evidence_write planner PASS' 't2_evidence_write transition PASS' \
+ 't2_evidence_write pre-gate-sync PASS' 't2_evidence_write T1 PASS' \
+ 't2_evidence_write T2 PASS' 't2_evidence_write NP08_HCC_AUTHORIZATION PASS'; do
+ grep -Fq "$timed_phase" "$T2"
+done
+grep -Fq 'duration=' "$T2"
grep -Fq 'GFS_RESTORE_ACTIVE_NOLOGIN=true' "$T1"
grep -Fq 'reconcile-gfs-deploy-credentials.sh' "$T1"
if grep -Fq "[ \"\$passed_files\" -ne \"\$expected\" ]" "$T1"; then
@@ -134,6 +152,8 @@ if grep -Eq 't2_kc[^|]*port-forward' "$T1"; then
exit 1
fi
grep -Fq 'kubectl --context="$T2_CONTEXT" -n "$PG_NAMESPACE" port-forward' "$T1"
+grep -Fq 'pf_owner_record_process "$PORT_FORWARD_RECORD" "$PORT_FORWARD_PID"' "$T1"
+grep -Fq 'pf_owner_cleanup_record "$PORT_FORWARD_RECORD"' "$T1"
grep -Fq 'set +x' "$T1" "$PREFLIGHT" "$T2"
# T1 must restore the branch-profile GFS credentials on the way out with the
@@ -257,6 +277,10 @@ grep -Fq 'T2_NP08_HCC_AUTHORIZATION_STATUS=NOT_RUN' "$T2"
grep -Fq 't2_lane_completed "$T2_T0_STATUS"' "$T2"
grep -Fq 't2_lane_completed "$T2_T1_STATUS"' "$T2"
grep -Fq 'T2_HEALTHCHECK_COMMAND' "$T2"
+grep -Fq 'validate_healthcheck_contract' "$T2"
+grep -Fq 'T2_HEALTHCHECK_REQUIRED=true' "$T2"
+grep -Fq -- '--label t2-user-facing-health' "$T2"
+grep -Fq 'Health=$T2_HEALTH_STATUS' "$T2"
grep -Fq 'run_np08_hcc_authorization' "$T2"
grep -Fq "CLERUM_PROFILE_PORTS_ENV=\"\$T2_PORTS_ENV\"" "$T2"
grep -Fq 'NP08_HCC_AUTHORIZATION PASS' "$T2"
@@ -527,16 +551,17 @@ for exit_code in 1 2 7 126 137; do
esac
shift
done
- printf "%s\\n" '\''{"success":true,"testResults":[{"status":"passed"}],"numPassedTests":1,"numFailedTests":0,"numPendingTestSuites":0,"numPendingTests":0,"numTotalTests":1}'\'' >"$output_file"
+ printf '\''{"success":true,"testResults":[{"name":"%s","status":"passed"}],"numTotalTestSuites":1,"numPassedTestSuites":1,"numFailedTestSuites":0,"numPassedTests":1,"numFailedTests":0,"numPendingTestSuites":0,"numPendingTests":0,"numTotalTests":1}\n'\'' \
+ "$T1_EXIT_ROOT/control-api/test/fake.realPostgres.test.ts" >"$output_file"
return "$EXIT_CODE"
}
- if run_suite control-api isolated postgresql://isolated; then
- exit 1
- fi
- ' bash "$T1"; then
+ run_suite control-api isolated postgresql://isolated
+ exit 0
+ ' bash "$T1" 2>"$tmp/t1-exit-$exit_code.err"; then
echo "FAIL: T1 accepted a green reporter after npm exit $exit_code" >&2
exit 1
fi
+ grep -Fq "Vitest process exited $exit_code" "$tmp/t1-exit-$exit_code.err"
done
bash "$ROOT/scripts/tests/test-minikube-t2-public-boundary.sh"
@@ -575,5 +600,6 @@ bash "$ROOT/scripts/tests/test-minikube-mutation-boundary.sh"
bash "$ROOT/scripts/tests/test-minikube-targeted-gfs-sync.sh"
bash "$ROOT/scripts/tests/test-minikube-t2-lock-race.sh"
bash "$ROOT/scripts/tests/test-minikube-t1-gfs-restore.sh"
+bash "$ROOT/scripts/tests/test-real-postgres-local-preflight.sh"
printf 'PASS: local Minikube T0/T1/T2 contract checks\n'
diff --git a/scripts/tests/test-minikube-t2-evidence.sh b/scripts/tests/test-minikube-t2-evidence.sh
new file mode 100755
index 000000000..595c10321
--- /dev/null
+++ b/scripts/tests/test-minikube-t2-evidence.sh
@@ -0,0 +1,77 @@
+#!/usr/bin/env bash
+# Behavioral contract for planner/certification evidence separation.
+set -euo pipefail
+set +x
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+COMMON="$ROOT/scripts/minikube/t2-common.sh"
+TMP_ROOT="${TMPDIR:-/tmp}"
+TEST_DIR="$(mktemp -d "$TMP_ROOT/evenfire-t2-evidence.XXXXXX")"
+trap 'rm -rf -- "$TEST_DIR"' EXIT
+
+write_evidence() {
+ local kind="$1" directory="$2" planner_result="${3:-PASS}"
+ T2_EVIDENCE_KIND="$kind" \
+ PLANNER_RESULT="$planner_result" \
+ T2_EVIDENCE_ROOT="$directory" \
+ T2_PROJECT_DIR="$ROOT" \
+ T2_PROFILE=clerum-feature-owner \
+ T2_CONTEXT=clerum-feature-owner \
+ bash -c '
+ set -euo pipefail
+ source "$1"
+ T2_BRANCH=feature/evidence
+ T2_HEAD=0123456789abcdef0123456789abcdef01234567
+ T2_ORIGIN_DEV=abcdef0123456789abcdef0123456789abcdef01
+ T2_MERGE_BASE="$T2_ORIGIN_DEV"
+ T2_WORKTREE_ID=worktree-owner
+ T2_CLUSTER_FINGERPRINT=cluster-fingerprint
+ t2_evidence_init
+ if [ "$T2_EVIDENCE_KIND" = planner ]; then
+ if [ "$PLANNER_RESULT" = PASS ]; then
+ t2_evidence_write preflight PASS "targeted-sync duration=1.25s"
+ else
+ t2_evidence_write failure FAIL "profile unavailable duration=250ms"
+ fi
+ else
+ t2_evidence_write T0 PASS "typecheck duration=120ms"
+ t2_evidence_write complete PASS "all required phases complete durationSeconds=2"
+ fi
+ printf "%s\n" "$T2_EVIDENCE_FILE"
+ ' _ "$COMMON"
+}
+
+planner_file="$(write_evidence planner "$TEST_DIR/planner")"
+planner_failure_file="$(write_evidence planner "$TEST_DIR/planner-failure" FAIL)"
+certification_file="$(write_evidence certification "$TEST_DIR/certification")"
+
+python3 - "$planner_file" "$planner_failure_file" "$certification_file" <<'PY'
+import json
+import sys
+from pathlib import Path
+
+planner = json.loads(Path(sys.argv[1]).read_text())
+assert planner["evidenceKind"] == "planner"
+assert "certificationVersion" not in planner
+assert planner["plannerStatus"] == "PASS"
+assert planner["attestationStatus"] == "NOT_APPLICABLE"
+assert planner["laneAttestationStatus"] == "NOT_APPLICABLE"
+planner_phase = planner["phases"][-1]
+assert planner_phase["detail"] == "targeted-sync duration=1.25s"
+assert planner_phase["durationSeconds"] == 1.25
+
+planner_failure = json.loads(Path(sys.argv[2]).read_text())
+assert planner_failure["plannerStatus"] == "FAIL"
+assert planner_failure["attestationStatus"] == "NOT_APPLICABLE"
+assert planner_failure["phases"][-1]["durationSeconds"] == 0.25
+
+certification = json.loads(Path(sys.argv[3]).read_text())
+assert certification["evidenceKind"] == "certification"
+assert certification["certificationVersion"] == 1
+assert certification["attestationStatus"] == "PASS"
+assert "plannerStatus" not in certification
+assert certification["phases"][-2]["durationSeconds"] == 0.12
+assert certification["phases"][-1]["durationSeconds"] == 2.0
+PY
+
+printf 'PASS: T2 planner and certification evidence have distinct terminal semantics\n'
diff --git a/scripts/tests/test-minikube-t2-process-owner.sh b/scripts/tests/test-minikube-t2-process-owner.sh
new file mode 100755
index 000000000..c6d1240e6
--- /dev/null
+++ b/scripts/tests/test-minikube-t2-process-owner.sh
@@ -0,0 +1,110 @@
+#!/usr/bin/env bash
+# T2 attributes only exact-context kubectl forwards and accepts exactly one
+# structured ownership record bound to this profile and worktree.
+set -euo pipefail
+set +x
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+COMMON="$ROOT/scripts/minikube/t2-common.sh"
+OWNER="$ROOT/scripts/minikube/port-forward-owner.sh"
+TMP_ROOT="${TMPDIR:-/tmp}"
+TEST_DIR="$(mktemp -d "$TMP_ROOT/evenfire-t2-process-owner.XXXXXX")"
+trap 'rm -rf -- "$TEST_DIR"' EXIT
+
+PROFILE=clerum-t2-pf-fixture
+PID=4242
+START='Sun Aug 23 10:11:12 2026'
+PROFILE_ROOT="$TEST_DIR/profiles"
+PID_DIR="$PROFILE_ROOT/$PROFILE/pids"
+PS_DATA="$TEST_DIR/ps-data"
+PS_BIN="$TEST_DIR/bin"
+PS_EF="$PS_DATA/ef"
+RECORD="$PID_DIR/control-ui.pid"
+mkdir -p "$PID_DIR" "$PS_DATA" "$PS_BIN"
+
+cat >"$PS_BIN/ps" <<'EOF_PS'
+#!/usr/bin/env bash
+pid=''
+mode=''
+if [ "${1:-}" = -ef ]; then
+ [ ! -f "${T2_PS_DATA_DIR:?}/ef" ] || cat "${T2_PS_DATA_DIR}/ef"
+ exit 0
+fi
+while [ "$#" -gt 0 ]; do
+ case "$1" in
+ -p) pid="$2"; shift 2 ;;
+ -o) mode="$2"; shift 2 ;;
+ *) shift ;;
+ esac
+done
+[ -n "$pid" ] || exit 1
+case "$mode" in
+ comm=) suffix=comm ;;
+ command=) suffix=command ;;
+ lstart=) suffix=start ;;
+ pid=) suffix=pid ;;
+ *) exit 1 ;;
+esac
+[ ! -f "${T2_PS_DATA_DIR:?}/${pid}.${suffix}" ] || cat "${T2_PS_DATA_DIR}/${pid}.${suffix}"
+EOF_PS
+chmod +x "$PS_BIN/ps"
+
+set_process_fixture() {
+ local comm="$1" command="$2"
+ printf '%s\n' "user $PID 1 0 10:00 ttys000 0:00 $command" >"$PS_EF"
+ printf '%s\n' "$comm" >"$PS_DATA/$PID.comm"
+ printf '%s\n' "$command" >"$PS_DATA/$PID.command"
+ printf '%s\n' "$START" >"$PS_DATA/$PID.start"
+ printf '%s\n' "$PID" >"$PS_DATA/$PID.pid"
+}
+
+run_process_check() {
+ local output="$1"
+ T2_PS_DATA_DIR="$PS_DATA" PATH="$PS_BIN:$PATH" \
+ T2_PROFILE="$PROFILE" T2_CONTEXT="$PROFILE" MINIKUBE_PROFILE="$PROFILE" \
+ T2_PROFILE_ROOT="$PROFILE_ROOT" T2_PROJECT_DIR="$ROOT" \
+ bash -c 'source "$1"; t2_process_check' _ "$COMMON" >"$output" 2>&1
+}
+
+exact_command="kubectl --context=$PROFILE -n control-plane port-forward --address=127.0.0.1 svc/control-ui 3000:3000"
+set_process_fixture kubectl "$exact_command"
+if run_process_check "$TEST_DIR/unowned.out"; then
+ printf 'FAIL: exact-context forward without a record was accepted\n' >&2
+ exit 1
+fi
+grep -Fq PORT_FORWARD_CONFLICT "$TEST_DIR/unowned.out"
+
+set_process_fixture bash "bash -c echo kubectl --context=$PROFILE port-forward"
+run_process_check "$TEST_DIR/wrapper.out"
+
+set_process_fixture kubectl 'kubectl -n control-plane port-forward svc/control-ui 3000:3000'
+run_process_check "$TEST_DIR/contextless.out"
+
+set_process_fixture kubectl 'kubectl --context=another-profile -n control-plane port-forward svc/control-ui 3000:3000'
+run_process_check "$TEST_DIR/other-context.out"
+
+# shellcheck source=scripts/minikube/port-forward-owner.sh
+source "$OWNER"
+set_process_fixture kubectl "$exact_command"
+pf_owner_write_record_atomic "$RECORD" "$PID" "$START" "$PROFILE" \
+ "$PROFILE" "$ROOT" control-plane control-ui 3000 3000
+run_process_check "$TEST_DIR/exact.out"
+
+printf '%s\n' 'different process start' >"$PS_DATA/$PID.start"
+if run_process_check "$TEST_DIR/reused.out"; then
+ printf 'FAIL: process-start mismatch was accepted\n' >&2
+ exit 1
+fi
+grep -Fq PORT_FORWARD_CONFLICT "$TEST_DIR/reused.out"
+printf '%s\n' "$START" >"$PS_DATA/$PID.start"
+
+SECOND_RECORD="$PID_DIR/control-ui-duplicate.pid"
+pf_owner_write_record_atomic "$SECOND_RECORD" "$PID" "$START" "$PROFILE" \
+ "$PROFILE" "$ROOT" control-plane control-ui 3000 3000
+if run_process_check "$TEST_DIR/duplicate.out"; then
+ printf 'FAIL: duplicate exact ownership records were accepted\n' >&2
+ exit 1
+fi
+grep -Fq PORT_FORWARD_CONFLICT "$TEST_DIR/duplicate.out"
+
+printf 'PASS: T2 process inventory requires one exact structured port-forward owner\n'
diff --git a/scripts/tests/test-minikube-t2-scenarios.sh b/scripts/tests/test-minikube-t2-scenarios.sh
index 475efb6ca..f3e0548c8 100755
--- a/scripts/tests/test-minikube-t2-scenarios.sh
+++ b/scripts/tests/test-minikube-t2-scenarios.sh
@@ -32,6 +32,8 @@ expect_code() {
}
}
+"$ROOT/scripts/tests/test-minikube-profile-owner.sh"
+
repo="$tmp/evenfire"
mkdir -p "$repo"
repo="$(cd "$repo" && pwd -P)"
@@ -55,7 +57,7 @@ profile_root="$tmp/profiles/$profile"
mkdir -p "$profile_root"
printf 'PROFILE=%s\nBRANCH=feat/scenario\nSHA_SHORT=%s\nDIRTY=false\nREPO_DIR=%s\n' \
"$profile" "$(git -C "$repo" rev-parse --short=8 HEAD)" "$repo" >"$profile_root/profile.env"
-printf 'PORT_BASE=23117\nCONTROL_API_URL=profile-owned-url\n' >"$profile_root/ports.env"
+printf 'PORT_BASE=23117\nCONTROL_API_PORT=23207\nCONTROL_API_URL=http://127.0.0.1:23207\n' >"$profile_root/ports.env"
repo_env=(
T2_PROJECT_DIR="$repo"
@@ -82,7 +84,7 @@ expect_code DEVELOPMENT_SCOPE_REQUIRED protected-branch protected-branch \
git -C "$repo" switch -q feat/scenario
missing_profile_env=("${repo_env[@]}" T2_PROFILE_ENV="$tmp/missing-profile.env")
-expect_code DEVELOPMENT_SCOPE_REQUIRED missing-profile missing-profile \
+expect_code PROFILE_OWNERSHIP_MISMATCH missing-profile missing-profile \
env "${missing_profile_env[@]}" bash -c 'source "$1"; t2_profile_scope' bash "$COMMON"
expect_code DEVELOPMENT_SCOPE_REQUIRED shared-profile shared-profile \
@@ -158,11 +160,39 @@ expect_code PROFILE_OWNERSHIP_MISMATCH profile-ownership profile-ownership \
env "${ownership_env[@]}" bash -c 'source "$1"; t2_profile_scope' bash "$COMMON"
stale_profile_env="$tmp/stale-profile.env"
-printf 'PROFILE=%s\nBRANCH=feat/scenario\nSHA_SHORT=oldsha1\nDIRTY=false\nREPO_DIR=%s\n' \
+printf 'PROFILE=%s\nBRANCH=feat/scenario\nSHA_SHORT=deadbeef\nDIRTY=false\nREPO_DIR=%s\n' \
"$profile" "$repo" >"$stale_profile_env"
env "${repo_env[@]}" T2_PROFILE_ENV="$stale_profile_env" \
bash -c 'source "$1"; t2_repo_metadata; t2_profile_scope' bash "$COMMON"
+worktree_id="$(bash -c 'source "$1"; t2_worktree_id "$2"' bash "$ROOT/scripts/minikube/t2-worktree-id.sh" "$repo")"
+owner_id="$(bash -c 'source "$1"; t2_profile_owner_id "$2" "$3"' bash "$ROOT/scripts/minikube/t2-worktree-id.sh" "$repo" feat/scenario)"
+v2_profile_env="$tmp/v2-profile.env"
+cat >"$v2_profile_env" <<EOF
+PROFILE_SCHEMA_VERSION=2
+WORKTREE_ID=$worktree_id
+OWNER_ID=$owner_id
+CREATED_HEAD=$feature_sha
+PROFILE=$profile
+REPO_DIR=$repo
+BRANCH=feat/scenario
+EOF
+env "${repo_env[@]}" T2_PROFILE_ENV="$v2_profile_env" \
+ bash -c 'source "$1"; t2_repo_metadata; t2_profile_scope' bash "$COMMON"
+
+bad_v2_owner_env="$tmp/bad-v2-owner.env"
+sed 's/^OWNER_ID=.*/OWNER_ID=aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa/' \
+ "$v2_profile_env" >"$bad_v2_owner_env"
+expect_code PROFILE_OWNERSHIP_MISMATCH v2-owner-mismatch v2-owner-mismatch \
+ env "${repo_env[@]}" T2_PROFILE_ENV="$bad_v2_owner_env" \
+ bash -c 'source "$1"; t2_repo_metadata; t2_profile_scope' bash "$COMMON"
+
+bad_ports_env="$tmp/bad-ports.env"
+printf 'PORT_BASE=not-a-port\n' >"$bad_ports_env"
+expect_code PROFILE_OWNERSHIP_MISMATCH corrupt-profile-ports corrupt-profile-ports \
+ env "${repo_env[@]}" T2_PROFILE_ENV="$v2_profile_env" T2_PORTS_ENV="$bad_ports_env" \
+ bash -c 'source "$1"; t2_repo_metadata; t2_profile_scope' bash "$COMMON"
+
marker_env=("${repo_env[@]}" T2_HEAD="$feature_sha" T2_WORKTREE_ID=worktree-a)
expect_code HEAD_MARKER_MISMATCH stale-marker stale-marker \
env "${marker_env[@]}" FAKE_MARKER='{"data":{"clusterFingerprint":"fp","gitHead":"old","worktreeId":"worktree-a","imageSource":"local","imageTag":"test"}}' \
@@ -394,112 +424,11 @@ grep -Fq 'T2_LOCK_ROOT' "$COMMON"
grep -Fq 't2_mutation_lock' "$COMMON"
grep -Fq 't2_lock_validate_inherited' "$COMMON"
grep -Fq 'PORT_FORWARD_CONFLICT' "$COMMON"
-grep -Fq 'while read -r uid pid ppid rest' "$COMMON" || fail 't2_process_check must split ps -ef fields'
-grep -Fq 'IFS= read -r uid pid ppid rest' "$COMMON" && fail 'IFS= read disables PID split and silences PORT_FORWARD_CONFLICT'
-grep -Fq '([^[:space:]]*\/)?kubectl([[:space:]]|$)' "$COMMON" || fail 't2_process_check awk must match a kubectl argv0/path token'
-grep -Fq '[[:space:]]port-forward([[:space:]]|$)' "$COMMON" || fail 't2_process_check awk must match a standalone port-forward token'
-grep -Fq 'kubectl[[:space:]]+port-forward' "$COMMON" && fail 't2_process_check awk still requires kubectl/port-forward adjacency'
-grep -Fq '[0-9]+:[0-9]+(:[0-9]+)?(\.[0-9]+)?' "$COMMON" && fail 't2_process_check awk still anchors on a TIME-column regex'
-
-# Real launcher argv (flags between kubectl and port-forward). TIME prefixes
-# are only ps -ef wrappers. The dead adjacency regex must miss these lines.
-pf_profile="clerum-t2-pf-fixture"
-pf_all='user 4242 1 0 10:00 ttys000 0:00 kubectl --context='"${pf_profile}"' -n control-plane port-forward --address=127.0.0.1 svc/control-ui 3000:3000'
-pf_ctl='user 4243 1 0 10:00 ttys000 0:00.03 kubectl -n control-plane port-forward svc/control-api 8090:8090'
-pf_e2e='user 4244 1 0 10:00 pts/0 00:00:00 kubectl --context '"${pf_profile}"' port-forward svc/control-ui -n control-plane 3000:3000'
-bare_pf='user 4245 1 0 10:00 ttys000 0:00 kubectl port-forward --context='"${pf_profile}"' svc/x 8080:80'
-path_pf='user 4246 1 0 10:00 pts/0 00:00:00 /usr/local/bin/kubectl --context='"${pf_profile}"' -n control-plane port-forward --address=127.0.0.1 svc/control-api 8090:8090'
-wrapper_pf='user 4247 1 0 10:00 ttys000 0:00 bash -c echo kubectl port-forward --context='"${pf_profile}"
-awk_pf="$(sed -n '/^t2_process_check()/,/^}/p' "$COMMON" | sed -n "s/.*awk '\\(.*\\)' .*/\\1/p")"
-[ -n "$awk_pf" ] || fail 'could not extract t2_process_check awk program'
-printf '%s\n' "$awk_pf" | grep -Fq 'kubectl[[:space:]]+port-forward' && fail 'extracted awk still requires kubectl/port-forward adjacency'
-dead_awk='/[[:space:]][0-9]+:[0-9]+(:[0-9]+)?(\.[0-9]+)?[[:space:]]+([^[:space:]]*\/)?kubectl[[:space:]]+port-forward([[:space:]]|$)/ {print}'
-printf '%s\n' "$pf_all" "$pf_ctl" "$pf_e2e" | awk "$dead_awk" >"$tmp/pf-dead-awk.out"
-[ ! -s "$tmp/pf-dead-awk.out" ] || fail 'real-launcher fixtures still match the dead adjacency regex — fixtures are wrong'
-printf '%s\n' "$pf_all" "$pf_ctl" "$pf_e2e" "$bare_pf" "$path_pf" "$wrapper_pf" | awk "$awk_pf" >"$tmp/pf-awk.out"
-grep -Fq -- "--context=${pf_profile} -n control-plane port-forward --address=127.0.0.1" "$tmp/pf-awk.out" || fail 'awk missed pf-all-stack real launcher'
-grep -Fq 'kubectl -n control-plane port-forward svc/control-api' "$tmp/pf-awk.out" || fail 'awk missed pf-control-stack real launcher'
-grep -Fq -- "--context ${pf_profile} port-forward svc/control-ui" "$tmp/pf-awk.out" || fail 'awk missed run-e2e real launcher'
-grep -Fq 'kubectl port-forward --context=' "$tmp/pf-awk.out" || fail 'awk missed a bare kubectl port-forward'
-grep -Fq '/usr/local/bin/kubectl --context=' "$tmp/pf-awk.out" || fail 'awk missed a path kubectl with flags before port-forward'
-read -r _ pid _ _ <<<"$pf_all"
-[ "$pid" = 4242 ] || fail "ps field split left pid='$pid' instead of 4242"
-
-pf_root="$tmp/pf-check-profiles"
-mkdir -p "$pf_root/$pf_profile/pids"
-pf_bin="$tmp/pf-ps-bin"
-mkdir -p "$pf_bin"
-cat >"$pf_bin/ps" <<'EOF'
-#!/usr/bin/env bash
-if [ "${1:-}" = "-ef" ]; then
- cat "${T2_PS_EF_FILE:?}"
- exit 0
-fi
-pid=""
-while [ "$#" -gt 0 ]; do
- case "$1" in
- -p)
- pid="$2"
- shift 2
- ;;
- -p*)
- pid="${1#-p}"
- shift
- ;;
- *)
- shift
- ;;
- esac
-done
-case "$pid" in
- 4242|4243|4244|4245|4246) printf '%s\n' kubectl ;;
- 4247) printf '%s\n' bash ;;
- *) printf '\n' ;;
-esac
-EOF
-chmod +x "$pf_bin/ps"
-
-run_process_check() {
- local ef_file="$1" out="$2"
- T2_PS_EF_FILE="$ef_file" PATH="$pf_bin:$PATH" \
- T2_PROFILE="$pf_profile" T2_CONTEXT="$pf_profile" \
- MINIKUBE_PROFILE="$pf_profile" \
- T2_PROFILE_ROOT="$pf_root" \
- T2_PROJECT_DIR="$ROOT" \
- T2_LOCK_ROOT="$tmp/locks" \
- bash -c 'source "$1"; t2_process_check' bash "$COMMON" >"$out" 2>&1
-}
-
-printf '%s\n' "$pf_all" >"$tmp/ps-ef-real.out"
-if run_process_check "$tmp/ps-ef-real.out" "$tmp/pf-real.err"; then
- fail 't2_process_check did not fail a foreign real launcher (PORT_FORWARD_CONFLICT)'
-fi
-grep -Fq 'PORT_FORWARD_CONFLICT' "$tmp/pf-real.err" || fail 'foreign real launcher did not report PORT_FORWARD_CONFLICT'
-
-printf '%s\n' "$wrapper_pf" >"$tmp/ps-ef-wrap.out"
-if ! run_process_check "$tmp/ps-ef-wrap.out" "$tmp/pf-wrap.err"; then
- fail 't2_process_check failed on a wrapper that only mentions kubectl (comm=bash)'
-fi
-
-printf '%s\n' "$pf_ctl" >"$tmp/ps-ef-contextless.out"
-if ! run_process_check "$tmp/ps-ef-contextless.out" "$tmp/pf-contextless.err"; then
- fail 't2_process_check rejected an unrelated context-less port-forward'
-fi
+grep -Fq 'port-forward-owner.sh' "$COMMON" || fail 'T2 must load the shared exact port-forward owner'
+grep -Fq 'matching_records' "$COMMON" || fail 'T2 must require exactly one structured ownership record'
+bash "$ROOT/scripts/tests/test-minikube-t2-process-owner.sh"
grep -Fq 'REUSE_DB=true' "$ROOT/scripts/minikube/t2.sh"
grep -Fq 'CONTROL_DB_RESET_PVC_UID' "$ROOT/scripts/minikube/t2.sh"
-forward_tmp="$tmp/port-forward-check"
-mkdir -p "$forward_tmp/bin" "$forward_tmp/profile-cache/profile-a/pids"
-cat >"$forward_tmp/bin/ps" <<'EOF'
-#!/usr/bin/env bash
-printf '%s\n' 'root 4242 1 0 00:00 ? 00:00:00 kubectl --context=profile-a -n control-plane port-forward svc/control-api 30100:8090'
-EOF
-chmod +x "$forward_tmp/bin/ps"
-printf '4242\nPROCESS_START=unavailable\n' >"/tmp/pf-profile-a-control-api.pid"
-forward_check="$(env PATH="$forward_tmp/bin:$PATH" T2_PROFILE=profile-a T2_CONTEXT=profile-a \
- T2_PROFILE_ROOT="$forward_tmp/profile-cache" bash -c 'source "$1"; t2_process_check; printf PASS' bash "$COMMON")"
-rm -f "/tmp/pf-profile-a-control-api.pid"
-[ "$forward_check" = PASS ] || fail "canonical /tmp port-forward ownership was rejected: $forward_check"
-
printf 'PASS: local Minikube T0/T1/T2 scenario checks\n'
diff --git a/scripts/tests/test-minikube-t2-setup-handoff.sh b/scripts/tests/test-minikube-t2-setup-handoff.sh
new file mode 100755
index 000000000..c18f94072
--- /dev/null
+++ b/scripts/tests/test-minikube-t2-setup-handoff.sh
@@ -0,0 +1,418 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+HANDOFF_HELPER="${ROOT}/scripts/minikube/t2-setup-handoff.sh"
+T2_SCRIPT="${ROOT}/scripts/minikube/t2.sh"
+PRE_GATE_SCRIPT="${ROOT}/scripts/minikube/pre-gate-sync.sh"
+FULL_SETUP_SCRIPT="${ROOT}/scripts/minikube/full-setup.sh"
+TEST_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/evenfire-t2-setup-handoff-test.XXXXXX")"
+
+cleanup() {
+ rm -rf -- "${TEST_ROOT}"
+}
+trap cleanup EXIT
+
+fail() {
+ printf 'FAIL: %s\n' "$*" >&2
+ exit 1
+}
+
+assert_file() {
+ [[ -f "$1" ]] || fail "expected file: $1"
+}
+
+assert_no_file() {
+ [[ ! -e "$1" && ! -L "$1" ]] || fail "unexpected path: $1"
+}
+
+PROJECT="${TEST_ROOT}/worktree"
+mkdir -p "${PROJECT}"
+git -C "${PROJECT}" init -q
+git -C "${PROJECT}" checkout -q -b fix/t2-handoff-test
+printf '.local-notes/\n' >"${PROJECT}/.gitignore"
+printf 'fixture\n' >"${PROJECT}/tracked.txt"
+git -C "${PROJECT}" add .gitignore tracked.txt
+git -C "${PROJECT}" -c user.name='T2 Handoff Test' -c user.email='[email protected]' \
+ -c commit.gpgsign=false commit -q -m 'test fixture'
+
+PROJECT="$(cd "${PROJECT}" && pwd -P)"
+BRANCH="$(git -C "${PROJECT}" branch --show-current)"
+HEAD="$(git -C "${PROJECT}" rev-parse --verify HEAD)"
+WORKTREE_ID="$(printf '%s' "${PROJECT}" | shasum | awk '{print $1}')"
+PROFILE=t2-handoff-profile
+CONTEXT=${PROFILE}
+TRANSITION=full-bootstrap
+LOCK_TOKEN='raw-test-lock-token-must-never-be-persisted'
+LOCK_KEY="$(printf '%s\0%s\0%s\0%s\0%s' \
+ "${PROJECT}" "${BRANCH}" "${HEAD}" "${PROFILE}" "${CONTEXT}" | shasum | awk '{print $1}')"
+MANIFEST="${PROJECT}/.local-notes/fixture/image-manifest.json"
+STATE_ROOT="${PROJECT}/.local-notes/state/t2-setup-handoffs"
+VERIFY_LOG="${TEST_ROOT}/verify.log"
+export VERIFY_LOG
+
+case_number=0
+RUN_ID=""
+HANDOFF_PATH=""
+CONSUMED_PATH=""
+CLAIM_PATH=""
+
+new_case() {
+ local label="$1"
+ case_number=$((case_number + 1))
+ RUN_ID="run-$(printf '%02d' "${case_number}")-${label}"
+ HANDOFF_PATH="${STATE_ROOT}/${WORKTREE_ID}/${RUN_ID}/setup-complete.json"
+ CONSUMED_PATH="${STATE_ROOT}/${WORKTREE_ID}/${RUN_ID}/setup-complete.consumed.json"
+ CLAIM_PATH="${STATE_ROOT}/${WORKTREE_ID}/${RUN_ID}/setup-complete.consume.claim"
+ mkdir -p "$(dirname "${MANIFEST}")"
+ printf '{"imageSource":"local","images":{"control-api":"sha256:test"}}\n' >"${MANIFEST}"
+}
+
+run_handoff() {
+ T2_PROJECT_DIR="${PROJECT}" T2_WORKTREE_ID="${WORKTREE_ID}" \
+ T2_RUN_ID="${RUN_ID}" T2_PROFILE="${PROFILE}" T2_CONTEXT="${CONTEXT}" \
+ T2_BRANCH="${BRANCH}" T2_HEAD="${HEAD}" T2_SKIP_LOCK=true \
+ T2_LOCK_KEY="${LOCK_KEY}" T2_LOCK_TOKEN="${LOCK_TOKEN}" \
+ T2_IMAGE_MANIFEST="${MANIFEST}" T2_SETUP_HANDOFF_ROOT="${STATE_ROOT}" \
+ T2_SETUP_HANDOFF_TTL_SECONDS=300 \
+ T2_SETUP_HANDOFF_TRANSITION="${TRANSITION}" \
+ bash "${HANDOFF_HELPER}" "$@"
+}
+
+create_complete() {
+ T2_SETUP_HANDOFF_SETUP_COMPLETE=true run_handoff create >/dev/null
+}
+
+verify_success() {
+ # VERIFY_LOG expands in the verifier subprocess.
+ # shellcheck disable=SC2016
+ run_handoff consume -- bash -c 'printf "verified\n" >>"${VERIFY_LOG}"'
+}
+
+verify_count() {
+ if [[ ! -f "${VERIFY_LOG}" ]]; then
+ printf '0'
+ return
+ fi
+ wc -l <"${VERIFY_LOG}" | tr -d '[:space:]'
+}
+
+mutate_string() {
+ local field="$1" value="$2"
+ python3 - "${HANDOFF_PATH}" "${field}" "${value}" <<'PY'
+import json
+import os
+import sys
+
+path, field, value = sys.argv[1:]
+with open(path, encoding="utf-8") as handle:
+ document = json.load(handle)
+target = document
+parts = field.split(".")
+for part in parts[:-1]:
+ target = target[part]
+target[parts[-1]] = value
+with open(path, "w", encoding="utf-8") as handle:
+ json.dump(document, handle, sort_keys=True, separators=(",", ":"))
+ handle.write("\n")
+os.chmod(path, 0o600)
+PY
+}
+
+mutate_incomplete() {
+ python3 - "${HANDOFF_PATH}" <<'PY'
+import json
+import os
+import sys
+
+path = sys.argv[1]
+with open(path, encoding="utf-8") as handle:
+ document = json.load(handle)
+document["setupComplete"] = False
+with open(path, "w", encoding="utf-8") as handle:
+ json.dump(document, handle, sort_keys=True, separators=(",", ":"))
+ handle.write("\n")
+os.chmod(path, 0o600)
+PY
+}
+
+mutate_stale() {
+ python3 - "${HANDOFF_PATH}" <<'PY'
+import json
+import os
+import sys
+import time
+from datetime import datetime, timezone
+
+def stamp(epoch):
+ return datetime.fromtimestamp(epoch, tz=timezone.utc).isoformat(timespec="seconds").replace("+00:00", "Z")
+
+path = sys.argv[1]
+with open(path, encoding="utf-8") as handle:
+ document = json.load(handle)
+issued = int(time.time()) - 900
+expires = issued + document["ttlSeconds"]
+document["issuedAtEpoch"] = issued
+document["expiresAtEpoch"] = expires
+document["issuedAt"] = stamp(issued)
+document["expiresAt"] = stamp(expires)
+with open(path, "w", encoding="utf-8") as handle:
+ json.dump(document, handle, sort_keys=True, separators=(",", ":"))
+ handle.write("\n")
+os.chmod(path, 0o600)
+PY
+}
+
+assert_rejected_plan() {
+ INCREMENTAL_FULL_IMAGE_BUILD=false
+ INCREMENTAL_FULL_DEPLOYMENT=false
+ INCREMENTAL_TARGETS=(unsafe-target)
+ t2_setup_handoff_apply_plan_result rejected
+ [[ "${INCREMENTAL_FULL_IMAGE_BUILD}" == true ]] \
+ || fail 'a rejected handoff did not restore the full image build'
+ [[ "${INCREMENTAL_FULL_DEPLOYMENT}" == true ]] \
+ || fail 'a rejected handoff did not restore the full deployment'
+ [[ "${#INCREMENTAL_TARGETS[@]}" -eq 0 ]] \
+ || fail 'a rejected handoff retained a partial target list'
+}
+
+expect_consume_rejected() {
+ local expected_code="$1" label="$2" before after log_file
+ before="$(verify_count)"
+ log_file="${TEST_ROOT}/${label}.log"
+ if verify_success >"${log_file}" 2>&1; then
+ fail "${label}: consume unexpectedly succeeded"
+ fi
+ grep -Fq "T2_SETUP_HANDOFF_REJECTED=${expected_code}" "${log_file}" \
+ || fail "${label}: expected rejection ${expected_code}"
+ after="$(verify_count)"
+ [[ "${after}" == "${before}" ]] || fail "${label}: image verifier ran before attestation acceptance"
+ assert_file "${HANDOFF_PATH}"
+ assert_no_file "${CONSUMED_PATH}"
+ assert_rejected_plan
+}
+
+# shellcheck source=scripts/minikube/t2-setup-handoff.sh
+source "${HANDOFF_HELPER}"
+
+# The production integration must not retain the former ambient boolean.
+if grep -Eq 'PRE_GATE_SYNC_SETUP_COMPLETE' \
+ "${T2_SCRIPT}" "${PRE_GATE_SCRIPT}" "${FULL_SETUP_SCRIPT}"; then
+ fail 'legacy PRE_GATE_SYNC_SETUP_COMPLETE optimization remains'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REQUIRED=true' "${T2_SCRIPT}" \
+ || fail 'T2 does not require a strict handoff from full setup'
+# These are literal production contracts.
+# shellcheck disable=SC2016
+grep -Fq 'T2_SETUP_HANDOFF_SETUP_COMPLETE="${all_ready}"' "${FULL_SETUP_SCRIPT}" \
+ || fail 'full setup does not bind handoff publication to readiness'
+grep -Fq 'if ! t2_deployment_check; then' "${FULL_SETUP_SCRIPT}" \
+ || fail 'T2 full setup does not enforce the complete deployment inventory'
+grep -Fq 't2-setup-handoff.sh" consume --' "${PRE_GATE_SCRIPT}" \
+ || fail 'pre-gate does not consume the attested handoff'
+# This is a literal production contract.
+# shellcheck disable=SC2016
+grep -Fq 't2_setup_handoff_apply_plan_result "${setup_handoff_result}"' "${PRE_GATE_SCRIPT}" \
+ || fail 'pre-gate does not apply the fail-closed handoff plan result'
+
+# A partial setup must not publish any handoff.
+new_case partial-setup
+partial_log="${TEST_ROOT}/partial-setup.log"
+if T2_SETUP_HANDOFF_SETUP_COMPLETE=false run_handoff create >"${partial_log}" 2>&1; then
+ fail 'partial setup unexpectedly created a handoff'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REJECTED=SETUP_INCOMPLETE' "${partial_log}" \
+ || fail 'partial setup did not fail with SETUP_INCOMPLETE'
+assert_no_file "${HANDOFF_PATH}"
+
+# A valid handoff is private, omits the raw token, verifies once, and is
+# consumed exactly once before the plan can be cleared.
+new_case valid-exact-once
+create_complete
+assert_file "${HANDOFF_PATH}"
+python3 - "${HANDOFF_PATH}" <<'PY'
+import os
+import stat
+import sys
+
+mode = stat.S_IMODE(os.stat(sys.argv[1], follow_symlinks=False).st_mode)
+if mode != 0o600:
+ raise SystemExit(f"handoff mode is {mode:o}, expected 600")
+PY
+if grep -Fq "${LOCK_TOKEN}" "${HANDOFF_PATH}"; then
+ fail 'raw lock token was persisted in the handoff'
+fi
+verify_success >/dev/null
+assert_no_file "${HANDOFF_PATH}"
+assert_file "${CONSUMED_PATH}"
+[[ -d "${CLAIM_PATH}" ]] || fail 'atomic consume claim is missing'
+[[ "$(verify_count)" == 1 ]] || fail 'valid handoff did not verify exactly once'
+if grep -Fq "${LOCK_TOKEN}" "${CONSUMED_PATH}"; then
+ fail 'raw lock token was persisted in the consumed handoff'
+fi
+INCREMENTAL_FULL_IMAGE_BUILD=true
+INCREMENTAL_FULL_DEPLOYMENT=true
+INCREMENTAL_TARGETS=(full)
+t2_setup_handoff_apply_plan_result consumed
+[[ "${INCREMENTAL_FULL_IMAGE_BUILD}" == false ]] \
+ || fail 'successful consume did not suppress the duplicate image build'
+[[ "${INCREMENTAL_FULL_DEPLOYMENT}" == false ]] \
+ || fail 'successful consume did not suppress the duplicate deployment'
+[[ "${#INCREMENTAL_TARGETS[@]}" -eq 0 ]] \
+ || fail 'successful consume retained a duplicate targeted deployment'
+replay_log="${TEST_ROOT}/replay.log"
+if verify_success >"${replay_log}" 2>&1; then
+ fail 'consumed handoff was replayable'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REJECTED=HANDOFF_REPLAYED' "${replay_log}" \
+ || fail 'replay did not fail with HANDOFF_REPLAYED'
+[[ "$(verify_count)" == 1 ]] || fail 'replay invoked the image verifier'
+assert_rejected_plan
+
+# Missing and malformed handoffs retain/force the safe plan and never verify.
+new_case missing
+missing_log="${TEST_ROOT}/missing.log"
+before="$(verify_count)"
+if verify_success >"${missing_log}" 2>&1; then
+ fail 'missing handoff unexpectedly succeeded'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REJECTED=HANDOFF_MISSING' "${missing_log}" \
+ || fail 'missing handoff did not fail closed'
+[[ "$(verify_count)" == "${before}" ]] || fail 'missing handoff invoked image verification'
+assert_rejected_plan
+
+new_case malformed
+create_complete
+printf '{malformed\n' >"${HANDOFF_PATH}"
+chmod 600 "${HANDOFF_PATH}"
+expect_consume_rejected HANDOFF_MALFORMED malformed
+
+new_case stale
+create_complete
+mutate_stale
+expect_consume_rejected HANDOFF_STALE stale
+
+new_case unsafe-mode
+create_complete
+chmod 644 "${HANDOFF_PATH}"
+expect_consume_rejected HANDOFF_UNSAFE_MODE unsafe-mode
+
+new_case incomplete-record
+create_complete
+mutate_incomplete
+expect_consume_rejected SETUP_INCOMPLETE incomplete-record
+
+# Every attested identity field is exact. None of these mismatches may reach
+# image verification or clear the build/deploy plan.
+new_case wrong-profile
+create_complete
+mutate_string profile wrong-profile
+expect_consume_rejected PROFILE_MISMATCH wrong-profile
+
+new_case wrong-context
+create_complete
+mutate_string context wrong-context
+expect_consume_rejected CONTEXT_MISMATCH wrong-context
+
+new_case wrong-worktree-id
+create_complete
+mutate_string worktree.id 0000000000000000000000000000000000000000
+expect_consume_rejected WORKTREE_ID_MISMATCH wrong-worktree-id
+
+new_case wrong-worktree-path
+create_complete
+mutate_string worktree.path "${PROJECT}/elsewhere"
+expect_consume_rejected WORKTREE_PATH_MISMATCH wrong-worktree-path
+
+new_case wrong-branch
+create_complete
+mutate_string branch fix/another-branch
+expect_consume_rejected BRANCH_MISMATCH wrong-branch
+
+new_case wrong-head
+create_complete
+mutate_string head 1111111111111111111111111111111111111111
+expect_consume_rejected HEAD_MISMATCH wrong-head
+
+new_case wrong-run
+create_complete
+mutate_string runId another-run
+expect_consume_rejected RUN_ID_MISMATCH wrong-run
+
+new_case wrong-lock-identity
+create_complete
+mutate_string lock.identity 2222222222222222222222222222222222222222
+expect_consume_rejected LOCK_IDENTITY_MISMATCH wrong-lock-identity
+
+new_case wrong-lock-token
+create_complete
+mutate_string lock.tokenSha256 3333333333333333333333333333333333333333333333333333333333333333
+expect_consume_rejected LOCK_TOKEN_MISMATCH wrong-lock-token
+
+new_case wrong-transition
+create_complete
+mutate_string transition full-reconcile
+expect_consume_rejected TRANSITION_MISMATCH wrong-transition
+
+new_case wrong-manifest-path
+create_complete
+mutate_string imageManifest.path "${PROJECT}/.local-notes/fixture/other.json"
+expect_consume_rejected IMAGE_MANIFEST_PATH_MISMATCH wrong-manifest-path
+
+new_case wrong-recorded-digest
+create_complete
+mutate_string imageManifest.sha256 4444444444444444444444444444444444444444444444444444444444444444
+expect_consume_rejected IMAGE_MANIFEST_DIGEST_MISMATCH wrong-recorded-digest
+
+new_case manifest-tamper
+create_complete
+printf 'tampered\n' >>"${MANIFEST}"
+expect_consume_rejected IMAGE_MANIFEST_DIGEST_MISMATCH manifest-tamper
+
+# Verification failure happens after attestation validation but before the
+# atomic claim. It must leave the handoff unconsumed and force full reconcile.
+new_case verify-failure
+create_complete
+verify_failure_log="${TEST_ROOT}/verify-failure.log"
+before="$(verify_count)"
+# VERIFY_LOG expands in the verifier subprocess.
+# shellcheck disable=SC2016
+if run_handoff consume -- bash -c 'printf "verified\n" >>"${VERIFY_LOG}"; exit 23' \
+ >"${verify_failure_log}" 2>&1; then
+ fail 'failed image verification unexpectedly consumed the handoff'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REJECTED=IMAGE_VERIFICATION_FAILED' "${verify_failure_log}" \
+ || fail 'verify failure did not fail closed'
+[[ "$(verify_count)" -eq $((before + 1)) ]] || fail 'verify-failure seam did not execute once'
+assert_file "${HANDOFF_PATH}"
+assert_no_file "${CONSUMED_PATH}"
+assert_no_file "${CLAIM_PATH}"
+assert_rejected_plan
+
+# Publication itself is one-shot for a run; it cannot overwrite an existing
+# or consumed attestation.
+new_case duplicate-create
+create_complete
+duplicate_log="${TEST_ROOT}/duplicate-create.log"
+if T2_SETUP_HANDOFF_SETUP_COMPLETE=true run_handoff create >"${duplicate_log}" 2>&1; then
+ fail 'duplicate handoff publication unexpectedly succeeded'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REJECTED=HANDOFF_ALREADY_EXISTS' "${duplicate_log}" \
+ || fail 'duplicate publication did not fail closed'
+
+# Even a symlink that resolves back inside the worktree is not an acceptable
+# local-state boundary: replacing it between validation and publication could
+# redirect a supposedly local-only attestation.
+new_case symlinked-local-notes
+mv "${PROJECT}/.local-notes" "${PROJECT}/.local-notes.real"
+ln -s "${PROJECT}/.local-notes.real" "${PROJECT}/.local-notes"
+symlinked_state_log="${TEST_ROOT}/symlinked-local-notes.log"
+if T2_SETUP_HANDOFF_SETUP_COMPLETE=true run_handoff create >"${symlinked_state_log}" 2>&1; then
+ fail 'symlinked .local-notes unexpectedly accepted a handoff publication'
+fi
+grep -Fq 'T2_SETUP_HANDOFF_REJECTED=STATE_ROOT_INVALID' "${symlinked_state_log}" \
+ || fail 'symlinked .local-notes did not fail with STATE_ROOT_INVALID'
+rm "${PROJECT}/.local-notes"
+mv "${PROJECT}/.local-notes.real" "${PROJECT}/.local-notes"
+
+printf 'PASS: Minikube T2 setup-complete handoff is strict, attested, and exact-once\n'
diff --git a/scripts/tests/test-minikube-targeted-health.sh b/scripts/tests/test-minikube-targeted-health.sh
new file mode 100755
index 000000000..4d8353662
--- /dev/null
+++ b/scripts/tests/test-minikube-targeted-health.sh
@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+T2="$ROOT/scripts/minikube/t2.sh"
+
+# shellcheck source=scripts/minikube/t2.sh
+source "$T2"
+
+fail() {
+ printf 'FAIL: %s\n' "$*" >&2
+ exit 1
+}
+
+T2_EVIDENCE_STATUS=''
+t2_evidence_write() {
+ T2_EVIDENCE_STATUS="$2"
+}
+
+T2_PLAN_STATE=targeted-sync
+T2_HEALTHCHECK_COMMAND=''
+T2_HEALTHCHECK_TIMEOUT_SECONDS=5
+if validate_healthcheck_contract >/dev/null 2>&1; then
+ fail 'targeted sync accepted a missing user-facing health command'
+fi
+[[ "$T2_ERROR_CODE" == PROFILE_UNHEALTHY ]] || \
+ fail "missing targeted health returned ${T2_ERROR_CODE:-<empty>}"
+
+T2_PLAN_STATE=already-synced
+T2_HEALTHCHECK_COMMAND=''
+validate_healthcheck_contract
+run_healthcheck_if_requested
+[[ "$T2_HEALTHCHECK_REQUIRED" == false && "$T2_HEALTH_STATUS" == NOT_RUN && \
+ "$T2_EVIDENCE_STATUS" == NOT_RUN ]] || \
+ fail 'an optional absent health check was not recorded as NOT_RUN'
+
+T2_PLAN_STATE=targeted-sync
+T2_HEALTHCHECK_COMMAND='printf "health-ok\\n" >/dev/null'
+T2_HEALTHCHECK_TIMEOUT_SECONDS=5
+validate_healthcheck_contract
+run_healthcheck_if_requested
+[[ "$T2_HEALTHCHECK_REQUIRED" == true && "$T2_HEALTH_STATUS" == PASS ]] || \
+ fail 'a bounded targeted health check did not pass'
+
+T2_PLAN_STATE=targeted-sync
+T2_HEALTHCHECK_COMMAND='sleep 10'
+T2_HEALTHCHECK_TIMEOUT_SECONDS=1
+validate_healthcheck_contract
+started="$SECONDS"
+if run_healthcheck_if_requested >/dev/null 2>&1; then
+ fail 'timed-out targeted health check unexpectedly passed'
+fi
+elapsed=$((SECONDS - started))
+[[ "$T2_ERROR_CODE" == PROFILE_UNHEALTHY && "$T2_HEALTH_STATUS" == FAIL ]] || \
+ fail 'timed-out targeted health check did not fail closed'
+(( elapsed < 8 )) || fail "targeted health deadline took ${elapsed}s"
+
+T2_HEALTHCHECK_TIMEOUT_SECONDS=0
+if validate_healthcheck_contract >/dev/null 2>&1; then
+ fail 'invalid health timeout was accepted'
+fi
+[[ "$T2_ERROR_CODE" == DEVELOPMENT_SCOPE_REQUIRED ]] || \
+ fail 'invalid health timeout returned the wrong stable code'
+
+printf 'PASS: targeted T2 requires a bounded user-facing health check\n'
diff --git a/scripts/tests/test-np08-e2e-guards.sh b/scripts/tests/test-np08-e2e-guards.sh
index 865420795..6809047bb 100755
--- a/scripts/tests/test-np08-e2e-guards.sh
+++ b/scripts/tests/test-np08-e2e-guards.sh
@@ -3,6 +3,8 @@ set -euo pipefail
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
E2E_SCRIPT="${ROOT}/scripts/e2e/e2e-np08-hcc-authorization.sh"
+RUNTIME_ACCESS_MODULE="${ROOT}/scripts/e2e/_lib/np08-runtime-access.mjs"
+RUNTIME_ACCESS_TEST="${ROOT}/scripts/tests/test-np08-runtime-access.mjs"
T2_SCRIPT="${ROOT}/scripts/minikube/t2.sh"
MAKEFILE="${ROOT}/Makefile"
@@ -159,6 +161,46 @@ if ! grep -Fq 'np08_cleanup_check_residual' "${E2E_SCRIPT}" ||
fi
pass 'the deployed E2E is wired to cleanup and provenance guards'
+forbidden_refresh_env='MCP_HOST_RUNTIME_'"REFRESH_TOKEN"
+forbidden_refresh_path='/api/v1/workflow-auth/'"refresh"
+forbidden_reissue_path='/api/v1/workflow-auth/'"reissue"
+for runtime_file in "${E2E_SCRIPT}" "${RUNTIME_ACCESS_MODULE}"; do
+ [[ -f "${runtime_file}" ]] || fail 'the NP-08 access-only runtime module is missing'
+ if grep -Fq "${forbidden_refresh_env}" "${runtime_file}" ||
+ grep -Fq "${forbidden_refresh_path}" "${runtime_file}" ||
+ grep -Fq "${forbidden_reissue_path}" "${runtime_file}"; then
+ fail 'the deployed NP-08 path can still consume or mutate the runtime credential lineage'
+ fi
+done
+pass 'the deployed NP-08 path is access-only and has no refresh or reissue route'
+
+if ! grep -Fq 'NP08_RUNTIME_ACTION=health' "${E2E_SCRIPT}" ||
+ ! grep -Fq 'NP08_RUNTIME_ACTION=journey' "${E2E_SCRIPT}" ||
+ ! grep -Fq "node --input-type=module - < \"\${NP08_RUNTIME_MODULE}\"" "${E2E_SCRIPT}" ||
+ ! grep -Fq 'approval-auth.json' "${RUNTIME_ACCESS_MODULE}" ||
+ ! grep -Fq 'hostRefs' "${RUNTIME_ACCESS_MODULE}" ||
+ ! grep -Fq 'recipeNamespace' "${RUNTIME_ACCESS_MODULE}" ||
+ ! grep -Fq 'recipeName' "${RUNTIME_ACCESS_MODULE}" ||
+ ! grep -Fq 'CLERUM_CONTEXT_MAPPER_URL' "${RUNTIME_ACCESS_MODULE}" ||
+ ! grep -Fq 'CLERUM_SERVER_PORT' "${RUNTIME_ACCESS_MODULE}" ||
+ ! grep -Fq 'runtime_access_rotation_timeout' "${RUNTIME_ACCESS_MODULE}"; then
+ fail 'the deployed E2E is not wired to the tested access observation and retry contract'
+fi
+pass 'the deployed journey streams the tested binding-aware access observer into mcp-host'
+
+health_line="$(grep -n -m1 'NP08_RUNTIME_ACTION=health' "${E2E_SCRIPT}" | cut -d: -f1)"
+fixture_line="$(grep -n -m1 'apply -f -' "${E2E_SCRIPT}" | cut -d: -f1)"
+if [[ -z "${health_line}" || -z "${fixture_line}" || "${health_line}" -ge "${fixture_line}" ]]; then
+ fail 'mcp-host runtime health is not required before fixture mutation'
+fi
+pass 'mcp-host runtime health is proven before fixture mutation'
+
+for expected_status in 200 404 400 401 410; do
+ grep -Fq "status !== ${expected_status}" "${RUNTIME_ACCESS_MODULE}" ||
+ fail "the access-only journey lost its HTTP ${expected_status} assertion"
+done
+pass 'the access-only journey preserves the deployed authorization status assertions'
+
if grep -Fq 'clerum-codex-np-08-cross-context-mcp-token-plan-' "${E2E_SCRIPT}"; then
fail 'the deployed E2E is still hard-wired to one historical branch profile'
fi
@@ -183,3 +225,7 @@ if grep -R -Fq 'e2e-np08-hcc-authorization.sh' "${ROOT}/.github/workflows"; then
fail 'CI directly invokes the cluster-mutating NP-08 deployed journey'
fi
pass 'CI validates NP-08 wiring statically without cluster writes'
+
+node --check "${RUNTIME_ACCESS_MODULE}"
+node --test "${RUNTIME_ACCESS_TEST}"
+pass 'NP-08 access selection and retry unit tests pass'
diff --git a/scripts/tests/test-np08-runtime-access.mjs b/scripts/tests/test-np08-runtime-access.mjs
new file mode 100644
index 000000000..968e41fc7
--- /dev/null
+++ b/scripts/tests/test-np08-runtime-access.mjs
@@ -0,0 +1,350 @@
+import assert from 'node:assert/strict'
+import { EventEmitter } from 'node:events'
+import { mkdtemp, readFile, rename, rm, writeFile } from 'node:fs/promises'
+import { tmpdir } from 'node:os'
+import { dirname, join } from 'node:path'
+import { fileURLToPath } from 'node:url'
+import { describe, it } from 'node:test'
+import {
+ NP08_RUNTIME_ACCESS_ROTATION_WAIT_MS,
+ observeFreshRuntimeAccess,
+ requestLocalRuntimeHealth,
+ requestWithRuntimeAccess,
+ requireLocalRuntimeHealth,
+} from '../e2e/_lib/np08-runtime-access.mjs'
+
+const NOW_SECONDS = 2_000_000_000
+const NOW_MS = NOW_SECONDS * 1000
+const BINDING = {
+ hostRefs: ['chatllm'],
+ recipeNamespace: 'mcp-host',
+ recipeName: 'standalone',
+}
+
+function encode(value) {
+ return Buffer.from(JSON.stringify(value)).toString('base64url')
+}
+
+function makeJwt({ exp, iat = NOW_SECONDS - 10, binding = BINDING, marker }) {
+ return `${encode({ alg: 'none', typ: 'JWT' })}.${encode({
+ ...binding,
+ exp,
+ iat,
+ marker,
+ })}.unsigned`
+}
+
+function persistedState(value) {
+ return JSON.stringify({ [['access', 'Token'].join('')]: value })
+}
+
+async function tempState() {
+ const directory = await mkdtemp(join(tmpdir(), 'np08-runtime-access-'))
+ return {
+ directory,
+ stateFilePath: join(directory, 'approval-auth.json'),
+ cleanup: () => rm(directory, { recursive: true, force: true }),
+ }
+}
+
+describe('NP-08 access-only state selection', () => {
+ it('uses a fresh mounted identity when no persisted state exists', async () => {
+ const state = await tempState()
+ try {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ nowMs: NOW_MS,
+ })
+
+ assert.equal(observed?.source, 'mounted')
+ assert.equal(observed?.value, mounted)
+ } finally {
+ await state.cleanup()
+ }
+ })
+
+ it('prefers fresher persisted state with the exact mounted binding', async () => {
+ const state = await tempState()
+ try {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ const persisted = makeJwt({
+ exp: NOW_SECONDS + 300,
+ iat: NOW_SECONDS + 1,
+ marker: 'persisted',
+ })
+ await writeFile(state.stateFilePath, persistedState(persisted), 'utf8')
+
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ nowMs: NOW_MS,
+ })
+
+ assert.equal(observed?.source, 'persisted')
+ assert.equal(observed?.value, persisted)
+ } finally {
+ await state.cleanup()
+ }
+ })
+
+ it('rejects persisted state when any canonical binding field differs', async () => {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ const variants = [
+ { ...BINDING, hostRefs: ['other-host'] },
+ { ...BINDING, hostRefs: ['chatllm', 'other-host'] },
+ { ...BINDING, recipeNamespace: 'other-namespace' },
+ { ...BINDING, recipeName: 'other-recipe' },
+ ]
+
+ for (const [index, binding] of variants.entries()) {
+ const state = await tempState()
+ try {
+ const persisted = makeJwt({
+ exp: NOW_SECONDS + 300,
+ iat: NOW_SECONDS + 1,
+ binding,
+ marker: `wrong-binding-${index}`,
+ })
+ await writeFile(state.stateFilePath, persistedState(persisted), 'utf8')
+
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ nowMs: NOW_MS,
+ })
+ assert.equal(observed?.source, 'mounted')
+ assert.equal(observed?.value, mounted)
+ } finally {
+ await state.cleanup()
+ }
+ }
+ })
+
+ it('ignores malformed persisted state without displacing a fresh mounted identity', async () => {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ for (const raw of ['{', persistedState('not-a-jwt')]) {
+ const state = await tempState()
+ try {
+ await writeFile(state.stateFilePath, raw, 'utf8')
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ nowMs: NOW_MS,
+ })
+ assert.equal(observed?.value, mounted)
+ } finally {
+ await state.cleanup()
+ }
+ }
+ })
+
+ it('rejects expired persisted state', async () => {
+ const state = await tempState()
+ try {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ const expired = makeJwt({
+ exp: NOW_SECONDS - 1,
+ iat: NOW_SECONDS - 20,
+ marker: 'expired',
+ })
+ await writeFile(state.stateFilePath, persistedState(expired), 'utf8')
+
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ nowMs: NOW_MS,
+ })
+ assert.equal(observed?.value, mounted)
+ } finally {
+ await state.cleanup()
+ }
+ })
+
+ it('rejects persisted state that is older by issue time or expiry', async () => {
+ const mounted = makeJwt({
+ exp: NOW_SECONDS + 180,
+ iat: NOW_SECONDS,
+ marker: 'mounted',
+ })
+ const olderCandidates = [
+ makeJwt({
+ exp: NOW_SECONDS + 300,
+ iat: NOW_SECONDS - 1,
+ marker: 'older-issue-time',
+ }),
+ makeJwt({
+ exp: NOW_SECONDS + 150,
+ iat: NOW_SECONDS + 1,
+ marker: 'older-expiry',
+ }),
+ makeJwt({
+ exp: NOW_SECONDS + 300,
+ iat: null,
+ marker: 'missing-issue-time',
+ }),
+ ]
+
+ for (const persisted of olderCandidates) {
+ const state = await tempState()
+ try {
+ await writeFile(state.stateFilePath, persistedState(persisted), 'utf8')
+ const observed = await observeFreshRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ nowMs: NOW_MS,
+ })
+ assert.equal(observed?.value, mounted)
+ } finally {
+ await state.cleanup()
+ }
+ }
+ })
+})
+
+describe('NP-08 authorization retry', () => {
+ it('observes an atomic persisted-state rotation after an HCC 401', async () => {
+ const state = await tempState()
+ try {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ const rotated = makeJwt({
+ exp: NOW_SECONDS + 300,
+ iat: NOW_SECONDS + 1,
+ marker: 'rotated',
+ })
+ const attempts = []
+ let clock = NOW_MS
+ let stateRotated = false
+
+ const outcome = await requestWithRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ timeoutMs: 2_000,
+ pollIntervalMs: 500,
+ now: () => clock,
+ sleep: async ms => {
+ clock += ms
+ if (!stateRotated) {
+ stateRotated = true
+ const nextPath = `${state.stateFilePath}.next`
+ await writeFile(nextPath, persistedState(rotated), 'utf8')
+ await rename(nextPath, state.stateFilePath)
+ }
+ },
+ request: async value => {
+ attempts.push(value)
+ return { status: value === rotated ? 200 : 401 }
+ },
+ })
+
+ assert.equal(outcome.result.status, 200)
+ assert.equal(outcome.rotatedAfterUnauthorized, true)
+ assert.equal(outcome.candidate.value, rotated)
+ assert.deepEqual(attempts, [mounted, rotated])
+ } finally {
+ await state.cleanup()
+ }
+ })
+
+ it('fails closed when no fresher state appears before the retry deadline', async () => {
+ const state = await tempState()
+ try {
+ const mounted = makeJwt({ exp: NOW_SECONDS + 120, marker: 'mounted' })
+ let clock = NOW_MS
+ let attempts = 0
+
+ await assert.rejects(
+ requestWithRuntimeAccess({
+ mountedAccessValue: mounted,
+ stateFilePath: state.stateFilePath,
+ timeoutMs: 1_500,
+ pollIntervalMs: 500,
+ now: () => clock,
+ sleep: async ms => {
+ clock += ms
+ },
+ request: async () => {
+ attempts += 1
+ return { status: 401 }
+ },
+ }),
+ /runtime_access_rotation_timeout/
+ )
+ assert.equal(attempts, 1)
+ assert.ok(NP08_RUNTIME_ACCESS_ROTATION_WAIT_MS > 30_000)
+ } finally {
+ await state.cleanup()
+ }
+ })
+})
+
+describe('NP-08 deployed-path guards', () => {
+ it('requires the local mcp-host runtime health contract', async () => {
+ const calls = []
+ await requireLocalRuntimeHealth({
+ env: { CLERUM_SERVER_PORT: '8080' },
+ requestImpl: async options => {
+ calls.push(options)
+ return { status: 200, body: { status: 'ok' } }
+ },
+ })
+
+ assert.equal(calls.length, 1)
+ assert.deepEqual(calls[0], { port: 8080, timeoutMs: 5_000 })
+ })
+
+ it('enforces an absolute local-health deadline against a drip-fed response', async () => {
+ let interval
+ const requestFactory = (_options, onResponse) => {
+ const request = new EventEmitter()
+ const response = new EventEmitter()
+ response.statusCode = 200
+ request.end = () => {
+ onResponse(response)
+ interval = setInterval(() => response.emit('data', Buffer.from(' ')), 10)
+ }
+ request.destroy = error => {
+ clearInterval(interval)
+ request.emit('error', error)
+ response.emit('aborted')
+ }
+ return request
+ }
+ let safetyTimer
+ try {
+ const request = requestLocalRuntimeHealth({
+ port: 8080,
+ timeoutMs: 100,
+ requestFactory,
+ })
+ const safety = new Promise((_, reject) => {
+ safetyTimer = setTimeout(
+ () => reject(new Error('absolute_runtime_health_deadline_not_enforced')),
+ 1_000
+ )
+ })
+ await assert.rejects(Promise.race([request, safety]), /mcp_host_runtime_health_timeout/)
+ } finally {
+ clearTimeout(safetyTimer)
+ clearInterval(interval)
+ }
+ })
+
+ it('contains no refresh credential dependency or write-line request path', async () => {
+ const modulePath = join(
+ dirname(fileURLToPath(import.meta.url)),
+ '../e2e/_lib/np08-runtime-access.mjs'
+ )
+ const source = await readFile(modulePath, 'utf8')
+ const forbidden = [
+ ['MCP_HOST_RUNTIME', 'REFRESH_TOKEN'].join('_'),
+ ['/api/v1/workflow-auth', 'refresh'].join('/'),
+ ['/api/v1/workflow-auth', 'reissue'].join('/'),
+ ]
+
+ for (const value of forbidden) {
+ assert.equal(source.includes(value), false)
+ }
+ })
+})
diff --git a/scripts/tests/test-real-postgres-local-preflight.sh b/scripts/tests/test-real-postgres-local-preflight.sh
new file mode 100755
index 000000000..1ffa00d88
--- /dev/null
+++ b/scripts/tests/test-real-postgres-local-preflight.sh
@@ -0,0 +1,91 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ROOT="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd -P)"
+PREFLIGHT="$ROOT/scripts/e2e/real-postgres-local-preflight.sh"
+tmp="$(mktemp -d)"
+trap 'rm -rf "$tmp"' EXIT
+
+project="$tmp/project"
+fake_bin="$tmp/bin"
+real_node="$(command -v node)"
+mkdir -p "$fake_bin"
+for package in control-api gfs-controller; do
+ mkdir -p "$project/$package/node_modules/.bin"
+ printf '{"name":"%s"}\n' "$package" >"$project/$package/package.json"
+ printf '#!/usr/bin/env bash\nexit 0\n' >"$project/$package/node_modules/.bin/vitest"
+ chmod +x "$project/$package/node_modules/.bin/vitest"
+done
+
+printf '%s\n' '#!/usr/bin/env bash' \
+ 'if [[ "${1:-}" == -p ]]; then printf "%s\n" "${FAKE_NODE_MAJOR:-24}"; exit 0; fi' \
+ 'if [[ "${1:-}" == -e ]]; then exit 0; fi' \
+ 'exec "$REAL_NODE" "$@"' >"$fake_bin/node"
+printf '%s\n' '#!/usr/bin/env bash' 'exit 0' >"$fake_bin/npm"
+printf '%s\n' '#!/usr/bin/env bash' \
+ 'if [[ "${1:-}" == context && "${2:-}" == inspect ]]; then' \
+ ' if [[ "$*" == *"TLSMaterial"* ]]; then' \
+ ' printf "unix:///private/tmp/evenfire-test-docker.sock\\tfalse\\t{}\\n"' \
+ ' else' \
+ ' printf "unix:///private/tmp/evenfire-test-docker.sock\\n"' \
+ ' fi' \
+ ' exit 0' \
+ 'fi' \
+ 'if [[ "${1:-}" == info ]]; then exit 0; fi' \
+ 'exit 1' >"$fake_bin/docker"
+chmod +x "$fake_bin/node" "$fake_bin/npm" "$fake_bin/docker"
+
+# shellcheck source=scripts/e2e/real-postgres-local-preflight.sh
+source "$PREFLIGHT"
+
+PATH="$fake_bin:$PATH"
+REAL_NODE="$real_node"
+export PATH REAL_NODE
+unset VITEST_MAX_WORKERS
+real_pg_local_preflight "$project" true >/dev/null
+[[ "$VITEST_MAX_WORKERS" == 1 ]] || {
+ echo 'FAIL: preflight did not pin T1 to one worker' >&2
+ exit 1
+}
+
+VITEST_MAX_WORKERS=4
+if real_pg_local_preflight "$project" true >"$tmp/workers.out" 2>&1; then
+ echo 'FAIL: preflight accepted an unsafe T1 worker count' >&2
+ exit 1
+fi
+[[ "$REAL_PG_PREFLIGHT_ERROR_CODE" == UNSUPPORTED_T1_CONCURRENCY ]]
+
+VITEST_MAX_WORKERS=1 FAKE_NODE_MAJOR=23
+export VITEST_MAX_WORKERS FAKE_NODE_MAJOR
+if real_pg_local_preflight "$project" true >"$tmp/node.out" 2>&1; then
+ echo 'FAIL: preflight accepted Node.js below the package contract' >&2
+ exit 1
+fi
+[[ "$REAL_PG_PREFLIGHT_ERROR_CODE" == LOCAL_DEPENDENCY_MISSING ]]
+unset FAKE_NODE_MAJOR
+
+chmod -x "$project/gfs-controller/node_modules/.bin/vitest"
+if real_pg_local_preflight "$project" true >"$tmp/deps.out" 2>&1; then
+ echo 'FAIL: preflight accepted missing package-local dependencies' >&2
+ exit 1
+fi
+[[ "$REAL_PG_PREFLIGHT_ERROR_CODE" == LOCAL_DEPENDENCY_MISSING ]]
+chmod +x "$project/gfs-controller/node_modules/.bin/vitest"
+
+printf '%s\n' '#!/usr/bin/env bash' 'exec sleep 5' >"$fake_bin/docker"
+chmod +x "$fake_bin/docker"
+REAL_PG_DOCKER_INFO_TIMEOUT_SECONDS=1
+export REAL_PG_DOCKER_INFO_TIMEOUT_SECONDS
+started="$SECONDS"
+if real_pg_local_preflight "$project" true >"$tmp/docker.out" 2>&1; then
+ echo 'FAIL: preflight accepted a Docker daemon timeout' >&2
+ exit 1
+fi
+elapsed=$((SECONDS - started))
+[[ "$REAL_PG_PREFLIGHT_ERROR_CODE" == LOCAL_DEPENDENCY_MISSING ]]
+(( elapsed < 4 )) || {
+ echo 'FAIL: Docker preflight timeout was not bounded' >&2
+ exit 1
+}
+
+printf 'PASS: Real PostgreSQL local preflight is early, serial, and bounded\n'
diff --git a/scripts/tests/test-repo-intake-packet.sh b/scripts/tests/test-repo-intake-packet.sh
index 5020863eb..aa0976b62 100755
--- a/scripts/tests/test-repo-intake-packet.sh
+++ b/scripts/tests/test-repo-intake-packet.sh
@@ -4,8 +4,11 @@ set -u
FAIL=0
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)"
SCRIPT="${ROOT}/scripts/dev/repo-intake-packet.sh"
+OWNER_SCRIPT="${ROOT}/scripts/minikube/profile-owner.sh"
TMP_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/clerum-intake-test.XXXXXX")"
TMP_ROOT="$(cd "${TMP_ROOT}" && pwd -P)"
+PROFILE_ROOT="${TMP_ROOT}/profiles"
+mkdir -p "${PROFILE_ROOT}"
cleanup() {
chmod -R u+w "${TMP_ROOT}" 2>/dev/null || true
@@ -54,10 +57,32 @@ run_packet() {
local cwd="$1"
(
cd "${cwd}" || exit 1
- "${SCRIPT}"
+ REPO_INTAKE_PROFILE_ROOT="${2:-${PROFILE_ROOT}}" "${SCRIPT}"
)
}
+run_packet_with_profile() {
+ local cwd="$1" profile_root="$2" profile="$3"
+ (
+ cd "${cwd}" || exit 1
+ REPO_INTAKE_PROFILE_ROOT="${profile_root}" MINIKUBE_PROFILE="${profile}" "${SCRIPT}"
+ )
+}
+
+payload_value() {
+ local payload="$1" key="$2"
+ awk -F= -v wanted="${key}" '$1 == wanted { sub(/^[^=]*=/, ""); print; exit }' <<<"${payload}"
+}
+
+write_ports() {
+ local destination="$1" base="$2"
+ cat >"${destination}" <<EOF_PORTS
+PORT_BASE=${base}
+CONTROL_API_PORT=$((base + 90))
+CONTROL_API_URL=http://127.0.0.1:$((base + 90))
+EOF_PORTS
+}
+
if bash -n "${SCRIPT}"; then
pass "repo intake packet has valid bash syntax"
else
@@ -98,12 +123,121 @@ assert_contains "${PRIMARY_OUTPUT}" "branch:" "primary output includes branch la
assert_contains "${PRIMARY_OUTPUT}" "dev" "primary output reports branch"
assert_contains "${PRIMARY_OUTPUT}" "profile_helper_exists:" "primary output includes profile helper label"
assert_contains "${PRIMARY_OUTPUT}" "yes" "primary output finds profile helper"
+assert_contains "${PRIMARY_OUTPUT}" "profile_owner_resolver:" "primary output includes profile owner resolver label"
+assert_contains "${PRIMARY_OUTPUT}" "profile_cache_state:" "primary output includes profile cache state"
+assert_contains "${PRIMARY_OUTPUT}" "absent" "primary output reports an uninitialized isolated profile root"
assert_not_contains "${PRIMARY_OUTPUT}" "SENSITIVE_MARKER_DO_NOT_PRINT" "primary output does not print file contents"
assert_not_contains "${PRIMARY_OUTPUT}" "local-sensitive.txt" "primary output does not print local sensitive filenames"
assert_not_contains "${PRIMARY_OUTPUT}" "unbound variable" "primary output has no shell unbound-variable warnings"
+CREATION_SHORT="$(git -C "${MAIN_REPO}" rev-parse --short=8 HEAD)"
+LEGACY_PROFILE="clerum-dev-${CREATION_SHORT}"
+LEGACY_DIR="${PROFILE_ROOT}/${LEGACY_PROFILE}"
+mkdir -p "${LEGACY_DIR}"
+cat >"${LEGACY_DIR}/profile.env" <<EOF_PROFILE
+PROFILE=${LEGACY_PROFILE}
+BRANCH=dev
+SHA_SHORT=${CREATION_SHORT}
+DIRTY=false
+REPO_DIR=${MAIN_REPO}
+EOF_PROFILE
+write_ports "${LEGACY_DIR}/ports.env" 28000
+
+printf 'advanced head\n' >"${MAIN_REPO}/tracked-after-profile.txt"
+git -C "${MAIN_REPO}" add tracked-after-profile.txt
+git -C "${MAIN_REPO}" commit -q -m "advance after profile creation"
+HISTORICAL_OUTPUT="$(run_packet "${MAIN_REPO}" 2>&1)"
+HISTORICAL_STATUS=$?
+if [[ "${HISTORICAL_STATUS}" -eq 0 ]]; then
+ pass "historical v1 profile remains valid after HEAD advances"
+else
+ fail "historical v1 profile remains valid after HEAD advances"
+ echo "${HISTORICAL_OUTPUT}"
+fi
+assert_contains "${HISTORICAL_OUTPUT}" "resolved_profile:" "historical output includes resolved profile"
+assert_contains "${HISTORICAL_OUTPUT}" "${LEGACY_PROFILE}" "historical output preserves the legacy profile name"
+assert_contains "${HISTORICAL_OUTPUT}" "profile_cache_schema:" "historical output includes source schema"
+assert_contains "${HISTORICAL_OUTPUT}" "profile_cache_creation_head:" "historical output treats SHA_SHORT as creation metadata"
+assert_contains "${HISTORICAL_OUTPUT}" "${CREATION_SHORT}" "historical output reports the legacy creation SHA"
+assert_contains "${HISTORICAL_OUTPUT}" "profile_cache_sha_match:" "historical output retains the compatibility label"
+assert_contains "${HISTORICAL_OUTPUT}" "not_applicable" "historical output does not compare creation SHA to HEAD"
+assert_not_contains "${HISTORICAL_OUTPUT}" "profile_cache_sha_mismatch" "historical profile does not become a blocker"
+
+V2_ROOT="${TMP_ROOT}/v2-profiles"
+CURRENT_HEAD="$(git -C "${MAIN_REPO}" rev-parse HEAD)"
+V2_IDENTITY="$("${OWNER_SCRIPT}" identity --repo-dir "${MAIN_REPO}" --branch dev --created-head "${CURRENT_HEAD}")"
+V2_PROFILE="$(payload_value "${V2_IDENTITY}" PROFILE)"
+V2_DIR="${V2_ROOT}/${V2_PROFILE}"
+mkdir -p "${V2_DIR}"
+cat >"${V2_DIR}/profile.env" <<EOF_PROFILE
+PROFILE_SCHEMA_VERSION=2
+WORKTREE_ID=$(payload_value "${V2_IDENTITY}" WORKTREE_ID)
+OWNER_ID=$(payload_value "${V2_IDENTITY}" OWNER_ID)
+CREATED_HEAD=${CURRENT_HEAD}
+PROFILE=${V2_PROFILE}
+REPO_DIR=${MAIN_REPO}
+BRANCH=dev
+EOF_PROFILE
+write_ports "${V2_DIR}/ports.env" 29000
+V2_OUTPUT="$(run_packet "${MAIN_REPO}" "${V2_ROOT}" 2>&1)"
+V2_STATUS=$?
+if [[ "${V2_STATUS}" -eq 0 ]]; then
+ pass "repo intake consumes schema-v2 profile metadata"
+else
+ fail "repo intake consumes schema-v2 profile metadata"
+ echo "${V2_OUTPUT}"
+fi
+assert_contains "${V2_OUTPUT}" "${V2_PROFILE}" "schema-v2 intake resolves the stable owner profile"
+assert_contains "${V2_OUTPUT}" "profile_cache_schema: 2" "schema-v2 intake reports schema 2"
+
+AMBIGUOUS_ROOT="${TMP_ROOT}/ambiguous-profiles"
+for suffix in aaaaaaaa bbbbbbbb; do
+ candidate="clerum-dev-${suffix}"
+ mkdir -p "${AMBIGUOUS_ROOT}/${candidate}"
+ cat >"${AMBIGUOUS_ROOT}/${candidate}/profile.env" <<EOF_PROFILE
+PROFILE=${candidate}
+BRANCH=dev
+SHA_SHORT=${suffix}
+DIRTY=false
+REPO_DIR=${MAIN_REPO}
+EOF_PROFILE
+ write_ports "${AMBIGUOUS_ROOT}/${candidate}/ports.env" $((30000 + ${#suffix}))
+done
+AMBIGUOUS_OUTPUT="$(run_packet "${MAIN_REPO}" "${AMBIGUOUS_ROOT}" 2>&1)"
+AMBIGUOUS_STATUS=$?
+if [[ "${AMBIGUOUS_STATUS}" -eq 2 ]]; then
+ pass "ambiguous profile selection blocks intake"
+else
+ fail "ambiguous profile selection blocks intake"
+fi
+assert_contains "${AMBIGUOUS_OUTPUT}" "profile_selection_ambiguous" "ambiguous intake reports the stable blocker"
+
+EXPLICIT_OUTPUT="$(run_packet_with_profile "${MAIN_REPO}" "${AMBIGUOUS_ROOT}" clerum-dev-aaaaaaaa 2>&1)"
+EXPLICIT_STATUS=$?
+if [[ "${EXPLICIT_STATUS}" -eq 0 ]]; then
+ pass "explicit profile disambiguates intake"
+else
+ fail "explicit profile disambiguates intake"
+ echo "${EXPLICIT_OUTPUT}"
+fi
+assert_contains "${EXPLICIT_OUTPUT}" "clerum-dev-aaaaaaaa" "explicit intake preserves selected profile"
+
+MISSING_PORTS_ROOT="${TMP_ROOT}/missing-ports-profile"
+mkdir -p "${MISSING_PORTS_ROOT}/${LEGACY_PROFILE}"
+cp "${LEGACY_DIR}/profile.env" "${MISSING_PORTS_ROOT}/${LEGACY_PROFILE}/profile.env"
+MISSING_PORTS_OUTPUT="$(run_packet_with_profile "${MAIN_REPO}" "${MISSING_PORTS_ROOT}" "${LEGACY_PROFILE}" 2>&1)"
+MISSING_PORTS_STATUS=$?
+if [[ "${MISSING_PORTS_STATUS}" -eq 2 ]]; then
+ pass "missing persisted ports block intake"
+else
+ fail "missing persisted ports block intake"
+fi
+assert_contains "${MISSING_PORTS_OUTPUT}" "profile_ports_missing" "missing ports report the stable blocker"
+
git -C "${MAIN_REPO}" worktree add -q --detach "${DETACHED_WT}" HEAD
-DETACHED_OUTPUT="$(run_packet "${DETACHED_WT}" 2>&1)"
+DETACHED_ROOT="${TMP_ROOT}/detached-profiles"
+mkdir -p "${DETACHED_ROOT}"
+DETACHED_OUTPUT="$(run_packet "${DETACHED_WT}" "${DETACHED_ROOT}" 2>&1)"
DETACHED_STATUS=$?
if [[ "${DETACHED_STATUS}" -eq 0 || "${DETACHED_STATUS}" -eq 2 ]]; then