Evenfire is an intelligence layer platform for running LLM agents that take real actions, on your infrastructure
TypeScript
fix: make a single minikube-t2 run self-heal GFS and unready deployments
Alfredo Lopez9e2c0e53084d
diff --git a/.cursor/rules/minikube-t0-t1-t2.mdc b/.cursor/rules/minikube-t0-t1-t2.mdcindex bfd48354b..b8020078e 100644--- a/.cursor/rules/minikube-t0-t1-t2.mdc+++ b/.cursor/rules/minikube-t0-t1-t2.mdc@@ -42,9 +42,16 @@ repository — never mix its PRs or clusters into this lane.current-context.- Bootstrap/reconcile must build the worktree: `IMAGE_SOURCE=local`.- T1 is fail-loud: unavailable DSN or zero executed tests is FAIL, never green.- The JSON reporter is the suite verdict; a leftover Vitest exit after a- complete green reporter is not a failed suite. Role-reset suites use the- throwaway `postgres:16-alpine`, never shared `control-postgres`.+ The JSON reporter is the suite verdict; a leftover Vitest exit after a+ complete green reporter is not a failed suite. Role-reset suites use the+ throwaway `postgres:16-alpine`, never shared `control-postgres`.+- Self-healing single run: the orchestrator planner (`T2_PLAN_MODE=true`)+ selects `full-reconcile` (naming the deployment) for a bootstrapped profile+ with an unready required deployment — never `PROFILE_UNHEALTHY` before a+ transition. Standalone preflight and the final exact-head check stay+ fail-loud. T1 restores branch-profile GFS credentials on exit, and+ `pre-gate-sync` provisions GFS serving with `GFS_RESTORE_ACTIVE_NOLOGIN=true`+ in every plan, so no manual repair script sits between plan and verdict.- Evidence lives under ignored `.local-notes/infra/runs/`; never commit it.Full workflow: skill `.cursor/skills/minikube-t0-t1-t2/SKILL.md`.diff --git a/.cursor/rules/minikube-t2-harness.mdc b/.cursor/rules/minikube-t2-harness.mdcindex e0988178c..1b2bba18a 100644--- a/.cursor/rules/minikube-t2-harness.mdc+++ b/.cursor/rules/minikube-t2-harness.mdc@@ -28,6 +28,15 @@ Invariants that must never regress:and plan state `already-synced`; anything else is `HEAD_MARKER_MISMATCH`.- Planner mode `T2_PLAN_MODE=true` exists only inside the orchestrator so`full-bootstrap` is reachable; standalone preflight stays fail-loud.+- Planner mode never dies `PROFILE_UNHEALTHY` on a bootstrapped profile with+ an unready required deployment: it records the deployment+ (`T2_UNREADY_DEPLOYMENTS`) and classifies `full-reconcile` with a reason+ naming it. `T2_PLAN_MODE=false` keeps failing `PROFILE_UNHEALTHY`.+- T1 (`minikube-real-postgres.sh`) restores branch-profile GFS credentials on+ exit via `reconcile-gfs-deploy-credentials.sh` with+ `GFS_RESTORE_ACTIVE_NOLOGIN=true` (armed only once suites started), and+ `pre-gate-sync.sh` `provision_gfs_serving` passes the same opt-in and runs+ in every sync plan, restarting an unready `gfsc-reader` after restore.- T1 stays fail-loud (no green on unavailable DSN or zero tests); the JSONreporter is the suite verdict, not a leftover Vitest process exit after acomplete green reporter. Role-reset suites keep the throwaway Postgres 16.diff --git a/.cursor/skills/minikube-t0-t1-t2/SKILL.md b/.cursor/skills/minikube-t0-t1-t2/SKILL.mdindex 882d7877d..d541a0fb0 100644--- a/.cursor/skills/minikube-t0-t1-t2/SKILL.md+++ b/.cursor/skills/minikube-t0-t1-t2/SKILL.md@@ -54,9 +54,12 @@ transition = full-bootstrap (fresh/uninitialized profile)?(its internal planner uses T2_PLAN_MODE=true so full-bootstrap isreachable; IMAGE_SOURCE=local is enforced by the orchestrator).-transition = full-reconcile (deploy/* or charts/* changed)?+transition = full-reconcile (deploy/* or charts/* changed, OR a required+ deployment is unready on a bootstrapped profile)?└── make minikube-t2. Never downgrade an infra change to a- service-only restart.+ service-only restart. An unready deployment is repaired inside the+ run (planner full-reconcile + pre-gate GFS restore), not by a manual+ script between runs.transition = targeted-sync (service-only diff)?└── make minikube-t2 performs the targeted deploy; record it as a@@ -85,6 +88,10 @@ T1 runs inside `make minikube-t2`, or explicitly viathrowaway `postgres:16-alpine`, never the shared `control-postgres`.- The admin DSN is resolved in-process from the cluster Secret; never print,export, or persist it.+- T1 restores branch-profile GFS credentials on exit (canonical+ `reconcile-gfs-deploy-credentials.sh` with `GFS_RESTORE_ACTIVE_NOLOGIN=true`,+ same as the GFS T1 gate), so a T1 run cannot leave `gfsc-reader` NOLOGIN+ and poison the T2 preflight. Do not run that restore by hand.## Step 4 — Verdict and evidence reportingdiff --git a/.cursor/skills/minikube-t0-t1-t2/reference.md b/.cursor/skills/minikube-t0-t1-t2/reference.mdindex 141d81b94..53e43975f 100644--- a/.cursor/skills/minikube-t0-t1-t2/reference.md+++ b/.cursor/skills/minikube-t0-t1-t2/reference.md@@ -49,7 +49,7 @@ Companion to `SKILL.md`. Source of truth: `scripts/minikube/t2.sh`,| `DEVELOPMENT_SCOPE_REQUIRED` | Preflight/final preflight failed a precondition, or T2-only mode was attempted without `already-synced`. | Repair the first reported condition; if T2-only was refused, run full `make minikube-t2`. || `ZERO_TESTS_EXECUTED` | A lane was configured off or executed nothing (including a required-but-missing Playwright journey). | Re-run with the lane enabled, or supply the required `T2_PLAYWRIGHT_COMMAND`. || `POSTGRES_NOT_READY` | PostgreSQL precondition failed, or a PVC reset was requested without the exact expected UID. | Fix DB readiness; never guess a PVC UID. |-| `PROFILE_UNHEALTHY` | Opt-in user-facing health command failed. | Fix the failing endpoint, re-run T2 on the same HEAD. |+| `PROFILE_UNHEALTHY` | A required deployment is unready in a fail-loud check (`T2_PLAN_MODE=false`: standalone preflight or the final exact-head T2), or the opt-in user-facing health command failed. The orchestrator planner never emits this for an unready deployment — it selects `full-reconcile` instead. | Re-run `make minikube-t2` on the same HEAD; the single run reconciles and restores GFS credentials itself. Do not run manual repair scripts between runs. |## GFS restore pointers (high level, no secrets)@@ -60,6 +60,13 @@ profile. Restore/DSN provisioning for `gfs` is ordered by full-setup /`minikube-deploy-all` note in the Makefile). Do not hand-provision GFS DSNs,and never copy DSNs or credentials between namespaces or into evidence.+The harness owns NOLOGIN recovery: T1 restores branch-profile GFS credentials+on exit, and `pre-gate-sync` provisions serving with+`GFS_RESTORE_ACTIVE_NOLOGIN=true` in every plan and restarts an unready+`gfsc-reader` after restore. Do not run+`reconcile-gfs-deploy-credentials.sh` or `kubectl rollout restart+deploy/gfsc-reader` by hand as a T2-repair step — re-run the entry point.+## Dual-repo noteThis contract belongs to `evenfire-ai/evenfire` only. `evenfire-infra` is adiff --git a/AGENTS.md b/AGENTS.mdindex 80924984c..356921130 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -92,7 +92,15 @@ path is valid only when the pre-gate marker already matches HEADA fresh or uninitialized profile must complete the supported bootstrap. Thestandalone preflight refuses to call `pre-gate-sync` on an incomplete profile;`make minikube-t2` runs the supported setup when its planner selects-`full-bootstrap`. Real PostgreSQL suites are opt-in in the ordinary test+`full-bootstrap`. The single run is self-healing: when a bootstrapped profile+has an unready required deployment, the orchestrator planner selects+`full-reconcile` (naming the deployment) instead of failing+`PROFILE_UNHEALTHY` before a transition exists; the T1 lane restores+branch-profile GFS credentials on exit and `pre-gate-sync` provisions GFS+serving with `GFS_RESTORE_ACTIVE_NOLOGIN=true` in every plan, so no manual+repair script belongs between plan and verdict. The standalone preflight and+the final exact-head T2 check stay fail-loud on an unready deployment.+Real PostgreSQL suites are opt-in in the ordinary testmatrix, but a T1 run that requires them must fail when the database/DSN isunavailable or when zero tests execute; a green run must never be produced bysilently skipping the suites. The JSON reporter is the suite verdict; a leftoverdiff --git a/CLAUDE.md b/CLAUDE.mdindex 26cb6cfeb..72635b17b 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -18,7 +18,12 @@ produced only by the final exact-head preflight inside `make minikube-t2` (or`make minikube-pre-gate-sync` alone is not T2. Playwright and product E2Escripts are separate lanes. T1 judges the JSON reporter (expected files,executed/passed, zero failures, zero pending); a leftover Vitest process exit-after a complete green reporter is not a failed suite. Follow the rule+after a complete green reporter is not a failed suite. The single run is+self-healing: the orchestrator planner selects `full-reconcile` for a+bootstrapped profile with an unready deployment (never `PROFILE_UNHEALTHY`+before a transition), T1 restores branch-profile GFS credentials on exit, and+`pre-gate-sync` provisions GFS serving with `GFS_RESTORE_ACTIVE_NOLOGIN=true`+in every plan; do not insert manual repair scripts between runs. Follow the rule`.cursor/rules/minikube-t0-t1-t2.mdc` and the skill`.cursor/skills/minikube-t0-t1-t2/SKILL.md` for the certification workflow.diff --git a/docs/testing/minikube-t2-runbook.md b/docs/testing/minikube-t2-runbook.mdindex f48c3f63b..22d6c2636 100644--- a/docs/testing/minikube-t2-runbook.md+++ b/docs/testing/minikube-t2-runbook.md@@ -77,6 +77,15 @@ the marker does not already match HEAD. Harness, documentation, Makefile, and`scripts/e2e` diffs do not force a full reconcile. The runner refuses todowngrade a `deploy/` or `charts/` change to a service-only restart.+A bootstrapped profile with an unready required deployment also selects+`full-reconcile` — but only in the orchestrator planner (`T2_PLAN_MODE=true`),+with a reason that names the unready deployment. The planner must never stop+with `PROFILE_UNHEALTHY` before a transition is selected: that turned every+mid-run failure into a manual repair script followed by another full run.+The standalone `make minikube-t2-preflight` and the final exact-head T2 check+(`T2_PLAN_MODE=false`) remain fail-loud if a deployment is still unready+after the reconcile.+## Ownership and concurrencyThe lock is keyed by repository, branch, `HEAD`, and profile, while the@@ -123,6 +132,19 @@ that drop or rewrite cluster-global roles (`db.realPostgresMigration`,never share live `control-postgres` (#412). CI continues to use`CONTROL_API_REAL_PG_ADMIN_URL` unchanged.+GFS runtime credentials are self-healing inside the single run. The+gfs-controller shared suites exercise cluster-global role names, so the T1+lane restores the branch-profile GFS credentials on exit (success, failure,+or interrupt) using the canonical+`deploy/scripts/reconcile-gfs-deploy-credentials.sh` with+`GFS_RESTORE_ACTIVE_NOLOGIN=true` — the same contract as the standalone GFS+T1 gate. `pre-gate-sync` provisions GFS serving with the same opt-in in every+sync plan (including "no cluster sync required") and restarts an unready+`gfsc-reader` after a successful restore. The recovery helper restores a+NOLOGIN role only from the committed Secret DSN and still fails loud when+that credential cannot authenticate; no password is ever invented and no DSN+is printed.+## T0, T1, and T2 boundaries* **T0** — shell syntax/ShellCheck where available, contract tests, affecteddiff --git a/scripts/e2e/minikube-real-postgres.sh b/scripts/e2e/minikube-real-postgres.shindex e93e24410..5d89f3332 100755--- a/scripts/e2e/minikube-real-postgres.sh+++ b/scripts/e2e/minikube-real-postgres.sh@@ -33,6 +33,10 @@ T1_NEXT_COMMAND='repair the first reported Real PostgreSQL precondition, then reT1_TOTAL_TESTS=0T1_PASSED_TESTS=0T1_PENDING_TESTS=0+# Armed just before the Real PostgreSQL suites run. The exit trap must not+# reconcile cluster credentials for a run that never reached the suites+# (e.g. a failed precondition, or this file being sourced by a harness test).+T1_GFS_RESTORE_REQUIRED=falsedie_t1() {local code="$1"@@ -52,6 +56,26 @@ stop_isolated_postgres() {fi}+restore_gfs_runtime_credentials() {+ # The gfs-controller shared suites exercise the production role names, which+ # are cluster-global even when their fixture databases are temporary.+ # Restore the branch profile before T1 exits — success, failure, or+ # interruption — so a T1 run cannot leave GFSC NOLOGIN and poison the+ # following exact-head T2 preflight. Same canonical restore contract as+ # scripts/e2e/gfs-real-pg-minikube-gate.sh; DSNs stay inside the helper.+ if [ "$T1_GFS_RESTORE_REQUIRED" != true ] || [ -z "$T2_CONTEXT" ]; then+ return 0+ fi+ if ! t2_kc -n gfs get secret gfs-controller-db >/dev/null 2>&1; then+ return 0+ fi+ if ! GFS_RESTORE_ACTIVE_NOLOGIN=true CONTEXT="$T2_CONTEXT" \+ bash "$PROJECT_DIR/deploy/scripts/reconcile-gfs-deploy-credentials.sh"; then+ printf '[minikube-t1] ERROR: failed to restore branch-profile GFS credentials\n' >&2+ return 1+ fi+}+cleanup_t1() {local status=$?if [ -n "$PORT_FORWARD_PID" ] && kill -0 "$PORT_FORWARD_PID" >/dev/null 2>&1; then@@ -63,6 +87,9 @@ cleanup_t1() {fifistop_isolated_postgres+ if ! restore_gfs_runtime_credentials; then+ status=1+ fiif [ -n "$T1_TMP_DIR" ] && [ -d "$T1_TMP_DIR" ]; then rm -rf "$T1_TMP_DIR"; fiexit "$status"}@@ -364,6 +391,7 @@ print("postgresql://" + quote(user.decode(), safe="") + ":" + quote(password.dec')"require_isolated_control_api_filesstart_isolated_postgres+ T1_GFS_RESTORE_REQUIRED=truerun_suite control-api isolated "$ISOLATED_DSN"run_suite control-api shared "$ADMIN_DSN"run_suite gfs-controller shared "$ADMIN_DSN"diff --git a/scripts/minikube/pre-gate-sync.sh b/scripts/minikube/pre-gate-sync.shindex 8615b9fff..312b6e8d6 100755--- a/scripts/minikube/pre-gate-sync.sh+++ b/scripts/minikube/pre-gate-sync.sh@@ -316,16 +316,43 @@ sync_mcp_host_auth_key() {bash "${PROJECT_DIR}/scripts/minikube/sync-auth-key.sh" --context "${PROFILE}"}+converge_gfs_reader_after_restore() {+ # A reader whose runtime role was NOLOGIN keeps serving 503 from its+ # already-started pod even after the role is restored. Converging it here+ # keeps the repair inside the single orchestrated run instead of a manual+ # `kubectl rollout restart deploy/gfsc-reader` side quest.+ local deployment=gfsc-reader desired ready+ if ! ${KC} get deployment "${deployment}" -n gfs >/dev/null 2>&1; then+ return 0+ fi+ desired="$(${KC} get deployment "${deployment}" -n gfs -o jsonpath='{.spec.replicas}' 2>/dev/null || true)"+ ready="$(${KC} get deployment "${deployment}" -n gfs -o jsonpath='{.status.readyReplicas}' 2>/dev/null || true)"+ if [[ -z "${desired}" || "${desired}" == "0" || "${ready:-0}" == "${desired}" ]]; then+ return 0+ fi+ log "Restarting gfs/${deployment} after credential restore (${ready:-0}/${desired} Ready)"+ rollout_restart_with_retry gfs "${deployment}"+ ${KC} rollout status "deployment/${deployment}" -n gfs --timeout=180s >/dev/null+}+provision_gfs_serving() {if ${KC} get configmap gfs-config -n gfs >/dev/null 2>&1; thenlog "Provisioning gfs serving before ${GATE_NAME}"# FAIL LOUD: with the GFS stack deployed, a broken gfs_controller credential# means every GFS operation 503s (issue #775). Continuing would burn the# whole gate run on a cluster that cannot pass.- if ! CONTEXT="${PROFILE}" bash "${PROJECT_DIR}/deploy/scripts/reconcile-gfs-deploy-credentials.sh"; then+ # GFS_RESTORE_ACTIVE_NOLOGIN=true: a role-reset T1 suite can leave the+ # cluster-global reader role NOLOGIN. On this local branch-owned profile+ # the committed Secret DSN is the source of truth, so the recovery helper+ # may restore the role from it — the same contract the GFS T1 gate already+ # applies on exit. The helper still fails loud when the restored+ # credential cannot authenticate; no password is invented here.+ if ! GFS_RESTORE_ACTIVE_NOLOGIN=true CONTEXT="${PROFILE}" \+ bash "${PROJECT_DIR}/deploy/scripts/reconcile-gfs-deploy-credentials.sh"; thenlog "ERROR: gfs DB provisioning FAILED — gfsc cannot authorize any operation. Aborting ${GATE_NAME} pre-gate sync."exit 1fi+ converge_gfs_reader_after_restoreelselog "Skipping gfs serving provisioning (gfs-config not found — GFS stack not deployed)"fi@@ -507,6 +534,15 @@ if [[ "${cluster_changed}" == "true" ]]; thenprovision_gfs_servingfi+ if ! incremental_requires_database_reconcile; then+ # No schema work is planned, but a role-reset T1 run may still have left+ # the cluster-global GFS runtime roles NOLOGIN while every image and+ # manifest matched. Serving provisioning is idempotent, so converge it in+ # every sync plan instead of leaving an unready gfsc-reader to fail the+ # exact-head T2 preflight after this sync reports success.+ provision_gfs_serving+ fi+# Schema/control-plane first: only after migrations, runtime roles, and# inventory gates have converged may the new consumer workloads be applied or# restarted. The workflow reconciler is fenced above for this window.@@ -574,6 +610,11 @@ if [[ "${cluster_changed}" == "true" ]]; thenlog "Pre-gate cluster sync complete"elselog "No cluster sync required before ${GATE_NAME}"+ # Images and manifests match, yet the GFS runtime credentials can still be+ # broken (a role-reset T1 leaves the reader NOLOGIN without changing any+ # fingerprint). Converge serving credentials on every pre-gate pass so a+ # single orchestrated run repairs them without a manual script.+ provision_gfs_servingassert_no_legacy_prompt_bridge_grantsensure_evenfire_registryrollout_if_present control-plane nginx-workflow-approval-gatewaydiff --git a/scripts/minikube/t2-common.sh b/scripts/minikube/t2-common.shindex 5dbdac6ae..5ab1a12be 100755--- a/scripts/minikube/t2-common.sh+++ b/scripts/minikube/t2-common.sh@@ -47,6 +47,7 @@ T2_PLAN_MODE="$T2_PLAN_MODE"if [ -z "$T2_PLAN_MODE" ]; then T2_PLAN_MODE=false; fiT2_BOOTSTRAP_REQUIRED=falseT2_PROFILE_HEALTHY=false+T2_UNREADY_DEPLOYMENTS=""T2_MARKER_MATCHES_HEAD=falseT2_MARKER_JSON=""T2_IMAGE_SOURCE=""@@ -460,7 +461,7 @@ PY}t2_deployment_check() {- local deployment_json+ local deployment_json unreadydeployment_json="$(t2_kc get deployments -A -o json 2>/dev/null || true)"if [ -z "$deployment_json" ]; thenif [ "$T2_BOOTSTRAP_REQUIRED" = true ]; then@@ -471,7 +472,7 @@ t2_deployment_check() {T2_NEXT_COMMAND="MINIKUBE_PROFILE=$T2_PROFILE make minikube-setup-local"t2_fail PROFILE_UNHEALTHY 'deployment readiness inventory is unavailable'fi- if ! python3 - "$deployment_json" <<'PY'+ unready="$(python3 - "$deployment_json" <<'PY'import jsonimport syspayload = json.loads(sys.argv[1])@@ -487,18 +488,27 @@ for item in payload.get("items", []):available = int(status.get("availableReplicas") or 0)if ready < desired or available < desired:bad.append("%s/%s %s/%s" % (metadata.get("namespace", "?"), metadata.get("name", "?"), ready, desired))-if bad:- print("; ".join(bad), file=sys.stderr)- raise SystemExit(1)+print("; ".join(bad))PY- then+ )"+ if [ -n "$unready" ]; thenif [ "$T2_BOOTSTRAP_REQUIRED" = true ]; thenT2_PLAN_STATE=full-bootstrap- T2_PLAN_REASON='one or more deployments are not Ready'+ T2_PLAN_REASON="one or more deployments are not Ready: $unready"+ return 0+ fi+ if [ "$T2_PLAN_MODE" = true ]; then+ # Orchestrator planner: a bootstrapped profile with an unready required+ # deployment is repaired by a full reconcile inside the same run.+ # Failing PROFILE_UNHEALTHY here would abort before any transition is+ # selected and force manual repair scripts. The standalone preflight and+ # the final exact-head check run with T2_PLAN_MODE=false and stay+ # fail-loud below.+ T2_UNREADY_DEPLOYMENTS="$unready"return 0fiT2_NEXT_COMMAND="MINIKUBE_PROFILE=$T2_PROFILE make minikube-t2"- t2_fail PROFILE_UNHEALTHY 'one or more deployments are not Ready'+ t2_fail PROFILE_UNHEALTHY "one or more deployments are not Ready: $unready"fi}@@ -545,6 +555,14 @@ t2_classify_transition() {[ -n "$T2_PLAN_REASON" ] || T2_PLAN_REASON='profile is not bootstrapped'return 0fi+ if [ -n "$T2_UNREADY_DEPLOYMENTS" ]; then+ # Recorded by t2_deployment_check in planner mode only. Reconcile the+ # running profile in place; an unready deployment must never certify as+ # already-synced nor stop the orchestrator before a transition exists.+ T2_PLAN_STATE=full-reconcile+ T2_PLAN_REASON="deployment not Ready: $T2_UNREADY_DEPLOYMENTS"+ return 0+ fiif [ "$T2_MARKER_MATCHES_HEAD" = true ]; thenT2_PLAN_STATE=already-syncedT2_PLAN_REASON='pre-gate marker already matches HEAD'diff --git a/scripts/tests/test-minikube-t2-contract.sh b/scripts/tests/test-minikube-t2-contract.shindex e416e469d..07b6d5837 100755--- a/scripts/tests/test-minikube-t2-contract.sh+++ b/scripts/tests/test-minikube-t2-contract.sh@@ -72,6 +72,28 @@ grep -Fq 'numTotalTests' "$T1"grep -Fq 'port-forward' "$T1"grep -Fq 'set +x' "$T1" "$PREFLIGHT" "$T2"+# T1 must restore the branch-profile GFS credentials on the way out with the+# same canonical script and NOLOGIN opt-in the GFS T1 gate uses, so a T1 run+# cannot leave gfsc-reader unready and fail the exact-head T2 preflight.+grep -Fq 'restore_gfs_runtime_credentials' "$T1"+grep -Fq 'GFS_RESTORE_ACTIVE_NOLOGIN=true' "$T1"+grep -Fq 'reconcile-gfs-deploy-credentials.sh' "$T1"+grep -Fq 'T1_GFS_RESTORE_REQUIRED=true' "$T1"+grep -Fq 'T2_UNREADY_DEPLOYMENTS' "$COMMON"++# pre-gate-sync GFS serving provisioning must opt into restoring a NOLOGIN+# role from the committed Secret DSN, and must run in every sync plan.+PRE_GATE="$MINIKUBE_DIR/pre-gate-sync.sh"+if ! sed -n '/^provision_gfs_serving()/,/^}/p' "$PRE_GATE" | grep -Fq 'GFS_RESTORE_ACTIVE_NOLOGIN=true'; then+ echo 'FAIL: provision_gfs_serving does not restore a NOLOGIN GFS role from the committed Secret DSN' >&2+ exit 1+fi+if ! sed -n '/No cluster sync required before/,$p' "$PRE_GATE" | grep -Fq 'provision_gfs_serving'; then+ echo 'FAIL: pre-gate-sync skips GFS serving convergence when no cluster sync is required' >&2+ exit 1+fi+grep -Fq 'converge_gfs_reader_after_restore' "$PRE_GATE"+if grep -Eq 'make minikube-pre-gate-sync|pre-gate-sync\.sh' "$PREFLIGHT"; thenecho 'FAIL: preflight invokes pre-gate-sync' >&2exit 1diff --git a/scripts/tests/test-minikube-t2-scenarios.sh b/scripts/tests/test-minikube-t2-scenarios.shindex 1f6c38440..297fc7d9f 100755--- a/scripts/tests/test-minikube-t2-scenarios.sh+++ b/scripts/tests/test-minikube-t2-scenarios.sh@@ -228,6 +228,25 @@ already_state="$(env "${repo_env[@]}" T2_PROJECT_DIR="$repo" T2_BOOTSTRAP_REQUIRbash -c 'source "$1"; T2_ORIGIN_DEV="$2"; T2_HEAD="$3"; T2_BOOTSTRAP_REQUIRED=false; T2_MARKER_MATCHES_HEAD=true; t2_classify_transition; printf "%s" "$T2_PLAN_STATE"' bash "$COMMON" "$base_sha" "$infra_sha")"[ "$already_state" = already-synced ] || fail "matching marker selected $already_state instead of already-synced"+# A bootstrapped profile with an unready required deployment must not stop+# the orchestrator planner before a transition exists (PROFILE_UNHEALTHY was+# the multi-script loop): planner mode selects full-reconcile and names the+# deployment, even when the marker already matches HEAD.+unready_json='{"items":[{"metadata":{"namespace":"gfs","name":"gfsc-reader"},"spec":{"replicas":1},"status":{"readyReplicas":0,"availableReplicas":0}}]}'+planner_unready="$(env "${repo_env[@]}" UNREADY_JSON="$unready_json" \+ bash -c 'source "$1"; T2_PLAN_MODE=true; T2_BOOTSTRAP_REQUIRED=false; T2_MARKER_MATCHES_HEAD=true; t2_kc(){ printf "%s" "$UNREADY_JSON"; }; t2_deployment_check; t2_classify_transition; printf "%s|%s" "$T2_PLAN_STATE" "$T2_PLAN_REASON"' bash "$COMMON")"+[ "${planner_unready%%|*}" = full-reconcile ] || fail "planner mode selected ${planner_unready%%|*} instead of full-reconcile for an unready deployment"+case "$planner_unready" in+ *gfs/gfsc-reader*) ;;+ *) fail 'planner full-reconcile reason does not name the unready deployment' ;;+esac++# Negative pin: the standalone preflight and the final exact-head check+# (T2_PLAN_MODE=false) stay fail-loud on an unready deployment.+expect_code PROFILE_UNHEALTHY unready-final-preflight unready-final-preflight \+ env "${repo_env[@]}" UNREADY_JSON="$unready_json" \+ bash -c 'source "$1"; T2_PLAN_MODE=false; T2_BOOTSTRAP_REQUIRED=false; t2_kc(){ printf "%s" "$UNREADY_JSON"; }; t2_deployment_check' bash "$COMMON"+plan_mode_head="$(env "${marker_env[@]}" T2_PLAN_MODE=true FAKE_MARKER='{"data":{"clusterFingerprint":"fp","gitHead":"old","worktreeId":"worktree-a","imageSource":"local","imageTag":"test"}}' \bash -c 'source "$1"; T2_WORKTREE_ID=worktree-a; T2_HEAD=feature; T2_PLAN_MODE=true; t2_kc(){ printf "%s" "$FAKE_MARKER"; }; t2_marker_check; printf "%s" "$T2_MARKER_MATCHES_HEAD"' bash "$COMMON")"[ "$plan_mode_head" = false ] || fail "planner mode still treated a stale marker as matching HEAD"@@ -248,6 +267,9 @@ grep -Fq 'ISOLATED_DSN' "$ROOT/scripts/e2e/minikube-real-postgres.sh"grep -Fq '|| suite_status=$?' "$ROOT/scripts/e2e/minikube-real-postgres.sh" || fail 'T1 must capture vitest exit without aborting before the reporter'grep -Fq 'complete green reporter' "$ROOT/scripts/e2e/minikube-real-postgres.sh" || fail 'T1 must document reporter-authoritative verdict'grep -Fq 'Real PostgreSQL suite failed in $package ($lane)' "$ROOT/scripts/e2e/minikube-real-postgres.sh" && fail 'T1 still treats npm exit as a suite failure before the JSON reporter'+grep -Fq 'restore_gfs_runtime_credentials' "$ROOT/scripts/e2e/minikube-real-postgres.sh" || fail 'T1 must restore branch-profile GFS credentials on exit'+grep -Fq 'GFS_RESTORE_ACTIVE_NOLOGIN=true' "$ROOT/scripts/e2e/minikube-real-postgres.sh" || fail 'T1 GFS restore must opt into the NOLOGIN recovery contract'+grep -Fq 'GFS_RESTORE_ACTIVE_NOLOGIN=true' "$ROOT/scripts/minikube/pre-gate-sync.sh" || fail 'pre-gate-sync must restore a NOLOGIN GFS role from the committed Secret DSN'grep -Fq 'T2_LOCK_ROOT' "$COMMON"grep -Fq 'trap t2_lock_release EXIT INT TERM' "$COMMON"grep -Fq 'PORT_FORWARD_CONFLICT' "$COMMON"