Skip to content

The ultimate harness for coding agents and beyond. All your agents. All your machines. One command center. Start with code, then follow your curiosity and build across disciplines: CAD, circuits, robots, games and music.

Authenticate CI to GCP without a long-lived service-account key (#30)

autonomousdevops0e9f6051bdfc
diff --git a/.github/workflows/production-be-build.yaml b/.github/workflows/production-be-build.yaml
index e514f067e..05bf635ed 100644
--- a/.github/workflows/production-be-build.yaml
+++ b/.github/workflows/production-be-build.yaml
@@ -1,5 +1,22 @@
name: Docker production backend build
+# Builds backend/Dockerfile.k8s and pushes it to gcr.io/autonomous-ecm/autonomous-code-be, on a
+# pushed `v*_api` tag. The `_api` suffix is what keeps this trigger and release.yml's `v*_cli` one
+# from firing on each other now that the CLI and the backend share a repo.
+#
+# The build itself lives in autonomous-ai/github-templates. This repo is the only caller of the `-v2`
+# template: a clone of docker-build-and-push.yaml with workload identity federation, SHA-pinned
+# actions, no caller value spliced into a shell, and deploy secrets as BuildKit mounts rather than
+# build args. The original is untouched and still serves autonomous-code, ecm-web and the standalone
+# harness backend, so nothing here can break their builds — they migrate when they choose.
+#
+# `@main` is a MOVING reference: whoever can push to that repo decides what runs here with GCP push
+# credentials attached. Pin it to a commit SHA and bump it deliberately — that matters more from a
+# public repo than a private one.
+#
+# Nothing here is reachable from a pull request: the only trigger is a tag push, which only a
+# maintainer with write access can make. A fork's PR cannot run this job and cannot see its secrets.
+
on:
push:
tags:
@@ -7,7 +24,14 @@ on:
jobs:
build-autonomous-code-be-production:
- uses: autonomous-ai/github-templates/.github/workflows/docker-build-and-push.yaml@main
+ uses: autonomous-ai/github-templates/.github/workflows/docker-build-and-push-v2.yaml@main
+ # Permissions belong to the CALLER: a called workflow can never hold more than it was granted, so
+ # the template deliberately declares none of its own. `contents: read` because the template's
+ # actions/checkout is the only token use; `id-token: write` to mint the OIDC token the federated
+ # auth step exchanges for a short-lived GCP credential.
+ permissions:
+ contents: read
+ id-token: write
with:
DOCKER_IMAGE_NAME: autonomous-code-be
DOCKER_CONTEXT: backend
@@ -15,4 +39,20 @@ jobs:
CONCURRENCY_GROUP: docker_build_web_production
RUNNER_TAG: ubuntu-latest
secrets:
+ # NOTE THE `_GCR` SUFFIX. The registry and the CLI release bucket live in DIFFERENT GCP
+ # projects, so they need different service accounts — this one in the registry's project,
+ # release.yml's plain GCP_SERVICE_ACCOUNT in the bucket's.
+ #
+ # The PROVIDER is shared, though. A pool in one project can be granted impersonation on a
+ # service account in another (the project number inside the pool's resource name is the pool's,
+ # never the service account's), so one pool and one provider serve both.
+ #
+ # Set all of them with scripts/setup-gcp-wif.sh in autonomous-ai/github-templates — run it
+ # once per service account.
+ # While they are unset these pass through as empty strings and the template falls back to the
+ # key below, so this file is correct before and after the migration.
+ GCP_WORKLOAD_IDENTITY_PROVIDER: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
+ GCP_SERVICE_ACCOUNT: ${{ secrets.GCP_SERVICE_ACCOUNT_GCR }}
+ # The legacy path. Delete this line once a tag build has gone green on federation, then delete
+ # the key itself in GCP — an unused key that still authenticates is still a liability.
GCLOUD_DEV_SERVICE_ACCOUNT_JSON: ${{ secrets.GCLOUD_DEV_SERVICE_ACCOUNT_JSON }}
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index ebd1dc67f..ac6417b84 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -15,8 +15,9 @@ name: Release
#
# Shape borrowed from autonomous-harness-desktop's release.yml, minus the parts that do not apply:
#
-# * SAME bucket, SAME `GCP_SA_KEY` service account, SAME preflight, SAME "the tag is the version"
-# rule, SAME annotated-tag release notes, SAME `metadata_path` dry run.
+# * SAME bucket, SAME preflight, SAME "the tag is the version" rule, SAME annotated-tag release
+# notes, SAME `metadata_path` dry run. NOT the same credential: this one authenticates with
+# workload identity federation instead of a long-lived service-account key (see below).
# * NO scratch-manifest merge. The desktop fans out to three build jobs that each own a manifest
# key, so it points each at its own scratch file and merges once at the end. The CLI is ONE
# platform-independent bundle under ONE key, published by ONE job — the race the desktop
@@ -56,7 +57,11 @@ on:
permissions:
contents: read
- id-token: write # only used by the Workload Identity Federation variant of the auth steps below
+ # Mints the OIDC token the auth steps exchange for a short-lived GCP credential. It is a
+ # capability, not a secret: what it can reach is decided entirely by the pool's
+ # attribute-condition and the service account's IAM bindings, both set by
+ # autonomous-ai/github-templates: scripts/setup-gcp-wif.sh (runbook: WORKLOAD-IDENTITY.md).
+ id-token: write
concurrency:
# Two releases overlapping would interleave their manifest writes, and the loser's version would
@@ -104,17 +109,45 @@ jobs:
# beside `version` and `tests`, so it is free in wall clock.
preflight:
runs-on: ubuntu-latest
+ env:
+ # The `secrets` context cannot be read from a step-level `if:`, so reduce it to a boolean here
+ # — job-level `env:` can see it. Holds "true"/"false", never the value itself.
+ HAS_WIF: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER != '' }}
steps:
- - uses: google-github-actions/auth@v2
+ # GCP auth, WITHOUT a long-lived key wherever possible.
+ #
+ # A service-account JSON key is a bearer credential with no expiry: it is only as safe as every
+ # place it has ever been copied to — a laptop, a password manager, a paste into chat, a CI log
+ # somebody widened. Workload Identity Federation replaces it with a short-lived token minted
+ # from this job's own OIDC identity, so there is nothing to leak and nothing to rotate. The GCP
+ # side is one script, in autonomous-ai/github-templates: scripts/setup-gcp-wif.sh.
+ #
+ # Two steps, not one, only for the duration of the cutover: WIF the moment the
+ # GCP_WORKLOAD_IDENTITY_PROVIDER secret exists, the key until then, so provisioning GCP and
+ # switching CI are not the same deploy.
+ # DELETE the second step (and the GCP_SA_KEY secret, and the key itself) once a release has
+ # gone green on the first — see docs/cicd.md.
+ - name: Authenticate to GCP (workload identity federation)
+ if: env.HAS_WIF == 'true'
+ uses: google-github-actions/auth@c200f3691d83b41bf9bbd8638997a462592937ed # v2
+ with:
+ workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
+ service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
+ - name: Authenticate to GCP (service-account key — legacy fallback)
+ if: env.HAS_WIF != 'true'
+ uses: google-github-actions/auth@c200f3691d83b41bf9bbd8638997a462592937ed # v2
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
- - uses: google-github-actions/setup-gcloud@v2
+ - uses: google-github-actions/setup-gcloud@e427ad8a34f8676edf47cf7d7925499adf3eb74f # v2
- name: Verify GCS write access
run: |
set -euo pipefail
probe="gs://${GCS_BUCKET}/harness/cli/.ci-preflight/${GITHUB_RUN_ID}"
- echo "$GITHUB_RUN_ID" | gsutil cp - "$probe"
- gsutil rm "$probe"
+ # `gcloud storage`, not `gsutil`: gsutil is a standalone Python tool that reads gcloud's
+ # credential store, and it does not handle the external-account (federated) credential WIF
+ # produces. `gcloud storage` is the same gcloud binary that did the token exchange.
+ echo "$GITHUB_RUN_ID" | gcloud storage cp - "$probe"
+ gcloud storage rm "$probe"
tests:
uses: ./.github/workflows/ci.yml
@@ -130,6 +163,8 @@ jobs:
# dry run it is the scratch manifest nothing polls.
METADATA_PATH: ${{ needs.version.outputs.metadata_path }}
VERSION: ${{ needs.version.outputs.version }}
+ # See `preflight` above: `secrets` is unreadable from a step `if:`, so it becomes a boolean.
+ HAS_WIF: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER != '' }}
steps:
- uses: actions/checkout@v7
@@ -143,10 +178,20 @@ jobs:
- run: npm ci
working-directory: cli
- - uses: google-github-actions/auth@v2
+ # Same two-step auth as `preflight` above, and the same cutover note applies: delete the key
+ # fallback once a release has gone green on federation.
+ - name: Authenticate to GCP (workload identity federation)
+ if: env.HAS_WIF == 'true'
+ uses: google-github-actions/auth@c200f3691d83b41bf9bbd8638997a462592937ed # v2
+ with:
+ workload_identity_provider: ${{ secrets.GCP_WORKLOAD_IDENTITY_PROVIDER }}
+ service_account: ${{ secrets.GCP_SERVICE_ACCOUNT }}
+ - name: Authenticate to GCP (service-account key — legacy fallback)
+ if: env.HAS_WIF != 'true'
+ uses: google-github-actions/auth@c200f3691d83b41bf9bbd8638997a462592937ed # v2
with:
credentials_json: ${{ secrets.GCP_SA_KEY }}
- - uses: google-github-actions/setup-gcloud@v2
+ - uses: google-github-actions/setup-gcloud@e427ad8a34f8676edf47cf7d7925499adf3eb74f # v2
# From the repo root, the way `make upload-cli` calls it. The script bundles with
# ADAPTER_VERSION="$VERSION" baked in, refuses to upload if `node dist/cli.js version` does
@@ -289,6 +334,8 @@ jobs:
EOF
echo "::group::RELEASE_NOTES.md"; cat RELEASE_NOTES.md; echo "::endgroup::"
- - uses: softprops/action-gh-release@v2
+ # Pinned to a commit, not a tag: this step runs with `contents: write` and a moving tag on a
+ # third-party action means whoever can move it can run code against that token.
+ - uses: softprops/action-gh-release@3bb12739c298aeb8a4eeaf626c5b8d85266b0e65 # v2
with:
body_path: RELEASE_NOTES.md
diff --git a/.gitignore b/.gitignore
index 0cb083645..d0fab79c0 100644
--- a/.gitignore
+++ b/.gitignore
@@ -2,6 +2,10 @@ node_modules/
dist/
.tsbuildinfo
.DS_Store
+# IDE state. workspace.xml records local paths, run configs and sometimes datasource
+# credentials — none of it belongs in a public repo.
+.idea/
+.vscode/
.codegraph
# Per-deployment state, never source:
diff --git a/Makefile b/Makefile
index 042acf33c..a87e710a7 100644
--- a/Makefile
+++ b/Makefile
@@ -31,7 +31,8 @@ install-cli:
bash cli/scripts/install-cli.sh $(ARGS)
## upload-cli: bump version -> bundle -> publish the CLI. MAINTAINER ONLY — it writes to the release
-## bucket, so it needs an authenticated `gsutil` with write access (plus node/npm for the bundle step).
+## bucket, so it needs an authenticated `gcloud storage` (or gsutil) with write access on it, plus
+## node/npm for the bundle step.
## Running daemons pick the new version up within ~1 min.
upload-cli:
bash cli/scripts/upload-cli.sh $(ARGS)
diff --git a/cli/scripts/upload-cli.sh b/cli/scripts/upload-cli.sh
index e0feea9d4..e46cca778 100755
--- a/cli/scripts/upload-cli.sh
+++ b/cli/scripts/upload-cli.sh
@@ -12,7 +12,8 @@
# The CURRENT version is read from the remote metadata.json on GCS (single source of truth) and the
# patch is bumped from there — nothing is git-committed (the version is injected into the bundle via
# ADAPTER_VERSION, so the running binary's version equals the published manifest version).
-# Prereqs: `gsutil` authenticated with WRITE access; the bucket/objects must be public-read; `node`+`npm`.
+# Prereqs: `gcloud storage` (or gsutil) authenticated with WRITE access on the bucket; the
+# bucket/objects must be public-read; `node`+`npm`.
set -euo pipefail
ADAPTER_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # this package
@@ -60,8 +61,38 @@ for arg in "$@"; do
esac
done
-command -v gsutil >/dev/null 2>&1 || { echo "error: gsutil not found — install/authenticate the gcloud SDK" >&2; exit 1; }
-command -v node >/dev/null 2>&1 || { echo "error: node not found" >&2; exit 1; }
+command -v node >/dev/null 2>&1 || { echo "error: node not found" >&2; exit 1; }
+
+# --- GCS client: `gcloud storage` if we have it, else gsutil ---
+# gsutil is a standalone Python tool that only understands gcloud's *user* and *service-account-key*
+# credentials. It cannot use the external-account (federated) credential that Workload Identity
+# Federation issues, so a CI job authenticated by WIF fails on every gsutil call while the identical
+# `gcloud storage` call works — it is the same gcloud binary that performed the token exchange.
+# gsutil stays as the fallback for a laptop whose SDK predates `gcloud storage`.
+if command -v gcloud >/dev/null 2>&1 && gcloud storage --help >/dev/null 2>&1; then
+ GCS_CLI=gcloud
+elif command -v gsutil >/dev/null 2>&1; then
+ GCS_CLI=gsutil
+ echo ">> note: falling back to gsutil (no 'gcloud storage'); this will not work under workload identity federation" >&2
+else
+ echo "error: neither 'gcloud storage' nor gsutil found — install/authenticate the gcloud SDK" >&2
+ exit 1
+fi
+
+# gcs_cp <src> <dst> [cache-control] [content-type] — either side may be gs:// or a local path or `-`.
+gcs_cp() {
+ local src="$1" dst="$2" cc="${3:-}" ct="${4:-}" args=()
+ if [ "$GCS_CLI" = gcloud ]; then
+ args=(storage cp)
+ if [ -n "$cc" ]; then args+=("--cache-control=$cc"); fi
+ if [ -n "$ct" ]; then args+=("--content-type=$ct"); fi
+ gcloud "${args[@]}" "$src" "$dst"
+ else
+ if [ -n "$cc" ]; then args+=(-h "Cache-Control:$cc"); fi
+ if [ -n "$ct" ]; then args+=(-h "Content-Type:$ct"); fi
+ gsutil "${args[@]}" cp "$src" "$dst"
+ fi
+}
cleanup() { rm -f "${SRC:-}" "${DST:-}"; }
trap cleanup EXIT
@@ -115,12 +146,12 @@ NOTIFY_SHA="$(sha256_of "$NOTIFY")"; NOTIFY_SIZE="$(wc -c < "$NOTIFY" | tr -d '
echo ">> uploading cli.js ($CLI_SIZE bytes) + notify.mjs ($NOTIFY_SIZE bytes)"
# Immutable per-version path — see the CDN_ASSET_BASE_URL note above. Long max-age here is what
# actually lets the CDN cache these instead of hitting GCS on every daemon's install/update.
-gsutil -h "Cache-Control:public, max-age=31536000, immutable" cp "$CLI" "gs://${GCS_BUCKET}/${CLI_GCS}"
-gsutil -h "Cache-Control:public, max-age=31536000, immutable" cp "$NOTIFY" "gs://${GCS_BUCKET}/${NOTIFY_GCS}"
+gcs_cp "$CLI" "gs://${GCS_BUCKET}/${CLI_GCS}" "public, max-age=31536000, immutable"
+gcs_cp "$NOTIFY" "gs://${GCS_BUCKET}/${NOTIFY_GCS}" "public, max-age=31536000, immutable"
echo ">> merging manifest: gs://${GCS_BUCKET}/${METADATA_PATH} (${OTA_KEY})"
SRC="$(mktemp)"; DST="$(mktemp)" # removed by cleanup() on EXIT
-if ! gsutil cp "gs://${GCS_BUCKET}/${METADATA_PATH}" "$SRC" 2>/dev/null; then
+if ! gcs_cp "gs://${GCS_BUCKET}/${METADATA_PATH}" "$SRC" 2>/dev/null; then
echo " (no existing metadata.json — creating a new one)"
printf '{}' > "$SRC"
fi
@@ -129,9 +160,8 @@ node "$ADAPTER_DIR/scripts/merge-manifest.mjs" \
"$SRC" "$DST" "$OTA_KEY" "$VER" \
"$CLI_URL" "$CLI_SHA" "$CLI_SIZE" \
"$NOTIFY_URL" "$NOTIFY_SHA" "$NOTIFY_SIZE"
-gsutil -h "Content-Type:application/json" \
- -h "Cache-Control:no-cache, no-store, must-revalidate" \
- cp "$DST" "gs://${GCS_BUCKET}/${METADATA_PATH}"
+gcs_cp "$DST" "gs://${GCS_BUCKET}/${METADATA_PATH}" \
+ "no-cache, no-store, must-revalidate" "application/json"
echo ""
echo ">> published adapter $VER"
diff --git a/docs/cicd.md b/docs/cicd.md
index 0265f28c5..7d5fd806c 100644
--- a/docs/cicd.md
+++ b/docs/cicd.md
@@ -1,31 +1,219 @@
# CI/CD
-Two workflows, both under [`.github/workflows/`](../.github/workflows/).
+Three workflows, all under [`.github/workflows/`](../.github/workflows/).
| File | Runs when | Does |
|---|---|---|
| `ci.yml` | every pull request against `main`; on demand | `npm ci` → `npx tsc --noEmit` → `npx vitest run`, i.e. `make cli-test` |
-| `release.yml` | a pushed `v*` tag; on demand | bundles and publishes the CLI, then verifies the published bytes |
+| `release.yml` | a pushed `v*_cli` tag; on demand | bundles and publishes the CLI, then verifies the published bytes |
+| `production-be-build.yaml` | a pushed `v*_api` tag | builds `backend/Dockerfile.k8s` and pushes it to GCR |
-They are modelled on the ones in `autonomous-harness-desktop` and share that repo's release
-infrastructure: the same GCS bucket (`s3-autonomous-upgrade-3`), the same `GCP_SA_KEY` service
+`ci.yml` and `release.yml` are modelled on the ones in `autonomous-harness-desktop` and share that
+repo's release infrastructure: the same GCS bucket (`s3-autonomous-upgrade-3`), the same service
account, the same "the tag is the version" rule, the same annotated-tag release notes, and the same
-`metadata_path` dry run. What follows is only what is specific to the CLI.
+`metadata_path` dry run. What follows is only what is specific to this repo.
+
+## Why the tag suffixes
+
+Both release triggers live in one repo, so each one has to be unmistakable: `v0.1.72_cli` publishes
+the CLI and `v1.4.0_api` builds the backend image. `release.yml` strips both the `v` and the `_cli`
+before anything treats the string as a version — the suffix is a routing marker for the trigger and
+never reaches the manifest, the release title or the daemon's self-update comparison.
## Setup
-One secret, in **Settings → Secrets and variables → Actions**:
+### Public-repo ground rules
+
+This repo is public and takes fork pull requests, which puts one hard line through the middle of CI:
+
+- **`ci.yml` has no access to any secret, and must keep it.** It is triggered by `pull_request`, never
+ `pull_request_target` — a fork's PR therefore runs with a read-only `GITHUB_TOKEN` and an empty
+ secrets context, which is the only safe way to execute a stranger's code. Never add a secret to
+ `ci.yml`, and never switch its trigger; a step that needs a credential belongs in a separate
+ workflow gated on something a maintainer does.
+- **Both release workflows are reachable only by a tag push**, which requires write access. Fork PRs
+ cannot run them and cannot read their credentials.
+- **Anything interpolated into a `run:` block goes through `env:` first.** Git allows quotes,
+ backticks and `$()` in a ref name, so `${{ github.ref_name }}` written inline in a shell script is
+ a command-injection primitive. `release.yml` routes every such value through `env:` and reads it as
+ `"$REF_NAME"`.
+
+### GCP credentials: workload identity federation
+
+`release.yml` authenticates to GCP with **Workload Identity Federation** — no long-lived
+service-account key. The job presents its own GitHub OIDC token, GCP validates it against a provider
+that is pinned to this org, and hands back a token that expires in an hour. There is nothing to leak
+and nothing to rotate.
+
+**The release bucket and the backend's container registry are in different GCP projects**, and that
+shapes the setup. It does *not* mean two pools: a pool in one project can be granted impersonation on
+a service account in another — the project number inside the pool's resource name is the pool's,
+never the service account's. So one pool and one provider serve both, and only the service account
+differs:
+
+```
+ ONE pool + provider (pick either project as home)
+ │
+ ┌─────────────┴─────────────┐
+ ▼ ▼
+ SA in the bucket project SA in autonomous-ecm
+ storage.objectAdmin artifactregistry.writer / gcr bucket writer
+ secret GCP_SERVICE_ACCOUNT secret GCP_SERVICE_ACCOUNT_GCR
+ used by release.yml used by production-be-build.yaml
+```
+
+Provision it from a laptop with an owner-ish `gcloud` login — **once per service account**, not once
+per project. The second run reuses the first run's pool:
+
+The script and the full runbook live in **`autonomous-ai/github-templates`**, because every repo that
+calls those templates needs them:
+
+```bash
+git clone https://github.com/autonomous-ai/github-templates && cd github-templates
+gcloud auth login
+
+# 1. pool, provider, and the release identity in the bucket's project
+GITHUB_REPO=autonomous-harness GCP_PROJECT=<bucket-project> ENVIRONMENT=release \
+ bash scripts/setup-gcp-wif.sh
+
+# 2. the image-push identity in the registry's project, against that same pool
+GITHUB_REPO=autonomous-harness GCP_PROJECT=autonomous-ecm POOL_PROJECT=<bucket-project> \
+ SA_NAME=gha-autonomous-harness-gcr SECRET_NAME=GCP_SERVICE_ACCOUNT_GCR \
+ bash scripts/setup-gcp-wif.sh
+```
+
+See `WORKLOAD-IDENTITY.md` in that repo for what each step does and how the failures present.
+
+It creates the pool, the provider, a dedicated service account, and the IAM binding that lets *only*
+this repo impersonate it — then prints the two values to add under **Settings → Secrets and variables
+→ Actions → Secrets**:
+
+| Secret | Value | Used by |
+|---|---|---|
+| `GCP_WORKLOAD_IDENTITY_PROVIDER` | `projects/<pool-project-number>/locations/global/workloadIdentityPools/github/providers/github` | both |
+| `GCP_SERVICE_ACCOUNT` | `gha-autonomous-harness@<bucket-project>.iam.gserviceaccount.com` | `release.yml` |
+| `GCP_SERVICE_ACCOUNT_GCR` | `gha-autonomous-harness-gcr@autonomous-ecm.iam.gserviceaccount.com` | `production-be-build.yaml` |
+
+The provider is one value for both — they share a pool. Only the service account is per project, and
+the number in the provider path is the **pool's** project, which is the mistake worth not making.
+
+Neither is truly sensitive — a provider resource name and a service-account email are public
+identifiers, useless without a matching OIDC token — but they are secrets so that everything CI
+authenticates with lives in one place. The cost is that GitHub will not show you the value again
+after you save it, and both are masked as `***` in logs, so a typo in the provider path surfaces as
+an auth failure rather than something you can read off the run. Keep a copy of what the script
+printed.
+
+**The provider's `--attribute-condition` is the control that matters.** A provider created without one
+trusts every repository on GitHub: anyone could push a workflow to their own repo, mint an OIDC token
+and exchange it against yours. The script sets
+
+```
+assertion.repository_owner == 'autonomous-ai' && assertion.event_name != 'pull_request'
+```
+
+and refuses to proceed if it finds an existing provider with an empty condition. A fork of this public
+repo has a different owner, so the first clause stops the obvious attack — fork it, add a workflow,
+mint a token. The second is belt as well as braces (GitHub does not grant `id-token: write` to a
+fork's pull request) but it is the clause that survives somebody later adding a PR-triggered job by
+mistake. The per-service-account binding then narrows it to one repo. All of it is required; no single
+layer is enough.
+
+Pass `ENVIRONMENT=release` (as above) and the binding becomes an exact match on
+`repo:autonomous-ai/autonomous-harness:environment:release`, so the credential only exists for a job
+that declares `environment: release`. That is how you put **required reviewers in front of the
+credential**, which is worth doing here for the reason spelled out under
+[What "publish" actually means](#what-publish-actually-means-here): the manifest this job writes is
+polled by every running daemon every 60 seconds. If you take that option, add `environment: release`
+to the `preflight` and `publish` jobs and configure the environment's reviewers in repo settings.
+
+### Cutover, and finishing it
+
+The auth steps try WIF when the `GCP_WORKLOAD_IDENTITY_PROVIDER` secret is set and fall back to the
+old `GCP_SA_KEY` secret when it is not, so provisioning GCP and switching CI are not the same deploy.
+That fallback is temporary. Once one release has gone green on WIF:
+
+1. Delete the `GCP_SA_KEY` secret from this repo and from any organization-level scope.
+2. Delete the key itself, so a copy that leaked earlier is dead:
+ ```bash
+ gcloud iam service-accounts keys list --iam-account=<old-sa> --project=<project>
+ gcloud iam service-accounts keys delete <KEY_ID> --iam-account=<old-sa> --project=<project>
+ ```
+3. Delete the two `credentials_json` fallback steps from `release.yml`, and the `HAS_WIF` job
+ variables that only exist to switch between the two paths.
+
+A key that is still valid is still a liability even when nothing uses it.
+
+### `gcloud storage`, not `gsutil`
+
+`gsutil` is a standalone Python tool that only understands gcloud's user and service-account-*key*
+credentials. It cannot use the external-account credential federation issues, so every `gsutil` call
+in a WIF-authenticated job fails while the identical `gcloud storage` call succeeds. `upload-cli.sh`
+prefers `gcloud storage` and keeps `gsutil` only as a fallback for an old local SDK — which it warns
+about, because that path cannot work in CI.
+
+### The backend image build
+
+`production-be-build.yaml` delegates the build to `autonomous-ai/github-templates`. It calls
+**`docker-build-and-push-v2.yaml`** — a clone of the long-standing `docker-build-and-push.yaml` with
+federation, SHA-pinned actions, no caller value spliced into a shell, and deploy secrets as BuildKit
+mounts instead of build args.
+
+**A clone, because one of those changes is not backwards compatible.** Moving
+`SECRET_DEPLOY_GITHUB_TOKEN` and friends out of `build-args` silently empties them for any Dockerfile
+still reading the `ARG`, and that template is shared with `autonomous-code`, `ecm-web` and
+`autonomous-harness-backend`. The original is untouched and still theirs; they migrate when they
+choose, and v1 can be deleted once the last one has. Nothing this repo does can break their builds.
+
+It shares `release.yml`'s provider secret but takes its **own** service account, because the registry
+is in a different project from the release bucket — hence `GCP_SERVICE_ACCOUNT_GCR`. Grant that one
+image push in the registry's project:
+
+```bash
+# Artifact Registry (preferred)
+gcloud artifacts repositories add-iam-policy-binding <REPO> --location=<LOC> --project=autonomous-ecm \
+ --member="serviceAccount:gha-autonomous-harness-gcr@autonomous-ecm.iam.gserviceaccount.com" \
+ --role="roles/artifactregistry.writer"
+# legacy gcr.io, which is a GCS bucket underneath
+gcloud storage buckets add-iam-policy-binding gs://artifacts.autonomous-ecm.appspot.com \
+ --member="serviceAccount:gha-autonomous-harness-gcr@autonomous-ecm.iam.gserviceaccount.com" \
+ --role="roles/storage.legacyBucketWriter"
+```
+
+Two identities rather than one is not ceremony here: it means a compromised image build cannot touch
+the release bucket that every daemon on the fleet polls, and vice versa.
+
+**Merge order matters.** `docker-build-and-push-v2.yaml` has to exist on `github-templates`' `main`
+before this repo's caller points at it. The reverse order breaks every `v*_api` tag build until it is
+fixed — though only this repo's, which is the point of the clone.
+
+**`permissions:` belongs to the caller.** A called workflow can never hold more permission than the
+caller granted it, and `id-token` is in no default set — a template that declared `id-token: write`
+itself would hard-fail every caller that had not granted it. So v2 declares nothing and each caller
+states what it grants, which is why `production-be-build.yaml` carries the block.
+
+Two details worth carrying into anything else that calls these templates:
+
+- **The `secrets` context is unreadable from a step-level `if:`.** Both here and in v2, the "use WIF
+ or use the key" switch reads a job-level `env:` boolean (`HAS_WIF`, `HAS_SA_KEY`) computed from the
+ secret, never the secret itself. `actionlint` catches the direct form; nothing else does.
+- **A build arg a Dockerfile consumes is readable from `docker history`.** It is not a private channel
+ into a build. Use `RUN --mount=type=secret` for anything that is actually a credential.
+
+`@main` on the `uses:` line is still a moving branch reference, and that is the remaining hole: pin it
+to a commit SHA once v2 has landed.
-| Secret | Value |
-|---|---|
-| `GCP_SA_KEY` | the JSON key of the service account with object read/write on `s3-autonomous-upgrade-3` |
+### Action pinning
-GitHub does not share secrets between repositories. This is the *same credential* the desktop repo
-uses, but it has to exist here as well — either added to this repo or promoted to an organization
-secret scoped to both. Nothing else is needed: there is no signing identity and no notarization,
-because the artifacts are two JavaScript files whose integrity is the sha256 in the manifest.
+Third-party actions in `release.yml` are pinned to commit SHAs with the tag in a trailing comment. A
+tag is a mutable pointer: `softprops/action-gh-release@v2` runs in the job that holds
+`contents: write`, and whoever can move that tag can run their own code against that token.
+`actions/*` are GitHub-owned and stay on major tags.
-`ci.yml` needs no secret at all, which is what makes it safe to run on a fork's pull request.
+The same argument applies to
+`uses: autonomous-ai/github-templates/....yaml@main` in `production-be-build.yaml`, which is a
+*moving branch* reference on a workflow that receives GCP push credentials. Pin it to a SHA and bump
+it deliberately.
## Cutting a release
@@ -46,8 +234,8 @@ The manual fallback, for when `make release` itself can't be used:
# What is live right now — the next tag is one above this. Someone may have released by hand.
curl -fsS https://storage.googleapis.com/s3-autonomous-upgrade-3/harness/cli/metadata.json
-git tag -a vX.Y.Z --cleanup=verbatim -F notes.md
-git push origin vX.Y.Z
+git tag -a vX.Y.Z_cli --cleanup=verbatim -F notes.md
+git push origin vX.Y.Z_cli
```
**Check the manifest first, and never re-use a version.** Uploading over `harness/cli/<version>/`