π OpenHands: AI-Driven Development
Fix stale Docker image tags & add version consistency CI + update-sdk skill (#13315)
Xingyao Wang83561701939f
diff --git a/.agents/skills/update-sdk/SKILL.md b/.agents/skills/update-sdk/SKILL.mdnew file mode 100644index 0000000000..a818db3641--- /dev/null+++ b/.agents/skills/update-sdk/SKILL.md@@ -0,0 +1,123 @@+---+name: update-sdk+description: This skill should be used when the user asks to "update SDK", "bump SDK version", "pin SDK to a commit", "test unreleased SDK", "update agent-server image", "bump the version", "prepare a release", "what files change for a release", or needs to know how SDK packages are managed in the OpenHands repository. For detailed reference material, see references/docker-image-locations.md and references/sdk-pinning-examples.md in this skill directory.+---++# Update SDK++Bump SDK packages (`openhands-sdk`, `openhands-agent-server`, `openhands-tools`), pin them to unreleased commits for testing, and cut an OpenHands release.++## Quick Summary β How Many Files Change?++| Activity | Manual edits | Auto-regenerated | Total |+|----------|:------------:|:----------------:|:-----:|+| **SDK bump** (released PyPI version) | 2 | 3 | **5** |+| **SDK pin** (unreleased git commit) | 3 | 3 | **6** |+| **Release commit** (version bump) | 3 | 0 | **3** |++The 3 auto-regenerated files are always: `poetry.lock`, `uv.lock`, `enterprise/poetry.lock`.++## SDK Package Bump β 2 Files + 3 Lock Files++Land as a separate PR before the release. Examples: `929dcc3` (SDK 1.11.5), `cd235cc` (SDK 1.11.4).++| File | What to change |+|------|----------------|+| `pyproject.toml` | `openhands-sdk`, `openhands-agent-server`, `openhands-tools` in **two** sections: the `dependencies` array (PEP 508) **and** `[tool.poetry.dependencies]` |+| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE` constant β set to `ghcr.io/openhands/agent-server:<version>-python` |++Then regenerate lock files:+```bash+poetry lock && uv lock && cd enterprise && poetry lock && cd ..+```++## Docker Image Locations β All Hardcoded References++For the complete inventory of every file containing a hardcoded Docker image tag or repository, see `references/docker-image-locations.md`. Key files that must stay in sync during an SDK bump:++| File | Image reference | Updated during SDK bump? |+|------|----------------|:------------------------:|+| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:<tag>-python'` | β Yes |+| `docker-compose.yml` | `AGENT_SERVER_IMAGE_TAG` default | β Should be |+| `containers/dev/compose.yml` | `AGENT_SERVER_IMAGE_REPOSITORY` + `_TAG` defaults | β Should be |++> **CI enforcement:** `.github/workflows/check-version-consistency.yml` validates version consistency and compose file image references on every PR and push to main.++### β οΈ Docker Image Tag Gotcha (merge-commit SHA)++The SDK CI in `software-agent-sdk` repo tags Docker images with the **GitHub Actions merge-commit SHA**, NOT the PR head-commit SHA. When pinning to an SDK PR branch:++1. Check the SDK PR description for the actual image tag (look for the `AGENT_SERVER_IMAGES` section)+2. Or query the CI logs: the "Consolidate Build Information" job prints `"short_sha": "<tag>"`+3. The merge-commit SHA differs from the head SHA shown in the PR++For released SDK versions, images use a version tag (e.g., `1.12.0-python`) β no merge-commit ambiguity.++## Cutting a Release β 3 Files++A release commit updates the version string across 3 files. Gold-standard examples: 1.3.0 (`d063c8c`), 1.4.0 (`495f48b`).++| File | What to change |+|------|----------------|+| `pyproject.toml` | `version = "X.Y.Z"` under `[tool.poetry]` |+| `frontend/package.json` | `"version": "X.Y.Z"` |+| `frontend/package-lock.json` | `"version": "X.Y.Z"` in **two** places (root object and `packages[""]`) |++> **Note:** `openhands/version.py` reads the version from `pyproject.toml` at runtime β no manual edit needed there.++### Compose Files (2 files)++Both compose files should use `ghcr.io/openhands/agent-server` with the current SDK version tag.++| File | What to verify |+|------|----------------|+| `docker-compose.yml` | `AGENT_SERVER_IMAGE_REPOSITORY` defaults to agent-server, `AGENT_SERVER_IMAGE_TAG` is current |+| `containers/dev/compose.yml` | Same β must use agent-server, not runtime |++### Release Workflow++#### Step 1: Verify the SDK bump has landed++```bash+grep -n "openhands-sdk\|openhands-agent-server\|openhands-tools" pyproject.toml+grep -n "AGENT_SERVER_IMAGE" openhands/app_server/sandbox/sandbox_spec_service.py+grep "AGENT_SERVER_IMAGE_TAG" docker-compose.yml containers/dev/compose.yml+```++#### Step 2: Bump version numbers++```bash+# Edit pyproject.toml, frontend/package.json, frontend/package-lock.json+git add pyproject.toml frontend/package.json frontend/package-lock.json+git commit -m "Release X.Y.Z"+git tag X.Y.Z+```++Create a `saas-rel-X.Y.Z` branch from the tagged commit for the SaaS deployment pipeline.++#### Step 3: CI builds Docker images automatically++The `ghcr-build.yml` workflow triggers on tag pushes and produces:+- `ghcr.io/openhands/openhands:X.Y.Z`, `X.Y`, `X`, `latest`+- `ghcr.io/openhands/runtime:X.Y.Z-nikolaik`, `X.Y-nikolaik`++The tagging logic lives in `containers/build.sh` β when `GITHUB_REF_NAME` matches a semver pattern (`^[0-9]+\.[0-9]+\.[0-9]+$`), it auto-generates major, major.minor, and `latest` tags.++## Development: Pin SDK to an Unreleased Commit++For detailed examples of all pinning formats (commit, branch, uv-only), see `references/sdk-pinning-examples.md`.++### Files to change (3 manual + 3 lock files)++| File | What to change |+|------|----------------|+| `pyproject.toml` | Pin all 3 SDK packages in **both** `dependencies` and `[tool.poetry.dependencies]` |+| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE` β use the merge-commit SHA tag, NOT the head-commit SHA |+| `docker-compose.yml` | `AGENT_SERVER_IMAGE_TAG` default (for local development) |+| `poetry.lock` | Auto-regenerated via `poetry lock` |+| `uv.lock` | Auto-regenerated via `uv lock` |+| `enterprise/poetry.lock` | Auto-regenerated via `cd enterprise && poetry lock` |++### CI guard++The `check-package-versions.yml` workflow blocks merging to `main` if `[tool.poetry.dependencies]` contains any `rev` fields. This ensures unreleased SDK pins do not accidentally ship in a release.diff --git a/.agents/skills/update-sdk/references/docker-image-locations.md b/.agents/skills/update-sdk/references/docker-image-locations.mdnew file mode 100644index 0000000000..c7a23bf43c--- /dev/null+++ b/.agents/skills/update-sdk/references/docker-image-locations.md@@ -0,0 +1,84 @@+# Docker Image Locations β Complete Inventory++Every file in the OpenHands repository containing a hardcoded Docker image tag, repository, or version-pinned image reference. Organized by update cadence.++## Updated During SDK Bump (must change)++These files contain image tags that **must** be updated whenever the SDK version or pinned commit changes.++### `openhands/app_server/sandbox/sandbox_spec_service.py`+- **Line:** `AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:<tag>-python'`+- **Format:** `<sdk-version>-python` for releases (e.g., `1.12.0-python`), `<7-char-commit-hash>-python` for dev pins+- **Source of truth** for which agent-server image the app server pulls at runtime+- **β οΈ Gotcha:** When pinning to an SDK PR, the image tag is the **merge-commit SHA** from GitHub Actions, not the PR head-commit SHA. Check the SDK PR description or CI logs for the correct tag.++### `docker-compose.yml`+- **Lines:**+ ```yaml+ - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-<tag>-python}+ ```+- Used by `docker compose up` for local development++### `containers/dev/compose.yml`+- **Lines:**+ ```yaml+ - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-<tag>-python}+ ```+- Used by the dev container setup+- **Known issue:** On main as of 1.4.0, this file still points to `ghcr.io/openhands/runtime` instead of `agent-server`, and the tag is `1.2-nikolaik` (stale from the V0 era). The `check-version-consistency.yml` CI workflow catches this.++## Updated During Release Commit (version string only)++### `pyproject.toml`+- **Line:** `version = "X.Y.Z"` under `[tool.poetry]`+- The Python version is derived from this at runtime via `openhands/version.py`++### `frontend/package.json`+- **Line:** `"version": "X.Y.Z"`++### `frontend/package-lock.json`+- **Two places:** root `"version": "X.Y.Z"` and `packages[""].version`++## Dynamic References (auto-derived, no manual update)++### `openhands/version.py`+- Reads version from `pyproject.toml` at runtime β `openhands.__version__`++### `openhands/resolver/issue_resolver.py`+- Builds `ghcr.io/openhands/runtime:{openhands.__version__}-nikolaik` dynamically++### `openhands/runtime/utils/runtime_build.py`+- Base repo URL `ghcr.io/openhands/runtime` is a constant; version comes from elsewhere++### `.github/scripts/update_pr_description.sh`+- Uses `${SHORT_SHA}` variable at CI runtime, not hardcoded++### `enterprise/Dockerfile`+- `ARG BASE="ghcr.io/openhands/openhands"` β base image, version supplied at build time++## V0 Legacy Files (separate update cadence)++These reference the V0 runtime image (`ghcr.io/openhands/runtime:X.Y-nikolaik`) for local Docker/Kubernetes paths. They are **not** updated as part of a V1 release but may be updated independently.++### `Development.md`+- `export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/openhands/runtime:X.Y-nikolaik`++### `openhands/runtime/impl/kubernetes/README.md`+- `runtime_container_image = "docker.openhands.dev/openhands/runtime:X.Y-nikolaik"`++### `enterprise/enterprise_local/README.md`+- Uses `ghcr.io/openhands/runtime:main-nikolaik` (points to `main`, not versioned)++### `third_party/runtime/impl/daytona/README.md`+- Uses `${OPENHANDS_VERSION}` variable, not hardcoded++## Image Registries++| Registry | Usage |+|----------|-------|+| `ghcr.io/openhands/agent-server` | V1 agent-server (sandbox) β built by SDK repo CI |+| `ghcr.io/openhands/openhands` | Main app image β built by `ghcr-build.yml` |+| `ghcr.io/openhands/runtime` | V0 runtime sandbox β built by `ghcr-build.yml` |+| `docker.openhands.dev/openhands/*` | Mirror/CDN for the above images |diff --git a/.agents/skills/update-sdk/references/sdk-pinning-examples.md b/.agents/skills/update-sdk/references/sdk-pinning-examples.mdnew file mode 100644index 0000000000..8cb036a72d--- /dev/null+++ b/.agents/skills/update-sdk/references/sdk-pinning-examples.md@@ -0,0 +1,103 @@+# SDK Pinning Examples++Examples from real commits showing how to pin SDK packages to unreleased commits, branches, or released versions.++## Pin to a Specific Commit++Example from commit `169fb76` (pinning all 3 packages to SDK commit `100e9af`):++### `dependencies` array (PEP 508 format)++```toml+"openhands-agent-server @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-agent-server",+"openhands-sdk @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-sdk",+"openhands-tools @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-tools",+```++### `[tool.poetry.dependencies]` (Poetry format)++```toml+openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-sdk" }+openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-agent-server" }+openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-tools" }+```++### `openhands/app_server/sandbox/sandbox_spec_service.py`++```python+AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:<merge-commit-sha>-python'+```++**β οΈ Important:** The image tag is the **merge-commit SHA** from the SDK CI, not the commit hash used in `pyproject.toml`. Look up the correct tag from the SDK PR description or CI logs.++## Pin to a Branch++Example from commit `430ee1c` (pinning to branch `openhands/issue-2228-sdk-settings-schema`):++### `[tool.poetry.dependencies]`++```toml+openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-sdk" }+openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-agent-server" }+openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-tools" }+```++## Using `[tool.uv.sources]` Override++When only `uv` needs the override (keep PyPI versions in the main arrays), add a `[tool.uv.sources]` section. Example from commit `1daca49`:++```toml+[tool.uv.sources]+openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-sdk", rev = "4170cca" }+openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-agent-server", rev = "4170cca" }+openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-tools", rev = "4170cca" }+```++## Released PyPI Version (standard release)++Example from commit `929dcc3` (SDK 1.11.5):++### `dependencies` array++```toml+"openhands-agent-server==1.11.5",+"openhands-sdk==1.11.5",+"openhands-tools==1.11.5",+```++### `[tool.poetry.dependencies]`++```toml+openhands-sdk = "1.11.5"+openhands-agent-server = "1.11.5"+openhands-tools = "1.11.5"+```++### `openhands/app_server/sandbox/sandbox_spec_service.py`++For released versions, the image tag uses the version number:++```python+AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:1.11.5-python'+```++However, **some releases use a commit-hash tag** even for the released version. Check which tag format exists on GHCR. Example from `929dcc3`:++```python+AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:010e847-python'+```++## Regenerate Lock Files++After any change to `pyproject.toml`, always regenerate:++```bash+poetry lock+uv lock+cd enterprise && poetry lock && cd ..+```++## CI Guards++- **`check-package-versions.yml`**: Blocks merge to `main` if `[tool.poetry.dependencies]` contains `rev` fields (prevents shipping unreleased SDK pins)+- **`check-version-consistency.yml`**: Validates version strings match across `pyproject.toml`, `package.json`, `package-lock.json`, and verifies compose files use `agent-server` imagesdiff --git a/.github/workflows/check-version-consistency.yml b/.github/workflows/check-version-consistency.ymlnew file mode 100644index 0000000000..7df1a1764a--- /dev/null+++ b/.github/workflows/check-version-consistency.yml@@ -0,0 +1,122 @@+name: Check Version Consistency++on:+ push:+ branches: [main]+ pull_request:+ workflow_dispatch:++jobs:+ check-version-consistency:+ runs-on: ubuntu-latest++ steps:+ - name: Checkout repository+ uses: actions/checkout@v4++ - name: Set up Python+ uses: actions/setup-python@v6+ with:+ python-version: "3.12"++ - name: Check version and Docker image tag consistency+ run: |+ python - <<'PY'+ import json+ import re+ import sys+ import tomllib++ errors = []+ warnings = []++ # ββ 1. Extract the canonical version from pyproject.toml ββββββββββ+ with open("pyproject.toml", "rb") as f:+ pyproject = tomllib.load(f)+ version = pyproject["tool"]["poetry"]["version"]+ major_minor = ".".join(version.split(".")[:2])+ print(f"π¦ pyproject.toml version: {version} (major.minor: {major_minor})")++ # ββ 2. Check frontend/package.json ββββββββββββββββββββββββββββββββ+ with open("frontend/package.json") as f:+ pkg = json.load(f)+ if pkg["version"] != version:+ errors.append(+ f"frontend/package.json version is '{pkg['version']}', expected '{version}'"+ )+ else:+ print(f" β frontend/package.json: {pkg['version']}")++ # ββ 3. Check frontend/package-lock.json (2 places) βββββββββββββββ+ with open("frontend/package-lock.json") as f:+ lock = json.load(f)+ for key, val in [+ ("root.version", lock.get("version")),+ ('packages[""].version', lock.get("packages", {}).get("", {}).get("version")),+ ]:+ if val != version:+ errors.append(+ f"frontend/package-lock.json {key} is '{val}', expected '{version}'"+ )+ else:+ print(f" β frontend/package-lock.json {key}: {val}")++ # ββ 4. Check compose files use agent-server images βββββββββββββββββ+ # Both compose files should use ghcr.io/.../agent-server (not runtime).+ # Agent-server tags use SDK version (e.g. "1.12.0-python") or commit+ # hashes (e.g. "31536c8-python") β both are acceptable.+ repo_pattern = re.compile(r"AGENT_SERVER_IMAGE_REPOSITORY[^}]*:-([^}]+)")+ tag_pattern = re.compile(r"AGENT_SERVER_IMAGE_TAG:-([^}]+)")++ for filepath in ["docker-compose.yml", "containers/dev/compose.yml"]:+ try:+ with open(filepath) as f:+ content = f.read()+ except FileNotFoundError:+ warnings.append(f"{filepath}: file not found")+ continue++ repos = repo_pattern.findall(content)+ tags = tag_pattern.findall(content)++ if not repos:+ warnings.append(f"{filepath}: no AGENT_SERVER_IMAGE_REPOSITORY default found")+ else:+ repo = repos[0]+ if "agent-server" not in repo:+ errors.append(+ f"{filepath}: AGENT_SERVER_IMAGE_REPOSITORY defaults to '{repo}', "+ f"expected an agent-server image (not runtime)"+ )+ else:+ print(f" β {filepath} image repository: {repo}")++ if not tags:+ warnings.append(f"{filepath}: no AGENT_SERVER_IMAGE_TAG default found")+ else:+ tag = tags[0]+ if not tag:+ errors.append(f"{filepath}: AGENT_SERVER_IMAGE_TAG default is empty")+ else:+ print(f" β {filepath} image tag: {tag}")++ # ββ 5. Report βββββββββββββββββββββββββββββββββββββββββββββββββββββ+ print()+ if warnings:+ print("β Warnings:")+ for w in warnings:+ print(f" {w}")+ print()++ if errors:+ print("β FAILED: Version inconsistencies found:\n")+ for e in errors:+ print(f" β {e}")+ print(+ "\nAll version numbers and Docker image tags must be consistent."+ "\nSee .agents/skills/update-sdk/SKILL.md for the full checklist."+ )+ sys.exit(1)+ else:+ print("β All version numbers and Docker image tags are consistent.")+ PYdiff --git a/containers/dev/compose.yml b/containers/dev/compose.ymlindex 1adb7e6b3f..66b72e8a20 100644--- a/containers/dev/compose.yml+++ b/containers/dev/compose.yml@@ -12,8 +12,8 @@ services:- SANDBOX_API_HOSTNAME=host.docker.internal- DOCKER_HOST_ADDR=host.docker.internal#- - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/runtime}- - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.2-nikolaik}+ - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.12.0-python}- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234}- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}ports:diff --git a/docker-compose.yml b/docker-compose.ymlindex 3fec30e057..089ed69c9a 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -8,7 +8,7 @@ services:container_name: openhands-app-${DATE:-}environment:- AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}- - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-31536c8-python}+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.12.0-python}#- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of ~/.openhands for this user- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}ports: