Skip to content

πŸ™Œ OpenHands: AI-Driven Development

Fix stale Docker image tags & add version consistency CI + update-sdk skill (#13315)

Xingyao Wang83561701939f
diff --git a/.agents/skills/update-sdk/SKILL.md b/.agents/skills/update-sdk/SKILL.md
new file mode 100644
index 0000000000..a818db3641
--- /dev/null
+++ b/.agents/skills/update-sdk/SKILL.md
@@ -0,0 +1,123 @@
+---
+name: update-sdk
+description: This skill should be used when the user asks to "update SDK", "bump SDK version", "pin SDK to a commit", "test unreleased SDK", "update agent-server image", "bump the version", "prepare a release", "what files change for a release", or needs to know how SDK packages are managed in the OpenHands repository. For detailed reference material, see references/docker-image-locations.md and references/sdk-pinning-examples.md in this skill directory.
+---
+
+# Update SDK
+
+Bump SDK packages (`openhands-sdk`, `openhands-agent-server`, `openhands-tools`), pin them to unreleased commits for testing, and cut an OpenHands release.
+
+## Quick Summary β€” How Many Files Change?
+
+| Activity | Manual edits | Auto-regenerated | Total |
+|----------|:------------:|:----------------:|:-----:|
+| **SDK bump** (released PyPI version) | 2 | 3 | **5** |
+| **SDK pin** (unreleased git commit) | 3 | 3 | **6** |
+| **Release commit** (version bump) | 3 | 0 | **3** |
+
+The 3 auto-regenerated files are always: `poetry.lock`, `uv.lock`, `enterprise/poetry.lock`.
+
+## SDK Package Bump β€” 2 Files + 3 Lock Files
+
+Land as a separate PR before the release. Examples: `929dcc3` (SDK 1.11.5), `cd235cc` (SDK 1.11.4).
+
+| File | What to change |
+|------|----------------|
+| `pyproject.toml` | `openhands-sdk`, `openhands-agent-server`, `openhands-tools` in **two** sections: the `dependencies` array (PEP 508) **and** `[tool.poetry.dependencies]` |
+| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE` constant β€” set to `ghcr.io/openhands/agent-server:<version>-python` |
+
+Then regenerate lock files:
+```bash
+poetry lock && uv lock && cd enterprise && poetry lock && cd ..
+```
+
+## Docker Image Locations β€” All Hardcoded References
+
+For the complete inventory of every file containing a hardcoded Docker image tag or repository, see `references/docker-image-locations.md`. Key files that must stay in sync during an SDK bump:
+
+| File | Image reference | Updated during SDK bump? |
+|------|----------------|:------------------------:|
+| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:<tag>-python'` | βœ… Yes |
+| `docker-compose.yml` | `AGENT_SERVER_IMAGE_TAG` default | βœ… Should be |
+| `containers/dev/compose.yml` | `AGENT_SERVER_IMAGE_REPOSITORY` + `_TAG` defaults | βœ… Should be |
+
+> **CI enforcement:** `.github/workflows/check-version-consistency.yml` validates version consistency and compose file image references on every PR and push to main.
+
+### ⚠️ Docker Image Tag Gotcha (merge-commit SHA)
+
+The SDK CI in `software-agent-sdk` repo tags Docker images with the **GitHub Actions merge-commit SHA**, NOT the PR head-commit SHA. When pinning to an SDK PR branch:
+
+1. Check the SDK PR description for the actual image tag (look for the `AGENT_SERVER_IMAGES` section)
+2. Or query the CI logs: the "Consolidate Build Information" job prints `"short_sha": "<tag>"`
+3. The merge-commit SHA differs from the head SHA shown in the PR
+
+For released SDK versions, images use a version tag (e.g., `1.12.0-python`) β€” no merge-commit ambiguity.
+
+## Cutting a Release β€” 3 Files
+
+A release commit updates the version string across 3 files. Gold-standard examples: 1.3.0 (`d063c8c`), 1.4.0 (`495f48b`).
+
+| File | What to change |
+|------|----------------|
+| `pyproject.toml` | `version = "X.Y.Z"` under `[tool.poetry]` |
+| `frontend/package.json` | `"version": "X.Y.Z"` |
+| `frontend/package-lock.json` | `"version": "X.Y.Z"` in **two** places (root object and `packages[""]`) |
+
+> **Note:** `openhands/version.py` reads the version from `pyproject.toml` at runtime β€” no manual edit needed there.
+
+### Compose Files (2 files)
+
+Both compose files should use `ghcr.io/openhands/agent-server` with the current SDK version tag.
+
+| File | What to verify |
+|------|----------------|
+| `docker-compose.yml` | `AGENT_SERVER_IMAGE_REPOSITORY` defaults to agent-server, `AGENT_SERVER_IMAGE_TAG` is current |
+| `containers/dev/compose.yml` | Same β€” must use agent-server, not runtime |
+
+### Release Workflow
+
+#### Step 1: Verify the SDK bump has landed
+
+```bash
+grep -n "openhands-sdk\|openhands-agent-server\|openhands-tools" pyproject.toml
+grep -n "AGENT_SERVER_IMAGE" openhands/app_server/sandbox/sandbox_spec_service.py
+grep "AGENT_SERVER_IMAGE_TAG" docker-compose.yml containers/dev/compose.yml
+```
+
+#### Step 2: Bump version numbers
+
+```bash
+# Edit pyproject.toml, frontend/package.json, frontend/package-lock.json
+git add pyproject.toml frontend/package.json frontend/package-lock.json
+git commit -m "Release X.Y.Z"
+git tag X.Y.Z
+```
+
+Create a `saas-rel-X.Y.Z` branch from the tagged commit for the SaaS deployment pipeline.
+
+#### Step 3: CI builds Docker images automatically
+
+The `ghcr-build.yml` workflow triggers on tag pushes and produces:
+- `ghcr.io/openhands/openhands:X.Y.Z`, `X.Y`, `X`, `latest`
+- `ghcr.io/openhands/runtime:X.Y.Z-nikolaik`, `X.Y-nikolaik`
+
+The tagging logic lives in `containers/build.sh` β€” when `GITHUB_REF_NAME` matches a semver pattern (`^[0-9]+\.[0-9]+\.[0-9]+$`), it auto-generates major, major.minor, and `latest` tags.
+
+## Development: Pin SDK to an Unreleased Commit
+
+For detailed examples of all pinning formats (commit, branch, uv-only), see `references/sdk-pinning-examples.md`.
+
+### Files to change (3 manual + 3 lock files)
+
+| File | What to change |
+|------|----------------|
+| `pyproject.toml` | Pin all 3 SDK packages in **both** `dependencies` and `[tool.poetry.dependencies]` |
+| `openhands/app_server/sandbox/sandbox_spec_service.py` | `AGENT_SERVER_IMAGE` β€” use the merge-commit SHA tag, NOT the head-commit SHA |
+| `docker-compose.yml` | `AGENT_SERVER_IMAGE_TAG` default (for local development) |
+| `poetry.lock` | Auto-regenerated via `poetry lock` |
+| `uv.lock` | Auto-regenerated via `uv lock` |
+| `enterprise/poetry.lock` | Auto-regenerated via `cd enterprise && poetry lock` |
+
+### CI guard
+
+The `check-package-versions.yml` workflow blocks merging to `main` if `[tool.poetry.dependencies]` contains any `rev` fields. This ensures unreleased SDK pins do not accidentally ship in a release.
diff --git a/.agents/skills/update-sdk/references/docker-image-locations.md b/.agents/skills/update-sdk/references/docker-image-locations.md
new file mode 100644
index 0000000000..c7a23bf43c
--- /dev/null
+++ b/.agents/skills/update-sdk/references/docker-image-locations.md
@@ -0,0 +1,84 @@
+# Docker Image Locations β€” Complete Inventory
+
+Every file in the OpenHands repository containing a hardcoded Docker image tag, repository, or version-pinned image reference. Organized by update cadence.
+
+## Updated During SDK Bump (must change)
+
+These files contain image tags that **must** be updated whenever the SDK version or pinned commit changes.
+
+### `openhands/app_server/sandbox/sandbox_spec_service.py`
+- **Line:** `AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:<tag>-python'`
+- **Format:** `<sdk-version>-python` for releases (e.g., `1.12.0-python`), `<7-char-commit-hash>-python` for dev pins
+- **Source of truth** for which agent-server image the app server pulls at runtime
+- **⚠️ Gotcha:** When pinning to an SDK PR, the image tag is the **merge-commit SHA** from GitHub Actions, not the PR head-commit SHA. Check the SDK PR description or CI logs for the correct tag.
+
+### `docker-compose.yml`
+- **Lines:**
+ ```yaml
+ - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}
+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-<tag>-python}
+ ```
+- Used by `docker compose up` for local development
+
+### `containers/dev/compose.yml`
+- **Lines:**
+ ```yaml
+ - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}
+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-<tag>-python}
+ ```
+- Used by the dev container setup
+- **Known issue:** On main as of 1.4.0, this file still points to `ghcr.io/openhands/runtime` instead of `agent-server`, and the tag is `1.2-nikolaik` (stale from the V0 era). The `check-version-consistency.yml` CI workflow catches this.
+
+## Updated During Release Commit (version string only)
+
+### `pyproject.toml`
+- **Line:** `version = "X.Y.Z"` under `[tool.poetry]`
+- The Python version is derived from this at runtime via `openhands/version.py`
+
+### `frontend/package.json`
+- **Line:** `"version": "X.Y.Z"`
+
+### `frontend/package-lock.json`
+- **Two places:** root `"version": "X.Y.Z"` and `packages[""].version`
+
+## Dynamic References (auto-derived, no manual update)
+
+### `openhands/version.py`
+- Reads version from `pyproject.toml` at runtime β†’ `openhands.__version__`
+
+### `openhands/resolver/issue_resolver.py`
+- Builds `ghcr.io/openhands/runtime:{openhands.__version__}-nikolaik` dynamically
+
+### `openhands/runtime/utils/runtime_build.py`
+- Base repo URL `ghcr.io/openhands/runtime` is a constant; version comes from elsewhere
+
+### `.github/scripts/update_pr_description.sh`
+- Uses `${SHORT_SHA}` variable at CI runtime, not hardcoded
+
+### `enterprise/Dockerfile`
+- `ARG BASE="ghcr.io/openhands/openhands"` β€” base image, version supplied at build time
+
+## V0 Legacy Files (separate update cadence)
+
+These reference the V0 runtime image (`ghcr.io/openhands/runtime:X.Y-nikolaik`) for local Docker/Kubernetes paths. They are **not** updated as part of a V1 release but may be updated independently.
+
+### `Development.md`
+- `export SANDBOX_RUNTIME_CONTAINER_IMAGE=ghcr.io/openhands/runtime:X.Y-nikolaik`
+
+### `openhands/runtime/impl/kubernetes/README.md`
+- `runtime_container_image = "docker.openhands.dev/openhands/runtime:X.Y-nikolaik"`
+
+### `enterprise/enterprise_local/README.md`
+- Uses `ghcr.io/openhands/runtime:main-nikolaik` (points to `main`, not versioned)
+
+### `third_party/runtime/impl/daytona/README.md`
+- Uses `${OPENHANDS_VERSION}` variable, not hardcoded
+
+## Image Registries
+
+| Registry | Usage |
+|----------|-------|
+| `ghcr.io/openhands/agent-server` | V1 agent-server (sandbox) β€” built by SDK repo CI |
+| `ghcr.io/openhands/openhands` | Main app image β€” built by `ghcr-build.yml` |
+| `ghcr.io/openhands/runtime` | V0 runtime sandbox β€” built by `ghcr-build.yml` |
+| `docker.openhands.dev/openhands/*` | Mirror/CDN for the above images |
diff --git a/.agents/skills/update-sdk/references/sdk-pinning-examples.md b/.agents/skills/update-sdk/references/sdk-pinning-examples.md
new file mode 100644
index 0000000000..8cb036a72d
--- /dev/null
+++ b/.agents/skills/update-sdk/references/sdk-pinning-examples.md
@@ -0,0 +1,103 @@
+# SDK Pinning Examples
+
+Examples from real commits showing how to pin SDK packages to unreleased commits, branches, or released versions.
+
+## Pin to a Specific Commit
+
+Example from commit `169fb76` (pinning all 3 packages to SDK commit `100e9af`):
+
+### `dependencies` array (PEP 508 format)
+
+```toml
+"openhands-agent-server @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-agent-server",
+"openhands-sdk @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-sdk",
+"openhands-tools @ git+https://github.com/OpenHands/software-agent-sdk.git@100e9af#subdirectory=openhands-tools",
+```
+
+### `[tool.poetry.dependencies]` (Poetry format)
+
+```toml
+openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-sdk" }
+openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-agent-server" }
+openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", rev = "100e9af", subdirectory = "openhands-tools" }
+```
+
+### `openhands/app_server/sandbox/sandbox_spec_service.py`
+
+```python
+AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:<merge-commit-sha>-python'
+```
+
+**⚠️ Important:** The image tag is the **merge-commit SHA** from the SDK CI, not the commit hash used in `pyproject.toml`. Look up the correct tag from the SDK PR description or CI logs.
+
+## Pin to a Branch
+
+Example from commit `430ee1c` (pinning to branch `openhands/issue-2228-sdk-settings-schema`):
+
+### `[tool.poetry.dependencies]`
+
+```toml
+openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-sdk" }
+openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-agent-server" }
+openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", branch = "openhands/issue-2228-sdk-settings-schema", subdirectory = "openhands-tools" }
+```
+
+## Using `[tool.uv.sources]` Override
+
+When only `uv` needs the override (keep PyPI versions in the main arrays), add a `[tool.uv.sources]` section. Example from commit `1daca49`:
+
+```toml
+[tool.uv.sources]
+openhands-sdk = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-sdk", rev = "4170cca" }
+openhands-agent-server = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-agent-server", rev = "4170cca" }
+openhands-tools = { git = "https://github.com/OpenHands/software-agent-sdk.git", subdirectory = "openhands-tools", rev = "4170cca" }
+```
+
+## Released PyPI Version (standard release)
+
+Example from commit `929dcc3` (SDK 1.11.5):
+
+### `dependencies` array
+
+```toml
+"openhands-agent-server==1.11.5",
+"openhands-sdk==1.11.5",
+"openhands-tools==1.11.5",
+```
+
+### `[tool.poetry.dependencies]`
+
+```toml
+openhands-sdk = "1.11.5"
+openhands-agent-server = "1.11.5"
+openhands-tools = "1.11.5"
+```
+
+### `openhands/app_server/sandbox/sandbox_spec_service.py`
+
+For released versions, the image tag uses the version number:
+
+```python
+AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:1.11.5-python'
+```
+
+However, **some releases use a commit-hash tag** even for the released version. Check which tag format exists on GHCR. Example from `929dcc3`:
+
+```python
+AGENT_SERVER_IMAGE = 'ghcr.io/openhands/agent-server:010e847-python'
+```
+
+## Regenerate Lock Files
+
+After any change to `pyproject.toml`, always regenerate:
+
+```bash
+poetry lock
+uv lock
+cd enterprise && poetry lock && cd ..
+```
+
+## CI Guards
+
+- **`check-package-versions.yml`**: Blocks merge to `main` if `[tool.poetry.dependencies]` contains `rev` fields (prevents shipping unreleased SDK pins)
+- **`check-version-consistency.yml`**: Validates version strings match across `pyproject.toml`, `package.json`, `package-lock.json`, and verifies compose files use `agent-server` images
diff --git a/.github/workflows/check-version-consistency.yml b/.github/workflows/check-version-consistency.yml
new file mode 100644
index 0000000000..7df1a1764a
--- /dev/null
+++ b/.github/workflows/check-version-consistency.yml
@@ -0,0 +1,122 @@
+name: Check Version Consistency
+
+on:
+ push:
+ branches: [main]
+ pull_request:
+ workflow_dispatch:
+
+jobs:
+ check-version-consistency:
+ runs-on: ubuntu-latest
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v4
+
+ - name: Set up Python
+ uses: actions/setup-python@v6
+ with:
+ python-version: "3.12"
+
+ - name: Check version and Docker image tag consistency
+ run: |
+ python - <<'PY'
+ import json
+ import re
+ import sys
+ import tomllib
+
+ errors = []
+ warnings = []
+
+ # ── 1. Extract the canonical version from pyproject.toml ──────────
+ with open("pyproject.toml", "rb") as f:
+ pyproject = tomllib.load(f)
+ version = pyproject["tool"]["poetry"]["version"]
+ major_minor = ".".join(version.split(".")[:2])
+ print(f"πŸ“¦ pyproject.toml version: {version} (major.minor: {major_minor})")
+
+ # ── 2. Check frontend/package.json ────────────────────────────────
+ with open("frontend/package.json") as f:
+ pkg = json.load(f)
+ if pkg["version"] != version:
+ errors.append(
+ f"frontend/package.json version is '{pkg['version']}', expected '{version}'"
+ )
+ else:
+ print(f" βœ” frontend/package.json: {pkg['version']}")
+
+ # ── 3. Check frontend/package-lock.json (2 places) ───────────────
+ with open("frontend/package-lock.json") as f:
+ lock = json.load(f)
+ for key, val in [
+ ("root.version", lock.get("version")),
+ ('packages[""].version', lock.get("packages", {}).get("", {}).get("version")),
+ ]:
+ if val != version:
+ errors.append(
+ f"frontend/package-lock.json {key} is '{val}', expected '{version}'"
+ )
+ else:
+ print(f" βœ” frontend/package-lock.json {key}: {val}")
+
+ # ── 4. Check compose files use agent-server images ─────────────────
+ # Both compose files should use ghcr.io/.../agent-server (not runtime).
+ # Agent-server tags use SDK version (e.g. "1.12.0-python") or commit
+ # hashes (e.g. "31536c8-python") β€” both are acceptable.
+ repo_pattern = re.compile(r"AGENT_SERVER_IMAGE_REPOSITORY[^}]*:-([^}]+)")
+ tag_pattern = re.compile(r"AGENT_SERVER_IMAGE_TAG:-([^}]+)")
+
+ for filepath in ["docker-compose.yml", "containers/dev/compose.yml"]:
+ try:
+ with open(filepath) as f:
+ content = f.read()
+ except FileNotFoundError:
+ warnings.append(f"{filepath}: file not found")
+ continue
+
+ repos = repo_pattern.findall(content)
+ tags = tag_pattern.findall(content)
+
+ if not repos:
+ warnings.append(f"{filepath}: no AGENT_SERVER_IMAGE_REPOSITORY default found")
+ else:
+ repo = repos[0]
+ if "agent-server" not in repo:
+ errors.append(
+ f"{filepath}: AGENT_SERVER_IMAGE_REPOSITORY defaults to '{repo}', "
+ f"expected an agent-server image (not runtime)"
+ )
+ else:
+ print(f" βœ” {filepath} image repository: {repo}")
+
+ if not tags:
+ warnings.append(f"{filepath}: no AGENT_SERVER_IMAGE_TAG default found")
+ else:
+ tag = tags[0]
+ if not tag:
+ errors.append(f"{filepath}: AGENT_SERVER_IMAGE_TAG default is empty")
+ else:
+ print(f" βœ” {filepath} image tag: {tag}")
+
+ # ── 5. Report ─────────────────────────────────────────────────────
+ print()
+ if warnings:
+ print("⚠ Warnings:")
+ for w in warnings:
+ print(f" {w}")
+ print()
+
+ if errors:
+ print("❌ FAILED: Version inconsistencies found:\n")
+ for e in errors:
+ print(f" βœ– {e}")
+ print(
+ "\nAll version numbers and Docker image tags must be consistent."
+ "\nSee .agents/skills/update-sdk/SKILL.md for the full checklist."
+ )
+ sys.exit(1)
+ else:
+ print("βœ… All version numbers and Docker image tags are consistent.")
+ PY
diff --git a/containers/dev/compose.yml b/containers/dev/compose.yml
index 1adb7e6b3f..66b72e8a20 100644
--- a/containers/dev/compose.yml
+++ b/containers/dev/compose.yml
@@ -12,8 +12,8 @@ services:
- SANDBOX_API_HOSTNAME=host.docker.internal
- DOCKER_HOST_ADDR=host.docker.internal
#
- - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/runtime}
- - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.2-nikolaik}
+ - AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}
+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.12.0-python}
- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234}
- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
ports:
diff --git a/docker-compose.yml b/docker-compose.yml
index 3fec30e057..089ed69c9a 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -8,7 +8,7 @@ services:
container_name: openhands-app-${DATE:-}
environment:
- AGENT_SERVER_IMAGE_REPOSITORY=${AGENT_SERVER_IMAGE_REPOSITORY:-ghcr.io/openhands/agent-server}
- - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-31536c8-python}
+ - AGENT_SERVER_IMAGE_TAG=${AGENT_SERVER_IMAGE_TAG:-1.12.0-python}
#- SANDBOX_USER_ID=${SANDBOX_USER_ID:-1234} # enable this only if you want a specific non-root sandbox user but you will have to manually adjust permissions of ~/.openhands for this user
- WORKSPACE_MOUNT_PATH=${WORKSPACE_BASE:-$PWD/workspace}
ports: