Skip to content

The open-source app everyone uses to manage agents at work

ci: cache compiled Docker Rust dependencies separately from source (#13329)

Devin Foley7435b2ee9cc6
diff --git a/.github/workflows/docker-runner-check.yml b/.github/workflows/docker-runner-check.yml
index e9705c3b5e..7c3b7820b5 100644
--- a/.github/workflows/docker-runner-check.yml
+++ b/.github/workflows/docker-runner-check.yml
@@ -6,6 +6,7 @@ on:
- .github/workflows/docker-runner-check.yml
- Dockerfile
- .dockerignore
+ - scripts/check-docker-runner-cache.sh
- packages/paperclip-runner/rust-toolchain.toml
- packages/paperclip-runner/runner/**
- packages/paperclip-runner/protocol/**
@@ -20,7 +21,7 @@ jobs:
runner:
name: Compile isolated native Runner
runs-on: ubuntu-latest
- timeout-minutes: 15
+ timeout-minutes: 20
permissions:
contents: read
steps:
@@ -30,8 +31,8 @@ jobs:
persist-credentials: false
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@37fe631027851001ddb9b187196cc803df7f5f0e # v4
- # Compile the real target with the real .dockerignore. This catches new
- # Cargo or embedded protocol inputs that the isolated COPY set omits.
- # No registry credentials, cache imports/exports, or image publication.
- - name: Compile the Runner from its isolated Docker context
- run: docker buildx build --target runner-build --progress plain .
+ # Compile the real target, then change source in a disposable context.
+ # Require dependency-layer reuse and changed metadata from the real binary.
+ # No registry credentials, external cache, or image publication.
+ - name: Verify native build and dependency cache reuse
+ run: bash scripts/check-docker-runner-cache.sh
diff --git a/Dockerfile b/Dockerfile
index b349251238..a3d046c96e 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -88,7 +88,27 @@ RUN set -eux; \
COPY packages/paperclip-runner/rust-toolchain.toml /tmp/runner-toolchain/rust-toolchain.toml
RUN cd /tmp/runner-toolchain && rustup show
-FROM rust-toolchain AS runner-build
+# Pin the recipe generator and its dependency lockfile. It is a build-only tool
+# and uses the same package-owned compiler as both native build stages.
+FROM rust-toolchain AS rust-chef
+RUN cd /tmp/runner-toolchain && cargo install cargo-chef --version 0.1.73 --locked
+
+FROM rust-chef AS runner-plan
+WORKDIR /app/packages/paperclip-runner
+COPY packages/paperclip-runner/rust-toolchain.toml ./
+COPY packages/paperclip-runner/runner ./runner
+RUN cd runner && cargo chef prepare --recipe-path /tmp/runner-recipe.json
+
+FROM rust-chef AS runner-deps
+WORKDIR /app/packages/paperclip-runner/runner
+COPY packages/paperclip-runner/rust-toolchain.toml ../
+# The recipe changes only when dependency manifests, the lockfile, or target
+# metadata change. Source edits can reuse this compiled dependency layer.
+COPY --from=runner-plan /tmp/runner-recipe.json /tmp/runner-recipe.json
+RUN cargo chef cook --release --locked --package paperclip-runner-core --bin paperclip-runnerd --recipe-path /tmp/runner-recipe.json \
+ && find . -mindepth 1 -maxdepth 1 ! -name target -exec rm -rf {} +
+
+FROM runner-deps AS runner-build
WORKDIR /app/packages/paperclip-runner
# Rust embeds protocol schemas and fixtures with include_str!. Keep those
# alongside the complete Cargo workspace so every compile-time input keys
diff --git a/doc/DOCKER.md b/doc/DOCKER.md
index 757a2c3094..2905bfe957 100644
--- a/doc/DOCKER.md
+++ b/doc/DOCKER.md
@@ -342,11 +342,20 @@ Notes:
## Native Runner build cache
The image compiles the native Runner in `runner-build`, before copying the
-application source. That stage includes the pinned Rust compiler, the complete
-Cargo workspace and lockfile, and the protocol schemas and fixtures embedded
-by Rust. Changes to those inputs rebuild the native binary. Ordinary server or
-UI changes can reuse it through the existing registry cache (`mode=max`). Each
-platform gets its own native build; no cross-architecture binary is reused.
+application source. A pinned `cargo-chef` generates a dependency recipe in
+`runner-plan`. The separate `runner-deps` stage compiles that recipe with the
+package-owned Rust compiler. Both the dependency build and the real binary use
+the release profile and locked Cargo dependencies. The recipe stage never
+modifies source in the checkout.
+
+Changes to Rust source or embedded protocol inputs rebuild the real binary but
+can reuse compiled dependencies when the recipe is unchanged. Dependency
+manifests, the Cargo lockfile, target metadata, or compiler changes invalidate
+the relevant cache. Ordinary server or UI changes can reuse the entire native
+build through the existing registry cache (`mode=max`). Each platform gets its
+own native build; no cross-architecture binary is reused. No additional GitHub
+Actions cache is created. A cold build also installs the recipe generator and
+compiles dependencies, so the savings apply after those layers are available.
The application build inherits that stage and still runs the normal server
build, including Cargo, binary staging, and generated-contract checks. Rust
@@ -357,6 +366,13 @@ directory as before. Cache misses only cost compilation time.
Pull requests that change the Dockerfile, Docker ignore rules, or Runner native
inputs also build the isolated `runner-build` target in `Docker Runner check`.
-This compiles against the actual reduced context and catches missing embedded
-inputs before the post-merge image build. It uses a GitHub-hosted runner with
-read-only repository access and does not publish images or cache artifacts.
+The check runs `bash scripts/check-docker-runner-cache.sh` against a disposable
+copy of tracked source and the actual Docker ignore rules. It compiles a baseline,
+changes a Rust metadata constant, and rebuilds. It requires a cached dependency
+build, an unchanged dependency recipe, and changed metadata from the real binary.
+It also verifies that a dependency declaration change alters the recipe. The
+probe exports only small metadata files, avoiding a large image import into the
+Docker daemon. It catches missing embedded inputs before the post-merge build.
+It uses a GitHub-hosted runner with read-only repository access and does not
+publish images or cache artifacts. Allow up to 20 minutes for its cold build and
+source rebuild.
diff --git a/scripts/check-docker-runner-cache.sh b/scripts/check-docker-runner-cache.sh
new file mode 100644
index 0000000000..0c8caf81d5
--- /dev/null
+++ b/scripts/check-docker-runner-cache.sh
@@ -0,0 +1,60 @@
+#!/usr/bin/env bash
+# Build the real Docker target twice in a disposable copy of tracked source.
+# Export only metadata, avoiding a multi-gigabyte test image in the daemon.
+set -euo pipefail
+repo_root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+probe_dir="$(mktemp -d "${TMPDIR:-/tmp}/paperclip-runner-cache.XXXXXX")"
+trap 'rm -rf "$probe_dir"' EXIT
+mkdir "$probe_dir/context"
+cd "$repo_root"
+git ls-files -z | tar -cf - --null -T - | tar -xf - -C "$probe_dir/context"
+cd "$probe_dir/context"
+export PROBE_DIR="$probe_dir"
+cp Dockerfile "$probe_dir/cache-probe.Dockerfile"
+cat >> "$probe_dir/cache-probe.Dockerfile" <<'DOCKER'
+FROM runner-build AS cache-proof
+RUN ./runner/target/release/paperclip-runnerd --build-metadata > /metadata.json
+FROM scratch AS cache-proof-export
+COPY --from=cache-proof /metadata.json /metadata.json
+COPY --from=runner-plan /tmp/runner-recipe.json /recipe.json
+FROM scratch AS recipe-proof-export
+COPY --from=runner-plan /tmp/runner-recipe.json /recipe.json
+DOCKER
+build_proof() {
+ docker buildx build --file "$probe_dir/cache-probe.Dockerfile" --target cache-proof-export --output "type=local,dest=$probe_dir/$1" --progress plain . 2>&1 | tee "$probe_dir/$1.log"
+}
+build_proof baseline
+python3 - <<'CHECK'
+from pathlib import Path
+p=Path('packages/paperclip-runner/runner/crates/runner-core/src/bin/paperclip-runnerd.rs')
+s=p.read_text(); needle='paperclip-runner/runnerd-build-metadata/v1'
+assert s.count(needle)==1
+p.write_text(s.replace(needle,needle+'-cache-probe'))
+CHECK
+build_proof source-change
+python3 - <<'CHECK'
+import os,json,re
+from pathlib import Path
+root=Path(os.environ['PROBE_DIR'])
+before=json.loads((root/'baseline/metadata.json').read_text())
+after=json.loads((root/'source-change/metadata.json').read_text())
+assert before['schema']=='paperclip-runner/runnerd-build-metadata/v1'
+assert after['schema']==before['schema']+'-cache-probe'
+assert (root/'baseline/recipe.json').read_bytes()==(root/'source-change/recipe.json').read_bytes()
+log=(root/'source-change.log').read_text()
+step=re.search(r'#(\d+) \[runner-deps[^\n]+ RUN cargo chef cook',log)[1]
+assert f'#{step} CACHED' in log
+assert 'Compiling paperclip-runner-core' in log
+print('PASS: unchanged dependency recipe and cached cook layer; real binary changed.')
+p=Path('packages/paperclip-runner/runner/Cargo.toml')
+s=p.read_text(); assert 'serde_json = "1.0"' in s
+p.write_text(s.replace('serde_json = "1.0"','serde_json = ">=1.0.0, <2.0.0"'))
+CHECK
+docker buildx build --file "$probe_dir/cache-probe.Dockerfile" --target recipe-proof-export --output "type=local,dest=$probe_dir/manifest-change" --progress plain .
+python3 - <<'CHECK'
+from pathlib import Path
+import os
+root=Path(os.environ['PROBE_DIR'])
+assert (root/'source-change/recipe.json').read_bytes()!=(root/'manifest-change/recipe.json').read_bytes()
+print('PASS: dependency declaration change invalidates the recipe.')
+CHECK
diff --git a/server/src/__tests__/docker-build-stamp.test.ts b/server/src/__tests__/docker-build-stamp.test.ts
index 4794989feb..76862952c8 100644
--- a/server/src/__tests__/docker-build-stamp.test.ts
+++ b/server/src/__tests__/docker-build-stamp.test.ts
@@ -76,3 +76,32 @@ describe("docker build-stamp wiring", () => {
).toBeGreaterThanOrEqual(2);
});
});
+
+
+describe("Docker Rust dependency cache", () => {
+ it("caches the locked dependency recipe separately from source and per-build metadata", () => {
+ const chef = stageBody(dockerfile, "rust-chef");
+ const planner = stageBody(dockerfile, "runner-plan");
+ const dependencies = stageBody(dockerfile, "runner-deps");
+ expect(chef).toContain("FROM rust-toolchain AS rust-chef");
+ expect(chef).toMatch(/cargo install cargo-chef --version \d+\.\d+\.\d+ --locked/);
+ expect(planner).toContain("COPY packages/paperclip-runner/runner ./runner");
+ expect(planner).toContain("cargo chef prepare --recipe-path /tmp/runner-recipe.json");
+ expect(dependencies).toContain("FROM rust-chef AS runner-deps");
+ expect(dependencies).toContain("COPY --from=runner-plan /tmp/runner-recipe.json /tmp/runner-recipe.json");
+ expect(dependencies).toContain("cargo chef cook --release --locked --package paperclip-runner-core --bin paperclip-runnerd");
+ expect(dependencies).not.toMatch(/COPY .*\.\/runner|COPY .*\.\/protocol|COPY \. \.|PAPERCLIP_BUILD_COMMIT/);
+ });
+
+ it("rebuilds real workspace code and embedded protocol inputs after cooking dependencies", () => {
+ const native = stageBody(dockerfile, "runner-build");
+ expect(native).toContain("FROM runner-deps AS runner-build");
+ for (const source of ["runner", "protocol"]) {
+ expect(native.indexOf(`COPY packages/paperclip-runner/${source} ./${source}`))
+ .toBeLessThan(native.indexOf("cargo build --release"));
+ expect(native).toContain(`COPY packages/paperclip-runner/${source} ./${source}`);
+ }
+ expect(native).toContain("cargo build --release --manifest-path runner/Cargo.toml --locked -p paperclip-runner-core --bin paperclip-runnerd");
+ expect(stageBody(dockerfile, "build")).toContain("FROM runner-build AS build");
+ });
+});