Skip to content

Open-source AI coworkers that each get a computer of their own: a browser, files and tools, with every action decided before it happens and recorded after. Bring any AG-UI agent.

Publish the service images as zstd rather than gzip (#369)

David McKay31e7fb98bfa0
diff --git a/.github/workflows/publish-release.yml b/.github/workflows/publish-release.yml
index 031bd405..e8f0a341 100644
--- a/.github/workflows/publish-release.yml
+++ b/.github/workflows/publish-release.yml
@@ -206,13 +206,27 @@ jobs:
# Pushed by digest and deliberately untagged. A tag written here would name one architecture,
# and the two jobs for one image would race to own it, so the last to finish would decide what
# the tag meant.
+ #
+ # zstd, not gzip. A pull is already a compressed transfer, so this is not about compressing
+ # something that was not; it is a better algorithm for the same job. Measured on
+ # `agent-computer`, the image that matters: 962 MB becomes 886 MB, and it inflates several
+ # times faster, which is worth more than the 8% on 2 GB of Chromium. `force-compression`
+ # is what reaches the layers that came from somebody else's registry, which is where almost
+ # all of the bytes are; without it only our own thin layers change and the saving rounds to
+ # nothing. The cost is that these images no longer share layers with a gzip pull of the same
+ # base, and that a client which cannot read zstd cannot read them, which is why this is here
+ # and not on the `openbot` image above: that one is pulled by servers with whatever they have,
+ # and these are pulled by an installer that ships Podman.
- id: push
uses: docker/build-push-action@53b7df96c91f9c12dcc8a07bcb9ccacbed38856a # v7.3.0
with:
context: .
file: ${{ matrix.image }}/Dockerfile
platforms: linux/${{ matrix.platform.arch }}
- outputs: type=image,name=ghcr.io/copilotkit/openbot-${{ matrix.image }},push-by-digest=true,name-canonical=true,push=true
+ # `oci-mediatypes=true` because zstd layers have no Docker-schema2 media type to be
+ # described by. It is buildx's default for this exporter and is stated rather than
+ # assumed, since the push fails without it and the reason would not be obvious.
+ outputs: type=image,name=ghcr.io/copilotkit/openbot-${{ matrix.image }},push-by-digest=true,name-canonical=true,push=true,oci-mediatypes=true,compression=zstd,force-compression=true
# Scoped per image and per architecture. One shared scope would have ten builds
# overwriting each other's cache and none of them reading their own.
cache-from: type=gha,scope=${{ matrix.image }}-${{ matrix.platform.arch }}
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c6832671..f56092d1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,19 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
## Unreleased
+### The published service images are zstd rather than gzip
+
+An image pull was already a compressed transfer, so this is not compression where there was none; it
+is a better algorithm for the same job. `agent-computer` goes from 962 MB to 886 MB on the wire, and
+zstd inflates several times faster, which on 2 GB of Chromium is worth more than the 8%. The saving
+comes from recompressing the layers that arrived from somebody else's registry, where nearly all of
+the bytes are, so the images no longer share layers with a gzip pull of the same base.
+
+This applies to the five `ghcr.io/copilotkit/openbot-<service>` images and not to
+`ghcr.io/copilotkit/openbot`. Reading a zstd layer needs a client that supports it, which Podman and
+current containerd do; the single image is pulled by deployments running whatever they have, so it
+stays gzip.
+
### A release publishes every service's image, not just the one
`ghcr.io/copilotkit/openbot` was the only image a release produced, so anything running the Compose