PostgreSQL backup tool with Point-In-Time-Recovery and restore verification
Client binaries — rules for changing this tree
Every file under assets/tools/ is a vendored third-party binary that runs in
production. ADR-0005 explains why they are committed instead of installed at
image build time. Two rules follow from that and apply to every change here:
the bytes in the repository are the bytes that ship, so a binary must be
verified in the runtime environment rather than on a developer machine, and a
directory name is a label chosen by us, which the startup check holds you to
but which nothing enforces while you are assembling the bundle.
The backend resolves a binary as
assets/tools/<arch>/<engine>/<engine>-<bundle>/bin/<command>, where <arch>
is x64 for linux/amd64 and arm for linux/arm64. The mapping lives in
backend/internal/util/tools/paths.go.
Before adding a version
Adding an engine version is mostly about the binary, not about the code that reads the version. Work through this list before committing anything; the first two items often end the job early.
- A new upstream release line needs a client here before it is supported. Until the bundle lands, a server on that line is refused at the connection test, by design: we never dump a server with a client from an older line. Both engines open a line roughly once a quarter, so this is the recurring task that keeps that promise.
- Is this a new line, or a new release inside a line we already serve?
Only a new line needs a bundle. A newer release inside a served line —
MariaDB 13.2 against our 13.0 client, MySQL 26.10 against our 26.7 one —
keeps working, and the bundle is refreshed when there is a reason to.
MariaDB's modern client additionally serves every line from 10.2 up by
designation, which is recorded in
README.mdin this directory. - Is the release line one upstream still patches? Short-term and innovation releases live about a quarter. Prefer the long-term release of that era; bundle a short-term release only when users run it. When a line we bundle later gets a long-term release, refresh the bundle to it rather than adding an identity.
- Do binaries exist for both architectures? Check, in order: the vendor's architecture-independent tarball, the vendor's Debian repository, the vendor's Ubuntu repository, then RPMs for an older enterprise distribution. Do not conclude a build is unavailable after checking one of them.
- Will the binary start in the runtime image? The image is
debian:bookworm-slimwith glibc 2.36. Compare withobjdump -T <binary> | grep -oE 'GLIBC_2\.[0-9]+' | sort -uV | tail -1and list the shared libraries withobjdump -p <binary> | grep NEEDED. Every library must already be in the image; see the table inREADME.mdin this directory. A matching soname is not enough: check the OpenSSL symbol versions the same way withOPENSSL_3\.[0-9]+, because bookworm ships OpenSSL 3.0 and a newer requirement fails only at run time. A development host with a newer OpenSSL hides the problem, so run the binary in the runtime image as the refresh checklist shows. - Know which of the two identity shapes the engine uses. MySQL's identity doubles as the name of its bundle directory, because each MySQL line gets a client of its own. MariaDB keeps two values, a server identity and a client version, because one client serves many server lines. Follow the engine's existing shape rather than introducing the other one, and never record a version the server did not report.
- Add the bundle and the identity together. The new directory goes into
the bundle set the startup check walks, in
backend/internal/util/tools/mysql.goormariadb.go, and the identity it serves is mapped to it. Detection refuses any version with no bundle, so these two land in one change or the version stays unsupported. - Check both architectures before claiming the version is supported.
A version whose client exists for only one architecture is supported only
there, and is refused on the other.
assets/tools/arm/mysql/has no 5.7 bundle for that reason. - Do not look for version tables to extend; there are none left. The restore downgrade order is computed from the numbers in the identity, the MySQL compression choice is derived from the identity rather than matched against a list of known ones, and the interface renders the identity string itself. All three used to be lookup tables whose missing entry changed behavior quietly. If you find yourself adding a version to a table, the table is the bug.
- An identity names a line, so its digits are not a promise about the
binary. Identity
26covers every MySQL 26.x and13.0every MariaDB 13.x, while the bundle holds one patch. That is intended. What is not intended is a bundle holding a version from a different line, which the startup check now rejects. - Add the identity constant on both sides. The backend constant and the
one in
frontend/src/entity/databases/model/<engine>/name the same value, and the enumeration is the type of the field, so a missing constant is a build error rather than a blank field. - Add the version to the test matrices. A unit test of the version
mapping proves nothing about a dump. The matrices in
backend/internal/features/databases/databases/<engine>/model_test.goandbackend/internal/features/tests/logical/<engine>/backup_restore_test.goboot a real server of each version. - Update the supported-version lists in the root
README.mdand in all five copies underassets/readme/, and the tables inREADME.mdin this directory.assets/readme/AGENTS.mdrequires every copy to follow the English file.
Refreshing a patch level
Refreshing is lower risk than adding a version, but it changes the client for
every server already using that bundle, so it is not a silent operation.
For MariaDB, tools/refresh-mariadb-bundle.sh <bundle> <arch> does the work;
edit the pinned package version in it rather than passing one in.
Keep every binary within one bundle on the same upstream build. A
pg_dumpolder than its siblingpg_basebackupis how issue #725 survived unnoticed.Pin the exact upstream package version in the refresh script. Installing "the newest patch of this line" makes the result depend on the day it ran, and two architectures refreshed a week apart then disagree.
Refresh both architectures from the same upstream source in the same change. Mixing sources is how one directory name comes to mean two different builds, and the difference surfaces as an engine-specific dump failure on one architecture only.
Run the refreshed binary in the runtime base image before committing:
docker run --rm --platform linux/amd64 -v "$PWD/assets/tools:/tools:ro" \ debian:bookworm-slim bash -c 'apt-get update -qq && \ apt-get install -y -qq libedit2 libncurses6 libtinfo6 libssl3 >/dev/null && \ /tools/x64/<engine>/<bundle>/bin/<command> --version'Confirm the executable bit is recorded in the index.
git ls-files -smust show100755; a binary committed as100644fails at runtime and not at build time.Run the engine's whole matrix, not only the refreshed version. A refresh changes the client for every server on that bundle.
Record the new patch and its download source in the tables in
README.mdin this directory, in the same change.
What not to do here
- Do not install client binaries in the
Dockerfile. ADR-0005 rejected that. - Do not add a shared library to the runtime image only to satisfy one client when a build against the libraries already present exists.
- Do not trust a directory name while you are building the bundle. Confirm
what a binary is by running
--version, or by reading its version string when the architecture cannot be executed locally. The startup check does compare the two — it runs every client's version flag and rejects a bundle whose version belongs to another line — but it is the last net, not the first. Note thatmariadb-dump --versionprints two numbers and only the one before-MariaDBis the server line;client 10.19is the protocol version.