Apache Solr open-source search software
SOLR-18032 Docker gradle build support multi-platform (#3964)
Jan Høydahl894fcbe15ccb
diff --git a/dev-docs/gradle-help/docker.txt b/dev-docs/gradle-help/docker.txtindex f02e1948349..b691a3359b1 100644--- a/dev-docs/gradle-help/docker.txt+++ b/dev-docs/gradle-help/docker.txt@@ -26,18 +26,32 @@ Solr Distribution: (Either Full or Slim, the solr binary distribution to build tEnvVar: SOLR_DOCKER_DISTGradle Property: -Psolr.docker.dist+Docker Platforms: (Comma-separated list of target platforms)+ Default: None (builds for current architecture only)+ EnvVar: SOLR_DOCKER_PLATFORM+ Gradle Property: -Psolr.docker.platform+ Examples:+ - "linux/amd64" (single platform - can be used with dockerBuild)+ - "linux/arm64,linux/amd64" (multi-platform - only supported by dockerPush)++ Note: Multi-platform builds require Docker Buildx and are ONLY supported+ by the dockerPush task. The dockerBuild task does not support multi-platform builds because+ Docker cannot load multi-platform images into the local daemon. Use dockerPush to build and push+ multi-platform images in a single step. You may need QEMU installed to run multi-platform+ builds.+Tagging and Pushing-------To tag the docker image, run the following command.This will also ensure that the docker image has been built as per the inputs detailed above.-gradlew dockerTag+./gradlew dockerTagAnd to push the image with the given tag, run the following command.Gradle will ensure that the docker image is built and tagged as the inputs describe before being pushed.-gradlew dockerPush+./gradlew dockerPushThe docker image tag can be customized via the following options, all accepted via both Environment Variables and Gradle Properties.@@ -61,13 +75,46 @@ Docker Image Name: (Use this to explicitly set a whole image name. If given, theEnvVar: SOLR_DOCKER_IMAGE_NAMEGradle Property: -Psolr.docker.imageName+Multi-Platform Builds+-------++To build a multi-platform Docker image, use the SOLR_DOCKER_PLATFORM environment variable or Gradle property.++IMPORTANT: Multi-platform builds (with multiple platforms) are ONLY supported by dockerPush, not dockerBuild.+This is because Docker cannot load multi-platform images into the local daemon.++Prerequisites:+ - Docker Buildx must be installed (included with Docker Desktop and recent Docker Engine versions)+ - A buildx builder must be configured. If you don't have one, create it:+ docker buildx create --name solr-builder --use+ - For cross-platform builds, QEMU may be required++Build and push for multiple platforms:+ SOLR_DOCKER_PLATFORM=linux/arm64,linux/amd64 ./gradlew dockerPush++Using Gradle property:+ ./gradlew dockerPush -Psolr.docker.platform=linux/arm64,linux/amd64++The dockerPush task will build and push the multi-platform image in a single step using Docker Buildx.+The task will validate that a suitable builder exists and that it supports all requested platforms before building.++Single-Platform Builds with Explicit Platform+--------------------------------------------------++You can build for a specific platform and load it into your local Docker daemon:++Build for a specific platform:+ SOLR_DOCKER_PLATFORM=linux/arm64 ./gradlew dockerBuild++This uses Docker Buildx with --load to make the image available locally for testing.+Testing-------To test the docker image, run the following command.This will also ensure that the docker image has been built as per the inputs detailed above in the "Building" section.-gradlew testDocker+./gradlew testDockerIf a docker image build parameters were used during building, then the same inputs must be used while testing.Otherwise a new docker image will be built for the tests to run with.diff --git a/solr/docker/README.md b/solr/docker/README.mdindex ba8fbdcef61..109fd18f979 100644--- a/solr/docker/README.md+++ b/solr/docker/README.md@@ -50,6 +50,29 @@ When building the image, Solr accepts arguments for customization. Currently onldocker build --build-arg BASE_IMAGE=custom/jdk:21-slim -f solr-X.Y.Z/docker/Dockerfile https://www.apache.org/dyn/closer.lua/solr/X.Y.Z/solr-X.Y.Z.tgz```+Multi-Platform Builds+----++To build multi-platform Docker images from the binary distribution, use Docker Buildx.++**Prerequisites**:+- Docker Buildx must be installed (included with Docker Desktop and recent Docker Engine)+- A buildx builder must be configured. Create one if needed:+ ```bash+ docker buildx create --name solr-builder --use+ ```+- For cross-platform builds, QEMU is be required++**Important**: When building for multiple platforms, you cannot use `--load` to load the image into your local Docker daemon. You must use `--push` to push directly to a registry.++```bash+# Build and push for multiple platforms+docker buildx build --platform linux/amd64,linux/arm64 -f solr-X.Y.Z/docker/Dockerfile --tag myrepo/solr:X.Y.Z --push - < solr-X.Y.Z.tgz++# Build for a single specific platform (can use --load for local testing)+docker buildx build --platform linux/arm64 -f solr-X.Y.Z/docker/Dockerfile --tag myrepo/solr:X.Y.Z-arm64 --load - < solr-X.Y.Z.tgz+```+Official Image Management----diff --git a/solr/docker/build.gradle b/solr/docker/build.gradleindex 25bba389465..e5f07c6aac9 100644--- a/solr/docker/build.gradle+++ b/solr/docker/build.gradle@@ -34,6 +34,19 @@ def dockerImageRepo = "${ -> propertyOrEnvOrDefault("solr.docker.imageRepo", "SOdef dockerImageTag = "${ -> propertyOrEnvOrDefault("solr.docker.imageTag", "SOLR_DOCKER_IMAGE_TAG", project.version + dockerImageDistSuffix) }"def dockerImageName = "${ -> propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}:${dockerImageTag}${dockerImageTagSuffix}") }"def baseDockerImage = "${ -> propertyOrEnvOrDefault("solr.docker.baseImage", "SOLR_DOCKER_BASE_IMAGE", 'eclipse-temurin:' + javaVersion + '-jre-noble') }"+def dockerImagePlatforms = "${ -> propertyOrEnvOrDefault("solr.docker.platform", "SOLR_DOCKER_PLATFORM", '') }"+def isMultiPlatform = { -> !dockerImagePlatforms.isEmpty() && dockerImagePlatforms.contains(',') }+def validatePlatformFormat = { platformValue ->+ if (!platformValue.isEmpty()) {+ def platformPattern = ~/^linux\/[\w-]+(\/[\w-]+)?(,\s*linux\/[\w-]+(\/[\w-]+)?)*$/+ if (!platformValue.matches(platformPattern)) {+ throw new GradleException("Invalid platform format: ${platformValue}\n" ++ "Platform must be in format 'linux/arch' or 'linux/arch/variant' (comma-separated for multiple).\n" ++ "Examples: 'linux/amd64', 'linux/arm64', 'linux/arm/v7', 'linux/amd64,linux/arm64'")+ }+ }+}+def getSolrTgzConfiguration = { -> isImageSlim() ? configurations.solrSlimTgz : configurations.solrFullTgz }def officialDockerImageName = {String dist -> "${ -> propertyOrEnvOrDefault("solr.docker.imageName", "SOLR_DOCKER_IMAGE_NAME", "${dockerImageRepo}-official:${dockerImageTag}${distToSuffix(dist)}${dockerImageTagSuffix}") }" }def releaseGpgFingerprint = "${ -> propertyOrDefault('signing.gnupg.keyName',propertyOrDefault('signing.keyId','')) }"@@ -134,6 +147,32 @@ def checksum = { file ->return new DigestUtils(DigestUtils.sha512Digest).digestAsHex(file).trim()}+// Helper method to get Docker Buildx builder info, throws exception if not available+def getBuildxBuilderInfo = {+ def builderCheckOutput = new ByteArrayOutputStream()+ def builderCheckResult = exec {+ ignoreExitValue = true+ standardOutput = builderCheckOutput+ errorOutput = new ByteArrayOutputStream()+ commandLine "docker", "buildx", "inspect"+ }++ if (builderCheckResult.exitValue != 0) {+ throw new GradleException("Docker Buildx is not available or no builder is configured.\n" ++ "Please ensure Docker Buildx is installed and create a builder:\n" ++ " docker buildx create --name solr-builder --use\n" ++ "Or use an existing builder:\n" ++ " docker buildx use <builder-name>")+ }++ return builderCheckOutput.toString()+}++// Helper method to validate Docker Buildx builder is available+def validateBuildxBuilder = {+ getBuildxBuilderInfo()+}+task assemblePackaging(type: Sync) {description = 'Assemble docker scripts and Dockerfile for Solr Packaging'@@ -158,21 +197,51 @@ task dockerBuild() {// Ensure that the docker image is rebuilt on build-arg changes or changes in the docker contextinputs.properties([- baseDockerImage: baseDockerImage+ baseDockerImage: baseDockerImage,+ dockerImagePlatforms: dockerImagePlatforms])- var solrTgzConfiguration = isImageSlim() ? configurations.solrSlimTgz : configurations.solrFullTgz+ var solrTgzConfiguration = getSolrTgzConfiguration()inputs.files(solrTgzConfiguration)inputs.property("isSlimImage", isImageSlim())+ inputs.property("isMultiPlatform", isMultiPlatform())dependsOn(solrTgzConfiguration)doLast {- exec {- standardInput = solrTgzConfiguration.singleFile.newDataInputStream()- commandLine "docker", "build",- "-f", "solr-${ -> project.version }${dockerImageDistSuffix}/docker/Dockerfile",- "--iidfile", imageIdFile,- "--build-arg", "BASE_IMAGE=${ -> inputs.properties.baseDockerImage}",- "-"+ def platformValue = inputs.properties.dockerImagePlatforms.toString().trim()+ validatePlatformFormat(platformValue)+ def useSinglePlatform = !platformValue.isEmpty() && !platformValue.contains(',')++ // Multi-platform builds are not supported by dockerBuild, only by dockerPush+ if (isMultiPlatform()) {+ throw new GradleException("Multi-platform builds (SOLR_DOCKER_PLATFORM with multiple platforms) are not supported by dockerBuild.\n" ++ "Please use 'dockerPush' instead, which will build and push the multi-platform image in a single step.\n" ++ "Example: SOLR_DOCKER_PLATFORM=linux/arm64,linux/amd64 ./gradlew dockerPush")+ }++ if (useSinglePlatform) {+ // Single-platform build with explicit platform using buildx+ validateBuildxBuilder()++ exec {+ standardInput = solrTgzConfiguration.singleFile.newDataInputStream()+ commandLine "docker", "buildx", "build",+ "-f", "solr-${ -> project.version }${dockerImageDistSuffix}/docker/Dockerfile",+ "--platform", platformValue,+ "--build-arg", "BASE_IMAGE=${ -> inputs.properties.baseDockerImage}",+ "--iidfile", imageIdFile,+ "--load",+ "-"+ }+ } else {+ // Standard build for current architecture+ exec {+ standardInput = solrTgzConfiguration.singleFile.newDataInputStream()+ commandLine "docker", "build",+ "-f", "solr-${ -> project.version }${dockerImageDistSuffix}/docker/Dockerfile",+ "--iidfile", imageIdFile,+ "--build-arg", "BASE_IMAGE=${ -> inputs.properties.baseDockerImage}",+ "-"+ }}}@@ -180,10 +249,13 @@ task dockerBuild() {doLast {def dockerImageId = file(imageIdFile).textproject.logger.lifecycle("Solr Docker Image Created")- project.logger.lifecycle("\tID: \t${ -> dockerImageId }")+ project.logger.lifecycle("\tID/Ref: \t${ -> dockerImageId }")project.logger.lifecycle("\tBase Image: \t${ -> baseDockerImage }")project.logger.lifecycle("\tSolr Version: \t${ -> project.version }")project.logger.lifecycle("\tSolr Distribution: \t${isImageSlim() ? "Slim" : "Full"}")+ if (!dockerImagePlatforms.isEmpty()) {+ project.logger.lifecycle("\tPlatforms: \t${ -> dockerImagePlatforms }")+ }}outputs.files(imageIdFile)@@ -234,6 +306,7 @@ task dockerPush(dependsOn: tasks.dockerTag) {// Ensure that the docker image is re-pushed if the image ID or tag changesinputs.properties([dockerImageName: dockerImageName,+ dockerImagePlatforms: dockerImagePlatforms])inputs.file(imageIdFile)@@ -241,12 +314,55 @@ task dockerPush(dependsOn: tasks.dockerTag) {mustRunAfter tasks.testDockerdoLast {- exec {- commandLine "docker", "push", dockerImageName- }+ def platformValue = inputs.properties.dockerImagePlatforms.toString().trim()+ validatePlatformFormat(platformValue)++ if (isMultiPlatform()) {+ // Multi-platform push requires buildx and rebuilding with --push++ // Get builder info and validate it exists+ def builderInfo = getBuildxBuilderInfo()++ // Verify the builder supports the requested platforms (fail fast)+ def requestedPlatforms = platformValue.split(",").collect { it.trim() }+ def missingPlatforms = []+ requestedPlatforms.each { platform ->+ // Check for exact platform match - use delimiters instead of word boundaries+ // Match platform at start/end or surrounded by commas/spaces+ def platformPattern = "(?:^|[,\\s])" + Pattern.quote(platform) + "(?:[,\\s]|\$)"+ if (!(builderInfo =~ platformPattern).find()) {+ missingPlatforms.add(platform)+ }+ }- // Print information on the image after it has been created- project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")+ if (!missingPlatforms.isEmpty()) {+ // Extract supported platforms from builder info using matcher+ def supportedPlatforms = (builderInfo =~ /linux\/[\w-]+(?:\/[\w-]+)?/).collect().join(', ')+ throw new GradleException("Current builder does not support all requested platforms: ${missingPlatforms}\n" ++ "Supported platforms: ${supportedPlatforms}\n" ++ "To add platform support, Install QEMU with proper binfmt support.")+ }++ def solrTgzConfiguration = getSolrTgzConfiguration()+ exec {+ standardInput = solrTgzConfiguration.singleFile.newDataInputStream()+ commandLine "docker", "buildx", "build",+ "-f", "solr-${ -> project.version }${dockerImageDistSuffix}/docker/Dockerfile",+ "--platform", platformValue,+ "--build-arg", "BASE_IMAGE=${ -> baseDockerImage}",+ "--tag", dockerImageName,+ "--push",+ "-"+ }+ project.logger.lifecycle("Solr Docker Multi-Platform Image Pushed: \t$dockerImageName")+ project.logger.lifecycle("\tPlatforms: \t$platformValue")+ } else {+ // Standard push (works for both standard builds and single-platform buildx builds)+ exec {+ commandLine "docker", "push", dockerImageName+ }+ project.logger.lifecycle("Solr Docker Image Pushed: \t$dockerImageName")+ }}}