Skip to content

Apache DataFusion SQL Query Engine

chore: Add script to protect RC branches during the release (#18660)

Oleks V2dd17b9e6c0f
diff --git a/.asf.yaml b/.asf.yaml
index 2f77331adf..0588a300a5 100644
--- a/.asf.yaml
+++ b/.asf.yaml
@@ -54,72 +54,14 @@ github:
# needs to be updated as part of the release process
# .asf.yaml doesn't support wildcard branch protection rules, only exact branch names
# https://github.com/apache/infrastructure-asfyaml?tab=readme-ov-file#branch-protection
- # Keeping set of protected branches for future releases
- # Meanwhile creating a prerelease script that will update the branch protection names
- # automatically. Keep track on it https://github.com/apache/datafusion/issues/17134
+ # these branches protection blocks autogenerated during release process which is described in
+ # https://github.com/apache/datafusion/tree/main/dev/release#2-add-a-protection-to-release-candidate-branch
branch-50:
required_pull_request_reviews:
required_approving_review_count: 1
branch-51:
required_pull_request_reviews:
required_approving_review_count: 1
-# branch-52:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-53:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-54:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-55:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-56:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-57:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-58:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-59:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-60:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-61:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-62:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-63:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-64:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-65:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-66:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-67:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-68:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-69:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
-# branch-70:
-# required_pull_request_reviews:
-# required_approving_review_count: 1
pull_requests:
# enable updating head branches of pull requests
allow_update_branch: true
diff --git a/dev/release/README.md b/dev/release/README.md
index d9bab521fa..898bceb6f7 100644
--- a/dev/release/README.md
+++ b/dev/release/README.md
@@ -26,11 +26,11 @@ Patch releases are made on an adhoc basis, but we try and avoid them given the f
## Release Process Overview
New development happens on the `main` branch.
-Releases are made from branches, e.g. `branch-37` for the `37.x.y` release series.
+Releases are made from branches, e.g. `branch-50` for the `50.x.y` release series.
To prepare for a new release series, we:
-- Create a new branch from `main`, such as `branch-37` in the Apache repository (not in a fork)
+- Create a new branch from `main`, such as `branch-50` in the Apache repository (not in a fork)
- Continue merging new features changes to `main` branch
- Prepare the release branch for release:
- Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`
@@ -55,7 +55,7 @@ these steps:
1. Find (or create) the issue for the incremental release ([example release issue]) and discuss the proposed change there with the maintainers.
2. Follow normal workflow to create PR to `main` branch and wait for its approval and merge.
-3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-37`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].
+3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-50`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].
For example, to backport commit `12345` from `main` to `branch-50`:
@@ -127,16 +127,35 @@ We then publish the code in the approved artifacts to crates.io.
First create a new release branch from `main` in the apache repository.
-For example, to create the `branch-51` branch for the `51.x.y` release series:
+For example, to create the `branch-50` branch for the `50.x.y` release series:
```shell
git fetch apache # make sure we are up to date
git checkout apache/main # checkout current latest development branch
-git checkout -b branch-51 # create local branch
-git push -u apache branch-51 # push branch to apache remote
+git checkout -b branch-50 # create local branch
+git push -u apache branch-50 # push branch to apache remote
```
-### 2. Prepare PR to Update Changelog and the Release Version
+### 2. Add a protection to release candidate branch
+
+To protect a release candidate branch from accidental merges, run:
+
+```shell
+./dev/release/add-branch-protection.sh 50
+```
+
+The script will modify `.asf.yaml` and add following block:
+
+```yaml
+branch-50:
+ required_pull_request_reviews:
+ required_approving_review_count: 1
+```
+
+- Create a PR.
+- Merge to `main`.
+
+### 3. Prepare PR to Update Changelog and the Release Version
First, prepare a PR to update the changelog and versions to reflect the planned
release. See [#18173](https://github.com/apache/datafusion/pull/18173) for an example.
@@ -202,17 +221,6 @@ git commit -a -m 'Update version'
Remember to merge any fixes back to `main` branch as well.
-### 3. Prepare a PR to Modify `asf.yaml`
-
-Modify `asf.yaml` to protect future release candidate branch to prevent accidental merges:
-
-```yaml
-# needs to be updated as part of the release process
-branch-50:
- required_pull_request_reviews:
- required_approving_review_count: 1
-```
-
### 4. Prepare Release Candidate Artifacts
After the PR gets merged, you are ready to create release artifacts based off the
@@ -425,7 +433,7 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusion
Delete a release candidate:
```shell
-svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-38.0.0-rc1/
+svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/
```
#### Deleting old releases from `release` svn
@@ -441,5 +449,5 @@ svn ls https://dist.apache.org/repos/dist/release/datafusion
Delete a release:
```shell
-svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-37.0.0
+svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0
```
diff --git a/dev/release/add-branch-protection.sh b/dev/release/add-branch-protection.sh
new file mode 100755
index 0000000000..735bae7f90
--- /dev/null
+++ b/dev/release/add-branch-protection.sh
@@ -0,0 +1,160 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eu
+
+# Script to add branch protection for a new release branch in .asf.yaml
+#
+# This script automates the process of adding branch protection rules to .asf.yaml
+# for new release branches. It ensures the branch protection block doesn't already
+# exist before adding it.
+#
+# Usage:
+# ./dev/release/add-branch-protection.sh <release_number>
+#
+# Examples:
+# ./dev/release/add-branch-protection.sh 52
+# ./dev/release/add-branch-protection.sh 53
+#
+# The script will:
+# 1. Validate the release number is a positive integer
+# 2. Check if branch protection already exists for branch-<release_number>
+# 3. Add the branch protection block to .asf.yaml if it doesn't exist
+# 4. Error out if the block already exists
+
+# Check if release number is provided
+if [ $# -eq 0 ]; then
+ echo "Error: Release number is required"
+ echo "Usage: $0 <release_number>"
+ echo "Example: $0 52"
+ exit 1
+fi
+
+RELEASE_NUM=$1
+BRANCH_NAME="branch-${RELEASE_NUM}"
+ASF_YAML_FILE=".asf.yaml"
+
+# Validate release number is a positive integer
+if ! [[ "$RELEASE_NUM" =~ ^[0-9]+$ ]]; then
+ echo "Error: Release number must be a positive integer"
+ echo "Provided: $RELEASE_NUM"
+ echo "Example: ./dev/release/add-branch-protection.sh 52"
+ exit 1
+fi
+
+# Check if .asf.yaml exists
+if [ ! -f "$ASF_YAML_FILE" ]; then
+ echo "Error: $ASF_YAML_FILE not found in current directory"
+ echo "Please run this script from the repository root"
+ exit 1
+fi
+
+# Check if the branch exists in the official Apache DataFusion repository
+GITHUB_API_URL="https://api.github.com/repos/apache/datafusion/branches/${BRANCH_NAME}"
+HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$GITHUB_API_URL")
+
+if [ "$HTTP_STATUS" != "200" ]; then
+ echo "Error: Branch ${BRANCH_NAME} does not exist in the official Apache DataFusion repository"
+ echo "Please create the branch '${BRANCH_NAME}' first before adding branch protection"
+ echo ""
+ echo "To check existing branches, visit:"
+ echo " https://github.com/apache/datafusion/branches"
+ exit 1
+fi
+
+# Check if branch protection already exists for this release
+if grep -q "^[[:space:]]*${BRANCH_NAME}:" "$ASF_YAML_FILE"; then
+ echo "Error: Branch protection for ${BRANCH_NAME} already exists in $ASF_YAML_FILE"
+ exit 1
+fi
+
+# Create a temporary file
+TEMP_FILE=$(mktemp)
+
+# Read the file and insert the new branch protection block
+# We'll insert it after the last branch-XX block
+awk -v branch="$BRANCH_NAME" '
+/^[[:space:]]*branch-[0-9]+:/ {
+ last_branch_line = NR
+ last_branch_content = $0
+}
+{
+ lines[NR] = $0
+}
+END {
+ if (last_branch_line == 0) {
+ print "Error: No existing branch protection blocks found" > "/dev/stderr"
+ exit 1
+ }
+
+ # Print all lines up to and including the last branch block
+ for (i = 1; i <= last_branch_line; i++) {
+ print lines[i]
+ }
+
+ # Print the required_pull_request_reviews lines after the last branch
+ for (i = last_branch_line + 1; i <= NR; i++) {
+ print lines[i]
+ # After printing the required_approving_review_count line, insert new branch
+ if (lines[i] ~ /required_approving_review_count:/) {
+ # Check if this belongs to the last branch block by looking ahead
+ next_non_empty = i + 1
+ while (next_non_empty <= NR && lines[next_non_empty] ~ /^[[:space:]]*$/) {
+ next_non_empty++
+ }
+ # If next non-empty line is not indented more than branch level, we found the end
+ if (next_non_empty > NR || lines[next_non_empty] !~ /^[[:space:]]{6,}/) {
+ print " " branch ":"
+ print " required_pull_request_reviews:"
+ print " required_approving_review_count: 1"
+ # Skip to next iteration to avoid double printing
+ for (j = i + 1; j <= NR; j++) {
+ i = j
+ if (j <= NR) print lines[j]
+ }
+ break
+ }
+ }
+ }
+}
+' "$ASF_YAML_FILE" > "$TEMP_FILE"
+
+# Check if awk succeeded
+if [ $? -ne 0 ]; then
+ rm -f "$TEMP_FILE"
+ exit 1
+fi
+
+# Verify the new content was added
+if ! grep -q "^[[:space:]]*${BRANCH_NAME}:" "$TEMP_FILE"; then
+ echo "Error: Failed to add branch protection block"
+ rm -f "$TEMP_FILE"
+ exit 1
+fi
+
+# Replace the original file with the modified version
+mv "$TEMP_FILE" "$ASF_YAML_FILE"
+
+echo "Successfully added branch protection for ${BRANCH_NAME} to $ASF_YAML_FILE"
+echo ""
+echo "Added block:"
+echo " ${BRANCH_NAME}:"
+echo " required_pull_request_reviews:"
+echo " required_approving_review_count: 1"
+echo ""
+echo "Please review the changes and commit them."
\ No newline at end of file