Apache DataFusion SQL Query Engine
chore: Add script to protect RC branches during the release (#18660)
Oleks V2dd17b9e6c0f
diff --git a/.asf.yaml b/.asf.yamlindex 2f77331adf..0588a300a5 100644--- a/.asf.yaml+++ b/.asf.yaml@@ -54,72 +54,14 @@ github:# needs to be updated as part of the release process# .asf.yaml doesn't support wildcard branch protection rules, only exact branch names# https://github.com/apache/infrastructure-asfyaml?tab=readme-ov-file#branch-protection- # Keeping set of protected branches for future releases- # Meanwhile creating a prerelease script that will update the branch protection names- # automatically. Keep track on it https://github.com/apache/datafusion/issues/17134+ # these branches protection blocks autogenerated during release process which is described in+ # https://github.com/apache/datafusion/tree/main/dev/release#2-add-a-protection-to-release-candidate-branchbranch-50:required_pull_request_reviews:required_approving_review_count: 1branch-51:required_pull_request_reviews:required_approving_review_count: 1-# branch-52:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-53:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-54:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-55:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-56:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-57:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-58:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-59:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-60:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-61:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-62:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-63:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-64:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-65:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-66:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-67:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-68:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-69:-# required_pull_request_reviews:-# required_approving_review_count: 1-# branch-70:-# required_pull_request_reviews:-# required_approving_review_count: 1pull_requests:# enable updating head branches of pull requestsallow_update_branch: truediff --git a/dev/release/README.md b/dev/release/README.mdindex d9bab521fa..898bceb6f7 100644--- a/dev/release/README.md+++ b/dev/release/README.md@@ -26,11 +26,11 @@ Patch releases are made on an adhoc basis, but we try and avoid them given the f## Release Process OverviewNew development happens on the `main` branch.-Releases are made from branches, e.g. `branch-37` for the `37.x.y` release series.+Releases are made from branches, e.g. `branch-50` for the `50.x.y` release series.To prepare for a new release series, we:-- Create a new branch from `main`, such as `branch-37` in the Apache repository (not in a fork)+- Create a new branch from `main`, such as `branch-50` in the Apache repository (not in a fork)- Continue merging new features changes to `main` branch- Prepare the release branch for release:- Update version numbers in `Cargo.toml` files and create `CHANGELOG.md`@@ -55,7 +55,7 @@ these steps:1. Find (or create) the issue for the incremental release ([example release issue]) and discuss the proposed change there with the maintainers.2. Follow normal workflow to create PR to `main` branch and wait for its approval and merge.-3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-37`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].+3. After PR is squash merged to `main`, branch from most recent release branch (e.g. `branch-50`), cherry-pick the commit and create a PR targeting the release branch [example backport PR].For example, to backport commit `12345` from `main` to `branch-50`:@@ -127,16 +127,35 @@ We then publish the code in the approved artifacts to crates.io.First create a new release branch from `main` in the apache repository.-For example, to create the `branch-51` branch for the `51.x.y` release series:+For example, to create the `branch-50` branch for the `50.x.y` release series:```shellgit fetch apache # make sure we are up to dategit checkout apache/main # checkout current latest development branch-git checkout -b branch-51 # create local branch-git push -u apache branch-51 # push branch to apache remote+git checkout -b branch-50 # create local branch+git push -u apache branch-50 # push branch to apache remote```-### 2. Prepare PR to Update Changelog and the Release Version+### 2. Add a protection to release candidate branch++To protect a release candidate branch from accidental merges, run:++```shell+./dev/release/add-branch-protection.sh 50+```++The script will modify `.asf.yaml` and add following block:++```yaml+branch-50:+ required_pull_request_reviews:+ required_approving_review_count: 1+```++- Create a PR.+- Merge to `main`.++### 3. Prepare PR to Update Changelog and the Release VersionFirst, prepare a PR to update the changelog and versions to reflect the plannedrelease. See [#18173](https://github.com/apache/datafusion/pull/18173) for an example.@@ -202,17 +221,6 @@ git commit -a -m 'Update version'Remember to merge any fixes back to `main` branch as well.-### 3. Prepare a PR to Modify `asf.yaml`--Modify `asf.yaml` to protect future release candidate branch to prevent accidental merges:--```yaml-# needs to be updated as part of the release process-branch-50:- required_pull_request_reviews:- required_approving_review_count: 1-```-### 4. Prepare Release Candidate ArtifactsAfter the PR gets merged, you are ready to create release artifacts based off the@@ -425,7 +433,7 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusionDelete a release candidate:```shell-svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-38.0.0-rc1/+svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/```#### Deleting old releases from `release` svn@@ -441,5 +449,5 @@ svn ls https://dist.apache.org/repos/dist/release/datafusionDelete a release:```shell-svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-37.0.0+svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0```diff --git a/dev/release/add-branch-protection.sh b/dev/release/add-branch-protection.shnew file mode 100755index 0000000000..735bae7f90--- /dev/null+++ b/dev/release/add-branch-protection.sh@@ -0,0 +1,160 @@+#!/usr/bin/env bash+# Licensed to the Apache Software Foundation (ASF) under one+# or more contributor license agreements. See the NOTICE file+# distributed with this work for additional information+# regarding copyright ownership. The ASF licenses this file+# to you under the Apache License, Version 2.0 (the+# "License"); you may not use this file except in compliance+# with the License. You may obtain a copy of the License at+#+# http://www.apache.org/licenses/LICENSE-2.0+#+# Unless required by applicable law or agreed to in writing,+# software distributed under the License is distributed on an+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY+# KIND, either express or implied. See the License for the+# specific language governing permissions and limitations+# under the License.++set -eu++# Script to add branch protection for a new release branch in .asf.yaml+#+# This script automates the process of adding branch protection rules to .asf.yaml+# for new release branches. It ensures the branch protection block doesn't already+# exist before adding it.+#+# Usage:+# ./dev/release/add-branch-protection.sh <release_number>+#+# Examples:+# ./dev/release/add-branch-protection.sh 52+# ./dev/release/add-branch-protection.sh 53+#+# The script will:+# 1. Validate the release number is a positive integer+# 2. Check if branch protection already exists for branch-<release_number>+# 3. Add the branch protection block to .asf.yaml if it doesn't exist+# 4. Error out if the block already exists++# Check if release number is provided+if [ $# -eq 0 ]; then+ echo "Error: Release number is required"+ echo "Usage: $0 <release_number>"+ echo "Example: $0 52"+ exit 1+fi++RELEASE_NUM=$1+BRANCH_NAME="branch-${RELEASE_NUM}"+ASF_YAML_FILE=".asf.yaml"++# Validate release number is a positive integer+if ! [[ "$RELEASE_NUM" =~ ^[0-9]+$ ]]; then+ echo "Error: Release number must be a positive integer"+ echo "Provided: $RELEASE_NUM"+ echo "Example: ./dev/release/add-branch-protection.sh 52"+ exit 1+fi++# Check if .asf.yaml exists+if [ ! -f "$ASF_YAML_FILE" ]; then+ echo "Error: $ASF_YAML_FILE not found in current directory"+ echo "Please run this script from the repository root"+ exit 1+fi++# Check if the branch exists in the official Apache DataFusion repository+GITHUB_API_URL="https://api.github.com/repos/apache/datafusion/branches/${BRANCH_NAME}"+HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$GITHUB_API_URL")++if [ "$HTTP_STATUS" != "200" ]; then+ echo "Error: Branch ${BRANCH_NAME} does not exist in the official Apache DataFusion repository"+ echo "Please create the branch '${BRANCH_NAME}' first before adding branch protection"+ echo ""+ echo "To check existing branches, visit:"+ echo " https://github.com/apache/datafusion/branches"+ exit 1+fi++# Check if branch protection already exists for this release+if grep -q "^[[:space:]]*${BRANCH_NAME}:" "$ASF_YAML_FILE"; then+ echo "Error: Branch protection for ${BRANCH_NAME} already exists in $ASF_YAML_FILE"+ exit 1+fi++# Create a temporary file+TEMP_FILE=$(mktemp)++# Read the file and insert the new branch protection block+# We'll insert it after the last branch-XX block+awk -v branch="$BRANCH_NAME" '+/^[[:space:]]*branch-[0-9]+:/ {+ last_branch_line = NR+ last_branch_content = $0+}+{+ lines[NR] = $0+}+END {+ if (last_branch_line == 0) {+ print "Error: No existing branch protection blocks found" > "/dev/stderr"+ exit 1+ }++ # Print all lines up to and including the last branch block+ for (i = 1; i <= last_branch_line; i++) {+ print lines[i]+ }++ # Print the required_pull_request_reviews lines after the last branch+ for (i = last_branch_line + 1; i <= NR; i++) {+ print lines[i]+ # After printing the required_approving_review_count line, insert new branch+ if (lines[i] ~ /required_approving_review_count:/) {+ # Check if this belongs to the last branch block by looking ahead+ next_non_empty = i + 1+ while (next_non_empty <= NR && lines[next_non_empty] ~ /^[[:space:]]*$/) {+ next_non_empty+++ }+ # If next non-empty line is not indented more than branch level, we found the end+ if (next_non_empty > NR || lines[next_non_empty] !~ /^[[:space:]]{6,}/) {+ print " " branch ":"+ print " required_pull_request_reviews:"+ print " required_approving_review_count: 1"+ # Skip to next iteration to avoid double printing+ for (j = i + 1; j <= NR; j++) {+ i = j+ if (j <= NR) print lines[j]+ }+ break+ }+ }+ }+}+' "$ASF_YAML_FILE" > "$TEMP_FILE"++# Check if awk succeeded+if [ $? -ne 0 ]; then+ rm -f "$TEMP_FILE"+ exit 1+fi++# Verify the new content was added+if ! grep -q "^[[:space:]]*${BRANCH_NAME}:" "$TEMP_FILE"; then+ echo "Error: Failed to add branch protection block"+ rm -f "$TEMP_FILE"+ exit 1+fi++# Replace the original file with the modified version+mv "$TEMP_FILE" "$ASF_YAML_FILE"++echo "Successfully added branch protection for ${BRANCH_NAME} to $ASF_YAML_FILE"+echo ""+echo "Added block:"+echo " ${BRANCH_NAME}:"+echo " required_pull_request_reviews:"+echo " required_approving_review_count: 1"+echo ""+echo "Please review the changes and commit them."\ No newline at end of file