Release Management Guide#

This page provides detailed information on the steps followed to perform a release. It can be used both as a guide to learn the ADBC release process and as a comprehensive checklist for the Release Manager when performing a release.

Principles#

The Apache Arrow Release follows the guidelines defined at the Apache Software Foundation Release Policy.

Preparing for the release#

Some steps of the release require being a committer or a PMC member.

A GPG key in the Apache Web of Trust to sign artifacts. This will have to be cross signed by other Apache committers/PMC members. You must set your GPG key ID in dev/release/.env (see dev/release/.env.example for a template).
The GPG key needs to be added to this SVN repo and this one.
Configure Maven to publish artifacts to Apache repositories. You will need to setup a master password at ~/.m2/settings-security.xml and settings.xml as specified on the Apache guide. It can be tested with the following command:
```
# You might need to export GPG_TTY=$(tty) to properly prompt for a passphrase
mvn clean install -Papache-release
```
An Artifactory API key (log in with your ASF credentials, then generate it from your profile in the upper-right). You must set the Artifactory API key in dev/release/.env (see dev/release/.env.example for a template).
Install en_US.UTF-8 locale. You can confirm available locales by locale -a.
Install Conda with conda-forge, and create and activate the environment.
```
mamba create -n adbc -c conda-forge --file ci/conda_env_dev.txt
```
This will install two tools used in the release process: commitizen (generates changelog from commit messages) and gh (submit jobs/download artifacts).
Install Docker.
Clone the main Arrow repository (https://github.com/apache/arrow) and symlink arrow-adbc/dev/release/.env to arrow/dev/release/.env. Some release scripts depend on the scripts in the main Arrow repository.

Before creating a Release Candidate#

# Setup gpg agent for signing artifacts
source dev/release/setup-gpg-agent.sh

# Activate conda environment
mamba activate adbc

Check Nightly Verification Job#

Ensure that the verification job is passing. This simulates part of the release verification workflow to detect issues ahead of time.

Creating a Release Candidate#

These are the different steps that are required to create a Release Candidate.

For the initial Release Candidate, we will create a maintenance branch from master. Follow up Release Candidates will update the maintenance branch by cherry-picking specific commits.

We have implemented a Feature Freeze policy between Release Candidates. This means that, in general, we should only add bug fixes between Release Candidates. In rare cases, critical features can be added between Release Candidates, if there is community consensus.

Create or update the corresponding maintenance branch#

Initial Release Candidate

# Execute the following from an up to date master branch.
# This will create a branch locally called maint-X.Y.Z.
# X.Y.Z corresponds with the Major, Minor and Patch version number
# of the release respectively. As an example 9.0.0
git branch maint-X.Y.Z
# Push the maintenance branch to the remote repository
git push -u apache maint-X.Y.Z

Follow up Release Candidates

git switch maint-X.Y.Z
# Remove the commits that created the changelog and bumped the
# versions, since 01-source.sh will redo those steps
git reset --hard HEAD~2
# Cherry-pick any commits by hand.
git cherry-pick ...
# Push the updated maintenance branch to the remote repository
git push -u apache maint-X.Y.Z

Create the Release Candidate tag from the updated maintenance branch#

# Start from the updated maintenance branch.
git switch maint-X.Y.Z

# The following script will create a branch for the Release Candidate,
# place the necessary commits updating the version number and changelog, and then create a git tag
# on OSX use gnu-sed with homebrew: brew install gnu-sed (and export to $PATH)
#
# <rc-number> starts at 0 and increments every time the Release Candidate is burned
# so for the first RC this would be: dev/release/01-prepare.sh 3.0.0 4.0.0 5.0.0 0

dev/release/01-prepare.sh <arrow-dir> <prev-version> <version> <next-version> <rc-number>

git push -u apache apache-arrow-adbc-<version>-rc<rc-number> maint-<version>

Build source and binaries and submit them#

# Download the produced source and binaries, sign them, and add the
# signatures to the GitHub release
#
# On macOS the only way I could get this to work was running "echo
# "UPDATESTARTUPTTY" | gpg-connect-agent" before running this
# comment otherwise I got errors referencing "ioctl" errors.
dev/release/02-sign.sh <prev-version> <version> <rc-number>

# Upload the source release tarball and signs to
# https://dist.apache.org/repos/dist/dev/arrow .
dev/release/03-source.sh <version> <rc-number>

# Upload the Java artifacts
#
# Note that you need to press the "Close" button manually by Web interface
# after you complete the script:
#   https://repository.apache.org/#stagingRepositories
dev/release/04-java-upload.sh <arrow-dir> <version> <rc-number>

# Sign and upload the deb/rpm packages and APT/Yum repositories
#
# This reuses release scripts in apache/arrow. So you need to
# specify cloned apache/arrow directory.
dev/release/05-linux-upload.sh <arrow-dir> <version> <rc-number>

# Start verifications for binaries and wheels
dev/release/06-binary-verify.sh <version> <rc-number>

Verify the Release#

Start the vote thread on dev@arrow.apache.org using the template email from 06-binary-verify.sh.

Voting and approval#

Start the vote thread on dev@arrow.apache.org and supply instructions for verifying the integrity of the release. Approval requires a net of 3 +1 votes from PMC members. A release cannot be vetoed.

How to Verify Release Candidates#

Install dependencies. At minimum, you will need:
- cURL
- Docker (to verify binaries)
- Git
- GnuPG
- shasum (built into macOS) or sha256sum/sha512sum (on Linux)
You will also need to install all dependencies to build and verify all languages. Roughly, this means:
- C and C++ compilers (or the equivalent of build-essential for your platform)
- Python 3
- Ruby with headers
- bundler, rake, red-arrow, and test-unit Ruby gems
- GLib and gobject-introspection with headers
- Java JRE and JDK (Java 8+)
- Go
- CMake, ninja-build, libpq (with headers), SQLite (with headers)
Alternatively, you can have the verification script download and install dependencies automatically via Conda. See the environment variables below.

Clone the project:

$ git clone https://github.com/apache/arrow-adbc.git

Run the verification script:
```
$ cd arrow-adbc
# Pass the version and the RC number
$ ./dev/release/verify-release-candidate.sh 0.1.0 6
```
These environment variables may be helpful:
- ARROW_TMPDIR=/path/to/directory to specify the temporary directory used. Using a fixed directory can help avoid repeating the same setup and build steps if the script has to be run multiple times.
- USE_CONDA=1 to download and set up Conda for dependencies. In this case, fewer dependencies are required from the system. (Git, GnuPG, cURL, and some others are still required.)
Once finished and once the script passes, reply to the mailing list vote thread with a +1 or a -1.

Post-release tasks#

After the release vote, we must undertake many tasks to update source artifacts, binary builds, and the Arrow website.

Be sure to go through on the following checklist: