diff --git a/dev/release/README.md b/dev/release/README.md index d49c1c9125235..42af909549d10 100644 --- a/dev/release/README.md +++ b/dev/release/README.md @@ -79,34 +79,34 @@ We then publish the code in the approved artifacts to crates.io. First create a new release branch from `main` in the `apache` repository. -For example, to create the `branch-50` branch for the `50.x.y` release series: +For example, to create the `branch-55` branch for the `55.x.y` release series: ```shell git fetch apache # make sure we are up to date git checkout apache/main # checkout current latest development branch -git checkout -b branch-50 # create local branch -git push -u apache branch-50 # push branch to apache remote +git checkout -b branch-55 # create local branch +git push -u apache branch-55 # push branch to apache remote ``` ### 2. Prepare PR to Update the Release Version -Manually update the DataFusion version in the root `Cargo.toml` to -reflect the new release version. Ensure `Cargo.lock` is updated accordingly by -running: - -```shell -cargo check -p datafusion -``` - -Within the user documentation there are references to the current version number. -Update these to the current version. At the time of this writing we need to manually -update the following files +Update the DataFusion version in the root `Cargo.toml`, in `Cargo.lock` and in the user +documentation. At the time of this writing the documentation files are - `docs/source/download.md` - `docs/source/user-guide/configs.md` - `docs/source/user-guide/crate-configuration.md` - `docs/source/user-guide/example-usage.md` +For example, to update the version from `55.1.0` to `55.2.0`: + +```shell +# version and lockfile +sed -i '' 's/"55\.1\.0"/"55.2.0"/g' Cargo.toml +sed -i '' 's/55\.1\.0/55.2.0/' docs/source/download.md docs/source/user-guide/{configs,crate-configuration,example-usage}.md +cargo update --workspace +``` + Then commit the changes and create a PR targeting the release branch `branch-N`. ```shell @@ -118,20 +118,20 @@ git commit -a -m 'Update version' To protect a release candidate branch from accidental merges, create PR against `main`: ```shell -git fetch apache && git checkout -b protect_branch_50 -./dev/release/add-branch-protection.sh 50 +git fetch apache && git checkout -b protect_branch_55 +./dev/release/add-branch-protection.sh 55 ``` The script will modify `.asf.yaml` and add following block: ```yaml -branch-50: +branch-55: required_pull_request_reviews: required_approving_review_count: 1 ``` - Commit changes -- Push to `origin/protect_branch_50` +- Push to `origin/protect_branch_55` - Create a PR against `main`. - Merge to `main`. - Notify community in Discord/Slack that release branch is created @@ -146,7 +146,7 @@ Backports are important and sometimes unexpected, so please proceed to next rele ### 5. Prepare PR to Update Changelog Update the changelog in `dev/changelog/`. Each release has its -own file, such as `dev/changelog/50.0.0.md`, which should include all changes +own file, such as `dev/changelog/55.2.0.md`, which should include all changes since the previous release. The changelog is generated using a Python script, which requires a GitHub @@ -160,7 +160,7 @@ uv sync To generate the changelog, set the `GITHUB_TOKEN` environment variable and run `./dev/release/generate-changelog.py` with two commit IDs or tags followed by the release version. For example, to generate a changelog of all changes -between the `50.3.0` tag and `branch-51` for release `51.0.0`: +between the `55.1.0` tag and `branch-55` for release `55.2.0`: > [!NOTE] > @@ -173,7 +173,7 @@ between the `50.3.0` tag and `branch-51` for release `51.0.0`: ```shell export GITHUB_TOKEN= -uv run ./dev/release/generate-changelog.py 50.3.0 branch-51 51.0.0 > dev/changelog/51.0.0.md +uv run ./dev/release/generate-changelog.py 55.1.0 branch-55 55.2.0 > dev/changelog/55.2.0.md ``` This script creates a changelog from GitHub PRs based on the labels associated with them as well as looking for @@ -182,7 +182,7 @@ titles starting with `feat:`, `fix:`, or `docs:`. Once the changelog is generated, run `prettier` to format the document: ```shell -prettier -w dev/changelog/51.0.0.md +prettier -w dev/changelog/55.2.0.md ``` Then commit the changes and create a PR targeting the release branch. @@ -209,17 +209,17 @@ Pick numbers in sequential order, with `1` for `rc1`, `2` for `rc2`, etc. While the official release artifacts are signed tarballs and zip files, we also tag the commit it was created from for convenience and code archaeology. Release tags -look like `50.3.0`, and release candidate tags look like `50.3.0-rc1`. See [the list of existing +look like `55.2.0`, and release candidate tags look like `55.2.0-rc1`. See [the list of existing tags]. [the list of existing tags]: https://github.com/apache/datafusion/tags -Create and push the RC tag, for example, to create the `50.3.0-rc1` tag from `branch-50`, use: +Create and push the RC tag, for example, to create the `55.2.0-rc1` tag from `branch-55`, use: ```shell git fetch apache -git tag 50.3.0-rc1 apache/branch-50 -git push apache 50.3.0-rc1 +git tag 55.2.0-rc1 apache/branch-55 +git push apache 55.2.0-rc1 ``` Please make sure the format is correct, tools like Homebrew listens for tags and in case of malformed tags users would be notified for non-existent version @@ -228,10 +228,10 @@ Please make sure the format is correct, tools like Homebrew listens for tags and Run the `create-tarball.sh` script with the `` tag and `` number you determined in previous steps: -For example, to create the `50.3.0-rc1` artifacts: +For example, to create the `55.2.0-rc1` artifacts: ```shell -GITHUB_TOKEN= ./dev/release/create-tarball.sh 50.3.0 1 +GITHUB_TOKEN= ./dev/release/create-tarball.sh 55.2.0 1 ``` The `create-tarball.sh` script @@ -243,6 +243,12 @@ The `create-tarball.sh` script 2. Provides you an email template to send to `dev@datafusion.apache.org` for release voting. +Check that the artifacts were uploaded to SVN: + +```shell +svn ls https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-55.2.0-rc1/ +``` + ### 7. Vote on Release Candidate Artifacts Send the email output from the script to `dev@datafusion.apache.org`. @@ -257,7 +263,7 @@ review the release candidate. `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like this: ```shell -./dev/release/verify-release-candidate.sh 50.3.0 1 +./dev/release/verify-release-candidate.sh 55.2.0 1 ``` #### If Changes Are Requested @@ -282,11 +288,17 @@ with the release verification. NOTE: steps in this section can only be done by PMC members after release is approved. Move artifacts to the release location in SVN, e.g. -https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.3.0/, using +https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0/, using the `release-tarball.sh` script: ```shell -./dev/release/release-tarball.sh 50.3.0 1 +./dev/release/release-tarball.sh 55.2.0 1 +``` + +Check that the artifacts are in the release location: + +```shell +svn ls https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0/ ``` Congratulations! The release is now official! @@ -296,9 +308,9 @@ Congratulations! The release is now official! Tag the same release candidate commit with the final release tag ```shell -git checkout 50.3.0-rc1 -git tag 50.3.0 -git push apache 50.3.0 +git checkout 55.2.0-rc1 +git tag 55.2.0 +git push apache 55.2.0 ``` ### 10. Publish on Crates.io @@ -315,10 +327,17 @@ instructions](https://doc.rust-lang.org/cargo/reference/publishing.html) to create an account and login to crates.io before asking to be added as an owner to all DataFusion crates. -Download and unpack the official release tarball +Download and unpack the official release tarball: + +```shell +svn export https://dist.apache.org/repos/dist/release/datafusion/datafusion-55.2.0 +cd datafusion-55.2.0 +tar xzf apache-datafusion-55.2.0.tar.gz +cd apache-datafusion-55.2.0 +``` Verify that the Cargo.toml in the tarball contains the correct version -(e.g. `version = "50.3.0"`) and then publish the crates by running the following commands +(e.g. `version = "55.2.0"`) and then publish the crates by running the following commands ```shell (cd datafusion/common && cargo publish) @@ -370,10 +389,10 @@ If it happens crates.io fails with wrong dependency message like below, just rer error: failed to prepare local package for uploading Caused by: - failed to select a version for the requirement `datafusion-proto = "^53.1.0"` - candidate versions found which didn't match: 53.0.0, 52.5.0, 52.4.0, ... + failed to select a version for the requirement `datafusion-proto = "^55.2.0"` + candidate versions found which didn't match: 55.1.0, 55.0.0, 54.1.0, ... location searched: crates.io index - required by package `datafusion-ffi v53.1.0 (/private/tmp/apache-datafusion-53.1.0/datafusion/ffi)` + required by package `datafusion-ffi v55.2.0 (/private/tmp/apache-datafusion-55.2.0/datafusion/ffi)` ``` ### Publish datafusion-cli on Homebrew @@ -423,22 +442,21 @@ svn ls https://dist.apache.org/repos/dist/dev/datafusion To delete a release candidate: ```shell -svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-50.0.0-rc1/ +svn delete -m "delete old DataFusion RC" https://dist.apache.org/repos/dist/dev/datafusion/apache-datafusion-55.2.0-rc1/ ``` #### Delete old releases from `release` SVN -Only the latest release should be available. Delete old releases after -publishing the new release. - -To get a list of DataFusion releases: - -```shell -svn ls https://dist.apache.org/repos/dist/release/datafusion -``` +Keep the last major release (`X.0.0`) and the releases after it. Delete older releases after +publishing the new release. For example, after publishing 55.2.0 keep `datafusion-55.0.0`, +`datafusion-55.1.0` and `datafusion-55.2.0`, and delete every 54.x release. -To delete a release: +The following prints, but does not run, the `svn delete` commands for all releases that do not +belong to the `major` version. Check the printed commands and then run them: ```shell -svn delete -m "delete old DataFusion release" https://dist.apache.org/repos/dist/release/datafusion/datafusion-50.0.0 +major=55 +for release in $(svn ls https://dist.apache.org/repos/dist/release/datafusion | grep -E "^datafusion-[0-9]+\." | grep -v "^datafusion-$major\."); do + echo "svn delete -m 'delete old DataFusion release' https://dist.apache.org/repos/dist/release/datafusion/$release" +done ```