Skip to content

Add a beta-promotion picker to release builds to the Play Store beta track#23079

Open
oguzkocer wants to merge 14 commits into
trunkfrom
task/promote-to-beta
Open

Add a beta-promotion picker to release builds to the Play Store beta track#23079
oguzkocer wants to merge 14 commits into
trunkfrom
task/promote-to-beta

Conversation

@oguzkocer

@oguzkocer oguzkocer commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

Adds a way to promote an already-uploaded build to the Play Store beta track without rebuilding. A developer picks a build from a Buildkite block step, and WordPress and Jetpack are promoted together (they share a version code, so one pick covers both).

On the term "promote": this isn't entirely Play Console's "Promote release" (which advances a track's current release). Here we pick any uploaded build and create a beta release for it, but it's still promotion up the track ladder. It's the terminology this PR proposes, but happy to take it to another direction if reviewers have a strong preference to use another term.

What's included:

  • fastlane/lanes/promote.rb — two lanes:
    • gather_beta_candidates lists the promotable builds (version codes present in both apps' Play libraries and above the current beta release, newest first), writes and uploads a Buildkite block step to choose one, and posts the candidate list to Slack.
    • promote_to_beta version_code:<code> creates a draft beta release for that version code in each app directly via the Play API (Supply::Client): open an edit, set the release on the beta track, commit.
  • .buildkite/promote-beta.yml plus commands/gather-beta-candidates.sh and commands/promote-to-beta.sh, wiring the gather → block → promote flow. Candidate discovery reads the beta track (google_play_track_version_codes) and the uploaded-bundle list (Supply::Client#aab_version_codes over a throwaway, aborted edit).
  • fastlane/Fastfile — adds the send_slack_message, notify_slack, and get_required_env helpers and the PLAY_STORE_VERSION_CODES_TO_RETAIN constant, and imports the new lane file.
  • fastlane/lanes/build.rbversion_codes_to_retain now references the shared PLAY_STORE_VERSION_CODES_TO_RETAIN constant.

Notes:

  • The beta release is created as release_status: 'draft', so nothing reaches beta testers yet. Flip to 'completed' in promote.rb when ready to distribute.
  • The promote lanes only run on trunk (ensure_promotion_on_trunk!); a local or off-branch run fails by design.

Testing instructions

Promotion is a Play Store track operation, so there is no app build to install. The flow runs entirely on Buildkite and ends in a draft release in the Play Console (which does not reach testers and can be discarded).

Because the promote lanes refuse to run off trunk, e2e-test from a throwaway branch with the guard removed:

Set up a test branch:

  1. Branch off task/promote-to-beta (e.g. task/promote-to-beta-e2e).
  2. Remove the ensure_promotion_on_trunk! calls in fastlane/lanes/promote.rb and commit.
  • Verify the gather and promote lanes no longer refuse to run off trunk.

Run the picker:

  1. Start a Buildkite build on the test branch with the env var PIPELINE=promote-beta.yml.
  • Verify the "Gather builds to promote to beta" step lists candidate version codes and posts them to Slack with a link to the block step.
  • Verify a "Promote to beta" block step is waiting for input.
  1. Open the Slack link (or the build) and unblock the step, choosing a build.
  • Verify the "Promote selected build to beta" step runs promote_to_beta.
  • Verify Slack posts a per-app result (WordPress and Jetpack submitted to beta).

Confirm in the Play Console:

  1. Open the Play Console for WordPress, then Jetpack.
  • Verify a draft release for the chosen version code appears on the beta track for both apps.
  • Discard the draft release(s) after verifying — draft releases do not reach testers.

@oguzkocer oguzkocer added this to the 27.0 milestone Jul 9, 2026
@dangermattic

dangermattic commented Jul 9, 2026

Copy link
Copy Markdown
Collaborator
1 Warning
⚠️ This PR is larger than 300 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.

Generated by 🚫 Danger

@wpmobilebot

wpmobilebot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

App Icon📲 You can test the changes from this Pull Request in WordPress Android by scanning the QR code below to install the corresponding build.

App NameWordPress Android
Build TypeDebug
Versionpr23079-83e0757
Build Number1498
Application IDorg.wordpress.android.prealpha
Commit83e0757
Installation URL3ot4ecqer618o
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot

wpmobilebot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

App Icon📲 You can test the changes from this Pull Request in Jetpack Android by scanning the QR code below to install the corresponding build.

App NameJetpack Android
Build TypeDebug
Versionpr23079-83e0757
Build Number1498
Application IDcom.jetpack.android.prealpha
Commit83e0757
Installation URL220hpmabbdo1g
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

oguzkocer added 14 commits July 10, 2026 23:50
First step of the beta-promotion picker: read the beta track's current
version codes for WordPress and Jetpack via google_play_track_version_codes
and log them plus the combined floor. Read-only for now.
Extend gather_beta_candidates to read every uploaded AAB version code per
app via the Play app bundle explorer (Supply::Client#aab_version_codes over
a throwaway edit), then compute the codes present for both apps and above
the beta floor. Read-only; logs the floor, the pool, and the candidates.
gather_beta_candidates now opens a Buildkite block step (candidate list
posted to Slack) and promote_to_beta promotes the chosen version code to
the beta track for WordPress and Jetpack as a draft, referencing the
existing bundle (no rebuild). Adds the Buildkite command scripts and the
promote-beta.yml pipeline.
The beta-track and bundle-explorer lookups rescued every error into [],
which is indistinguishable from a legitimately empty result: a transient
failure would collapse the candidate intersection (reading as 'nothing to
promote') or null out the floor (reading as 'promote everything'). Raise a
clear error instead, mirroring the iOS promote lanes. Also wrap the
throwaway edit's read in ensure so it's aborted even when the read raises.
upload_promotion_steps hardcoded build/promote-steps.yml while
PROMOTION_STEPS_FILE built the same path from PROJECT_ROOT_FOLDER; the two
could silently diverge. Both now derive from PROMOTION_STEPS_RELATIVE_PATH.
buildkite_api_get had no open/read timeout, so a hung response could stall
the gather lane across all 5 block-step poll attempts. Bound both to 10s.
The promote path hand-rolled its own upload_to_play_store call, duplicating
the retry, skip flags, and retained pinned codes that upload_build_to_play_store
already owns. Generalize that lane to also promote an existing build by
version_code (reference the code, skip the AAB), and have the beta promotion
call it. The pinned RETAINED_VERSION_CODES constant now lives in one place.
Use Array#max(n) instead of sort.reverse.first(n) for the newest N
candidates, and reject a non-integer version_code up front with a clear
message rather than letting Integer() fail per app.
Route the candidate list, per-app results, and the pipeline failure-notify
to #test-wpmobile-slack-integration while validating the flow. Revert to
#build-and-ship before merge.
Cut over-explaining and session-flavored comments down to the factual
point, and drop a stale comment that claimed available_aab_version_codes
returns [] on failure (it raises).
- Candidates are the version code itself; drop the version_name/build_number
  decode, which was wrong for hotfix (patch) versions and unused beyond the
  block-step/Slack label.
- Move the generic send_slack_message/notify_slack/get_required_env helpers
  out of promote.rb into the Fastfile.
- Keep [1440] inline in upload_build_to_play_store and drop the extracted
  RETAINED_VERSION_CODES constant (out of scope now that promote reuses the lane).
- Raise the Buildkite API poll timeout from 10s to 30s.
Replace the ad-hoc buildkite_api_token helper and the lone SLACK_WEBHOOK
read with the shared get_required_env helper (matching wpios), so required
secrets are read one consistent way. Optional env vars keep ENV.fetch with
a default.
upload_to_play_store(version_code:, skip_upload_aab: true) commits an empty
edit — supply only builds a track release from binaries uploaded in the same
run, so the bare version_code was ignored and no beta release was created.

Create the draft beta release directly with Supply::Client (build a
TrackRelease for the chosen version code, set it on the beta track, commit),
mirroring supply's own update_track. Revert upload_build_to_play_store to its
original AAB-only form.
…script shebangs

- Replace inline [1440] in upload_build_to_play_store and promote_version_code_to_beta with PLAY_STORE_VERSION_CODES_TO_RETAIN
- Change gather-beta-candidates.sh and promote-to-beta.sh shebangs to '#!/usr/bin/env bash' with 'set -eu'
- Update promote-beta.yml notify and send_slack_message channel comments
@oguzkocer oguzkocer force-pushed the task/promote-to-beta branch from ce4b529 to 83e0757 Compare July 11, 2026 04:37
@oguzkocer oguzkocer marked this pull request as ready for review July 11, 2026 06:12
@oguzkocer oguzkocer requested a review from a team as a code owner July 11, 2026 06:12
@oguzkocer oguzkocer requested a review from jkmassel July 11, 2026 06:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants