Skip to content

Commit 4a35c55

Browse files
fix(ci): recover a release whose release-PR merge run failed (#289)
Only the release-PR merge run builds and publishes a release. When that run fails, the manifest version is never tagged, the release PR stays "autorelease: pending", and release-please aborts on every later push ("There are untagged, merged release PRs outstanding"). 2.10.0 and 2.11.0 both failed validation on that run and had to be released by hand. - ci/recover_release.sh runs before release-please on every push to main. If the manifest version has no GitHub release and no release-PR merge or dispatched run is still in progress, it dispatches main.yaml. - The "Mark the release PR as tagged" step now also runs for dispatched releases, relabelling every merged release PR still marked pending. - The workflow gains actions: write so it can dispatch the run. Co-authored-by: bougyman's bot <ruby-automation@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 3608a53 commit 4a35c55

3 files changed

Lines changed: 195 additions & 4 deletions

File tree

‎.github/workflows/main.yaml‎

Lines changed: 34 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,9 @@ on: # yamllint disable-line rule:truthy
1010
workflow_dispatch:
1111

1212
permissions:
13+
# actions: write lets manage-release-pr dispatch a release run whose
14+
# release-PR merge run failed (ci/recover_release.sh).
15+
actions: write
1316
contents: write
1417
packages: write
1518
pull-requests: write
@@ -50,6 +53,15 @@ jobs:
5053
uses: actions/checkout@v7
5154
with:
5255
fetch-tags: true
56+
-
57+
# Only the release-PR merge run builds and publishes a release. If
58+
# that run fails (as 2.10.0 and 2.11.0 did, on validation), the
59+
# manifest version is never tagged and release-please below aborts
60+
# on every push. This dispatches the release run for that version.
61+
name: Recover a release whose run failed
62+
env:
63+
GH_TOKEN: ${{ github.token }}
64+
run: ./ci/recover_release.sh
5365
-
5466
# skip-github-release: this only ever manages the version-bump PR
5567
# (manifest/CHANGELOG.md/mix.exs) - it never creates a tag or
@@ -338,14 +350,32 @@ jobs:
338350
# keeps finding this PR stuck "pending" forever and refuses to
339351
# open any future release PR ("There are untagged, merged release
340352
# PRs outstanding - aborting").
353+
# A dispatched run (a manual or recovered release) has no PR in its
354+
# event, so it relabels every merged release PR still marked pending.
341355
name: Mark the release PR as tagged
342-
if: github.event_name == 'pull_request'
343356
env:
344357
GH_TOKEN: ${{ github.token }}
358+
PR_NUMBER: ${{ github.event.pull_request.number }}
345359
run: |
346-
gh pr edit "${{ github.event.pull_request.number }}" \
347-
--remove-label "autorelease: pending" \
348-
--add-label "autorelease: tagged"
360+
if [ -n "$PR_NUMBER" ]
361+
then
362+
numbers="$PR_NUMBER"
363+
elif ! numbers=$(gh pr list --state merged --label "autorelease: pending" \
364+
--head release-please--branches--main --json number --jq '.[].number')
365+
then
366+
printf 'ERROR: unable to list pending release PRs\n' >&2
367+
exit 1
368+
fi
369+
for number in $numbers
370+
do
371+
if ! gh pr edit "$number" \
372+
--remove-label "autorelease: pending" \
373+
--add-label "autorelease: tagged"
374+
then
375+
printf 'ERROR: unable to relabel release PR #%s\n' "$number" >&2
376+
exit 1
377+
fi
378+
done
349379
working-directory: .
350380

351381
container:

‎ci/recover_release.sh‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
#!/usr/bin/env bash
2+
# Re-runs a release whose run failed. Called by the manage-release-pr job on
3+
# every push to main.
4+
#
5+
# Only the release-PR merge event builds and publishes a release. If that one
6+
# run fails, the manifest version is never tagged, and release-please refuses
7+
# to open any later release PR ("There are untagged, merged release PRs
8+
# outstanding"). This script finds that state and dispatches the release run.
9+
#
10+
# Environment:
11+
# GH_TOKEN token for gh; needs actions: write to dispatch
12+
# RELEASE_WORKFLOW workflow file to dispatch (default: main.yaml)
13+
# RELEASE_MANIFEST release-please manifest (default: .release-please-manifest.json)
14+
15+
workflow="${RELEASE_WORKFLOW:-main.yaml}"
16+
manifest="${RELEASE_MANIFEST:-.release-please-manifest.json}"
17+
18+
if ! version=$(jq -er '."."' "$manifest")
19+
then
20+
printf 'ERROR: unable to read the version from %s\n' "$manifest" >&2
21+
exit 1
22+
fi
23+
tag="v$version"
24+
25+
if gh release view "$tag" --json tagName >/dev/null 2>&1
26+
then
27+
printf 'Release %s exists; nothing to recover\n' "$tag"
28+
exit 0
29+
fi
30+
31+
# A release-PR merge run (pull_request) or an earlier recovery
32+
# (workflow_dispatch) may still be building this release. Leave it alone; the
33+
# next push to main checks again.
34+
if ! in_flight=$(gh run list --workflow "$workflow" --limit 50 \
35+
--json event,status \
36+
--jq '[.[] | select(.status != "completed")
37+
| select(.event == "pull_request" or .event == "workflow_dispatch")]
38+
| length')
39+
then
40+
printf 'ERROR: unable to list %s runs\n' "$workflow" >&2
41+
exit 1
42+
fi
43+
44+
if [ "$in_flight" != "0" ]
45+
then
46+
printf 'Release %s is missing, but %s release-capable run(s) are in progress; not dispatching\n' \
47+
"$tag" "$in_flight"
48+
exit 0
49+
fi
50+
51+
if ! gh workflow run "$workflow" --ref main
52+
then
53+
printf 'ERROR: release %s is missing and dispatching %s failed\n' "$tag" "$workflow" >&2
54+
exit 1
55+
fi
56+
57+
printf '::warning::Release %s was missing; dispatched %s to build and publish it\n' "$tag" "$workflow"
58+
exit 0

‎test/recover_release_test.exs‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
defmodule RecoverReleaseTest do
2+
use ExUnit.Case, async: true
3+
4+
# ci/recover_release.sh re-dispatches a release whose release-PR merge run
5+
# failed. 2.10.0 and 2.11.0 both failed validation on that run, which left
6+
# release-please aborting on every later push until the release was
7+
# dispatched by hand.
8+
9+
@script Path.expand("../ci/recover_release.sh", __DIR__)
10+
11+
@fake_gh """
12+
#!/bin/sh
13+
printf '%s\\n' "$*" >> "$FAKE_GH_LOG"
14+
case "$1 $2" in
15+
"release view")
16+
[ "$FAKE_RELEASE_EXISTS" = "yes" ] && exit 0
17+
exit 1
18+
;;
19+
"run list")
20+
[ -n "$FAKE_RUN_LIST_FAILS" ] && exit 1
21+
printf '%s\\n' "${FAKE_IN_FLIGHT:-0}"
22+
;;
23+
"workflow run")
24+
[ -n "$FAKE_DISPATCH_FAILS" ] && exit 1
25+
exit 0
26+
;;
27+
esac
28+
"""
29+
30+
setup do
31+
dir = tmp_dir!()
32+
bin = Path.join(dir, "bin")
33+
File.mkdir_p!(bin)
34+
gh = Path.join(bin, "gh")
35+
File.write!(gh, @fake_gh)
36+
File.chmod!(gh, 0o755)
37+
38+
manifest = Path.join(dir, "manifest.json")
39+
File.write!(manifest, ~s({".": "2.11.0"}\n))
40+
41+
%{dir: dir, bin: bin, manifest: manifest, log: Path.join(dir, "gh.log")}
42+
end
43+
44+
test "does nothing when the manifest version is released", ctx do
45+
assert {output, 0} = run(ctx, [{"FAKE_RELEASE_EXISTS", "yes"}])
46+
assert output =~ "Release v2.11.0 exists; nothing to recover"
47+
assert gh_calls(ctx) == ["release view v2.11.0 --json tagName"]
48+
end
49+
50+
test "dispatches the release run when the version was never released", ctx do
51+
assert {output, 0} = run(ctx)
52+
assert output =~ "::warning::Release v2.11.0 was missing; dispatched main.yaml"
53+
assert List.last(gh_calls(ctx)) == "workflow run main.yaml --ref main"
54+
end
55+
56+
test "waits while a release-capable run is still in progress", ctx do
57+
assert {output, 0} = run(ctx, [{"FAKE_IN_FLIGHT", "1"}])
58+
assert output =~ "1 release-capable run(s) are in progress; not dispatching"
59+
refute Enum.any?(gh_calls(ctx), &String.starts_with?(&1, "workflow run"))
60+
end
61+
62+
test "fails when the dispatch fails", ctx do
63+
assert {output, 1} = run(ctx, [{"FAKE_DISPATCH_FAILS", "1"}])
64+
assert output =~ "ERROR: release v2.11.0 is missing and dispatching main.yaml failed"
65+
end
66+
67+
test "fails when runs cannot be listed", ctx do
68+
assert {output, 1} = run(ctx, [{"FAKE_RUN_LIST_FAILS", "1"}])
69+
assert output =~ "ERROR: unable to list main.yaml runs"
70+
end
71+
72+
test "fails when the manifest has no version", ctx do
73+
File.write!(ctx.manifest, "{}\n")
74+
75+
assert {output, 1} = run(ctx)
76+
assert output =~ "ERROR: unable to read the version from"
77+
end
78+
79+
defp run(ctx, env \\ []) do
80+
base = [
81+
{"PATH", "#{ctx.bin}:#{System.get_env("PATH")}"},
82+
{"FAKE_GH_LOG", ctx.log},
83+
{"RELEASE_MANIFEST", ctx.manifest}
84+
]
85+
86+
System.cmd(@script, [], env: base ++ env, stderr_to_stdout: true, cd: ctx.dir)
87+
end
88+
89+
defp gh_calls(ctx) do
90+
case File.read(ctx.log) do
91+
{:ok, log} -> String.split(log, "\n", trim: true)
92+
{:error, :enoent} -> []
93+
end
94+
end
95+
96+
defp tmp_dir! do
97+
nonce = :crypto.strong_rand_bytes(16) |> Base.url_encode64(padding: false)
98+
path = Path.join(System.tmp_dir!(), "linear_cli_recover_release_#{nonce}")
99+
File.mkdir!(path)
100+
on_exit(fn -> File.rm_rf!(path) end)
101+
path
102+
end
103+
end

0 commit comments

Comments
 (0)