installer: automate checklist via AutoHotkey UI tests#696
Draft
dscho wants to merge 30 commits into
Draft
Conversation
dscho
force-pushed
the
run-remaining-installer-tests-via-autohotkey
branch
19 times, most recently
from
4f325da to
46cc689
Compare
dscho
force-pushed
the
run-remaining-installer-tests-via-autohotkey
branch
9 times, most recently
from
392ef82 to
36d29e6
Compare
Add two checks to run-checklist.sh:
Scan all HTML files in the installed doc directory for raw linkgit:
macros that were not converted to hyperlinks. These occur when the
AsciiDoc source has the macro inside a literal block (indented text
or an explicit .... block), where Asciidoctor does not expand macros
by design. Four files are currently affected (gitformat-commit-graph,
gitformat-index, gitformat-pack, MyFirstContribution) and are
excluded by their exact file:line match. See commits b3ac6e737db8
("doc: fix accidental literal blocks") and 399694384bf9 ("doc:
patch-id: fix accidental literal blocks") in git/git for prior fixes
of the same class of issue. Any new occurrence causes the check to
fail.
Verify that `git help git` opens the correct page by temporarily
configuring a fake browser via browser.fake.cmd that writes the URL
to a temp file. The script then checks that the URL ends in
git.html.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho
force-pushed
the
run-remaining-installer-tests-via-autohotkey
branch
from
90b6818 to
6f0cf3f
Compare
Add LaunchGitBashViaStartMenu() to the library, which presses the Windows key, types "Git Bash", presses Enter, and waits for a new mintty window to appear. This mirrors the way a user would actually launch Git Bash after installation, and also verifies that the installer correctly registered the Start Menu shortcut. The checklist script now performs its first real test: launch Git Bash via the Start Menu, wait for the bash prompt to appear (verified via mintty's HTML buffer export), and then cleanly exit. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The UI tests need a Git repository with enough history to exercise git log, gitk, and other commands that inspect commit history. Rather than shipping a binary bundle, this shell script outputs a git fast-import stream that produces a reproducible repository with 18 commits across two branches (main and feature) including a merge. The fixed committer identity and timestamps ensure the resulting repository is byte-identical on every run. The script accepts an optional --create-test-repo=<dir> argument that initializes the repository and feeds the stream into git fast-import in one step. This is designed for easy invocation from AutoHotkey, which has no built-in way to pipe the stdout of one subprocess into the stdin of another without buffering the entire output first. Without the argument, the stream is written to stdout for use with a manual pipe. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The subsequent checklist tests (prompt with branch, git log, gitk, git gui) all need a Git repository to work with. Rather than relying on a pre-existing repository, the script creates one in /tmp using the generate-test-repo.sh script via RunWaitOne, which runs git.exe with a shell alias outside of any UI. This happens before Git Bash is launched since repo creation is not itself a UI concern. This also hoists the gitExe path check and the scriptDirUnix conversion to the top of the script so they are available for both the repo creation and any later steps that need them. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
After cd-ing into a Git repository, the Git for Windows bash prompt should display the current branch in parentheses (e.g. "(main)"). This test types "cd <test-repo>" into the Git Bash window and verifies via the mintty HTML export that the prompt contains "(main)". The regex uses the AHK "s)" flag (dot-matches-newline) because the prompt string and the dollar-sign prompt marker are on separate lines in the captured buffer. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Run git log in the test repository and verify two things: that the output is colorful (by checking the raw HTML export for fg-color CSS classes that mintty uses for colored text), and that the pager stops after the first page (by detecting the less pager's ":" prompt in the captured text). After the checks, the pager is dismissed by sending "q", and the test proceeds to the next step. Also add a small sleep in CloseMinTTYWindow before looking for the confirmation dialog, giving MinTTY time to display it. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Launch gitk from the Git Bash window (the way a user would) and verify it renders the expected commit graph by analyzing the screenshot structurally rather than comparing pixel-by-pixel against a reference image. Testing on a GitHub Actions Windows Server 2025 runner revealed that the pixel-level appearance of gitk varies too much between Windows versions (Win11 vs Win2025) for a reference-image comparison to work reliably: different window chrome, font metrics, theme colors, and DPI settings produced diff ratios well above any reasonable threshold. The new VerifyGitkLayout() function captures an 80x60 thumbnail and checks structural properties instead: that the window is not blank, that the left columns contain colored graph decorations (chroma > 40), that the center columns contain commit text rows (darker than the white background), that at least 5 such rows exist, and that a selection highlight is present. This is robust across Windows versions while still catching real failures (empty window, broken rendering, missing content). The gitk-reference.png file is removed since it is no longer needed. The exit code is also checked via a shell redirect (`gitk; echo $? >gitk-exit.code`). Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Launch git gui from the Git Bash window and verify it opens without
complaining about a missing repository. The window title is checked
for error strings ("not a git repository", "Error"), and a
downscaled screenshot is compared against a reference image to
confirm the expected layout (menu bar, staging area, commit message
pane). The exit code is verified via a shell redirect, same as the
gitk test.
The same CaptureUntilMatchesReference approach is used, which
retries until the screenshot matches the reference within 15% pixel
difference, handling git gui's progressive rendering and any
blinking cursor in the commit message field.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add functions for capturing terminal output from Windows Terminal, needed for the Git CMD checklist tests. Windows Terminal's exportBuffer action writes the terminal buffer to a file when triggered via a configurable key binding, similar to how MinTTY's Ctrl+F5 exports to HTML. ReadWindowsTerminalExportBufferConfig() parses the user's settings.json (checking all three known locations documented at https://learn.microsoft.com/en-us/windows/terminal/install: the Microsoft Store stable and preview paths, and the unpackaged install path) to find the exportBuffer action's file path and key binding. The test will skip gracefully if this is not configured, just like the MinTTY tests skip when ~/.minttyrc is missing the required settings. WindowsTerminalHotkeyToAHK() converts Windows Terminal key binding notation (e.g. "ctrl+shift+e") to AutoHotkey Send format ("^+e"). CaptureBufferFromWindowsTerminal() and WaitForRegExInWindowsTerminal() mirror the corresponding MinTTY functions, adapted from the MSYS2 runtime's ui-test-library.ahk. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Launch Git CMD via the Start Menu and verify it opens in Windows
Terminal. The test tracks existing CASCADIA_HOSTING_WINDOW_CLASS
windows before typing "Git CMD" into the Start Menu search, then
waits for a new one to appear.
The Git CMD tests are only run if the user's Windows Terminal has
exportBuffer configured with a file path and key binding. If not,
the tests are skipped with an informational message, just like the
Git Bash tests are skipped when ~/.minttyrc is not configured.
The buffer is captured via the user's configured exportBuffer key
binding and verified to contain a CMD prompt ("> " at end of line).
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Extend the Git CMD section to cover all three remaining checklist items: git log is colorful and paged, gitk runs and shows history, and git gui runs without error. The git log color check uses a screenshot comparison against a reference image since Windows Terminal's exportBuffer output is plain text with ANSI escape sequences stripped. The pager check still uses the text buffer (looking for the ":" prompt). For gitk, the test verifies that the CMD prompt returns immediately after typing "gitk" (confirming gitk.exe is a GUI wrapper that does not block the terminal), then waits for the Tk window and compares its screenshot against the same reference image used by the Git Bash gitk test. The git gui test similarly launches and verifies via screenshot comparison, reusing the Git Bash reference image. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Accept --git-bash, --git-cmd, and --git-gui arguments to run only specific phases of the checklist. With no arguments all phases run, preserving backward compatibility. This makes iterating on a single failing phase much faster since the other phases can be skipped. MinTTY prerequisites are only checked when --git-bash is active, and the export file path is only resolved in that case. This allows running --git-gui or --git-cmd alone without needing ~/.minttyrc. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Add the reference image for the Git GUI chooser dialog, which is the
window that appears when launching Git GUI from the Start Menu
(without being inside a repository). The test uses this to verify
the chooser rendered correctly before clicking the recent repo link.
The screenshot was generated with this AutoHotkey script:
#Requires AutoHotkey v2.0
#Include D:\git-sdk-64\usr\src\build-extra\installer\ui-tests\ui-test-library.ahk
SetLogFile(EnvGet('TEMP') . '\gen-chooser-ref.log')
gitExe := 'C:\Program Files\Git\cmd\git.exe'
testRepoWin := 'C:\Users\johasc\AppData\Local\Temp\git-bash-checklist-test-repo'
testRepoUnix := StrReplace(testRepoWin, '\', '/')
; Backup and replace .gitconfig
gitconfigPath := EnvGet('USERPROFILE') . '\.gitconfig'
gitconfigBackup := gitconfigPath . '.ui-test-backup'
FileMove(gitconfigPath, gitconfigBackup)
OnExit((*) => (FileExist(gitconfigBackup) && (FileDelete(gitconfigPath), FileMove(gitconfigBackup, gitconfigPath))))
FileAppend('[gui]`n`trecentrepo = ' testRepoUnix '`n', gitconfigPath)
chooserHwnd := LaunchViaStartMenu('Git GUI', 'TkTopLevel', 'Git Gui')
WinMove(100, 100, 500, 400, 'ahk_id ' chooserHwnd)
WinActivate('ahk_id ' chooserHwnd)
Sleep 5000
outFile := 'D:\git-sdk-64\usr\src\build-extra\installer\ui-tests\git-gui-chooser-reference.png'
CaptureAndDownscaleWindow(chooserHwnd, 80, 60, outFile)
FileAppend("Saved: " FileGetSize(outFile) " bytes`n", "*")
WinClose('ahk_id ' chooserHwnd)
Sleep 500
FileDelete gitconfigPath
FileMove gitconfigBackup, gitconfigPath
The heavy downscale from the physical window size (625x500 at 125%
DPI) to 80x60 averages away pixel-level differences between DPI
settings, making the reference usable across machines. The test uses
a 20% diff threshold for this particular comparison to provide
additional margin.
Assisted-by: Claude Opus 4.6
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
At the end of run-checklist.sh, after all the headless checks pass, look for AutoHotkey in PATH and check whether ~/.minttyrc has the required KeyFunctions and SaveFilename settings for the MinTTY buffer export. If both prerequisites are met, run the AHK checklist script that verifies Git Bash starts, the prompt shows the branch, git log is colorful and paged, gitk shows history, and git gui runs without error. If either prerequisite is missing, print an informational message suggesting the user follow checklist.txt for manual verification. The check never fails for missing prerequisites so that the existing headless tests remain usable everywhere. Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Install AutoHotkey v2 and configure ~/.minttyrc with the required KeyFunctions and SaveFilename settings before running run-checklist.sh in both validate steps. This enables the automated UI tests (Git Bash checklist) to run in CI. The AutoHotkey download is cached to avoid re-downloading on every run. Windows Terminal is not available on GitHub Actions runners, so the Git CMD tests (which require Windows Terminal's exportBuffer) will skip gracefully with an informational message. The Git GUI standalone test (which launches via the Start Menu) will also be skipped in CI because GitHub Actions runners do not have the Git for Windows Start Menu shortcut installed (the installer runs in silent mode without desktop integration). Assisted-by: Claude Opus 4.6 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The `windows-latest` runner image installs Git for Windows with `Bash Terminal Option: ConHost` in `install-options.txt` (the packer image build chose this; see e.g. https://github.com/actions/runner-images/blob/main/images/windows/scripts/build/Install-Git.ps1). InnoSetup's `ReplayChoice` function reads these previous settings during `/VERYSILENT` upgrades, so our own `/VERYSILENT` reinstall keeps ConHost as the terminal default instead of MinTTY. That breaks the UI tests for Git Bash because `git-bash.exe` then opens in a console window, not a MinTTY one. Uninstall the pre-installed Git first so the subsequent install starts from a clean slate and applies our defaults (MinTTY). Assisted-by: Claude Opus 4.7 (Extra high reasoning)(Internal only) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On the headless `windows-latest` runner the Explorer shell starts without its taskbar / Start Menu fully initialised, so `LaunchVia StartMenu` (Win key + type + Enter) fails to bring up the Start Menu search and the UI tests time out. Killing `explorer.exe` lets Windows re-spawn it cleanly with the full shell UI, which then lets the search box accept input. This may not be strictly necessary on every future runner image; if the tests start passing without it, the restart can simply be removed. Assisted-by: Claude Opus 4.7 (Extra high reasoning)(Internal only) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Added during the CI bring-up to verify that the AutoHotkey interpreter's `FileAppend ..., '*'` writes a usable byte to the log file when invoked from MSYS2 bash. That suspicion turned out to be wrong (the missing pieces were `/ErrorStdOut /force` plus a piped consumer), so the smoke-test is no longer needed. The log file rename to `git-bash-checklist-stdout.log` is left over from the same debugging session and should be reverted before landing. Assisted-by: Claude Opus 4.7 (Extra high reasoning)(Internal only) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The previous test compared git gui screenshots against a single reference image (git-gui-reference.png), which produced false negatives across Windows versions where Tk themes, font rendering, and DPI differ subtly. Switch to a structural verifier that downscales the window to a small thumbnail and looks for the characteristic layout: a colored staging area (yellow/green hues), a light diff/commit area, and enough non-blank rows overall. The window title is also checked for "Error" or "not a git repository", so layout problems are surfaced immediately rather than after a timeout. Apply the new VerifyGitGuiLayout to both "git gui" call sites (Git Bash and Git CMD flows), and remove the now unused git-gui-reference.png. Assisted-by: Claude Opus 4.7 (Extra high reasoning) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho
force-pushed
the
run-remaining-installer-tests-via-autohotkey
branch
from
5ff8e82 to
e92365a
Compare
Exercise the chooser dialog that git gui opens when launched outside of a repository: write a temporary .gitconfig that lists the test repo as the only recent repository, launch Git GUI from the Start Menu, verify the chooser appears with the right title, then click the recent-repo link to open the repo. The chooser content cannot be verified by pixel comparison because it depends on the user's recent-repo list and Tk theme. Instead, capture the chooser at full window size and scan for the row containing the blue link text via FindChooserLinkRow, then click the center of that row in window-relative coordinates. Once the repo window opens, run the same structural git gui verification as the other phases. On CI runners the Start Menu can become unresponsive after the first test cycle, so restart Explorer before this phase to get a fresh shell UI. Back up the user's existing .gitconfig if present, and restore (or remove) it on exit so the test leaves no trace. Drop the now unused git-gui-chooser-reference.png. Assisted-by: Claude Opus 4.7 (Extra high reasoning) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Diagnostic noise accumulated while chasing the Git GUI chooser-click
failure on CI:
- cat install-options.txt after the installer run.
- Info() dumps of scriptDirUnix, testRepoWin, generate-test-repo
output, AHK TEMP/USERPROFILE, git.exe path/exec-path/ls /tmp/.
- In-bash diagnostic dump that probes TEMP/TMP, cygpath -aw /tmp,
and presence of testRepoWin from inside the mintty session.
- LogAllWindows() + CaptureScreen() calls around the CI Explorer
restart and before the Git GUI launch.
- In LaunchViaStartMenu, an active-window dump + LogAllWindows +
start-menu-failure.png capture when the Start Menu fails to appear.
- CaptureScreen + LogAllWindows on chooser click failure.
- Click-retry loop trying y+5, y+10, y+15, y-5 if the detected link
row doesn't open the repo, and a Tab+Enter keyboard fallback.
- CaptureScreen() and LogAllWindows() helper functions in the library.
Drop before merging once the chooser test is reliably passing.
Assisted-by: Claude Opus 4.7 (Extra high reasoning)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When the UI tests fail, install and run Copilot CLI in --yolo mode with a diagnostic prompt that instructs it to write a standalone AHK script exercising multiple click strategies on the Git GUI chooser window, then report findings to copilot-diagnosis.md. The entire working directory and Copilot session state are uploaded as artifacts for post-mortem inspection.
…link FindChooserLinkRow restricted its blue-pixel scan to the lower half of the chooser screenshot, on the assumption that the "Open Recent Repository" link always sits below the vertical midpoint of the window. On at least one CI runner image the link is rendered at y=173 in a 500x400 capture, well above the y=200 cut-off. The function then returns the first stray "blue" row it finds further down, which on the runner is bleed-through pixels from windows behind the chooser leaking into the BitBlt capture from the screen DC. Clicking that y lands on dead space, no <Button-1> event fires inside the Tk text widget, and the test hangs waiting for a new TkTopLevel window. Scan the whole image instead, group contiguous blue rows into bands, and return the last band whose peak blue-pixel count is at least 20. Real link bands have dozens to hundreds of blue pixels per row, while bleed-through bands have only a handful, so a noise threshold cleanly separates the two. Picking the last qualifying band selects the recent- repo link (which is below the Create/Clone/Open links and other text elements that also use blue). Diagnosed by a Copilot CLI session running on the affected runner; the full diagnosis is preserved in copilot-session.tar.gz on the failing CI run.
The previous prompt was hard-wired for the chooser-click bug: it told Copilot exactly what was suspected (Tk Win32 event delivery), instructed it to write a click-strategies experiment, and listed line numbers that have since moved. With the click bug fixed, the next failure mode is a script hang after the test passes (e.g. an uncaught FileMove exception that opens an invisible error dialog on the headless runner). Rewrite the prompt as a general "the test hangs or fails, diagnose" playbook: read the AHK log to find the last logged line, locate that line in the source, follow control flow to identify what runs next, enumerate common AHK v2 hang causes (uncaught exceptions, modal dialogs, WinWait without timeout), verify with a small standalone diagnostic script wrapped in try/catch, and write the diagnosis with file:line root cause, evidence, a unified diff, and a verification step to copilot-diagnosis.md. Assisted-by: Claude Opus 4.7 (Extra high reasoning) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The preceding TO-DROP commit narrowed the matrix to x86_64 to speed up iteration on the installer UI tests on the more common architecture. Now that those tests pass on x86_64, swap the matrix to aarch64 to exercise the same path on a Windows 11 ARM64 runner. The `owenthereal/action-upterm` action depends on MSYS2 being present on the runner so it can install `tmux` via `pacman`. On `windows-11-arm` GitHub-hosted runners, MSYS2 is not pre-installed (the same workaround is documented in the upterm action's own manual-test workflow at https://github.com/owenthereal/action-upterm/blob/master/.github/workflows/manual-test.yml). Add a conditional `msys2/setup-msys2@v2` step ahead of the upterm step on ARM64 only, using `msystem: CLANGARM64` and `location: C:\.\` (the non-trivial location is required to defeat the setup-msys2 action's sanity check that incorrectly believes an MSYS2 already exists at `C:\msys64` on this runner image). AutoHotkey itself has no native ARM64 build as of v2.0.26 (verified by listing the contents of the release zips: only `AutoHotkey32.exe` and `AutoHotkey64.exe` are shipped). The x64 binary will run under Windows 11 ARM64's x64 emulation layer; if that proves unworkable for the UI tests, this is the first place to revisit. Assisted-by: Claude Opus 4.7 (Extra high reasoning) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
On GitHub-hosted windows-11-arm runners the desktop has been observed
to be stuck in the Out-Of-Box Experience flow when the runner agent
comes up; the screenshot captured at the first failure on this branch
shows the "Choose privacy settings for your device" page occluding
everything. LaunchViaStartMenu() then has nothing to send {LWin} to
and fails with "Start Menu did not appear".
Reboot is not an option: GitHub-hosted runners are ephemeral; the
runner agent is killed along with the OS.
Add a DismissOOBE() helper to ui-test-library.ahk that escalates
through three phases: (1) keyboard walkthrough with {Enter} (every
page's default button is Next/Accept), (2) WinClose on the
Shell_OOBEProxy HWND (which on this image is hosted by explorer.exe
itself), and (3) stopping the known OOBE host processes when the
previous phases were insufficient. Diagnostic screenshots are
captured at each phase boundary so the Copilot CLI failure-debug
step has actionable evidence when dismissal is not enough.
Detection is done by FindOOBEWindow() (which scans WinGetList() for
the Shell_OOBEProxy class or known OOBE process names) and by
IsOOBEActive() (which additionally checks whether the foreground
active window belongs to an OOBE process). The latter is needed
because on windows-11-arm the actual OOBE WebView lives in
WWAHost.exe with a cloaked Windows.UI.Core.CoreWindow that
WinGetList() never sees; after WinClose removes Shell_OOBEProxy,
FindOOBEWindow() returns 0 while the cloaked CoreWindow still holds
foreground focus and swallows {LWin}. The OOBE process list is
factored into OOBEProcessNames() to avoid duplicating it across
FindOOBEWindow() and the Phase 3 kill loop.
Call DismissOOBE() once at the top of git-bash-checklist.ahk, right
after prerequisite validation, so all three phases (Git Bash, Git
CMD, Git GUI) benefit from a clean shell.
Assisted-by: Claude Opus 4.6 (1M context)
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
dscho
force-pushed
the
run-remaining-installer-tests-via-autohotkey
branch
from
e92365a to
ddad9cf
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every Git for Windows release requires a manual checklist of UI verification steps: launch Git Bash, confirm the prompt shows the branch name, verify that
git logis colorful and paged, check thatgitkandgit guistart correctly, and so on for Git CMD and the standalone Git GUI. This has been a tedious, error-prone process for years, and one I always itched to automate.The MSYS2 runtime repository already has AutoHotkey v2-based UI tests (
background-hook.ahk,ctrl-c.ahk,keystroke-order.ahk, etc.) that exercise MinTTY and Windows Terminal interactions. Those tests demonstrated that AHK v2 is a viable tool for automated UI testing on Windows, even for applications like MinTTY that do not expose their content through standard Win32 accessibility APIs.This PR applies the same approach to the installer checklist. It automates all of the manual verification steps listed in
installer/checklist.txt:git logis colorful and paged,gitkruns,git guiruns,git help gitopens the right page, and the existingrun-checklist.shtests pass.git logis colorful and paged,gitkruns,git guiruns.The tests are integrated into
run-checklist.sh: when AutoHotkey is available and the MinTTY prerequisites are configured, the UI tests run automatically; otherwise they skip gracefully with an informational message. This keepsrun-checklist.shworking in headless/CI environments exactly as before.A few design choices worth calling out:
gitk,git gui) do not expose their content via Win32 APIs, so the tests use screenshot-based verification with heavily downscaled reference thumbnails (80x60). This keeps comparisons robust across minor rendering differences while still catching real problems (wrong window, missing content).git fast-import, so they do not depend on any pre-existing checkout.ui-test-library.ahkis adapted from the MSYS2 runtime's version for the installer's specific needs (MinTTY buffer capture, screenshot comparison, Start Menu launching).