Make test_tests_collected_once actually verify coverage#3130
Merged
adamtheturtle merged 4 commits intomainfrom Apr 25, 2026
Merged
Make test_tests_collected_once actually verify coverage#3130adamtheturtle merged 4 commits intomainfrom
adamtheturtle merged 4 commits intomainfrom
Conversation
Use a pytest plugin to record collected node IDs directly instead of parsing captured stdout, and disable pytest-retry in the nested run so it does not exit with INTERNAL_ERROR. Also remove a stray breakpoint and assert that the baseline collection is non-empty. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The matrix in test.yml was missing entries for several test files and docs, so those tests were never run in CI. Add them now that test_tests_collected_once actually verifies coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the per-file rst entries with a glob. Enable bash globstar in the workflow run step so the pattern expands, and expand globs in the linter helper since pytest.main is invoked without a shell. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Test plan
🤖 Generated with Claude Code
Note
Low Risk
Low risk: changes are limited to CI configuration and a custom linter test that validates pytest collection patterns; main risk is CI runtime/flake if pytest plugin interactions differ across environments.
Overview
Fixes the custom CI linter so it actually validates test coverage across the CI matrix.
ci/test_custom_linters.pynow collects tests via a lightweight pytest plugin (pytest_itemcollected) instead of parsing captured stdout, disablespytest-retryin the nestedpytest.mainrun to avoid internal errors, and adds a sanityassert all_tests.Closes CI coverage gaps. The workflow matrix in
.github/workflows/test.ymlis updated to include previously-missed patterns (additional mock VWS tests,ci/test_custom_linters.py, and thedocs/tree), andpyproject.tomladdspytest_itemcollectedto Vulture’s ignore list to avoid false positives.Reviewed by Cursor Bugbot for commit 413ab1a. Bugbot is set up for automated code reviews on this repo. Configure here.