Skip to content

Make test_tests_collected_once actually verify coverage#3130

Merged
adamtheturtle merged 4 commits intomainfrom
adamtheturtle/fix-linter-all-tests
Apr 25, 2026
Merged

Make test_tests_collected_once actually verify coverage#3130
adamtheturtle merged 4 commits intomainfrom
adamtheturtle/fix-linter-all-tests

Conversation

@adamtheturtle
Copy link
Copy Markdown
Member

@adamtheturtle adamtheturtle commented Apr 25, 2026

Summary

  • The custom linter test `test_tests_collected_once` was passing vacuously: `capsys` could not see the inner `pytest.main` output, and the nested run was also exiting with `INTERNAL_ERROR` because of `pytest-retry`, so 0 items were ever collected. Replaced the stdout-parsing approach with a small pytest plugin that records collected node IDs directly, and disabled `pytest-retry` in the nested run (matching the existing workaround in `test_ci_patterns_valid`). Removed a stray `breakpoint()` and added `assert all_tests` as a sanity check.
  • Added the missing test files and docs (`ci/test_custom_linters.py`, `tests/mock_vws/test_respx_mock_usage.py`, `tests/mock_vws/test_target_validators.py`, `docs/source/httpx-example.rst`) to the CI matrix in `.github/workflows/test.yml` — these were the real coverage gaps the now-working linter test surfaced.

Test plan

  • `uv run pytest ci/test_custom_linters.py` passes locally.

🤖 Generated with Claude Code


Note

Low Risk
Low risk: changes are limited to CI configuration and a custom linter test that validates pytest collection patterns; main risk is CI runtime/flake if pytest plugin interactions differ across environments.

Overview
Fixes the custom CI linter so it actually validates test coverage across the CI matrix. ci/test_custom_linters.py now collects tests via a lightweight pytest plugin (pytest_itemcollected) instead of parsing captured stdout, disables pytest-retry in the nested pytest.main run to avoid internal errors, and adds a sanity assert all_tests.

Closes CI coverage gaps. The workflow matrix in .github/workflows/test.yml is updated to include previously-missed patterns (additional mock VWS tests, ci/test_custom_linters.py, and the docs/ tree), and pyproject.toml adds pytest_itemcollected to Vulture’s ignore list to avoid false positives.

Reviewed by Cursor Bugbot for commit 413ab1a. Bugbot is set up for automated code reviews on this repo. Configure here.

adamtheturtle and others added 2 commits April 25, 2026 21:16
Use a pytest plugin to record collected node IDs directly instead of
parsing captured stdout, and disable pytest-retry in the nested run so
it does not exit with INTERNAL_ERROR. Also remove a stray breakpoint
and assert that the baseline collection is non-empty.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The matrix in test.yml was missing entries for several test files and
docs, so those tests were never run in CI. Add them now that
test_tests_collected_once actually verifies coverage.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
adamtheturtle and others added 2 commits April 25, 2026 21:37
Replaces the per-file rst entries with a glob. Enable bash globstar
in the workflow run step so the pattern expands, and expand globs in
the linter helper since pytest.main is invoked without a shell.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant