PYTHON-4232 Add Fuzzing Tests

[blink1073]

PYTHON-4232

Changes in this PR

Adds coverage-guided fuzz tests for bson: each target checks that the C extension and pure-Python implementations agree on arbitrary input, with committed reproducers for known bugs. Runs on every PR plus a daily Evergreen task; no driver source is changed.

Test Plan

• just lint, just lint-manual, and just typing pass.
• Both Evergreen runner paths verified locally; a deliberately broken target fails as expected.
• An Evergreen patch build will confirm real atheris execution (not runnable on local hardware).

Checklist

Checklist for Author

• Did you update the changelog (if necessary)? Internal test infrastructure only, no user-facing change.
• Is there test coverage? This PR adds the fuzz tests themselves and changes no driver source.
• Is any followup work tracked in a JIRA ticket? If so, add link(s). (PYTHON-6169, PYTHON-6178)

Checklist for Reviewer

• Does the title of the PR reference a JIRA Ticket?
• Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
• Is all relevant documentation (README or docstring) updated?

[Copilot]

🟡 Changes recommended

The targets lack BSON parser coverage feedback, miss decoded-value differences, and skip a committed regression input.

7 open findings

• Regression run skips the final corpus input · New
• Coverage run omits the final corpus file · New
• Pure-Python import failures are silently ignored · New
• Pure-Python BSON parser is not instrumented · New
• Decoder comparison ignores document content · New
• Encoder failures are misclassified as decode failures · New
• Decode-all comparison checks length but not document content · New

What changed in this PR

This PR adds BSON fuzz testing to the driver’s test infrastructure without changing driver source code.

Changes:

• Adds three C-versus-pure-Python BSON fuzz targets, a seed corpus builder, and regression inputs.
• Runs regression inputs on PRs and time-boxed fuzzing daily through Evergreen.
• Documents local use and adds the Atheris dependency.

File	Description
uv.lock	Locks Atheris for fuzz testing.
pyproject.toml	Adds the fuzz dependency group and lint exceptions.
fuzz/​README.md	Documents targets, inputs, and commands.
fuzz/​fuzz_is_valid.py	Tests BSON validity across implementations.
fuzz/​fuzz_decode.py	Tests single-document decoding and re-encoding.
fuzz/​fuzz_decode_all.py	Tests multi-document decoding and re-encoding.
fuzz/​build_corpus.py	Builds BSON seed inputs.
CONTRIBUTING.md	Adds contributor instructions for fuzz tests.
AGENTS.md	Describes the new fuzz directory.
.gitignore	Ignores generated fuzz scratch files.
.evergreen/​scripts/​utils.py	Registers the fuzz test category.
.evergreen/​scripts/​setup_tests.py	Prepares the fuzz corpus and environment.
.evergreen/​scripts/​run_tests.py	Runs regression and timed fuzz targets.
.evergreen/​scripts/​generate_config.py	Defines Evergreen fuzz tasks and variants.
.evergreen/​generated_configs/​variants.yml	Adds fuzz build variants.
.evergreen/​generated_configs/​tasks.yml	Adds fuzz tasks.
.evergreen/​generated_configs/​functions.yml	Passes the fuzz time setting.

🧠 Review effort: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The documented fuzz group cannot install on several supported environments, and the seed builder omits valid raw BSON variants.

3 open findings

• Document or support fuzz dependencies across platforms · New
• Coverage run omits the final corpus file
• Regression run skips the final corpus input

5 resolved since last review

• Decode-all comparison checks length but not document content
• Encoder failures are misclassified as decode failures
• Decoder comparison ignores document content
• Pure-Python BSON parser is not instrumented
• Pure-Python import failures are silently ignored

Previously missed (1)

In code that hasn't changed since last review

Include all raw BSON variants as fuzzing seeds

fuzz/​build_corpus.py:86

The builder only adds canonical_bson bytes, although valid spec cases also include degenerate_bson (for example, test/bson_corpus/array.json:18) and converted_bson (test/bson_corpus/undefined.json:11). Encoding their Extended JSON does not retain these distinct wire representations, so nightly fuzzing misses those cases. Add all available raw BSON variants as separate seeds.

🧠 Review effort: Lite

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[Copilot]

🔵 Needs a closer look

Unresolved moderate findings affect fuzz coverage, artifact retention, resource cleanup, and exception handling.

3 open findings

• Document or support fuzz dependencies across platforms
• Coverage run omits the final corpus file
• Regression run skips the final corpus input

Previously missed (1)

In code that hasn't changed since last review

Do not swallow decode exceptions in is_valid

fuzz/​fuzz_is_valid.py:70

is_valid catches every exception and returns False, so a True result guarantees that decode will not raise any exception, not just InvalidBSON. Swallowing ValueError, struct.error, and the other exceptions here means this target will miss a regression where is_valid() returns True but decode() fails with one of them. Let any exception from decode fail the fuzz case (while retaining the diagnostic).

This issue also appears on line 82 of the same file.

🧠 Review effort: Lite

[Copilot]

🟡 Changes recommended

The fuzz runner must invoke targets with the intended uv interpreter.

4 open findings

• Use sys.executable to launch targets with the correct environment · New
• Document or support fuzz dependencies across platforms
• Coverage run omits the final corpus file
• Regression run skips the final corpus input

🧠 Review effort: Lite

[Copilot]

🔵 Needs a closer look

Evergreen fuzz failures do not currently archive fuzz-scratch/ reproducers.

0 open findings

4 resolved since last review

• Use sys.executable to launch targets with the correct environment
• Document or support fuzz dependencies across platforms
• Coverage run omits the final corpus file
• Regression run skips the final corpus input

🧠 Review effort: Lite