Skip to content

Refactor validator return type to support warnings via ValidationResult wrapper (#1479) - #1480

Merged
rly merged 23 commits into
hdmf-dev:devfrom
sejalpunwatkar:feature/1479-validation-result
Sep 4, 2026
Merged

Refactor validator return type to support warnings via ValidationResult wrapper (#1479)#1480
rly merged 23 commits into
hdmf-dev:devfrom
sejalpunwatkar:feature/1479-validation-result

Conversation

@sejalpunwatkar

Copy link
Copy Markdown
Contributor

Motivation

Addresses #1479

This PR implements Phase 1 (the foundational plumbing) to introduce ValidationWarning infrastructure without breaking the existing error reporting model. This change ensures that downstream packages (like PyNWB and NWB Inspector) can eventually consume warning-level messages without modifying their immediate logic or incorrectly marking files as invalid.

How to test the behavior?

  1. Added Core Types: Introduced ValidationWarning class as a sibling to Error inside errors.py, preserving the matching internal shape (.name, .reason, .location).
  2. Built the Smart Container: Implemented the ValidationResult wrapper class in errors.py with custom Python overrides (__bool__, __len__, __iter__, __getitem__) pointing explicitly to the errors bucket to ensure backward compatibility.
  3. Wired the Plumbing: Updated ValidatorMap.validate() inside validator.py to capture the errors_list, wrap it into the new ValidationResult container with an empty warnings container (warnings=[]), and adjusted its strict @docval return type check to rtype=(list, ValidationResult).

Checklist

  • Did you update CHANGELOG.md with your changes?
  • Does the PR clearly describe the problem and the solution?
  • Have you reviewed our Contributing Guide?
  • Does the PR use "Fix #XXX" notation to tell GitHub to close the relevant issue numbered XXX when the PR is merged?

@codecov

codecov Bot commented May 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 85.71429% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.15%. Comparing base (7c03ebf) to head (b96980b).

Files with missing lines Patch % Lines
src/hdmf/validate/validator.py 81.48% 3 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev    #1480      +/-   ##
==========================================
- Coverage   93.18%   93.15%   -0.04%     
==========================================
  Files          41       41              
  Lines       10269    10296      +27     
  Branches     2126     2128       +2     
==========================================
+ Hits         9569     9591      +22     
- Misses        421      424       +3     
- Partials      279      281       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread src/hdmf/validate/errors.py Outdated
Comment thread src/hdmf/validate/errors.py
Comment thread src/hdmf/validate/validator.py Outdated
@rly

rly commented May 30, 2026

Copy link
Copy Markdown
Contributor

Thanks! Looks good overall. Could you please add some basic tests of ValidationResult:

  • validate() returns a ValidationResult with .warnings == [].
  • Backward compatibility: bool, len, iteration, indexing all reflect errors only.
  • Construct ValidationResult(warnings=[w]) and assert the warning survives.

Could you please also add a changelog entry that points to this PR?

Comment thread src/hdmf/validate/errors.py Outdated
@sejalpunwatkar
sejalpunwatkar force-pushed the feature/1479-validation-result branch 2 times, most recently from 3a072bb to 669261e Compare June 2, 2026 11:09
@sejalpunwatkar
sejalpunwatkar force-pushed the feature/1479-validation-result branch from 9d3ea64 to 88bb8e9 Compare June 2, 2026 11:13
@rly

rly commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

@sejalpunwatkar Thanks for your work on this. Let me know when it is ready for re-review.

@sejalpunwatkar

Copy link
Copy Markdown
Contributor Author

Hi @rly, the PR is now ready for your re-review!

I have fully addressed your feedback from the initial code review:

  1. Bug Fix: Fixed the initialization logic typo in ValidationResult.__init__ (warnings vs errors).
  2. Ecosystem Safety: Added the new classes to __all__ in errors.py.
  3. Type-Checking & Docs: Tightened @docval in validator.py to strictly enforce returning ValidationResult, and updated the corresponding docstrings and imports in src/hdmf/common/__init__.py.
  4. Enhanced Diagnostics: Added a human-readable __repr__ method to ValidationResult.
  5. Testing & Coverage: Added comprehensive unit tests at the bottom of test_validate.py covering the wrapper construction, magic methods, and empty state evaluations. 100% of the test suite is passing cleanly.
  6. Changelog: Added a release tracking entry under the ## HDMF 5.0.0 (Upcoming) section in CHANGELOG.md.

Comment thread src/hdmf/common/__init__.py Outdated
Comment thread src/hdmf/common/__init__.py Outdated
Comment thread CHANGELOG.md Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread tests/unit/validator_tests/test_validate.py Outdated
Comment thread src/hdmf/validate/errors.py Outdated
@rly

rly commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Thanks @sejalpunwatkar . This is looking much better. I have a few more comments and suggestions above

@sejalpunwatkar
sejalpunwatkar force-pushed the feature/1479-validation-result branch from 5e33150 to 964acbf Compare June 17, 2026 11:12
@sejalpunwatkar

Copy link
Copy Markdown
Contributor Author

Hi @rly, I've addressed all your feedback: fixed the changelog section, updated the docstrings, corrected the test indentation, implemented the missing test, and updated the ValidationWarning.eq check. Ready for review!

Comment thread src/hdmf/common/__init__.py Outdated
rly and others added 2 commits July 23, 2026 17:34
Change "Validate an file" to "Validate a file" per code review suggestion.
@sejalpunwatkar

Copy link
Copy Markdown
Contributor Author

Thanks for catching that, @rly ! Fixed now.

sejalpunwatkar and others added 8 commits August 2, 2026 05:00
…ning

- Error and ValidationWarning now share __init__, properties, __str__,
  __hash__, and __eq__ logic via a common ValidationIssue base class
- Error.__eq__ now correctly checks type, matching ValidationWarning
- Add test confirming Error and ValidationWarning with same fields are not equal
- Update changelog entry and placement
- Move ValidationResult into validator module
- Add TestValidate for testing a clean validation with no warnings
- Refactor TestValidationResultWrapper
@rly

rly commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Thanks for your patience and contributions @sejalpunwatkar ! I made a few minor adjustments and this is now good to go.

@rly
rly merged commit d8dcae3 into hdmf-dev:dev Sep 4, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants