Skip to content

Fail smoke tests when any reflected invocation fails - #99

Closed
krotname wants to merge 1 commit into
mainfrom
codex/fix-smoke-test-harness-failure-handling
Closed

Fail smoke tests when any reflected invocation fails#99
krotname wants to merge 1 commit into
mainfrom
codex/fix-smoke-test-harness-failure-handling

Conversation

@krotname

@krotname krotname commented Aug 1, 2026

Copy link
Copy Markdown
Owner

Motivation

  • The smoke harness previously counted reflective invocation failures but only asserted that at least one invocation was attempted, allowing crashes to be masked by a single successful invocation.
  • The change aims to strengthen the smoke-test signal so that any crashing executable path causes the harness to fail, preserving the original intention of exposing regressions.

Description

  • Change the final check in quality.SmokeMethodTestHarness.verify(...) to Assertions.assertEquals(0, failed, ...) so the harness fails when any reflected invocation throws.
  • Update quality.SmokeMethodTestHarnessTest to assert that a class with any failing executable path triggers an AssertionError, renaming the test to reflect the stricter behaviour.
  • Remove the now-unused assertDoesNotThrow import and keep the existing summary diagnostics in the failure message.

Testing

  • Ran mvn -Dtest=quality.SmokeMethodTestHarnessTest test and the harness regression tests passed (2 tests, 0 failures).
  • Ran mvn test and the full test suite completed successfully (Tests run: 597, Failures: 0, Errors: 0).
  • Ran mvn -DskipTests checkstyle:check and there were no Checkstyle violations (build success).

Codex Task

@krotname

krotname commented Aug 1, 2026

Copy link
Copy Markdown
Owner Author

Superseded by the validated combined merge in #115: #115

@krotname krotname closed this Aug 1, 2026
@krotname
krotname deleted the codex/fix-smoke-test-harness-failure-handling branch August 1, 2026 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant