Skip to content

HBASE-29968 Upgrade hbase-mapreduce to use junit5#7861

Open
liuxiaocs7 wants to merge 1 commit intoapache:masterfrom
liuxiaocs7:HBASE-29968
Open

HBASE-29968 Upgrade hbase-mapreduce to use junit5#7861
liuxiaocs7 wants to merge 1 commit intoapache:masterfrom
liuxiaocs7:HBASE-29968

Conversation

@liuxiaocs7
Copy link
Member

@liuxiaocs7 liuxiaocs7 marked this pull request as draft March 5, 2026 18:08
@liuxiaocs7
Copy link
Member Author

wait #7887

@liuxiaocs7
Copy link
Member Author

wait: #7945

@liuxiaocs7 liuxiaocs7 force-pushed the HBASE-29968 branch 2 times, most recently from 36361af to 3a271d7 Compare March 17, 2026 09:01
@liuxiaocs7 liuxiaocs7 marked this pull request as ready for review March 17, 2026 09:01
@liuxiaocs7
Copy link
Member Author

liuxiaocs7 commented Mar 19, 2026

TestVerifyReplicationAdjunct fails

java.io.IOException: Cluster already running

@liuxiaocs7 liuxiaocs7 marked this pull request as draft March 24, 2026 02:19
@liuxiaocs7 liuxiaocs7 marked this pull request as ready for review March 24, 2026 07:48
htable2 = connection2.getTable(tableName);
}

@BeforeAll
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add this for TestVerifyReplicationAdjunct, will remove in hbase-server upgrading!

Comment on lines 99 to -100
public static void setUpBeforeClass() throws Exception {
TestReplicationBase.setUpBeforeClass();
Copy link
Member Author

@liuxiaocs7 liuxiaocs7 Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will be invoked in parent class TestReplicationBase automatically, will refactor this in hbase-server

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Migrates the hbase-mapreduce module’s test suite from JUnit 4 to JUnit Jupiter (JUnit 5) as part of HBASE-29968, updating annotations/rules/categories/parameterization patterns and aligning build dependencies accordingly.

Changes:

  • Converted many tests from org.junit.* + @Category/@Rule/@RunWith(Parameterized.class) to org.junit.jupiter.* + @Tag, TestInfo, and Jupiter parameterized tests.
  • Replaced JUnit4 rules/runners (e.g., TestName, MockitoJUnitRunner) with Jupiter equivalents (TestInfo, MockitoExtension).
  • Updated hbase-mapreduce/pom.xml test dependencies (Mockito Jupiter integration; removal of Vintage engine).

Reviewed changes

Copilot reviewed 87 out of 87 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationBase.java Adds Jupiter lifecycle annotations alongside JUnit4 to support mixed test hierarchies.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshotV2NoCluster.java Minor formatting change in a Jupiter-tagged test.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/VerifyReplicationTestBase.java Migrates assertions/lifecycle to Jupiter; introduces TestInfo usage.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationZkConnectionUri.java Replaces JUnit4 categories/class rule with Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationZkClusterKey.java Replaces JUnit4 categories/class rule with Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationSecureClusterCredentials.java Migrates to Jupiter parameterized tests (@MethodSource).
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationRpcConnectionUri.java Replaces JUnit4 categories/class rule with Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationRecompareRunnable.java Moves to Jupiter + MockitoExtension; adjusts stubbing with lenient().
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationCrossDiffHdfs.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationAdjunct.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionToolNpeFix.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/regionserver/TestCompactionTool.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALRecordReader.java Migrates lifecycle/assertions/categories; replaces “assertTrue(false)” with fail(...).
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALPlayer.java Migrates lifecycle/categories; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestWALInputFormat.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTimeRangeMapRed.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSplit.java Migrates to Jupiter; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableSnapshotInputFormat.java Migrates to Jupiter; replaces TestName with TestInfo-derived name.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableRecordReader.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableOutputFormat.java Migrates lifecycle/assertions/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceUtil.java Migrates categories/class rule to Jupiter tags + Jupiter assertions.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduceBase.java Migrates lifecycle to Jupiter.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableMapReduce.java Migrates negative tests to assertThrows.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanYZYToEmpty.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanYYYToEmpty.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanYYXToEmpty.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanOPPToEmpty.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanOBBToQPP.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanOBBToOPP.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanEmptyToOPP.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanEmptyToEmpty.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanEmptyToBBB.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanEmptyToBBA.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanEmptyToAPP.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScanBase.java Migrates lifecycle/assertions to Jupiter.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatScan.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormatBase.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestTableInputFormat.java Migrates lifecycle/categories; replaces expected-exception style with assertThrows.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSyncTable.java Migrates lifecycle/categories; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestSimpleTotalOrderPartitioner.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRowCounter.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRoundRobinTableInputFormat.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestRegionSizeCalculator.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultithreadedTableMapper.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormatImpl.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableSnapshotInputFormat.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableInputFormatBase.java Migrates categories; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMultiTableInputFormat.java Migrates categories/lifecycle to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMRIncrementalLoadWithLocality.java Migrates JUnit4 Parameterized runner to Jupiter parameterized test.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestMRIncrementalLoad.java Migrates JUnit4 Parameterized runner to Jupiter parameterized test.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestJarFinder.java Partially migrated (tags) but still uses JUnit4 test API.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsvParser.java Migrates expected-exception tests to assertThrows.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportTsv.java Migrates rules/expected exceptions to Jupiter lifecycle + assertThrows.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestImportExport.java Normalizes Jupiter tag usage via imports.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHashTable.java Migrates lifecycle/categories; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHRegionPartitioner.java Migrates lifecycle/categories; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2WithSecurity.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHFileOutputFormat2.java Migrates assertions; replaces @Ignore with @Disabled.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestHBaseMRTestingUtility.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestGroupingTableMapper.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTableToPeerClusterWithZkUri.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTableToPeerClusterWithRpcUri.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTableToPeerClusterWithClusterKey.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCopyTable.java Migrates lifecycle/categories; replaces TestName with TestInfo plumbing.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestConfigurePartitioner.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TestCellCounter.java Migrates lifecycle/categories; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/TableSnapshotInputFormatTestBase.java Migrates lifecycle/assertions to Jupiter.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/MultiTableInputFormatTestBase.java Migrates lifecycle/assertions to Jupiter.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/MRIncrementalLoadTestBase.java Refactors parameterized setup/teardown to be invoked from Jupiter parameterized tests.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat2TestBase.java Migrates assertions to Jupiter.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/CopyTableToPeerClusterTestBase.java Migrates lifecycle; replaces TestName usage with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapreduce/CopyTableTestBase.java Migrates to Jupiter assertions; replaces TestName usage with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestTableSnapshotInputFormat.java Migrates mapred snapshot tests to Jupiter (incl. @Disabled).
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestTableOutputFormatConnectionExhaust.java Migrates lifecycle/categories to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduceUtil.java Migrates lifecycle/categories/assertions to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestTableMapReduce.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestTableInputFormat.java Migrates lifecycle/categories; replaces expected-exception style with assertThrows.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestSplitTable.java Migrates categories/assertions; replaces TestName with TestInfo.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestRowCounter.java Migrates categories/assertions to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestMultiTableSnapshotInputFormat.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestIdentityTableMap.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestGroupingTableMap.java Migrates categories/assertions to Jupiter tags.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/mapred/TestDriver.java Migrates categories/class rule to Jupiter tags.
hbase-mapreduce/pom.xml Switches Mockito test dependency to Jupiter integration and removes Vintage engine.
Comments suppressed due to low confidence (2)

hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/VerifyReplicationTestBase.java:95

  • VerifyReplicationTestBase declares a static setUpBeforeClass with the same signature as TestReplicationBase.setUpBeforeClass, but it no longer invokes the superclass setup. In JUnit 5, hiding lifecycle methods can prevent the superclass @BeforeAll from being executed, which would leave CONF1/CONF2, connections, and the mini clusters uninitialized before this method uses CONF2.

Suggested fix: either call TestReplicationBase.setUpBeforeClass() as the first statement here, or rename this method (e.g., setUpPeerTable) to avoid hiding the superclass lifecycle method and let JUnit run both setup methods.
hbase-mapreduce/src/test/java/org/apache/hadoop/hbase/replication/TestVerifyReplicationAdjunct.java:100

  • This class defines a static setUpBeforeClass that hides TestReplicationBase.setUpBeforeClass, but it does not invoke the superclass setup. This method immediately uses CONF2 to create a connection, which will fail if the mini clusters/config in TestReplicationBase were not initialized.

Suggested fix: call TestReplicationBase.setUpBeforeClass() first (or rename this method to avoid hiding the superclass lifecycle method and rely on inherited @BeforeAll).


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +344 to 347
@AfterAll
public static void tearDownAfterClass() throws Exception {
htable3.close();
TestReplicationBase.tearDownAfterClass();
}
Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tearDownAfterClass no longer delegates to TestReplicationBase.tearDownAfterClass(). Since this class also hides the superclass method, the mini clusters/connections started by TestReplicationBase may never be shut down, leading to leaked resources and follow-on test failures.

Suggested fix: add TestReplicationBase.tearDownAfterClass() after closing htable3 (or rename this method to avoid hiding and rely on inherited @AfterAll).

Copilot uses AI. Check for mistakes.
Comment on lines 30 to 47
import java.util.jar.JarInputStream;
import java.util.jar.JarOutputStream;
import java.util.jar.Manifest;
import org.apache.hadoop.hbase.HBaseClassTestRule;
import org.apache.hadoop.hbase.testclassification.SmallTests;
import org.junit.Assert;
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.experimental.categories.Category;
import org.junit.jupiter.api.Tag;
import org.slf4j.LoggerFactory;

/**
* This file was forked from hadoop/common/branches/branch-2@1350012.
*/
@Category(SmallTests.class)
@Tag(SmallTests.TAG)
public class TestJarFinder {

@ClassRule
public static final HBaseClassTestRule CLASS_RULE =
HBaseClassTestRule.forClass(TestJarFinder.class);

@Test
public void testJar() throws Exception {

Copy link

Copilot AI Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test still imports and uses JUnit 4 (org.junit.Test / org.junit.Assert). With the removal of the junit-vintage-engine in this module, it will no longer be executed on the JUnit Platform.

Suggested fix: migrate the remaining JUnit 4 annotations/assertions to JUnit Jupiter (e.g., org.junit.jupiter.api.Test and org.junit.jupiter.api.Assertions).

Copilot uses AI. Check for mistakes.
@liuxiaocs7
Copy link
Member Author

liuxiaocs7 commented Mar 24, 2026

master branch has the same result with applying the patch (56+366)

mvn clean install -DskipTests && mvn test -pl hbase-mapreduce -PrunAllTests -Dsurefire.firstPartForkCount=1 -Dsurefire.secondPartForkCount=1
default-test

[INFO] Results:
[INFO] 
[INFO] Tests run: 56, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- surefire:3.5.3:test (secondPartTestsExecution) @ hbase-mapreduce ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO] 
[INFO] Results:
[INFO] 
[WARNING] Tests run: 366, Failures: 0, Errors: 0, Skipped: 12
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 87 out of 87 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@liuxiaocs7 liuxiaocs7 requested a review from Apache9 March 24, 2026 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants