Skip to content

HBASE-30288 Fix WALPlayer bulk output table mappings - #8557

Open
Sigma-Ma wants to merge 1 commit into
apache:masterfrom
Sigma-Ma:HBASE-30288-walplayer-bulk-table-mapping
Open

HBASE-30288 Fix WALPlayer bulk output table mappings#8557
Sigma-Ma wants to merge 1 commit into
apache:masterfrom
Sigma-Ma:HBASE-30288-walplayer-bulk-table-mapping

Conversation

@Sigma-Ma

Copy link
Copy Markdown
Contributor

JIRA: https://issues.apache.org/jira/browse/HBASE-30288

What changes were proposed in this pull request?

This PR makes WALPlayer honor source-to-target table mappings when generating bulk-output HFiles.

The bulk mapper now builds the mapping from TABLES_KEY and TABLE_MAP_KEY, keeps source table names for WAL filtering, and emits mapped target table names in multi-table composite keys. The bulk job is also configured with target table descriptors and region locators.

The existing bulk-load test now uses distinct source and target tables, with the target table pre-split using a different region layout, and verifies that the generated HFiles can be loaded into the mapped target table.

Why are the changes needed?

Before this change, the bulk-output path ignored the target table mappings. WALKeyValueMapper emitted source table names, and createSubmittableJob configured HFileOutputFormat with source table metadata.

As a result, HFiles could be generated under the wrong table path and with region boundaries or table descriptors belonging to the source table, preventing them from being correctly bulk-loaded into the intended target table.

How was this patch tested?

mvn -pl hbase-mapreduce -am \
  '-Dtest=TestWALPlayer#testWALPlayerBulkLoadWithTableMappingAndOverriddenTimestamps' \
  -Dsurefire.failIfNoSpecifiedTests=false \
  test

mvn -pl hbase-mapreduce -am -Dtest=TestWALPlayer -Dsurefire.failIfNoSpecifiedTests=false test

Signed-off-by: mazhengxuan <mazhengxuan@didiglobal.com>
@charlesconnell
charlesconnell requested a review from hgromer August 20, 2026 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant