Skip to content

HDDS-16629. Allow same-rack placement when no other rack has a writable node - #11404

Draft
peterxcli wants to merge 7 commits into
apache:masterfrom
peterxcli:HDDS-16629
Draft

peterxcli wants to merge 7 commits into
apache:masterfrom
peterxcli:HDDS-16629

Conversation

@peterxcli

@peterxcli peterxcli commented Oct 4, 2026 •

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

This builds on #11403 (HDDS-16628) and includes its commits. To see only what this PR adds, use this comparison.

Take a cluster with two racks and decommission one whole rack. Every container with a replica on that rack needs a new copy, and SCMContainerPlacementRackAware wants that copy on a different rack from the existing replicas. The only other rack is the one going away, so placement fails, the containers stay under-replicated, and the decommission never finishes.

This PR makes two changes:

  • If no usable node turns up on the other racks, placement now falls back to any rack (when fallback is enabled). It already did this when the other racks had no candidates at all; now it also does it when the candidates it tried were all unusable.
  • MisReplicationHandler now checks whether copying the container to the chosen node would actually improve its placement. A copy to a rack that already has a replica doesn't: the container becomes over-replicated, the extra copy is deleted, it is mis-replicated again, and the cycle starts over. In that case the handler now skips the copy.

We found this with the SCM simulation (HDDS-16627).

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16629

How was this patch tested?

Copilot AI balanced review requested due to automatic review settings October 4, 2026 14:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions github-actions Bot added the scm label Oct 4, 2026
# Conflicts:
#	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRackAware.java
# Conflicts:
#	hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/placement/algorithms/SCMContainerPlacementRackAware.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants