HDDS-15969. Prioritize Ratis Streaming capable DataNodes during Pipeline creation with graceful fallback

[chungen0126]

What changes were proposed in this pull request?

During rolling upgrades or in heterogeneous clusters, random node selection makes forming a 100% Ratis Streaming pipeline rare (e.g., (1/2)^3 = 12.5% chance when 50% nodes support it), leading to frequent client fallbacks.

To maximize streaming pipeline creation without sacrificing availability, we introduce a two-phase node selection strategy for THREE-factor pipelines:

1. Strict Phase: Exclude nodes lacking Ratis Streaming support (along with over-utilized nodes) and attempt allocation.
2. Fallback Phase: If allocation fails due to insufficient streaming nodes, fall back to standard allocation with relaxed capability checks.

This prioritizes Ratis Streaming nodes when available while ensuring full cluster fallback resilience.

Key Implementation Details:

• Refactored filterPipelineEngagement() to filterNodes(boolean filterRatisStreaming) to check for DatanodeDetails.Port.Name.RATIS_DATASTREAM.
• Safe handling of currentExcluded lists to avoid polluting original excludedNodes during the fallback attempt.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15969

How was this patch tested?

Added new unit tests in TestRatisPipelineProvider:

• testCreatePipelinePrioritizesRatisStreamingNodes: Verifies that SCM strictly selects DataNodes with RATIS_DATASTREAM when sufficient nodes exist.
• testCreatePipelineFallsBackWhenNotEnoughRatisStreamingNodes: Verifies that SCM gracefully falls back and includes non-streaming DataNodes when streaming-capable nodes are fewer than 3.

ci: https://github.com/chungen0126/ozone/actions/runs/30141420657

[chungen0126]

@yandrey321 PTAL

[szetszwo]

@chungen0126 , thanks for working on this! Please see if the suggested change would work.

[jojochuang]

This is fine. A little rewrite can make the code more self explanatory. The pipeline choosing algorithm could end up becoming too complex to maintain.

case THREE:
  dns = chooseThreeFactorDatanodes(excludedNodes, favoredNodes);
  break;

and then:

private List<DatanodeDetails> chooseThreeFactorDatanodes(
    List<DatanodeDetails> excludedNodes, List<DatanodeDetails> favoredNodes) throws IOException {

  List<DatanodeDetails> strictExcluded = new ArrayList<>(excludedNodes);
  strictExcluded.addAll(filterNodes(true));

  try {
    return placementPolicy.chooseDatanodes(
        strictExcluded, favoredNodes, 3, minRatisVolumeSizeBytes, containerSizeBytes);
  } catch (SCMException e) {
    List<DatanodeDetails> relaxedExcluded = new ArrayList<>(excludedNodes);
    relaxedExcluded.addAll(filterNodes(false));
    return placementPolicy.chooseDatanodes(
        relaxedExcluded, favoredNodes, 3, minRatisVolumeSizeBytes, containerSizeBytes);
  }
}

[Copilot]

Pull request overview

Note

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Implements a two-phase datanode selection strategy for 3-way Ratis pipelines to prefer RATIS_DATASTREAM-capable nodes and gracefully fall back to standard selection when streaming-capable nodes are insufficient.

Changes:

• Add strict-then-fallback node selection in RatisPipelineProvider for factor THREE pipelines based on RATIS_DATASTREAM port presence.
• Refactor node exclusion filtering to support optional RATIS_DATASTREAM capability filtering.
• Add unit tests to validate prioritization and fallback behavior.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java	Adds two-phase placement logic and refactors exclusion filtering to optionally require RATIS_DATASTREAM.
hadoop-hdds/server-scm/src/test/java/org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineProvider.java	Adds tests and test utilities to verify streaming-capable prioritization and fallback behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.