Skip to content

Removes pauseless FSM scheme#17852

Merged
Jackie-Jiang merged 2 commits intoapache:masterfrom
noob-se7en:remove_scheme_pauseless
Mar 19, 2026
Merged

Removes pauseless FSM scheme#17852
Jackie-Jiang merged 2 commits intoapache:masterfrom
noob-se7en:remove_scheme_pauseless

Conversation

@noob-se7en
Copy link
Copy Markdown
Contributor

@noob-se7en noob-se7en commented Mar 10, 2026

Problem
Server and controller can disagree on the segment commit protocol. The server decides whether to use pauseless or normal commit based on the pauselessConsumptionEnabled flag in StreamIngestionConfig.
But the controller's createFsm() first checks for a segment.completion.fsm.scheme override in the stream config — if someone sets this to "pauseless" without enabling pauselessConsumptionEnabled, the controller creates a PauselessSegmentCompletionFSM while the server does a normal commit.

Solution
Remove the segment.completion.fsm.scheme stream config override from SegmentCompletionManager.createFsm(). The controller now always derives the FSM scheme from PauselessConsumptionUtils.isPauselessEnabled() — the same source of truth the server uses. This eliminates any possibility of mismatch.

Backward Compatibility

Config pauselessEnabled Before (buggy) After (fixed)
scheme=pauseless true Pauseless FSM Pauseless FSM (no change)
scheme=pauseless false Pauseless FSM ❌ Normal FSM ✅
scheme=default true Normal FSM ❌ Pauseless FSM ✅
scheme=default false Normal FSM Normal FSM (no change)
not set true Pauseless FSM Pauseless FSM (no change)
not set false Normal FSM Normal FSM (no change)

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 10, 2026

Codecov Report

❌ Patch coverage is 40.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.28%. Comparing base (58daa99) to head (a939c13).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
.../helix/core/realtime/SegmentCompletionManager.java 40.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17852      +/-   ##
============================================
+ Coverage     63.23%   63.28%   +0.04%     
  Complexity     1466     1466              
============================================
  Files          3190     3190              
  Lines        192009   192003       -6     
  Branches      29412    29411       -1     
============================================
+ Hits         121424   121502      +78     
+ Misses        61063    60987      -76     
+ Partials       9522     9514       -8     
Flag Coverage Δ
custom-integration1 100.00% <ø> (ø)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-11 63.22% <40.00%> (+0.01%) ⬆️
java-21 63.24% <40.00%> (+0.02%) ⬆️
temurin 63.28% <40.00%> (+0.04%) ⬆️
unittests 63.27% <40.00%> (+0.04%) ⬆️
unittests1 55.58% <ø> (+0.02%) ⬆️
unittests2 34.27% <40.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@9aman
Copy link
Copy Markdown
Contributor

9aman commented Mar 11, 2026

@noob-se7en can we update the corresponding docs, if any, an mention about the need for config for older versions.

@9aman
Copy link
Copy Markdown
Contributor

9aman commented Mar 11, 2026

I don't see any backward compatibility issues too as the existing code defaulted to using the flag to figure out the fsmClass.

Copy link
Copy Markdown
Contributor

@9aman 9aman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Jackie-Jiang Jackie-Jiang added cleanup Code cleanup or removal of dead code ingestion Related to data ingestion pipeline real-time Related to realtime table ingestion and serving labels Mar 19, 2026
@Jackie-Jiang Jackie-Jiang merged commit e2882f9 into apache:master Mar 19, 2026
14 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cleanup Code cleanup or removal of dead code ingestion Related to data ingestion pipeline real-time Related to realtime table ingestion and serving

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants