Conversation
…er as disabled
## Proposed changes
Issue Number: close #<ISSUE_ID>
Problem:
RuntimeFilter::serialize() marks the outgoing filter as `disabled` for any
state other than READY, so a filter that is merely UNINITED (no filter
content produced yet) is broadcast to consumers as `disabled`. On the
consumer side, `disabled` is an absorbing state:
RuntimeFilterWrapper::signal()
-> set_state(DISABLED, "get disabled from remote")
so the consumer permanently gives up the filter and the probe side degrades
to a full scan, even though nothing actually disabled the filter.
The merger decides readiness only by producer count:
RuntimeFilterMerger::merge_from()
*ready = (_received_producer_num == _expected_producer_num)
which is independent of whether any producer really produced filter content.
When all producers report without content, the adopted wrapper is still
UNINITED while merge_from() already returns ready == true, and the caller
publishes it as disabled.
What this PR does:
1. be/src/exec/runtime_filter/runtime_filter.h
In serialize(), only State::DISABLED sets `disabled=true`. State::UNINITED
is left as "publish nothing": DCHECK in debug builds plus a WARNING log in
release builds, and return OK. This keeps the worst case as "the filter is
lost and the probe side falls back to a full scan", instead of returning
InternalError and failing the whole query.
2. be/src/exec/runtime_filter/runtime_filter_merger.h
Add is_wrapper_uninited() helper to tell "collected all products by count
but there is no real filter content (nor a real DISABLED)" from a really
disabled filter.
3. be/src/exec/runtime_filter/runtime_filter_producer.cpp
In publish()/do_merge, skip sending (local and remote targets) when the
merged wrapper is still UNINITED.
4. be/src/exec/runtime_filter/runtime_filter_mgr.cpp
In RuntimeFilterMergeControllerEntity::_send_rf_to_target(), skip the
broadcast when the merged wrapper is still UNINITED, so consumers keep
waiting until runtime_filter_wait_time_ms and then time out normally.
This aligns with the legacy (branch-3.1) runtime filter semantics: only a
really disabled filter (reach max_in_num / join spill / rpc error) is
published as disabled; a filter that is simply not ready must not be.
## Further comments
No user-visible interface or configuration change. In the worst case a
runtime filter is dropped and the query falls back to a full scan, which is
the same behavior as a filter timeout, and strictly better than the current
behavior of permanently disabling the filter on the consumer side.
lide-reed
requested review from
Gabriel39,
airborne12,
csun5285,
eldenmoon,
gavinchou,
hello-stephen,
liaoxin01,
luwei16,
morningman and
yiguolei
as code owners
September 15, 2026 06:48
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
Fix #67997
Problem
RuntimeFilter::serialize()marks the outgoing filter asdisabledfor any state otherthan
READY:UNINITEDmeans "no filter content has been produced yet", not "this filter wascancelled", but the consumer cannot distinguish the two. It treats
disabledas anabsorbing state (
set_state(DISABLED, "get disabled from remote")) and permanently givesup the filter, so the probe side degrades to a full scan.
The merger decides readiness only by producer count, which is independent of whether any
producer really produced content:
So when all producers report without content,
merge_from()returnsready == truewhilethe merged wrapper is still
UNINITED, and the caller publishes it asdisabledto everyconsumer.
Fix
Align with the legacy (branch-3.1) semantics: only a really disabled filter
(
max_in_num/ join spill / rpc error ->State::DISABLED) is published asdisabled; afilter that is simply not ready is skipped and consumers wait until
runtime_filter_wait_time_ms.runtime_filter.hserialize(): onlyDISABLEDsetsdisabled=true;UNINITEDpublishes nothing (DCHECK + WARNING +OK)runtime_filter_merger.his_wrapper_uninited()helperruntime_filter_producer.cppdo_merge: skip send when merged wrapper isUNINITEDruntime_filter_mgr.cpp_send_rf_to_target(): skip broadcast when merged wrapper isUNINITEDWhy
UNINITEDreturnsOKinstead ofInternalErrorThe old behavior for
UNINITEDwas a silent performance degradation (filter disabled),not an error. Turning it into
InternalErrorwould escalate a performance issue into aquery failure. Publishing nothing keeps the failure mode conservative: the filter is lost,
the consumer times out, and at worst the probe side falls back to a full scan.
DCHECKiskept so unexpected paths still fail fast in debug builds.
Checklist
Further comments
Silent performance regression is hard to notice in production; the fix restores the
previous (branch-3.1) semantics and makes the worst case a filter timeout rather than a
permanently disabled filter.