[fix](fe) Check all aggregate function arguments#65537
Open
morrySnow wants to merge 1 commit into
Open
Conversation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: NormalizeAggregate only checked the first child of each aggregate function for nested aggregate functions. Aggregate functions in later arguments, such as SUM in the ORDER BY argument of GROUP_CONCAT, could bypass analysis validation. Check every aggregate child and add FE unit and regression coverage for the invalid query.
### Release note
Reject nested aggregate functions in every aggregate function argument.
### Check List (For Author)
- Test: Unit Test and Regression test
- Unit Test: NormalizeAggregateTest
- Regression test: query_p0/aggregate/agg_group_concat
- Behavior changed: Yes, invalid nested aggregate functions in non-first arguments are rejected during analysis
- Does this need documentation: No
morrySnow
requested review from
924060929,
englefly and
starocean999
as code owners
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
/review |
Contributor
Author
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
I found one correctness issue in the aggregate-parameter validation.
Critical checkpoints:
- Goal: the PR intends to reject aggregate functions nested in aggregate parameters. It covers the simple non-distinct
group_concat(k order by sum(k))case, but the validation still runs after some aggregate-containing children can be normalized to slots. - Scope/focus: the code change is small, but the affected normalization path has parallel distinct and pushdown-triggered order-key branches.
- Concurrency/lifecycle/config/persistence/FE-BE protocol: not involved.
- Parallel paths: issue found in distinct non-slot children and non-distinct order keys containing pushdown-triggering expressions.
- Tests: the new FE and regression tests use the right expected-error pattern, but they only cover the simple non-distinct order-key case and miss the variants in the inline comment.
- User focus: no additional user-provided focus was supplied.
Validation was static only. I did not run builds or tests because this review prompt forbids builds/code modifications, and this checkout is not worktree-initialized and lacks thirdparty/installed / thirdparty/installed/bin/protoc.
| bottomSlotContext.normalizeToUseSlotRef(SessionVarGuardExpr.getExprWithGuard(aggFuncs)); | ||
| if (normalizedAggFuncs.stream().anyMatch(agg -> !agg.children().isEmpty() | ||
| && agg.child(0).containsType(AggregateFunction.class))) { | ||
| && agg.children().stream().anyMatch(e -> e.containsType(AggregateFunction.class)))) { |
Contributor
There was a problem hiding this comment.
This check still runs too late for children that were put into needPushDownSelfExprs. For example, group_concat(distinct k order by sum(k)) unwraps the order key in the distinct branch and maps sum(k) to a bottom-project alias slot before this block runs; similarly, count(distinct sum(k)) or an order key like sum(k) + row_number() over(...) can be replaced by a slot first. After normalizeToUseSlotRef, the aggregate child no longer contains an AggregateFunction, so this check misses the nested aggregate and may only fail later as an aggregate pushed into the wrong plan node. Please validate the original aggregate children/order-key children for non-window aggregate functions before any slot normalization, and add coverage for the distinct and pushdown-triggered variants.
Contributor
TPC-H: Total hot run time: 29293 ms |
Contributor
TPC-DS: Total hot run time: 180871 ms |
Contributor
ClickBench: Total hot run time: 25.04 s |
Contributor
FE UT Coverage ReportIncrement line coverage |
Contributor
FE Regression Coverage ReportIncrement line coverage |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: NormalizeAggregate only checked the first child of each aggregate function for nested aggregate functions. Aggregate functions in later arguments, such as SUM in the ORDER BY argument of GROUP_CONCAT, could bypass analysis validation. Check every aggregate child and add FE unit and regression coverage for the invalid query.
Release note
Reject nested aggregate functions in every aggregate function argument.
Check List (For Author)