fix(tdigest): merge into existing dest without OVERRIDE#3412
fix(tdigest): merge into existing dest without OVERRIDE#3412Tangruilin wants to merge 4 commits intoapache:unstablefrom
Conversation
|
@jihuayu Hi, this PR is ready for review. Thanks! |
There was a problem hiding this comment.
Pull request overview
Aligns Kvrocks’ TDIGEST.MERGE semantics with Redis when the destination key already exists and OVERRIDE is not specified, by merging destination + sources instead of returning an error.
Changes:
- Update
TDigest::Mergeto include the destination digest’s centroids in the merge set when dest exists withoutOVERRIDE. - Refactor centroid extraction logic into a shared helper (
getCentroidsForMerge) to handle buffered/unbuffered digests consistently. - Extend Go unit tests to validate merging into an existing destination and
OVERRIDEoverwrite behavior.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/gocase/unit/type/tdigest/tdigest_test.go |
Updates merge tests to assert existing-dest merge semantics and override overwrite semantics. |
src/types/redis_tdigest.h |
Declares a new helper for extracting centroids for merge operations. |
src/types/redis_tdigest.cc |
Implements the new merge behavior and the centroid-extraction helper. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@jihuayu "I'm sorry the CI didn't pass. However, it seems the CI format configuration is not using the default configuration. I ran ./x.py check format locally and it passed, but the CI command is ./x.py check format This can be confusing. Below is a screenshot of my local run. Perhaps we should keep the CI consistent with the default configuration and use configuration files instead of command-line
|
089460d to
f8ff985
Compare
|
@Tangruilin Yes, you are right! |
eec4eb0 to
0ffe467
Compare
LindaSummer
left a comment
There was a problem hiding this comment.
Hi @Tangruilin ,
Generally LGTM with some comments. 😊
Could we add some cpp unit tests for this change?
Sure. I will add cpp unit before merge |
|
@Tangruilin It looks like the code incorrectly overrides the COMPRESSION parameter in the following scenario. Could you add more test cases for this? TDIGEST.CREATE src COMPRESSION 200
TDIGEST.ADD src 1
TDIGEST.CREATE dest COMPRESSION 100
TDIGEST.ADD dest 2
TDIGEST.INFO dest
TDIGEST.MERGE dest 1 src
TDIGEST.INFO destThe Redis results are as follows: 127.0.0.1:6379> TDIGEST.INFO dest
1) Compression
2) (integer) 100 |
Done |
WIP: fix TDIGEST.MERGE to match Redis behavior Issue: apache#3410 When destination key already exists without OVERRIDE flag: - Redis: merge dest + sources together - Kvrocks (before): return error Changes: 1. Remove error return when dest exists without OVERRIDE 2. Read dest centroids and add to merge list 3. Add check to prevent duplicate merge if dest in sources 4. Update tests to verify merge and override behavior
|
@jihuayu ji |
jihuayu
left a comment
There was a problem hiding this comment.
It looks OK.
However, the logic in this function is a bit complex, so I’ll need some help from other reviewers.
|
LindaSummer
left a comment
There was a problem hiding this comment.
Hi @Tangruilin ,
LGTM 🚀
With the compression parameter improvement, its behavior should match the Redis' now.
Thanks very much for your effort!
Co-authored-by: Twice <twice@apache.org>





WIP: fix TDIGEST.MERGE to match Redis behavior
Issue: #3410
When destination key already exists without OVERRIDE flag:
Changes: