fix(index): keep scalar indexes on a LIKE with a mixed string literal - #9054
Open
wombatu-kun wants to merge 1 commit into
Open
fix(index): keep scalar indexes on a LIKE with a mixed string literal#9054wombatu-kun wants to merge 1 commit into
wombatu-kun wants to merge 1 commit into
Conversation
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PgoMt71tk3ZDbctk73ggx2
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The planner now preserves scalar-index discovery only across the value- and ordering-preserving Utf8, LargeUtf8, and Utf8View casts, then normalizes LIKE bounds to the indexed type before parser dispatch. Non-string casts still fall back to refinement, and exact BTree paths cannot receive a mismatched bound.
Contributor
Author
|
The The rest of the matrix is green. I re-ran it by reopening the PR, which is why the timeline shows a close/reopen pair. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #7310.
DataFusion's type coercion promotes both operands of a
LIKEto a common string type, so aUtf8column matched against aUtf8VieworLargeUtf8pattern arrives asCast(column, <wider>) LIKE <wider>.maybe_indexed_columnrecognises only a bare column reference, so index discovery gives up and the predicate falls back to a full scan with a refine: the results stay correct and the index is silently lost. Comparisons are unaffected because DataFusion's simplifier unwraps the column cast forBinaryExprandInList, and Lance's ownresolve_exprcovers the same three shapes, but neither has a rule forExpr::Like, so the coercion handling added in #6985 never reached it. That same cast is also what keeps theUtf8Viewprefix support merged in #7351 from being reachable.Index discovery now looks through a cast whose source and target are both plain string types. The three string encodings carry the same values in the same order, so the index still answers the original predicate exactly, which matters because an exact answer skips the recheck; a cast that leaves the string types, or one over a non-string indexed column, is still refused.
visit_like_exprnow also coerces the pattern to the indexed column's type, which every other visitor already does throughmaybe_scalar. Without that, looking through the cast would let aLargeUtf8pattern reach aUtf8index and emit aLikePrefix(LargeUtf8)bound, which the zone map cannot compare against itsUtf8statistics (pruning every zone) and which trips the BTree's physicalLIKEassertion that both sides share a type.apply_regex_flagsgains theUtf8Viewvariant:regexp_likehas a per-argument coercible signature that never unifies its arguments, so its flags literal reaches the parser in whatever variant the caller built. The tests drivePlanner::create_filter_planrather than the parsers directly, and the LIKE-prefix case is parameterised over the column type as well as the literal variant, because the two hazards sit in different cells of that matrix.