🚧 Simplify Parquet AST expressions by pruning - #23992
Draft
mhaseeb123 wants to merge 3 commits into
Draft
Conversation
… row groups A pruning expression is built over per-row-group summaries and answers "might some row here satisfy the filter?" - an existential, not the predicate. Existentials only combine under conjunction and disjunction, so every other node has to relax, that is, decline to constrain the chunk at all. The converters expressed relaxation as an always-true literal tracked alongside the expression tree, duplicated per converter. Replace it with std::optional: absent means unconstrained. Add `pruning_expression_builder`, which owns the traversal and the combination rules, and port `stats_expression_converter` onto it, leaving that class with only the leaves statistics can evaluate. Two behaviour changes, both toward more pruning: - `false AND (col_a < 50)` drops the unconstrained conjunct and prunes on the other, rather than relaxing the whole conjunction - `(a == 1) == (b == 2)` relaxes rather than applying `==` to two summaries, which yielded false and pruned row groups holding rows that do satisfy the predicate
…ion builder Both membership converters carried their own copy of the traversal, the operand extraction and the always-true bookkeeping. Derive them from `pruning_expression_builder` instead, leaving each with only its `build_comparison`: equality for bloom filters, equality and inequality for dictionary pages. This makes the negation bug structurally unrepresentable rather than merely fixed. A converter cannot see a `NOT` unless it overrides `build_negated_*`, and a membership answer must not be negated: not(some row is 5) means "no row is 5", not "some row is not 5". No behaviour change beyond what the shared combination rules already give.
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
TBA
Checklist