Skip to content

[Improvement] Push Hive partition filters to HMS during planning #67724

Description

@zhaorongsheng

Problem

For highly partitioned Hive tables, FE planning may enumerate every HMS partition name before applying selective partition predicates. This makes planning latency depend on a full metastore listing and can cause timeouts.

Proposal

Materialize safe equality and IN partition predicates through HMS get_partitions_by_filter before the generic selected-partition map is built. Keep the existing local full-list pruning path as a compatibility fallback when the HMS filter API or dialect is unavailable.

Expected benefit

Selective partition queries avoid the full partition-name enumeration on the planner path when HMS supports the filter API.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions