-
Notifications
You must be signed in to change notification settings - Fork 3.9k
[fix](point query) Guard prepared point-query short-circuit reuse #67885
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
morrySnow
wants to merge
6
commits into
apache:master
Choose a base branch
from
morrySnow:fix/enforce-prepared-point-policy-key-constraints
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
812a7a1
[fix](nereids) Enforce fixed key predicates in prepared point queries
morrySnow be8a0aa
[fix](fe) Revalidate security before prepared point-query reuse
morrySnow 6aa418c
[fix](point query) Version cached built-in security decisions
morrySnow d3ac58e
[fix](point query) Restrict reusable prepared plans
morrySnow e5c5fbb
[fix](point query) Simplify prepared-plan security checks
morrySnow 4ccff13
[fix](point query) Scope placeholder tracking to filters
morrySnow File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
217 changes: 217 additions & 0 deletions
217
fe/fe-core/src/main/java/org/apache/doris/nereids/SecurityDependencyContext.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,217 @@ | ||
| // Licensed to the Apache Software Foundation (ASF) under one | ||
| // or more contributor license agreements. See the NOTICE file | ||
| // distributed with this work for additional information | ||
| // regarding copyright ownership. The ASF licenses this file | ||
| // to you under the Apache License, Version 2.0 (the | ||
| // "License"); you may not use this file except in compliance | ||
| // with the License. You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, | ||
| // software distributed under the License is distributed on an | ||
| // "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| // KIND, either express or implied. See the License for the | ||
| // specific language governing permissions and limitations | ||
| // under the License. | ||
|
|
||
| package org.apache.doris.nereids; | ||
|
|
||
| import org.apache.doris.analysis.UserIdentity; | ||
| import org.apache.doris.authorization.DataMaskSpec; | ||
| import org.apache.doris.authorization.RowFilterSpec; | ||
| import org.apache.doris.catalog.DatabaseIf; | ||
| import org.apache.doris.catalog.Env; | ||
| import org.apache.doris.catalog.OlapTable; | ||
| import org.apache.doris.catalog.TableIf; | ||
| import org.apache.doris.datasource.CatalogIf; | ||
| import org.apache.doris.datasource.InternalCatalog; | ||
| import org.apache.doris.mysql.privilege.InternalAuthorizationPlugin; | ||
| import org.apache.doris.nereids.rules.analysis.UserAuthentication; | ||
| import org.apache.doris.policy.PolicyMgr; | ||
| import org.apache.doris.qe.ConnectContext; | ||
| import org.apache.doris.qe.SessionVariable; | ||
|
|
||
| import com.google.common.collect.ImmutableSet; | ||
|
|
||
| import java.util.ArrayList; | ||
| import java.util.List; | ||
| import java.util.Objects; | ||
| import java.util.Optional; | ||
| import java.util.Set; | ||
|
|
||
| /** Security dependencies of a reusable prepared point-query plan. */ | ||
| public class SecurityDependencyContext { | ||
| private final UserIdentity planningUserIdentity; | ||
| private final Set<String> planningAuthenticatedRoles; | ||
| private final Env planningEnv; | ||
| private final boolean authorizationChecksEnabled; | ||
| private final List<CheckedPrivilege> checkedPrivileges = new ArrayList<>(); | ||
| private boolean hasEffectiveRowPolicy; | ||
| private boolean hasDataMask; | ||
| private boolean complete; | ||
|
|
||
| /** Create an incomplete context for tests and callers without a connection. */ | ||
| public SecurityDependencyContext() { | ||
| this(null, ImmutableSet.of(), null, false); | ||
| } | ||
|
|
||
| /** Capture the authorization subject before analysis starts. */ | ||
| public SecurityDependencyContext(ConnectContext connectContext) { | ||
| this(connectContext == null ? null : connectContext.getCurrentUserIdentity(), | ||
| authenticatedRoles(connectContext), | ||
| connectContext == null ? null : connectContext.getEnv(), | ||
| usesAuthorizationChecks(connectContext)); | ||
| } | ||
|
|
||
| private SecurityDependencyContext(UserIdentity planningUserIdentity, Set<String> planningAuthenticatedRoles, | ||
| Env planningEnv, boolean authorizationChecksEnabled) { | ||
| this.planningUserIdentity = planningUserIdentity; | ||
| this.planningAuthenticatedRoles = planningAuthenticatedRoles; | ||
| this.planningEnv = planningEnv; | ||
| this.authorizationChecksEnabled = authorizationChecksEnabled; | ||
| this.complete = authorizationChecksEnabled; | ||
| } | ||
|
|
||
| /** Record the exact SELECT check which must be repeated before direct reuse. */ | ||
| public synchronized void addCheckedPrivilege(TableIf table, Set<String> usedColumns) { | ||
| if (table == null) { | ||
| complete = false; | ||
| return; | ||
| } | ||
| DatabaseIf<?> database = table.getDatabase(); | ||
| CatalogIf<?> catalog = database == null ? null : database.getCatalog(); | ||
| if (catalog == null | ||
| || !(table instanceof OlapTable) | ||
| || !InternalCatalog.INTERNAL_CATALOG_NAME.equals(catalog.getName())) { | ||
| complete = false; | ||
| return; | ||
| } | ||
| checkedPrivileges.add(new CheckedPrivilege(table, database, catalog, catalog.getName(), | ||
| database.getFullName(), table.getName(), | ||
| usedColumns == null ? ImmutableSet.of() : ImmutableSet.copyOf(usedColumns))); | ||
| } | ||
|
|
||
| /** Record mask presence so a masked plan is never reused without policy analysis. */ | ||
| public synchronized void addDataMask( | ||
| String catalog, String database, String table, String column, Optional<DataMaskSpec> mask) { | ||
| hasDataMask |= mask.isPresent(); | ||
| } | ||
|
|
||
| public synchronized boolean hasDataMask() { | ||
| return hasDataMask; | ||
| } | ||
|
|
||
| /** Record only whether external policy analysis produced a row filter; definitions are not retained. */ | ||
| public synchronized void addRowPolicies(List<RowFilterSpec> policies) { | ||
| hasEffectiveRowPolicy |= policies != null && !policies.isEmpty(); | ||
| } | ||
|
|
||
| public synchronized boolean hasEffectiveRowPolicy() { | ||
| return hasEffectiveRowPolicy; | ||
| } | ||
|
|
||
| /** Freeze the completed dependency set before storing it with the prepared plan. */ | ||
| public synchronized SecurityDependencyContext snapshotForShortCircuit() { | ||
| SecurityDependencyContext snapshot = new SecurityDependencyContext( | ||
| planningUserIdentity, planningAuthenticatedRoles, planningEnv, authorizationChecksEnabled); | ||
| snapshot.checkedPrivileges.addAll(checkedPrivileges); | ||
| snapshot.hasEffectiveRowPolicy = hasEffectiveRowPolicy; | ||
| snapshot.hasDataMask = hasDataMask; | ||
| snapshot.complete = complete && !checkedPrivileges.isEmpty() | ||
| && !hasEffectiveRowPolicy && !hasDataMask | ||
| && checkedPrivileges.stream().allMatch(CheckedPrivilege::matchesNamespace); | ||
| return snapshot; | ||
| } | ||
|
|
||
| /** | ||
| * Recheck the small set of facts needed to bypass planning. Returning false only rejects direct reuse; normal | ||
| * planning then performs the authoritative check and reports its standard error. | ||
| */ | ||
| public boolean isValid(ConnectContext connectContext) { | ||
| if (!complete || !authorizationChecksEnabled || connectContext == null | ||
| || planningEnv == null || planningEnv != connectContext.getEnv() | ||
| || !Objects.equals(planningUserIdentity, connectContext.getCurrentUserIdentity()) | ||
| || !planningAuthenticatedRoles.equals(authenticatedRoles(connectContext)) | ||
| || !usesAuthorizationChecks(connectContext) | ||
| || !usesBuiltInAuthorization(planningEnv)) { | ||
| return false; | ||
| } | ||
| try { | ||
| PolicyMgr policyMgr = planningEnv.getPolicyMgr(); | ||
| if (policyMgr == null) { | ||
| return false; | ||
| } | ||
| for (CheckedPrivilege checkedPrivilege : checkedPrivileges) { | ||
| if (!checkedPrivilege.matchesNamespace()) { | ||
| return false; | ||
| } | ||
| if (policyMgr.hasRowPolicy(checkedPrivilege.catalog, checkedPrivilege.database, | ||
| checkedPrivilege.tableName)) { | ||
| return false; | ||
| } | ||
| UserAuthentication.checkPermission( | ||
| checkedPrivilege.table, connectContext, checkedPrivilege.usedColumns); | ||
| if (!checkedPrivilege.matchesNamespace()) { | ||
| return false; | ||
| } | ||
| } | ||
| return true; | ||
| } catch (Exception e) { | ||
| return false; | ||
| } | ||
| } | ||
|
|
||
| private static boolean usesBuiltInAuthorization(Env env) { | ||
| return env != null && env.getAccessManager() != null | ||
| && env.getAccessManager().getAccessControllerOrDefault(InternalCatalog.INTERNAL_CATALOG_NAME) | ||
| instanceof InternalAuthorizationPlugin; | ||
| } | ||
|
|
||
| private static boolean usesAuthorizationChecks(ConnectContext connectContext) { | ||
| if (connectContext == null || connectContext.isSkipAuth()) { | ||
| return false; | ||
| } | ||
| SessionVariable sessionVariable = connectContext.getSessionVariable(); | ||
| return sessionVariable != null && !sessionVariable.isPlayNereidsDump(); | ||
| } | ||
|
|
||
| private static Set<String> authenticatedRoles(ConnectContext connectContext) { | ||
| if (connectContext == null) { | ||
| return ImmutableSet.of(); | ||
| } | ||
| Set<String> roles = connectContext.getAuthenticatedRoles(); | ||
| return roles == null || roles.isEmpty() ? ImmutableSet.of() : ImmutableSet.copyOf(roles); | ||
| } | ||
|
|
||
| private static class CheckedPrivilege { | ||
| private final TableIf table; | ||
| private final DatabaseIf<?> databaseObject; | ||
| private final CatalogIf<?> catalogObject; | ||
| private final String catalog; | ||
| private final String database; | ||
| private final String tableName; | ||
| private final Set<String> usedColumns; | ||
|
|
||
| private CheckedPrivilege(TableIf table, DatabaseIf<?> databaseObject, CatalogIf<?> catalogObject, | ||
| String catalog, String database, String tableName, Set<String> usedColumns) { | ||
| this.table = table; | ||
| this.databaseObject = databaseObject; | ||
| this.catalogObject = catalogObject; | ||
| this.catalog = catalog; | ||
| this.database = database; | ||
| this.tableName = tableName; | ||
| this.usedColumns = usedColumns; | ||
| } | ||
|
|
||
| private boolean matchesNamespace() { | ||
| DatabaseIf<?> currentDatabase = table.getDatabase(); | ||
| CatalogIf<?> currentCatalog = currentDatabase == null ? null : currentDatabase.getCatalog(); | ||
| return currentDatabase == databaseObject | ||
| && currentCatalog == catalogObject | ||
| && Objects.equals(database, currentDatabase == null ? null : currentDatabase.getFullName()) | ||
| && Objects.equals(catalog, currentCatalog == null ? null : currentCatalog.getName()) | ||
| && Objects.equals(tableName, table.getName()); | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[P1] Preserve the relation namespace across security refresh
For an unqualified prepared query, execute
SELECT ... FROM tindb_ato publish this cache, then runUSE db_b. While the security facts stay unchanged, this check still passes and direct execution remains ondb_a.t. If a privilege, role, or row-policy change now makes the new security dependency fail,refreshPreparedPlanreparses the one-part name under the current connection andBindRelationbindsdb_b.t, permanently switching the same handle to a different table. Thus a security invalidation determines relation resolution. Capture the prepare/bound resolution namespace and use it during refresh (or make cached and normal executions consistently follow the chosen Doris semantics), and add a two-databaseUSEplus policy/revoke regression.