[fix](fk) Isolate primary key proofs by relation instance - #67891
Open
morrySnow wants to merge 1 commit into
Open
[fix](fk) Isolate primary key proofs by relation instance#67891morrySnow wants to merge 1 commit into
morrySnow wants to merge 1 commit into
Conversation
### What problem does this PR solve?
Issue Number: None
Related PR: None
Problem Summary: Join elimination could return extra rows for a self-referencing foreign key when the primary-key relation was limited. A query joining (select id from self_ref order by id limit 1) back to another scan of self_ref should keep the join and return one row, but the optimizer eliminated the join and returned both foreign-side rows. ForeignKeyContext stored primary-key validity by catalog column identity, so expiring the limited relation removed a shared catalog key that a later scan of the same table registered again. This accidentally revived the invalid proof for the earlier relation instance. Track declared primary-key metadata separately from active Slot/ExprId proofs, expire only the exact plan output slots, and propagate activity through aliases only when the origin slot remains active.
### Release note
Fix incorrect results when a self-referencing foreign-key join has LIMIT or TopN on the primary-key relation.
### Check List (For Author)
- Test: Unit Test, Regression test, and Manual test
- Added optimizer tests for plain, limited, aliased, projected, filtered, reversed, and unrelated relation cases.
- Added a regression test that verifies the limited join is retained and returns one row while the unrestricted join remains eliminable.
- Ran the focused FE unit test, full FE build with checkstyle, and the regression suite on a local FE/BE deployment.
- Behavior changed: Yes. Unsafe join elimination is disabled when LIMIT or TopN invalidates a relation-instance primary-key proof or when primary/foreign predicates are not compatible; proven safe elimination remains enabled.
- Does this need documentation: No
morrySnow
requested review from
924060929,
englefly and
starocean999
as code owners
September 11, 2026 20:07
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
FE UT Coverage ReportIncrement line coverage |
morrySnow
changed the title
morrySnow
marked this pull request as draft
September 12, 2026 03:30
Contributor
TPC-H: Total hot run time: 16724 ms |
Contributor
TPC-DS: Total hot run time: 81595 ms |
Contributor
ClickBench: Total hot run time: 14.63 s |
morrySnow
marked this pull request as ready for review
September 12, 2026 07:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Foreign-key join elimination could produce incorrect results for a self-referencing table when LIMIT or TopN invalidated the primary-key side. The optimizer could remove the join and return rows that had no match in the limited primary input.
Root cause
ForeignKeyContextrepresented both catalog-declared primary keys and the validity of a particular relation instance with the same set of qualified catalog columns. When a limited scan expired its proof, visiting another scan of the same physical table registered the catalog primary key again. Because both scans shared the same catalog identity, this revived the expired proof for the first scan even though their output slots had different ExprIds.Reproduction
Create a self-referencing table containing
(id, parent_id) = (1, 1), (2, 2), declareidas its primary key andparent_idas a foreign key toid, then join the table to(select id from the same table order by id limit 1). The valid join returns only1. Before this change, join elimination removed the limited primary side and incorrectly returned1, 2.Fix
declaredPrimaryKeys.activePrimaryKeySlots.Tests
./run-fe-ut.sh --run org.apache.doris.nereids.rules.rewrite.EliminateJoinByFkTest— 17 tests, 0 failures, 0 errors.DISABLE_BUILD_UI=ON ./build.sh --fe— full FE reactor build passed; FE checkstyle reported 0 violations.-forceGenOut— 1 suite passed.