feat: add union query support by seladb · Pull Request #2146 · tortoise/tortoise-orm

seladb · 2026-03-17T05:47:22Z

Add union query support in an API similar to Django's API

Addresses these issues:
#1507
#2056 (partially)

Description

Add union query support:

qs1 = Tournament.filter(name="T1").only("id")
qs2 = Reporter.filter(name="R1").only("id")

result = await qs1.union(qs2)

As opposed to Django, this API supports result of multiple models (in Django all results will be instances of the first model).
However, similar to Django, the requested field names and types should be the same across all queried models. This is because of how DBs support union queries.

Motivation and Context

It addresses the following issues:
#1507
#2056 (partially)

Also - it brings more feature parity with Django.

How Has This Been Tested?

Added test coverage for all written code. It was tested with MySQL, Postgres and SQLite.

Checklist:

My code follows the code style of this project.
My change requires a change to the documentation.
I have updated the documentation accordingly.
I have added the changelog accordingly.
I have read the CONTRIBUTING document.
I have added tests to cover my changes.
All new and existing tests passed.

# Conflicts: # tortoise/queryset.py

codspeed-hq · 2026-03-17T06:16:26Z

Merging this PR will not alter performance

✅ 24 untouched benchmarks

_{Comparing seladb:add-union (5f4ad80) with develop (4874995)}

tortoise/queryset.py

seladb · 2026-03-18T09:09:39Z

tests/test_queryset.py

+    assert result == expected
+
+
+@requireCapability(dialect=NotEQ("mssql"))


MSSql limit syntax is different but it's not currently supported in _SetOperation which has its own implementation of _limit_sql() instead of using the dialect SQL. I guess we can leave it as a TODO for later

https://github.com/tortoise/pypika-tortoise/blob/378f6145f7529d88fa58510851d80454b742406e/pypika_tortoise/queries.py#L672

tortoise/queryset.py

tests/test_queryset.py

seladb · 2026-04-03T08:08:59Z

@abondar I addressed your comments, can you please re-review this PR?

seladb · 2026-04-08T03:56:05Z

@abondar I addressed your comments, can you please re-review this PR?

@abondar a gentle reminder to re-review this PR - I addressed all of your comments

abondar · 2026-04-08T09:57:58Z

tortoise/backends/base/executor.py

+                    model._meta.app == row[app_field]
+                    and model._meta._model.__name__ == row[model_field]
+                ):
+                    instance_list.append(model._init_from_db(**row))


I think it is not good idea to pass row like that, with extra values of app field and model field. I think we should strip them. I understand that it works now like that, but it seems fragile and can interfere with actual fields declared on model. Maybe even consider making these special field names more unique, to lower chance of overlapping with actual fields (maybt "_*" prefix?)

I'm not sure what you mean. We add 2 annotation fields: tortoise_app and tortoise_model here:
https://github.com/seladb/tortoise-orm/blob/074a61e59e88bbe221aa835f7ba6aa8390e7a605/tortoise/queryset.py#L2316-L2319

I can change the field names to __tortoise_app__ and __tortoise_model__ if you think it's better, but there's always a (very small) chance it will interfere with existing fields.

I think we should strip them

What do you mean by that?

You are passing these field values inside model init, I think we shouldn't do it, we should pop them from row dict

Oh ok, it makes sense. Addressed in 9d541c2

abondar · 2026-04-08T10:00:39Z

tortoise/queryset.py

+            if getattr(select, "alias") not in [cls.TORTOISE_APP_FIELD, cls.TORTOISE_MODEL_FIELD]
+        ]
+
+    def _make_query(self) -> None:


I feel we still should make it more idempotent, maybe just add self._union_query = None at start?

Added in 7ddbc5e

abondar · 2026-04-08T10:09:11Z

tortoise/queryset.py

+        return list(dict(result[0]).values())[0]
+
+
+class UnionQuery(AwaitableQuery[MODEL]):


I am skeptical about inheretence here - we inherit AwaitableQuery, but don't use many of it's fields, also redefining __slots__ and not setting those base fields in _clone

Is there reason why we want that inheretence at all? only for choose_db methods? Maybe we should extract choose_db methods to some mixing used by both?

Do you want me to implement it in this PR or move it to a separate PR?
I'm a bit concerned that this PR already contains quite a lot of changes, maybe we should avoid making even more changes?

Well, it's still change that concerns new class, it would be strange to ship it with bad inheritance model and start refactoring it right after

I made this change in 5f4ad80, please let me know what you think

- Change `UnionQuery` to inherit from `_ChooseDBMixin` instead of `AwaitableQuery`

seladb · 2026-04-09T07:56:55Z

@abondar I addressed your PR comments, please let me know what you think

seladb · 2026-04-10T02:15:00Z

Thank you @abondar for the thorough review and for merging this PR, much appreciated! 🙏
I'll update my other PRs and would love if you can review them also

seladb added 4 commits March 16, 2026 22:54

Merge from tortoise/develop

ee4ca6a

# Conflicts: # tortoise/queryset.py

Add test and some fixes

483f720

Fix union count

7d761b8

Add documentation and changelog

1484a9b

seladb force-pushed the add-union branch from b5bce1c to 1484a9b Compare March 17, 2026 06:13

seladb added 6 commits March 16, 2026 23:51

Fix checks

808c302

Fix count query

3e4dee7

Another fix for count query

9080bfb

Small fix

18dfd14

Add debug SQL

852faa0

Fixed test for MSSql

da59410

seladb commented Mar 18, 2026

View reviewed changes

seladb marked this pull request as ready for review March 18, 2026 09:10

seladb added 3 commits March 18, 2026 02:31

Fix count query

e96065b

Cleanup

7618481

Remove unused variable

ba5973e

abondar reviewed Mar 19, 2026

View reviewed changes

seladb added 5 commits March 24, 2026 22:21

Address PR comments

8f6a104

Merge branch 'develop' into add-union

d64f5b7

Raise a proper exception when qs uses annotations

709ddfa

Add offset support

dbbed50

Exclude MS-SQL from offset tests

a1accfd

seladb commented Mar 25, 2026

View reviewed changes

tests/test_queryset.py Show resolved Hide resolved

seladb added 3 commits March 25, 2026 00:15

Fix UnionCountQuery to properly build union query before counting

211bbfe

Fix style

7c1225a

Trigger CI

f8dd7c8

seladb requested a review from abondar March 25, 2026 07:37

seladb changed the title ~~Add union query support~~ feat: add union query support Apr 4, 2026

Merge branch 'tortoise:develop' into add-union

074a61e

abondar reviewed Apr 8, 2026

View reviewed changes

seladb added 4 commits April 8, 2026 23:57

Address PR comments

7ddbc5e

Remove model and app fields when initializing the model

7cd1817

Fix for "remove model and app fields when initializing the model"

9d541c2

- Extract choose DB methods to a mixin

5f4ad80

- Change `UnionQuery` to inherit from `_ChooseDBMixin` instead of `AwaitableQuery`

seladb requested a review from abondar April 9, 2026 07:56

abondar merged commit 56335a1 into tortoise:develop Apr 9, 2026
25 checks passed

seladb deleted the add-union branch April 10, 2026 02:20

		assert result == expected


		@requireCapability(dialect=NotEQ("mssql"))

		return list(dict(result[0]).values())[0]


		class UnionQuery(AwaitableQuery[MODEL]):

Conversation

seladb commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Checklist:

Uh oh!

codspeed-hq bot commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

seladb commented Apr 3, 2026

Uh oh!

seladb commented Apr 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seladb Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seladb commented Apr 9, 2026

Uh oh!

Uh oh!

seladb commented Apr 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

seladb commented Mar 17, 2026 •

edited

Loading

codspeed-hq bot commented Mar 17, 2026 •

edited

Loading

seladb Apr 9, 2026 •

edited

Loading