Skip to content

[DNM] feat: Support apache uniffle remote shuffle service#4884

Draft
wForget wants to merge 3 commits into
apache:mainfrom
wForget:rss_uniffle
Draft

[DNM] feat: Support apache uniffle remote shuffle service#4884
wForget wants to merge 3 commits into
apache:mainfrom
wForget:rss_uniffle

Conversation

@wForget

@wForget wForget commented Jul 10, 2026

Copy link
Copy Markdown
Member

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

Comment thread .github/workflows/pr_build_linux.yml Dismissed
@wForget wForget changed the title [DNM] feat: Support rss shuffle service [DNM] feat: Support apache uniffle remote shuffle service Jul 10, 2026
@wForget wForget self-assigned this Jul 13, 2026
@wForget

wForget commented Jul 14, 2026

Copy link
Copy Markdown
Member Author

Most Comet Spark unit tests pass successfully. See: https://github.com/apache/datafusion-comet/actions/runs/29303133000/job/86994439958?pr=4884

The remaining failures are due to:

  • The Uniffle Shuffle Manager supports only native shuffle mode.
  • The data ordering produced by the remote shuffle service differs from that of local shuffle, so results without explicit sorting are nondeterministic.

In a follow-up, I’ll enable only the TPC-H or TPC-DS integration tests for Uniffle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants