bazel: upgrade liburing to 2.14 by travisdowns · Pull Request #29955 · redpanda-data/redpanda

travisdowns · 2026-03-25T14:10:45Z

Upgrade liburing from 2.5 to 2.14. Includes a patch to work around
out-of-source build issues (bazelbuild/bazel-central-registry#8079)
until we can pick up the upstream fix (axboe/liburing#1558).

We don't currently use liburing, so this is a no-op upgrade to stay
current.

I ran all unit tests with uring (see other commit) and had no new failures (one test failed even w/o io_uring, I think disk cache corruption).

Backports Required

Release Notes

none

Copilot

Pull request overview

This PR upgrades the Bazel BCR liburing dependency from 2.5 to 2.14 and applies a temporary patch to the upstream BCR build definition to avoid in-source build assumptions (to work around known out-of-source build issues).

Changes:

Bump liburing Bazel module version to 2.14 and apply a local patch via single_version_override.
Add a liburing BUILD patch to run configure without changing CWD (symlink-based out-of-source workaround) and to avoid relying on realpath.
Refresh MODULE.bazel.lock to reflect the new module versions/resolution.

Reviewed changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.

File	Description
bazel/thirdparty/liburing.patch	Adjusts upstream module’s `BUILD.bazel` to work around out-of-source build issues and propagate Bazel C/C++ flags into `configure`.
MODULE.bazel	Upgrades `liburing` to 2.14 and applies the patch using `single_version_override`.
MODULE.bazel.lock	Updates resolved module metadata/hashes for the new dependency graph.

travisdowns

Some notes for reviewers on the test.bzl changes.

travisdowns · 2026-03-25T15:31:39Z

+    return select({
+        "//bazel:io_uring_enabled": ["--reactor-backend=io_uring"],
+        "//conditions:default": [],
+    })


This is extracted into its own function because it's used in two places: _redpanda_cc_test (unit tests) and redpanda_cc_bench (benchmark smoke tests). Both are Seastar-based and accept --reactor-backend.

travisdowns · 2026-03-25T15:31:39Z

+
    test_data, test_env = _test_options()
    cc_test(
        name = name,


This must go here (after line 155) and not earlier because select() returns a Starlark select type that is not iterable or mutable. The code above iterates args in _has_flags() (line 140-142), calls args.append() (lines 145-146), and checks if args truthiness (line 154) — all of which require a plain list. Once we concatenate the select(), the result can only be passed to a rule attribute.

travisdowns · 2026-03-25T15:31:39Z

    )

-    args = ["$(rootpath :{})".format(binary_name)] + args
+    args = ["$(rootpath :{})".format(binary_name)] + args + _reactor_args()


For benchmarks, the reactor arg is concatenated into args here so it flows through to both the py_binary (the actual benchmark runner, line 496) and the py_test smoke test (line 509, via test_args which is derived from args on line 505). The select() + list concatenation chains work fine in Starlark as long as the final value lands in a rule attribute.

StephanDollberg · 2026-03-25T15:34:25Z

Why don't we enable it unconditionally? Isn't that what we do in cmake land? That's why we have the seastar patch to change order.

travisdowns

Reviewer notes for the liburing patch.

travisdowns · 2026-03-25T15:36:32Z

+         "src/include/liburing/io_uring_version.h",
+     ],
+     cmd = """
+-            export CC=$(CC) CXX=$(CC)++


We now pass CFLAGS="$(CC_FLAGS)" so that the configure step picks up the sysroot and target flags from the Bazel toolchain. Without this, configure's feature-detection probes compile against the host system headers instead of the hermetic toolchain, which can produce wrong results (e.g. detecting __kernel_timespec or open_how incorrectly).

travisdowns · 2026-03-25T15:36:32Z

+            # We do an out-of-source build of liburing here, so that
+            # CWD does not change and CC and other relative paths work
+            # (e.g. those embedded in sysroot). Currently, liburing
+            # doesn't support out of source builds, so we fake it


This is the core of the patch. The upstream BCR BUILD.bazel does pushd into the package directory to run configure, which changes CWD away from the execroot. That breaks CC because Bazel's CC wrapper path is relative to the execroot, not the package dir.

Instead we do an out-of-source build: stay in the execroot (so CC works) and symlink the two files that configure expects to find in CWD (Makefile.common and liburing.spec). This is a workaround until upstream liburing supports out-of-source builds properly (axboe/liburing#1558).

travisdowns · 2026-03-25T15:36:32Z

+            mkdir -p src/include/liburing
+            $$pkg_dir/configure --use-libc
+
+             # collect the outputs


The upstream patch used realpath --relative-to=$(BINDIR) to copy generated headers from the package dir back to the output locations. That doesn't work with our out-of-source layout since the generated files are now in the execroot, not the package dir. ${out#$(RULEDIR)/} strips the output-dir prefix to get the relative path of each generated file, which is already correct relative to CWD.

travisdowns · 2026-03-25T15:36:32Z

+             done
+           """,
+-    toolchains = ["@bazel_tools//tools/cpp:current_cc_toolchain"],
+    toolchains = [


Adding @bazel_tools//tools/cpp:cc_flags makes $(CC_FLAGS) available in the genrule cmd. This is the standard Bazel mechanism for propagating toolchain flags (like --sysroot) into genrules.

travisdowns · 2026-03-25T15:46:13Z

Why don't we enable it unconditionally? Isn't that what we do in cmake land? That's why we have the seastar patch to change order.

Support is built-in to seastar by default (I think that's what you are asking). See the .bazelrc for comments, LMK if that is unclear.

StephanDollberg · 2026-03-25T15:51:01Z

Support is built-in to seastar by default (I think that's what you are asking). See the .bazelrc for comments, LMK if that is unclear.

Oh yes I didn't read the comment to the end.

Upgrade from 2.5 to 2.14 with a patch to work around out-of-source build issues (bazelbuild/bazel-central-registry#8079) which itself is hard to fix until BCR picks up axboe/liburing#1558.

Add a bool_flag //bazel:io_uring (default false) that passes --reactor-backend=io_uring to all unit tests and benchmarks when enabled. Usage: bazel test //... --config=io_uring

travisdowns · 2026-03-25T18:26:30Z

70738a2 is rebase to fix build conflict

vbotbuildovich · 2026-03-25T19:37:44Z

CI test results

test results on build#82342

test_class	test_method	test_arguments	test_kind	job_url	test_status	passed	reason	test_history
PartitionMovementTest	test_cross_core_movements	{"cloud_storage_type": 1}	integration	https://buildkite.com/redpanda/redpanda/builds/82342#019d2646-8eb0-47e4-9f1e-b661f2234176	FLAKY	10/11	Test PASSES after retries.No significant increase in flaky rate(baseline=0.0000, p0=1.0000, reject_threshold=0.0100. adj_baseline=0.1000, p1=0.3487, trust_threshold=0.5000)	https://redpanda.metabaseapp.com/dashboard/87-tests?tab=142-dt-individual-test-history&test_class=PartitionMovementTest&test_method=test_cross_core_movements

WillemKauf · 2026-04-01T02:45:04Z

ERROR: /home/willem/.cache/bazel/_bazel_willem/6fc2d9663f4e6fbf4046ab3d95678674/external/liburing+/BUILD.bazel:48:11: Compiling src/queue.c failed: (Exit 1): cc_wrapper.sh failed: error executing CppCompile command (from target @@liburing+//:uring) external/toolchains_llvm++llvm+current_llvm_toolchain/bin/cc_wrapper.sh -U_FORTIFY_SOURCE '--target=x86_64-unknown-linux-gnu' '-march=westmere' -U_FORTIFY_SOURCE -fstack-protector ... (remaining 38 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
In file included from external/liburing+/src/queue.c:5:
In file included from external/liburing+/src/syscall.h:13:
In file included from external/liburing+/src/include/liburing.h:17:
bazel-out/k8-fastbuild/bin/external/liburing+/src/include/liburing/compat.h:55:3: error: redefinition of enumerator 'P_ALL'
   55 |   P_ALL,                /* Wait for any child.  */
      |   ^
external/+non_module_dependencies+x86_64_sysroot/usr/include/x86_64-linux-gnu/sys/wait.h:76:3: note: previous definition is here
   76 |   P_ALL,                /* Wait for any child.  */
      |   ^

https://redpandadata.slack.com/archives/C07HD9U0EHL/p1775011039916519

WillemKauf · 2026-04-01T04:09:13Z

#30029

Copilot AI review requested due to automatic review settings March 25, 2026 14:10

github-actions bot added the area/build label Mar 25, 2026

travisdowns marked this pull request as draft March 25, 2026 14:11

Copilot started reviewing on behalf of travisdowns March 25, 2026 14:11 View session

Copilot AI reviewed Mar 25, 2026

View reviewed changes

travisdowns requested a review from StephanDollberg March 25, 2026 15:29

travisdowns marked this pull request as ready for review March 25, 2026 15:29

travisdowns commented Mar 25, 2026

View reviewed changes

rockwotj previously approved these changes Mar 25, 2026

View reviewed changes

travisdowns added 2 commits March 25, 2026 15:17

bazel: upgrade liburing to 2.14

91ce6ca

Upgrade from 2.5 to 2.14 with a patch to work around out-of-source build issues (bazelbuild/bazel-central-registry#8079) which itself is hard to fix until BCR picks up axboe/liburing#1558.

bazel: add io_uring build setting for reactor backend

70738a2

Add a bool_flag //bazel:io_uring (default false) that passes --reactor-backend=io_uring to all unit tests and benchmarks when enabled. Usage: bazel test //... --config=io_uring

travisdowns dismissed rockwotj’s stale review via 70738a2 March 25, 2026 18:19

travisdowns force-pushed the td-uring-upgrade branch from 14fce50 to 70738a2 Compare March 25, 2026 18:19

rockwotj approved these changes Mar 25, 2026

View reviewed changes

StephanDollberg approved these changes Mar 25, 2026

View reviewed changes

travisdowns merged commit 609937b into redpanda-data:dev Apr 1, 2026
18 checks passed

Conversation

travisdowns commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Backports Required

Release Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

travisdowns left a comment

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

StephanDollberg commented Mar 25, 2026

Uh oh!

travisdowns left a comment

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

travisdowns Mar 25, 2026

Choose a reason for hiding this comment

Uh oh!

travisdowns commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

StephanDollberg commented Mar 25, 2026

Uh oh!

travisdowns commented Mar 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vbotbuildovich commented Mar 25, 2026

CI test results

Uh oh!

Uh oh!

WillemKauf commented Apr 1, 2026

Uh oh!

WillemKauf commented Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

travisdowns commented Mar 25, 2026 •

edited

Loading

travisdowns commented Mar 25, 2026 •

edited

Loading

travisdowns commented Mar 25, 2026 •

edited

Loading