Generate target_compatible_with for whl_library_targets

[nacljht]

🚀 feature request

Relevant Rules

whl_library_targets macro in python/private/pypi/whl_library_targets.bzl

Description

When rules_python generates BUILD targets for platform-specific wheels (e.g., torch-2.8.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl), the resulting py_library does not set target_compatible_with. This means Bazel cannot distinguish platform-specific wheels from pure-Python ones at analysis time.

We maintain a repo that targets both x86_64 and aarch64. Our pyproject.toml uses PEP 508 environment markers to restrict certain dependencies to x86_64:

dependencies = [
    "torch==2.8.0; platform_machine == 'x86_64'",
    "opencv-python==4.11.0.86; platform_machine == 'x86_64'",
    "tensorboard==2.20.0; platform_machine == 'x86_64'",
    # ... many more x86_64-only deps
]

We generate separate lock files per platform via uv pip compile. The pip.parse extension correctly fetches platform-specific wheels. However, the generated py_library targets for these wheels have no target_compatible_with constraint.

This causes two issues:

1. bazel build //... fails on aarch64: Bazel attempts to build targets that transitively depend on x86_64-only wheels, which fail because the wheel files don't exist for aarch64. With target_compatible_with, Bazel would instead skip incompatible targets (printing SKIPPED in the build output).

2. Manual target_compatible_with annotations required everywhere: As a workaround, we must add target_compatible_with = ["@platforms//cpu:x86_64"] to every py_test and py_binary that transitively depends on an x86_64-only wheel. This is error-prone and tedious — our repo currently has dozens of such annotations scattered across BUILD files.

Describe the solution you'd like

The whl_library_targets macro should automatically infer target_compatible_with from the wheel filename's platform tag and set it on the generated py_library.

Wheel filenames follow a well-defined format (PEP 427): {distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl. The platform tag encodes the target architecture:

Platform tag suffix	Constraint
_x86_64, _amd64	@platforms//cpu:x86_64
_aarch64, _arm64	@platforms//cpu:aarch64
_i686	@platforms//cpu:x86_32
_ppc64le	@platforms//cpu:ppc
_s390x	@platforms//cpu:s390x
any (pure-Python)	(no constraint)

A possible implementation in whl_library_targets.bzl:

load(":parse_whl_name.bzl", "parse_whl_name")

_PLATFORM_TAG_CPU_MAP = {
    "x86_64": "@platforms//cpu:x86_64",
    "amd64": "@platforms//cpu:x86_64",
    "aarch64": "@platforms//cpu:aarch64",
    "arm64": "@platforms//cpu:aarch64",
    "i686": "@platforms//cpu:x86_32",
    "ppc64le": "@platforms//cpu:ppc",
    "s390x": "@platforms//cpu:s390x",
}

def _target_compatible_with_from_whl_name(whl_filename):
    if not whl_filename or not whl_filename.endswith(".whl"):
        return []
    parsed = parse_whl_name(whl_filename)
    if parsed.platform_tag == "any":
        return []
    for suffix, constraint in _PLATFORM_TAG_CPU_MAP.items():
        if parsed.platform_tag.endswith("_" + suffix):
            return [constraint]
    return []

Then in the whl_library_targets function, pass it to py_library:

rules.py_library(
    ...
    target_compatible_with = _target_compatible_with_from_whl_name(name),
    ...
)

This leverages the existing parse_whl_name utility already available in rules_python.

Describe alternatives you've considered

1. Manual target_compatible_with on downstream targets: This is what we currently do — every py_test/py_binary that transitively depends on an x86_64-only wheel gets a manual target_compatible_with = ["@platforms//cpu:x86_64"]. This is tedious, error-prone, and doesn't scale. Each new test file requires the developer to figure out whether any transitive dependency is platform-specific.

2. Patching rules_python locally via single_version_override: We currently apply a local patch to whl_library_targets.bzl (via single_version_override in MODULE.bazel) that implements the solution described above. This works but is a maintenance burden — we have to keep the patch in sync with upstream rules_python updates.

3. Using pip.parse whl_modifications to post-hoc add target_compatible_with: This would require listing every platform-specific wheel individually and is even more error-prone than option 1.

The information to infer target_compatible_with is already embedded in the wheel filename. Having rules_python set it automatically would eliminate all three workarounds and make cross-platform Bazel builds work correctly out of the box.

[aignas]

This has come up many times, so I am happy for this code to be added in the whl_library_targets macro, however, I would like to somehow have a way to toggle this behaviour on and off before committing to one way or another. It seems that this is something that the end user might want to toggle, so we can use the internal_config repo and an env variable to toggle the behaviour. So the DoD for this could be:

• Update whl_library_targets macro with tests.
• Add an env var for the feature flag. We can leave it disabled in the first PR. Document the env var as part of our regular docs. This will make it easy for users to disable the behaviour when they report bugs. For example, too many targets are excluded because of this. Maybe they have some custom rules and are not using transitions, etc.
• Add changelog notes.

One other solution would be to actually changes this in the hub repo and ensure that alias invocations have the right target_compatible_with based on what target_platfroms we use.