pyiceberg-core 0.9.0 requires building with rust on x86_64

[wufh43]

Apache Iceberg version

0.11.0 (latest release)

Please describe the bug 🐞

We've been using pyiceberg-core with uv like so:

uv run --native-tls --with pyiceberg-core

Before 0.9.0, this worked without having any rust tooling installed.

With 0.9.0, it tries to build the pyiceberg-core package from sources, which requires rust.

Here is the output for 0.8.0

$ uv run -v --native-tls --with pyiceberg-core==0.8.0
DEBUG uv 0.10.12 (x86_64-unknown-linux-gnu)
DEBUG No project found; searching for Python interpreter
DEBUG No Python version file found in ancestors of working directory: /home/user
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Searching for default Python interpreter in virtual environments, managed installations, or search path
DEBUG Searching for managed installations at `.local/share/uv/python`
DEBUG Found managed installation `cpython-3.15.0a2-linux-x86_64-gnu`
DEBUG Found `cpython-3.15.0a2-linux-x86_64-gnu` at `/home/user/.local/share/uv/python/cpython-3.15.0a2-linux-x86_64-gnu/bin/python3.15` (managed installations)
DEBUG Skipping pre-release installation cpython-3.15.0a2-linux-x86_64-gnu
DEBUG Found managed installation `cpython-3.12.12-linux-x86_64-gnu`
DEBUG Found `cpython-3.12.12-linux-x86_64-gnu` at `/home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12` (managed installations)
DEBUG Using Python 3.12.12 interpreter at: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG At least one requirement is not satisfied in the base environment: pyiceberg-core==0.8.0
DEBUG Syncing `--with` requirements to cached environment
DEBUG Assessing Python executable as base candidate: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Caching via base interpreter: `/home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12`
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Solving with installed Python version: 3.12.12
DEBUG Solving with target Python version: >=3.12.12
DEBUG Adding direct dependency: pyiceberg-core>=0.8.0, <0.8.0+
DEBUG No cache entry for: https://pypi.org/simple/pyiceberg-core/
DEBUG Sending fresh GET request for: https://pypi.org/simple/pyiceberg-core/
DEBUG Searching for a compatible version of pyiceberg-core (>=0.8.0, <0.8.0+)
DEBUG Selecting: pyiceberg-core==0.8.0 [compatible] (pyiceberg_core-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl)
DEBUG No cache entry for: https://files.pythonhosted.org/packages/95/4b/f799e5c7a2b2ede75514e64901503358a7a134ca1ea217fd86535af533b6/pyiceberg_core-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG Sending fresh GET request for: https://files.pythonhosted.org/packages/95/4b/f799e5c7a2b2ede75514e64901503358a7a134ca1ea217fd86535af533b6/pyiceberg_core-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata
DEBUG Tried 1 versions: pyiceberg-core 1
DEBUG marker environment resolution took 0.066s
Resolved 1 package in 67ms
DEBUG Assessing Python executable as base candidate: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Using base executable for virtual environment: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Identified uncached distribution: pyiceberg-core==0.8.0
DEBUG No cache entry for: https://files.pythonhosted.org/packages/95/4b/f799e5c7a2b2ede75514e64901503358a7a134ca1ea217fd86535af533b6/pyiceberg_core-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
DEBUG Sending fresh GET request for: https://files.pythonhosted.org/packages/95/4b/f799e5c7a2b2ede75514e64901503358a7a134ca1ea217fd86535af533b6/pyiceberg_core-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl
Downloading pyiceberg-core (13.2MiB)
 Downloaded pyiceberg-core
Prepared 1 package in 1.02s
Installed 1 package in 95ms
 + pyiceberg-core==0.8.0

And for 0.9.0:

DEBUG uv 0.10.12 (x86_64-unknown-linux-gnu)
DEBUG No project found; searching for Python interpreter
DEBUG No Python version file found in ancestors of working directory: /home/user
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Searching for default Python interpreter in virtual environments, managed installations, or search path
DEBUG Searching for managed installations at `.local/share/uv/python`
DEBUG Found managed installation `cpython-3.15.0a2-linux-x86_64-gnu`
DEBUG Found `cpython-3.15.0a2-linux-x86_64-gnu` at `/home/user/.local/share/uv/python/cpython-3.15.0a2-linux-x86_64-gnu/bin/python3.15` (managed installations)
DEBUG Skipping pre-release installation cpython-3.15.0a2-linux-x86_64-gnu
DEBUG Found managed installation `cpython-3.12.12-linux-x86_64-gnu`
DEBUG Found `cpython-3.12.12-linux-x86_64-gnu` at `/home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12` (managed installations)
DEBUG Using Python 3.12.12 interpreter at: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG At least one requirement is not satisfied in the base environment: pyiceberg-core==0.9.0
DEBUG Syncing `--with` requirements to cached environment
DEBUG Assessing Python executable as base candidate: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Caching via base interpreter: `/home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12`
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Solving with installed Python version: 3.12.12
DEBUG Solving with target Python version: >=3.12.12
DEBUG Adding direct dependency: pyiceberg-core>=0.9.0, <0.9.0+
DEBUG Found fresh response for: https://pypi.org/simple/pyiceberg-core/
DEBUG Searching for a compatible version of pyiceberg-core (>=0.9.0, <0.9.0+)
DEBUG Selecting: pyiceberg-core==0.9.0 [compatible] (pyiceberg_core-0.9.0.tar.gz)
DEBUG No cache entry for: https://files.pythonhosted.org/packages/fa/2f/ccb605b8f09dc417dc01345f13273688efcd08e9a4c8297749fab20a1441/pyiceberg_core-0.9.0-cp310-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl.metadata
DEBUG Sending fresh GET request for: https://files.pythonhosted.org/packages/fa/2f/ccb605b8f09dc417dc01345f13273688efcd08e9a4c8297749fab20a1441/pyiceberg_core-0.9.0-cp310-abi3-macosx_10_12_x86_64.macosx_11_0_arm64.macosx_10_12_universal2.whl.metadata
DEBUG Tried 1 versions: pyiceberg-core 1
DEBUG marker environment resolution took 0.024s
Resolved 1 package in 25ms
DEBUG Assessing Python executable as base candidate: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Using base executable for virtual environment: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Using request connect timeout of 10s and read timeout of 30s
DEBUG Identified uncached distribution: pyiceberg-core==0.9.0
DEBUG Found fresh response for: https://files.pythonhosted.org/packages/ab/a3/2fd5953aa1cdc8dca3b498b4d7691db8431a1199df95a4e826d120860880/pyiceberg_core-0.9.0.tar.gz
   Building pyiceberg-core==0.9.0
DEBUG Building: pyiceberg-core==0.9.0
DEBUG Assessing Python executable as base candidate: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Reusing existing build environment for: pyiceberg-core==0.9.0
DEBUG No workspace root found, using project root
DEBUG Assessing Python executable as base candidate: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Using base executable for virtual environment: /home/user/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/bin/python3.12
DEBUG Resolving build requirements
DEBUG Solving with installed Python version: 3.12.12
DEBUG Solving with target Python version: >=3.12.12
DEBUG Adding direct dependency: maturin>=1.0, <2.0
DEBUG No cache entry for: https://pypi.org/simple/maturin/
DEBUG Sending fresh GET request for: https://pypi.org/simple/maturin/
DEBUG Searching for a compatible version of maturin (>=1.0, <2.0)
DEBUG Selecting: maturin==1.12.6 [compatible] (maturin-1.12.6-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl)
DEBUG No cache entry for: https://files.pythonhosted.org/packages/59/ea/f1d6ad95c0a12fbe761a7c28a57540341f188564dbe8ad730a4d1788cd32/maturin-1.12.6-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl.metadata
DEBUG Sending fresh GET request for: https://files.pythonhosted.org/packages/59/ea/f1d6ad95c0a12fbe761a7c28a57540341f188564dbe8ad730a4d1788cd32/maturin-1.12.6-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl.metadata
DEBUG Tried 1 versions: maturin 1
DEBUG marker environment resolution took 0.174s
DEBUG Installing in maturin==1.12.6 in /home/user/.cache/uv/builds-v0/.tmpetFzII
DEBUG Identified uncached distribution: maturin==1.12.6
DEBUG Downloading and building requirement for build: maturin==1.12.6
DEBUG No cache entry for: https://files.pythonhosted.org/packages/59/ea/f1d6ad95c0a12fbe761a7c28a57540341f188564dbe8ad730a4d1788cd32/maturin-1.12.6-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl
DEBUG Sending fresh GET request for: https://files.pythonhosted.org/packages/59/ea/f1d6ad95c0a12fbe761a7c28a57540341f188564dbe8ad730a4d1788cd32/maturin-1.12.6-py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.musllinux_1_1_x86_64.whl
DEBUG Installing build requirement: maturin==1.12.6
DEBUG Creating PEP 517 build environment
DEBUG Calling `maturin.get_requires_for_build_wheel()`
DEBUG No workspace root found, using project root
DEBUG Calling `maturin.build_wheel("/home/user/.cache/uv/builds-v0/.tmpB4x5Ua", {}, None)`
DEBUG Running `maturin pep517 build-wheel -i /home/user/.cache/uv/builds-v0/.tmpetFzII/bin/python --compatibility off`
^C

Willingness to contribute

• I can contribute a fix for this bug independently
• I would be willing to contribute a fix for this bug with guidance from the Iceberg community
• I cannot contribute a fix for this bug at this time

[orlp]

We have the same issue, this is breaking Polars' CI.

It seems that 0.9.0 is missing the 3.10+ manylinux wheel that 0.8.0 has:

• https://pypi.org/project/pyiceberg-core/0.8.0/#files has pyiceberg_core-0.8.0-cp310-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

• https://pypi.org/project/pyiceberg-core/0.9.0/#files doesn't have a cp310-abi3 for linux x86_64.

[kevinjqliu]

Thanks for reporting! Let me look into this

[kevinjqliu]

Found the issue, the github workflow run picked up python 3.8 instead of the specified python 3.12!

https://github.com/apache/iceberg-rust/actions/runs/23356765173/job/67949517681

🔗 Found pyo3 bindings with abi3 support
🐍 Found CPython 3.8 at /opt/python/cp38-cp38/bin/python3
📡 Using build options features from pyproject.toml
⚠️ Warning: CPython 3.8 at /opt/python/cp38-cp38/bin/python3 does not yet support abi3 so the build artifacts will be version-specific.

[kevinjqliu]

@wufh43 @orlp thank you both for the heads up. I've created a fix at apache/iceberg-rust#2277

Ran the branch in our nightly pipeline

I now see manylinux2014_x86_64 with cp310-abi3
https://test.pypi.org/project/pyiceberg-core/0.9.0.dev20260323172059/#files

We'll likely do a patch release for this