Skip to content

CI Fix compilation issue with OpenMP inside conda env - #237

Open
lesteve wants to merge 23 commits into
joblib:masterfrom
lesteve:debug
Open

CI Fix compilation issue with OpenMP inside conda env#237
lesteve wants to merge 23 commits into
joblib:masterfrom
lesteve:debug

Conversation

@lesteve

@lesteve lesteve commented Sep 6, 2026

Copy link
Copy Markdown
Member

Fix #232

Here is a summary of my investigation:

  • there is a bug on macOS for nested login shells Conda folders added to end of PATH in sub login shell conda/conda#13500, the PATH is in the wrong order so system clang is picked instead of the clang symlink in the conda env. This cause the error about clang no knowing what to do with -fopenmp
  • minimally activated conda compilers is part of the picture GCC v15 & Clang v21 conda-forge/conda-forge-pinning-feedstock#8795. Now what happens is that CC is not set when you install compilers. So clang is the compiler rather than the full conda compiler executable i.e. x86_64-conda-<something>. Before CC being set would hide the macOS bug from previous bullet point
  • there is a number of combinatorial stuff that I tried before converging to something that works and doesn't seem to brittle/dodgy. I would rather avoid trying more things 😉.

@lesteve

lesteve commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Really? build log

continuous_integration/run_tests.sh: line 24:   742 Segmentation fault         PYTHONPATH=. python tests/empirical_scope_observation.py blas
Error: Process completed with exit code 139.

No idea where that was coming from but maybe when I was using source continuous_integration/<script>.sh, I haven't been able to reproduce with just ./continuous_integration/<script>.sh. I don't think it's worth investigating, but feel free to investigate if you don't agree with me 😉.

@lesteve lesteve changed the title Debug with upterm CI Fix compilation issue with OpenMP inside conda env Sep 6, 2026

@lesteve lesteve left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments about the changes to help review

# a new commit on the same branch
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}

@lesteve lesteve Sep 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This wasn't cancelling the build before when pushing a new commit, I took it from scikit-learn

# Need to use this shell to get conda working properly.
# See https://github.com/marketplace/actions/setup-miniconda#important
shell: ${{ matrix.os == 'windows-latest' && 'cmd /C CALL {0}' || 'bash -el {0}' }}
shell: 'bash -el {0}'

@lesteve lesteve Sep 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This made my life easier to always use bash, probably an historical thing to use CMD on Windows, not 100% sure

- name: Install dependencies
run: |
bash -el continuous_integration/install.sh
./continuous_integration/install.sh

@lesteve lesteve Sep 6, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not using bash -l avoids the nested login shell bug in macOS where system clang is picked first rather than conda env symlink

fi

# Need to source conda.sh before first conda command
source "$CONDA/etc/profile.d/conda.sh"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sourcing the conda.sh file is the simpler way to make conda command work on all OSes inside a script.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI is failing on macOS due to some OpenMP compilation issues

1 participant