Skip to content

DPL: add option to force parentFile colocation#15391

Open
ktf wants to merge 1 commit into
AliceO2Group:devfrom
ktf:pr15391
Open

DPL: add option to force parentFile colocation#15391
ktf wants to merge 1 commit into
AliceO2Group:devfrom
ktf:pr15391

Conversation

@ktf

@ktf ktf commented May 11, 2026

Copy link
Copy Markdown
Member

No description provided.

@ktf ktf requested a review from a team as a code owner May 11, 2026 20:07
@alibuild

alibuild commented Jun 10, 2026

Copy link
Copy Markdown
Collaborator

Error while checking build/O2/fullCI_slc9 for 883a847 at 2026-07-06 02:19:

## sw/BUILD/O2-sim-challenge-test-latest/log
./sim-challenge.log[FATAL] TGrid::Connect returned nullptr. May be due to missing alien token
./grp.log[FATAL] TGrid::Connect returned nullptr. May be due to missing alien token


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/15590201305405d349786042af881d9822814aaf/slc9_x86-64/o2checkcode/1.0-local192/etc/modulefiles
++ alibuild-generate-module
+ HAS_BIN=
+ HAS_LIB=
+ HAS_CMAKE=
+ HAS_ROOT=
+ '[' 0 '!=' 0 ']'
+ cat
+ printf 'if ![ is-loaded '\''BASE/1.0'\'' ] {\n module load BASE/1.0\n}'
+ echo 'BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release'
BUILD_REQUIRES=CMake alibuild-recipe-tools defaults-release
+ FULL_BUILD_REQUIRES='date defaults-release pytorch_cpuinfo FairCMakeModules ninja-fortran CMake googlebenchmark safe_int UUID MPFR Alice-GRID-Utils bz2 double-conversion ninja Python-modules-list O2-customization json-c alibuild-recipe-tools'
+ echo 'FULL_BUILD_REQUIRES=date defaults-release pytorch_cpuinfo FairCMakeModules ninja-fortran CMake googlebenchmark safe_int UUID MPFR Alice-GRID-Utils bz2 double-conversion ninja Python-modules-list O2-customization json-c alibuild-recipe-tools'
FULL_BUILD_REQUIRES=date defaults-release pytorch_cpuinfo FairCMakeModules ninja-fortran CMake googlebenchmark safe_int UUID MPFR Alice-GRID-Utils bz2 double-conversion ninja Python-modules-list O2-customization json-c alibuild-recipe-tools
++ env
++ cut -f1 -d=
++ grep -v '^DEFAULT_'
++ grep -v PKGREVISION
++ grep -v ALIBUILD_RECIPE_TOOLS
++ grep REVISION
++ sed -e s/_REVISION//
+ for x in $(env | cut -f1 -d= | grep -v "^DEFAULT_" | grep -v PKGREVISION | grep -v ALIBUILD_RECIPE_TOOLS | grep REVISION | sed -e 's/_REVISION//')
++ eval 'echo $LIBFFI_REVISION'
+++ echo 10
+ REVISION_VALUE=10
++ eval 'echo $LIBFFI_VERSION'
+++ echo v3.2.1-alice1
+ VERSION_VALUE=v3.2.1-alice1
++ eval 'echo $LIBFFI_ROOT'
+++ echo /sw/slc9_x86-64/libffi/v3.2.1-alice1-10
+ ROOT_PATH_VALUE=/sw/slc9_x86-64/libffi/v3.2.1-alice1-10
+ '[' -z 10 ']'
+ echo 'date defaults-release pytorch_cpuinfo FairCMakeModules ninja-fortran CMake googlebenchmark safe_int UUID MPFR Alice-GRID-Utils bz2 double-conversion ninja Python-modules-list O2-customization json-c alibuild-recipe-tools'
+ tr '[:lower:]' '[:upper:]'
+ tr - _
+ tr ' ' '\n'
[0 more errors; see full log]

Full log here.

@jgrosseo

Copy link
Copy Markdown
Collaborator

@costing I vaguely remember that there are SEs which are counted equivalent because they are close by. Would the check created by @ktf work for this case?

@costing

costing commented Jun 11, 2026

Copy link
Copy Markdown
Collaborator

I think the exact SE name is checked in the whereis output. Which is ok, the secondary files are supposed to be stored in the same locations as the original ones, not in nearby SEs.

Just that enabling this check for every file will double the load on the catalogue. We already return the list of PFNs at a ::Open() of the original and the secondary files. So if the check could be deferred to when the files are actually about to be used it would not put additional pressure on the databases. In this form it can be used for one time debugging / optimizing step, but it cannot be done regularly or by default.

@jgrosseo

Copy link
Copy Markdown
Collaborator

I agree. This should run as an extra special train and not be enabled for user trains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants