OSU Microbenchmarks on Elja

[p-costa]

Running the OSU Microbenchmarks on Elja

Installation script

The benchmarks can be installed using the following bash script.

#!/bin/bash
#
BENCH_NAME=osu-micro-benchmarks-5.8
TARGET_DIR=$HOME/$BENCH_NAME
#
# load gnu programming environment
#
module purge
module load gnu9 openmpi4
#
# download library and compile benchmarks
#
wget https://mvapich.cse.ohio-state.edu/download/mvapich/$BENCH_NAME.tgz
tar -xzvf $BENCH_NAME.tgz
cd $BENCH_NAME
MPI_BIN_DIR=/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/bin
mkdir -p $TARGET_DIR
./configure CC=$MPI_BIN_DIR/mpicc CXX=$MPI_BIN_DIR/mpicxx --prefix=$TARGET_DIR && make && make install
echo "*** Done! MPI benchmarks under folder: $TARGET_DIR/mpi ***"

Running the MPI benchmarks

The suite of MPI benchmarks is under the folder mpi/. I was interested in the benchmarks of collective operations, which are located under mpi/collective/. The following SLURM script needs to be run from one of the subfolders (such as mpi/collectives/), and loops through all the executables, and outputs the results in a results/ folder:

#!/bin/bash -l
#
#SBATCH -N 1
#SBATCH --ntasks-per-node=48
#SBATCH --job-name=OSU_BENCHMARKS_COLLECTIVE_MPI
#SBATCH --time=0-01:00:00
#SBATCH --get-user-env
#
RESULTS_DIR=results/
mkdir -p $RESULTS_DIR
for i in $(ls osu_* | grep -v '\.'); do
  log_file=${RESULTS_DIR}out_N_${SLURM_JOB_NUM_NODES}_n_${SLURM_NTASKS}_$i.log
  err_file=${RESULTS_DIR}err_N_${SLURM_JOB_NUM_NODES}_n_${SLURM_NTASKS}_$i.log
  echo "# Job name, ID: $SLURM_JOB_NAME, $SLURM_JOB_ID"        >  $log_file
  echo "# Submission directory: $SLURM_SUBMIT_DIR"             >> $log_file
  echo "# Number of nodes: $SLURM_JOB_NUM_NODES"               >> $log_file
  echo "# List of nodes: $SLURM_JOB_NODELIST"                  >> $log_file
  echo "# List of number of tasks/node: $SLURM_TASKS_PER_NODE" >> $log_file
  echo "# Total number of tasks: $SLURM_NTASKS"                >> $log_file

  mpirun $i >> $log_file 2> $err_file
done