Midway-UChicago¶

Midway is the HPC cluster of the University of Chicago, maintained by UChicago’s RCC.

$ ssh -Y <username>@midway.rcc.uchicago.edu

Building WEST¶

WEST executables can be compiled using the following script:

$ cat build_west.sh
#!/bin/bash

module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5

export F77=mpiifort
export CC=mpiicc
export MPIF90=mpiifort
export FC=mpiifort
export CFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
export FFLAGS="-O3 -fno-alias -ansi-alias -g -mkl -Bdynamic"
export BLAS_LIBS_SWITCH="external"
export BLAS_LIBS=" -lmkl_intel_lp64  -lmkl_sequential -lmkl_core"
export LAPACK_LIBS_SWITCH="external"
export LAPACK_LIBS=" "
export SCALAPACK_LIBS=" -lmkl_scalapack_lp64 -Wl,--start-group  -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_blacs_intelmpi_lp64 -Wl,--end-group"

./configure --enable-openmp=yes --enable-parallel=yes --with-scalapack=intel --with-hdf5=no
make -j 2 pw

cd West
make conf PYT=python3 PYT_LDFLAGS="`python3-config --ldflags --embed`"
make all

To use the script do:

$ bash build_west.sh

Running WEST Jobs¶

The following is an example executable script run_west.sh to run the wstat.x WEST executable on two nodes of the broadwl partition with 28 MPI ranks per node.

$ cat run_west.sh
#!/bin/bash
#SBATCH --time=00:30:00
#SBATCH --partition=broadwl
#SBATCH --nodes=2
#SBATCH --ntasks-per-node=28
#SBATCH --cpus-per-task=1

module load intelmpi/5.1+intel-16.0 mkl/2017.up4 python/cpython-3.8.5

export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
NTASKS=$(($SLURM_NTASKS_PER_NODE * $SLURM_JOB_NUM_NODES))

mpirun -np $NTASKS ./wstat.x -i wstat.in > wstat.out

Job submission is done with the following:

$ sbatch run_west.sh