Installation

This section describes the basic installation of the framework. See also the Clusters page in case you are installing the code to some computing cluster.

Git + GitHub access with SSH

Before we start, we need a developer level access to GitHub. Most importantly, you need to ensure that you can access git repositories via the SSH connection. Follow these tutorials to link your SSH key to your GitHub account:

Downloading/Cloning

The framework relies on various small (template) libraries that are automatically downloaded along the main code as git submodules. Because of this, remember to always issue a recursive clone when downloading the code from GitHub:

git clone --recursive git@github.com:natj/runko.git

You also need to update all the submodules (so that various branches are in sync) with

git submodule update --recursive

Note

Note that the above git command uses the ssh address of the repository (git@github.com) that requires that you have the SSH+GitHub access set. If you want to revert back to the old https access, replace the first command with git clone --recursive https://github.com/natj/runko.git.

External libraries

External libraries installed together with the framework include:

corgi massively-parallel grid infrastructure
PyBind11 binding library for seamless operability between C++ and Python
mpi4cpp high-level interface to MPI library
ezh5 lazy template interface to HDF5 for easier usage (originally from mileschen )
cppitertools for various python-like C++ iterators etc.

Requirements

Before proceeding with the code compilation, your system needs to fulfill these requirements:

Modern C++ compiler (such g++, Clang++, etc.)
MPI message passing interface
python3 (with mpi4py, numpy, scipy, matplotlib)
HDF5 for I/O (serial mode is enough)
CMake (>v3.0) for building and compiling

Note

Note that g++-8 does not work because of a (known) compiler bug. Therefore, g++-9 or newer is the current recommended choice.

MacOS

On MacOS these are easily installed with homebrew by running:

brew install gcc hdf5 python3 cmake wget

Make also sure that Xcode developer tools are installed by running

xcode-select --install

MPI needs to be compiled separately because, by default, it uses the AppleClang compiler (instead of the g++ that you just installed).

You can compile OpenMPI via homebrew by first modifying your ~/.bash_profile to link the new compilers:

export HOMEBREW_CC=gcc-13
export HOMEBREW_CXX=g++-13
export OMPI_CC=gcc-13
export OMPI_CXX=g++-13

Then restart the terminal to reload the newly added environment variables. After restarting, install OpenMPI from source with

brew reinstall openmpi -cc=gcc-13 --build-from-source

and

brew reinstall mpi4py

Linux (Ubuntu)

When compiling runko and running the scripts, it is critical that you always use the same Python interpreter, C/C++ compiler, and associated OpenMPI distribution, otherwise this can give several errors during the installation. For this reason we recommend using vanilla python3 and disabling anaconda (if you are using it) by commenting out its activation in your ~/.bashrc file.

# >>> conda initialize >>>
# ...
# <<< conda initialize <<<

You may find it also necessary to delete folders containing the older Python versions than your current one at /usr/bin/python3.*. In order to get a completely clean OpenMPI distribution first run:

sudo apt-get remove mpich libopenmpi-dev openmpi-bin
sudo apt-get update && sudo apt-get autoclean && sudo apt-get clean && sudo apt-get autoremove

Then run:

sudo -E apt-add-repository -y "ppa:ubuntu-toolchain-r/test"
sudo apt-get install libopenmpi-dev libhdf5-serial-dev hdf5-helpers openmpi-bin libblas-dev liblapack-dev python3 python3-pip

Note

Recent Ubuntu (bionic) comes with gcc-7 which makes the installation easier. For previous versions you, additionally, need to install gcc-7 (or 9) and manually compile MPI similar to the MacOS discussed above.

You also need to export the HDF5 library location (since it is non-standard at least in Ubuntu) with

export HDF5_INCLUDE_PATH=/usr/include/hdf5/serial

Manual installation of OpenMPI (optional)

Alternatively, if you want even more control of the operation, you can compile it manually yourself by running:

export MPI_IMPL=openmpi41
mkdir -p $HOME/local/$MPI_IMPL/bin/openmpi
cd $HOME/local/$MPI_IMPL/bin/openmpi
wget --no-check-certificate http://www.open-mpi.org/software/ompi/v4.1/downloads/openmpi-4.1.5.tar.bz2
tar -xjf openmpi-4.1.5.tar.bz2
cd openmpi-4.1.5
export OMPI_CC=gcc-13
export OMPI_CXX=g++-13
./configure CC=gcc-13 CXX=g++-13 --prefix=$HOME/bin/$MPI_IMPL
make -j 4
make install
make clean

This installs OpenMPI to ~/bin/ and exports the correct directories so that the mpic++ compiler wrapper becomes available. You should then add to your .bash_profile (or .zshrc in latest MacOS) these exports (in case you need to re-compile the library):

export OMPI_CC=gcc-13
export OMPI_CXX=g++-13
export MPI_IMPL=openmpi41
export PATH=$PATH:$HOME/bin/$MPI_IMPL/bin
export PATH=$PATH:$HOME/bin/$MPI_IMPL/include
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/bin/$MPI_IMPL/lib

After OpenMPI is installed we also need to re-install mpi4py because it uses the system-default mpi installation

pip3 uninstall mpi4py --break-system-packages
pip3 install mpi4py --break-system-packages

Note the additional --break-system-packages keyword that is needed for the latest python versions >3.12 to install packages with pip and homebrew/apt-get.

Python libraries

All the python requirements can be installed via pip3 as

pip3 install -r requirements.txt

Compiling

After installing all the pre-requisites, you can proceed to compiling. First you need to configure the build. To use your (freshly installed) modern C++ compiler we need to export them as

export CC=mpicc
export CXX=mpic++

Then make sure that everything works, check the output of

$CC --version
$CXX --version

This should indicate that the newly installed compilers are used.

You should also put this part into your ~/.bashrc (or ~/.zshrc in the latest MacOS) so that correct compilers are automatically exported during the startup.

You should also add the python script directories into PYTHONPATH environment variable. Modify your ~/.zshrc (MacOS) or ~/.bashrc (Linux) by appending corgi and runko libraries to the path by exporting

export RUNKO=/path2repo
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}$RUNKO/"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}$RUNKO/lib"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}$RUNKO/corgi/lib"
PYTHONPATH="${PYTHONPATH:+${PYTHONPATH}:}$RUNKO/bindings/old"
export PYTHONPATH

where path2repo points to the location where you cloned the repository (i.e. path to runko directory). Note that there is no trailing slash / symbol in the commands. As an example, the path can be e.g., /Users/natj/runko.

Next we can proceed to compiling. Out-of-source builds are recommended: inside the repository directory, make a new build directory, go into that, and only then run the CMake configuration commands. This can be done by running (inside runko directory):

mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DPYTHON_EXECUTABLE=$(which python3) ..

And make sure to check that CMake finishes successfully. After that, you are ready to compile the framework with

make

When compiling and linking is finished, CMake runs few automated tests to check that everything is working. You should see a message “XX tests finished succesfully” in the end, if the build was successful.

Note

Since the compiling can take quite a while, you can use the multi-core compilation by make -j8 (or whatever number of tasks you want).

Testing of the new installation

Runko comes with multiple tests (found in runko/tests/) that are ran after every compilation. In general, if you see “All tests passed.”, after the compilation, your installation should be succesfull.

The next step is to run an actual simulation with the code. For that, see the premade projects setups in runko/projects/.