Multi-Resolution Coupling for Exascale Engineering
ARCHER2-eCSE01-28
PI: Prof Alistair Revell (University of Manchester)
Co-Is: Dr Charles Moulinec (Science and Technology Facilities Council – Daresbury Laboratory), Dr Alessandro De Rosis (University of Manchester), Dr Jianping Meng (Science and Technology Facilities Council – Daresbury Laboratory)
Technical staff: Dr Marta Camps Santasmasas (University of Manchester), Dr Ian Hinder (University of Manchester), Dr Chrysovalantis Tsinginos (Science and Technology Facilities Council – Daresbury Laboratory)
Subject Area:
This project has made important progress in demonstrating a practical and efficient framework to enable computational science to bridge scales from both macro- and microscale physics, to mesoscale modelling, with the increasingly popular Lattice Boltzmann method (LBM). The primary objective of this project was to provide a highly parallel and versatile framework for the coupling of Code_Saturne, a proven pre-Exascale-ready Finite Volume Method (FVM) code, to LUMA, an efficient and versatile LBM code, in order to leverage the potential of mesoscale modelling for industrial engineering simulation. In order to prepare for full multi-scale and a 3-way coupling, we have also coupled LUMA to LAMMPS, a popular and extensive code for molecular dynamics (MD) and particle evolution simulations through the discrete element method (DEM).
The Lattice Boltzmann method is referred to as a mesoscale method on account of its ability to capture the mechanics of matter that lies between the gap of what we can see (macroscale) and what we know constitutes matter itself (microscale). Mesoscale methods are able to capture some of the physics and mechanisms of statistical mechanics in a compact and direct manner. It is very well suited to the modelling of multi-physics processes across a range of scales, as has been confirmed by the continuous success of the UK Consortium on Mesoscale Engineering Sciences (UKCOMES), a HEC Consortia formed in 2013 and renewed in 2018. Nevertheless, used alone, LBM struggles to span physical scales necessary for system-level computation, which has thus limited its use in engineering context. Indeed, usage of LBM on ARCHER2 has to date been modest. In contrast, the FVM is an excellent and highly scalable technique to simulate single-phase processes at the macro-scale with proven capabilities to meet system-scale simulation challenges. Code_Saturne is a highly capable unstructured FVM code which is massively-parallel oriented and has a strong track record on leadership-class systems.
Motivation for so-called multi-scale modelling is all around us in the everyday world, where small changes at a very small scale can impact on large, system-level, engineering process, e.g. the impact of surface degradation on performance of a heat exchanger, or clogging-up over time of porous surfaces. In coupling the LBM method to the popular MD and DEM software (LAMMPS) we have demonstrated the potential for LBM to act as a bridge from macro to microscale simulation. This work provides first steps and much more physics can be incorporated to truly explore its potential. For more details check out this tutorial. Instructions for building the codes from their source repositories on the correct branches for coupling are provided.
Figures (clockwise from top left): diagram of coupling between LUMA and Code_Saturne; flow field for lid driven cavity; velocity profiles across coupling interface; weak scaling test up to 500 nodes
Information about the code
Versions of Code_Saturne and LUMA with support for coupling to each other are installed in /work/ecseaa28/shared/software. To add them to your environment, use the command:
source /work/ecseaa28/shared/software/cs-luma-ecse0128/setup.sh
A tutorial for running a coupled lid-driven cavity case is provided.