Introduction
Lattice Boltzmann Methods (LBM) are a relatively recent approach to the simulation of fluid flows and transport problems in general. Due to their very good parallelizability LBM are particularly suited to perform large scale simulations on modern HPC clusters.
The open source LBM library OpenLB is developed by the Lattice Boltzmann Research Group (LBRG, KIT) together with partners as a foundation for deepening the understanding of transport problems in process engineering and medicine. This ensures not only the reproducability of existing research but also aids future research by providing a trusted and easily extensible simulation toolbox.
| Principal Investigator: | PD Dr. Mathias J. Krause |
| Project Manager: | PD Dr. Mathias J. Krause |
| Researchers: | M.Sc. Fedor Bukreev, M.Sc. Nicolas Hafen, M.Sc. Julius Jeßberger, M.Sc. Adrian Kummerländer, M.Sc. Jan E. Marquardt, M.Sc. Johanna Mödl, M.Sc. Anas Selmi, M.Sc. Stephan Simonis, M.Sc. Dennis Teutscher, M.Sc. Mingliang Zhong, Dr. Davide Dapelo, Ing. B.Sc. František Prinz |
| Date published: | June 2021 |
| HPC Platform used: | HoreKa |
| Institutes: | Institute for Applied and Numerical Mathematics, KIT Institute for Mechanical Process Engineering and Mechanics, KIT |
| Affiliations: | Karlsruhe Institute of Technology (KIT) University of Bradford Brno University of Technology (VUT) |
| Research Field: | Computational Engineering Sciences |
| Project ID: | CPE |
Project description
Approaches to modeling fluid dynamics may be coarsely grouped into microscropic, mesoscopic and macroscopic approach. The first directly model the individual particles of which the fluid behaviour is but an emergent property while the latter directly consider macroscopic quantities such as velocity and pressure. Mesoscopic approaches model the probability of some particle density moving at a given speed at given points in time and space. A common example for such a model is the Boltzmann transport equation which can be connected to microscopic and macroscopic models using the mathematical field of kinetic theory.
Lattice Boltzmann Methods discretize such mesoscopic equations on a regular lattice, yielding a two-step algorithm consisting of a purely local and thus perfectly parallel collision step and a streaming step that only communicates in a local cell neighborhood. These properties render LBM particularly suited to highly parallel execution on heterogeneous HPC clusters.
OpenLB https://www.openlb.net/ implements LBM in a hybrid parallelization scheme consiting of a block-decomposition that is parallelized using MPI while individual blocks may be processed by CPUs and accelerators such as GPUs on the level of computation nodes. In this basic framework a wide variety of collision operators and boundary conditions are implemented, allowing for the efficient simulation of:
- Particulate fluid flows
- Thermal flows
- Turbulent flows
- Material transport and chemical reactive flows
- Light transport
- Fluid-structure interaction
- Flows in porous media and complex geometries.
This rich selection of simulatable phenomena allows for the investigation of various problems both in process engineering and medicine. For instance simulations of resolved particles allow for the detailed study of shape-dependent settling velocities in particle sedimentations which is an essential question in process engineering. Simulations of multi-component chemical reactive flows can aid in the development of more efficient batteries and light transport simulations furthen the understanding of algae growth in photobioreactors. In medicine, OpenLB is used to simulate both air flows in patient-specific airway geometries and hemodynamics of the human aorta. The short time-to-solution that can be achieved by combining LBM with modern HPC clusters is essential for establishing the usage of simulations as a convenient tool to understand patient-specific issues as well as to explore possible treatments.
The C++-framework OpenLB is in active development since 15 years and started out as a MPI-only CPU code with optional support for utilizing OpenMP on the node level in a hybrid parallelization scheme. Automatic code generation is utilized to transform high-level descriptions of collision operators and boundary conditions into efficient computation kernels. Motivated by the availability of HoreKa as a state-of-the-art heterogeneous HPC system, large scale refactoring is currently ongoing to allow for the utilization of all collected models on modern SIMD CPUs and GPGPUs. This is augmented by extensive coverage of the codebase by unit and benchmark tests in an continuous integration (CI) pipeline.
References
M.J. Krause, A. Kummerländer, S.J. Avis, H. Kusumaatmaja, D. Dapelo, F. Klemens, M. Gaedtke, N. Hafen, A. Mink, R. Trunk, J.E. Marquardt, M.L. Maier, M. Haussmann, and S. Simonis. "OpenLB-Open source lattice Boltzmann code." Computers & Mathematics with Applications (2020). https://doi.org/10.1016/j.camwa.2020.04.033.
R. Trunk, C. Bretl, G. Thäter, H. Nirschl, M. Dorn, and M.J. Krause. "A Study on Shape-Dependent Settling of Single Particles with Equal Volume Using Surface Resolved Simulations." Computation 9, no. 4 (March 25, 2021): 40. https://doi.org/10.3390/computation9040040.
T. Henn, G. Thäter, W. Dörfler, H. Nirschl, and M.J. Krause. "Parallel dilute particulate flow simulations in the human nasal cavity." Computers & Fluids 124 (2016), http://dx.doi.org/10.1016/j.compfluid.2015.08.002.