GPU-Native Adaptive Mesh Refinement with Application to Lattice Boltzmann Simulations

The Lattice Boltzmann Method (LBM) has garnered significant interest in General-Purpose GPU (GPGPU) programming for computational fluid dynamics due to its straightforward GPU parallelization. Adaptive Mesh Refinement (AMR) can assist in efficiently resolving flows with regions of interest requiring a high degree of resolution while providing a means of coarsening when this degree of resolution is no longer needed at runtime. AMR is commonly implemented in parallel on the CPU or in a hybrid frameworks with GPU-acceleration rather than natively on the GPU due to lack of a recursive data structure. An AMR scheme that could manage a computational mesh entirely on the GPU without intermediate data transfers to/from the host device would provide a substantial speedup, however, this is tackled scarcely in the literature and open-source codes are not available. Consequently, the LBM with AMR continues to be parallelized in hybrid frameworks in recent literature. The current work addresses these two concerns: 1) a block-based GPU-native algorithm is developed for AMR in two- and three-dimensions and implemented in an open-source C++ code, and 2) a Lattice Boltzmann solver for incompressible hydrodynamic flows is constructed and equipped to the mesh. Although the implementation is tailored for LBM simulations, the outlined grid refinement procedure is compatible with other solvers over cell-centered block-based grids. Validation for three choices of velocity set and analysis of performance in terms of solver speed with single- and double-precision, fraction of time communicating between grids, and fraction of time in refinement are conducted using the lid-driven cavity and flow past a square cylinder test cases. Consumer- and datacenter-grade GPUs are both used to run the test cases in order to demonstrate versatility of the algorithm. Link to repository: https://github.com/KhodrJ/AGAL