Lawrence Livermore National Laboratory

We work with a number of turbulence and turbulent-reaction codes developed with Department of Energy Funding, some by the national labs, some by academic researchers. All contain state-of-the-art numerics, turbulent transport, mixing and combustion capabilities. TASC staff members are also developing further capabilities of several of these codes, for particular application to problems encountered at LLNL and elsewhere within the DOE complex.


MARGOT has been developed by the TASC staff for high-fidelity large-eddy and direct-numerical simulations of unsteady hypersonic flows in massively-parallel configurations. MARGOT contains an underlying numerical framework similar to the Charles and Chris codes, described below, and in addition, hypersonic aerothermodynamics up to approximately Mach 15. MARGOT treats nitrogen-oxygen chemistry of the air, using a five-equation finite rate system, and ionization chemistry with an eleven-equation system. Temperatures are partitioned between vibrational, rotational, and translational modes and tied to the transport properties of the flow, such as the viscosity and the thermal diffusivity. In addition, the code contains a one-dimensional conjugate heat transfer (CHT) model, and a full three-dimensional CHT model is currently under development. A radiative heat transfer capability is also combined with the CHT model. Extensions to treating ablative problems at M > 20 are currently under development, including on-the-fly boundary motion while simulations progress. We believe MARGOT to be the first-of-its-kind code for LES and DNS of unsteady hypersonic flows in the nation.


Developed by Stanford University's Center for Turbulence Research (CTR), CHARLES is a high-fidelity unstructured compressible flow solver for Large Eddy Simulation (LES) and is ideally suited for aeroacoustic applications involving unsteady high-speed flows and complex geometries. Charles solves the spatially-filtered compressible Navier-Stokes equations using a novel finite-volume method involving a blended flux approach using non-dissipative central flux and dissipative upwind flux. The Dynamic Smagorinsky and Vreman models are available to model the physical effects of unresolved turbulence. Shocks are handled using a hybrid central-ENO scheme along with the HLLC approximate Riemann solver. There are a number of additional features, such as digital filtering for synthetic inflow turbulence and a wall-model for high-Reynolds number wall bounded flows. CHARLES scales extremely well for massively parallel simulations and in 2013 demonstrated nearly linear scaling up to 1.5 million cores on LLNL’s Sequoia supercomputer.


JOE is CTR’s state-of-the-art multiphysics RANS/URANS solver. It uses a cell-centered finite-volume method. Multiple time integration schemes are available. Several turbulence models are available to the user including the Spalart-Allmaras, Menter-SST, k-omega, and V2F models. JOE uses the Flamelet Progress Variable Approach for combustion modeling.


The BRADY code is a substantially modifed version of the Structured Pierce Code, described below, for incompressible and variable-density incompressible turbulent mixing and reactive flows. The code includes a complete implementation of the nLES Method on structured grids, as well as 4th-order centered derivative and filtering operators throughout. BRADY is currently being upgraded with a hybrid MPI/OpenMP capability that will allow for rapid turnaround time of exceedingly large simulations, so far tested in configurations of over 1GB points on over 20K cores.


The DOOLIN code is a substantially modified version of the NGA code, described below, for incompressible and variable-density incompressible turbulent flows with or without reaction. DOOLIN contains a full implementation of the nLES Method on structured cartesian and cylindrical meshes, and scales well for massively parallel (10K+ cores) simulation configurations.


CharLES is Cascade Technology Inc.’s flagship product, and shares a common progeny with the CTR's CharlesX code, described above.


Cascade Technology Inc.’s unstructured compressible reacting flow solver, CHRIS, has been developed for LES of reacting flows where compressibility effects are important. CHRIS solves the filtered compressible Navier-Stokes equations in conservative form. A novel hybrid flux using a blend of upwind and central flux, similar to that used in CharLES, is employed to minimize dissipation and dispersion errors. A 3rd order TVD RK scheme is used to advance the solution in time. A hybrid central-ENO scheme is used to capture shocks and material interfaces, along with the HLLC approximate Riemann solver. CHRIS’s chemistry modeling is based on the flamelet progress variable approach.


Cascade’s multi-physics variable density flow solver VIDA provides an enabling technology for high-fidelity LES of mixing and reacting flows in low-Mach number regimes. VIDA’s formulation decouples pressure from density and temperature, removing any acoustic restrictions on the time step. This property, combined with an implicit fractional step method results in a very efficient time integration method. VIDA uses the flamelet progress-variable chemistry model in which a turbulent flame is described as an ensemble of laminar flamelets using a presumed pdf. In addition, the Lagrangian spray models in the VIDA solver allow for the coupled simulation of liquid-phase fuel spray and evaporation.


CLIFF is Cascade Technologies’ unstructured incompressible flow solver. CLIFF solves the incompressible Navier-Stokes equations using a node-based finite volume method with time integration based on the fractional-step method.


ADAPT is a massively parallel tool developed in Cascade’s solver infrastructure that gives the user detailed control over the local mesh resolution in their grid. The underlying parallel refinement algorithm can refine elements locally to match a target length scale. This target length scale can vary throughout the domain, and can even be different in each direction. The length scale can even come from a solution on an un-adapted or partially adapted mesh. The ability to refine elements in the direction or directions necessary to meet target length scale requirements dramatically reduces the overall mesh size and prevents the addition of stiffness to the problem due to excessively small elements. ADAPT also provides a surface projection algorithm to respect non-planar mesh boundaries during the refinement process, ensuring accurate representation of the underlying geometry.

Structured Pierce Code

Developed principally by Charles D. Pierce (2003) at Stanford's Center for Turbulence Research, the Structured Pierce Code (SPC) is a structured variable-density incompressible Navier-Stokes solver for large eddy simulation (LES) and direct numerical simulation (DNS) of multi-physics multi-scale turbulent flows involving radiation, heat transfer, reaction and high Schmidt-number mixing.


NGA is a structured variable-density incompressible solver for LES and DNS of reacting turbulent flows, using derivative operators of arbitriably high order.


Developed principally by Drs. Andrew Cook and William Cabot in A-Division, MIRANDA is a structured Navier Stokes flow solver, with multi-physics capability including radiation and combustion, using high-order compact derivative and filtering operators. MIRANDA was used to run the largest-ever (N = 3072^3) landmark DNS study of 0.5 Atwood-number Rayleigh-Taylor mixing, using over 132K cores on LLNL's BlueGene/L supercomputer.


Multiphase coal combustion RANS code developed by the National Energy Technology Laboratory (NETL) in Morgantown, West Virginia. It includes a capability to run in a massively-parallel configuration.