Recently we have developed a new, publicly available, large eddy simulation code for the simulation of atmospheric turbulence (Pressel et al., 2015). The code is called PyCLES and is available on Github. Here we discuss aspects of its design that position it well for answering fundamental questions regarding clouds in the climate system.  

Perhaps the most widely discussed uncertainties in predictions of climate sensitivity are related to boundary layer clouds and their climate feedbacks (e.g., Bony and Dufresne, 2005; Vial et al., 2013), both because of the size of the uncertainties and how defiant they have been to reduction. Clouds are problematic because they exert strong control on the large-scale climate, mostly through their radiative effects, yet the energetic turbulent dynamics of clouds themselves occur on scales down to meters or centimeters. These scales are much smaller than those directly resolved in general circulation models (GCM).  This makes them critically important, but at the same time incredibly difficult to parameterize in GCMs.

Developing a general theory for clouds that can serve as a basis for parameterization in large-scale models is difficult for many reasons, not the least of which is the fact that observing the instantaneous, three-dimensional turbulent structure of clouds, say from aircraft, is essentially impossible. This, along with ever increasing high-performance computing resources, has led to reliance on computational methods to directly simulate important parts of the unobservable turbulent dynamics of clouds.  In essence, we rely on supercomputers to solve equations, namely those dictated by Newton’s laws of motion and fairly well understood thermodynamic principles, that generate three-dimensional cloud fields. These simulated cloud fields provide a three-dimensional picture of cloud dynamics to aid in the development of GCM parameterization.

One of the most widely used approaches to simulating clouds is called Large Eddy Simulation (LES). Because resolving all scales of turbulent motion in clouds, even for limited areas, remains beyond the capability of modern computing, LES seeks only to directly represent the large features (on scales of meters) of the three-dimensional turbulent motions in clouds, and then parameterize the rest. Despite what their name suggests,  large eddy simulations are high-resolution simulations in the hierarchy of computational models for the atmosphere, which resolve much smaller eddies than climate models. However, since they do not fully resolve all length scales relevant to clouds, LES provide a simplified view of reality, and not a panacea. LES results are sensitive to fine details of the equations used to represent the dynamics and thermodynamics of clouds and the way the equations are solved numerically (Ghosal, 1996; Chow and Moin, 2003). With PyCLES (Python Cloud Large Eddy Simulation), we have attempted to address many of these potential sensitivities and have developed a modern LES code ready to tackle many of the challanges posed by clouds in the climate system. Below, we will describe the specific features that make PyCLES an important tool for studying cloud-climate interactions.  We begin by discussing the novel software design of PyCLES and then turn to a discussion of unique aspects of its dynamical equations and numerical implementation.

Software Design 

We developed PyCLES from the ground up, and the code base is not derivative of any other LES code. This gave us the freedom to completely re-envision the software aspects of the design. For example, we prioritized ease of use and extensibility. To this end, the code has been written in a combination of Python, Cython, and C. Python and statically-typed Cython source code are used to construct interfaces to pure C code that performs the bulk of the numerical computations. The use of C for the most computationally expensive portions of the code allows the use of Python and Cython with only a minor compromise in performance.  (For the Python aficionados: Most of the data structures are based on Numpy arrays, allowing use of the full Numpy and Scipy tool stack from within PyCLES itself.)

Governing Equations

The most general set of equations that govern the turbulent motions of clouds are the compressible Navier-Stokes equations, which are a statement of Newton’s laws of motion for a fluid. By being so general, these equations permit the propagation of sound waves, which are fast and therefore make solving these equation numerically very expensive. What’s more, these sound waves are of little consequence to the dynamics of clouds. Therefore, it is advantageous to eliminate sound waves from the system by modifying the equations of motion. There are many ways to do this, but some formulations require approximations to the equations of motion that can lead to significant bias, particularly in thermodynamic variables such as temperature. In PyCLES, we have sought to avoid these biases, while still eliminating sound waves, by solving the energetically consistent form of the anelastic equations of motion described by Pauluis (2008). This form of the anelastic approximation remains thermodynamically consistent even for deep atmospheric motions, as occur in tropical deep convection, by computing buoyant accelerations caused by density fluctuations around a moist isentropic reference state. This makes it natural to use moist entropy as prognosed variable, and indeed PyCLES is the only LES of its type to use moist specific entropy as a directly prognosed model variable. An additional benefit of using moist entropy as a prognostic variable is that assumptions used in modeling sub-gridscale turbulent mixing are more well posed.

Numerics 

Selecting the governing equations is only part of the process of formulating a LES code. Given a set of governing equations, these equations must be recast in such a way that they can be solved numerically on computers. The typical approach to this, and the one employed in PyCLES, is to represent the spatial domain as a discrete mesh and approximate derivatives in the equations of motion as finite differences on that mesh. However, there are many finite difference approximations, each having their own numerical properties and error characteristics. Of particular importance are the terms in the governing equations representing advective transport of mass and momentum. The form of these terms make them particularly susceptible to oscillatory numerical error that can seriously degrade the quality of simulations. We equipped PyCLES with relatively recently developed Weighted Essentially Non-Oscillatory (WENO) finite difference schemes, which seek to avoid the spurious numerical oscillations (Liu et al., 1994). The benefit of using WENO schemes is striking. For example, Figure 1 shows the vertical velocity variance in non-precipitating stratocumulus clouds as simulated by PyCLES using both a traditional centered difference schemes and a WENO scheme. Stratocumulus cloud fields of this type are know to pose significant challenges for LES (e.g., Stevens 2005). Observations for this case are depicted with black circles.  Clearly, the simulation using WENO schemes maintains much greater fidelity to observations.

Figure 1: Vertical velocity variance simulated using PyCLES. The blue and red lines show the results from simulations using 5th order WENO and 6th order central schemes, respectively. Observations from Stevens (2005) are shown in black circles. 

Simulations: 

The development of PyCLES began in earnest in January of 2014, and since then the code base has matured and is now being used by a growing number of researchers. PyCLES has been used to simulate a wide variety of boundary layer cloud types. In the movie below we show a numerical simulation of the DyCOMS-II RF01 stratocumulus-topped boundary layer, the same case as shown in Figure 1. The movie shows the liquid water path, which is roughly a proxy for what you would see if you looked at the cloud from above. Its remarkable realism suggests the power of LES to improve our understanding of clouds.

Simulation of stratocumulus clouds with PyCLES.