Purdue University

A research group from Columbia University has utilized Purdue’s Anvil supercomputer to run computational fluid dynamics simulations in order to learn more about atmospheric turbulence and the interactions between land surfaces and the atmosphere.

Dr. Marco Giometto is an Assistant Professor in the Civil Engineering and Engineering Mechanics Department at Columbia University, as well as the head of Columbia University’s Environmental Flow Physics Lab (EFPL). He and the rest of the EFPL group use supercomputers to conduct research that focuses on the study of flow phenomena involving turbulence, heat transfer, and evaporation, specifically within the atmospheric boundary layer (bottom layer of the atmosphere which is in contact with the surface of the earth). The team’s research varies widely, and includes topics such as turbulence and turbulent transport in urban and plant canopies, flow in complex terrain, hurricane boundary layer turbulence, and uncertainty quantification and artificial intelligence techniques for turbulence modeling. Though the specifics of each research project can be quite different, the group’s overall goal is to advance the understanding and ability to model turbulent transport in the atmosphere. The EFPL’s work has wide-ranging implications. From forecasting local weather and climate variability to air quality monitoring and urban planning, the applications of this research are nearly unending.

Atmospheric turbulence is classified as small-scale, chaotic airflow driven by winds that vary in speed and direction. Turbulence in the atmosphere transports pollutants, heat, moisture, etc., mixing and churning these substances and distributing them both vertically and horizontally throughout the landscape. Physical structures, such as houses, buildings, and trees, heavily influence atmospheric turbulence, changing how things like heat, energy, and air pollutants are exchanged between the earth’s surface and the atmosphere. Urbanization, therefore, has a major impact on local weather and climate variability.

As a crude example, imagine a swift-flowing creek. If a small rock is placed in the creek, it will impact the flow of the water, albeit not much, and perhaps not noticeably so. But if a boulder or a log were placed in the creek, it would alter the water’s flow significantly. Imagine a dam being built in the creek, and now imagine tiny holes or slits being cut through the dam. The water would continue to flow, but would pass through much more slowly than before. Some water would become trapped for a time before finally making its way through the gaps in the dam. Adding more twigs, rocks, logs—structures—would further change the flow, perhaps creating eddies and whirlpools. Leaves or runoff carried by the water could become trapped. The flow could slow enough to cause it to freeze over quicker than before, or perhaps so much so that it becomes completely frozen. The entire system could change, thanks to the newly added structures.

These same types of changes can occur in atmospheric turbulence when new structures are introduced to the landscape. Scientists have long known this to be the case, but recently, more effort has been placed into predicting these changes, or even into predicting turbulent airflow based on current structures, in order to improve city planning, local weather forecasting, and climate projections, to name a few. And this is exactly the type of research that the EFPL group conducts.

“We are a computational fluid dynamics group,” says Dr. Giometto. “We develop high-fidelity numerical algorithms for the simulation of turbulent transport phenomena in the atmosphere, and derive theories that describe the underlying physics controlling such processes”

As noted earlier, atmospheric turbulence transports things such as pollutants and heat, and the EFPL group studies the physics of that transport. Without understanding these physics at a fundamental level, it would be difficult to make any advancement in the field and drive the decision-making process in applications. The group also develops newer and better ways to model atmospheric turbulence and transport, experimenting with machine learning and uncertainty quantification techniques. The goal is to advance the field and allow for more efficient and more accurate predictions of atmospheric boundary layer processes.

The real-world impact of this type of research is hard to overstate. Being able to accurately model the interaction between the atmosphere and land will allow us to build more sustainable and equitable cities, reducing energy consumption and air pollution within urban areas that are already experiencing unprecedented levels of population growth. In short, it can help enhance our civilization through improved infrastructure.

“Our research not only helps improve our ability to forecast weather and climate variability, but also to understand how weather impacts our everyday life,” says Dr. Giometto. “Weather results are typically at a very coarse resolution, so our model can be used to downscale the wind and determine where wind gusts might happen. They can help better understand how if you place trees in a certain location, how that will affect the local airflow, and how that might benefit the energy usage of buildings or heat in that street. It also helps us better understand how urban geometry affects the ‘urban heat island effect’ [cities heat up more than rural surroundings] and how we can design cities that are more resilient and sustainable in their use of resources.”

The models developed by the EFPL group are also extraordinarily important for addressing equity-related issues. Numerous studies have been conducted that highlight how some neighborhoods heat up more or have poorer air quality than others within the same city. Existing atmospheric models, however, are not very accurate for the study of such problems.

“Many ethics-related questions are on a scale of 100 to 200 meters, so you need to be able to predict the temperature, wind, and air pollution at such small scales. Weather forecasting products cannot help you because they have a resolution of one by one kilometer, and existing weather downscaling approaches are based on assumptions that are not valid at small scales. Models we are developing are tailored to be accurate at the 10-meter scale and can therefore help answer questions related to ethics, like ‘Why is this street so much warmer than that street?’”

The EFPL’s high-fidelity simulations are beneficial in other ways, too. For instance, automated drone flights for drone delivery operations require knowledge of how the wind varies in roughly 10-meter increments in order to plan optimal flight trajectories. But regardless of the specific project, running these simulations is computationally demanding and requires access to high-performance computing systems.

“Turbulence is a multiscale phenomenon,” says Dr. Giometto, “so if you want to simulate it accurately, you need very powerful computers because you have a lot of degrees of freedom. Our simulations can easily have billions of grid nodes…[these simulations] have a lot of modes of variability, from large scale to very, very tiny scales. So to run these, you need very powerful clusters.”

The powerful cluster that the EFPL relied on was Purdue’s Anvil cluster, which turned out to be a boon for the group’s research efforts. The group initially requested 7 million core hours on Anvil, which they quickly burned through. They have since extended their allocation request and received approval for nearly 35 million CPU core hours, all on the Anvil system. According to Dr. Giometto, two unexpected benefits to using Anvil were the queue times and max allowable simulation time. On other systems the group has used in the past, the max time allowed per simulation was two days, after which they would be placed back in a long queue to wait until their next turn. This was not the case with Anvil, as the maximum simulation time was four days, and the queue was very short.

“With Anvil, one nice thing was that it was not oversubscribed, and it had a four-day simulation duration. And these, I think, enabled us to close on a project that we wouldn’t have been able to close on otherwise.”

All in all, Anvil has proven to be an excellent resource for the EFPL group. Multiple journal articles have been published surrounding their work, with many more to come. The group intends to continue using Anvil for the foreseeable future, with notable research involving surrogate modeling and more work on uncertainty quantification.

Anvil is Purdue University’s most powerful supercomputer, providing researchers from diverse backgrounds with advanced computing capabilities. Built through a $10 million system acquisition grant from the National Science Foundation (NSF), Anvil supports scientific discovery by providing resources through the NSF’s Advanced Cyberinfrastructure Coordination Ecosystem: Services & Support (ACCESS), a program that serves tens of thousands of researchers across the United States.

For more information regarding HPC and how it can help you, please visit our “Why HPC?” page.

Researchers may request access to Anvil via the ACCESS allocations process. More information about Anvil is available on Purdue’s Anvil website. Anyone with questions should contact anvil@purdue.edu. Anvil is funded under NSF award No. 2005632.