2024

What do physics-informed DeepONets learn? Understanding and improving training for scientific computing applications
Williams, E., Howard, A., Meuris, B., and Stinis, P.

Physics-informed deep operator networks (DeepONets) have emerged as a promising approach toward numerically approximating the solution of partial differential equations (PDEs). In this work, we aim to develop further understanding of what is being learned by physics-informed DeepONets by assessing the universality of the extracted basis functions and demonstrating their potential toward model reduction with spectral methods. Results provide clarity about measuring the performance of a physics-informed DeepONet through the decays of singular values and expansion coefficients. In addition, we propose a transfer learning approach for improving training for physics-informed DeepONets between parameters of the same PDE as well as across different, but related, PDEs where these models struggle to train well. This approach results in significant error reduction and learned basis functions that are more effective in representing the solution of a PDE.

Deep Operator Networks for Reduced-Order Modeling
Williams, E., Howard, A., and Stinis, P.
Department of Energy Computational Science Graduate Fellowship Annual Program Review (July 2024)

Many multiscale systems representing complex physical phenomena contain too many degrees of freedom to simulate accurately given limited computational resources. Reduced-order modeling techniques reduce the prohibitively large system to a computationally feasible size without sacrificing essential dynamical features. The Mori-Zwanzig (MZ) formalism is an exact formalism for the reduction of the dynamics of a full system to the dynamics of a reduced set of variables. MZ-based model reduction which involves coarsening a representation using standard basis functions, e.g. Fourier functions, is well developed. In this work, we employ machine-learning extracted basis functions from deep operator networks which are custom-made for the particular system, with the goal of reduced-order modeling with spectral methods.

Learning Diffusion for Stabilizing Linearized Chaotic Systems
Williams, E., Trono Figueras, R., and Darmofal, D.
Society for Industrial and Applied Mathematics Annual Meeting (July 2024)

Stochastic processes are used often in mathematical modeling of phenomena that appear to vary chaotically or randomly. We aim to incorporate stochastic modeling and explore the potential stabilizing effect on linearized chaotic systems. We introduce a stochastic diffusive term to obtain the Euler-Maruyama integration step as $\tilde{\mathbf{x}}_{n+1} = \tilde{\mathbf{x}}_n + \mathbf{J}(\mathbf{x}_n)\tilde{\mathbf{x}}_n\Delta t + \boldsymbol{\Sigma}(\mathbf{x}_n,\tilde{\mathbf{x}}_n)\Delta W_n$, where $\mathbf{J}$ is the Jacobian (drift), $\boldsymbol{\Sigma}$ is the diffusion matrix, and $\Delta W_n$ is the Wiener increment. Dietrich et al. (2023) use a neural network with a probabilistic loss function inspired by the structure of stochastic integrators to approximate the true drift and true diffusion used to generate the training data [Dietrich et al., Chaos 33, 023121, 2023]. Our work trains the network using data from a nonlinear dynamical system subject to perturbations. The resulting diffusion matrix $\boldsymbol{\Sigma}$ of the stochastic equation learned by the network is shown to be stabilizing. The motivation in using a neural network stabilization function is to enable controllable diffusion. The performance is assessed by comparing sample trajectories generated by the neural network to a scalar stabilization model derived through theory.

2023

Towards a stochastic subgrid-scale model for turbulence
Williams, E., and Darmofal, D.
Advances in Computational Mechanics (October 2023)

Stochastic processes are used often in mathematical modeling of phenomena that appear to vary chaotically or in a random manner. Stochastic differential equations are ubiquitous in the formulation of these models, including population dynamics, neuron activity, blood clotting, turbulent diffusion, and more. Stochastic processes may be defined for all time instants in a bounded interval or in an unbounded interval, in which case they are continuous time stochastic processes. We aim to incorporate stochastic modeling to improve the time integration schemes for chaotic systems. Many stochastic integration numerical schemes recursively update the state using the Brownian increment, taken as independent and identically distributed normal random variables with expected value zero and variance equal to the timestep. We propose that this increment should be sampled from some function that is representative of the subgrid (i.e., unresolved) dynamics of the chaotic system.

Stochastic Integration for Chaotic Dynamical Systems
Williams, E., and Darmofal, D.
Department of Energy Computational Science Graduate Fellowship Annual Program Review (July 2023)

Stochastic processes are used often in mathematical modeling of phenomena that appear to vary chaotically or in a random manner. Stochastic differential equations are ubiquitous in the formulation of these models, including population dynamics, neuron activity, blood clotting, turbulent diffusion, and more. Stochastic processes may be defined for all time instants in a bounded interval or in an unbounded interval, in which case they are continuous time stochastic processes. We aim to incorporate stochastic modeling to improve the time integration schemes for chaotic systems. Many stochastic integration numerical schemes recursively update the state using the Brownian increment, taken as independent and identically distributed normal random variables with expected value zero and variance equal to the timestep. We propose that this increment should be sampled from some function that is representative of the subgrid (i.e., unresolved) dynamics of the chaotic system. Further, we want to explore the potential stabilizing effect that stochastic models could have on linearized sensitivity of chaotic systems. Initial efforts toward this investigation include developing a higher-order finite-element-based time-marching approach for stochastic initial value problems.

Near-Field Wall-Modeled Large-Eddy Simulation of the NASA X-59 Low-Boom Flight Demonstrator
Williams, E., Arranz, G., and Lozano-Durán, A.

Wall-modeled large-eddy simulation (WMLES) is utilized to analyze the experimental aircraft X-59 Quiet SuperSonic Technology (QueSST) developed by Lockheed Martin at Skunk Works for NASA's Low-Boom Flight Demonstrator project. The simulations utilize the charLES solver and aim to assess the ability of WMLES to predict near-field noise levels under cruise conditions, considering various subgrid-scale (SGS) models and grid resolutions. The results are compared with previous numerical studies based on the Reynolds-averaged Navier-Stokes (RANS) equations. Our findings demonstrate that WMLES produces near-field pressure predictions that are similar to those of RANS simulations at a comparable computational cost. Some mild discrepancies are observed between the WMLES and RANS predictions downstream the aircraft. These differences persist for finest grid refinement considered, suggesting that they might be attributed to the intricate interactions of shock waves and expansions waves at the trailing edge.

Assessment of wall-modeled large-eddy simulation for high-speed flows and novel modeling strategies
Williams, E.
Massachusetts Institute of Technology Master's Thesis (June 2023)

Turbulent flows are ubiquitous in aerospace engineering. External aerodynamic applications have exposed limitations in current state-of-the-art computational fluid dynamics solvers due to the presence of chaotic and multiscale turbulent flows. Wall-modeled large-eddy simulation (WMLES) stands as a realistic contender for turbulence modeling for aerospace applications. However, there remain important challenges of WMLES in the accurate prediction of certain quantities of interest within the stringest tolerance demanded by the industry. In this work, we evaluate the performance of WMLES for high-speed flows and explore new modeling venues. We assess the performance of WMLES for canonical compressible channel flows and a realistic external aerodynamic application, the Lockheed Martin X-59 Quiet SuperSonic Technology (QueSST) aircraft. Additionally, we propose potential modeling improvements to enhance the predictive capabilities of WMLES for high-speed flows.

2022

Wall-Modeled Large-Eddy Simulation of the Lockheed Martin X-59 QueSST
Williams, E., Arranz, G., and Lozano-Durán, A.
American Physical Society Division of Fluid Dynamics (November 2022)

A wall-modeled large-eddy simulation of the experimental aircraft X-59 Quiet SuperSonic Technology (QueSST) developed by Lockheed Martin at Skunk Works for NASA's Low-Boom Flight Demonstrator project is conducted using the solver charLES. The capabilities of large-eddy simulation to predict the noise level in cruise conditions are evaluated and compared with other numerical studies. The main quantities of interest are the farfield pressure field and the intensities and locations of the shock waves. This approach will enable the detection of design deficiencies prior to aircraft construction, resulting in financial benefits and accelerating certification by analysis efforts.

Information-Theoretic Approach for Subgrid-Scale Modeling for High-Speed Compressible Wall Turbulence
Williams, E., and Lozano-Durán, A.
AIAA AVIATION Forum (June 2022)

The problem of modeling for turbulent flows is investigated within the framework of information theory. A wall-modeled large-eddy simulation (WMLES) of a compressible turbulent channel flow is conducted using an equilibrium wall model and either the dynamic Smagorinsky (DSM) or information-preserving (IP) subgrid-scale (SGS) model. The IP SGS model is formulated using the Kullback-Leibler (KL) divergence. The model aims at minimizing the information lost between the probability mass distribution of the interscale energy transfer and viscous dissipation at different scales. The statistical quantities of interest are the mean velocity and mean temperature profiles. It is found that the IP SGS model matches or exceeds the accuracy of the DSM SGS model when compared to direct numerical simulation (DNS) data for the compressible channel.

2021

Error Scaling of Wall-Modeled Large-Eddy Simulation of Compressible Wall Turbulence
Williams, E., and Lozano-Durán, A.
American Physical Society Division of Fluid Dynamics (November 2021)

Error scaling properties of large-eddy simulation of compressible wall-bounded turbulent flows are characterized. The statistical quantities of interest investigated are the mean velocity profile, wall stress, and wall heat transfer. The errors scale as $(\Delta/L)^\alpha Re_\tau^\gamma M^\beta$, where $\Delta$ is the characteristic grid resolution, $Re_\tau$ is the friction Reynolds number, $L$ is the meaningful length-scale to normalize $\Delta$ in order to collapse the errors across the wall-normal distance, and $M$ is the Mach number to account for compressibility effects in the channel flow. Different length-scales are used in determining $L$ such that errors along the wall-normal distance are effectively collapsed. The exponents $\alpha,$ $\gamma,$ and $\beta$ are estimated using theoretical analysis and validated through numerical simulations.

Numerical Schlieren of the X-59 QueSST
Williams, E., Ling. Y., Arranz, G., and Lozano-Durán, A.
American Physical Society Division of Fluid Dynamics Gallery of Fluid Motion (November 2021)

Numerical schlieren visualizations were created in ParaView by contouring the magnitude of the density gradients in the flow to capture the shock waves. The aircraft is also contoured based on the temperature at the surface during flight at cruise conditions. Wall-modeled large-eddy simulations were run through the charLES solver by Cascade Technologies, Inc. The solver integrates the filtered Navier-Stokes equations using a second-order finite volume method in space and a third- order explicit Runge-Kutta method in time. The mesh generator uses a Voronoi hexagonal close-packed point-seeding method with $\mathcal{O}(100\text{M})$ control volumes. An equilibrium wall model is used assuming an isothermal wall with a wall-stress boundary condition with the dynamic Smagorinsky subgrid-scale model.

2020

Relation of Dissociation Rates to the Centrifugal Barrier
Williams, E., Sharma, M.P., Venturi, S., and Panesi, M.
University of Illinois Undergraduate Research Symposium (April 2020)

Hypersonic re-entry flight experiences extreme temperatures leading to a state of chemical non-equilibrium in the flow. To analyze the state-to-state (StS) kinetics for these flows, we propose to fit the kinetics data using spare regression techniques. The advantages of obtaining an expression for the state-to-state dissociation rates in terms of the position of the centrifugal barrier are two-fold: it provides an insight into the relation of dissociation via various levels to the centrifugal barrier and can be used to arrive at a physics-informed grouping strategy to reduce the dimension of the kinetics system (for example from the order of 10000 to 10). This would further aid in modelling the reactions that take place in the flow in a computationally efficient way. For the purpose of fitting, we select a large basis set of functions that include exponentials and polynomials motivated by expressions derived in Transition State theory. It is followed by a sparse regression over this function basis set to obtain a rather simple looking expression.