Variational Physics-Informed Neural Networks For Solving Partial Differential Equations (1912.00873v1)

Abstract: Physics-informed neural networks (PINNs) [31] use automatic differentiation to solve partial differential equations (PDEs) by penalizing the PDE in the loss function at a random set of points in the domain of interest. Here, we develop a Petrov-Galerkin version of PINNs based on the nonlinear approximation of deep neural networks (DNNs) by selecting the {\em trial space} to be the space of neural networks and the {\em test space} to be the space of Legendre polynomials. We formulate the \textit{variational residual} of the PDE using the DNN approximation by incorporating the variational form of the problem into the loss function of the network and construct a \textit{variational physics-informed neural network} (VPINN). By integrating by parts the integrand in the variational form, we lower the order of the differential operators represented by the neural networks, hence effectively reducing the training cost in VPINNs while increasing their accuracy compared to PINNs that essentially employ delta test functions. For shallow networks with one hidden layer, we analytically obtain explicit forms of the \textit{variational residual}. We demonstrate the performance of the new formulation for several examples that show clear advantages of VPINNs over PINNs in terms of both accuracy and speed.

• The paper introduces VPINNs, which incorporate the variational residual of PDEs into a neural network’s loss function to reduce differential operator order.
• The method employs a Petrov-Galerkin framework with Legendre polynomial test spaces, enhancing accuracy and lowering training costs.
• Numerical experiments show that VPINNs effectively capture boundary layers and steep gradients compared to traditional PINNs.

Overview of VPINNs: Variational Physics-Informed Neural Networks for Solving PDEs

The paper "VPINNs: Variational Physics-Informed Neural Networks For Solving Partial Differential Equations" presents a novel approach to the integration of physics-informed neural networks (PINNs) into the framework of variational formulations for solving partial differential equations (PDEs). The authors introduce a method they denote as Variational Physics-Informed Neural Networks (VPINN), which leverages the variational form of the underlying mathematical models, providing several advantages over traditional PINNs.

The authors develop VPINNs within the Petrov-Galerkin framework, whereby the trial space is designated as the space of neural networks while the test space is chosen as the space of Legendre polynomials. This is a significant departure from standard PINNs, which typically employ a strong form of residuals. In the variational approach, the weak form allows for integration by parts, effectively reducing the order of differential operators and enhancing computational efficiency. This integration by parts is particularly beneficial for lowering the regularity demands on the solution space, which in turn improves accuracy and reduces the training costs of VPINNs.

Key Contributions and Methodology

The core methodological innovation lies in the formulation of the variational loss function, which incorporates the variational residual of the PDE into the neural network's loss function. This variational approach enables the reduction of high-order differential operators and simplifies the mathematical complexity involved in gradient-based optimization of the neural network parameters. The authors explore shallow networks with a single hidden layer to analytically express the variational residuals and extend these concepts to deep networks, necessitating numerical integration techniques such as Gauss quadrature.

One of the notable advantages of VPINNs as demonstrated in the paper is their ability to solve complex PDEs more accurately and efficiently than PINNs, particularly in scenarios involving steep gradients or boundary layers. The authors demonstrate that the use of variational forms leads to a reduced number of quadtrature points compared to the penalizing points necessary for PINNs, further reducing computational demands.

Numerical Validation

The performance of VPINNs is validated through a series of numerical experiments involving canonical problems such as the steady-state Burgers' equation and one-dimensional and two-dimensional Poisson's equations. These experiments illustrate the efficacy of VPINNs in capturing the underlying physics with high accuracy, particularly in boundary layer problems where traditional PINNs struggle without a high concentration of training points around the boundary layer.

Implications and Future Developments

The promising results of VPINNs suggest potential widespread applicability across various fields where PDEs are pivotal. The reduction in computational cost and increase in accuracy make VPINNs attractive for tackling more complex, large-scale problems, such as those found in climate modeling, fluid dynamics, and biomedical engineering.

Looking forward, one of the paper's insights is the possibility of extending the numerical analysis of VPINNs to comprehend their behavior under different conditions better and to refine their integration schemes further. Future research could delve into alternative variational formulations, explore multi-dimensional VPINNs, and robustly establish the theoretical underpinning of numerical integrations involving DNNs.

By addressing the limitations of conventional PINNs and capitalizing on the mathematical strength of variational formulations, VPINNs stand poised to advance the computational capabilities for solving PDEs in both classical and emerging scientific domains.