An exploratory study on machine learning to couple numerical solutions of partial differential equations (Q2656809)

The authors propose a machine learning based algorithm for the solution of dynamical systems with governing equations that can be written as PDEs/coupled PDEs to understand the most complex coupled multi-physics and multiscale phenomena. The coupled governing equations involving the multiscale and multi-physical characteristics are solved with high efficiency and high numerical stability. If we could use machine learning to speed up the process of solving such complex phenomena, it could herald a new era in doing a whole lot of good for scientific inquiry and engineering. One of the key features of the machine learning approach is the fact that, unlike other commonly used numerical approaches such as FDM/FEM methods, the calculation efficiency is superlative. As such, it does not suffer (as much as other numerical methods) from the curse of dimensionality associated with high-dimensional PDEs and PDE systems. The paper represents a paradigm shift from the traditional approach based on grids to a machine-learning approach that couples numerical solutions of PDEs. It solves PDEs using domain decomposition as in a conventional method but develops and trains artificial neural networks (ANN) to couple the PDEs at their interfaces, leading to their solutions in the whole domains. The key concepts and algorithms for the ML coupling are discussed via computation of coupled Poisson equations and coupled advection-diffusion equations as model equations. One can look at the results provided in this work as a preliminary finding, and substantial research is required for extending these frameworks to complex multi-dimensional, multiscale problems. The authors, based on their research, are of the view that this exploratory study indicates that the ML paradigm is promising in terms of feasibility and performance and deserves further research. The paper surveys ongoing research on the coupling of partial differential equations and pinpoints that progress in theoretical and applied research is limited because of the complexity in coupling PDEs, especially different types of PDEs. One of the key findings of this survey is: further progress in coupling PDEs, especially distinct PDEs, via existing approaches is hindered due to the difficulties stemming from the fact that the coupling is a heterogeneous domain decomposition problem; it involves different PDEs, numerical methods, and computational meshes. The difficulties, the authors argue, are hard to overcome in a conventional coupling approach, which directly exchanges PDEs' solutions at their interfaces according to certain interface conditions. Taking inspiration from the fact that rapid progress of machine learning (ML) is taking place also in PDE-based learning of fluid flows, the authors explore the idea of coupling PDEs via machine learning forecasting that coupling PDEs via ML methods may help overcome these difficulties and lead to a new avenue to accurate simulation of real-world multiscale and multi-physics problems. The first part of the technical section begins with the boundary value problem of the Poisson equation, over a domain, which is divided into two overlapping subdomains with interfaces. An ML Schwarz iteration algorithm to solve the problem is presented. As in a standard approach, the ML solution over the interface zone is constructed as a function of neural network function, and \(\alpha\) denotes its parameters, including weights and biases. The first thing to understand here is that neural networks are fundamentally function approximators. Conveniently, this function approximation process is what we need to solve a PDE. Since an ML solution, \(s\), is a neural network function, a key to its construction becomes to find the best weights and biases, within the network. In correspondence, this leads to a minimization problem. Thankfully, researchers who have made a breakthrough and developed software packages that can efficiently solve the minimization problem. The ML Schwarz iterative algorithm is executed using MatLab, and the resulting ML solution, being the neural network solution in subdomains. It is seen that the solution obtained with the ML method matches the exact solution, and the error mainly occurs near the interfaces. Interestingly, it is noticed that the ML Schwarz iteration may converge faster than the classic Schwarz iteration. In order to illustrate the feasibility and performance of the ML-based coupling methods proposed above, a numerical example is considered. In the second part, the paper considers an initial value problem of two coupled advection-diffusion equations over a domain, which is divided into two sub-domains that overlap with each other with interfaces. Three training approaches in terms of the place where the initial condition resides are explored. Two numerical experiments are performed to assess the feasibility and performance, first considering two identical one dimension equations and second a heterogeneous problem. Based on the findings of the experiments, it is observed that ML interface solutions are not merely repeating or data fitting. They could have certain capability of predictions; with input different from those used to train them, the interface solutions can predict output with accuracy. To illustrate and explore such predictability, experiments are presented for the above sections' problems, but with perturbed or different initial and boundary conditions. Highlights of the paper: In ML coupling frameworks of model equations, numerical solutions in subdomains are coupled via an ML solution in between. This coupling is different from conventional coupling methods, in which the numerical solutions are coupled directly with each other. It is expected that such a difference could enable the ML-based methods to exhibit advantages over the conventional methods in the following ways. \begin{itemize} \item[$\bullet$] An ML approach may avoid difficulties/singularities encountered in a conventional approach, e.g., non-physical solutions at interfaces. \item[$\bullet$] Without Schwarz iteration, ML coupling works for the advection-diffusion equations, or the Schwarz iteration is unnecessary for computation. \item[$\bullet$] The approach of coupling PDEs via ML is built on ML methods' capabilities, which are anticipated to bear a solid foundation in mathematics. Empirical results are documented using tables and graphs and the paper places strong emphasis on both algorithmic and mathematical aspects. \end{itemize} Finally and most importantly, it is worth remarking that all neural networks are rarely a ``one-size-fits-all'' tool. Just as is the case with numerical methods, they need to be modified based on the problem. Continual experimentation and reflection is the key to improving results, but a solid understanding of the underlying processes is vital to avoiding pitfalls in solving numerically. Observations and comments 1) Machine learning seems to be a powerful tool for high-dimensional problems. The coupling of model equations has infinitely many variables rendering them high dimensional. 2) In spite of vastly expanded limits on computational power, efforts are on to understand the most complex coupled multiphysics and multiscale phenomena using newer and innovative techniques. Discovery of ML based methods is a step in the right direction. 3) The iterative training process of neural networks solves an optimization problem that finds for parameters (weights and biases) that result in a minimum error or loss when evaluating the numerical examples. The error of the ML solution primarily comes from the error of the training data against the exact solution. 4) The accuracy of the solution is governed by how the information is exchanged between these solvers at the interface, and several methods have been devised for such coupling problems. it is seen that the ML solution is close to the numerical solution, and larger error occurs around the interfaces. 5) When talking about minimization in the context of neural networks, we are discussing non-convex optimization. Thanks to those researchers who made a breakthrough and developed software packages that can efficiently solve the minimization problem. 6) Although numerical methods have been widely used with good performance, researchers are still searching for new methods for solving partial differential equations. In recent years, machine learning has achieved great success in many diverse fields, such as image classification and natural language processing. Studies have shown that artificial neural networks have powerful function-fitting (approximating) capabilities and have great potential in the study of partial differential equations. Though exploratory in nature, the paper aims to show that ML paradigm is promising in terms of feasibility and performance and deserves further research. 7) ML based methods can also be interpreted in a broad sense as domain decomposition methods, where two solvers simultaneously advance the multiscale and multi-physics problems, and the information is exchanged across the interface between solvers. 8) This study is exploratory and preliminary, and further research is necessary on the ML coupling to explore the above listed potential advantages via systematic numerical and theoretical analyses. Nevertheless, this paper indicates that the ML coupling is promising, and, hopefully, it will attract the community's attention to the topic and its further study. 9) Deep learning has achieved remarkable success in diverse applications; however, its use in solving partial differential equations (PDEs) has emerged only recently. Therefore, the present study being exploratory and preliminary, further research is necessary on the ML coupling to explore the above listed potential advantages via systematic numerical and theoretical analyses. Nevertheless, by the authors' own admission, this paper indicates that the ML coupling is promising, and, hopefully, it will attract the community's attention to the topic and its further study. 10) A major challenge in multiscale as well as multi-physics modelling lies in coupling the sub-models such that the overall model is both accurate enough to be scientifically relevant and reproducible, and efficient enough to be executed conveniently by modern computing resources. Precisely for this reason, research has been initiated on the mathematical foundation and mechanism of ML.