IDENT: identifying differential equations with numerical time evolution (Q1995989)

Deriving PDEs mathematically or physically through basic conservation laws, is a well established procedure, empirical data playing the supportive role. However, in modern applications, the mechanism of many complex systems is still unclear, and it is difficult to identify or uncover PDEs. Uncovering the underlying differential equations directly from measured data, poses a challenging task. The task becomes daunting and often impractical if the data are corrupted. With the rapid development of computing power and data storage, the cost of data collection and computing has been greatly reduced so that a large amount of experimental data could be obtained. The advent of machine learning has also provided a reliable tool to discover the potential laws of the system from large datasets, ushering a new era in the studies and research efforts of climate science, neuroscience, ecology, finance, and epidemiology, to name only a few examples. With abundant data and elusive laws, data-driven discovery of dynamics will continue to play an important role, significantly broadening the applicability to a wide variety of complex systems. Directly recognising the underlying PDE, from the experimental data is referred to as an inverse problem in the terminology of computational mathematics. The main objective of the paper is the identification of a PDE from given time-dependent discrete data together with the boundary conditions and assuming that the given data do not have any noise. A new algorithm IDENT is explored for evolutionary data with non-periodic boundary conditions for the solution of the inverse problem, and generalised to include noisy data and PDEs with nonlinearity and variable coefficients. The authors propose a new approach, based on the fundamental principle of consistency, convergence and stability of numerical PDE schemes. They also utilize LASSO for efficiency and incoherence property for performance guarantee. A brief literature survey tells about ongoing research on learning dynamical systems through machine learning. Symbolic and sparse regression and various extensions of these are two pioneering works. One group of researchers referenced in the paper have used a spectral method, L1-minimization, random sampling theory, Bayesian approach etc. for analysing dynamical systems on the basis of observed data. Related works of another group of researchers focus on using deep learning techniques, neural network etc. The present paper aims to cross-fertilize between dynamical systems and machine learning each benefiting from the other. The mathematical formulation of the inverse problem consists of assuming that governing equation is a linear combination of a few linear and nonlinear differential terms in a prescribed dictionary and arriving at the discrete linear system, by approximating derivatives by FDM. Assuming that the coefficients are sparse, L1 regularized minimization is utilized to find candidates for non-zero coefficients. Finally, using TEE and least square fit the identification of PDE is achieved. Key assumptions made in the development of the algorithm are: 1) The given data do not have any measurement noise or are denoised if noisy; 2) Coefficients in the linear combination of time derivative are sparse since sparsity plays a key role in minimisation of TEE. To solve the problem that existing PDE identification methods are ineffective to low NSR data, the authors state and prove a theorem (details of which are given in the appendix) leading to a new definition of NSR. The authors, first, demonstrate the algorithm on two benchmark problems for the discovery of the underlying PDE when data were obtained from sampling the exact or simulated solution of the inviscid Burgers' equation and the one with diffusive term satisfying Dirichlet zero boundary conditions, throughout the paper. Incidentally, the authors are of the view that the method is capable of generalization with regard to boundary conditions. As a part of the implementation, TEE is computed and term with smallest TEE is identified as the outcome of IDENT. The obtained results are displayed graphically to capture the qualitative aspects of the given data and establishing the validity of theoretical observations. In another subsection, the effects of inaccuracy of data generation, noise and downsampling are explored. An error formula is derived for these three aspects, which provides a theoretical tool to figure out the difficulties in identifying higher order PDEs especially. To establish the fact that denoising is helpful in general. The authors present a numerical treatment of Burger's equation with noisy data, and apply directly IDENT without denoising and demonstrating the failure of identification correctly. This necessitates the proposal of a new order preserving denoising method LSMA by the authors and the next set of numerical experiments show that LSMA+IDENT improve the results drastically. Proceeding further, Burger's equation with different down-samplig rates is considered. IDENT is executed on the down-sampled noisy data, denoised by LS and LSMA respectively. Findings of the experiment reveal that LSMA improves the performance of IDENT significantly. In the final set of experiments, PDEs with variable coefficients are considered. An extra procedure BEE is invoked so as to ensure the stability of the algorithm. With this added procedure, it is claimed that IDENT continues to perform satisfactorily in the refinement of the selection. To establish the validity of their claim, the authors consider Burger's equation with a varying diffusive coefficient. To the author's surprise, even for the data with noise, IDENT+BEE without any denoising gives a good identification of the structure of PDE. However, varying coefficient approximation can be improved if LSMA denoising is applied. As a fitting finale to their research effort, all the coefficients are allowed to vary in space. Even in this case, IDENT with BEE successfully identifies the correct terms. This example shows that the method seems robust to noise. Yet, the method is able to detect the correct features and approximate the true value of the coefficients. Comments and observations \begin{itemize} \item[1)] Appropriate function spaces and norms are used from functional analysis for estimation of error and least square fit, minimization of energy etc. \item[2)] The author's objective is to develop stable algorithm against a high level of noise, varying down-sampling rates and varying coefficients. Numerical experiments to validate their claims about robustness are presented. \item[3)] To emphasize the role of computational framework, without underestimating the power of theoretical background and to make the paper as self-contained as possible, the details of the stated proof are presented in the appendix. \item[4)] In this work, a new sparse identification criterion is also proposed to identify the structure of PDE, and the method can identify the terms with small coefficients in PDE. \item[5)] It is difficult to identify PDEs from low noise-to- signal ratio (NSR) data, because the numerical differentiation is ill-posed to noisy data. Further, noise affects different PDEs differently. To remedy the situation a new definition of NSR from the insights of theorems is given to measure the level of noise. \item[6)] The proposed method is comparable to existing methods with regard to the size of the training data, flexibility with boundary conditions, and originality in addressing varying coefficients. The novel methodology has several advantageous practical characteristics. \item[7)] Inverse problems arise when our objective is to recover information about a system from observations of the system. In this work, the authors are concerned with systems that are described by PDEs and the information they wish to discover are coefficients in the differential equation. \item[8)] This work develops a novel framework to discover governing equations underlying a dynamical system simply from data measurements, leveraging advances in sparsity techniques and machine learning. \item[9)] To my knowledge, this is the first data-driven regression technique that explicitly accounts for spatial derivatives in discovering physical laws, thus the success of this methodology suggests that many concepts from statistical and machine learning can be integrated with traditional scientific computing and dynamical systems theory to discover dynamical models from data. \item[10)] This paper presents insights into different research gaps which will provide the direction for further research in in the field of data-driven dynamical systems. Particularly, in the use of L1 regularized least-squares minimization to select the correct features that learn the governing PDE. The identification of the terms in the PDE and the approximation of the coefficients come directly from the data. The numerical experiments show that the proposed method is robust to data noise and size, and is able to capture the true features of the data accurately. \end{itemize}