Probabilistic partition of unity networks for high‐dimensional regression problems
From MaRDI portal
Publication:6062830
Nonlinear parabolic equations (35K55) Approximation by polynomials (41A10) Nonparametric regression and quantile regression (62G08) Factor analysis and principal components; correspondence analysis (62H25) Probabilistic models, generic numerical methods in probability and statistics (65C20) Quantum computation (81P68)
Abstract: We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems and propose a general framework focusing on adaptive dimensionality reduction. With the proposed framework, the target function is approximated by a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a local fixed-degree polynomial. We present a training strategy that leverages the expectation maximization (EM) algorithm. During the training, we alternate between (i) applying gradient descent to update the DNN coefficients; and (ii) using closed-form formulae derived from the EM algorithm to update the mixture of experts model parameters. Under the probabilistic formulation, step (ii) admits the form of embarrassingly parallelizable weighted least-squares solves. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. We also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.
Recommendations
- Learning high-dimensional parametric maps via reduced basis adaptive residual networks
- Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
- Deep UQ: learning deep neural network surrogate models for high dimensional uncertainty quantification
- Variational Monte Carlo -- bridging concepts of machine learning and high-dimensional partial differential equations
- Adaptive deep neural networks methods for high-dimensional partial differential equations
Cites work
- A Sparse Grid Stochastic Collocation Method for Partial Differential Equations with Random Input Data
- A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems
- Active subspaces. Emerging ideas for dimension reduction in parameter studies
- Bayesian deep convolutional encoder-decoder networks for surrogate modeling and uncertainty quantification
- Breaking the curse of dimensionality with convex neural networks
- Chebyshev and Fourier spectral methods.
- Data-driven computational mechanics
- Deep learning in high dimension: neural network expression rates for generalized polynomial chaos expansions in UQ
- Deep ReLU networks and high-order finite element methods
- Deep UQ: learning deep neural network surrogate models for high dimensional uncertainty quantification
- DGM: a deep learning algorithm for solving partial differential equations
- Error bounds for approximations with deep ReLU networks
- Exploiting active subspaces to quantify uncertainty in the numerical simulation of the hyshot II scramjet
- EXTENDING CLASSICAL SURROGATE MODELING TO HIGH DIMENSIONS THROUGH SUPERVISED DIMENSIONALITY REDUCTION: A DATA-DRIVEN APPROACH
- Gradient-based optimization for regression in the functional tensor-train format
- scientific article; zbMATH DE number 1465030 (Why is no real title available?)
- scientific article; zbMATH DE number 784362 (Why is no real title available?)
- Kernel PCA for novelty detection
- Learning constitutive relations from indirect observations using deep neural networks
- Learning generative neural networks with physics knowledge
- Lebesgue functions and Lebesgue constants in polynomial interpolation
- Non-intrusive low-rank separated approximation of high-dimensional stochastic models
- Physics constrained learning for data-driven inverse modeling from sparse observations
- Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data
- Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations
- Projection-based approximation and a duality with kernel methods
- Radial Basis Functions
- Reducing the Dimensionality of Data with Neural Networks
- Solving high-dimensional partial differential equations using deep learning
- Solving parametric PDE problems with artificial neural networks
- Spectral tensor-train decomposition
- Splines in Statistics
This page was built for publication: Probabilistic partition of unity networks for high‐dimensional regression problems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6062830)
