Stable architectures for deep neural networks

DOI10.1088/1361-6420/aa9a90zbMath1426.68236arXiv1705.03341OpenAlexW3098011980MaRDI QIDQ4607800

Publication date: 14 March 2018

Published in: Inverse Problems (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1705.03341

zbMATH Keywords

parameter estimation machine learning dynamic inverse problems image classification deep neural networks

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Learning and adaptive systems in artificial intelligence (68T05) Inverse problems involving ordinary differential equations (34A55)

Related Items

Parameter calibration with stochastic gradient descent for interacting particle systems driven by neural networks, Interpolation and approximation via momentum ResNets and neural ODEs, EnResNet: ResNets Ensemble via the Feynman--Kac Formalism for Adversarial Defense and Beyond, Stability of Deep Neural Networks via Discrete Rough Paths, A survey on deep learning and its applications, Turnpike in optimal control of PDEs, ResNets, and beyond, ODE-RU: a dynamical system view on recurrent neural networks, Machine learning from a continuous viewpoint. I, Analytic continuation of noisy data using Adams Bashforth residual neural network, Meta-mgnet: meta multigrid networks for solving parameterized partial differential equations, Activation function design for deep networks: linearity and effective initialisation, Feasibility-based fixed point networks, Wasserstein-Based Projections with Applications to Inverse Problems, Designing rotationally invariant neural networks from PDEs and variational methods, On the regularized risk of distributionally robust learning over deep neural networks, Personalized Algorithm Generation: A Case Study in Learning ODE Integrators, A non-intrusive correction algorithm for classification problems with corrupted data, slimTrain---A Stochastic Approximation Method for Training Separable Deep Neural Networks, Large Sample Mean-Field Stochastic Optimization, Deep Neural Networks and PIDE Discretizations, A backward SDE method for uncertainty quantification in deep learning, Generative modeling via tensor train sketching, A literature survey of matrix methods for data science, Deep learning methods for partial differential equations and related parameter identification problems, Deep limits of residual neural networks, A framework for machine learning of model error in dynamical systems, Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training, Neural network representation of time integrators, Dual non-autonomous deep convolutional neural network for image denoising, Deep neural networks on diffeomorphism groups for optimal shape reparametrization, CD-ROM: complemented deep -- reduced order model, Locally-symplectic neural networks for learning volume-preserving dynamics, Deep learning approximation of diffeomorphisms via linear-control systems, Sparsity in long-time control of neural ODEs, Accuracy and architecture studies of residual neural network method for ordinary differential equations, Neural ODE Control for Classification, Approximation, and Transport, A mathematical perspective of machine learning, Unnamed Item, Physics-Informed Probabilistic Learning of Linear Embeddings of Nonlinear Dynamics with Guaranteed Stability, A functional approach to interpreting the role of the adjoint equation in machine learning, Estimating a Potential Without the Agony of the Partition Function, Multi‐fidelity data fusion through parameter space reduction with applications to automotive engineering, Connections between numerical algorithms for PDEs and neural networks, Applications of time parallelization, Relational intelligence recognition in online social networks -- a survey, Deep learning via dynamical systems: an approximation perspective, A Multilevel Method for Many-Electron Schrödinger Equations Based on the Atomic Cluster Expansion, Learning Hamiltonian systems with mono-implicit Runge-Kutta methods, Optimal control using to approximate probability distribution of observation set, Numerical Analysis for Convergence of a Sample-Wise Backpropagation Method for Training Stochastic Neural Networks, Convex and concave envelopes of artificial neural network activation functions for deterministic global optimization, On mathematical modeling in image reconstruction and beyond, Unnamed Item, An artificial neural network framework for reduced order modeling of transient flows, PDE-Net 2.0: learning PDEs from data with a numeric-symbolic hybrid deep network, Generalization of partitioned Runge-Kutta methods for adjoint systems, Mini-workshop: Deep learning and inverse problems. Abstracts from the mini-workshop held March 4--10, 2018, Dynamic inverse problems: modelling—regularization—numerics, State-Space Representations of Deep Neural Networks, Forward stability of ResNet and its variants, Deep neural networks motivated by partial differential equations, Residual networks as flows of diffeomorphisms, Variational networks: an optimal control approach to early stopping variational methods for image restoration, Networks for nonlinear diffusion problems in imaging, ADMM-softmax: an ADMM approach for multinomial logistic regression, Unnamed Item, PNKH-B: A Projected Newton--Krylov Method for Large-Scale Bound-Constrained Optimization, Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection, How Deep Are Deep Gaussian Processes?, Selection dynamics for deep neural networks, Unnamed Item, Learning adaptive regularization for image labeling using geometric assignment, Continuous-time system identification with neural networks: model structures and fitting criteria, An Efficient Parallel-in-Time Method for Optimization with Parabolic PDEs, On Functions Computed on Trees, Deep learning as optimal control problems: models and numerical methods, Model reduction and neural networks for parametric PDEs, The Random Feature Model for Input-Output Maps between Banach Spaces, Solving inverse problems using data-driven models, A sequential quadratic Hamiltonian algorithm for training explicit RK neural networks, A mean-field optimal control formulation of deep learning, Time-series learning of latent-space dynamics for reduced-order model closure, How does momentum benefit deep neural networks architecture design? A few case studies, Quantized convolutional neural networks through the lens of partial differential equations, Geometric numerical integration of the assignment flow, Least-squares finite element method for ordinary differential equations, Unnamed Item, Artificial Neural Networks for the Estimation of Pedestrian Interaction Forces, Nonlinear Power Method for Computing Eigenvectors of Proximal Operators and Neural Networks, Structure-preserving deep learning, Continuous-domain assignment flows, Stabilize deep ResNet with a sharp scaling factor \(\tau\), A measure theoretical approach to the mean-field maximum principle for training NeurODEs, Do ideas have shape? Idea registration as the continuous limit of artificial neural networks, Optimization with learning-informed differential equation constraints and its applications, A Nonautonomous Equation Discovery Method for Time Signal Classification, Layer-Parallel Training of Deep Residual Neural Networks, Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network

Uses Software

Cites Work