Projected Nonlinear Least Squares for Exponential Fitting

DOI10.1137/16M1084067zbMATH Open1432.65082arXiv1508.05890OpenAlexW2626218069WikidataQ115525631 ScholiaQ115525631MaRDI QIDQ4600007FDOQ4600007

Jeffrey M. Hokanson

Publication date: 5 January 2018

Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)

Abstract: The modern ability to collect vast quantities of data poses a challenge for parameter estimation problems. When posed as a nonlinear least squares problem fitting a model to data, the cost of each iteration grows linearly with the amount of data and it can easily become prohibitively expensive to perform many iterations. Here we develop an approach that projects the data onto a low-dimensional subspace of the high-dimensional data that preserves the information in the original data. We provide results from both optimization and statistical perspectives showing that the information is preserved when the subspace angles between this projection and the Jacobian of the model at the current iterate remain small. However, for this approach to reduce computational complexity, both the projected model and Jacobian must be computed inexpensively. This is a constraint on the pairs of models and subspaces for which this approach provides a computational speedup. Here we consider the exponential fitting problem projected onto the range of Vandermonde matrix, for which the projected model and Jacobian can be computed in closed form using a generalized geometric sum formula. We further provide an inexpensive heuristic that picks this Vandermonde matrix so that the subspace angles with the Jacobian remain small and use this heuristic to update the subspace during optimization. Although the asymptotic cost still depends on the data dimension, the overall cost of this sequence of projected nonlinear least squares problems is less expensive than the original nonlinear least squares problem. Applied to the exponential fitting problem, this provides an algorithm that is not only faster in the limit of large data than the conventional nonlinear least squares approach, but is also faster than subspace based approaches such as HSVD.

Full work available at URL: https://arxiv.org/abs/1508.05890

zbMATH Keywords

parameter estimationdimension reductionexperimental designspectral analysisexponential fittingnonlinear least squaresmodal analysisharmonic estimation

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Methods of successive quadratic programming type (90C55) Parametric tolerance and confidence regions (62F25) Design of statistical experiments (62K99)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Matrix Analysis
• The Differentiation of Pseudo-Inverses and Nonlinear Least Squares Problems Whose Variables Separate
• Separable nonlinear least squares: the variable projection method and its applications
• Introduction to spectral analysis
• Matrix pencil method for estimating parameters of exponentially damped/undamped sinusoids in noise
• Inexact Newton Methods
• Incremental Least Squares Methods and the Extended Kalman Filter
• Compressed Sensing Off the Grid
• A New Class of Incremental Gradient Methods for Least Squares Problems
• Exponential Data Fitting and its Applications
• Functional approach to optimal experimental design.
• Hybrid deterministic-stochastic methods for data fitting
• An inexact Levenberg-Marquardt method for large sparse nonlinear least squres
• Perturbation analysis of an SVD-based linear prediction method for estimating the frequencies of multiple sinusoids

Cited In (3)

• Sampling-free model reduction of systems with low-rank parameterization
• The p-AAA Algorithm for Data-Driven Modeling of Parametric Dynamical Systems
• $\mathcal{H}_2$-Optimal Model Reduction Using Projected Nonlinear Least Squares

Uses Software

• KELLEY
• mpmath

This page was built for publication: Projected Nonlinear Least Squares for Exponential Fitting

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4600007)