Linear models based on noisy data and the Frisch scheme

DOI10.1137/130921179zbMATH Open1341.62166DBLPjournals/siamrev/NingGTB15arXiv1304.3877OpenAlexW1729590385WikidataQ31095413 ScholiaQ31095413MaRDI QIDQ2808247FDOQ2808247

Authors: Lipeng Ning, Tryphon T. Georgiou, Allen Tannenbaum, Stephen Boyd 

Publication date: 20 May 2016

Published in: SIAM Review (Search for Journal in Brave)

Abstract: We address the problem of identifying linear relations among variables based on noisy measurements. This is, of course, a central question in problems involving "Big Data." Often a key assumption is that measurement errors in each variable are independent. This precise formulation has its roots in the work of Charles Spearman in 1904 and of Ragnar Frisch in the 1930's. Various topics such as errors-in-variables, factor analysis, and instrumental variables, all refer to alternative formulations of the problem of how to account for the anticipated way that noise enters in the data. In the present paper we begin by describing the basic theory and provide alternative modern proofs to some key results. We then go on to consider certain generalizations of the theory as well applying certain novel numerical techniques to the problem. A central role is played by the Frisch-Kalman dictum which aims at a noise contribution that allows a maximal set of simultaneous linear relations among the noise-free variables --a rank minimization problem. In the years since Frisch's original formulation, there have been several insights including trace minimization as a convenient heuristic to replace rank minimization. We discuss convex relaxations and certificates guaranteeing global optimality. A complementary point of view to the Frisch-Kalman dictum is introduced in which models lead to a min-max quadratic estimation error for the error-free variables. Points of contact between the two formalisms are discussed and various alternative regularization schemes are indicated.

Full work available at URL: https://arxiv.org/abs/1304.3877

Recommendations

• Identification of the maximal number of linear relations from noisy data
• Certain models from uncertain data: The algebraic case
• scientific article; zbMATH DE number 37094
• An upper bound on the number of linear relations identified from noisy data by the Frisch scheme
• scientific article; zbMATH DE number 5620939

zbMATH Keywords

factor analysisidentificationlinear models

Mathematics Subject Classification ID

Factor analysis and principal components; correspondence analysis (62H25) Linear regression; mixed models (62J05) Applications of statistics to social sciences (62P25)

Cites Work

• Matrix completion from noisy entries
• Principal component analysis.
• Title not available (Why is that?)
• Title not available (Why is that?)
• The Generalized Dynamic Factor Model
• Differential-geometrical methods in statistics
• Dynamic factor models with infinite-dimensional factor spaces: one-sided representations
• Latent variable models and factor analysis. A unified approach
• Title not available (Why is that?)
• Factor modeling for high-dimensional time series: inference for the number of factors
• Errors in Variables
• Exact matrix completion via convex optimization
• Estimation of (near) low-rank matrices with noise and high-dimensional scaling
• Generic global identification in factor analysis
• Title not available (Why is that?)
• Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization
• Robust techniques for signal processing: A survey
• Estimation of high-dimensional low-rank matrices
• Rank-reducibility of a symmetric matrix and sampling theory of minimum trace factor analysis
• Minimum rank and minimum trace of covariance matrices
• Weighted minimum trace factor analysis
• Some contributions to maximum likelihood factor analysis
• Rank-Sparsity Incoherence for Matrix Decomposition
• Title not available (Why is that?)
• Confluence Analysis by Means of Lag Moments and Other Methods of Confluence Analysis
• Matrix Completion From a Few Entries
• Errors-in-variables methods in system identification
• Consistent Sets of Estimates for Regressions with Errors in All Variables
• A penalized maximum likelihood approach to sparse factor analysis
• Noisy matrix decomposition via convex relaxation: optimal rates in high dimensions
• Title not available (Why is that?)
• Diagonal and low-rank matrix decompositions, correlation matrices, and ellipsoid fitting
• On Majorization, Factorization, and Range Inclusion of Operators on Hilbert Space
• Positive definite matrices
• A Riemannian framework for tensor computing
• The distance between two random vectors wigh given dispersion matrices
• Generalized linear dynamic factor models: an approach via singular autoregressions
• Title not available (Why is that?)
• A Competitive Minimax Approach to Robust Estimation of Random Parameters
• On minimax robustness: A general approach and applications
• Estimating linear statistical relationships
• Identifiability of factor analysis: Some results and open problems
• System Identification by Dynamic Factor Models
• Least-Squares Approximation of Structured Covariances
• Some necessary conditions for common-factor analysis
• A Minimax Theorem with Applications to Machine Learning, Signal Processing, and Finance
• Title not available (Why is that?)
• Identification of the maximal number of linear relations from noisy data
• Identification of noisy systems
• Identification of a linear system from inexact data: A three-variable example
• Matrices with prescribed off-diagonal elements
• An upper bound on the number of linear relations identified from noisy data by the Frisch scheme
• A Structure Theory for Linear Dynamic Errors-in-Variables Models
• Linear dynamic errors-in-variables models. Some structure theory
• The general dynamic factor model: one-sided representation results
• The foundations of factor analysis
• Information geometry of covariance matrix: Cartan-Siegel homogeneous bounded domains, Mostow/Berger fibration and Fréchet median
• Dynamic errors-in-variables systems with three variables
• Title not available (Why is that?)

Cited In (9)

• Bayesian Factor Model Shrinkage for Linear IV Regression With Many Instruments
• Title not available (Why is that?)
• A general system for heuristic minimization of convex functions over non-convex sets
• Identification of a linear system from inexact data: A three-variable example
• Identification of linear relations from noisy data: Geometrical aspects
• The prejudices of least squares, principal components and common factors schemes
• Certain models from uncertain data: The algebraic case
• An alternating minimization algorithm for factor analysis.
• Title not available (Why is that?)

This page was built for publication: Linear models based on noisy data and the Frisch scheme

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2808247)