Variational analysis of constrained M-estimators
From MaRDI portal
Abstract: We propose a unified framework for establishing existence of nonparametric M-estimators, computing the corresponding estimates, and proving their strong consistency when the class of functions is exceptionally rich. In particular, the framework addresses situations where the class of functions is complex involving information and assumptions about shape, pointwise bounds, location of modes, height at modes, location of level-sets, values of moments, size of subgradients, continuity, distance to a "prior" function, multivariate total positivity, and any combination of the above. The class might be engineered to perform well in a specific setting even in the presence of little data. The framework views the class of functions as a subset of a particular metric space of upper semicontinuous functions under the Attouch-Wets distance. In addition to allowing a systematic treatment of numerous M-estimators, the framework yields consistency of plug-in estimators of modes of densities, maximizers of regression functions, level-sets of classifiers, and related quantities, and also enables computation by means of approximating parametric classes. We establish consistency through a one-sided law of large numbers, here extended to sieves, that relaxes assumptions of uniform laws, while ensuring global approximations even under model misspecification.
For computing nonparametric \(M\)-estimators, more general versions of the optimization problem, as used for parametric \(M\)-estimators, are considered. Here, the unknown quantity is a certain function \(f\), lying in a given set \(F\) of functions on \(\mathbb{R}^d\) with dimension \(d\). An example is the probability density to be estimated in case of nonparametric maximum likelihood (ML)-estimators. With a certain loss function \(\psi = \psi(x,f)\), the objective function \(\Psi=\Psi(f)\) to be minimized with respect to \(f \in F\) is defined by the average of \(\psi(x_i,f)\), \(i=1,\dots,n\), with samples \(x_i\) of random variables \(X_i\), \(i=1,\dots,n\); moreover, \(\Psi(f)\) involves still an additive penalty term \(\pi = \pi(f)\) for possible smoothing or regulation. Under certain assumptions, results on the existence and consistency of the resulting solutions of the constrained \(M\)-estimator problem are given. Moreover, an algorithm for an approximate solution of the \(M\)-estimator problem is suggested and examples are given.
Recommendations
Cites work
- A Characterization of Epi-Convergence in Terms of Convergence of Level Sets
- A multivariate version of isotonic regression
- A Neyman–Pearson Approach to Statistical Learning
- Approximation by log-concave distributions, with applications to regression
- Approximations and solution estimates in optimization
- Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems
- Asymptotic Statistics
- Asymptotics of least-squares estimators for constrained nonlinear regression
- Consistency for the least squares estimator in nonparametric regression
- Consistency of Support Vector Machines and Other Regularized Kernel Classifiers
- Constrained penalized splines
- Erratum to: ``Multivariate epi-splines and evolving function identification problems
- Estimating density functions: a constrained maximum likelihood approach*
- Estimation of unimodal densities without smoothness assumptions
- Fusion of hard and soft information in nonparametric density estimation
- scientific article; zbMATH DE number 4013613 (Why is no real title available?)
- scientific article; zbMATH DE number 5819433 (Why is no real title available?)
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 611183 (Why is no real title available?)
- scientific article; zbMATH DE number 3797051 (Why is no real title available?)
- scientific article; zbMATH DE number 823368 (Why is no real title available?)
- scientific article; zbMATH DE number 3336465 (Why is no real title available?)
- Lopsided convergence: an extension and its quantification
- Maximum likelihood estimation of a multi-dimensional log-concave density. With discussion and authors' reply
- Monotone Approximation by Polynomials
- Monotone Approximation by Splines
- Nonparametric estimation of a smooth density with shape restrictions
- Nonparametric estimation of density and hazard rate functions with shape restrictions
- Nonparametric estimation of multivariate convex-transformed densities
- Nonparametric least squares estimation of a multivariate convex regression function
- Nonparametric maximum likelihood estimation by the method of sieves
- Note on the Consistency of the Maximum Likelihood Estimate
- On shape-preserving probabilistic wavelet approximators
- On the Convergence in Distribution of Measurable Multifunctions (Random Sets) Normal Integrands, Stochastic Processes and Stochastic Infima
- Quasi-concave density estimation
- Random LSC Functions: An Ergodic Theorem
- Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density
- Total positivity in Markov structures
- Tunability: importance of hyperparameters of machine learning algorithms
- Univariate log-concave density estimation with symmetry or modal constraints
- Variational Analysis
- Variational theory for optimization under stochastic ambiguity
- Weak convergence and empirical processes. With applications to statistics
Cited in
(10)- On estimation of variance components with constraints
- Optimization hierarchy for fair statistical decision problems
- scientific article; zbMATH DE number 4181268 (Why is no real title available?)
- Asymptotic Properties of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization
- Analytic approach to variance optimization under an \(\mathcal{l}_1\) constraint
- Performance Bounds for PDE-Constrained Optimization under Uncertainty
- scientific article; zbMATH DE number 7733444 (Why is no real title available?)
- A variational approach to a cumulative distribution function estimation problem under stochastic ambiguity
- Epi-consistent approximationof stochastic dynamic programs
- Set-convergence and its application: a tutorial
This page was built for publication: Variational analysis of constrained M-estimators
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2215758)
