Variational analysis of constrained M-estimators
From MaRDI portal
Abstract: We propose a unified framework for establishing existence of nonparametric M-estimators, computing the corresponding estimates, and proving their strong consistency when the class of functions is exceptionally rich. In particular, the framework addresses situations where the class of functions is complex involving information and assumptions about shape, pointwise bounds, location of modes, height at modes, location of level-sets, values of moments, size of subgradients, continuity, distance to a "prior" function, multivariate total positivity, and any combination of the above. The class might be engineered to perform well in a specific setting even in the presence of little data. The framework views the class of functions as a subset of a particular metric space of upper semicontinuous functions under the Attouch-Wets distance. In addition to allowing a systematic treatment of numerous M-estimators, the framework yields consistency of plug-in estimators of modes of densities, maximizers of regression functions, level-sets of classifiers, and related quantities, and also enables computation by means of approximating parametric classes. We establish consistency through a one-sided law of large numbers, here extended to sieves, that relaxes assumptions of uniform laws, while ensuring global approximations even under model misspecification.
Recommendations
Cites work
- scientific article; zbMATH DE number 4013613 (Why is no real title available?)
- scientific article; zbMATH DE number 5819433 (Why is no real title available?)
- scientific article; zbMATH DE number 5654889 (Why is no real title available?)
- scientific article; zbMATH DE number 611183 (Why is no real title available?)
- scientific article; zbMATH DE number 3797051 (Why is no real title available?)
- scientific article; zbMATH DE number 823368 (Why is no real title available?)
- scientific article; zbMATH DE number 3336465 (Why is no real title available?)
- A Characterization of Epi-Convergence in Terms of Convergence of Level Sets
- A Neyman–Pearson Approach to Statistical Learning
- A multivariate version of isotonic regression
- Approximation by log-concave distributions, with applications to regression
- Approximations and solution estimates in optimization
- Asymptotic Statistics
- Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems
- Asymptotics of least-squares estimators for constrained nonlinear regression
- Consistency for the least squares estimator in nonparametric regression
- Consistency of Support Vector Machines and Other Regularized Kernel Classifiers
- Constrained penalized splines
- Erratum to: ``Multivariate epi-splines and evolving function identification problems
- Estimating density functions: a constrained maximum likelihood approach*
- Estimation of unimodal densities without smoothness assumptions
- Fusion of hard and soft information in nonparametric density estimation
- Lopsided convergence: an extension and its quantification
- Maximum likelihood estimation of a multi-dimensional log-concave density. With discussion and authors' reply
- Monotone Approximation by Polynomials
- Monotone Approximation by Splines
- Nonparametric estimation of a smooth density with shape restrictions
- Nonparametric estimation of density and hazard rate functions with shape restrictions
- Nonparametric estimation of multivariate convex-transformed densities
- Nonparametric least squares estimation of a multivariate convex regression function
- Nonparametric maximum likelihood estimation by the method of sieves
- Note on the Consistency of the Maximum Likelihood Estimate
- On shape-preserving probabilistic wavelet approximators
- On the Convergence in Distribution of Measurable Multifunctions (Random Sets) Normal Integrands, Stochastic Processes and Stochastic Infima
- Quasi-concave density estimation
- Random LSC Functions: An Ergodic Theorem
- Theoretical properties of the log-concave maximum likelihood estimator of a multidimensional density
- Total positivity in Markov structures
- Tunability: importance of hyperparameters of machine learning algorithms
- Univariate log-concave density estimation with symmetry or modal constraints
- Variational Analysis
- Variational theory for optimization under stochastic ambiguity
- Weak convergence and empirical processes. With applications to statistics
Cited in
(9)- Set-convergence and its application: a tutorial
- scientific article; zbMATH DE number 4181268 (Why is no real title available?)
- On estimation of variance components with constraints
- Asymptotic Properties of Stationary Solutions of Coupled Nonconvex Nonsmooth Empirical Risk Minimization
- Analytic approach to variance optimization under an \(\mathcal{l}_1\) constraint
- scientific article; zbMATH DE number 7733444 (Why is no real title available?)
- A variational approach to a cumulative distribution function estimation problem under stochastic ambiguity
- Optimization hierarchy for fair statistical decision problems
- Performance Bounds for PDE-Constrained Optimization under Uncertainty
This page was built for publication: Variational analysis of constrained M-estimators
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2215758)
