A Statistical Learning Approach to Modal Regression

zbMATH Open1497.68418arXiv1702.05960MaRDI QIDQ4969033FDOQ4969033

Authors: Yun-Long Feng, Jun Fan, Johan A. K. Suykens 

Publication date: 5 October 2020

Abstract: This paper studies the nonparametric modal regression problem systematically from a statistical learning view. Originally motivated by pursuing a theoretical understanding of the maximum correntropy criterion based regression (MCCR), our study reveals that MCCR with a tending-to-zero scale parameter is essentially modal regression. We show that nonparametric modal regression problem can be approached via the classical empirical risk minimization. Some efforts are then made to develop a framework for analyzing and implementing modal regression. For instance, the modal regression function is described, the modal regression risk is defined explicitly and its extit{Bayes} rule is characterized; for the sake of computational tractability, the surrogate modal regression risk, which is termed as the generalization risk in our study, is introduced. On the theoretical side, the excess modal regression risk, the excess generalization risk, the function estimation error, and the relations among the above three quantities are studied rigorously. It turns out that under mild conditions, function estimation consistency and convergence may be pursued in modal regression as in vanilla regression protocols, such as mean regression, median regression, and quantile regression. However, it outperforms these regression models in terms of robustness as shown in our study from a re-descending M-estimation view. This coincides with and in return explains the merits of MCCR on robustness. On the practical side, the implementation issues of modal regression including the computational algorithm and the tuning parameters selection are discussed. Numerical assessments on modal regression are also conducted to verify our findings empirically.

Full work available at URL: https://arxiv.org/abs/1702.05960

Recommendations

• Nonparametric statistical learning based on modal regression
• Regularized modal regression with data-dependent hypothesis spaces
• Empirical likelihood based modal regression
• Nonparametric modal regression
• A new regression model: modal linear regression
• Modal linear regression using log-concave distributions
• Modal regression estimation for linear models with longitudinal data
• Modal regression based on nonparametric quantile estimator
• Robust nonparametric regression and modality
• Statistical learning from a regression perspective

zbMATH Keywords

kernel density estimationempirical risk minimizationgeneralization boundsstatistical learning theorynonparametric modal regression

Mathematics Subject Classification ID

Nonparametric regression and quantile regression (62G08) General nonlinear regression (62J02) Learning and adaptive systems in artificial intelligence (68T05)

Cites Work

• Nonparametric modal regression in the presence of measurement error
• Learning Theory
• Support Vector Machines
• Correntropy: Properties and Applications in Non-Gaussian Signal Processing
• On Estimation of a Probability Density Function and Mode
• Statistical behavior and consistency of classification methods based on convex risk minimization.
• Local Rademacher complexities
• The estimation of the gradient of a density function, with applications in pattern recognition
• Robust Inference for Generalized Linear Models
• Semiparametric mixtures of nonparametric regressions
• A note on prediction via estimation of the conditional mode function
• Rates of consistency for nonparametric estimation of the mode in absence of smoothness assumptions
• Estimation of the mode
• Non-parametric estimation of the conditional mode
• A Note on Ergodic Processes Prediction via Estimation of the Conditional Mode Function
• A nonparametric conditional mode estimate
• Information theoretic learning. Renyi's entropy and kernel perspectives
• Convexity, Classification, and Risk Bounds
• Optimum kernel estimators
• Robust variable selection for nonlinear models with diverging number of parameters
• Local modal regression
• Robust Variable Selection With Exponential Squared Loss
• Maximum likelihood estimation of isotonic modal regression
• Nonparametric modal regression
• Consistency analysis of an empirical minimum error entropy algorithm
• A new regression model: modal linear regression
• Neural Network Learning
• Learning theory approach to minimum error entropy criterion
• Modelling Beyond Regression Functions: An Application of Multimodal Regression to Speed–Flow Data
• Mode regression
• Global rates of convergence of the MLEs of log-concave and \(s\)-concave densities
• Multi-kernel regularized classifiers
• The mode functional is not elicitable
• Learning with the maximum correntropy criterion induced losses for regression
• Generalized Correntropy for Robust<?Pub _newline ?>Adaptive Filtering
• Quadratic mode regression
• Robust Principal Component Analysis Based on Maximum Correntropy Criterion
• Functional time series prediction via conditional mode estimation
• Regression towards the mode
• On semiparametric mode regression estimation
• Semiparametric econometric estimators for a truncated regression model: a review with an extension
• Learning with correntropy-induced losses for regression with mixture of symmetric stable noise
• Learning under \((1 + \epsilon)\)-moment conditions
• Non linear parametric mode regression
• An Iterative Procedure for Estimating the Mode
• Nonparametric forecasting: a comparison of three kernel-based methods
• Optimal learning with Gaussians and correntropy loss

Cited In (35)

• Nonparametric modal regression with mixed variables and application to analyze the GDP data
• Half-quadratic alternating direction method of multipliers for robust orthogonal tensor approximation
• Learning with correntropy-induced losses for regression with mixture of symmetric stable noise
• Distributed penalized modal regression for massive data
• On empirical estimation of mode based on weakly dependent samples
• Semiparametric partially linear varying coefficient modal regression
• On the convergence of gradient descent for robust functional linear regression
• Statistical Inference with Local Optima
• Nonparametric statistical learning based on modal regression
• Nonparametric modal regression
• Online minimum error entropy algorithm with unbounded sampling
• Robust kernel-based distribution regression
• Regularized modal regression with data-dependent hypothesis spaces
• Large margin unified machines with non-i.i.d. process
• Error analysis on regularized regression based on the maximum correntropy criterion
• Modal linear regression models with additive distortion measurement errors
• Bootstrap Inference for Quantile-based Modal Regression
• Modal regression based on nonparametric quantile estimator
• Comparison theorems on large-margin learning
• Modal additive models with data-driven structure identification
• Robust partially linear trend filtering for regression estimation and structure discovery
• Empirical likelihood based modal regression
• Learning under \((1 + \epsilon)\)-moment conditions
• Robust estimation for nonrandomly distributed data
• New insights into learning with correntropy-based regression
• Kernel-based maximum correntropy criterion with gradient descent method
• Fuzzy modeling using the similarity-based approximate reasoning system
• Modal regression models based on B-splines
• A Framework of Learning Through Empirical Gain Maximization
• Optimal subsampling for modal regression in massive data
• Maximum correntropy criterion regression models with tending-to-zero scale parameters
• Quantile regression approach to conditional mode estimation
• Optimal learning with Gaussians and correntropy loss
• Learning theory of minimum error entropy under weak moment conditions
• Some asymptotic properties of kernel regression estimators of the mode for stationary and ergodic continuous time processes

This page was built for publication: A Statistical Learning Approach to Modal Regression

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4969033)