Suboptimality of Penalized Empirical Risk Minimization in Classification

DOI10.1007/978-3-540-72927-3_12MaRDI QIDQ5432431zbMATH OpenOpenAlexFDO

Authors Guillaume Lecué

Publication date 3 January 2008

Published in Learning Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/math/0703811

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: Let $cF$ be a set of $M$ classification procedures with values in $[-1,1]$. Given a loss function, we want to construct a procedure which mimics at the best possible rate the best procedure in $cF$. This fastest rate is called optimal rate of aggregation. Considering a continuous scale of loss functions with various types of convexity, we prove that optimal rates of aggregation can be either $((logM)/n{)}^{1/2}$ or $(logM)/n$. We prove that, if all the $M$ classifiers are binary, the (penalized) Empirical Risk Minimization procedures are suboptimal (even under the margin/low noise condition) when the loss function is somewhat more than convex, whereas, in that case, aggregation procedures with exponential weights achieve the optimal rate of aggregation.

Recommendations

• On the optimality of the empirical risk minimization procedure for the convex aggregation problem
• Optimal rates of aggregation in classification under low noise assumption
• Empirical risk minimization is optimal for the convex aggregation problem
• Aggregation via empirical risk minimization
• Optimal aggregation of classifiers in statistical learning.

Cited in

(15)

• On the optimality of the empirical risk minimization procedure for the convex aggregation problem
• Oracle inequalities for cross-validation type procedures
• General oracle inequalities for model selection
• Penalized empirical risk minimization over Besov spaces
• Aggregation of estimators and stochastic optimization
• Fast learning rates in statistical inference through aggregation
• Empirical risk minimization is optimal for the convex aggregation problem
• Optimal learning with \textit{Q}-aggregation
• Optimal rates of aggregation in classification under low noise assumption
• Margin-adaptive model selection in statistical learning
• Aggregated hold-out
• Optimal Oracle Inequality for Aggregation of Classifiers Under Low Noise Condition
• Sharper lower bounds on the performance of the empirical risk minimization algorithm
• Aggregation via empirical risk minimization
• A survey of cross-validation procedures for model selection

This page was built for publication: Suboptimality of Penalized Empirical Risk Minimization in Classification

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5432431)