A novel multi-scale loss function for classification problems in machine learning

DOI10.1016/J.JCP.2023.112679MaRDI QIDQ6147046zbMATH OpenFDO

Authors Leonid Berlyand, Robert Creese, Pierre-Emmanuel Jabin

Publication date 31 January 2024

Published in Journal of Computational Physics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2106.02676

zbMATH Keywords

deep neural networksmachine learningmulti-scale

Mathematics Subject Classification ID

Numerical methods for partial differential equations, boundary value problems (65Nxx) Artificial intelligence (68Txx) Computing methodologies and applications (68Uxx)

Abstract: We introduce two-scale loss functions for use in various gradient descent algorithms applied to classification problems via deep neural networks. This new method is generic in the sense that it can be applied to a wide range of machine learning architectures, from deep neural networks to support vector machines for example. These two-scale loss functions allow to focus the training onto objects in the training set which are not well classified. This leads to an increase in several measures of performance for appropriately-defined two-scale loss functions with respect to the more classical cross-entropy when tested on traditional deep neural networks on the MNIST, CIFAR10, and CIFAR100 data-sets.

Recommendations

• A modification of adaptive moment estimation (Adam) for machine learning
• Method for improving gradient boosting learning efficiency based on modified loss functions
• Employing different loss functions for the classification of images via supervised learning
• Artificial neural networks with uniform norm-based loss functions
• Levenberg-Marquardt multi-classification using hinge loss function

Cites work

• A multi-scale DNN algorithm for nonlinear elliptic equations with multiple scales
• Deep convolutional neural networks for image classification: a comprehensive review
• scientific article; zbMATH DE number 49594 (Why is no real title available?)
• On the convergence of formally diverging neural net-based classifiers

Cited in

(4)

• Artificial neural networks with uniform norm-based loss functions
• Generalization of cross-entropy loss function for image classification
• An error analysis for deep binary classification with sigmoid loss
• Enhancing deep learning algorithm accuracy and stability using multicriteria optimization: an application to distributed learning with MNIST digits

This page was built for publication: A novel multi-scale loss function for classification problems in machine learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6147046)