The geometry of adversarial training in binary classification

DOI10.1093/IMAIAI/IAAC029MaRDI QIDQ6039764zbMATH OpenOpenAlexFDO

Authors Leon Bungert, Nicolás García Trillos, Ryan W. Murray

Publication date 23 May 2023

Published in Information and Inference: A Journal of the IMA (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/2111.13613

zbMATH Keywords

adversarial trainingexistence of solutionsnonlocal perimeternonlocal total variationregularity

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Numerical optimization and variational techniques (65K10) Learning and adaptive systems in artificial intelligence (68T05)

Abstract: We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type $L^1+$ (nonlocal) $operatornameTV$, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows us to establish a series of properties of optimal solutions of the original problem, including the existence of minimal and maximal solutions (interpreted in a suitable sense), and the existence of regular solutions (also interpreted in a suitable sense). In addition, we highlight how the connection between adversarial training and perimeter minimization problems provides a novel, directly interpretable, statistical motivation for a family of regularized risk minimization problems involving perimeter/total variation. The majority of our theoretical results are independent of the distance used to define adversarial attacks.

Recommendations

• Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning
• Adversarial classification via distributional robustness with Wasserstein ambiguity
• Precise statistical analysis of classification accuracies for adversarial training
• An Analytical and Geometric Perspective on Adversarial Robustness
• On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it

Cited in

(17)

• Adversarial classification via distributional robustness with Wasserstein ambiguity
• Neural ODE Control for Classification, Approximation, and Transport
• Gamma-convergence of a nonlocal perimeter arising in adversarial machine learning
• Variational methods for evolution. Abstracts from the workshop held December 5--8, 2023
• Eikonal depth: an optimal control approach to statistical depths
• Nonasymptotic bounds for adversarial excess risk under misspecified models
• On adversarial robustness and the use of Wasserstein ascent-descent dynamics to enforce it
• A mean curvature flow arising in adversarial training
• A notion of uniqueness for the adversarial Bayes classifier
• Nonlocal perimeters and variations: extremality and decomposability for finite and infinite horizons
• Instability in deep learning -- when algorithms cannot compute uncertainty quantifications for neural networks
• The mathematics of adversarial attacks in AI -- why deep learning is unstable despite the existence of stable neural networks
• Do stable neural networks exist for classification problems? -- A new view on stability in AI
• Adversarial flows: a gradient flow characterization of adversarial attacks
• Uniform convergence of adversarially robust classifiers
• On the existence of solutions to adversarial training in multiclass classification
• Distributionally robust optimization

This page was built for publication: The geometry of adversarial training in binary classification

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6039764)