Unifying adversarial training algorithms with data gradient regularization

DOI10.1162/NECO_A_00928MaRDI QIDQ5380676zbMATH OpenDBLPWikidataFDO

Authors Alexander G. Ororbia II, Daniel Kifer, C. Lee Giles

Publication date 6 June 2019

Published in Neural Computation (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1601.07213

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07)

Abstract: Many previous proposals for adversarial training of deep neural nets have included di- rectly modifying the gradient, training on a mix of original and adversarial examples, using contractive penalties, and approximately optimizing constrained adversarial ob- jective functions. In this paper, we show these proposals are actually all instances of optimizing a general, regularized objective we call DataGrad. Our proposed DataGrad framework, which can be viewed as a deep extension of the layerwise contractive au- toencoder penalty, cleanly simplifies prior work and easily allows extensions such as adversarial training with multi-task cues. In our experiments, we find that the deep gra- dient regularization of DataGrad (which also has L1 and L2 flavors of regularization) outperforms alternative forms of regularization, including classical L1, L2, and multi- task, both on the original dataset as well as on adversarial sets. Furthermore, we find that combining multi-task optimization with DataGrad adversarial training results in the most robust performance.

Recommendations

• Adversarial defense via the data-dependent activation, total variation minimization, and adversarial training
• A fast saddle-point dynamical system approach to robust deep learning
• Manifold adversarial training for supervised and semi-supervised learning
• Adversarial perturbations of deep neural networks
• Theoretical investigation of generalization bounds for adversarial learning of deep neural networks

Cited in

(9)

• Adversarial defense via the data-dependent activation, total variation minimization, and adversarial training
• Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
• Graph interpolating activation improves both natural and robust accuracies in data-efficient deep learning
• LADDER: latent boundary-guided adversarial training
• Wavelet regularization benefits adversarial training
• A fast saddle-point dynamical system approach to robust deep learning
• Manifold adversarial training for supervised and semi-supervised learning
• Lagrangian objective function leads to improved unforeseen attack generalization
• A3T: accuracy aware adversarial training

This page was built for publication: Unifying adversarial training algorithms with data gradient regularization

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5380676)