The power of localization for efficiently learning linear separators with noise

DOI10.1145/3006384zbMATH Open1426.68234arXiv1307.8371OpenAlexW2620857485MaRDI QIDQ3177877FDOQ3177877

Philip M. Long, Pranjal Awasthi, Maria-Florina Balcan

Publication date: 2 August 2018

Published in: Journal of the ACM, Proceedings of the forty-sixth annual ACM symposium on Theory of computing (Search for Journal in Brave)

Abstract: We introduce a new approach for designing computationally efficient learning algorithms that are tolerant to noise, and demonstrate its effectiveness by designing algorithms with improved noise tolerance guarantees for learning linear separators. We consider both the malicious noise model and the adversarial label noise model. For malicious noise, where the adversary can corrupt both the label and the features, we provide a polynomial-time algorithm for learning linear separators in $R{e}^d$ under isotropic log-concave distributions that can tolerate a nearly information-theoretically optimal noise rate of $eta=Omega(epsilon)$. For the adversarial label noise model, where the distribution over the feature vectors is unchanged, and the overall probability of a noisy label is constrained to be at most $eta$, we also give a polynomial-time algorithm for learning linear separators in $R{e}^d$ under isotropic log-concave distributions that can handle a noise rate of $eta=Omegaleft(epsilonight)$. We show that, in the active learning model, our algorithms achieve a label complexity whose dependence on the error parameter $epsilon$ is polylogarithmic. This provides the first polynomial-time active learning algorithm for learning linear separators in the presence of malicious noise or adversarial label noise.

Full work available at URL: https://arxiv.org/abs/1307.8371

Recommendations

• The power of localization for efficiently learning linear separators with noise
• Learning Halfspaces with Malicious Noise
• Learning halfspaces with malicious noise
• Learning geometric concepts with nasty noise
• A polynomial-time algorithm for learning noisy linear threshold functions

zbMATH Keywords

learning theorylocalizationactive learninglinear classificationagnostic learningnoise-tolerant learningmalicious noiseadversarial label noisepassive and active learning

Mathematics Subject Classification ID

Learning and adaptive systems in artificial intelligence (68T05) Computational learning theory (68Q32)

Cites Work

• Theory of Cryptography
• The geometry of differential privacy: the small database and approximate cases
• On the geometry of differential privacy
• Interactive privacy via the median mechanism
• The price of privately releasing contingency tables and the spectra of random matrices with correlated rows
• Lower Bounds in Differential Privacy
• Iterative Constructions and Private Data Release
• Differential Privacy and the Fat-Shattering Dimension of Linear Queries
• Our Data, Ourselves: Privacy Via Distributed Noise Generation
• On the complexity of differentially private data release
• Title not available (Why is that?)
• Collusion-secure fingerprinting for digital data
• Advances in Cryptology – CRYPTO 2004
• Title not available (Why is that?)
• New Efficient Attacks on Statistical Disclosure Control Mechanisms
• Bounds on the Sample Complexity for Private Learning and Private Data Release
• Answering n _{2+o(1)} counting queries with differential privacy is hard
• Characterizing the sample complexity of private learners
• Faster Algorithms for Privately Releasing Marginals
• Faster private release of marginals on small databases
• Private Learning and Sanitization: Pure vs. Approximate Differential Privacy
• Privately releasing conjunctions and the statistical query barrier
• Efficient algorithms for privately releasing marginals via convex relaxations

Cited In (13)

• Active Nearest-Neighbor Learning in Metric Spaces
• Agnostically Learning Boolean Functions with Finite Polynomial Representation
• Stronger data poisoning attacks break data sanitization defenses
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Active learning by query by committee with robust divergences
• Learning with a Drifting Target Concept
• Robust Estimators in High-Dimensions Without the Computational Intractability
• Learning from binary labels with instance-dependent noise
• Testing distributional assumptions of learning algorithms
• Statistical active learning algorithms for noise tolerance and differential privacy
• Learning under \(p\)-tampering poisoning attacks

Uses Software

• SuLQ

This page was built for publication: The power of localization for efficiently learning linear separators with noise

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3177877)