The adaptive projected subgradient method constrained by families of quasi-nonexpansive mappings and its application to online learning

DOI10.1137/100807004zbMATH Open1266.47113arXiv1008.5231OpenAlexW2963338986MaRDI QIDQ5300520FDOQ5300520

Isao Yamada, Konstantinos Slavakis

Publication date: 27 June 2013

Published in: SIAM Journal on Optimization (Search for Journal in Brave)

Abstract: Many online, i.e., time-adaptive, inverse problems in signal processing and machine learning fall under the wide umbrella of the asymptotic minimization of a sequence of non-negative, convex, and continuous functions. To incorporate a-priori knowledge into the design, the asymptotic minimization task is usually constrained on a fixed closed convex set, which is dictated by the available a-priori information. To increase versatility towards the usage of the available information, the present manuscript extends the Adaptive Projected Subgradient Method (APSM) by introducing an algorithmic scheme which incorporates a-priori knowledge in the design via a sequence of strongly attracting quasi-nonexpansive mappings in a real Hilbert space. In such a way, the benefits offered to online learning tasks by the proposed method unfold in two ways: 1) the rich class of quasi-nonexpansive mappings provides a plethora of ways to cast a-priori knowledge, and 2) by introducing a sequence of such mappings, the proposed scheme is able to capture the time-varying nature of a-priori information. The convergence properties of the algorithm are studied, several special cases of the method with wide applicability are shown, and the potential of the proposed scheme is demonstrated by considering an increasingly important, nowadays, online sparse system/signal recovery task.

Full work available at URL: https://arxiv.org/abs/1008.5231

Recommendations

• The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings
• An adaptive family of projection methods for constrained monotone nonlinear equations with applications
• Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions
• Sliding window generalized kernel affine projection algorithm using projection mappings
• scientific article; zbMATH DE number 2196506

zbMATH Keywords

sparsityonline learningquasi-nonexpansive mappingsubgradient methodadaptive filteringasymptotic minimization

Mathematics Subject Classification ID

Numerical optimization and variational techniques (65K10) Learning and adaptive systems in artificial intelligence (68T05) Convex programming (90C25) Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Contraction-type mappings, nonexpansive mappings, (A)-proper mappings, etc. (47H09) Applications of operator theory in optimization, convex analysis, mathematical programming, economics (47N10) Numerical solution to inverse problems in abstract spaces (65J22) Computer-assisted instruction, e-learning (aspects of mathematics education) (97U50)

Cited In (8)

• On the finite convergence of a projected cutter method
• Acoustic feedback cancellation based on weighted adaptive projection subgradient method in hearing aids
• The Method of Cyclic Intrepid Projections: Convergence Analysis and Numerical Experiments
• On Subgradient Projectors
• Data-driven control of nonlinear systems: an on-line direct approach
• The Adaptive Projected Subgradient Method over the Fixed Point Set of Strongly Attracting Nonexpansive Mappings
• Adaptive Projected Subgradient Method for Asymptotic Minimization of Sequence of Nonnegative Convex Functions
• Quasi-Nonexpansive Iterations on the Affine Hull of Orbits: From Mann's Mean Value Algorithm to Inertial Methods

Uses Software

• UNLocBoX

This page was built for publication: The adaptive projected subgradient method constrained by families of quasi-nonexpansive mappings and its application to online learning

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5300520)