Optimization with Sparsity-Inducing Penalties

DOI10.1561/2200000015MaRDI QIDQ2903300

Julien Mairal, Rodolphe Jenatton, Francis Bach, Guillaume Obozinski

Publication date: 8 August 2012

Published in: Foundations and Trends® in Machine Learning (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1108.0775

Mathematics Subject Classification ID

62-XX: Statistics

Related Items

Unnamed Item, Unnamed Item, Low-Rank Inducing Norms with Optimality Interpretations, Transaction cost optimization for online portfolio selection, A Symmetric Alternating Direction Method of Multipliers for Separable Nonconvex Minimization Problems, Variational Gram Functions: Convex Analysis and Optimization, Polynomial Norms, Optimization Methods for Large-Scale Machine Learning, Adapting Regularized Low-Rank Models for Parallel Architectures, Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice, Implementing the Alternating Direction Method of Multipliers for Big Datasets: A Case Study of Least Absolute Shrinkage and Selection Operator, Cut Pursuit: Fast Algorithms to Learn Piecewise Constant Functions on General Weighted Graphs, Sharp Oracle Inequalities for Square Root Regularization, Gap Safe screening rules for sparsity enforcing penalties, Distributed Learning with Sparse Communications by Identification, Error Estimates for Multivariate Regression on Discretized Function Spaces, Strong convergence of over-relaxed multi-parameter proximal scaled gradient algorithm and superiorization, A Derivative-Free Method for Structured Optimization Problems, Locally sparse reconstruction using the \(\ell^{1,\infty}\)-norm, Combining a relaxed EM algorithm with Occam's razor for Bayesian variable selection in high-dimensional regression, Minimum distance Lasso for robust high-dimensional regression, On some steplength approaches for proximal algorithms, A lasso for hierarchical interactions, Block coordinate descent algorithms for large-scale sparse multiclass classification, Proximal methods for the latent group lasso penalty, Sparse time-frequency representation of nonlinear and nonstationary data, Conditional gradient algorithms for norm-regularized smooth convex optimization, Beyond support in two-stage variable selection, Sparse and constrained stochastic predictive control for networked systems, Gelfand numbers related to structured sparsity and Besov space embeddings with small mixed smoothness, Regularized optimization with spatial coupling for robust decision making, Convex relaxations of penalties for sparse correlated variables with bounded total variation, Strong convergence and bounded perturbation resilience of a modified proximal gradient algorithm, Nonsmoothness in machine learning: specific structure, proximal identification, and applications, A reproducing kernel Hilbert space approach to high dimensional partially varying coefficient model, Statistical properties of convex clustering, An iterative SVM approach to feature selection and classification in high-dimensional datasets, Feature selection in machine learning: an exact penalty approach using a difference of convex function algorithm, A primal dual active set with continuation algorithm for the \(\ell^0\)-regularized optimization problem, Efficient nonconvex sparse group feature selection via continuous and discrete optimization, Sampling in the analysis transform domain, A joint convex penalty for inverse covariance matrix estimation, Sparse and kernel OPLS feature extraction based on eigenvalue problem solving, A conjugate subgradient algorithm with adaptive preconditioning for the least absolute shrinkage and selection operator minimization, A unified formulation for generalized oilfield development optimization, Compressed sensing with structured sparsity and structured acquisition, Global convergence of ADMM in nonconvex nonsmooth optimization, Robust graph regularized nonnegative matrix factorization for clustering, Continuous relaxation for discrete DC programming, Sparse inference of the drift of a high-dimensional Ornstein-Uhlenbeck process, Fast projections onto mixed-norm balls with applications, The dual step size of the alternating direction method can be larger than 1.618 when one function is strongly convex, Compressed sparse tensor based quadrature for vibrational quantum mechanics integrals, Robust tracking via locally structured representation, Local and global convergence of a general inertial proximal splitting scheme for minimizing composite functions, Point process estimation with Mirror Prox algorithms, Tractable ADMM schemes for computing KKT points and local minimizers for \(\ell_0\)-minimization problems, Block-based refitting in \(\ell_{12}\) sparse regularization, Convergent inexact penalty decomposition methods for cardinality-constrained problems, A Laplacian approach to \(\ell_1\)-norm minimization, An effective procedure for feature subset selection in logistic regression based on information criteria, Nonconvex and nonsmooth sparse optimization via adaptively iterative reweighted methods, Sequential convergence of AdaGrad algorithm for smooth convex optimization, Variational analysis perspective on linear convergence of some first order methods for nonsmooth convex optimization problems, Sparse classification: a scalable discrete optimization perspective, Sparsest piecewise-linear regression of one-dimensional data, Dualize, split, randomize: toward fast nonsmooth optimization algorithms, On the local convergence of a stochastic semismooth Newton method for nonsmooth nonconvex optimization, A phase transition for finding needles in nonlinear haystacks with LASSO artificial neural networks, An active-set algorithm for norm constrained quadratic problems, A stochastic extra-step quasi-Newton method for nonsmooth nonconvex optimization, Sparse optimization on measures with over-parameterized gradient descent, Inertial alternating direction method of multipliers for non-convex non-smooth optimization, Efficient inexact proximal gradient algorithms for structured sparsity-inducing norm, A characterization of proximity operators, Block-sparsity regularized maximum correntropy criterion for structured-sparse system identification, Multi-block nonconvex nonsmooth proximal ADMM: convergence and rates under Kurdyka-Łojasiewicz property, An evaluation of the sparsity degree for sparse recovery with deterministic measurement matrices, Data-driven time-frequency analysis, Perspective maximum likelihood-type estimation via proximal decomposition, Data scarcity, robustness and extreme multi-label classification, Sparse low rank approximation of potential energy surfaces with applications in estimation of anharmonic zero point energies and frequencies, A telescopic Bregmanian proximal gradient method without the global Lipschitz continuity assumption, A proximal framework for fuzzy subspace clustering, A two-step fixed-point proximity algorithm for a class of non-differentiable optimization models in machine learning, An efficient primal dual prox method for non-smooth optimization, Structured regularization for conditional Gaussian graphical models, Performance of first- and second-order methods for \(\ell_1\)-regularized least squares problems, Multi-view kernel completion, An optimal subgradient algorithm with subspace search for costly convex optimization problems, A random block-coordinate Douglas-Rachford splitting method with low computational complexity for binary logistic regression, Learning with optimal interpolation norms, An unexpected connection between Bayes \(A\)-optimal designs and the group Lasso, Safe feature screening rules for the regularized Huber regression, A second-order method for convex₁-regularized optimization with active-set prediction, Sparse Learning for Large-Scale and High-Dimensional Data: A Randomized Convex-Concave Optimization Approach, Robust Decoding from 1-Bit Compressive Sampling with Ordinary and Regularized Least Squares, Oracle Inequalities for Local and Global Empirical Risk Minimizers, A Least-Squares Method for Sparse Low Rank Approximation of Multivariate Functions, A Convex Approach for Image Restoration with Exact Poisson--Gaussian Likelihood, Unnamed Item, Unnamed Item, Sparsity-promoting algorithms for the discovery of informative Koopman-invariant subspaces, Convergence of proximal gradient algorithm in the presence of adjoint mismatch ^*, Unnamed Item, Unnamed Item, Unnamed Item, On the Effectiveness of Richardson Extrapolation in Data Science, Proximal Gradient Methods with Adaptive Subspace Sampling, Proximal Gradient Methods for Machine Learning and Imaging, Global optimization for sparse solution of least squares problems, A penalty decomposition approach for multi-objective cardinality-constrained optimization problems, Parameter choices for sparse regularization with the ℓ1 norm ^*, Unnamed Item, A Subspace Acceleration Method for Minimization Involving a Group Sparsity-Inducing Regularizer, Multivariate Monotone Inclusions in Saddle Form, Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning, The Signature Kernel Is the Solution of a Goursat PDE, A Distributed Flexible Delay-Tolerant Proximal Gradient Algorithm, A Stochastic Semismooth Newton Method for Nonsmooth Nonconvex Optimization, A new convergence analysis and perturbation resilience of some accelerated proximal forward–backward algorithms with errors, Convex Optimization and Parsimony of $L_p$-balls Representation, Structured sparsity through convex optimization, On Algorithms for Difference of Monotone Operators, On Algorithms for Difference of Monotone Operators, A Trust-region Method for Nonsmooth Nonconvex Optimization, Safe Rules for the Identification of Zeros in the Solutions of the SLOPE Problem, Newton acceleration on manifolds identified by proximal gradient methods, Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists, Distributed Sparse Composite Quantile Regression in Ultrahigh Dimensions, A computational framework for edge-preserving regularization in dynamic inverse problems, Penalized wavelet nonparametric univariate logistic regression for irregular spaced data, A unifying framework for sparsity-constrained optimization, Root-finding approaches for computing conformal prediction set, Smooth over-parameterized solvers for non-smooth structured optimization, Global optimization using random embeddings, Compression and data similarity: combination of two techniques for communication-efficient solving of distributed variational inequalities, Recent Theoretical Advances in Non-Convex Optimization