Stochastic proximal-gradient algorithms for penalized mixed models
From MaRDI portal
Publication:2329762
Abstract: Motivated by penalized likelihood maximization in complex models, we study optimization problems where neither the function to optimize nor its gradient have an explicit expression, but its gradient can be approximated by a Monte Carlo technique. We propose a new algorithm based on a stochastic approximation of the Proximal-Gradient (PG) algorithm. This new algorithm, named Stochastic Approximation PG (SAPG) is the combination of a stochastic gradient descent step which - roughly speaking - computes a smoothed approximation of the past gradient along the iterations, and a proximal step. The choice of the step size and the Monte Carlo batch size for the stochastic gradient descent step in SAPG are discussed. Our convergence results cover the cases of biased and unbiased Monte Carlo approximations. While the convergence analysis of the Monte Carlo-PG is already addressed in the literature (see Atchad'e et al. [2016]), the convergence analysis of SAPG is new. The two algorithms are compared on a linear mixed effect model as a toy example. A more challenging application is proposed on non-linear mixed effect models in high dimension with a pharmacokinetic data set including genomic covariates. To our best knowledge, our work provides the first convergence result of a numerical method designed to solve penalized Maximum Likelihood in a non-linear mixed effect model.
Recommendations
- \(\ell_{1}\)-penalization for mixture regression models
- Stochastic proximal linear method for structured non-convex problems
- Penalized maximum likelihood estimation of a stochastic multivariate regression model
- A globally convergent algorithm for Lasso-penalized mixture of linear regression models
- Computation for latent variable model estimation: a unified stochastic proximal framework
- Partially projected gradient algorithms for computing nonparametric maximum likelihood estimates of mixing distributions
- Mixed Lasso estimator for stochastic restricted regression models
- Inexact proximal stochastic gradient method for convex composite optimization
- Proximal penalty-duality algorithms for mixed optimality conditions
Cites work
- \(\ell_{1}\)-penalization for mixture regression models
- A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems
- A SAEM algorithm for fused Lasso penalized nonlinear mixed effect models: application to group comparison in pharmacokinetics
- A stochastic inertial forward–backward splitting algorithm for multivariate monotone inclusions
- Adaptive equi-energy sampler: convergence and illustration
- An automated (Markov chain) Monte Carlo EM algorithm
- Convergence of a stochastic approximation version of the EM algorithm
- Convergence of adaptive and interacting Markov chain Monte Carlo algorithms
- Convergence of stochastic proximal gradient algorithm
- Convergence of the Wang-Landau algorithm
- Convex analysis and monotone operator theory in Hilbert spaces
- Coupling and Ergodicity of Adaptive Markov Chain Monte Carlo Algorithms
- Extended Bayesian information criteria for model selection with large model spaces
- scientific article; zbMATH DE number 48727 (Why is no real title available?)
- scientific article; zbMATH DE number 3567782 (Why is no real title available?)
- scientific article; zbMATH DE number 1220667 (Why is no real title available?)
- scientific article; zbMATH DE number 1972910 (Why is no real title available?)
- scientific article; zbMATH DE number 2117879 (Why is no real title available?)
- scientific article; zbMATH DE number 3310599 (Why is no real title available?)
- scientific article; zbMATH DE number 3192366 (Why is no real title available?)
- Latent variable models and factor analysis. A unified approach
- Markov Chains and Stochastic Stability
- Mathematical statistics. Basic ideas and selected topics. Volume I
- Modified Fejér sequences and applications
- On perturbed proximal gradient algorithms
- On the convergence properties of the EM algorithm
- On the ergodicity of the adaptive Metropolis algorithm on unbounded domains
- On the ergodicity properties of some adaptive MCMC algorithms
- Penalized nonlinear mixed effects model to identify biomarkers that predict disease progression
- Signal Recovery by Proximal Forward-Backward Splitting
- Stochastic approximations and perturbations in forward-backward splitting for monotone operators
- Stochastic quasi-Fejér block-coordinate fixed point iterations with random sweeping
- Sufficient conditions for the convergence of monotonic mathematical programming algorithms
- The EM Algorithm and Extensions, 2E
Cited in
(6)- Fast selection of nonlinear mixed effect models using penalized likelihood
- Maximum likelihood estimation of regularization parameters in high-dimensional inverse problems: an empirical Bayesian approach. I: Methodology and experiments
- On perturbed proximal gradient algorithms
- The Split Gibbs Sampler Revisited: Improvements to Its Algorithmic Structure and Augmented Target Distribution
- Penalized Poisson model for network meta-analysis of individual patient time-to-event data
- A stability-enhanced Lasso approach for covariate selection in non-linear mixed effects model
This page was built for publication: Stochastic proximal-gradient algorithms for penalized mixed models
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2329762)
