Quantile regression for large-scale applications
From MaRDI portal
Publication:2940044
DOI10.1137/130919258zbMATH Open1314.68391arXiv1305.0087OpenAlexW2586258055MaRDI QIDQ2940044FDOQ2940044
Authors: Jiyan Yang, Xiangrui Meng, Michael W. Mahoney 
Publication date: 23 January 2015
Published in: SIAM Journal on Scientific Computing (Search for Journal in Brave)
Abstract: Quantile regression is a method to estimate the quantiles of the conditional distribution of a response variable, and as such it permits a much more accurate portrayal of the relationship between the response variable and observed covariates than methods such as Least-squares or Least Absolute Deviations regression. It can be expressed as a linear program, and, with appropriate preprocessing, interior-point methods can be used to find a solution for moderately large problems. Dealing with very large problems, emph(e.g.), involving data up to and beyond the terabyte regime, remains a challenge. Here, we present a randomized algorithm that runs in nearly linear time in the size of the input and that, with constant probability, computes a approximate solution to an arbitrary quantile regression problem. As a key step, our algorithm computes a low-distortion subspace-preserving embedding with respect to the loss function of quantile regression. Our empirical evaluation illustrates that our algorithm is competitive with the best previous work on small to medium-sized problems, and that in addition it can be implemented in MapReduce-like environments and applied to terabyte-sized problems.
Full work available at URL: https://arxiv.org/abs/1305.0087
Recommendations
Cited In (26)
- Distributed quantile regression for longitudinal big data
- An Implementation for Regression Quantile Estimation
- Random projections for linear programming
- First-Order Newton-Type Estimator for Distributed Estimation and Inference
- Random projections for linear programming: an improved retrieval phase
- Weighted SGD for \(\ell_p\) regression with randomized preconditioning
- Communication-efficient surrogate quantile regression for non-randomly distributed system
- Penalized and constrained LAD estimation in fixed and high dimension
- Random projections for quadratic programs
- Communication-efficient estimation of high-dimensional quantile regression
- Side-constrained minimum sum-of-squares clustering: mathematical programming and random projections
- Composite quantile regression for massive datasets
- ADMM for Penalized Quantile Regression in Big Data
- Computational issues for quantile regression
- Quantile regression for robust bank efficiency score estimation
- Tight bounds for the subspace sketch problem with applications
- Residual projection for quantile regression in vertically partitioned big data
- Sampling Lasso quantile regression for large-scale data
- Advanced algorithms for penalized quantile and composite quantile regression
- Quantile Regression for Analyzing Heterogeneity in Ultra-High Dimension
- Block average quantile regression for massive dataset
- Penalized Quantile Regression for Distributed Big Data Using the Slack Variable Representation
- Improving linear quantile regression for replicated data
- Vector quantile regression beyond the specified case
- A distributed quantile estimation algorithm of heavy-tailed distribution with massive datasets
- Quantile regression for large-scale data via sparse exponential transform method
This page was built for publication: Quantile regression for large-scale applications
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2940044)
