Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators With Massive Data
From MaRDI portal
Abstract: Nonuniform subsampling methods are effective to reduce computational burden and maintain estimation efficiency for massive data. Existing methods mostly focus on subsampling with replacement due to its high computational efficiency. If the data volume is so large that nonuniform subsampling probabilities cannot be calculated all at once, then subsampling with replacement is infeasible to implement. This paper solves this problem using Poisson subsampling. We first derive optimal Poisson subsampling probabilities in the context of quasi-likelihood estimation under the A- and L-optimality criteria. For a practically implementable algorithm with approximated optimal subsampling probabilities, we establish the consistency and asymptotic normality of the resultant estimators. To deal with the situation that the full data are stored in different blocks or at multiple locations, we develop a distributed subsampling framework, in which statistics are computed simultaneously on smaller partitions of the full data. Asymptotic properties of the resultant aggregated estimator are investigated. We illustrate and evaluate the proposed strategies through numerical experiments on simulated and real data sets.
Recommendations
- Optimal subsampling for large-scale quantile regression
- Optimal subsampling algorithms for big data regressions
- Optimal subsampling for large sample logistic regression
- Optimal subsample selection for massive logistic regression with distributed data
- Optimal subsampling for quantile regression in big data
Cites work
- A note on the uniqueness of the quasi-likelihood estimator
- A Scalable Bootstrap for Massive Data
- A statistical perspective on algorithmic leveraging
- Aggregated estimating equation estimation
- Asymptotic Statistics
- Communication-efficient distributed statistical inference
- Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
- Empirical likelihood confidence intervals for complex sampling designs
- Faster least squares approximation
- scientific article; zbMATH DE number 47310 (Why is no real title available?)
- Information-Based Optimal Subdata Selection for Big Data Linear Regression
- Local polynomial regresssion estimators in survey sampling.
- Model assisted survey sampling
- Multivariate statistical modelling based on generalized linear models.
- Optimal Design of Experiments
- Optimal subsampling for large sample logistic regression
- Speeding Up MCMC by Efficient Data Subsampling
- Strong consistency of maximum quasi-likelihood estimators in generalized linear models with fixed and adaptive designs
Cited in
(92)- Efficient distribution estimation for data with unobserved sub-population identifiers
- Optimal subsampling for large-scale quantile regression
- Optimal subsampling for least absolute relative error estimators with massive data
- Model-free global likelihood subsampling for massive data
- Optimal subsample selection for massive logistic regression with distributed data
- A two-stage optimal subsampling estimation for missing data problems with large-scale data
- Optimal subsampling algorithms for big data regressions
- Smoothing Splines Approximation Using Hilbert Curve Basis Selection
- LowCon: A Design-based Subsampling Approach in a Misspecified Linear Model
- Fast Calibration for Computer Models with Massive Physical Observations
- Optimal subsampling for multiplicative regression with massive data
- Subsampling spectral clustering for stochastic block models in large-scale networks
- Subsampling and Jackknifing: A Practically Convenient Solution for Large Data Analysis With Limited Computational Resources
- Subdata selection based on orthogonal array for big data
- Generalized linear models for massive data via doubly-sketching
- Optimal subsampling algorithms for composite quantile regression in massive data
- Optimal sampling designs for multidimensional streaming time series with application to power grid sensor data
- Efficient Approximation of Gromov-Wasserstein Distance Using Importance Sparsification
- Optimal decorrelated score subsampling for generalized linear models with massive data
- Subsampling in longitudinal models
- Analyzing Big EHR Data—Optimal Cox Regression Subsampling Procedure with Rare Events
- Optimal subsampling for functional quantile regression
- Unweighted estimation based on optimal sample under measurement constraints
- Subsampling approach for least squares fitting of semi-parametric accelerated failure time models to massive survival data
- Optimal subsampling for modal regression in massive data
- Emerging directions in Bayesian computation
- A note on centering in subsample selection for linear regression
- A review on design inspired subsampling for big data
- Approximating Partial Likelihood Estimators via Optimal Subsampling
- Robust optimal subsampling based on weighted asymmetric least squares
- Robust and efficient subsampling algorithms for massive data logistic regression
- Poisson subsampling-based estimation for growing-dimensional expectile regression in massive data
- Distributed subsampling for multiplicative regression
- The COR criterion for optimal subset selection in distributed estimation
- A selective review on statistical methods for massive data computation: distributed computing, subsampling, and minibatch techniques
- Distributed hypothesis testing for large dimensional two-sample mean vectors
- Core-elements for large-scale least squares estimation
- Optimal Poisson subsampling decorrelated score for high-dimensional generalized linear models
- D-optimal subsampling design for multiple linear regression on massive data
- Communication-efficient distributed composite quantile regression via convolution smoothing and Poisson subsampling
- Refitted cross-validation estimation for high-dimensional subsamples from low-dimension full data
- Big data subsampling: a review
- Optimal distributed subsampling for high-dimensional linear measurement error models via doubly bias-corrected score
- Optimal distributed subsampling for accelerated failure time models with massive censored data
- LPRE estimation for functional multiplicative model and optimal subsampling
- D-criterion based optimal subsampling in Poisson regression with one covariate
- Optimal subsampling algorithm for composite quantile regression with distributed data
- A synthetic subsampling and estimation procedure for imbalanced big data
- Optimal Subsampling for Data Streams with Measurement Constrained Categorical Responses
- Optimal subsampling for principal component analysis
- Subsampled one-step estimation for fast statistical inference
- Fast and efficient causal inference in large-scale data via subsampling and projection calibration
- Mastering rare event analysis: subsample-size determination in Cox and logistic regressions
- DsubCox: a fast subsampling algorithm for Cox model with distributed and massive survival data
- Multi-resolution subsampling for linear classification with massive data
- CluBear: a subsampling package for interactive statistical analysis with massive data on a single machine
- Distributed subsampling and quasi decorrelated score for cluster data: an application to Beijing multisite air quality
- Optimal subsampling for multinomial logistic models with big data
- Leverage classifier: another look at support vector machine
- Optimal distributed Poisson subsampling for modal regression with massive data
- A Subsampling Strategy for AIC-based Model Averaging with Generalized Linear Models
- Active Sampling: A Machine-Learning-Assisted Framework for Finite Population Inference with Optimal Subsamples
- Distributed estimation and algorithm for distributed outcome dependent subsampling in generalized linear regression with large-scale data
- Analyzing the dissemination of news by model averaging and subsampling
- Subsampling for big data linear models with measurement errors
- Optimal response-free Poisson subsampling for generalized linear models in increasing dimension
- Nonparametric Additive Models for Billion Observations
- Renewable Quantile Regression with Heterogeneous Streaming Datasets
- Group-Orthogonal Subsampling for Hierarchical Data Based on Linear Mixed Models
- Supervised Stratified Subsampling for Predictive Analytics
- Subsampling from features in large regression to find ``winning features
- Efficient distributed estimation for expectile regression in increasing dimensions
- Outcome dependent subsampling divide and conquer in generalized linear models for massive data
- Sampling-based methods for multi-block optimization problems over transport polytopes
- Power enhancing probability subsampling using side information
- Optimal distributed subsampling under heterogeneity
- Optimal subsampling for generalized additive models on large-scale datasets
- Random perturbation subsampling for rank regression with massive data
- Subsampling-based modified Bayesian information criterion for large-scale stochastic block models
- Efficient Approximation of Leverage Scores in Two-Dimensional Autoregressive Models with Application to Image Anomaly Detection
- Communication-Efficient Pilot Estimation for Non-Randomly Distributed Data in Diverging Dimensions
- Efficient subsampling for exponential family models
- Cor: an R package for optimal subset selection in distributed estimation
- Double-robust and heterogeneity-aware distributed Poisson subsampling for misspecified measurement error models
- Optimal subsampling algorithm for the marginal model with massive longitudinal data
- Poisson subsampling for large-scale functional data analysis
- Optimal Poisson subsampling for multiplicative regressions with massive data
- Nearly optimal two-step Poisson sampling and empirical likelihood weighting estimation for M-estimation with big data
- Distributed information-based optimal sub-data selection algorithm for big data logistic regression
- Optimal subsampling for functional composite quantile regression in massive data
- An adaptive-to-model distributed hybrid test for conditional independence
- Conformal prediction for distributed data
This page was built for publication: Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators With Massive Data
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5881082)
