Real-Time Regression Analysis of Streaming Clustered Data With Possible Abnormal Data Batches
From MaRDI portal
Publication:6077590
Abstract: This paper develops an incremental learning algorithm based on quadratic inference function (QIF) to analyze streaming datasets with correlated outcomes such as longitudinal data and clustered data. We propose a renewable QIF (RenewQIF) method within a paradigm of renewable estimation and incremental inference, in which parameter estimates are recursively renewed with current data and summary statistics of historical data, but with no use of any historical subject-level raw data. We compare our renewable estimation method with both offline QIF and offline generalized estimating equations (GEE) approach that process the entire cumulative subject-level data, and show theoretically and numerically that our renewable procedure enjoys statistical and computational efficiency. We also propose an approach to diagnose the homogeneity assumption of regression coefficients via a sequential goodness-of-fit test as a screening procedure on occurrences of abnormal data batches. We implement the proposed methodology by expanding existing Spark's Lambda architecture for the operation of statistical inference and data quality diagnosis. We illustrate the proposed methodology by extensive simulation studies and an analysis of streaming car crash datasets from the National Automotive Sampling System-Crashworthiness Data System (NASS CDS).
Cites work
- A nonparametric exponentially weighted moving average control scheme
- A note on improving quadratic inference functions using a linear shrinkage approach
- A Stochastic Approximation Method
- Assessing robustness of generalised estimating equations and quadratic inference functions
- Asymptotic and finite-sample properties of estimators based on stochastic gradients
- CONTINUOUS INSPECTION SCHEMES
- Correlated data analysis: modeling, analytics, and applications
- Determination of A. R. L. and a Contour Nomogram for Cusum Charts to Control Normal Mean
- Efficient estimation for longitudinal data by combining large-dimensional moment conditions
- Efficient recursive estimation. Application to estimating the parameters of a covariance function
- scientific article; zbMATH DE number 125629 (Why is no real title available?)
- scientific article; zbMATH DE number 870530 (Why is no real title available?)
- scientific article; zbMATH DE number 5060482 (Why is no real title available?)
- Improving generalised estimating equations using quadratic inference functions
- Inference functions and quadratic score tests
- Large Sample Properties of Generalized Method of Moments Estimators
- Likelihood Ratio Identities and Their Applications to Sequential Analysis
- Logarithmic regret algorithms for online convex optimization
- Longitudinal data analysis using generalized linear models
- Models for Longitudinal Data: A Generalized Estimating Equation Approach
- Nonparametric quality control charts based on the sign statistic
- On Optimum Methods in Quickest Detection Problems
- Renewable estimation and incremental inference in generalized linear models with streaming data sets
- Scalable statistical inference for averaged implicit stochastic gradient descent
- Sequential detection of a change in a normal mean when the initial value is unknown
- SGD-QN: careful quasi-Newton stochastic gradient descent
Cited in
(15)- A discussion on “A selective review of statistical methods using calibration information from similar studies”
- Renewable Huber estimation method for streaming datasets
- The effect of the working correlation on fitting models to longitudinal data
- Renewable risk assessment of heterogeneous streaming time-to-event cohorts
- Collaborative inference for treatment effect with distributed data-sharing management in multicenter studies
- Non-crossing quantile double-autoregression for the analysis of streaming time series data
- Online sequential leveraging sampling method for streaming autoregressive time series with application to seismic data
- Collaborative inference for accelerated failure time model using clinical center-level summary statistics
- An online updating approach for estimating and testing mediation effects with big data streams
- Online updating mode learning for streaming datasets
- Estimation of two-layer Gaussian mixture model for streaming longitudinal data in Bayesian framework
- Inference for high-dimensional streamed longitudinal data
- Real-time inference for smoothing quantile regression on streaming datasets with heterogeneity detection
- Unconditional Quantile Regression for Streaming Datasets
- Comments on: ``Data integration via analysis of subspaces (DIVAS)
This page was built for publication: Real-Time Regression Analysis of Streaming Clustered Data With Possible Abnormal Data Batches
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6077590)
