Efficient and adaptive linear regression in semi-supervised settings

DOI10.1214/17-AOS1594MaRDI QIDQ2413601zbMATH OpenOpenAlexWikidataFDO

Authors Abhishek Chakrabortty, Tianxi Cai

Publication date 14 September 2018

Published in The Annals of Statistics (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1701.04889

zbMATH Keywords

adaptive estimationmodel mis-specificationsemi-nonparametric imputationsemi-supervised linear regressionsemiparametric inference

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Robustness and adaptive procedures (parametric inference) (62F35) Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05)

Abstract: We consider the linear regression problem under semi-supervised settings wherein the available data typically consists of: (i) a small or moderate sized 'labeled' data, and (ii) a much larger sized 'unlabeled' data. Such data arises naturally from settings where the outcome, unlike the covariates, is expensive to obtain, a frequent scenario in modern studies involving large databases like electronic medical records (EMR). Supervised estimators like the ordinary least squares (OLS) estimator utilize only the labeled data. It is often of interest to investigate if and when the unlabeled data can be exploited to improve estimation of the regression parameter in the adopted linear model. In this paper, we propose a class of 'Efficient and Adaptive Semi-Supervised Estimators' (EASE) to improve estimation efficiency. The EASE are two-step estimators adaptive to model mis-specification, leading to improved (optimal in some cases) efficiency under model mis-specification, and equal (optimal) efficiency under a linear model. This adaptive property, often unaddressed in the existing literature, is crucial for advocating 'safe' use of the unlabeled data. The construction of EASE primarily involves a flexible 'semi-non-parametric' imputation, including a smoothing step that works well even when the number of covariates is not small; and a follow up 'refitting' step along with a cross-validation (CV) strategy both of which have useful practical as well as theoretical implications towards addressing two important issues: under-smoothing and over-fitting. We establish asymptotic results including consistency, asymptotic normality and the adaptive properties of EASE. We also provide influence function expansions and a 'double' CV strategy for inference. The results are further validated through extensive simulations, followed by application to an EMR study on auto-immunity.

Recommendations

• Semi-Supervised Linear Regression
• On semi-supervised linear regression in covariate shift problems
• An efficient kernel learning algorithm for semisupervised regression problems
• Semi-supervised partial least squares
• Regularized semi-supervised least squares regression with dependent samples
• On adaptive linear regression
• Improved estimators for semi-supervised high-dimensional regression model
• The performance of semi-supervised Laplacian regularized regression with the least square loss

Cites work

• Comment
• Dimension Reduction in Binary Response Regression
• scientific article; zbMATH DE number 788275 (Why is no real title available?)
• Manifold regularization: a geometric framework for learning from labeled and unlabeled examples
• MULTIVARIATE LOCAL POLYNOMIAL REGRESSION FOR TIME SERIES:UNIFORM STRONG CONSISTENCY AND RATES
• On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma
• Principal Hessian Directions Revisited
• Semi-supervised learning with density-ratio estimation
• Sliced Inverse Regression for Dimension Reduction
• Slicing regression: A link-free regression method
• Text classification from labeled and unlabeled documents using EM
• The relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter
• UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA

Cited in

(43)

• An efficient kernel learning algorithm for semisupervised regression problems
• Subdata selection algorithm for linear model discrimination
• Semi-supervised empirical risk minimization: using unlabeled data to improve prediction
• On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning
• Improved estimation of the extreme value index using related variables
• Semi-supervised inference: general theory and estimation of means
• On semi-supervised linear regression in covariate shift problems
• Optimal Sampling for Generalized Linear Models Under Measurement Constraints
• Semi-supervised validation of multiple surrogate outcomes with application to electronic medical records phenotyping
• Robust and efficient semi‐supervised estimation of average treatment effects with application to electronic health records data
• The Generalized Oaxaca-Blinder Estimator
• Semi-Supervised Linear Regression
• Semi-supervised inference for case-control binary data under possibly mis-specified logistic models
• Autoencoding slow representations for semi-supervised data-efficient regression
• Improved estimation of average treatment effects under covariate-adaptive randomization methods
• A General M-estimation Theory in Semi-Supervised Framework
• Efficient and multiply robust risk estimation under general forms of dataset shift
• Semi-supervised inference for nonparametric logistic regression
• Dimension reduction-based adaptive-to-model semi-supervised classification
• Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
• Semi-supervised inference for the high-dimensional quantile regression
• Improve efficiency of doubly robust estimator when propensity score is misspecified
• Mixed semi-supervised generalized linear regression with applications to deep learning and interpolators
• Semi-supervised U-statistics
• Group inference of high-dimensional single-index models
• Assumption-lean and data-adaptive post-prediction inference
• Semi-supervised linear regression: enhancing efficiency and robustness in high dimensions
• Optimal surrogate-assisted sampling for cost-efficient validation of electronic health record outcomes
• On the role of surrogates in the efficient estimation of treatment effects with limited outcome data
• Sparse estimation and inference for prediction-powered semi-supervised linear regression
• Optimal Decorrelated Score Subsampling for High-Dimensional Generalized Linear Models Under Measurement Constraints
• Efficient semiparametric estimation in two-sample comparison via semisupervised learning
• Semi-Supervised Triply Robust Inductive Transfer Learning
• A general semi-parametric elliptical distribution model for semi-supervised learning
• Doubly Robust Augmented Model Accuracy Transfer Inference with High Dimensional Features
• Sharp-SSL: Selective High-Dimensional Axis-Aligned Random Projections for Semi-Supervised Learning
• Extreme Value Statistics in Semi-Supervised Models
• Smoothed estimation on optimal treatment regime under semisupervised setting in randomized trials
• Semi-supervised learning for various comparison functions across two populations
• Prediction de-correlated inference: a safe approach for post-prediction inference
• Leveraging Unlabeled Data for Superior ROC Curve Estimation via a Semiparametric Approach
• Generative semi-supervised classification
• Semi-supervised sufficient dimension reduction via predictive inference

This page was built for publication: Efficient and adaptive linear regression in semi-supervised settings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2413601)