Efficient and adaptive linear regression in semi-supervised settings

DOI10.1214/17-AOS1594zbMATH Open1403.62041arXiv1701.04889OpenAlexW2581201870WikidataQ129619236 ScholiaQ129619236MaRDI QIDQ2413601FDOQ2413601

Abhishek Chakrabortty, Tianxi Cai

Publication date: 14 September 2018

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: We consider the linear regression problem under semi-supervised settings wherein the available data typically consists of: (i) a small or moderate sized 'labeled' data, and (ii) a much larger sized 'unlabeled' data. Such data arises naturally from settings where the outcome, unlike the covariates, is expensive to obtain, a frequent scenario in modern studies involving large databases like electronic medical records (EMR). Supervised estimators like the ordinary least squares (OLS) estimator utilize only the labeled data. It is often of interest to investigate if and when the unlabeled data can be exploited to improve estimation of the regression parameter in the adopted linear model. In this paper, we propose a class of 'Efficient and Adaptive Semi-Supervised Estimators' (EASE) to improve estimation efficiency. The EASE are two-step estimators adaptive to model mis-specification, leading to improved (optimal in some cases) efficiency under model mis-specification, and equal (optimal) efficiency under a linear model. This adaptive property, often unaddressed in the existing literature, is crucial for advocating 'safe' use of the unlabeled data. The construction of EASE primarily involves a flexible 'semi-non-parametric' imputation, including a smoothing step that works well even when the number of covariates is not small; and a follow up 'refitting' step along with a cross-validation (CV) strategy both of which have useful practical as well as theoretical implications towards addressing two important issues: under-smoothing and over-fitting. We establish asymptotic results including consistency, asymptotic normality and the adaptive properties of EASE. We also provide influence function expansions and a 'double' CV strategy for inference. The results are further validated through extensive simulations, followed by application to an EMR study on auto-immunity.

Full work available at URL: https://arxiv.org/abs/1701.04889

zbMATH Keywords

semiparametric inferenceadaptive estimationmodel mis-specificationsemi-nonparametric imputationsemi-supervised linear regression

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Nonparametric regression and quantile regression (62G08) Linear regression; mixed models (62J05) Robustness and adaptive procedures (parametric inference) (62F35)

Cites Work

• Sliced Inverse Regression for Dimension Reduction
• Comment
• Slicing regression: A link-free regression method
• UNIFORM CONVERGENCE RATES FOR KERNEL ESTIMATION WITH DEPENDENT DATA
• Text classification from labeled and unlabeled documents using EM
• MULTIVARIATE LOCAL POLYNOMIAL REGRESSION FOR TIME SERIES:UNIFORM STRONG CONSISTENCY AND RATES
• On Principal Hessian Directions for Data Visualization and Dimension Reduction: Another Application of Stein's Lemma
• Principal Hessian Directions Revisited
• Title not available (Why is that?)
• Semi-supervised learning with density-ratio estimation
• The relative value of labeled and unlabeled samples in pattern recognition with an unknown mixing parameter
• Dimension Reduction in Binary Response Regression
• Title not available (Why is that?)

Cited In (18)

• Improved estimation of average treatment effects under covariate-adaptive randomization methods
• On nearly assumption-free tests of nominal confidence interval coverage for causal parameters estimated by machine learning
• A General M-estimation Theory in Semi-Supervised Framework
• Subdata selection algorithm for linear model discrimination
• An efficient kernel learning algorithm for semisupervised regression problems
• Semi-Supervised Linear Regression
• Robust and efficient semi‐supervised estimation of average treatment effects with application to electronic health records data
• Semi-supervised empirical risk minimization: using unlabeled data to improve prediction
• The Generalized Oaxaca-Blinder Estimator
• Efficient and multiply robust risk estimation under general forms of dataset shift
• Semi-supervised inference for nonparametric logistic regression
• Dimension reduction-based adaptive-to-model semi-supervised classification
• Optimal Sampling for Generalized Linear Models Under Measurement Constraints
• Improved estimation of the extreme value index using related variables
• Optimal and Safe Estimation for High-Dimensional Semi-Supervised Learning
• Semi-supervised inference: general theory and estimation of means
• Title not available (Why is that?)
• Autoencoding slow representations for semi-supervised data-efficient regression

   Recommendations

• Semi-Supervised Linear Regression 👍 👎
• On semi-supervised linear regression in covariate shift problems 👍 👎
• An efficient kernel learning algorithm for semisupervised regression problems 👍 👎
• Semi-supervised partial least squares 👍 👎
• Regularized semi-supervised least squares regression with dependent samples 👍 👎
• On adaptive linear regression 👍 👎
• Improved estimators for semi-supervised high-dimensional regression model 👍 👎
• The performance of semi-supervised Laplacian regularized regression with the least square loss 👍 👎

This page was built for publication: Efficient and adaptive linear regression in semi-supervised settings

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2413601)