How many iterations are sufficient for efficient semiparametric estimation?

DOI10.1002/SJOS.12005zbMATH Open1364.62077arXiv1009.2111OpenAlexW1601848613MaRDI QIDQ2852630FDOQ2852630

Authors: Guang Cheng 

Publication date: 9 October 2013

Published in: Scandinavian Journal of Statistics (Search for Journal in Brave)

Abstract: A common practice in obtaining a semiparametric efficient estimate is through iteratively maximizing the (penalized) log-likelihood w.r.t. its Euclidean parameter and functional nuisance parameter via Newton-Raphson algorithm. The purpose of this paper is to provide a formula in calculating the minimal number of iterations $k^{a}st$ needed to produce an efficient estimate $hathet{a}_n^{(k^{a}st)}$ from a theoretical point of view. We discover that (a) $k^{a}st$ depends on the convergence rates of the initial estimate and nuisance estimate; (b) more than $k^{a}st$ iterations, i.e., $k$, will only improve the higher order asymptotic efficiency of $hathet{a}_n^{(k)}$; (c) $k^{a}st$ iterations are also sufficient for recovering the estimation sparsity in high dimensional data. These general conclusions hold, in particular, when the nuisance parameter is not estimable at root-n rate, and apply to semiparametric models estimated under various regularizations, e.g., kernel or penalized estimation. This paper provides a first general theoretical justification for the "one-/two-step iteration" phenomena observed in the literature, and may be useful in reducing the bootstrap computational cost for the semiparametric models.

Full work available at URL: https://arxiv.org/abs/1009.2111

Recommendations

• SOME CONVERGENCE THEORY FOR ITERATIVE ESTIMATION PROCEDURES WITH AN APPLICATION TO SEMIPARAMETRIC ESTIMATION
• Iterative estimating equations: Linear convergence and asymptotic properties
• A simple iterative Z-estimator for semiparametric models
• Estimating a real parameter in a class of semiparametric models
• Efficient estimation and computation of parameters and nonparametric functions in generalized semi/non-parametric regression models

zbMATH Keywords

semiparametric modelsNewton-Raphson algorithmhigher order asymptotic efficiencygeneralized profile likelihood\(k\)-step estimation

Mathematics Subject Classification ID

Asymptotic properties of parametric estimators (62F12) Nonparametric estimation (62G05) Asymptotic properties of nonparametric inference (62G20) Functional limit theorems; invariance principles (60F17)

Cites Work

• Least angle regression. (With discussion)
• Introduction to empirical processes and semiparametric inference
• Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation
• Generalized Partially Linear Single-Index Models
• Profile likelihood and conditionally parametric models
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Efficient estimation for the proportional hazards model with interval censoring
• Semiparametric additive isotonic regression
• On adaptive estimation
• Penalized quasi-likelihood estimation in partial linear models
• The Stochastic Difference Between Econometric Statistics
• Local Polynomial Kernel Regression for Generalized Linear Models and Quasi-Likelihood Functions
• Title not available (Why is that?)
• Distribution-Free Maximum Likelihood Estimator of the Binary Choice Model
• Iterative estimating equations: Linear convergence and asymptotic properties
• On Profile Likelihood
• On asymptotically efficient estimation in semiparametric models
• General frequentist properties of the posterior profile distribution
• An elementary estimator of the partial linear model
• A Semiparametric Mixture Approach to Case-Control Studies With Errors in Covariables
• The penalized profile sampler
• Higher order semiparametric frequentist inference with the profile sampler
• Observed information in semi-parametric models
• SOME CONVERGENCE THEORY FOR ITERATIVE ESTIMATION PROCEDURES WITH AN APPLICATION TO SEMIPARAMETRIC ESTIMATION
• The Kernel Estimate of a Regression Function in Likelihood-Based Models
• The Profile Sampler
• The cluster bootstrap consistency in generalized estimating equations

Cited In (3)

• A simple iterative Z-estimator for semiparametric models
• SOME CONVERGENCE THEORY FOR ITERATIVE ESTIMATION PROCEDURES WITH AN APPLICATION TO SEMIPARAMETRIC ESTIMATION
• Fast Generalized Linear Models by Database Sampling and One-Step Polishing

This page was built for publication: How many iterations are sufficient for efficient semiparametric estimation?

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2852630)