Estimating multi-index models with response-conditional least squares (Q2044313)

In the general area of sufficient dimension reduction, a novel methodology is derived and analyzed for estimation of the index space \(A\) of a high dimensional multi-index regression model \(E(Y|X) = g(A^T X)\) with \(g\) being a link function. The estimation method is based on the solution of localized least squares problems and uses the span of linear regression slope coefficients computed over level sets of the data. The propagation error of the index space estimate in the regression of the link function is studied in detail. The proposed method is shown to be attractive to practitioners by being computationally efficient and easy to implement as only one hyperparameter (the number of level sets) needs to be specified. Furthermore, finite sample generalization bounds for the regression model are provided which account for the projection error; in turn the bounds are used in quantifying the generalization bound for derived regression estimate and obtain that when response-conditional least squares are paired with piecewise polynomial regression, this leads to optimal estimation of the multi-index model.