Efficient Derivative Codes through Automatic Differentiation and Interface Contraction: An Application in Biostatistics
Let \(C(f)\) be the computational cost of the function \(f: \mathbb{R}^n\to\mathbb{R}^p\). The additional cost of computing the derivatives of \(f(x)\) with respect to \(x\) is approximately \(nC(f)\), if they are performed using the forward mode of automatic differentiation (FMAD). In case of \(f= h\circ g\), where \(h:\mathbb{R}^m\to \mathbb{R}^p\), \(g:\mathbb{R}^n\to \mathbb{R}^m\), the total cost of computing \({\partial g\over\partial x}\) and \({\partial h\over\partial g}\) independently, is approximately \(C_1= nC(g)+ mC(h)\) plus the cost \(C_0\) of performing the matrix multiplication \({\partial h\over\partial g}\times{\partial g\over\partial x}\). Usually \(C_0< C_1\). The FMAD reduces the cost of computing derivatives if \(m<n\). If, in addition, \(C(h)\gg C(g)\), this method has a cost of \(mC(f)\), possibly less than \(nC(f)\). The width of the ``interface between \(g\) and \(h\) is the value of \(m\). The interface contraction takes place when \(m<n\). The problem of computing the first- and higher-order derivatives of a function is considered. In this report, the authors demonstrate how the user of an automatic differentiation tool can easily modify the code produced by this tool to take advantage of interface contraction. That is, to reduce the computational cost. This technique depends upon an easily recognized program feature: the width of the interface between program units. Two sections are devoted to describe the results of some experiments. Here it is used the macroscopic version of interface contraction in the development of derivative codes for two log-likelihood functions used for biostatistical analysis.
- On computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation.
- Fitting psychometric models with methods based on automatic differentiation
- Estimation in Dirichlet random effects models
- Sensitivity tools vs. Poincaré sections
- Segmented Regression in the Presence of Covariate Measurement Error in Main Study/Validation Study Designs
- Stochastic volatility: Bayesian computation using automatic differentiation and the extended Kalman filter
- Modeling the performance of interface contraction
- Development of an adjoint system-based hydrofoil optimisation framework using algorithmic differentiation
- Statistical unfolding of elementary particle spectra: empirical Bayes estimation and bias-corrected uncertainty quantification
This page was built for publication: Efficient Derivative Codes through Automatic Differentiation and Interface Contraction: An Application in Biostatistics
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4376221)
