Adaptive random Fourier features with Metropolis sampling

arXiv2007.10683MaRDI QIDQ6345533FDOQ6345533

Authors: Aku Kammonen, Jonas Kiessling, Petr Plecháč, Mattias Sandberg, Anders Szepessy 

Publication date: 21 July 2020

Abstract: The supervised learning problem to determine a neural network approximation $$ with one hidden layer is studied as a random Fourier features algorithm. The Fourier features, i.e., the frequencies $omeg{a}_{k}inmathbb{R}^d$, are sampled using an adaptive Metropolis sampler. The Metropolis test accepts proposal frequencies $omeg{a_k}^{\prime }$, having corresponding amplitudes $$, with the probability $$, for a certain positive parameter $gamma$, determined by minimizing the approximation error for given computational work. This adaptive, non-parametric stochastic method leads asymptotically, as $Koinfty$, to equidistributed amplitudes $$, analogous to deterministic adaptive algorithms for differential equations. The equidistributed amplitudes are shown to asymptotically correspond to the optimal density for independent samples in random Fourier features methods. Numerical evidence is provided in order to demonstrate the approximation properties and efficiency of the proposed algorithm. The algorithm is tested both on synthetic data and a real-world high-dimensional benchmark.

Mathematics Subject Classification ID

Monte Carlo methods (65C05) Algorithms for approximation of functions (65D15) Numerical approximation of high-dimensional functions; sparse grids (65D40)

This page was built for publication: Adaptive random Fourier features with Metropolis sampling

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6345533)