Stochastic expansions using continuous dictionaries: Lévy adaptive regression kernels

DOI10.1214/11-AOS889zbMATH Open1227.62030arXiv1112.3149OpenAlexW1969508365MaRDI QIDQ98918FDOQ98918

Authors: Robert L. Wolpert, Merlise Clyde, Chong Tu, Robert L. Wolpert, Merlise A. Clyde, Chong Tu 

Publication date: 1 August 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This article describes a new class of prior distributions for nonparametric function estimation. The unknown function is modeled as a limit of weighted sums of kernels or generator functions indexed by continuous parameters that control local and global features such as their translation, dilation, modulation and shape. L'{e}vy random fields and their stochastic integrals are employed to induce prior distributions for the unknown functions or, equivalently, for the number of kernels and for the parameters governing their features. Scaling, shape, and other features of the generating functions are location-specific to allow quite different function properties in different parts of the space, as with wavelet bases and other methods employing overcomplete dictionaries. We provide conditions under which the stochastic expansions converge in specified Besov or Sobolev norms. Under a Gaussian error model, this may be viewed as a sparse regression problem, with regularization induced via the L'{e}vy random field prior distribution. Posterior inference for the unknown functions is based on a reversible jump Markov chain Monte Carlo algorithm. We compare the L'{e}vy Adaptive Regression Kernel (LARK) method to wavelet-based methods using some of the standard test functions, and illustrate its flexibility and adaptability in nonstationary applications.

Full work available at URL: https://arxiv.org/abs/1112.3149

Recommendations

• A variational inference for the Lévy adaptive regression with multiple kernels
• Kernel estimation for Lévy driven stochastic convolutions
• Kernel-based regression of drift and diffusion coefficients of stochastic processes
• Iteratively reweighted least square for kernel expectile regression with random features
• The stochastic approximation method for semi-recursive multivariate kernel-type regression estimation
• Conditional stochastic kernel estimation by nonparametric methods
• Kernel regression estimation for random fields
• Kernel continuum regression
• scientific article; zbMATH DE number 751035
• scientific article; zbMATH DE number 7408882

zbMATH Keywords

waveletssplineskernel regressionnonparametric regressionBayesBesovLARKrelevance vector machinereversible jump Markov chain Monte Carlosupport vector machineLévy random field

Mathematics Subject Classification ID

Bayesian inference (62F15) Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Random fields; image analysis (62M40) Monte Carlo methods (65C05) Nontrigonometric harmonic analysis involving wavelets and other special systems (42C40)

Cites Work

• Title not available (Why is that?)
• Ideal spatial adaptation by wavelet shrinkage
• Financial Modelling with Jump Processes
• Bayesian Density Estimation and Inference Using Mixtures
• Title not available (Why is that?)
• Title not available (Why is that?)
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• Interpolation of spatial data. Some theory for kriging
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• Title not available (Why is that?)
• Atomic Decomposition by Basis Pursuit
• Estimating normal means with a conjugate style dirichlet process prior
• Title not available (Why is that?)
• Fractional Ornstein-Uhlenbeck Lévy processes and the telecom process: Upstairs and downstairs
• Spectral representations of infinitely divisible processes
• Nonparametric regression using Bayesian variable selection
• Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems
• Stochastic expansions in an overcomplete wavelet dictionary
• Choosing a kernel regression estimator. With comments and a rejoinder by the authors
• Nonparametric Bayesian data analysis
• Prior distributions on spaces of probability measures
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• On posterior consistency in nonparametric regression problems
• Completely random measures
• A Bayesian analysis of some nonparametric problems
• Empirical Bayes selection of wavelet thresholds
• An introduction to support vector machines and other kernel-based learning methods.
• Bayesian curve-fitting with free-knot splines
• Title not available (Why is that?)
• Title not available (Why is that?)
• Bayesian function estimation using continuous wavelet dictionaries
• Title not available (Why is that?)
• Orthonormal bases of compactly supported wavelets
• Title not available (Why is that?)
• Title not available (Why is that?)
• Ten Lectures on Wavelets
• Title not available (Why is that?)
• Title not available (Why is that?)
• Automatic Bayesian Curve Fitting
• Wavelet Thresholding via A Bayesian Approach
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Estimating Normal Means with a Dirichlet Process Prior
• 10.1162/15324430152748236
• The use of unlabeled data in predictive modeling
• Title not available (Why is that?)
• Title not available (Why is that?)
• Bayesian Estimation of the Spectral Density of a Time Series
• Title not available (Why is that?)
• Bayesian Variable Selection and Regularization for Time–Frequency Surface Estimation
• Title not available (Why is that?)
• Matching pursuits with time-frequency dictionaries
• Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization
• Title not available (Why is that?)
• Transdimensional Markov Chains
• Title not available (Why is that?)
• Bayesian methods for support vector machines: Evidence and predictive class probabilities
• Convergence rates of posterior distributions for non iid observations

Cited In (8)

• Stochastic expansions in an overcomplete wavelet dictionary
• Locally Adaptive Bayes Nonparametric Regression via Nested Gaussian Processes
• Some aspects of symmetric Gamma process mixtures
• Lévy Adaptive B-spline Regression via Overcomplete Systems
• bark
• Bayesian nonparametric models for peak identification in MALDI-TOF mass spectroscopy
• Bayesian curve fitting for discontinuous functions using an overcomplete system with multiple kernels
• A variational inference for the Lévy adaptive regression with multiple kernels

Uses Software

• R
• PDCO
• EbayesThresh
• EBayesThresh

This page was built for publication: Stochastic expansions using continuous dictionaries: Lévy adaptive regression kernels

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q98918)