Stochastic expansions using continuous dictionaries: Lévy adaptive regression kernels

DOI10.1214/11-AOS889zbMATH Open1227.62030OpenAlexW1969508365MaRDI QIDQ98918FDOQ98918

Authors: Robert L. Wolpert, Merlise Clyde, Chong Tu, Robert L. Wolpert, Merlise A. Clyde, Chong Tu 

Publication date: 1 August 2011

Published in: The Annals of Statistics (Search for Journal in Brave)

Abstract: This article describes a new class of prior distributions for nonparametric function estimation. The unknown function is modeled as a limit of weighted sums of kernels or generator functions indexed by continuous parameters that control local and global features such as their translation, dilation, modulation and shape. L'{e}vy random fields and their stochastic integrals are employed to induce prior distributions for the unknown functions or, equivalently, for the number of kernels and for the parameters governing their features. Scaling, shape, and other features of the generating functions are location-specific to allow quite different function properties in different parts of the space, as with wavelet bases and other methods employing overcomplete dictionaries. We provide conditions under which the stochastic expansions converge in specified Besov or Sobolev norms. Under a Gaussian error model, this may be viewed as a sparse regression problem, with regularization induced via the L'{e}vy random field prior distribution. Posterior inference for the unknown functions is based on a reversible jump Markov chain Monte Carlo algorithm. We compare the L'{e}vy Adaptive Regression Kernel (LARK) method to wavelet-based methods using some of the standard test functions, and illustrate its flexibility and adaptability in nonstationary applications.

Full work available at URL: https://arxiv.org/abs/1112.3149

Recommendations

• A variational inference for the Lévy adaptive regression with multiple kernels
• Kernel estimation for Lévy driven stochastic convolutions
• Kernel-based regression of drift and diffusion coefficients of stochastic processes
• Iteratively reweighted least square for kernel expectile regression with random features
• The stochastic approximation method for semi-recursive multivariate kernel-type regression estimation
• Conditional stochastic kernel estimation by nonparametric methods
• Kernel regression estimation for random fields
• Kernel continuum regression
• scientific article; zbMATH DE number 751035
• scientific article; zbMATH DE number 7408882

zbMATH Keywords

waveletssplineskernel regressionnonparametric regressionBayesBesovLARKrelevance vector machinereversible jump Markov chain Monte Carlosupport vector machineLévy random field

Mathematics Subject Classification ID

Bayesian inference (62F15) Density estimation (62G07) Nonparametric regression and quantile regression (62G08) Random fields; image analysis (62M40) Monte Carlo methods (65C05) Nontrigonometric harmonic analysis involving wavelets and other special systems (42C40)

Cites Work

• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• 10.1162/15324430152748236
• A Bayesian analysis of some nonparametric problems
• An introduction to support vector machines and other kernel-based learning methods.
• Atomic Decomposition by Basis Pursuit
• Automatic Bayesian Curve Fitting
• Bayesian Density Estimation and Inference Using Mixtures
• Bayesian Estimation of the Spectral Density of a Time Series
• Bayesian Variable Selection and Regularization for Time–Frequency Surface Estimation
• Bayesian curve-fitting with free-knot splines
• Bayesian function estimation using continuous wavelet dictionaries
• Bayesian methods for support vector machines: Evidence and predictive class probabilities
• Characterizing the function space for Bayesian kernel models
• Choosing a kernel regression estimator. With comments and a rejoinder by the authors
• Completely random measures
• Convergence rates of posterior distributions for non iid observations
• Empirical Bayes selection of wavelet thresholds
• Estimating Normal Means with a Dirichlet Process Prior
• Estimating normal means with a conjugate style dirichlet process prior
• Financial Modelling with Jump Processes
• Fractional Ornstein-Uhlenbeck Lévy processes and the telecom process: Upstairs and downstairs
• Ideal spatial adaptation by wavelet shrinkage
• Interpolation of spatial data. Some theory for kriging
• Matching pursuits with time-frequency dictionaries
• Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems
• Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences
• Nonparametric Bayesian data analysis
• Nonparametric regression using Bayesian variable selection
• On posterior consistency in nonparametric regression problems
• Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ ¹ minimization
• Orthonormal bases of compactly supported wavelets
• Prior distributions on spaces of probability measures
• Reversible jump Markov chain Monte Carlo computation and Bayesian model determination
• Spectral representations of infinitely divisible processes
• Stochastic expansions in an overcomplete wavelet dictionary
• Ten Lectures on Wavelets
• The Dantzig selector: statistical estimation when \(p\) is much larger than \(n\). (With discussions and rejoinder).
• The use of unlabeled data in predictive modeling
• Transdimensional Markov Chains
• Wavelet Thresholding via A Bayesian Approach

Cited In (8)

• Stochastic expansions in an overcomplete wavelet dictionary
• Some aspects of symmetric Gamma process mixtures
• Locally adaptive Bayes nonparametric regression via nested Gaussian processes
• Lévy Adaptive B-spline Regression via Overcomplete Systems
• bark
• Bayesian nonparametric models for peak identification in MALDI-TOF mass spectroscopy
• Bayesian curve fitting for discontinuous functions using an overcomplete system with multiple kernels
• A variational inference for the Lévy adaptive regression with multiple kernels

Uses Software

• R
• PDCO
• EbayesThresh
• EBayesThresh

This page was built for publication: Stochastic expansions using continuous dictionaries: Lévy adaptive regression kernels

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q98918)