On the sample information about parameter and prediction

DOI10.1214/10-STS329zbMATH Open1329.62046arXiv1101.0899OpenAlexW1979937369MaRDI QIDQ906533FDOQ906533

Authors: Nader Ebrahimi, Ehsan S. Soofi, Refik Soyer 

Publication date: 22 January 2016

Published in: Statistical Science (Search for Journal in Brave)

Abstract: The Bayesian measure of sample information about the parameter, known as Lindley's measure, is widely used in various problems such as developing prior distributions, models for the likelihood functions and optimal designs. The predictive information is defined similarly and used for model selection and optimal designs, though to a lesser extent. The parameter and predictive information measures are proper utility functions and have been also used in combination. Yet the relationship between the two measures and the effects of conditional dependence between the observable quantities on the Bayesian information measures remain unexplored. We address both issues. The relationship between the two information measures is explored through the information provided by the sample about the parameter and prediction jointly. The role of dependence is explored along with the interplay between the information measures, prior and sampling design. For the conditionally independent sequence of observable quantities, decompositions of the joint information characterize Lindley's measure as the sample information about the parameter and prediction jointly and the predictive information as part of it. For the conditionally dependent case, the joint information about parameter and prediction exceeds Lindley's measure by an amount due to the dependence. More specific results are shown for the normal linear models and a broad subfamily of the exponential family. Conditionally independent samples provide relatively little information for prediction, and the gap between the parameter and predictive information measures grows rapidly with the sample size.

Full work available at URL: https://arxiv.org/abs/1101.0899

Recommendations

• scientific article; zbMATH DE number 4188970
• An extension of bayesian measure of information to regression
• On the association between a random parameter and an observable
• Publication:3492634
• Measures of dependence in normal models and exponential models by information gain

zbMATH Keywords

entropyserial correlationoptimal designmutual informationreference priororder statisticsintraclass correlationBayesian predictive distribution

Mathematics Subject Classification ID

Statistical aspects of information-theoretic topics (62B10) Bayesian inference (62F15) Linear regression; mixed models (62J05) Sampling theory, sample surveys (62D05) Optimal statistical designs (62K05) Order statistics; empirical distribution functions (62G30)

Cites Work

• Title not available (Why is that?)
• Bayesian experimental design: A review
• Title not available (Why is that?)
• A Mathematical Theory of Communication
• Title not available (Why is that?)
• Expected information as ecpected utility
• Estimating a Product of Means: Bayesian Analysis with Reference Priors
• Title not available (Why is that?)
• Title not available (Why is that?)
• Maximum likelihood prediction
• On a Measure of the Information Provided by an Experiment
• BINOMIAL SAMPLING SCHEMES AND THE CONCEPT OF INFORMATION
• Information Properties of Order Statistics and Spacings
• Title not available (Why is that?)
• A Prediction Problem concerning Samples from the Exponential Distribution, with Application in Life Testing
• Diagnostic Measures for Model Criticism
• Goodness of prediction fit
• An information criterion for likelihood selection
• Bayesian Hypothesis Testing: A Reference Approach
• Title not available (Why is that?)
• Title not available (Why is that?)
• Admissible predictive density estimation
• Relative information loss under Type II censored exponential data
• Title not available (Why is that?)
• Title not available (Why is that?)
• Principal component regression under exchangeability
• Effects of collinearity on information about regression coefficients
• Collinearity and least squares regression. With discussion
• Prediction intervals for future failures in the exponential distribution under hybrid censoring
• A note on Bayes designs for inference using a hierarchical linear model
• Application of a Measure of Information to the Design and Comparison of Regression Experiments
• Title not available (Why is that?)
• Prediction variance and information worth of observations in time series
• Goodness of Prediction Fit for Multivariate Linear Models
• On the relative efficiency of grouped and censored survival data
• Title not available (Why is that?)
• Title not available (Why is that?)
• Title not available (Why is that?)
• Information and Sampling from the Exponential Distribution

Cited In (10)

• I Got More Data, My Model is More Refined, but My Estimator is Getting Worse! Am I Just Dumb?
• Statistical Problem Classes and Their Links to Information Theory
• Comparison, utility, and partition of dependence under absolutely continuous and singular distributions
• Bayesian testing of a point null hypothesis based on the latent information prior
• Component importance based on dependence measures
• Bayesian predictive densities based on latent information priors
• Normality of Posterior Distribution Under Misspecification and Nonsmoothness, and Bayes Factor for Davies' Problem
• A past inaccuracy measure based on the reversed relevation transform
• On the effect of dependency in information properties of series and parallel systems
• Title not available (Why is that?)

This page was built for publication: On the sample information about parameter and prediction

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q906533)