The Rate-Distortion Risk in Estimation From Compressed Data

DOI10.1109/TIT.2021.3068981MaRDI QIDQ5001589zbMATH OpenOpenAlexFDO

Authors Alon Kipnis, Stefano Rini, Andrea J. Goldsmith

Publication date 22 July 2021

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1602.02201

zbMATH Keywords

datasets undergoing lossy compressionestimation of latent signal

Mathematics Subject Classification ID

Signal theory (characterization, reconstruction, filtering, etc.) (94A12) Source coding (94A29)

Abstract: Consider the problem of estimating a latent signal from a lossy compressed version of the data when the compressor is agnostic to the relation between the signal and the data. This situation arises in a host of modern applications when data is transmitted or stored prior to determining the downstream inference task. Given a bitrate constraint and a distortion measure between the data and its compressed version, let us consider the joint distribution achieving Shannon's rate-distortion (RD) function. Given an estimator and a loss function associated with the downstream inference task, define the rate-distortion risk as the expected loss under the RD-achieving distribution. We provide general conditions under which the operational risk in estimating from the compressed data is asymptotically equivalent to the RD risk. The main theoretical tools to prove this equivalence are transportation-cost inequalities in conjunction with properties of compression codes achieving Shannon's RD function. Whenever such equivalence holds, a recipe for designing estimators from datasets undergoing lossy compression without specifying the actual compression technique emerges: design the estimator to minimize the RD risk. Our conditions simplified in the special cases of discrete memoryless or multivariate normal data. For these scenarios, we derive explicit expressions for the RD risk of several estimators and compare them to the optimal source coding performance associated with full knowledge of the relation between the latent signal and the data.

Recommendations

• Estimation via compressed information
• Threshold Effects in Parameter Estimation From Compressed Data
• Gaussian Approximation of Quantization Error for Estimation From Compressed Data
• Bounds on Data Compression Ratio with a Given Tolerable Error Probability
• scientific article; zbMATH DE number 3932242
• Rate Distortion and Denoising of Individual Data Using Kolmogorov Complexity
• scientific article; zbMATH DE number 3318548
• On the amount of statistical side information required for lossy data compression
• Rate distortion theory for general sources with potential application to image compression

Cited in

(5)

• Compressed and Quantized Correlation Estimators
• Gaussian Approximation of Quantization Error for Estimation From Compressed Data
• scientific article; zbMATH DE number 3318548 (Why is no real title available?)
• On the rate-distortion-perception-semantics tradeoff in low-rate regime for lossy compression
• Common pitfalls using the normalized compression distance: what to watch out for in a compressor

This page was built for publication: The Rate-Distortion Risk in Estimation From Compressed Data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q5001589)