Lattice quantization error for redundant representations (Q734330)

Redundant systems such as frames are often used to represent a signal for error correction, denoising and general robustness. In the digital domain quantization needs to be performed. Given the redundancy, the distribution of quantization errors can be rather complex. This paper studies the stronger form of the asymptotic White Noise Hypothesis for general lattice by characterizing the asymptotic distribution of the quantization error in the most general setting. Let \(L\) be a full rank lattice in \(\mathbb{R}^{N}\) and \(G\) be a \(d\times N\) matrix. Let \(X\) be an absolutely continuous random vector in \(\mathbb{R}^{d}\) (the signal). For the cell size of the lattice \(\Delta >0\), let \(Z_{\Delta}=\frac{1}{\Delta}\tau (G^{T}X,\Delta L)\) be the normalized quantization error. The distribution of \(Z_{\Delta}\) as \(\Delta\) becomes small is characterized by the following property: If the rows of the matrix \(A^{-1}G^{T}\) are linearly independent over the rationals, then \(Z_{\Delta}\) is asymptotically uniformly distributed in the Voronoi cell \(V(\mathbf \{0\})\) of the lattice \(L\) as \(\Delta \rightarrow 0\). The case when the maximal number of linearly independent rows of this matrix is equal to \(m\) is also considered with respect to the \(m\)-dimensional Hausdorff measure.