Achievable Rates for Pattern Recognition

DOI10.1109/TIT.2007.911296MaRDI QIDQ3604593zbMATH OpenOpenAlexWikidataFDO

Authors M. Brandon Westover, Joseph A. O'Sullivan

Publication date 24 February 2009

Published in IEEE Transactions on Information Theory (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/cs/0509022

zbMATH Keywords

pattern recognitionmultiterminal information theoryDistributed source coding

Mathematics Subject Classification ID

Pattern recognition, speech recognition (68T10) Problem solving in the context of artificial intelligence (heuristics, search strategies, etc.) (68T20) Source coding (94A29)

Abstract: Biological and machine pattern recognition systems face a common challenge: Given sensory data about an unknown object, classify the object by comparing the sensory data with a library of internal representations stored in memory. In many cases of interest, the number of patterns to be discriminated and the richness of the raw data force recognition systems to internally represent memory and sensory information in a compressed format. However, these representations must preserve enough information to accommodate the variability and complexity of the environment, or else recognition will be unreliable. Thus, there is an intrinsic tradeoff between the amount of resources devoted to data representation and the complexity of the environment in which a recognition system may reliably operate. In this paper we describe a general mathematical model for pattern recognition systems subject to resource constraints, and show how the aforementioned resource-complexity tradeoff can be characterized in terms of three rates related to number of bits available for representing memory and sensory data, and the number of patterns populating a given statistical environment. We prove single-letter information theoretic bounds governing the achievable rates, and illustrate the theory by analyzing the elementary cases where the pattern data is either binary or Gaussian.

Recommendations

• Generalized error bounds in pattern recognition
• On the role of pattern matching in information theory
• On limits to perception and pattern recognition
• Algorithmic Learning Theory
• Information approach to the problems of pattern recognition

Cited in

(2)

• LEARNING RATES FOR DENSITY LEVEL DETECTION
• SIMBAD: emergence of pattern similarity

This page was built for publication: Achievable Rates for Pattern Recognition

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3604593)