Abstract: The study of complex systems is limited by the fact that only few variables are accessible for modeling and sampling, which are not necessarily the most relevant ones to explain the systems behavior. In addition, empirical data typically under sample the space of possible states. We study a generic framework where a complex system is seen as a system of many interacting degrees of freedom, which are known only in part, that optimize a given function. We show that the underlying distribution with respect to the known variables has the Boltzmann form, with a temperature that depends on the number of unknown variables. In particular, when the unknown part of the objective function decays faster than exponential, the temperature decreases as the number of variables increases. We show in the representative case of the Gaussian distribution, that models are predictable only when the number of relevant variables is less than a critical threshold. As a further consequence, we show that the information that a sample contains on the behavior of the system is quantified by the entropy of the frequency with which different states occur. This allows us to characterize the properties of maximally informative samples: in the under-sampling regime, the most informative frequency size distributions have power law behavior and Zipf's law emerges at the crossover between the under sampled regime and the regime where the sample contains enough statistics to make inference on the behavior of the system. These ideas are illustrated in some applications, showing that they can be used to identify relevant variables or to select most informative representations of data, e.g. in data clustering.
Recommendations
Cites work
- Algorithms of maximum likelihood data clustering with applications
- Are biological systems poised at criticality?
- Counting probability distributions: Differential geometry and model selection
- ENTROPIC ANALYSIS OF THE ROLE OF WORDS IN LITERARY TEXTS
- Financial applications of random matrix theory: old laces and new pieces
- scientific article; zbMATH DE number 107482 (Why is no real title available?)
- scientific article; zbMATH DE number 3592801 (Why is no real title available?)
- scientific article; zbMATH DE number 1059167 (Why is no real title available?)
- Information Theory and Statistical Mechanics
- Information, Physics, and Computation
- Power-law distributions in empirical data
- The unreasonable effectiveness of mathematics in the natural sciences. Richard courant lecture in mathematical sciences delivered at New York University, May 11, 1959
- Zipf's law unzipped
Cited in
(12)- Multiscale relevance and informative encoding in neuronal spike trains
- Criticality of mostly informative samples: a Bayesian model selection approach
- Cooperation, competition and the emergence of criticality in communities of adaptive systems
- Resolution and relevance trade-offs in deep learning
- Statistical criticality arises in most informative representations
- Sampling Strategies for Data-Driven Inference of Input–Output System Properties
- Maximal relevance and optimal learning machines
- ``Universal fitting function for complex systems: case of the short samplings
- A random energy approach to deep learning
- The peculiar statistical mechanics of optimal learning machines
- A simple probabilistic neural network for machine understanding
- Absolute abstraction: a renormalisation group approach
This page was built for publication: On sampling and modeling complex systems
Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q3301678)
