Composable models for online Bayesian analysis of streaming data

DOI10.1007/S11222-017-9783-1MaRDI QIDQ2329734zbMATH OpenOpenAlexWikidataFDO

Authors Jonathan Law, Darren J. Wilkinson

Publication date 18 October 2019

Published in Statistics and Computing (Search for Journal in Brave)

Copyright license Creative Commons Attribution 4.0 International

Full work available at URL https://arxiv.org/abs/1609.00635

zbMATH Keywords

Bayesian analysisdynamic modelsMCMConline analysisparticle filtering

Mathematics Subject Classification ID

Bayesian inference (62F15) Statistical aspects of big data and data science (62R07) Numerical analysis or methods applied to Markov chains (65C40)

Abstract: Data is rapidly increasing in volume and velocity and the Internet of Things (IoT) is one important source of this data. The IoT is a collection of connected devices (things) which are constantly recording data from their surroundings using on-board sensors. These devices can record and stream data to the cloud at a very high rate, leading to high storage and analysis costs. In order to ameliorate these costs, we can analyse the data as it arrives in a stream to learn about the underlying process, perform interpolation and smoothing and make forecasts and predictions. Conventional tools of state space modelling assume data on a fixed regular time grid. However, many sensors change their sampling frequency, sometimes adaptively, or get interrupted and re-started out of sync with the previous sampling grid, or just generate event data at irregular times. It is therefore desirable to model the system as a partially and irregularly observed Markov process which evolves in continuous time. Both the process and the observation model are potentially non-linear. Particle filters therefore represent the simplest approach to online analysis. A functional Scala library of composable continuous time Markov process models has been developed in order to model the wide variety of data captured in the IoT.

Recommendations

• Using Storm for scaleable sequential statistical inference
• On-Line Inference for Hidden Markov Models via Particle Filters
• Learning continuous time Bayesian network classifiers
• Renewable estimation and incremental inference in generalized linear models with streaming data sets
• Online data processing: comparison of Bayesian regularized particle filters

Cites work

• Bayesian forecasting and dynamic models.
• Bayesian model selection and model averaging
• Book review of: L. Gawarecki and V. Mandrekar, Stochastic differential equations in infinite dimensions with applications to stochastic partial differential equations
• Efficient implementation of Markov chain Monte Carlo when using an unbiased likelihood estimator
• Factorial hidden Markov models
• Higher-order implicit strong numerical schemes for stochastic differential equations
• scientific article; zbMATH DE number 2106098 (Why is no real title available?)
• Machine learning. A probabilistic perspective
• Particle learning and smoothing
• Particle Markov chain Monte Carlo for efficient numerical simulation
• Sequential Monte Carlo Methods in Practice
• The pseudo-marginal approach for efficient Monte Carlo computations

Describes a project that uses

Uses Software

• R
• Scala
• pomp
• PMTK
• LibBi
• WinBUGS
• Spark

This page was built for publication: Composable models for online Bayesian analysis of streaming data

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2329734)