Dynamic Bayesian networks for audio-visual speech recognition

MaRDI QIDQ1424537zbMATH OpenFDO

Authors D. Massart

Publication date 16 March 2004

Published in EURASIP Journal on Applied Signal Processing (Search for Journal in Brave)

zbMATH Keywords

coupled hidden Markov models factorial hidden Markov models hidden Markov models

Mathematics Subject Classification ID

Pattern recognition, speech recognition (68T10) Natural language processing (68T50)

Review text

Summary: The use of visual features in audio-visual speech recognition (AVSR) is justified by both the speech generation mechanism, which is essentially bimodal in audio and visual representation, and by the need for features that are invariant to acoustic noise perturbation. As a result, current AVSR systems demonstrate significant accuracy improvements in environments affected by acoustic noise. In this paper, we describe the use of two statistical models for audio-visual integration, the coupled HMM (CHMM) and the factorial HMM (FHMM), and compare the performance of these models with the existing models used in speaker dependent audio-visual isolated word recognition. The statistical properties of both the CHMM and FHMM allow to model the state asynchrony of the audio and visual observation sequences while preserving their natural correlation over time. In our experiments, the CHMM performs best overall, outperforming all the existing models and the FHMM.

Recommendations

Cited in

(12)

This page was built for publication: Dynamic Bayesian networks for audio-visual speech recognition

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q1424537)