Video primal sketch: a unified middle-level representation for video

DOI10.1007/S10851-015-0563-2MaRDI QIDQ890113zbMATHOpenAlexFDO

Authors Zhi Han, Song-Chun Zhu, Zongben Xu

Publication date 9 November 2015

Published in Journal of Mathematical Imaging and Vision (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1502.02965

zbMATH Keywords

video representationdynamic texture synthesismiddle-level visionprimal sketchtextured motion

Mathematics Subject Classification ID

Image processing (compression, reconstruction, etc.) in information and communication theory (94A08)

Abstract: This paper presents a middle-level video representation named Video Primal Sketch (VPS), which integrates two regimes of models: i) sparse coding model using static or moving primitives to explicitly represent moving corners, lines, feature points, etc., ii) FRAME /MRF model reproducing feature statistics extracted from input video to implicitly represent textured motion, such as water and fire. The feature statistics include histograms of spatio-temporal filters and velocity distributions. This paper makes three contributions to the literature: i) Learning a dictionary of video primitives using parametric generative models; ii) Proposing the Spatio-Temporal FRAME (ST-FRAME) and Motion-Appearance FRAME (MA-FRAME) models for modeling and synthesizing textured motion; and iii) Developing a parsimonious hybrid model for generic video representation. Given an input video, VPS selects the proper models automatically for different motion patterns and is compatible with high-level action representations. In the experiments, we synthesize a number of textured motion; reconstruct real videos using the VPS; report a series of human perception experiments to verify the quality of reconstructed videos; demonstrate how the VPS changes over the scale transition in videos; and present the close connection between VPS and high-level action models.

Recommendations

• scientific article; zbMATH DE number 2062513
• A Generative Method for Textured Motion: Analysis and Synthesis
• scientific article; zbMATH DE number 2013310
• Computer Vision - ECCV 2004
• Dynamic textures

Cites work

• scientific article; zbMATH DE number 3513115 (Why is no real title available?)
• A parametric texture model based on joint statistics of complex wavelet coefficients
• A probabilistic exclusion principle for tracking multiple objects
• Dynamic textures
• Equivalence of Julesz ensembles and FRAME models
• Intrackability: characterizing video statistics and pursuing video representations
• Matching pursuits with time-frequency dictionaries
• Mixed-state auto-models and motion texture modeling
• Probabilistic detection and tracking of motion boundaries
• Video primal sketch: a unified middle-level representation for video

Cited in

(4)

• Video primal sketch: a unified middle-level representation for video
• scientific article; zbMATH DE number 2013310 (Why is no real title available?)
• Temporal scale selection in time-causal scale space
• Multi-video synopsis for video representation

Describes a project that uses

Uses Software

• Steerable pyramid

This page was built for publication: Video primal sketch: a unified middle-level representation for video

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q890113)