Multimodal Task-Driven Dictionary Learning for Image Classification

DOI10.1109/TIP.2015.2496275zbMATH Open1408.94039arXiv1502.01094OpenAlexW3100063120WikidataQ40352112 ScholiaQ40352112MaRDI QIDQ4616124FDOQ4616124

Authors:

Publication date: 4 February 2019

Published in: IEEE Transactions on Image Processing (Search for Journal in Brave)

Abstract: Dictionary learning algorithms have been successfully used for both reconstructive and discriminative tasks, where an input signal is represented with a sparse linear combination of dictionary atoms. While these methods are mostly developed for single-modality scenarios, recent studies have demonstrated the advantages of feature-level fusion based on the joint sparse representation of the multimodal inputs. In this paper, we propose a multimodal task-driven dictionary learning algorithm under the joint sparsity constraint (prior) to enforce collaborations among multiple homogeneous/heterogeneous sources of information. In this task-driven formulation, the multimodal dictionaries are learned simultaneously with their corresponding classifiers. The resulting multimodal dictionaries can generate discriminative latent features (sparse codes) from the data that are optimized for a given task such as binary or multiclass classification. Moreover, we present an extension of the proposed formulation using a mixed joint and independent sparsity prior which facilitates more flexible fusion of the modalities at feature level. The efficacy of the proposed algorithms for multimodal classification is illustrated on four different applications -- multimodal face recognition, multi-view face recognition, multi-view action recognition, and multimodal biometric recognition. It is also shown that, compared to the counterpart reconstructive-based dictionary learning algorithms, the task-driven formulations are more computationally efficient in the sense that they can be equipped with more compact dictionaries and still achieve superior performance.

Full work available at URL: https://arxiv.org/abs/1502.01094

Mathematics Subject Classification ID

Classification and discrimination; cluster analysis (statistical aspects) (62H30) Image analysis in multivariate analysis (62H35) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08)

Cited In (9)

• Multi-Modal Dictionary Learning for Image Separation With Application in Art Investigation
• Manifold optimization-based analysis dictionary learning with an \(\ell_{1/2}\)-norm regularizer
• Multi-Modal Curriculum Learning for Semi-Supervised Image Classification
• A dictionary learning approach for fractal image coding
• Joint specific and correlated information exploration for multi-view action clustering
• A classification-oriented dictionary learning model: explicitly learning the particularity and commonality across categories
• Multi-view low-rank dictionary learning for image classification
• An outer-inner linearization method for non-convex and nondifferentiable composite regularization problems
• Title not available (Why is that?)

This page was built for publication: Multimodal Task-Driven Dictionary Learning for Image Classification

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4616124)