Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

DOI10.1109/TIP.2017.2655449MaRDI QIDQ4618680zbMATH OpenDBLPWikidataFDO

Authors Yang Wang, Xuemin Lin, Lin Wu, Wenjie Zhang

Publication date 5 February 2019

Published in IEEE Transactions on Image Processing (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1701.05003

Mathematics Subject Classification ID

Information storage and retrieval of data (68P20) Image processing (compression, reconstruction, etc.) in information and communication theory (94A08)

Abstract: Given a query photo issued by a user (q-user), the landmark retrieval is to return a set of photos with their landmarks similar to those of the query, while the existing studies on the landmark retrieval focus on exploiting geometries of landmarks for similarity matches between candidate photos and a query photo. We observe that the same landmarks provided by different users over social media community may convey different geometry information depending on the viewpoints and/or angles, and may subsequently yield very different results. In fact, dealing with the landmarks with illshapes caused by the photography of q-users is often nontrivial and has seldom been studied. In this paper we propose a novel framework, namely multi-query expansions, to retrieve semantically robust landmarks by two steps. Firstly, we identify the top-$k$ photos regarding the latent topics of a query landmark to construct multi-query set so as to remedy its possible illshape. For this purpose, we significantly extend the techniques of Latent Dirichlet Allocation. Then, motivated by the typical emph{collaborative filtering} methods, we propose to learn a emph{collaborative} deep networks based semantically, nonlinear and high-level features over the latent factor for landmark photo as the training set, which is formed by matrix factorization over emph{collaborative} user-photo matrix regarding the multi-query set. The learned deep network is further applied to generate the features for all the other photos, meanwhile resulting into a compact multi-query set within such space. Extensive experiments are conducted on real-world social media data with both landmark photos together with their user information to show the superior performance over the existing methods.

Recommendations

• Deep Context-Sensitive Facial Landmark Detection With Tree-Structured Modeling
• Cascaded Collaborative Regression for Robust Facial Landmark Detection Trained Using a Mixture of Synthetic and Real Images With Dynamic Weighting

Cited in

(2)

• Topic extraction from extremely short texts with variational manifold regularization
• Beyond low-rank representations: orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering

This page was built for publication: Effective Multi-Query Expansions: Collaborative Deep Networks for Robust Landmark Retrieval

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q4618680)