Sharing hash codes for multiple purposes

DOI10.1007/S42081-018-0010-XMaRDI QIDQ2329853zbMATH OpenOpenAlexFDO

Authors Wiktor Pronobis, Danny Panknin, Johannes Kirschnick, Vignesh Srinivasan, Wojciech Samek, Volker Markl, Manohar Kaul, Klaus-Robert Müller, Shinichi Nakajima

Publication date 18 October 2019

Published in Japanese Journal of Statistics and Data Science (Search for Journal in Brave)

Full work available at URL https://arxiv.org/abs/1609.03219

zbMATH Keywords

collaborative filteringinformation retrievallocality sensitive hashingapproximate near neighbor search

Mathematics Subject Classification ID

Applications of statistics in engineering and industry; control charts (62P30) Statistical aspects of big data and data science (62R07)

Abstract: Locality sensitive hashing (LSH) is a powerful tool for sublinear-time approximate nearest neighbor search, and a variety of hashing schemes have been proposed for different dissimilarity measures. However, hash codes significantly depend on the dissimilarity, which prohibits users from adjusting the dissimilarity at query time. In this paper, we propose {multiple purpose LSH (mp-LSH) which shares the hash codes for different dissimilarities. mp-LSH supports L2, cosine, and inner product dissimilarities, and their corresponding weighted sums, where the weights can be adjusted at query time. It also allows us to modify the importance of pre-defined groups of features. Thus, mp-LSH enables us, for example, to retrieve similar items to a query with the user preference taken into account, to find a similar material to a query with some properties (stability, utility, etc.) optimized, and to turn on or off a part of multi-modal information (brightness, color, audio, text, etc.) in image/video retrieval. We theoretically and empirically analyze the performance of three variants of mp-LSH, and demonstrate their usefulness on real-world data sets.

Recommendations

• The distortion of locality sensitive hashing
• On the Distortion of Locality Sensitive Hashing
• Latent structure preserving hashing
• Why locality sensitive hashing works: a practical perspective
• Locality-sensitive hashing without false negatives

Cites work

• scientific article; zbMATH DE number 1775450 (Why is no real title available?)
• How to explain individual classification decisions
• Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming
• Learning deep architectures for AI
• Lectures on analysis on metric spaces
• Locality-sensitive hashing scheme based on p-stable distributions
• Pattern recognition and machine learning.
• Similarity estimation techniques from rounding algorithms
• The elements of statistical learning. Data mining, inference, and prediction

Cited in

(2)

• Consensus hashing
• A locality sensitive hashing filter for encrypted vector databases

Describes a project that uses

Uses Software

• PRMLT
• AlexNet
• AnnexML
• ImageNet
• SIFT

This page was built for publication: Sharing hash codes for multiple purposes

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q2329853)