Dimensionality Reduction by Learning an Invariant Mapping¶
Authors: Raia Hadsell, Sumit Chopra, Yann LeCun
Published: 2006 (Conference Paper)
Source: IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Algorithm: DrLIM
DOI: 10.1109/CVPR.2006.100
Summary¶
Introduces DrLIM and the contrastive-loss training setup for learning invariant embeddings from neighborhood relationships. It is an important ancestor of Siamese metric learning and modern contrastive representation learning.
Abstract¶
Dimensionality reduction involves mapping a set of high dimensional input points onto a low dimensional manifold so that similar points in input space are mapped to nearby points on the manifold. Most existing techniques for solving the problem suffer from two drawbacks. First, most of them depend on a meaningful and computable distance metric in input space. Second, they do not compute a function that can accurately map new input samples whose relationship to the training data is unknown. We present a method - called Dimensionality Reduction by Learning an Invariant Mapping (DrLIM) - for learning a globally coherent non-linear function that maps the data evenly to the output manifold. The learning relies solely on neighborhood relationships and does not require any distance measure in the input space. The method can learn mappings that are invariant to certain transformations of the inputs, as is demonstrated with a number of experiments. Comparisons are made to other techniques, in particular LLE.
Links¶
Primary
Standard
Tags¶
-
Dimensionality reduction
-
Metric learning
-
Contrastive loss
-
Siamese networks
-
Invariant mapping
-
Representation learning