Artem

2015-03-18 12:39:26 UTC

Hello everyone

Recently I mentioned metric learning as one of possible projects for this

years' GSoC, and would like to hear your comments.

Metric learning, as follows from the name, is about learning distance

functions. Usually the metric that is learned is a Mahalanobis metric, thus

the problem reduces to finding a PSD matrix A that minimizes some

functional.

Metric learning is usually done in a supervised way, that is, a user tells

which points should be closer and which should be more distant. It can be

expressed either in form of "similar" / "dissimilar", or "A is closer to B

than to C".

Since metric learning is (mostly) about a PSD matrix A, one can do Cholesky

decomposition on it to obtain a matrix G to transform the data. It could

lead to something like guided clustering, where we first transform the data

space according to our prior knowledge of similarity.

Metric learning seems to be quite an active field of research ([1

<http://www.icml2010.org/tutorials.html>], [2

<http://www.ariel.ac.il/sites/ofirpele/DFML_ECCV2010_tutorial/>], [3

<http://nips.cc/Conferences/2011/Program/event.php?ID=2543>]). There are 2

somewhat up-to date surveys: [1

<http://web.cse.ohio-state.edu/~kulis/pubs/ftml_metric_learning.pdf>] and [2

<http://arxiv.org/abs/1306.6709>].

Top 3 seemingly most cited methods (according to Google Scholar) are

- MMC by Xing et al.

<http://papers.nips.cc/paper/2164-distance-metric-learning-with-application-to-clustering-with-side-information.pdf>

This

is a pioneering work and, according to the survey #2

The algorithm used to solve (1) is a simple projected gradient approach

<http://papers.nips.cc/paper/2795-distance-metric-learning-for-large-margin-nearest-neighbor-classification.pdf>.

The survey 2 acknowledges this method as "one of the most widely-used

Mahalanobis distance learning methods"

LMNN generally performs very well in practice, although it is sometimes

<http://dl.acm.org/citation.cfm?id=1273523> This one features a special

kind of regularizer called logDet.

- There are many other methods. If you guys know that other methods

rock, let me know.

So the project I'm proposing is about implementing 2nd or 3rd (or both?)

algorithms along with a relevant transformer.

Recently I mentioned metric learning as one of possible projects for this

years' GSoC, and would like to hear your comments.

Metric learning, as follows from the name, is about learning distance

functions. Usually the metric that is learned is a Mahalanobis metric, thus

the problem reduces to finding a PSD matrix A that minimizes some

functional.

Metric learning is usually done in a supervised way, that is, a user tells

which points should be closer and which should be more distant. It can be

expressed either in form of "similar" / "dissimilar", or "A is closer to B

than to C".

Since metric learning is (mostly) about a PSD matrix A, one can do Cholesky

decomposition on it to obtain a matrix G to transform the data. It could

lead to something like guided clustering, where we first transform the data

space according to our prior knowledge of similarity.

Metric learning seems to be quite an active field of research ([1

<http://www.icml2010.org/tutorials.html>], [2

<http://www.ariel.ac.il/sites/ofirpele/DFML_ECCV2010_tutorial/>], [3

<http://nips.cc/Conferences/2011/Program/event.php?ID=2543>]). There are 2

somewhat up-to date surveys: [1

<http://web.cse.ohio-state.edu/~kulis/pubs/ftml_metric_learning.pdf>] and [2

<http://arxiv.org/abs/1306.6709>].

Top 3 seemingly most cited methods (according to Google Scholar) are

- MMC by Xing et al.

<http://papers.nips.cc/paper/2164-distance-metric-learning-with-application-to-clustering-with-side-information.pdf>

This

is a pioneering work and, according to the survey #2

The algorithm used to solve (1) is a simple projected gradient approach

requiring the full

â â

eigenvalue decomposition of

â â

M

â â

at each iteration. This is typically intractable for medium

â â

and high-dimensional problems

- âLarge Margin Nearest Neighbor by Weinberger et alâ â

eigenvalue decomposition of

â â

M

â â

at each iteration. This is typically intractable for medium

â â

and high-dimensional problems

<http://papers.nips.cc/paper/2795-distance-metric-learning-for-large-margin-nearest-neighbor-classification.pdf>.

The survey 2 acknowledges this method as "one of the most widely-used

Mahalanobis distance learning methods"

LMNN generally performs very well in practice, although it is sometimes

prone to overfitting due to the absence of regularization, especially in

high dimension

- Information-theoretic metric learning by Davis et al.high dimension

<http://dl.acm.org/citation.cfm?id=1273523> This one features a special

kind of regularizer called logDet.

- There are many other methods. If you guys know that other methods

rock, let me know.

So the project I'm proposing is about implementing 2nd or 3rd (or both?)

algorithms along with a relevant transformer.