[Scikit-learn-general] Confidence-weighted learning

[Daniel Dahlmeier]

Dear scikit-learn community,

is there an implementation of confidence-weighted (CW) for scikit-learn or
is there an interest to have CW learning implemented for scikit? I have
noticed that the related passive-aggressive algorithm is already
implemented as part of the linear learner but CW learning is not.

I have implemented CW learning during my graduate school days and would be
interested to port the implementation to scikit-learn if this is seen as
useful.

regards,
Daniel


References

=============

Confidence-Weighted Linear Classification

Mark Dredze, Koby Crammer and Fernando Pereira

Proceedings of the 25th International Conference on Machine Learning

(ICML), 2008


Multi-Class Confidence Weighted Algorithms

Koby Crammer, Mark Dredze and Alex Kulesza

Empirical Methods in Natural Language Processing (EMNLP), 2009

[Mathieu Blondel]

Hi Daniel,

I think CW is a bit outdated and also a bit too specific (it supports only
the hinge loss). Algorithms like Adagrad are more generic. Thus, I think CW
is not a good candidate for inclusion in scikit-learn.

That said, I would welcome a contribution in lightning:
https://github.com/scikit-learn-contrib/lightning

In addition to the references you gave, there is also an ICML paper:
Exact Soft Confidence-Weighted Learning by J. Wang.

Mathieu

On Tue, Mar 29, 2016 at 12:34 PM, Daniel Dahlmeier <

Post by Daniel Dahlmeier
Dear scikit-learn community,
is there an implementation of confidence-weighted (CW) for scikit-learn
or is there an interest to have CW learning implemented for scikit? I
have noticed that the related passive-aggressive algorithm is already
implemented as part of the linear learner but CW learning is not.
I have implemented CW learning during my graduate school days and would be
interested to port the implementation to scikit-learn if this is seen as
useful.
regards,
Daniel
References
=============
Confidence-Weighted Linear Classification
Mark Dredze, Koby Crammer and Fernando Pereira
Proceedings of the 25th International Conference on Machine Learning
(ICML), 2008
Multi-Class Confidence Weighted Algorithms
Koby Crammer, Mark Dredze and Alex Kulesza
Empirical Methods in Natural Language Processing (EMNLP), 2009
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785471&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

[Daniel Dahlmeier]

Hi Mathieu,

thanks for the response and the feedback. It is correct that there are
other more recent algorithms available, on the other hand CW learning only
requires minor extensions from passive-aggressive learning (PA) which is
already available in scikit-learn and it achieves very competitive
performance (better than PA, SVM, maxent) and very good performance for
high dimensional, sparse classification problems, like text classification.

Thanks for the offer to contribute to lightning, I will have a look.

regards,
Daniel

Post by Mathieu Blondel
Hi Daniel,
I think CW is a bit outdated and also a bit too specific (it supports only
the hinge loss). Algorithms like Adagrad are more generic. Thus, I think

CW

Post by Mathieu Blondel
is not a good candidate for inclusion in scikit-learn.
https://github.com/scikit-learn-contrib/lightning
Exact Soft Confidence-Weighted Learning by J. Wang.
Mathieu