[Scikit-learn-general] maximum and minimum regularization for NMF

[James Jensen]

For ElasticNetCV, inside the function _alpha_grid() it computes the maximum
regularization strength alpha, with a given dataset X, target Y, and L1
ratio, for which there will be at least one nonzero coefficient. I'm
wondering if/how the same could be computed for sklearn's L1/L2-regularized
NMF. I'm also interested in computing a minimum alpha (the smallest at
which there are more nonzero coefficients than with alpha=0).

Does anyone know how this could be done?

Thanks,

James Jensen
PhD student, Bioinformatics and Systems Biology
Trey Ideker lab
University of California, San Diego

[Vlad Niculae]

Hi James,

I'm not sure how useful a minimum alpha would be. Even if no weights
are shrunk quite to zero, the regularization can still impact
performance metrics. I would be curious what application you have in
mind for this.

The max alpha question is interesting, I am curious as well. (Sorry my
reply can't be more helpful!)

Yours,
Vlad

Post by James Jensen
For ElasticNetCV, inside the function _alpha_grid() it computes the maximum
regularization strength alpha, with a given dataset X, target Y, and L1
ratio, for which there will be at least one nonzero coefficient. I'm
wondering if/how the same could be computed for sklearn's L1/L2-regularized
NMF. I'm also interested in computing a minimum alpha (the smallest at which
there are more nonzero coefficients than with alpha=0).
Does anyone know how this could be done?
Thanks,
James Jensen
PhD student, Bioinformatics and Systems Biology
Trey Ideker lab
University of California, San Diego
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

[Mathieu Blondel]

I guess knowing the max alpha is useful to know where to start your grid
search from. However, I think deriving max alpha for NMF should be more
difficult since the problem is non-convex.

Mathieu

Post by Vlad Niculae
Hi James,
I'm not sure how useful a minimum alpha would be. Even if no weights
are shrunk quite to zero, the regularization can still impact
performance metrics. I would be curious what application you have in
mind for this.
The max alpha question is interesting, I am curious as well. (Sorry my
reply can't be more helpful!)
Yours,
Vlad

Post by James Jensen
For ElasticNetCV, inside the function _alpha_grid() it computes the

maximum

Post by James Jensen
regularization strength alpha, with a given dataset X, target Y, and L1
ratio, for which there will be at least one nonzero coefficient. I'm
wondering if/how the same could be computed for sklearn's

L1/L2-regularized

Post by James Jensen
NMF. I'm also interested in computing a minimum alpha (the smallest at

which

Post by James Jensen
there are more nonzero coefficients than with alpha=0).
Does anyone know how this could be done?
Thanks,
James Jensen
PhD student, Bioinformatics and Systems Biology
Trey Ideker lab
University of California, San Diego

------------------------------------------------------------------------------

Post by James Jensen
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general