On Fri, Nov 4, 2011 at 10:49 AM, Andreas Müller
Post by Andreas Müller Post by Alexandre Passos Post by Lars Buitinck Post by Alexandre Passos
I have a question: why not just use Theano for this? I doubt that we
can write neural network code that's as fast as their automatically
Would that mean an extra run-time dependency?
Yes, as theano needs a compiler (gcc or nvcc if you want to use cuda)
available at run time, but even still it's faster even than most
hand-coded implementations of neural networks. James Bergstra reads
this list occasionally, and he's one of the main people behind theano,
so he can give more info here.
I think sklearn does not aim at beating cuda implementations.
For using theano: that's a huge and imho unnecessary dependency.
For a simple mlp, I think theano will not beat a hand implemented version.
Afaik, torch7 is faster than theano for cnns and mlps and there
is no compilation of algorithms there. If you want to vary your
a lot and want to do fancy things, theano is probably faster.
But I thought more about an easy to use classifier.
Yes Theano is a huge dependency. Yes there is no MLP implementation in
Theano, but there is the Deep Learning Tutorial that have one. It
also have implementation for Logistic Regression, Deep Convolutional
Network, Stacked Denoising Auto-Encoders, Restricted Boltzmann
Machines and Deep Belief Networks.
Those implementation are in a tutorial format. They are not in a
librarie with an easy to use interface like what scikit.learn use. But
many people already modified them to there need. We want from a long
time to have an easier interface, but we don't know how well to do
automatically the hyper-parameter selection.
I don't have a good understanding of scikit.learn, but I think that
all the hyper-parameter selection is a hot research topic for now. How
do you plan to include this in the current scikit.learn interface of
the fit method?
About torch7 being faster then Theano. I have heard that a few times,
but never see the papers, numbers, code or whatever subtential for
this. I would love to have any number with something about that. Do
you have some? But don't forget that in the Theano framework, we can
just implement all the trick that other people used to beat Theano. So
if torch7 is faster in some case, this will tell us where we can make
Theano faster! Can you tell us more about the comparison you refer to?
Just a side note. I don't imply the comparison you refer to is biased,
but benchmarking is VERY HARD. So I like to have information on how
the comparison is done. We tried to make the Theano comparison as fair
as we could at that time. We spend days compiling each applications
with the same blas and other stuff like that. But since them torch
have new version released.
Thanks and I hope to have more info on the comparison people used to
tell that torch7 is faster then Theano and how you plan to work around
the hyper-paramete selection problem. That would be very valuable to
every body I think.
p.s. I'm one of the core Theano developer.