Discussion:
[Scikit-learn-general] Machine Learning in Numba
Mathieu Blondel
2013-08-06 16:34:17 UTC
Permalink
Hi everyone,

I have been interested in trying my hand at Numba [1] for quite a long
time. For those who don't know, Numba is a just-in-time compiler for Python
and NumPy. When a function is called for the first time, Numba infers its
types and compiles a specialized instance of the function to LLVM
instructions. Numba integrates seamlessly in Python source code: to
accelerate a function, one simply needs to annotate it with a Python
decorator. Therefore, unlike Cython, there is no need for prior compilation
at all.

This evening, I finally gave Numba a shot and reimplemented my sparse
multiclass classification paper [2] using it. The code source is available
in a gist [3].

My motivation was two-fold: 1) I wanted to compare Numba and Cython in
terms of ease of use and speed for real-world machine learning algorithms
2) I wanted a more self-contained implementation of my paper since the
reference Cython implementation is quite packed with features and hard to
follow.

I bencharked the Numba implementation against the reference Cython
implementation on the 20 newgroups dataset (18,846 samples; 130,107
features; 20 classes). The Numba implementation takes 55 seconds to make 20
outer iterations over the dataset while the Cython implementation takes 26
seconds. So there is only roughly a 2x performance hit (I'm pretty sure
that both implementations have some room for improvement, though).

I'm quite impressed with Numba and think it has the potential to be a game
changer for the Python scientific community. While reimplementing my paper,
I encountered a few cryptic error messages which make me think it may be a
bit too early to use Numba in scikit-learn yet. However, in a year or so,
after Numba reaches 1.0, I think we should definitely consider making it a
dependency. It should allow us to get rid of much compiled code in our code
base.

Cheers,
Mathieu

[1] http://numba.pydata.org/
[2] http://www.mblondel.org/code/mlj2013/
[3] https://gist.github.com/mblondel/6165747
Yaroslav Halchenko
2013-08-06 16:52:31 UTC
Permalink
coolio!

from numba page: "For e.g., llvm-gcc can be used to generate the LLVM
assembly for a given C source file, which can then be loaded and
manipulated (adding profiling code to every function, say) using a
llvm-py based Python script." I wonder if you have tried to profile the
beast to see if there is an obvious single bottleneck? ;)

Cheers,
Post by Mathieu Blondel
Hi everyone,
I have been interested in trying my hand at Numba [1] for quite a long
time. For those who don't know, Numba is a just-in-time compiler for
Python and NumPy. When a function is called for the first time, Numba
infers its types and compiles a specialized instance of the function to
LLVM instructions. Numba integrates seamlessly in Python source code: to
accelerate a function, one simply needs to annotate it with a Python
decorator. Therefore, unlike Cython, there is no need for prior
compilation at all.
This evening, I finally gave Numba a shot and reimplemented my sparse
multiclass classification paper [2] using it. The code source is available
in a gist [3].
My motivation was two-fold: 1) I wanted to compare Numba and Cython in
terms of ease of use and speed for real-world machine learning algorithms
2) I wanted a more self-contained implementation of my paper since the
reference Cython implementation is quite packed with features and hard to
follow.
I bencharked the Numba implementation against the reference Cython
implementation on the 20 newgroups dataset (18,846 samples; 130,107
features; 20 classes). The Numba implementation takes 55 seconds to make
20 outer iterations over the dataset while the Cython implementation takes
26 seconds. So there is only roughly a 2x performance hit (I'm pretty sure
that both implementations have some room for improvement, though).
I'm quite impressed with Numba and think it has the potential to be a game
changer for the Python scientific community. While reimplementing my
paper, I encountered a few cryptic error messages which make me think it
may be a bit too early to use Numba in scikit-learn yet. However, in a
year or so, after Numba reaches 1.0, I think we should definitely consider
making it a dependency. It should allow us to get rid of much compiled
code in our code base.
Cheers,
Mathieu
[1] [1]http://numba.pydata.org/
[2] [2]http://www.mblondel.org/code/mlj2013/
[3] [3]https://gist.github.com/mblondel/6165747
References
Visible links
1. http://numba.pydata.org/
2. http://www.mblondel.org/code/mlj2013/
3. https://gist.github.com/mblondel/6165747
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
Olivier Grisel
2013-08-06 17:20:35 UTC
Permalink
Thanks for sharing your insights.

If you are interested in contributing more benchmarks to help python
compiler developers make performance optimizations based on
semi-realistic usecases, please check out this github repo:

https://github.com/numfocus/python-benchmarks

I think a python + numba and a cython implementation of an (averaged)
perceptron algorithm would be very interesting.

I share you conclusions:

- numba (and similar JIT compilers such as parakeet) is a game changer
for high performance Python dev
- it's still too early to consider it as production replacement for
cython (cryptic error messages, unsupported python constructs, API and
design still evolving...)
--
Olivier
Yaroslav Halchenko
2013-08-06 17:32:12 UTC
Permalink
Post by Olivier Grisel
If you are interested in contributing more benchmarks to help python
compiler developers make performance optimizations based on
https://github.com/numfocus/python-benchmarks
and if happen you are interested about numpy performance (for now just a
single build using cPython with stock libraries on Debian system but
across few branches) -- this repo:
https://github.com/yarikoptic/numpy-vbench/ with reports as of few days
back: http://www.onerussian.com/tmp/numpy-vbench/ .
--
Yaroslav O. Halchenko, Ph.D.
http://neuro.debian.net http://www.pymvpa.org http://www.fail2ban.org
Senior Research Associate, Psychological and Brain Sciences Dept.
Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755
Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419
WWW: http://www.linkedin.com/in/yarik
Mathieu Blondel
2013-08-07 00:16:07 UTC
Permalink
Post by Olivier Grisel
- it's still too early to consider it as production replacement for
cython (cryptic error messages, unsupported python constructs, API and
design still evolving...)
As an example, something as simple as this didn't work yet:

violation_max = max(violation, violation_max)

Mathieu

Loading...