Mathieu Blondel
2013-08-06 16:34:17 UTC
Hi everyone,
I have been interested in trying my hand at Numba [1] for quite a long
time. For those who don't know, Numba is a just-in-time compiler for Python
and NumPy. When a function is called for the first time, Numba infers its
types and compiles a specialized instance of the function to LLVM
instructions. Numba integrates seamlessly in Python source code: to
accelerate a function, one simply needs to annotate it with a Python
decorator. Therefore, unlike Cython, there is no need for prior compilation
at all.
This evening, I finally gave Numba a shot and reimplemented my sparse
multiclass classification paper [2] using it. The code source is available
in a gist [3].
My motivation was two-fold: 1) I wanted to compare Numba and Cython in
terms of ease of use and speed for real-world machine learning algorithms
2) I wanted a more self-contained implementation of my paper since the
reference Cython implementation is quite packed with features and hard to
follow.
I bencharked the Numba implementation against the reference Cython
implementation on the 20 newgroups dataset (18,846 samples; 130,107
features; 20 classes). The Numba implementation takes 55 seconds to make 20
outer iterations over the dataset while the Cython implementation takes 26
seconds. So there is only roughly a 2x performance hit (I'm pretty sure
that both implementations have some room for improvement, though).
I'm quite impressed with Numba and think it has the potential to be a game
changer for the Python scientific community. While reimplementing my paper,
I encountered a few cryptic error messages which make me think it may be a
bit too early to use Numba in scikit-learn yet. However, in a year or so,
after Numba reaches 1.0, I think we should definitely consider making it a
dependency. It should allow us to get rid of much compiled code in our code
base.
Cheers,
Mathieu
[1] http://numba.pydata.org/
[2] http://www.mblondel.org/code/mlj2013/
[3] https://gist.github.com/mblondel/6165747
I have been interested in trying my hand at Numba [1] for quite a long
time. For those who don't know, Numba is a just-in-time compiler for Python
and NumPy. When a function is called for the first time, Numba infers its
types and compiles a specialized instance of the function to LLVM
instructions. Numba integrates seamlessly in Python source code: to
accelerate a function, one simply needs to annotate it with a Python
decorator. Therefore, unlike Cython, there is no need for prior compilation
at all.
This evening, I finally gave Numba a shot and reimplemented my sparse
multiclass classification paper [2] using it. The code source is available
in a gist [3].
My motivation was two-fold: 1) I wanted to compare Numba and Cython in
terms of ease of use and speed for real-world machine learning algorithms
2) I wanted a more self-contained implementation of my paper since the
reference Cython implementation is quite packed with features and hard to
follow.
I bencharked the Numba implementation against the reference Cython
implementation on the 20 newgroups dataset (18,846 samples; 130,107
features; 20 classes). The Numba implementation takes 55 seconds to make 20
outer iterations over the dataset while the Cython implementation takes 26
seconds. So there is only roughly a 2x performance hit (I'm pretty sure
that both implementations have some room for improvement, though).
I'm quite impressed with Numba and think it has the potential to be a game
changer for the Python scientific community. While reimplementing my paper,
I encountered a few cryptic error messages which make me think it may be a
bit too early to use Numba in scikit-learn yet. However, in a year or so,
after Numba reaches 1.0, I think we should definitely consider making it a
dependency. It should allow us to get rid of much compiled code in our code
base.
Cheers,
Mathieu
[1] http://numba.pydata.org/
[2] http://www.mblondel.org/code/mlj2013/
[3] https://gist.github.com/mblondel/6165747