Discussion:
Fwd: [Scikit-learn-commits] buildbot failure in Scikit Learn on ubuntu-64bit
(too old to reply)
Olivier Grisel
2010-11-25 10:22:08 UTC
Permalink
Hi Vincent,

Could you please subscribe to the scikit-learn-commits mailing list
and fix the doctests that were broken during the merge of your branch
please?

---------- Forwarded message ----------
From: <***@ensta.org>
Date: 2010/11/25
Subject: [Scikit-learn-commits] buildbot failure in Scikit Learn on ubuntu-64bit
To: scikit-learn-***@lists.sourceforge.net


The Buildbot has detected a failed build on builder ubuntu-64bit while
building Scikit Learn.
Full details are available at:
 http://buildbot.afpy.org/scikit-learn/builders/ubuntu-64bit/builds/945

Buildbot URL: http://buildbot.afpy.org/scikit-learn/

Buildslave for this Build: scikit-learn-slave

Build Reason: The Periodic scheduler named 'periodic' triggered this build
Build Source Stamp: HEAD
Blamelist:

BUILD FAILED: failed test

sincerely,
 -The Buildbot
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Vincent Dubourg
2010-11-25 19:20:09 UTC
Permalink
Hi list,

It seems that Fabian has done the work for me. Thanks!
@Olivier: I didn't know about the commits mailing list: I just suscribed.

@Gael: Why don't you like lambda functions?
f = lambda x: x * np.sin(x)
return x * np.sin(x)
@Fabian: I don't understand why you removed the gaussian_process relative
import from the toplevel __init__ file? As all the other submodules are
systematically imported in that file. In any case, if you decide to maintain
this change, you have to change the examples (or the auto_examples' at doc
build will fail accordingly). I would do it, but I don't have my laptop
tonight... and won't have time to do it before Saturday. But I find that
importing the whole module just as we usually do with numpy (as np) is a bit
heavy for the user and it's not homogeneous with the rest of the scikit as
from scikits.learn.svm import SVC
from scikits.learn.gaussian_process import GaussianProcess
Hi Vincent,
Could you please subscribe to the scikit-learn-commits mailing list
and fix the doctests that were broken during the merge of your branch
please?
---------- Forwarded message ----------
Date: 2010/11/25
Subject: [Scikit-learn-commits] buildbot failure in Scikit Learn on ubuntu-64bit
The Buildbot has detected a failed build on builder ubuntu-64bit while
building Scikit Learn.
http://buildbot.afpy.org/scikit-learn/builders/ubuntu-64bit/builds/945
Buildbot URL: http://buildbot.afpy.org/scikit-learn/
Buildslave for this Build: scikit-learn-slave
Build Reason: The Periodic scheduler named 'periodic' triggered this build
Build Source Stamp: HEAD
BUILD FAILED: failed test
sincerely,
-The Buildbot
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-11-25 19:51:05 UTC
Permalink
Vincent Dubourg
2010-11-25 19:59:52 UTC
Permalink
@Gael: You're right about the incomprehensible Pickling Errors about lambda
functions.

@all: What about the deletion of the relative import of the gaussian_process
submodule in the top level __init__file: what's your opinion?
Post by Vincent Dubourg
@Gael: Why don't you like lambda functions?
f = lambda x: x * np.sin(x)
return x * np.sin(x)
I don't have particularly strong feelings against lambda functions.
However, I removed the lambda in the documentation because the notation
'lambda' is extremely confusing for beginners. Of course an advanced
Python user will know what a lambda is. However, I work with people who
come from Matlab, or other environments. I think that lowering the bar as
much as possible in the documentation is a good thing.
On top of that, I don't think that we should encourage users to use
lambdas that much: they don't pickle, and as a result if you start using
them a lot, you will have incomprehensible error message when using the
parallel processing features of the scikit.
Cheers,
G
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-11-25 20:13:10 UTC
Permalink
Post by Vincent Dubourg
@all: What about the deletion of the relative import of the
gaussian_process submodule in the top level __init__file: what's your
opinion?
I am still hesitate on have imports in the base __init__. I am wondering
if we shouldn't go the scipy way, and have no imports in the __init__.

There are several reasons to do this: one is to limit the import time. As
time goes the features in the scikit will grow, and so will the import
time. Another one is to localize breakage: if some code does not import
(because of a bug in the module, some binary incompatibility, or just an
installation bug as we had yesterday evening with Joel. Finally, one
reason for not importing subpackages is that if we change our minds to
import them, it won't break user's script, whereas the other way around
will. Scipy changed recently, and it created havok amongst users.

However, we should be consistent: either all the subpackages should be
imported, or none. My current gut feeling is to go for none, and only
import a few things, like from base.py.

What are other people's opinions?

G
Olivier Grisel
2010-11-25 23:42:46 UTC
Permalink
Post by Gael Varoquaux
   gaussian_process submodule in the top level __init__file: what's your
   opinion?
I am still hesitate on have imports in the base __init__. I am wondering
if we shouldn't go the scipy way, and have no imports in the __init__.
There are several reasons to do this: one is to limit the import time. As
time goes the features in the scikit will grow, and so will the import
time. Another one is to localize breakage: if some code does not import
(because of a bug in the module, some binary incompatibility, or just an
installation bug as we had yesterday evening with Joel. Finally, one
reason for not importing subpackages is that if we change our minds to
import them, it won't break user's script, whereas the other way around
will. Scipy changed recently, and it created havok amongst users.
However, we should be consistent: either all the subpackages should be
imported, or none. My current gut feeling is to go for none, and only
import a few things, like from base.py.
What are other people's opinions?
I would go for __init__.py empty unless special requirements because
it feels simpler and more predictable. But no strong opinion either.
--
Olivier
http://twitter.com/ogrisel - http://github.com/ogrisel
Matthieu PERROT
2010-11-26 14:40:02 UTC
Permalink
Post by Gael Varoquaux
Post by Vincent Dubourg
@all: What about the deletion of the relative import of the
gaussian_process submodule in the top level __init__file: what's your
opinion?
I am still hesitate on have imports in the base __init__. I am wondering
if we shouldn't go the scipy way, and have no imports in the __init__.
There are several reasons to do this: one is to limit the import time. As
time goes the features in the scikit will grow, and so will the import
time. Another one is to localize breakage: if some code does not import
(because of a bug in the module, some binary incompatibility, or just an
installation bug as we had yesterday evening with Joel. Finally, one
reason for not importing subpackages is that if we change our minds to
import them, it won't break user's script, whereas the other way around
will. Scipy changed recently, and it created havok amongst users.
However, we should be consistent: either all the subpackages should be
imported, or none. My current gut feeling is to go for none, and only
import a few things, like from base.py.
What are other people's opinions?
+1, you give good reasons to follow this behaviour (import time and
consistency).
--
Matthieu Perrot
Mathieu Blondel
2010-11-26 15:23:11 UTC
Permalink
On Fri, Nov 26, 2010 at 5:13 AM, Gael Varoquaux
Post by Gael Varoquaux
What are other people's opinions?
I'm +1 for not importing any module. I always explicitly import the
modules I need anyway.

Mathieu
Fabian Pedregosa
2010-11-26 20:56:17 UTC
Permalink
Post by Mathieu Blondel
On Fri, Nov 26, 2010 at 5:13 AM, Gael Varoquaux
Post by Gael Varoquaux
What are other people's opinions?
I'm +1 for not importing any module. I always explicitly import the
modules I need anyway.
Excellent. Vincent, could you take care of this (cleaning __init__.py
file + fixing tests and examples) when you have time ?

Thanks,

Fabian
Post by Mathieu Blondel
Mathieu
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Vincent Dubourg
2010-11-27 10:05:21 UTC
Permalink
Post by Fabian Pedregosa
Post by Mathieu Blondel
from scikits.learn.gaussian_process import GaussianProcess
still works on my machine despites the relative import has disappeared
from the top level init file right after a "make clean && make inplace".
How does python know about the existence of a submodule that is not
imported in the top level init file?

Also, can someone tell me which global tests I should run before
committing so that I am sure the build bot won't fail.
Until now, I just ran "nosetests *.py --with-doctest" in the
gaussian_process subdirectories but I should now run more global tests.
Note that the command "make test test-doc" is not producing any test on
my machine (see attached file for output).

Thanks,
Vincent
Post by Fabian Pedregosa
Post by Mathieu Blondel
On Fri, Nov 26, 2010 at 5:13 AM, Gael Varoquaux
What are other people's opinions?
I'm +1 for not importing any module. I always explicitly import the
modules I need anyway.
Excellent. Vincent, could you take care of this (cleaning __init__.py
file + fixing tests and examples) when you have time ?
Thanks,
Fabian
Post by Mathieu Blondel
Mathieu
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
Tap into the largest installed PC base& get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
Tap into the largest installed PC base& get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-11-27 10:41:07 UTC
Permalink
Post by Vincent Dubourg
from scikits.learn.gaussian_process import GaussianProcess
still works on my machine despites the relative import has disappeared
from the top level init file right after a "make clean && make inplace".
How does python know about the existence of a submodule that is not
imported in the top level init file?
Yes. This is why we think that it is pretty harmless to remove them.

One thing that will change, is that you will no longer be able to do:

from scikits import learn
learn.gaussian_process.GaussianProcess
Post by Vincent Dubourg
Also, can someone tell me which global tests I should run before
committing so that I am sure the build bot won't fail.
Until now, I just ran "nosetests *.py --with-doctest" in the
gaussian_process subdirectories but I should now run more global tests.
Run 'nosetests scikits.learn --width-doctest'
Post by Vincent Dubourg
Note that the command "make test test-doc" is not producing any test on
my machine (see attached file for output).
Hum, not good. Something is wrong, I am not sure what.
Post by Vincent Dubourg
nosetests scikits/learn
.
----------------------------------------------------------------------
Ran 1 test in 0.030s
OK
nosetests --with-doctest --doctest-tests --doctest-extension=rst doc/ doc/modules/
----------------------------------------------------------------------
Ran 0 tests in 0.009s
Vincent Dubourg
2010-11-27 11:02:09 UTC
Permalink
Gael, I pushed the changed __init__ file in my branch
(***@github.com:dubourg/scikit-learn.git). You can now pull it and check
that it does not change anything.
Post by Gael Varoquaux
from scikits import learn
learn.gaussian_process.GaussianProcess
It seems that nobody's working like that in the examples, so it's fine.
Post by Gael Varoquaux
Post by Vincent Dubourg
from scikits.learn.gaussian_process import GaussianProcess
still works on my machine despites the relative import has disappeared
from the top level init file right after a "make clean&& make inplace".
How does python know about the existence of a submodule that is not
imported in the top level init file?
Yes. This is why we think that it is pretty harmless to remove them.
from scikits import learn
learn.gaussian_process.GaussianProcess
Post by Vincent Dubourg
Also, can someone tell me which global tests I should run before
committing so that I am sure the build bot won't fail.
Until now, I just ran "nosetests *.py --with-doctest" in the
gaussian_process subdirectories but I should now run more global tests.
Run 'nosetests scikits.learn --width-doctest'
Post by Vincent Dubourg
Note that the command "make test test-doc" is not producing any test on
my machine (see attached file for output).
Hum, not good. Something is wrong, I am not sure what.
Post by Vincent Dubourg
nosetests scikits/learn
.
----------------------------------------------------------------------
Ran 1 test in 0.030s
OK
nosetests --with-doctest --doctest-tests --doctest-extension=rst doc/ doc/modules/
----------------------------------------------------------------------
Ran 0 tests in 0.009s
Gael Varoquaux
2010-11-27 11:59:49 UTC
Permalink
Post by Vincent Dubourg
Gael, I pushed the changed __init__ file in my branch
that it does not change anything.
OK, I have run all thes tests and the examples. Everything seems good.
You good go ahead and push in the master.

Thanks for your work,

Gaël
Gael Varoquaux
2010-11-27 12:35:05 UTC
Permalink
Post by Gael Varoquaux
OK, I have run all thes tests and the examples. Everything seems good.
You good go ahead and push in the master.
I've done the merge, because I had other things to commit too.

Gaël
Fabian Pedregosa
2010-11-29 12:44:15 UTC
Permalink
On Sat, Nov 27, 2010 at 12:02 PM, Vincent Dubourg
Post by Vincent Dubourg
Gael, I pushed the changed __init__ file in my branch
Thanks Vincent!

Fabian.

Vincent Dubourg
2010-11-27 10:32:52 UTC
Permalink
Forget about the tests not working on my machine. I used to clone the
git repository on an ntfs-3g partition which caused some unexpected
behavior.
Now that I checked on a real (ext4) partition, I am able to run the
tests (both test and doc-test).

However the deletion of the submodule relative imports in the init file
is not producing any error nor failure (after commenting lines 17-26 of
the init file + "make clean inplace").
Can someone deny/confirm they are completely useless?

Vincent
Post by Fabian Pedregosa
Post by Mathieu Blondel
On Fri, Nov 26, 2010 at 5:13 AM, Gael Varoquaux
Post by Gael Varoquaux
What are other people's opinions?
I'm +1 for not importing any module. I always explicitly import the
modules I need anyway.
Excellent. Vincent, could you take care of this (cleaning __init__.py
file + fixing tests and examples) when you have time ?
Thanks,
Fabian
Post by Mathieu Blondel
Mathieu
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
Tap into the largest installed PC base& get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Increase Visibility of Your 3D Game App& Earn a Chance To Win $500!
Tap into the largest installed PC base& get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2010-11-27 10:38:18 UTC
Permalink
Post by Vincent Dubourg
However the deletion of the submodule relative imports in the init file
is not producing any error nor failure (after commenting lines 17-26 of
the init file + "make clean inplace").
It's quite common to have remaining '.pyc' that mask the deletion of
'.py', but as you did a 'make clean', I suspect that you are safe against
that problem.
Post by Vincent Dubourg
Can someone deny/confirm they are completely useless?
I can't :). However, I am willing to have a look.

Could you:

1. Run all the examples
2. Commit and push on your github branch.

I'll pull from your github and have a look. If both of us come to the
same conclusion, we can say that we are safe, and push to the main
master.

Cheers,

Gaël
Loading...