Discussion:
[Scikit-learn-general] Implementation of Bag-of-Features
Guillaume Lemaître
2016-02-15 02:03:09 UTC
Permalink
Dear all,

My group and I, are currently working on image classification applied to
medical images. We are using the Bag-of-Features (or Bag-of-Visual-Words,
Bag-of-Words) which was inspired originally from the text classification.
In fact, we have a kind of dirty implementation [here](
https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py)
which I would like to, somehow, even only if it is for a personal branch,
integrate to the scikit-learn.

However, I have some philosophical questions before to mess around, which
in fact are feeding some discussions in our lab. Checking the API, the BoF
approach could be part of the `feature_extraction` module. BoF is really
similar to the implementation of the BoW for text as previously mentioned.

Nevertheless, I am questioning if the BoF shall rather not be integrated to
the `decomposition` module. By looking at it, the method consists of: (i)
dictionary learning (base K-Means, Mean-Shift, etc.), (ii) encoding (or
voting in that case using k-NN), and (iii) pooling (histogram).

Thus, in some sort the BoF can be seen as any of the decomposition (even
more similar to sparse coding). For instance the sparse learning follow
exactly the same scheme: dictionary learning with K-SVD, encoding, and
pooling (min/max/etc.). Similar thing for PCA, if you tackle the problem of
dictionary as finding the eigenvectors/eigenvalues.

My questions are thus the following:
- what are you thinking about such thing;
- where the BoF implementation of this approach is the most judicious;
- would it be judicious to think about the different decomposition methods
as the three steps earlier mentioned or it would be not at all intuitive?

Hope that the topic is not to weird.

Cheers,
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Andreas Mueller
2016-02-22 17:10:58 UTC
Permalink
Hi Guillaume.

I was a big user of BoW myself, but I don't think it should go into
scikit-learn.
BoW doesn't really operate on a "flat" dataset, as scikit-learn usually
does. It works on groups of data points.
Each sample is usually a concatenation of feature vectors, which you
summarize as a histogram.
That doesn't really fit into the scikit-learn API.

For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here
https://github.com/amueller/segmentation/blob/master/bow.py#L184)

Cheers,
Andy
Post by Guillaume Lemaître
Dear all,
My group and I, are currently working on image classification applied
to medical images. We are using the Bag-of-Features (or
Bag-of-Visual-Words, Bag-of-Words) which was inspired originally from
the text classification. In fact, we have a kind of dirty
implementation
[here](https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py)
which I would like to, somehow, even only if it is for a personal
branch, integrate to the scikit-learn.
However, I have some philosophical questions before to mess around,
which in fact are feeding some discussions in our lab. Checking the
API, the BoF approach could be part of the `feature_extraction`
module. BoF is really similar to the implementation of the BoW for
text as previously mentioned.
Nevertheless, I am questioning if the BoF shall rather not be
integrated to the `decomposition` module. By looking at it, the method
consists of: (i) dictionary learning (base K-Means, Mean-Shift, etc.),
(ii) encoding (or voting in that case using k-NN), and (iii) pooling
(histogram).
Thus, in some sort the BoF can be seen as any of the decomposition
(even more similar to sparse coding). For instance the sparse learning
follow exactly the same scheme: dictionary learning with K-SVD,
encoding, and pooling (min/max/etc.). Similar thing for PCA, if you
tackle the problem of dictionary as finding the eigenvectors/eigenvalues.
- what are you thinking about such thing;
- where the BoF implementation of this approach is the most judicious;
- would it be judicious to think about the different decomposition
methods as the three steps earlier mentioned or it would be not at all
intuitive?
Hope that the topic is not to weird.
Cheers,
--
*LEMAÎTRE Guillaume
PhD Candidate
MSc Erasmus Mundus ViBOT (Vision-roBOTic)
MSc Business Innovation and Technology Management
**
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot
*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Gael Varoquaux
2016-02-22 17:41:41 UTC
Permalink
Post by Andreas Mueller
For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here https://github.com/amueller/segmentation/blob/master/bow.py#
L184)
Any chance that this can go in an example? I guess this depends on
whether we can find a dataset that is simple, yet can illustrate visual
BoWs.

G
Andreas Mueller
2016-02-22 18:37:41 UTC
Permalink
Post by Gael Varoquaux
Post by Andreas Mueller
For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here https://github.com/amueller/segmentation/blob/master/bow.py#
L184)
Any chance that this can go in an example? I guess this depends on
whether we can find a dataset that is simple, yet can illustrate visual
BoWs.
You need a supervised learning datasets, and need to extract features
for each image.
Not sure how to extract features without skimage, unless we use some
sparse coding for that.
And that would probably take a long time.
I guess the most simple datasets for this would be cifar 10 and caltech 101.
And it would probably not run fast enough for a plot example.
Guillaume Lemaitre
2016-02-22 19:04:27 UTC
Permalink
Maybe the simplest one should be to have texton (patches 9x9) with a PCA behind then the clustering. That would be the one without skimage dependences.

Guillaume Lemaitre 
PhD candidate
MSc Erasmus Mundus Vision and robotic (ViBOT)
Master in Business Innovation and Technology Management (BITM)
Univertité de Bourgogne - LE2I
Universitat de Girona - ViCOROB
  Original Message  
From: Andreas Mueller
Sent: Monday, 22 February 2016 19:38
To: scikit-learn-***@lists.sourceforge.net
Reply To: scikit-learn-***@lists.sourceforge.net
Subject: Re: [Scikit-learn-general] Implementation of Bag-of-Features
Post by Gael Varoquaux
Post by Andreas Mueller
For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here https://github.com/amueller/segmentation/blob/master/bow.py#
L184)
Any chance that this can go in an example? I guess this depends on
whether we can find a dataset that is simple, yet can illustrate visual
BoWs.
You need a supervised learning datasets, and need to extract features
for each image.
Not sure how to extract features without skimage, unless we use some
sparse coding for that.
And that would probably take a long time.
I guess the most simple datasets for this would be cifar 10 and caltech 101.
And it would probably not run fast enough for a plot example.

------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Andreas Mueller
2016-02-22 19:10:40 UTC
Permalink
Post by Guillaume Lemaitre
Maybe the simplest one should be to have texton (patches 9x9) with a PCA behind then the clustering. That would be the one without skimage dependences.
yeah... but what dataset? Actually one that has unequal size images
would be nice, but I'm not sure if I want a caltech 101 loader in sklearn ;)
Guillaume Lemaître
2016-02-23 10:41:32 UTC
Permalink
That's a point :D
What would be the requirement regarding the dataset. Does it need to be
academic dependent?

Other solution kinda crazy: scikit-learn currently has 555 contributors. If
each of them take picture of some objects predefined, it would be actually
possible to get a small set just to illustrate such concept.
Post by Guillaume Lemaitre
Post by Guillaume Lemaitre
Maybe the simplest one should be to have texton (patches 9x9) with a PCA
behind then the clustering. That would be the one without skimage
dependences.
yeah... but what dataset? Actually one that has unequal size images
would be nice, but I'm not sure if I want a caltech 101 loader in sklearn ;)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Andreas Mueller
2016-02-23 16:01:06 UTC
Permalink
Post by Guillaume Lemaître
That's a point :D
What would be the requirement regarding the dataset. Does it need to
be academic dependent?
Other solution kinda crazy: scikit-learn currently has 555
contributors. If each of them take picture of some objects predefined,
it would be actually possible to get a small set just to illustrate
such concept.
I wish I knew a way to make all 555 contributors read an email ;)
And creating a dataset is even more work than downloading an existing one...
I guess we could download some smaller dataset.

Maybe cows vs cars vs motorcycles?
http://host.robots.ox.ac.uk/pascal/VOC/databases.html#TUD

It's 50 mb.
How long do you think it'll take to generate textons from that?
Guillaume Lemaitre
2016-02-23 16:49:47 UTC
Permalink
I will make a try to have  an accurate estimate. Give me one or two days.

Guillaume Lemaitre 
PhD candidate
MSc Erasmus Mundus Vision and robotic (ViBOT)‎
Master in Business Innovation and Technology Management (BITM)
Univertité de Bourgogne - LE2I
Universitat de Girona - ViCOROB
  Original Message  
From: Andreas Mueller
Sent: Tuesday, 23 February 2016 17:02
To: scikit-learn-***@lists.sourceforge.net
Reply To: scikit-learn-***@lists.sourceforge.net
Subject: Re: [Scikit-learn-general] Implementation of Bag-of-Features
Post by Guillaume Lemaître
That's a point :D
What would be the requirement regarding the dataset. Does it need to
be academic dependent?
Other solution kinda crazy: scikit-learn currently has 555
contributors. If each of them take picture of some objects predefined,
it would be actually possible to get a small set just to illustrate
such concept.
I wish I knew a way to make all 555 contributors read an email ;)
And creating a dataset is even more work than downloading an existing one...
I guess we could download some smaller dataset.

Maybe cows vs cars vs motorcycles?
http://host.robots.ox.ac.uk/pascal/VOC/databases.html#TUD

It's 50 mb.
How long do you think it'll take to generate textons from that?


------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-***@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Nadim Farhat
2016-02-23 19:39:16 UTC
Permalink
HI Andreas,

Sorry for Jumping into the conversation and getting a bit off topic, what
does it mean "flat data" sets in sklearn ?

Bests

Nadim Farhat
Phd Bioengineering candidate
Center for Ultrasound and therapeutics
University of Pittsburgh
Post by Andreas Mueller
Hi Guillaume.
I was a big user of BoW myself, but I don't think it should go into
scikit-learn.
BoW doesn't really operate on a "flat" dataset, as scikit-learn usually
does. It works on groups of data points.
Each sample is usually a concatenation of feature vectors, which you
summarize as a histogram.
That doesn't really fit into the scikit-learn API.
For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here
https://github.com/amueller/segmentation/blob/master/bow.py#L184)
Cheers,
Andy
Dear all,
My group and I, are currently working on image classification applied to
medical images. We are using the Bag-of-Features (or Bag-of-Visual-Words,
Bag-of-Words) which was inspired originally from the text classification.
In fact, we have a kind of dirty implementation [here](
<https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py>
https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py)
which I would like to, somehow, even only if it is for a personal branch,
integrate to the scikit-learn.
However, I have some philosophical questions before to mess around, which
in fact are feeding some discussions in our lab. Checking the API, the BoF
approach could be part of the `feature_extraction` module. BoF is really
similar to the implementation of the BoW for text as previously mentioned.
Nevertheless, I am questioning if the BoF shall rather not be integrated
(i) dictionary learning (base K-Means, Mean-Shift, etc.), (ii) encoding (or
voting in that case using k-NN), and (iii) pooling (histogram).
Thus, in some sort the BoF can be seen as any of the decomposition (even
more similar to sparse coding). For instance the sparse learning follow
exactly the same scheme: dictionary learning with K-SVD, encoding, and
pooling (min/max/etc.). Similar thing for PCA, if you tackle the problem of
dictionary as finding the eigenvectors/eigenvalues.
- what are you thinking about such thing;
- where the BoF implementation of this approach is the most judicious;
- would it be judicious to think about the different decomposition methods
as the three steps earlier mentioned or it would be not at all intuitive?
Hope that the topic is not to weird.
Cheers,
--
*LEMAÎTRE Guillaume PhD Candidate MSc Erasmus Mundus ViBOT
(Vision-roBOTic) MSc Business Innovation and Technology Management *
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
<http://vicorob.udg.es/>http://vicorob.udg.es/
*LE2I - Le Creusot *IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
<http://le2i.cnrs.fr>http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Guillaume Lemaitre
2016-02-23 21:32:54 UTC
Permalink
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
Andy
2016-02-23 23:41:47 UTC
Permalink
Since that I was working on a cluster I did not realize but loading
all the image in memory will be problematic with a laptop-desktop
configuration.
Or we can learn the PCA projection on a subset and to apply the
dimension reduction right after the patch extraction. However, I am
not sure that all data will fit in memory.
We have out of core versions for PCA and KMeans.

I think the way I'd do it is to go over all images, extract only a
couple of patches from each image, store them.
After we have some patches from all images, I'd learn the PCA model.
Then we can go over the data again, transforming the patches. If they
don't fit into memory after dimensionality reduction, we can
use minibatch k-means to do the clustering without loading all the data.
then we need to go over the data one more time to get the cluster
centers and compute the BoW (which will fit in memory)
Guillaume Lemaitre
2016-02-24 05:19:19 UTC
Permalink
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
Guillaume Lemaître
2016-03-08 20:21:52 UTC
Permalink
Hi,

I made a pull-request with the draft:
https://github.com/scikit-learn/scikit-learn/pull/6509
Extracting the feature is taking a honest amount of time (around 30 sec.)
The codebook generation through MiniBatchKMeans is more problematic. I am
still running it but it could be a couple of hours.

Let me know what do you think about it,

Cheers,
Since that I was working on a cluster I did not realize but loading all
the image in memory will be problematic with a laptop-desktop configuration.
Or we can learn the PCA projection on a subset and to apply the dimension
reduction right after the patch extraction. However, I am not sure that all
data will fit in memory.
We have out of core versions for PCA and KMeans.
I think the way I'd do it is to go over all images, extract only a couple
of patches from each image, store them.
After we have some patches from all images, I'd learn the PCA model.
Then we can go over the data again, transforming the patches. If they
don't fit into memory after dimensionality reduction, we can
use minibatch k-means to do the clustering without loading all the data.
then we need to go over the data one more time to get the cluster centers
and compute the BoW (which will fit in memory)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Andreas Mueller
2016-03-08 21:09:42 UTC
Permalink
Hey Guillaume.
If it is a couple of hours, I'm not sure it is worth adding.
You can probably aggressively subsample or just do fewer iterations
(like, one pass over the data)
How do you run MiniBatchKMeans?

Cheers,
Andy
Post by Guillaume Lemaître
Hi,
https://github.com/scikit-learn/scikit-learn/pull/6509
Extracting the feature is taking a honest amount of time (around 30 sec.)
The codebook generation through MiniBatchKMeans is more problematic. I
am still running it but it could be a couple of hours.
Let me know what do you think about it,
Cheers,
Since that I was working on a cluster I did not realize but
loading all the image in memory will be problematic with a
laptop-desktop configuration.
Or we can learn the PCA projection on a subset and to apply the
dimension reduction right after the patch extraction. However, I
am not sure that all data will fit in memory.
We have out of core versions for PCA and KMeans.
I think the way I'd do it is to go over all images, extract only a
couple of patches from each image, store them.
After we have some patches from all images, I'd learn the PCA model.
Then we can go over the data again, transforming the patches. If
they don't fit into memory after dimensionality reduction, we can
use minibatch k-means to do the clustering without loading all the data.
then we need to go over the data one more time to get the cluster
centers and compute the BoW (which will fit in memory)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE Guillaume
PhD Candidate
MSc Erasmus Mundus ViBOT (Vision-roBOTic)
MSc Business Innovation and Technology Management
**
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot
*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://makebettercode.com/inteldaal-eval
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Guillaume Lemaître
2016-03-08 22:23:16 UTC
Permalink
Sorry I was wrong. The MiniBatchKMeans converge after 20 minutes.
So for one iteration of the CV, I get something like that:

Classification performed
[[21 2 0]
[ 0 20 0]
[ 0 0 23]]
It took 1253.23589396 seconds.

Probably this is not desirable to have a cross-validation. I don't know if
you consider 20 minutes as reasonable?
Post by Andreas Mueller
Hey Guillaume.
If it is a couple of hours, I'm not sure it is worth adding.
You can probably aggressively subsample or just do fewer iterations (like,
one pass over the data)
How do you run MiniBatchKMeans?
Cheers,
Andy
Hi,
<https://github.com/scikit-learn/scikit-learn/pull/6509>
https://github.com/scikit-learn/scikit-learn/pull/6509
Extracting the feature is taking a honest amount of time (around 30 sec.)
The codebook generation through MiniBatchKMeans is more problematic. I am
still running it but it could be a couple of hours.
Let me know what do you think about it,
Cheers,
Since that I was working on a cluster I did not realize but loading all
the image in memory will be problematic with a laptop-desktop configuration.
Or we can learn the PCA projection on a subset and to apply the dimension
reduction right after the patch extraction. However, I am not sure that all
data will fit in memory.
We have out of core versions for PCA and KMeans.
I think the way I'd do it is to go over all images, extract only a couple
of patches from each image, store them.
After we have some patches from all images, I'd learn the PCA model.
Then we can go over the data again, transforming the patches. If they
don't fit into memory after dimensionality reduction, we can
use minibatch k-means to do the clustering without loading all the data.
then we need to go over the data one more time to get the cluster centers
and compute the BoW (which will fit in memory)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE Guillaume PhD Candidate MSc Erasmus Mundus ViBOT
(Vision-roBOTic) MSc Business Innovation and Technology Management *
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot *IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.http://makebettercode.com/inteldaal-eval
_______________________________________________
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://makebettercode.com/inteldaal-eval
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Guillaume Lemaître
2016-03-08 22:44:15 UTC
Permalink
Regarding the MiniBatchKMeans, I use the following parameters

MiniBatchKMeans(n_clusters=nb_words, verbose=1, init='random', batch_size=10
* nb_words, compute_labels=False, reassignment_ratio=0.0, random_state=1,
n_init=3)

With 1000 words. I am not sure about the batch size as well as the
initialisation. Does 'k-means++' should improve the convergence with the
mini-batch?
Post by Guillaume Lemaître
Sorry I was wrong. The MiniBatchKMeans converge after 20 minutes.
Classification performed
[[21 2 0]
[ 0 20 0]
[ 0 0 23]]
It took 1253.23589396 seconds.
Probably this is not desirable to have a cross-validation. I don't know if
you consider 20 minutes as reasonable?
Post by Andreas Mueller
Hey Guillaume.
If it is a couple of hours, I'm not sure it is worth adding.
You can probably aggressively subsample or just do fewer iterations
(like, one pass over the data)
How do you run MiniBatchKMeans?
Cheers,
Andy
Hi,
<https://github.com/scikit-learn/scikit-learn/pull/6509>
https://github.com/scikit-learn/scikit-learn/pull/6509
Extracting the feature is taking a honest amount of time (around 30 sec.)
The codebook generation through MiniBatchKMeans is more problematic. I am
still running it but it could be a couple of hours.
Let me know what do you think about it,
Cheers,
Since that I was working on a cluster I did not realize but loading all
the image in memory will be problematic with a laptop-desktop configuration.
Or we can learn the PCA projection on a subset and to apply the
dimension reduction right after the patch extraction. However, I am not
sure that all data will fit in memory.
We have out of core versions for PCA and KMeans.
I think the way I'd do it is to go over all images, extract only a
couple of patches from each image, store them.
After we have some patches from all images, I'd learn the PCA model.
Then we can go over the data again, transforming the patches. If they
don't fit into memory after dimensionality reduction, we can
use minibatch k-means to do the clustering without loading all the data.
then we need to go over the data one more time to get the cluster
centers and compute the BoW (which will fit in memory)
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE Guillaume PhD Candidate MSc Erasmus Mundus ViBOT
(Vision-roBOTic) MSc Business Innovation and Technology Management *
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot *IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.http://makebettercode.com/inteldaal-eval
_______________________________________________
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://makebettercode.com/inteldaal-eval
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Guillaume Lemaître
2016-02-29 20:15:12 UTC
Permalink
Hi guys,

I got a bit of time meanwhile in a conference:
https://github.com/glemaitre/bow-sklearn-example
The extraction is in the source. I have to include the part to measure the
timing. Let me know of that make sense.

Cheers,
Post by Nadim Farhat
HI Andreas,
Sorry for Jumping into the conversation and getting a bit off topic, what
does it mean "flat data" sets in sklearn ?
Bests
Nadim Farhat
Phd Bioengineering candidate
Center for Ultrasound and therapeutics
University of Pittsburgh
Post by Andreas Mueller
Hi Guillaume.
I was a big user of BoW myself, but I don't think it should go into
scikit-learn.
BoW doesn't really operate on a "flat" dataset, as scikit-learn usually
does. It works on groups of data points.
Each sample is usually a concatenation of feature vectors, which you
summarize as a histogram.
That doesn't really fit into the scikit-learn API.
For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here
https://github.com/amueller/segmentation/blob/master/bow.py#L184)
Cheers,
Andy
Dear all,
My group and I, are currently working on image classification applied to
medical images. We are using the Bag-of-Features (or Bag-of-Visual-Words,
Bag-of-Words) which was inspired originally from the text classification.
In fact, we have a kind of dirty implementation [here](
<https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py>
https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py)
which I would like to, somehow, even only if it is for a personal branch,
integrate to the scikit-learn.
However, I have some philosophical questions before to mess around, which
in fact are feeding some discussions in our lab. Checking the API, the BoF
approach could be part of the `feature_extraction` module. BoF is really
similar to the implementation of the BoW for text as previously mentioned.
Nevertheless, I am questioning if the BoF shall rather not be integrated
(i) dictionary learning (base K-Means, Mean-Shift, etc.), (ii) encoding (or
voting in that case using k-NN), and (iii) pooling (histogram).
Thus, in some sort the BoF can be seen as any of the decomposition (even
more similar to sparse coding). For instance the sparse learning follow
exactly the same scheme: dictionary learning with K-SVD, encoding, and
pooling (min/max/etc.). Similar thing for PCA, if you tackle the problem of
dictionary as finding the eigenvectors/eigenvalues.
- what are you thinking about such thing;
- where the BoF implementation of this approach is the most judicious;
- would it be judicious to think about the different decomposition
methods as the three steps earlier mentioned or it would be not at all
intuitive?
Hope that the topic is not to weird.
Cheers,
--
*LEMAÎTRE Guillaume PhD Candidate MSc Erasmus Mundus ViBOT
(Vision-roBOTic) MSc Business Innovation and Technology Management *
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
<http://vicorob.udg.es/>http://vicorob.udg.es/
*LE2I - Le Creusot *IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
<http://le2i.cnrs.fr>http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Andreas Mueller
2016-02-29 20:56:08 UTC
Permalink
Can you open a PR with an example and attach the plots and output and
timing?
The example needs to download the dataset.

Cheers,
Andy
Post by Guillaume Lemaître
Hi guys,
https://github.com/glemaitre/bow-sklearn-example
The extraction is in the source. I have to include the part to measure
the timing. Let me know of that make sense.
Cheers,
HI Andreas,
Sorry for Jumping into the conversation and getting a bit off
topic, what does it mean "flat data" sets in sklearn ?
Bests
Nadim Farhat
Phd Bioengineering candidate
Center for Ultrasound and therapeutics
University of Pittsburgh
Hi Guillaume.
I was a big user of BoW myself, but I don't think it should go
into scikit-learn.
BoW doesn't really operate on a "flat" dataset, as
scikit-learn usually does. It works on groups of data points.
Each sample is usually a concatenation of feature vectors,
which you summarize as a histogram.
That doesn't really fit into the scikit-learn API.
For any particular application (I did bag of visual words),
creating an implementation using the kmeans or sparse coding
in scikit-learn
is only a couple of lines (you can find my visual bow for
per-superpixel descriptors here
https://github.com/amueller/segmentation/blob/master/bow.py#L184)
Cheers,
Andy
Post by Guillaume Lemaître
Dear all,
My group and I, are currently working on image classification
applied to medical images. We are using the Bag-of-Features
(or Bag-of-Visual-Words, Bag-of-Words) which was inspired
originally from the text classification. In fact, we have a
kind of dirty implementation
[here](https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py)
which I would like to, somehow, even only if it is for a
personal branch, integrate to the scikit-learn.
However, I have some philosophical questions before to mess
around, which in fact are feeding some discussions in our
lab. Checking the API, the BoF approach could be part of the
`feature_extraction` module. BoF is really similar to the
implementation of the BoW for text as previously mentioned.
Nevertheless, I am questioning if the BoF shall rather not be
integrated to the `decomposition` module. By looking at it,
the method consists of: (i) dictionary learning (base
K-Means, Mean-Shift, etc.), (ii) encoding (or voting in that
case using k-NN), and (iii) pooling (histogram).
Thus, in some sort the BoF can be seen as any of the
decomposition (even more similar to sparse coding). For
dictionary learning with K-SVD, encoding, and pooling
(min/max/etc.). Similar thing for PCA, if you tackle the
problem of dictionary as finding the eigenvectors/eigenvalues.
- what are you thinking about such thing;
- where the BoF implementation of this approach is the most judicious;
- would it be judicious to think about the different
decomposition methods as the three steps earlier mentioned or
it would be not at all intuitive?
Hope that the topic is not to weird.
Cheers,
--
*LEMAÎTRE Guillaume
PhD Candidate
MSc Erasmus Mundus ViBOT (Vision-roBOTic)
MSc Business Innovation and Technology Management
**
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 <tel:%2B34%20972%2041%2098%2012> - Fax.
+34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot
*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 <tel:%2B33%203%2085%2073%2010%2090> -
Fax. +33 3 85 73 10 97 <tel:%2B33%203%2085%2073%2010%2097>
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE Guillaume
PhD Candidate
MSc Erasmus Mundus ViBOT (Vision-roBOTic)
MSc Business Innovation and Technology Management
**
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot
*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
Guillaume Lemaître
2016-03-01 17:32:55 UTC
Permalink
Shall I download the dataset through the module dataset or just fetch
directly in the example? If so, do you have an example that I can see the
desired structure.

Cheers,
Post by Andreas Mueller
Can you open a PR with an example and attach the plots and output and
timing?
The example needs to download the dataset.
Cheers,
Andy
Hi guys,
<https://github.com/glemaitre/bow-sklearn-example>
https://github.com/glemaitre/bow-sklearn-example
The extraction is in the source. I have to include the part to measure the
timing. Let me know of that make sense.
Cheers,
Post by Nadim Farhat
HI Andreas,
Sorry for Jumping into the conversation and getting a bit off topic, what
does it mean "flat data" sets in sklearn ?
Bests
Nadim Farhat
Phd Bioengineering candidate
Center for Ultrasound and therapeutics
University of Pittsburgh
Post by Andreas Mueller
Hi Guillaume.
I was a big user of BoW myself, but I don't think it should go into
scikit-learn.
BoW doesn't really operate on a "flat" dataset, as scikit-learn usually
does. It works on groups of data points.
Each sample is usually a concatenation of feature vectors, which you
summarize as a histogram.
That doesn't really fit into the scikit-learn API.
For any particular application (I did bag of visual words), creating an
implementation using the kmeans or sparse coding in scikit-learn
is only a couple of lines (you can find my visual bow for per-superpixel
descriptors here
https://github.com/amueller/segmentation/blob/master/bow.py#L184)
Cheers,
Andy
Dear all,
My group and I, are currently working on image classification applied to
medical images. We are using the Bag-of-Features (or Bag-of-Visual-Words,
Bag-of-Words) which was inspired originally from the text classification.
In fact, we have a kind of dirty implementation [here](
<https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py>
https://github.com/glemaitre/protoclass/blob/master/protoclass/extraction/codebook.py)
which I would like to, somehow, even only if it is for a personal branch,
integrate to the scikit-learn.
However, I have some philosophical questions before to mess around,
which in fact are feeding some discussions in our lab. Checking the API,
the BoF approach could be part of the `feature_extraction` module. BoF is
really similar to the implementation of the BoW for text as previously
mentioned.
Nevertheless, I am questioning if the BoF shall rather not be integrated
(i) dictionary learning (base K-Means, Mean-Shift, etc.), (ii) encoding (or
voting in that case using k-NN), and (iii) pooling (histogram).
Thus, in some sort the BoF can be seen as any of the decomposition (even
more similar to sparse coding). For instance the sparse learning follow
exactly the same scheme: dictionary learning with K-SVD, encoding, and
pooling (min/max/etc.). Similar thing for PCA, if you tackle the problem of
dictionary as finding the eigenvectors/eigenvalues.
- what are you thinking about such thing;
- where the BoF implementation of this approach is the most judicious;
- would it be judicious to think about the different decomposition
methods as the three steps earlier mentioned or it would be not at all
intuitive?
Hope that the topic is not to weird.
Cheers,
--
*LEMAÎTRE Guillaume PhD Candidate MSc Erasmus Mundus ViBOT
(Vision-roBOTic) MSc Business Innovation and Technology Management *
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 <%2B34%20972%2041%2098%2012> - Fax. +34 972 41 82
59
<http://vicorob.udg.es/>http://vicorob.udg.es/
*LE2I - Le Creusot *IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 <%2B33%203%2085%2073%2010%2090> - Fax. +33 3 85
73 10 97 <%2B33%203%2085%2073%2010%2097>
<http://le2i.cnrs.fr>http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of <http://ascfours.free.fr>http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE Guillaume PhD Candidate MSc Erasmus Mundus ViBOT
(Vision-roBOTic) MSc Business Innovation and Technology Management *
*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/
*LE2I - Le Creusot *IUT Le Creusot, Laboratoire LE2I, 12 rue de la
Fonderie, 71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr
https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
------------------------------------------------------------------------------
Site24x7 APM Insight: Get Deep Visibility into Application Performance
APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
Monitor end-to-end web transactions and take corrective actions now
Troubleshoot faster and improve end-user experience. Signup Now!
http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
_______________________________________________
Scikit-learn-general mailing list
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
--
*LEMAÎTRE GuillaumePhD CandidateMSc Erasmus Mundus ViBOT
(Vision-roBOTic)MSc Business Innovation and Technology Management*
***@gmail.com

*ViCOROB - Computer Vision and Robotic Team*
Universitat de Girona, Campus Montilivi, Edifici P-IV 17071 Girona
Tel. +34 972 41 98 12 - Fax. +34 972 41 82 59
http://vicorob.udg.es/

*LE2I - Le Creusot*IUT Le Creusot, Laboratoire LE2I, 12 rue de la Fonderie,
71200 Le Creusot
Tel. +33 3 85 73 10 90 - Fax. +33 3 85 73 10 97
http://le2i.cnrs.fr

https://sites.google.com/site/glemaitre58/
Vice - Chairman of A.S.C. Fours UFOLEP
Chairman of A.S.C. Fours FFC
Webmaster of http://ascfours.free.fr
Loading...