# Fancier indexing Classic List Threaded 8 messages Open this post in threaded view
|

## Fancier indexing

 After poking around for a bit, I was wondering if there was a faster method for the following:# Array of index values 0..nitems = numpy.array([0,3,2,1,4,2],dtype=int) # Count the number of occurrences of each indexcounts = numpy.zeros(5, dtype=int)for i in items:  counts[i] += 1In my real code, 'items' contain up to a million values and this loop will be in a performance critical area of code.  If there is no simple solution, I can trivially code this using the C-API. Thanks,-Kevin _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Fancier indexing

 On Thu, May 22, 2008 at 8:59 AM, Kevin Jacobs <[hidden email]> <[hidden email]> wrote: > After poking around for a bit, I was wondering if there was a faster method > for the following: > > # Array of index values 0..n > items = numpy.array([0,3,2,1,4,2],dtype=int) > > # Count the number of occurrences of each index > counts = numpy.zeros(5, dtype=int) > for i in items: >   counts[i] += 1 > > In my real code, 'items' contain up to a million values and this loop will > be in a performance critical area of code.  If there is no simple solution, > I can trivially code this using the C-API. How big is n? If it is much smaller than a million then loop over that instead. _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Fancier indexing

 On Thu, May 22, 2008 at 12:08 PM, Keith Goodman <[hidden email]> wrote: How big is n? If it is much smaller than a million then loop over that instead. n is always relatively small, but I'd rather not do:for i in range(n):  counts[i] = (items==i).sum() If that was the best alternative, I'd just bite the bullet and code this in C.Thanks,-Kevin _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Fancier indexing

 In reply to this post by Keith Goodman On Thu, May 22, 2008 at 9:08 AM, Keith Goodman <[hidden email]> wrote: > On Thu, May 22, 2008 at 8:59 AM, Kevin Jacobs <[hidden email]> > <[hidden email]> wrote: >> After poking around for a bit, I was wondering if there was a faster method >> for the following: >> >> # Array of index values 0..n >> items = numpy.array([0,3,2,1,4,2],dtype=int) >> >> # Count the number of occurrences of each index >> counts = numpy.zeros(5, dtype=int) >> for i in items: >>   counts[i] += 1 >> >> In my real code, 'items' contain up to a million values and this loop will >> be in a performance critical area of code.  If there is no simple solution, >> I can trivially code this using the C-API. > > How big is n? If it is much smaller than a million then loop over that instead. Or how about using a list instead: >> items = [0,3,2,1,4,2] >> uitems = frozenset(items) >> count = [items.count(i) for i in uitems] >> count    [1, 1, 2, 1, 1] _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Fancier indexing

 In reply to this post by Keith Goodman You're just trying to do this...correct?>>> import numpy>>> items = numpy.array([0,3,2,1,4,2],dtype=int)>>> unique = numpy.unique(items)>>> uniquearray([0, 1, 2, 3, 4]) >>> counts=numpy.histogram(items,unique)>>> counts(array([1, 1, 2, 1, 1]), array([0, 1, 2, 3, 4]))>>> countsarray([1, 1, 2, 1, 1])>>> On Thu, May 22, 2008 at 9:08 AM, Keith Goodman <[hidden email]> wrote: On Thu, May 22, 2008 at 8:59 AM, Kevin Jacobs <[hidden email]> <[hidden email]> wrote: > After poking around for a bit, I was wondering if there was a faster method > for the following: > > # Array of index values 0..n > items = numpy.array([0,3,2,1,4,2],dtype=int) > > # Count the number of occurrences of each index > counts = numpy.zeros(5, dtype=int) > for i in items: >   counts[i] += 1 > > In my real code, 'items' contain up to a million values and this loop will > be in a performance critical area of code.  If there is no simple solution, > I can trivially code this using the C-API. How big is n? If it is much smaller than a million then loop over that instead. _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Fancier indexing

 In reply to this post by Keith Goodman On Thu, May 22, 2008 at 9:15 AM, Keith Goodman <[hidden email]> wrote: > On Thu, May 22, 2008 at 9:08 AM, Keith Goodman <[hidden email]> wrote: >> On Thu, May 22, 2008 at 8:59 AM, Kevin Jacobs <[hidden email]> >> <[hidden email]> wrote: >>> After poking around for a bit, I was wondering if there was a faster method >>> for the following: >>> >>> # Array of index values 0..n >>> items = numpy.array([0,3,2,1,4,2],dtype=int) >>> >>> # Count the number of occurrences of each index >>> counts = numpy.zeros(5, dtype=int) >>> for i in items: >>>   counts[i] += 1 >>> >>> In my real code, 'items' contain up to a million values and this loop will >>> be in a performance critical area of code.  If there is no simple solution, >>> I can trivially code this using the C-API. >> >> How big is n? If it is much smaller than a million then loop over that instead. > > Or how about using a list instead: > >>> items = [0,3,2,1,4,2] >>> uitems = frozenset(items) >>> count = [items.count(i) for i in uitems] >>> count >   [1, 1, 2, 1, 1] Oh, I see, so uitems should be range(n) _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion