Tests for empty arrays

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Tests for empty arrays

Andy Cheesman
Hi nice numpy people

I was wondering if anyone could shed some light on how to distinguish an
empty array of a given shape and an zeros array of the same dimensions.

Thanks
Andy

_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Tests for empty arrays

Tim Hochberg


On Tue, May 6, 2008 at 9:31 AM, Andy Cheesman <[hidden email]> wrote:
Hi nice numpy people

I was wondering if anyone could shed some light on how to distinguish an
empty array of a given shape and an zeros array of the same dimensions.

An empty array is just uninitialized, while a zeros array is initialized to zeros. Short of checking whether the zeros array is all zeros, which only tells you that it looks like it was coming from zeros; it still could have created by by empty or zeros or some other method.

Why do you need to know? If the array is coming from an unknown source, why not just use a.fill(0) to force everything to be zero and start from a known state?


 


Thanks
Andy

_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion



--
. __
. |-\
.
. [hidden email]
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Tests for empty arrays

Anne Archibald
In reply to this post by Andy Cheesman
2008/5/6 Andy Cheesman <[hidden email]>:

>  I was wondering if anyone could shed some light on how to distinguish an
>  empty array of a given shape and an zeros array of the same dimensions.

An "empty" array, that is, an array returned by the function empty(),
just means an uninitialized array. The only difference between it and
an array returned by zeros() is the values in the array: from zeros()
they are, obviously, all zero, while from empty() they can be
anything, including zero(). In practice they tend to be crazy numbers
of order 1e300 or 1e-300, but one can't count on this.

As a general rule, I recommend against using empty() on your first
pass through the code. Only once your code is working and you have
tests running should you consider switching to empty() for speed
reasons. In fact, if you want to use empty() down the road, it may
make sense to initialize your array to zeros()/0., so that if you ever
use the values, the NaNs will propagate and become obvious.
Unfortunately, some CPUs implement NaNs in software, so there can be a
huge speed hit.

Anne
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Tests for empty arrays

Keith Goodman
On Tue, May 6, 2008 at 9:45 AM, Anne Archibald
<[hidden email]> wrote:
>  In fact, if you want to use empty() down the road, it may
>  make sense to initialize your array to zeros()/0., so that if you ever
>  use the values, the NaNs will propagate and become obvious.

Numpy has ones and zeros. Could we add a nans?

I often initialize using x = nan * ones((n ,m)). But if it's in a
loop, I'll avoid one copy by doing

x = np.ones((n, m))
x *= np.nan

To many on the list using nans for missing values is like chewing gum
you found on the sidewalk. But I use it all the time so I'd use a
nans.
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Tests for empty arrays

Tim Hochberg


On Tue, May 6, 2008 at 9:53 AM, Keith Goodman <[hidden email]> wrote:
On Tue, May 6, 2008 at 9:45 AM, Anne Archibald
<[hidden email]> wrote:
>  In fact, if you want to use empty() down the road, it may
>  make sense to initialize your array to zeros()/0., so that if you ever
>  use the values, the NaNs will propagate and become obvious.

Numpy has ones and zeros. Could we add a nans?

I often initialize using x = nan * ones((n ,m)). But if it's in a
loop, I'll avoid one copy by doing

x = np.ones((n, m))
x *= np.nan

To many on the list using nans for missing values is like chewing gum
you found on the sidewalk. But I use it all the time so I'd use a
nans.

Why don't you just roll your own?

>>> def nans(shape, dtype=float):
...     a = np.empty(shape, dtype)
...     a.fill(np.nan)
...     return a
...
>>> nans([3,4])
array([[ NaN,  NaN,  NaN,  NaN],
       [ NaN,  NaN,  NaN,  NaN],
       [ NaN,  NaN,  NaN,  NaN]])
 

--
. __
. |-\
.
. [hidden email]
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Tests for empty arrays

Keith Goodman
On Tue, May 6, 2008 at 10:03 AM, Timothy Hochberg <[hidden email]> wrote:

> Why don't you just roll your own?
>
> >>> def nans(shape, dtype=float):
> ...     a = np.empty(shape, dtype)
> ...     a.fill(np.nan)
> ...     return a
>  ...
> >>> nans([3,4])
> array([[ NaN,  NaN,  NaN,  NaN],
>        [ NaN,  NaN,  NaN,  NaN],
>        [ NaN,  NaN,  NaN,  NaN]])

I learn a lot from this list. I didn't know about fill. Looks like it
is much faster than adding nan.

>> timeit nans0((500,500))
10 loops, best of 3: 30.5 ms per loop
>> timeit nans1((500,500))
1000 loops, best of 3: 956 ┬Ás per loop

def nans0(shape, dtype=float):
    a = np.ones(shape, dtype)
    a += np.nan
    return a

def nans1(shape, dtype=float):
    a = np.empty(shape, dtype)
    a.fill(np.nan)

No need to roll my own. I'll smoke yours.
    return a
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion