How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?

Martin.Gfeller
Dear all

I have object array of arrays, which I compare element-wise to None in various places:

>>> a = numpy.array([numpy.arange(5),None,numpy.nan,numpy.arange(6),None],dtype=numpy.object)
>>> a
array([array([0, 1, 2, 3, 4]), None, nan, array([0, 1, 2, 3, 4, 5]), None], dtype=object)
>>> numpy.equal(a,None)
FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.


So far, I always ignored the warning, for lack of an idea how to resolve it.

Now, with Numpy 1.13, I have to resolve the issue, because it fails with:

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

It seem that the numpy.equal is applied to each inner array, returning a Boolean array for each element, which cannot be coerced to a single Boolean.

The expression

>>> numpy.vectorize(operator.is_)(a,None)

gives the desired result, but feels a bit clumsy.

Is there a cleaner, efficient way to do an element-wise (but shallow) comparison?

Thank you and best regards,
Martin Gfeller, Swisscom

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?

Sebastian Berg
On Mon, 2017-07-17 at 09:13 +0000, [hidden email] wrote:

> Dear all
>
> I have object array of arrays, which I compare element-wise to None
> in various places:
>
> > > > a =
> > > > numpy.array([numpy.arange(5),None,numpy.nan,numpy.arange(6),Non
> > > > e],dtype=numpy.object)
> > > > a
>
> array([array([0, 1, 2, 3, 4]), None, nan, array([0, 1, 2, 3, 4, 5]),
> None], dtype=object)
> > > > numpy.equal(a,None)
>
> FutureWarning: comparison to `None` will result in an elementwise
> object comparison in the future.
>
>
> So far, I always ignored the warning, for lack of an idea how to
> resolve it. 
>
> Now, with Numpy 1.13, I have to resolve the issue, because it fails
> with: 
>
> ValueError: The truth value of an array with more than one element is
> ambiguous. Use a.any() or a.all() 
>
> It seem that the numpy.equal is applied to each inner array,
> returning a Boolean array for each element, which cannot be coerced
> to a single Boolean.
>
> The expression 
>
> > > > numpy.vectorize(operator.is_)(a,None)
>
> gives the desired result, but feels a bit clumsy. 
>
Yes, I guess ones bug is someone elses feature :(, if it is very bad,
we could delay the deprecation probably. For a solutions, maybe we
could add a ufunc  for elementwise `is` on object arrays (dunno about
the name, maybe `object_identity`.

Just some quick thoughts.

- Sebastian


> Is there a cleaner, efficient way to do an element-wise (but shallow)
> comparison? 
>
> Thank you and best regards,
> Martin Gfeller, Swisscom
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

signature.asc (817 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?

Robert Kern-2
In reply to this post by Martin.Gfeller
On Mon, Jul 17, 2017 at 2:13 AM, <[hidden email]> wrote:

>
> Dear all
>
> I have object array of arrays, which I compare element-wise to None in various places:
>
> >>> a = numpy.array([numpy.arange(5),None,numpy.nan,numpy.arange(6),None],dtype=numpy.object)
> >>> a
> array([array([0, 1, 2, 3, 4]), None, nan, array([0, 1, 2, 3, 4, 5]), None], dtype=object)
> >>> numpy.equal(a,None)
> FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
>
>
> So far, I always ignored the warning, for lack of an idea how to resolve it.
>
> Now, with Numpy 1.13, I have to resolve the issue, because it fails with:
>
> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
>
> It seem that the numpy.equal is applied to each inner array, returning a Boolean array for each element, which cannot be coerced to a single Boolean.
>
> The expression
>
> >>> numpy.vectorize(operator.is_)(a,None)
>
> gives the desired result, but feels a bit clumsy.

Wrap the clumsiness up in a documented, tested utility function with a descriptive name and use that function everywhere instead.

--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?

Eric Wieser

Here’s a hack that lets you keep using ==:

class IsCompare:
    __array_priority__ = 999999  # needed to make it work on either side of `==`
    def __init__(self, val): self._val = val
    def __eq__(self, other): return other is self._val
    def __neq__(self, other): return other is not self._val

a == IsCompare(None)  # a is None
a == np.array(IsCompare(None)) # broadcasted a is None

Eric


On Mon, 17 Jul 2017 at 17:45 Robert Kern <[hidden email]> wrote:
On Mon, Jul 17, 2017 at 2:13 AM, <[hidden email]> wrote:

>
> Dear all
>
> I have object array of arrays, which I compare element-wise to None in various places:
>
> >>> a = numpy.array([numpy.arange(5),None,numpy.nan,numpy.arange(6),None],dtype=numpy.object)
> >>> a
> array([array([0, 1, 2, 3, 4]), None, nan, array([0, 1, 2, 3, 4, 5]), None], dtype=object)
> >>> numpy.equal(a,None)
> FutureWarning: comparison to `None` will result in an elementwise object comparison in the future.
>
>
> So far, I always ignored the warning, for lack of an idea how to resolve it.
>
> Now, with Numpy 1.13, I have to resolve the issue, because it fails with:
>
> ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
>
> It seem that the numpy.equal is applied to each inner array, returning a Boolean array for each element, which cannot be coerced to a single Boolean.
>
> The expression
>
> >>> numpy.vectorize(operator.is_)(a,None)
>
> gives the desired result, but feels a bit clumsy.

Wrap the clumsiness up in a documented, tested utility function with a descriptive name and use that function everywhere instead.

--
Robert Kern
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: How to compare an array of arrays elementwise to None in Numpy 1.13 (was easy before)?

Robert Kern-2
On Mon, Jul 17, 2017 at 10:52 AM, Eric Wieser <[hidden email]> wrote:

Here’s a hack that lets you keep using ==:

class IsCompare:
    __array_priority__ = 999999  # needed to make it work on either side of `==`
    def __init__(self, val): self._val = val
    def __eq__(self, other): return other is self._val
    def __neq__(self, other): return other is not self._val

a == IsCompare(None)  # a is None
a == np.array(IsCompare(None)) # broadcasted a is None

Frankly, I'd stick with a well-named utility function. It's much more kind to those who have to read the code (e.g. you in 6 months). :-)

--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Loading...