Truth value of an array

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

Truth value of an array

Olivier Verdier-2
In mathematics, if I compare two function, it means that I compare on
all its "coordinates". If I say "f < g" I mean "f(x) < g(x) for all
x".

The same holds for a vector, if I write "v == w" I mean "v[i] == w[i]
for all i".

How come this doesn't work in numpy? And why the message about the
truth value of an array being ambiguous? What is ambiguous?

In my opinion any boolean array should be cast with all()
automatically so that people can simply write:
if v == w:
  ...

Or is there a good reason why this is not possible?

Thank you!
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

David Douard-2
On Fri, Apr 18, 2008 at 01:11:37PM +0200, Olivier Verdier wrote:

> In mathematics, if I compare two function, it means that I compare on
> all its "coordinates". If I say "f < g" I mean "f(x) < g(x) for all
> x".
>
> The same holds for a vector, if I write "v == w" I mean "v[i] == w[i]
> for all i".
>
> How come this doesn't work in numpy? And why the message about the
> truth value of an array being ambiguous? What is ambiguous?
>
> In my opinion any boolean array should be cast with all()
> automatically so that people can simply write:
> if v == w:
>   ...
No. A comparison operator on ndarray produce an ndarray of booleans. So
you have to write

if numpy.alltrue(u == v):
   blah

This is a very good idea, since it allows you to also write things
like:

u[v<0] = 0

Which you could definitely not do if the comparison returns a scalar
bool instead of a ndarray.

More, you may also want to write things like:

if numpy.sometrue(u<0):
   blah

Same remark.



>
> Or is there a good reason why this is not possible?
>
> Thank you!
> _______________________________________________
> Numpy-discussion mailing list
> [hidden email]
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>

--
David Douard                        LOGILAB, Paris (France), +33 1 45 32 03 12
Formations Python, Zope, Debian :   http://www.logilab.fr/formations
Développement logiciel sur mesure : http://www.logilab.fr/services
Informatique scientifique :         http://www.logilab.fr/science

_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Olivier Verdier-2
I certainly didn't mean that "A==B" should return a boolean!!

"A==B" should return an array of boolean as it does now. This is all right.

*However* "bool(A==B)" should return a boolean, *not* raise an
exception. Why raise an exception? What is ambiguous about
"bool(A==B)"??

This is what happens when you write "if A==B" because then
"bool(A==B)" is somehow triggered and bang, an exception is raised.

As I tried to explain, the default behaviour should be that "bool(A)"
return "A.all()" but not an exception. Why is there an exception
raising at all?

I hope that my question is clearer now...

Thanks.


On 18/04/2008, David Douard <[hidden email]> wrote:

> On Fri, Apr 18, 2008 at 01:11:37PM +0200, Olivier Verdier wrote:
>  > In mathematics, if I compare two function, it means that I compare on
>  > all its "coordinates". If I say "f < g" I mean "f(x) < g(x) for all
>  > x".
>  >
>  > The same holds for a vector, if I write "v == w" I mean "v[i] == w[i]
>  > for all i".
>  >
>  > How come this doesn't work in numpy? And why the message about the
>  > truth value of an array being ambiguous? What is ambiguous?
>  >
>  > In my opinion any boolean array should be cast with all()
>  > automatically so that people can simply write:
>  > if v == w:
>  >   ...
>
>
> No. A comparison operator on ndarray produce an ndarray of booleans. So
>  you have to write
>
>  if numpy.alltrue(u == v):
>    blah
>
>  This is a very good idea, since it allows you to also write things
>  like:
>
>  u[v<0] = 0
>
>  Which you could definitely not do if the comparison returns a scalar
>  bool instead of a ndarray.
>
>  More, you may also want to write things like:
>
>  if numpy.sometrue(u<0):
>    blah
>
>  Same remark.
>
>
>
>
>  >
>  > Or is there a good reason why this is not possible?
>  >
>  > Thank you!
>
> > _______________________________________________
>  > Numpy-discussion mailing list
>  > [hidden email]
>  > http://projects.scipy.org/mailman/listinfo/numpy-discussion
>  >
>
>
>  --
>  David Douard                        LOGILAB, Paris (France), +33 1 45 32 03 12
>  Formations Python, Zope, Debian :   http://www.logilab.fr/formations
>  Développement logiciel sur mesure : http://www.logilab.fr/services
>  Informatique scientifique :         http://www.logilab.fr/science
>
> -----BEGIN PGP SIGNATURE-----
>  Version: GnuPG v1.4.6 (GNU/Linux)
>
>  iD8DBQFICIwl15pXfQ0A2uMRAm/3AJ9nqNHSH7vY5NpIULCXjgXdqkCUUgCfdxyi
>  NIYA8A4grCPDp5lShTKhcoY=
>  =S6VB
>  -----END PGP SIGNATURE-----
>
> _______________________________________________
>  Numpy-discussion mailing list
>  [hidden email]
>  http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Matthieu Brucher-2


2008/4/18, Olivier Verdier <[hidden email]>:
I certainly didn't mean that "A==B" should return a boolean!!

"A==B" should return an array of boolean as it does now. This is all right.

*However* "bool(A==B)" should return a boolean, *not* raise an
exception. Why raise an exception? What is ambiguous about
"bool(A==B)"??

This is what happens when you write "if A==B" because then
"bool(A==B)" is somehow triggered and bang, an exception is raised.

As I tried to explain, the default behaviour should be that "bool(A)"
return "A.all()" but not an exception. Why is there an exception
raising at all?

Sometimes, you want .all(), sometimes .any(). It really depends on the question you are asking. Even for bool(A), there is no easy answer. In some case I want True if some elements are true, in other cases only if all elements are true.
I would agree with you some years ago, but after using bool(A) = A.all(), I started noticing that it brakes my coding instead of speeding it and does not give me any benefit at all.

Matthieu
--
French PhD student
Website : http://matthieu-brucher.developpez.com/
Blogs : http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn : http://www.linkedin.com/in/matthieubrucher
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Alexander Michael
On Fri, Apr 18, 2008 at 8:43 AM, Matthieu Brucher
<[hidden email]> wrote:

>
>
> 2008/4/18, Olivier Verdier <[hidden email]>:
> > I certainly didn't mean that "A==B" should return a boolean!!
> >
> > "A==B" should return an array of boolean as it does now. This is all
> right.
> >
> > *However* "bool(A==B)" should return a boolean, *not* raise an
> > exception. Why raise an exception? What is ambiguous about
> > "bool(A==B)"??
> >
> > This is what happens when you write "if A==B" because then
> > "bool(A==B)" is somehow triggered and bang, an exception is raised.
> >
> > As I tried to explain, the default behaviour should be that "bool(A)"
> > return "A.all()" but not an exception. Why is there an exception
> > raising at all?
>
> Sometimes, you want .all(), sometimes .any(). It really depends on the
> question you are asking. Even for bool(A), there is no easy answer. In some
> case I want True if some elements are true, in other cases only if all
> elements are true.
>  I would agree with you some years ago, but after using bool(A) = A.all(), I
> started noticing that it brakes my coding instead of speeding it and does
> not give me any benefit at all.

Not only might reasonable people differ on want any or all semantics,
it is also quite reasonable to want __nonzero__ to merely indicate
whether or not the ndarray has any elements in the first dimension
(i.e. len >0) like the built-in list and array. At least I think this
is reasonable as this is the semantic that I often want! But seeing
how there are multiple prior conceptions, perhaps the current behavior
is best because it forces us to make sure what we think is being
returned is in fact being returned?

Regards,
Alex
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Alan G Isaac
In reply to this post by Olivier Verdier-2
On Fri, 18 Apr 2008, Olivier Verdier apparently wrote:
> What is ambiguous about "bool(A==B)"?

A==B is an array. Compare:

    >>> bool([])
    False
    >>> bool([0])
    True

Even if you decide the second should be false,
what about [0,1]?  (I.e., all or any?)

Cheers,
Alan Isaac



_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Olivier Verdier-2
Let's restrict the discussion the case to boolean arrays (dtype bool),
since all the comparisons (A==B, A!=B, A<B etc. return boolean
arrays).

So I have an array filled with booleans. Is there a reason not to map
"bool(A)" to "A.all()" but instead raise an exception?

As far as I can see, "if A==B" is clear enough; one always means
"(A==B).all()". Isn't that so?

I would like to see examples where there is clearly an ambiguity so
that I understand the benefit of having an exception raised for
boolean matrices instead of simply using all().

If there is no such clear example, then why not map "bool(A)" to
"A.all()" in numpy?

Thanks!

On 18 avr, 15:20, Alan G Isaac <[hidden email]> wrote:

> On Fri, 18 Apr 2008, Olivier Verdier apparently wrote:
> > What is ambiguous about "bool(A==B)"?
>
> A==B is an array. Compare:
>
>     >>> bool([])
>     False
>     >>> bool([0])
>     True
>
> Even if you decide the second should be false,
> what about [0,1]?  (I.e., all or any?)
>
> Cheers,
> Alan Isaac
>
> _______________________________________________
> Numpy-discussion mailing list
> [hidden email]://projects.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Matthew Brett
Hi,

I must say,  I agree with the other posters here, that it is not
completely obvious to me that:

a = np.array([True, False])
bool(a)

should return False.   Especially given:

L  = [True, False]
bool(L)

returns True.

Given that it's not completely obvious, and

a.all()

is completely obvious, and readable, the current behavior seems right
to me...  Otherwise someone somewhere is going to assume that bool(a)
does the same as a.any(), and run into problems.   Explicit is better
than implicit...

Best,

Matthew
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Anne Archibald
In reply to this post by Olivier Verdier-2
On 18/04/2008, Olivier <[hidden email]> wrote:

> Let's restrict the discussion the case to boolean arrays (dtype bool),
>  since all the comparisons (A==B, A!=B, A<B etc. return boolean
>  arrays).
>
>  So I have an array filled with booleans. Is there a reason not to map
>  "bool(A)" to "A.all()" but instead raise an exception?
>
>  As far as I can see, "if A==B" is clear enough; one always means
>  "(A==B).all()". Isn't that so?
>
>  I would like to see examples where there is clearly an ambiguity so
>  that I understand the benefit of having an exception raised for
>  boolean matrices instead of simply using all().
>
>  If there is no such clear example, then why not map "bool(A)" to
>  "A.all()" in numpy?

In [1]: import numpy as np

In [2]: A = np.array([0,1])

In [3]: B = np.array([0,0])

In [4]: (A==B).all()
Out[4]: False

In [5]: (A!=B).all()
Out[5]: False

One would expect A!=B to be the logical opposite of A==B, but with
your proposed suggestion it is not.

In math, when comparing functions, one can compare the functions as a
whole, or one can compare them pointwise. numpy's implementation does
pointwise comparison, for a variety of good reasons. As for why
converting an array to a boolean doesn't automatically do all(), if
you don't know you are dealing with an array and using all(), you will
surely shoot yourself in the foot sooner rather than later (as the
above example shows). If you do, simply wrapping it in all() is easy
and clear.

Anne
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Joe Harrington
In reply to this post by Olivier Verdier-2
For that matter, is there a reason logical operations don't work on
arrays other than booleans?  What about:

import numpy

x = numpy.ones((10), dtype='Bool')
y = numpy.ones((10), dtype='Bool')

y[6] = False
z = x and y   # logical AND: this one fails with an error about arrays
---------------------------------------------------------------------------
<type 'exceptions.ValueError'>            Traceback (most recent call last)

/home/jh/<ipython console> in <module>()

<type 'exceptions.ValueError'>: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
---------------------------------------------------------------------------
z = x & y     # bitwise AND: this one succeeds
print z
[ True  True  True  True  True  True False  True  True  True]

But, both operators should return the same result, since booleans have
one bit.  Where this really matters is ANDing ints:

x = numpy.ones((10), dtype='uint8')
y = numpy.ones((10), dtype='uint8')

y[6] = 2      # still True this time: http://docs.python.org/ref/Booleans.html
z = x and y   # logical AND: this one fails with an error about arrays
---------------------------------------------------------------------------
<type 'exceptions.ValueError'>            Traceback (most recent call last)

/home/jh/<ipython console> in <module>()

<type 'exceptions.ValueError'>: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
---------------------------------------------------------------------------
z = x & y     # bitwise AND: this one succeeds...but it isn't the AND I want!
print z
[1 1 1 1 1 1 0 1 1 1]  # the AND I want makes this array all True

Here, both x and y are completely True, but you can't trivially do a
logical AND, and the bitwise & isn't the one you want.  You have to
cast to boolean manually and do the logical instead.

Can this be fixed or is there a reason for the exception?

--jh--
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Robert Kern-2
On Fri, Apr 18, 2008 at 3:33 PM, Joe Harrington <[hidden email]> wrote:
> For that matter, is there a reason logical operations don't work on
>  arrays other than booleans?  What about:

The keywords "and", "or", and "not" only work on bool objects; Python
tries to convert the operands using bool(). bool(some_array) raises
the exception just as we've been discussing.

>  Here, both x and y are completely True, but you can't trivially do a
>  logical AND, and the bitwise & isn't the one you want.  You have to
>  cast to boolean manually and do the logical instead.
>
>  Can this be fixed or is there a reason for the exception?

No, it cannot be fixed. Python does not give us a way to overload the
behavior of these keywords.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Alan G Isaac
In reply to this post by Joe Harrington
On Fri, 18 Apr 2008, Joe Harrington apparently wrote:
> For that matter, is there a reason logical operations don't work on
> arrays other than booleans?  What about:
> import numpy
> x = numpy.ones((10), dtype='Bool')
> y = numpy.ones((10), dtype='Bool')
> y[6] = False
> z = x and y   # logical AND: this one fails with an error about arrays

It's the same reason.
(See the other comments in this thread.)
Do this::

    x = numpy.ones((10), dtype='Bool')
    y = numpy.ones((10), dtype='Bool')
    y[6] = False
    z= x&y

hth,
Alan Isaac


_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Anne Archibald
In reply to this post by Robert Kern-2
On 18/04/2008, Robert Kern <[hidden email]> wrote:
> On Fri, Apr 18, 2008 at 3:33 PM, Joe Harrington <[hidden email]> wrote:
>  > For that matter, is there a reason logical operations don't work on
>  >  arrays other than booleans?  What about:
>
> The keywords "and", "or", and "not" only work on bool objects; Python
>  tries to convert the operands using bool(). bool(some_array) raises
>  the exception just as we've been discussing.

This is a bit misleading, as the actual value is returned:

In [167]: "" or "A"
Out[167]: 'A'

In [168]: "A" and "B"
Out[168]: 'B'

In fact the problem is that "and" and "or" are short-circuiting, which
means that "return X or Y" is equivalent to something like:
if X:
    return X
elif Y:
    return Y
else:
    return False

These are handled specially by syntax so that, for example, you can do
"False and 1/0" and not raise a ZeroDivisionError. So "and" and "or"
aren't even really operators, and it's not just impossible to change
them but unlikely to ever become possible. Just cast your arrays to
booleans if you want to do boolean operations on them.

Anne
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Joe Harrington
In reply to this post by Olivier Verdier-2
> Just cast your arrays to booleans if you want to do boolean operations
> on them.

It turns out there's an even better way: logical_and() and its friends
do boolean operations on arrays.

IDL solves the problem exactly as numpy does, erroring on arrays in
conditionals and short-circuiting boolean operators.  They, too,
provide logical_and() and friends to do guarranteed-to-execute-on-all-
elements logical array operations, which was in fact why I thought to
look for numpy's.  I'm curious what Matlab does (I don't have access).

What wasn't obvious was how to cast an array to bool.  Doing that the
obvious way doesn't work, as is the subject of this thread:

numpy.bool(x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

My next thought was:

numpy.array(x, dtype=bool)

array([ True,  True,  True,  True,  True,  True,  True,  True,  True,  True], dtype=bool)

This works, but is a lot to type, and it's pretty opaque to a new user
that you'd have to do it this way.  Then I discovered the numpy
version of bool() (took 45 minutes):

numpy.bool_(x)                   # note trailing underscore

array([ True,  True,  True,  True,  True,  True,  True,  True,  True,  True], dtype=bool)

I just added bool_ and the other casting functions to the categorized
list of functions on the web site.  Each of these links points to the
Numpy_Example_List_With_Doc page, where there is no example, however.
I hope that's ok, for now.

--jh--
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Robert Kern-2
On Sat, Apr 19, 2008 at 1:32 AM, Joe Harrington <[hidden email]> wrote:
>  What wasn't obvious was how to cast an array to bool.

Just like any other type:

  x.astype(bool)

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Truth value of an array

Olivier Verdier-2
In reply to this post by Anne Archibald
Anne, thank you, this was the example I was looking for. Indeed A!=B
would not work as expected if the bool(A) always returned A.all(). Now
I can teach my student why there is no automatic conversion from
boolean arrays to booleans.

== Olivier

On 18/04/2008, Anne Archibald <[hidden email]> wrote:

> On 18/04/2008, Olivier <[hidden email]> wrote:
>  > Let's restrict the discussion the case to boolean arrays (dtype bool),
>  >  since all the comparisons (A==B, A!=B, A<B etc. return boolean
>  >  arrays).
>  >
>  >  So I have an array filled with booleans. Is there a reason not to map
>  >  "bool(A)" to "A.all()" but instead raise an exception?
>  >
>  >  As far as I can see, "if A==B" is clear enough; one always means
>  >  "(A==B).all()". Isn't that so?
>  >
>  >  I would like to see examples where there is clearly an ambiguity so
>  >  that I understand the benefit of having an exception raised for
>  >  boolean matrices instead of simply using all().
>  >
>  >  If there is no such clear example, then why not map "bool(A)" to
>  >  "A.all()" in numpy?
>
>
> In [1]: import numpy as np
>
>  In [2]: A = np.array([0,1])
>
>  In [3]: B = np.array([0,0])
>
>  In [4]: (A==B).all()
>  Out[4]: False
>
>  In [5]: (A!=B).all()
>  Out[5]: False
>
>  One would expect A!=B to be the logical opposite of A==B, but with
>  your proposed suggestion it is not.
>
>  In math, when comparing functions, one can compare the functions as a
>  whole, or one can compare them pointwise. numpy's implementation does
>  pointwise comparison, for a variety of good reasons. As for why
>  converting an array to a boolean doesn't automatically do all(), if
>  you don't know you are dealing with an array and using all(), you will
>  surely shoot yourself in the foot sooner rather than later (as the
>  above example shows). If you do, simply wrapping it in all() is easy
>  and clear.
>
>
>  Anne
>
> _______________________________________________
>  Numpy-discussion mailing list
>  [hidden email]
>  http://projects.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion