

Hi,
I’m a bit surprised with the following code:
>>> import numpy as np
>>> np.seterr(all='warn')
>>> Z = np.array([128], dtype=np.byte)
>>> print(np.abs(Z))
[128]
Obviously, it does not return the absolute value and I get no warning.
Is it something expected ? (numpy 1.16.4, python 3.7.3)
Nicolas
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion


Hi,
It turns out you're running into a biterror. In general, the two's complement of 2 ** (n1) with the bitlength being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
Regards,
Hameer Abbasi
On 03.07.19, 17:03, "NumPyDiscussion on behalf of Nicolas Rougier" <numpydiscussionbounces+einstein.edison= [hidden email] on behalf of [hidden email]> wrote:
Hi,
I’m a bit surprised with the following code:
>>> import numpy as np
>>> np.seterr(all='warn')
>>> Z = np.array([128], dtype=np.byte)
>>> print(np.abs(Z))
[128]
Obviously, it does not return the absolute value and I get no warning.
Is it something expected ? (numpy 1.16.4, python 3.7.3)
Nicolas
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion


Hi,
It turns out you're running into a biterror. In general, the two's complement of 2 ** (n1) with the bitlength being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
Regards,
Hameer Abbasi
More like the eight bit twos complement of 128 is 128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
<snip>
Chuck
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion


Hi,
On Wed, Jul 3, 2019 at 9:08 AM Charles R Harris
< [hidden email]> wrote:
>
>
>
> On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi < [hidden email]> wrote:
>>
>> Hi,
>>
>> It turns out you're running into a biterror. In general, the two's complement of 2 ** (n1) with the bitlength being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
>>
>> TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
>>
>> Regards,
>> Hameer Abbasi
>
>
> More like the eight bit twos complement of 128 is 128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
I still think this is a major wart in numpy's abs.
I'd really like to add a new function, `uabs` which would return an
unsigned int for integer inputs. I'm happy to do a pull request if
that also seems sensible to y'all,
Cheers,
Matthew
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion


Hi,
On Wed, Jul 3, 2019 at 9:08 AM Charles R Harris
<[hidden email]> wrote:
>
>
>
> On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi <[hidden email]> wrote:
>>
>> Hi,
>>
>> It turns out you're running into a biterror. In general, the two's complement of 2 ** (n1) with the bitlength being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
>>
>> TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
>>
>> Regards,
>> Hameer Abbasi
>
>
> More like the eight bit twos complement of 128 is 128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
I still think this is a major wart in numpy's abs.
I'd really like to add a new function, `uabs` which would return an
unsigned int for integer inputs. I'm happy to do a pull request if
that also seems sensible to y'all,
Seems reasonable, it would provide something for discussion. Note that the current abs is in the slot provided by Python numeric types. The main problem is that as soon as the unsigned result is combined with a signed type it will get promoted. Which would not be much of a problem except int64 > double.
Chuck
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion


Thanks for the quick explanation & answers. I was mostly puzzled by the absence of warning.
In C you would get an overflow warning in such a case, no ?
Nicolas
> On 3 Jul 2019, at 20:01, Charles R Harris < [hidden email]> wrote:
>
>
>
> On Wed, Jul 3, 2019 at 11:39 AM Matthew Brett < [hidden email]> wrote:
> Hi,
>
> On Wed, Jul 3, 2019 at 9:08 AM Charles R Harris
> < [hidden email]> wrote:
> >
> >
> >
> > On Wed, Jul 3, 2019 at 9:08 AM Hameer Abbasi < [hidden email]> wrote:
> >>
> >> Hi,
> >>
> >> It turns out you're running into a biterror. In general, the two's complement of 2 ** (n1) with the bitlength being limited to n bits is itself... No way around that. And integers don't set hardware exceptions so checking for errors like these is hard as well.
> >>
> >> TL;DR: It's an error with how the integer is stored in memory and how you're running out of space.
> >>
> >> Regards,
> >> Hameer Abbasi
> >
> >
> > More like the eight bit twos complement of 128 is 128, bytes cannot represent 128. Matlab used to (still does?) solve this problem by returning 127 instead :) Basically, the data needs more precision. Returning an unsigned type would lead to it's own problems with unexpected promotions when the result was used. We could, I suppose, raise a warning, although that might be considered noisy. If you use `absolute` instead, you can specify the dtype.
>
> I still think this is a major wart in numpy's abs.
>
> I'd really like to add a new function, `uabs` which would return an
> unsigned int for integer inputs. I'm happy to do a pull request if
> that also seems sensible to y'all,
>
>
> Seems reasonable, it would provide something for discussion. Note that the current abs is in the slot provided by Python numeric types. The main problem is that as soon as the unsigned result is combined with a signed type it will get promoted. Which would not be much of a problem except int64 > double.
>
> Chuck
> _______________________________________________
> NumPyDiscussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpydiscussion_______________________________________________
NumPyDiscussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpydiscussion

