round / set_printoptions discrepancy

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

round / set_printoptions discrepancy

Irvin Probst
Hi,
Is it expected/documented that np.round and np.set_printoptions do not
output the same result on screen ?
I tumbled into this running this code:

import numpy as np
mes = np.array([
     [16.06, 16.13, 16.06, 16.00, 16.06, 16.00, 16.13, 16.00]
])

avg = np.mean(mes, axis=1)
print(np.round(avg, 2))
np.set_printoptions(precision=2)
print(avg)


Which outputs:

[16.06]
[16.05]

Is that worth a bug report or did I miss something ? I've been able to
reproduce this on many windows/linux PCs with python/numpy releases from
2017 up to last week.

Thanks.
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Andras Deak
On Fri, Sep 13, 2019 at 12:58 PM Irvin Probst
<[hidden email]> wrote:

>
> Hi,
> Is it expected/documented that np.round and np.set_printoptions do not
> output the same result on screen ?
> I tumbled into this running this code:
>
> import numpy as np
> mes = np.array([
>      [16.06, 16.13, 16.06, 16.00, 16.06, 16.00, 16.13, 16.00]
> ])
>
> avg = np.mean(mes, axis=1)
> print(np.round(avg, 2))
> np.set_printoptions(precision=2)
> print(avg)
>
>
> Which outputs:
>
> [16.06]
> [16.05]
>
> Is that worth a bug report or did I miss something ? I've been able to
> reproduce this on many windows/linux PCs with python/numpy releases from
> 2017 up to last week.
>
> Thanks.
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

Hi,

I just want to add that you can use literal 16.055 to reproduce this:
>>> import numpy as np
>>> np.set_printoptions(precision=2)
>>> np.array([16.055]).round(2)
array([16.06])
>>> np.array([16.055])
array([16.05])

I would think it has to do with "round to nearest even":
>>> np.array(16.055)
array(16.05)
>>> np.array(16.065)
array(16.07)
>>> np.array(16.065).round(2)
16.07

But it's as if `round` rounded decimal digits upwards (16.055 ->
16.06, 16.065 -> 16.07), whereas the `repr` rounded to the nearest
odd(!) digit (16.055 -> 16.05, 16.065 -> 16.07). Does this make any
sense? I'm on numpy 1.17.2.
(Scalars or 1-length 1d arrays don't seem to make a difference).
Regards,

András
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Philip Hodge


On 9/13/19 7:23 AM, Andras Deak wrote:
I just want to add that you can use literal 16.055 to reproduce this:
import numpy as np
np.set_printoptions(precision=2)
np.array([16.055]).round(2)
array([16.06])
np.array([16.055])
array([16.05])

I would think it has to do with "round to nearest even":
np.array(16.055)
array(16.05)
np.array(16.065)
array(16.07)
np.array(16.065).round(2)
16.07

But it's as if `round` rounded decimal digits upwards (16.055 ->
16.06, 16.065 -> 16.07), whereas the `repr` rounded to the nearest
odd(!) digit (16.055 -> 16.05, 16.065 -> 16.07). Does this make any
sense? I'm on numpy 1.17.2.
(Scalars or 1-length 1d arrays don't seem to make a difference).
Regards,

András


Isn't that just for consistency with Python 3 round()?  I agree that the discrepancy with np.set_printoptions is not necessarily expected, except possibly for backwards compatibility.

Phil


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Irvin Probst
On 13/09/2019 14:05, Philip Hodge wrote:
>
> Isn't that just for consistency with Python 3 round()?  I agree that
> the discrepancy with np.set_printoptions is not necessarily expected,
> except possibly for backwards compatibility.
>
>

I've just checked and np.set_printoptions behaves as python's round:

 >>> round(16.055,2)
16.05
 >>> np.round(16.055,2)
16.06

I don't know why round and np.round do not behave the same, actually I
would even dare to say that I don't care :-)
However np.round and np.set_printoptions should provide the same output,
shouldn't they ? This discrepancy is really disturbing whereas
consistency with python's round looks like the icing on the cake but in
no way a required feature.

--
Irvin
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Philip Hodge
On 9/13/19 8:45 AM, Irvin Probst wrote:

> On 13/09/2019 14:05, Philip Hodge wrote:
>>
>> Isn't that just for consistency with Python 3 round()?  I agree that
>> the discrepancy with np.set_printoptions is not necessarily expected,
>> except possibly for backwards compatibility.
>>
>>
>
> I've just checked and np.set_printoptions behaves as python's round:
>
> >>> round(16.055,2)
> 16.05
> >>> np.round(16.055,2)
> 16.06
>
> I don't know why round and np.round do not behave the same, actually I
> would even dare to say that I don't care :-)
> However np.round and np.set_printoptions should provide the same
> output, shouldn't they ? This discrepancy is really disturbing whereas
> consistency with python's round looks like the icing on the cake but
> in no way a required feature.
>

Python round() is supposed to round to the nearest even value, if the
two closest values are equally close.  So round(16.055, 2) returning
16.05 was a surprise to me.  I checked the documentation and found a
note that explained that this was because "most decimal fractions can't
be represented exactly as a float."  round(16.55) returns 16.6.

Phil

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Andras Deak
On Fri, Sep 13, 2019 at 2:59 PM Philip Hodge <[hidden email]> wrote:

>
> On 9/13/19 8:45 AM, Irvin Probst wrote:
> > On 13/09/2019 14:05, Philip Hodge wrote:
> >>
> >> Isn't that just for consistency with Python 3 round()?  I agree that
> >> the discrepancy with np.set_printoptions is not necessarily expected,
> >> except possibly for backwards compatibility.
> >>
> >>
> >
> > I've just checked and np.set_printoptions behaves as python's round:
> >
> > >>> round(16.055,2)
> > 16.05
> > >>> np.round(16.055,2)
> > 16.06
> >
> > I don't know why round and np.round do not behave the same, actually I
> > would even dare to say that I don't care :-)
> > However np.round and np.set_printoptions should provide the same
> > output, shouldn't they ? This discrepancy is really disturbing whereas
> > consistency with python's round looks like the icing on the cake but
> > in no way a required feature.
> >
>
> Python round() is supposed to round to the nearest even value, if the
> two closest values are equally close.  So round(16.055, 2) returning
> 16.05 was a surprise to me.  I checked the documentation and found a
> note that explained that this was because "most decimal fractions can't
> be represented exactly as a float."  round(16.55) returns 16.6.
>
> Phil
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

Ah, of course, endless double-precision shenanigans...
>>> format(16.055, '.30f')
'16.054999999999999715782905695960'

>>> format(16.55, '.30f')
'16.550000000000000710542735760100'

András
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Eric Moore-11
See the notes section here.   https://numpy.org/devdocs/reference/generated/numpy.around.html.

This note was recently added in https://github.com/numpy/numpy/pull/14392

Eric

On Fri, Sep 13, 2019 at 9:20 AM Andras Deak <[hidden email]> wrote:
On Fri, Sep 13, 2019 at 2:59 PM Philip Hodge <[hidden email]> wrote:
>
> On 9/13/19 8:45 AM, Irvin Probst wrote:
> > On 13/09/2019 14:05, Philip Hodge wrote:
> >>
> >> Isn't that just for consistency with Python 3 round()?  I agree that
> >> the discrepancy with np.set_printoptions is not necessarily expected,
> >> except possibly for backwards compatibility.
> >>
> >>
> >
> > I've just checked and np.set_printoptions behaves as python's round:
> >
> > >>> round(16.055,2)
> > 16.05
> > >>> np.round(16.055,2)
> > 16.06
> >
> > I don't know why round and np.round do not behave the same, actually I
> > would even dare to say that I don't care :-)
> > However np.round and np.set_printoptions should provide the same
> > output, shouldn't they ? This discrepancy is really disturbing whereas
> > consistency with python's round looks like the icing on the cake but
> > in no way a required feature.
> >
>
> Python round() is supposed to round to the nearest even value, if the
> two closest values are equally close.  So round(16.055, 2) returning
> 16.05 was a surprise to me.  I checked the documentation and found a
> note that explained that this was because "most decimal fractions can't
> be represented exactly as a float."  round(16.55) returns 16.6.
>
> Phil
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

Ah, of course, endless double-precision shenanigans...
>>> format(16.055, '.30f')
'16.054999999999999715782905695960'

>>> format(16.55, '.30f')
'16.550000000000000710542735760100'

András
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Irvin Probst
On 13/09/2019 15:26, Eric Moore wrote:
> See the notes section here.
> https://numpy.org/devdocs/reference/generated/numpy.around.html.
>
> This note was recently added in https://github.com/numpy/numpy/pull/14392
>
>
Thanks, it indeed explains the discrepancy.

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Stefano Miccoli
In reply to this post by Irvin Probst
In my opinion the problem is that numpy floats break the Liskov substitution principle,

>>> pyfloat = 16.055
>>> npfloat = np.float64(pyfloat)
>>> isinstance(npfloat, float)
True
>>> round(pyfloat, 2)
16.05
>>> round(npfloat, 2)
16.06

Since numpy.float64 is a subclass of builtins.float I would expect that

>>> round(x, j) == round(np.float64(x), j)

is an invariant, but unfortunately this is not the case.

Moreover the python3 semantics of the round function require that when the number of digits is None,
the return value should be of integral type:

>>> round(pyfloat)
16
>>> round(pyfloat, None)
16
>>> round(pyfloat, 0)
16.0
>>> round(npfloat)
16.0
>>> round(npfloat, None)
16.0
>>> round(npfloat, 0)
16.0

see also https://github.com/numpy/numpy/issues/11810

Stefano
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: round / set_printoptions discrepancy

Allan Haldane
In reply to this post by Eric Moore-11
On 9/13/19 9:26 AM, Eric Moore wrote:
> See the notes section here.  
> https://numpy.org/devdocs/reference/generated/numpy.around.html.
>
> This note was recently added in https://github.com/numpy/numpy/pull/14392
>
> Eric

Hmm, but this example with 16.055 shows the note still isn't quite right.

The doc suggests that the floating point error only matters for large
values or large `decimals`, but this shows it also happens for small
values. Makes sense now that I see the example.

We should tweak the docstring.

Also, I did make some notes in
https://github.com/numpy/numpy/issues/14391
for how we could "fix" this problem efficiently. Unfortunately it's far
from trivial to write a correct rounding algorithm, and I'm not sure
it's worth the effort: The round error is comparable to normal
floating-point error, and I don't think round is heavily used.

Best,
Allan


> On Fri, Sep 13, 2019 at 9:20 AM Andras Deak <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     On Fri, Sep 13, 2019 at 2:59 PM Philip Hodge <[hidden email]
>     <mailto:[hidden email]>> wrote:
>     >
>     > On 9/13/19 8:45 AM, Irvin Probst wrote:
>     > > On 13/09/2019 14:05, Philip Hodge wrote:
>     > >>
>     > >> Isn't that just for consistency with Python 3 round()?  I agree
>     that
>     > >> the discrepancy with np.set_printoptions is not necessarily
>     expected,
>     > >> except possibly for backwards compatibility.
>     > >>
>     > >>
>     > >
>     > > I've just checked and np.set_printoptions behaves as python's round:
>     > >
>     > > >>> round(16.055,2)
>     > > 16.05
>     > > >>> np.round(16.055,2)
>     > > 16.06
>     > >
>     > > I don't know why round and np.round do not behave the same,
>     actually I
>     > > would even dare to say that I don't care :-)
>     > > However np.round and np.set_printoptions should provide the same
>     > > output, shouldn't they ? This discrepancy is really disturbing
>     whereas
>     > > consistency with python's round looks like the icing on the cake but
>     > > in no way a required feature.
>     > >
>     >
>     > Python round() is supposed to round to the nearest even value, if the
>     > two closest values are equally close.  So round(16.055, 2) returning
>     > 16.05 was a surprise to me.  I checked the documentation and found a
>     > note that explained that this was because "most decimal fractions
>     can't
>     > be represented exactly as a float."  round(16.55) returns 16.6.
>     >
>     > Phil
>     >
>     > _______________________________________________
>     > NumPy-Discussion mailing list
>     > [hidden email] <mailto:[hidden email]>
>     > https://mail.python.org/mailman/listinfo/numpy-discussion
>
>     Ah, of course, endless double-precision shenanigans...
>     >>> format(16.055, '.30f')
>     '16.054999999999999715782905695960'
>
>     >>> format(16.55, '.30f')
>     '16.550000000000000710542735760100'
>
>     András
>     _______________________________________________
>     NumPy-Discussion mailing list
>     [hidden email] <mailto:[hidden email]>
>     https://mail.python.org/mailman/listinfo/numpy-discussion
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion