# Adding "maximum difference" to np.testing.assert_array_equal errors

5 messages
Open this post in threaded view
|

## Adding "maximum difference" to np.testing.assert_array_equal errors

 I have a PR up for review that adds "maximum difference" to the error messages produced by NumPy's testing functions for comparing arrays:https://github.com/numpy/numpy/pull/12591Because this changes NumPy's public interface, I'm running it by the mailing list to see if there are any objections or suggestions for improvement.Example behavior:>>> x = np.array([1, 2, 3])>>> y = np.array([1, 2, 3.0001]) >>> np.testing.assert_allclose(x, y) AssertionError:Not equal to tolerance rtol=1e-07, atol=0(mismatch 33.333333333333336%, maximum difference 0.00010000000000021103) x: array([1, 2, 3]) y: array([1. , 2. , 3.0001]) Motivation: when writing numerical algorithms, I frequently find myself experimenting to pick the right value of atol and rtol for np.testing.assert_allclose(). If I make the tolerance too generous, I risk missing regressions in accuracy, so I usually try to pick the smallest values for which tests pass. This change immediately reveals appropriate values to use for these parameters, so I don't need to guess and check.One alternative would be print both "atol" and "rtol" numbers directly that could be immediately used in assert_allclose(). Here we effectively only have "atol". The main reason why I didn't do this is that I felt it would add more clutter (rtol is slightly redundant with the displayed array values). But we could probably do this if we're willing to split the error message onto a few more lines, e.g.,AssertionError:Not equal to tolerance rtol=1e-07, atol=0(mismatch 33.333333333333336%, maximum absolute difference 0.00010000000000021103, maximum relative difference 3.333333333340368e-05) x: array([1, 2, 3]) y: array([1. , 2. , 3.0001]) Best,Stephan _______________________________________________ NumPy-Discussion mailing list [hidden email] https://mail.python.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Adding "maximum difference" to np.testing.assert_array_equal errors

 On Wed, 19 Dec 2018 09:47:01 -0800, Stephan Hoyer wrote: > Example behavior: > > >>> x = np.array([1, 2, 3]) > >>> y = np.array([1, 2, 3.0001]) > >>> np.testing.assert_allclose(x, y) > AssertionError: > Not equal to tolerance rtol=1e-07, atol=0 > > (mismatch 33.333333333333336%, maximum difference > 0.00010000000000021103) This is a helpful addition; thank you!  I don't have a strong preference around whether to also include rtol. Stéfan _______________________________________________ NumPy-Discussion mailing list [hidden email] https://mail.python.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: Adding "maximum difference" to np.testing.assert_array_equal errors

 > On 19. Dec 2018, at 19:50, Stefan van der Walt <[hidden email]> wrote: > > On Wed, 19 Dec 2018 09:47:01 -0800, Stephan Hoyer wrote: >> Example behavior: >> >>>>> x = np.array([1, 2, 3]) >>>>> y = np.array([1, 2, 3.0001]) >>>>> np.testing.assert_allclose(x, y) >> AssertionError: >> Not equal to tolerance rtol=1e-07, atol=0 >> >> (mismatch 33.333333333333336%, maximum difference >> 0.00010000000000021103) > > This is a helpful addition; thank you!  I don't have a strong preference > around whether to also include rtol. > > Stéfan > _______________________________________________ > NumPy-Discussion mailing list > [hidden email] > https://mail.python.org/mailman/listinfo/numpy-discussion+1 to this addition. And +1 to also print rtol. I frequently use rtol with assert_allclose, and having the value printed helps quickly choose an appropriate value. Christoph _______________________________________________ NumPy-Discussion mailing list [hidden email] https://mail.python.org/mailman/listinfo/numpy-discussion