

Hi everyone, >>> import numpy as np >>> np.__version__ '1.9.0' >>> np.gradient(np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float)) [array([[ 2., 2., 1.], [ 2., 2., 1.]]), array([[0.5, 2.5, 5.5], [ 1. , 1. , 1. ]])]
On the other hand: >>> import numpy as np >>> np.__version__ '1.8.2' >>> np.gradient(np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float)) [array([[ 2., 2., 1.], [ 2., 2., 1.]]), array([[ 1. , 2.5, 4. ], [ 1. , 1. , 1. ]])]
For what it's worth, the 1.8 version of this function seems to be in agreement with the Matlab equivalent function ('gradient'): >> gradient([[1, 2, 6]; [3, 4, 5]])
ans =
1.0000 2.5000 4.0000 1.0000 1.0000 1.0000
This seems like a regression to me, but maybe it's an improvement?
Cheers,
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On 4 Oct 2014, at 08:37 pm, Ariel Rokem < [hidden email]> wrote:
> >>> import numpy as np
> >>> np.__version__
> '1.9.0'
> >>> np.gradient(np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float))
> [array([[ 2., 2., 1.],
> [ 2., 2., 1.]]), array([[0.5, 2.5, 5.5],
> [ 1. , 1. , 1. ]])]
>
> On the other hand:
> >>> import numpy as np
> >>> np.__version__
> '1.8.2'
> >>> np.gradient(np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float))
> [array([[ 2., 2., 1.],
> [ 2., 2., 1.]]), array([[ 1. , 2.5, 4. ],
> [ 1. , 1. , 1. ]])]
>
> For what it's worth, the 1.8 version of this function seems to be in agreement with the Matlab equivalent function ('gradient'):
> >> gradient([[1, 2, 6]; [3, 4, 5]])
> ans =
> 1.0000 2.5000 4.0000
> 1.0000 1.0000 1.0000
>
> This seems like a regression to me, but maybe it's an improvement?
>
Technically yes, the function has been changed to use 2ndorder differences where possible,
as is described in the docstring. Someone missed to update the example though, which still
quotes the 1.8 results.
And if the loss of Matlabcompliance is seen as a disadvantage, maybe there is a case for
reenabling the old behaviour via keyword argument?
Cheers,
Derek
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


Hi Ariel,
> I think that the docstring in 1.9 is fine (has the 1.9 result). The docs online (for all of numpy) are still on version 1.8, though.
>
> I think that enabling the old behavior might be useful, if only so that I can write code that behaves consistently across these two versions of numpy. For now, I might just copy over the 1.8 code into my project.
>
Hmm, I got this with 1.9.0:
Examples

>>> x = np.array([1, 2, 4, 7, 11, 16], dtype=np.float)
>>> np.gradient(x)
array([ 1. , 1.5, 2.5, 3.5, 4.5, 5. ])
>>> np.gradient(x, 2)
array([ 0.5 , 0.75, 1.25, 1.75, 2.25, 2.5 ])
>>> np.gradient(np.array([[1, 2, 6], [3, 4, 5]], dtype=np.float))
[array([[ 2., 2., 1.],
[ 2., 2., 1.]]),
array([[ 1. , 2.5, 4. ],
[ 1. , 1. , 1. ]])]
In [5]: x =np.array([1, 2, 4, 7, 11, 16], dtype=np.float)
In [6]: print(np.gradient(x))
[ 0.5 1.5 2.5 3.5 4.5 5.5]
In [7]: print(np.gradient(x, 2))
[ 0.25 0.75 1.25 1.75 2.25 2.75]
…
I think there is a point for supporting the old behaviour besides backwardscompatibility or any sort of Matlabcompliance
as I’d probably like to be able to restrict a function to linear/1st order differences in cases where I know the input to be
not wellbehaved.
+1 for an order=2 or maxorder=2 flag
Cheers,
Derek
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On Oct 4, 2014 10:14 PM, "Derek Homeier" <[hidden email]> wrote:
>
> +1 for an order=2 or maxorder=2 flag
If you parameterize that flag, users will want to change its value (above two). Perhaps rather use a boolean flag such as "second_order" or "high_order", unless it seems feasible to include additional orders in the future.
Stéfan
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On 4 Oct 2014 22:17, "Stéfan van der Walt" <[hidden email]> wrote:
>
> On Oct 4, 2014 10:14 PM, "Derek Homeier" <[hidden email]> wrote:
> >
> > +1 for an order=2 or maxorder=2 flag
>
> If you parameterize that flag, users will want to change its value (above two). Perhaps rather use a boolean flag such as "second_order" or "high_order", unless it seems feasible to include additional orders in the future.
Predicting the future is hard :). And in particular high_order= would create all kinds of confusion if in the future we added 3rd order approximations but high_order=True continued to mean 2nd order because of compatibility. I like maxorder (or max_order would be more pep8ish I guess) because it leaves our options open. (Similar to how it's often better to have a kwarg that can take two possible string values than to have a boolean kwarg. It makes current code more explicit and makes future enhancements easier.)
n
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On 14 Oct 2014 18:29, "Charles R Harris" <[hidden email]> wrote:
>
>
>
> On Tue, Oct 14, 2014 at 10:57 AM, Nathaniel Smith <[hidden email]> wrote:
>>
>> On 4 Oct 2014 22:17, "Stéfan van der Walt" <[hidden email]> wrote:
>> >
>> > On Oct 4, 2014 10:14 PM, "Derek Homeier" <[hidden email]> wrote:
>> > >
>> > > +1 for an order=2 or maxorder=2 flag
>> >
>> > If you parameterize that flag, users will want to change its value (above two). Perhaps rather use a boolean flag such as "second_order" or "high_order", unless it seems feasible to include additional orders in the future.
>>
>> Predicting the future is hard :). And in particular high_order= would create all kinds of confusion if in the future we added 3rd order approximations but high_order=True continued to mean 2nd order because of compatibility. I like maxorder (or max_order would be more pep8ish I guess) because it leaves our options open. (Similar to how it's often better to have a kwarg that can take two possible string values than to have a boolean kwarg. It makes current code more explicit and makes future enhancements easier.)
>
>
> I think maxorder is a bit misleading. The both versions are second order in the interior while at the ends the old is first order and the new is second order. Maybe edge_order?
Ah, that makes sense. edge_order makes more sense to me too then  and we can always add interior_order to complement it later, if appropriate.
The other thing to decide on is the default. Is the 2nd order version generally preferred (modulo compatibility)? If so then it might make sense to keep it the default, given that there are already numpy's in the wild with that version, so we can't fully guarantee compatibility even if we wanted to. But what do others think?
n
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On Tue, Oct 14, 2014 at 10:33 PM, Charles R Harris
< [hidden email]> wrote:
>
> On Tue, Oct 14, 2014 at 11:50 AM, Nathaniel Smith < [hidden email]> wrote:
>>
>> On 14 Oct 2014 18:29, "Charles R Harris" < [hidden email]>
>> wrote:
>> >
>> >
>> >
>> > On Tue, Oct 14, 2014 at 10:57 AM, Nathaniel Smith < [hidden email]> wrote:
>> >>
>> >> On 4 Oct 2014 22:17, "Stéfan van der Walt" < [hidden email]> wrote:
>> >> >
>> >> > On Oct 4, 2014 10:14 PM, "Derek Homeier"
>> >> > < [hidden email]> wrote:
>> >> > >
>> >> > > +1 for an order=2 or maxorder=2 flag
>> >> >
>> >> > If you parameterize that flag, users will want to change its value
>> >> > (above two). Perhaps rather use a boolean flag such as "second_order" or
>> >> > "high_order", unless it seems feasible to include additional orders in the
>> >> > future.
>> >>
>> >> Predicting the future is hard :). And in particular high_order= would
>> >> create all kinds of confusion if in the future we added 3rd order
>> >> approximations but high_order=True continued to mean 2nd order because of
>> >> compatibility. I like maxorder (or max_order would be more pep8ish I guess)
>> >> because it leaves our options open. (Similar to how it's often better to
>> >> have a kwarg that can take two possible string values than to have a boolean
>> >> kwarg. It makes current code more explicit and makes future enhancements
>> >> easier.)
>> >
>> >
>> > I think maxorder is a bit misleading. The both versions are second order
>> > in the interior while at the ends the old is first order and the new is
>> > second order. Maybe edge_order?
>>
>> Ah, that makes sense. edge_order makes more sense to me too then  and we
>> can always add interior_order to complement it later, if appropriate.
>>
>> The other thing to decide on is the default. Is the 2nd order version
>> generally preferred (modulo compatibility)? If so then it might make sense
>> to keep it the default, given that there are already numpy's in the wild
>> with that version, so we can't fully guarantee compatibility even if we
>> wanted to. But what do others think?
>
> I'd be inclined to keep the older as the default and regard adding the
> keyword as a bugfix. I should have caught the incompatibility in review.
I don't have any code that uses gradient, so I don't have a great
sense of the tradeoffs here.
 Usually if we have a change that produces increased accuracy, we
make the increased accuracy the default. Otherwise noone ever uses
it, and everyone gets less accurate results than they would otherwise.
(I don't have a great sense of how much this change affects accuracy
though.)
 If the change in output per se is a serious problem for people, then
it's not one we can fix at this point  1.9.0 is out there and people
are using it anyway, so those who have the problem already need to
take some affirmative action to fix it. (Also, it's kinda weird to
change a function's behaviour and add a new argument in a point
release!)
So I'd like to hear from people affected by this  would you prefer
to have the 2nd order boundary calculations by default, you just need
some way to workaround the immediate problems in existing code? Or do
you prefer the old default remain the default, with 2nd order boundary
calculations something that must be requested by hand every time?
n

Nathaniel J. Smith
Postdoctoral researcher  Informatics  University of Edinburgh
http://vorpus.org_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On Fri, Oct 17, 2014 at 2:23 AM, Benjamin Root < [hidden email]> wrote:
> It isn't really a question of accuracy. It breaks unit tests and
> reproducibility elsewhere. My vote is to revert to the old behavior in
> 1.9.1.
Why would one want the 2nd order differences at all, if they're not
more accurate? Should we just revert the patch entirely? I assumed the
change had some benefit...

Nathaniel J. Smith
Postdoctoral researcher  Informatics  University of Edinburgh
http://vorpus.org_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


That isn't what I meant. Higher order doesn't "necessarily" mean more accurate. The results simply have different properties. The user needs to choose the differentiation order that they need. One interesting effect in data assimilation/modeling is that evenorder differentiation can often have detrimental effects while higher odd order differentiation are better, but it is highly dependent upon the model.
This change in gradient broke a unit test in matplotlib (for a new feature, so it isn't *that* critical). We didn't notice it at first because we weren't testing numpy 1.9 at the time. I want the feature (I have need for it elsewhere), but I don't want the change in default behavior. Cheers!
Ben Root
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


Hi,
On Thu, Oct 16, 2014 at 6:38 PM, Benjamin Root < [hidden email]> wrote:
> That isn't what I meant. Higher order doesn't "necessarily" mean more
> accurate. The results simply have different properties. The user needs to
> choose the differentiation order that they need. One interesting effect in
> data assimilation/modeling is that evenorder differentiation can often have
> detrimental effects while higher odd order differentiation are better, but
> it is highly dependent upon the model.
>
> This change in gradient broke a unit test in matplotlib (for a new feature,
> so it isn't *that* critical). We didn't notice it at first because we
> weren't testing numpy 1.9 at the time. I want the feature (I have need for
> it elsewhere), but I don't want the change in default behavior.
I think it would be a bad idea to revert now.
I suspect, if you revert, then a lot of other code will assume the <
1.9.0, >= 1.9.1 behavior. In that case, the code will work as
expected most of the time, except when combined with 1.9.0, which
could be seriously surprising, and often missed. If you keep the new
behavior, then it will be clearer that other code will have to adapt
to this change >= 1.9.0  surprise, but predictable surprise, if you
see what I mean...
Matthew
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion


On 17 Oct 2014 02:38, "Benjamin Root" <[hidden email]> wrote:
>
> That isn't what I meant. Higher order doesn't "necessarily" mean more accurate. The results simply have different properties. The user needs to choose the differentiation order that they need. One interesting effect in data assimilation/modeling is that evenorder differentiation can often have detrimental effects while higher odd order differentiation are better, but it is highly dependent upon the model.
To be clear, we aren't talking about different degrees of differentiation, we're talking about different approximations to the first derivative. I just looked up the original pull request and it contains a pretty convincing graph in which the old code has large systematic errors and the new code doesn't:
https://github.com/numpy/numpy/issues/3603
I think the claim is that the old code had approximation error that grows like O(1/n), and the new code has errors like O(1/n**2). (Don't ask me what n is though.)
> This change in gradient broke a unit test in matplotlib (for a new feature, so it isn't *that* critical). We didn't notice it at first because we weren't testing numpy 1.9 at the time. I want the feature (I have need for it elsewhere), but I don't want the change in default behavior.
You say it's bad, the original poster says it's good, how are we poor maintainers to know what to do? :) Can you say any more about why you you prefer socalled lower accuracy approximations here by default?
n
_______________________________________________
NumPyDiscussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpydiscussion

