Possible Deprecation of np.ediff1d

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Possible Deprecation of np.ediff1d

Tyler Reddy
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Best wishes,
Tyler

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Robert Kern-2
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).

--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Robert Cimrman
In reply to this post by Tyler Reddy
Hi,

it was originally used in other functions of arraysetops (culprit yours truly)
which used to use the padding functionality. Nowadays the only use I can see
git-grepping the numpy sources is in the tests, so +1 for deprecating.

Cheers,
r.

On 08/27/2018 07:28 PM, Tyler Reddy wrote:

> Chuck suggested (
> https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we
> may want to consider deprecating np.ediff1d, which is perhaps not much more
> useful than np.diff, apart from having some arguably strange prepend /
> append behavior added in.
>
> Related discussion on SO:
> https://stackoverflow.com/questions/39014324/difference-between-numpy-ediff1d-and-diff
>
> Thoughts?
>
> Best wishes,
> Tyler
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Charles R Harris
In reply to this post by Robert Kern-2


On Mon, Aug 27, 2018 at 11:37 AM Robert Kern <[hidden email]> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).

I was thinking we might want to add something to `np.diff`, maybe using `np.pad`.

Chuck 

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Eric Wieser

There is already a patch to add such a feature to np.diff at https://github.com/numpy/numpy/pull/8206


On Mon, 27 Aug 2018 at 10:47 Charles R Harris <[hidden email]> wrote:
On Mon, Aug 27, 2018 at 11:37 AM Robert Kern <[hidden email]> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).

I was thinking we might want to add something to `np.diff`, maybe using `np.pad`.

Chuck 
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

josef.pktd
In reply to this post by Charles R Harris


On Mon, Aug 27, 2018 at 1:46 PM, Charles R Harris <[hidden email]> wrote:


On Mon, Aug 27, 2018 at 11:37 AM Robert Kern <[hidden email]> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Huh. Never knew this existed. I'd say about 50% of the time I use np.diff(), I'm doing that prepend/append behavior manually (and less readably, possibly inefficiently, but most importantly annoyingly).

same for me
never seen, never used

 

I was thinking we might want to add something to `np.diff`, maybe using `np.pad`.

ediff1d avoids one extra temp array compared to simple padding.

However, ediff1d pads the final, difference array instead of the original data array.

AFAIR, I almost always need padding of the original array.

eg as in the standard example of round tripping between cumsum and diff

Josef

 

Chuck 

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Stephan Hoyer-2
In reply to this post by Tyler Reddy
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Best wishes,
Tyler

I don't think there's much to be gained by dropping edit1d from NumPy. It's really not a maintenance burden to keep it around unchanged.

My preference, in keeping with our tradition of not unnecessarily causing disruption, would be to keep this function around but mention that np.diff should be preferred for almost all use cases in the docs. This is "Official discouragement" strategy that came up in the recent discussion about our deprecation policy:

I did a search in Google's codebase and turned up only a handful of uses (~20 uses total) but in a variety of different projects:
- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.
- It used in six different internal projects

Cheers,
Stephan


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Charles R Harris


On Mon, Aug 27, 2018 at 8:05 PM Stephan Hoyer <[hidden email]> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Best wishes,
Tyler

I don't think there's much to be gained by dropping edit1d from NumPy. It's really not a maintenance burden to keep it around unchanged.

My preference, in keeping with our tradition of not unnecessarily causing disruption, would be to keep this function around but mention that np.diff should be preferred for almost all use cases in the docs. This is "Official discouragement" strategy that came up in the recent discussion about our deprecation policy:

I did a search in Google's codebase and turned up only a handful of uses (~20 uses total) but in a variety of different projects:
- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.
- It used in six different internal projects

Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.

Chuck 

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

ralfgommers


On Tue, Aug 28, 2018 at 8:04 AM Charles R Harris <[hidden email]> wrote:


On Mon, Aug 27, 2018 at 8:05 PM Stephan Hoyer <[hidden email]> wrote:
On Mon, Aug 27, 2018 at 10:30 AM Tyler Reddy <[hidden email]> wrote:
Chuck suggested ( https://github.com/numpy/numpy/pull/11805#issuecomment-416069436 ) that we may want to consider deprecating np.ediff1d, which is perhaps not much more useful than np.diff, apart from having some arguably strange prepend / append behavior added in.


Thoughts?

Best wishes,
Tyler

I don't think there's much to be gained by dropping edit1d from NumPy. It's really not a maintenance burden to keep it around unchanged.

My preference, in keeping with our tradition of not unnecessarily causing disruption, would be to keep this function around but mention that np.diff should be preferred for almost all use cases in the docs. This is "Official discouragement" strategy that came up in the recent discussion about our deprecation policy:

I did a search in Google's codebase and turned up only a handful of uses (~20 uses total) but in a variety of different projects:
- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.
- It used in six different internal projects

Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.

Given the list of uses Stephan turned up and Robert saying it's a useful function, I'm -1 on any warning. If np.diff gets the same padding behavior, we can document ediff1d in its document as being superceded with a recommendation to use np.diff instead.

In such a docstring warning we could include an easily searchable phrase that we start using for all such functions, but I don't think there's much value in that.

Cheers,
Ralf


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Stephan Hoyer-2


On Tue, Aug 28, 2018 at 9:03 AM Ralf Gommers <[hidden email]> wrote:
Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.

Given the list of uses Stephan turned up and Robert saying it's a useful function, I'm -1 on any warning. If np.diff gets the same padding behavior, we can document ediff1d in its document as being superceded with a recommendation to use np.diff instead.

To be clear, I don't think np.ediff1d is particularly useful or necessary, despite these uses. Most of these uses don't even use the optional arguments, so the author was probably simply ignorant of np.diff. This is more or less inevitable for most corners of NumPy's API, given how many users we have.

"PendingDeprecationWarning" is Python's built-in warning for signaling that something is obsolete but not deprecated yet. It might be appropriate to use in these cases. The default warning filters silence it for users, so it doesn't show up unless you're very aggressive about enabling all warnings.
 

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Ilhan Polat
In the meantime I'll make a PR to get rid of it from SciPy. We can also signal other libraries to do so. Anything frees up the already-very-crowded namespace of NumPy dot is worth it in my opinion.

On Tue, Aug 28, 2018 at 7:40 PM Stephan Hoyer <[hidden email]> wrote:


On Tue, Aug 28, 2018 at 9:03 AM Ralf Gommers <[hidden email]> wrote:
Maybe we need a "NumpyObsoleteWarning" :) At the least, we should probably have a list of obsolete functions in the documentation somewhere. My main concern is that as we go forward we might end up supporting a bunch of functions that are seldom used and have better replacements. We need some method of pruning.

Given the list of uses Stephan turned up and Robert saying it's a useful function, I'm -1 on any warning. If np.diff gets the same padding behavior, we can document ediff1d in its document as being superceded with a recommendation to use np.diff instead.

To be clear, I don't think np.ediff1d is particularly useful or necessary, despite these uses. Most of these uses don't even use the optional arguments, so the author was probably simply ignorant of np.diff. This is more or less inevitable for most corners of NumPy's API, given how many users we have.

"PendingDeprecationWarning" is Python's built-in warning for signaling that something is obsolete but not deprecated yet. It might be appropriate to use in these cases. The default warning filters silence it for users, so it doesn't show up unless you're very aggressive about enabling all warnings.
 
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Marten van Kerkwijk
In reply to this post by Stephan Hoyer-2


On Mon, Aug 27, 2018 at 10:05 PM Stephan Hoyer <[hidden email]> wrote:

- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.

The only reason it appears in astropy is because of tests that Quantity works correctly with it; we do not actually use it...

So that's at least a few hits that do not count as arguments to keep it!  I'm in favour of a PendingDeprecationWarning.

-- Marten


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

ralfgommers


On Tue, Aug 28, 2018 at 12:21 PM Marten van Kerkwijk <[hidden email]> wrote:


On Mon, Aug 27, 2018 at 10:05 PM Stephan Hoyer <[hidden email]> wrote:

- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.

The only reason it appears in astropy is because of tests that Quantity works correctly with it; we do not actually use it...

So that's at least a few hits that do not count as arguments to keep it!  I'm in favour of a PendingDeprecationWarning.

We should at least first merge the PR that adds the same padding behavior to np.diff before doing this, then such a warning could say to just use that and get unchanged behavior.

Ralf
 

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Possible Deprecation of np.ediff1d

Stephan Hoyer-2
On Tue, Aug 28, 2018 at 12:27 PM Ralf Gommers <[hidden email]> wrote:
On Tue, Aug 28, 2018 at 12:21 PM Marten van Kerkwijk <[hidden email]> wrote:


On Mon, Aug 27, 2018 at 10:05 PM Stephan Hoyer <[hidden email]> wrote:

- It appears in astropy, dask, pandas, pint, scipy and TensorFlow.

The only reason it appears in astropy is because of tests that Quantity works correctly with it; we do not actually use it...

So that's at least a few hits that do not count as arguments to keep it!  I'm in favour of a PendingDeprecationWarning.

We should at least first merge the PR that adds the same padding behavior to np.diff before doing this, then such a warning could say to just use that and get unchanged behavior.

The proposed behavior for np.diff() is different, but it should solve the same use-cases. 

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion