__numpy_ufunc__

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

__numpy_ufunc__

Charles R Harris
Hi All,

For those interested in continuing the __numpy_ufunc__ saga, there is a pull request enabling it. Likely we will want to make some changes up front before merging that, so some discussion is in order.

Chuck

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

Charles R Harris


On Sun, Nov 6, 2016 at 11:44 AM, Charles R Harris <[hidden email]> wrote:
Hi All,

For those interested in continuing the __numpy_ufunc__ saga, there is a pull request enabling it. Likely we will want to make some changes up front before merging that, so some discussion is in order.


As a first order of business, let's decide whether to remove the index and rename `__numpy_ufunc__`. The motivation for this is discussed in issue #5986.
If we decide positive on that (I'm in favor), I would be happy with the proposed name `__array_ufunc__`, although something more descriptive like `__handle_ufunc__` might be better. This is a wonderful opportunity for bike shedding for those so inclined ;)

Chuck


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

Nathan Goldbaum
I'm interested in this, but was never able to dive in to the lengthy discussions on this. I'm curious if there is a summary of the current state of things somewhere that I could read to catch up.

On Sunday, November 6, 2016, Charles R Harris <[hidden email]> wrote:


On Sun, Nov 6, 2016 at 11:44 AM, Charles R Harris <<a href="javascript:_e(%7B%7D,&#39;cvml&#39;,&#39;charlesr.harris@gmail.com&#39;);" target="_blank">charlesr.harris@...> wrote:
Hi All,

For those interested in continuing the __numpy_ufunc__ saga, there is a pull request enabling it. Likely we will want to make some changes up front before merging that, so some discussion is in order.


As a first order of business, let's decide whether to remove the index and rename `__numpy_ufunc__`. The motivation for this is discussed in issue #5986.
If we decide positive on that (I'm in favor), I would be happy with the proposed name `__array_ufunc__`, although something more descriptive like `__handle_ufunc__` might be better. This is a wonderful opportunity for bike shedding for those so inclined ;)

Chuck


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

ralfgommers
In reply to this post by Charles R Harris


On Mon, Nov 7, 2016 at 9:10 AM, Charles R Harris <[hidden email]> wrote:


On Sun, Nov 6, 2016 at 11:44 AM, Charles R Harris <[hidden email]> wrote:
Hi All,

For those interested in continuing the __numpy_ufunc__ saga, there is a pull request enabling it. Likely we will want to make some changes up front before merging that, so some discussion is in order.


As a first order of business, let's decide whether to remove the index and rename `__numpy_ufunc__`. The motivation for this is discussed in issue #5986.
If we decide positive on that (I'm in favor),

It seems like everyone on that issue is in favor or at least +0. So +1 from me too.
 
I would be happy with the proposed name `__array_ufunc__`, although something more descriptive like `__handle_ufunc__` might be better.

 
This is a wonderful opportunity for bike shedding for those so inclined ;)

pass :)

Ralf


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

ralfgommers
In reply to this post by Nathan Goldbaum


On Mon, Nov 7, 2016 at 9:21 AM, Nathan Goldbaum <[hidden email]> wrote:
I'm interested in this, but was never able to dive in to the lengthy discussions on this. I'm curious if there is a summary of the current state of things somewhere that I could read to catch up.

Main issue for discussion: https://github.com/numpy/numpy/issues/5844. That really needs a summary, but I'm not sure there's an up-to-date one.

Ralf



On Sunday, November 6, 2016, Charles R Harris <[hidden email]> wrote:


On Sun, Nov 6, 2016 at 11:44 AM, Charles R Harris <[hidden email]> wrote:
Hi All,

For those interested in continuing the __numpy_ufunc__ saga, there is a pull request enabling it. Likely we will want to make some changes up front before merging that, so some discussion is in order.


As a first order of business, let's decide whether to remove the index and rename `__numpy_ufunc__`. The motivation for this is discussed in issue #5986.
If we decide positive on that (I'm in favor), I would be happy with the proposed name `__array_ufunc__`, although something more descriptive like `__handle_ufunc__` might be better. This is a wonderful opportunity for bike shedding for those so inclined ;)

Chuck


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

Charles R Harris
In reply to this post by ralfgommers


On Mon, Nov 7, 2016 at 1:08 AM, Ralf Gommers <[hidden email]> wrote:


On Mon, Nov 7, 2016 at 9:10 AM, Charles R Harris <[hidden email]> wrote:


On Sun, Nov 6, 2016 at 11:44 AM, Charles R Harris <[hidden email]> wrote:
Hi All,

For those interested in continuing the __numpy_ufunc__ saga, there is a pull request enabling it. Likely we will want to make some changes up front before merging that, so some discussion is in order.


As a first order of business, let's decide whether to remove the index and rename `__numpy_ufunc__`. The motivation for this is discussed in issue #5986.
If we decide positive on that (I'm in favor),

It seems like everyone on that issue is in favor or at least +0. So +1 from me too.
 
I would be happy with the proposed name `__array_ufunc__`, although something more descriptive like `__handle_ufunc__` might be better.

 
This is a wonderful opportunity for bike shedding for those so inclined ;)

Let me try to break things down an bit.


Uncontroversial

  • Rename __numpy_ufunc__ to __array_ufunc__
  • Remove index
  • Out is always a tuple

I think this much is useful even if nothing else is done.

Goals

  • Deprecate __array_priority__
  • Ufuncs should succeed or error, never return NotImplemented
  • Add __array_ufunc__ stub to ndarray.

I don't think these are controversial either, but they are longer term except possibly the last. Note that never returning NotImplemented disentangles ufuncs from ndarray binops, which I think is a good thing.

Binops


Here we come to the crux of the last arguments. The functions used for binops can currently be set dynamically, the method that is currently used to set them when the ufunc module is loaded. I think we want to do away with that at some point and fix a set of ufuncs with which they are implemented. This allows folks to overide the binop behavior using __array_ufunc__. I think that is mostly of interest to people who are subclassing ndarray and with that restriction doesn't bother me except that it entangles ufuncs with binops. However, what I'd like to see as well is an opt out for objects that don't care about ufuncs, but want to override the python numeric operators, something simple like `__me_me_me__`, or,  more seriously, `__array_opt_out__`  that will only come into play if the defining object is on the right hand side of an instance of ndarray. In that case the binop would return NotImplemented so as to defer to the Python machinery.  Note that  __array_priority__ is currently (ab)used for this.

Numpy scalars

Numpy scalars default to the corresponding PyArray_Type or PyGenericArrType_Type unless both arguments can be converted to the same c type as the calling scalar, so I don't think there is a problem there. Note that they do check _array_priority_ before trying to convert unknown objects to array scalars in a fallback case.

Chuck


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

Marten van Kerkwijk
Hi All,

I'd very much like to get `__array_ufunc__` in, and am willing to do
some work, but fear we need to get past the last sticking point. As I
noted in Chuck's PR [1], in python 3.6 there is now an explicit
language change [2], which I think is relevant:
```
It is now possible to set a special method to None to indicate that
the corresponding operation is not available. For example, if a class
sets __iter__() to None, the class is not iterable.
```

It seems to me entirely logical (but then it would, I suggested it
before...) that we allow opting out by setting `__array_ufunc__` to
None; in that case, binops return NotImplemented and ufuncs raise
errors. (In addtion, or alternatively, one could allow setting
`__array__` to None, which would generally disable something to be
turned into an array object).

But I should note that I much prefer to get something in over wait yet
another round! In astropy, there is now more and more clamouring to
offer options for pure ndarray functions where quantities are more
logical because quantities are twice as slow -- this would instantly
be solved with __array_ufunc__... If we can decide on this, then I'd
gladly help with remaining issues (e.g., the `ndarray.__array_ufunc__`
method, so super can be used).

All the best,

Marten

[1] https://github.com/numpy/numpy/pull/8247
[2] https://docs.python.org/3.6/whatsnew/3.6.html#other-language-changes
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

Stephan Hoyer-2
On Wed, Feb 22, 2017 at 6:31 AM, Marten van Kerkwijk <[hidden email]> wrote:
It seems to me entirely logical (but then it would, I suggested it
before...) that we allow opting out by setting `__array_ufunc__` to
None; in that case, binops return NotImplemented and ufuncs raise
errors. (In addtion, or alternatively, one could allow setting
`__array__` to None, which would generally disable something to be
turned into an array object).

This is indeed appealing, but I recall this was still a point of contention because it leaves intact two different ways to override arithmetic involving numpy arrays. Mimicking all this logic on classes designed to wrap well-behaved array-like classes (e.g., xarray, which can wrap NumPy or Dask arrays) could be painful -- it's easier to just call np.add and let it handle all the dispatching rather than also worrying about NotImplemented. That said, I think the opt-out is probably OK, as long we make it clear that defining __array_ufunc__ to do arithmetic overriding is the preferred solution (and provide appropriate Mixin classes to make this easier).

Just to be clear: if __array__ = None but __array_ufunc__ is defined, this would be a class that defines array-like operations but can't be directly converted into a NumPy array? For example, a scipy.sparse matrix?

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: __numpy_ufunc__

Marten van Kerkwijk
HI Stephan,

Indeed, `__array_ufunc__` is None would be for classes that interact
with arrays only as if they were any other numeric type, and thus have
no use for ufuncs, but may need normal operations (astropy's `Unit`
class is a reasonably good example).

Your example also makes clear that, indeed, setting __array__ or
__array_ufunc__ to None implies different things, so concretely here
the proposal is that if `__array_ufunc__` is None, ndarray methods
will return `NotImplemented`.

As an aside, I think that if we do not have something like that, we'll
be stuck with supporting `__array_priority__`. (Which is OK by me too,
but it might as well be a conscious choice.)

All the best,

Marten
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Loading...