Request for comments on PEP 637 - Support for indexing with keyword arguments

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Request for comments on PEP 637 - Support for indexing with keyword arguments

Stefano Borini
Dear all,

I would like to know your opinion on how to address a specific need of
the new PEP 637:

https://github.com/python/peps/blob/master/pep-0637.txt

Such PEP would make a syntax like the following valid

    obj[2, x=23]
    obj[2, 4, x=23]

Which would resolve to a call in the form

    type(obj).__getitem__(obj, 2, x=23)
    type(obj).__getitem__(obj, (2, 4), x=23)

and similar for set and del.
After discussion, we currently have one open point we are unsure how
to address, that is what to pass when no positional index is passed,
that is:

    obj[x=23]

We are unsure if we should resolve this call with None or the empty
tuple in the positional index:

    type(obj).__getitem__(obj, None, x=23)
    type(obj).__getitem__(obj, (), x=23)

You can see a detailed discussion in the PEP at L913

https://github.com/python/peps/blob/1fb19ac3a57c9793669ea9532fb840861d4d7566/pep-0637.txt#L913

One of the commenters on python-ideas reported that using None might
introduce an issue in numpy, as None is used to create new axes, hence
the proposal for rejection of None as a solution.
However, we are unsure how strongly this would affect numpy and
similar packages, as well as what developer will find more natural to
receive in that case. We would like to hear your opinion on the topic.

Thank you for your help.


--
Kind regards,

Stefano Borini
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Request for comments on PEP 637 - Support for indexing with keyword arguments

Stephan Hoyer-2


On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini <[hidden email]> wrote:
Dear all,





I would like to know your opinion on how to address a specific need of


the new PEP 637:





https://github.com/python/peps/blob/master/pep-0637.txt





Such PEP would make a syntax like the following valid





    obj[2, x=23]


    obj[2, 4, x=23]





Which would resolve to a call in the form





    type(obj).__getitem__(obj, 2, x=23)


    type(obj).__getitem__(obj, (2, 4), x=23)





and similar for set and del.


After discussion, we currently have one open point we are unsure how


to address, that is what to pass when no positional index is passed,


that is:





    obj[x=23]





We are unsure if we should resolve this call with None or the empty


tuple in the positional index:





    type(obj).__getitem__(obj, None, x=23)


    type(obj).__getitem__(obj, (), x=23)




Hi Stefano -- thanks for pushing this proposal forward! I am sure that support for keyword indexing will be very welcome in the scientific Python ecosystem.

I have not been following the full discussion on PEP 637, but I recall seeing another suggestion earlier for what this could be resolved into:

    type(obj).__getitem__(obj, x=23)

I.e., not passing a positional argument at all.

The author of a class that supports keyword indexing could check this sort of case with a positional only argument with a default value, e.g.,

    def __getitem__(self, key=MY_SENTINEL, /, **kwargs):

where MY_SENTINEL could be any desired sentinel value, including either None or (). Is there a reason for rejecting this option? It seems like a nice explicit alternative to prespecifing the choice of sentinel value.

I guess the concern might be that this would not suffice for __setitem__?
 


You can see a detailed discussion in the PEP at L913





https://github.com/python/peps/blob/1fb19ac3a57c9793669ea9532fb840861d4d7566/pep-0637.txt#L913





One of the commenters on python-ideas reported that using None might


introduce an issue in numpy, as None is used to create new axes, hence


the proposal for rejection of None as a solution.


However, we are unsure how strongly this would affect numpy and


similar packages, as well as what developer will find more natural to


receive in that case. We would like to hear your opinion on the topic.

I guess the case this would disallow is distinguishing between obj[None, x=23] and obj[x=23]?

Yes, this could be a little awkward potentially. The tuple would definitely be more natural for NumPy users, given the that first step of __getitem__/__setitem__ methods in the broader NumPy ecosystem is typically packing non-tuple keys into a tuple, e.g.,

    def __getitem__(self, key):
        if not isinstance(key, tuple):
            key = (key,)
        ...

That said:
- NumPy itself is unlikely to support keyword indexing anytime soon.
- New packages could encourage using explicit aliases like "np.newaxis" instead of "None", which in general is a best practice already.
- The combined use of keyword indexing *and* insertion of new axes at the same time strikes me as something that would be unusual in practice. From what I've seen, it is most useful to either use entirely unnamed or entirely named axes. In the later case, I might write something like obj[x=None] to indicate inserting a new dimension with the name "x".

I think we could definitely live with it either way. I would lean towards using an empty tuple, but I agree that it feels a little uglier than using None (though perhaps not surprising, given the already ugly special cases for tuples in the indexing protocol).

Best,
Stephan



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Request for comments on PEP 637 - Support for indexing with keyword arguments

Sebastian Berg
On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
> On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini <
> [hidden email]>
> wrote:

<snip>

> >
> Hi Stefano -- thanks for pushing this proposal forward! I am sure
> that
> support for keyword indexing will be very welcome in the scientific
> Python
> ecosystem.
>
> I have not been following the full discussion on PEP 637, but I
> recall
> seeing another suggestion earlier for what this could be resolved
> into:
>
>     type(obj).__getitem__(obj, x=23)
>
> I.e., not passing a positional argument at all.
>
> The author of a class that supports keyword indexing could check this
> sort
> of case with a positional only argument with a default value, e.g.,
>
>     def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
>
> where MY_SENTINEL could be any desired sentinel value, including
> either
> None or (). Is there a reason for rejecting this option? It seems
> like a
> nice explicit alternative to prespecifing the choice of sentinel
> value.
>
> I guess the concern might be that this would not suffice for
> __setitem__?
>
>
> >
> > You can see a detailed discussion in the PEP at L913
<snip>

> I guess the case this would disallow is distinguishing between
> obj[None,
> x=23] and obj[x=23]?
>
> Yes, this could be a little awkward potentially. The tuple would
> definitely
> be more natural for NumPy users, given the that first step of
> __getitem__/__setitem__ methods in the broader NumPy ecosystem is
> typically
> packing non-tuple keys into a tuple, e.g.,
>
>     def __getitem__(self, key):
>         if not isinstance(key, tuple):
>             key = (key,)
>         ...
>
> That said:
> - NumPy itself is unlikely to support keyword indexing anytime soon.
> - New packages could encourage using explicit aliases like
> "np.newaxis"
> instead of "None", which in general is a best practice already.
> - The combined use of keyword indexing *and* insertion of new axes at
> the
> same time strikes me as something that would be unusual in practice.
> From
> what I've seen, it is most useful to either use entirely unnamed or
> entirely named axes. In the later case, I might write something like
> obj[x=None] to indicate inserting a new dimension with the name "x".
>
Just to briefly second these points and the general support thanks for
the hard work!  I do also wonder about the `key=custom_default`
solution, or whether there may be other option to address this.

For NumPy, I do hope we can play with the idea of adding support if
this PEP lands. But agree that labeled axes in NumPy is unlikely to be
on the immediate horizon, and I am not sure how feasible it is.

My main in the discussion on python-ideas was `arr[]` doing something
unexpected, but it was long decided that it will remain a SyntaxError.

For the question at hand, it seems to me that mixing labeled and
unlabeled indexing would be an error for array-like objects.
In that case, the worst we get seems a quirk where `arr[None, x=4]` is
not an error, when it should be an error.
That does not really strike me as a blocker.

I am not a fan of such quirks. But some trade-off seems unavoidable
considering the backward compatibility constraints and differences
between array-likes and typing use of `__getitem__`.

- Sebastian


> I think we could definitely live with it either way. I would lean
> towards
> using an empty tuple, but I agree that it feels a little uglier than
> using
> None (though perhaps not surprising, given the already ugly special
> cases
> for tuples in the indexing protocol).
>
> Best,
> Stephan
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Request for comments on PEP 637 - Support for indexing with keyword arguments

Aaron Meurer
This PEP also opens the possibility of allowing a[] with nothing in the getitem. Has that been considered?

Aaron Meurer

On Thu, Sep 24, 2020 at 1:48 PM Sebastian Berg <[hidden email]> wrote:
On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
> On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini <
> [hidden email]>
> wrote:

<snip>

> >
> Hi Stefano -- thanks for pushing this proposal forward! I am sure
> that
> support for keyword indexing will be very welcome in the scientific
> Python
> ecosystem.
>
> I have not been following the full discussion on PEP 637, but I
> recall
> seeing another suggestion earlier for what this could be resolved
> into:
>
>     type(obj).__getitem__(obj, x=23)
>
> I.e., not passing a positional argument at all.
>
> The author of a class that supports keyword indexing could check this
> sort
> of case with a positional only argument with a default value, e.g.,
>
>     def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
>
> where MY_SENTINEL could be any desired sentinel value, including
> either
> None or (). Is there a reason for rejecting this option? It seems
> like a
> nice explicit alternative to prespecifing the choice of sentinel
> value.
>
> I guess the concern might be that this would not suffice for
> __setitem__?
>
>
> >
> > You can see a detailed discussion in the PEP at L913

<snip>

> I guess the case this would disallow is distinguishing between
> obj[None,
> x=23] and obj[x=23]?
>
> Yes, this could be a little awkward potentially. The tuple would
> definitely
> be more natural for NumPy users, given the that first step of
> __getitem__/__setitem__ methods in the broader NumPy ecosystem is
> typically
> packing non-tuple keys into a tuple, e.g.,
>
>     def __getitem__(self, key):
>         if not isinstance(key, tuple):
>             key = (key,)
>         ...
>
> That said:
> - NumPy itself is unlikely to support keyword indexing anytime soon.
> - New packages could encourage using explicit aliases like
> "np.newaxis"
> instead of "None", which in general is a best practice already.
> - The combined use of keyword indexing *and* insertion of new axes at
> the
> same time strikes me as something that would be unusual in practice.
> From
> what I've seen, it is most useful to either use entirely unnamed or
> entirely named axes. In the later case, I might write something like
> obj[x=None] to indicate inserting a new dimension with the name "x".
>

Just to briefly second these points and the general support thanks for
the hard work!  I do also wonder about the `key=custom_default`
solution, or whether there may be other option to address this.

For NumPy, I do hope we can play with the idea of adding support if
this PEP lands. But agree that labeled axes in NumPy is unlikely to be
on the immediate horizon, and I am not sure how feasible it is.

My main in the discussion on python-ideas was `arr[]` doing something
unexpected, but it was long decided that it will remain a SyntaxError.

For the question at hand, it seems to me that mixing labeled and
unlabeled indexing would be an error for array-like objects.
In that case, the worst we get seems a quirk where `arr[None, x=4]` is
not an error, when it should be an error.
That does not really strike me as a blocker.

I am not a fan of such quirks. But some trade-off seems unavoidable
considering the backward compatibility constraints and differences
between array-likes and typing use of `__getitem__`.

- Sebastian


> I think we could definitely live with it either way. I would lean
> towards
> using an empty tuple, but I agree that it feels a little uglier than
> using
> None (though perhaps not surprising, given the already ugly special
> cases
> for tuples in the indexing protocol).
>
> Best,
> Stephan
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Request for comments on PEP 637 - Support for indexing with keyword arguments

Eric Wieser
I agree with Stephan's suggestion of having no default value for positional indices, and letting the user supply it.

It seems I replied badly to the mail on the python-ideas list, and my response ended up as a separate thread, at  https://mail.python.org/archives/list/python-ideas@.../thread/ZQ4CHT37YUHYEFYFBQ423QNZMFCRULAW/.

Eric

On Thu, 24 Sep 2020 at 22:22, Aaron Meurer <[hidden email]> wrote:
This PEP also opens the possibility of allowing a[] with nothing in the getitem. Has that been considered?

Aaron Meurer

On Thu, Sep 24, 2020 at 1:48 PM Sebastian Berg <[hidden email]> wrote:
On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
> On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini <
> [hidden email]>
> wrote:

<snip>

> >
> Hi Stefano -- thanks for pushing this proposal forward! I am sure
> that
> support for keyword indexing will be very welcome in the scientific
> Python
> ecosystem.
>
> I have not been following the full discussion on PEP 637, but I
> recall
> seeing another suggestion earlier for what this could be resolved
> into:
>
>     type(obj).__getitem__(obj, x=23)
>
> I.e., not passing a positional argument at all.
>
> The author of a class that supports keyword indexing could check this
> sort
> of case with a positional only argument with a default value, e.g.,
>
>     def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
>
> where MY_SENTINEL could be any desired sentinel value, including
> either
> None or (). Is there a reason for rejecting this option? It seems
> like a
> nice explicit alternative to prespecifing the choice of sentinel
> value.
>
> I guess the concern might be that this would not suffice for
> __setitem__?
>
>
> >
> > You can see a detailed discussion in the PEP at L913

<snip>

> I guess the case this would disallow is distinguishing between
> obj[None,
> x=23] and obj[x=23]?
>
> Yes, this could be a little awkward potentially. The tuple would
> definitely
> be more natural for NumPy users, given the that first step of
> __getitem__/__setitem__ methods in the broader NumPy ecosystem is
> typically
> packing non-tuple keys into a tuple, e.g.,
>
>     def __getitem__(self, key):
>         if not isinstance(key, tuple):
>             key = (key,)
>         ...
>
> That said:
> - NumPy itself is unlikely to support keyword indexing anytime soon.
> - New packages could encourage using explicit aliases like
> "np.newaxis"
> instead of "None", which in general is a best practice already.
> - The combined use of keyword indexing *and* insertion of new axes at
> the
> same time strikes me as something that would be unusual in practice.
> From
> what I've seen, it is most useful to either use entirely unnamed or
> entirely named axes. In the later case, I might write something like
> obj[x=None] to indicate inserting a new dimension with the name "x".
>

Just to briefly second these points and the general support thanks for
the hard work!  I do also wonder about the `key=custom_default`
solution, or whether there may be other option to address this.

For NumPy, I do hope we can play with the idea of adding support if
this PEP lands. But agree that labeled axes in NumPy is unlikely to be
on the immediate horizon, and I am not sure how feasible it is.

My main in the discussion on python-ideas was `arr[]` doing something
unexpected, but it was long decided that it will remain a SyntaxError.

For the question at hand, it seems to me that mixing labeled and
unlabeled indexing would be an error for array-like objects.
In that case, the worst we get seems a quirk where `arr[None, x=4]` is
not an error, when it should be an error.
That does not really strike me as a blocker.

I am not a fan of such quirks. But some trade-off seems unavoidable
considering the backward compatibility constraints and differences
between array-likes and typing use of `__getitem__`.

- Sebastian


> I think we could definitely live with it either way. I would lean
> towards
> using an empty tuple, but I agree that it feels a little uglier than
> using
> None (though perhaps not surprising, given the already ugly special
> cases
> for tuples in the indexing protocol).
>
> Best,
> Stephan
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Request for comments on PEP 637 - Support for indexing with keyword arguments

Hameer Abbasi
Hello,

For consistency with the current API, I’d suggest an empty tuple.

arr[] # arr.__setitem__(())
arr[1] # arr.__setitem__(1) (could be the only exception)
arr[1, 2] # arr.__setitem__((1, 2)) # NOT a.__setitem__(1, 2), note the missing parantheses
arr[*a, **kw] # arr.__setitem__(a, **kw), perhaps a future extension

If we do
a[] # arr.__setitem__()
a[1] # arr.__setitem__(1)
a[1, 2] # arr.__setitem__((1, 2))
a[1, 2, x=3] # arr.__setitem__((1, 2), x=3)
a[*a, **kw] # arr.__setitem__(a, **kw)

But honestly, if I’m being candid, the form I like most is:

arr[*a, **kw] # arr.__setitem__(a, kw)
arr[*a] # arr.__setitem__(a)
arr[**kw] # arr.__setitem__((), kw)

This is driven by my love for generality and hatred of special cases. So I’m +1 for empty tuple or the third form.

Best regards,
Hameer Abbasi
--
Sent from Canary

On Friday, Sep 25, 2020 at 4:22 PM, Eric Wieser <[hidden email]> wrote:
I agree with Stephan's suggestion of having no default value for positional indices, and letting the user supply it.

It seems I replied badly to the mail on the python-ideas list, and my response ended up as a separate thread, at  https://mail.python.org/archives/list/python-ideas@.../thread/ZQ4CHT37YUHYEFYFBQ423QNZMFCRULAW/.

Eric

On Thu, 24 Sep 2020 at 22:22, Aaron Meurer <[hidden email]> wrote:
This PEP also opens the possibility of allowing a[] with nothing in the getitem. Has that been considered?

Aaron Meurer

On Thu, Sep 24, 2020 at 1:48 PM Sebastian Berg <[hidden email]> wrote:
On Wed, 2020-09-23 at 21:41 -0700, Stephan Hoyer wrote:
> On Wed, Sep 23, 2020 at 2:22 PM Stefano Borini <
> [hidden email]>
> wrote:

<snip>

> >
> Hi Stefano -- thanks for pushing this proposal forward! I am sure
> that
> support for keyword indexing will be very welcome in the scientific
> Python
> ecosystem.
>
> I have not been following the full discussion on PEP 637, but I
> recall
> seeing another suggestion earlier for what this could be resolved
> into:
>
>     type(obj).__getitem__(obj, x=23)
>
> I.e., not passing a positional argument at all.
>
> The author of a class that supports keyword indexing could check this
> sort
> of case with a positional only argument with a default value, e.g.,
>
>     def __getitem__(self, key=MY_SENTINEL, /, **kwargs):
>
> where MY_SENTINEL could be any desired sentinel value, including
> either
> None or (). Is there a reason for rejecting this option? It seems
> like a
> nice explicit alternative to prespecifing the choice of sentinel
> value.
>
> I guess the concern might be that this would not suffice for
> __setitem__?
>
>
> >
> > You can see a detailed discussion in the PEP at L913

<snip>

> I guess the case this would disallow is distinguishing between
> obj[None,
> x=23] and obj[x=23]?
>
> Yes, this could be a little awkward potentially. The tuple would
> definitely
> be more natural for NumPy users, given the that first step of
> __getitem__/__setitem__ methods in the broader NumPy ecosystem is
> typically
> packing non-tuple keys into a tuple, e.g.,
>
>     def __getitem__(self, key):
>         if not isinstance(key, tuple):
>             key = (key,)
>         ...
>
> That said:
> - NumPy itself is unlikely to support keyword indexing anytime soon.
> - New packages could encourage using explicit aliases like
> "np.newaxis"
> instead of "None", which in general is a best practice already.
> - The combined use of keyword indexing *and* insertion of new axes at
> the
> same time strikes me as something that would be unusual in practice.
> From
> what I've seen, it is most useful to either use entirely unnamed or
> entirely named axes. In the later case, I might write something like
> obj[x=None] to indicate inserting a new dimension with the name "x".
>

Just to briefly second these points and the general support thanks for
the hard work!  I do also wonder about the `key=custom_default`
solution, or whether there may be other option to address this.

For NumPy, I do hope we can play with the idea of adding support if
this PEP lands. But agree that labeled axes in NumPy is unlikely to be
on the immediate horizon, and I am not sure how feasible it is.

My main in the discussion on python-ideas was `arr[]` doing something
unexpected, but it was long decided that it will remain a SyntaxError.

For the question at hand, it seems to me that mixing labeled and
unlabeled indexing would be an error for array-like objects.
In that case, the worst we get seems a quirk where `arr[None, x=4]` is
not an error, when it should be an error.
That does not really strike me as a blocker.

I am not a fan of such quirks. But some trade-off seems unavoidable
considering the backward compatibility constraints and differences
between array-likes and typing use of `__getitem__`.

- Sebastian


> I think we could definitely live with it either way. I would lean
> towards
> using an empty tuple, but I agree that it feels a little uglier than
> using
> None (though perhaps not surprising, given the already ugly special
> cases
> for tuples in the indexing protocol).
>
> Best,
> Stephan
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion