Dropping support for Accelerate

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Dropping support for Accelerate

Ilhan Polat
A few months ago, I had the innocent intention to wrap LDLt decomposition routines of LAPACK into SciPy but then I am made aware that the minimum required version of LAPACK/BLAS was due to Accelerate framework. Since then I've been following the core SciPy team and others' discussion on this issue.

We have been exchanging opinions for quite a while now within various SciPy issues and PRs about the ever-increasing Accelerate-related issues and I've compiled a brief summary about the ongoing discussions to reduce the clutter.

First, I would like to kindly invite everyone to contribute and sharpen the cases presented here

https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate

The reason I specifically wanted to post this also in NumPy mailing list is to probe for the situation from the NumPy-Accelerate perspective. Is there any NumPy specific problem that would indirectly effect SciPy should the support for Accelerate is dropped?




_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Nathaniel Smith
I've been wishing we'd stop shipping Accelerate for years, because of
how it breaks multiprocessing – that doesn't seem to be on your list
yet.

On Sat, Jul 22, 2017 at 3:50 AM, Ilhan Polat <[hidden email]> wrote:

> A few months ago, I had the innocent intention to wrap LDLt decomposition
> routines of LAPACK into SciPy but then I am made aware that the minimum
> required version of LAPACK/BLAS was due to Accelerate framework. Since then
> I've been following the core SciPy team and others' discussion on this
> issue.
>
> We have been exchanging opinions for quite a while now within various SciPy
> issues and PRs about the ever-increasing Accelerate-related issues and I've
> compiled a brief summary about the ongoing discussions to reduce the
> clutter.
>
> First, I would like to kindly invite everyone to contribute and sharpen the
> cases presented here
>
> https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
>
> The reason I specifically wanted to post this also in NumPy mailing list is
> to probe for the situation from the NumPy-Accelerate perspective. Is there
> any NumPy specific problem that would indirectly effect SciPy should the
> support for Accelerate is dropped?
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>



--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Ilhan Polat
That's probably because I know nothing about the issue, is there any reference I can read about?

But in general, please feel free populate new items in the wiki page.

On Sun, Jul 23, 2017 at 11:15 AM, Nathaniel Smith <[hidden email]> wrote:
I've been wishing we'd stop shipping Accelerate for years, because of
how it breaks multiprocessing – that doesn't seem to be on your list
yet.

On Sat, Jul 22, 2017 at 3:50 AM, Ilhan Polat <[hidden email]> wrote:
> A few months ago, I had the innocent intention to wrap LDLt decomposition
> routines of LAPACK into SciPy but then I am made aware that the minimum
> required version of LAPACK/BLAS was due to Accelerate framework. Since then
> I've been following the core SciPy team and others' discussion on this
> issue.
>
> We have been exchanging opinions for quite a while now within various SciPy
> issues and PRs about the ever-increasing Accelerate-related issues and I've
> compiled a brief summary about the ongoing discussions to reduce the
> clutter.
>
> First, I would like to kindly invite everyone to contribute and sharpen the
> cases presented here
>
> https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
>
> The reason I specifically wanted to post this also in NumPy mailing list is
> to probe for the situation from the NumPy-Accelerate perspective. Is there
> any NumPy specific problem that would indirectly effect SciPy should the
> support for Accelerate is dropped?
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>



--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Nathan Goldbaum
See https://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html and replies in that thread.

Quote from an Apple engineer in that thread:

"For API outside of POSIX, including GCD and technologies like Accelerate, we do not support usage on both sides of a fork(). For this reason among others, use of fork() without exec is discouraged in general in processes that use layers above POSIX."

On Sun, Jul 23, 2017 at 10:16 AM, Ilhan Polat <[hidden email]> wrote:
That's probably because I know nothing about the issue, is there any reference I can read about?

But in general, please feel free populate new items in the wiki page.

On Sun, Jul 23, 2017 at 11:15 AM, Nathaniel Smith <[hidden email]> wrote:
I've been wishing we'd stop shipping Accelerate for years, because of
how it breaks multiprocessing – that doesn't seem to be on your list
yet.

On Sat, Jul 22, 2017 at 3:50 AM, Ilhan Polat <[hidden email]> wrote:
> A few months ago, I had the innocent intention to wrap LDLt decomposition
> routines of LAPACK into SciPy but then I am made aware that the minimum
> required version of LAPACK/BLAS was due to Accelerate framework. Since then
> I've been following the core SciPy team and others' discussion on this
> issue.
>
> We have been exchanging opinions for quite a while now within various SciPy
> issues and PRs about the ever-increasing Accelerate-related issues and I've
> compiled a brief summary about the ongoing discussions to reduce the
> clutter.
>
> First, I would like to kindly invite everyone to contribute and sharpen the
> cases presented here
>
> https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
>
> The reason I specifically wanted to post this also in NumPy mailing list is
> to probe for the situation from the NumPy-Accelerate perspective. Is there
> any NumPy specific problem that would indirectly effect SciPy should the
> support for Accelerate is dropped?
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>



--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Ilhan Polat
Ouch, that's from 2012 :(  I'll add this thread as a reference to the wiki list.


On Sun, Jul 23, 2017 at 5:22 PM, Nathan Goldbaum <[hidden email]> wrote:
See https://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html and replies in that thread.

Quote from an Apple engineer in that thread:

"For API outside of POSIX, including GCD and technologies like Accelerate, we do not support usage on both sides of a fork(). For this reason among others, use of fork() without exec is discouraged in general in processes that use layers above POSIX."

On Sun, Jul 23, 2017 at 10:16 AM, Ilhan Polat <[hidden email]> wrote:
That's probably because I know nothing about the issue, is there any reference I can read about?

But in general, please feel free populate new items in the wiki page.

On Sun, Jul 23, 2017 at 11:15 AM, Nathaniel Smith <[hidden email]> wrote:
I've been wishing we'd stop shipping Accelerate for years, because of
how it breaks multiprocessing – that doesn't seem to be on your list
yet.

On Sat, Jul 22, 2017 at 3:50 AM, Ilhan Polat <[hidden email]> wrote:
> A few months ago, I had the innocent intention to wrap LDLt decomposition
> routines of LAPACK into SciPy but then I am made aware that the minimum
> required version of LAPACK/BLAS was due to Accelerate framework. Since then
> I've been following the core SciPy team and others' discussion on this
> issue.
>
> We have been exchanging opinions for quite a while now within various SciPy
> issues and PRs about the ever-increasing Accelerate-related issues and I've
> compiled a brief summary about the ongoing discussions to reduce the
> clutter.
>
> First, I would like to kindly invite everyone to contribute and sharpen the
> cases presented here
>
> https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
>
> The reason I specifically wanted to post this also in NumPy mailing list is
> to probe for the situation from the NumPy-Accelerate perspective. Is there
> any NumPy specific problem that would indirectly effect SciPy should the
> support for Accelerate is dropped?
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion
>



--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Matthew Brett
Hi,

On Sun, Jul 23, 2017 at 5:07 PM, Ilhan Polat <[hidden email]> wrote:

> Ouch, that's from 2012 :(  I'll add this thread as a reference to the wiki
> list.
>
>
> On Sun, Jul 23, 2017 at 5:22 PM, Nathan Goldbaum <[hidden email]>
> wrote:
>>
>> See
>> https://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html
>> and replies in that thread.
>>
>> Quote from an Apple engineer in that thread:
>>
>> "For API outside of POSIX, including GCD and technologies like Accelerate,
>> we do not support usage on both sides of a fork(). For this reason among
>> others, use of fork() without exec is discouraged in general in processes
>> that use layers above POSIX."
>>
>> On Sun, Jul 23, 2017 at 10:16 AM, Ilhan Polat <[hidden email]>
>> wrote:
>>>
>>> That's probably because I know nothing about the issue, is there any
>>> reference I can read about?
>>>
>>> But in general, please feel free populate new items in the wiki page.
>>>
>>> On Sun, Jul 23, 2017 at 11:15 AM, Nathaniel Smith <[hidden email]> wrote:
>>>>
>>>> I've been wishing we'd stop shipping Accelerate for years, because of
>>>> how it breaks multiprocessing – that doesn't seem to be on your list
>>>> yet.
>>>>
>>>> On Sat, Jul 22, 2017 at 3:50 AM, Ilhan Polat <[hidden email]>
>>>> wrote:
>>>> > A few months ago, I had the innocent intention to wrap LDLt
>>>> > decomposition
>>>> > routines of LAPACK into SciPy but then I am made aware that the
>>>> > minimum
>>>> > required version of LAPACK/BLAS was due to Accelerate framework. Since
>>>> > then
>>>> > I've been following the core SciPy team and others' discussion on this
>>>> > issue.
>>>> >
>>>> > We have been exchanging opinions for quite a while now within various
>>>> > SciPy
>>>> > issues and PRs about the ever-increasing Accelerate-related issues and
>>>> > I've
>>>> > compiled a brief summary about the ongoing discussions to reduce the
>>>> > clutter.
>>>> >
>>>> > First, I would like to kindly invite everyone to contribute and
>>>> > sharpen the
>>>> > cases presented here
>>>> >
>>>> > https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
>>>> >
>>>> > The reason I specifically wanted to post this also in NumPy mailing
>>>> > list is
>>>> > to probe for the situation from the NumPy-Accelerate perspective. Is
>>>> > there
>>>> > any NumPy specific problem that would indirectly effect SciPy should
>>>> > the
>>>> > support for Accelerate is dropped?
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > _______________________________________________
>>>> > NumPy-Discussion mailing list
>>>> > [hidden email]
>>>> > https://mail.python.org/mailman/listinfo/numpy-discussion
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Nathaniel J. Smith -- https://vorpus.org
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> [hidden email]
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> [hidden email]
>>> https://mail.python.org/mailman/listinfo/numpy-discussion

I added some more discussion, and some links to previous discussion on
the mailing list.

I also pointed to this PR :
https://github.com/MacPython/numpy-wheels/pull/1 - which builds
OpenBLAS wheels for numpy.  The same kind of thing would work fine for
Scipy.

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Nathaniel Smith
I updated the bit about OpenBLAS wheel with some more information on
the status of that work. It's not super important, but FYI.

I also want to disagree with this characterization of the
Accelerate/multiprocessing issue: "This problem was due to a bug in
multiprocessing and is fixed in Python 3.4 and later; Accelerate was
POSIX compliant but multiprocessing was not."

In 3.4 it became possible to *work around* this issue, but it requires
configuring the multiprocessing module in a non-default way, which
means that the common end-user experience is still that they try using
multiprocessing, and they get random hangs with no other feedback, and
then spend hours or days debugging before they discover this
configuration option. (And the problem occurs on MacOS only, so you
get extra fun when e.g. a module is developed on Windows or Linux and
then you give it to a less-technical collaborator on MacOS and it
breaks on their computer and you have no idea why.)

And the workaround is suboptimal -- fork()'s memory-sharing semantics
are very powerful. I've had cases where I could easily and efficiently
solve a problem using multiprocessing in fork() mode, but where
enabling the workaround for Accelerate would have made it impossible.
(Specifically this happened because I had a huge read-only data
structure that I could load once in the parent process, and then all
the child processes could share it through fork's virtual memory
magic; I didn't have enough memory to load two copies of it, yet fork
let me have 10 or 20 virtual copies.)

Technically, yes, mixing threads and fork can't be done in a
POSIX-compliant manner. But no-one runs their code on an abstract
POSIX machine, and on actual systems it's totally possible to make
this work reliably. OpenBLAS does it. Users don't care if Apple is
technically correct, they just want their stuff to work.

-n

On Tue, Jul 25, 2017 at 4:57 AM, Matthew Brett <[hidden email]> wrote:

> Hi,
>
> On Sun, Jul 23, 2017 at 5:07 PM, Ilhan Polat <[hidden email]> wrote:
>> Ouch, that's from 2012 :(  I'll add this thread as a reference to the wiki
>> list.
>>
>>
>> On Sun, Jul 23, 2017 at 5:22 PM, Nathan Goldbaum <[hidden email]>
>> wrote:
>>>
>>> See
>>> https://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html
>>> and replies in that thread.
>>>
>>> Quote from an Apple engineer in that thread:
>>>
>>> "For API outside of POSIX, including GCD and technologies like Accelerate,
>>> we do not support usage on both sides of a fork(). For this reason among
>>> others, use of fork() without exec is discouraged in general in processes
>>> that use layers above POSIX."
>>>
>>> On Sun, Jul 23, 2017 at 10:16 AM, Ilhan Polat <[hidden email]>
>>> wrote:
>>>>
>>>> That's probably because I know nothing about the issue, is there any
>>>> reference I can read about?
>>>>
>>>> But in general, please feel free populate new items in the wiki page.
>>>>
>>>> On Sun, Jul 23, 2017 at 11:15 AM, Nathaniel Smith <[hidden email]> wrote:
>>>>>
>>>>> I've been wishing we'd stop shipping Accelerate for years, because of
>>>>> how it breaks multiprocessing – that doesn't seem to be on your list
>>>>> yet.
>>>>>
>>>>> On Sat, Jul 22, 2017 at 3:50 AM, Ilhan Polat <[hidden email]>
>>>>> wrote:
>>>>> > A few months ago, I had the innocent intention to wrap LDLt
>>>>> > decomposition
>>>>> > routines of LAPACK into SciPy but then I am made aware that the
>>>>> > minimum
>>>>> > required version of LAPACK/BLAS was due to Accelerate framework. Since
>>>>> > then
>>>>> > I've been following the core SciPy team and others' discussion on this
>>>>> > issue.
>>>>> >
>>>>> > We have been exchanging opinions for quite a while now within various
>>>>> > SciPy
>>>>> > issues and PRs about the ever-increasing Accelerate-related issues and
>>>>> > I've
>>>>> > compiled a brief summary about the ongoing discussions to reduce the
>>>>> > clutter.
>>>>> >
>>>>> > First, I would like to kindly invite everyone to contribute and
>>>>> > sharpen the
>>>>> > cases presented here
>>>>> >
>>>>> > https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate
>>>>> >
>>>>> > The reason I specifically wanted to post this also in NumPy mailing
>>>>> > list is
>>>>> > to probe for the situation from the NumPy-Accelerate perspective. Is
>>>>> > there
>>>>> > any NumPy specific problem that would indirectly effect SciPy should
>>>>> > the
>>>>> > support for Accelerate is dropped?
>>>>> >
>>>>> >
>>>>> >
>>>>> >
>>>>> > _______________________________________________
>>>>> > NumPy-Discussion mailing list
>>>>> > [hidden email]
>>>>> > https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Nathaniel J. Smith -- https://vorpus.org
>>>>> _______________________________________________
>>>>> NumPy-Discussion mailing list
>>>>> [hidden email]
>>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> [hidden email]
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>
> I added some more discussion, and some links to previous discussion on
> the mailing list.
>
> I also pointed to this PR :
> https://github.com/MacPython/numpy-wheels/pull/1 - which builds
> OpenBLAS wheels for numpy.  The same kind of thing would work fine for
> Scipy.
>
> Cheers,
>
> Matthew
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion



--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Matthew Brett
On Tue, Jul 25, 2017 at 2:19 PM, Nathaniel Smith <[hidden email]> wrote:
> I updated the bit about OpenBLAS wheel with some more information on
> the status of that work. It's not super important, but FYI.

Maybe remove the bit (of my text) that you crossed out, or removed the
strikethrough and qualify?  At the moment it's confusing, because I
believe what I wrote is correct, so leaving in there and crossed out
looks kinda weird.

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Nathaniel Smith
On Tue, Jul 25, 2017 at 6:48 AM, Matthew Brett <[hidden email]> wrote:
> On Tue, Jul 25, 2017 at 2:19 PM, Nathaniel Smith <[hidden email]> wrote:
>> I updated the bit about OpenBLAS wheel with some more information on
>> the status of that work. It's not super important, but FYI.
>
> Maybe remove the bit (of my text) that you crossed out, or removed the
> strikethrough and qualify?  At the moment it's confusing, because I
> believe what I wrote is correct, so leaving in there and crossed out
> looks kinda weird.

Eh, it's a little weird because there's no specification needed
really, we can implement it any time we want to. It was stalled for a
long time because I ran into arcane technical problems dealing with
the MacOS linker, but that's solved and now it's just stalled due to
lack of attention.

I deleted the text but feel free to qualify further if you think it's useful.

-n

--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Matthew Brett
On Tue, Jul 25, 2017 at 3:00 PM, Nathaniel Smith <[hidden email]> wrote:

> On Tue, Jul 25, 2017 at 6:48 AM, Matthew Brett <[hidden email]> wrote:
>> On Tue, Jul 25, 2017 at 2:19 PM, Nathaniel Smith <[hidden email]> wrote:
>>> I updated the bit about OpenBLAS wheel with some more information on
>>> the status of that work. It's not super important, but FYI.
>>
>> Maybe remove the bit (of my text) that you crossed out, or removed the
>> strikethrough and qualify?  At the moment it's confusing, because I
>> believe what I wrote is correct, so leaving in there and crossed out
>> looks kinda weird.
>
> Eh, it's a little weird because there's no specification needed
> really, we can implement it any time we want to. It was stalled for a
> long time because I ran into arcane technical problems dealing with
> the MacOS linker, but that's solved and now it's just stalled due to
> lack of attention.
>
> I deleted the text but feel free to qualify further if you think it's useful.

Are you saying that we should consider this specification approved
already?  Or that we should go ahead without waiting for approval?  I
guess the latter.  I guess you're saying you think there would be no
bad consequences for doing this if the spec subsequently changed
before being approved?  It might be worth adding something like that
to the text, in case there's somebody who wants to do some work on
that.

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Nathaniel Smith
On Tue, Jul 25, 2017 at 7:05 AM, Matthew Brett <[hidden email]> wrote:

> On Tue, Jul 25, 2017 at 3:00 PM, Nathaniel Smith <[hidden email]> wrote:
>> On Tue, Jul 25, 2017 at 6:48 AM, Matthew Brett <[hidden email]> wrote:
>>> On Tue, Jul 25, 2017 at 2:19 PM, Nathaniel Smith <[hidden email]> wrote:
>>>> I updated the bit about OpenBLAS wheel with some more information on
>>>> the status of that work. It's not super important, but FYI.
>>>
>>> Maybe remove the bit (of my text) that you crossed out, or removed the
>>> strikethrough and qualify?  At the moment it's confusing, because I
>>> believe what I wrote is correct, so leaving in there and crossed out
>>> looks kinda weird.
>>
>> Eh, it's a little weird because there's no specification needed
>> really, we can implement it any time we want to. It was stalled for a
>> long time because I ran into arcane technical problems dealing with
>> the MacOS linker, but that's solved and now it's just stalled due to
>> lack of attention.
>>
>> I deleted the text but feel free to qualify further if you think it's useful.
>
> Are you saying that we should consider this specification approved
> already?  Or that we should go ahead without waiting for approval?  I
> guess the latter.  I guess you're saying you think there would be no
> bad consequences for doing this if the spec subsequently changed
> before being approved?  It might be worth adding something like that
> to the text, in case there's somebody who wants to do some work on
> that.

It's not a PEP. It will never be approved because there is no-one to
approve it :-). The only reason for writing it as a spec is to
potentially help coordinate with others who want to get in on making
these kinds of packages themselves, and the main motivator for that
will be if one of us starts doing it and proves it works...

-n

--
Nathaniel J. Smith -- https://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Matthew Brett
On Tue, Jul 25, 2017 at 3:14 PM, Nathaniel Smith <[hidden email]> wrote:

> On Tue, Jul 25, 2017 at 7:05 AM, Matthew Brett <[hidden email]> wrote:
>> On Tue, Jul 25, 2017 at 3:00 PM, Nathaniel Smith <[hidden email]> wrote:
>>> On Tue, Jul 25, 2017 at 6:48 AM, Matthew Brett <[hidden email]> wrote:
>>>> On Tue, Jul 25, 2017 at 2:19 PM, Nathaniel Smith <[hidden email]> wrote:
>>>>> I updated the bit about OpenBLAS wheel with some more information on
>>>>> the status of that work. It's not super important, but FYI.
>>>>
>>>> Maybe remove the bit (of my text) that you crossed out, or removed the
>>>> strikethrough and qualify?  At the moment it's confusing, because I
>>>> believe what I wrote is correct, so leaving in there and crossed out
>>>> looks kinda weird.
>>>
>>> Eh, it's a little weird because there's no specification needed
>>> really, we can implement it any time we want to. It was stalled for a
>>> long time because I ran into arcane technical problems dealing with
>>> the MacOS linker, but that's solved and now it's just stalled due to
>>> lack of attention.
>>>
>>> I deleted the text but feel free to qualify further if you think it's useful.
>>
>> Are you saying that we should consider this specification approved
>> already?  Or that we should go ahead without waiting for approval?  I
>> guess the latter.  I guess you're saying you think there would be no
>> bad consequences for doing this if the spec subsequently changed
>> before being approved?  It might be worth adding something like that
>> to the text, in case there's somebody who wants to do some work on
>> that.
>
> It's not a PEP. It will never be approved because there is no-one to
> approve it :-).

Sure, but it is a pull-request, it hasn't been merged - so I assume
that someone is expecting to make or receive more feedback on it.

> The only reason for writing it as a spec is to
> potentially help coordinate with others who want to get in on making
> these kinds of packages themselves, and the main motivator for that
> will be if one of us starts doing it and proves it works...

If I had to guess, I'd guess that you are saying Yes to "no bad
consequences" (above)?  Would you mind adding something about that in
the text to make it clear?

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Ilhan Polat

On Tue, Jul 25, 2017 at 4:23 PM, Matthew Brett <[hidden email]> wrote:
On Tue, Jul 25, 2017 at 3:14 PM, Nathaniel Smith <[hidden email]> wrote:
> On Tue, Jul 25, 2017 at 7:05 AM, Matthew Brett <[hidden email]> wrote:
>> On Tue, Jul 25, 2017 at 3:00 PM, Nathaniel Smith <[hidden email]> wrote:
>>> On Tue, Jul 25, 2017 at 6:48 AM, Matthew Brett <[hidden email]> wrote:
>>>> On Tue, Jul 25, 2017 at 2:19 PM, Nathaniel Smith <[hidden email]> wrote:
>>>>> I updated the bit about OpenBLAS wheel with some more information on
>>>>> the status of that work. It's not super important, but FYI.
>>>>
>>>> Maybe remove the bit (of my text) that you crossed out, or removed the
>>>> strikethrough and qualify?  At the moment it's confusing, because I
>>>> believe what I wrote is correct, so leaving in there and crossed out
>>>> looks kinda weird.
>>>
>>> Eh, it's a little weird because there's no specification needed
>>> really, we can implement it any time we want to. It was stalled for a
>>> long time because I ran into arcane technical problems dealing with
>>> the MacOS linker, but that's solved and now it's just stalled due to
>>> lack of attention.
>>>
>>> I deleted the text but feel free to qualify further if you think it's useful.
>>
>> Are you saying that we should consider this specification approved
>> already?  Or that we should go ahead without waiting for approval?  I
>> guess the latter.  I guess you're saying you think there would be no
>> bad consequences for doing this if the spec subsequently changed
>> before being approved?  It might be worth adding something like that
>> to the text, in case there's somebody who wants to do some work on
>> that.
>
> It's not a PEP. It will never be approved because there is no-one to
> approve it :-).

Sure, but it is a pull-request, it hasn't been merged - so I assume
that someone is expecting to make or receive more feedback on it.

> The only reason for writing it as a spec is to
> potentially help coordinate with others who want to get in on making
> these kinds of packages themselves, and the main motivator for that
> will be if one of us starts doing it and proves it works...

If I had to guess, I'd guess that you are saying Yes to "no bad
consequences" (above)?  Would you mind adding something about that in
the text to make it clear?

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Matthew Brett
Hi,

On Sat, Jul 29, 2017 at 11:26 AM, Ilhan Polat <[hidden email]> wrote:
> Yet another twirl to the existing spaghetti
>
> https://www.continuum.io/blog/developer-blog/open-sourcing-anaconda-accelerate
>

Just to avoid some obvious confusion, Anaconda Accelerate is nothing
to do with macOS Accelerate:

"""
Accelerate currently is composed of three different feature sets:

Python wrappers around NVIDIA GPU libraries for linear algebra, FFT,
sparse matrix operations, sorting and searching.
Python wrappers around some of Intel’s MKL Vector Math Library functions
A “data profiler” tool based on cProfile and SnakeViz.
"""

Cheers,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Matthew Brett
On Sat, Jul 29, 2017 at 11:34 AM, Matthew Brett <[hidden email]> wrote:

> Hi,
>
> On Sat, Jul 29, 2017 at 11:26 AM, Ilhan Polat <[hidden email]> wrote:
>> Yet another twirl to the existing spaghetti
>>
>> https://www.continuum.io/blog/developer-blog/open-sourcing-anaconda-accelerate
>>
>
> Just to avoid some obvious confusion, Anaconda Accelerate is nothing
> to do with macOS Accelerate:

Sorry - I didn't mean to imply that anyone was confused apart from me :)

Cheers2,

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Dropping support for Accelerate

Ilhan Polat
If it can confuse you, imagine what would happen to regular users like me. That's why I wanted to mention this in advance that this also needs some sort of a "No this is not related to Anaconda Accelerate" disclaimer at some place if need be.



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Loading...