Choosing between NumPy and SciPy functions

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Choosing between NumPy and SciPy functions

D. Michael McFarland
A recent post raised a question about differences in results obtained
with numpy.linalg.eigh() and scipy.linalg.eigh(), documented at
http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eigh.html#numpy.linalg.eigh
and
http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigh.html#scipy.linalg.eigh,
respectively.  It is clear that these functions address different
mathematical problems (among other things, the SciPy routine can solve
the generalized as well as standard eigenproblems); I am not concerned
here with numerical differences in the results for problems both should
be able to solve (the author of the original post received useful
replies in that thread).

What I would like to ask about is the situation this illustrates, where
both NumPy and SciPy provide similar functionality (sometimes identical,
to judge by the documentation).  Is there some guidance on which is to
be preferred?  I could argue that using only NumPy when possible avoids
unnecessary dependence on SciPy in some code, or that using SciPy
consistently makes for a single interface and so is less error prone.
Is there a rule of thumb for cases where SciPy names shadow NumPy names?

I've used Python for a long time, but have only recently returned to
doing serious numerical work with it.  The tools are very much improved,
but sometimes, like now, I feel I'm missing the obvious.  I would
appreciate pointers to any relevant documentation, or just a summary of
conventional wisdom on the topic.

Regards,
Michael
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Eelco Hoogendoorn
The same occurred to me when reading that question. My personal opinion is that such functionality should be deprecated from numpy. I don't know who said this, but it really stuck with me: but the power of numpy is first and foremost in it being a fantastic interface, not in being a library.

There is nothing more annoying than every project having its own array type. The fact that the whole scientific python stack can so seamlessly communicate is where all good things begin.

In my opinion, that is what numpy should focus on; basic data structures, and tools for manipulating them. Linear algebra is way too high level for numpy imo, and used by only a small subsets of its 'matlab-like' users.

When I get serious about linear algebra or ffts or what have you, id rather import an extra module that wraps a specific library.

On Mon, Oct 27, 2014 at 2:26 PM, D. Michael McFarland <[hidden email]> wrote:
A recent post raised a question about differences in results obtained
with numpy.linalg.eigh() and scipy.linalg.eigh(), documented at
http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eigh.html#numpy.linalg.eigh
and
http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigh.html#scipy.linalg.eigh,
respectively.  It is clear that these functions address different
mathematical problems (among other things, the SciPy routine can solve
the generalized as well as standard eigenproblems); I am not concerned
here with numerical differences in the results for problems both should
be able to solve (the author of the original post received useful
replies in that thread).

What I would like to ask about is the situation this illustrates, where
both NumPy and SciPy provide similar functionality (sometimes identical,
to judge by the documentation).  Is there some guidance on which is to
be preferred?  I could argue that using only NumPy when possible avoids
unnecessary dependence on SciPy in some code, or that using SciPy
consistently makes for a single interface and so is less error prone.
Is there a rule of thumb for cases where SciPy names shadow NumPy names?

I've used Python for a long time, but have only recently returned to
doing serious numerical work with it.  The tools are very much improved,
but sometimes, like now, I feel I'm missing the obvious.  I would
appreciate pointers to any relevant documentation, or just a summary of
conventional wisdom on the topic.

Regards,
Michael
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

josef.pktd


On Mon, Oct 27, 2014 at 2:24 PM, Eelco Hoogendoorn <[hidden email]> wrote:
The same occurred to me when reading that question. My personal opinion is that such functionality should be deprecated from numpy. I don't know who said this, but it really stuck with me: but the power of numpy is first and foremost in it being a fantastic interface, not in being a library.

There is nothing more annoying than every project having its own array type. The fact that the whole scientific python stack can so seamlessly communicate is where all good things begin.

In my opinion, that is what numpy should focus on; basic data structures, and tools for manipulating them. Linear algebra is way too high level for numpy imo, and used by only a small subsets of its 'matlab-like' users.

When I get serious about linear algebra or ffts or what have you, id rather import an extra module that wraps a specific library.

We are not always "getting serious" about linalg, just a quick call to pinv or qr or matrix_rank or similar doesn't necessarily mean we need a linalg library with all advanced options.

"@"  matrix operations and linear algebra are "basic" stuff.

 

On Mon, Oct 27, 2014 at 2:26 PM, D. Michael McFarland <[hidden email]> wrote:
A recent post raised a question about differences in results obtained
with numpy.linalg.eigh() and scipy.linalg.eigh(), documented at
http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eigh.html#numpy.linalg.eigh
and
http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigh.html#scipy.linalg.eigh,
respectively.  It is clear that these functions address different
mathematical problems (among other things, the SciPy routine can solve
the generalized as well as standard eigenproblems); I am not concerned
here with numerical differences in the results for problems both should
be able to solve (the author of the original post received useful
replies in that thread).

What I would like to ask about is the situation this illustrates, where
both NumPy and SciPy provide similar functionality (sometimes identical,
to judge by the documentation).  Is there some guidance on which is to
be preferred?  I could argue that using only NumPy when possible avoids
unnecessary dependence on SciPy in some code, or that using SciPy
consistently makes for a single interface and so is less error prone.
Is there a rule of thumb for cases where SciPy names shadow NumPy names?

I've used Python for a long time, but have only recently returned to
doing serious numerical work with it.  The tools are very much improved,
but sometimes, like now, I feel I'm missing the obvious.  I would
appreciate pointers to any relevant documentation, or just a summary of
conventional wisdom on the topic.


Just as opinion as user:

Most of the time I don't care and treat this just as different versions.
For example in the linalg case, I use by default numpy.linalg and switch to scipy if I need the extras.
pinv is the only one that I ever seriously compared. 

Some details are nicer, np.linalg.qr(x, mode='r') returns the reduced matrix instead of the full matrix as does scipy.linalg.
np.linalg.pinv is faster but maybe slightly less accurate (or defaults that make it less accurate in corner cases). scipy often has more overhead (and isfinite check by default).

I just checked, I didn't even know scipy.linalg also has an `inv`. One of my arguments for np.linalg would have been that it's easy to switch between inv and pinv.

For fft I use mostly scipy, IIRC.   (scipy's fft imports numpy's fft, partially?)


Essentially, I don't care most of the time that there are different ways of doing essentially the same thing, but some better information about the differences would be useful.

Josef

 

Regards,
Michael
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Sturla Molden
<[hidden email]> wrote:

> For fft I use mostly scipy, IIRC.   (scipy's fft imports numpy's fft,
> partially?)

No. SciPy uses the Fortran library FFTPACK (wrapped with f2py) and NumPy
uses a smaller C library called fftpack_lite. Algorithmically they are are
similar, but fftpack_lite has fewer features (e.g. no DCT). scipy.fftpack
does not import numpy.fft. Neither of these libraries are very "fast", but
usually they are "fast enough" for practical purposes. If we really need a
kick-ass fast FFT we need to go to libraries like FFTW, Intel MKL or
Apple's Accelerate Framework, or even use tools like CUDA or OpenCL to run
the FFT on the GPU. But using such tools takes more coding (and reading API
specifications) than the convinience of just using the FFTs already in
NumPy or SciPy. So if you count in your own time as well, it might not be
that FFTW or MKL are the "faster" FFTs.

Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Sturla Molden
Sturla Molden <[hidden email]> wrote:

> If we really need a
> kick-ass fast FFT we need to go to libraries like FFTW, Intel MKL or
> Apple's Accelerate Framework,

I should perhaps also mention FFTS here, which claim to be faster than FFTW
and has a BSD licence:

http://anthonix.com/ffts/index.html

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

josef.pktd
In reply to this post by Sturla Molden


On Mon, Oct 27, 2014 at 10:50 PM, Sturla Molden <[hidden email]> wrote:
<[hidden email]> wrote:

> For fft I use mostly scipy, IIRC.   (scipy's fft imports numpy's fft,
> partially?)

No. SciPy uses the Fortran library FFTPACK (wrapped with f2py) and NumPy
uses a smaller C library called fftpack_lite. Algorithmically they are are
similar, but fftpack_lite has fewer features (e.g. no DCT). scipy.fftpack
does not import numpy.fft. Neither of these libraries are very "fast", but
usually they are "fast enough" for practical purposes. If we really need a
kick-ass fast FFT we need to go to libraries like FFTW, Intel MKL or
Apple's Accelerate Framework, or even use tools like CUDA or OpenCL to run
the FFT on the GPU. But using such tools takes more coding (and reading API
specifications) than the convinience of just using the FFTs already in
NumPy or SciPy. So if you count in your own time as well, it might not be
that FFTW or MKL are the "faster" FFTs.


Ok, I didn't remember correctly.

I didn't use much fft recently, I never used DCT. My favorite "fft function" is fftconvolve.
https://github.com/scipy/scipy/blob/e758c482efb8829685dcf494bdf71eeca3dd77f0/scipy/signal/signaltools.py#L13    doesn't seem to mind mixing numpy and scipy  (quick github search)


It's sometimes useful to have simplified functions that are "good enough" where we don't have to figure out all the extras that the docstring of the fancy version is mentioning.

Josef

 

Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

josef.pktd


On Mon, Oct 27, 2014 at 11:31 PM, <[hidden email]> wrote:


On Mon, Oct 27, 2014 at 10:50 PM, Sturla Molden <[hidden email]> wrote:
<[hidden email]> wrote:

> For fft I use mostly scipy, IIRC.   (scipy's fft imports numpy's fft,
> partially?)

No. SciPy uses the Fortran library FFTPACK (wrapped with f2py) and NumPy
uses a smaller C library called fftpack_lite. Algorithmically they are are
similar, but fftpack_lite has fewer features (e.g. no DCT). scipy.fftpack
does not import numpy.fft. Neither of these libraries are very "fast", but
usually they are "fast enough" for practical purposes. If we really need a
kick-ass fast FFT we need to go to libraries like FFTW, Intel MKL or
Apple's Accelerate Framework, or even use tools like CUDA or OpenCL to run
the FFT on the GPU. But using such tools takes more coding (and reading API
specifications) than the convinience of just using the FFTs already in
NumPy or SciPy. So if you count in your own time as well, it might not be
that FFTW or MKL are the "faster" FFTs.


Ok, I didn't remember correctly.

I didn't use much fft recently, I never used DCT. My favorite "fft function" is fftconvolve.
https://github.com/scipy/scipy/blob/e758c482efb8829685dcf494bdf71eeca3dd77f0/scipy/signal/signaltools.py#L13    doesn't seem to mind mixing numpy and scipy  (quick github search)


It's sometimes useful to have simplified functions that are "good enough" where we don't have to figure out all the extras that the docstring of the fancy version is mentioning.

I take this back (even if it's true), 
because IMO the defaults should work, and I have a tendency to pile on options in my code that are intended for experts.

Josef

 


Josef

 

Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Matthew Brett
In reply to this post by Sturla Molden
Hi,

On Mon, Oct 27, 2014 at 8:07 PM, Sturla Molden <[hidden email]> wrote:

> Sturla Molden <[hidden email]> wrote:
>
>> If we really need a
>> kick-ass fast FFT we need to go to libraries like FFTW, Intel MKL or
>> Apple's Accelerate Framework,
>
> I should perhaps also mention FFTS here, which claim to be faster than FFTW
> and has a BSD licence:
>
> http://anthonix.com/ffts/index.html

Nice.  And a funny New Zealand name too.

Is this an option for us?  Aren't we a little behind the performance
curve on FFT after we lost FFTW?

Matthew
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Sturla Molden
In reply to this post by josef.pktd
<[hidden email]> wrote:

<ahref="https://github.com/scipy/scipy/blob/e758c482efb8829685dcf494bdf71eeca3dd77f0/scipy/signal/signaltools.py#L13">https://github.com/scipy/scipy/blob/e758c482efb8829685dcf494bdf71eeca3dd77f0/scipy/signal/signaltools.py#L13</a>
>    doesn't seem to mind mixing numpy and scipy  (quick github search)

I believe it is because NumPy's FFTs (beginning with 1.9.0) are
thread-safe. But FFTs from numpy.fft and scipy.fftpack should be rather
similar in performance. (Except if you use Enthought, in which case the
former is much faster.)

It seems from the code that fftconvolve does not use overlap-add or
overlap-save. I seem to remember that it did before, but I might be wrong.
Personally I prefer to use overlap-add instead of a very long FFT.

There is also a scipy.fftpack.convolve module. I have not used it though.


Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Sturla Molden
In reply to this post by Matthew Brett
Matthew Brett <[hidden email]> wrote:

> Is this an option for us?  Aren't we a little behind the performance
> curve on FFT after we lost FFTW?

It does not run on Windows because it uses POSIX to allocate executable
memory for tasklets, as i understand it.

By the way, why did we loose FFTW, apart from GPL? One thing to mention
here is that MKL supports the FFTW APIs. If we can use MKL for linalg and
numpy.dot I don't see why we cannot use it for FFT.

On Mac there is also vDSP in Accelerate framework which has an insanely
fast FFT (also claimed to be faster than FFTW). Since it is a system
library there should be no license problems.

There are clearly options if someone wants to work on it and maintain it.

Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

David Cournapeau


On Tue, Oct 28, 2014 at 5:24 AM, Sturla Molden <[hidden email]> wrote:
Matthew Brett <[hidden email]> wrote:

> Is this an option for us?  Aren't we a little behind the performance
> curve on FFT after we lost FFTW?

It does not run on Windows because it uses POSIX to allocate executable
memory for tasklets, as i understand it.

By the way, why did we loose FFTW, apart from GPL? One thing to mention
here is that MKL supports the FFTW APIs. If we can use MKL for linalg and
numpy.dot I don't see why we cannot use it for FFT.

The problem is APIs: MKL, Accelerate, etc... all use a standard API (BLAS/LAPACK), but for FFT, you need to reimplement pretty much the whole thing. Unsurprisingly, this meant the code was not well maintained.

Wrapping non standard, non-BSD libraries makes much more sense in separate libraries in general.

David



On Mac there is also vDSP in Accelerate framework which has an insanely
fast FFT (also claimed to be faster than FFTW). Since it is a system
library there should be no license problems.

There are clearly options if someone wants to work on it and maintain it.

Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Stéfan van der Walt
In reply to this post by D. Michael McFarland
Hi Michael

On 2014-10-27 15:26:58, D. Michael McFarland <[hidden email]> wrote:
> What I would like to ask about is the situation this illustrates, where
> both NumPy and SciPy provide similar functionality (sometimes identical,
> to judge by the documentation).  Is there some guidance on which is to
> be preferred?  I could argue that using only NumPy when possible avoids
> unnecessary dependence on SciPy in some code, or that using SciPy
> consistently makes for a single interface and so is less error prone.
> Is there a rule of thumb for cases where SciPy names shadow NumPy names?

I'm not sure if you've received an answer to your question so far. My
advice: use the SciPy functions.  SciPy is often built on more extensive
Fortran libraries not available during NumPy compilation, and I am not
aware of any cases where a function in NumPy is faster or more extensive
than the equivalent in SciPy.

If you want code that falls back gracefully when SciPy is not available,
you may use the ``numpy.dual`` library.

Regards
Stéfan
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Pierre Barbier de Reuille-2
I would add one element to the discussion: for some (odd) reasons, SciPy is lacking the functions `rfftn` and `irfftn`, functions using half the memory space compared to their non-real equivalent `fftn` and `ifftn`. However, I haven't (yet) seriously tested `scipy.fftpack.fftn` vs. `np.fft.rfftn` to check if there is a serious performance gain (beside memory usage).

Cheers,

Pierre

On Tue Oct 28 2014 at 10:54:00 Stefan van der Walt <[hidden email]> wrote:
Hi Michael

On 2014-10-27 15:26:58, D. Michael McFarland <[hidden email]> wrote:
> What I would like to ask about is the situation this illustrates, where
> both NumPy and SciPy provide similar functionality (sometimes identical,
> to judge by the documentation).  Is there some guidance on which is to
> be preferred?  I could argue that using only NumPy when possible avoids
> unnecessary dependence on SciPy in some code, or that using SciPy
> consistently makes for a single interface and so is less error prone.
> Is there a rule of thumb for cases where SciPy names shadow NumPy names?

I'm not sure if you've received an answer to your question so far. My
advice: use the SciPy functions.  SciPy is often built on more extensive
Fortran libraries not available during NumPy compilation, and I am not
aware of any cases where a function in NumPy is faster or more extensive
than the equivalent in SciPy.

If you want code that falls back gracefully when SciPy is not available,
you may use the ``numpy.dual`` library.

Regards
Stéfan
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Sturla Molden
Pierre Barbier de Reuille <[hidden email]> wrote:

> I would add one element to the discussion: for some (odd) reasons, SciPy is
> lacking the functions `rfftn` and `irfftn`, functions using half the memory
> space compared to their non-real equivalent `fftn` and `ifftn`.

In both NumPy and SciPy the N-dimensional FFTs are implemented in Python.
It is just a Python loop over all the axes, calling fft or rfft on each
axis.

> However, I
> haven't (yet) seriously tested `scipy.fftpack.fftn` vs. `np.fft.rfftn` to
> check if there is a serious performance gain (beside memory usage).

Real-value FFT is implemented with complex-value FFT. You save half the
memory, but not quite half the computation. Apart from that, the FFT in
SciPy is written in Fortran and the FFT in NumPy is written in C, but they
are algorithmically similar. I don't see any good reason why the Fortran
code in SciPy should be faster than the C code in NumPy. It used to be the
case that Fortran was "faster than C", everything else being equal, but
with modern C compilers and CPUs with deep pipelines and branch prediction
this is rarely the case. So I would expect the NumPy rfftn to be slightly
faster than SciPy fftn, but keep in mind that both have a huge Python
overhead.

Sturla

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

D. Michael McFarland
In reply to this post by Stéfan van der Walt
Stefan van der Walt <[hidden email]> writes:

> On 2014-10-27 15:26:58, D. Michael McFarland <[hidden email]> wrote:
>> What I would like to ask about is the situation this illustrates, where
>> both NumPy and SciPy provide similar functionality (sometimes identical,
>> to judge by the documentation).  Is there some guidance on which is to
>> be preferred?
>
> I'm not sure if you've received an answer to your question so far. My
> advice: use the SciPy functions.  SciPy is often built on more extensive
> Fortran libraries not available during NumPy compilation, and I am not
> aware of any cases where a function in NumPy is faster or more extensive
> than the equivalent in SciPy.

The whole thread has been interesting reading (now that I've finally
come back to it...got busy for a few days), but this is the sort of
answer I was hoping for.  Thank you.

Best,
Michael
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Benjamin Root-2
Just to throw in my two cents here. I feel that sometimes, features are tried out first elsewhere (possibly in scipy) and then brought down into numpy after sufficient shakedown time. So, in some cases, I wonder if the numpy version is actually more refined than the scipy version? Of course, there is no way to know this from the documentation, which is a problem. Didn't scipy have nanmean() for a while before Numpy added it in version 1.8?

Ben Root

On Fri, Oct 31, 2014 at 10:26 AM, D. Michael McFarland <[hidden email]> wrote:
Stefan van der Walt <[hidden email]> writes:

> On 2014-10-27 15:26:58, D. Michael McFarland <[hidden email]> wrote:
>> What I would like to ask about is the situation this illustrates, where
>> both NumPy and SciPy provide similar functionality (sometimes identical,
>> to judge by the documentation).  Is there some guidance on which is to
>> be preferred?
>
> I'm not sure if you've received an answer to your question so far. My
> advice: use the SciPy functions.  SciPy is often built on more extensive
> Fortran libraries not available during NumPy compilation, and I am not
> aware of any cases where a function in NumPy is faster or more extensive
> than the equivalent in SciPy.

The whole thread has been interesting reading (now that I've finally
come back to it...got busy for a few days), but this is the sort of
answer I was hoping for.  Thank you.

Best,
Michael
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

josef.pktd


On Fri, Oct 31, 2014 at 11:07 AM, Benjamin Root <[hidden email]> wrote:
Just to throw in my two cents here. I feel that sometimes, features are tried out first elsewhere (possibly in scipy) and then brought down into numpy after sufficient shakedown time. So, in some cases, I wonder if the numpy version is actually more refined than the scipy version? Of course, there is no way to know this from the documentation, which is a problem. Didn't scipy have nanmean() for a while before Numpy added it in version 1.8?

That's true for several functions in scipy.stats. And we have more deprecation in scipy.stats in favor of numpy pending.

part of polynomials is another case, kind of.

But I don't remember any other ones in my time.

(There is also a reverse extension for scipy binned_stats based on the np.histogram code.)

Josef


 

Ben Root

On Fri, Oct 31, 2014 at 10:26 AM, D. Michael McFarland <[hidden email]> wrote:
Stefan van der Walt <[hidden email]> writes:

> On 2014-10-27 15:26:58, D. Michael McFarland <[hidden email]> wrote:
>> What I would like to ask about is the situation this illustrates, where
>> both NumPy and SciPy provide similar functionality (sometimes identical,
>> to judge by the documentation).  Is there some guidance on which is to
>> be preferred?
>
> I'm not sure if you've received an answer to your question so far. My
> advice: use the SciPy functions.  SciPy is often built on more extensive
> Fortran libraries not available during NumPy compilation, and I am not
> aware of any cases where a function in NumPy is faster or more extensive
> than the equivalent in SciPy.

The whole thread has been interesting reading (now that I've finally
come back to it...got busy for a few days), but this is the sort of
answer I was hoping for.  Thank you.

Best,
Michael
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Choosing between NumPy and SciPy functions

Robert Kern-2
In reply to this post by Benjamin Root-2
On Fri, Oct 31, 2014 at 3:07 PM, Benjamin Root <[hidden email]> wrote:
> Just to throw in my two cents here. I feel that sometimes, features are
> tried out first elsewhere (possibly in scipy) and then brought down into
> numpy after sufficient shakedown time. So, in some cases, I wonder if the
> numpy version is actually more refined than the scipy version? Of course,
> there is no way to know this from the documentation, which is a problem.
> Didn't scipy have nanmean() for a while before Numpy added it in version
> 1.8?

Not that often, and these usually get actively deprecated eventually.
Most duplications are of the form Stefan discusses.

--
Robert Kern
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion