numpy1.4 dtype issues: scipy.stats & pytables

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

numpy1.4 dtype issues: scipy.stats & pytables

Timmie
Administrator
Hello,
I experienced the following issue with numpy 1.4:

scipy.stats:

Python 2.6.2 (r262:71605, Apr 14 2009, 22:40:02) [MSC v.1500 32 bit (Intel)] on
win32

import scipy.stats as st

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python26\lib\site-packages\scipy\stats\__init__.py", line 7, in <module>
    from stats import *
  File "C:\Python26\lib\site-packages\scipy\stats\stats.py", line 203, in <module>
    from morestats import find_repeats #is only reference to scipy.stats
  File "C:\Python26\lib\site-packages\scipy\stats\morestats.py", line 7, in <module>
    import distributions
  File "C:\Python26\lib\site-packages\scipy\stats\distributions.py", line 27, in
<module>
    import vonmises_cython
  File "numpy.pxd", line 30, in scipy.stats.vonmises_cython
(scipy\stats\vonmises_cython.c:2939)
ValueError: numpy.dtype does not appear to be the correct type object

pytables:

import tables

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python26\lib\site-packages\tables\__init__.py", line 56, in <module>
    from tables.utilsExtension import getPyTablesVersion, getHDF5Version
  File "definitions.pxd", line 138, in tables.utilsExtension
ValueError: numpy.dtype does not appear to be the correct type object


Is this an error in numpy or no the other packages require update in the code?

Thanks,
Timmie

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

Pierre GM-2
On Jan 11, 2010, at 5:10 AM, Tim Michelsen wrote:
> Hello,
> I experienced the following issue with numpy 1.4:
> ...
>
> Is this an error in numpy or no the other packages require update in the code?


Let me guess, you just recently updated numpy ? I'd bet ybut forgot to recompile scipy and pytables...

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

Denis-B
Only 2 of the 21 top-level subpackages draw that warning
with numpy-1.4.0-py2.6-python.org.dmg
scipy-0.7.1-py2.6-python.org.dmg
on my mac 10.4 ppc, python 2.6.4:

try:
     import scipy.cluster
except ValueError, e:
     print "scipy.cluster error", e
try:
     import scipy.constants
except ValueError, e:
     print "scipy.constants error", e
...

scipy.cluster error numpy.dtype does not appear to be the correct type object
.../linsolve/__init__.py:4: DeprecationWarning: scipy.linsolve has moved to scipy.sparse.linalg.dsolve
   warn('scipy.linsolve has moved to scipy.sparse.linalg.dsolve', DeprecationWarning)
scipy.stats error numpy.dtype does not appear to be the correct type object

cheers
   -- denis

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

josef.pktd
On Mon, Jan 11, 2010 at 11:56 AM, denis <[hidden email]> wrote:

> Only 2 of the 21 top-level subpackages draw that warning
> with numpy-1.4.0-py2.6-python.org.dmg
> scipy-0.7.1-py2.6-python.org.dmg
> on my mac 10.4 ppc, python 2.6.4:
>
> try:
>     import scipy.cluster
> except ValueError, e:
>     print "scipy.cluster error", e
> try:
>     import scipy.constants
> except ValueError, e:
>     print "scipy.constants error", e
> ...
>
> scipy.cluster error numpy.dtype does not appear to be the correct type object
> .../linsolve/__init__.py:4: DeprecationWarning: scipy.linsolve has moved to scipy.sparse.linalg.dsolve
>   warn('scipy.linsolve has moved to scipy.sparse.linalg.dsolve', DeprecationWarning)
> scipy.stats error numpy.dtype does not appear to be the correct type object

For this problem, it's supposed to be only those packages that have or
import cython generated code.

Josef

>
> cheers
>   -- denis
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

Denis-B
On 11/01/2010 18:10, [hidden email] wrote:

> For this problem, it's supposed to be only those packages that have or
> import cython generated code.

Right; is this a known bug, is there a known fix  for mac dmgs ?
(Whisper, how'd it get past testing ?)

scipy/stats/__init__.py has an apparent patch which doesn't work
     #remove vonmises_cython from __all__, I don't know why it is included
     __all__ = filter(lambda s:not (s.startswith('_') or s.endswith('cython')),dir())

but just removing vonmises_cython in distributions.py
=> import scipy.stats then works.

Similarly import scipy.cluster => trace
   File "numpy.pxd", line 30, in scipy.spatial.ckdtree (scipy/spatial/ckdtree.c:6087)
ValueError: numpy.dtype does not appear to be the correct type object

I like the naming convention xx_cython.so.

cheers
   -- denis


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

Robert Kern-2
On Tue, Jan 12, 2010 at 10:33, denis <[hidden email]> wrote:
> On 11/01/2010 18:10, [hidden email] wrote:
>
>> For this problem, it's supposed to be only those packages that have or
>> import cython generated code.
>
> Right; is this a known bug, is there a known fix  for mac dmgs ?
> (Whisper, how'd it get past testing ?)

It's not a bug, but it is a known issue. We tried very hard to keep
numpy 1.4 binary compatible; however, Pyrex and Cython impose
additional runtime checks above and beyond binary compatibility.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

josef.pktd
In reply to this post by Denis-B
On Tue, Jan 12, 2010 at 11:33 AM, denis <[hidden email]> wrote:
> On 11/01/2010 18:10, [hidden email] wrote:
>
>> For this problem, it's supposed to be only those packages that have or
>> import cython generated code.
>
> Right; is this a known bug, is there a known fix  for mac dmgs ?
> (Whisper, how'd it get past testing ?)

Switching to numpy 1.4 requires recompiling cython code (i.e. scipy),
there's a lot of information on the details in the mailing lists.

>
> scipy/stats/__init__.py has an apparent patch which doesn't work
>     #remove vonmises_cython from __all__, I don't know why it is included
>     __all__ = filter(lambda s:not (s.startswith('_') or s.endswith('cython')),dir())

No this is unrelated, this is just to reduce namespace pollution in __all__

vonmises_cython is still imported as an internal module and  functions
in distributions.

Josef

>
> but just removing vonmises_cython in distributions.py
> => import scipy.stats then works.

Then, I expect you will get an import error or some other exception
when you try to use stats.vonmises.

>
> Similarly import scipy.cluster => trace
>   File "numpy.pxd", line 30, in scipy.spatial.ckdtree (scipy/spatial/ckdtree.c:6087)
> ValueError: numpy.dtype does not appear to be the correct type object
>
> I like the naming convention xx_cython.so.
>
> cheers
>   -- denis
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

Denis-B
In reply to this post by Robert Kern-2
On 12/01/2010 17:41, Robert Kern wrote:

> It's not a bug, but it is a known issue. We tried very hard to keep
> numpy 1.4 binary compatible; however, Pyrex and Cython impose
> additional runtime checks above and beyond binary compatibility.

Robert, Josef,
   are you saying that mac users shouldn't expect
numpy-1.4.0-py2.6-python.org.dmg
scipy-0.7.1-py2.6-python.org.dmg
to "just work" together, download and go ?
If not, then the download pages should clearly say "... may not work with ..."
(If they weren't tested together, that's imho a problem in the process;
I realize that testing is hard work, no glory.)

cheers
   -- denis





_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy1.4 dtype issues: scipy.stats & pytables

David Cournapeau-3
denis wrote:

> On 12/01/2010 17:41, Robert Kern wrote:
>
>> It's not a bug, but it is a known issue. We tried very hard to keep
>> numpy 1.4 binary compatible; however, Pyrex and Cython impose
>> additional runtime checks above and beyond binary compatibility.
>
> Robert, Josef,
>    are you saying that mac users shouldn't expect
> numpy-1.4.0-py2.6-python.org.dmg
> scipy-0.7.1-py2.6-python.org.dmg
> to "just work" together, download and go ?

It would not work for the concerned subpackages, no.

> If not, then the download pages should clearly say "... may not work with ..."
> (If they weren't tested together, that's imho a problem in the process;
> I realize that testing is hard work, no glory.)

It is not so much hard-work than time consuming, at least as long as we
don't have automated testing of binaries.

Unfortunately, the problem was not caught properly during the beta phase,

David
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion