Re: numpyx.pyx (recent svn) works?

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
Hey,

On Thu, Feb 28, 2008 at 9:17 AM, Robert Kern <[hidden email]> wrote:
> On Thu, Feb 28, 2008 at 7:35 AM, Neal Becker <[hidden email]> wrote:
>  > I tried numpyx.pyx with cython-0.9.6.12.
>
>  These were written for and still work with Pyrex. If it doesn't work
>  with Cython then that is either a bug in Cython or an intentional
>  incompatibility of Cython.

I just updated  all this code to run with cython here:

https://code.launchpad.net/~fdo.perez/+junk/numpy-cython

Do we want to just switch over to cython and use this instead?  If
people think that the cython switch is OK for 1.0.5 I can do it, but
I'm not about to touch this before others approve...

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Stéfan van der Walt
On 09/04/2008, Fernando Perez <[hidden email]> wrote:
>  Do we want to just switch over to cython and use this instead?  If
>  people think that the cython switch is OK for 1.0.5 I can do it, but
>  I'm not about to touch this before others approve...

I'm in favour of switching -- pyrex is as good as unmaintained, and in
contrast cython is improving by the day.  Do the changes cause
problems with pyrex, or can we even have both working at the same
time?  Either way, we should add a note to the docstring.

Regards
Stéfan
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Gael Varoquaux
On Wed, Apr 09, 2008 at 01:15:23AM +0200, Stéfan van der Walt wrote:
> I'm in favour of switching -- pyrex is as good as unmaintained, and in
> contrast cython is improving by the day.  Do the changes cause
> problems with pyrex, or can we even have both working at the same
> time?  Either way, we should add a note to the docstring.

How about waiting for after 1.0.5 release. It should be anytime soon now,
and let us not risk breaking the workflow of potential contributors for
little gain.

I have little to say on the subject, as I am not helping to get the
release out, but here are my 2 cents.

Gaël
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Robert Kern-2
On Tue, Apr 8, 2008 at 4:17 PM, Gael Varoquaux
<[hidden email]> wrote:
> On Wed, Apr 09, 2008 at 01:15:23AM +0200, Stéfan van der Walt wrote:
>  > I'm in favour of switching -- pyrex is as good as unmaintained, and in
>  > contrast cython is improving by the day.  Do the changes cause
>  > problems with pyrex, or can we even have both working at the same
>  > time?  Either way, we should add a note to the docstring.
>
>  How about waiting for after 1.0.5 release. It should be anytime soon now,
>  and let us not risk breaking the workflow of potential contributors for
>  little gain.

I'm not sure how it could. It's example code, not part of numpy itself.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Gael Varoquaux
On Tue, Apr 08, 2008 at 04:20:28PM -0700, Robert Kern wrote:
> I'm not sure how it could. It's example code, not part of numpy itself.

OK, maybe I should keep my 2 cents. They appear to be forged money, and
worth nothing.

:-).

Gaël
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
On Tue, Apr 8, 2008 at 4:21 PM, Gael Varoquaux
<[hidden email]> wrote:
> On Tue, Apr 08, 2008 at 04:20:28PM -0700, Robert Kern wrote:
>  > I'm not sure how it could. It's example code, not part of numpy itself.
>
>  OK, maybe I should keep my 2 cents. They appear to be forged money, and
>  worth nothing.
>
>  :-).

I *could* make it pyrex/cython-valid, it's trivial but just adds noise
IMHO...  As Stefan said, pyrex is essentially unmaintained as far as
publicly-visible development goes, while cython is very actively
moving ahead and likely picking up better numpy support soon (thanks
to Dag and other GSoC work), so why not just follow that?

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Gael Varoquaux
On Tue, Apr 08, 2008 at 04:28:11PM -0700, Fernando Perez wrote:
> I *could* make it pyrex/cython-valid, it's trivial but just adds noise
> IMHO...  As Stefan said, pyrex is essentially unmaintained as far as
> publicly-visible development goes, while cython is very actively
> moving ahead and likely picking up better numpy support soon (thanks
> to Dag and other GSoC work), so why not just follow that?

Because Cython is not yet shipped be all the major distros (including not
Enthought's EPD current Beta, AFAIK), so it rules out a fair amount of
people who cannot/do not want to install something not shipped.

Note that I don't care that much amount this issue. I am just explaining
why I think it should be after 1.0.5 ships.

Cheers,

Gaël
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
On Tue, Apr 8, 2008 at 4:31 PM, Gael Varoquaux
<[hidden email]> wrote:

> On Tue, Apr 08, 2008 at 04:28:11PM -0700, Fernando Perez wrote:
>  > I *could* make it pyrex/cython-valid, it's trivial but just adds noise
>  > IMHO...  As Stefan said, pyrex is essentially unmaintained as far as
>  > publicly-visible development goes, while cython is very actively
>  > moving ahead and likely picking up better numpy support soon (thanks
>  > to Dag and other GSoC work), so why not just follow that?
>
>  Because Cython is not yet shipped be all the major distros (including not
>  Enthought's EPD current Beta, AFAIK), so it rules out a fair amount of
>  people who cannot/do not want to install something not shipped.

Unfortunately this is a case where the tool that is distributed
(pyrex) has serious limitations for our purposes, and will hopefully
soon (as Cython grows numpy support) be simply unusable: anyone
wanting numpy/.pyx use will HAVE to use cython.   So in this case, the
above argument doesn't really apply: if the shipped tool simply
doesn't do what you need, having it available by default does you zero
good.

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Travis Oliphant-5
In reply to this post by Fernando Perez
Fernando Perez wrote:

> On Tue, Apr 8, 2008 at 4:21 PM, Gael Varoquaux
> <[hidden email]> wrote:
>  
>> On Tue, Apr 08, 2008 at 04:20:28PM -0700, Robert Kern wrote:
>>  > I'm not sure how it could. It's example code, not part of numpy itself.
>>
>>  OK, maybe I should keep my 2 cents. They appear to be forged money, and
>>  worth nothing.
>>
>>  :-).
>>    
>
> I *could* make it pyrex/cython-valid, it's trivial but just adds noise
> IMHO...  As Stefan said, pyrex is essentially unmaintained as far as
> publicly-visible development goes, while cython is very actively
> moving ahead and likely picking up better numpy support soon (thanks
> to Dag and other GSoC work), so why not just follow that?
>  
I say just add it.  We should move forward with Cython.   More important
is to see if random actually builds with Cython right now.  There was an
issue that I recall from a few weeks ago that Cython could not build the
pyrex extension in NumPy.

-Travis



_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
On Tue, Apr 8, 2008 at 4:38 PM, Travis E. Oliphant
<[hidden email]> wrote:

>  I say just add it.  We should move forward with Cython.   More important
>  is to see if random actually builds with Cython right now.  There was an
>  issue that I recall from a few weeks ago that Cython could not build the
>  pyrex extension in NumPy.

OK, I'll play with random for a bit.

BTW, the original 'bug' that started this thread is due to a change in
Cython's casting behavior explained here:

http://wiki.cython.org/DifferencesFromPyrex

it's fixed with a simple extra (void *) cast as shown here:

    print 'Printing array info for ndarray at 0x%0lx'% \
          (<Py_intptr_t><void *>arr,)


I just committed that code to the bzr branch.

In summary:

1. Do you want me to obliterate the old numpy/doc/pyrex and replace it
with numpy/doc/cython? That would make it clear what the tool to use
is...

2. I'll work on random and report shortly.

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Andrew Straw
This is off-topic and should be directed to the pyrex/cython list, but
since we're on the subject:

I suppose the following is true, but let me ask, since I have not used
Cython. Please correct me if I'm wrong.

I have a bunch of pyrex compiled .pyx code. If I start adding some
Cython compiled code, all the extensions will live happily without
colliding, even if I don't re-compile my old .pyx files with Cython.

(I can't imagine how they'd conflict, since they both translate to .c
extensions without anything to collide. Or are there linker-level
symbols that collide or something difficult?)

-Andrew

Fernando Perez wrote:

> On Tue, Apr 8, 2008 at 4:38 PM, Travis E. Oliphant
> <[hidden email]> wrote:
>
>  
>>  I say just add it.  We should move forward with Cython.   More important
>>  is to see if random actually builds with Cython right now.  There was an
>>  issue that I recall from a few weeks ago that Cython could not build the
>>  pyrex extension in NumPy.
>>    
>
> OK, I'll play with random for a bit.
>
> BTW, the original 'bug' that started this thread is due to a change in
> Cython's casting behavior explained here:
>
> http://wiki.cython.org/DifferencesFromPyrex
>
> it's fixed with a simple extra (void *) cast as shown here:
>
>     print 'Printing array info for ndarray at 0x%0lx'% \
>           (<Py_intptr_t><void *>arr,)
>
>
> I just committed that code to the bzr branch.
>
> In summary:
>
> 1. Do you want me to obliterate the old numpy/doc/pyrex and replace it
> with numpy/doc/cython? That would make it clear what the tool to use
> is...
>
> 2. I'll work on random and report shortly.
>
> Cheers,
>
> f
> _______________________________________________
> Numpy-discussion mailing list
> [hidden email]
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>  

_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
In reply to this post by Fernando Perez
Howdy,

On Tue, Apr 8, 2008 at 4:55 PM, Fernando Perez <[hidden email]> wrote:

>  1. Do you want me to obliterate the old numpy/doc/pyrex and replace it
>  with numpy/doc/cython? That would make it clear what the tool to use
>  is...

OK, the code is ready and I just updated the docs/makefile a little to
make it more new user-friendly.

I went ahead and made the commit here:

http://projects.scipy.org/scipy/numpy/changeset/4994

Instead of deleting the old pyrex dir, I just put deprecation markers
in there.  I can remove it if everyone agrees, but I figured it would
be best to be careful (this is not code I work on regularly...)

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
In reply to this post by Andrew Straw
On Tue, Apr 8, 2008 at 6:52 PM, Andrew Straw <[hidden email]> wrote:

> This is off-topic and should be directed to the pyrex/cython list, but
>  since we're on the subject:
>
>  I suppose the following is true, but let me ask, since I have not used
>  Cython. Please correct me if I'm wrong.
>
>  I have a bunch of pyrex compiled .pyx code. If I start adding some
>  Cython compiled code, all the extensions will live happily without
>  colliding, even if I don't re-compile my old .pyx files with Cython.
>
>  (I can't imagine how they'd conflict, since they both translate to .c
>  extensions without anything to collide. Or are there linker-level
>  symbols that collide or something difficult?)

I think you're correct, I fail to see how it could be any different.
But I'm far from a cython expert, so I may be missing something.

cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
In reply to this post by Fernando Perez
On Tue, Apr 8, 2008 at 4:55 PM, Fernando Perez <[hidden email]> wrote:

>  2. I'll work on random and report shortly.

As far as I can tell, there are no problems at all.  I'm running

bic128[scipy]$ cython --version
Cython version 0.9.6.12

This is both on 64-bit Fedora 8 and 32-bit Ubuntu 7.10.  I made sure I
was actually running off the newly generated C code written out by
cython instead of the old pyrex one.

If all agree, we can push the C sources and the generate_mtrand script
as a single commit where I don't touch *anything* else. That way the
buildbots can run the tests, and if that C commit changes anything,
then it's easy to back off.

I scanned the sources for a while, and the only other file that has
significant amounts of pyrex mention is build_src.py:

http://projects.scipy.org/scipy/numpy/browser/trunk/numpy/distutils/command/build_src.py

I'd rather not touch that file yet, since distutils isn't exactly my
favorite code to mess with :)

So I'd like to know whether people would want to see this one isolated
commit to happen, for the buildbots to test things out on other
platforms or not.

I can also make the patch available somewhere (it's huge, since it
involves the auto-generated mtrand.c file) if you all prefer...

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Francesc Altet
In reply to this post by Andrew Straw
A Wednesday 09 April 2008, Andrew Straw escrigué:

> This is off-topic and should be directed to the pyrex/cython list,
> but since we're on the subject:
>
> I suppose the following is true, but let me ask, since I have not
> used Cython. Please correct me if I'm wrong.
>
> I have a bunch of pyrex compiled .pyx code. If I start adding some
> Cython compiled code, all the extensions will live happily without
> colliding, even if I don't re-compile my old .pyx files with Cython.
>
> (I can't imagine how they'd conflict, since they both translate to .c
> extensions without anything to collide. Or are there linker-level
> symbols that collide or something difficult?)

I don't expect you having problems in that regard either.  However, I've
been having problems compiling perfectly valid Pyrex code with the
Cython compiler.  I just haven't had time to locate where the problem
is and report it, but people using Pyrex and planning to migrate to
Cython must be aware that these sort of things happen.

Cheers,

--
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Francesc Altet
A Wednesday 09 April 2008, Francesc Altet escrigué:

> A Wednesday 09 April 2008, Andrew Straw escrigué:
> > This is off-topic and should be directed to the pyrex/cython list,
> > but since we're on the subject:
> >
> > I suppose the following is true, but let me ask, since I have not
> > used Cython. Please correct me if I'm wrong.
> >
> > I have a bunch of pyrex compiled .pyx code. If I start adding some
> > Cython compiled code, all the extensions will live happily without
> > colliding, even if I don't re-compile my old .pyx files with
> > Cython.
> >
> > (I can't imagine how they'd conflict, since they both translate to
> > .c extensions without anything to collide. Or are there
> > linker-level symbols that collide or something difficult?)
>
> I don't expect you having problems in that regard either.  However,
> I've been having problems compiling perfectly valid Pyrex code with
> the Cython compiler.  I just haven't had time to locate where the
> problem is and report it, but people using Pyrex and planning to
> migrate to Cython must be aware that these sort of things happen.

Just to clarify, my problems were more due to file arrangement to create
extensions (.pxd, .pxi, etc...) than the issue in the relatively simple
Pyrex example distributed with NumPy.

Cheers,

--
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
In reply to this post by Francesc Altet
On Wed, Apr 9, 2008 at 12:25 AM, Francesc Altet <[hidden email]> wrote:

>  I don't expect you having problems in that regard either.  However, I've
>  been having problems compiling perfectly valid Pyrex code with the
>  Cython compiler.  I just haven't had time to locate where the problem
>  is and report it, but people using Pyrex and planning to migrate to
>  Cython must be aware that these sort of things happen.

It would be great to have some examples of this, so we can better
understand whether they are bugs in cython or deliberate changes in
behavior.  There are some of the latter, described here:

http://wiki.cython.org/DifferencesFromPyrex

and in fact one of those already bit us in the tiny numpy/pyrex
example, as reported originally in this thread.  If something more
serious arises we certainly need to know about it, but I know the
cython devs are a pretty responsive bunch, so I hope we'll have a good
chance of clean resolutions for those.  Considering how pyrex has
essentially no public development process to speak of, I find it a
worrying tool to commit to for the long run, while cython is very
actively developed, which is always a bonus.

But I'm not trying to ram anything down people's throats, which is why
I didn't even want to put in the mtrand.c changes before there was
more feedback (even as a self-contained single commit).  Here's the
patch that commit would contain, against current SVN, so others can
test it without putting anything into SVN itself:

http://amath.colorado.edu/faculty/fperez/tmp/mtrand-cython.patch.gz

I'll be happy to leave it as a new trac ticket with an attachment for
it if it will make the testing/review process easier in the long run.

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Robert Kern-2
On Wed, Apr 9, 2008 at 2:47 AM, Fernando Perez <[hidden email]> wrote:
>  I'll be happy to leave it as a new trac ticket with an attachment for
>  it if it will make the testing/review process easier in the long run.

Please do. I don't want to change mtrand over until after 1.0.5 is released.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Fernando Perez
On Wed, Apr 9, 2008 at 12:49 AM, Robert Kern <[hidden email]> wrote:
> On Wed, Apr 9, 2008 at 2:47 AM, Fernando Perez <[hidden email]> wrote:
>  >  I'll be happy to leave it as a new trac ticket with an attachment for
>  >  it if it will make the testing/review process easier in the long run.
>
>  Please do. I don't want to change mtrand over until after 1.0.5 is released.

http://scipy.org/scipy/numpy/ticket/727

I assigned it to you for now, since I figured you'd have the most to
say on the fate of mtrand.

Cheers,

f
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: numpyx.pyx (recent svn) works?

Francesc Altet
In reply to this post by Fernando Perez
A Wednesday 09 April 2008, Fernando Perez escrigué:
> On Wed, Apr 9, 2008 at 12:25 AM, Francesc Altet <[hidden email]>
wrote:

> >  I don't expect you having problems in that regard either.
> > However, I've been having problems compiling perfectly valid Pyrex
> > code with the Cython compiler.  I just haven't had time to locate
> > where the problem is and report it, but people using Pyrex and
> > planning to migrate to Cython must be aware that these sort of
> > things happen.
>
> It would be great to have some examples of this, so we can better
> understand whether they are bugs in cython or deliberate changes in
> behavior.

This is in my TODO list, yes.

> There are some of the latter, described here:
>
> http://wiki.cython.org/DifferencesFromPyrex
>
> and in fact one of those already bit us in the tiny numpy/pyrex
> example, as reported originally in this thread.  If something more
> serious arises we certainly need to know about it, but I know the
> cython devs are a pretty responsive bunch, so I hope we'll have a
> good chance of clean resolutions for those.  Considering how pyrex
> has essentially no public development process to speak of, I find it
> a worrying tool to commit to for the long run, while cython is very
> actively developed, which is always a bonus.

Well, I agree that Greg Ewing (the Pyrex creator) has possibly not been
very speedy in adding the suggested patches (Greg has his own thoughts
on what should be added to Pyrex and what not), which ultimately
brought to the need of the Cython fork, but let me stress out that he
has always been *very* responsive to the questions on the Pyrex list,
and quick enough in fixing real problems in Pyrex.  I'm personally very
satisfied with Pyrex as does its job extremely well.  And I'm specially
grateful to Greg for his *huge* contribution to make the job of doing
Python extensions a pretty simple job.

So, I don't really think that Pyrex should be considered a "worrying
tool" at all (even in the "long run"), rather the contrary, it is a
extremely useful tool.  Having said that, I'm all for pushing Cython
forward (specially if the integration with NumPy becomes a reality :).

Cheers,

--
>0,0<   Francesc Altet     http://www.carabos.com/
V   V   Cárabos Coop. V.   Enjoy Data
 "-"
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
12
Loading...