
123

2008/4/4, Joe Harrington <[hidden email]>:
import numpy as N import numpy.math as N.M import numpy.trig as N.T import numpy.stat as N.S
I don't think the issue is whether to put everything in the base namespace // everything in individual namespace, but rather to find an optimal and intuitive mix between the two. For instance, the io functions would be easier to find by typing np.io.loadtxt than by sifting through the 500+ items of the base namespace. The stats functions could equally well be in a separate namespace, given that the most used are implemented as array methods. I think this would allow numpy to grow more gracefully.
As for the financial functions, being specific to a discipline, I think they rather belongs with scipy. The numpy namespace will quickly become a mess if we add np.geology, np.biology, np.material, etc.
Of course, this raises the problem of distributing scipy, and here is a suggestion:
Change the structure of scipy so that it looks like the scikits:
scipy/ sparse/ cluster/ financial/ ... fftpack/ setup.py scipy/
__init__.py fftpack/
The advantage is that each subpackage can be installed independently of the others. For distribution, we could lump all the pure python or easy to compile packages into scipy.common, and distribute the other packages such as sparse and fftpack independently. My feeling is that such a lighter structure would encourage projects with large code base to join the scipy community. It would also allow folks with 56k modems to download only what they need.
David
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On 07/04/2008, David Huard < [hidden email]> wrote:
>
> 2008/4/4, Joe Harrington < [hidden email]>:
> > import numpy as N
> > import numpy.math as N.M
> > import numpy.trig as N.T
> > import numpy.stat as N.S
>
>
> I don't think the issue is whether to put everything in the base namespace
> // everything in individual namespace, but rather to find an optimal and
> intuitive mix between the two. For instance, the io functions would be
> easier to find by typing np.io.loadtxt than by sifting through the 500+
> items of the base namespace. The stats functions could equally well be in a
> separate namespace, given that the most used are implemented as array
> methods. I think this would allow numpy to grow more gracefully.
I agree, and I think we can come to some compromise  maybe a
numpy.all namespace, that simply imports all the other subpackages.
Regards
Stéfan
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 07, 2008 at 05:20:47PM +0200, Stéfan van der Walt wrote:
> I agree, and I think we can come to some compromise  maybe a
> numpy.all namespace, that simply imports all the other subpackages.
For the beginner, "from numpy.all import *" is more confusing than "from
numpy import *" (which is already confusing).
I know namespace are good things, but the beginner struggles with them.
This is why I used the "import *" in my above examples.
My 2 cents,
Gaël
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On 07/04/2008, Gael Varoquaux < [hidden email]> wrote:
> On Mon, Apr 07, 2008 at 05:20:47PM +0200, Stéfan van der Walt wrote:
> > I agree, and I think we can come to some compromise  maybe a
> > numpy.all namespace, that simply imports all the other subpackages.
>
>
> For the beginner, "from numpy.all import *" is more confusing than "from
> numpy import *" (which is already confusing).
>
> I know namespace are good things, but the beginner struggles with them.
> This is why I used the "import *" in my above examples.
You're only a beginner for a short while, and after that the lack of
namespaces really start to bite. I am all in favour of catering for
those who are busy learning numpy, but should we do that at the cost
of our advanced users?
There must be a way to make both groups happy  any suggestions?
Regards
Stéfan
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 07, 2008 at 06:22:28PM +0200, Stéfan van der Walt wrote:
> You're only a beginner for a short while, and after that the lack of
> namespaces really start to bite. I am all in favour of catering for
> those who are busy learning numpy, but should we do that at the cost
> of our advanced users?
I agree with you. However lowering the bar is a good thing.
> There must be a way to make both groups happy  any suggestions?
Hum, I am still trying to find ideas. If only "from foo.bar import baz"
didn't import all what is in foo.__init__ !!! By the way, the standard
solution to this problem is to use a module called "api" and not "all".
That's what people have been doing to solve the problem we are faced
with. If we are going to go this way, I suggest we stick to the "api"
convention (eventhought it sucks).
Cheers,
Gaël
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 7, 2008 at 9:57 AM, Gael Varoquaux < [hidden email]> wrote:
On Mon, Apr 07, 2008 at 06:22:28PM +0200, Stéfan van der Walt wrote:
> You're only a beginner for a short while, and after that the lack of
> namespaces really start to bite. I am all in favour of catering for
> those who are busy learning numpy, but should we do that at the cost
> of our advanced users?
I agree with you. However lowering the bar is a good thing.
> There must be a way to make both groups happy  any suggestions?
Hum, I am still trying to find ideas. If only "from foo.bar import baz"
didn't import all what is in foo.__init__ !!! By the way, the standard
solution to this problem is to use a module called "api" and not "all".
That's what people have been doing to solve the problem we are faced
with. If we are going to go this way, I suggest we stick to the "api"
convention (eventhought it sucks). I prefer 'all' for this since it has the correct meaning. 'api' assuming that one can remember what it means doesn't fit. The 'all' module would not contain the api, at least not the preferred api (in my book at least), but it would contain everything.
If "from numpy.all import *" is really too complicated, which although possible, seems a little disheartening, I suspect it would be easy enough to have a separate module that pulled everything in so that you could use "from big_numpy import *". Or, to preserve backward compatibility, make numpy the unsplit namespace and expose the split namespace under a different name, let's say 'np' because that's what I already use as a numpy abbreviation. Then "import np" would get you just the core np functions (which I imagine we could argue about endlessly) and the various subpackages would be imported separately. 'np' is 'numpy' with some stuff removed: get it? OK, so that's a weak joke, sorry.
 . __ . \ . . [hidden email]
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, April 7, 2008 11:16, Timothy Hochberg wrote:
>
> If "from numpy.all import *" is really too complicated, which although
> possible, seems a little disheartening, I suspect it would be easy enough
> to
> have a separate module that pulled everything in so that you could use
> "from
> big_numpy import *". Or, to preserve backward compatibility, make numpy
> the
> unsplit namespace and expose the split namespace under a different name,
> let's say 'np' because that's what I already use as a numpy abbreviation.
> Then "import np" would get you just the core np functions (which I imagine
> we could argue about endlessly) and the various subpackages would be
> imported separately. 'np' is 'numpy' with some stuff removed: get it? OK,
> so
> that's a weak joke, sorry.
>
May not be the epitome of wit, but not bad.
+1 for np package being a minimalist numpy and numpy being bigger.
# Steve
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 07, 2008 at 10:16:22AM 0700, Timothy Hochberg wrote:
> I prefer 'all' for this since it has the correct meaning. 'api' assuming
> that one can remember what it means doesn't fit. The 'all' module would
> not contain the api, at least not the preferred api (in my book at least),
> but it would contain everything.
Sure, but everybody does it different. Convention are important,
especially in coding. See http://ivory.idyll.org/blog/sep07/notsuckingfor a good argumentation about the point. I agree 100% with the author.
Especially the conclusion.
> If "from numpy.all import *" is really too complicated, which although
> possible, seems a little disheartening,
How much have you tried forcing Python on people who don't care at all
about computers. In my work we spend maybe 2% of our time dealing with
computers, and the rest struggling with electronics, optics, lasers,
mechanical design... People don't want to have to learn _anything_ about
computers. I am not saying they are right, I am however saying that we
need to provide easy entry point, from where they can evolve and learn.
> I suspect it would be easy enough to have a separate module that
> pulled everything in so that you could use "from big_numpy import
> *". Or, to preserve backward compatibility, make numpy the unsplit
> namespace and expose the split namespace under a different name,
> let's say 'np' because that's what I already use as a numpy
> abbreviation.
That's the only solution I see wich would make everybody happy. IMHO the
pylab option is quite nice: matplotlib is nice and modular, but pylab has
it all. Use whichever you want.
Now the difficulty is to find a good name for the latter
module/namespace.
Cheers,
Ga�l
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 7, 2008 at 10:30 AM, Gael Varoquaux < [hidden email]> wrote:
On Mon, Apr 07, 2008 at 10:16:22AM 0700, Timothy Hochberg wrote:
> I prefer 'all' for this since it has the correct meaning. 'api' assuming
> that one can remember what it means doesn't fit. The 'all' module would
> not contain the api, at least not the preferred api (in my book at least),
> but it would contain everything.
Sure, but everybody does it different. Convention are important,
especially in coding. See http://ivory.idyll.org/blog/sep07/notsucking
for a good argumentation about the point. I agree 100% with the author.
Especially the conclusion. This is all moot since we agree below, but I don't see that the page your reference, which seem uncontroversial on a casual reading, is all that relevant. It's not that I disagree, that following convention is important where reasonable, I just don't see that this is a case where there is a convention to follow.
I'm at a bit of a disadvantage since the convention in question hasn't penetrated the parts of of Python land that I inhabit (which could either imply something about my experience or about the universality of the 'api' convention, take your pick). However, I think that I vaguely recall it from back in my Cprogramming days, and as I recall/infer/guess the 'api' namespace is how you are supposed to use the functions in question, while the actual modules are split out for implementation purposes only.
However, in numpy, that is not the case. Any splitting of the namespace would be to support a better, more organized interface, not as an implementation details. So. referring to the collected, flat namespace as 'api' would be confusing at best since it would imply that was the official approved way to access the python functions rather than one of two equivalent apis, where the flat namespace is provided primarily for beginners.
> If "from numpy.all import *" is really too complicated, which although
> possible, seems a little disheartening,
How much have you tried forcing Python on people who don't care at all
about computers.
Almost none, thankfully.
In my work we spend maybe 2% of our time dealing with
computers, and the rest struggling with electronics, optics, lasers,
mechanical design... People don't want to have to learn _anything_ about
computers. I am not saying they are right, I am however saying that we
need to provide easy entry point, from where they can evolve and learn.
> I suspect it would be easy enough to have a separate module that
> pulled everything in so that you could use "from big_numpy import
> *". Or, to preserve backward compatibility, make numpy the unsplit
> namespace and expose the split namespace under a different name,
> let's say 'np' because that's what I already use as a numpy
> abbreviation.
That's the only solution I see wich would make everybody happy. IMHO the
pylab option is quite nice: matplotlib is nice and modular, but pylab has
it all. Use whichever you want.
Now the difficulty is to find a good name for the latter
module/namespace.
Cheers,
Gaël
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion
 . __ . \ . . [hidden email]
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


Steven H. Rogers wrote:
> On Mon, April 7, 2008 11:16, Timothy Hochberg wrote:
>
>> If "from numpy.all import *" is really too complicated, which although
>> possible, seems a little disheartening, I suspect it would be easy enough
>> to
>> have a separate module that pulled everything in so that you could use
>> "from
>> big_numpy import *". Or, to preserve backward compatibility, make numpy
>> the
>> unsplit namespace and expose the split namespace under a different name,
>> let's say 'np' because that's what I already use as a numpy abbreviation.
>> Then "import np" would get you just the core np functions (which I imagine
>> we could argue about endlessly) and the various subpackages would be
>> imported separately. 'np' is 'numpy' with some stuff removed: get it? OK,
>> so
>> that's a weak joke, sorry.
>>
>>
> May not be the epitome of wit, but not bad.
>
> +1 for np package being a minimalist numpy and numpy being bigger.
>
> # Steve
>
> _______________________________________________
> Numpydiscussion mailing list
> [hidden email]
> http://projects.scipy.org/mailman/listinfo/numpydiscussion>
>
Hi,
I think that splitting the NumPy namespace should not happen within a
major release series because it would cause too many breakages. Rather
it should be in a forthcoming release like the 2.0 series where it may
be very feasible to have a true core functionality (NumPy), extended
functionality (SciPy) and specific applications (Scikits). At the same
time, Python 3K would be fully supported because it will break lots of
code.
It is really nice to think about having NumPy Core, NumPy Full,
NumPyKits, SciPy Core, SciPy Full and SciPyKits. But splitting
namespaces like core and complete brings into the problem of conflicts
and how to resolve them. Regardless of the content of each, I have the
suspicion that most people would just take the full versions of each
eventhough most of them only use a very small fraction of NumPy (just
probably different amongst users).
In the past, the real distinction between Numpy and SciPy for me was the
requirement of having a full Lapack installation and a Fortran compiler
for SciPy. This made the current scheme very usable especially the
frustrations of getting SciPy to install. Fortunately Linux and GCC
Fortran has really developed over the years that these are not as big as
they were although these still cause issues (especially if packages are
broken). However, it remains a big concern if everything has to be built
from scratch (perhaps with different compilers) or if developers
continue to tell users to get the latest version from svn (problem if
you used a precompiled version).
Regards
Bruce
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 7, 2008 at 11:19 AM, Bruce Southey < [hidden email]> wrote:
> Hi,
> I think that splitting the NumPy namespace should not happen within a
> major release series because it would cause too many breakages. Rather
> it should be in a forthcoming release like the 2.0 series where it may
> be very feasible to have a true core functionality (NumPy), extended
> functionality (SciPy) and specific applications (Scikits). At the same
> time, Python 3K would be fully supported because it will break lots of
> code.
I would prefer not to do it at all. We've just gotten people moved
over from Numeric; I'd hate to break their trust again.
> It is really nice to think about having NumPy Core, NumPy Full,
> NumPyKits, SciPy Core, SciPy Full and SciPyKits.
Really? It gives me the shivers, frankly.

Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 Umberto Eco
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 7, 2008 at 11:29 AM, Gael Varoquaux
< [hidden email]> wrote:
> On Mon, Apr 07, 2008 at 11:29:41AM 0700, Robert Kern wrote:
> > I would prefer not to do it at all. We've just gotten people moved
> > over from Numeric; I'd hate to break their trust again.
+1
I also think we have a big enough proliferation of namespaces (with
numpy, scipy, and scikits).

Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Montag 07 April 2008, Robert Kern wrote:
> I would prefer not to do it at all. We've just gotten people moved
> over from Numeric; I'd hate to break their trust again.
+1.
IMO, numpy has arrived at a state where there's just enough namespace clutter
to allow most use cases to get by without importing much subnamespace junk,
and I think that's a good place to be (and to stay).
For now, I'd be very careful about adding more.
> > It is really nice to think about having NumPy Core, NumPy Full,
> > NumPyKits, SciPy Core, SciPy Full and SciPyKits.
>
> Really? It gives me the shivers, frankly.
Couldn't agree more.
Andreas
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 7, 2008 at 11:02 AM, Timothy Hochberg < [hidden email]> wrote:
> I'm at a bit of a disadvantage since the convention in question hasn't
> penetrated the parts of of Python land that I inhabit (which could either
> imply something about my experience or about the universality of the 'api'
> convention, take your pick). However, I think that I vaguely recall it from
> back in my Cprogramming days, and as I recall/infer/guess the 'api'
> namespace is how you are supposed to use the functions in question, while
> the actual modules are split out for implementation purposes only.
I haven't been following how many projects have been using the api.py
convention, but when I last looked about a year ago there was
enthought, peak, zope, trac, etc. See this note for a bit more
information: http://neuroimaging.scipy.org/neuroimaging/ni/ticket/86Hope this helps,

Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On 07/04/2008, Andreas Klöckner < [hidden email]> wrote:
> On Montag 07 April 2008, Robert Kern wrote:
> > I would prefer not to do it at all. We've just gotten people moved
> > over from Numeric; I'd hate to break their trust again.
>
>
> +1.
>
> IMO, numpy has arrived at a state where there's just enough namespace clutter
> to allow most use cases to get by without importing much subnamespace junk,
> and I think that's a good place to be (and to stay).
I wouldn't exactly call 494 functions "just enough namespace clutter";
I'd much prefer to have a clean api to work with.
I certainly don't propose forcing such an api upon all users, but it
should be available as an option, at least. Tim's suggestion for a
separate package that pulls in a "structured" numpy would suit my
needs.
As Gael mentioned, __init__'s are cursed, otherwise we'd be able to
provide numpy.* for the flat earth society (all in friendly jest ;)
and numpy.api to expose a somewhat organised underlying structure. As
it is, importing numpy.api would trigger the __init__ of the flat
namespace as well; but I'd still be amenable to this solution since
the import doesn't take long, and the organisation of the api is more
important to me.
Would it therefore make sense to
a) Reorganise numpy to expose functionality as numpy.api.*
b) Do a series of imports in numpy.__init__ which pulls in from numpy.api.
This way, numpy.* would look exactly as it does now, bar the added member 'api'.
Regards
Stéfan
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 07, 2008 at 10:16:57PM +0200, Stéfan van der Walt wrote:
> Would it therefore make sense to
> a) Reorganise numpy to expose functionality as numpy.api.*
> b) Do a series of imports in numpy.__init__ which pulls in from numpy.api.
> This way, numpy.* would look exactly as it does now, bar the added member 'api'.
+1. That way you don't break compatibility, but you provide nested
namespace for people interested in them. You still get the import
overhead. That's too bad.
With some very good engineering, you might even make it possible to ship
only part of numpy for custom installations.
Cheers,
Gaël
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


> I wouldn't exactly call 494 functions "just enough namespace clutter";
> I'd much prefer to have a clean api to work with
I don't know. The 494 functions do not seem like many to me.
Apparently, I tend to come down in the "flat earth society" although I
do like some structure (after all that's why numpy has numpy.linalg and
numpy.fft). I don't think this is the most pressing issue we are facing.
> Would it therefore make sense to
>
> a) Reorganise numpy to expose functionality as numpy.api.*
> b) Do a series of imports in numpy.__init__ which pulls in from numpy.api.
>
> This way, numpy.* would look exactly as it does now, bar the added member 'api'.
>
This discussion is interesting, but it is really better suited for 1.1,
isn't it?
0 on adding the .api name in 1.0.5
Travis
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Montag 07 April 2008, Stéfan van der Walt wrote:
> I wouldn't exactly call 494 functions "just enough namespace clutter";
> I'd much prefer to have a clean api to work with.
Not to bicker, but...
>>> import numpy
>>> len(dir(numpy))
494
>>> numpy.__version__
'1.0.4'
>>> funcs = [s for s in dir(numpy) if type(getattr(numpy, s)) in
[type(numpy.array), type(numpy.who)]]
>>> len(funcs)
251
>>> classes = [s for s in dir(numpy) if type(getattr(numpy, s)) ==
type(numpy.ndarray)]
>>> len(classes)
88
>>> ufuncs = [s for s in dir(numpy) if type(getattr(numpy, s)) ==
type(numpy.sin)]
>>> len(ufuncs)
69
>>>
(and, therefore, another 69 names of "fluff")
I honestly don't see much of a problem.
The only things that maybe should not have been added to numpy.* are the
polynomial functions and the convolution windows, conceptually. But in my
book that's not big enough to even think of breaking people's code for.
Andreas
Proud Member of the Flat Earth Society
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Mon, Apr 07, 2008 at 03:52:45PM 0500, Travis E. Oliphant wrote:
> This discussion is interesting, but it is really better suited for 1.1,
> isn't it?
Yes. The original dicussion was about adding simple financial functions
for Numpy.
I think the functions you wrote should land somewhere. We may disagree on
where, and you should choose based on feedback here and your personnal
feeling, but I hope they will indeed be published somewhere.
Cheers,
Gaël
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion

123
