New python/numpy user

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

New python/numpy user

paul.carrico
Dear All
 
I'm a like matlab user (more specifically a Scilab one) for years, and because I've to deal with huge ascii files (with dozens of millions of lines), I decided to have a look to Python and Numpy, including vectorization topics.
 
Obviously I've been influenced by my current feedbacks.
 
I've a basic question concerning the current code: why it is necessary to transpose the column vector (still in the right format in my mind)? does it make sens?
 
Thanks
 
Paul
 
####################################
import numpy as np ## np = raccourci

## works with a row vector
vect0 = np.random.rand(5); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np.transpose(vect0); print mat

## works while the vector is still in column i.e. in a right format, isn't it?
vect0 = np.random.rand(5,1); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np.transpose(vect0); print mat

## does not work
vect0 = np.random.rand(5,1); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np(vect0); print mat

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: New python/numpy user

Brian Blais-2
There are a couple of interesting observations here.  In your first bit, you have:

## works with a row vector
vect0 = np.random.rand(5)
mat[:,0]=np.transpose(vect0)

(or I prefer vect0.T).  Did you happen to notice that this works too:

mat[:,0]=vect0


The transpose or the original work as well.  Unlike Scilab, python’s arrays can be literally 1-dimensional.  Not 5x1 but just 5,  which doesn’t have a transpose, because it doesn’t have a 2nd dimension.

you can see that in vect0.shape

so np.random.rand(5) doesn’t make a row-vector but a length 5 array, which is different than np.random.rand(5,1) or np.random.rand(1,5).  Thus, you have to make sure the shapes all work. 

in your second example, with the column vector, you can also slice along the 2nd dimension without transposing like:

mat[:,0]=vect0[:,0]

mat[:,0] seems to have shape of (5,) which is just length-5 array, so setting it equal to 1xN or Nx1 arrays seems to cause some issues.


- Brian

On Jul 3, 2017, 15:57 -0400, [hidden email], wrote:
Dear All
 
I'm a like matlab user (more specifically a Scilab one) for years, and because I've to deal with huge ascii files (with dozens of millions of lines), I decided to have a look to Python and Numpy, including vectorization topics.
 
Obviously I've been influenced by my current feedbacks.
 
I've a basic question concerning the current code: why it is necessary to transpose the column vector (still in the right format in my mind)? does it make sens?
 
Thanks
 
Paul
 
####################################
import numpy as np ## np = raccourci

## works with a row vector
vect0 = np.random.rand(5); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np.transpose(vect0); print mat

## works while the vector is still in column i.e. in a right format, isn't it?
vect0 = np.random.rand(5,1); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np.transpose(vect0); print mat

## does not work
vect0 = np.random.rand(5,1); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np(vect0); print mat
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: New python/numpy user

paul.carrico

Hi Brian


First of all thanks for the answer, the explanations and the advices; as mentioned I've to think differently in order to code with efficiency.

1.: yes 'mat[:,0]=vect0' works fine and I understand why :-)

2. more generally, I've read some tutorials or presentations saying that Numpy is faster than the native Python; regarding the (huge) size of my matrices, vectorization, Numpy (and others) + optimzation of calls are the basics

Thanks


Paul



Le 2017-07-04 01:04, Brian Blais a écrit :

There are a couple of interesting observations here.  In your first bit, you have:
 
## works with a row vector
vect0 = np.random.rand(5)
mat[:,0]=np.transpose(vect0)

(or I prefer vect0.T).  Did you happen to notice that this works too:
 
mat[:,0]=vect0
 

The transpose or the original work as well.  Unlike Scilab, python's arrays can be literally 1-dimensional.  Not 5x1 but just 5,  which doesn't have a transpose, because it doesn't have a 2nd dimension.
 
you can see that in vect0.shape
 
so np.random.rand(5) doesn't make a row-vector but a length 5 array, which is different than np.random.rand(5,1) or np.random.rand(1,5).  Thus, you have to make sure the shapes all work. 
 
in your second example, with the column vector, you can also slice along the 2nd dimension without transposing like:
 
mat[:,0]=vect0[:,0]
 
mat[:,0] seems to have shape of (5,) which is just length-5 array, so setting it equal to 1xN or Nx1 arrays seems to cause some issues.
 

- Brian

On Jul 3, 2017, 15:57 -0400, [hidden email], wrote:
Dear All
 
I'm a like matlab user (more specifically a Scilab one) for years, and because I've to deal with huge ascii files (with dozens of millions of lines), I decided to have a look to Python and Numpy, including vectorization topics.
 
Obviously I've been influenced by my current feedbacks.
 
I've a basic question concerning the current code: why it is necessary to transpose the column vector (still in the right format in my mind)? does it make sens?
 
Thanks
 
Paul
 
####################################
import numpy as np ## np = raccourci

## works with a row vector
vect0 = np.random.rand(5); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np.transpose(vect0); print mat

## works while the vector is still in column i.e. in a right format, isn't it?
vect0 = np.random.rand(5,1); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np.transpose(vect0); print mat

## does not work
vect0 = np.random.rand(5,1); print vect0; print("\n")
mat = np.zeros((5,4),dtype=float)
mat[:,0]=np(vect0); print mat
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: New python/numpy user

Charles R Harris


On Tue, Jul 4, 2017 at 1:32 AM, <[hidden email]> wrote:

Hi Brian


First of all thanks for the answer, the explanations and the advices; as mentioned I've to think differently in order to code with efficiency.

1.: yes 'mat[:,0]=vect0' works fine and I understand why :-)

2. more generally, I've read some tutorials or presentations saying that Numpy is faster than the native Python; regarding the (huge) size of my matrices, vectorization, Numpy (and others) + optimzation of calls are the basics

Thanks



Note that NumPy is not the best for large text files, for instance, pandas is faster.

Also note that the custom here is bottom posting.  And welcome to Python ;)

<snip>

Chuck



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Loading...