f2py: ram usage

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

f2py: ram usage

yellowhat
Hi all,
I am trying to write some code to do calculation onto an array: for each row I need to do some computation and have a number as return.
To speed up the process I wrote a fortran subroutine that is called from python [using f2py] for each row of the array, so the input of this subroutine is a row and the output is a number.
This method works but I saw some speed advantage if I pass the entire array to fortran and then, inside fortran, call the subroutine that does the math; so in this case I pass an array and return a vector.
But I noticed that when python pass the array to fortran, the array is copied and the RAM usage double.
Is there a way to "move" the array to fortran, I don't care if the array is lost after the call to fortran.
The pyd module is generated using: python f2py.py -c --opt="-ffree-form -Ofast" -m F2PYMOD F2PYMOD.f90

Thanks
Vasco

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: f2py: ram usage

Sebastian Berg
On So, 2016-04-10 at 12:04 +0200, Vasco Gervasi wrote:

> Hi all,
> I am trying to write some code to do calculation onto an array: for
> each row I need to do some computation and have a number as return.
> To speed up the process I wrote a fortran subroutine that is called
> from python [using f2py] for each row of the array, so the input of
> this subroutine is a row and the output is a number.
> This method works but I saw some speed advantage if I pass the entire
> array to fortran and then, inside fortran, call the subroutine that
> does the math; so in this case I pass an array and return a vector.
> But I noticed that when python pass the array to fortran, the array
> is copied and the RAM usage double.
I expect that the fortran code needs your arrays to be fortran
contiguous, so the wrappers need to copy them.

The easiest solution may be to create your array in python with the
`order="F"` flag. NumPy will have a tendency to prefer C-order and uses
it as default though when doing something with an "F" ordered array.

That said, I have never used f2py, so these are just well founded
guesses.

- Sebastian



> Is there a way to "move" the array to fortran, I don't care if the
> array is lost after the call to fortran.
> The pyd module is generated using: python f2py.py -c --opt="-ffree
> -form -Ofast" -m F2PYMOD F2PYMOD.f90
>
> Thanks
> Vasco
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: f2py: ram usage

George Nurser-4
Yes, f2py is probably copying the arrays; you can check this by appending
 -DF2PY_REPORT_ON_ARRAY_COPY=1 to your call to f2py.

I normally prefer to keep the numpy arrays C-order (most efficient for numpy) and simply pass the array transpose to the f2py-ized fortran routine.
This means that the fortran array indices are reversed, but this is the most natural  way in any case.

--George Nurser

On 10 April 2016 at 11:53, Sebastian Berg <[hidden email]> wrote:
On So, 2016-04-10 at 12:04 +0200, Vasco Gervasi wrote:
> Hi all,
> I am trying to write some code to do calculation onto an array: for
> each row I need to do some computation and have a number as return.
> To speed up the process I wrote a fortran subroutine that is called
> from python [using f2py] for each row of the array, so the input of
> this subroutine is a row and the output is a number.
> This method works but I saw some speed advantage if I pass the entire
> array to fortran and then, inside fortran, call the subroutine that
> does the math; so in this case I pass an array and return a vector.
> But I noticed that when python pass the array to fortran, the array
> is copied and the RAM usage double.

I expect that the fortran code needs your arrays to be fortran
contiguous, so the wrappers need to copy them.

The easiest solution may be to create your array in python with the
`order="F"` flag. NumPy will have a tendency to prefer C-order and uses
it as default though when doing something with an "F" ordered array.

That said, I have never used f2py, so these are just well founded
guesses.

- Sebastian



> Is there a way to "move" the array to fortran, I don't care if the
> array is lost after the call to fortran.
> The pyd module is generated using: python f2py.py -c --opt="-ffree
> -form -Ofast" -m F2PYMOD F2PYMOD.f90
>
> Thanks
> Vasco
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: f2py: ram usage

yellowhat

Using order='F' solved the problem.

Thanks for reply.


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion