Numpy support for Microblaze

classic Classic list List threaded Threaded
6 messages Options
pnp
Reply | Threaded
Open this post in threaded view
|

Numpy support for Microblaze

pnp
Are there some steps to add support to a new cpu such as microblaze?

Regards.



--
Sent from: http://numpy-discussion.10968.n7.nabble.com/
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Numpy support for Microblaze

mattip
Administrator
On 9/1/20 10:03 pm, pnp wrote:

> Are there some steps to add support to a new cpu such as microblaze?
>
> Regards.
>
>
>
> --
> Sent from: http://numpy-discussion.10968.n7.nabble.com/
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion


I don't think we have a documented procedure for adding a new CPU
architecture: it is not that common.

The microblaze is a soft core built in HDL, targeting an FPGA. I would
try to break the question into three parts:

- Does the python support for the microblaze implement any the Python C-API?

- Does the C compiler for microblase support enough of the c99 language
so that NumPy can be synthesized?

and once NumPy is running:

- Are there acceleration libraries that could supply a BLAS-like interface?


My approach would be to "make it run" then "make it fast", but I doubt
the high-level (c) tools are up to the task of sythesizing the NumPy C
code to HDL without alot of work.

It might be better to approach this task in a different direction: take
one of the NumPy-like projects (CuPy might be a good starting point) and
rewrite the C code parts in a way that can be synthesized.

Matti

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Numpy support for Microblaze

Frederic Bastien
I like your breakdown. But the fact that it is a softcore is not related. We do not need to compile it to an HDL.
We need it to be compiled to the binary format that the Microblaze support.

Now, after one full version is working as a software, it is possible to optimize part of NumPy by having those part synthesized to hardware.
But from my understanding, it wasn't the question. This would be a bigger effort and I suppose there is no guide from NumPy on how to do this.

Frederic

-----Original Message-----
From: NumPy-Discussion <numpy-discussion-bounces+fbastien=[hidden email]> On Behalf Of Matti Picus
Sent: Thursday, January 9, 2020 5:34 PM
To: [hidden email]
Subject: Re: [Numpy-discussion] Numpy support for Microblaze

External email: Use caution opening links or attachments


On 9/1/20 10:03 pm, pnp wrote:

> Are there some steps to add support to a new cpu such as microblaze?
>
> Regards.
>
>
>
> --
> Sent from: http://numpy-discussion.10968.n7.nabble.com/
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> https://mail.python.org/mailman/listinfo/numpy-discussion


I don't think we have a documented procedure for adding a new CPU
architecture: it is not that common.

The microblaze is a soft core built in HDL, targeting an FPGA. I would try to break the question into three parts:

- Does the python support for the microblaze implement any the Python C-API?

- Does the C compiler for microblase support enough of the c99 language so that NumPy can be synthesized?

and once NumPy is running:

- Are there acceleration libraries that could supply a BLAS-like interface?


My approach would be to "make it run" then "make it fast", but I doubt the high-level (c) tools are up to the task of sythesizing the NumPy C code to HDL without alot of work.

It might be better to approach this task in a different direction: take one of the NumPy-like projects (CuPy might be a good starting point) and rewrite the C code parts in a way that can be synthesized.

Matti

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may contain
confidential information.  Any unauthorized review, use, disclosure or distribution
is prohibited.  If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Numpy support for Microblaze

Robert Kern-2
On Fri, Jan 10, 2020 at 8:34 AM Frederic Bastien <[hidden email]> wrote:
I like your breakdown. But the fact that it is a softcore is not related. We do not need to compile it to an HDL.
We need it to be compiled to the binary format that the Microblaze support.

Now, after one full version is working as a software, it is possible to optimize part of NumPy by having those part synthesized to hardware.
But from my understanding, it wasn't the question. This would be a bigger effort and I suppose there is no guide from NumPy on how to do this.

Indeed, there is no particular guide. Porting Python first will likely tell you a whole lot about the general process. You can search in the numpy source for "NPY_CPU" (and "PPC" and "ARM") to see where we have applied architecture-specific customizations. I suspect that you will have more trouble dealing with the OS (or lack of OS, if that's the case) than the architecture itself.

--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
pnp
Reply | Threaded
Open this post in threaded view
|

Re: Numpy support for Microblaze

pnp
First of, many thanks to each one of you for taking your time to post your
valuable comments and suggestions. My apologies for not filling in the
details. Here goes:

1.We have linux running on the Microblaze. So, what I am looking at this
point is a software only solution.
2. We have cross compilation tools and buildroot. I was able to compile
python3.8 externally and run it on Microblaze. Note that I compiled python
outside of buildroot since the version of python built from buildroot did
not work.
3. Buildroot did not have support for Numpy. So I added it to the
configuration. To resolve the compilation issues, here is what I did:

diff -r -u
/home/gokul/numpyExpt/numpy-1.16.4/numpy/core/include/numpy/npy_cpu.h
python-numpy-1.16.4.BAK/numpy/core/include/numpy/npy_cpu.h
--- /home/gokul/numpyExpt/numpy-1.16.4/numpy/core/include/numpy/npy_cpu.h      
2019-02-22 06:03:41.000000000 +0530
+++ python-numpy-1.16.4.BAK/numpy/core/include/numpy/npy_cpu.h  2019-12-22
18:52:16.468085099 +0530
@@ -102,6 +102,8 @@
     #define NPY_CPU_ARCEB
 #elif defined(__riscv) && defined(__riscv_xlen) && __riscv_xlen == 64
     #define NPY_CPU_RISCV64
+#elif defined(__MICROBLAZEEL__)
+    #define NPY_CPU_MICROBLAZEEL
 #else
     #error Unknown CPU, please report this to numpy maintainers with \
     information about your platform (OS, CPU and compiler)
diff -r -u
/home/gokul/numpyExpt/numpy-1.16.4/numpy/core/include/numpy/npy_endian.h
python-numpy-1.16.4.BAK/numpy/core/include/numpy/npy_endian.h
--- /home/gokul/numpyExpt/numpy-1.16.4/numpy/core/include/numpy/npy_endian.h  
2019-02-22 06:03:41.000000000 +0530
+++ python-numpy-1.16.4.BAK/numpy/core/include/numpy/npy_endian.h      
2019-12-22 18:54:08.912859083 +0530
@@ -48,7 +48,8 @@
             || defined(NPY_CPU_MIPSEL)        \
             || defined(NPY_CPU_PPC64LE)       \
             || defined(NPY_CPU_ARCEL)         \
-            || defined(NPY_CPU_RISCV64)
+            || defined(NPY_CPU_RISCV64)       \
+            || defined(NPY_CPU_MICROBLAZEEL)
         #define NPY_BYTE_ORDER NPY_LITTLE_ENDIAN
     #elif defined(NPY_CPU_PPC)                \
             || defined(NPY_CPU_SPARC)         \

diff -ruN -x '*.py'
/home/gokul/numpyExpt/numpy-1.16.4/numpy/core/include/microblaze-fenv.h
python-numpy-1.16.4/numpy/core/include/microblaze-fenv.h
--- /home/gokul/numpyExpt/numpy-1.16.4/numpy/core/include/microblaze-fenv.h    
1970-01-01 05:30:00.000000000 +0530
+++ python-numpy-1.16.4/numpy/core/include/microblaze-fenv.h    2020-01-02
14:06:30.410908275 +0530
@@ -0,0 +1,36 @@
+#ifndef MICROBLAZE_FENV_H
+#define MICROBLAZE_FENV_H 1
+
+enum
+  {
+    FE_INEXACT   =
+#define FE_INEXACT      (0x00)
+      FE_INEXACT,
+    FE_UNDERFLOW =
+#define FE_UNDERFLOW    (0x00)
+      FE_UNDERFLOW,
+    FE_OVERFLOW  =
+#define FE_OVERFLOW     (0x00)
+      FE_OVERFLOW,
+    FE_DIVBYZERO =
+#define FE_DIVBYZERO    (0x00)
+      FE_DIVBYZERO,
+    FE_INVALID   =
+#define FE_INVALID      (0x00)
+      FE_INVALID
+  };
+
+enum
+  {
+    FE_TOWARDZERO =
+#define FE_TOWARDZERO   (0x0)
+      FE_TOWARDZERO,
+    FE_DOWNWARD   =
+#define FE_DOWNWARD     (0x0)
+      FE_DOWNWARD,
+    FE_UPWARD     =
+#define FE_UPWARD       (0x0)
+      FE_UPWARD
+  };
+
+#endif

The last change above was to take care of floating point exception errors in
compilation. Not sure if this is the right approach- I had to do this since
Microblaze didnt support all these exceptions

4. When I run with these changes, I get the following runtime error:
~# /usr/local/bin/python

Python 3.8.0 (default, Jan  3 2020, 16:10:54)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/numpy/__init__.py", line 150, in
<module>
    from . import random
  File "/usr/lib/python3.8/site-packages/numpy/random/__init__.py", line
143, in <module>
    from .mtrand import *
SystemError: execution of module numpy.random.mtrand failed without setting
an exception
>>>





--
Sent from: http://numpy-discussion.10968.n7.nabble.com/
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: Numpy support for Microblaze

Robert Kern-2
On Sat, Jan 11, 2020 at 1:30 AM pnp <[hidden email]> wrote:
4. When I run with these changes, I get the following runtime error:
~# /usr/local/bin/python

Python 3.8.0 (default, Jan  3 2020, 16:10:54)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.8/site-packages/numpy/__init__.py", line 150, in
<module>
    from . import random
  File "/usr/lib/python3.8/site-packages/numpy/random/__init__.py", line
143, in <module>
    from .mtrand import *
SystemError: execution of module numpy.random.mtrand failed without setting
an exception
>>>

I'd suggest trying to build a small Cython module by itself to see if this is a platform-specific problem with the general Cython-generated code for initializing the module or a problem that we can fix in the mtrand code.

--
Robert Kern

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.python.org/mailman/listinfo/numpy-discussion