printing structured arrays

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

printing structured arrays

Bruce Schultz-2
Hi,

I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?

Here an example of what I mean:

data = [ (1, 2), (3, 4.1) ]
dtype = [('x', float), ('y', float)]
print '### ndarray'
a = numpy.array(data)
print a
print '### structured array'
a = numpy.array(data, dtype=dtype)
print a

Output is:
### ndarray
[[ 1.   2. ]
 [ 3.   4.1]]
### structured array
[(1.0, 2.0) (3.0, 4.0999999999999996)]


Thanks
Bruce


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Gökhan SEVER-2


On Fri, Mar 5, 2010 at 8:00 AM, Bruce Schultz <[hidden email]> wrote:
Hi,

I've just started playing with numpy and have noticed that when printing a structured array that the output is not nicely formatted. Is there a way to make the formatting look the same as it does for an unstructured array?

Here an example of what I mean:

data = [ (1, 2), (3, 4.1) ]
dtype = [('x', float), ('y', float)]
print '### ndarray'
a = numpy.array(data)
print a
print '### structured array'
a = numpy.array(data, dtype=dtype)
print a

Output is:
### ndarray
[[ 1.   2. ]
 [ 3.   4.1]]
### structured array
[(1.0, 2.0) (3.0, 4.0999999999999996)]


Thanks
Bruce


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion


I still couldn't figure out how floating point numbers look nicely on screen in cases like yours (i.e., trying numpy.array2string()) but you can make sure by using numpy.savetxt("file", array, fmt="%.1f") you will always have specified precision in the written file.

--
Gökhan

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Timmie
Administrator
In reply to this post by Bruce Schultz-2
Hello,
I am also looking into the convertsion from strcutured arrays to ndarray.

> I've just started playing with numpy and have noticed that when printing
> a structured array that the output is not nicely formatted. Is there a
> way to make the formatting look the same as it does for an unstructured
> array?

> Output is:
> ### ndarray
> [[ 1.   2. ]
>  [ 3.   4.1]]
> ### structured array
> [(1.0, 2.0) (3.0, 4.0999999999999996)]
How could we make this structured array look like the above shown
ndarray with shape (2, 2)?

Thanks for any additional hint,
Timmie

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

josef.pktd
On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen
<[hidden email]> wrote:

> Hello,
> I am also looking into the convertsion from strcutured arrays to ndarray.
>
>> I've just started playing with numpy and have noticed that when printing
>> a structured array that the output is not nicely formatted. Is there a
>> way to make the formatting look the same as it does for an unstructured
>> array?
>
>> Output is:
>> ### ndarray
>> [[ 1.   2. ]
>>  [ 3.   4.1]]
>> ### structured array
>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
> How could we make this structured array look like the above shown
> ndarray with shape (2, 2)?

.view(float) should do it, to created a ndarray view of the structured
array data

Josef

>
> Thanks for any additional hint,
> Timmie
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

jseabold
On Mon, Mar 8, 2010 at 2:01 PM,  <[hidden email]> wrote:

> On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen
> <[hidden email]> wrote:
>> Hello,
>> I am also looking into the convertsion from strcutured arrays to ndarray.
>>
>>> I've just started playing with numpy and have noticed that when printing
>>> a structured array that the output is not nicely formatted. Is there a
>>> way to make the formatting look the same as it does for an unstructured
>>> array?
>>
>>> Output is:
>>> ### ndarray
>>> [[ 1.   2. ]
>>>  [ 3.   4.1]]
>>> ### structured array
>>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
>> How could we make this structured array look like the above shown
>> ndarray with shape (2, 2)?
>
> .view(float) should do it, to created a ndarray view of the structured
> array data
>

Plus a reshape.  I usually know how many columns I have, so I put in
axis 1 and leave axis 0 as -1.

In [21]: a.view(float).reshape(-1,2)
Out[21]:
array([[ 1. ,  2. ],
       [ 3. ,  4.1]])


Skipper
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

josef.pktd
On Mon, Mar 8, 2010 at 2:04 PM, Skipper Seabold <[hidden email]> wrote:

> On Mon, Mar 8, 2010 at 2:01 PM,  <[hidden email]> wrote:
>> On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen
>> <[hidden email]> wrote:
>>> Hello,
>>> I am also looking into the convertsion from strcutured arrays to ndarray.
>>>
>>>> I've just started playing with numpy and have noticed that when printing
>>>> a structured array that the output is not nicely formatted. Is there a
>>>> way to make the formatting look the same as it does for an unstructured
>>>> array?
>>>
>>>> Output is:
>>>> ### ndarray
>>>> [[ 1.   2. ]
>>>>  [ 3.   4.1]]
>>>> ### structured array
>>>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
>>> How could we make this structured array look like the above shown
>>> ndarray with shape (2, 2)?
>>
>> .view(float) should do it, to created a ndarray view of the structured
>> array data
>>
>
> Plus a reshape.  I usually know how many columns I have, so I put in
> axis 1 and leave axis 0 as -1.
>
> In [21]: a.view(float).reshape(-1,2)
> Out[21]:
> array([[ 1. ,  2. ],
>       [ 3. ,  4.1]])


a.view(float).reshape(len(a),-1)     #if you don't want to count columns

I obviously haven't done this in a while.
And of course, it only works if all elements of the structured array
have the same type.

Josef

>
> Skipper
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

jseabold
On Mon, Mar 8, 2010 at 2:17 PM,  <[hidden email]> wrote:

> On Mon, Mar 8, 2010 at 2:04 PM, Skipper Seabold <[hidden email]> wrote:
>> On Mon, Mar 8, 2010 at 2:01 PM,  <[hidden email]> wrote:
>>> On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen
>>> <[hidden email]> wrote:
>>>> Hello,
>>>> I am also looking into the convertsion from strcutured arrays to ndarray.
>>>>
>>>>> I've just started playing with numpy and have noticed that when printing
>>>>> a structured array that the output is not nicely formatted. Is there a
>>>>> way to make the formatting look the same as it does for an unstructured
>>>>> array?
>>>>
>>>>> Output is:
>>>>> ### ndarray
>>>>> [[ 1.   2. ]
>>>>>  [ 3.   4.1]]
>>>>> ### structured array
>>>>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
>>>> How could we make this structured array look like the above shown
>>>> ndarray with shape (2, 2)?
>>>
>>> .view(float) should do it, to created a ndarray view of the structured
>>> array data
>>>
>>
>> Plus a reshape.  I usually know how many columns I have, so I put in
>> axis 1 and leave axis 0 as -1.
>>
>> In [21]: a.view(float).reshape(-1,2)
>> Out[21]:
>> array([[ 1. ,  2. ],
>>       [ 3. ,  4.1]])
>
>
> a.view(float).reshape(len(a),-1)     #if you don't want to count columns
>
> I obviously haven't done this in a while.
> And of course, it only works if all elements of the structured array
> have the same type.
>

For the archives with heterogeneous dtype.

import numpy as np

b = np.array([(1.0, 'string1', 2.0), (3.0, 'string2', 4.1)],
dtype=[('x', float),('str_var', 'a7'),('y',float)])

b[['x','y']].view(float).reshape(len(b),-1) # note the list within list syntax

#array([[ 1. ,  2. ],
#       [ 3. ,  4.1]])

Skipper
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

josef.pktd
On Mon, Mar 8, 2010 at 2:24 PM, Skipper Seabold <[hidden email]> wrote:

> On Mon, Mar 8, 2010 at 2:17 PM,  <[hidden email]> wrote:
>> On Mon, Mar 8, 2010 at 2:04 PM, Skipper Seabold <[hidden email]> wrote:
>>> On Mon, Mar 8, 2010 at 2:01 PM,  <[hidden email]> wrote:
>>>> On Mon, Mar 8, 2010 at 1:55 PM, Tim Michelsen
>>>> <[hidden email]> wrote:
>>>>> Hello,
>>>>> I am also looking into the convertsion from strcutured arrays to ndarray.
>>>>>
>>>>>> I've just started playing with numpy and have noticed that when printing
>>>>>> a structured array that the output is not nicely formatted. Is there a
>>>>>> way to make the formatting look the same as it does for an unstructured
>>>>>> array?
>>>>>
>>>>>> Output is:
>>>>>> ### ndarray
>>>>>> [[ 1.   2. ]
>>>>>>  [ 3.   4.1]]
>>>>>> ### structured array
>>>>>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
>>>>> How could we make this structured array look like the above shown
>>>>> ndarray with shape (2, 2)?
>>>>
>>>> .view(float) should do it, to created a ndarray view of the structured
>>>> array data
>>>>
>>>
>>> Plus a reshape.  I usually know how many columns I have, so I put in
>>> axis 1 and leave axis 0 as -1.
>>>
>>> In [21]: a.view(float).reshape(-1,2)
>>> Out[21]:
>>> array([[ 1. ,  2. ],
>>>       [ 3. ,  4.1]])
>>
>>
>> a.view(float).reshape(len(a),-1)     #if you don't want to count columns
>>
>> I obviously haven't done this in a while.
>> And of course, it only works if all elements of the structured array
>> have the same type.
>>
>
> For the archives with heterogeneous dtype.
>
> import numpy as np
>
> b = np.array([(1.0, 'string1', 2.0), (3.0, 'string2', 4.1)],
> dtype=[('x', float),('str_var', 'a7'),('y',float)])
>
> b[['x','y']].view(float).reshape(len(b),-1) # note the list within list syntax
>
> #array([[ 1. ,  2. ],
> #       [ 3. ,  4.1]])

nice, I've never seen selection of multiple columns before. I didn't
know it is possible to get a subset of columns this way

>>> b[['x','y']]
array([(1.0, 2.0), (3.0, 4.0999999999999996)],
      dtype=[('x', '<f8'), ('y', '<f8')])
>>> b['x']
array([ 1.,  3.])
>>> b[['x']]
array([(1.0,), (3.0,)],
      dtype=[('x', '<f8')])

Josef


>
> Skipper
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Pierre GM-2
In reply to this post by Timmie
On Mar 8, 2010, at 1:55 PM, Tim Michelsen wrote:

> Hello,
> I am also looking into the convertsion from strcutured arrays to ndarray.
>
>> I've just started playing with numpy and have noticed that when printing
>> a structured array that the output is not nicely formatted. Is there a
>> way to make the formatting look the same as it does for an unstructured
>> array?
>
>> Output is:
>> ### ndarray
>> [[ 1.   2. ]
>> [ 3.   4.1]]
>> ### structured array
>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
> How could we make this structured array look like the above shown
> ndarray with shape (2, 2)?


if you're 100% sure all your fields have the same dtype (float), ``a.view((float,2))`` is the simplest. Note the tuple (dtype, len(a.dtype.names)).
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Timmie
Administrator
In reply to this post by josef.pktd
Hello,
thanks to all who responded and have their input here.

I added a little code snippet to show the view and reshape:

http://www.scipy.org/Cookbook/Recarray

What do you think?
Is this worth to go into the official docs?
The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite
sparse...

I still wonder why there is not a quick function for such a view /
reshape conversion.


Best regards,
Timmie

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

josef.pktd
On Mon, Mar 8, 2010 at 5:50 PM, Tim Michelsen
<[hidden email]> wrote:

> Hello,
> thanks to all who responded and have their input here.
>
> I added a little code snippet to show the view and reshape:
>
> http://www.scipy.org/Cookbook/Recarray
>
> What do you think?
> Is this worth to go into the official docs?
> The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite
> sparse...
>
> I still wonder why there is not a quick function for such a view /
> reshape conversion.


Thanks, the docs for working with arrays with structured dtypes are sparse

Note that in your example .view(np.ndarray) doesn't do anything

>>> struct_diffdtype[['str_var', 'x', 'y']].view(np.ndarray).reshape(len(struct_diffdtype),-1)
array([[(1.0, 'string1', 2.0)],
       [(3.0, 'string2', 4.0999999999999996)]],
      dtype=[('x', '<f8'), ('str_var', '|S7'), ('y', '<f8')])

>>> struct_diffdtype[['x', 'y']].view(np.ndarray).reshape(len(struct_diffdtype),-1)
array([[(1.0, 2.0)],
       [(3.0, 4.0999999999999996)]],
      dtype=[('x', '<f8'), ('y', '<f8')])


view on columns with floating values  (is this a copy???)

>>> struct_diffdtype[['x', 'y']].view(float).reshape(len(struct_diffdtype),-1)
array([[ 1. ,  2. ],
       [ 3. ,  4.1]])

and float view on strings is not possible

>>> struct_diffdtype[['str_var', 'x', 'y']].view(float).reshape(len(struct_diffdtype),-1)
Traceback (most recent call last):
ValueError: new type not compatible with array.

Josef

>
>
> Best regards,
> Timmie
>
> _______________________________________________
> NumPy-Discussion mailing list
> [hidden email]
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Chris Barker - NOAA Federal
In reply to this post by Timmie
Tim Michelsen wrote:
> I still wonder why there is not a quick function for such a view /
> reshape conversion.

Because it is difficult (impossible?) to do in the general case. .view()
really isn't that bad, in fact, it remarkably powerful and flexible!

-Chris


--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

[hidden email]
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Timmie
Administrator
>> I still wonder why there is not a quick function for such a view /
>> reshape conversion.
>
> Because it is difficult (impossible?) to do in the general case. .view()
> really isn't that bad, in fact, it remarkably powerful and flexible!
I would not drop .view() but rather add a convenience function for
struct_1dtype_float_alt = struct_1dtype.view((np.float,
len(struct_1dtype.dtype.names)))

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Timmie
Administrator
In reply to this post by josef.pktd
[hidden email] schrieb:

> On Mon, Mar 8, 2010 at 5:50 PM, Tim Michelsen
> <[hidden email]> wrote:
>> Hello,
>> thanks to all who responded and have their input here.
>>
>> I added a little code snippet to show the view and reshape:
>>
>> http://www.scipy.org/Cookbook/Recarray
>>
>> What do you think?
>> Is this worth to go into the official docs?
>> The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite
>> sparse...
>>
>> I still wonder why there is not a quick function for such a view /
>> reshape conversion.
>
>
> Thanks, the docs for working with arrays with structured dtypes are sparse
>
> Note that in your example .view(np.ndarray) doesn't do anything
Please note that I wanted to demonstrate many different ways of view &
reshape.

I updated the page:
http://www.scipy.org/Cookbook/Recarray

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Timmie
Administrator
In reply to this post by josef.pktd
>> Is this worth to go into the official docs?
>> The page http://docs.scipy.org/doc/numpy/user/basics.rec.html is quite
>> sparse...
>>
>> I still wonder why there is not a quick function for such a view /
>> reshape conversion.
>
>
> Thanks, the docs for working with arrays with structured dtypes are sparse
I cannot recover my longin for docs.scipy.org.

Would you advice to add the elaborated example?

_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Bruce Schultz-2
In reply to this post by Gökhan SEVER-2
On Sat, Mar 6, 2010 at 8:35 AM, Gökhan Sever <[hidden email]> wrote:

>
>
> On Fri, Mar 5, 2010 at 8:00 AM, Bruce Schultz <[hidden email]>
> wrote:
>>
>> Output is:
>> ### ndarray
>> [[ 1.   2. ]
>>  [ 3.   4.1]]
>> ### structured array
>> [(1.0, 2.0) (3.0, 4.0999999999999996)]
>>
>>
>> Thanks
>> Bruce
>>
>
> I still couldn't figure out how floating point numbers look nicely on screen
> in cases like yours (i.e., trying numpy.array2string()) but you can make
> sure by using numpy.savetxt("file", array, fmt="%.1f") you will always have
> specified precision in the written file.

Using numpy.array2string() gives the same format as the output above.

Using numpy.savetxt() creates the same nicely formatted file
containing the lines below for both structured and unstructured
arrays.
1.0 2.0
3.0 4.1

But I was mainly curious about this because I just want to quickly
dump data out to the console for debugging, and the unstructured
format is obviously much easier to read. It seems like from other
discussion in the thread that the quick solution is to convert back to
a unstructured array with something like view((float, 2)), but that
seems a bit clumsy.

Cheers
Bruce
_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: printing structured arrays

Bruce Schultz-2


On 10/03/10 10:09, Bruce Schultz wrote:
On Sat, Mar 6, 2010 at 8:35 AM, Gökhan Sever [hidden email] wrote:
  
On Fri, Mar 5, 2010 at 8:00 AM, Bruce Schultz [hidden email]
wrote:
    
Output is:
### ndarray
[[ 1.   2. ]
 [ 3.   4.1]]
### structured array
[(1.0, 2.0) (3.0, 4.0999999999999996)]
      
I still couldn't figure out how floating point numbers look nicely on screen
in cases like yours (i.e., trying numpy.array2string()) but you can make
sure by using numpy.savetxt("file", array, fmt="%.1f") you will always have
specified precision in the written file.
    

Using numpy.array2string() gives the same format as the output above.
  
I started looking at how array2string() is implemented, and came up with this patch which formats my structured array nicely, the same as an unstructured array. It was mainly done as a proof of concept, so it only works for floats and I'm probably doing the wrong thing to detect a structured array by comparing the dtype to void.  Maybe someone with more numpy experience can tell me if I'm on the right track...

=== modified file 'numpy/core/arrayprint.py'
--- numpy/core/arrayprint.py    2010-02-21 16:16:34 +0000
+++ numpy/core/arrayprint.py    2010-03-10 13:48:22 +0000
@@ -219,6 +219,10 @@
         elif issubclass(dtypeobj, _nt.unicode_) or \
                  issubclass(dtypeobj, _nt.string_):
             format_function = repr
+        elif issubclass(dtypeobj, _nt.void):
+            #XXX this is for structured arrays....
+            format_function = StructuredFormatter(a)
+            separator = '\n '
         else:
             format_function = str
 
@@ -231,6 +235,17 @@
 
     return lst
 
+class StructuredFormatter:
+    def __init__(self, a):
+        self.data = a
+        self.dtype = a.dtype  #XXX use the dtype to build column formatters
+
+    def __call__(self, x):
+        ff = FloatFormat(self.data.view(float), _float_output_precision,
+                          _float_output_suppress_small)
+        return '[' + ' '.join([ff(n) for n in x]) + ']'
+
+   
 def _convert_arrays(obj):
     import numeric as _nc
     newtup = []



Cheers
Bruce


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
http://mail.scipy.org/mailman/listinfo/numpy-discussion