# first recarray steps

10 messages
Open this post in threaded view
|

## first recarray steps

 Hi, I'm trying to get into recarrays. Unfortunately documentation is a bit on the short side... Lets say I have a rgb image of arbitrary size, as a normal ndarray (that's what my image reading lib gives me). Thus shape is (3,ysize,xsize), dtype = int8. How would I convert/view this as a recarray of shape (ysize, xsize) with the first dimension split up into 'r', 'g', 'b' fields? No need for 'x' and 'y' fields. I tried creating a numpy dtype {names: ('r','g','b'), formats: (numpy.int8,)*3}, but when I try to raw_img.view(rgb_dtype) I get: "ValueError: new type not compatible with array." Now this probably should not be too difficult, but I just don't see it... Thanks, Vincent. _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 Hi Vincent 2008/5/20 Vincent Schut <[hidden email]>: > Hi, I'm trying to get into recarrays. Unfortunately documentation is a > bit on the short side... > > Lets say I have a rgb image of arbitrary size, as a normal ndarray > (that's what my image reading lib gives me). Thus shape is > (3,ysize,xsize), dtype = int8. How would I convert/view this as a > recarray of shape (ysize, xsize) with the first dimension split up into > 'r', 'g', 'b' fields? No need for 'x' and 'y' fields. First, you need to flatten the array so you have one (r,g,b) element per row.  Say you have x with shape (3, 4, 4): x = x.T.reshape((-1,3)) Then you can view it with your new dtype: dt = np.dtype([('r',np.int8),('g',np.int8),('b',np.int8)]) x = x.view(dt) Then you must reshape it back to your original pixel arrangement: x = x.reshape((4,4)).T Or you can do it all in one go: x.T.reshape((-1,x.shape[0])).view(dt).reshape(x.shape[1:]).T Maybe someone else comes up with an easier way. Cheers Stéfan _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 In reply to this post by Vincent Schut-2 Vincent Schut wrote: > Lets say I have a rgb image of arbitrary size, as a normal ndarray > (that's what my image reading lib gives me). Thus shape is > (3,ysize,xsize), dtype = int8. How would I convert/view this as a > recarray of shape (ysize, xsize) with the first dimension split up into > 'r', 'g', 'b' fields? No need for 'x' and 'y' fields. Take a look in this list for a thread entitled "recarray fun" about a month ago -- you'll find some more discussion of approaches. Also, if you image data is rgb, usually, that's a (width, height, 3) array: rgbrgbrgbrgb... in memory. If you have a (3, width, height) array, then that's rrrrrrr....gggggggg......bbbbbbbb. Some image libs may give you that, I'm not sure. Also, you probably want a uint8 dtype, giving you 0-255 for each byte. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R            (206) 526-6959   voice 7600 Sand Point Way NE   (206) 526-6329   fax Seattle, WA  98115       (206) 526-6317   main reception [hidden email] _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 Christopher Barker wrote: > > Vincent Schut wrote: >> Lets say I have a rgb image of arbitrary size, as a normal ndarray >> (that's what my image reading lib gives me). Thus shape is >> (3,ysize,xsize), dtype = int8. How would I convert/view this as a >> recarray of shape (ysize, xsize) with the first dimension split up into >> 'r', 'g', 'b' fields? No need for 'x' and 'y' fields. > > Take a look in this list for a thread entitled "recarray fun" about a > month ago -- you'll find some more discussion of approaches. Well, actually that thread was my inspiration to take a closer look into recarrays... > > Also, if you image data is rgb, usually, that's a (width, height, 3) > array: rgbrgbrgbrgb... in memory. If you have a (3, width, height) > array, then that's rrrrrrr....gggggggg......bbbbbbbb. Some image libs > may give you that, I'm not sure. My data is. In fact, this is a simplification of my situation; I'm processing satellite data, which usually has more (and other) bands than just rgb. But the data is definitely in shape (bands, y, x). > > Also, you probably want a uint8 dtype, giving you 0-255 for each byte. Same story. In fact, in this case it's int16, but can actually be any data type, even floats, even complex. But thanks for the thoughts :-) > > -Chris > > > _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 On Wed, May 21, 2008 at 1:48 AM, Vincent Schut <[hidden email]> wrote: > Christopher Barker wrote: >> Also, if you image data is rgb, usually, that's a (width, height, 3) >> array: rgbrgbrgbrgb... in memory. If you have a (3, width, height) >> array, then that's rrrrrrr....gggggggg......bbbbbbbb. Some image libs >> may give you that, I'm not sure. > > My data is. In fact, this is a simplification of my situation; I'm > processing satellite data, which usually has more (and other) bands than > just rgb. But the data is definitely in shape (bands, y, x). I don't think record arrays will help you much, then. Individual records need to be contiguous (bar padding). You can't interleave them. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth."  -- Umberto Eco _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 Robert Kern wrote: > On Wed, May 21, 2008 at 1:48 AM, Vincent Schut <[hidden email]> wrote: >> Christopher Barker wrote: > >>> Also, if you image data is rgb, usually, that's a (width, height, 3) >>> array: rgbrgbrgbrgb... in memory. If you have a (3, width, height) >>> array, then that's rrrrrrr....gggggggg......bbbbbbbb. Some image libs >>> may give you that, I'm not sure. >> My data is. In fact, this is a simplification of my situation; I'm >> processing satellite data, which usually has more (and other) bands than >> just rgb. But the data is definitely in shape (bands, y, x). > > I don't think record arrays will help you much, then. Individual > records need to be contiguous (bar padding). You can't interleave > them. > Hmm, that was just what I was wondering about, when reading Stefan's reply. So in fact, recarrays aren't just another way to view some data, no matter in what shape it is. So his solution: x.T.reshape((-1,x.shape[0])).view(dt).reshape(x.shape[1:]).T won't work, than. Or, at least, won't give me a view on my original dat, but would give me a recarray with a copy of my data. I guess I was misled by this text on the recarray wiki page: "We would like to represent a small colour image. The image is two pixels high and two pixels wide. Each pixel has a red, green and blue colour component, which is represented by a 32-bit floating point number between 0 and 1. Intuitively, we could represent the image as a 3x2x2 array, where the first dimension represents the color, and the last two the pixel positions, i.e. " Note the "3x2x2", which suggested imho that this would work with an image with (bands,y,x) shape, not with (x,y,bands) shape. But I understand that it's not shape, but internal representation in memory (contiguous or not, C/Fortran, etc) that matters? I know I can change the wiki text, but I'm afraid I still don't feel confident on this matter... _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 On Wed, May 21, 2008 at 2:03 AM, Vincent Schut <[hidden email]> wrote: > Robert Kern wrote: >> On Wed, May 21, 2008 at 1:48 AM, Vincent Schut <[hidden email]> wrote: >>> Christopher Barker wrote: >> >>>> Also, if you image data is rgb, usually, that's a (width, height, 3) >>>> array: rgbrgbrgbrgb... in memory. If you have a (3, width, height) >>>> array, then that's rrrrrrr....gggggggg......bbbbbbbb. Some image libs >>>> may give you that, I'm not sure. >>> My data is. In fact, this is a simplification of my situation; I'm >>> processing satellite data, which usually has more (and other) bands than >>> just rgb. But the data is definitely in shape (bands, y, x). >> >> I don't think record arrays will help you much, then. Individual >> records need to be contiguous (bar padding). You can't interleave >> them. >> > Hmm, that was just what I was wondering about, when reading Stefan's > reply. So in fact, recarrays aren't just another way to view some data, > no matter in what shape it is. > > So his solution: > x.T.reshape((-1,x.shape[0])).view(dt).reshape(x.shape[1:]).T won't work, > than. Or, at least, won't give me a view on my original dat, but would > give me a recarray with a copy of my data. Right. > I guess I was misled by this text on the recarray wiki page: > > "We would like to represent a small colour image. The image is two > pixels high and two pixels wide. Each pixel has a red, green and blue > colour component, which is represented by a 32-bit floating point number > between 0 and 1. > > Intuitively, we could represent the image as a 3x2x2 array, where the > first dimension represents the color, and the last two the pixel > positions, i.e. " > > Note the "3x2x2", which suggested imho that this would work with an > image with (bands,y,x) shape, not with (x,y,bands) shape. Yes, the tutorial goes on to use record arrays as a view onto an (x,y,bands) array and also make a (bands,x,y) view from that, too. That is, in fact, quite a confusing presentation of the subject. Now, there is a way to use record arrays here; it's a bit ugly but can be quite useful when parsing data formats. Each item in the record can also be an array. So let's pretend we have a (3,nx,ny) RGB array. nbands, nx, ny = a.shape dtype = numpy.dtype([   ('r', a.dtype, [nx, ny]),   ('g', a.dtype, [nx, ny]),   ('b', a.dtype, [nx, ny]), ]) # The flatten() is necessary to pre-empt numpy from # trying to do too much interpretation of a's shape. rec = a.flatten().view(dtype) print rec['r'] print rec['g'] print rec['b'] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth."  -- Umberto Eco _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion
Open this post in threaded view
|

## Re: first recarray steps

 Robert Kern wrote: > On Wed, May 21, 2008 at 2:03 AM, Vincent Schut <[hidden email]> wrote: >> Robert Kern wrote: >>> On Wed, May 21, 2008 at 1:48 AM, Vincent Schut <[hidden email]> wrote: >>>> Christopher Barker wrote: >>>>> Also, if you image data is rgb, usually, that's a (width, height, 3) >>>>> array: rgbrgbrgbrgb... in memory. If you have a (3, width, height) >>>>> array, then that's rrrrrrr....gggggggg......bbbbbbbb. Some image libs >>>>> may give you that, I'm not sure. >>>> My data is. In fact, this is a simplification of my situation; I'm >>>> processing satellite data, which usually has more (and other) bands than >>>> just rgb. But the data is definitely in shape (bands, y, x). >>> I don't think record arrays will help you much, then. Individual >>> records need to be contiguous (bar padding). You can't interleave >>> them. >>> >> Hmm, that was just what I was wondering about, when reading Stefan's >> reply. So in fact, recarrays aren't just another way to view some data, >> no matter in what shape it is. >> >> So his solution: >> x.T.reshape((-1,x.shape[0])).view(dt).reshape(x.shape[1:]).T won't work, >> than. Or, at least, won't give me a view on my original dat, but would >> give me a recarray with a copy of my data. > > Right. > >> I guess I was misled by this text on the recarray wiki page: >> >> "We would like to represent a small colour image. The image is two >> pixels high and two pixels wide. Each pixel has a red, green and blue >> colour component, which is represented by a 32-bit floating point number >> between 0 and 1. >> >> Intuitively, we could represent the image as a 3x2x2 array, where the >> first dimension represents the color, and the last two the pixel >> positions, i.e. " >> >> Note the "3x2x2", which suggested imho that this would work with an >> image with (bands,y,x) shape, not with (x,y,bands) shape. > > Yes, the tutorial goes on to use record arrays as a view onto an > (x,y,bands) array and also make a (bands,x,y) view from that, too. > That is, in fact, quite a confusing presentation of the subject. > > Now, there is a way to use record arrays here; it's a bit ugly but can > be quite useful when parsing data formats. Each item in the record can > also be an array. So let's pretend we have a (3,nx,ny) RGB array. > > nbands, nx, ny = a.shape > dtype = numpy.dtype([ >   ('r', a.dtype, [nx, ny]), >   ('g', a.dtype, [nx, ny]), >   ('b', a.dtype, [nx, ny]), > ]) > > # The flatten() is necessary to pre-empt numpy from > # trying to do too much interpretation of a's shape. > rec = a.flatten().view(dtype) > print rec['r'] > print rec['g'] > print rec['b'] > Ah, now that is clarifying! Thanks a lot. I'll do some experiments to see whether this way of viewing my data is useful to me (in a sense that making may code more readable is already very useful). Cheers, Vincent. _______________________________________________ Numpy-discussion mailing list [hidden email] http://projects.scipy.org/mailman/listinfo/numpy-discussion