

All,
Most of the errors that are reported in 1.1.0rc1 are related to the .tolist()
method in numpy.ma, such as :
############################
ERROR: Tests fields retrieval

Traceback (most recent call last):
File
"C:\Apps\Python\Lib\sitepackages\numpy\ma\tests\test_mrecords.py", line
62, in test_get
assert_equal(getattr(mbase,field), mbase[field])
File "C:\Apps\Python\Lib\sitepackages\numpy\ma\testutils.py", line
104, in assert_equal
desired.tolist(),
File "C:\Apps\Python\Lib\sitepackages\numpy\ma\core.py", line 2552,
in tolist
result = self.filled().tolist()
RuntimeError: array_item not returning smallerdimensional array
#####################################
Note that the method seems to work, still: for example, the following command
gives the proper output, without RuntimeError
python c "import numpy as np, numpy.ma as ma; x=ma.array(np.random.rand(5),
mask=[1,0,0,0,0]); print x.tolist()"
The problem looks quite recent, and not related to numpy.ma itself: what
changed recently in the .tolist() method of ndarrays ? Why do we get these
RuntimeErrors ?
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 9:28 AM, Pierre GM < [hidden email]> wrote:
All,
Most of the errors that are reported in 1.1.0rc1 are related to the .tolist()
method in numpy.ma, such as :
############################
ERROR: Tests fields retrieval

Traceback (most recent call last):
File
"C:\Apps\Python\Lib\sitepackages\numpy\ma\tests\test_mrecords.py", line
62, in test_get
assert_equal(getattr(mbase,field), mbase[field])
File "C:\Apps\Python\Lib\sitepackages\numpy\ma\testutils.py", line
104, in assert_equal
desired.tolist(),
File "C:\Apps\Python\Lib\sitepackages\numpy\ma\core.py", line 2552,
in tolist
result = self.filled().tolist()
RuntimeError: array_item not returning smallerdimensional array
#####################################
Note that the method seems to work, still: for example, the following command
gives the proper output, without RuntimeError
python c "import numpy as np, numpy.ma as ma; x=ma.array(np.random.rand(5),
mask=[1,0,0,0,0]); print x.tolist()"
The problem looks quite recent, and not related to numpy.ma itself: what
changed recently in the .tolist() method of ndarrays ? Why do we get these
RuntimeErrors ?
I expect it comes from the matrix churn. The tolist method was one of those with a workaround for the failure to reduce dimensions. I'll look at it later if someone doesn't beat me to it.
Chuck
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 11:35 AM, Charles R Harris
< [hidden email]> wrote:
> On Wed, May 21, 2008 at 9:28 AM, Pierre GM < [hidden email]> wrote:
>> The problem looks quite recent, and not related to numpy.ma itself: what
>> changed recently in the .tolist() method of ndarrays ? Why do we get these
>> RuntimeErrors ?
>
> I expect it comes from the matrix churn. The tolist method was one of those
> with a workaround for the failure to reduce dimensions. I'll look at it
> later if someone doesn't beat me to it.
There's some commentary and a patch on NumPy ticket 793 on this issue:
http://scipy.org/scipy/numpy/ticket/793Hope it's helpful.
Alan
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 10:07 AM, Alan McIntyre < [hidden email]> wrote:
On Wed, May 21, 2008 at 11:56 AM, Pierre GM < [hidden email]> wrote:
> On Wednesday 21 May 2008 11:39:32 Alan McIntyre wrote:
>> There's some commentary and a patch on NumPy ticket 793 on this issue:
>>
>> http://scipy.org/scipy/numpy/ticket/793
>
> OK, thanks a lot ! That's a C problem, then...
It's probably worth mentioning that I'm not that familiar with all the
innards of NumPy yet, so take my comments and patch on that issue with
a (fairly large) grain of salt. ;) This was introduced by Travis in r5138 as part of the matrix changes.
Chuck
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 2:39 PM, Charles R Harris < [hidden email]> wrote:
On Wed, May 21, 2008 at 10:07 AM, Alan McIntyre < [hidden email]> wrote:
On Wed, May 21, 2008 at 11:56 AM, Pierre GM < [hidden email]> wrote:
> On Wednesday 21 May 2008 11:39:32 Alan McIntyre wrote:
>> There's some commentary and a patch on NumPy ticket 793 on this issue:
>>
>> http://scipy.org/scipy/numpy/ticket/793
>
> OK, thanks a lot ! That's a C problem, then...
It's probably worth mentioning that I'm not that familiar with all the
innards of NumPy yet, so take my comments and patch on that issue with
a (fairly large) grain of salt. ;) This was introduced by Travis in r5138 as part of the matrix changes.
Alan's change looks a bit iffy to me because the looser check would pass things like matrices and there would be no check for decreasing dimensions to throw an error. So I think the safest thing for 1.1 is to back out Travis' change and rethink this for 1.2.
Chuck
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 3:10 PM, Charles R Harris < [hidden email]> wrote:
On Wed, May 21, 2008 at 2:39 PM, Charles R Harris < [hidden email]> wrote:
On Wed, May 21, 2008 at 10:07 AM, Alan McIntyre < [hidden email]> wrote:
On Wed, May 21, 2008 at 11:56 AM, Pierre GM < [hidden email]> wrote:
> On Wednesday 21 May 2008 11:39:32 Alan McIntyre wrote:
>> There's some commentary and a patch on NumPy ticket 793 on this issue:
>>
>> http://scipy.org/scipy/numpy/ticket/793
>
> OK, thanks a lot ! That's a C problem, then...
It's probably worth mentioning that I'm not that familiar with all the
innards of NumPy yet, so take my comments and patch on that issue with
a (fairly large) grain of salt. ;) This was introduced by Travis in r5138 as part of the matrix changes.
Alan's change looks a bit iffy to me because the looser check would pass things like matrices and there would be no check for decreasing dimensions to throw an error. So I think the safest thing for 1.1 is to back out Travis' change and rethink this for 1.2.
Chuck
I backed out r5138 and, with a few test fixes, everything passes. However, there is now a warning exposed in the masked array tests:
/usr/lib/python2.5/sitepackages/numpy/ma/core.py:1357: UserWarning: MaskedArray.__setitem__ on fields: The mask is NOT affected!
warnings.warn("MaskedArray.__setitem__ on fields: "
Pierre?
Chuck
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wednesday 21 May 2008 17:57:30 Charles R Harris wrote:
> On Wed, May 21, 2008 at 3:10 PM, Charles R Harris
> < [hidden email]>
>
> wrote:
> > On Wed, May 21, 2008 at 2:39 PM, Charles R Harris <
> >
> > [hidden email]> wrote:
> >> On Wed, May 21, 2008 at 10:07 AM, Alan McIntyre
> >> < [hidden email]>
> >>
> >> wrote:
> >>> On Wed, May 21, 2008 at 11:56 AM, Pierre GM < [hidden email]>
> >>>
> >>> wrote:
> >>> > On Wednesday 21 May 2008 11:39:32 Alan McIntyre wrote:
> >>> >> There's some commentary and a patch on NumPy ticket 793 on this
> >>> >> issue:
> >>> >>
> >>> >> http://scipy.org/scipy/numpy/ticket/793> >>> >
> >>> > OK, thanks a lot ! That's a C problem, then...
> >>>
> >>> It's probably worth mentioning that I'm not that familiar with all the
> >>> innards of NumPy yet, so take my comments and patch on that issue with
> >>> a (fairly large) grain of salt. ;)
> >>
> >> This was introduced by Travis in r5138 as part of the matrix changes.
> >
> > Alan's change looks a bit iffy to me because the looser check would pass
> > things like matrices and there would be no check for decreasing
> > dimensions to throw an error. So I think the safest thing for 1.1 is to
> > back out Travis' change and rethink this for 1.2.
> >
> > Chuck
>
> I backed out r5138 and, with a few test fixes, everything passes. However,
> there is now a warning exposed in the masked array tests:
>
> /usr/lib/python2.5/sitepackages/numpy/ma/core.py:1357: UserWarning:
> MaskedArray.__setitem__ on fields: The mask is NOT affected!
> warnings.warn("MaskedArray.__setitem__ on fields: "
>
> Pierre?
Well, that's a warning all right, not an error.
With "exotic" dtypes (understand, not int,bool,float or complex but named
fields), setting a field doesn't affect the mask, hence the warning.
The reason behind this behavior is that the mask of a MaskedArray is only a
booleanarray: you have (at most) one boolean per element/record. Therefore,
you can mask/unmask a full record, but not a specific field. Masking
particular fields is possible with MaskedRecords, however, but with an
overhead that wasn't worth putting in MaskedArray.
Because I've been bitten a couple of times by this mechanism, I figured that a
warning would be the easiest way to remember that MaskedArrays don't handle
records very well.
So, nothing to worry about.
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 5:56 PM, Pierre GM < [hidden email]> wrote:
On Wednesday 21 May 2008 17:57:30 Charles R Harris wrote:
> On Wed, May 21, 2008 at 3:10 PM, Charles R Harris
> < [hidden email]>
>
> wrote:
> > On Wed, May 21, 2008 at 2:39 PM, Charles R Harris <
> >
> > [hidden email]> wrote:
> >> On Wed, May 21, 2008 at 10:07 AM, Alan McIntyre
> >> < [hidden email]>
> >>
> >> wrote:
> >>> On Wed, May 21, 2008 at 11:56 AM, Pierre GM < [hidden email]>
> >>>
> >>> wrote:
> >>> > On Wednesday 21 May 2008 11:39:32 Alan McIntyre wrote:
> >>> >> There's some commentary and a patch on NumPy ticket 793 on this
> >>> >> issue:
> >>> >>
> >>> >> http://scipy.org/scipy/numpy/ticket/793
> >>> >
> >>> > OK, thanks a lot ! That's a C problem, then...
> >>>
> >>> It's probably worth mentioning that I'm not that familiar with all the
> >>> innards of NumPy yet, so take my comments and patch on that issue with
> >>> a (fairly large) grain of salt. ;)
> >>
> >> This was introduced by Travis in r5138 as part of the matrix changes.
> >
> > Alan's change looks a bit iffy to me because the looser check would pass
> > things like matrices and there would be no check for decreasing
> > dimensions to throw an error. So I think the safest thing for 1.1 is to
> > back out Travis' change and rethink this for 1.2.
> >
> > Chuck
>
> I backed out r5138 and, with a few test fixes, everything passes. However,
> there is now a warning exposed in the masked array tests:
>
> /usr/lib/python2.5/sitepackages/numpy/ma/core.py:1357: UserWarning:
> MaskedArray.__setitem__ on fields: The mask is NOT affected!
> warnings.warn("MaskedArray.__setitem__ on fields: "
>
> Pierre?
Well, that's a warning all right, not an error.
With "exotic" dtypes (understand, not int,bool,float or complex but named
fields), setting a field doesn't affect the mask, hence the warning.
The reason behind this behavior is that the mask of a MaskedArray is only a
booleanarray: you have (at most) one boolean per element/record. Therefore,
you can mask/unmask a full record, but not a specific field. Masking
particular fields is possible with MaskedRecords, however, but with an
overhead that wasn't worth putting in MaskedArray.
Because I've been bitten a couple of times by this mechanism, I figured that a
warning would be the easiest way to remember that MaskedArrays don't handle
records very well.
So, nothing to worry about.
Should we disable the warning for the tests? It's a bit unnerving and likely to generate mail calling attention to it.
Chuck
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


> Should we disable the warning for the tests? It's a bit unnerving and
> likely to generate mail calling attention to it.
I don't have a problem with that. Is there some kind of trapping mechanism we
can use ? A la "fail_unless_raise" ?
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 7:12 PM, Charles R Harris
< [hidden email]> wrote:
> Should we disable the warning for the tests? It's a bit unnerving and likely
> to generate mail calling attention to it.
Yes. Known warnings should be explicitly silenced in tests.

Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 Umberto Eco
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 5:12 PM, Charles R Harris
< [hidden email]> wrote:
> Should we disable the warning for the tests? It's a bit unnerving and likely
> to generate mail calling attention to it.
+1
I tend to agree that it is disconcerting to have warnings pop up in the tests.

Jarrod Millman
Computational Infrastructure for Research Labs
10 Giannini Hall, UC Berkeley
phone: 510.643.4014
http://cirl.berkeley.edu/_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion


On Wed, May 21, 2008 at 7:23 PM, Pierre GM < [hidden email]> wrote:
>
>> Should we disable the warning for the tests? It's a bit unnerving and
>> likely to generate mail calling attention to it.
>
> I don't have a problem with that. Is there some kind of trapping mechanism we
> can use ?
You can filter warnings using warning.filterwarnings() or
warning.simplefilter().
http://docs.python.org/dev/library/warnings> A la "fail_unless_raise" ?
I'm not sure I see the connection. Do you need to test that the
warning is issued? If so, then make a filter with the action 'error'
and use self.assertRaises().

Robert Kern
"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 Umberto Eco
_______________________________________________
Numpydiscussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpydiscussion

