Possible to pickle new state in NDArray subclasses?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Possible to pickle new state in NDArray subclasses?

Stuart Reynolds
I'm trying to subclass an NDArray as shown here:

My problem is that when I save the new class' state with pickle, the new attributes are lost. I don't seem to be able to override __getstate__ or __setstate__ to achieve this?

Is it possible to allow new state to serialized when overriding an NDArray?

In my example below, __setstate__ gets called by pickle but not __getstate__.
In the final line, a RealisticInfoArray has been deserialized, but it has no .info attribute.

----

import cPickle as pickle
import numpy as np

class RealisticInfoArray(np.ndarray):
    def __new__(cls, arr, info):
        obj = np.asarray(arr).view(cls)
        obj.info = info
        return obj

    def __array_finalize__(self, obj):
        if obj is None: return
        self.info = getattr(obj,"info",None)

    def __setstate__(self, *args):
        print "SET"
        return np.ndarray.__setstate__(self,*args)

    def __getstate__(self):
        print "GET"
        assert False, "EXPLODE"
        return np.ndarray.__getstate__(self)

arr = np.zeros((2,3), int)
arr = RealisticInfoArray(arr, "blarg")
print arr.info
arr2 = pickle.loads(pickle.dumps(arr))
print arr2.info  # no .info attribute!


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible to pickle new state in NDArray subclasses?

Nathan Goldbaum
I'm able to do this in my ndarrary subclass using __reduce__ and __setstate__:


Here it's being used to save the unit information into the pickle for a unit-aware ndarray subclass.

On Wed, Dec 14, 2016 at 1:45 PM, Stuart Reynolds <[hidden email]> wrote:
I'm trying to subclass an NDArray as shown here:

My problem is that when I save the new class' state with pickle, the new attributes are lost. I don't seem to be able to override __getstate__ or __setstate__ to achieve this?

Is it possible to allow new state to serialized when overriding an NDArray?

In my example below, __setstate__ gets called by pickle but not __getstate__.
In the final line, a RealisticInfoArray has been deserialized, but it has no .info attribute.

----

import cPickle as pickle
import numpy as np

class RealisticInfoArray(np.ndarray):
    def __new__(cls, arr, info):
        obj = np.asarray(arr).view(cls)
        obj.info = info
        return obj

    def __array_finalize__(self, obj):
        if obj is None: return
        self.info = getattr(obj,"info",None)

    def __setstate__(self, *args):
        print "SET"
        return np.ndarray.__setstate__(self,*args)

    def __getstate__(self):
        print "GET"
        assert False, "EXPLODE"
        return np.ndarray.__getstate__(self)

arr = np.zeros((2,3), int)
arr = RealisticInfoArray(arr, "blarg")
print arr.info
arr2 = pickle.loads(pickle.dumps(arr))
print arr2.info  # no .info attribute!


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Possible to pickle new state in NDArray subclasses?

Stuart Reynolds
Works great! Thank you.

On Wed, Dec 14, 2016 at 11:51 AM, Nathan Goldbaum <[hidden email]> wrote:
I'm able to do this in my ndarrary subclass using __reduce__ and __setstate__:


Here it's being used to save the unit information into the pickle for a unit-aware ndarray subclass.

On Wed, Dec 14, 2016 at 1:45 PM, Stuart Reynolds <[hidden email]> wrote:
I'm trying to subclass an NDArray as shown here:

My problem is that when I save the new class' state with pickle, the new attributes are lost. I don't seem to be able to override __getstate__ or __setstate__ to achieve this?

Is it possible to allow new state to serialized when overriding an NDArray?

In my example below, __setstate__ gets called by pickle but not __getstate__.
In the final line, a RealisticInfoArray has been deserialized, but it has no .info attribute.

----

import cPickle as pickle
import numpy as np

class RealisticInfoArray(np.ndarray):
    def __new__(cls, arr, info):
        obj = np.asarray(arr).view(cls)
        obj.info = info
        return obj

    def __array_finalize__(self, obj):
        if obj is None: return
        self.info = getattr(obj,"info",None)

    def __setstate__(self, *args):
        print "SET"
        return np.ndarray.__setstate__(self,*args)

    def __getstate__(self):
        print "GET"
        assert False, "EXPLODE"
        return np.ndarray.__getstate__(self)

arr = np.zeros((2,3), int)
arr = RealisticInfoArray(arr, "blarg")
print arr.info
arr2 = pickle.loads(pickle.dumps(arr))
print arr2.info  # no .info attribute!


_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion



_______________________________________________
NumPy-Discussion mailing list
[hidden email]
https://mail.scipy.org/mailman/listinfo/numpy-discussion
Loading...