Custom Serialization in Pyro4

I am a pretty big fan of the Python remote object library Pyro. It has a great interface that's small and easy to use. However, there's been one big problem for me. I've been working on a project that composes applications into components running in separate processes to achieve some isolation properties. These components need to communicate with each other, so they do this through a normal Unix socket. I used Pyro to set up the RPCs easily.

The problem is that Pyro uses Python's pickle module. If you click on that link, you should see a nice warning right there at the top. Never unpickle data from an untrusted source. Suddenly my nice isolation properties have vanished! This is a problem that's looked at very nicely over at the blog Why Python Pickle is Insecure. Now, that article and numerous other places on the internet offer suggestions for how to make pickle more safe or to use safer serialization libraries.

What's the problem then? Well, Pyro doesn't really expose a great interface for replacing the serialization library. Or at least they don't advertise one. But I didn't want to give up Pyro, so I poked around in the codebase of Pyro4 for a little bit and found this in Pyro4/util.py:

class Serializer(object):
    """
    A (de)serializer that wraps a certain serialization protocol.
    Currently it only supports the standard pickle protocol.
    It can optionally compress the serialized data, and is thread safe.
    """

    def serialize(self, data, compress = False):
        ...
    def deserialize(self, data, compressed=False):
        ...
    def __eq__(self, other):
        ...
    def __ne__(self, other):
        ...

So it turns out there's this class that defines serialize() and deserialize() methods for use with Pyro. But how do we get our Proxy objects and Daemons to use a custom serializer? Looking at those two class definitions, we find that each has an instance variable for storing that instance's serializer object. So to install your own serializer looks like this:

def start_daemon():
    daemon = Pyro4.Daemon()
    daemon.serializer = customSerializer()
    daemon.register(foo, "foo_name")
    daemon.requestLoop()

def get_proxy():
    proxy = Pyro4.Proxy(uri)
    proxy._pyroSerializer = customSerializer()
    return proxy

Great! So this will work for you as long as your serializer defines the functions that the default serializer does. You only need to be able to serialize and deserialize the objects that your project uses, but remember that Pyro will try to serialize Exceptions and the like.