Python Tornado, Apache Thrift and Javascript

Hello everyone,
It’s been a while since my last post, a lot has been going on in my personal life.

I’ve recently started working on a project to implement a virtual joystick device on Microsoft Windows using open source software.
The idea is simple: your mobile phone can be used as a joystick on your Windows machine.The project is still under development, however, you can take peek at the code on Github and watch it working on Youtube, the architecture is as follows:

1. Virtual joystick device driver from a Sourceforge project – controls a virtual joystick on your computer
2. Python Tornado web server – receives RPC calls from the joystick client and hosts an HTML5 application – the joystick client itself
3. HTML5 Joystick client

Note that the version shown on Youtube uses the phone accelerometer, the version currently on Github uses touch screen interface. I might upload a new video soon…

So, how does this relates to Thrift you’re asking? Well, the underlying transport protocol I’ve used for my RPC is a protocol I made up. I’ve used HTTP POST or Web sockets (depending on browser support. Apparently, Android’s native browser doesn’t support web sockets [tested on jellybean]) – and simple JSON for requests & responses.

After getting a working prototype, I started thinking on how to improve it. Aside from additional functionality I want to implement (such as multi-player support, multiple joysticks support, etc.), the RPC architecture bothered me – there has to be a nicer, cleaner way of doing it. Well, apparently, there is – and it’s called Apache Thrift. From their website:

The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.

Investigating it revealed that Thrift supports HTTP as transport protocol (HTTP POST, not web sockets. But more on that later).
I’ve tested it both as a client and a server on Python – works as expected. The next phase was to make it play along with Tornado.
I wanted Tornado to serve the HTTP Thrift calls, both because I’ve already been using Tornado for this project, and because it’d be much easier to implement web sockets supports later on.

Although WebSocketHandler inherits from RequestHandler, Tornado doesn’t let you implement both on the same class. To do that, I had to “trick” Tornado. Here’s how I did it:
The idea was simple. get() function calls will get handled on WebSocketHandler, all the rest by the regular RequestHandler.
Here’s the implementation:

from tornado.web import RequestHandler
from tornado.websocket import WebSocketHandler

class DualHandler(WebSocketHandler):
    """This class lets you implement WebSocket and RequestHandler
    (except for the get() method) in the same class.
    I couldn't find any reason why not to use the same handler both for
    websockets and POST calls, however, tornado doesn't let you do it.
    This is my workaround.
    """

    def _execute(self, *args, **kwargs):
        """Does the exact same thing like WebSocketHandler._execute() for GET
        requests. Otherwise, acts like RequestHandler.
        """
        if self.request.method == 'GET':
            # dispatch to WebSocketHandler
            WebSocketHandler._execute(self, *args, **kwargs)
        else:
            # remap unsupported WebSocketHandler methods back to RequestHandler
            # the __get__(self, DualHandler) trick is used to bind the methods
            # back to the object, otherwise it's unbound
            for method in ["write", "redirect", "set_header", "send_error",
                           "set_cookie", "set_status", "flush", "finish"]:
                    setattr(self, method,
                            getattr(RequestHandler, method).__get__(
                                    self, DualHandler))

            # dispatch to RequestHandler
            RequestHandler._execute(self, *args, **kwargs)

Next, handling Thrift with Tornado. I won’t be touching Tornado much here (you can check out the sources of everything on this post on Github).
I wrote this Tornado Handler:

from thrift.server import TServer
from thrift.transport import TTransport

class ThriftDualHandler(DualHandler, TServer.TServer):
    def initialize(self, processor, inputProtocolFactory,
                  outputProtocolFactory):
        TServer.TServer.__init__(self, processor, None, None, None,
            inputProtocolFactory, outputProtocolFactory)

    def post(self):
        self.set_header('Content-Type', 'application/x-thrift')
        self.write(self.handle_request(self.request.body))

    def on_message(self, message):
        self.write_message(self.handle_request(message))

    def handle_request(self, data):
        itrans = TTransport.TMemoryBuffer(data)
        otrans = TTransport.TMemoryBuffer()
        iprot = self.inputProtocolFactory.getProtocol(itrans)
        oprot = self.outputProtocolFactory.getProtocol(otrans)
        self.processor.process(iprot, oprot)
        return otrans.getvalue()

One more thing you should know before diving into the code: I’ve used TJSONPrtocol.py for transport – the Java script client library doesn’t support other transport methods, also, the Python web sockets library gave me trouble when trying to use TBinaryProtocol.

Here’s the Github project layout:

1. server.py – regular Thrift HTTP server
2. server_tornado.py – Tornado powered Thrift HTTP server – accepts both POST and web socket transport protocols
3. client_post.py – regular Thrift client that uses POST (will work with both servers)
4. client_websocket.py – Thrift client that uses web sockets for transport
5. static/index.html – HTML client with Java script Thrift library – accessible through http://localhost:8888/static/index.html when running server_tornado.py