Ryan's Ramblings

There are a couple of really exciting projects underway at Mozilla right now.

The first is BrowserID, a distributed single-signon system from the Mozilla Identity Team. BrowserID sports a very slick sign-in process, simple integration with your existing software stack, and is designed from the ground up to provide strong user privacy guarantees.

The second is Project Sagrada, a platform for quickly building secure, scalable web applications from the folks I work with on the Mozilla Services Team. While initial development is being driven by Mozilla's internal projects and needs, the hope is that one day developers will be able to just write some simple service definitions then deploy their applications across Mozilla's infrastructure.

Of course, we're not there yet, but we're building up to it piece by piece. One piece I've been working on, and which is coming along quite nicely if I do say so myself, is authentication. We are building a secure, flexible and efficient authentication stack on top of some key Python web technologies such a Pyramid and repoze.who.

So without further ado, here's a glimpse of my first tiny piece of Project Sagrada: adding BrowserID support to a Pyramid app with just a few lines of configuration.

I'm really impressed by Mozilla's new BrowserID project.

A robust distributed identity infrastructure will make the internet a better place, and I'm keen to help that goal along however I can. I was quick to jump on the OpenID bandwagon when it came rolling by, and still use "http://www.rfk.id.au" as a delegated identity on sites that support it. But I've never really liked OpenID – it has always felt clunky to me, both a UI level and as a protocol.

BrowserID fixes pretty much all of the things I dislike about OpenID, and a couple more problems that I didn't even realise I had.

It's been two months since I posted Passionate Developer seeks Dream Job, and I've been overwhelmed by the positive response it has generated - lots of well wishes, kind words, advice, and several really interesting opportunities. Even better, I'm happy to announce that the story has a happy ending.

The short version: I started today with Mozilla, working as a contractor with the Services team. It's been quite a ride to get here, so read on if you're interested in the long-version debrief.

I need something new in my career.

Don't get me wrong. I love what I do and I've no intention of abandoning the tech scene to, say, start an emu farm. But the current combination of work, family and life in general is starting to wear us all down. It's time for a change.

And frankly, if you're going to make a change, you might as well make it the absolute best one you can. So for the duration of this post I'm going to try something rather difficult – I'm going to switch off my usual unassuming, undemanding nature and give myself permission to be greedy. What's my dream job? My serious, no-holds-barred, anything-is-possible dream job. Hopefully I don't come off sounding like an arrogant jerk.

Because I need to know: does it exist, and do I have what it takes to land it?

I've got a little bit of downtime at the moment, and I'm determined to spend it keeping my Python skills and projects up to date. First on the list: testing.

Almost all of my open-source projects come with automated tests of some kind. Some are woefully under-tested (m2wsgi, myppy) while others have collected a test case for just about every feature (pyenchant, esky). But they could all stand to benefit from a little more testing discipline.

If you have any of your own python projects, ask yourself: do they work on older versions of python? How about on 2.7? On 3.2? How about on Windows or OSX? Are you sure?

I want to be able to answer a confident yes to all these questions, so I'm embarking on a rather ambitious plan over the coming months: full test suites, with 100% code coverage, tested on Linux, Windows and OSX, across python 2.5, 2.6, 2.7 and 3.2.

If you've seen any of my latest python projects, you know that I spend a lot of time thinking about freezing python programs – taking a python script and packaging it up into a stand-alone application that can be deployed to an end user. My latest quest has been freezing an application that uses PySide for its GUI, and trying to make the resulting distribution bundle as small as possible. The result is a neat little tool called Hatchet.

This post is part motivational, part example. I'll show you a basic "hello world" app in PySide, take you through the process of freezing it into a stand-alone application, then show how to use Hatchet to shrink the distribution down to a manageable size. If that seems a little too academic for you, consider this for real-world motivation: using the techniques shown in this post, I was able to chop over 40MB off of the SaltDrive application bundle for Mac OSX.

The latest release of PyEnchant now contains an experimental binary distribution for OSX, as both an mpkg installer and a python egg. In theory, users on OSX 10.4 or later should be able to just drop pyenchant-1.6.3-py2.6-macosx-10.4-universal.egg somewhere on sys.path and be up and running and spellchecking with ease.

If you're a Mac user, please try it out and let me know if anything doesn't work the way you expect.

Inspired by a recent discussion on Reddit about a Python-to-C++ compiler called Shed Skin, I decided to write up my own experiences on compiling (a restricted subset of) Python to a stand-alone executable. My tool of choice is the translation toolchain from the PyPy project – a project, by the way, that every Python programmer should take a look at.

Take this very exciting (EDIT: and needlessly inefficient) python script, which we'll assume is in a file "factors.py":

def factors(n):
    """Calculate all the factors of n."""
    for i in xrange(2,n / 2 + 1):
       if n % i == 0:
           return [i] + factors(n / i)
    return [n]

def main(argv):
    n = int(argv[1])
    print "factors of", n, "are", factors(n)

if __name__ == "__main__":
    import sys
    main(sys.argv)

We can of course run this from the command-line using the python interpreter, but gosh that's boring:

$> python factors.py 987654321
factors of 987654321 are [3, 3, 17, 17, 379721]

Instead, let's compile it into a stand-alone executable! Grab the latest source tarball from the PyPy downloads page and unzip it in your work directory: read more >>

I've just spent a few days trying to improve the performance of a frozen Python app - specifically, the time it takes to start up and present a login window. Most of the improvements were down to good old-fashioned writing of better code, but I also put together a couple of tricks to help shave off even more milliseconds. They both target one of the major sources of slowness when starting up a Python app: imports.

I just halved the running time of one of my test suites.

The tests in question are multi-threaded, and while they perform a lot of IO they still push the CPU pretty hard. For some time now, nose has been reporting a happy little message along these lines:

Ran 35 tests in 24.893s

I wouldn't have though anything of it, but every so often this number would drop dramatically – often down to as little as 15 seconds. After a lot of puzzling, I realised that the tests would run faster whenever I had another test suite running at the same time. Making my computer work harder made these tests run almost twice as fast!