Testing file uploads in Django

Following my previous post on testing Django with Windmill, I quickly ran into a common snag with in-browser web app testing: it's not possible to programmatically set the value of file input fields. This makes it very difficult to test file upload functionality using frameworks such as Windmill or Selenium.

In Firefox it's possible to request elevated permissions for your unit tests, but this is far from ideal. It means the tests are no longer automatic (you have to click "yes, grant this page extra permissions" whenever the tests are run) and it takes other browsers out of the testing loop. Like many things in life, the easiest solution seems to be simply to fake it.

But like any convincing fakery, the details are never that simple in practice. Uploading a big file from a web browser will take a long time, but could be nearly instantaneous if you fake it using a server-side file. And what if you have custom upload handlers to enable things like upload progress reporting? How can we make fake file uploads as transparent and convincing as possible?

Presenting FakeFileUploadMiddleware.

This middleware class hooks into the standard Django file upload mechanism, re-writing requests and responses to transparently provide support for fake file uploads. At the heart of its operation is the setting FAKEUPLOAD_FILE_SPEC, which specifies a set of available files for fake upload. Here's a representative example:

FAKEUPLOAD_FILE_SPEC = {

  "smallfile":  { "filename": "test1.txt",
                      "contents": "I am a small text file"},
  "slowfile":   { "filename": "example.bin",
                      "file": "/path/on/server/to/example.bin",
                      "chunk_size": 1024,
                      "sleep_time": 1 }
}```

This specifies two fake files.  The one with id "smallfile" is a simple text file whose contents are specified directly.  The one with id "slowfile" takes its contents from a file on the server, will read from this file in 1KB chunks, and will sleep for 1 second between each read from the file.  As you can probably guess, this makes the simulated upload quite slow-and-steady, which is very useful for testing the behaviour of any fancy AJAX progress bars you might have on the site.

Internally, the middleware manages fake uploads by inserting additional fields into any forms being sent to the client.  Suppose we have this simple form:

```html 
<form method='POST' enctype='multipart/form-data'>
  <input type='file' name='myfile' />
  <input type='submit' name='upload' value='upload' />
</form>

As it passes through FakeFileUploadMiddleware on its way to the client, it will be re-written to:

<form method='POST' enctype='multipart/form-data'>
  <input type='hidden' name='fakefile_myfile' />
  <input type='file' name='myfile' />
  <input type='submit' name='upload' value='upload' />
</form>

In-browser test scripts can then set the value of this hidden form field to the id of a fake file ("smallfile" or "slowfile" in the example specification above). When these fields are received by the middleware, they are translated into raw file upload data which is then passed through the standard Django file upload mechanism. The resulting request object is just about indistinguishable from a real file upload.

I implemented this as middleware so that I can use it in true set-and-forget fashion – I simply include FakeFileUploadMiddleware in the middleware list on my testing server, but remove it on the deployment servers. None of the rest of the application needs to be modified to support fake uploads. Of course, if you want to apply this ability to just a specific view function, you can always use the decorator_from_middleware function to do so.