I just started a small side project last week, and I finally decided to take the opportunity to really make TDD (test-driven development) happen for me using Django. Though I’ve dabbled with testing in Django and done quite a bit more with testing in Ruby, I never managed to embrace TDD fully. It’s about time, I figure.

Well, I hit some snags as I got underway. Partly this had to do with me being something of a novice when it comes to testing, and partly it has to do with the state of testing in Django at present.

My mini-gripe with testing in Django is that it’s just kind of loosey-goosey. You can use doctest, or you can use unittest. You can put your tests in your functions, or in your models, or in tests.py, or in a tests/ subfolder.

For me at least, I want my framework to be a bit more opinionated. I want a bunch of guys who are all smarter than me to have sat around and thought about (or better yet, generalized from a working production code) things like the best way to configure testing for a Python web framework, and then to tell me, in no uncertain terms, the exact way to do it.

I’m obviously exaggerating a bit, and I don’t really mean at all to rant about a phenomally awesome, not to mention open source, framework like Django. What all of the above amounts to is that I had to ponder deeply on the problem space myself and come up with a way of structuring my test code that would work for me.

So that’s what I did, and here is the organizational structure I intend to use as I proceed.

An Organizational Structure for Test Code in Django

This is how Wikipedia defines unit testing:

In computer programming, unit testing is a software design and development method where the programmer verifies that individual units of source code are working properly. A unit is the smallest testable part of an application.

One of the things that kept frustrating me about writing tests was that my tests were all methods of a test case class that referred to a model class or a view module. Most of the examples you see are written in that form, such as the following from the Django testing documentation:

class AnimalTestCase(unittest.TestCase):
    def setUp(self):
        self.lion = Animal.objects.create(name="lion", sound="roar")
        self.cat = Animal.objects.create(name="cat", sound="meow")

    def testSpeaking(self):
        self.assertEquals(self.lion.speak(), 'The lion says "roar"')
        self.assertEquals(self.cat.speak(), 'The cat says "meow"')

What frustated me was that you’d end up with a big long catalog of test methods for all the various methods of Animal. It would also be hard to determine what the test was referring to by looking at the test method name. Sometimes a method might be testSpeaking(); other times it might be testShouldSpeak(). In either case, I didn’t feel like the methods were serving very well as documentation, which is one of the supposed benefits of user testing.

Then as I re-read the definition of a unit test to myself for the hundredth time, it dawned on me. Unit tests are supposed to focus on *the smallest testable part of an application.* But all the examples I was seeing had tests that were organized around the class, which is certainly not the smallest testable part. That distinction belongs to methods and class attributes.

I stumbled upon my first rule: Organize test methods around the individual methods and attributes of a class, instead of the class itself. How so? The easiest is to create a separate test case subclass for *each* method and attribute. Each test method of that subclass is then a test of a behavior of the method or attribute under test.

This mini-breakthrough also helped me flesh out a rule for writing better test method names: Start every test method name with “test_should_”. This borrows from a page in behavior-driven development, but basically “should” is the perfect word to use to express a desired behavior.

In Practice

Here’s what all this looks like in practice, taking the Animal class as an example again from Django’s testing documentation:

# models.py
from django.db import models

class Animal(models.Model):
    name = models.CharField(max_length=20)
    sound = models.CharField(max_length=20)

    def speak(self):
        return 'The %s says "%s"' % (self.name, self.sound)

And here’s how I would propose that test code for this class be structured (with a focus on the organization, not thoroughness of testing or appropriateness of implementation):

from django.test import TestCase
from myapp.models import Animal

class AnimalTestCase(TestCase):
    def setUp(self):
        self.lion = Animal.objects.create(name="lion", sound="roar")

class Class(AnimalTestCase):
    # obviously a silly, trivial test
    def test_should_instantiate_object_of_its_own_type(self):
        self.failUnlessEqual(type(Animal()), Animal)

class NameAttribute(AnimalTestCase):
    def test_should_be_defined(self):
        self.failUnless(hasattr(self.lion, 'name'))

    def test_should_accept_string_value(self):
        a = Animal()
        a.name = "bear"
        a.save()
        self.failUnless(a.id) # object was saved successfully

    def test_should_allow_no_more_than_20_chars(self):
        self.lion.name = "a"*21
        self.failUnlessRaises(Warning, self.lion.save)

class SoundAttribute(AnimalTestCase):
    def test_should_be_defined(self):
        self.failUnless(hasattr(self.lion, 'sound'))

    # similar test_shoulds as with 'name' here

class SpeakMethod(AnimalTestCase):
    def test_should_be_defined(self):
        self.failUnless(hasattr(self.new_pp, 'speak'))

    def test_should_return_appropriate_string(self):
        self.failUnlessEquals(self.lion.speak(), 'The lion says "roar"')

(I have not executed the above code, so please don’t be surprised if it has errors.)

So, it’s pretty simple really. Each attribute or method being tested is given an explicitly named test class, which is a subclass of a single test case class for the class under test. Each specified behavior of that attribute or method is assigned a test method beneath the appropriate test class.

Here’s what I like about this approach:

• You don’t have to “think” while writing tests. If you’ve decided to define a method, you just list off to yourself all of the things it should do and/or require.
• The tests actually read like documentation. Not that you’ll ever see them expressed anywhere in this format, but I like to read them as “NamedAttribute.should_accept_test_value” or “SpeakMethod.should_return_appropriate_string”. It’s just so easy to make sense out of exactly what each part of your model is supposed to do.
• The class names provide some grouping structure to the test methods. Ten groups of ten methods, for example, are a lot easier to make sense of and keep organized than 100 test methods without any other structure.

Getting It To Work

It’s actually quite easy to get this all to work. The only trick involved is that you have to specify which suites to run to the django test runner. (This may not in fact be a requirement, but I wanted adding new test case subclasses to be quick and easy; I didn’t want to spend my life configuring which tests to run in a bunch of places.)

Here’s a quick function I wrote to handle this:

def build_test_suite_from(test_cases):
    """
    Returns a single or group of unittest test suite(s) that's ready to be
    run. The function expects a list of classes that are subclasses of
    TestCase.

    The function will search the module where each class resides and
    build a test suite from that class and all subclasses of it.
    """
    test_suites = []
    for test_case in test_cases:
        mod = __import__(test_case.__module__)
        components = test_case.__module__.split('.')
        for comp in components[1:]:
            mod = getattr(mod, comp)
        tests = []
        for item in mod.__dict__.values():
            if type(item) is type and issubclass(item, test_case):
                tests.append(item)
        test_suites.append(unittest.TestSuite(map(unittest.TestLoader().loadTestsFromTestCase, tests)))
    return unittest.TestSuite(test_suites)

Django allows you to define a suite() function in your test module, which is then automatically detected by the test runner when tests are executed. So we’ll just embed the above function in that, like this, for example:

from myapp.test_utils import build_test_suite_from
from myapp.tests.models.some_model import SomeModelTestCase
from myapp.tests.models.another_model import AnotherModelTestCase

test_cases = [
    SomeModelTestCase,
    AnotherModelTestCase
]

def suite():
    return build_test_suite_from(test_cases)

Here’s what happens: build_test_suite_from() runs through each TestCase you pass to it, and searches the module where that TestCase is found for subclasses of that TestCase (including the TestCase itself). It then builds a test suite containing all of the test methods out of each of these, and then combines all of them together into one giant test suite which Django will run automatically when you do ./manage.py test.

The idea again is to keep configuration to a minimum. For each model you create test cases for you have to import the model’s TestCase and add it to the test_cases list, but no additional configuration is required for subclasses you create of that TestCase.

I expect that this structure will evolve as I actually start using it more. In particular, tests for forms and views may or may not work as cleanly within this structure as do those for models.

If you have any comments or feedback (good or bad), I’d love to hear about it below.