Pretty nearly every developer I’ve ever worked with either hates writing automated tests, or doesn’t do it at all.  And why shouldn’t they?  After all, it’s a ton of tedious work which doesn’t impress anyone looking at the final product.  Yeah, yeah, it improves quality a bit, but still… it take so much time and effort in the first place, and even more effort to keep them from breaking all the time.  Right?

Of course not.

The problem is that we’ve mostly not been taught to write tests, and our testing frameworks tend to lead us in the wrong direction.  For example, consider this made-up little example which follows a pattern I’ve seen all too often:

class MyObnoxiousUnitTest(TestBase):

    @before
    def setup():
        # Do a little work to set things up.  Maybe this is creating
        # a database connection, maybe clearing out a directory of
        # stale test results, etc.
    
    @test
    def test_something(self):
        # here's about 5-10 lines of code to set up some test data
        # ...
        # ...
        # ...
        # ...

        # and here's another 5-10 lines of code to verify the results
        # ...
        # ...
        # ...
        # ...

        # now let's have another 3-4 lines of code to tweak some
        # little thing
        # ...

        # and now another one or two lines to verify that
        # ...

Had enough? And that’s just one test… what about your next one?  I suspect it will look very much the same, and be documented just as well.  Except, you’ll copy-n-paste a little bit from the first setup block, and tweak it some so it looks similar without being quite the same.  The same for the next one… and the next…  And good luck if someone else wrote the tests in the first place.

Before long, you’ve got a test file which is hundreds of lines long with code which has been copy-pasted into existence, but none of which is documented or easy to follow.  So, now what happens when you want to add another test?  More copy-pasta?  Probably.  And the problem gets even worse.  No wonder everyone hates testing.

 

A Better Way

Fortunately, this isn’t the only way to write automated tests, and there are even a number of frameworks which can help (e.g., rspec in Ruby, mamba in Python, or mocha in JavaScript).  This “better” style of testing grew out of a movement called Behavior-Driven Development (BDD).¹

Let’s start with a pretty typical example testing a hypothetical CSV reader class², and then pick it apart:

with description("with a CSV file full of valid data") as self:

    with before.each:
        self.csv_doc = CsvDoc("my-test-data.csv")

    with it("should contain a list of headers"):
        self.csv_doc.headers.should.equal(["alpha", "bravo"])

    with it("should contains only elements of the right form"):
        self.csv_doc.data.should.be.a(list)
        for element in self.csv_doc.data:
            element.should.be.a(dict)
            sorted(element.keys()).should.equal(["alpha", "bravo"])

    with description("when the data is modified and re-read"):

        with before.each:
            self.csv_doc.data.append({"alpha": "a", "bravo": "b"})
            self.csv_doc.save("saved-data.csv")

            self.csv_doc2 = CsvDoc("saved-data.csv")

        with it("should have the same contents as the first doc"):
            self.csv_doc2.should.equal(self.csv_doc)

The first thing you’ll notice is that this approach is a lot more structured. There isn’t just one big test function with a bunch of code in it.  Instead we have a definite pattern where we:

1. 1. Define, in English, what the state we’re testing is (i.e., with description)
   2. Write some code to make that state true (i.e., with before.each)
   3. State, in English, one specific thing which should be true now (i.e., with it)
   4. Write some code to verify that really did happen.

You can see that exact pattern repeated several times in this example.  This makes the tests much easier to follow, and gives a great deal of built-in documentation as to exactly what conditions are being tested, and what the expected outcomes are—in plain English.

The second thing you’ll notice is that this pattern not only repeats, but becomes progressively more nested.  Each nesting means that any state which happened in the outer layers will also be applied to the inner layers.  So, in our final test case (i.e., the last with it statement) we get both of the with before.each statements run before our test.  This provides an exceptionally easy way to share state between individual tests, thus saving us the massively problematic copy-pasta in the more conventional approach.

Finally, the third thing you’ll probably notice is that each with it block is super short.  Since each one only has assertions, and each one only asserts a single condition (described with an English sentence), it really doesn’t need much code.  This makes the tests both extremely well-documented, and very easy to modify.  Stop for a moment, and think how eager you would be to add a missing test case at 16:45 on a Friday with each approach…

✧✧✧

The important take-away here is that the architecture of your tests matters.  We’re often fed the line that test code is throw-away code, and therefore the same rules don’t apply as when writing “real” code.  This is a colossal mistake.  Written badly, your test code will massively slow down a development team, and be a major source of conflict among its members.  Written to the same standards as any other code, it can be fast to write, easy to change, and save you a ton of time and trouble.

 

---

¹ While BDD gets the credit for originating this mode of testing, it recommends going way, way beyond what I personally do or would recommend. ↩︎

² I’m using Python along with the mamba and sure libraries in this example only because that’s what I happen to be working in these days. ↩︎