Writing useful unit tests
Let's face it, writing unit tests is hard.
Many new coders take a long time to get into unit testing. They're scary, a lot of effort, and it's difficult to understand how they work. When we do eek out a few unit tests, they don't seem to be proving anything. They're just a big waste of yours, and your client's time.
It's widely recognised in the industry that testing is important. In general though, most of the tests that get written are complete garbage that don't actually test anything. I'm going to give you some pointers that will help streamline your tests to produce fewer, more useful unit tests.
Keep your functions concise
As anyone who has ever tried Test Driven Development before knows, tests make up an important part of the specification of your code.
Many of the challenges people face when they come to write unit tests are that the unit they are testing is far too complex. The battle for effective tests is fought and won in how you structure your code.
It can take some practice, but you want to divide your code into focussed units that you can easily test, and place your more complex code into their own sections which you then integration test.
What defines a unit is really down to the context, and the developers own preferences. The only hard rules is: If it has to access an API, or reference a different file, it's not a unit.
Here is one of my classic, contrived examples: An add function, that also updates an API and instead of returning a value, updates the page:
function add(a, b) {
let total = a + b;
api.update('/add', {
total: total
});
document.getElementsByClassName('add').innerHtml = total;
}
While it is possible to test this function, we've made it hard for ourselves. In order to test a simple add function, we now need to mock an API, and a DOM. Throw that stuff into a different function - we can write a broader set of tests later.
function add(a, b) {
return a + b;
}
Think about your functions
In unit tests (and general software specifications), we need to think about both the inputs and outputs for our functions, and what we expect them to be - an important point, however, is that we want to write our unit test as though we had no idea how it was written.
In more formal processes, you would define pre- and post-conditions: logic rules for what your inputs should be, and a separate set of rules that the output will meet. The formal definitions of pre- and post-conditions are too weighty for day-to-day coding, but the general concept can be very useful
In this case, I want to say that both our inputs and outputs will be a number
, simply so we can avoid the quirks of +
in Javascript. Additionally, the return value must equal a + b
– looking at the code that seems obvious, but it still forms a crucial part of the spec.
We should expand our function to meet the number
limitation, so we can test it:
function add(a, b) {
if (typeof(a) !== 'number' || typeof(b) !== 'number') {
throw new TypeError('Both inputs must be a Number');
}
return a + b;
}
Now we can write some tests around these concepts:
describe('add()', function() {
it('should add two numbers', function() {
assert.equal(add(1, 2), 3);
});
it('should throw an error when an input is not a number', function() {
assert.throws(add.bind(null, 'a', 2), TypeError);
assert.throws(add.bind(null, 1, false), TypeError);
assert.throws(add.bind(null, {}, 2), TypeError);
})
});
We have asserted our function works. Good job everyone. But before you congratulate yourself too much, I'm going to tell you now that a lot of tests are missing.
Cover your input space
Usually, the metric people tout when evaluating the effectiveness of their tests is code coverage. I'm going to tell you now that code coverage is only barely useful to evaluate your tests. After all, the above tests have 100% coverage of our add
function.
Think about this though: in unit testing, we don't control the code we're testing - we control the inputs. The real golden metric is how much of the input space we have covered.
For the un-initiated, the input space for a function is every possible valid value. The big problem here is that in a lot of cases, the input space is practically infinite. So how do you get 90-100% coverage of infinity?
You don't need to!
What we want to do is identify clusters of values that are similar to each other, and create a test for each of those. In our add
function, we have tested two positive integers – This is good enough to be representative for about a quarter of our input space. So what we need to do now is test some of the behaviour with negative integers, and non-integer numbers.
describe('add()', function() {
it('should add two positive numbers', function() {
assert.equal(add(1, 2), 3);
});
it('should add two negative numbers', function() {
assert.equal(add(-1, -2), -3);
});
it('should add a positive and negative number', function() {
assert.equal(add(-1, 2), 1);
});
it('should add decimals', function() {
assert.equal(add(1.5, 2.3), 3.8);
});
it('should throw an error when an input is not a number', function() {
assert.throws(add.bind(null, 'a', 2), TypeError);
assert.throws(add.bind(null, 1, false), TypeError);
assert.throws(add.bind(null, {}, 2), TypeError);
})
});
We now have the vast majority of our input space covered. We are able to reason that it can handle almost all positive and negative numbers, including fractions - after all, there is no functional difference between 1 and 1,000,000 as far as addition is concerned.
The one thing we didn't explicitly check is whether positive and negative decimals work in the same way as integers. Since we've verified both work independently, we can be fairly confident that they do, but without actively checking we can never be 100%.
... then try to break everything
Far and away the most useful tests you can write are ones that will throw your function a curve ball. We call these edge cases.
What kind of curve balls can we throw basic addition? Well, let's look at our inputs: We're using number
, and that has some special cases which might break everything.
The first is the value Infinity
. This has defined behaviour in mathematics - Infinity
+ anything = Infinity
. That makes it easy to write a test for.
it('should add to infinity', function() {
assert.equal(add(Number.POSITIVE_INFINITY, 2), Infinity);
});
The second value is less clear (and thus a better test). Number.MAX_VALUE
. This number is an actual, finite integer, but it exists due to a technical limit. Javascript will let you add things to MAX_VALUE
, but it will all just equal MAX_VALUE
, but mathematically, there is no reason any special rules should apply.
Since we can't actually add anything to MAX_VALUE
, we can choose to either accept the default behaviour, and say that MAX_VALUE + 1 == MAX_VALUE
, or we can check to see if we're going to end up higher than MAX_VALUE
and throw an error.
it('should not add to Number.MAX_VALUE', function() {
assert.equal(add(Number.MAX_VALUE, 3), Number.MAX_VALUE);
});
The latter behaviour is more difficult to define, and requires more thorough tests.
it('should not add to Number.MAX_VALUE', function() {
assert.throws(add(Number.MAX_VALUE, 3), InputError);
});
it('should allow Number.MAX_VALUE as an input', function() {
assert.equal(add(Number.MAX_VALUE, 0), Number.MAX_VALUE);
});
it('should allow addition to Number.MAX_VALUE', function() {
let input = Number.MAX_VALUE - 3;
assert.equal(add(input, 3), Number.MAX_VALUE);
});
it('should not allow numbers to add to higher than Number.MAX_VALUE', function() {
let input = Number.MAX_VALUE - 3;
assert.throws(add(input, 4), InputError);
});
There is one final edge case that is a little strange, but is probably worth testing anyway: We should test that the order of the parameters doesn't matter.
it('should work both ways', function() {
assert.equal(add(2, 1), 3);
});
It seems like a weird thing to test, but it is an important part of the spec: a + b
should equal b + a
, so we should make it explicit.
Cost of Testing
Just as a word of warning, some testing just isn't worth it.
Testing isn't free. Tests themselves are a codebase that need work, maintenance and time. As a developer, you need to weight the cost of testing the correctness of a function against its importance.
For instance, in general a 1 line add
function would normally not need extensive unit testing - check that it adds numbers together, and then you're away. In a banking application where that function is responsible for millions of dollars though, you suddenly want your testing to be as extensive as it can.
One of the reason we write unit tests is because they are cheaper to write and run than integration tests. We can afford to be more thorough, and test for specific failures in parts of our application, that way we can write fewer integration tests, and save them for the happier, less tricky paths through our code.
As always, get out and try it yourself.