Testing complex high-level functions
In You probably shouldn’t test the leafs directly, except when…, I snuck in a pesky “reasonably” somewhere mid-sentence:
If you can reasonably test the entire tree by testing the root, then do it.
What do you do if you can’t reasonably test the entire tree by testing the root? Well, you write tests against some of your lower-level functions.
We can’t really stop there, though. You still need to the test the high-level functions somehow. How do you prevent the tests for your high-level function from becoming unbelievably complex?
Do you mock away the lower-level functions, so that you can focus on testing the logic that is directly in the high-level function? I guess that that would technically work for unit tests, but you usually shouldn’t. And for integration tests, that idea is of no help at all.
What you can do instead is express the tests for your high-level function in terms of the lower-level functions that it uses.
You make sure that the set-up code for testing your lower-level functions is reusable, so that you can compose it when writing the set-up code for your high-level function. When you then write your assertions, you can assume that your lower-level functions are correct (you’ve tested them, after all), and you can use them freely when computing the (complex) expected result of your high-level function.
One of the most powerful ways to make the set-up code for your lower-level functions reusable is to write Property-Based Tests, where the set-up code for your tests are essentially your test data generators. I owe a lot of my thoughts on how property-based tests can help when testing high-level functions to Mark Seemann’s excellent article series about the epistemology of interaction testing, and I highly recommend that you check out if you want to learn more about this subject. Don’t forget, however, that you can make your tests’ set-up code composable even if you can’t do Property-Based testing. Perfect is the enemy of good.
That raises another question, however.
If your lower-level tests contain a whole slew of test cases with entirely different set-up code (or test data generators in case you’re doing Property-Based Testing), then which ones do you pick when testing the high-level function?
I didn’t have a satisfying answer to that question for a really long time, until I came across an interesting presentation by John Hughes on building on developer’s intuition to create effective property-based tests. It proposes a solution to reduce the burden and risk involved with writing a lot of custom test data generators: For every function that you test, you write one universal test data generator that covers all(!) cases, you make sure that your tests can handle everything that you throw at them, and then you simply measure if every test still covers all relevant cases, and adjust the common test data generator accordingly.
If the tests for your lower-level functions are written like that, then I can finally give a satisfying answer on which lower-level test cases to repeat when testing your high-level function: All of them. And it won’t even cost you, since you simply plug in the universal test data generators that already you wrote for each lower-level function.
If the tests for your lower-level functions aren’t written like that, or aren’t Property-Based tests at all, then just focus on covering every branch in the high-level function. Don’t fall into the trap of trying to cover every branch further down in the tree as well, lest you will end up repeating yourself endlessly.