Writing the steps to validate a test in the commit message
I previously argued that there is value in verifying that a commit that purports to add a failing test actually adds a failing test, and that we might want to encode that information in the commit message somehow.
What if, however, you want to insist on making what has sometimes been called “The Perfect Commit”, which adds both the production code and the passing test at the same time?
Or what if you’re simply writing a Characterization Test for existing code, and are following some variation of the following recipe?
- Write a test that exercises the SUT, including the correct assertion you have in mind.
- Run the test to see it pass.
- Sabotage the SUT so that it fails the assertion. If there are several assertions, do this for each, one after the other.
- Run the test to see it fail.
- Revert the sabotage.
- Run the test again to see it pass.
- Repeat.
— Empirical Characterization Testing, Mark Seemann
Should you commit the sabotage, so that you can verify that the test fails? Probably not. Committing a failing test before making it pass in the next commit is one thing. Sabotaging existing production code to show that a test fails and then undoing it again in the next commit is something else entirely.
Who says that assertions in commit messages have to be about the exact code that that is in the commit, however?
If we can write an assertion in a commit message that says that the build succeeds and the tests pass (or don’t pass), then why not also write an assertion in the commit message that says that “when you sabotage the code in this way, the test fails”? Or “if you undo the change to the production code in this commit, the added test won’t pass”?
Edit: I gave it a go, check out git-check-assertions.