Week 11, 2019: Testing testing testing

As we are spreading and building up our skills in software testing (particularly unit testing), this week’s issue explores why it matters and how to effectively improve your code’s testability. Let’s start with some gentle introductions before moving to a team-specific discussion at the end.

History of Software Testing (list of facts)

Given the short history of the software development industry, one may think that testing is a recent occurrence. The history of testing actually goes back centuries and is almost as old as computing itself. This is a list of significant events in the field, with a move from theoretical research in the early days to a more practical approach in recent years.

A Brief Essay on Software Testing (very long read)

Timeless and in-depth literature review on software testing. Since each section in this essay is a gem by itself, I decided not to summarise it here. However, if you don’t have time to go through the whole paper (12 pages, double-column, excluding references), have a read of subsection 3.3 “Objectives of testing” (page 4) and the whole section 4 “Test Levels”. They provide a 360° overview of the testing world.

Intro to BDD and TDD (blog post)

Here’s an honorary mention for Behaviour Driven Development (BDD), which will be discussed in a future update.  Meanwhile, I think it is still beneficial to have a broad perspective of testing methodologies and understand their relationships. To recap: BDD safeguard the business requirements whereas TDD ensures that all the gears of the machine work as intended. They work together to facilitate the creation and maintenance of functional and beautifully built software system.

The rest of this week’s update will focus on unit testing, which is used inside our team on a day-to-day basis.

What is a Unit Test? (blog post)

The term “unit test” is sometimes loosely misunderstood as “a test for every public method”. A “unit” actually refers to the behaviour of a component. For example, imagine we have a function with 3 different behaviours depending on its input (think of a switch statement, an if or try/catch block), there should ideally be (at least) 3 test cases corresponding to those branches/paths. If one “writes tests for the sake of writing tests”, he/she may combine all those cases into one, which is a classic example of bad testing methodology.

On the other hand, not all functions represent behaviour and therefore does not require a unit test(s).

How Much Unit Test Coverage Do You Need? – The Testivus Answer (forum post)

The above article is an entertaining read. If you are about to skip everything else on this update, read this one. The post summarises my general philosophy on software testing: a developer with a good understanding of the project and testing methodology intuitively knows where and how much testing is needed. Considering our team’s limited testing experience, if you asked me the same test-coverage question in our current project(s), I am very likely to give one of the following answers:

  • Stage 1 – If you are totally new to unit testing: You can start by writing tests for codes you have already written. This is for you to get started with the syntax and grammar of unit testing. Consider this a hello world tutorial.
  • Stage 2 – Once you have learned the robe: You should write tests for important functions or features before writing the code; You may skip the trivial one-liners. This is the second milestone in the journey where you learn to do test-driven development (TDD). It is a significant paradigm shift from a code-and-pray development methodology (and thus requires a good amount of mental power). Skipping the trivial one-liner reduces mental load and is good for your learning.
  • Stage 3 – After you have learned the basic of TDD: You should do TDD for everything. This is an extreme but necessary step. At this stage, you can gradually write tests without looking at the Jest documentations. You have mastered all important testing concepts intuitively (fake, mock, spy, etc.). My intention at this stage is to improve your test-first thinking, such as how to structure the code in a testable manner (e.g. via inversion of control, loading external libraries via interfaces). Once you have mastered these, you will notice a natural reduction in the number of test cases you need to write alongside an improvement in code quality. You now feel very confident about your own testing skill.
  • Stage 4 – You have attained the Tao of testing: You know what to test and what not to. You will realise it yourself once you have reached this stage. You now live and breadth TDD. You know what testable code looks like. Your tests flow naturally without hindering the possibility of refactoring. Your code looks like poetry and other developers will find it hard to add or remove anything to/from it. At this stage, you no longer need nor aim for 100% coverage. You have realised that there is no end to the craft and unit testing is only one small part of an enlightened developer. A major difference between stage 3 and 4 is that in stage 4, you are humbled after realising what you know is still very limited. Disclaimer: I haven’t achieved this stage myself.
  • Stage 5 – You no longer write any test or code. At this level, your power is so great that bad codes crumble and good code flourish when you look at them. A one-liner from you can fix bugs that even thousands of unit tests could not detect. I only heard of this stage through words of mouth and I doubt whether I will ever get there in this lifetime. Nevertheless, it’s a worthy goal to aim for.

I’m afraid that’s it, folks! It was a long weekend after all. See you next week!