A What Test?

We all know what happens when you make assumptions. Of course, when things are obvious to ourselves, we tend not to notice the possibility they may not be obvious to others. Mea culpa, I made exactly that mistake recently. But in my aspirations to teach software development, that provides me with a great opportunity. I get to explain something that I understand so well, I thought it was obvious to everyone. So what are we talking about today? What is a unit test?

I work with a lot of Fortran programmers that wouldn’t give themselves that title. They are scientists and engineers who happen to write code, because it seems the most efficient way for them to get their work done. That the product of this is software seems almost a strange afterthought to many who see themselves in this role. I started my career this way, so I can understand the sentiment.

I was in the middle of a pair programming session to implement some new functionality in a program, and I was writing a unit test when the people I was working with said something that took me back. They said “How does *main program* know to call this? Is it a new option in the input file?” And then I realized I hadn’t explained what a unit test is.

So what did they think a test was? You run the whole program, and look at the outputs. Maybe it’s modeling some experiment and seeing that things look like the measurements, or we compare to a case we can solve analytically, or just make some basic sanity checks. The point is, it’s a very manual process, and involves executing the whole program. They’re not software developers, why would they think anything different?

Let’s take a look at an example. It’s a bit contrived, but it’s something like what you mind find an engineer writing. I’ve organized it a bit more than you might find in the wild, but that should hopefully make the exercise easier to follow. We’re going to imagine a program that generates values for polynomial functions. Something like the following.

program polynomial_point_generator
    use polynomials_m, only: constant, linear, quadratic
    implicit none
    integer :: num_points, polynomial_degree, i
    real :: x_start, x_increment
    real, allocatable :: xs(:), ys(:), polynomial_coefficients(:)

    print *, "Enter the " &
            // "number of points desired, " &
            // "polynomial degree, " &
            // "x starting point, " &
            // "and x increment:"
    read(*, *)  &
            num_points, &
            polynomial_degree, &
            x_start, &
            x_increment
    allocate(polynomial_coefficients(0:polynomial_degree))
    print *, "Enter polynomial coefficients:"
    read(*, *) polynomial_coefficients
    xs = [(x_start + (i-1)*x_increment, i = 1, num_points)]
    select case (polynomial_degree)
    case (0)
        ys = constant(xs, polynomial_coefficients(0))
    case (1)
        ys = linear( &
                xs, &
                polynomial_coefficients(0), &
                polynomial_coefficients(1))
    case (2)
        ys = quadratic(&
                xs, &
                polynomial_coefficients(0), &
                polynomial_coefficients(1), &
                polynomial_coefficients(2))
    end select
    do i = 1, num_points
        print *, xs(i), ys(i)
    end do
end program

module polynomials_m
    implicit none
    private
    public :: constant, linear, quadratic
contains
    elemental function constant(x, c0) result(y)
        real, intent(in) :: x, c0
        real :: y

        y = c0
    end function

    elemental function linear(x, c0, c1) result(y)
        real, intent(in) :: x, c0, c1
        real :: y

        y = c0 + x*c1
    end function

    elemental function quadratic(x, c0, c1, c2) result(y)
        real, intent(in) :: x, c0, c1, c2
        real :: y

        y = c0 + x*c1 + x**2*c2
    end function
end module

You might test such a program manually by having it generate a handful of points for a handful of cases and checking the outputs, perhaps even graphing the linear and quadratic cases to visually verify they look correct. For something like this example that makes sense, and is perfectly reasonable. The problem comes when we’re working on substantially more complex programs, where the relationships between the inputs and outputs is complicated and non-linear. In that case, trying to test all the possible variations in inputs becomes very labor intensive, and verifying that the outputs are correct becomes very difficult. Not to mention it leaves you vulnerable to the logical fallacy of confirmation bias; I.e. it looks like I expected, so it must be right.

That’s where unit testing comes in. Unit testing is a technique that allows us to test a part of our code, independently from the rest of the program. Preferably, we automate those tests and define objective criteria by which to judge whether they pass or fail so that we can remove a lot of the manual effort and potential for human error in testing our code.

To continue on with our example, imagine we are tasked with adding the capability of generating cubic functions to our program. I highly encourage you to write unit tests for any new functionality you add. Even if you’re not going to go back and write unit tests for the existing stuff, at least write unit tests for the new stuff. And so I wrote something like the following.

BIG DISCLAIMER: These tests do not follow best practices that I recommend. I’ve stripped away all the extra complexity associated with those patterns and techniques to make the example easier to follow. Please don’t write your real tests like this. Although, even these are better than no tests at all.

module cubic_test
    use polynomials_m, only: cubic, quadratic

    implicit none
    private
    public :: &
            test_all_zero_coefficients, &
            test_just_x_cubed, &
            test_matching_quadratic
contains
    function test_all_zero_coefficients() result(passed)
        logical :: passed

        if (0.0 == cubic(42.0, 0.0, 0.0, 0.0, 0.0)) then
            passed = .true.
        else
            passed = .false.
        end if
    end function

    function test_just_x_cubed() result(passed)
        logical :: passed

        if (42.0**3 == cubic(42.0, 0.0, 0.0, 0.0, 1.0)) then
            passed = .true.
        else
            passed = .false.
        end if
    end function

    function test_matching_quadratic() result(passed)
        logical :: passed

        if ( &
                quadratic(42.0, 1.0, 2.0, 3.0) &
                == cubic(42.0, 1.0, 2.0, 3.0, 0.0)) then
            passed = .true.
        else
            passed = .false.
        end if
    end function
end module

So the people I was working with saw me writing these functions and said “where do you call these in the program?” And that’s when I realized we were still thinking completely differently about testing. These functions are not called from the program the user runs. They are called from a separate program that looks something like the following.

program unit_tests
    use cubic_test, only: &
            test_all_zero_coefficients, &
            test_just_x_cubed, &
            test_matching_quadratic

    implicit none

    print *, "Test cubic with all zero coefficients"
    if (test_all_zero_coefficients()) then
        print *, "  passed"
    else
        print *, "  failed"
    end if

    print *, "Test calculating just x**3"
    if (test_just_x_cubed()) then
        print *, "  passed"
    else
        print *, "  failed"
    end if

    print *, "Test with cubic coefficient of zero " &
            // "matches quadratic function"
    if (test_matching_quadratic()) then
        print *, "  passed"
    else
        print *, "  failed"
    end if
end program

Note how these tests do not require any human input, nor human judgement to tell us whether they passed or failed. We can run these tests at any time to easily determine if the code we are testing works as expected. We have taken what would have been a laborious process and automated it. And by testing our program in pieces, we can easily narrow down where any bugs might be if something goes wrong. We also do not have to devise all the inputs to our whole program in order to test one piece of it.

There is a lot more to be said about how to write good unit tests, and how to design your software to make it easier to write unit tests. Stay tuned, cause those are things I intend to keep talking and writing about. Also, let me know was this article helpful, confusing, was there something you’d like to hear more about? I want to produce the stuff you guys find valuable so feedback is always appreciated. And if you’re interested in learning more and applying this stuff to your own projects, I’m available for training and coaching, so please do contact me.