Rick van den Bosch - Blog

... on software development, architecture and more

This is NOT a test.

Test Driven Development is hot, just like unit testing your software and any other kinds of (automated) testing. And as we all know: sometimes stuff that's hot is misinterpreted, explained wrong or just simply used in a really bad way. Unfortunately, testing is no exception to this rule...

Not too long ago I had a discussion with a project manager who told me the code in his project had a code coverage close to 100%. Because of that, so he felt, his software would be darn close to perfect. I then asked him if testers had validated the unit tests his developers had made. The answer was no, but he didn't see the need for that because "the code coverage was so high". I'll spare you the details of the discussion, but let's say it led to a few new insights for at least one of us ;).

I'l try to make my point over here with some examples. Let's first make a very simple (trivial) method.

public
static string LowercaseConcatenateStrings(string first, string second)
{
    string
result;

    result = first.ToLower();
    result += second.ToLower();

    return
result;
}

As you can see this is a simple method which converts both input parameters to lowercase and concatenates them. Now let's see a unittest which will result in 100% code coverage:

[TestMethod]
public void TestLowercaseConcatenate()
{
   
string result = UnitTestSample.LowercaseConcatenateStrings("test1", "test2");
   
Assert.IsNotNull(result, "Returnvalue is null!");
}

This unittest does result in 100% code coverage, but does not test all the scenario's applicable to the method. A null value for one of either parameters results in an unhandled exception. It doesn't even check if the value of the returned string is anything like it should be! These tests (and the code) are far from perfect, even though the coverage is 100%.

This is a simple example where it's easy to determine the flaws, but you can imagine a more complex method would require an expert's eye to ensure the quality of the asserts is up to par. A method of 20 lines with lots of calculations and object modifications, cannot be asserted with a check to see if the object merely exists.
Of course, and this should come as no surprise, testers are able to look at a piece of code or software a little bit different than developers do, so why not use their expertise. In short: check the quality of your assertions!

Comments

Sanderke said:

Please use a StringBuilder in your examples.

And public methods need to be XML-commented.

Greetz, The Aov Service Team

# November 6, 2007 9:05 AM

Dennis van der Stelt said:

Using StringBuilder would be slower because of initialization time of the object. And everyone knows about the XML comments, but it'd clutter up the sample.

Greetz, The BloggingAbout.NET Team ;)

# November 6, 2007 9:57 AM

Vrieler said:

Don't you think that validating the values of the arguments and the return value fall under Entry/exit coverage (one of the criteria of code coverage)? Meaning that your example would not have a 100% code coverage?

# November 12, 2007 2:14 PM

Rick van den Bosch said:

Hey Vrieler,

Long time no see!

Of course you are right, but in this case I'm talking about Visual Studio's interpretation of Code Coverage. This number can be generated based on all unittests you have run in your solution. And, looking at this example, the Visual Studio number for Code Coverage would be 100%.

Unfortunately, that's the only interpretation of Code Coverage most people are interested in ;)

# November 12, 2007 4:20 PM

Rick van den Bosch - Blog said:

When asked about my point of view about a guideline concerning code coverage, my answer always is: go

# November 26, 2007 8:48 PM
Leave a Comment

(required) 

(required) 

(optional)

(required) 


Please add 2 and 6 and type the answer here: