out >> m_Conscientia; » code coverage

Code Coverage in Visual Studio 2010 Beta 2

Alex Peck — Thu, 03 Dec 2009 21:38:05 +0000

I spent a while trying to enable code coverage in Visual Studio 2010 Beta 2. After I ran my unit tests, when I tried to view the code coverage I got this message:

Empty results generated: none of the instrumented binary was used. Look at test run details for any instrumentation problems.

In LocalTestRun.testrunconfig I had code coverage enabled, but it took me a few attempts to realise how to get to the page where you can select which artifacts to instrument.

Double click code coverage row to get to next page

I’m not really too sure what visual cues indicate that I can double click the row to open the code coverage detail page. This is poor UI, hopefully it will be changed prior to release.

Choose which artifacts to instrument here

100% code coverage

Alex Peck — Fri, 25 Sep 2009 21:31:45 +0000

Unit tests can’t expose bugs in code they don’t execute. Therefore, unit tests should have a high degree of code coverage. If you test first, inject all dependencies and use mocks, 100% block coverage isn’t an onerous a goal. But what does block coverage really tell you?

The answer is not much. It’s easy to spot which code you didn’t execute. For code that did execute, if neither the behaviour nor resultant state is verified, all you really know is that there were no unexpected errors. In other words, this code coverage is merely incidental, and no actual testing took place.

Furthermore, even with 100% coverage you have no indication of the code you should have written, but didn’t. For example, consider this simple one liner:

public float Divide(float dividend, float divisor)
{
   return dividend / divisor;
}
 
public void TestDivide()
{
   float quotient = Divide(10, 2);
   Assert.AreEqual<float>(5, quotient);
}

The test case gives us 100% block coverage. However, we will get different behaviour if we use a divisor of 0. This different behaviour may very well constitute a bug. Interestingly, we won’t find this bug even if we use a more robust coverage metric (e.g path coverage) because it is a boundary problem.

Dependency Injection & Testability

Alex Peck — Wed, 17 Jun 2009 19:01:06 +0000

When objects have no (or few) collaborators it’s relatively easy to get high unit test code coverage. As the number and complexity of collaborators increases, so too does the difficulty in obtaining decent coverage. This talk illustrates why dependency injection principles improve the testability of your code, and ease mock/stub based unit testing.

Code Coverage in VSTS 2005

Alex Peck — Fri, 06 Feb 2009 15:30:46 +0000

I’m measuring test code coverage on an application stack with both .NET (managed) and native (unmanaged) components. For some reason the help doesn’t explicitly state how this is accomplished, and some googling suggested that you need to go outside the IDE to instrument the native code.

This isn’t the case. With a Visual Studio 2005 solution consisting of:

a native .dll project
a .NET .dll project
an mstest project

Instrumentation can be enabled by editing the localtestrun.testrunconfig solution item. Look for the code coverage section and select the artifacts you want to instrument.

Caveat: the native binary will not be instrumented unless it is linked with the /PROFILE switch. There is no error message if you forget, it just silently fails to collect coverage for the .dll.