Friday, August 26, 2011

The problem with sequence coverage.

Sequence coverage is probably the simplest coverage metric, the information is packaged in PDB files and can be read using tools like Mono.Cecil, but just because a method has 100% sequence coverage does not mean you have 100% code coverage.

I'll use an example from OpenCover's own dogfood tests to demonstrate what I mean. Here is a method which shows that it has 100% coverage (sequence point that is).


However I see an issue and that is that on line 101 there is a condition, i.e. a branch, and yet if the visit count is 1 then there is no possibility that both paths for that branch could have been tested. We can therefore infer that even if we had 10000 visits there is no guarantee that every path would be covered even in such a simple method.

Looking at the OpenCover results from which the coverage report was generated. We get


We can see that each sequence point has been visited once, however the branch coverage shows that only one of the paths for the condition we have identified had been visited (in this case it is the true path); which is good as that is what we deduced.

So if you are using code coverage tools do NOT just rely on sequence point coverage alone to determine how well covered your code is. Luckily OpenCover now, as of 25th Aug 2011, supports branch coverage and ReportGenerator 1.2 displays most of the information to help you identify possible coverage mismatches.

No comments:

Post a Comment