Using GMock with Visual Studio CppUnitTestFramework

One of the things I have been a bit disappointed with myself during the development of OpenCover is the lack of unit testing around the C++ code that makes up the profiler. I did toy with GTest and got some decent tests around the instrumentation engine but I was never able to actually test the profiler callbacks, also I found the lack of GTest integration with Visual Studio quite irritating; I know I have been spoilt by ReSharper. Recently however, during handling Fakes through OpenCover, I had an opportunity to work out how to load the profiler using registry free loading and realised that perhaps such testing might be within my reach, what I was missing however was a mocking library and one that I could use with Visual Studio tooling.

Frankly GMock was the only candidate, the commercial alternatives being out as this was for an OSS project, but the instructions all seemed to want to build a number of libraries (64/32 bit Debug/Release) that I would have to statically link to and maintain these builds should the source or build options change. I decided to try a different tack that wouldn't involve building libraries and it has worked out reasonably successful, so I thought it would be worth commenting on here.

Step 1 

Get the latest GMock (1.7.0) library as a zip file and uncompress it somewhere within your repository.

Step 2

From within Visual Studio update the Additional Include Directories to include the following paths

$(SolutionDir)lib\gmock-1.7.0
$(SolutionDir)lib\gmock-1.7.0\include
$(SolutionDir)lib\gmock-1.7.0\gtest
$(SolutionDir)lib\gmock-1.7.0\gtest\include

Step 3

Add the following to your "stdafx.h"

#include "gmock/gmock.h"
#include "gtest/gtest.h"

Step 4

Add the following to your "stdafx.cpp"

// The following lines pull in the real gmock *.cc files.
#include "src/gmock-cardinalities.cc"
#include "src/gmock-internal-utils.cc"
#include "src/gmock-matchers.cc"
#include "src/gmock-spec-builders.cc"
#include "src/gmock.cc"

// The following lines pull in the real gtest *.cc files.
#include "src/gtest.cc"
#include "src/gtest-death-test.cc"
#include "src/gtest-filepath.cc"
#include "src/gtest-port.cc"
#include "src/gtest-printers.cc"
#include "src/gtest-test-part.cc"
#include "src/gtest-typed-test.cc"

Step 5

Now all you need to do is add initialise GMock and you are ready; as I am using the CppUnitTestFramework I do the following.

TEST_MODULE_INITIALIZE(ModuleInitialize)
{
    // enable google mock
    ::testing::GTEST_FLAG(throw_on_failure) = true;
    int argc = 0;
    TCHAR **argv = NULL;
    ::testing::InitGoogleMock(&argc, argv);
}

Now all you need to do is follow the GMock documentation and add some expectations etc you can as I discovered even mock COM objects and have expectations on them e.g.

EXPECT_CALL(*profilerInfo, SetEventMask(EVENT_MASK_WHEN_FAKES))
                .Times(1)
                .WillRepeatedly(Return(S_OK));

Bonus Round

There were a few little niggles however the first of which is that if an expectation fails, the Visual Studio test runner takes a little too long to close down (I suspect this may be something on my machine related to DrWatson). 

The second was that if an expectation did fail I could only initially see the result using DebugView - ugh - however I found a solution at http://www.durwella.com/post/96457792632/extending-microsoft-cppunittestframework which involves using some extra macros, which I added to my "stdafx.h" and voila the results are now available in Visual Studio.

Finally, I found the mocks were not very lightweight and in fact if I left them hooked in caused performance issues however replacing them with an admittedly less useful stub I could avoid this when necessary.

Excluding code from coverage...

This may (no guarantees) turn into a series of posts on how to refactor your code for testing using simple examples.

This particular example came from a request to add an "Exclude Lines from Coverage" feature to OpenCover. Now there are many ways this could be achieved, none of which I had any appetite for as they were either too clunky and/or could make OpenCover very slow. I am also not a big fan on excluding anything from code coverage; though OpenCover has several exclude options I just thought that this was one step too far in order to achieve that 100% coverage value as it could too easily abused. Even if I did think the feature was useful it still may not get implemented by myself for several days, weeks or months.

But sometimes there are other ways to cover your code without a big refactoring and mocking exercise which can act as a deterrent to doing the right thing.

In this case the user was using EntityFramework and wanted to exclude the code in the catch handlers because they couldn't force EntityFramework to crash on demand - this is quite a common problem in my experience. The user also knew that one approach was to push all that EntityFramework stuff out to another class and could then test their exception handling via mocks but didn't have the time/appetite to go down that path and thus wanted to exclude that code.

I imagined that the user has code that looked something like this:

public void SaveCustomers(ILogger logger)
{
  CustomersEntities ctx = CustomersEntities.Context;//)
  try
  {
    // awsome stuff with EntityFramework
    ctx.SaveChanges();
  }
  catch(Exception ex)
  {
    // do some awesome logging
    logger.Write(ex);
    throw;
  }
}

and I could see why this would be hard (but not impossible) to test the exception handling. Now instead of extracting out all the interactions with the EntityFramework so it is possible to throw an exception during testing I suggested the following refactoring:

internal void CallWrapper(Action doSomething, ILogger logger)
{
  try
  {
    doSomething();
  }
  catch(Exception ex)
  {
    // do some awesome logging
    logger.Write(ex);
    throw;
  }
}

which I would then use like this:

public void SaveCustomers(ILogger logger)
{
  CustomersEntities ctx = CustomersEntities.Context;//)
  CallWrapper(() => {
    // awsome stuff with EntityFramework
    ctx.SaveChanges();
  }, logger);
}

My original tests should still continue as before and now I have a new method that I can now test independently.

I know this isn't the only way to tackle this sort of problem and I'd love to hear about other approaches.

A simple TDD example

I recently posted a response to StackOverflow wrt TDD and Coverage and I thought it would be worth re-posting the response here. The example is simple but hopefully shows how writing the right tests using TDD gives you a better suite of tests for your code than you would probably write if you wrote the tests after the code (which may have been re-factored as you developed).

"As the [original] accepted answer has pointed out your actual scenario reduces to collection.Sum() however you will not be able to get away with this every time.

If we use TDD to develop this (overkill I agree but easy to explain) we would [possibly] do the following (I am also using NUnit in this example out of preference).

[Test]
public void Sum_Is_Zero_When_No_Entries()
{
    var bomManager = new BomManager();
    Assert.AreEqual(0, bomManager.MethodToTest(new Collection<int>()));
}

and then write the following code (note: we write the minimum to meet the current set of tests)

public int MethodToTest(Collection<int> collection)
{
    var sum = 0;
    return sum;
}

We would then write a new test e.g.

[Test]
[TestCase(new[] { 0 }, 0)]
public void Sum_Is_Calculated_Correctly_When_Entries_Supplied(int[] data, int expected)
{
    var bomManager = new BomManager();
    Assert.AreEqual(expected, bomManager.MethodToTest(new Collection<int>(data)));
}

If we ran our tests they would all pass (green) so we need a new test(cases)

[TestCase(new[] { 1 }, 1)]
[TestCase(new[] { 1, 2, 3 }, 6)]

In order to satisfy those tests I would need to modify my code e.g.

public int MethodToTest(Collection<int> collection)
{
    var sum = 0;
    foreach (var value in collection)
    {
        sum += value;
    }
    return sum;
}

Now all my tests work and if I run that through OpenCover I get 100% sequence and branch coverage - Hurrah!.... And I did so without using coverage as my control but writing the right tests to support my code.

BUT there is a 'possible' defect... what if I pass in null? Time for a new test to investigate

[Test]
public void Sum_Is_Zero_When_Null_Collection()
{
    var bomManager = new BomManager();
    Assert.AreEqual(0, bomManager.MethodToTest(null));
}

The test fails so we need to update our code e.g.

public int MethodToTest(Collection<int> collection)
{
    var sum = 0;
    if (collection != null)
    {
        foreach (var value in collection)
        {
            sum += value;
        }
    }
    return sum;
}

Now we have tests that support our code rather than tests that test our code i.e. our tests do not care about how we went about writing our code.

Now we have a good set of tests so we can now safely refactor our code e.g.

public int MethodToTest(IEnumerable<int> collection)
{
    return (collection ?? new int[0]).Sum();
}

And I did so without affecting any of the existing tests."