Howto get AutoFixture.NUnit2 working with NCrunch

Now that the AutoFixture.NUnit2 is out there in the wild, a few people have commented that the deployment story has a bit of friction, and I agree with that statement, but that is how the NUnit Extensibility model works.

After a few attempts to make NCrunch work with AutoFixture.NUnit2 resorted to looking on NCrunch support forum it mentioned to include the Addin in the test assembly. This got me thinking… what if we created a class inherited from AutoFixture.NUnit2 Addin. To my surprise this worked.

Here is how:

  • 1. Install-Package AutoFixture.NUnit2
  • 2. Install-Package NUnit.Runners (will have to manually reference tools\nunit.core.interfaces)
  • 3. Remove this line [assembly: NUnit.Framework.RequiredAddin(Ploeh.AutoFixture.NUnit2.Addins.Constants.AutoDataExtension)] from AssemblyInfo.cs
  • 4. Create the below class in each test assembly
using NUnit.Core.Extensibility;

namespace ...
{
    [NUnitAddin]
    public class LocalAddin : Ploeh.AutoFixture.NUnit2.Addins.Addin
    {   
    }
}

To make NCrunch work with AutoFixture.NUnit2 change the “solution configuration” – “framework utilisation type for NUnit” option from “Dynamic Analysis” to “Static Analysis” this is explained in the support documentation

Why does this work?
When you are developing NUnit Addins, NUnit will load any classes that implements IAddin into the test runner’s AddinRegistry this is intended for development only.

Interesting side effects
You can use the buildin TeamCity NUnit Runner with AutoFixture.NUnit2 no need to use msbuild script as previously explained.

Howto setup TeamCity NUnit Runner with AutoFixture.NUnit2

In a previous post I explained how setup AutoFixture.NUnit2 to work with a few 3rd party applications, to complete the AutoFixture.NUnit2 story we need to be able to configure TeamCity for Continuous integration according to Teamcity documentations there are a few ways to do this, but only way I have been successful so far was to configure msbuild script to run nunit-console.exe

Example of msbuild script to copy Teamcity NUnit Addin and AutoFixture.NUnit2.Addins.dll

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" DefaultTargets="CompleteBuild" ToolsVersion="4.0">
    <PropertyGroup>
    	<NUnitHome>packages\nunit.runners.2.6.2\tools</NUnitHome>
    </PropertyGroup>

    <ItemGroup>
    	<NUnitAddinFiles Include="$(teamcity_dotnet_nunitaddin)-2.6.2.*" />
    	<NUnitAddinFiles Include="packages\AutoFixture.NUnit2.3.9.0\lib\net40\Ploeh.AutoFixture.NUnit2.Addins.dll" />
    </ItemGroup>

    <Target Name="RunTests">
    	<MakeDir Directories="$(NUnitHome)\addins" />
	<Copy SourceFiles="@(NUnitAddinFiles)" DestinationFolder="$(NUnitHome)\addins" />
	<Exec Command="$(NUnitHome)\nunit-console.exe $(TestAssembly)" />
    </Target>

    <Target Name="CompleteBuild" DependsOnTargets="RunTests" />
</Project>

Howto guide for AutoFixture.NUnit2

With the release of AutoFixture.NUnit2 earlier today, here is some guidance howto get it working

NUnit Documentation: Addin Identification, Loading and Installation

NUnit examines all assemblies in the bin/addins directory, looking for public classes with the NUnitAddinAttribute and implementing the IAddin interface. It loads all those it finds as Addins.

After package install you need to copy packages\AutoFixture.NUnit2.3.9.0\lib\net40\Ploeh.AutoFixture.NUnit2.Addins.dll to bin\addins folder

But where exactly is bin/addins located. The short answer is where is the nunit.exe or nunit-console.exe located

Here is some default locations

  • NUnit2.6.2 -> C:\Program Files (x86)\NUnit 2.6.2\bin\addins
  • R# 8 -> C:\Program Files (x86)\JetBrains\ReSharper\v8.0\Bin\addins
  • TestDriven.Net -> C:\Program Files (x86)\TestDriven.NET 3\NUnit\2.6\addins

Mark Seemann better explains how to use AutoDataAttribute here:

NUnit example

[Test, AutoData]
public void IntroductoryTest(
    int expectedNumber, MyClass sut)
{
    int result = sut.Echo(expectedNumber);
    Assert.Equal(expectedNumber, result);
}

NUnit Extensibility and AutoFixture

As a software consultant we can’t always choose the libraries we use on a client site, and so the story begins.

I have fallen in love with a library called AutoFixture it really simplifies the Arrange phase in writing unit tests. There are many patterns on how to use AutoFixture, but one pattern in particular that I like is where your unit test doesn’t take any dependency on AutoFixture.

Mark Seemann the author of AutoFixture also wrote an extension for xUnit.net  AutoDataAttribute which supplies auto generated arguments to a parameterised unit test.

[Theory, AutoData]
public void IntroductoryTest(int expectedNumber, MyClass sut)
{
    int result = sut.Echo(expectedNumber);
    Assert.Equal(expectedNumber, result);
}

Extending AutoDataAttribute allows you to customise the internal Fixture that is used during auto generate process.

public class TestConventionsAttribute
  : AutoDataAttribute
{
   public TestConventionsAttribute()
        : base(new Fixture().Customize(new AutoNSubstituteCustomization())
    {
    }
}

I wanted to be able to do the same using NUnit as said above, we can’t always choose what unit testing library we use. So how do you extend NUnit? After reading a few blog posts and NUnit Documentation on extensibility I was more confused than when I started.

Now where to start… NUnit contained a TheoryAttribute, but the implementation is really different from what I am used to in xUnit, and the NUnit DataAttribute is a property only attribute, so this wasn’t even an option to extend.

Some more investigation, I found the TestCaseAttribute and some examples of parameterised unit tests with NUnit.

[TestCase(12,3,4)]
[TestCase(12,2,6)]
[TestCase(12,4,3)]
public void DivideTest(int n, int d, int q)
{
    Assert.AreEqual( q, n / d );
}

This looked much closer to what I was after, so I created an AutoTestCaseAttribute that inherited TestCaseAttribute, after many attempts to dynamically provide the arguments, I found that the property that I needed to set was an internal property, after some reading found out this was by design. #sadface

More reading, found TestCaseSourceAttribute which allowed you to define the parameters else where,

static int[] EvenNumbers = new int[] { 2, 4, 6, 8 };

[Test, TestCaseSource("EvenNumbers")]
public void TestMethod(int num)
{
    Assert.IsTrue( num % 2 == 0 );
}

This really looked like the way to go, but wasn’t meant to be. The arguments that you passed had to be statically defined and again extending the attribute had the same issue as TestCaseAttribute.

The conclusion was that I needed to write an NUnitAddin with a implementation of ITestCaseProvider2

public interface ITestCaseProvider
{
    bool HasTestCasesFor(MethodInfo method);
    IEnumerable GetTestCasesFor(MethodInfo method);
}

public interface ITestCaseProvider2 : ITestCaseProvider
{
    bool HasTestCasesFor( MethodInfo method, Test suite );
    IEnumerable GetTestCasesFor( MethodInfo method, Test suite );
}

Below is where the magic happens.

public class AutoTestCaseProvider : ITestCaseProvider2
{
    ...

    public bool HasTestCasesFor(MethodInfo method)
    {
        return Reflect.HasAttribute(method, AutoFixtureNUnitFramework.AutoTestCaseAttribute, false);
    }

    ...

    public IEnumerable GetTestCasesFor(MethodInfo method, Test parentSuite)
    {
        Type[] parameterTypes = method.GetParameters().Select(o => o.ParameterType).ToArray();

        ArrayList parameterList = new ArrayList();

        var attributes = Reflect.GetAttributes(method, AutoFixtureNUnitFramework.AutoTestCaseAttribute, false);

        foreach (TestCaseDataAttribute attr in attributes)
        {
            foreach (var arguments in attr.GetArguments(method, parameterTypes))
            {
                ParameterSet parms = new ParameterSet();
                parms.Arguments = arguments;

                parameterList.Add(parms);
            }
        }

        return parameterList;
    }

    ...
}

Using reflection the NUnit.Framework identifies any unit tests with the AutoTestCaseAttribute applied, and then calls the GetArguments method which in turn uses the configured Fixture to generate any arguments and passes them to the parameterised unit test.

public abstract class TestCaseDataAttribute : Attribute
{
    ...
 
    public abstract IEnumerable<object[]> GetArguments(MethodInfo method);
 
    ...
}
 
public class AutoTestCaseAttribute : TestCaseDataAttribute
{
    ...
     
    public override IEnumerable<object[]> GetArguments(MethodInfo method)
    {
        if (method == null)
        {
            throw new ArgumentNullException("method");
        }
 
        var specimens = new List<object>();
        foreach (var p in method.GetParameters())
        {
            CustomizeFixture(p);
 
            var specimen = Resolve(p);
            specimens.Add(specimen);
        }
 
        return new[] { specimens.ToArray() };
    }    
 
    ...
}

An example of how to use.

[TestFixture]
public class Tests
{
    [Test, AutoTestCase]
    public void IntroductoryTest(int expectedNumber, MyClass sut)
    {
        int result = sut.Echo(expectedNumber);
        Assert.Equal(expectedNumber, result);
    }
}

As I am writing this, I have submitted a pull request and awaiting a code review hopefully soon my code will be merged and set free for the world to use.

Thanks for reading this, and if anything needs more explaining or isn’t clear just comment below.

There are 10 kinds of people in the world those who can read binary and those who can't