December 2006 - Posts

Design for Testability, NMock and CodeCoverage

In one of my previous posts I showed some code calling the Team Foundation Server. During test runs I had a reference to our production TFS box. You can imagine that carefull testing was the first law to obey, hence the built-in option 'Emulate' to not really destroy anything.

For Unit testing this approach was somewhat cumbersome so I needed a better approach. My first class model looked like this:

In this model I have two dependencies on two external types in the assembly Microsoft.TeamFoundation.Build.Proxy. During Unit testing I want to call an instance of those classes that have the same signature. NMock provide such functionality. It simulates an actual class. To instantiate an Mock class you need to provide it an interface. Time for some refactoring.

As you can see the Action classes now depend on the new Interface ITeamFoundationServer. The  class TeamFoundationServer implements that interface and has dependencies on the  TFS classes.   The Action classes obtain an instance to an implementation of ITeamFoundationServer by calling the static property Instance on the TeamFoundationServer class. So this class also acts as a Factory.

Mock

When we are running a unit test we want to provide our own implementation of the ITeamFoundationServer interface. For this to happen we need a form of dependency injection. As I do not want to introduce a complete, full blown dependency injection framework (I could have used ObjectBuilder), I simply provide a dependency injection constructor on the TeamFoundationServer class. The constructor accepts an ITeamFoundationServer implementation and sets the corresponding private static field _Instance. The default constructor is private which prevents instantiating the TeamFoundationServer class from non-testing code.

After you added a reference to the NMock assembly you can start creating Mocks in your test method. First you start off with creating a Mockery object and then you're ready for creating a mock object for the interface. The instance returned from that call is injected in our code by the means of the dependency constructor as shown in the following snippet:

Mockery mock = new Mockery();
ITeamFoundationServer itfsMock = (ITeamFoundationServer) mock.NewMock(typeof(ITeamFoundationServer));
TeamFoundationServer tfsDummy = new TeamFoundationServer(itfsMock);

The next step is to define what call will be expected on the interface and which return values should be delivered in that case. For example:

Expect.Once.On(itfsMock).Method("GetListOfBuilds").With(new object[] {"par1", "par2"}).Will(Return.Value(TestData()));

This tells the NMock framework to expect one call to GetListOfBuilds with two string parameters. That call will return testdata. The testdata is choosen so that it exercises different execution paths to maximize codecoverage. Now you're ready to  make the call to your class/classes under test. As the call returns one final call to the mock framework will check that all of our expected calls have been taking place.

mock.VerifyAllExpectationsHaveBeenMet();

We now have a unit test that can safely run without the need for an up-and-running teamfoundationserver. So everybody can relax...

CodeCoverage

The best thing of the mock object is the flexibility in accepting and returning data for testcases that would require running complex database scripts or infrastructure. You can even throw exceptions. Having said that there is no reason for a codecoverage of less than 100%.

If you have no ambition you're close to death. It is however hard to reach 100%. In my case the real implementation was also taken into account for the coverage and was off-course not covered by any test. In the first run only 60% of the code was covered. By adding more testcases (and richer testdata), refactor out common code and having the mock framework throw excpetions I achieved over 80%.

To reach the 90% I needed some trickery. Inspection of the non-coveraged code blocks revealed that some private constructors were not called. With some reflection magic that can be solved:

Type tfsType = typeof(TeamFoundationServer);
ConstructorInfo[] call = tfsType.GetConstructors(BindingFlags.NonPublic | BindingFlags.CreateInstance | BindingFlags.Instance);
object tfsInstance = call[0].Invoke(null);

Also the Dispose method was not fully covered because some internal fields where never set. Also for this CodeCoverageCase some reflection magic was needed. This is how you achieve that. Pay attention to the fact that this field is private so it can not be set from the testcode.

tfsInstance.InvokeMember("_Instance",
  BindingFlags.Instance | BindingFlags.SetField | BindingFlags.NonPublic,
  null,
  tfsInstance,
  new object[] { tfsInstance });

Calling Dispose now did exercise all code paths.

Click to enlarge 

Conclusion

If you're serious about Unit testing you need to design for testability. A good test design enables the Unit test to insert Mock objects without cluttering you're code with test artifacts. Using Mock objects gives you the ability to test even complex sceanrio's relatively simple. Using Code coverage to measure the effectiveness of your unit tests is a best practise. To achieve top code coverage ratings you might need some magic.

EntLib 3.0 CTP Validation block

The first CTP release of the Enterprise Library, which is available for download from CodePlex, contains the first release of the long awaited Validation block.

Tom already gave some insight in what could be expected. Today I downloaded the bits and investigated the current implementation.

First of all: the help documentation is not ready and the EntLib Config application does not handle the validation block yet.

I will demo here two of the posible validation scenario's. First of all the validation based on Attributes. You simply take the data class and decorate the properties or fields with the validation attributes.

public class Customer
{
  [StringLengthValidator(2,15)]
  public string Name;

  [TodayValidator()]
  public DateTime BirthDate;
}

On the Name field we see StringLengthValidator that will validate if a Name has a length of minimum 2 and maximum 15.

On the BirthDate field I created a Validator myself. It validates if the Birthdate is less than the current date. Creating a custom validator is simple by just inheriting ValidatorAttribute:

public class TodayValidator : ValidatorAttribute
{
  public override IValidator CreateValidator() 
  {
    return new RangeValidator<DateTime>(DateTime.Now);
  }
}

To check if the defined validations are met the following code is used:

ValidationResults attributedValidation = Validation.Validate<Customer>(customer);

foreach (ValidationResult result in attributedValidation)
{
  Console.WriteLine("{0}:{1}", result.Key, result.Message);
}

The ValidationResults class has also an IsValid property which is set accordingly.

One of the nice things is the posibility to define the validation rules in a configurationsource, for example the app.config file.

The following config defines two rulesets and the stringlength validator on the name attribute of the Customer class:

<configSections>
  <section name="validation"   type="Microsoft.Practices.EnterpriseLibrary.Validation.Configuration.ValidationSettings, Microsoft.Practices.EnterpriseLibrary.Validation" />
</configSections>
<validation>
  <type name="Entlib3Ctp.Customer" defaultRule="relaxed">
    <rule name="relaxed">
      <fields>
        <field name="Name">
          <add name="NameMaxLength" type="Microsoft.Practices.EnterpriseLibrary.Validation.Validators.StringLengthValidator, Microsoft.Practices.EnterpriseLibrary.Validation" upperBound="15" upperBoundType="Inclusive"/>
        </field>
     </fields>
   </rule>
   <rule name="strong">
     <fields>
        <field name="Name">
         <add name="NameMaxLength" type="Microsoft.Practices.EnterpriseLibrary.Validation.Validators.StringLengthValidator, Microsoft.Practices.EnterpriseLibrary.Validation" upperBound="10" upperBoundType="Inclusive"/>
        </field>
      </fields>
    </rule>
  </type>
</validation>

In the validaton config first define the Type for which the rules are to be used. Then you define one or more rules. Within each rule you configurate the field  the validators  should act on.

By using a name for the ruleset you can switch in code between relaxed and a more tight validation. The relaxed validation can be used for example if the user needs to store the data temporary.

To choose which ruleset to use call the ValidationFactory:

IValidator<Customer> strongRules = ValidationFactory.CreateValidatorFromConfiguration<Customer>("strong");

In this case the "strong" rules are loaded and therefore the Name field of the Customer object should be no more than 10 characters.

I do not want to give any indications on performance of the validation logic but I will share some of my observations. First of all the call to Validation.Validate will create the validators based on the configuration source AND attributes in each call. If you need to validate the same object type multiple times create the validators only once by calling the appropiate ValidationFactory method. In the current CTP version creating the validators based on attributes takes approximately twice the time than creation of the validators defined in the app.config. If you know you're ruleset is in the config file use the methods ...FromConfiguration to prevent reflecting over your type.

The first look at the Validation block from the Enterprise Library 3.0 looks very promising. It will be a good fit for most of the projects.

In the attached zip you will find a small demo project.

Posted by Rene Schrieken | 3 comment(s)
Filed under:

Delete builds during the build

Now that is an intriguing title. But if you are using Team Foundation Server and especially Team Build you know where I'm talking about.

If you use the team build as the heart beat of your project and therefore build daily (or nightly if you're in the java camp) you end up with a large list  of Builds. Most of them are probably failed (just kidding), some of them have a build quality of unexamined and most of them are so old you probalby never want to touch or install any of it.

So you want to get rid of the old builds. Luckily you can. Microsoft provided us with a nice command line utility called tfsbuild which enable you to remove completed builds. Just provide a projectname and a buildnumber. Repeat 400 times until all unwanted builds are deleted.

There must be an easier way. I already found this solution but in our development shop I'm not the owner of the buildbox. Handing over the control of which builds to delete to an uncontrolled service didn't appealed to me much.

So reinvent the wheel to come up with a for me more suitable solution. Create a msbuild Task that deletes builds based on criteria that are provided in the build script. In the download you'll find the source (so you can see that there is no virus in it) of the build task that enables you to delete builds from the BuildStore. If you place the compiled assembly in the same source control folder where tfsbuild.proj resides you can start configuring.

In your tfsbuild.proj script do this:

<UsingTask AssemblyFile="DeleteBuildEasy.dll" TaskName="DeleteBuilds" />

<Target Name="AfterEndToEndIteration">
<DeleteBuilds 
   TeamServer="[your tfs server]" <!-- tfs:8080 -->
   TeamProject="$(TeamProject)"   
   BuildType="$(BuildType)" 
   Status="Failed"      
   Quality="Unexamined"

   Days="20" />
</Target>

Check-in all your changes, kick-off your build and you'll see the builds disappear as snow for the sun. Having the Days as selection mechanism quarantees that you do not have to modify the script day after day. If you come up with a good set of thresholds you can leave it running without any aftercare.

Best of all is that not only you're builds from the BuildStore will disappaer but also the files on you're buildserver will be removed, so it really cleans up.

This attached source code shows the basic mechanism. I have a more advanced version available. That one will also be able to run as service (as mentioned earlier) and has a more sophisticated query mechanism. Just contact me if you're intereseted in that version.

[UPDATE]

When you use this task on a TFS2008 based server keep in mind that this task DOESN'T honor the KeepForever=true property. As a matter of fact it will happily remove the build.

Source code is here