Vagif Abilov's blog on .NET : TypeMock, Isolator

.NET mocking framework comparison

Vagif Abilov — Wed, 14 Oct 2009 14:03:21 GMT

A couple of good sources for those who need to decide:

Capability comparison made by Morgan Soley. Half year old but with a feature comparison table.

Recently updated comparison by Andrew Kazyrevich.

TypeMock Isolator is a clear winner if it fits in your budget. Otherwise Rhino Mocks or Moq seem to be the most mature frameworks. According to a recent Roy Osherove’s pole they are in the lead with Rhino Mocks having 47% and Moq on second place with 34%. If I had to choose a free mocking framework today, I’d probably go with Moq because of Rhino Mocks’ confusing “very large array of syntaxes”. Or wait for Rhino Mocks 4.0 that will be radically simplified and, like Isolator, will go for AAA abd single “fake” concept.

But for mocking without severe limitations there is no alternative to Isolator. Still in the lead.

ASP.NET Bundle from TypeMock

Vagif Abilov — Mon, 18 May 2009 16:28:50 GMT

TypeMock is has announced ASP.NET Bundle (consisting of Isolator and Ivonna). Being TypeMock user and customer, I am distributing their announcement. The text below is not mine – it’s written by TypeMock and explains how to get a free ASP.NET Bundle license (if you hurry up, like me :-))

Unit Testing ASP.NET? ASP.NET unit testing has never been this easy.

Typemock is launching a new product for ASP.NET developers – the ASP.NET Bundle - and for the launch will be giving out FREE licenses to bloggers and their readers.

The ASP.NET Bundle is the ultimate ASP.NET unit testing solution, and offers both Typemock Isolator, a unit test tool and Ivonna, the Isolator add-on for ASP.NET unit testing, for a bargain price.

Typemock Isolator is a leading .NET unit testing tool (C# and VB.NET) for many ‘hard to test’ technologies such as SharePoint, ASP.NET, MVC, WCF, WPF, Silverlight and more. Note that for unit testing Silverlight there is an open source Isolator add-on called SilverUnit.

The first 60 bloggers who will blog this text in their blog and tell us about it, will get a Free Isolator ASP.NET Bundle license (Typemock Isolator + Ivonna). If you post this in an ASP.NET dedicated blog, you'll get a license automatically (even if more than 60 submit) during the first week of this announcement.

Also 8 bloggers will get an additional 2 licenses (each) to give away to their readers / friends.

Go ahead, click the following link for more information on how to get your free license.

TypeMock Isolator and matching faked method’s arguments – part 3

Vagif Abilov — Sat, 14 Mar 2009 22:18:26 GMT

Part 1

Part 2

I promised to address matching reference and output arguments using lambda-based syntax that I proposed for TypeMock Isolator. Before getting there I would like to retract my original comment that faked method’s arguments should be matched by default. I still think that, like strong typing, strong matching is a best practice to follow. However I should not forget the main goal of my proposal: increase code simplicity and compactnes. My first examples used very simple method calls consisting of one-two arguments, so an extra effort of typing “(int x, int y)” went unnoticed. Using real API methods immediately showed how inconvenient it can become to faked methods with large number of arguments if developer is forced to list argument types even when no argument match is required. Here how it might look:

Isolate.WhenCalled((string a, Evidence b, byte[] c, AssemblyHashAlgorithm d) => 
        Assembly.LoadFrom(a, b, c, d)).WillReturn(null);

Note where it gets really terrible: you have to type the whole argument list of LoadFrom: “string a, Evidence b, byte[] c, AssemblyHashAlgorithm d”. All this without help from Intellisense! The argument list comes before the method name.

Now compare it with current use of TypeMock API:

Isolate.WhenCalled(() => Assembly.LoadFrom(null, null, null, 0)).WillReturn(null);

What would you choose? With all respect to strong matching the choice is clear.

Eli suggested defining a different method for faking a method a method when arguments matter, and now I came to the same conclusion. So I will leave WhenCalled alone and use a different method name, let it be called WhenCalledArgs (I am not sure about using word “Exact” in such method name, because this method will support partial match, and in case you only match one argument out of five, the word “exact” can be confusing).

Now back to agenda: matching reference and output arguments. In my previous post for each size of argument list I defined two overloads (I will be using new name WhenCalledArgs):

public static IPublicNonVoidMethodHandler WhenCalledArgs(Func func);
public static IPublicNonVoidMethodHandler WhenCalledArgs(Func func, Func predicate);

The definitions above cover the case of a method with single argument. The first overload corresponds to a call with no argument match and no checker. The second overload can be used to define a custom checker for an argument. What is missing here is support for reference and output arguments, when the passed value is assigned to an argument marked as “ref” or “out” on method return. Fortunately, this is easy to achive using the same notation. It just requires new oveloads.

To assign reference and output value we need to extend WhenCalledArgs with a new parameter: an Action<T> delegate:

public static IPublicNonVoidMethodHandler WhenCalledArgs(Func func, Action output);

And to enable specifying both custom checker and output delegate, we will have an overload that combines them all:

public static IPublicNonVoidMethodHandler WhenCalledArgs(Func func, Func predicate, Action output);

Similar overloads we will need to provide for void methods (in such case WhenCalledArgs will be returning IVoidActionHandler instead of IPublicNonVoidMethodHandler). Now let’s look at the test code. For the sake of simplicity we assume to be dealing with a static class called MyClass, and the method that we are going to fake has the following signature:

public static void LookupCustomer(int customerId, out string firstName, out string lastName);

First we will fake its call but assign on return the values “John” and Smith” to the firstName and lastName respectively. Here how the code will look:

Isolate.WhenCalledArgs((int x, string y, string z) => 
        MyClass.LookupCustomer(x, out y, out z), (x, y, z) => { y = "John"; z = "Smith"; }).IgnoreCall();

Note that the customerId argument will not be matched. If we want to match it we slightly change the code:

Isolate.WhenCalledArgs((string x, string y) => 
        MyClass.LookupCustomer(123, out x, out y), (x, y) => { x = "John"; y = "Smith"; }).IgnoreCall();

What happens now is that the method LookupCustomer will only be faked for customerId equals 123, and upon return firstName and lastName arguments will be assigned values “John” and “Smith”.

Using “ref” instead of “out” does not really change much. It just gives an opportunity to assign (and match) a value to an argument and in addition specify another one on return.

And of course we can combine assignment of ret/out values with custom argument checker. Here’s an example:

Isolate.WhenCalledArgs((int x, string y, string z) => 
        MyClass.LookupCustomer(x, out y, out z), (x, y, z) => x > 100, 
        (x, y, z) => { y = "John"; z = "Smith"; }).IgnoreCall();

This is quite compact notation, however it does various things:

It fakes calls to MyClass.LookupCustomer without argument match
Even though arguments are not matched, only calls with customerId greater than 100 will be faked, as speficied in custom checker predicate
On return output arguments firstName and lastName will be assigned values “John” and “Smith” respectively.

Conclusion

In these blogs posts I tried to demonstrate how we can take advantage of C# 3.0 language features and provide compact notation for TypeMock Isolator AAA syntax. I am pretty sure there is someting uncovered, and I haven’t even thought about how this approach can be applied to VB (I am afraid it can’t, at least not until Visual Basic gets action-based lambda expression support). However I believe this notation can be successfully applied to Isolator AAA syntax, as it is based on the same principles and can further improve compactness and readability of the code.

TypeMock Isolator and matching faked method’s arguments – part 2

Vagif Abilov — Sat, 14 Mar 2009 10:49:25 GMT

I was glad to see that my thoughts about figuring out the best syntax to control the match of faked method’s arguments haven’t been unnoticed by TypeMockians. Eli Lopian raised a few questions, and addressing them seems to be crucial for materialization of the whole idea.

1. Backward compatibility with existing tests. If WhenCalled(() => some_method(a, b, c)) will be interpreted as a call with exact argument match, existing tests will fail. Parameterless delegates in such calls will have to be replaced with parameterized ones, e.g. WhenCalled((x, y, z) => some_method(x, y, z)).

I was quick enough to respond that I would be willing to update all tests just to get the syntax right, but I admit such response is childish. You can’t force your customers to use many hours on fixing their tests just because you discovered great syntax. Of course you need to provide a compatibility mode. There are different ways to achieve that. The simplest is to use a global flag:

Isolate.DefaultArgumentMatchBehavior = ArgumentBehavior.ExactMatch;

Exact match can be used as default for backward compatibility, at least during transition period. Or argument match mode can be implemented as an attribute that can later be deprecated. I don’t think this looks very elegant (global flags can’t be elegant), this is more like a temporary solution to let tests survive until the next major TypeMock release.

If this sounds too messy, then alternative approach might be to place functionality based on lambda syntax in a new namespace. Perhaps this is cleaner. I can compare it with approach taken by LINQ developers that divided LINQ classes between two namespaces and placed Expression and its derivations in System.Linq.Expressions. Later in this post I will show new extensions that solve custom checkers, and the more such extensions come, the more it is justified to define a dedicated namespace for lambda-based argument match control. What can work best, I don’t really know. Anyway, I believe resolving backward compatibility is more like configuration issue that should not compromise syntax selection.

What I find however important is to let a developer set up the tests so the arguments would be matched by default. Already now indexers are matched, this change broke many of our tests (I like defining custom indexers), but I think this was a right thing. When mock framework encounters ar[1], I believe it should respect the index value unless tests are specifically configured to ignore it. The same behavior I expect from faked methods: mock framework should not ignore that a method DoSomething(a, b, c) is expected to be faked with arguments (a, b, c). I do agree that this may not be turned on by default for backward compatibility purpose, but I would like to have a way to turn it on – at least to unifier indexers’ and methods’ behaviour.

2. Support for custom checkers. This is perhaps the most important requirement. If we find binary approach to argument matchings (match/nomatch) not sufficient because in some cases we expect partial argument match, we should also expect support for matching with custom checkers, for example when a string value begins with certain pattern, numerical value is within a certain range etc. I agree that without providing an easy way to define custom checkers the idea that I described in my previous post will stay a purely academical excersise. So I started playing with example code.

MyIsolate.WhenCalled((int x, int y) => MyClass.GetSum(x, y)).WillReturn(6);

So how we can extend this code to support custom argument checkers, for example to fake a method only if “x” is greater than 0? Something like this:

MyIsolate.WhenCalled((int x, int y) => MyClass.GetSum(x, y) where x > 0).WillReturn(6);

Of course this code will never compile, but I admit I imported System.Linq namespace and spent some time shuffling “where” keyword around. Not to resolve it using LINQ – just to get some inspiration. Making mock framework depedent on LINQ would be really mad (and would never work), not to say that both “where” clause is just a syntactic sugar around Where method that requires support for IEnumerable. And what IEnumerable can have with argument matching?

However turning inspecting implementation of “where” paid of, because the its core part is a predicate delegate Func<T, bool>. This is what we need to extend our WhenCalled with support for custom checkers.

Remember our new WhenCalled overloads listed in the previous post:

public static IPublicNonVoidMethodHandler WhenCalled(Func func)
public static IPublicNonVoidMethodHandler WhenCalled(Func func); 
public static IPublicNonVoidMethodHandler WhenCalled(Func func);
…

If for each parameterized Func delegate we provide an additional overload that takes a predicate delegate, then we enable custom checkers.

public static IPublicNonVoidMethodHandler WhenCalled(Func func);
public static IPublicNonVoidMethodHandler WhenCalled(Func func)
public static IPublicNonVoidMethodHandler WhenCalled(Func func, Func predicate)
public static IPublicNonVoidMethodHandler WhenCalled(Func func)
public static IPublicNonVoidMethodHandler WhenCalled(Func func, Func predicate)
…

Now look at that tests we can write now:

[Test, Isolated]
public void TestWithCustomCheckers()
{
    Isolate.WhenCalled((int x, int y) => MyClass.GetSum(x, y), (x, y) => x > 0).WillReturn(6);
    Assert.AreEqual(6, MyClass.GetSum(1, 2));
    Assert.AreEqual(1, MyClass.GetSum(-1, 2));

    Isolate.WhenCalled((int x, int y) => MyClass.GetSum(x, y), (x, y) => x > 10 && y > 10).WillReturn(6);
    Assert.AreEqual(6, MyClass.GetSum(20, 20));
    Assert.AreEqual(10, MyClass.GetSum(5, 5));
 
    Isolate.WhenCalled((int y) => MyClass.GetSum(1, y), y => y == 3).WillReturn(7);
    Assert.AreEqual(7, MyClass.GetSum(1, 3));
    Assert.AreEqual(3, MyClass.GetSum(1, 2));
 
    Isolate.WhenCalled((int x) => MyClass.Elements[x], x => x == 10).WillReturn(9);
    Assert.AreEqual(9, MyClass.Elements[10]);
 
    Isolate.WhenCalled((string x, string y) => MyClass.Concatenate(x, y), (x, y) => x.StartsWith("abc") && y.EndsWith("xyz")).WillReturn("bingo!");
    Assert.AreEqual("bingo!", MyClass.Concatenate("abc", "xyz"));
    Assert.AreEqual("ab", MyClass.Concatenate("a", "b"));
}

What do we gain with this syntax?

1. I think it is more readable. In fact, I think it is much more readable. Compare the line 19 with something like Isolate.WhenCalled(MyClass.Concatenate(null, null)).AndArgumentsMatch(1, Arg.StartsWith("abc")).AndArgumentsMatch(2, Arg.EndsWith("xyz")).WillReturn("bingo!").

2. I believe when we extend functionality, we should prefer using built-in language constructions over new methods definition. The advanage of Func<T, bool> delegate is that it is based on generics and if we use it to define predicates then the predicates will naturally support methods and properties of respective types. You can simply write “x == 10” or “x.StartsWith(“abc”)”.

3. You can take advantage of boolean logic using built-in language operators. Writing “x > 0 && y > 0” is equally simple as writing “x > 0 || y > 0”, but how would you specify the latter checker using method-based notation? AndArgumentsMatch(1, Arg.GreaterThan(0)).OrArgumentsMatch(2, Arg.GreaterThan(0)?

There is one unanswered question from Eli: handing “ref” and “out” arguments. I need to think how this can be resolved.

How I would match faked method's arguments if I was TypeMock Isolator

Vagif Abilov — Thu, 12 Mar 2009 20:43:00 GMT

I enjoy compactness and clarity of TypeMock Isolator AAA syntax. But it still lacks certain features in comparison with the original RecordExpectation-based syntax. One of such features is support for exact match of method's attributes. Here's an example (I assume that MyClass is a static class):

Isolate.WhenCalled(() => MyClass.GetSum(1, 2)).WillReturn(10);
Isolate.WhenCalled(() => MyClass.GetSum(3, 4)).WillReturn(20);

At the time of writing (TypeMock Isolator version 5.2.2) this will not trigger return value of 10 on all calls to GetSum with arguments 1 and 2. Argument values are currently ignored. However, recently Isolator was enhanced with exact argument match for indexers:

Isolate.WhenCalled(() => MyClass.Elements[1]).WillReturn(10);
Isolate.WhenCalled(() => MyClass.Elements[2]).WillReturn(20);

In the case above any subsequent call to MyClass.Elements with an index value of 2 will return 20.

Needless to say that support for exact argument match is one of the most requested features for AAA syntax. And the wait is soon over - there will be new methods that will make it possible to require exact argument matching. But is it really necessary to define new methods for this purpose? And what about partial argument match? I think there is another way to go.

Before I move on to my proposal, let me express first what does not make me fully satisfied with current TypeMock approach.

1. Indexers' and methods' arguments should not be handled differently.

I understand what was the motivation for exact match of arguments on indexers. Indexers are mostly used with arrays, so when you set different expectations on ar[1] and ar[2], you will most likely want Isolator to map indexes to corresponding values and always return values depending on the passed index. The problem however is that C# makes it very easy to implement indexers as alternatives for getters and setters. So calls MyClass.GetValue(1) and MyClass.Values[1] can mean the same, but when you set expectations on GetValue and Values, the first one will ignore the argument value, while the second one will memorize it.

2. All or nothing is not enough.

Alright, we are soon getting support for exact match of arguments. All arguments. But will it cover all needs? If you can come up with scenario when you need to fake a method with exact argument match, and you can come up with scenario when you need to fake a method with no argument match, then why can't you have a scenario when you need to match just certain method arguments? I can clearly see some typical scenarios: for example, you may fake a method that receives as part of its input current time, generated GUIDs etc. We can of course ask if this is a good practice to make tests dependent on non-deterministic data, but don't forget that Isolator can be successfully used not just for pure unit tests. Smoke tests and even integration tests can also be simplified by isolating some aspects, and once there is support for both exact match and no match of method arguments, there is nothing wrong with requesting a partial match.

But how can it be provided? Extend ExactMatch or whatever the new method will be called with an overload that would take an array of argument ordinals? Cumbersome and hard to read and maintain.

Let's use lambda expressions at its full strength!

One thing that makes TypeMock Isolator AAA syntax charming (although if I am not mistaken it was Moq that introduced it first) is its use of lambda expressions:

Isolate(() => SomeCall(...)).WillReturn(...);

First time I saw this syntax, I spent some time trying to understand what I can use instead of "()" to the left from "=>". Maybe depending on the faked method's signature I could sometimes write (x)? Or even (x,y)? Then I realized that this was not an option: static class Isolate supports two overloads of WhenCalled:

public static IVoidActionHandler WhenCalled(Action action); 
public static IPublicNonVoidMethodHandler<TResult> WhenCalled<TResult>(Func<TResult> func)

So no matter whether you fake a void method or a method that returns some value, in both cases you pass a parameterless delegate. You can't have unbound variables: all method's arguments must be resolved.

But what if WhenCalled supported overload for each form of Action and Func? Like this:

public static IPublicNonVoidMethodHandler WhenCalled(Func func) {...}
public static IPublicNonVoidMethodHandler WhenCalled(Func func)  {...}
public static IPublicNonVoidMethodHandler WhenCalled(Func func) {...} 
public static IPublicNonVoidMethodHandler WhenCalled(Func func) {...} 
public static IPublicNonVoidMethodHandler WhenCalled(Func func) {...}

This would open quite interesting way of setting expectations with exact match of only selected arguments! Because then we could write Isolate.WhenCalled(() => ..., Isolate.WhenCalled((x) => ..., Isolate.WhenCalled((x, y) => .... And use these variables in a method call to indicate the unspecified arguments that will not be matched exactly.

Let me illustrate this with an example. Let's say we have a static class MyClass with a method GetSum(int x, int y). In TypeMock Isolator the following code will work fine:

Isolate.WhenCalled(() => MyClass.GetSum(1, 2)).WillReturn(4);
Assert.AreEqual(4, MyClass.GetSum(1, 2));
Assert.AreEqual(4, MyClass.GetSum(10, 20));

The code above uses currently supported syntax, except that I believe it's semantics should be identical to the one used by indexers: method arguments should be matched exactly.

And here how we can disregard argument values:

Isolate.WhenCalled((int x, int y) => MyClass.GetSum(x, y)).WillReturn(6);
Assert.AreEqual(6, MyClass.GetSum(1, 2));
Assert.AreEqual(6, MyClass.GetSum(2, 1));

In the code above it does not matter what arguments are passed to GetSum: we used a new overload of WhenCalled (unfortunately only supported in this blog entry) that let us speficy what arguments should be treated as unbound.

And what about partial argument match? Now it's easy:

Isolate.WhenCalled((int y) => MyClass.GetSum(1, y)).WillReturn(7);
Assert.AreEqual(7, MyClass.GetSum(1, 2));
Assert.AreEqual(4, MyClass.GetSum(2, 2));

Here Isolator is instructed to set expectations only on calls with first argument equal to 1. So GetSum(2,2) will not be faked.

Finally setting expectations on an indexer:

Isolate.WhenCalled(() => MyClass.Elements[0]).WillReturn(8);
Assert.AreEqual(8, MyClass.Elements[0]);
Assert.AreNotEqual(8, MyClass.Elements[123]);
Isolate.WhenCalled((int x) => MyClass.Elements[x]).WillReturn(9);
Assert.AreEqual(9, MyClass.Elements[123]);

What do we gain by this?

1. Consistent semantics for argument match on indexers and methods.

2. Support for partial argument match.

3. No need to introduce new method names - just overloads to WhenCalled.

4. And isn't such use of labmda expressions cool?

To demonstrate the proposed syntax, I wrote a simple example with unit tests that include the code I showed above. Of course I could not use real TypeMock Isolator in these tests: it does not support (yet!) proposed WhenCalled overloads. So I cheated a little and implemented a toy MyIsolate class with just enough overloaded methods to make these example work. The test project can be downloaded from this post. You can compile and run the tests - they will pass as if they were real. I wish they were.

Part 2

Part 3