Performance Testing in VS2005
Until recently we were using Microsoft's Application Center Test for performance testing our ASP.NET web appliction. Because our appliction is mostly about flexible workflows and we're using guids as identifiers we had to do a lot of VB-scripting (arf!) to get some reusable tests.
Last week I was assigned to do the performance testing for the new release. Because of all the scripting, the structure of testscripts was a mess. So I decided to do some cleaning up. The first thing I did was loading the scripts in VS2003 in stead of ACT. This helped a little because it gave me some colour to the code (in ACT you're code is in a downgraded version of notepad). But debugging the code to try to understand it was still not possible. My next bright idea was to use the ACT type library and write the test in C#. By using the .NET->COM interfacing I was able to call my C# assembly from ACT.
Running the tests appeared to be working very wel because no HTTP-errors were thrown. But looking more closely revealed that a lot of 302 codes were received. This means that the "browser" is redirected to another page. By debugging the responses I found out that most of the requests were redirected to either our logon or error page. The 302's were noticed in previous tests but ignored because they're warnings and not errors.....
So much for this performance test. We are using forms authentication in our application and it turned out that for some reason this did not work very well with ACT. I tried to get it to work, but it refused to do so.
So I was up to my third bright idea of the day (just an avarage day for me ;-): A Visual Studio 2005 Test Project. After some fiddling around I recorded my first webtest and this is sweeeeet! In IE you get an extra pane with recording buttons and you can follow the requests being made. After recording you get the results in a treeview. The viewstate is automatically handled by Extraction Rules and this is working very well. So it was time to play back the recording. And this is another major improvement: you can see the result of each request after the test has finished. You can see the browser result, the request, the response and validations in a tabbed window. Great stuff, if something is going wrong you can now actually see what the problem is.
The next step was parameterizing the host name and making the test data driven. You can create context parameters for the host name and add one or more data sources. I connected the test to our application's user table and was able to simulate different users in the test.
It's very easy to customize the test further with your own extraction rules, validations, parameters etc. You can even write you're own test and request plug-ins to capture events from the test. And for those who want full control: you can also convert the test to code.
When you're webtest is complete you can put it into a load test. A load test can contain multiple webtests and you can add percentages to them to balance your test. You can also adjust the simulated browser and network mix, set up a load pattern and your own set of performance counters. From this point it's a mather of running the load test and analyze the results. You get a very comprehensive set of data which can be stored in a SQL database. One thing that I find very usefull is the possibility to define sets of related requests in the webtest (transactions). Data about these transactions is recorded during the load test. This makes it possible to see how long it takes for a user to perform certain tasks.
The test suite in VS2005 is fabulous and although some features are still missing (please give me nested webtests!) it's going to make our lives a lot easier. Because this stuff is new, there is not a lot of information available on the web. The site that helped me a lot was this one:
http://lab.msdn.microsoft.com/teamsystem/teamcenters/tester/default.aspx
and the related forums:
http://forums.microsoft.com/msdn/ShowForum.aspx?ForumID=19