I've been playing around with the performance test features of VS2005 and as I said in previous post: I really like it!
Some handy things that I figured out so far:
In VS2005 you can create webtests to test the pages of your website. A nice feature is that you can parameterize your test with context parameters. When you place you webtests in a load test you can add context parameters to your load test. If the parameters have the same names as in the webtest they overrule the values in the webtest. This is very useful, for instance when you want to change the host name for all your webtests.
You can create a plugin for your webtest by creating a class that derives from Microsoft.VisualStudio.TestTools.WebTesting.WebTestPlugin. This class has PreWebTest and PostWebTest events which you can use to set the parameters for your webtest. You can do the same trick for individual requests (WebRequestPlugin) and for load test (ILoadTestPlugin).
When you add a data source to a webtest, you can only select tables from that data source, there are no views available. To solve this, just edit the xml from the webtest and replace the table name with the name of the view you want to use. Works without a problem.
When you have added a data source to a webtest, you cannot easily change the connection string. It's greyed out in the properties box and for some reason you can't reach it through code. You can work around this by creating an UDL file for your data source. When you add the data source click on the "Advanced" button and add the UDL-file. When you use the UDL file in the data sources of all your webtests you can change them all at once by simply changing the connection in the file.
That's all so far, but more will follow.
Until recently we were using Microsoft's Application Center Test for performance testing our ASP.NET web appliction. Because our appliction is mostly about flexible workflows and we're using guids as identifiers we had to do a lot of VB-scripting (arf!) to get some reusable tests.
Last week I was assigned to do the performance testing for the new release. Because of all the scripting, the structure of testscripts was a mess. So I decided to do some cleaning up. The first thing I did was loading the scripts in VS2003 in stead of ACT. This helped a little because it gave me some colour to the code (in ACT you're code is in a downgraded version of notepad). But debugging the code to try to understand it was still not possible. My next bright idea was to use the ACT type library and write the test in C#. By using the .NET->COM interfacing I was able to call my C# assembly from ACT.
Running the tests appeared to be working very wel because no HTTP-errors were thrown. But looking more closely revealed that a lot of 302 codes were received. This means that the "browser" is redirected to another page. By debugging the responses I found out that most of the requests were redirected to either our logon or error page. The 302's were noticed in previous tests but ignored because they're warnings and not errors.....
So much for this performance test. We are using forms authentication in our application and it turned out that for some reason this did not work very well with ACT. I tried to get it to work, but it refused to do so.
So I was up to my third bright idea of the day (just an avarage day for me ;-): A Visual Studio 2005 Test Project. After some fiddling around I recorded my first webtest and this is sweeeeet! In IE you get an extra pane with recording buttons and you can follow the requests being made. After recording you get the results in a treeview. The viewstate is automatically handled by Extraction Rules and this is working very well. So it was time to play back the recording. And this is another major improvement: you can see the result of each request after the test has finished. You can see the browser result, the request, the response and validations in a tabbed window. Great stuff, if something is going wrong you can now actually see what the problem is.
The next step was parameterizing the host name and making the test data driven. You can create context parameters for the host name and add one or more data sources. I connected the test to our application's user table and was able to simulate different users in the test.
It's very easy to customize the test further with your own extraction rules, validations, parameters etc. You can even write you're own test and request plug-ins to capture events from the test. And for those who want full control: you can also convert the test to code.
When you're webtest is complete you can put it into a load test. A load test can contain multiple webtests and you can add percentages to them to balance your test. You can also adjust the simulated browser and network mix, set up a load pattern and your own set of performance counters. From this point it's a mather of running the load test and analyze the results. You get a very comprehensive set of data which can be stored in a SQL database. One thing that I find very usefull is the possibility to define sets of related requests in the webtest (transactions). Data about these transactions is recorded during the load test. This makes it possible to see how long it takes for a user to perform certain tasks.
The test suite in VS2005 is fabulous and although some features are still missing (please give me nested webtests!) it's going to make our lives a lot easier. Because this stuff is new, there is not a lot of information available on the web. The site that helped me a lot was this one:
http://lab.msdn.microsoft.com/teamsystem/teamcenters/tester/default.aspx
and the related forums:
http://forums.microsoft.com/msdn/ShowForum.aspx?ForumID=19