Search results matching tag 'performance'

Don’t use Activator.CreateInstance or ConstructorInfo.Invoke, use compiled lambda expressions

VagifAbilov — Fri, 02 Apr 2010 15:43:00 GMT

For a long time I’ve been under impression that rule engine that comes with Microsoft Windows Workflow Foundation is slow, very slow. We used it to execute some of our business rules, and soon found out that rule processing slows down application execution. What was strange is that the rules were really simple and it was hard to believe that an industry-strength rule engine uses so long time on processing them.

Eventually I ran a profiler, and it immediately showed where the time was spent. In fact it was not the actual rule validation but preparation for it: instantiation of the WF rule parser. The parser class is not public, so we used a trick to obtain its instance: retrieved its non-public constructor information via reflection and then called Invoke.

This was sloooow. So slow that we noticed it in our integration tests. What made it noticeable is that after we converted some of our business rules to use WF rule engine, we were encouraged by results and move more rules to use the same techinque. In the end rule validation was called many times during a single business operation, and performance deteriorated.

My first approach to this issue was to cache validated rules, and it worked. The total time to execute integration tests was reduced by 50%. However, I was not quite happy with the situation: we have various places where objects are instantiated using reflection, caching is not always possible and in general complicates component design. Recently I reviewed the code of StructureMap (I like reading good code), and remembered that it no longer used reflection and instead instantiated dependencies using compiled lambda expressions.

Roger Alsing in this post presented a generic method that can be used as a replacement for constructor method invocation. The essence of the method is in it’s last three lines:

// Make a NewExpression that calls the ctor with the args we just created
NewExpression newExp = Expression.New(ctor, argsExp);                  

// Create a lambda with the New expression as body and our param object[] as arg
LambdaExpression lambda = Expression.Lambda(typeof(ObjectActivator), newExp, param);              

// Compile it
ObjectActivator compiled = (ObjectActivator)lambda.Compile();

Although lambda expressions are part of LINQ (and you have to import System.Linq.Expressions namespace to manage them), as we can see from this example, they can be very useful to solve core language tasks, such as object instantiation.

But what is the gain? The gain is huge bringing the performance close to the speed of native IL code. I wrote a few tests to measure performance of object creation in different scenarios: by calling ‘new’ constructor, Activator.CreateInstance, ConstructorInfo.Invoke and using compiled lambda expression. Here are the results:

DefaultConstructor_Activator: (0,20 ms per 1000 calls)
DefaultConstructor_CompiledExpression: (0,04 ms per 1000 calls)
DefaultConstructor_Invoke: (1,07 ms per 1000 calls)
DefaultConstructor_New: (0,02 ms per 1000 calls)
DefaultConstructor_NotCompiledExpression: (169,00 ms per 1000 calls)
NonDefaultConstructor_Activator: (3,39 ms per 1000 calls)
NonDefaultConstructor_CompiledExpression: (0,07 ms per 1000 calls)
NonDefaultConstructor_Invoke: (1,57 ms per 1000 calls)
NonDefaultConstructor_New: (0,02 ms per 1000 calls)
NonDefaultConstructor_NotCompiledExpression: (293,00 ms per 1000 calls)

Results says it all, especially when arranged in a graph.

I removed from the graph the results for not compiled lambda expressions, because they make other figures insignificant. But it is important to remember that compilation should be performed only once, so the code should be carefully reviewed to avoid occasional recompilcation of lambdas.

P.S. The title of this blog post may look provocative, and of course I don’t really mean somebody should completely stop using Activator.CreateInstance. Compilation of labmda expressions is expensive enough to limit it’s practical use. But there are certain use cases (like IoC frameworks) when it gives clear performance advantage.

Speeding up startup time for applications that use NHibernate

Ramon Smits — Tue, 30 Mar 2010 20:57:00 GMT

I just read a very cool NHibernate trick to let your application start faster that was mentioned by Ricardo Peres:

Configuration cfg = new Configuration().Configure();
IFormatter serializer = new BinaryFormatter();

using (Stream stream = File.OpenWrite("Configuration.serialized"))
{
   serializer.Serialize(stream, cfg);
}

...

using (Stream stream = File.OpenRead("Configuration.serialized"))
{
   cfg = serializer.Deserialize(stream) as Configuration;
}

Using the CrossListQueryInfo and CrossListQueryCache

Bas — Fri, 27 Mar 2009 09:56:00 GMT

Lately, I am relatively a lot working with the CrossListQueryInfo. This is a way to query multiple lists at once, crossing multiple SPWebs, if you want to. And, the main reason why I used it, it's possible to use the cached lists of the sitecollection, to improve performance! The first steps with this CrossListQueryInfo can be quite frustrating, as there isnt too much documentation to find on.

I will start with a code sample, to show how to use this to query multuple webs to retrieve Items with the contentType News Item OR Location News Item from the publishing pages. When you are going to the CrossListQueryInfo, it´s smart to make use of U2U´s Caml Query Builder, to quickly build up thecaml that you want use.
Things to keep in mind while using the CrossListQueryInfo:

The results that are returned are ONLY PUBLISHED items. I was breaking my head around this one, because I was sure that my query was good, the CAML Query builder also returned a result, but the CrossListQueryCache didn't. After publishing, all the results showed up properly
ViewFields: Insert the fields that you want to see. If there is a field inside that doesnt exist in the list that you query, your result will be nill, nada, nothing. Make sure that you put in the INTERNAL field names!
RowLimit: When filling in value '0', you won't see any results
When using the CrossListQueryCache , make sure not to use the GetSiteData that uses the SPWeb param: GetSiteData(SPWeb web) and GetSiteData(SPWeb web, SPSiteDataQuery query) DO NOT use caching!!
and make sure to use Microsoft.SharePoint.WebPartPages.WebPart! This WebPart inherits from System.Web.UI.WebControls.WebParts.WebPart and enhances it with some extra functionality for connected webparts, client-side programming and data caching!! Thanks to Waldek Mastykarz for sharing this info.

Below is an example of a query that queries all the Publishing pages libraries that resides in the current SPWeb and all childWebs.

private DataTable ExampleQuery()
        {
            clqi = new CrossListQueryInfo();
 
            // Insert the list types that you want to use. In this case, its the publishing page library (850, see code below)
            clqi.Lists = "<Lists ServerTemplate=\"" + (int)ListServerTemplateCodes.PageLibrary + "\" />"; 
 
            // Insert the fields that you want to see. If there is a field inside that doesnt exist in the list that you query, your result will be nill, nada, nothing.
            // Make sure that you put in the INTERNAL field names!
            clqi.ViewFields = "<FieldRef Name=\"Title\" /><FieldRef Name=\"FileLeafRef\" /><FieldRef Name=\"Nieuws_x0020_Type\" /><FieldRef Name=\"Nieuws_x0020_Leverancier\" /><FieldRef Name=\"Uitgelicht\" /><FieldRef Name=\"Created\" /><FieldRef Name=\"Comments\" />";
 
            // scop to use. Another possibility is SiteCollection
            clqi.Webs = "<Webs Scope=\"Recursive\" />";
 
            // turn the cache on
            clqi.UseCache = true;
 
            // if row limit == 0, you will get 0 results
            clqi.RowLimit = 100;
 
            // I know a stringbuilder would be better, but i wanted to show the markup of the query
            clqi.Query = "<OrderBy>" + 
                            "<FieldRef Name='Title' />" +
                        "</OrderBy>" +
                        "<Where>" +
                            "<Or>" +
                                "<Eq>" +
                                    "<FieldRef Name='ContentType' />" +
                                    "<Value Type='Text'>News Item</Value>" +
                                "</Eq>" +
                                "<Eq>" +
                                    "<FieldRef Name='ContentType' />" +
                                    "<Value Type='Text'>LocationNews Item</Value>" +
                                "</Eq>" +
                            "</Or>" +
                        "</Where>";
 
            // put the CrossListQueryInfo object into the CrossListQueryCache
            CrossListQueryCache clqc = new CrossListQueryCache(clqi);
 
            // and query the data!
            // make sure: the GetSiteData(SPWeb web) and GetSiteData(SPWeb web, SPSiteDataQuery query) DO NOT use caching!!!
            DataTable tbl = clqc.GetSiteData((SPContext.Current.Site, CrossListQueryCache.ContextUrl());
 
            // return the datatable
            return tbl;
        }

The enum below can be used to make life a little bit easier. I got it from: http://www.aspenhorizons.com/devblog/?p=29

public enum ListServerTemplateCodes

    {
        GenericList = 100,
        DocumentLibrary = 101,
        Survey = 102,
        LinksList = 103,
        AnnouncementsList = 104,
        ContactsList = 105,
        EventsList = 106,
        TasksList = 107,
        DiscussionBoard = 108,
        PictureLibrary = 109,
        DataSources = 110,
        SiteTemplateGallery = 111, 
        UserInformationList = 112,
        WebPartGallery = 113,
        ListTemplateGallery = 114,
        XMLFormLibrary = 115,
        MasterPagesGallery = 116,
        NoCodeWorkflows = 117,
        CustomWorkflowProcess = 118,
        WikiPageLibrary = 119,
        CustomGridForAList = 120,
        DataConnectionLibrary = 130,
        WorkflowHistory = 140,
        GanttTasksList = 150,
        MeetingSeriesList = 200,
        MeetingAgendaList = 201,
        MeetingAttendeesList = 202,
        MeetingDecisionsList = 204,
        MeetingObjectivesList = 207,
        MeetingTextBox = 210,
        MeetingThingsToBringList = 211,
        MeetingWorkspacePagesList = 212,
        PortalSitesList = 300,
        BlogPostsList = 301,
        BlogCommentsList = 302,
        BlogCategoriesList = 303,
        PageLibrary = 850,
        IssueTracking = 1100,
        AdministratorTasksList = 1200, 
        PersonalDocumentLibrary = 2002,
        PrivateDocumentLibrary = 2003
    }
 
more information about this subject can be found at: 
http://sharepoint.nailhead.net/2008/04/musing-on-crosslistquerycache-class.html 

The Myth of Stored Procedures Preference

Adel Khalil — Sat, 05 Jan 2008 22:16:28 GMT

When looking to the Stored Proscedures debate, there is always those three factors you should measure by.

Productivity

- Span code base over multiple staging environments is a pain and harder to maintain.

- Versioning sp code is way harder than versioning application code.

- Minor change to the design require changing in both the SPs and the DAL code.

- Todays IDEs are more advanced than most of the RDBMS offers, implementing on IDEs is obviously preferable.

- Switching between two seprate world to implement single method is always pain.

- It's impossible to cover every single scenario and write SP for it, which will lead to write these SP as you go, huge consistency problem.

- There is no way to only update single param in the Update method using SPs as there isn't optional parameters, on every update you need to supply full param collection.

- SPs are not portable if you want to develop application that run over multiple DBMS you will be writing SPs for each DBMS, standard SQL is portable.

Security

- Big myth over here, using SPs not guarante best security practice and certinly dosn't mean that your application is SQL Injection proof you can write code like this

string s = "EXEC sp_GetCustomerByEmail '" + txtEmailAddress.Text + "'";

and you will be using SP and still open to all kind of SQL Injection.

- Another myth regarding security is that if you are using Ad-hoc queries you *most likely* grand permissions for CRUD operations for your application user on the database, no you are not, that's why Views are invented.

Performance

often when SPs vs. Ad-hoc queries debate intoduced the performance card played, SPs advocates says SPs are pre-compiled which is not let met quate like Frans did from SQL Server Books Online

SQL Server 2000 and SQL Server version 7.0 incorporate a number of changes to statement processing that extend many of the performance benefits of stored procedures to all SQL statements. SQL Server 2000 and SQL Server 7.0 do not save a partially compiled plan for stored procedures when they are created. A stored procedure is compiled at execution time, like any other Transact-SQL statement. SQL Server 2000 and SQL Server 7.0 retain execution plans for all SQL statements in the procedure cache, not just stored procedure execution plans.

So with no pre-compilation and caching for both SPs and SQL statments there is no advantage for SPs here, in some other databases the SPs compiled into C or C++ but this isn't the case in SQL Server 7.0/2000.

I have introuduced my view on the SP vs. Dynamic SQL i don't see any benfit of SPs over the huge amount of productivity, performance that you will gain with dynamic SQL, the only benfit in peformance you will get it when using Managed SPs (SQL Server 2005) but for 0.7/2000 SPs isn't the right choice for most of the scenarios.

Tiny Tip: is vs. as

Adel Khalil — Mon, 08 Oct 2007 02:33:00 GMT

Today's tip is a quick look at both not widely used C# operators and why prefer one to another..

Casting using is-operator:

if(p is Product) // CLR?: Could i cast this to product?

{

Product x = (Product)p; // CLR again??: Could i cast this to Product?

}

This work but comes with performance cost so you better use the as-operator as follows:

Casting using as-operator:

Product x = p as Product; // CLR?: Could i cast this to product?

if(x!=null)

{

// your logic here..

}

Here the CLR checks for type compatibility only one time and as-operator never throw an exception only the result will be null if it's not type compatible.

Tiny Tools, Huge Benefits

Adel Khalil — Wed, 12 Sep 2007 19:03:12 GMT

Those are a subset of Hanselman's tool list, those are very helpful during my daily job... so i thought i share with you.. to boost your productivity as well.

Query Express - A light weight Query analyzer like application

.NET Reflector - Amazing utility reflect over .NET Assemblies you can use it as code converter as well between number of .NET Languages.

SysInternals Tools - Those are amazing now part of Microsoft.. Process Explorer, FileMon, RegMon and other goodies.

IIS Diagnostic Tools - Number of great tools allow debugging, tracing and other stuff as well.

IE Developer Toolbar - A FireBug equivalent.

FireBug - Can't debug without it... must have tool that looks underneath your page.

MemProfiler - Track memory leaks

if you have any tiny little goodies you use frequently please do share.

How fickle the SQL!

Richthofen — Wed, 12 Sep 2007 13:55:00 GMT

So I was optimizing a query that displays a timetable of television programming. I'm working to try to optimize it because its currently taking 3-10 seconds per execution, which is less than optimal, considering that the query needs to be made 48 times per pageload, to load 48 timetables.

The query looks something like this:

SELECT * FROM TimeSlotsInADay CROSS JOIN (

SELECT DISTINCT dayofweek FROM Dashboard_TEST.dbo.broadcastDates

)

INNER JOIN ProgrammingData

Now normally you'd think that wouldn't be too bad. Basically we have a list of timeslots, and we make an entry for every timeslot, every day. But it turns out, the bottleneck according to the Execution Planner was actually the cross join. So I ran the subquery and found out that the subquery was returning days in reverse order, while all my Indexes on those day fields were set to ASC order. So basically when the JOIN happened on programming data, because the list was reversed, the index scanning ended up being equivalent to a table scan.

So I modified the list to ORDER BY on the subquery (which requires a TOP statement), so the Subquery now looks like:

SELECT DISTINCT TOP 7 dayofweek FROM Dashboard_TEST.dbo.broadcastDates ORDER BY 1

And now the entire parent query executes in under a second. Hope this helps someone

In search for the fastest IList to IList<T> conversion

Ramon Smits — Sun, 17 Jun 2007 22:47:00 GMT

I was yet again busy with a generic data layer. I like type safety as it provides compile time safety instead of run-time. The current released Db4o build (6.1) does not support type safe result list thus I had a need to convert these. My first approach was to just do a foreach like the following example:

Conversion one

        public static System.Collections.Generic.IList<T> Convert1<T>(
            System.Collections.IList source
            )
        {
            System.Collections.Generic.IList<T> result = new System.Collections.Generic.List<T>(source.Count);
            foreach (T o in source)
                result.Add(o);
            return result;
        }

But after I had this working I thought that this could be slow and that another solution was probably faster. I did a quick google search and ended up at a post of Bil Simser. There I saw some approaches in the comments but without additional information about its performance. So I made a little performance test where I tested not only the first example but also two others:

Conversion two

        public static System.Collections.Generic.IList<T> Convert2<T>(
            System.Collections.IList source
            )
        {
            System.Collections.ArrayList list = new System.Collections.ArrayList(source);
            return new List<T>(list.ToArray(typeof(T)) as T[]);
        }

Conversion three

        public static System.Collections.Generic.IList<T> Convert3<T>(
            System.Collections.IList source
            )
        {
            T[] output = new T[source.Count];
            source.CopyTo(output, 0);
            return output;
        }

After creating an ArrayList I initialized it with 20.000.000 instances of my test class and then converted it to a type-safe IList<T>. After running it several times with no load on my system I got the following results:

Conversion one : ~2175ms
Conversion two : ~ 884ms
Conversion three : ~133ms

So my "vingerspitsgevoel" was right again! What amazes me is that the conversions timings are so far apart. At first I didn't think that the second conversion would differ that much compared with the first but I think the .ToArray(..) method has CLR optimizations just as .CopyTo(..) method in the third example. The third conversion is more then 16 times faster then the first and even more then 6,5 times faster then the second.

The third conversion is even faster with much smaller arrays and lists (200.000 items). Then it is ~26 times faster then the first and ~7 times faster then the second conversion. I did this test on a Intel 3GHz D processor with 2GB of RAM on Windows XP Pro with the .net 2.0 framework.

So if you were searching for (probably the fastest) ArrayList / IList to IList<T> conversion then you have found it. I immediately added the third conversion method to my personal framework code and I suggest that you do to after seeing these results.

Software RAID and virtualisation

Ramon Smits — Mon, 14 May 2007 12:30:00 GMT

I have 3 harddrives in my home computer and today I though lets experiment with software RAID. So I first made some room on my drives and then created a 3 disk wide software RAID partition in Windows. After that I migrated my development virtual machine and started it. The damn thing is flying now! So if you have some room free on your partitions and don't want all your drives to be hardware RAID 0 because of failure then this is a good thing to do to have better virtualisation performance on your working machine.

By the way, I'm using VMWare Server 6.0.3 as host.

Improve Windows Forms performance

Adel Khalil — Fri, 09 Feb 2007 14:52:00 GMT

This post is particularly for me as I always forgot those great tips and hard to find them back in MSDN Magazine or other resources, so I figure by putting them here will end up using them more also dear reader enjoy them too I'm sure you will find something you don't know or forgot a while ago.

fire your application faster

You can accomplish this by many ways starting with not populating the databoud controls in the load event directly instead have them on serrate thread so they don't block the UI while they linking to data source and rendering.

If you are using .NET 2.0 then you pleased with the new BackgroundWorker component that will delegate the work into background thread - will talk in details about BackgroundWorker in other post .

Now the most important part which is will boost your application startup is using native image generator NGen which is available with .NET Framework SDK since version 1.1 at least.

couple of days ago I have attended session about improving application startup performance by NGen within MDC presented by Surupa Biswas Program Manager on CLR Team and with the demo she showed us the difference between launching Microsoft Expression Blend with and without NGen images like the difference between shooting a bullet and throwing it and this also speak of a seprate post.

faster UI and Controls rendering

Always use BeginUpdate() and EndUpdate() to supress repaiting the controls while it's collection changing like the case of ListBox and ComboBox.

The neat trick i wouldn't thought of before by Milena Salman is when you hard code the data binding to a control is to assign the Control.ValueMemebr first then Control.DataSource couse assining the data source first will couse the control to repopulate.

Also notice that if you add controls at runtime, change or resize any existing controls will couas the parent control to fire Control.Layout event, so be sure to use SuspendLayout and ResumeLayout on the parent control when playing with the child controls.

Don't lean much on garbage collector

Not becouse GC will let you down but for instant finalizing managed resourses within your code is less expensive by letting GC do the job also don't forget to use the using keyword with unmanaged resources, if you do finalize an unmanaged resource by yourself call the GC.SuppressFinalization to save a GC cycle.