Orchestration performance

Published Wed, Apr 28 2004 12:38 AM

The following test is done in a somewhat uncontrolled environment and can not be seen as a benchmark test. It might contain errors.

I've done some testing with BizTalk performance the last two days. We had some good and some bad results, but the general feeling that we had in the project was that the performance was less than expected.

So I decided to do some performance testing tonight on my BizTalk Server which is hosted by VirtualPC which is hosted by my 3.2 GHz Pentium 4 with 2GB Memory home computer. The results were odd and unexpected. Here are the results.

Scenario 1: Receive 1 message and send out 1000 messages in a loop
Scenario 2: Receive 1 message and send out 1000 messages in a loop. Mark the orchestration as long-running
Scenario 3: Receive 1 message and call 1000 orchestrations in a loop. Send out 1 message in every called orchestration (so send out 1000 message).
Scenario 4: Receive 1 message and start 1000 orchestration in a loop. Send out 1 message in every started orchestration (so send out 1000 message) and initialize a correlationset. Wait for 1 message to receive (but do not actually receive the message).
Scenario 5: Receive 1 message and start 1000 orchestration in a loop. Send out 1 message in every started orchestration (so send out 1000 message).
Scenario 6: Receive 1 message and send out 1000 messages in a loop. Receive those 1000 messages in separate orchestration (1000 instances) and send out 1000 messages.
Scenario 7: Repeat Scenario 2
Scenario 8: Reboot the virtual machine and repeat Scenario 2

I've cleaned up folders between every run and restarted the BizTalkApplication Host

Here are the numbers:

1) 97 seconds -> 10 messages per second
2) 90 seconds -> 11 messages per second
3) 94 seconds -> 11 messages per second
4) 409 seconds -> 2.5 messages per second
5) 516 seconds -> 1.9 messages per second
6) 1248 seconds -> 0.8 messages per second
7) 379 seconds !! -> 2.6 messages per second
8) 369 seconds !! -> 2.7 messages per second

I didn't expect to see the drop in throughput between scenario 3 and 4, but the figures in 7 and 8 are more frightning if you ask me. All of a sudden the same orchestration takes almost 4 times as long to complete.

The figures at the project are more complex with more orchestrations and messagetypes involved, but I do recognize the same pattern to some extend. It tends to slow down when more message have been processed. It makes me curious into where this curve will go over a longer period of time. Will my BizTalk processes come to a slow stop?

It's 0:30 AM here, when I write this so I won't investigate further now ;-), but I'm very interested in performance figures from others and any clue to what could cause this behaviour. 

by Carlo Poli
Filed under:

Comments

# Carlo Poli said on Wednesday, April 28, 2004 9:10 AM

Carlo,

Have a look in the eventlog and see if BT2004 was throtteling the messageboxdb.

If so, the figures of the last to tests are coloured.....

# Carlo Poli said on Wednesday, April 28, 2004 1:59 PM

@Patrick

Interesting thought. I can't check that from here, unfortunately. But in that case it would reach the opposite of what it's meant to do. It lowers the throughput. Isn't that strange?

# Carlo Poli said on Wednesday, April 28, 2004 5:46 PM

What is the implication of the throttling? I have seen this entry in the event log and I have experienced poorer BTS04 performance than expected. Any idea what a "typical" BTS04 solution looks like? How much underlying hardware horsepower to accomplish what sort of throughput and worload goals?

# Carlo Poli said on Thursday, April 29, 2004 12:03 PM

No....

Throtteling will ( ok not precise but....) will slow down the processing of new messages.

I dont know what it really does but this is what it seems to be doing....

More processing power to handeling a message
Less processing power to Receive stuff

So messages will be handled but it will take a longer time. This is done to prevent the server from thrashing the haddisk
(At a certein point in time it is only swapping memory that SQL Needs...)


# Carlo Poli said on Tuesday, May 11, 2004 1:12 PM

The Start shape causes BTS to generate an async call to the ExecService() method of the BTXService, whereas the Call shape causes the generation of a sync call to the StartCall() method of the current context. An instance of the called orchestration is passed directly to StartCall, whereas ExecService takes a Type object for the orchestraion to be called. I wonder if the slow-down in scenarios 4 and 5 are due to the use of reflection to indirectly invoke orchestrations? Using reflection and a late-bound approach 1000 times in a loop is going to inroduce some noticable overhead!

# Carlo Poli said on Friday, August 13, 2004 2:29 PM

Explanations of why steps 7 and 8 are so slow :

The BTS 2004 configuration create about 6 scheduled jobs in SQL Server. There jobs clean up messagesbox, process, queue, etc... Be sure that the user running the jobs have effective right to run SPs. By default, the configuration put user doing setup for seting up user running in SQL server... Somes of this jobs are fired every minutes.

Also, if you're having some messages waiting for something in messagebox, this can really slow down your tests.

# Start/Call Orchestrations Performance | keyongtech said on Thursday, January 22, 2009 4:50 AM

Pingback from  Start/Call Orchestrations Performance | keyongtech