April 2004 - Posts

Orchestration performance
Wed, Apr 28 2004 12:38 AM

The following test is done in a somewhat uncontrolled environment and can not be seen as a benchmark test. It might contain errors.

I've done some testing with BizTalk performance the last two days. We had some good and some bad results, but the general feeling that we had in the project was that the performance was less than expected.

So I decided to do some performance testing tonight on my BizTalk Server which is hosted by VirtualPC which is hosted by my 3.2 GHz Pentium 4 with 2GB Memory home computer. The results were odd and unexpected. Here are the results.

Scenario 1: Receive 1 message and send out 1000 messages in a loop
Scenario 2: Receive 1 message and send out 1000 messages in a loop. Mark the orchestration as long-running
Scenario 3: Receive 1 message and call 1000 orchestrations in a loop. Send out 1 message in every called orchestration (so send out 1000 message).
Scenario 4: Receive 1 message and start 1000 orchestration in a loop. Send out 1 message in every started orchestration (so send out 1000 message) and initialize a correlationset. Wait for 1 message to receive (but do not actually receive the message).
Scenario 5: Receive 1 message and start 1000 orchestration in a loop. Send out 1 message in every started orchestration (so send out 1000 message).
Scenario 6: Receive 1 message and send out 1000 messages in a loop. Receive those 1000 messages in separate orchestration (1000 instances) and send out 1000 messages.
Scenario 7: Repeat Scenario 2
Scenario 8: Reboot the virtual machine and repeat Scenario 2

I've cleaned up folders between every run and restarted the BizTalkApplication Host

Here are the numbers:

1) 97 seconds -> 10 messages per second
2) 90 seconds -> 11 messages per second
3) 94 seconds -> 11 messages per second
4) 409 seconds -> 2.5 messages per second
5) 516 seconds -> 1.9 messages per second
6) 1248 seconds -> 0.8 messages per second
7) 379 seconds !! -> 2.6 messages per second
8) 369 seconds !! -> 2.7 messages per second

I didn't expect to see the drop in throughput between scenario 3 and 4, but the figures in 7 and 8 are more frightning if you ask me. All of a sudden the same orchestration takes almost 4 times as long to complete.

The figures at the project are more complex with more orchestrations and messagetypes involved, but I do recognize the same pattern to some extend. It tends to slow down when more message have been processed. It makes me curious into where this curve will go over a longer period of time. Will my BizTalk processes come to a slow stop?

It's 0:30 AM here, when I write this so I won't investigate further now ;-), but I'm very interested in performance figures from others and any clue to what could cause this behaviour. 

Testing BlogJet
Tue, Apr 27 2004 11:39 PM

I have installed an interesting application - BlogJet. It's a cool Windows client for my blog tool (as well as for other tools). Get your copy here: http://blogjet.com

The cool thing I like about BlogJet is that you can immediately use pictures in your post and let them be FTP'ed to whereever you like.

Here's such an interesting picture ;-). It's a shame though that you can't directly paste a picture in the blogtext and let BlogJet name and publish the picture.

Validiting messages in MSMQT - Excellent post from Darren Jefford
Wed, Apr 21 2004 9:57 AM

I just came across the strange behaviour that invalid messages weren't captured by the HAT if send via MSMQT and were captured when send via FILE.

Turned out that Darren Jefford published about it a couple of weeks ago in an excellent article and came up with a workable (although strange) solution. He says it's not a bug. I can understand what he means, but stills find the behaviour odd. You can't do anything other with a nonresumable message but to look at it, so why don't I just get it.

by Carlo Poli | with no comments
Filed under:
Issue in my MightCorrelateSample
Tue, Apr 20 2004 11:08 PM

I published last week about how to handle messages that might correlate (see this post). When implementing it, I stumbled upon the issue that although I handled the exception, the messages that failed to correlate show up as suspended in the HAT (twice).

That is a serious problem with this nice solution. In my project we decided that these messages are expected and I wrote a Windows Service, based on a post I found on Darren Jefford's blog, that kills of those messages. Mail me if you're interested in the code, I can send you a copy of the Windows Service.

Although problem solved, I'm now left with the feeling that there has to be a better way to handle messages that might correlate. Any idea's?

Versioning in BizTalk
Tue, Apr 20 2004 9:54 PM

My current project will go live this weekend with the first phase and we're looking to go live with a next phase within days/weeks after that. 

But our solution relies heavily on long-running transactions (long being about a month), with thousands of concurrent transactions. So, we can't just undeploy an existing version and deploy a new one.

The good news is: BizTalk has build-in support for versioning which relies heavily on .NET assembly versioning (which is good).
The bad news is: The documentation about it is vage and incomplete. Things didn't work like I first understood from the documentation. Later I found out that the documentation might be right, but isn't very clear about action to take.

So I went off to see how versioning really works. I started easy. I tested two scenario's on a simple solution.

The solution I used for testing

My solution consists of two project. One with 2 schemas and a map and one with 1 orchestration that receives a message, maps it and send it out. I deployed both assemblies with version number 1.0.0.0 and then increased the version number of the schema assembly to 1.1.0.0 and started changing it.

Scenario 1

The idea: I fixed some bugs in my maps, the assemblies interface is exactly identical and I just want to replace the existing version without any impact.

I decided that the easiest way to do this is to NOT deploy the assembly to BizTalk. Instead, just deploy it to the GAC using gacutil and let .NET's versioning make sure the right assembly is used. The way to do this in .NET, is to add assemblybinding information to the application configuration file. For BizTalk this is the BTSNTSvc.exe.config file which can be found in the root of the BizTalk installation folder. You can create the binding directly in that file or create a seperate configuration file and reference it from BTSNTSvc.exe.config. This does require you to create a seperate appdomain for your assemblies, which might not always be what you want.

I added the following info to BTSNTSvc.exe.config, just below the opening configuration tag.

<configSections>
  <section name="xlangs" type="Microsoft.XLANGs.BizTalk.CrossProcess.XmlSerializationConfigurationSectionHandler, Microsoft.XLANGs.BizTalk.CrossProcess"
/>
</configSections>

<xlangs>
  <Configuration
>
    <AppDomains AssembliesPerDomain
="10">
     
<AppDomainSpecs
>
       
<AppDomainSpec Name="MyAppDomain" SecondsIdleBeforeShutdown="-1" SecondsEmptyBeforeShutdown
="12000">
          <BaseSetup
>
            <ConfigurationFile>c:\myAppBase\myConfig.config</ConfigurationFile
>
          </BaseSetup
>
        </AppDomainSpec
>
      </AppDomainSpecs
>
      <PatternAssignmentRules
>
        <PatternAssignmentRule AssemblyNamePattern="myAssembly*" AppDomainName="MyAppDomain" />

      </PatternAssignmentRules
>
    </AppDomains
>
  </Configuration
>
</xlangs>

My configuration file contained the following

<?xml version="1.0" ?>
<configuration>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
        <dependentAssembly>
           <assemblyIdentity name="myAssembly" publicKeyToken="32ab4ba45e0a69a1" culture="neutral" />
           <bindingRedirect oldVersion="1.0.0.0" newVersion="1.1.0.0"/>
        </dependentAssembly> 
     </assemblyBinding> 
  </runtime>
</configuration>
NOTE: Above code is not tested, so it might contain some issues, just tell me and I will update it

After restarting BiztalkServerApplication the new version was used perfectly.

Scenario 2

The idea: I extended the assembly with new schemas and maps, which will be used by new assembly. My existing orchestration assembly remains unchanged.

Now I am required to deploy the assembly to BizTalk. The issue that arises after you deploy the assembly is the existance of two definitions of all the existing schemas and maps. This will especially cause errors when receive and sending messages using the default pipelines. The solution to this issue is pretty simple. You have to tell BizTalk which version of the schema you want to use.

I extended my version 1.1.0.0 with two pipelines, one for sending and one for receive my messages. I added the XML (dis)assembler to the (dis)assembler stage and added the schemas to the document definition. Since the pipelines reside in the same assembly as the schema, it will always reference the latest version. I used these pipelines in the send and receive port and it worked perfectly.

If you need to reference the version 1.0.0.0 of schemas in your pipeline (which might be needed if you change schema definitions along the way), you have to put the pipeline in a seperate assembly that is build against the version 1.0.0.0 of the assembly. Here's where it started to become complex and I haven't gone through this root (yet ;-)

I guess I justed scratched the surface and didn't cover all the things that can go wrong in scenario 2, but this looks pretty promising.

I would recommend though that if you expect versioning of your assembly that you quickly start using your own pipelines. This eases the stuff.

My next question is how to version orchestrations. The documentation mentions two scenario that might apply to my situation, side-by-side execution and major application upgrade. So, to be continued I guess.

Good webcasts about HWS and InfoPath
Tue, Apr 20 2004 9:43 PM

I 'attended' the live webcasts about HWS (view here) and InfoPath (view here) last week. The Human Workflow Services webcast was pretty good, although I was left with the feeling that the webcast wasn't really clear on what it exactly takes to implement a HWS-solution. HWS still looks pretty complex to me, but I did get enthousiastic about it.

The InfoPath session that Scott Woodgate gave yesterday was brilliant. One demo was particularly impressive. Scott showed how to publish an orchestration as webservice and then build an InfoPath form against the webservice. 5 minutes of work and a piece of cake. The user friendlyness of that solution is tremendous. Send the user an InfoPath document via mail or SharePoint, let him fill it out and let him push publish. Your orchestration in BizTalk will do the handling. Scott's published the samples he demo-ed here. Make sure you look at his webcast first.

Nice takeway -> InfoPath is to BizTalk what Outlook is to Exchange.

by Carlo Poli | with no comments
Filed under:
Finally some time to blog about my BizTalk experiences
Tue, Apr 20 2004 9:25 PM
I've been diving into updategrams with the SQL-Adapter, implementing BAM at my current project and looking into versioning. And I have seen two pretty good webcast about HWS and InfoPath. And I just didn't get around to write about it. And I also disappointed some commenters by not reacting to their comments. Let's see how far I will get tonight ;-)
by Carlo Poli | with no comments
Filed under:
BizTalk Tutorial available for download
Tue, Apr 13 2004 4:17 PM

I have used the beta material of the tutorial to learn BizTalk 2004 in a week, and that worked out very well.

So, for anyone starting with BizTalk, I would recommend to follow this tutorial. You can find the tutorial here.

How to undeploy a BAM Schema after you lost the right Excel sheet
Mon, Apr 12 2004 10:04 PM

If you make an BAM Excel sheet and deploy it, you either need that same Excel sheet or it's exported XML to undeploy it. What happened to me two times now, is that I changed the sheet and it's XML export, before I undeployed it.

Now, when I try to deploy the new version, I get an “The activities and/or views have already been deployed. Please undeploy first before redeploying.”

And when I try to undeploy using the new version, I get an “BAM undeployment failed. The view MainView cannot be undeployed. The definition does not match.”

The solution to this is pretty simple. The definition you used to deploy the earlier schema is stored in the table bam_Metadata_Transactions in the BAMPrimaryImport database. Find the last record with deploy in the optype column and the correct filename in the BamDefFileName column. Copy the contents of the BAMDefinitionXML column to a textfile, save it as something.xml and use that file to undeploy your BAM Schema.

Note: The XML can get quite large and if that happens, you can not get the complete XML using the Query Analyzer or the Enterprise Manager. I used Access to attach to the table and get the correct XML.

How to handle messages that might correlate (or might not)
Sat, Apr 10 2004 11:00 PM

In my current project we are in a situation where we receive messages that might correlate with existing orchestration, but just might not correlate as well. The messages that don't correlate are perfectly valid, but just don't belong to an already running orchestration instance. I discovered a way tonight that handles this situation very elegantly. The sample showing exactly how can be found here. Unzip it to the root of c:. Build and deploy the project and then use the drop directory to run the process. First throw in the FirstMessages in their in folder and then the SecondMessages in their in folder.

The key to the solution is the use of seperate orchestration to handle the incoming messages that might correlate. This orchestration uses a direct port which is somehow described in my other post of today.

In short the solution is as follows:

  • Build an orchestration with the correlating flow. Let the correlating messages listen to a direct port.
  • Build an additional orchestration that listens to the messages that might correlated.
  • Send the message to main orchestration inside a scope shape
  • Connect an exception catch block to the scope shape, and start the alternative orchestration for noncorrelating message (or do the stuff there and then).

Refer to this article at Jan Tielen's blog for a guide on using exception blocks (I needed it ;-).

 

Direct ports in BizTalk
Sat, Apr 10 2004 10:53 PM

I ran into an interesting feature of BizTalk tonight that I have overlooked uptill now. It's called a direct port and it enables you to send messages between orchestrations via the messagebox, without having to specify a transport.

The particular cool thing about this, is that it supports the correlation of messages to running orchestration instances, something you can't do when you call or start orchestration using the shapes. I find this feature so interesting because it solves an issue I have at my current project with incoming messages that might correlate or not (see my other post today).

You can create a direct port using the Create Port wizard, just select Direct instead of Specify now or Specify later. Two notes:

1) I was able to create an endless loop by creating a port that just listened to the messagebox and pick up all messages of a particular type and send that one out to a folder. Because that message also entered the messagebox another orchestration got started (and then another one and another one...). So watch out, don't start the process and then get yourself some coffee. Your computer might be dead by time your back ;-).

2) I got the build error 'Illegal port binding' when I tried to connect two orchestrations. It disappeared at some point to never come back, even resetting all settings to non-working configurations didn't recreate it (strange, isn't it). If anyone knows what causes this error and how to resolve it, I would very much like to know. -> UPDATE 4/14/04: I noticed that I connected the listening port to itself and the sending port to the listening port to resolve the problem. See my sample for more details.

Problems with validating a BizTalk-mapfile in VS.NET
Thu, Apr 1 2004 10:41 AM

I just spend two hours trying to figure out why my very simple map wasn't valid when using Validate Map. It turned out that I had changed a field name in the source schema, which was in a different assembly and I hadn't compiled that assembly yet. After I compiled it, I could succesfully test my map. 

The error message I got was “Exception Caught: The map contains a reference to a schema node that is not valid.  Perhaps the schema has changed.  Try reloading the map in the BizTalk Mapper.  The XSD XPath of the node is: <XPath for new nodename>

by Carlo Poli | with no comments
Filed under: