Limit the number of instances of any BizTalk Service
At my current project we had a problem. We received a batch of files (more then 500) and we had to process all the files. For every file we needed to call a webservice and sometimes a response would take about 1 minute. There was another limiting factor. The webservice we needed to call only could have 5 instances running at the same time. So we needed a way to limit the number of running service instances for that particular web service. On top of that we needed to give Interactive requests to the webservice priority over batch request. So interactive calls should be executed immediately.
I set up a POC an thanks to Carlo Poli we had a working prototype pretty quickly. This prototype had only one problem, we had to re-apply the logic again if we wanted to limit another service. So it had to be a generic BizTalk Solution capable of limiting any service to given number.
First we needed the propertyschemas. Below is a sample ofthe propertyschema
- CorrelationToken is a correlation Token used to correlate the messages on
- Service is the Servicename we want to limit
- Priority is the priority the message has can be LOW or HIGH
- Retrycount could be the retrycount of a message. Nothing is implemented yet.
Then we needed a couple of schema's. The schemas are quite simple and all have the same layout. Only the namespace is different.
- InstanceController Start is used to Start the controller.
- InstanceController Request is used to send the request to the Endpoint
- InstanceController Response is the response from the endpoint
Next we needed a way to configure the InstanceController I chose the Business Rule Engine for that. For every service there should be a BusinessRule.
- CallbackTimeout, If a service is limited we need a way to force a callback from the limiter in case the answer from the limited service never comes.
- HighPrioTimeout, same as above but now for the High Priority messages.
- Max Instances, The maximum concurrent Webservice calls the Instance Controller will start.
- Send XXPrio Failure, If the callback timeout is expired should a message be routed to the initiating service. (notifying it of the failure)
Using the instance controller.
Now how do you use the instance controller ? First of all we need to construct a message of InstanceControllerStart. And set the payload to the message we want to limit.
// Construct message from xml
xmlLimiterStart = new System.Xml.XmlDocument();
msgInstanceControllerStart = xmlLimiterStart;
xmlPayload = new System.Xml.XmlDocument();
xmlPayload = msgFromDisk ;
// Set Payload
msgInstanceControllerStart.Payload = xmlPayload.InnerXml;
Then we have to set some context properties
// Set context properties
msgInstanceControllerStart(InstanceController.Schemas.CorrelationToken) = System.Convert.ToString( System.Guid.NewGuid());
msgInstanceControllerStart(InstanceController.Schemas.RetryCount) = 0 ;
msgInstanceControllerStart(InstanceController.Schemas.Priority) = "LOW"
msgInstanceControllerStart(InstanceController.Schemas.Service) = "SampleService"
Then when we send the message to the message box, we start a correlation on the Correlation Token so we can correlate the response. Next to that we have to promote the other properties as well, so we start a correlation on them as well. So we initialize 2 correlations, One on the CorrelationToken and one on the other properties. Then we Add a receive shape in our orchestration and set the messagetype to InstancelimiterResponse and follow the correlation on the CorrelationToken.
So the message is send off to the messagebox and the limiter will do it's work. We have to modify the orchestration that does the actual work a little. We will receive a InstanceControllerRequest message, so we have to filter the service by message type. The filter for the receive is BTS.MessageType="http://InstanceController.Schemas.InstanceControllerRequest#Container and "http://InstanceController.Schemas.InstanceControllerRequest#Container" = "ServiceName". Where ServiceName is the name of the service we are limiting. Once the orchestration has done it's work we must send a response back to the messagebox. The message we are sending back is of the type InstancelimiterResponse. So we have to put the actual message in the payload node again. And promote some properties so the frontend will receive the response.
// Get webresponse to XML
xmlPayload = new System.Xml.XmlDocument();
xmlPayload = msgResponse ;
// Construct Response Message
xmlResponse = new System.Xml.XmlDocument();
msgInstanceControllerResponse = xmlResponse ;
msgInstanceControllerResponse.Result = "SUCCESS";
// Set properties
msgInstanceControllerResponse(InstanceController.Schemas.CorrelationToken) = strCorrelationToken;
msgInstanceControllerResponse(InstanceController.Schemas.Priority) = strPriority ;
msgInstanceControllerResponse(InstanceController.Schemas.RetryCount) = intRetryCount ;
msgInstanceControllerResponse(InstanceController.Schemas.Service) = strService ;
Then we have to start a correlation again on the message we are sending. We only have to correlate on th ecorrelationToken. By doing so the frontend will get the response from the webservice.
Something about callbacks
The InstanceController does it's work by counting the number of instances send to the endpoint service. Everytime a message is send of th ecounter is increased. If we get a response the counter is decreased. So we always know how much instances we have running. But what if the webservice doesn't give a response ( is bad design but it could happen ) ....
The counter will never decrease... So we need a way of making sure the counter is decreased is a webservice does't give a response. That's where the callback comes in handy. The callback will wait for a very long time ( bussiness rule ) and if a response is not received by that time it will give a callback to the limiter. So the counter is decreased.
You can find the complete at codeplex http://www.codeplex.com/InstanceController/Release/ProjectReleases.aspx?ReleaseId=1863