Gerben van Loon

IBM WebSphere MQ WCF Channels comparison update

Gerben van Loon — Tue, 16 Aug 2011 07:46:00 GMT

Wrote a lot about the IBM WebSphere MQ WCF Channels in the past. Check out my original post and an update on that one.

What changed in the meantime is that Microsoft added backout queue support to their channel in a hotfix. My comparison table now looks like this:

	IBM	HIS2009	HIS2010
Fire-and-forget (oneway) client	+	+	+
Fire-and-forget (oneway) service	+	+	+
Request-reply client	+	-	+
Request-reply service	+	-	+
Local transactions	+	-	+
Distributed transactions (DTC)	-	+	+
Client side transactions (DTC)	-	+	+
Backout queue support	+	-	+
Message format	-	+	+
AppFabric support / IIS Hosting (WAS listener for IIS7)	-	-	+
Works with .NET 3.x	+	+	-
Works with .NET 4.0	+	+	+
WMQ6 support	-	+	+
WMQ7 support	+	+	+
Sample code	+	+	+
Documentation	+	-	-
Licensing	+	-	-

Conclusion
We'll consider starting to use the HIS channel for services only. Hosting services with the HIS channel can be interesting due to support for AppFabric which can save development and maintenance time. For client side we’ll probably stay on the IBM channel anyway due to the licensing costs of the Microsoft channel.

Mobile app development overrated?

Gerben van Loon — Mon, 01 Aug 2011 11:40:00 GMT

Fowler did a nice sum-up of different approaches for cross platform mobile development: http://martinfowler.com/bliki/CrossPlatformMobile.html

A little more structured I see these six approaches for mobile development:

Approach	Build UI	Build Libs	Dev platforms	UI Control experience	User Experience
Cross-platform toolkit with native components from each platform	1x	1x	1	Native. But lowest common denominator of functionality and UI controls from platforms available	No native experience form device
Cross-platform toolkit with emulation	1x	1x	1	Emulated. Differences in behavior per platform. Less performance / responsiveness	No native experience form device
Cross-platform toolkit only for libraries	Multiple x	1x	Multiple	Native	Native experience form device
Cross-platform toolkit but building seperate apps for each platform	Multiple x	1x	1	Native	Native experience form device
Mobile web apps instead of mobile apps	1x	1x	1	Web instead of device	No native experience form device. No interaction with mobile hardware like GPS, Camera, etc. No offline usage possible

My conclusion can only be that building apps that work on all platforms today is quite hard.

But everybody wants apps these days. Why? I don't know, probably because it sounds cool.

My advice: if you don't need to interact with hardware from the device or need offline usage just keep doing web apps. Nothing wrong with that! In fact they're even easier to build. Or am I missing a point?

Duet Enterprise - What is it really?

Gerben van Loon — Thu, 30 Dec 2010 12:04:00 GMT

Update: some changes to my original post. Thanks to William van Strien for pointing extra things out that I missed.

We’ve been participating in the Rapid Deployment Program (RDP) and Duet Enterprise (DuetE) seems to be GA so I can talk about it a bit more.

First of all there is loads of marketing stuff surrounding DuetE which makes it hard to really nail down what it is. This is logical because the market of companies who do have some SAP back office systems and SharePoint for front ends is pretty large. So apparently SAP and MS put some money in marketing DuetE. The website and standard presentations are full with terms like “Meet changing business needs, manage costs, boost productivity, Flexible, Rapid, Focus, Accelerate, etc.” so it seems a product of which every manager think he or she needs it.

But what is it really?

Well it’s three fold. The three parts overview is given by this picture which comes from this presentation:

I'll take the view from the SharePoint side since I'm not too familiar with SAP software development. Although note that the center of gravity for DuetE is on the SAP side. So what is it really for SharePoint for these three parts?

Ready-to-use Capabilities: SharePoint web parts delivered by DuetE to enable these ready-to-use capabilities. Good stuff because now you don’t need to build this yourself anymore. The web parts are mainly around use for collaboration / internal SharePoint websites. Besides the webparts DuetE also ships extra plumbing to get the SAP reporting functionality working in SharePoint and a profile sync mechanism to get SAP HR data to SharePoint.

Tools and Business Content: Just some SharePoint site templates for some SAP objects. Also mainly around use for internal collaboration sites.

Foundation: Provides a SAP Role sync provider to get SAP roles to work with SharePoint. Besides that all relying on SharePoint BCS here. DuetE just adds Single Sign On (SSO) and user mapping to SAP for you. Pretty nice, you don’t want to build this yourself. Could also be used for external facing SharePoint websites. According to the product team you can’t use this foundation without BCS. So disadvantage is that DuetE inherently suffers from the BCS minuses that I wrote about earlier.

Conclusion
DuetE on the SharePoint side gives you some ready-to-use capabilities (webparts + extra plumbing), site templates and can handle security stuff like roles and SSO to SAP for you. That´s it!

When to use SharePoint 2010 Business Connectivity Services (BCS)

Gerben van Loon — Thu, 30 Dec 2010 11:13:00 GMT

After viewing the usual conference sessions I was pretty enthusiastic about the possibilities of BCS. Seems to be nice at first, but when starting to design something real in the field I was thrown back to reality. BCS does give “free” things on your external data without too much coding. But there are some serious downsides like the one I described before.

My current plusses en minuses lists for making decisions about when to use BCS usage in solutions is:

+ Simple UI with SharePoint styling on your data without coding
+ Offline support via SharePoint workspaces
+ Single Sign On possibilities
+ Includes your data in search
+ MS office integration
+ The external data can be accessed by sandbox solutions
+ Out of the box throtling posibilities
- BCS directly calls into the external system, no caching. Is the external system fast enough to support your users? Or will it slow down your SharePoint site?
- No transaction support
- No bulk processing?
- Complex and custom types not properly supported
- Strict requirements on the WSDL/Type schema of the external system. Will almost never work in the real world unless you can control the schema. No complex types allowed.
- External Lists aren’t accessible via the SharePoint REST interface
- Very limited control over WCF configuration settings. Or only in very bad ways like for example timeout.

Hope it helps you to decide if BCS is the right solution. Let me know if I missed some plusses or minuses.

SharePoint 2010 Business Connectivity Services (BCS) - Complex and custom types not properly supported

Gerben van Loon — Thu, 30 Dec 2010 10:41:00 GMT

This one is annoying because data is almost never flat. Besides the limitation on complex and custom types there are some more field limitations, check them out here.

There is an article available about how complex types are supported. But I don't like the options outlined there.

You have to work either via the Lists API or the lower-level APIs of the BDC OM. Seems to be nice at first but the BDC OM API’s are really low level without any strong typing on your data available. The Lists API seems to be a better option since it now supports strong typing via SPMetal. But unfortunately SPMetal doesn’t work on external lists at the moment. So both APIs currently aren’t viable solutions for serious enterprise development, working with them feels like going back in time.

You can do some workarounds to flatten your types out. You could add an extra service layer especially for SharePoint or a BCS .NET Connectivity assembly. This all adds quite some extra work while normally when you build an “advanced custom code” solution type you do want to work with complex strongly typed data in your code. You don't have a need for flat data from that standpoint.

Another creative workaround is to create an extra normal SharePoint list and to use lookups in that one to your external lists. In that way strongly typing via SPMetal is supported again. Although this only works read only.

The BCS team once outlined the types of solutions that you can do with BCS. My current conclusion for BCS usage is to fully stay out of the advanced solution types and only use BCS only for the simple and intermediate solution types if that fits. Unless you badly need a feature that BCS gives you above custom coding.

IBM WMQ WCF Transport channel issues

Gerben van Loon — Mon, 20 Dec 2010 14:51:00 GMT

There are some known nasty issues that my colleagues discovered while using the IBM WMQ WCF Transport channel. These where discovered with client version 7.0.1.3.

The service will hit the faulted state and will not pick up further messages when:

- A wrong, empty or no soap action is in the message
- The codepage is different then UTF-8
- The quemanager is reset or the (network) connection to the queue manager is (temporary) lost

Another one is that the service will throw "This buffer cannot be returned to the buffer manager because it is the wrong size" when the MQ message format (MQ header) is filled with MQSTR.

Reported all these issues to IBM. Hope they will resolve them (very) soon.

IBM WebSphere MQ WCF Channels updated

Gerben van Loon — Mon, 20 Dec 2010 13:58:00 GMT

Some updates to my original post since Microsoft Host Integration Server 2010 is released. My comparison now looks like this:

	IBM	HIS2009	HIS2010
Fire-and-forget (oneway) client	+	+	+
Fire-and-forget (oneway) service	+	+	+
Request-reply client	+	-	+
Request-reply service	+	-	+
Local transactions	+	-	+
Distributed transactions (DTC)	-	+	+
Client side transactions (DTC)	-	+	+
Backout queue support	+	-	-
Message format	-	+	+
AppFabric support / IIS Hosting (WAS listener for IIS7)	-	-	+
Works with .NET 3.x	+	+	-
Works with .NET 4.0	+	+	+
WMQ6 support	-	+	+
WMQ7 support	+	+	+
Sample code	+	+	+
Documentation	+	-	-
Licensing	+	-	-

So what has changed in the meanwhile with HIS2010 since HIS2009?

Request-reply: Microsoft added request reply support to their channel for both the client and the service side. Not a big surprise that this was coming since when you paid attention you saw with Reflector that already some bits and pieces where available in the HIS2009 one. Request reply is available in a separate binding. Unfortunately the Microsoft implementation also suffers from the #2 Oneway calls over WMQ can still block issue. Hope they resolve that soon.

Local transaction support through the WCF 4 ReceiveContext feature. Means you don't always need DTC anymore when local transactions are sufficient.

AppFabric support: HIS2010 added an IIS7 WAS process listener for the WebSphere MQ (WMQ) protocol. This opens up AppFabric integration with WMQ. This means that you can now host WMQ services in IIS7 with the AppFabric dashboard functionality. No more custom build windows services!

Support for AppFabric and the WCF 4 ReceiveContext feature also means the HIS2010 channel now only works with .NET 4.

Documentation mains pretty poor, haven't found too much info.

Licensing unfortunately still means that you have to license via BizTalk. See my original post for details.

Still no backout queue support: this in fact means that while transaction support (local and distributed) seems to be nice but is pretty useless. You always hit a poison message that goes on forever when a transaction fails. Normally you want these messages to go to a defined backout queue after n times tried.

Conclusion
We’re still staying on the IBM channel for now. Maybe if the HIS team adds backout support to a future version we’ll consider switching for services. Services become more interesting in the HIS channel due to support for AppFabric which can save development and maintenance time. For client side we’ll probably stay on the IBM channel anyway due to the licensing costs of the Microsoft channel.

IBM WMQ WCF Transport channel tips and tricks

Gerben van Loon — Wed, 21 Jul 2010 08:24:00 GMT

Working with this transport channel now for a while, and discovered some interesting things which I’d like to share. With the transport channel I mean the “IBM.XMS.WCF.SoapJmsIbmTransportChannel” which you can find in the IBM.XMS.WCF assembly after installing WMQ 7 server or client. I’m currently using the 7.0.1.0 version.

#1 Skip SOAP/JMS over WMQ in a oneway service provider
The transport channel seemed to be tied to SOAP/JMS over WMQ. Although I discovered that you can skip this when you build a service provider.

You can set the WCF service AddressFilterMode to any: [ServiceBehavior(AddressFilterMode = AddressFilterMode.Any)]. If you do this then the service can just read plain text encoded SOAP envelopes without JMS and MQHRF2 stuff. But when doing request-reply then the replies that are send back by the service will again be in the SOAP/JMS format.

#2 Oneway calls over WMQ can still block
This one is interesting: WMQ is an asynchronous transport but WCF oneway calls can still block in the scenario described here. Although I hoped that IBM did this right I was already afraid that they didn’t since the WCF channel model is a bit quirky in this area. I blogged about this before for standard WCF bindings. The difference here is that even the operation call is blocking, not the proxy.Close().

WMQ is an asynchronous transport but IBM does support request-reply in their WCF implementation by using the correlation identifier pattern: http://www.eaipatterns.com/ramblings/09_correlation.html
Microsoft doesn’t do this in their WCF MSMQ implementation. It only supports oneway, no request reply.

WCF makes a difference between servicecontracts with oneway only operations and contracts that have oneway and request-reply operations mixed.

“Oneway only”

“Mixed”

So here is where it gets interesting: in WCF the choice of channel is just made once:

“The proxy is a model of the service contract. The choice of channel type is made once against the service contract rather than on a per-operation contract basis. The channel shape with highest preference that supports all of the operations is selected.” (Nicholas Allen, Microsoft WCF Team)

So here is where it goes wrong: The IBM WMQ WCF transport channel returns an IRequestChannel (request-reply) in the case of a mixed contract .This is fine for the request-reply operations but not for the oneway operations. This results in that the service consumer call on the oneway operation (like SayHello in the sample) blocks until the service side responded. This can take a while when the service is busy, so this can cause timeouts on the service consumer side:

“WCFCH0310E: The timeout '00:01:00' was exceeeded while attempting to receive data from endpoint. The operation did not complete in the time allowed. Review the system status and configuration and increase the timeout if required.”

One of the reasons why I would pick the WMQ transport is that you have a disconnected service consumer and provider. But due to the implementation of the WCF transport and depending on how you contract look you might get different results. Microsoft doesn’t suffer from the problem with MSMQ and WCF since mixed contracts aren’t supported.

The only work around that I’m aware of if to change the mixed contract to oneway only contracts.

#3 Assured delivery does not support multiple endpoints
You can only use the assured delivery mechanism when you only have a WMQ endpoint on your service. The behaviour will crash when you also for example add an HTTP endpoint. A HTTP endpoint for just WSDL publishing is no problem. Makes sense though because for the WMQ endpoint with assured delivery you could make some promises that you don’t loose messages. You can’t make these promises for an HTTP endpoint.

#4 Sharing conversations
The “sharing conversations” setting from the WMQ connection channel trough which you connect may not be 0. In this case with channel I mean the thing on the WMQ server side to connect, has nothing to do with WCF channels. Otherwise you’ll hit some errors like “The specified value 'IBM.XMS.Client.WMQ' is not allowed for 'Expected ConnectionType = '.” or 2058 or 2059 reason codes.

Updated my IBM WebSphere MQ WCF post again

Gerben van Loon — Tue, 20 Jul 2010 13:51:00 GMT

Updated the post again: http://bloggingabout.net/blogs/gerben/archive/2010/02/25/ibm-websphere-mq-wcf-channels.aspx.

Now a clearer overview with a feature comparison.

So what happened to “Oslo”?

Gerben van Loon — Tue, 13 Jul 2010 07:10:00 GMT

This question still pops up every now and then. You can find quite a clear explanation on MSDN: http://msdn.microsoft.com/en-us/data/bb525059.aspx

Microsoft beta exams results published

Gerben van Loon — Thu, 01 Jul 2010 15:18:00 GMT

The results for the .NET 4 wave beta exams are there. Received an email from Microsoft this morning. I only took the 70-513: TS: WCF Development with Microsoft .NET Framework 4, I found it pretty difficult but apparently I managed to pass :-)

BizTalk vs AppFabric

Gerben van Loon — Mon, 28 Jun 2010 07:28:00 GMT

Lately I hear more and more people saying that BizTalk server has become obsolete. This because of the developments in the WCF4, WF4 and AppFabric space. While there is some truth in there, BizTalk server offers the following things today that makes it useful for several scenarios that are not in WCF4/WF4/AppFabric:

- Publish / Subscribe model (messagebox)
- Adapters (Legacy application access, non based WCF ones)
- EDI, RFID, EDIFACT, ANSCI X12, HL7, HIPAA, RosettaNet and SWIFT support
- Configurable retry logic with send and receive of messages
- Flat file parsing
- Debatching of messages
- Business Activity Monitoring (BAM)
- Visual mapper (BizTalk mapper)
- Business Rules Engine (BRE)

The new things in the WCF4, WF4 and AppFabric space that where previously unique to BizTalk:

- Host out of the Box: Scalable, reliable, load balancedmonitorable
- Durability and persistence
- Suspend / resume functionality
- Extensive set of tools provided for IT pros to administer and track execution
- Health monitoring
- Failed-message management
- Message tracking + archiving and purging
- Long running and atomic (ACID) transactions with compensation mechanisms

So I think for quite some projects that I’m doing I’ll pick AppFabric now as the preferred platform where previously I would choose BizTalk. I prefer AppFabric since the development frameworks (WCF + WF) are more close to the core .NET platform.

Let me know if you have any remarks or things to add.

What's new in WCF4?

Gerben van Loon — Wed, 21 Apr 2010 07:51:00 GMT

Not showing up on the first page if you search around, but I think it currently is the best article around that shows the new stuff:

http://msdn.microsoft.com/en-us/library/ee354381.aspx

DevDays 2010 WCF Best Practices session slides

Gerben van Loon — Tue, 06 Apr 2010 07:19:00 GMT

Wow, I was really surprised when starting my session. Had the last presentation slot at the last day so didn't expect a whole lot of people. But it turned out to be pretty different, there actually wasn't enough space for everybody, people had to stand in the back.

Find the slide deck of my presentation below if you want to re read some stuff.

Also I blogged about some of these practices with some more detail as in the slidedecks. You can find them here: http://bloggingabout.net/blogs/gerben/archive/2010/02/01/wcf-best-practices.aspx.

Just let me know if you have any additional questions!

Spolsky stops blogging

Gerben van Loon — Mon, 22 Mar 2010 07:47:00 GMT

Always enjoyed reading his articles and books. His writing really changed how I approach things. Good content combined with a great sense of humor.... sniff... sad to read that he stops blogging....

http://www.inc.com/magazine/20100301/lets-take-this-offline.html