CaPo on Becoming an Integration Architect

The foundations of Service Oriented architecting

2009-01-05T20:31:06Z

A lot has been said about the foundations of Service Orientation. But still, way to often I see people deciding to use SOAP/XML and thinking they are doing Service Orientation. You might recognize the following examples I see as getting Service Orientation wrong:

Services are very technically oriented: service-designers know they should do something on business alignment, but say that the business doesn’t get it and using web service standards is the best we can.
Services are point-to-point interfaces: ‘services’ are designed on the fact that some information needs to go from system A to system B and therefore a service needs to be created. They then call the service something like the InterfaceInformationService or the SetSomeDataService. Also recognizable by the fact that they always have one operation.
Services with little responsibility: services are designed around one function and are therefore responsible for only a small bit of data.

I think that, when you talk about architecting Service Oriented solutions, the architecting part is the hardest part. There’s nothing special about that. It’s just decomposing systems into subsystems with clear coherent responsibilities and logical interfaces. But the decomposition is at a higher, abstracter level, which makes it harder. When decomposing applications, you assign responsibilities to concrete components. When decomposing systems, you assign responsibilities to abstract services. But still you need to be concrete and accurate about the responsibility of the service.

Clearly defining the responsibility of the service is the best way to get a service contract right.

Imagine you are designing an Order Management service (OM) in an environment where there is an Customer Management service (CM) available. Now you have to decide whether your OM knows the CM. Whether you want that is based on the way you assign responsibilities. Now let’s say that it’s the CM’s responsibility to manage a customers credit rating. Your options:

If you make the OM responsible for validating the customers credit rating before accepting an order, your OM will need to know the CM.
You could also decide that it’s the service consumers responsibility to perform a credit check (the service consumer could very well be a process manager) and the OM doesn’t need to know the CM.

The first option is more strict. There is now way to create an order if the customer doesn’t have a required credit rating. The second option is more flexible, the OM can even handle orders for customers unknown to the CM (imagine merger scenarios). I think that no one option is better than the other, it depends on where you assign responsibilities and that is driven by architecting requirements.

More generally I think that in any service interaction pattern there are three main parties:

The service consumer. The service consumer knows the service provider. It knows it’s contract, it’s policy, it’s endpoint location. Ideally, the service provider doesn’t know the consumer.
The service provider. The service provider defines the contract and provides the implementation. It’s implementation might (and often will) include a datastore.
Other services. These services are used by the service provider to delegate work that’s necessary to realize the responsibility of the service. The service provider plays the role of service consumer to these services.

Deciding on the interacting pattern between the parties is the key to good service contract design. So far, this looks pretty straightforward, isn’t it? Well, it’s often harder then it looks. It’s easier to directly contact a service to get some work done and if your not careful on that, soon you get a spaghetti of communication services. In following posts I dive deeper into the subject “Deciding on the contract”.

I really value any feedback you have on this topic. I hope it sparks some discussion.

Restarting my blog

2009-01-01T20:10:11Z

I haven’t been blogging for over 2,5 years (except for one off topic post). This was partly because I didn’t bother, but the main reason was that I didn’t have a goal, a story I wanted to tell the world. I’ve been doing a lot of stuff around integration and SOA/BPM, ranging from guiding BizTalk development to developing an enterprise integration strategy.

2008 to me was the year I dived into the theory of architecting and architecture on one side and understanding the core foundations of Service Orientation and Business Process Management on the other side.

On the theory of architecting I was greatly inspired by the work done by the Software Engineering Institute. I dived into Software Architecture in Practice (2nd Edition) and was lucky to receive an eight-day training from the writers through Logica, my employer.

On SOA I was greatly inspired by the work done by Thomas Erl.

These sources gave me a lot of background on the complex, and still evolving, playground of the Integration Architect.

For 2009, I’ll take up the next challenge, bringing all this theory into practice. Now, I do think that I already bring a lot of the theory in practice. In my day to day job, I work in an environment where the integration we do is greatly driven (and constrained) by existing systems, but with a vision of having a Service Oriented environment in the long run.

In the past year I’ve learned a lot on what the real key features of integration architectures are and what kind of decisions are important to make steps toward Service Orientation.

My goal for this blog is sharing my view on these topics with the world, sparking some discussion. I already have some stuff to talk about, so I guess in the short run I will sound like a teacher telling his story, In the long run I see this blog more as the diary of a tourist touring an environment that is only roughly explored and documented.

As a side note, I don’t write today as a result of some new year blues with big new intentions. It’s more the holiday season with a lot of days off that allowed me to think about what I want to achieve. But still it is a good day to restart a blog ;-).

An accident waiting to happen

2007-01-04T04:10:00Z

When I saw a post some weeks ago about 5 things I didn't know about Robert Scoble I could only hope it didn't reach me. Since then I saw dozens more until the last couple of days it hit our bloggingabout.net community. Just over the last two days many of us wrote about our private 'secrets'. And thanks to Edward it's my turn now. A good reason to reincarnate my blog.

So here we go:

I've been a major (let's say MAJOR) Guns n' Roses fan since the day I switched on MTV and saw Paradise City. Which BTW is still my favorite GnR-song by far. Not so interesting in itself, but people that know me, can hardly think of me as a:
1. Long-haired
2. Head-banging
3. Black t-shirt wearing
4. Hard-rock fan
I could say this would be 5 things by itself but the stories continues.
I learned programming on the Commodore 64. I was the book-buying kind that used to type evening after evening on a program that in the end didn't do what I expected it to do and was disappointed. The funniest thing I remember is Hobbyscoop, a radioshow about programming broadcasted on Sunday evening. The show always ended with the broadcast of the bytes of a compiled program that you could record on tape and then try to play (which I have never seen anyone get to work, BTW). The fun part was that I was always in the car with my parent travelling home from my grandmothers house, because it was right after football ended on the TV. So we were sitting in the car listening to peep-peeeep-peep-peeeep-grgrrgr-peep-grrrgrr (imagine the sound a modem makes when connecting) and back then there wasn´t much else so we just had to listen ;-).
My wife always was unhappy with my little belly, which I found strange, because it wasn't so bad. Until the little belly went into a somewhat more descend belly and once I found myself laying my hands upon it to rest. That was the day I decided to loose weight (I guess that day is almost 4 years ago). I did that by starting running and in combination with a normalized eating pattern (which is, no candy but three descend meals a day). I lost 10 kilos in 8 weeks. I still run 3 times a week and I again eat everything I want and haven't gained a kilo since then. I once tried to run a marathon but that failed for several reasons, including major illness a couple a weeks before the event. I now focus on shorter distances, but I will once finish a marathon. Oh BTW, my wife is still unhappy because I now run a race every 3 or 4 weeks and am gone for most of that Sunday. Well, it's hard to please them ;-).
I don't think there is one line of code gone into production in 2006 that was written by me. I merely just talked, written documents, supported testing, building environments and lots more, all well received (at least that's what my customer still tells me), but hardly any programming. And I haven't really missed it too.
I can't end this list, without introducing my lovely children. Stefan (6) and Julia (3) really changed my life. A couple of years ago, work was everything to me, everything was build around it. Now, my family is in the center and I build everything around that. I enjoy coming home to eat and play with them more than anything else I do. If I come home from work, it´s my task to shower them, read them a last story and then bring them to bed and I love doing that every day of the week.

Mmm, now the hard part, I'm supposed to tag someone. They won't like it but I'll tag Robert, Wellink, Edward and Fijsjan. I guess there will be some swearing going around in a crowded room in Rotterdam ;-).

Office 2007 RTM

2006-11-06T07:12:00Z

Here it is: http://www.microsoft.com/presspass/press/2006/nov06/11-062007OfficeRTMPR.mspx

Now, give me the bits ;-).

Deploying a BizTalk web service using setup project

2006-09-24T03:14:55Z

Here’s another thing we bumped into recently that took a lot of time to find and fix.

In a project in which I’m sideways involved a BizTalk generated WebService was deployed using a setup project. These projects by default include all dependencies, which are all BizTalk assemblies. Deployed this way, our webservice did not work, without any error logged in either the HAT or the event log. We did see that BizTalk was involved by incomplete messages in the HAT. Turned out that when we removed all dependency assemblies from the bin folder of the webservice it worked great. We choose to remove dependencies from the setup project and that fixed it.

Don’t close the original DataStream in a Custom Adapter

2006-09-24T03:07:58Z

I have written a custom adapter over two years ago that worked a charm back then. Recently I had to use it again in a little project. It worked, but not when an error occurred on transmission. In that case it failed with the following errors in the eventlog:

“The Messaging Engine encountered an error while suspending one or more messages”

and

“The Messaging Engine encountered an error while deleting one or more messages”

After attaching the Visual Studio debugger to the BizTalk Host process and setting breakpoints on all Exceptions, I saw it failed on a closed stream. As clean as I am, my adapter closed and disposed everything inclused the stream I got from GetOriginalDataStream. And that is not allowed. After I fixed that, everything worked great!

Back on blogging?

2006-09-24T01:59:24Z

I haven’t written anything here for a long time, but since I installed Vista and Office 2007, I should also try the blogging service. I know, I know, half the world tried. In fact, I did too, but didn’t get it to work because of a bug that is supposed to be fixed. In the meantime I was involved in two nasty BizTalk issues, which I will blog about later on.

Windows SDK February 2006 CTP installation trouble

2006-04-17T23:21:00Z

I've been trying to install the Windows SDK CTP versions numerous times now, failing every time. None of the workarounds/resolvements I found on the net did work for me. I found another way of resolving the issue though, so just for some search engine juice, here's my solution.

The error I received uptill now was:

Installation of the "Microsoft Windows Software Development Kit" product has reported the following error: Fatal error during installation.

My enviroment is as follows:

I run Windows Server 2003 R2 in a Virtual PC session. I have bound the Windows SDK image to a drive with Virtual Daemon Manager on the host and bound that drive to the guest session.

I ran a verbose install:

d:\setup\msiexec /package WinSDK-x86.msi /quiet /lv c:\out.log

When checking out the log I found the following error at just around 80% of the 30 MB log file:

"invalid digital signature" "WinSDK-SDK_WinFX_DOC-common.8.cab"

I resolved the issue by copying the entire setup disk content to a local folder in the guest OS and run the setup from there.

BizTalk 2006 RTM

2006-03-23T22:06:00Z

This post from Don Box really amazed me. Didn't think it was quite that far. Anyone to confirm?

In any way, you might want to check the webcast calender for which an update on RTM is also announced.

What kind of architect am I?

2005-12-13T10:34:00Z

When I was young I wanted to become an architect ;-). But whenever this goal came closer, the target became more obscure. What is an architect? What I thought an architect did, seemed more a software designers function, while architects did infrastructural stuff and application selection and more. Since a while I know that Enterprise Architects, Infrastructure Architects and Application Architects each do high-profile work but require very different skills. Yet, in my projects I'm oftenly faced with challenges that requires bits of all skills. Which left me wondering for a while which of the three I actually am or will become.

But now Simon Guest proposed a good way to lighten up the discussion. It's not black and white, I'm floating somewhere in the middle. But now the question still remains where I should place the dot. Which is something I will think about...

Building my personal radio station

2005-11-18T21:51:00Z

Brilliant,

build your personal radio station in seconds. You can even build multiple within minutes.

[via Dennis Pilarinos]

WinFX November CTP

2005-11-18T21:48:00Z

The November CTP of WinFX is available. Guess that this is the first version to run on .NET 2.0 RTM. Jevdemon has all the links.

Code snippet library

2005-11-15T10:37:00Z

Just came across this source of VS2005 code snippets. Still a little empty but looks promising. Anyone knows better sources for code snippets? I saw Don Box using snippets for Indigo bindings but I can't find them. Anyone?

BPM, reinventing the UI every time?

2005-11-09T22:19:00Z

I'm currently involved in introducing a workflow package within my customer's organization. The last couple of weeks I had numerous discussions with various persons inside this company about the way end users would interact within these processes.

The point that arose everytime around is they certainly don't want to rebuild interfaces they already have. SAP screens, Oracle Forms, etc. On the other hand they want to give users an integrated experience. A user executes a task within a workflow and he shouldn't take aditional steps to inform the workflow itself on what they did. Ideally he is presented the right screen when he selects the task at hand and information about the task done flows automatically between the workflow and the user screen.

Everytime these requirements come up, my brain start to run overtime ;-). It doesn't fit my view on architecture. I don't know of any technology that would allow me do this and when I look around in the service oriented world I don't see any discussions about reusing existing UI. You see a lot of samples with InfoPath and portals, but then I do start running down the path of rebuilding the UI for the processes.

What do you think, should I start discussing the fact with the customer that in a message oriented, workflow enabled world I should accept the fact that I can't reuse existing UI's? Or do you see alternatives I missed? Or should the customer and I drastically change views on the subject? Do you know of workflow tools that can reuse UI's more or less seamlessly?

On the last question, I do think the problem lies in the fact that we now view the workflow as the dominator, the initiator, in the process. The problem would shift if it would be viewed as more of a coordinator (which doesn't fits the products view, but that might be a different question). However, in that case the question would arise how the coordinator knows what is happening without requiring the user to enter 'workflow ids' all over the place. Interesting food for thought I think. Anxiously awaiting your reactions.

IBF, .NET 2.0 and VSTO 2005

2005-10-04T01:40:00Z

A few weeks ago I had an interesting demo script in mind where the Outlook extensions in VSTO2005 and IBF together would create an very compelling experience for a certain customer.

When we started trying to implement that, we immediately ran into the issue that IBF requires .Net 1.1 while VSTO2005 requires .NET 2.0 and Outlook can only host one framework version. Which basically meant, end of the idea, wait for next version of IBF. I thought.

When browsing the newsgroups today I came along this post that describes how to host IBF in .NET 2.0. You still need VS2003 for the Metadata Designer but at least you use both VSTO 2005 and IBF in one solution for rich experiences. Which is great news. Now if I only could find the time to build the demo I had in mind ;-).