Nathan J Pledger

Program.X musings from the Isle of Man concerning ASP.NET, in particular accessibility, web standards and neat ideas.

October 2006 - Posts

How do you solve a problem like Open Source?

A provocative title maybe, but I have become a little disenchanted with open source of late, due to my various involvement or consumption of such projects. In my experience, open source projects have been poorly managed, overly prescriptive in terms of implementation, poorly documented and are a breeding ground for politics.

Media Portal

In the absence of any commitment to the UK market, I 'upgraded' my UK Season 1 TiVo to an HTPC (Home Theatre PC) solution. I did my research and found 2 open source solutions and several commercial solutions. One of the open source solutions was instantly rejected because it ran on Linux - and unfortunately I cannot afford the tinker time that this would require, much as I would love to use Linux as a platform. So I looked at the other one, Media Portal.

Media Portal (http://www.team-mediaportal.com/) is a Windows application that is designed to provide the features of an HTPC; such as DVD Playback, Music collection playback and management, Recorded and Live TV and Picture galleries. It does a whole lot more, too, thanks to its open source community. It has many things going for it. It certainly has a devoted set of followers, a lot of whom have contributed in some form, be it coding, design, skinning, translation or just general feedback.

It is fantastic what has been achieved with this application. Working with video and sound is never easy, but combine that with the myriad of hardware combinations possible on the Windows platform, much of which has to be "reverse engineered" to some extent as API's are not readily available. The support forum is also excellent, and the guys there were very helpful in my initial construction project of my HTPC.

Obviously, this is a direct competitor to Microsoft's Media Centre application, which is actually currently shipped as an entire operating system on an OEM basis. The applications are very similar in terms of core functionality, but Media Portal is more extensible, due to the open nature of the application. I'm not sure how it is now, but certainly when I was testing it I was amazed at how it was implemented. It ran as an application, so if you needed to use your PC, you had to close (or ALT+TAB out of it, resulting in the usual clunky black screens while DirectX or whatever video layer was being used was working) the application. This also stopped any chance of scheduled recordings being recorded. Microsoft, on the other hand, have theirs running as a service, which the client UI connects to. Therefore, no lost recordings. Even if the UI crashes, mid recording, the service is still going and no disruption is evident.

This issue has been highlighted, and the problem that rears its head here is change control. In a diverse community of developers, how easy is it to call a checkpoint and perform fundamental and breaking changes that would implement this functionality? Is it possible to expect developers to then revise their code accordingly? In a corporate environment, this would be simple: you get paid to do your job; but in a jobby scenario, it becomes much more difficult to control. So I wonder if we will ever see this particular change. As it happens, this is one of the main reasons why I dropped Media Portal in favour of Windows Media Centre.

I have contributed to the Media Portal project, as I was asked to provide my feedback on the usability of the program, and the configuration in particular. At the time of testing, the configuration was dysfunctional and was very difficult to understand, let alone configure in some places (TV Guide, encoders, etc.). My argument was that people should be isolated from this, as 90% of users wouldn't need to know about it. Obviously, this is an open source project, and one of its main strengths is the ability for advanced users to tinker with their set-up. It is the task of effectively marrying these two requirements that is the key. So, I developed an article (which I can provide on request) on 11-foot usability and was asked to help identify the improvements that could be made in the configuration section of the application. Whether my comments were taken on board is irrelevant, by the way! After submitting these recommendations, I was asked who commissioned the work. Apparently I had inadvertently walked into a political situation where senior developers in the team had disagreed somewhere along the line. This ensued with my work being disregarded to some extent, and a series of arguments about why we were even looking at this area. I will say that I worked with two people on this usability study, both of whom were most professional and helpful. I resolved that I wasn't prepared to apply time to a project that was, essentially disorganised and dysfunctional, so moved on. Besides, by this time I had installed and tried MCE and found that suited most requirements.

nHibernate and Castle ActiveRecord

I was recently in the market for an alternative data layer, to use instead of the CSLA 1.4 implementation we previously used. I was recommended by a friend to look at ActiveRecord, nHIbernate and Spring.NET. Starting with ActiveRecord, I chose a suitably low profile project to try it with. I have to say it was an absolute joy to use. Simple, quick. Then I approached an existing project I was working on that used a "real life" database, that is, a database that hasn't come straight out from a university lecture. You might have worked on one of these yourselves. They are evident by their use of character fields for number fields, composite keys and human-defined keys.

Then I started to experience problems. I could not understand the inflexibility of the initialisation procedure, which effectively required the consuming application to "know" about the classes and/or assemblies it was likely to use. I didn't agree with this, and asked why ont he forums and it took many attempts to extract the reason from them. I wasn't saying it was bad, or good; I was just interested in the reasoning behind it. Then, gaining confidence, I hit a hurdle: composite keys. Active Record could not deal with composite keys effectively, or at all elegantly. So I started looking at nHibernate. This is the core of ActiveRecord, I figured if I could use the main engine, I could avoid the wrappers which may be causing my problem. nHibernate is an excellent piece of work, and no mistake. You'll know it was based on the Java Hibernate project.

However, the use of nHibernate was not so easy. The documentation was good, so long as you didn't want to do any debugging. I was recommended to turn on the log4net logging service; a simple task you may think. It would be, if any web site I consulted (including nHibernates own wiki on the subject and the log4net home page at Apache) gave correct and complete information. It took myself and my colleague to figure out that you actually have to initialise it such a way (not using the Basic Configurator), a forum post to figure out that you have to tell nHIbernate to turn on the SQL logging and then a whole load of experimentation to figure out how to format the configuration. By the time I'd got this working, I'd lost sight of the original problem: composite keys.

It seems that both projects don't agree with composite keys. Whether they are bad or not is entirely irrelevant. The fact is, they are used and therefore a data-layer is going to have to know how to deal with them. Support for them was disjointed and dysfunctional. Documentation on them was poor. I really do appreciate the help I received on the forums, but when a problem hits a wall and people can't help, the issue just dies and the post started sinking down the list.

And yet ....

... we have Linux. That epitome of open source software collaboration. It is amazing to think that such a complex and diverse product is with us today, considering the issues I have had with just three projects. So I guess the key is to understand how Linux succeeded and how come other projects have problems.

A benefit of Linux is that it was headed by Linus Torvalds from the outset. The start of any project is always its most vulnerable time, and the ability to have the confidence and direction at this stage is essential. Linux can be argued to also be very modular. The kernal can be unit tested and then compiled together, then the individual applications that contribute to the full package can be developed by other multi-person projects or single developers. The point is, Linux is not a success or failure due to the success or failure of a single project; it is a product of many individual projects. This, I feel, is where open source can often stumble.

The problems I have experienced with Open Source can be clearly identified as being political. There are no technical issues at stake here, only technical issues as a result of politics. Arguments amongst project stakeholders is a major distraction and problem. Steps need to be taken to limit the opportunity of these to affect the project. When I was involved in the Media Portal project, I had clearly walked right into a situation that had developed between two of the developers. Sure, differences are clearly going to occur, but lets keep it to ourselves and not involve people who want to contirbute to the project.

The issue with composite keys is, I feel, a result of the drive of the various projects. We know that there will always be ways to bind the UI to the data, and frameworks will be developed and revised to meet these objectives. But these projects should be a solution to an existing problem, not a solution for solutions sake. The lack of effective support for something as basic as composite keys is very disappointing. As a result, coding for them becomes clunky and difficult. It took me 2 or 3 weeks to start and finish with nHibernate and ActiveRecord. It took 4 days for me to code my own alternative. 

On the other side

There is another way. That is by spending money on a commercial product. Sure, a commercial product is entirely controlled by a single vendor, and updates can be slower. In the same period as I have been involved with these open source options, I have purchased products from a number of companies, most significant of which is Telerik. Telerik offer a product, radControls, wich includes a load of AJAX rich-client controls. Coding for them is easy, but there can be times when you need the support of the product team or a fellow developer who has had the experience. Posting to a forum is free, no matter the license, and members of the product teams do actively reply to posts. And if the problem can't be solved, then it is rarely left waiting, as you get the option to create a support ticket. Even users of trial licenses are able to escalate tickets. This results in effective responses that you can point at a payment and demand some service instead of lying frustrated at the feet of developers with no accountancy.

Conclusion

Open Source is a fantastic thing, lets not disagree there. It encourages innovation, personal and professional development and can be an excellent way to give something back. In no way do I disagree with the premise of open source.

What I have a problem with is that these projects are unnaccountable. A project needs to be accountable to ensure that it meets the problem it is intending to address and that the solution is still relevant and has not drifted or become irrelevant due to external factors. Open source projects are increasingly getting more and more elaborate and change control systems are being used to try and manage diverse development efforts within a single project body. These systems onyl work if there is direct control from above. While democracy is good, it isn't always useful. At the end of the day, if a decision is made, the person who made the decision has to be accountable to that decision. Whether this comes to pass as a result of a vote or a single arbitrary decision.

Support is a major issue for open source development. In the real world, developers don't have time to learn who things should be done, or how clever a particular framework is. They need to address a problem and address it quick. If a tool or framework is available, one should be able to get into it as soon as possible after a series of tests. A quickstart framework would be essential to this, where code can be literally cut and pasted into place. Any resources that are needed should be adequately documented and made available with the absolute minimum of effort. I'm not interested in searching for a particular version to compile with a particular version of a project. I want a single MSI that can be downloaded and installed. Kudos to the Castle Project for providing this! The documentation has to be suitably comprehensive and correct. There is no point in being clever when no-one knows how to use it. Trawling through papers and having to resort to blogs to address problems that should be covered on the core site is not convenient to most people.

Maybe projects could encourage a bit of funding by providing support services, whereby tickets can be raised on a pay-per-ticket basis or a subscription basis. Have a couple of people dedicated to addressing these tickets, and then have them address the forums that support the product.

I am willing to accept the limitations in man-power of open source projects, but they do operate within a commercial world. Unless they want to maintain their own eco-system of open source projects that use open source projects that don't make money or cost money, then open source will have to wisen up and embrace means of supporting and managing thier projects and people.