Erwyn van der Meer

You probably already noticed that Microsoft has enabled access to the source code of parts of the .NET Framework 3.5. Here is the announcement of availability in case you missed it. And there are detailed instructions on how to enable this as well.

This morning I started stepping into the Windows Presentation Foundation. Before I knew it I was deep down into the internals of dependency properties and the Dispatcher object. The dispatcher has a nice old-fashioned message pump. I stepped into the dispatch of a message call onto a non-UI thread. Even though I already knew this, it is still startling to see that this translates into old-fashioned User32 Windows messages being send.

What really surprised me was to see a bit of VB code in a file that is called ExceptionFilterHelper.vb. According to the comment this file was needed because C#, unlike VB, does not support exception filters. However, the filter always returns true. According to the comment in the source code it is a "dummy stub to keep compiler happy, will not be replaced".

Something weird is going on here. The debugger shows I am in WindowsBase.dll!System.Windows.Threading.ExceptionWrapper.TryCatchWhen. The source code file shows the class MS.Internal.Threading.ExceptionWrapper.TryCatchWhen. Reflector can only find the former class. The disassembly of this class shows source code that is very different than what the MS source code server gives me.

This doesn't make much sense to me. Is the source code for this class deliberately hidden or is there some error in the source code lookup?

Since I haven't had my lawyer take a look at the license agreement that governs the source code access, I will not add a screen shot of the VB code ;)

Best wishes to everyone for 2008!

In the first day of the new year, I've released version 0.9.0.0 of my Flickr Meta Synchr tool. You can always find the latest version on CodePlex.

I had been working on this new version for a while now, but didn't get around to finishing it. Improvements in this new version:

• Added much better activity logging. The activity log can now be shown in an additional window and is persisted to disk.
• Added the option to match pictures on title and filename. This is useful when images have been timeshifted and cannot be matched on date taken.
• Bug fixes. Improved stability when corrupt image files are encountered. Fixed GPS roundtripping bug.
• Should run better on 64-bit versions of Windows XP and Vista.
• Solution is now built in Visual Studio 2008 without the need for any additional WPF extensions.

Here is a screenshot of version 0.9.0.0:

Microsoft used the keynote in the SOA & Business Process Conference in Redmond to present its vision on the future of Service Oriented Architecture on the Microsoft platform. That vision and the wave of technology that will come with it, is codenamed "Oslo".

MVP Charles Young has a solid write up in a blog post called "Microsoft 'Oslo' - the vNext SOA platform". No need to repeat all that here.

Long time Microsoft watcher Mary Jo Foley is very critical in her post called "Microsoft talks SOA futures but not dates". Mary Jo ends with "Microsoft has been struggling to prove to the market that it has a real SOA strategy. While the Redmondians are talking the right talk, the company is still a ways away from walking the SOA walk. Will customers wait or run off with other SOA vendors before Microsoft rolls out more than just a piecemeal SOA strategy?"

"Oslo" is obviously a Grand Vision. It will take a couple of years before this next wave of Microsoft technologies will ship. I thought that after the Longhorn reset/WinFX debacle and the "Whidbey" delays, Microsoft would not attempt to align so many technologies again in the future. But it is! "Oslo" comprises of at least:

• BizTalk Server "6"
• Visual Studio "10"
• .NET Framework "4"
• Systems Center "5"
• BizTalk Services "1"

Some of the stuff presented reminded me of the grand WinFX, especially WinFS, vision that Microsoft presented at PDC03. We all know that WinFS never RTM-ed, despite enormous effort (many, many man years) put into it by Microsoft. Especially the term "Universal Editor" for the "Oslo" integrated modeling tool gave me the creeps. Sounds too much like: One tool to rules them all. One tool that spans the entire application development lifecycle: from its inception to its deployment.

Here are some screenshots from the new "Universal Editor" modeling tool that was demoed during the keynote:

Scott Guthrie did a major announcement on his blog yesterday: Microsoft will be releasing the source code for most .NET Framework libraries with the release of Visual Studio 2008. There will even be integrated support for debugging into framework classes and on-demand dynamic downloading of source files and debug symbols in Visual Studio 2008.

This is great news for .NET developers and a major step forward for Microsoft in my opinion. In and by itself it is enough reason to warrant an upgrade to Visual Studio 2008. In fact, I can think of no reason to keep using Visual Studio 2005 after the release of VS2008.

The source will be released under the Microsoft Reference License which basically means you can view and debug but not change or reuse the source code.

If you want a more liberal license you can look into Rotor aka the Shared Source CLI. Rotor was Microsoft's first effort for open sourcing a .NET CLI implementation. But Microsoft does not guarantee that Rotor has exactly the same codebase as the real .NET Framework.

Check out the full details and screenshots of VS2008 integration on Scott's blog.

Today I encountered a problem with accessing the metadata for a WCF service that was deployed on a Windows Server 2003 machine.

The WSDL part worked just fine for the metadata exchange endpoint (url?wsdl, url?wsdl=wsdl0, etc.). These WSDL files refer to XSD files for the message types. Requesting these files (url?xsd=xsd0, url?xsd=xsd1, etc.) resulted in an empty response from the webserver. Checking the IIS logs indicated a HTTP 200 OK response with 0 bytes transferred. A very weird problem. Checking the config files did not lead anywhere.

Eventually I found a hint in a reply by James Zhang in this MSDN Forum post. The identity that is used for the application pool that hosts the WCF service must have the correct NTFS permissions on the %WINDIR%\temp folder. The identity that I used is a domain account. After setting the right NTFS permissions, the problem disappeared.

The funny thing was that this particular answer wasn't the answer for the original question in this forum post.

James Zhang does not indicate what type of permissions are needed, so I had to experiment a little.

First I added the account to the local Users group. This gives it special access permissions: Traverse folder/execute file, create files/write data, create folders/append data  on this folder and subfolders. This is not enough. Then I realized, the domain account is already implicitly a member of this group because the Users group contains the NT Authority\Authenticated Users group. Next, I duplicated the extra rights that the NETWORK SERVICE account had for the domain account. These are list folder/read data and delete permissions for this folder, subfolders and files. This was enough. But it doesn't seem very secure. Now the service account can access temporary files created by other accounts.

So I experimented a bit more. I tuned back the NTFS permissions for the service account on %WINDIR%\temp to list folder/read data on this folder only. This is just enough. This allows the account to see which files are in the temp folder, but it doesn't allow it to read the data in files that are owned by other accounts.

It is very unfortunate that WCF didn't give any clue about why it couldn't generate metadata in this case. It is also unfortunate that it needs just slightly more permissions that a standard user on the folder for temporary files.

Note that if you run your WCF service in an IIS application pool under the default NETWORK SERVICE account you won't run into this problem, because it has more than enough permissions.

PS: Best practices indicate you shouldn't deploy your services with metadata enabled. We will turn this off eventually. However, of course it should work if you do want to enable this.

After a long day and night of coding, I released version 0.8.0.0 of my Flickr Metadata Synchr tool on CodePlex this morning. I finally solved the long-standing problem I was having with the Windows Imaging Component (WIC) to update metadata. So this is the first fully-functional release of my application.

Functionality of FlickrMetadataSynchr v0.8.0.0

This is what the app does:

• It allows you to select a set of your photos on Flickr and a folder on your hard drive with images.
• It reads the metadata for both Flickr images and the local images. The metadata that is read is:
  
  • Title
  • Description
  • Author
  • Tags
  • Geo-info (GPS coordinates)
  • Date and time taken
  • Last update date and time
• It matches images on Flickr with local images based on the date and time taken.
• It determines on a per picture basis in what direction the metadata should by synced, i.e., which side should be updated, if any. Currently the most recently updated side wins. I am getting help from Timo Proescholdt for a better algorithm that will allow for a merge of metadata, i.e., a two-way synch.
• It updates the metadata on Flickr and in the local images.

This is a screenshot of the app:

Previous posts on this tool

• Here is more info on why and how you should store the metadata in your images.
• Read here how I was previously forced to use C++ to update metadata in images stored on the hard drive (I call these local images).
• Read about how CodePlex lost my source code.

Workaround for WIC problems

During my holiday in France in July I received e-mail from Robert A. Wlodarczyk who works at Microsoft. He pinged me to say that he had released new sample code to update metadata using WIC. Yesterday, I tried to incorporate similar code into my application and ran into the same type of problems as before.

Because his sample was working, I wasn't ready to give up again. I finally tracked the problem down to a threading issue. WIC is throwing strange InvalidOperationException and InvalidFormatException exceptions with messages like "Cannot write to the stream" when it is called from a background thread. My app is multi-threaded so that the UI doesn't hang when it is busy syncing.

After I got confirmation from Robert that WIC indeed suffers from a threading issue, I solved the problem with a work around. I now marshal the call to the code that uses WIC to update metadata to the UI thread using the WPF Dispatcher object. This causes the app to become non-responsive for small amounts of time during the update of local metadata. But that is better than a non-fully-functional app.

Fully functional, give it a try

So all is well that ends well. After finally getting WIC to work, I could do away with the C++ code that was causing me headaches ;) And my app now works on Windows XP again. You just need to have the .NET Framework 3.0 installed.

If you have images on Flickr and you have been busy tagging them, give my app a spin! You can always find the latest release on CodePlex. The source code is also available under a GPL license on CodePlex.

Installing is easy. You just need to unzip the ZIP-file, which contains three files, to a folder. Start the FlickrMetadataSynchr.exe file and you are done. The app remembers the last settings.

If you find any issues, please report them using the Issue Tracker for my app on CodePlex.

The Future

Even though the app is now able to sync metadata in both Flickr images and local images, there is always room for improvement. Here are my ideas, some of which are based of suggestions by people on CodePlex:

• Improve the synchronization to also allow two-way synchronization for a picture pair. I.e., one side doesn't have to win. For example, if the Flickr image has just the title set and the local image the description, the metadata should be merged.
• Add UI to see the match that is made by the tool and how it proposes to sync the metadata.
• Allow you to exclude images if the match isn't good.
• Allow you to overrule the sync proposal and sync the metadata in a different direction (on a per property basis) .
• Add UI to store multiple mappings between Flickr sets and local folders. Currently the app only remembers the last folder and Flickr set that was used.
• Add click-once deployment. That way the app can automatically check for new versions and update itself.

If you have any other ideas please post them at the discussions page for my app on CodePlex.

If you are interested in the history of LINQ to SQL and how it works internally, go watch Charlie Calvert's video with Matt Warren and Luca Bolognese.

The first 30 minutes of this video are primarily about the internals. The last 10 minutes focus on the history of LINQ to SQL and how it relates to its "predecessors" Cω  and ObjectSpaces. Matt and Luca also tell how Erik Meijer and Anders Hejlsberg got involved.

Related posts on this blog:

• .NET Language Integrated Query Framework (July 17, 2005)
• History of LINQ to SQL (May 31, 2007)

Back in January 2006 I wrote a blog post about a way to add input validation to declarative query string parameters in ASP.NET 2.0.

This involved adding code to the code-behind of each ASP.NET page to hook an event handler. This quickly becomes tedious if you have a lot of pages.

Will Shaver has come up with a better approach. He derives a custom class ValidatedQueryStringParameter from the ASP.NET class QueryStringParameter.

Check out the download on his blog post. It contains a more complete ValidatedQueryStringParameter class that supports other types besides Int32 like DateTime and Boolean. The QueryStringParameter already supports a Type property (of type TypeCode) to specify the type for the value of the parameter, so you can use it from markup like so:

<cc:ValidatedQueryStringParameter Name="CategoryId" QueryStringField="CategoryId" DefaultValue="0" Type="Int32" />

PDC07 has been postponed indefinitely. Dennis already warned me just before MIX07 that Microsoft wouldn't have anything to announce at the PDC after the MIX conference.

No new date for a PDC has been given, yet Microsoft already announced the date for MIX08! It will be held March 5-7, 2008 in Las Vegas.

So when will the next PDC be? The official statement by Microsoft is that it will be when they are ready to announce the "next wave of platform technologies".  The .NET technology is still forbidden from entering large parts of the Windows Operating System.

Will Microsoft attempt to pull off a synchronized wave of platform technologies  again? Synchronizing the release of

• Visual Studio .NET 2005, .NET 2.0 and SQL Server 2005
• .NET 3.0 and Windows Vista

proved to be mighty difficult for Microsoft to pull off. They only managed to release Windows Vista by cutting the dependencies between the OS and .NET. .NET 3.0 is only bundled with Windows Vista. Vista doesn't make use of it.

Vista even makes it very difficult to pull some things of using .NET. See my metadata API difficulty requiring me to use C++ and the fact that is is very difficult to create a decent Sidebar gadget using WPF.

The synchronization of .NET 3.5 and the ADO.NET Entity Framework failed. ADO.NET vNext has been delayed and will be released after the LINQ wave.

However, there is hope glimmering on the horizon. The new CoreCLR used by Silverlight 1.1 is able to co-exist with other CLR versions in one process. Silverlight will be a great technology for the Vista Sidebar. ASP.NET Ajax was released out-of-band. Microsoft has said that the next revision of the full CLR (remember .NET 3.5 still uses CLR 2.0) will also have this feature.

Martin Fowler notices a trend: interest in the .NET platform is declining. Ruby is all the rage. He has some recommendations for Microsoft on how to better collaborate with the community instead of going its own way. Martin noticed how Microsoft created its own unit testing variant that is incompatible with NUnit.

Change is visible in some parts of Microsoft, for example Microsoft is embracing dynamic languages, but boats the size of aircraft carries can only slowly change course. There are no clear signs from upper management at Microsoft that they are indeed changing course. The recent threats against the open source community over infringement on software patents seem to indicate that the old school closed-source thinking still has big strongholds within Microsoft.

Google is about to set a standard for offline storage for web applications with Google Gears. Even better, it's open source. Does Microsoft have the guts to embrace this technology instead of creating its own variant? I doubt it.

I am still pretty excited about the .NET platform. Especially about the LINQ wave of technologies coming with .NET Framework 3.5 and Silverlight. But I am also thinking about looking into Ruby on Rails more and more.

Not many Microsoft employees dare to publicly refer to WinFS as a black hole. Matt Warren does in this blog post. He gives us great insight into the origin of LINQ to SQL and how it managed to survive. He ends with:

So, for the sake of LINQ and the customer in general, we took up the OR/M torch officially, announcing our intention internally and starting the political nightmare that became my life for the next three years.

I briefly talked with Matt at PDC05. It was pretty clear already that he had had a big role in the development of LINQ to SQL even though the public presentation was done by Anders Hejlsberg. His blog is pretty quiet, but Matt is much more active on the MSDN Forums answering questions about LINQ to SQL.

WPF comes with great support for animation using XAML without needing to code this in for example C#. With Silverlight (fka "WPF/E") you can also do animations from XAML.

If you want to perform custom animations in code that you can't do using XAML, you need timers. In the full blown WPF you have several options, e.g., System.Threading.Timer, System.Timers.Timer and System.Windows.Forms.Timer.

You normally provide a callback that gets called when the timer elapses from a background thread. Properties on WPF objects can only be set from the foreground thread, so you have to queue a call on the UI thread to perform the actual animation. You can do that by calling the Invoke or BeginInvoke method on the System.Windows.Threading.Dispatcher class. You can access the correct Dispatcher instance to use through the Dispatcher property on the UI element (*).

Another option in WPF is to use the Rendering event of a CompositionTarget instance. In that case you get called when WPF is ready to render a frame. The frame rate depends on CPU speed, GPU performance, graphics complexity and other factors, so it fluctuates. This means that the interval after which you get called also fluctuates. However this is great for some scenarios.

In the current Silverlight 1.1 alpha your options are more limited. The CoreCLR libraries do have a System.Threading.Timer, but there is no Dispatcher class to delegate work to the UI thread. So it is useless for doing custom animation. In the source of the Monotone sample by Lutz Roeder I found there is an HtmlTimer class in Silverlight 1.1. This class is undocumented and marked obsolete. Visual Studio shows a warning after compilation:

'System.Windows.Browser.HtmlTimer' is obsolete: 'This is not a high resolution timer and is not suitable for short-interval animations. A new timer type will be available in a future release.

Lutz shows how to use an HtmlTimer in his sample. HtmlTimer has a Tick event. Any event handler that you wire-up to that event gets called from the UI thread. So that solves the problem for the time being.

When I tried to Google for more info on HtmlTimer, all I found was this blog post by Mike Taulty which mentions this class in passing.

(*) In fact any .NET class that derives from DispatcherObject has a Dispatcher property.

Via Brad Abram's blog: Microsoft has made available a Silverlight 1.1 Developer Reference as a beautiful poster. Go check it out to get an overview of what Silverlight is all about.

The only thing missing is an image of a penguin ;)

If you haven't heard the news yet, you must be living under a rock ;) There is a new CLR in town. Silverlight (fka "WPF/E") 1.1 comes with its own CLR. And the best news is that it runs both on the Mac and the PC.

Go check it out at silverlight.net.

One of the greatest achievements is that the download is a mere 4 MB. If you have a reasonably fast Internet connection you can download and install it in under a minute. Try that with .NET 3.0! That will take at least an order of magnitude longer to install.

Before the announcment at MIX07, I wasn't sure if Microsoft would be able to pull this off. They have certainly gone beyond my expectations.

This is what the Silverlight directory looks like after the install:

It shows you this CLR is completely separate from the standard CLR 2.0 runtime and has no dependency on it. This CLR has 2.1.x.x as version number. It has no Global Assembly Cache (GAC). You can see that support for dynamic languages like Python and JScript is included. Support for Ruby is in the works.

Go listen to the Channel 9 interview with Scott Guthrie if you want to know how Microsoft succeeded in trimming down the CLR 2.0 and the Base Class Libraries to this size.

The Silverlight .NET assemblies have the same format as standard .NET assemblies, so you can view them using Reflector. The type system is the same, so Silverlight supports generics. It will also support C# 3.0, VB 9.0 and LINQ.

These are exciting times for the .NET world. The reach of .NET has been substantially increased. Not just because the few percent of Mac users can run .NET applications now, but because it is such an easy deployment for Firefox users on the PC. And soon for Opera users as well.

Microsoft has announced that the little cross-platform, cross-browser cousin of Windows Presentation Foundation will be called Silverlight. This technology which was first announced at PDC05 was codenamed "WPF/E".

Tim Sneath has the best list so far of the features and power of this "Flash killer" technology. Microsoft doesn't ever call Silverlight a Flash killer, but the overlap in feature set is so large, that it cannot be viewed as anything other than a direct Flash competitor.

However, I do believe that Silverlight leapfrogs Flash in a couple of ways. The programmability and ease of use is better than Flash. You can build Silverlight sites using just Notepad if you want. The direct integration of the Silverlight DOM (Document Object Model) with JavaScript in the browser and the ability to create Silverlight UI elements on the fly with the createFromXaml method is a killer feature.

Tim Sneath mentions a secret number 10 feature on his list:

"Ah... #10. I can't reveal this yet - there's a big surprise up our collective corporate sleeve that will be announced at MIX. I hate to hold back on you, but anticipation is part of the pleasure, as my mother used to tell me as a child when I was waiting impatiently for Christmas to come!"

Could this be the .NET programmability that was previously speculated about? Soma spills some more details in his announcement:

"As I mentioned, this Silverlight announcement at NAB is only part of the story, the rest will be unveiled at MIX including details about how Silverlight is a core component of Microsoft’s broader .NET platform."

I commented previously on Robert McLaws' blog that I didn't think that Microsoft wasn't going to release a lightweight crossplatform CLR for Silverlight programmability. But I also speculated that Microsoft was working on a bigger crossplatform CLR based on the .NET Compact Framework.

What I am pretty sure about, is that Microsoft will announce ASP.NET controls that will allow you to very easily integrate Silverlight on your web pages and to expose dynamic data as XAML to Silverlight controls. I.e., AJAX on steroids UI-wise.

Adding ASP.NET to the mix shows that there is no direct need for a CLR on the client in order to enable C# or VB.NET programmability: coding in C#, compiling to IL and converting that IL to JavaScript on the fly! Prototype efforts by Nikhil Kothari with Script# show that this is quite possible. Check out Nikhil's example.