Jan Schreuder on .Net

Sandcastle, First glance

I managed to find some time to try the Sandcastle CTP on the CommandBuilder component I blogged about recently. Here are the observations I've made so far:

At the moment, it is all command-line based. So you need to know which application you need to run, what the arguments are and in which order you need to issue them. This is not that easy, but with the help of the information on the Sandcastle blog, you should be able to get it to work. You could also use the batch file created by Mikael Söderström. This works even better and it saves you from having to re-type all the commands. I also advise you to keep an eye on Mikael's blog, because he is building a GUI application to control all this. I had similar plans, but I think I'll just wait for Mikael to finish his. The completed CHM file can be downloaded here.

I was waiting for Sandcastle, so that I could build a help file for my provate set of class libraries. But looking at the way I can now use Sandcastle, I really think I need an NDoc like application with which I can control the entire process. Best thing would be to have an application with which you can define a project which contains the locations of all the assemblies and generated XML files (like NDoc) and which would then allow you to generate the help file.

Another feature of such an application would have to be an option to create batch files, NANT, MSBuild or CruiseControl projects. That would make it easier to integrate Sandcastle into your automated build proces. The way Sandcastle currently operates, it is a lot of work to get it set up for your project. But once it is set up, it works. The help files that it generates also look good, so I'm happy with the results so far. And this is of course the first CTP. There are plans to release more CTPs before the product reaches Sandcastle 1.0 RTW.

There are some others that have tried Sandcastle as well. If you're interested in what they have to say, then check out these blogs:

• Scot Hanselman
• P.J. van der Sande
• Szymon Kobalczyk's Blog

Generate documentation for .Net 2.0, Sandcastle CTP released!

It was due to arrive, and it finally has. Microsoft have released the first CTP of Sandcastle this weekend. I'm already busy installing it, and trying out stuff for my own set of class libraries. I will let you all know how I got along soon.

Some important links if you want to take a look at Sandcastle your self:

1. Sandcastle overview. A powerpoint sheet providing a high-level overview.
2. Creating a CHM using Sandcastle. A quick start in getting a CHM file
3. HTML Help Workshop. You'll need to download and install this before you can use Sandcastle.
4. FAQs about Sandcastle. This contains some interesting answers to questions about Sandcastle, as well as links to blogs of people who have already had time to look at this new product.

Generate documentation for .Net 2.0, update 3

I'm still waiting for the CTP of Sandcastle to be released. I've been expecting it for some weeks now. This was fueled by posts on the Microsoft Forums by Anand Raman, a Group Manager at Microsoft's development division. But more information about Sandcastle is still being released to the general public.

A new thread at the Microsoft Forum describes the pre-requisites for Sandcastle and the steps you need to take to generate a CHM file for a class library. The prerequisites for Sandcastle:

• .Net Framework 2.0
• For Chm generation download HTML Help Workshop.

One thing that immediately caught my attention is the fact that there seems to be no GUI for Sandcastle. When you read the steps that need to be taken to generate a CHM file, you will see that it's all command-line stuff. The Sandcastle team will provide a batch file (!!!) that will generate the CHM file. This is great news when you want to include generating the CHM file in a NANT or Microsoft Build process. But a GUI would be great to build projects and test the generation process. Kinda like NDOC. I'm already thinking on building a small GUI application that will convert NDOC projects to Sandcastle builds. When I do, I'll post it on my blog.

And then there's the new Sandcastle team blog on MSDN. At the moment, there's not much more information than what was already known about Sandcastle. There's a mission statement, though:

"Enable managed class library developers throughout the world to easily create accurate, informative documentation with a common look and feel."

I'm surprised that the information from the new thread at the Forum is not there. Still, it does show that there's a lot happening with regards to Documentation compilers. One thing that is interesting though, is the Powerpoint presentation explaining more about Sandcastle. The tension is building!!!

The author of Ndoc, Kevin Downs, appears to have stopped development on a version for .Net framework 2.0. As quoted from one of the forums:

"Once 'Sandcastle' is released, it is my belief that it will become the de-facto standard and that NDoc will slowly become a stagnant side-water."

Another reason for Kevin to abandon further development of NDoc seems to be the mail bombs and threats. These were made by people in the community, who felt that they had the right to demand a new version of NDoc. This despite the fact that NDoc was a private project. I feel it's a real shame that this happened. I can fully understand his reasons to stop working on a new NDoc. I would loved to have seen this great tool continue.

Free Download: Virtual PC 2004 & 2007

Microsoft has announced that Virtual PC is now available free of charge. I read this last week already, on several blogs and websites, but I still found that not everyone is ware of this yet.

Whether Microsoft virtualization technology is an important component of your existing infrastructure or you're just a Virtual PC enthusiast, you can now download Virtual PC 2004 absolutely free. Microsoft is also offering the free download of Virtual PC 2007, with support for Windows Vista in 2007.

From: Microsoft Virtual PC 2004

This is of course great news.

Generate documentation for .Net 2.0, update 2

Microsoft will have released the CTP for Sandcastle by next week, according to the post on this thread by Anand Raman, who is a group manager at Microsoft's Developer Division. There are also plans to include Sandcastle in the August CTP of the Visual Studio 2005 SDK.

Sandcastle Overview:

• Producesa quality, comprehensive, familiar MSDN-like documentation.
• Works with or without authored comments.
• Supports Generics and .Net Framework 2.0.
• Sandcastle has 2 main components (MrefBuilder and Build Assembler).
• MrefBuilder generates reflection XML file for Build Assembler.
• Sandcastle is used internally at Microsoft to build .Net Framework documentation.

This should mean that we will be able to produce MSDN-like documentation, using the tools used by Microsoft them selves. Sounds promising, right?

I will download the CTP as soon as it is available and let you all know if it was worth the wait, and if it is a good replacement for NDoc.

Custom SqlCommandBuilder

About a year ago, I felt I had to write my own SqlCommandBuilder object, because I had some issues with the one provided by the .Net framework:

1. You need to assign a SqlDataAdapter, a SqlCommand object with a select statement and a SqlDatabase connection in order for the SqlCommandBuilder to generate the Insert, Delete or Update commands.
2. The SqlCommandBuilder is registered as a listener for RowUpdating events whenever you set the DataAdapter property. So the commands are only generated when an attempt is made to update the database using the SqlDataAdapter.
3. The SqlCommandBuilder requires a round-trip to the server to get the required metadata. And since it is a listener on RowUpdating events, it does so ever ytime you call the DataAdapter.Update method, which has a major impact on the performance.
4. When you have an auto-identifier column as the primary key, the value inserted for this column is not returned by the insert command created by the CommandBuilder object. That means that any parent-child relations that use that primary key are not properly updated.
5. The Update and Delete commands have very large and complex "where-clauses" to ensure the correct row is affected. The commands are unaware of primary keys. So if there is a proper primary key, the updates and deletes are relatively slow.

So last year, I decided to "roll my own" command builder object. This custom made version only needs a database connection and the name of the table to create the CUD commands. The command objects are made available as properties of the object, allowing you to cache them. This eliminates the need for a round-trip for every call to DataAdapter.Update. In addition, it also creates the select command you normally need to add to the DataAdapter. You can find that post here, as well as the code I used at that time.

Today I finally found some time to convert it to .Net 2.0. With this version, it is now possible to do something like this:

void UpdateDataSet(DataSet dataSet)

{

    // Create the database connection

    using (SqlConnection connection = new SqlConnection("Server=servername;Database=database;uid=username;pwd=password"))

    {

        foreach (DataTable dataTable in dataSet.Tables)

        {

            // Create a new data adapter 

            using (SqlDataAdapter dataAdapter = new SqlDataAdapter())

            {

                // Use the custom command builder to create commands for the 

                using (LCommandBuilder cm = new LCommandBuilder(connection, dataTable.TableName))

                {

                    dataAdapter.UpdateCommand = LCommandBuilder.PrepareCommand(cm.UpdateCommand, connection);

                    dataAdapter.InsertCommand = LCommandBuilder.PrepareCommand(cm.InsertCommand, connection);

                    dataAdapter.DeleteCommand = LCommandBuilder.PrepareCommand(cm.DeleteCommand, connection);

                }

                // Update the dataset changes in the data source

                dataAdapter.Update(dataSet, dataTable.TableName);

                // Commit all the changes made to the DataSet

                dataTable.AcceptChanges();

            }

        }

    }

}

There's no need to create stored procedures for these basic actions on the database, as this custom command builder will create queries and command objects to take care of those tasks. I've added a number of constructors to allow the object to be used in transactions as well, as you can see in this sample:

void UpdateDataSet(SqlTransaction transaction, DataSet dataSet)

{

    // Create the database connection

    foreach (DataTable dataTable in dataSet.Tables)

    {

        // Create a new data adapter 

        using (SqlDataAdapter dataAdapter = new SqlDataAdapter())

        {

            // Use the custom command builder to create commands for the 

            using (LCommandBuilder cm = new LCommandBuilder(transaction, dataTable.TableName))

            {

                dataAdapter.UpdateCommand = LCommandBuilder.PrepareCommand(cm.UpdateCommand, transaction);

                dataAdapter.InsertCommand = LCommandBuilder.PrepareCommand(cm.InsertCommand, transaction);

                dataAdapter.DeleteCommand = LCommandBuilder.PrepareCommand(cm.DeleteCommand, transaction);

            }

            // Update the dataset changes in the data source

            dataAdapter.Update(dataSet, dataTable.TableName);

            // Commit all the changes made to the DataSet

            dataTable.AcceptChanges();

        }

    }

}

As Marc Jacobi was kind enough to point out, these examples will not work if you have parent-child relations in your dataset. Unless you are absolutely sure that the tables are in the correct order. ;-)

Further improvements in this version:

1. The SqlParameter objects which are created for the columns in the table are assigned with the following meta data: Column size, SqlDBType, Scale and Precision. The last two only for those types to which they apply.
2. A helper class, DataSetHelper, is added. This class uses the command builder object to persist data from a DataSet or DataTable object in the database.
3. Support for use in transactions has been improved.

There is still room for improvement though. The following is still on my to-do list:

1. Support for OleDb.
2. Improve the way meta data is retrieved for tables.

You can download a project with the code here. The project will build a class library which you can use in your project, as long as you read the disclaimers in the code and here.

I need to switch from C# to VB.Net...

I always claimed that the discussion about C# being a better language compared to VB.Net was nonsense. In fact, I argued that it really doesn't matter. But after almost four years of developing C# applications, I think I'm starting to favour C#. That may sound late, after four years, but I hardly ever did anything in VB.Net, until now.

In 1989, when 20Mb was enormous for a harddisk drive, I switched from Basic to C. Yes, it's not a typo, regular Kernighan-Ritchie C. It drove me bezerk for a few weeks, but after that I really felt at home. In 1996, I made the switch from C to VB6. The same thing happened, it took me some weeks to get used to it, but after that I was fine. When I made the move to C# a few years back, I had no problems. It just felt right from the beginning.

For my current assignment, the applications need to be developed in VB.Net. I've been doing C# projects for the last couple of years and I'm having a hard time making the switch again. I've even installed SharpDevelop to help me. For those of you that never heard of SharpDevelop; SharpDevelop is a free IDE for C# and VB.Net which features a pretty good conversion tool between these two languages. The first application I built was written by me in C# and then converted to VB.Net using SharpDevelop. This works fine, although you need to solve some stuff.

But what strikes me most is the clean syntax that C# offers, compared to VB.Net. Of cource, C# was written from the ground up for .Net, whereas VB.Net had to be somewhat compatible with old versions of VB. But have a look at the following piece of code which reads a text file and stores the informaton in a database. Assume MyDataSet is a typed DataSet and UpdateDataSet stores the dataset information in the database.

private void Process(string fileName, string connectionString)

{

    using (FileStream importStream = new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.None))

    {

        using (StreamReader importReader = new StreamReader(importStream))

        {

            using (SqlConnection con = new SqlConnection(connectionString))

            {

                using (DataSet ds = new MyDataSet())

                {

                    string line = null;

                    while ((line = importReader.ReadLine()) != null)

                    {

                        ProcessLine(line, ds);

                    }

                    UpdateDataSet(connectionString, ds);

                }

            }

        }

    }        

}

This C# code looks pretty clean and is quite easy to understand. But have a look at the VB version of this method:

Private Sub Process(ByVal fileName As String, ByVal connectionString As String)

    ' Using 

    Dim importStream As FileStream = New FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.None)

    Try

        ' Using 

        Dim importReader As StreamReader = New StreamReader(importStream)

        Try

            ' Using 

            Dim con As SqlConnection = New SqlConnection(connectionString)

            Try

                ' Using 

                Dim ds As DataSet = New MyDataSet

                Try

                    Dim line As String = importReader.ReadLine()

                    While Not (line Is Nothing)

                        ProcessLine(line, ds)

                        line = importReader.ReadLine()

                    End While

                    UpdateDataSet(connectionString, ds)

                Finally

                    CType(ds, IDisposable).Dispose()

                End Try

            Finally

                CType(con, IDisposable).Dispose()

            End Try

        Finally

            CType(importReader, IDisposable).Dispose()

        End Try

    Finally

        CType(importStream, IDisposable).Dispose()

    End Try

End Sub

It's not bad, but I think it doesn't look as clean as the C# version. And what I found even more annoying is the fact that VB.Net is not case sensitive, or that it doesn't immediately compile because I type a semi-colon (;) at the end of each line. Although I've been told that VB.Net afficinados actually like this about the language.

Another example I just cannot get used to yet. I use interface and base classes in my projects. In C#, I just list the base class and interface I want to implement:

public class NewImport : ImportSetting, IDisposable

In VB.Net I have to explain what they are?

Public Class NewImport

    Inherits ImportSetting

    Implements IDisposable

I know, it's probably just me. But I'm having a hard time getting used to these differences at this moment. In a few weeks time, I'll probably be OK. But for now, I think I prefer C# over VB.Net. Mind you, I'm not saying VB.Net is worse than C#. I just feel more at home in C#.

I wonder what other peoples experiences are, switching between these two languages.

Using section handlers to group settings in the configuration file.

Introduction

It is common practice to store application settings in the app.config (or web.config) configuration files. The larger the application, the more settings you will tend to have. However, as the number of settings grow for an application, so does the need to bring more structure in these configuration files. Let's look at a small configuration file:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="ConnectionString" value="data source=.;initial catalog=NorthWind;integrated security=SSPI"/>
    <add key="ImportDirectory" value="C:\Import\Inbox"/>
    <add key="ProcessedDirectory" value="C:\Import\Processed"/>
    <add key="RejectedDirectory" value="C:\Import\Rejected"/>
  </appSettings>
</configuration>

Because of the small number of settings here, the readability is pretty good. But it might be useful if you could structure these settings into two groups: Database and Locations.

The .Net framework offers a few options in the form of SectionHandlers that reside in the System.Configuration namespace that will allow you to create these groups. This article describes how to use these section to introduce custom sections. A zip file containing a demonstration project can be downloaded from this location.

NameValueSectionHandler

The NameValueSectionHandler is the handler used by most .Net developers, without even knowing it. This section handler allows you to add settings to the configuration file in the form of key value pairs. The example above shows you just that.

But as said, the idea is to group different types of settings into appropriate sections. To do this  you first need to change the configuration file. First step is to define the sections in the configuration file that we want to use. In the configuration file, add a new section with the name configSections. In this group you can define the sections and the handlers that will be used to handle these settings. Here is an example that uses the NameValueSectionHandler:

<configuration>
   <configSections>
      <section name="Database" type="System.Configuration.NameValueSectionHandler" />
      <section name="Locations" type="System.Configuration.NameValueSectionHandler" />
   </configSections>
</configuration>

Now that the sections have been defined, it is time to place the settings into the appropriate groups. The completed configuration file now looks like this:

<configuration>
   <configSections>
      <section name="Database" type="System.Configuration.NameValueSectionHandler" />
      <section name="Locations" type="System.Configuration.NameValueSectionHandler" />
   </configSections>
   <Database>
      <add key="ConnectionString" value="data source=.;initial catalog=NorthWind;integrated security=SSPI"/>
   </Database>
   <Locations>
      <add key="ImportDirectory" value="C:\Import\Inbox"/>
      <add key="ProcessedDirectory" value="C:\Import\Processed"/>
      <add key="RejectedDirectory" value="C:\Import\Rejected"/>
   </Locations>
</configuration>

The NameValueSectionHandler returns a specialised NameValueCollection which contains all entries after you call ConfigurationSettings.GetConfig(). Reading this information from the configuration file can be accomplished in the following manner:

private void ReadSettings()

{

    NameValueCollection db = (NameValueCollection)ConfigurationSettings.GetConfig("Database");

    labelConnection2.Text = db["ConnectionString"];

    NameValueCollection loc = (NameValueCollection)ConfigurationSettings.GetConfig("Locations");

    labelImport2.Text = loc["ImportDirectory"];

    labelProcessed2.Text = loc["ProcessedDirectory"];

    labelRejected2.Text = loc["RejectedDirectory"];

}

As you can see, this is a very simple way of grouping settings into appropriate sections. This example only contains a few settings, so grouping the settings is not something you would normally do. But with larger amounts of settings, grouping of settings becomes really useful.

DictionarySectionHandler

The DictionarySectionHandler is similar to the NameValueSectionHandler. The difference is that this handler returns a HashTable rather than the specialised NameValueCollection. HashTables are faster than the NameValueCollection when you have large amounts of settings.

You can use the DictionarySectionHandler in the same way as the NameValueSectionHandler. In the configuration file, you will need to use DictionarySectionHandler as the type for the section. Defining values for this handler type is identical to the NameValueSectionHandler. The following code sample shows how you can access the settings when you use the DictionarySectionHandler:

private void ReadSettings()

{

    Hashtable db = (Hashtable)ConfigurationSettings.GetConfig("Database");

    labelConnection2.Text = db["ConnectionString"].ToString();

}

As said, this is pretty similar to the NameValueSectionHandler with the difference of using a HashTable instead of the NameValueCollection.

SingleTagSectionHandler

<configuration>
   <configSections>
      <section name="Database" type="System.Configuration.NameValueSectionHandler" />
      <section name="Locations" type="System.Configuration.SingleTagSectionHandler" />
   </configSections>
</configuration>

<Locations 
   ImportDirectory="C:\Import\Inbox"
   ProcessedDirectory ="C:\Import\Processed"
   RejectedDirectory ="C:\Import\Rejected"
/>

private void ReadLocations()

{

    Hashtable loc = (Hashtable)ConfigurationSettings.GetConfig("Locations"); 

    labelImport2.Text = loc["ImportDirectory"].ToString();

    labelProcessed2.Text = loc["ProcessedDirectory"].ToString();

    labelRejected2.Text = loc["RejectedDirectory"].ToString();

}

Conclusion

Feel free to download the demonstration project attached to this article and see for yourself how these handlers work.

How to: Safely use FileSystemWatcher.Created events

On my current assignment, I was asked to build a simple windows service to monitor a directory on the server. Any file placed in that directory had to be moved to another location as soon as it arrived. The locations differed based on the file's contents.

Nothing special of course. I soon had a service running which used the following code to monitor that directory:

private FileSystemWatcher    _watcher;

private void SetupWatcher()

{

    _watcher = new FileSystemWatcher(@"C:\Inbox", "*.csv");

    _watcher.Created += new FileSystemEventHandler(_watcher_Created);

}

private void _watcher_Created(object sender, FileSystemEventArgs e)

{

    this.Process(e.FullPath);

}

The Process method would handle everything. It worked perfectly, except for the odd IOException telling me the file was in use by another process. The FileSystemWatcher object raises the Created event whenever a file is created. But it does so as soon as the first byte is written. So in fact, my application was telling me it couldn't process the file, because another process was still writing the file to the specified directory.

In one of the forums on TheScripts.com, I found a solution which I modified slightly to suit my needs. Here's what I did to solve the problem. I now have a method that will try to open the file exclusively. When the File.Open method fails, it will throw an IOException. The method is then able to tell me if the file is still being used:

/// <summary>

/// Check if the file upload has been completed.

/// </summary>

/// <param name="filename">The name of file to check.</param>

/// <returns>

/// Returns true if the specified file has completed uploading and 

/// is ready for processing.

/// </returns>

private static bool FileUploadCompleted(string filename)

{

    // If the file can be opened for exclusive access it means that the file

    // is no longer locked by another process.

    try

    {

        using (FileStream inputStream = File.Open(filename, FileMode.Open,

                    FileAccess.Read,

                    FileShare.None))

        {

            return true;

        }

    }

    catch (IOException)

    {

        return false;

    }

}

Using this method, the event handler for the Created Event is modified. A loop is introduced in which the application checks if the file is in use. If that is the case, the loop will wait for a short period of time and try again. To make sure the application doesn't retry indefinitely, a check is implemented to see if a predefined retry period has passed. If that is the case, an error is written to the eventlog and the loop is stopped:

private void _watcher_Created(object sender, FileSystemEventArgs e)

{

    DateTime fileReceived = DateTime.Now;

    while (true)

    {

        if (FileUploadCompleted(e.FullPath))

        {

            this.Process(e.FullPath);

            break;

        }

        // Calculate the elapsed time and stop if the maximum retry

        // period has been reached.

        TimeSpan timeElapsed = DateTime.Now - fileReceived;

        if (timeElapsed.TotalMinutes > MaximumRetryPeriod)

        {

            WriteToEventLog(string.Format(CultureInfo.CurrentCulture, "The file \"{0}\" could not be processed.", e.FullPath));

            break;

        }

        System.Threading.Thread.Sleep(RetryDelay);

    }

}

This method works really well. I hope it will be of some use to some of you.

Generate documentation for .Net 2.0, update

A few weeks back, I blogged about NDoc not being available for .Net 2.0. In that same blog post, I mentioned that Microsoft will be releasing a documentation compiler somewhere in June. This morning, I saw the following update in this thread on http://forums.microsoft.com/MSDN which states the following:

We did a code complete of "Sandcastle" CTP on June 15th and currently we are testing the tool building our .Net Framework documentation. We would like to release the CTP in Microsoft download center by next week. The perf in our testing has been great as we are able to build the entire framework content in less than 1 hour. I am in the process of going through final check and code signing required to post this in our download center.

Please expect the CTP in the next week or so and I will provide an update here as soon I post the CTP. 

Anand..

---

Group Manager, Developer Division

This all looks very promising. Can't wait to get my hands on "Sandcastle"!

Update: A separate thread has been started for "SandCastle". Info about the new documentation compiler will be available in that thread.