24/02/2010 21:02:32

Wireshark is an open source network protocol analyzer and quite probably the best
of its kind. If you are a developer working with a lot of networking code, it’s a must have!

It can recognize many standard protocols such as POP3, NTP, Jabber, etc…
Most of the development I do is related to communicating with hardware devices
using TCP/IP. These devices implement their own communication protocol unknown to
Wireshark. While I can still use Wireshark to monitor the raw data, I thought it would
be much better if I could actually tell it what my data is so it can be displayed properly.

1/02/2010 14:27:26

For reasons explained in my previous post, I decided to embed IronPython.

In this post I’m going to show you how to do it yourself. In order to get started
you’ll need a build of IronPython. If you’re targeting .NET 4.0 Beta 1 you’ll have to
get “IronPython 2.6 CTP for .NET 4.0 Beta 1”, otherwise you can get the normal release.
(The new features of C# 4.0 do make it a lot easier to embed dynamic languages).

31/01/2010 19:27:33

Most of the applications that we and our partners work on are highly customized
for a very specific purpose with point-and-click end users in mind, but some of the
applications are designed only to control and configure hardware devices.

The highly customized applications usually integrate our hardware as part of a much
larger system. The focus of those applications is not the hardware.

The focus of the other applications is really the hardware itself.
Those applications simply try to expose all of the hardware’s functionality
with just enough abstraction so that the end users (most likely technicians, engineers or
others with detailed knowledge about the hardware) can consume it easily.

Consider the software to control our LED panels. It allows the user
to configure the LED hardware’s parameters, perform diagnostics and
put various components on the screen as a means to “demo” it to potential customers.

This software is not on our highest priority list, so it doesn’t really get special features
or new additions unless the hardware changes.

However, on more than one occasion we’ve had customers asking:
”Hey it’s great you can control the hardware like that, and we like the designer you
provided, but some of our engineers would like the ability to do some of those things
by using code.” and we’ve had situations where potential customers want to see more
then the static demo where you just add some text, pictures, etc… to the screen.

Now with IronPython we can solve both problems in one go. By embedding IronPython
into our application the end user can manipulate the objects we expose directly.
The customer wants to see the latest tweets on an LED panel or perhaps a simulation
of traffic information on an LED panel? No problem!

A screenshot of the initial version of our IronPython powered DSL.
(Completely outdated, but more on that in my next blog entry)

I’m going to post more information on how to embed IronPython in .NET applications
very soon. If you’re an IronRuby fan, don’t worry the same concepts apply.

10/01/2010 18:55:58

I went to the SPAR supermarket for the first time since my return today.
It provided me with a real *facepalm* moment.

At the cash register they put up a sign:

Due to the use of new software we can no longer offer cash refunds for foodcheques.

I don’t get it… is the ‘new software’ thing just an excuse or did they really
pay someone to write software which reduces customer satisfaction?

If you’re going to replace a software system, at least make sure the new
system is a real improvement. Software is supposed to make our lives easier, not harder.

To bad I didn’t get to take a look at the software to see who made it,
in any case whoever they are… as far as I’m concerned they can join the Isabel developers.

FAIL!

30/10/2009 10:36:47

I don’t know about most developers, but whenever I write code I keep wondering
if I’m writing it the way I should be writing it. I created a question on the subject
over at StackOverflow recently, but I wanted to expand a bit on the subject.

I started out with the following F# code: (Yes I realize I should have used a foreach)

let data = Array.zeroCreate(3 + (int)firmwareVersions.Count * 27)
data.[0] <- 0x09uy                              //drcode
data.[1..2] <- firmwareVersionBytes             //Number of firmware versions

let mutable index = 0
let loops = firmwareVersions.Count - 1
for i = 0 to loops do
    let nameBytes = ASCIIEncoding.ASCII.GetBytes(firmwareVersions.[i].Name)
    let timestampBytes = this.getTimeStampBytes firmwareVersions.[i].Timestamp
    let sizeBytes = BitConverter.GetBytes(firmwareVersions.[i].Size) |> Array.rev

    data.[index + 3 .. index + 10] <- nameBytes
    data.[index + 11 .. index + 24] <- timestampBytes
    data.[index + 25 .. index + 28] <- sizeBytes
    data.[index + 29] <- firmwareVersions.[i].Status
    index <- index + 27

The code above is part of a library which parses binary data from a communication protocol.
”firmwareVersions” is a List of class which is defined in a c# library.
It has no knowledge of how it will be converted into an array of bytes.
I realized the code above is not exactly written in a functional way and figured I’d probably
be burned alive for it so I modified it like this:

let headerData = Array.zeroCreate(3)
headerData.[0] <- 0x09uy
headerData.[1..2] <- firmwareVersionBytes

let getFirmwareVersionBytes (firmware : FirmwareVersion) =
    let nameBytes = ASCIIEncoding.ASCII.GetBytes(firmware.Name)
    let timestampBytes = this.getTimeStampBytes firmware.Timestamp
    let sizeBytes = BitConverter.GetBytes(firmware.Size) |> Array.rev
    Array.concat [nameBytes; timestampBytes; sizeBytes]

let data = 
    firmwareVersions.ToArray()
    |> Array.map (fun f -> getFirmwareVersionBytes f)
    |> Array.reduce (fun acc b -> Array.concat [acc; b])

let fullData = Array.concat [headerData;data]

The implementation of the protocol handler serves as an example to other developers who
will be implementing the same protocol in different languages (hardware developers) and those
developers hardly understand English so the protocol specification document is pretty useless
to them. Keeping this in mind I thought (and still do) that the first implementation would
give them a much better idea of how to parse the data. The exact positioning of the bytes
is very clear in the first version while in the second that information is abstracted away.

To my surprise the people who responded seemed to agree and disagree with each other:

I like your first version better because the indexing gives a better picture of the offsets, which are an important piece of the problem (I assume). The imperative code features the byte offsets prominently, which might be important if your partners can't/don't read the documentation. The functional code emphasises sticking together structures, which would be OK if the byte offsets are not important enough to be mentioned in the documentation either.

(By Nathan Sanders)

The advantage of 'array concatenation' is that it does make it easier to 'see' the logical portions. The disadvantage is that it creates a lot of garbage (allocating temporary arrays) and may also be slower if used in a tight loop.

(By Brain)

These responses made me wonder, what do I actually gain from writing this code
in a more functional way? The details are less obvious to the other developers and
I might get a decent amount of memory overhead. As I understand it, in the second
version I care more about what I want to do then how I want to do it, but given the
fact that the code also serves as documentation in a “universal” language I still prefer
the first “imperative” version of the code. (But maybe I’m missing the point somewhere)

To add to my own confusion (and loss of confidence) I was going over some old code:

public void DefineClockFields(byte address, List<ClockField> clockFields)
{
    var data = new byte[clockFields.Count * 19 + 3];
    data[0] = 0x0D;
    data.InsertByteArray(BitConverter.GetBytes((ushort)clockFields.Count).Invert(), 1);
    var position = 3;

    foreach (var field in clockFields)
    {
        var definitionData = dataFactory.CreateDefinitionData(field);
        data.InsertByteArray(definitionData, position);
        position += definitionData.Length;
    }

    SendData(data, DEFAULT_MESSAGEGROUP_VERSION, address, 701, 74, AckResponseProcessor);
}

This code is part of a prototype test client I used to test the hardware emulator for the
protocol. Not the most memory efficient code as you can see. Slapping myself in the head
(Although I was suffering from jetlag and other things when I wrote this) I refactored the
sources and the above method was translated into this:

public void DefineClockFields(byte address, List<ClockField> clockFields)
{
    using (var stream = new MemoryStream(clockFields.Count * 19 + 3))
    using (var writer = new BinaryWriter(stream))
    {
        writer.Write((byte)0x0D)
        writer.Write(BitConverter.GetBytes((ushort)clockFields.Count).Invert());
        
        foreach(var field in clockFields)
        {
            var definitionData = dataFactory.CreateDefinitionData(field);
            writer.Write(definitionData);
        }

        SendData(stream.ToArray(), DEFAULT_MESSAGEGROUP_VERSION, 
                 address, 701, 74, AckResponseProcessor);
    }
}

It’s certainly makes more efficient use of memory and it’s still quite readable.
I don’t feel any relevant information was lost either. I decided to go functional crazy:

public void DefineClockFields(byte address, List<ClockField> clockFields)
{
    using (var stream = new MemoryStream(clockFields.Count * 19 + 3))
    using (var writer = new BinaryWriter(stream))
    {
        writer.Write((byte)0x0D)
        writer.Write(BitConverter.GetBytes((ushort)clockFields.Count).Invert());
        
        clockFields.ForEach(f => writer.Write(dataFactory.CreateDefinitionData(f));

        SendData(stream.ToArray(), DEFAULT_MESSAGEGROUP_VERSION, 
                 address, 701, 74, AckResponseProcessor);
    }
}

There you go, even less code… but at what cost? Is it still readable?
I’m really wondering what others think of all of this… how far should we take this?

I’d really like some feedback, what would you do, what would you avoid, etc…

15/12/2008 21:13:42

ClickOnce is a great way of distributing software and it's been
a real life saver for one of the projects I've been working on.

Basically it allows you to publish an application every time you
have a working build and all customers will get the new version automatically
the next time they start the software. Users don't have to go through the motions
of uninstalling old versions, copying files and so on.

On more than one occasion I had the customer calling me to report a problem
and by the time the conversation was over I had already published an updated version.
It's kinda nice to tell the customer "Please restart the application, there ya go... it works".

There's a lot more to ClickOnce than easy deployment, such as sandboxing
and fine grained security but that's not what this post is about.

While using ClickOnce I tend not to worry about deployment problems,
and without giving it any further thought I published the application from
my laptop after making a few tweaks while I was on the road.

I've learned never to try that again...

The software published just fine... but after updating the clients
started crashing and the entire update system collapsed.

After quite a long time of bug hunting I finally realized it might
have had something to do with the fact that I published from another pc.

I came home, updated the source repository, published from my dev-machine
and all was well again... Bottom line: only publish from a single computer.

PS: Google either uses or mimicks the ClickOnce deployment technology,
can anybody confirm this?

18/11/2008 15:47:13

One of the most important changes in the upcoming C# 4.0 is the "dynamic" type.
You can find information and code samples on the subject all over the web,
so I'm not going to delve to deep into the subject here, but instead I want explain the
difference between the "object" type, the "dynamic" type and the "var" keyword.

5/10/2008 19:17:37

Controller actions in an ASP.NET MVC application return an instance of a class
which derives from the ActionResult class, usually an instance of the ViewResult class.
Other classes which derive from the same base class are EmptyResult, RedirectResult,
RedirectToRouteResult, JsonResult and ContentResult.

We needed the ability to send a file to the browser and none of the existing classes
is well suited for that, so I created a custom ActionResult and called it FileActionResult.

30/07/2008 14:52:25

I was asked to install Isabel on a Vista computer yesterday. A quad core machine with
2GB's of RAM, the installation shouldn't have taken more than a few minutes.

Of course, if that were the case I wouldn't be blogging about it, now would I?
The first problem I ran into was the installer refusing to install on an "unkown"
version of Windows. The solution was to get an updated version of the installer
without the restriction. 
Along with the update I was also told to install a hotfix for Windows.  Ready to move on.

NOT! After installing the application I was able to boot it up, but it stopped responding
while loading the various "resources". After terminating the process I was unable to start
the application again because it had locked some files... solution: reboot Windows.

Remembering Isabel from the old days, I started searching the docs to find out all the
files the user should have "full access" to in order for the application to work properly.
Half of them were in the %Windows% directory, a few in the %system32% directory, etc...

After making all the changes I was able to boot the application, only to be presented with
the same exception message box over and over again, something about missing data.
Instead of simply telling me once and shutting down the application I was forced to
once again terminate the process and reboot the system. Eventually I managed to get
everything working at the expense of precious time and security.

So who's fault is this? Who's to blame? I wonder if I told this story in class how many
would raise their hand when I ask them who thinks Vista is to blame... probably way to many.

Then I'd have to tell them the same problems apply to Windows 2000 and Windows XP and
the only one to blame here is the company that made Isabel. The project was probably started
years ago by some wannabe developer who didn't see the point of designing good software.

An application should never consider certain files or directories to exist on the target system.
It should only rely on the directory in which the executable exists and the virtual directories
that are available to store user specific data. It should never assume to have "administrative"
rights on any files or directories. And finally, the Windows registry should be avoided aoc!

If the Isabel developers had kept this in mind they would have been able to "xcopy deploy"
their application without the need for users to have administrative access.

Applications should "realize" the following: (These simple assumptions are very important)

• I will run in a sandbox with limited access to the system
• I should only access virtual directories
• I should never assume full access to any other files or directories

Working with the Windows Forms ListBox

lstProducts.AddItem "Product A"
lstProducts.AddItem "Product B"