Mark Gilbert's Blog

Science and technology, served light and fluffy.

Resisting the woodpeckers – Builder Pattern

If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization.
– (Gerald) Weinberg’s Second Law

A couple of months ago, I found a recommendation on my company’s intranet for "Growing Object-Oriented Software, Guided by Tests" by Steve Freeman and Nat Pryce.  I am an avid reader, but believe or not, I haven’t actually managed to read all the books yet, so I checked this one out from the library, and went through it. 

One of the real gems I found inside was a pattern I hadn’t seen before – the builder pattern (for those of you who have the book, this appears starting on page 258).  The goal here is to create data for a test in a way that keeps your test clean, but also gives you the flexibility to change that data when you need to.  It is also very expressive – you know exactly what part of the data is being changed just by reading the setup code.

Their example, which is what sold me on this pattern looks like this:

new AddressBuilder()
       .build();

This would get you a simple address, with (presumably) default, valid, values.  Next, the builder class would allow you to modify the defaults using methods like so:

new AddressBuilder()
       .withStreet("221b Baker Street")
       .withStreet2("London")
       .withPostCode("NW1 6XE")
       .build();

The methods withStreet(), withStreet2(), and withPostCode() would override the address 1, address 2, and postal code, respectively.  What’s more, this is far clearer than writing a method like UpdateTestData("221b Baker Street", “London”, "NW1 6XE") to do the same thing – is the second parameter modifying the street address or the city?  You no longer know at a glance, you now have to dig into the code to find out.

I’ve had to do things like this in the past numerous times, and my setup code for this has been wearisome-at-best to maintain.  Not only is this pattern clean and expressive, but it could start out simple and grow with my tests.  I could start out with the default case, and if I found that I later needed to test a case involving a custom postal code, I could add a method that allowed me to inject that value.  I wouldn’t have to touch any of my other tests, or do anything crazy with my setup logic – I would just have to add a method, and chain it for the one test that needed it.

I vowed that my next project would use this pattern, and in the last couple of weeks, I got the opportunity to put it to good use.  I was building a mechanism for NLog that would allow me to configure it from a database (rather than from web.config/app.config; I won’t go into detail on how this is done, but it turns out to be rather straightforward: https://github.com/nlog/NLog/wiki/Configuration-API ).  I would pass in a Dictionary of name-value pairs for the properties, and then I wanted to test that the proper NLog target and logger was configured (out of the gate I wanted to support a database target and a mail target).

After some back and forth – some of which was me getting comfortable with the pattern, and some of which was me letting the tests drive the functionality that I needed – I arrived at the following structure:

private class NLogSettingsBuilder
{
    private List<KeyValuePair<String, String>> _Settings;

    public NLogSettingsBuilder()
    {
        this._Settings = new List<KeyValuePair<String, String>>();
    }

    public NLogSettingsBuilder WithAllDatabaseSettings()
    {
        this._Settings = new List<KeyValuePair<String, String>>();
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.connectionStringName", "SomeName"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.commandText", "insert into EventLog ([Origin], [LogLevel], [Message], [StackTrace], SourceMachine) values (@origin, @logLevel, @message, @stackTrace, @sourceMachine);"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.name", "database"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.Parameter.origin", "Services"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.Parameter.logLevel", "${level:uppercase=true}"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.Parameter.message", "${message}"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.Parameter.stackTrace", "${stacktrace}"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Database.Parameter.sourceMachine", "${machinename}"));

        this._Settings.Add(new KeyValuePair<String, String>("Logger.Database.minlevel", "Error"));
        return this;
    }

    public NLogSettingsBuilder WithAllMailSettings()
    {
        this._Settings = new List<KeyValuePair<String, String>>();
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.name", "email"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.subject", "Blah Local Dev Error"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.to", "mgilbert@blah.com"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.from", "nlog@blah.com"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.smtpServer", "smtp.blah.com"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.encoding", "UTF-8"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.body", "Timestamp: ${longdate}${newline}Level: ${level:uppercase=true}${newline}Logger: ${logger}${newline}Machine Name: ${machinename}${newline}${newline}Message: ${message}${newline}${newline}Stacktrace: ${stacktrace}"));
        this._Settings.Add(new KeyValuePair<String, String>("Target.Mail.html", "true"));

        this._Settings.Add(new KeyValuePair<String, String>("Logger.Mail.minlevel", "Error"));
        return this;
    }

    public NLogSettingsBuilder WithoutSetting(String Key)
    {
        this._Settings.RemoveAll(setting => setting.Key == Key);
        return this;
    }

    public NLogSettingsBuilder WithThisSettingAltered(String Key, String NewValue)
    {
        this.WithoutSetting(Key);
        this._Settings.Add(new KeyValuePair<String, String>(Key, NewValue));
        return this;
    }

    public Dictionary<String, String> Build()
    {
        Dictionary<String, String> NewSettings = new Dictionary<String, String>();
        if (this._Settings != null)
        {
            foreach (KeyValuePair<String, String> CurrentPair in this._Settings) { NewSettings.Add(CurrentPair.Key, CurrentPair.Value); }
        }
        return NewSettings;
    }
}

That allowed me to test things like:

this._NLogSettings = (new NLogSettingsBuilder())

                               .Build();

The default case, where there are no properties configured, and therefore no NLog targets will be configured.

this._NLogSettings = (new NLogSettingsBuilder())

                               .WithAllDatabaseSettings()

                               .Build();

All of the correct database settings will be present, so I should expect the database target and logger to be configured.

this._NLogSettings = (new NLogSettingsBuilder())

                               .WithAllMailSettings()

                               .Build();

All of the correct mail settings will be present, so I should expect the mail target and logger to be configured.

this._NLogSettings = (new NLogSettingsBuilder())

                               .WithAllDatabaseSettings()

                               .WithoutSetting("Target.Database.connectionStringName")

                               .Build();

All of the correct database settings will be present except for "Target.Database.connectionStringName".  Since this is a required property, I should not expect the database target and logger to be configured.

this._NLogSettings = (new NLogSettingsBuilder())

                               .WithAllDatabaseSettings()

                               .WithThisSettingAltered("Target.Database.connectionStringName", "Blah")

                               .Build();

All of the correct database settings will be present, and "Target.Database.connectionStringName" will have the value of "Blah".  I should not expect the database target and logger to be configured, and I should be able to test the connectionStringName property and confirm that its value matches "Blah".

As I said before, as soon as I read this, I was hooked.  I’ve struggled in the past to keep my test data manageable, and have had to bit a lot of bullets to make sweeping changes to keep it up to date.  This kind of pattern will help that immensely, and will go a great way, I think, to keeping the digital woodpeckers at bay.

April 23, 2014 Posted by | Visual Studio/.NET | Leave a comment

Account, Development, and the Tension Between

Debbie: "You sold what?!"

Steve: "What do you mean you can’t build that?!"

Debbie: "You told the client we’d have it ready when?!"

Steve: "You can’t spend three weeks polishing the code – we need to go out tomorrow!"

If you are a developer or an account person in the software/digital marketing industry for any length of time, you will probably encounter or make some variation of the above statements.  What I’ve come to realize is that there is a fundamental tension between the account person – let’s call him Steve – and the development person – let’s call her Debbie. 

The typical Debbie can wax poetic about the ridiculous deadlines, insane requirements, and unbelievable client requests that account commits her to.  The typical Steve can do the same for the infuriatingly obstinate and short-sighted developers that he has to drag kicking and screaming through a project.  So this tension is bad, and we need to find ways to reduce and even eliminate it.  Right?

Wrong.

This tension serves an absolutely vital purpose in a company.  It mustn’t be banned – it just needs to be balanced.  To do that, we need communication.  Let’s say that our two personas, Steve and Debbie, actually go into business together and create a company called Spelltronix*.  We’re going to explore the tension between the two of them in three scenarios.

 

Scenario 1
Let’s say that Steve committed Spelltronix to building a new web site that does X, Y, and Z for A dollars in B weeks, without ever consulting Debbie.  Debbie is a more than a little offended when he brings back those details and says "Here you go".  Here are just a few problems Debbie might find with this scenario:

  1. Feature X is not feasible in the time allotted.
  2. Feature Y will cost more to develop than what you negotiated, or will require the client to buy licenses for third-party software for more than what you negotiated.
  3. I’m already working on Projects J and K, and therefore can’t start on this new one for another month, so the deadline is unreasonable.
  4. I have no expertise in this domain, so Feature Z is beyond me right now.  I’ll need to attend some classes, or do some extra reading to get up to speed, and that will require additional time.

My guess is Debbie won’t enjoy working with Steve for very long if he keeps committing them to projects like this.

 

Scenario 2
Now let’s say that Debbie agreed to build a system with requirements X, Y, and Z, for A dollars in B weeks.  Steve gets the agreement signed by the client, and Debbie goes off and starts work on the new system.  Steve comes back after B weeks to see how things have gone.  Debbie reports:

  1. Feature X is complete, Y is mostly done but buggy, and Z has not yet been started.
  2. Debbie spent a lot more time trying out new third-party libraries for feature Y, finding several that did the job, but most weren’t "neat and clean" in her opinion.
  3. She didn’t have any domain expertise with Feature Z, so she has been doing a lot of reading about it, trying to get up to speed.  That’s part of the reason why she hasn’t started it yet.
  4. She’s logged a lot of overtime, and spent a lot in licensing on those third party libraries, charging all of it back to the client.  As a result, costs on this project have already exceeded the A dollars that they bid, and that doesn’t include the time needed to finish the unstarted Feature Z.
  5. It will take another 2 weeks to finish up work on Feature Y, and then start and finish feature Z.

My guess is Steve won’t enjoy working with Debbie for very long if she keeps executing projects like this.

In both of these scenarios, the tension is very one-sided.  There was no balance.  What would help avoid these?  Project Managers, you say?  Maybe, but when you boil the role of a PM down, what does he or she really do on a project?  And no, the answer is not "Gantt charts".  A PM’s primary job function is make sure all of the stakeholders in a project have the information they need, when they need it, in order to keep the project on track, on budget, etc.  In a small shop like Spelltronix, I wouldn’t start by adding a PM.  I would start by getting Steve and Debbie to ACTUALLY TALK TO EACH OTHER! 

 

Scenario 3
Steve and Debbie visit a new prospective client together.  They spend a couple of hours gathering information about the client’s needs, asking questions about the new proposed system, and finding out where the client is headed in the future.  They return to their office, and spend the rest of the afternoon pouring over their notes individually.  They spend much of the next two days discussing the project together, where several questions consume the conversation:

  1. Do we think we understand all of the major requirements well enough to bid on this?
  2. How much do we think this will cost us to do?
  3. Do we have the necessary expertise?
  4. Can we build this in the timeframes that the client is requesting?
  5. Can we hire or contract pieces of this out?
  6. What about Projects J and K – can we wrap those up before we start this one, or even run all three concurrently for a period of time without getting into trouble?

They work up the proposal together, and are eventually awarded the work.  They bring on two contractors for the specific piece of the project that Debbie is not familiar with.  Debbie is able to manage the contractors while she wraps up her other two projects, and then joins them full-bore when those wind down.  Steve, Debbie, and the contractors hold daily meetings to coordinate efforts.  Debbie evaluates several third party libraries to use for one of the features, and while she doesn’t like how any of them look, the team all agrees that several will do the job.  To meet the deadline she selects the one she feels is the best of the bunch and allows them to keep things moving along.  The contractors are released when their piece is finished, and Debbie wraps up Phase 1 on time, without any major surprises.

In the first scenario, Steve is committing their company to things that Debbie can’t do for one reason or another, but if he had waited and talked through those proposed commitments with Debbie, they could come up with more sensible ones to propose. 

In the second scenario, Debbie got overzealous with her approach with the design and implementation of the solution, and needed to be reined in.  She also needed to say she needed help sooner on the feature that she had no expertise in.  Having regular touch points between Steve and Debbie (weekly-, or even daily stand-ups) would have kept her on task, and kept the project moving forward.

In both cases, more communication was needed between Steve and Debbie.  The lack of communication caused the tension between the two sides of the company to be one-sided, and therefore unbalanced.  That led to unbalanced results.

However, when the two sides communicated as in Scenario 3, the tension was balanced.  Steve and Debbie had to contend with all of the same issues in the third scenario – with all of the same tension.  They undoubtedly had to discuss and go back and forth on the issues.  Debbie couldn’t magically conjure up more time in a day to get her existing work done AND take on this new project.  Steve couldn’t magically convince the client to cut out the feature that Debbie wasn’t familiar with.  But because they were communicating, that tension was balanced, and they were able to work through those issues, and arrived at a much better result.

Is communication and balancing tension between someone like Steve and someone like Debbie going to ensure unicorns and rainbows at the end of the project?  Of course not.  But if Steve and Debbie DON’T communicate, the tension between them will be unbalanced from the start, which will make it more likely that dumb luck and 11th-hour heroics will be required to prevent their project from going off the rails.  And that’s no way to run a project, a team, or a company.

 

*I just made this up.  Any resemblance to a real company, living or dead, is purely coincidental.

April 11, 2014 Posted by | Software Process | Leave a comment

The X Conference is coming

Registration for the 6th annual Kalamazoo X Conference opened today! 

Are you a knowledge worker?  Can you rip out IoC frameworks in your sleep?  Do you know every Excel shortcut there is?  Can you dazzle in T-SQL?  There are a plethora of technical conferences for you to hone your skills with.

The X Conference is different.  The X Conference is for all of the other skills you need to become a GREAT knowledge worker.

Join me on Saturday, April 26 at the Fetzer Center on the central campus of Western Michigan University in Kalamazoo, Michigan.  Mike Eaton and his exception crew have put what promises to be another mind-bending day together.  There is also chance to win one of three free tickets to this year’s conference by playing the Session Matchmaker Ticket Giveaway.  Check out the site for full details.

February 1, 2014 Posted by | Kalamazoo X Conference | Comments Off

Reducing the Tedium: Redux

Back in July, I wrote “Reducing the Tedium: Generalized Unit Tests via Reflection” which introduced a generalized unit test for strings and IDs in a pre-defined list of classes.  This was my first stab at trying to automate some of the more tedious tests I typically write for properties of this sort.  With my latest project, I was able to extend this even further with a new “PropertyValidator” class.

My first task was to eliminate the need for the explicit “TypesToCheck” array that defined which classes to test.  Since a lot of my use cases was checking business classes in the same assembly, I started by building a method that could take the name of the assembly to test as a parameter, open that assembly up via Reflection, and find all of the classes in it.

I started by repackaging my original code into two main functions – one for validating strings:

private static void ValidateStrings(Type CurrentType)
{
    String TestValue, ClassName;
    PropertyInfo[] ClassProperties;
    Object ClassInstance;
    
    ClassName = CurrentType.Name;
    ClassProperties = CurrentType.GetProperties().Where(p => p.PropertyType == typeof(String) && p.GetSetMethod() != null).ToArray();
    ClassInstance = Activator.CreateInstance(CurrentType);

    foreach (var PropertyUnderTest in ClassProperties)
    {
        if (ShouldSkipProperty(PropertyUnderTest)) { continue; }

        TestValue = (String)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.IsEmpty(TestValue, String.Format("{0}.{1} did not initialize properly", ClassName, PropertyUnderTest.Name));

        PropertyUnderTest.SetValue(ClassInstance, null, null);
        TestValue = (String)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.IsEmpty(TestValue, String.Format("{0}.{1} did not handle null properly", ClassName, PropertyUnderTest.Name));

        PropertyUnderTest.SetValue(ClassInstance, "", null);
        TestValue = (String)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.IsEmpty(TestValue, String.Format("{0}.{1} did not handle an empty string properly", ClassName, PropertyUnderTest.Name));

        PropertyUnderTest.SetValue(ClassInstance, "  ", null);
        TestValue = (String)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.IsEmpty(TestValue, String.Format("{0}.{1} did not handle a blank string properly", ClassName, PropertyUnderTest.Name));

        PropertyUnderTest.SetValue(ClassInstance, "abc123", null);
        TestValue = (String)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.AreEqual("abc123", TestValue, String.Format("{0}.{1} did not handle a valid string properly", ClassName, PropertyUnderTest.Name));
    }
}

 

And the other for validating IDs:

private static void ValidateIDs(Type CurrentType)
{
    long TestValue;
    String ClassName;
    PropertyInfo[] ClassProperties;
    Object ClassInstance;

    ClassName = CurrentType.Name;
    ClassProperties = CurrentType.GetProperties();
    ClassInstance = Activator.CreateInstance(CurrentType);

    foreach (var PropertyUnderTest in ClassProperties.Where(p => IsIDToValidate(p)))
    {
        if (PropertyUnderTest.GetCustomAttributes(typeof(ObsoleteAttribute), true).Count() > 0) { continue; }

        TestValue = (long)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.AreEqual(0, TestValue, String.Format("{0}.{1} did not initialize properly", ClassName, PropertyUnderTest.Name));

        PropertyUnderTest.SetValue(ClassInstance, 0, null);
        TestValue = (long)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.AreEqual(0, TestValue, String.Format("{0}.{1} did not handle being set to 0 properly", ClassName, PropertyUnderTest.Name));

        PropertyUnderTest.SetValue(ClassInstance, -1, null);
        TestValue = (long)PropertyUnderTest.GetValue(ClassInstance, null);
        Assert.AreEqual(0, TestValue, String.Format("{0}.{1} did not handle being set to a negative properly", ClassName, PropertyUnderTest.Name));
    }
}

 

These do the same basic checks on each string or ID property as described in the July post.  They make use of three helper functions – ShouldSkipType, ShouldSkipProperty, and IsIDToValidate.  I’ll describe these in turn:

private static bool ShouldSkipType(Type CurrentType)
{
    return CurrentType.GetCustomAttributes(typeof(PropertyValidator.Attributes.DoNotPerformBasicValidation), true).Length > 0 
        || CurrentType.IsGenericType 
        || CurrentType.IsAbstract
        || CurrentType.FullName.StartsWith("PostSharp");
}

The ShouldSkipType function returns True if the class:

  • is a generic type
  • is an abstract type
  • starts with “PostSharp” (this is a logging library that I started using this summer, and its classes are added to mine post-build; trying to run my test code against these classes cause problems, so I just skip them entirely)
  • has the custom “DoNotPerformBasicValidation” attribute attached.

If any of these conditions is met, the class is skipped.  I found a need to create exceptions to the rule of “check every class in this assembly”, so I created the “DoNotPerformBasicValidation” attribute that I could decorate a specific class in an assembly to be skipped.  This attribute has no logic of its own – it is merely used as a flag on the class:

using System;
using System.Collections.Generic;
using System.Linq;

namespace PropertyValidator.Attributes
{
    [AttributeUsage(AttributeTargets.All)]
    public class DoNotPerformBasicValidation : Attribute 
    { 
    }
}

This attribute can be applied to not only classes, but also individual properties, and the ShouldSkipProperty function looks for that:

private static bool ShouldSkipProperty(PropertyInfo CurrentProperty)
{
    return CurrentProperty.GetCustomAttributes(typeof(PropertyValidator.Attributes.DoNotPerformBasicValidation), true).Length > 0;
}

This function returns True if the custom attribute is found on the current property being evaluated.

Finally, to determine if a given property is a string, “ValidateStrings” merely looks at the property’s type.  To determine if it is an ID to be checked is a little trickier.  I can’t rely solely on its base type, so instead I require that property to be treated as an ID is marked as such, with another custom attribute:

using System;
using System.Collections.Generic;
using System.Linq;

namespace PropertyValidator.Attributes
{
    [AttributeUsage(AttributeTargets.Property)]
    public class ValidateAsID : Attribute 
    {
    }
}

When the “ValidateAsID” attribute is applied to a property, the “IsIDToValidate” function returns true:

private static bool IsIDToValidate(PropertyInfo CurrentProperty)
{
    return CurrentProperty.GetCustomAttributes(typeof(PropertyValidator.Attributes.ValidateAsID), true).Length > 0;
}

 

The “ValidateStrings” and “ValidateIDs” functions need a list of classes to open and examine.  That list is provided by the PropertyValidator’s “Validate” methods:

public static void Validate(String AssemblyPartialName)
{
    Validate(AssemblyPartialName, null);
}
public static void Validate(String AssemblyPartialName, String ClassName)
{
    Assembly AssemblyToValidate;
    Type[] AssemblyTypes;

    AssemblyToValidate = Assembly.Load(AssemblyPartialName);
    if (AssemblyToValidate == null) { throw new Exception(String.Format("Could not load {0}", AssemblyPartialName)); }

    AssemblyTypes = AssemblyToValidate.GetTypes();

    foreach (Type CurrentType in AssemblyTypes)
    {
        try
        {
            if (ShouldSkipType(CurrentType)) { continue; }
            if (ClassName != null && CurrentType.Name != ClassName) { continue; }

            System.Diagnostics.Trace.WriteLine(String.Format("Now testing '{0}'", CurrentType.Name));

            ValidateStrings(CurrentType);
            ValidateIDs(CurrentType);
        }
        catch (Exception ex)
        {
            throw new Exception(String.Format("Error validating the type '{0}'", CurrentType.Name), ex);
        }
    }
}

The first takes only the assembly name, and will attempt to check every class in that assembly.   The second will allow you to check a single specific class in an assembly.  For each class being checked, then, the method will invoke both “ValidateStrings” and “ValidateIDs”.

To use it, then, I can do something as simple as

PropertyValidator.Validate(“MyAssembly”);

or

PropertyValidator.Validate(“MyAssembly”, “MyClass”);

***

PropertyValidator gives me a much better testing framework for properties than before, and is very easy to use.  I still have a dependency on NUnit, however, and it doesn’t do anything with other data types such as datetimes.  Perhaps the next iteration.

December 23, 2013 Posted by | Visual Studio/.NET | Comments Off

Two-word searches

Years ago I came across a game you could play with Google where you tried to find a two-word search that would turn up exactly one result.  The fun part of this game was that once you found one of these magical searches, you’d post it somewhere on the ‘net, and then after a little while Google would come along, crawl your post, and then *poof*, there would be two results for that pair of search terms – the original and your port – thus making your find very short-lived.  I tried a few crazy searches of my own, seeing if I could stumble upon one of these, but I wasn’t ever able to.

Fast forward several years.  I was troubleshooting an NHibernate issue I was working on, and I did a search using two terms, and it came up with exactly one hit:

Google Results

I ran this search on December 10, and at the time the link returned a 404.  Google has updated since then to remove the search result, so re-running the search for these two terms turns up 0 hits now.

Short-lived, true, but hey, I finally found one!

December 20, 2013 Posted by | General | Comments Off

Now you see it, now you… still see it? Hiding Inherited Properties

A while ago, a colleague of mine, Doug, and I got into a conversation about inherited properties, and the potential for hiding them.  The precise scenario we were looking at was a base class with a "public” property, and a child class that descends from it with the same property redefined as "private".  We weren’t sure if we could do this at all, and even if we could, was it a good idea?  We ended up going a different way for the problem we were working on, but the conversation stuck in my head and I decided that I would look into it more when I had the chance.

Let’s start by defining a very simple base class:

    public class BaseClass
    {
        public String StringProperty { get; set; }
    }

And then define a child class to descend from it:

    public class ChildClass : BaseClass
    {
    }

Now, we’ll add a new version of "StringProperty", and let’s begin by keeping the scope the same:

    public class ChildClass : BaseClass
    {
        public new String StringProperty { get; set; }
    }

Next, let’s have ChildClass initialize "StringProperty" with a value:

    public class ChildClass : BaseClass
    {
        public new String StringProperty { get; set; }

        
        public ChildClass()
        {
            this.StringProperty = "INITIALIZED By Child";
        }
    }

Now, let’s write a small program that instantiates ChildClass, and reassigns "StringProperty":

    class Program
    {
        static void Main(string[] args)
        {
            ChildClass MyChildClass = new ChildClass();

            Console.WriteLine("Initialized:\t{0}", MyChildClass.StringProperty);
            MyChildClass.StringProperty = "ASSIGNED By Program";
            Console.WriteLine("Assigned:\t{0}", MyChildClass.StringProperty);

            Console.ReadLine();
        }
    }

As soon as I compile this, I’ll get a warning:

‘ParentChildProperties.ChildClass.StringProperty’ hides inherited member ‘ParentChildProperties.BaseClass.StringProperty’. Use the new keyword if hiding was intended.

(Let me point out a little grammatical issue here.  When I read this, my first question, “Ok, what’s the new keyword I’m supposed to use?”  What they meant was that the keyword to use is literally “new”.  That should have been enclosed in quotation marks in the above message.")

A little research turns up a couple of MSDN articles talking about this warning, and the larger topic of "hiding" inherited properties:

  1. http://msdn.microsoft.com/en-us/library/aa691135(v=vs.71).aspx
  2. http://msdn.microsoft.com/en-us/library/vstudio/435f1dw2.aspx

Adding the "new" modifier sounds like it only suppresses the warning, and doesn’t actually change the behavior of the properties, but let’s test that, too.

Scenario 1 – Same scopes, no "new" modifier

When I run the application built so far, I get the following output:

Scenario 1

The property is initialized and assigned as I would expect it.  Furthermore, when I inspect the initialized version of the property in the Visual Studio debugger, I see that ChildClass.StringProperty has the initialized value, but BaseClass’s version is null.  When I inspect the assigned value, ChildClass.StringProperty now has the assigned value, and the BaseClass version is still "null".

Scenario 2 – Scope is narrowed, no "new" modifier

If I change ChildClass.StringProperty to be "private", thus narrowing the scope of the property, this is the output:

Scenario 2

When I inspect the initialized value, the ChildClass and BaseClass properties are exactly the same as they were in Scenario 1, but this time the initialized value doesn’t get printed.  Once I assign a new value to it, that assigned value is printed out.  However, when I inspect StringProperty, I find that the ChildClass version still has the initialized value, but BaseClass’s version now has the assigned value.

Scenario 3 – Same scopes, added the "new" modifier

When I add the "new" modifier to ChildClass.StringProperty, and change the scope back to "public", I get the exact same results as Scenario 1.  This seems to confirm that the “new” modifier doesn’t change behavior – it only suppresses a warning.

Scenario 4 – Scope is narrowed, added the "new" modifier

When I changed the scope to "private" again, I get the exact same results as Scenario 3.  Again, this seems to confirm that the “new” modifier doesn’t change behavior.

 

So what’s going on here?  I believe this line from the first MSDN article explains it:

A declaration of a new member hides an inherited member only within the scope of the new member.

In this case, the new member’s scope is ChildClass.  When I declare "StringProperty" to be private, and then tried to access it from the program (now outside the scope of ChildClass), it couldn’t find that version, so it fell back to using on the base class version.  Since that version is never assigned, the program can only print out the "null". 

How do we explain what we were seeing through the debugger?  The debugger has always been able to inspect the full properties of a class – regardless of their scopes – so that’s why it can show me the assigned values properly every time.  However, assigning MyChildClass.StringProperty, then, must actually assign the base class version of this property, and it’s the base class version that is printed – not ChildClass’s version.

As the second article describes, this is really not so much "hiding" as it is "replacing".  That sends me back to the original question.  Is there a way to truly hide a base class property, so that it doesn’t even show up in Intellisense?

Jon Skeet responded to this exact question on a forum post on Bytes.com in 2006.  In short, no, this is not allowed:

…However, you can’t hide them in terms of preventing people from calling them. Doing so would violate Liskov’s Substitution Principle – you should always be able to treat a derived type as the base type.

***

So, the technique of hiding a property I think will be related to my "use sparingly" bucket.

Furthermore, while narrowing the property’s scope in the child class is allowed by the language, it

  1. makes things harder to troubleshoot
  2. will downright mislead me when I’m inspecting properties in the debugger
  3. seems unfit for any real-world scenario

QED – Narrowing scope is bad and should be avoided.

October 18, 2013 Posted by | Visual Studio/.NET | Comments Off

Science Podcast, Episode 18 – Battery Pack

Mark and Katherine, with Lucy’s help, take the instrument pack on its first ever roadtrip to test the GPS sensitivity.  They also work out a diagnostic procedure for testing the pack on launch day.

http://markofquality.wordpress.com/2013/10/16/science-podcast-episode-18-battery-pack/

October 16, 2013 Posted by | Podcast, Science | Comments Off

He just cut me off! SSMS and Display Limits in the Query Grid

I’m using NLog in my current project to record errors, tracing, and other information while my application is running.  Some of the log messages, particularly the errors, can get quite long – thousands of kilobytes – because part of what I’m recording is the raw XML being passed into the app.

I have NLog configured to both email me when an error occurs and record the details to a table in SQL Server 2008.  I was running a load test recently, and noticed a few errors being recorded.  When I looked at the "Message" field in the table, I found that the message was being truncated – in fact, it was cutting the raw XML off.  Since I use this for reproducing the error, figuring out why it was being truncated was fairly important.

As I investigated more deeply, I found several things that were odd about this:

  1. First, I checked the "Message" field in the table to make sure it was large enough to store everything.  I confirmed that it was defined as NVARCHAR(max) field, so there should have been ample room.
  2. Next, I looked at the version of the error that got emailed to me.  That version had the complete message.
  3. I next added a "File" target to my NLog.config file.  I wanted to see what would get recorded if I wrote the error message out to a text file.  That version also had the complete message.
  4. Finally, the messages were always being truncated at 43,679 bytes.

So, it seemed that I was only having problems writing NLog error messages to SQL Server.  Perhaps there was something wonky with the "database" NLog target source.  I dug that up on the internets, and didn’t see anything that should be truncating the messages, let alone at 43,679 bytes.

I next decided to try inserting a large message directly into SQL Server, via some quickly-crafted C# logic.  I wanted to see if I could eliminate NLog as being the culprit here.  I inserted 30KB and 40KB messages, and those worked fine – I could insert them without errors, and I could get them out of SQL Server Management Studio (SSMS) completely intact.  When I tried a 50KB file, though, that failed in the same way – no errors on the insert, but when I queried the table in SSMS and copied the message to a text file, it was truncated.

What. The. Heck?!?

I did some more digging on the internets, this time for "sql server truncate" and other variants, and came across this post on SQLServerCentral.com.  From Jim McLeod’s answer:

"Yep – you got it. There’s a setting in Management Studio – Tools, Options, Query Results, SQL Server, Results To Grid, Maximum Characters Retrieved – Non XML Data, which defaults to 64KB."

So SSMS is limiting the text it’s going to display in the grid.  Ok, easy enough, I just have to increase this default limit.  I found the option:

SSMS Options

And tried to change it to 100KB.  The dialog looked like it saved it, but when I went back in later it had been reverted to 65KB.  I tried the spinner controls, and it wouldn’t let me go above 65KB.

SSMS – why do you taunt me so?

Since Management Studio won’t let me view this via a query, perhaps I just need to extract it.  I first tried extracting this to a flat file, but that choked because the Message field was an NVARCHAR, and there was a problem with the character set.  I next tried exporting to Excel, but that choked because individual cells in Excel can’t hold more than 32KB.  In the end, I just wrote a little app in C# that extracts the message and saves it to a text file.

<rant>
I’m going to officially write this up as a case of "tool-fail", if only because it was failing silently.  Microsoft: At least tell me that the query I ran returned text results that were greater than 65KB, and would be truncated.  If you really want brownie points, tell me where this limit is defined in the Tools dialog, and even better LET ME INCREASE IT.
</rant>

Ok, I’m done.

September 11, 2013 Posted by | SQL Server | Comments Off

Breaking the Mold – Hacking together an Interface for Constants

It seems like once a project gets to a certain size, magic strings – that is, character values hard-coded into the application – become inevitable.  When a piece of software needs to read data generated by another piece of software, the use of magic strings becomes a requirement for sanity.  The common pattern for dealing with magic strings is to encapsulate them in a constant:

        public const String FRUITS_APPLE = "Apple";

That works fine when you have a small number of constants, but what happens when you have 10, 20, or more?  Furthermore, what do you do when subsets of them are logically related to each other?  One approach is to name them according to that logical group:

        public const String APPLES_RED_DELICIOUS = "Red Delicious";
        public const String APPLES_GALA = "Gala";
        public const String APPLES_FUJI = "Fiji";

        public const String ORANGES_BLOOD = "Blood";
        public const String ORANGES_NAVEL = "Navel";
        public const String ORANGES_VALENCIA = "Valencia";

        public const String LEMONS_MEYER = "Meyer";
        public const String LEMONS_EUREKA = "Eureka";
        public const String LEMONS_LISBON = "Lisbon";

What I’d really prefer to do is to create enumerations for each of these groups.  The problem is an enumeration can’t have string values behind it:

        public enum Apples
        {
            RedDelicious = "Red Delicious",
            Gala = "Gala",
            Fuji = "Fuji"
        }

This will return a compile-time error of "Cannot implicitly convert type ‘string’ to ‘int’".

I’ve gotten around this in the last few years by creating one or more subclasses, one for each logical grouping, and then creating constants within.  I call these "constants-classes":

        public class Apples
        {
            public const String RedDelicious = "Red Delicious";
            public const String Gala = "Gala";
            public const String Fuji = "Fiji";
        }
        public class Oranges
        {
            public const String Blood = "Blood";
            public const String Navel = "Navel";
            public const String Valencia = "Valencia";
        }
        public class Lemons
        {
            public const String Meyer = "Meyer";
            public const String Eureka = "Eureka";
            public const String Lsibon = "Lisbon";
        }

In code, then, I can access these values just like I would the values of a true enumeration:

        String CurrentApple = Apples.RedDelicious;
        String CurrentOranage = Oranges.Navel;

This pattern has served me well over the years, but I ran into a new challenge with it in my current project.  The application imports XML messages pulled off of a queue, and there are a small number of major types of entities that need to be processed.  (For the purposes of this post, I’ll continue my fruit analogy and say that the types of entities are apples, oranges, and lemons; I’m not really writing a fruit app.)

Large chunks of the XML for these different types are actually the same, so I built a series of System.Xml.Linq.XElement extension methods that operate on a specific slice of the XML.  When I import a given entity – let’s say a lemon – I break off the appropriate piece of the full XML document for that lemon, then call the appropriate extension method to de-serialize it and turn it into a strongly-typed object (“FruitProperties” in this case):

        public static FruitProperties ExtractProperties(this XElement XmlToProcess)
        {
        }

The XML was largely made up of collections of <Attribute /> blocks, each of which contained the same basic components.  One of these components was the "name" of the attribute – a unique handle that will never change, and allows me to safely identify the various bits of data for an entity.  I defined several constants-classes to centralize these names. 

Let’s say I had attributes like "PeelTexture", "Color", and "HasSeeds".  The challenge arose when I realized that although all of my entity types had these attributes, the actual attribute names used in the apple-XML versus the orange-XML were different.  For example, the PeelTexture attribute might appear as "Peel_Texture" for an apple, but "Orange_Peel" for an orange.   
   
My first attempt to address this was to try to move the constants to an “IFruit” interface.  My thought was to define the template for the constants, and then I would "implement" them (assign the real, apple-, orange-, and lemon-specific values) in the concrete classes that implemented that interface.  However, I got another compile-time error because C# (and presumably all .NET languages) don’t allow interfaces to have static members, fields, or properties.

I tried several other approaches before I arrived at this slightly-hacky solution.  First, I defined my constants-classes to hold all of my entities’ unique values:

        public class Constants
        {
            public class Apple
            {
                public const String PeelTexture = "Peel_Texture";
                public const String Color = "Color";
                public const String HasSeeds = "Apple_HasSeeds";
            }
            public class Orange
            {
                public const String PeelTexture = "Orange_Peel";
                public const String Color = "Orange_Color";
                public const String HasSeeds = "Has_Seeds";
            }
            public class Lemon
            {
                public const String PeelTexture = "PeelTexture_2";
                public const String Color = "Color_Lemon";
                public const String HasSeeds = "Has_Any_Seeds";
            }
        }

I then modified my generic ExtractProperties method to take a Type object.  This object would be the actual constants-class for the entity being processed:

        public static FruitProperties ExtractProperties(this XElement XmlToProcess, Type FruitConstants)
        {
        }

This method would be invoked as follows:

        XDocument MyDocument;
        FruitProperties MyProperties;

        // Load XML into MyDocument here

        XElement SliceToImport = MyDocument.Element("Root").Element("Properties");
        MyProperties = SliceToImport.ExtractProperties(typeof(Constants.Apple));

Within the ExtractProperties method, I then naively tried to do this to get the actual value of the constant:

        String FieldToExtract = FruitConstants.PeelTexture;

But ExtractProperties had no idea what "FruitConstants" type really was, so this also throws a compile-time error.  So, I then created a simple enumeration that became my "template" for my constants-classes.  This enumeration’s members would have the same name as the constants defined in my classes:

        public enum FruitConstantsTemplate
        {
            PeelTexture,
            Color,
            HasSeeds
        }

I could use that template – and a little trust – to pull the desired value out of the constants-class by its name using reflection.  I wrote a custom function to do just that:

        public static String GetFieldName(Type T, String ConstantName)
        {
            FieldInfo CandidateConstant;

            // Validate the parameters being passed in
            ConstantName = (ConstantName ?? "").Trim();
            if (String.IsNullOrEmpty(ConstantName)) { throw new ConstantException(); }

            if (T == null) { throw new ClassException(); }

            // Get the list of constants from type T using reflection
            var Constants = T.GetFields(BindingFlags.Public | BindingFlags.Static | BindingFlags.FlattenHierarchy);

            // Find the one I'm looking for, based on name.
            CandidateConstant = Constants.FirstOrDefault(c => c.Name == ConstantName);
            if (CandidateConstant == null) { throw new ConstantNotFoundException(ConstantName, T.Name); }
            return (String)CandidateConstant.GetValue(CandidateConstant);
        }

I modified ExtractProperties to use this new function:

            String FieldToExtract = GetFieldName(typeof(Constants.Apple), FruitConstants.PeelTexture.ToString());

I say "a little trust" because this function only returns a constant’s value if both of the following hold true:

  1. the FruitConstantsTemplate defines it; and
  2. the constants-class passed in as type T contains it

This sort of thing is what an interface would enforce for methods (for example) at compile-time.  With this approach, however, there’s nothing to stop me from adding a value to the enumeration, and forgetting to add it to the constants-classes (or vice versa).  There’s also nothing at compile time to stop me from passing in a completely invalid type to GetFieldName(), such as "String" or "DateTime" (again, something a true interface would catch at compile-time).

This approach does have its share of advantages, though:

  1. It allows me to concentrate the "generic-ness" in one function (rather than having the same basic logic duplicated in several functions, differing only by the constants it was using).
  2. It also allows me to define and continue using the class-constants just as I have always done.  The only catch is when I add a constant, I have to remember to update the template with it.

***

While I have a functional solution, I also have a nagging thought.  This is organic solution, grown (no pun intended) around the constraints of the framework and the language, but is there a better way to accomplish what I’m trying to do?

A question to revisit for my next project.

September 9, 2013 Posted by | Visual Studio/.NET | Comments Off

Queue them up! Unit testing HttpClient

Early in my current project, I was looking for a way to test some logic that relied on HttpClient.PostAsync.  I was writing a routine that would process items coming off a queue that was exposed via a series of REST services.  Subscribing to the queue, pulling messages down, and cleaning the subscription up once the queue was empty all required calls to the services, and in most cases several calls.  Ideally I’d be able to mock the calls to HttpClient.PostAsync out, but I can’t because that method isn’t virtual.  The usual way to handle methods like that is to build an interface that I CAN mock out, and then build a wrapper-implementation of that interface which calls HttpClient.PostAsync.  My unit tests would use a mock of that interface, and the real code would use my wrapper.  Adding interfaces adds flexibility to a piece of software, at the cost of readibility and maintainability, so before I went that route I wanted to see if there was a better way.

I came across this post by Gabriel Perez where he shows how to build a fake handler that returns an “injected” response when PostAsync is invoked.  (He credits Glenn Bock for the original source.)  This “russian doll model” as he calls it looked very promising.  In fact, I thought I could even extend it to inject the responses from a queue, allowing me to line them for a specific scenario, and then test my code against it.

I decided that a simple list would suffice for the queue.  FakeHandler would just keep track of which responses it had already handed out:

// FakeHandler, adapted from: http://perezgb.com/2012/02/21/web-api-testing-with-httpclient
        public class FakeHandler : DelegatingHandler
        {
            public List<HttpResponseMessage> Responses { get; set; }
            private int _Index;
public FakeHandler() : base()
            {
                this.Responses = new List<HttpResponseMessage>();
                this._Index = -1;
            }

            protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
            {
this._Index++;
                if (this._Index >= this.Responses.Count) { return Task.Factory.StartNew(() => (HttpResponseMessage)null); }
                return Task.Factory.StartNew(() => Responses[this._Index]);
            }
        }

Here is how a test would look using this structure:

        [Test]
        public void SomeTest()
        {
            HttpResponseMessage ConfiguredResponse;
            HttpClient ConfiguredClient;
            FakeHandler NewHandler;
            QueueProcessor MyProcessor;

            // Configure the response.  Set properties (headers, etc.) as needed.
            ConfiguredResponse = new HttpResponseMessage();

            // Build a handler around this response
            NewHandler = new FakeHandler() { InnerHandler = new HttpClientHandler() };
            NewHandler.Responses.Add(ConfiguredResponse);

            // Create a client object around this handler
            ConfiguredClient = new HttpClient(NewHandler);

            // Pass in the pre-configured client to the class under test (dependency injection)
            MyProcessor = new QueueProcessor(ConfiguredClient);

            // Test the method
            var Results = MyProcessor.DoSomething();

            // TODO: Evaluate the results
        }

For “real” use, I would simply pass in a new HttpClient() object to the QueueProcessor constructor.  This all worked quite well to mock out the responses, but my tests could only evaluate what came out at the very end.  I also wanted to make sure the individual responses were being handled properly.

I could have simply injected a set of Asserts into FakeHandler.SendAsync, but I wanted to build it so it could potentially handle completely different tests each time.  To do that, I added an event handler called “OnEntryVerificationMethod”.  This became a property of the FakeHandler class that I could define in my unit test, which would be run at a known point in the execution (on entry), and would have access to both the current and previous responses (see bolded additions below).

// Define the prototype for the delegate that will perform our validation
        public delegate void VerifyOnEntry(HttpRequestMessage CurrentRequest, HttpResponseMessage PreviousResponse);

        // FakeHandler, adapted from: http://perezgb.com/2012/02/21/web-api-testing-with-httpclient
        public class FakeHandler : DelegatingHandler
        {
            public List<HttpResponseMessage> Responses { get; set; }
            private int _Index;

            // Define a delegate that allows me to pass in a custom function at runtime
            public VerifyOnEntry OnEntryVerificationMethod;

            public FakeHandler() : base()
            {
                this.Responses = new List<HttpResponseMessage>();
                this._Index = -1;
            }

            protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
            {
                // If a custom function was defined, invoke it, passing in the parameters to be verified.
                if (OnEntryVerificationMethod != null)
                {
                    OnEntryVerificationMethod(request, (this._Index < 0 ? (HttpResponseMessage)null : Responses[this._Index]));
                }

                this._Index++;
                if (this._Index >= this.Responses.Count) { return Task.Factory.StartNew(() => (HttpResponseMessage)null); }
                return Task.Factory.StartNew(() => Responses[this._Index]);
            }
        }

I could define this in my unit test using an anonymous function:

            int NumberOfMessagesProcessed = 0;
            NewHandler.OnEntryVerificationMethod = (CurrentRequest, PreviousResponse) =>
            {
                NumberOfMessagesProcessed++;

                if (PreviousResponse == null) { return; }

                switch (NumberOfMessagesProcessed)
                {
                    case 1:
                        // TODO: Perform tests for the first response processed
                        break;

                    case 2:
                        // TODO: Perform tests for the second response processed
                        break;
                }
            };

Now I had a window into FakeHandler’s execution, and could inspect anything I needed.

If HttpClient.PostAsync was virtual (and therefore directly mockable), it would have made testing one call far easier, but trying to simulate multiple calls in a row would probably have pushed me into this structure anyway.

You could argue that in lining up multiple responses like this I’m trying to test too much; that what I should be doing is setting up everything for a given scenario (e.g., responses have already come back as “fail” twice), and then pass in the one I really want to test (the third attempt is a success).  However, I think that would result in a more complicated code base because I would have to make several internal properties available to my test (like the number of times an HttpClient.PostAsync call results in a failure), or make more extensive use of interfaces, or both.  Lining up a series of responses, and then testing that greater scenario, is a more straightforward approach.

Sometimes, I will totally trade having code that is awesomely-flexible and unit test-pure for code that I can more easily get my head wrapped around.

September 5, 2013 Posted by | Visual Studio/.NET, Web API | Comments Off

Follow

Get every new post delivered to your Inbox.