Unit test your web application your own way

I'm likely doing it wrong but I don't care

The goal of unit testing (in my mind) is to employ tiny workers (the tests) to go out and double check that things are still working as expected (my application). Nothing more, nothing less. So let me preface this article with the acknowledgement that in the eyes of the purists, I'm probably doing testing all wrong. But I'm ok with it because my testing has given me both peace of mind and has actually caught issues in my code. Up to this point, I've been using the term 'unit testing' liberally to encompass all the sorts of tests that could be written with code. It turns out that there are many types of testing and it took me awhile to figure out that not all tests are unit tests. For the discussion of this article, we will take a look at how using NUnit can be used to perform unit, functional and smoke testing. We won't get terribly deep as the goal here is to get something useful going in your solutions and not try to understand all of the nuances. Another reason to keep this simple is because I'm not a unit testing pro just yet but wanted to share my experience of what I have discovered thus far.

The goal of unit testing (in my mind) is to employ tiny workers (the tests) to go out and double check that things are still working as expected (my application). Nothing more, nothing less.

Why should you test?

You should write tests to confirm what you think is true to actually be true. By doing so you can be much more confident the next time you deploy changes. And given enough check and balances, your night and weekend time can be better spent with your family. That sounds all great and well but how does this translate into testing that is relevant to my web application? Let's break down the four types of testing that I currently employ by writing tests with code:

  • Unit Testing - For me, this means taking a simple method and 'fuzzing' it with many inputs and checking it against the expected output. The value here is that if you later optimize the method, the test will confirm that it all works just the way you intended it. Unit testing has the positive side effect of regression testing aka "did I just break anything depending on this?". Unit tests I build typically test methods that transform values from x to y.
  • Smoke Testing - In my mind, this is a simple test that determines if some basic things are working as intended but at a larger scope than a unit test. These tests are written with the thought of, 'does it seem to work? The key word being 'seem'. These tests won't catch everything, but they prove generalized things to be generally true. It's more for early warning, if there is smoke, then there might be a fire. For instance, these are the sorts of things I can write a smoke test for:
    • Does page x load with a 200 status code? I can infer I don't have a global issue.
    • Does page x have the words 'server error' on it? I can infer that the page loads, but not much else.
  • Functional Testing - Functional testing is a means to test my custom application logic and/or configurations on a particular system with an end-to-end mindset. The sorts of tests I write for this category look like this list below:
    • Can a user login?
    • Can a user do a specific task like submit a form?
    • Does a DB call actually pull back the expected results?
  • Tasks - I like to think that I don't have to do repetitive tasks over and over given that I'm a programmer. Let's make some code do those trivial yet essential tasks. These are real 'tests', but it gives me an opportunity to run some tasks on remote systems right/before my smoke tests complete. For instance, I write tests that can do the following:
    • Check that a particular system has custom errors turned on remotely (i.e. my local test can reach out to my production boxes). 
    • Republish cache on my preproduction box just after deployment.
    • Rebuild an Examine index on preproduction after deployment.

As you can see, some of the concepts overlap and that tripped me up because I thought everything was a unit test. Defining what type of test is pretty subjective, but as long as the test proves your code is doing what is expected, it doesn't matter what you call it. 

As an aside, I'm not against test driven development (TDD) but I must say it's a bit counter to how I'm hard-wired when I build applications. It turns out that writing tests that actually helped me find issues after my app was built was more beneficial than writing a test ahead of time. With TDD, you are supposed to write all of the tests firsts. By doing so all of your tests should fail since there is no actual code to test yet. You then add the functionality as desired for the application until eventually all of the tests pass. For me when I begin a new application, I have an idea of how the application will work, but I can't wrap my head (or workflow) around foreseeing all the tests I will need upfront.

What should we use for a testing framework and what does it look like?

Well, there are a few options out there and I've only used one. So take that for what it's worth when you make your decision. I'm using NUnit v2 though I concede there is a new version of NUnit (version 3) available. I like NUnit because it integrates quite nicely with Visual Studio via an extension. As you can see below, my tests are categorized by a few traits, we can list out several 'test cases' and my tests live in their own project:

 

Dispel the myths and set expectations

So if you do enough Googling about unit testing, you'll find plenty of flame war type articles about why you shouldn't ever write code that is hard to unit test. They'll even go so far as to say "if it's hard to unit test, you've written bad code". I find most of it to be discouraging to new testers like myself. For instance, Singletons are the biggest target of these types of discussions because due to the nature of its pattern. Yes, they create hard to test circumstances, but with inversion of control (IoC) and dependency injection; you can test them.

One thing that is truly difficult is testing methods that require a runtime, a context or an object that can't be otherwise created easily. For such circumstances you will need to use fakes, mocks and stubs to get around these issues. The purpose of this article isn't to get into the weeds of of how to do those things as they can truly become time consuming. Andy Butland has put together a blog on unit testing Umbraco (and another here) methods/controllers that depend on things like IPublishedContent and the UmbracoContext. 

Let's make our first test (unit test)

I've assumed at this point you've installed NUnit via Nuget to your tests project and installed the NUnit adapter via the Visual Studio extensions manager. Our first test will be pretty simple and I'll classify it as a unit test. The goal of the test is to simple choose likely values that you might encounter during execution and verify that the output is what you expect. Let's start with the method that we're testing:

//a simple extension method
//given a file path, return a enum of type MediaType
public static Constants.MediaType GetMediaType(this string input)
{
    input = input.ToLower();

    if (input.EndsWith(".jpg"))
    {
        return Constants.MediaType.Jpg;
    }

    if (input.EndsWith(".png"))
    {
        return Constants.MediaType.Png;
    }

    if (input.EndsWith(".gif"))
    {
        return Constants.MediaType.Gif;
    }

    return Constants.MediaType.Jpg;
}

My first stab at writing a test for this looked like this:

using NUnit.Framework;
using MyNamespace.Extensions;

namespace MyTests.DigitalMedia
{
    [TestFixture]//required
    public class Foo
    {
        //let's test for png
        [Test]//required
        public void Can_Get_Media_Type_Png()
        {
            var input = "/foo/test.png";

            var type = input.GetMediaType();

            //this is there the testing passes or fails
            Assert.AreEqual(Constants.MediaType.Png, type);
        }

        //let's test for png again
        [Test]
        public void Can_Get_Media_Type_Png2()
        {
            var input = "/bar/test.PNG";

            var type = input.GetMediaType();

            Assert.AreEqual(Constants.MediaType.Png, type);
        }
    }
}

These tests end up passing without issue when I run them, but clearly this won't scale nor does it pass the DRY standard.

It turns out NUnit has an attribute named 'TestCase' that allows us to write DRY code and add many test cases very quickly. The only drawback here is that any information you put into the attribute must be available at compile time. Meaning you can't just 'new' an object in the attribute. So let's try another attempt at our unit test:

[TestCase("foo.jpg", Constants.MediaType.Jpg)]
[TestCase("/foo/foo.jpg", Constants.MediaType.Jpg)]
[TestCase(".jpg", Constants.MediaType.Jpg)]
[TestCase("foo.JPG", Constants.MediaType.Jpg)]

[TestCase("jpg.png", Constants.MediaType.Png)]
[TestCase("foo.png", Constants.MediaType.Png)]
[TestCase("/foo/foo.png", Constants.MediaType.Png)]
[TestCase("/foo/foo.PNG", Constants.MediaType.Png)]
[TestCase(".png", Constants.MediaType.Png)]

[TestCase("foo.gif", Constants.MediaType.Gif)]
[TestCase("/foo/foo.gif", Constants.MediaType.Gif)]
[TestCase("/foo/foo.GIF", Constants.MediaType.Gif)]
[TestCase(".gif", Constants.MediaType.Gif)]
[TestCase("", Constants.MediaType.Jpg)]
public void Can_Get_Media_Type(string input, Constants.MediaType expectedValue)
{
    var type = input.GetMediaType();

    Assert.AreEqual(expectedValue, type);
}

Ah! As you see we only have to write the method once and pass in as many variations as we'd like to test. Each attribute we add will translate into it's own test in the test explorer on the left:

For a lot of my unit tests, this is all that is needed; given a value as input, is the output as what is expected? If you're method requires a context or an object at runtime, you'll have to get creative inside your test to get the object you need for a method signature. Again, that is outside of the scope of this article, perhaps in the future as I figure out better ways to pull that off myself.

Our second test (smoke test)

Our second test is something that I'd qualify as a smoke test. We are running Umbraco as a single core with several websites/domains hanging off of it. Every time I push code I'd have to wait until the deployment is complete then visit each site to verify that yes indeed, the page is still up and appears to be working. That's at least 8-10 pages I'd have to visit which is very boring to do. I asked myself, why not write a test to visit each page and verify that the page is loading with a 200 status code and/or not returning a server error message? So let's look at that sort of test:

using System.Net;
using System.Net.Http;
using MyNamespace.Helpers;
using NUnit.Framework;

namespace MyNamespace.DigitalMedia
{
    [TestFixture]
    [Category("Smoke")]
    [Category("Digital Media")]
    public class SmokeTests
    {
        [TestCase(Domains.DIGITAL_MEDIA_PRODUCTION)]//a constant that resolves to a domain name
        [TestCase(Domains.DIGITAL_MEDIA_PREPRODUCTION)]
        [TestCase(Domains.DIGITAL_MEDIA_LOCAL)]
        public void Can_Reach_DigitalMedia(string domain)
        {
            //this is a custom helper that uses HttpClient to visit the given domain
            HttpResponseMessage response = WebHelper.Get(domain).Result;

            //require that the status code is 200
            Assert.AreEqual(HttpStatusCode.OK, response.StatusCode);

            //crawl through the text with HtmlAgilityPack and look for certain words that indicate an issue
            Assert.That(!WebHelper.ContainsServerErrorText(response));
        }
    }
}

So now my smoke test can reach out to my local, preproduction and production domains and verify that the pages are indeed up and not showing any error text. Does this prove there are no issues? Nope. Does it cause alarm if these start failing on preproduction? Yep and it's designed to be an early warning system before you push code to production. I repeat this sort of test for my additional domains and the tests can all run just after I deploy my code:

Our third test (functional)

How can you test to see if someone can log into a protected website? Well, it seems I've been able to come up with a test that tests that functionality out. What made this test especially difficult is the fact that I'm using Umbraco which handles form submission a little differently along with a CSRF token that needs to be submitted. To be sure, this doesn't prove all cases to be true, but it does prove that someone can login. To pass this test, the test must login and verify that the link to logout includes the users email address in the source that returns. You can adjust yours to your own needs. This test was quite tricky as I had to visit the page, scrape the CSRF token along with another hidden property found on Umbraco forms named 'ufprt'. I don't feel comfortable posting the complete code due to the fact that it might help someone spam forms, so a redacted version is provided here:

[TestCase(Domains.MY_PREPRODUCTION,
    "Targaryen, Daenerys",
    "[email protected]",
    "[email protected]$"
)]

[TestCase(Domains.MY_PRODUCTION,
    "Tarly, Samwell",
    "[email protected]",
    "[email protected]"
)]

//the test
public void Ensure_User(string domain, string email, string password)
{
    _ensureLogin(domain, email, password);
}

//create an HttpClient and call additional helpers
private void _ensureLogin(string domain, string email, string password)
{
    using (var client = new HttpClient())
    {
        client.BaseAddress = new Uri(domain);

        _ensureLogout(_login(client, email, password), email);
    }
}

//handle the login and grab the markup after success
private string _login(HttpClient client, string email, string password)
{
    //secret code removed

    return "this will be markup of the first page after you login successfully";
}

//verify that the page text contains the expected markup proving success
private void _ensureLogout(string resultString, string email)
{
    var htmlDocument = new HtmlDocument();
    htmlDocument.LoadHtml(resultString);

    var logoutNode = htmlDocument.DocumentNode.SelectSingleNode(string.Format("//a [@href='{0}']", "/link/to/logout"));

    Assert.That(logoutNode, Is.Not.Null);
    Assert.That(logoutNode.InnerText.ToLower().Contains(email.ToLower()));
}

Ok, so now we can test out whether an end-to-end behavior is working as expected. I've also created methods that reach out to management API's that I've built that allow for a test account to be 'reset' to a test condition on a remote server. We'll cover a use-case of that in the next section.

Our last test (task)

So as I mentioned before, this isn't really a test. This is more of making NUnit do something that it wasn't intended on doing (that I know of). These 'tests' simply perform a task such as republish a site remotely. I typically run these tests right after a deployment so I don't have to manually login and perform these tasks via the UI. Let's look at my publish example:

[Test] //no need for test cases here
public void Update_Cache_On_Preprod()
{
   //preprod only
 
  //let's visit a URL and send a shared key (over HTTPS) with a helper method I wrote
   var response = WebHelper.Get(Domains.MY_PREPRODUCTION + "/umbraco/api/somecustomcontroller/updatecache", true).Result;
   
   //make sure you got a thumbs up
   Assert.AreEqual(HttpStatusCode.OK, response.StatusCode);

   //test for a specific response
   Assert.That(response.Content.ReadAsStringAsync().Result.Contains("Blast off"));
}

To reiterate, I'm using this test to reach out to a server authenticating with a shared key over HTTPS and it runs a particular method. The net effect is that I can perform an normally trivial administrative task via a test. The endpoint on the server is just a typical controller that will only execute should the provided key be presented.

Summary

Ok, so there we have it. I'm learning how to use testing to my advantage and I'm certainly probably doing it wrong in the eyes of the testing gods but I don't care. My tests have caught issues on both fresh code and on remote systems. Better yet, I feel the tests are testing things relevant to what I'm building and not just building tests for testing sake. I'm leveling up everyday and feel like I'm starting to make progress in the realm of unit testing and will certainly pass along any pearls of wisdom should I stumble upon one. If you have any suggestions or feedback, please reach out on Twitter (oh and be nice).