Let's reinvent more wheels!

When I learned math in elementary school, I would reach for my calculator. But my father stopped me: “You only get to use the calculator when you can do math without it.” As you can imagine, at the time I though this was unfair and unreasonable, but later I have discovered what advantage it is to understand the basics before you reach for a powerful tool.

Many developers are focused on what fancy framework they will learn next. Or what programming language might solve all their problems. Before we reach for the tools, it’s useful to learn how they really work. My motto is: “I will not use a framework I would be unable to recreate.” It is of course, too time consuming to create a framework as complete as many of those available, but I should at least be able to solve all the normal cases myself.

Having recreated the behavior of a framework means that I will understand how the framework is implemented. I get better intuition about how to use the framework, I understand more quickly what the problem might be when something doesn’t work, and last, but not least: I understand when the framework is harming me more than it’s helping me.

An example I have enjoyed using is to create a web application in Java without framework. I may use a simple domain like creating an address book where you can register your contacts and search for them. In honor of my C#-friends, I have solved the same task in C#: How to make a web application without MVC, without ASP.NET or even without IIS.

Test-driven development is an essential tool for me to think clearly. I’ve made an exception from the calculator rule above and used a few testing libraries: SimpleBrowser.WebDriver, FluentAssertions and NUnit. This test demonstrates the behavior that I want from the application when I’m done:

[Test]
public void ShouldFindSavedPerson()
{
    // Start a web server INSIDE THE TEST :-D
    var server = new My.Application.WebServer();
    server.Start();

    var browser = new SimpleBrowser.WebDriver.SimpleBrowserDriver();
    browser.Url = server.BaseUrl;

    // Navigate to the "add contact" page
    browser.FindElement(By.LinkText("Add contact")).Click();

    // Add a new contact
    browser.FindElement(By.Name("fullName")).SendKeys("Darth Vader");
    browser.FindElement(By.Name("address")).SendKeys("Death Star");
    browser.FindElement(By.Name("saveContact")).Submit();

    // Navigate to the "find contact" page
    browser.FindElement(By.LinkText("Find contact")).Click();

    // Execute some queries:
    browser.FindElement(By.Name("nameQuery")).SendKeys("vader");
    browser.FindElement(By.Name("nameQuery")).Submit();
    browser.FindElement(By.CssSelector("#contacts li")).Text
        .Should().Be("Darth Vader (Death Star)");
    browser.FindElement(By.Name("nameQuery")).SendKeys("anakin");
    browser.FindElement(By.Name("nameQuery")).Submit();
    browser.FindElements(By.CssSelector("#contacts li"))
       .Should().BeEmpty();
}

I add an empty class for My.Application.WebServer and the test will fail at the line browser.Url = server.BaseUrl as there is not real server.

To implement the server, I’m using a cute small class which is part of the .NET core library: System.Net.HttpListener. Here are the essentials:

class WebServer
{
    public void Start()
    {
        var listener = new System.Net.HttpListener();
        listener.Prefixes.Add(BaseUrl);
        listener.Start();
        new Thread(HttpThread).Start(listener);
    }

    private void HttpThread(object listenerObj)
    {
        HttpListener listener = (HttpListener)listenerObj;
        while (true)
        {
            var context = listener.GetContext();
            using (context.Response)
            {
            }
        }
    }
}

Running the test again, I get one step further. This time, I am told that the test can’t find the link to “Add contact”. No big surprise, as I’m not serving any HTML! A small change in the WebServer code will fix this:

    var context = listener.GetContext();
    using (context.Response)
    {
        new AddressBookController().Service(context);
    }

Then we just have to create a simple implementation for AddressBookController.Service:

class AddressBookController
{
    internal void Service(HttpListenerContext context)
    {
        var html = "" +
            "

[Add contact](/contact/create)

" +
            "

[Find contact](/contact/)

" +
            "";
        var buffer = Encoding.UTF8.GetBytes(html);
        context.Response.OutputStream.Write(buffer, 0, buffer.Length);
    }
}

Again, the test will get one step further. Now we can see that the main page is presented with the links “Add contact” and “Find contact”. After Click()ing “Add contact” the test will of course fail to find the field fullName as we have not created the form yet. The method HandleGetRequest inspects the URL to determine which page should be displayed:

internal void Service(HttpListenerContext context)
{
    var html = HandleGetRequest(context.Request);
    var buffer = Encoding.UTF8.GetBytes(html);
    context.Response.OutputStream.Write(buffer, 0, buffer.Length);
}

private string HandleGetRequest(HttpListenerRequest request)
{
    if (request.Url.LocalPath == "/contact/create")
    {
        return "" +
                "

" +
                "

" +
                "

" +
                "

" +
                "

" +
                "";
    }
    else
    {
        // As before
    }
}

We’re almost done saving contacts. The test fails to find the link “Find contact” after submitting the form. The method Service must be modified to handle POST requests and perform a redirect back to the menu:

internal void Service(HttpListenerContext context)
{
    if (context.Request.HttpMethod == "GET")
    {
        var html = HandleGetRequest(context.Request);
        var buffer = Encoding.UTF8.GetBytes(html);
        context.Response.OutputStream.Write(buffer, 0, buffer.Length);
    }
    else
    {
        context.Response.Redirect(context.Request.Url.GetLeftPart(UriPartial.Authority));
    }
}

We are still missing the form to search for contacts. We’ll get some help from the copy/paste pattern:

private string HandleGetRequest(HttpListenerRequest request)
{
    if (request.Url.LocalPath == "/contact/create") ...
    else if (request.Url.LocalPath == "/contact/")
    {
        return "" +
                "

" +
                "

" +
                "

" +
                "

" +
                "";
    }
    else ...
}

The next error is obvious - we need to actually include the contacts in the response:

    class Contact
    {
        public string FullName { get; set; }
        public string Address { get; set; }
    }

    private static List<Contact> contacts = new List<Contact>();

    private string HandleGetRequest(HttpListenerRequest request)
    {
        else if (request.Url.LocalPath == "/contact/")
        {
                var contactsHtml = string.Join("",
                    contacts.Select(c => "*   " + c.FullName + " (" + c.Address + ")
")))
                return string.Format("" + ...
                        "

{0}

" +
                        "", contactsHtml);

        }

The only thing missing is to store the contacts when we post the “Add contact” form:

    internal void Service(HttpListenerContext context)
    {
        if (context.Request.HttpMethod == "GET") ...
        else
        {
            // Read the parameters from the POST body (Request.InputStream)
            var request = context.Request;
            var encoding = context.Request.ContentEncoding;
            var reader = new StreamReader(context.Request.InputStream, encoding);
            var parameters = HttpUtility.ParseQueryString(reader.ReadToEnd(), encoding);

            context.Response.Redirect(HandlePostRequest(request, parameters));
        }
    }

    private string HandlePostRequest(HttpListenerRequest request, NameValueCollection parameters)
    {
        contacts.Add(new Contact() { FullName = parameters["fullName"], Address = parameters["address"] });
        return request.Url.GetLeftPart(UriPartial.Authority);
    }

One last check is failing: We’re failing to filter contacts to the query:

    private string HandleGetRequest(HttpListenerRequest request)
    {
        if (request.Url.LocalPath == "/contact/create") ...
        else if (request.Url.LocalPath == "/contact/")
        {
            var query = request.QueryString["nameQuery"];
            var contactsHtml = string.Join("",
                contacts
                    .Where(c => query == null || c.FullName.ToLower().Contains(query.ToLower()))
                    .Select(c => "*   " + c.FullName + " (" + c.Address + ")
"));
            return string.Format("" +
                    ...
                    "

{0}

" +
                    "", contactsHtml);
        }
        else ...
    }

All that remains to turn this into a real application is to use a real database and correct the obvious security vulnerability when we display contacts. The AddressBookWebServer could have a Main method to enable you to run the code. But I’ll leave these issues as an exercise to you, dear reader.

This article has showed how HTTP really works and how frameworks like ASP.NET MVC work behind the curtains. There are many details that we’re glad that the framework can fix for us, like the character encoding and reading the contents of a POST request. And there are many things that turn out to be not as hard as you’d think, like a real “redirect-on-post” pattern. In more than one project, I’ve realized that after spending a few days understanding the underlying technology, I could deliver the project much better and faster without the “obvious”, popular frameworks that everyone recommend that you use.

Did I reinvent the wheel in this article? You could argue that I did, but let me stretch the metaphor of “reinventing the wheel” as far as possible:

My experience is that many “cars” today have misaligned wheels where the axel isn’t mounted in the center. Maybe the wheel was poorly constructed, or maybe the car was just assembled wrong. Maybe we notice that the car is bouncing because two of the wheels have a misaligned axel. And then we spend a lot of work trying to adjust these wheels to synchronize the bouncing. Finally we publish articles about the nice even undulations of our car after aligning the errors in the wheels.

If we have some experience contructing one or two “wheels”, it’s possible that we’re able to identify the real problems with the “wheels” we were given, so we can determine which “wheels” are good and which “wheels” are bad. Not to mention: We may learn how to use them properly.

Reinvent wheels you don’t understand, don’t use a framework you couldn’t have made yourself and don’t use a calculatore before you understand math.

This article was previously published in Norwegian in Torbjørn Marø’s programming blog

Comments:

Manuel Palacio - Feb 2, 2013

I see your point but more often than not this is not practical.

Johannes Brodwall - Feb 2, 2013

Well, I am suggesting it as a learning exercise, not necessarily for commercial projects. That being said: how do you know it’s not practical?

[Nils Harald Berge] - Feb 4, 2013

Nice! Is there anyway to use HttpListener with MVC3/4 (and throw away IIS)? That would make my day….

Dileepa Jayasuriya - Feb 5, 2013

practical for me ,but by the time you are done with some useful project this way ,you would end up with your own mini javascript library/framework which would be optimized for such project

Johannes Brodwall - Feb 5, 2013

Sadly, the closest I’ve found is using System.Web.Http.SelfHost.HttpSelfHostConfiguration (hat tip to my colleague Adipa for that one). You give up considerable control once you introduce it, though.

I’d love to hear about something in between - basically, run a HttpListenerContext through the router etc.

Johannes Brodwall - Feb 5, 2013

True, Dileepa. I’ve been in this situation with Java. It had its upsides (very easy for me to get good test coverage) and downsides. The downsides are greater the less you know. If you create your own mini-framework a few times (and throw it away), you eventually get pretty good at it.

• C#
• Code
• English