Thinking Inside a Bigger Box

A jQuery inspired server side view model for Java

Johannes Brodwall — Tue, 07 May 2013 06:33:13 +0000

In HTML applications, jQuery has changed the way people thing about view rendering. Instead of an input or a text field in the view pulling data into it, the jQuery code pushes data into the view. How could this look in a server side situation like Java?

In this code example, I read an HTML template file from the classpath, set the value of an input field and append more data based on a template (also in the HTML file).

public class ContactServlet extends HttpServlet {
 
    @Override
    protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        String nameQuery = req.getParameter("nameQuery");
 
        // Read a pure html template from the classpath
        Xhtml document = Xhtml.fromResource(getClass().getResource("html/contact/index.html"));
 
        // Set the value of the nameQuery input field in the HTML
        document.getForm("#findForm").set("nameQuery", nameQuery);
 
        Element contactsEl = document.select("#contacts");
        Element contactTmpl = contactsEl.take(".contact");
        for (Contact contact : findContacts(nameQuery)) {
            // Fill in the .contact template of the HTML with data
            Element contactHtml = contactTmpl.copy().text(contact.print());
            // And add it to the #contacts element
            contactsEl.add(contactHtml);
        }
 
        document.write(resp.getWriter());
        resp.setContentType("text/html");
    }
}

This is a simplified version of the HTML:


  
	
    
  
  
    Johannes (4444444)

This is a third way from the alternatives of templated views like Velocity and JSP and from component models like JSF. In this model, the view, the model and the binding of the model variables to the view are all separate.

Disclaimer: In this example, I’ve used my still in pre-alpha XML library with the working name of Eaxy. You can get similiar results with libraries like jSoup and JOOX.

Caveat: I’ve never tried this on a grand scale. It’s an idea that compels me for three reasons: First, it’s very explicit. Nothing happens through @annotation, conventions or some special syntax in a template. Second, it’s very unit testable. There is nothing tying this code to running in a web application server. Finally, it’s easy to get to this code through incremental steps. I initially wrote the example application with code that embedded the HTML as strings in Java code and refactored to use the Java Query approach.

Could this approach be worth trying out more?

Scrum as an impediment to Agility

Johannes Brodwall — Wed, 24 Apr 2013 08:07:26 +0000

As I’m working with smaller and more agile projects, I’m increasingly seeing the classic way that Scrum is executed as more of an impediment to agility than a helper.

This is especially the case when it comes to the classic Sprint Planning as described in the Scrum Guide:

“For example, two-week Sprints have four-hour Sprint Planning Meetings”
In the Sprint Planning Meeting part 1: “Development Team works to forecast the functionality that will be developed during the Sprint.”
In the Sprint Planning Meeting part 1: “Product Owner presents ordered Product Backlog items”
“Work planned for the first days of the Sprint by the Development Team is decomposed to units of one day or less by the end of this meeting”

I’ve seen many sprint planning meetings struggle for the same reasons again and again:

The user stories described by the product owner doesn’t fit the team’s way of working
The team dives into too many details on each user story to be able to break it down to the level required
The team blames the product owner for not providing enough details to the user stories
Most of the design discussions are considered to be over once the sprint starts
The forecasting/commitment to future velocity becomes a heated negotiation

If your project experienced these sorts of Sprint planning meetings, I would expect that the reaction of the project was to add meetings (“backlog grooming”), documentation and checkpoint prior to starting a new sprint. These activities would probably resulted in the product owner (team) spending less amount of time with the development team.

Scrum’s Sprint planning is assuming a situation where the product backlog is detailed for a considerable amount of time and where the ideal is that the product owner spends their time adding more details to the product backlog all the time.

The resulting projects have huge rigid backlogs describing the details for several months into the future. They communication between the users and developers is limited to the acceptance criteria that the product owner writes down before each sprint planning. They spend a considerable amount of the sprint planning the rest of the sprint. Deviations from the sprint backlog are considered problematic.

I think this is misguided. I think this is why we left waterfall in the first place.

In order for Scrum to work better, we have to abandon the idea that the product owner comes to the planning with a perfect set of stories, we have to abandon the sprint backlog detailing the work and design for several weeks and we probably should be very careful with what estimates we ask for.

Instead I would suggest the following approach to planning a sprint:

The product owner and the team comes into the room informed by their current understanding of the value the system can deliver
The product owner describes the current most important gaps in the value available to stakeholders
The team already knows their current trajectory and together with the product owner, they can describe “what’s the next meaningful thing we could demonstrate to closing these gaps” as a script for the next demonstration
The team isn’t asked to estimate their work, but the product owner, project managers and others are free to make qualified guesses based on the team’s past performance
Keep it short and frequent!

Scrum was developed in the time where it had to match the perception of projects that did huge batches of planning and design. In response, it does smaller batches of planning and design. But “give a man an inch and he’ll take a yard”. The smaller batches leads to frustration over lack of details and the sprints become more and more plan driven and the connection between the users and the developers more and more document-driven.

A new approach is needed.

How to start an agile project

Johannes Brodwall — Mon, 08 Apr 2013 09:57:29 +0000

During the recent panel debate in Colombo Agile Meetup my colleague Lasantha Bandara asked the following question:

How do you start an agile project and ensure room for future enhancements? How can we achieve flexibity at the beginning?

This is my answer:

Flexibility is about having an acceptable cost of change. Sometimes, the best cost of change is to create something and throw it away early to try something else if it doesn’t work out.

The first technical decision I make on projects is what I call the deployment model. That is: How will the users access the application and where will the data reside. The most common categories of deployment model include web, disconnected client, rich client with server, and mobile client. There are other less common as well, for example an email responder. A mobile web client could also be considered a category of its own and so may perhaps a rich JavaScript web application.

As Sabri Sawaad said during in the panel: You must make decisions, but consider the cost of changing them.

You have to choose a deployment model and client-server communication protocol before you write any code that you can demo. Often, the project comes with an assumed deployment model, but it’s not always set in stone. For example, a project where the customer was aiming for a traditional web application, we suggested a JavaScript application due to the skillset of the developers. In another project, we challenged the customer to try a rich client instead of a web application, but after the first sprint, we discarded the code and did the web application instead. In a final example, discussing the requirements further with the client it turned out that a web client was more appropriate than the initial idea of a mobile application, so we had to replace team members to get the correct skillset.

As an architect, it’s part of your job to help the customer find a deployment model that matches the requirements of the users and the skillset of the team.

On the panel, Subuki Shihabdeen and Buddhima Wickramasinghe both pointed out the importance of early feedback. In my opinion, this implies that the beginning of a project is a bad time to learn new technologies. When you know the deployment model, use the technologies and frameworks you are familiar with to quickly show something to the customer. If this means delaying using a database, as Hasith Yaga suggested in the panel, do this. If it means using Entity Framework and SQL Server, because the team is familiar with this, do that. But as Sabri said: Invest in strategies to minimize the cost of changing these decisions, such as encapsulating the data layer.

In order to quickly choose and roll out a first demo, the only think I’ve found to work is to practice. I’ve created applications repeatedly from scratch in many frameworks and languages for practice. Each time I do it faster.

So here is my process, summarized:

Spend time to practice technologies so you can choose good options with confidence
Help the customer choose an appropriate deployment model based on the user needs and developer skills (including your own)
Put something in front of the customer quickly by using your existing skills
Invest to reduce the cost of changing decisions on frameworks and technologies
Change course if you find a better technology or even deployment model

A canonical Repository test

Johannes Brodwall — Fri, 08 Mar 2013 07:31:58 +0000

There are only so many ways to test that your persistence layer is implemented correctly or that you’re using an ORM correctly. Here’s my canonical tests for a repository (Java-version):

import static org.fest.assertions.api.Assertions.*;
 
public class PersonRepositoryTest {
    private PersonRepository repository; // TODO < == you must initialize this
 
    @Test
    public void shouldSaveAllProperties() {
        Person person = randomPerson();
        repository.save(person); // TODO: Make sure your repository flushes!
        assertThat(repository.find(person.getId())
            .isNotSameAs(person)
            .isEqualTo(person)
            .isEqualsToByComparingFields(person);
    }
 
    @Test
    public void shouldFindByCaseInsensitiveSubstringOfName() {
        Person matching = randomPerson();
        Person nonMatching = randomPerson();
        matching.setName("A. Matching Person");
        nonMatching.setName("A. Random Person");
        repository.save(matching);
        repository.save(nonMatching);
        assertThat(repository.findByNameLike("MATCH"))
            .contains(matching)
            .doesNotContain(nonMatching);
    }
}

Very simple. The randomPerson test helper generates actually random people:

public class PersonTest {
    // ....
    public static Person randomPerson() {
        Person person = new Pesron();
        person.setName(randomName());
        // TODO Initialize all properties
        return person;
    }
 
    public static String randomName() {
        return RandomData.randomWord() + " " + RandomData.randomWord() + "son";
    }
}
 
 
public class RandomData {
    public static String randomString() {
        return random("foo", "bar", "baz", "qux", "quux"); // TODO: Add more!
    }
 
    public static <T> T random(T... options) {
        return options[random(options.length)];
    }
 
    public static int random(int max) {
        return random.nextInt(max);
    }
 
    private static Random random = new Random();
}

If your data has relationships with other entities, you may want to include those as well:

public class OrderRepositoryTest {
    private OrderRepository repository; // TODO < == you must initialize this
    private PersonRepository personRepository; // TODO <== you must initialize this
 
    private Person person = PersonTest.randomPerson();
 
    @Before
    public void insertData() {
        personRepository.save(person);
    }
 
    @Test
    public void shouldSaveAllProperties() {
        Order order = randomOrder(person);
        repository.save(order); // TODO: Make sure your repository flushes!
        assertThat(repository.find(order.getId())
            .isNotSameAs(order)
            .isEqualTo(order)
            .isEqualsToByComparingFields(order);
    }

A simple and easy way to simplify your Repository testing.

(The tests use FEST assert 2 for the syntax. Look at FluentAssertions for a similar API in .NET)

(Yes, this is what some people would call an integration test. Personally, I can't be bothered with this sort of classifications)

Sweet C# test syntax

Johannes Brodwall — Fri, 22 Feb 2013 11:23:48 +0000

Two of my favorite libraries in C#: FluentAssertions and NUnit (of course). NUnit has a “hidden” gem (that is, it’s well documented, yet few developers use it): [TestCase]. Look at this:

using FluentAssertions;
using NUnit.Framework;
 
public class RomanNumeralsTest
{
    [TestCase(3333, "MMMCCCXXXIII")]
    [TestCase(555, "DLV")]
    [TestCase(999, "CMXCIX")]
    [TestCase(444, "CDXLIV")]
    public void ItConvertsNumbersToRomanNumerals(int number, string roman)
    {
        ToRoman(number).Should().Be(roman);
    }
}

The [TestCase] lets us easily have several test cases implemented by the same method. FluentAssertions adds the Should() extension method to all objects (and primitives, too) that let’s us write things like ToRoman(3333).Should().Be("MMMCCCXXXIII").

Add some sugar to your C# tests today. What’s your favorite ways of writing sweeter tests?

Better Scrum sprint planning – look to the demo

Johannes Brodwall — Fri, 08 Feb 2013 09:28:06 +0000

After having worked with Scrum for a number of years, I still witness sprint reviews where the team’s demonstration of the product is confusing and the value produced in the sprint is unclear. The demo may consist of just a bunch of different functions and screens without any meaning. Or maybe the team is just talking about what happens behind the curtains in the database. Or maybe the demo just doesn’t display the value that the team was supposed to give to the stakeholders. Most teams have okay demos most of the time, but every now and then, it’s a complete train wreck.

If you’ve experienced a sprint like this, you probably noticed some problems from the very beginning. The sprint planning may have been chaotic and the work during the sprint may have felt purposeless. Chances are that the team was most of the time discussing technical terms that didn’t make much sense to the product owner.

If your team learns to be clear about the sprint goal, you can avoid anemic demos, unstructured planning and undirected work. “But wait”, you may say, “we had a sprint goal and our demo still sucked”. You may think you have a clear sprint goal, but very few teams know what a sprint goal looks like. You may have one, but you might have gotten there by accident.

Here is what a sprint goal looks like: At the end of this period of time, we will demonstrate something to our stakeholders. What we will demonstrate will tell a compelling story that demonstrates real value. We can create the first draft plan of what the demo will look like already at the sprint planning, and we can use this description both to verify our understanding and plan our work.

Let’s say that the product owner says “the goal for the next sprint is to verify the payment solution.” What would a plan for this sprint look like?

At a sprint planning meeting, after the product owner has described the goal, the team plans its work. Then they come back to the product owner to verify that their plan matches the goal. This is what a good plan may sound like: “We will set up so that all users on the web site has a random set of items in their shopping cart. Then we will go to the checkout page. Here, we will see that the shopping cart is displayed in a reasonable way. When we click the payment button, the user will be redirected to the test site for payment provider. We’ll input credit card details and pay. The user will be redirected back to the web site and the web site will display the success or the failure of the payment. We’ll also show the order along with the payment status in a early mockup of the order list page”

The product owner may agree to this sprint plan. If the team knows their technologies well, it is now easy to break this down into tasks, such as “create a shopping cart model”, “display shopping cart page”, “retrieve the payment status from the payment provider” and “store the payment status in the database”.

This demo script will guide the team both during the construction and during the actual sprint review. During the construction a team member is now in a position to solve the sprint goal in the simplest way possible. By focusing on how the team will demonstrate value instead of what technical tasks may or may not be required (e.g. “construct a new order facade service” – whatever that may mean!) we can dramatically cut down on wasteful and convoluted design.

Agile methods emphasize “adapting to change over following a plan”. The same holds true for a demo script. The purpose of the script is not to create a perfect plan (which is of limited value), but to get a clear picture of what we need to create and how we will demonstrate that we have indeed delivered real value.

Let’s reinvent more wheels!

Johannes Brodwall — Thu, 31 Jan 2013 10:21:48 +0000

When I learned math in elementary school, I would reach for my calculator. But my father stopped me: “You only get to use the calculator when you can do math without it.” As you can imagine, at the time I though this was unfair and unreasonable, but later I have discovered what advantage it is to understand the basics before you reach for a powerful tool.

Many developers are focused on what fancy framework they will learn next. Or what programming language might solve all their problems. Before we reach for the tools, it’s useful to learn how they really work. My motto is: “I will not use a framework I would be unable to recreate.” It is of course, too time consuming to create a framework as complete as many of those available, but I should at least be able to solve all the normal cases myself.

Having recreated the behavior of a framework means that I will understand how the framework is implemented. I get better intuition about how to use the framework, I understand more quickly what the problem might be when something doesn’t work, and last, but not least: I understand when the framework is harming me more than it’s helping me.

An example I have enjoyed using is to create a web application in Java without framework. I may use a simple domain like creating an address book where you can register your contacts and search for them. In honor of my C#-friends, I have solved the same task in C#: How to make a web application without MVC, without ASP.NET or even without IIS.

Test-driven development is an essential tool for me to think clearly. I’ve made an exception from the calculator rule above and used a few testing libraries: SimpleBrowser.WebDriver, FluentAssertions and NUnit. This test demonstrates the behavior that I want from the application when I’m done:

[Test]
public void ShouldFindSavedPerson()
{
    // Start a web server INSIDE THE TEST :-D
    var server = new My.Application.WebServer();
    server.Start();
 
    var browser = new SimpleBrowser.WebDriver.SimpleBrowserDriver();
    browser.Url = server.BaseUrl;
 
    // Navigate to the "add contact" page
    browser.FindElement(By.LinkText("Add contact")).Click();
 
    // Add a new contact
    browser.FindElement(By.Name("fullName")).SendKeys("Darth Vader");
    browser.FindElement(By.Name("address")).SendKeys("Death Star");
    browser.FindElement(By.Name("saveContact")).Submit();
 
    // Navigate to the "find contact" page
    browser.FindElement(By.LinkText("Find contact")).Click();
 
    // Execute some queries:
    browser.FindElement(By.Name("nameQuery")).SendKeys("vader");
    browser.FindElement(By.Name("nameQuery")).Submit();
    browser.FindElement(By.CssSelector("#contacts li")).Text
        .Should().Be("Darth Vader (Death Star)");
    browser.FindElement(By.Name("nameQuery")).SendKeys("anakin");
    browser.FindElement(By.Name("nameQuery")).Submit();
    browser.FindElements(By.CssSelector("#contacts li"))
       .Should().BeEmpty();
}

I add an empty class for My.Application.WebServer and the test will fail at the line browser.Url = server.BaseUrl as there is not real server.

To implement the server, I’m using a cute small class which is part of the .NET core library: System.Net.HttpListener. Here are the essentials:

class WebServer
{
    public void Start()
    {
        var listener = new System.Net.HttpListener();
        listener.Prefixes.Add(BaseUrl);
        listener.Start();
        new Thread(HttpThread).Start(listener);
    }
 
    private void HttpThread(object listenerObj)
    {
        HttpListener listener = (HttpListener)listenerObj;
        while (true)
        {
            var context = listener.GetContext();
            using (context.Response)
            {
            }
        }
    }
}

Running the test again, I get one step further. This time, I am told that the test can’t find the link to “Add contact”. No big surprise, as I’m not serving any HTML! A small change in the WebServer code will fix this:

    var context = listener.GetContext();
    using (context.Response)
    {
        new AddressBookController().Service(context);
    }

Then we just have to create a simple implementation for AddressBookController.Service:

class AddressBookController
{
    internal void Service(HttpListenerContext context)
    {
        var html = "" +
            "Add contact
" +
            "Find contact
" +
            "";
        var buffer = Encoding.UTF8.GetBytes(html);
        context.Response.OutputStream.Write(buffer, 0, buffer.Length);
    }
}

Again, the test will get one step further. Now we can see that the main page is presented with the links “Add contact” and “Find contact“. After Click()ing “Add contact” the test will of course fail to find the field fullName as we have not created the form yet. The method HandleGetRequest inspects the URL to determine which page should be displayed:

internal void Service(HttpListenerContext context)
{
    var html = HandleGetRequest(context.Request);
    var buffer = Encoding.UTF8.GetBytes(html);
    context.Response.OutputStream.Write(buffer, 0, buffer.Length);
}
 
private string HandleGetRequest(HttpListenerRequest request)
{
    if (request.Url.LocalPath == "/contact/create")
    {
        return "" +
                "" +
                "
" +
                "
" +
                "
" +
                ""
 +
                "";
    }
    else
    {
        // As before
    }
}

We’re almost done saving contacts. The test fails to find the link “Find contact” after submitting the form. The method Service must be modified to handle POST requests and perform a redirect back to the menu:

internal void Service(HttpListenerContext context)
{
    if (context.Request.HttpMethod == "GET")
    {
        var html = HandleGetRequest(context.Request);
        var buffer = Encoding.UTF8.GetBytes(html);
        context.Response.OutputStream.Write(buffer, 0, buffer.Length);
    }
    else
    {
        context.Response.Redirect(context.Request.Url.GetLeftPart(UriPartial.Authority));
    }
}

We are still missing the form to search for contacts. We’ll get some help from the copy/paste pattern:

private string HandleGetRequest(HttpListenerRequest request)
{
    if (request.Url.LocalPath == "/contact/create") ...
    else if (request.Url.LocalPath == "/contact/")
    {
        return "" +
                "" +
                "
" +
                "
" +
                ""
 +
                "";
    }
    else ...
}

The next error is obvious – we need to actually include the contacts in the response:

    class Contact
    {
        public string FullName { get; set; }
        public string Address { get; set; }
    }
 
    private static List<Contact> contacts = new List<Contact>();
 
    private string HandleGetRequest(HttpListenerRequest request)
    {
        else if (request.Url.LocalPath == "/contact/")
        {
                var contactsHtml = string.Join("",
                    contacts.Select(c => "" + c.FullName + " (" + c.Address + ")
")))
                return string.Format("" + ...
                        "{0}
" +
                        "", contactsHtml);
 
        }

The only thing missing is to store the contacts when we post the “Add contact” form:

    internal void Service(HttpListenerContext context)
    {
        if (context.Request.HttpMethod == "GET") ...
        else
        {
            // Read the parameters from the POST body (Request.InputStream)
            var request = context.Request;
            var encoding = context.Request.ContentEncoding;
            var reader = new StreamReader(context.Request.InputStream, encoding);
            var parameters = HttpUtility.ParseQueryString(reader.ReadToEnd(), encoding);
 
            context.Response.Redirect(HandlePostRequest(request, parameters));
        }
    }
 
    private string HandlePostRequest(HttpListenerRequest request, NameValueCollection parameters)
    {
        contacts.Add(new Contact() { FullName = parameters["fullName"], Address = parameters["address"] });
        return request.Url.GetLeftPart(UriPartial.Authority);
    }

One last check is failing: We’re failing to filter contacts to the query:

    private string HandleGetRequest(HttpListenerRequest request)
    {
        if (request.Url.LocalPath == "/contact/create") ...
        else if (request.Url.LocalPath == "/contact/")
        {
            var query = request.QueryString["nameQuery"];
            var contactsHtml = string.Join("",
                contacts
                    .Where(c => query == null || c.FullName.ToLower().Contains(query.ToLower()))
                    .Select(c => "" + c.FullName + " (" + c.Address + ")
"));
            return string.Format("" +
                    ...
                    "{0}
" +
                    "", contactsHtml);
        }
        else ...
    }

All that remains to turn this into a real application is to use a real database and correct the obvious security vulnerability when we display contacts. The AddressBookWebServer could have a Main method to enable you to run the code. But I’ll leave these issues as an exercise to you, dear reader.

This article has showed how HTTP really works and how frameworks like ASP.NET MVC work behind the curtains. There are many details that we’re glad that the framework can fix for us, like the character encoding and reading the contents of a POST request. And there are many things that turn out to be not as hard as you’d think, like a real “redirect-on-post” pattern. In more than one project, I’ve realized that after spending a few days understanding the underlying technology, I could deliver the project much better and faster without the “obvious”, popular frameworks that everyone recommend that you use.

Did I reinvent the wheel in this article? You could argue that I did, but let me stretch the metaphor of “reinventing the wheel” as far as possible:

My experience is that many “cars” today have misaligned wheels where the axel isn’t mounted in the center. Maybe the wheel was poorly constructed, or maybe the car was just assembled wrong. Maybe we notice that the car is bouncing because two of the wheels have a misaligned axel. And then we spend a lot of work trying to adjust these wheels to synchronize the bouncing. Finally we publish articles about the nice even undulations of our car after aligning the errors in the wheels.

If we have some experience contructing one or two “wheels”, it’s possible that we’re able to identify the real problems with the “wheels” we were given, so we can determine which “wheels” are good and which “wheels” are bad. Not to mention: We may learn how to use them properly.

Reinvent wheels you don’t understand, don’t use a framework you couldn’t have made yourself and don’t use a calculatore before you understand math.

This article was previously published in Norwegian in Torbjørn Marø’s programming blog

If you’re an architect, knowledge is your enemy

Johannes Brodwall — Thu, 13 Dec 2012 19:51:43 +0000

When a software architect gets a good idea or learns something new, he has a problem. The main job of the architect is to ensure that the right information in present inside the heads of the people who should build the application. Every new piece of information in the architect’s head represents a broader gap between his brain and that of the rest of the team.

The classical ways of adressing this gap is for the architect to write huge documents or sets of wiki pages. When they realize that there’s not sufficient time set aside in the project schedule for the developers to read all this information, the architect may present the material to developers who sit and nod their heads. But what did they really understand?

Instead of the read-listen-and-nod approach, I prefer an approach that I sometimes call “dragging the information throught the heads of the team and looking at what comes out in the other end.” I provide as little processed information as possible, but instead give the team a structured workshop to uncover and structure the information by asking me or business stakeholders. The outcomes of the workshop should be some tanglible results presented by the team. This result is always different from what I had in mind. Sometimes the difference shows a critical misunderstanding, which allows me to go more in depth in this area. Sometimes the difference represents a trivial misunderstanding or difference in opinion and the architect has the difficult task of accepting a small disgreement without distracting the team. Sometimes, the team has discovered something much smarter than the original idea of the architect.

I find it most useful to do workshops in small groups of three people per group. Each group should produce something that they can show to the whole team afterwards. Here are some examples of workshops that I run:

Divide in groups of three with the users/business and the developers represented in each group. Each group should discuss and fill in a template for the vision of the product being created: “For some user who performs some business function the name of system is a type of system which gives a capability related to the task. Unlike most interesting alternative our solution has an important advantage“. The groups get 10 minutes before a debrief with the whole team.
Each group then brainstorms a list of users, consumers and others affected by the system and write these on sticky notes. This should be about 20-30 roles. The whole team decides on a few interesting users and the groups then write down for some these: What characterizes the user, what tasks do they perform and what do they value?
Based on the list of tasks that stakeholders perform, we create a sketch of a usage flow. I like to refine the documented usage flow with a small task group which takes a few hours to prepare a description of the flow of interaction between the system and external actors
Groups of three go through the usage flow to come up with Actors (users and systems), Domain concepts (classes) or Containers (deployment diagram) mentioned or implied in the usage flow and write these on sticky notes. After showing the Actors, Concepts or Containers to the whole group, each workgroup then organizes these on flipcharts to create a Context Model, a Domain Model and a Deployment Model.

Many of these workshops can also be run with distributed groups over video conference and screen sharing.

I like to collect all of these artifacts (vision, users, usage flow, context model, domain model and deployment model) in a PowerPoint presentation so it can be easily showed by the team to external stakeholders. Sometimes someone on the team feel that photographed flipcharts with sticky notes are too informal and decide to draw something in Visio or another fancy tool. This is just a plus.

By asking the team to produce something and present it, rather than explaining the architecture to the team, I ensure that the information is really in their heads and not just my fooling myself by my own understanding.

The Rainbow Sprint Plan

Johannes Brodwall — Wed, 24 Oct 2012 16:40:57 +0000

Do you ever feel it’s hard to get real progress in a sprint towards the business goal? Do you feel the feedback from a iteration picks on all the details you didn’t mean to cover this sprint? Do you feel like sprint planning meetings are dragging out? Then a Rainbow Sprint Plan may be for you.

Here is an example of a Rainbow Sprint plan:

A customer wants cheap vacations
The customer signs up for daily or weekly notifications of special flight offers
Periodically the System checks which customers should get notifications
The System checks for offers that matches the customer’s travel preference by looking up flights with the travel provider system
The System notifies customer of any matching offers via SMS
1. Variation: The System notifies customer of any matching offers via email
The customer accepts the offer via SMS
The System books the tickets on behalf of the customer
The system confirms the booking by sending an SMS to the customer
The customer can at any point see their active offers and accepted offers on the system website
The customer enjoys a cheap vacation!

What you can see from this plan:

Use case overview: The plan gives a high-level picture of the next release. We can see how the work we are doing is fitting together and how it ends up satisfying a customer need. This is a requirement technique that is basically Use Cases as per Alistair Cockburn’s “Writing Effective Use Cases“. I’ve been writing use cases at this level for the last three years and found it to be a good way to understand requirements. The trick of good use cases is to stay at a the right level. In this example, each step is some interaction between the system and a user or the system and another system. How this communication is handled is something I find best to leave for an individual sprint.

Iterative completion: Each step has a color code:

Black: The team hasn’t started looking into this
Red: We have made something, but it’s a dummy version just to show something
Orange: We have made something, but we expect lots of work remaining
Yellow: We’re almost done, we’re ready to receive feedback
Green: Development is complete, we have done reasonable verification and documentation

So the plan accepts that we revisit a feature. As we get closer to the next release, things will move further and further into the rainbow. But we can choose whether we want to get everything to orange first, or whether we will leave some things at red (or even black) while bringing other steps all the way to green.

Demonstration script: When we get to the end of the sprint and demonstrate what we’ve created, this plan gives a pretty good idea of what the demo will look like: We will sign up the customer to a dummy signup page (red), we will register some flights in another dummy page (red), trigger the actual scheduling code (orange), then we will see that an SMS is received on an actual phone (yellow). Then we will simulate an SMS response (orange), see that they system made some communication to a dummy system (red), and send “ok” back as an SMS to the customer (orange). This will focus the team around a shared vision of what to do in this sprint.

I have been thinking in terms of a Rainbow Plan in my last projects, but I’ve never used the term before. I think the plan addresses three of the most common problems that I see in Scrum implementations:

The team doesn’t see where it’s going, because user stories are too fine grained to get the big picture. User story mapping and use cases address this, and rainbow plans put it into a sprint-context
The team dives into technical details during sprint planning. With rainbow plans, the sprint plan becomes the demo plan which coincides with the requirements.
The project has a purely incremental approach, where each feature should be completed in a single sprint. This means that it’s hard to keep the big picture and the product owner is forced to look for even small bugs in everything that’s done in a sprint. With rainbow plans, the team agrees on the completeness of each feature.

May you always become more goal oriented and productive in your sprints.

Loud failures are better than silent, faulty behavior

Johannes Brodwall — Fri, 12 Oct 2012 08:11:05 +0000

Sometimes, small questions lead to big answers. Sometimes these answers are controversial. One such question is “What does this warning about serialVersionUID mean”? All the advice out there basically is for developers who don’t know what’s going on to write code that will ignore errors when something unexpected happens. In my view – this is exactly the wrong approach. The safe way to act is to make sure that your program crashes if you don’t have control.

Java programmers usually get this warning when they write code that looks like this:

public class MyServlet extends HttpServlet {
 
  public void doGet(HttpRequest req, HttpResponse resp) {
    // Some logic goes here
  }
}

On stackoverflow this question has an answer that is extensive, well-written, accepted and, in my opinion, wrong. In short, the answer just recommends to implement something like the following:

public class MyServlet extends HttpServlet {
 
  private static final long serialVersionUID = 123L;

Let’s dig deeper.

The reason we get this warning is that HttpServlet implements the interface Serializable. This was a design flaw in the first version of the servlet-api, and now we’re stuck with it. A serialized object can be written to and read from byte streams, such as a file. Here is an example:

        ObjectOutputStream outputStream = new ObjectOutputStream(new FileOutputStream(file));
        outputStream.writeObject(new Person("Johannes", "Brodwall"));
 
       // ... somewhere else:
 
        ObjectInputStream inputStream = new ObjectInputStream(new FileInputStream(file));
        Object object = inputStream.readObject();
        System.out.println(object);

In this case, everything is fine. But let’s imagine that some time passes between the write and the read. For example, what if we try to read the file with the next version of the program. A version in which the Person class is changed? For example, what if we changed the implementation of Person to store the name as a single field fullName, instead of two fields firstName and lastName?

In this case, we would get an error message like the following:

java.io.InvalidClassException: Person; local class incompatible:
                             stream classdesc serialVersionUID = -4897183855179110397,
                             local class serialVersionUID = -1928642322738440913

In other words: If we had set the serialVersionUID, we could still have read the file. Now we’re stuck.

This is why the stackoverflow answer recommends putting a serialVersionUID field in the class.

But wait, there’s another option. Let’s say we found this problem when we tested if our new version was backwards compatible. Now, we could just cut and paste the serialVersionUID from the stack trace. If we do test our software, this is just as easy as putting the serialVersionUID there in the first place. So, let’s fix the Person class:

public class Person implements Serializable {
 
    private static final long serialVersionUID = -4897183855179110397L;
    private final String fullName;
 
    public Person(String firstName, String lastName) {
        this.fullName = firstName + " " + lastName;
    }
 
    @Override
    public String toString() {
        return getClass().getSimpleName() + "[name=" + fullName + "]";
    }
}

Voila! Problem fixed. Our code will run again. Here’s the output of printing the object:

Person[name=null]

Whoops! By forcing different versions of class Person to have the same serialVersionUID, the code now has a serious bug. FullName should never be null (especially since it’s final!). And what’s worse, the bug is a silent one. If we don’t examine the contents of System.out (in this case), we might not catch it before it escapes into the wild.

When you’re not sure, the correct behavior should be to fail, not to silently do the wrong thing.

TL;DR

If you omit a serialVersionUID field from your class, many changes will cause serialized objects to no longer be readable. Even for trivial changes.

Sadly, classes like HttpServlet, AbstractAction and JFrame which are meant to be subclassed implements Serializable, even though they are almost never serialized in practice. Adding serialVersionUID to these classes would only be noise.

The serialVersionUID field can be added to a class afterward if you actually want to read old objects in a new version of the program. This leaves you no worse off than if you added the serialVersionUID field in the first place.

If the old and the new version of the class are deeply incompatible, giving the class a serialVersionUID when you first create it will cause silent faulty behavior.

I prefer loud, failing behavior that is easy to detect during testing over quiet, faulty behavior that may escape into production. I think you do, too.