Thinking Inside a Bigger Box

This blog post is a summary of my lightning talk at XP2011

I needed to fail with modern methods for requirement gathering in order to understand old methods for requirements gathering. Many software projects write requirements in what is refered to as “user stories”. But a use story is not a story at all. There’s no drama, no action and no resolution. Instead, user stories are often just a bunch of interactions between the user and the system laying in a big heap in a shoe box. Or even worse: In an issue tracking system. Using the form of use cases, a requirement form that had failed me in the past, I have reinserted the drama into my projects.

Humans have used stories to communicate and to think about the world for as long as we’ve had language. Some elements are common to all stories: A conflict upsets the order of the world. A hero enters the scene. Through the actions of the hero, the world is set back into order and the all is well in the world.

Here is a for a project that makes a webshop, the story of desire:

1. Some person craves stuff (the conflict)
2. The webshop(the hero) welcomes the craving user to its front page
3. The webshop lets the user picks stuff
4. The webshop lets the user pay
5. Some more stuff happens through the webshop so that the stuff is delivered
6. Even more stuff happens
7. The craving person gets his desire fullfilled and the world is set back to the way it was supposed to be

Not the most exciting story, you say? Well, you asked for software system requirements, not for literary masterpieces! The use case illustrates my point: A narrative lets us understand the purpose of the system (the hero) in the world and the operations the system needs to perform in order to fulfill its purpose. The first and the last step of this narrative happen without reference to the hero, setting the stage and providing the coda (look it up!), respectively. Each of the action steps between clearly states what sort of capability needs to be built for the system.

The story of desire is a made-up story to illustrate the point. Here is a simplified version of a real story in my current project – the story of disturbance:

1. There is a disturbance in the balance of production and consumption of electricity (the conflict)
2. Already, the system (the hero) has received from power plants information about their reserve capacity (this is a story of its own, but for another day)
3. The system shows the Operator (the real hero!) power plants with reserve capacity
   • Variation: The operator filters the list of power plants using some criteria
4. The system lets the Operator activate the reserve capacity at a power plant
5. The system sends an activation request to the power plant in question so they can be activated
6. The system shows the operator what reserves are currently activated
7. The system lets the user deactivate reserves
8. The system sends all activation requests to the accounting system, so the power plant can get paid
9. The balance is restored

If you wonder why this disturbance in the force is a bad thing, take a look at my blog post about xxx. You’ll quickly see that the balance of electricity may impact whether or not my beer stays cool. Clearly an important concern.

This story is a bit messier, since it deals with more real-world problems. Together with all the details I let out, it also has the disadvantage that it will take one team about a year to develop. Our customer is a bit too eager to get some software out the door to wait for all that.

So, we present: The impatient story of disturbance

1. There is a disturbance in the balance of production and consumption of electricity
2. Already, the system has received from the old system we’re replacing information about power plant reserve capacities
3. The system shows the operator power plants with reserve capacity
   • Variation: The operator filters the list of power plants using some criteria
   • Variation: The system updates the view it receives new reserves
4. The system lets the operator activate the reserve capacity at a power plant
5. The system sends the information about the activation to the old system we’re replacing for further distribution
6. The system shows the operator what reserves are currently activated
• Variation: The system shows the operator what reserves have been activated at a given power plant
• Variation: The system displays a chart of what reserves have been activated at a given power plant
7. The balance is restored

During the start of our project we wrote about 10 stories like the original story of disturbance. The stories were written by small gangs each consisting of developers and users or other domain experts. The gangs would swap stories and improve each others work. Meanwhile, the product owner and the project architect would try and pry apart a story so it could be delivered by a single programming team in the order of about 4 months.

For a single delivery, we would write an impatient story. Each action step or variation of the “impatient story of disturbance” became an item on the backlog of the programming team. I’ve left out about a thirds of the stories to keep the text of the blog post down. In addition, the team had about 5 technical tasks, such as performance testing. Each week, the team would deliver on average two such items.

After a few months, we have delivered the first increment in a system that tells a new story to the users, but a story they recognize. We have taken the task that they use their existing system for the most and made it easier and faster to use. The result: Happy users. And cold beer. We hope.

Notes

This approach has very much the same vision as Jeff Patton’s user story mapping and Effect maps. The approach of linear textual stories is one that works well for me. I urge you to find an approach that works even better for you. My stories use Alistair Cockburn’s style of use cases from “Writing effective use cases”, but with one changes: I simplify the writing of variations to better support the linear flow and avoid detail. They are roughly at what Cockburn describes as “kite level”.

Teams gather around task boards to plan their day. If the conversation and the task board match, we stay on topic and we understand what the rest of the team talks about.

The last month, my project has redesigned the task board with this in mind. Here is a picture of our task board as it looked a few weeks ago:

Each column represents a day. The top swimline represents testing tasks, the second swim line represents development tasks, and the bottom represents outside influences. In general, each swim line represents a theme for the next few weeks. Each card represents a task.

Every morning, we review the board:

• The Scrum Master reads the title of every card from yesterday that has been marked with a green checkmark. These tasks are defined as done. The people who worked on the task may comment.
• For every task for yesterday that still has a magnet with a face on it, the person on the magnet declares the task as done or continuing. Done tasks get a green checkmark. Continuing tasks are moved to today.
• For every task planned today, someone puts their magnet on it and thus takes responsibility for the task. Preferably, tasks are claimed by two people who will pair program.
• The Scrum Master reads every task for today that hasn’t been claimed. The team decides whether they they can commit to it in addition to the rest of the task. If not, it’s moved to tomorrow.
• The team looks at the rest of the week to make sure that tasks are evenly distributed. They may have to decide to drop tasks in order to get done or to ask for another user story for the sprint in order to have enough to do.

We’ve also used marker pens to indicate tasks that take several days (one task on Tuesday and one task on Friday, in this case), and tasks that are dependent on other tasks. We only indicate this when we feel it’s important. With these modifications, the observant among you may have noticed that we have reinvented the Gantt diagram!

For the stand-up meeting to be meaningful and stay on-topic, you need to relate it to what you’re actually doing. Have your meeting by a task board that reflects the way you work.

Thank you to Ram Yoga for the brilliant pictures of our task board

One of best ways to get the full effect of pair programming is if everyone programs with everyone else. This maximizes learning in the team. Here are three simple tricks we’ve found useful to get the pair programming to flow better.

• Magnets: Getting away from task ownership is essential to get pair programming to work. Instead of writing names on the cards that represents the tasks the team is working on, we place magnets with pictures of the team members on top of the tasks on our task board. This helps people remember that they don’t have to permanently commit to a task. Moving the magnets around during the stand up meeting is a good way to discuss how to organize the work today. We try to keep one person and exchange one person on every task every day.
• Pair programming star: Deliberate rotation fosters learning in the team. Write the names of all team members in a circle on a whiteboard in the team area. Every day two people pair program, draw a line from the name of one to the name of the other. This both becomes the team’s own indicator of success, and an “excuse” to ask someone you haven’t programmed with if you could pair up with them today.
• Don’t sweat it: When I started pair programming, I found it exhausting! It felt like I never had time that I needed to ruminate over the problems and that I never had chance to explore solutions on my own. During this period, it’s helpful to remember to give people the time they need to think through problems on their own. I do prefer that the code that’s actually checked in is pair programmed, though.

When I started pair programming, I found it to be an obstacle for getting into flow. After a few weeks, working solo has become an obstacle to getting into flow. You got to give it some time and take the breaks that you need along the way.

With pair programming, the teams share responsibility, pull together and have more fun!

I’m getting a little fed up with descriptions of project development lifecycles starting with waterfall, and then describing iterative and agile development in waterfall terms. What happens if we start on the other side?

Project development lifecycles

• Agile: The project creates a roadmap for a year or two, commits to the scope a delivery in a few weeks to a few months, adapts their commitment as they learn more and irons out the details of each task just prior to when it’s performed.
• Incremental: The project commits to the scope the whole roadmap at the beginning of the project and delivers bits every few months according to this plan. The details of a task are ironed a few months ahead of time. Changes to the full project scope are handled as exceptions.
• Waterfall: The project commits to the whole scope of the roadmap at the beginning of the project and postpones delivery of all functionality until the end of the project. The project avoids spending time to verify details after any implementation has started. Learning is handled as exceptions.

Agile described in terms of waterfall (“do a full project every few weeks”) sounds chaotic. Waterfall explained in terms of agile sounds insane.

What do you do if you want to have a reliable system? You make sure you have redundancy: More than one component can do a certain job. You back up you valuable data to a separate system. You have two servers to provide your critical service, in case on fails. Yet, many projects have no plan for the inevitable absence of people with critical knowledge.

We practice pair programming to get redundancy. This manifests itself in many ways:

• When someone is sick, on vacation or leaves the project (to become a doctor, say), the project doesn’t suffer. Or rather, it doesn’t suffer as much. We do get less work done if fewer people are present, of course. And many of our developers have areas where everyone loves to get their help.
• We can add new people to the team. Learning how the system works is just a matter of pair programming as part of the normal rotation.
• We are very adaptive to varying focus at different times in the project. If we had someone who could only work with, say, databases, he wouldn’t have much to do at some times, and too much to do at other times. With pair programming, we still have people who’re especially good in some areas, but everyone can help out with whatever needs doing.
• No parts of the code are off-limits. As every commit of the code is written by two people, no one person has the right to block changes to “his” code. Anyone can make the changes that they need to complete their current task as good as they can.
• The team is working with fewer tasks at a time. As teams grow in size, there can be a lot of different things happening at the same time. This both increases the chances that people will step on each other’s toes (that is, work on the same source code files) and increases the amount of “Work in progress” at any one time. Pair programming effectively halves the amount of work in progress.
• Everyone has a understanding of the whole system. During planning and design discussions, everyone can contribute. Instead of focusing on technical components and subsystems, the discussion is about how to solve the user story as a whole.

Some studies focuses on the impact of pair programming on efficiency, quality, progress, or even focus. I think it’s much more interesting to see how pair programming changes the dynamic of the team. From a group of individuals, we form teams where everyone is appreciated as a contributor. Everyone has their individual strengths, but more importantly, everyone has the same goal in mind: The success of the team.

When I started pair programming daily, it changed my life and my projects for the better. I often point to pair programming as one of the most influential possible interventions on any projects. Very often, an audience member with a healthy degree skepticism will inquire about the research into pair programming.

Sadly, the practice of software engineering research has not come very far in this area, so the questioner is left unsatisfied. In this blogpost, I will summarize the current state of research into pair programming, and describe my hopes for the future.

Read the rest of this entry »

“Refactoring” is the practice of “improving the design of existing code without changing its behavior”. It’s an essential part of software maintenance. Done well, refactoring will make sure your code base is easy to maintain. Done poorly, refactoring will lead you into a dangerous swamp where you’ll be stuck forever.

Read the rest of this entry »

I’ve published a Norwegian language article titled: “Min supre produkteier” (“My excellent product owner”) at the company blog for Steria Norway. Go check it out if you understand Norwegian!

The more I code, the more I’ve learned to appreciate keeping the code clean even during complex refactorings. By “clean”, I mean that the code always compiles and the test always run.

I often find myself in a situation where I have a method call that’s starting to accumulate parameters. Something like this:

showPersonCreateForm(writer, firstName, firstNameErrorMessage, lastName, lastNameErrorMessage,....);

After three or four parameters, the need to refactor is starting to become evident. I would rather have something like this:

CreatePersonForm form = new CreatePersonForm();
form.setFirstName(firstName);
form.setFirstNameErrorMessage(firstName);
form.setLastName(firstName);
form.setLastNameErrorMessage(firstName);
form.show(writer);

This is one of the more complex simple refactorings you can make, and it requires several steps. In this five minute video, I show how to perform such a refactoring without any steps that break my code:

The screencast was created using the free BB FlashBack Express on Windows. All the magic you see happening while I program is either ctrl-space (complete) or ctrl-1 (quick fix).

Can you modify your code without going thought long stages of nothing working? I think you can!

I’ve published a Norwegian language article titled: “Slik lykkes du med smidig utvikling” (“How to succeed with agile development”) at the company blog for Steria Norway. Go check it out if you understand Norwegian!