The Malmö Experiment: Estimation Techniques Shootout
At ØreDev I ran into Lasse Koskela. We started talking about estimation techniques, and we both felt that the dominant estimation technique of relative estimation with planning poker has been unchallenged for a very long time. We found ourselves wondering what the next big idea about estimation will be. After throwing a couple of ideas back and forth, we decided to invite to a workshop comparing a few estimation techniques. We decided to call the workshop “The Malmö Experiment.”
The results of the experiment were interesting, but far from conclusive.
During the experiment, we gave the same set of requirements to three teams, each consisting of three estimators. Each team was told to use a different technique. We decided on the following techniques:
- Planning poker: The purpose is to give all requirements a relative number (the meaning of these numbers will later be measured based on the output of the iterations). Each estimator has a deck of cards and choose a card with the number he feels is appropriate for the current requirement. Everyone reveals the numbers at the same time to avoid anchoring. The team discusses and reestimates a requirement until their estimates converge.
- Table spread estimation: This is one of the new techniques we proposed. Each requirement is written on a card. The estimators spread the cards along a large table according to the relative effort required per requirement. Numbers can be imposed later if desired.
- Goldilocks estimation: The purpose is to restructure requirements until they all have roughly equal size. Instead of assigning a number to a requirement, the estimators pick one of three options: Too big (split up and estimate the parts again), Too small (merge with other requirements), or Just right. When all requirements have been split or merged into “Just Right” size, the estimation is complete.
All teams found their estimation techniques to be motivating, but the Table Spread and Goldilocks groups managed to complete the estimation much faster. The Table Spread estimation would obviously need more space if we had a lot of requirements, while the Goldilocks estimation would generate a large number of requirements.
Based on these experiences, we propose the following experiment in a project:
- Use Table Spread Estimation for Release planning. This will encourage the team to keep the number of requirements low instead of trying to plan too detailed too far ahead. Since the table spread is quick it can be redone every iteration.
- Use Goldilocks Estimation for the next few upcoming iterations to split up the requirements into equal sized items. This will generate a better set of work items. The shorter planning window will ensure that we won’t have an unmanageable number of requirements.
These are currently very rough ideas and we have no idea of whether it will work as we expect. Let me know if you have any relevant experience or if you want more information.
Comments:
Anders johansson - Nov 11, 2009
The Table spread estimation is not something new. This is a technique used in many scrum teams i have been a part of. This one together with the Use Goldilocks Estimation are already commonly used.
chrishedgate - Nov 9, 2009
Johannes, thanks for the inspiring and thoughtful session at Øredev. Here is a link to a short description of the sorting exercise that I mentioned, similar to the Table Estimation Technique you described.
http://jkarlsson.com/blog/2008/11/08/why-estima…
[jwgrenning] - Nov 10, 2009
I call the table spread estimation technique a Planning Poker Party. I find it is the right tool for the job when there is a large number of stories to estimate. It works for smaller numbers, but is great for a larger backlog. If you need to create a longer term plan, with large and small stories, it gives visibility into the work remaining, showing the work that is small enough and ready to be worked on, and the work that must be broken down further before it can be worked on.
I find that planning poker works well when there are few stories being added to an existing backlog.
jhannes - Nov 10, 2009
Hi, James
Great comment, name and link. A small question: You say it works well for “a large number of stories”. How large is “large”? Your pictures look like 30-40 stories. Is this the comfortable range?
[jwgrenning] - Nov 10, 2009
I’ve used the technique with as many as 150 stories, where we had a first estimate in half a day. I do not know where the bounds are. Its a helpful technique whenever there is no established baseline. a.k.a. A value of “1” that everyone has a feel for.
[undefined] - Jul 9, 2011
Index table spread with the planning. This will encourage the team to limit the requirements, rather than trying to plan too far in detail over time,. Table is a rapidly spreading, it can be reused in each iteration.
[undefined] - Jul 5, 2011
Our goal is to reorganize the requirements until they are roughly the same size. But to identify the required number of evaluation, selection of the three options. Â Table Cover