Pair programming research misses the most important point

When I started pair programming daily, it changed my life and my projects for the better. I often point to pair programming as one of the most influential possible interventions on any projects. Very often, an audience member with a healthy degree skepticism will inquire about the research into pair programming.

Sadly, the practice of software engineering research has not come very far in this area, so the questioner is left unsatisfied. In this blogpost, I will summarize the current state of research into pair programming, and describe my hopes for the future.

Lately, I’ve been talking with researchers at SINTEF about project in-situ studies into different aspects of agile development, including pair programming. As we have several teams that practice pair programming on a daily basis in Steria, I’m hoping that this may finally shine some scientific light on what I believe to be a really good practice.

The current situation is not satisfactory. The body of research into Pair Programming is summarized in the meta-study Are Two Heads Better than One? On the effectiveness of pair programming by Dybå, Arisholm, Sjøberg, Hannay, and Shull in IEEE Software November, 2007.

The meta-analysis looks at twelve papers on pair programming in their assessment of pair programming on three project metrics: Quality, Duration and Effort.

The studies are mostly pointing to the same results: Pair programming seems to improve quality significantly, decrease duration slightly, and increase effort slightly. That is: Pair programming will make your project ship a little earlier, with significantly better quality, but at a higher cost.

Sadly, all the research so far miss some significant factors:

  • First: Better quality in the short term may transform into lower cost in the long term. As the studies are all short term, this effect cannot be assessed.
  • Second: One of the main reasons we do pair programming is to decrease the effect of absence (vacations, illness, turnover) and suboptimization due to over-specialization. As far as I know, there are no studies that include these effects.
  • Third: I’ve found that pair programming is a skill in itself. It takes up to a few months to learn it. All studies of pair programming have been with people who are inexperienced in the practice.

Finally, studies on pair programming seem to conclude that pair programming may be appropriate for juniors or mid-level developers, but that it seldom makes sense for senior developers. I don’t know if I should take that as an insult or not, as I practice pair programming in almost all my programming time. And I would like to consider myself a senior developer. Feel free to disagree.

Copyright © 2011 Johannes Brodwall. All Rights Reserved.

About Johannes Brodwall

Johannes is Principal Software Engineer in SopraSteria. In his spare time he likes to coach teams and developers on better coding, collaboration, planning and product understanding.
This entry was posted in English, Extreme Programming, Pair programming, Software Development. Bookmark the permalink.
  • Karianne Berg

    I’ve read some of these studies, and had the same objections to how they measure “better”. Would love to see a long term study with people that actually pair program daily.

  • Niklas Bjørnerstedt

    There is one more thing that seems to be omitted in these studies. Pair programming is likely to lead to increased learning. This effect is probably stronger for inexperienced programmers.

  • http://twitter.com/YvesHanoulle Yves Hanoulle

    Another reason to pair is knowledge sharing. None of the studies have looked at the knowledge sharing aspect.

  • Willem van den Ende

    I find, that when working in a pair, I’m more likely to go and figure out how something really works (as opposed to assuming I understand what it does). It does help if at least one of the pairs has not much experience in the subject matter. Experience in programming, I’m still not sure how relevant that variable is.

  • Willem van den Ende

    I find, that when working in a pair, I’m more likely to go and figure out how something really works (as opposed to assuming I understand what it does). It does help if at least one of the pairs has not much experience in the subject matter. Experience in programming, I’m still not sure how relevant that variable is.

  • http://twitter.com/YvesHanoulle Yves Hanoulle

    A few years ago I did a pairprogramming workshop with students and professors. The workshop was 3 hours. 1 hour me talking, then 2 hours pomodore promiscuous paring (changing pairs every pomodore) to solve a few different problems. We had students from the 1, 2 and 3 year (and professors).
    The first feedback at the end came from a professor who said he learned something from all students he paired with. Even the 1ste years. The first year students had started their year 6 weeks before. I think he was the head developer professor.
    They would talk amoungst the professors to see if they could have students work on assignments in pairs in their classes.

  • Pingback: Tweets that mention Thinking Inside a Bigger Box » Pair programming research misses the most important point -- Topsy.com()

  • http://www.johannesbrodwall.com/ Johannes Brodwall

    Great comments. I think I’ll have to write another blog post on the intended effects of pair programming.

  • Jim Cooper

    Pair programming is also an excellent way to mentor people. Make the mentoree drive though!

  • Brunopedroso

    Interestingly, the state of the art in TDD empirical research suffers from pretty similar problems!

  • Pingback: The Benefits of Pair Programming – Why is it important? | Michael Tang's Blog – IT Agility and Leadership()

  • SallyannFreudenberg

    I have published quite a few papers about pair programming and nearly always studied pairs who were used to pairing and had done so for at least 6 months. Take care when you suggest ALL studies have looked at novices.

  • http://www.johannesbrodwall.com/ Johannes Brodwall

    It’s good to hear that you’re studying experienced teams, Sallyann. Do you have references to your research? I guess it’s newer than the meta-analysis I mention?

  • kavitha

    We are practicing pair programming in our regular laboratory sessions for PG classes. We are interested to analyse aspects related to knowledge transfer and learning. Will be happy if you could suggest some metrics that can be used for the analysis

  • http://www.johannesbrodwall.com/ Johannes Brodwall

    I’ve had a hard time formulating metrics, but the questions I’d like to shed light on include: 1) Will the team be better able to deal with someone being sick or on leave, 2) Will the team be more willing and able to execute a sprint with tasks that only cover a subset of their skills, 3) will the team come up with simpler solution, because they have a better overview, 4) will the team learn more and faster?

    I would expect these effects to be measurable after 2-4 months. I’m not sure what would be good ways to benchmark it, though.

  • Pingback: Does pair programming work? | Dave Nicolette()

  • Jake

    I’ve read quite of few research papers on pair programming and have been practicing it for over three years daily and I think proponents of pair programming blow the conclusions way out of proportion. You see its very hard to draw general conclusion about a social matter such as pair programming. Any conclusion about a social/human matter is of course highly related to environment, social norms and taboos (culture), and the individuals involved. Some papers on pair programming concur with this claim and cautiously narrow the scope of its findings.

    In my real life professional pair programming experience, as opposed to a controlled academic setting where the goals of the programmers are very different (e.g get good grade rather than get promoted or stand out) I have seen pair programming both succeed and fail miserably. I’ve seen learning both accelerated and stifled. I seen productivity bloom and on the other hand I’ve seen programmers become dependent on pairing an loose the capability to work effectively alone to the detriment of productivity. I have seen the confidence of less experienced programmers increase at a rate that is significantly retarded by pair programming. So when I read post that just objectively declares pair programming the right thing to do — I just chuckle.