I was sitting in my Operations Management class yesterday, and we were discussing statistical quality concepts such as the Six Sigma movement which the business world has been obsessed with for the last several years. As we were going through the concepts and the statistical calculations, I realized that the same calculations could demonstrate the value of some key ideas in the agile software movement, such as team size and the value of pair programming.
The math goes something like this. Assume you have a process that produces a product at 98% quality. That means 2 in 100 units is bad and has to be thrown away. That doesn’t sound so bad. But then imagine that you have a product that is composed of 20 parts, and all of those parts have 98% quality. That works out to the overall quality of the product being only 67% (%98 to the power of 20). Over 1 in 4 have to be thrown away, which is obviously too much.
Now let’s assume that each of those “parts” is a developer on an agile team. For each story point worked by a developer, there is a 2% chance they will introduce a defect in that story. As with good agile, every developer on the team works with every story, so you end up with every developer contributing their 2% defect chance. If you limit the team size to 5 developers, the lower bound for recommended agile team size, the overall chance of a defect for each story point is between 90%, which is probably manageable. If you increase the team size 9 people, the upper bound of the recommendation, quality drops to 83%. Each additional person has a multiplicative effect on the chance for defects, due to misunderstandings, different approaches, communication gaps, etc. A team of 15 people would have an overall quality of only 73%, which is far less than desired.
Note that the actual numbers here don’t matter much. The core idea is that everyone has some chance of causing a defect, and those chances are multiplied together when those people work in a team environment. The quality of each individual contributor has a major impact on overall team quality.
Now let’s say you have no choice but to use a team size of 15, and your developers are stuck at their 98% quality level. Obviously, anything you can do to raise that 73% overall quality number would be a huge benefit. Let’s look at what effect pair programming has. Imagine that you never have an individual developer working on the code. Instead, you always have one developer looking over another’s shoulder and reviewing the code for problems. If both of those developers individually had a 2% chance of introducing a defect, together they only have a 0.04% chance (2% squared). This means that your team of 15 people, if they are looking over each others’ shoulders, have an overall quality of 99.4% (1 – 0.04% to the power of 15). What a huge difference!
Management always looks at pair programming as using two developers to produce what could be produced by only one, effectively doubling the cost of each “code unit”. However, it isn’t about initial cost, it is about initial quality. It is well understood that defects introduced into the software development lifecycle cost exponentially more the later in the process they are caught. Raising the initial quality from 73% to 99.4%, meaning defects are caught at the earliest point in the process, would have a huge positive impact on overall SDLC costs.
“Top Quality” Developers
There is an idea in the industry that the “top” developers are worth the cost because you get an exponentially better product from them for only a linear increase in salary. The quality calculations above show some of why this is true. Let’s say you can have a team of 9 newbies that have an individual quality of 95%, or a team of top shelf, experienced developers that have an individual quality of 99%. Using the (admittedly contrived) math above, the newbies would have an overall quality of 63% (95% to the power of 9), and even introducing pair programming would only get you to 97.7%. However, our top quality team would have an overall quality of 91% even without pair programming, and would get to 99.9% with it. If you look at overall SDLC costs, the difference between these two teams are astronomical. These numbers reveal some of the hidden value that top quality developers can bring to a team.
Math is Fun
I’m not sure if this application of statistical quality analysis to software development is revolutionary. But I’ve never run into the concept before, and I found it interesting how some of the ideas that seem valuable intuitively actually work out that way objectively if you apply a little math. These types of analyses can be used to demonstrate to upper management some of the “hard numbers” around agile concepts, and might help you to convince the powers that be to let you give it a shot.