Every now and then, I come across some useful bits of information related to software engineering and outsourcing. Although they may not be needed for anything immediately, at some point, you will find out that you need them to better plan your project, estimate ROI, or prove some point. I decided to publish them in this blog, making it easier for anybody, including myself, to find and use retrieved figures and other research results any time.
Today, my post is dedicated to the value of using agile methodologies in a software project. We’ve all heard how good it is for performance and project results to go agile. But has anybody actually measured the difference? In fact, someone has.
David Consulting Group (DCG) published a great report on agile productivity. In fact, I discovered it after gathering my own collection of agile-related metrics from Michael Mah (QSM Associates), CHAOS Manifesto from Standish Group, Donald Reifer (Reifer Consultants), and other sources. What I discovered is that the DCG report had 90% of the information I had gathered already in one place. So read the entire report if you are interested. And I will provide some of the selected results from the various reports that I considered noteworthy.
I can also recommend reading the Cutter Consortium report by Michael Mah, which contains some interesting case studies in addition to the statistics.
Here are some of the useful figures from various sources. I tried to filter out the more subjective results (i.e., cases when researchers were collecting respondents’ perceptions on what they thought were the most important features of agile, what area of their lifecycle it affected most, etc.). In order to stand on a sound base, I kept only hard metrics and the data relevant to slightly subjective but much better defined and more easily measured aspects of software development as successful vs. unsuccessful projects.
Although many authors of research papers on the subject caution that the results should be taken with a grain of salt due to a limited statistical base or the fact that correlation doesn’t necessarily mean causation, there are a number of good results there. The provided figures can help to estimate the results of switching to agile methodologies for software companies and to determine if the proposed budgets and efforts for that initiative are adequate.
- Agile projects are 16% more productive. [DCG, 2006, quoting Michael Mah, 2008]
- Small agile projects show a 12% to 35% increase in productivity. [DCG, 2006]
- Agile projects have a 37% faster time to market. [DCG, 2006, quoting Michael Mah, 2008]
- Highly mature XP and Scrum teams had 30%–50% less QA/regression test defects than average. Less mature teams were not much different from waterfall teams in that aspect. [Michael Mah, 2008]
- Agile projects are typically smaller: Projects under $1 million in total costs constitute 45% of agile projects and only 14% of waterfall projects. [Standish Group, 2013]
- For small projects, agile and waterfall projects have almost the same success rates: successful/challenged/failed = 46/48/6% for agile and 49/43/8% for waterfall. [Standish Group, 2013]
– Andrey Pronin, SVP, Strategy, Technology, Marketing, Auriga
