Skip to main content
SpringerLink
Log in

Quantifying the impact of requirements definition and management process maturity on project outcome in large business application development

  • Original Article
  • Published:
Requirements Engineering Aims and scope Submit manuscript

Abstract

Using data from two surveys of people knowledgeable about requirements for, and the success of the development of, large commercial applications (CAs) in hundreds of large organizations from around the world, this paper reports a high positive correlation between an organization’s requirements definition and management (RDM) maturity and that organization’s successful performance on CA development projects. Among the organizations that responded with a filled survey, an organization that is assessed at a high RDM maturity is significantly more successful in its CA development projects than is an organization that is assessed at a low RDM maturity, when success in CA development projects is measured as (1) delivering CAs on-time, on-budget, and on-function, (2) meeting the business objectives of these projects, and (3) the perceived success of these projects. This paper presents a comprehensive framework for RDM, describes a quality RDM process, and describes RDM maturity and how to measure it. It describes the two surveys, the first of which ended up being a pilot for the second, which was designed taking into account what was learned from the first survey. The paper concludes with advice to practitioners on the application of the RDM maturity framework in any organization that wishes to improve its RDM and its performance in the development of large CAs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Commercial applications are known also as business applications. This paper uses “commercial application” and its acronym “CA” to reserve the “BA” acronym for its common usage to mean “business analyst”. No acronym is given to “business analysis”.

  2. These percentages are guestimated by the first author, Ellis, based on his more than 19 years of full-time experience as a software development methods consultant, and the second author, Berry, agrees from his more limited but longer experience.

  3. “CA” continues to mean “commercial application” and not “capability area”.

  4. While it would have been appropriate to use a score of 1 to 1.499, the sampling of companies with a score of under 1.5 was too small to accommodate this segmentation.

  5. While this level is defined, in fact no respondent had a high enough maturity score to achieve this level.

  6. Ellis’s expectation is based on his 18 years of experience conducting marketing surveys.

References

  1. Abbott B (2001) Requirements set the mark. InfoWorld 23(10):45–46

    Google Scholar 

  2. Agile Alliance Principles (2001) The agile alliance 2001. http://www.agilealliance.org/

  3. Alexander L, Davis A (1991) Criteria for the selection of a software process model. In: Knafl G (ed) Proceedings of the fifteenth annual computer software and application conference. pp 521–528

  4. Berry DM, Damian D, Finkelstein A, Gause D, Hall R, Wassyng A (2005) To do or not to do: if the requirements engineering payoff is so good, why aren’t more companies doing it? In: Proceedings of the IEEE international conference on requirements engineering (RE), p 447, http://www.requirements-engineering.org/REnotDonePanelRE05/

  5. Boehm BW, Abts C, Brown AW, Chulani S, Clark BK, Horowitz E, Madachy R, Reifer DJ, Steece B (2000) Software cost estimation with COCOMO II. Prentice Hall, Upper Saddle River, NJ

    Google Scholar 

  6. Bourque P, Dupuis R, Abran A, Moore JW, Tripp L (1999) The guide to the software engineering body of knowledge. IEEE Softw 16:35–44

    Article  Google Scholar 

  7. Brooks F (1987) No silver bullet—essence and accidents of software engineering. IEEE Comput 20(4):10–19

    Article  MathSciNet  Google Scholar 

  8. Curtis B, Krasner H, Iscoe N (1988) A field study of the software design process for large systems. Commun ACM 31(11):1268–1287

    Article  Google Scholar 

  9. Damian D, Chisan J (2006) An empirical study of the complex relationships between requirements engineering processes and other processes that lead to payoffs in productivity, quality, and risk management. IEEE Trans Softw Eng 32(7):433–453

    Article  Google Scholar 

  10. Damian D, Chisan J, Vaidyanathasamy L, Pal Y (2005) Requirements engineering and downstream software development: findings from a case study. Empir Softw Eng 10(3):255–283

    Article  Google Scholar 

  11. Damian D, Zowghi D, Vaidyanathasamy L, Pal Y (2004) An industrial case study of immediate benefits of requirements engineering process improvement at the australian center for unisys software. Empir Softw Eng 9(1–2):45–75

    Article  Google Scholar 

  12. El Emam K, Madhavji NH (1995) A field study of requirements engineering practices in information systems development. In: Proceedings of the second IEEE international symposium on requirements engineering (RE), p 68

  13. Ellis K (2008) Business analysis benchmark. Technical report, IAG Consulting, Oakville, ON, Canada, http://www2.iag.biz/benchmark-2008

  14. Ellis K (2009) Business analysis benchmark—2009. Technical report. IAG Consulting, Oakville, ON, Canada http://www.iag.biz/resources/library/download-business-analysis-benchmark-2009.html

  15. Fenton N, Pfleeger SL, Glass RL (1994) Science and substance: a challenge to software engineers. IEEE Softw 11:86–95

    Article  Google Scholar 

  16. Galin D, Avrahami M (2006) Are CMM program investments beneficial? Analyzing past studies. IEEE Softw 23(6):81–87

    Article  Google Scholar 

  17. Gibson DL, Goldenson DR, Kost K (2006) Performance results of CMMI-based process improvement. Technical report. The Software Engineering Institute, Carnegie Mellon University, Pittsburgh, PA, USA. http://www.sei.cmu.edu/reports/06tr004.pdf

  18. Haase V, Messnarz R, Koch G, Kugler HJ, Decrinis P (1994) Bootstrap: fine-tuning process assessment. IEEE Softw 11:25–35

    Article  Google Scholar 

  19. Hall T, Beecham S, Rainer A (2002) Requirements problems in twelve software companies: an empirical analysis. IEE Proc Softw 149:153–160

    Google Scholar 

  20. Harry M, Schroeder R (2000) Six Sigma: the breakthrough management strategy revolutionizing the world’s top corporations. Currency, New York, NY

  21. Herbsleb JD, Goldenson DR (1996) A systematic survey of CMM experience and results. In: Proceedings of the eighteenth international conference on software engineering (ICSE). pp 323–330

  22. Herzberg F (1959) The motivation to work. Wiley, New York, NY

    Google Scholar 

  23. Hofmann HF, Lehner F (2001) Requirements engineering as a success factor in software projects. IEEE Softw 18(4):58–66

    Article  Google Scholar 

  24. IAG Consulting (2002). IAG performance benchmark. http://www.iag.biz

  25. International Institute of Business Analysis (2010) Business analysis body of knowledge (BABOK), Version 2.0. http://www.theiiba.org

  26. International Standards Organization (2010) ISO 9000—Quality management. Viewed 10 August 2010. http://www.iso.org/iso/iso_catalogue/management_standards/quality_management.htm

  27. International Standards Organization (2010) ISO 9001:2000, Quality management systems—requirements. Viewed 10 August 2010. http://www.iso.org/iso/iso_catalogue/catalogue_tc/catalogue_detail.htm?csnumber=21823

  28. Jones C (1996) Applied software measurement: assuring productivity and quality. McGraw-Hill, New York, NY

    Google Scholar 

  29. Kerton B (2006) Flawed requirements trigger 70% of project failures. Infotech Research. http://www.infotech.com/research/flawed-requirements-trigger-70-of-project-failures

  30. Kitchenham B, Pfleeger SL (2002) Principles of survey research, part 4: questionnaire evaluation. SIGSOFT Softw Eng Note 27:20–23

    Google Scholar 

  31. Lauesen S, Vinter O (2001) Preventing requirement defects: an experiment in process improvement. Requir Eng 6:37–50

    Google Scholar 

  32. Leffingwell D (1997) Calculating the return on investment from more effective requirements management. Am Prog 10(4):13–16

    Google Scholar 

  33. Lubars M, Potts C, Richter C (1993) A review of the state of the practice in requirements modeling. In: Proceedings of the IEEE international symposium on requirements engineering (RE), pp 2–14

  34. Paech B, Koenig T, Borner L, Aurum A (2005) An analysis of empirical requirements engineering survey data. In: Engineering and managing software requirements, Part 3. Springer, Berlin. pp 427–452

  35. Paulk MC, Curtis B, Chrissis MB, Weber CV (1993) Capability maturity model, version 1.1. IEEE Softw 10(4):18–27

    Article  Google Scholar 

  36. Requirements-engineering.org. RE Day 2005 Website. Viewed 10 August 2010. http://www.requirements-engineering.org/REday05/

  37. Sadraei E, Aurum A, Beydoun G, Paech B (2007) A field study of the requirements engineering practice in australian software industry. Requir Eng J 12:145–162

    Article  Google Scholar 

  38. Sawyer P, Sommerville I, Viller S (1999) Capturing the benefits of requirements engineering. IEEE Softw 16(2):78–85

    Article  Google Scholar 

  39. Software Engineering Institute (2010) CMMI overview. Viewed 10 August 2010. http://www.sei.cmu.edu/cmmi/

  40. Sommerville I, Sawyer P (1997) Requirements engineering: a good practice guide. Wiley, Chichester

    MATH  Google Scholar 

  41. The Standish Group (2009) CHAOS summary 2009. http://www1.standishgroup.com/newsroom/chaos_2009.php

  42. United States Government Accountability Office (2008). Report to congressional requesters: information technology: agencies need to establish comprehensive policies to address changes to projects’ cost, schedule, and performance goals, http://www.gao.gov/new.items/d08925.pdf

  43. Wang Y, Court I, Ross M, Staples G, King G, Dorling A (1997) Quantitative evaluation of the SPICE, CMM, IS0 9000 and BOOTSTRAP. In: Proceedings of the third international software engineering standards symposium (ISESS), pp 57–68

  44. Wiegers KE (2003) Software requirements. 2nd edn. Microsoft Press, Redmond, WA

    Google Scholar 

  45. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2000) Experimentation in software engineering: an introduction. Kluwer Academic Publishers, Norwell, MA

    Book  Google Scholar 

Download references

Acknowledgments

This paper was developed in a partnership between IAG Consulting, a technology services firm specialized in RDM, the University of Waterloo, and Lancaster University. The authors thank Carol Deutschlander for introducing each to the other and Ross Little, CEO of IAG Consulting, for endorsing this collaboration. The authors thank Pete Sawyer and the anonymous reviewers of a previous version of this paper for their comments. Daniel Berry’s work was supported in parts by a Canadian NSERC grant NSERC-RGPIN227055-00 and by a Canadian NSERC–Scotia Bank Industrial Research Chair NSERC-IRCPJ365473-05.

Author information

Authors and Affiliations

  1. Department of Computing, Lancaster University, LA1 4WA, Lancaster, UK

    Keith Ellis

  2. David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, ON, N2L 3G1, Canada

    Daniel M. Berry

Authors
  1. Keith Ellis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel M. Berry
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Daniel M. Berry.

Additional information

Written while Keith Ellis was at IAG Consulting, Oakville, ON L6M 3E6, Canada.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ellis, K., Berry, D.M. Quantifying the impact of requirements definition and management process maturity on project outcome in large business application development. Requirements Eng 18, 223–249 (2013). https://doi.org/10.1007/s00766-012-0146-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00766-012-0146-3

Keywords

Search

Navigation