Soft systems analysis of the unification of test and evaluation and program management: A study of a federal aviation administration's strategy

Systems Engineering

Volumn 13, Issue 3, 2010, Pages 298-310

  Eigbe, Arekhandia Patrick ^a   Sauser, Brian J ^b   Boardman, John ^b  

^a FEDERAL AVIATION ADMINISTRATION   (United States)

^b STEVENS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Program management; Program strategy; Soft systems methodology; Test and evaluation

Indexed keywords

CRITICAL DEFECTS; E-STRATEGY; ESCALATING COSTS; EXECUTION STRATEGIES; FEDERAL AVIATION ADMINISTRATION; HOLISTIC VIEW; OPERATIONAL NEEDS; PERFORMANCE OBJECTIVE; PROCESS IMPROVEMENT MODELS; PROGRAM MANAGEMENT; PROGRAM STRATEGY; QUALITY IMPROVEMENT; SOFT SYSTEMS ANALYSIS; SOFT SYSTEMS METHODOLOGY; STRATEGIC OBJECTIVES; SYSTEM DEPLOYMENT; SYSTEMS THINKING; TECHNICAL TOOLS; TEST AND EVALUATION;

AVIATION; DEFECTS; PLANNING; SALES;

CUSTOMER SATISFACTION;

EID: 78649661897     PISSN: 10981241     EISSN: 15206858     Source Type: Journal    
DOI: 10.1002/sys.20150     Document Type: Article

References (83)

1. P. Adolph, C. Dipetto, and E. Seglie, Defense Science Board Task Force study of developmental test and evaluation, ITEA J 29(3) (2008), 215-220.
2. W.D. Bell and D. Eng., The 800-pound (364 kg) gorilla, ITEA J(3) 29 (2008), 275-277.
3. C.D. Blair, J.T. Boardman, and B.J. Sauser, Communicating strategic intent with Systemigrams: Application to the network-enabled challenge, Syst Eng 10(4) (2007), 309-322.
4. J. Boardman, Wholes and parts-a systems approach, IEEE Trans Syst Man Cybernet 25 (1995), 1150-1161.
5. J.T. Boardman, A process model for unifying systems engineering and project management, Eng Management J 4(1) (1994), 25-35.
6. J.T. Boardman, The evolution of Systemigrams, 2005, available from the author.
7. J.T. Boardman, Boardman Soft Systems Methodology and Systemigrams, communications with the author, 2006.
8. J.T. Boardman and A.J. Cole, Integrated process improvement in design and manufacture using a systems approach, IEE Process- Control Theory Appl 143(2) (1996), 171-185.
9. J. Boardman and B. Sauser, Systems thinking: Coping with 21st century problems (industrial innovation), CRC Press, Boca Raton, FL, 2008, Chap. 6.
10. W.M. Branch, The systems approach to management: A paradigm for the test and evaluation process, Master's Thesis, Naval Postgraduate School, Monterey, CA, 1976.
11. I. Burnstein, The testing maturity model and test process assessment, Springer, London, 2003, pp. 537-585.
12. I. Burnstein, T. Suwanassart, and R. Carlson, Developing a testing maturity model for software test process evaluation and improvement, Int Test and Evaluation Conf, ITEA, Hawaii, 1996, 581-589.
13. Carnegie Mellon, S.E.I., "Introduction to CMMI, http://www.sei.cmu. edu/programs/acquisition-support/present ations/tutorialmod2.pdf, 2005.
14. D. Castellano, Identifying systemic program issues based on real program findings, PEO/SYSCOM Conf, 2006.
15. D. Castellano, Sharing lessons learned based on systemic program findings, ITEA Int Symp, Kaua'i, Hawaii, 2007.
16. J.N. Chang and D.J. Rohall, Improving program success by applying systems engineering and reliability growth analysis, ITEA J 29(3) (2008), 267-274.
17. P. Checkland, Systems thinking, systems practice, a formal system model, Wiley, Chichester, 1999, pp. 1-330.
18. P. Checkland, Soft systems methodology: A thirty year retrospective, Syst Res Behav Sci 17(1) (2000), 11-58.
19. P.B. Checkland, Systems thinking, systems practice, Wiley, Chichester, 1981, pp. 1-330.
20. B.T. Clegg, Building a holarchy using business Process-Oriented Holonic (PROH) modeling, IEEE Trans Syst Man Cybernet-Part A Syst Hum 37(1) (2007), 23-40.
21. B.T. Clegg and J.T. Boardman, Process integration and improvement using systemic diagrams and a human-centred approach, Concurrent Eng 4(2) (1996a), 119-136.
22. B.T. Clegg and J.T. Boardman, Systems approach to process improvement in design and manufacture. 1996b = IEE Colloquia (1996/171), 311-319. Available: http://ieeexplore.ieee.org/ stamp/stamp.jsp?tp=&arnumber= 640527&isnumber=13888
23. B.T. Clegg and J.T. Boardman, A soft systems methodology to support business process change towards concurrent engineering, Int Conf INCOSE, Vancouver, BC, Canada, 1998, pp. 697- 704.
24. R.J. Cloutier and D. Verma, Applying the concept of patterns to systems architecture, Syst Eng 10(2) (2007), 138-154.
25. A.J. Cole, S.J. Wolak, and J.T. Boardman, A computer-based process handbook for a systems engineering business, Proc Seventh Int Workshop Computer-Aided Software Eng,, IEEE Computer Society, 1995, p. 172.
26. D.C. Conner, Managing at the speed of change: how resilient managers succeed and prosper where others fail, 1992, p. 282.
27. C.L. Conwell, R. Enright, M.A. Stutzman, J. A. Joines, R.R. Burton, K. Kung, and P.A. Fishwick (Editors), Capability maturity models support of modeling and simulation verification, validation, and accreditation, Proc 2000 Winter Simulation Conf, IEEE, 2000, pp. 819-828.
28. P.B. Crosby, Quality is free: The art of making quality certain, McGraw-Hill, New York, 1979, pp. 1-309.
29. C. Deephouse, D. Goldenson, M. Kellner, and T. Mukhopadhyay, The effects of software processes on meeting targets and quality, Proc 28th Hawaii Int Conf Syst Sci, IEEE Computer Society, 1995, pp. 710-719.
30. E.W. Deming, Out of the crisis, Massachusetts Institute of Technology, Center for Advanced Educational Study (MIT-CAES), Cambridge, MA, 1986, pp. 1-305.
31. FAA, Project management: Current state assessment, DOT (Editor), Federal Aviation Administration WJHTC, Atlantic City, NJ, 2003.
32. FAA, Integrated engineering services: Project office implementation proposal, DOT (Editor), Federal Aviation Administration WJHTC, Atlantic City, NJ, 2004.
33. FAA, Acquisition management policy: Test and evaluation," DOT (Editor), Federal Aviation Administration, Washington, DC, 2006a.
34. FAA, FAA ATO strategy map, DOT (Editor), Federal Aviation Administration, Washington, DC, 2006b.
35. FAA, Verification and validation protocol of operations-White Paper: Improvement recommendations, DOT (Editor), Federal Aviation Administration, Atlantic City, NJ, 2006, pp. 1-10.
36. FAA, FAA ATO operations planning strategy map, DOT (Editor), Federal Aviation Administration Washington, DC, 2007a.
37. FAA, Project management: Current State assessment, DOT (Editor), Federal Aviation Administration WJHTC, Atlantic City, NJ, 2007b.
38. B. Fox, M. Boito, J.C. Graser and O. Younossi, Test and evaluation trends and costs for aircraft and guided weapons, RAND Corporation, prepared for the US Air Force, Pittsburgh, PA, 2004.
39. L. Franz and J.C. Shih, Estimating the value of inspections and early testing for software projects, Hewlett-Packard J 45(6) (1994), 60-67.
40. GAO, Best practices: A more constructive test approach is key to better weapon system outcomes, Government Accounting Office, Washington, DC, 2000, pp. 1-71.
41. GAO and M. Sullivan, Best practices: Increased focus on requirements and oversight needed to improve DOD's acquisition environment and weapon system quality, Government Accounting Office, Washington, DC, 2008, 1-61.
42. GAO, K. Siggerud, and T. Conquest, Air traffic control: FAA needs to ensure better coordination when approving air traffic control systems, Report To The Chairman, Subcommittee On Aviation, Committee On Transportation And Infrastructure, House Of Representatives, Washington, DC, 2004.
43. B. Hobbs and M. Aubry, A multi-phase research program investigating Project Management Offices (PMOs): The results of phase 1, Project Management J 38(1) (2007), 74-86.
44. M. Hoppe and A. Engel, Improving the VVT process: Evaluating the SYSTEST results in six industrial pilot projects, INCOSE, Seattle, WA, 2005, 17 pages.
45. M. Hoppe, A. Engel, and S. Shachar, Systems test: Improving the verification, validation, and testing process- assessing six industrial pilot projects, Syst Eng 10(4) (2007), 323-347.
46. ISO/IEC, IEE Std 15288:2008, Systems and software engineering- system life cycle processes, International Organization For Standardization/IEC-IEEE, Geneva, Switzerland, 2008.
47. J. Juran and A.B. Godfrey, Juran?s quality handbook, McGraw Hill, New York, 1999, pp. 1-85.
48. J. Kasser, Enhancing the role of test and evaluation in the acquisition process to increase the probability of the delivery of equipment that meets the needs of the users, Proc Syst Test Eval Conf (SETE 2000), 2000, 5 pages.
49. H. Kerzner, Applied project management best practices on implementation, Wiley, New York, 2000, pp. 1-534.
50. H. Kerzner, Project management-a systems approach to planning, scheduling & controlling, Wiley, Hoboken, NJ, 2003, pp. 1-891.
51. J. Knutson, The project office: An evolutionary implementation plan, PM Network, Project Management Institute, Newtown Square, PA, 1998, pp. 14-16.
52. J. Knutson, Project office: An evolutionary implementation plan, Proc 30th Annu Project Management Inst, Project Management Institute, Newtown Square, PA, 1999, 5 pages.
53. H. Kreher, Some recurring themes in using SSM, J Oper Res Soc 45(11) (1994), 1293-1303.
54. J.H. Lambert, R.K. Jennings, and N.N. Joshi, Integration of risk identification with business process models, Syst Eng 9(3) (2006), 187-198.
55. P.W.J. Ledington and J. Donaldson, Soft OR and management practice: A study of the adoption and use of soft systems methodology, J Oper Res Soc 48(3) (1997), 229-240..
56. V. Lévárdy, M. Hoppe, and E. Honour, Verification, validation & testing strategy and planning procedure, 14th Annu Int Symp INCOSE, 2004, 17 pages.
57. B. Mazoyer, N. Tzourio-Mazoyer, A. Mazard, M. Denis, and E. Mellet, Neural bases of image and language interactions, Int J Psychol 37(4) (2002), 204-208.
58. D. Milosevic and P. Patanakul, Standardized project management may increase development projects success, Int J Project Management 23(3) (2005), 181-192.
59. J. Mingers and S. Taylor, The use of soft systems methodology in practice, J Oper Res Soc 43(4) (1992), 321-332.
60. P. Morris and A. Jamieson, Moving from corporate strategy to project strategy, Project Management J 36(4) (2005), 5-18.
61. I. Munro and J. Mingers, The use of multimethodology in practice- results of a survey of practitioners, J Oper Res Soc 53(4) (2002), 369-378.
62. S. Nagano, Space systems verification program and management process, Syst Eng 11(1) (2008), 27-38.
63. NRC, V. Nair, and M.L. Cohen, Testing of defense systems in an evolutionary acquisition environment, V. Nair And M.L. Cohen (Editors), National Academy of Sciences, Washington, DC, 2006, pp. 1-76.
64. A. Paivio, Dual Coding Theory: Retrospect And Current Status, Canada Journal Of Psychology 45 (1991), 255-287.
65. PMI, Government extension to a guide to the project management body of knowledge (PMBOK® guide), Project Management Institute, Newtown Square, PA, 2000, 1-89.
66. PMI, An executive guide to OPM; transform strategy into results, Project Management Institute, Newtown Square, PA, 2004a, http://www.pmi.org/prod/groups/ public/documents/info/ pp-opm3execguide.pdf.
67. PMI®, A guide to the project management body of knowledge (PMBOK® guide), 3rd edition, Project Management Institute, Newtown Square, PA, 2004b, pp. 1-216.
68. M. Poli and A.J. Shenhar, Project strategy: The key to project success, PICMET '03, Portland Int Conf Technol Management for Reshaping the World, Management of Engineering and Technology, 2003, pp. 231-235.
69. G. Rao, K. Timmaraju, and T. Weigert, Network element testing using ttcn-3: Benefits and comparison, Lecture Notes in Computer Science, Springer, New York, 2005, Vol. 3530, pp. 265- 280.
70. J. Riemer, T&E investment, capacity, and program execution, ITEA Int Symp, ITEA, Kaua'i, Hawaii, 2007, pp. 1-5.
71. J. Rothman, What does it cost to fix a defect?, 2002.
72. B.J. Sauser, Toward mission assurance: A framework for systems engineering management, Syst Eng 9(3) (2006), 213-227.
73. B.J. Sauser, R.R. Reilly, and A.J. Shenhar, Why projects fail? How contingency theory can provide new insights-a comparative analysis of NASA?s Mars Climate Orbiter loss, Int J Project Management (2009), in press, corrected proof available online February 14, 2009.
74. A.J. Shenhar, Strategic Project Leadership® toward a strategic approach to project management, R&D Management 34(5) (2004), 569-578.
75. A. Shenhar and D. Dvir, Reinventing project management: The diamond approach to successful growth and innovation, Harvard Bus Rev (2007), 274.
76. A.J. Shenhar, D. Dvir. W. Guth, T. Lechler, P. Panatakul, M. Poli, and J. Stefanovic, A new framework for strategic project management, Acad Management Annu Meeting, Honolulu, HI, 2005.
77. B. Simmons and C.M. Wilcox, The four-element framework: A mission-focused test and evaluation strategy, ITEA J Test Eval 28(3) (2007), 61-69.
78. D.E. Stem, M. Boito, and O. Younossi, Systems engineering and program management trends and costs for aircraft and guided weapons programs, RAND Corp., prepared for the US Air Force, Pittsburgh, PA, 2006.
79. J.S. Sutterfield, S.S. Friday-Stroud, and S.L. Shivers-Blackwell, A case study of project and stakeholder management failures: Lessons learned, Project Management J 37(5) (2006), 26-35.
80. F. Toney and R. Powers, A practical use of key success factors to improve the effectiveness of project management, Int J Project Management 17(3) (1999), 139-145.
81. S. Voetsch and S. Whitehead, Operational test agencies: Enhancing operational realism in T&E, ITEA J 29(2) (2008), 142-146.
82. H.v.d. Water, M. Schinkel, and R. Rozier, Fields of application of SSM: A categorization of publications, J Oper Res Soc 58(3) (2007), 271-287.
83. R.K. Wysocki, Project management process improvement -Chapter 3: Assessing and reporting maturity level, Artech House iv(5896) (2004), 52-62.