The focus group method as an empirical tool in software engineering

Guide to Advanced Empirical Software Engineering

Volumn , Issue , 2008, Pages 93-116

  Kontio, Jyrki ^a   Bragge, Johanna ^b   Lehtola, Laura ^a  

^a AALTO UNIVERSITY   (Finland)

^b HELSINKI SCHOOL OF ECONOMICS   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 70749147914     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-84800-044-5_4     Document Type: Chapter

References (87)

1. Anon., Focus Group Kit, Vol. 1-6, Sage Publications, Thousand Oaks, CA, 1997. Adler, M., Ziglio, E., Gazing into the Oracle: The Delphi Method and Its Application to Social Policy and Public Health, Jessica Kingsley Pub, 1996
2. Austin, T., Drakos, N., Mann, J., Web Conferencing Amplifies Dysfunctional Meeting Practices. Gartner Research Report Nr. G00138101, Gartner Inc., 2006
3. Baker, S.L., Improving Business Services through the Use of Focus Groups, Reference Quarterly, 30(Spring):377-385, 1991
4. Basili, V.R., Selby, R.W., Hutchens, D.H., Experimentation in Software Engineering, IEEE Transactions on Software Engineering, 12(7):758-773, 1986
5. Benbasat, I., Rethinking Diversity in Information Systems Research, Information Systems Research, 7(4):389-399, 1996
6. Boehm, B., Gruenbacher, P., Briggs, R.O., Developing Groupware for Requirements Negotiation: Lessons Learned, IEEE Software, 18(3):46-55, 2001
7. Bogdan, R.C., Biklen, S.K., Qualitative Research for Education: An Introduction to Theory and Methods, Allyn and Bacon Inc., Boston, MA, 1982
8. Bragge, J., den Hengst, M., Tuunanen, T., Virtanen, V., A Repeatable Collaboration Process for Developing a Road Map for Mobile Marketing. In Proceedings of the 11th Americas Conference on Information Systems AMCIS, 2005a
9. Bragge, J., Marttiin, P., Tuunanen, T., Developing Innovative IS Services Together with Wide Audience End-Users, In Proceedings of the 38th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1-10, 2005b
10. Bragge, J., Merisalo-Rantanen, H., Hallikainen, P., Gathering Innovative End-User Feedback for Continuous Development of Information Systems: A Repeatable and Transferable E-Collaboration Process, IEEE Transactions on Professional Communication, 48(1):55-67, 2005c
11. Bragge, J., Merisalo-Rantanen, H., Nurmi, A., Tanner, L., A Repeatable E-Collaboration Process Based on ThinkLets for Multi-Organization Strategy Development, Group Decision and Negotiation, 16(4):363-379, 2007a
12. Bragge, J., Relander, S., Sunikka, A., Mannonen, P., Enriching Literature Reviews with Computer-Assisted Research Mining. Case: Profiling Group Support Systems Research. In Proceedings of the 40th Annual Hawaii International Conference on System Sciences (HICSS'07), IEEE, Los Alamitos, CA, pp. 1-10, 2007b
13. Briggs, R.O., De Vreede, G.J., Nunamaker, J.F., Collaboration Engineering with ThinkLets to Pursue Sustained Success with Group Support Systems, Journal of Management Information Systems, 19(4):31-64, 2003
14. Buhrer, H.K., Software Development: What It is, What It should be, and How to get There, ACM SIGSOFT Software Engineering Notes, 28(2):1-4, 2007
15. Chen, M., Nunamaker, J.F., The Architecture and Design of a Collaborative Environment for Systems Definition, Data-Base, 22(1-2):22-29, 1991
16. Clapper, D.L., Massey, A.P., Electronic Focus Groups: A Framework for Exploration, Information and Management, 30(1):43-50, 1996
17. Daft, R., Lengel, R., Organizational Information Requirements, Media Richness, and Structural Design, Management Science, 32(5):554-570, 1986
18. De Vreede, G.J., Fruhling, A., Chakrapani, A., A Repeatable Collaboration Process for Usability Testing, In Proceedings of the 38th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1-10, 2005
19. Easton, G., Easton, A., Belch, M., An Experimental Investigation of Electronic Focus Groups, Information and Management, 40:717-727, 2003
20. Edmunds, H., The Focus Group Research Handbook, NTC Business Books, Lincolnwood, IL, 1991
21. Edmunds, H., The Focus Group Research Handbook, NTC Business Books and American Marketing Association, Lincolnwood, IL, 1999
22. Elfvengren, K., Karkkainen, H., Torkkeli, M., Tuominen, M., A GDSS Based Approach for the Assessment of Customer Needs in Industrial Markets, International Journal of Production Economics, 89(3):272-292, 2004
23. Feig, B., How to Run a Focus Group, American Demographics, 11(December):36-37, 1989
24. Fern, E.F., The Use of Focus Groups for Idea Generation: The Effects of Group Size, Acquaintanceship, and Moderator on Response Quantity and Quality, Journal of Marketing Research, 19(1):1-13, 1982
25. Fjermestad, J., Hiltz, S.R., An Assessment of Group Support Systems Experimental Research: Methodology and Results, Journal of Management Information Systems, 15(3):7-150, 1999
26. Fjermestad, J., Hiltz, S.R., Group Support Systems: A Descriptive Evaluation of Case and Field Studies, Journal of Management Information Systems, 17(3):112-157, 2000
27. Fraunhofer USA Inc., Summary of the Third eWorkshop on Agile Methods, Center for Experimental Software Engineering, http://fc-md.umd.edu/projects/ Agile/3rd-eWorkshop/summary3rdeWorksh.htm, 2002
28. Galliers, R.D., Choosing Appropriate Information Systems Research Approaches: A Revised Taxonomy, in: Information Systems Research: Contemporary Approaches and Emerging Traditions, H.-E. Nissen, H.K. Klein and R. Hirschheim, eds. Elsevier Science Publishers, Amsterdam, pp. 327-345, 1991
29. Ghauri, P., Grønhaug, K., Kristianslund, I., Research Methods in Business Studies, Prentice-Hall, Englewood Cliffs, NJ, 1995
30. Glass, R.A., A Structure-Based Critique of Contemporary Computing Research, Journal of Systems and Software, 28(1):3-7, 1995
31. Gruenbacher, P., Halling, M., Biffl, S., Kitapci, H., Boehm, B.W., Repeatable Quality Assurance Techniques for Requirements Negotiation, In Proceedings of the 36th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1-9, 2003
32. Halling, M., Gruenbacher, P., Biffl, S., Tailoring a COTS Group Support System for Software Requirements Inspection, In Proceedings of the 16th Annual International Conference on Automated Software Engineering, IEEE, Los Alamitos, CA 201-208, 2001
33. Hansen, K., Hansen, R.S., Using an Asynchronous Discussion Board for Online Focus Groups: A Protocol and Lessons Learned, In Proceedings of the College Teaching and Learning Conference, Clute Institute for Academic Research, Littleton, Colorado 1-8, 2006
34. Judd, C.M., Smith, E.R., Kidder, L.H., Research Methods in Social Relations, Harcourt Brace Jovanovich College Publishers, New York, 1991
35. Juristo, N., Moreno, A.M., Basics of Software Engineering Experimentation, Kluwer Academic Publishers, Boston, MA, 2001
36. Kitchenham, B., Dyba, T., Jorgensen, M., Evidence-Based Software Engineering. In Proceedings of 26th International Conference on Software Engineering, IEEE, Los Alamitos, CA, pp. 273-281, 2004
37. Klein, E.E., Tellefsen, T., Herskovitz, P.J., The Use of Group Support Systems in Focus Groups: Information Technology Meets Qualitative Research, Computers in Human Behavior, 23(5):2113-2132, 2007
38. Kontio, J., The Riskit Method for Software Risk Management, version 1.00. (College Park, MD, University of Maryland, 1997) CS-TR-3782/UMIACS-TR-97-38, Computer Science Technical Reports
39. Kontio, J., Software Engineering Risk Management: A Method, Improvement Framework, and Empirical Evaluation. Doctoral dissertation. (2001), Helsinki University of Technology, publisher: Center of Excellence, ISBN: 952-5136-22-1
40. Kontio, J., Bragge, J., Lehtola, L., Using the Focus Group Method in Software Engineering: Obtaining Practitioner and User Experiences. In Proceedings of the International Symposium on Empirical Software Engineering (ISESE), ACM-IEEE, Los Alamitos, CA pp. 271-280, 2004
41. Krueger, R.A., Casey, M.A., Focus Groups: A Practical Guide for Applied Research, Sage Publications, Thousand Oaks, CA, 2000
42. Langford, J., McDonaugh, D., Focus Groups. Supporting Effective Product Development, Taylor and Francis, London, 2003
43. Lehtola, L., Kauppinen, M., Kujala, S., Requirements-Prioritization- Challenges-in-Practice. In Fifth International Conference on Product Focused Software Process Improvement, 2004
44. Liou, Y.I., Chen, M., Using Group Support Systems and Joint Application Development for Requirements Specification, Journal of Management Information Systems, 10(3):25-41, 1993
45. Massey, A.P., Wallace, W.A., Focus Groups as a Knowledge Elicitation Technique, IEEE Transactions on Knowledge and Data Engineering, 3(2):193-200, 1991
46. McQuarrie, E.F., New Books in Review: The Handbook for Focus Group Research & Successful Focus Groups: Advancing the State of the Art, Journal of Marketing Research, 31:377-380, 1994
47. McQuarrie, E.F., New Books in Review: The Mirrored Window: Focus Groups from a Moderator's Point of View & Advanced Focus Group Research, Journal of Marketing Research, 38(November):515-516, 2001
48. McQuarrie, E.F., McIntyre, S.H., Focus Groups and the Development of New Products by Technologically Driven Companies: Some Guidelines, Journal of Product Innovation Management, 1:40-47, 1986
49. Miles, M.B., Huberman, A.M., Qualitative Data Analysis: A Sourcebook of New Methods, Sage Publications, Thousand Oaks, CA, 1984
50. Montoya-Weiss, M.M., Massey, A.P., Clapper, D.L., On-line Focus Groups: Conceptual Issues and a Research Tool, European Journal of Marketing, 32(7/8):713-723, 1998
51. Morgan, D.L., Designing Focus Group Research, in: Tools for Primary Care Research. Volume 2: Research Methods for Primary Care
52. M. Stewart, F. Tudiver, M.J. Bass, E.V. Dunn and P.G. Norton, eds. Sage Publications, Thousand Oaks, CA, 1992
53. Morgan, D.L., Focus Groups, Annual Review of Sociology, 22(August):129-152, 1996
54. Morgan, D.L., Focus Groups as Qualitative Research, Sage Publications, Thousand Oaks, CA, 1997
55. Myers, M., Qualitative Research in Information Systems, http://www.qual.auckland.ac.nz /, 2004.
56. Nambisan, S., Information Systems as a Reference Discipline for New Product Development, MIS Quarterly, 27(1):1-18, 2003
57. Nambisan, S., Wilemon, D., Software Development and New Product Development: Potentials for Cross-Domain Knowledge Sharing, IEEE Transactions on Engineering Management, 47(2):211-220, 2000
58. Neter, J., Waksberg, J., A Study of Response Errors in Expenditure Data from Household Interviews, Journal of the American Statistical Association, 59:18-55, 1964
59. Newby, R., Soutar, G., Watson, J., Comparing Traditional Focus Groups with a Group Support Systems (GSS) Approach for Use in SME Research, International Small Business Journal, 21(4):421-433, 2003
60. Nielsen, J., The Use and Misuse of Focus Groups, IEEE Software, 14(January):94-95, 1997
61. Nunamaker, J.F., Dennis, A.R., Valacich, J.S., Vogel, D.R., George, J.F., Electronic Meeting Systems to Support Group Work, Communications of the ACM, 34(7):40-61, 1991
62. O'Connor, H., Madge, C., "Focus Groups in Cyberspace": Using the Internet for Qualitative Research, Qualitative Market Research, 6(2):133-143, 2003
63. Oringderff, J., 'My Way': Piloting and Online Focus Group, International Journal of Qualitative Methods, 3(3):1-10, 2004
64. Parent, M., Gallupe, R.B., Salisbury, W.D., Handelman, J.M., Knowledge Creation in Focus Groups: Can Group Technologies Help?, Information and Management, 38(1):47-58, 2000
65. Patton, M.Q., Qualitative Evaluation and Research Methods, Sage Publications, Thousand Oaks, CA, 1990
66. Reid, D.J., Reid, F.J.M., Online Focus Groups. An In-Depth Comparison of Computer-Mediated and Conventional Focus Group Discussions, International Journal of Market Research, 47(2):131-162, 2005
67. Rodgers, T.L., Dean, D.L., Nunamaker, J.F., Increasing Inspection Efficiency through Group Support Systems, In Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, pp. 1-10, 2004
68. Rubin, J., Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests, Wiley, New York, 1994
69. Stewart, D.W., Shamdasani, P.N., Focus Groups: Theory and Practice, Sage Publications, Thousand Oaks, CA, 1990
70. Sunikka, A., Usability evaluation of the Helsinki School of Economics Website. Master's thesis, Helsinki School of Economics, 2004
71. Sweet, C., Designing and Conducting Virtual Focus Groups, Qualitative Market Research, 4(3):130-135, 2001
72. Taylor, S.J., Bogdan, R., Introduction to Qualitative Research Methods, Wiley, New York, 1984
73. Templeton, J.F., The Focus Group: A Strategic Guide to Organizing, Conducting and Analyzing the Focus Group Interview, McGraw-Hill Professional Publishing, New York, 1994
74. Ten-Pow, J., Fundamentals for Those Considering Online Focus Groups, On Survey Research Intelligence, http://www.onsurvey.ca/supplemental/onfocus.pdf, 2003
75. Tichy, W.F., Should Computer Scientists Experiment More?, IEEE Computer, 31(5):32-40, 1998
76. Tichy, W.F., Lukowicz, P., Prechelt, L., Heinz, E.A., Experimental Evaluation in Computer Science: A Quantitative Study, Journal of Systems and Software, 28(1):9-18, 1995
77. Turney, L., Pocknee, C., Virtual focus groups: New technologies, new opportunities, new learning environments, Proceedings of the 21st ASCILITE Conference, University of Wollongong, New South Wales, Australia, pp. 905-912.
78. Tuunanen, T., Rossi, M., Engineering a Method for Wide Audience Requirements Elicitatation and Integrating It to Software Development, In Proceedings of the 37th Annual Hawaii International Conference on System Sciences, Los Alamitos, CA, 2004
79. Van Genuchten, M., Cornelissen, W., Van Dijk, C., Supporting Inspections with an Electronic Meeting System, Journal of Management Information Systems, 14(3):165-178, 1997
80. Van Genuchten, M., Van Dijk, C., Scholten, H., Using Group Support Systems for Software Inspections, IEEE Software, 18(3):60-65, 2001
81. Vitharana, P., Ramamurthy, K., Computer-Mediated Group Support, Anonymity, and the Software Inspection Process: An Empirical Investigation, IEEE Transactions on Software Management, 29(2):167-180, 2003
82. Wellner, A.S., The New Science of Focus Groups, American Demographics, March 1 29-33, 2003
83. Widdows, R., Hensler, T.A., Wyncott, M.H., The Focus Group Interview: A Method for Assessing User's Evaluation of Library Service, College and Research Libraries, 52(July):352-359, 1991
84. Wohlin, C., Runeson, P., Host, M., Ohlsson, M.C., Experimentation in Software Engineering: An Introduction, Kluwer Academic Publishers, Boston, MA, 1999
85. Wohlin, C., Höst, M., Henningsson, K., Empirical Research Methods in Software Engineering, Lecture Notes in Computer Science, Vol. 2765 7-23, 2003
86. Zelkowitz, M.V., Wallace, D.R., Experimental Models for Validating Technology, IEEE Computer, 31(5):23-31, 1998
87. Zinchiak, M., Online Focus Group FAQs, Quirk's Marketing Research Review, http://www.quirks. com/articles/a2001/20010712.aspx?searchID=2619905, July/August 2001