Improving the quality of mixed research reports in the field of human resource development and beyond: A call for rigor as an ethical practice

Human Resource Development Quarterly

Volumn 25, Issue 3, 2014, Pages 273-299

  Onwuegbuzie, Anthony J ^a   Corrigan, Julie A ^b  

^a SAM HOUSTON STATE UNIVERSITY   (United States)

^b UNIVERSITY OF OTTAWA   (Canada)

Author keywords

13 step mixed research process; Evidence based guidelines; Guidelines for conducting and reporting mixed research; Mixed methods research; Mixed research; Prevalence rate studies; Reporting standards; Research rigor

Indexed keywords

EID: 84906835819     PISSN: 10448004     EISSN: 15321096     Source Type: Journal    
DOI: 10.1002/hrdq.21197     Document Type: Editorial

References (98)

1. Alise, M. A., & Teddlie, C. (2010). A continuation of the paradigm wars? Prevalence rates of methodological approaches across the social/behavioral sciences. Journal of Mixed Methods Research, 4, 103-126. doi:10.1177/1558689809360805
2. American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33-40.
3. American Psychological Association (APA). (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
4. Benge, C. L. (2012). Effect of cartoon mnemonics and revised definitions on the acquisition of tier-two vocabulary words among selected fifth-grade students. (Unpublished dissertation), Sam Houston State University, Huntsville, Texas.
5. Biesta, G. (2010). Pragmatism and the philosophical foundations of mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social & behavioral research (2nd ed., pp. 95-117). Thousand Oaks, CA: Sage.
6. Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3-15. doi:10.3102/0013189X034006003
7. Bronfenbrenner, U. (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.
8. Callahan, J. L., & Reio, T. (2006). Making subjective judgments in quantitative studies: The importance of using effect sizes and confidence intervals. Human Resource Development Quarterly, 17, 159-174. doi:10.1002/hrdq.167
9. Cameron, R., & Molina-Azorín, J. F. (2011). The acceptance of mixed methods in business and management research. International Journal of Organizational Analysis, 19, 256-271. doi:10.1108/19348831111149204
10. Collins, K. M. T., & Onwuegbuzie, A. J. (2013). Establishing interpretive consistency when mixing approaches: Role of sampling designs in program evaluations. New Directions for Evaluation, 138, 85-95. doi:10.1002/ev.20060
11. Collins, K. M. T., Onwuegbuzie, A. J., & Johnson, R. B. (2012). Securing a place at the table: Introducing legitimation criteria for the conduct of mixed research. American Behavioral Scientist, 56, 849-865. doi:10.1177/0002764211433799
12. Collins, K. M. T., Onwuegbuzie, A. J., Johnson, R. B., & Frels, R. K. (2013). Practice note: Using debriefing interviews to promote authenticity and transparency in mixed research. International Journal of Multiple Research Approaches, 7, 271-283.
13. Collins, K. M. T., Onwuegbuzie, A. J., & Sutton, I. L. (2006). A model incorporating the rationale and purpose for conducting mixed methods research in special education and beyond. Learning Disabilities: A Contemporary Journal, 4, 67-100.
14. Collins, K. M. T., Onwuegbuzie, A. J., & Sutton, I. L. (2007, February). The role of mixed methods in special education. Paper presented at the annual meeting of the Southwest Educational Research Association, San Antonio, TX.
15. Constas, M. A. (1992). Qualitative data analysis as a public event: The documentation of category development procedures. American Educational Research Journal, 29, 253-266. doi:10.3102/00028312029002253
16. Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches (3rded.). Thousand Oaks, CA: Sage.
17. Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith, K. C. (2011). Best practices for mixed methods research in the health sciences. Office of Behavioral and Social Sciences Research National Institutes of Health. Retrieved from http://obssr.od.nih.gov/mixed_methods_research
18. Curtis, S., Gesler, W., Smith, G., & Washburn, S. (2000). Approaches to sampling and case selection in qualitative research: Examples in the geography of health. Social Science and Medicine, 50, 1001-1014. doi:10.1016/S0277-9536(99)00350-0
19. Dellinger, A., & Leech, N. L. (2007). A validity framework: A unified approach to evaluating validity of empirical research. Journal of Mixed Methods Research, 1, 309-332. doi:10.1177/1558689807306147
20. Denzin, N. K., & Lincoln, Y. S. (2011). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Sage handbook of qualitative research (4th ed., pp. 1-25). Thousand Oaks, CA: Sage.
21. Ellinger, A. D., Anderson, V., Gubbins, C., Lunn, M. L., Nimon, K. F., Sheehan, M., et al. (2013). The generous spirit of the peer review process: Perspectives and insights from the HRDQ editorial team on providing high-quality reviews. Human Resource Development Quarterly, 22, 417-428. doi:10.1002/hrdq
22. Greene, J. C. (2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.
23. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11, 255-274. doi:10.3102/01623737011003255
24. Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18, 59-82. doi:10.1177/1525822X05279903
25. Halpern, E. S. (1983). Auditing naturalistic inquiries: The development and application of a model. Unpublished doctoral dissertation, Indiana University.
26. Hart, L. C., Smith, S. Z., Swars, S. L., & Smith, M. E. (2009). An examination of research methods in mathematics education. Journal of Mixed Methods Research, 3, 26-41. doi:10.1177/1558689808325771
27. Hibbard, S., & Onwuegbuzie, A. J. (2012, April). Trends of mixed methods designs in evaluation studies: From 2003 to 2011. Paper presented at the annual meeting of the American Educational Research Association, Vancouver, British Columbia, Canada.
28. Hurmerinta-Peltomaki, L., & Nummela, N. (2006). Mixed methods in international business research: A value-added perspective. Management International Review, 46, 439-459.
29. Ivankova, N., & Kawamura, Y. (2010). Emerging trends in the utilization of integrated designs in the social, behavioral, and health science. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social & behavioral research (2nd ed., pp. 581-612). Thousand Oaks, CA: Sage.
30. Janesick, V. J. (2000). The choreography of qualitative research design. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 379-399). Thousand Oaks, CA: Sage.
31. Janesick V. J. (2004). Stretching exercises for qualitative researchers (3rd ed.). Thousand Oaks, CA: Sage.
32. Johnson, R. B., & Christensen, L. (2013). Educational research: Quantitative, qualitative, and mixed approaches (5th ed.). Thousand Oaks, CA: Sage.
33. Johnson, R. B., McGowan, M. W., & Turner, L. A. (2010). Grounded theory in practice: Is it inherently a mixed method? Research in the Schools, 17(2), 65-78.
34. Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14-26. doi:10.1177/1558689806298224
35. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1, 112-133. doi:10.1525/sp.1960.8.2.03a00030
36. Leech, N. L., Dellinger, A. B., Brannagan, K. B., & Tanaka, H. (2010). Evaluating mixed research studies: A mixed methods approach. Journal of Mixed Methods Research, 4, 17-31. doi:10.1177/1558689809345262
37. Leech, N. L., & Onwuegbuzie, A. J. (2004). A proposed fourth measure of significance: The role of economic significance in educational research. Evaluation and Research in Education, 18, 179-198. doi:10.1080/09500790408668317
38. Leech, N. L., & Onwuegbuzie, A. J. (2009). A typology of mixed methods research designs. Quality & Quantity: International Journal of Methodology, 43, 265-275. doi:10.1007/s11135-007-9105-3
39. Leech, N. L., & Onwuegbuzie, A. J. (2010a). Guidelines for conducting and reporting mixed research in the field of stress and coping and beyond. In G. S. Gates, W. H. Gmelch, & M. Wolverton (Series Eds.) & K. M. T. Collins, A. J. Onwuegbuzie, & Q. G. Jiao (Vol. Eds.), Toward a broader understanding of stress and coping: Mixed methods approaches (pp. 77-104). The Research on Stress and Coping in Education Series (Vol. 5). Charlotte, NC: Information Age Publishing.
40. Leech, N. L., & Onwuegbuzie, A. J. (2010b). Guidelines for conducting and reporting mixed research in the field of counseling and beyond. Journal of Counseling and Development, 88, 61-69. doi:10.1002/j.1556-6678.2010.tb00151.x
41. Leech, N. L., & Onwuegbuzie, A. J. (2011). Mixed research in counseling: Trends in the literature. Measurement and Evaluation in Counseling Development, 44, 169-180. doi:10.1177/0748175611409848
42. Leech, N. L., Onwuegbuzie, A. J., & Combs, J. C. (2011). Writing publishable mixed research articles: Guidelines for emerging scholars in the health sciences and beyond. International Journal of Multiple Research Approaches, 5, 7-24. doi:10.5172/mra.2011.5.1.7
43. Lester, F. K. (2005). On the theoretical, conceptual, and philosophical foundations for research in mathematics education. ZDM, 37, 457-467. doi:10.1007/BF02655854
44. Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded sourcebook (2nd ed.). Thousand Oaks, CA: Sage.
45. Molina-Azorín, J. F., & Cameron, R. (2010). The application of mixed methods in organisational research: A literature review. Electronic Journal of Business Research Methods, 8, 95-105. Retrieved from http://www.ejbrm.com
46. Nastasi, B. K., Hitchcock, J. H., & Brown, L. M. (2010). An inclusive framework for conceptualizing mixed methods design typologies. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed., pp. 305-338). Thousand Oaks, CA: Sage.
47. Neuendorf, K. A. (2001). The content analysis guidebook. Thousand Oaks, CA: Sage.
48. Newman, I., Ridenour, C. S., Newman, C., & DeMarco, G. M. P. (2003). A typology of research purposes and its relationship to mixed methods. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 167-188). Thousand Oaks, CA: Sage.
49. Niglas, K. (2004). The combined use of qualitative and quantitative methods in educational research. (Unpublished doctoral dissertation), Tallinn Pedagogical University, Tallinn, Estonia.
50. Nimon, K. (2011). Improving the quality of quantitative research reports: A call for action. Human Resource Development Quarterly, 22, 387-394. doi:10.1002/hrdq.20091
51. O'Cathain, A. (2010). Assessing the quality of mixed methods research: Towards a comprehensive framework. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp. 531-555). Thousand Oaks, CA: Sage.
52. Onwuegbuzie, A. J. (2003). Expanding the framework of internal and external validity in quantitative research. Research in the Schools, 10(1), 71-90.
53. Onwuegbuzie, A. J. (2012). Introduction: Putting the mixed back into quantitative and qualitative research in educational research and beyond: Moving towards the radical middle. International Journal of Multiple Research Approaches, 6, 192-219.
54. Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A typology of mixed methods sampling designs in social science research. Qualitative Report, 12, 281-316. Retrieved from http://www.nova.edu/ssss/QR/QR12-2/onwuegbuzie2.pdf
55. Onwuegbuzie, A. J., & Collins, K. M. T. (2014). Using Bronfenbrenner's ecological systems theory to enhance interpretive consistency in mixed research. International Journal of Research in Education Methodology, 5, 651-661.
56. Onwuegbuzie, A. J., Collins, K. M. T., & Frels, R. K. (2013). Foreword: Using Bronfenbrenner's ecological systems theory to frame quantitative, qualitative, and mixed research. International Journal of Multiple Research Approaches, 7, 2-8.
57. Onwuegbuzie, A. J., & Combs, J. P. (2010). Emergent data analysis techniques in mixed methods research: A synthesis. In A. Tashakkori & C. Teddlie (Eds.), Sage handbook of mixed methods in social and behavioral research (2nd ed., pp. 397-430). Thousand Oaks, CA: Sage.
58. Onwuegbuzie, A. J., & Daniel, L. G. (2002). A framework for reporting and interpreting internal consistency reliability estimates. Measurement and Evaluation in Counseling and Development, 35(2), 89-103.
59. Onwuegbuzie, A. J., & Daniel, L. G. (2003, February 19). Typology of analytical and interpretational errors in quantitative and qualitative educational research. Current Issues in Education [Online], 6(2). Retrieved from http://cie.asu.edu/volume6/number2
60. Onwuegbuzie, A. J., & Daniel, L. G. (2004). Reliability generalization: The importance of considering sample specificity, confidence intervals, and subgroup differences. Research in the Schools, 11(1), 60-71.
61. Onwuegbuzie, A. J., & Daniel, L. G. (2005). Evidence-based guidelines for publishing articles in Research in the Schools and beyond [Editorial]. Research in the Schools, 12(2), 1-11. Retrieved from www.msera.org/download/Rits_editorial_12_2.pdf
62. Onwuegbuzie, A. J., Daniel, L. G., & Collins, K. M. T. (2009). A meta-validation model for assessing the score-validity of student teacher evaluations. Quality & Quantity: International Journal of Methodology, 43, 197-209. doi:10.1007/s11135-007-9112-4
63. Onwuegbuzie, A. J., & Denham, M. (2014). Qualitative data analysis techniques. In L. Meyer (Ed.), Oxford bibliographies in education. New York, NY: Oxford University Press. Retrieved from www.oxfordbibliographies.com
64. Onwuegbuzie, A. J., & Frels, R. K. (in press). Seven steps to a comprehensive literature review: A multimodal and cultural approach. London, England: Sage.
65. Onwuegbuzie, A. J., Frels, R. K., Leech, N. L., & Collins, K. M. T. (2011a, April). A mixed research study of approaches used by mixed research instructors. Invited paper presented at the annual meeting of the American Educational Research Association, New Orleans, LA.
66. Onwuegbuzie, A. J., Frels, R. K., Leech, N. L., & Collins, K. M. T. (2011b). A mixed research study of pedagogical approaches and student learning in doctoral-level mixed research courses. International Journal of Multiple Research Approaches, 5, 169-199. doi:10.5172/mra.2011.5.2.169
67. Onwuegbuzie, A. J., & Johnson, R. B. (2006). The validity issue in mixed research. Research in the Schools, 13(1), 48-63.
68. Onwuegbuzie, A. J., Johnson, R. B., & Collins, K. M. T. (2009). A call for mixed analysis: A philosophical framework for combining qualitative and quantitative. International Journal of Multiple Research Approaches, 3, 114-139. doi:10.5172/mra.3.2.114
69. Onwuegbuzie, A. J., & Leech, N. L. (2004a). Enhancing the interpretation of "significant" findings: The role of mixed methods research. Qualitative Report, 9(4), 770-792. Retrieved from http://www.nova.edu/ssss/QR/QR9-4/Onwuegbuzie.pdf
70. Onwuegbuzie, A. J., & Leech, N. L. (2004b). Post-hoc power: A concept whose time has come. Understanding Statistics, 3, 201-230. doi:10.1207/s15328031us0304_1
71. Onwuegbuzie, A. J., & Leech, N. L. (2006). Linking research questions to mixed methods data analysis procedures. Qualitative Report, 11, 474-498. Retrieved from http://www.nova.edu/ssss/QR/QR11-3/onwuegbuzie.pdf
72. Onwuegbuzie, A. J., & Leech, N. L. (2007a). A call for qualitative power analyses. Quality & Quantity: International Journal of Methodology, 41, 105-121. doi:10.1007/s11135-005-1098-1
73. Onwuegbuzie, A. J., & Leech, N. L. (2007b). Validity and qualitative research: An oxymoron? Quality & Quantity: International Journal of Methodology, 41, 233-249. doi:10.1007/s11135-005-1098-1
74. Onwuegbuzie, A. J., Leech, N. L., & Collins, K. M. T. (2011). Toward a new era for conducting mixed analyses: The role of quantitative dominant and qualitative dominant crossover mixed analyses. In M. Williams & W. P. Vogt (Eds.), The Sage handbook of innovation in social research methods (pp. 353-384). Thousand Oaks, CA: Sage.
75. Onwuegbuzie, A. J., Slate, J. R., Leech, N. L., & Collins, K. M. T. (2009). Mixed data analysis: Advanced integration techniques. International Journal of Multiple Research Approaches, 3, 13-33. doi:10.5172/mra.455.3.1.13
76. Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing data in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 351-383). Thousand Oaks, CA: Sage.
77. Onwuegbuzie, A. J., Witcher, A. E., Collins, K. M. T., Filer, J. D., Wiedmaier, C. D., & Moore, C. W. (2007). Students' perceptions of characteristics of effective college teachers: A validity study of a teaching evaluation form using a mixed-methods analysis. American Educational Research Journal, 44, 113-160. doi:10.3102/0002831206298169
78. Plano Clark, V. L., & Badice, M. (2010). Research questions in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed., pp. 275-304). Thousand Oaks, CA: Sage.
79. Powell, H., Mihalas, S., Onwuegbuzie, A. J., Suldo, S., & Daley, C. E. (2008). Mixed methods research in school psychology: A mixed methods investigation of trends in the literature. Psychology in the Schools, 45, 291-309. doi:10.1002/pits.20296
80. Ray, D. C., Hull, D. M., Thacker, A. J., Pace, L. S., Swan, K. L., Carlson, S. E., et al. (2011). Research in counseling: A 10-year review to inform practice. Journal of Counseling & Development, 89, 349-359. doi:10.1002/j.1556-6678.2011.tb00099.x
81. Rocco, T. S. (2010). Criteria for evaluating qualitative studies. Human Resource Development International, 13, 375-378. doi:10.1080/13678868.2010.501959
82. Ross, A., & Onwuegbuzie, A. J. (2010). Mixed methods research design: A comparison of prevalence in JRME and AERJ. International Journal of Multiple Research Approaches, 4, 233-245.
83. Ross, A., & Onwuegbuzie, A. J. (2012). Prevalence of mixed methods research in mathematics education. Mathematics Educator, 22(1), 84-113.
84. Ross, A., & Onwuegbuzie, A. J. (2014). Complexity of quantitative analyses used in mixed research articles published in a flagship mathematics education journal. International Journal of Multiple Research Approaches, 8, 80-90.
85. Saldaña, J. (2012). The coding manual for qualitative researchers (2nd ed.). Thousand Oaks, CA: Sage.
86. Sandelowski, M. (2008). Theoretical saturation. In L. M. Given (Ed.), The Sage encyclopedia of qualitative methods (Vol. 1, pp. 875-876). Thousand Oaks, CA: Sage.
87. Saumure, K., & Given, L. M. (2008). Data saturation. In L. M. Given (Ed.), The Sage encyclopedia of qualitative methods (Vol. 1, pp. 195-196). Thousand Oaks, CA: Sage.
88. Stake, R. E. (2005). Qualitative case studies. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage handbook of qualitative research (3rd ed., pp. 443-466). Thousand Oaks, CA: Sage.
89. Stake, R. E., & Trumbull, D. J. (1982). Naturalistic generalizations. Review Journal of Philosophy and Social Science, 7, 3-12.
90. Storberg-Walker, J. (2012). Instructor's corner: Tips for publishing and reviewing qualitative studies in applied disciplines. Human Resource Development Review, 11, 254-261. doi:10.1177/1534484312436709
91. Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage.
92. Teddlie, C., & Tashakkori, A. (2009). Foundations of mixed methods research: Integrating quantitative and qualitative approaches in the social and behavioral sciences. Thousand Oaks, CA: Sage.
93. Teddlie, C., & Tashakkori, A. (2010). Overview of contemporary issues in mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (2nd ed., pp. 1-41). Thousand Oaks, CA: Sage.
94. Teddlie, C., & Yu, F. (2007). Mixed methods sampling: A typology with examples. Journal of Mixed Methods Research, 1, 77-100. doi:10.1177/2345678906292430
95. Thompson, B. (Ed.). (2003). Score reliability: Contemporary thinking on reliability issues. Newbury Park, CA: Sage.
96. Tracy, S. J. (2010, December). Qualitative inquiry: Eight "big-ten" criteria for excellent qualitative research. Qualitative Inquiry, 16, 837-851.
97. Wisdom, J. P., Cavaleri, M. A., Onwuegbuzie, A. J., & Green, C. A. (2012). Methodological reporting in qualitative, quantitative, and mixed methods health services research articles. Health Services Research Journal, 47(2), 721-745. doi:10.1111/j.1475-6773.2011.01344.x
98. Yin, R. K. (2009). Case study research: Design and methods (4th ed.). Thousand Oaks, CA: Sage.