Evaluating journal quality and the association for information systems senior scholars' journal basket via bibliometric measures: Do expert journal assessments add value?

MIS Quarterly: Management Information Systems

Volumn 37, Issue 4, 2013, Pages 993-1012

  Lowry, Paul Benjamin ^a   Moody, Gregory D ^b   Gaskin, James ^c   Galletta, Dennis F ^d   Humpherys, Sean L ^e   Barlow, Jordan B ^f   Wilson, David W ^g  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^b UNIVERSITY OF NEVADA   (United States)

^c BRIGHAM YOUNG UNIVERSITY   (United States)

^d UNIVERSITY OF PITTSBURGH   (United States)

^e WEST TEXAS A AND M UNIVERSITY   (United States)

^f INDIANA UNIVERSITY   (United States)

^g UNIVERSITY OF ARIZONA   (United States)

Author keywords

AIS Senior Scholars basket of journals; Bibliometrics; Composite ranking or rating; Expert opinion; H index; Impact factor; Information systems journal rankings; Journal quality; Nomologies for dissemination of scientific knowledge; Scientometrics; Self citation; SenS 6; SenS 8; Social network analysis

Indexed keywords

CLUSTER ANALYSIS; FACTOR ANALYSIS; INDEXING (OF INFORMATION); INFORMATION SYSTEMS; PUBLISHING; QUALITY CONTROL; SOCIAL NETWORKING (ONLINE);

BIBLIOMETRICS; COMPOSITE RANKING; EXPERT OPINION; H INDICES; IMPACT FACTOR; JOURNAL QUALITY; JOURNAL RANKINGS; SCIENTIFIC KNOWLEDGE; SCIENTOMETRICS; SELF-CITATION; SENS-6; SENS-8;

INFORMATION MANAGEMENT;

EID: 84887453422     PISSN: 02767783     EISSN: 21629730     Source Type: Journal    
DOI: 10.25300/MISQ/2013/37.4.01     Document Type: Article

References (83)

1. Acuna, D. E., Allesina, S., and Kording, K. P. 2012a. "Future Impact: Predicting Scientific Success," Nature (489:7415), pp. 201-202.
2. Acuna, D. E., Allesina, S., and Kording, K. P. 2012b. "Predicting Scientific Success," Nature (489:13), pp. 201-202.
3. Adler, N. J., and Bartholomew, S. 1992. "Academic and Professional Communities of Discourse: Generating Knowledge on Transnational Human Resource Management," Journal of International Business Studies (23:3), pp. 551-569.
4. AIS. 2011. "Senior Scholars' Basket of Journals," Association for Information Systems http://home.aisnet.org/displaycommon. cfm?an=1&subarticlenbr=346).
5. Allen, L., Jones, C., Dolby, K., Lynn, D., and Walport, M. 2009. "Looking for Landmarks: The Role of Expert Review and Bibliometric Analysis in Evaluating Scientific Publication Outputs," PloS ONE (4:6).
6. Axarloglou, K., and Theoharakis, V. 2003. "Diversity in Economics: An Analysis of Journal Quality Perceptions," Journal of the European Economic Association (1:6), pp. 1402-1423.
7. Baskerville, R. 2008. "For Better or Worse: How We Apply Journal Ranking Lists," European Journal of Information Systems (17:2), pp. 156-157.
8. Baskerville, R., and Wood-Harper, A.T. 1998. "Diversity in Information Systems Action Research Methods," Journal of the Operational Research Society (7:2), pp. 90-107.
9. Bonacich, P. 1987. "Power and Centrality: A Family of Measures," American Journal of Sociology (92:5), pp. 1170-1182.
10. Bonner, S. E., Hesford, J. W., Van der Stede, W. A., and Young, S. M. 2006. "The Most Influential Journals in Academic Accounting," Accounting, Organizations, and Society (31:7), pp. 663-685.
11. Bornmann, L., and Daniel, H. D. 2009. "The State of h Index Research: Is the h Index the Ideal Way to Measure Research Performance?," EMBO Reports (10:1), pp. 2-6.
12. Butler, L. 2008. "Using a Balanced Approach to Bibliometrics: Quantitative Performance Measures in the Australian Research Quality Framework," Ethics in Science and Environmental Politics (8:1), pp. 83-92.
13. Cabell, D., and English, D. L. 2004. Cabell's Directory of Publishing Opportunities in Management (9th ed.), Beaumont, TX: Cabell Publishing.
14. Caliński, T., and Harabasz, J. 1974. "A Dendrite Method for Cluster Analysis," Communications in Statistics-theory and Methods (3:1), pp. 1-27.
15. Campbell, J. P., Daft, R. L., and Hulin, C. L. 1982. What to Study: Generating and Developing Research Questions, Beverly Hills, CA: Sage Publications.
16. Carnegie Foundation. 2010. "The Carnegie Classification of Institutions of Higher Education" (http://classifications. carnegiefoundation.org/; retrieved January 1, 2011).
17. Chen, C. R., and Huang, Y. 2007. "Author Affiliation Index, Finance Journal Ranking, and the Pattern of Authorship," Journal of Corporate Finance (13:5), pp. 1008-1026.
18. Chua, C., Cao, L., Cousins, K., and Straub, D. W. 2003. "Measuring Researcher-Production in Information Systems," Journal of the Association for Information Systems (3:4), pp. 145-215.
19. Clarke, R., Davison, R., and Beath, C. M. 2009. "Journal Self- Citation Xi: Regulation of 'Journal Self-Referencing'-The Substantive Role of the AIS Code of Research Conduct," Communications of the Association for Information Systems (25:1), pp. 91-96.
20. Crews, J. M., McLeod, A., and Simkin, M. G. 2009. "Journal Self- Citation XII: The Ethics of Forced Journal Citations," Communications of the Association for Information Systems (25:1), pp. 97-110.
21. Culnan, M. J., and Swanson, E. B. 1986. "Research in Management Information Systems, 1980-1984: Points of Work and Reference," MIS Quarterly (10:3), pp. 289-302.
22. Daft, R. L., Griffin, R. W., and Yates, V. 1987. "Retrospective Accounts of Research Factors Associated with Significant and Not-So-Significant Research Outcomes," The Academy of Management Journal (30:4), pp. 763-785.
23. Davis, M. S. 1971. "That's Interesting: Towards a Phenomenology of Sociology and a Sociology of Phenomenology," Philosophy of Social Science (1:4), pp. 309-344.
24. Dean, D. L., Lowry, P. B., and Humpherys, S. L. 2011. "Profiling the Research Productivity of Tenured Information Systems Faculty at U.S. Institutions," MIS Quarterly (35:1), pp. 1-15.
25. Dennis, A., Valacich, J., Fuller, M., and Schneider, C. 2006. "Empirical Benchmarks for Promotion and Tenure in Information Systems," MIS Quarterly (30:1), pp. 1-13.
26. Dreher, G. F., and Bretz, R. D. 1991. "Cognitive Ability and Career Attainment: Moderating Effects of Early Career Success," Journal of Applied Psychology (76:3), pp. 392-397.
27. Egghe, L. 2006. "Theory and Practice of the G-Index," Scientometrics (69:1), pp. 131-152.
28. Ferratt, T. W., Gorman, M. F., Kanet, J. J., and Salisbury, W. D. 2007. "IS Journal Quality Assessment Using the Author Affiliation Index," Communications of the Association for Information Systems (19:1), pp. 710-724.
29. Freeman, L. C. 1979. "Centrality in Social Networks: Conceptual Clarification," Social Networks (1:1979), pp. 215-239.
30. Galletta, D. 2010. "The Senior Scholar' Basket of Journals. Panel Presentation," in Proceedings of the 31st International Conference of Information Systems, St. Louis, MO, December 12-15.
31. Galliers, R. D., and Meadows, M. 2003. "A Discipline Divided: Globalization and Parochialism in Information Systems Research," Communications of the Association for Information Systems (11), pp. 108-117.
32. Gallivan, M., and Benbunan-Fich, R. 2007. "Analyzing IS Research Productivity: An Inclusive Approach to Global IS Scholarship," European Journal of Information Systems (16:1), pp. 36-53.
33. Grant, A. M. 2008. "Does Intrinsic Motivation Fuel the Prosocial Fire? Motivational Synergy in Predicting Persistence, Performance, and Productivity," Journal of Applied Psychology (93:1), pp. 48-58.
34. Gray, P. 2009. "Journal Self-Citation I: The Overview of the Journal Self-Citation Paper-The Wisdom of the IS Crowd," Communications of the Association for Information Systems (25:1), pp. 1-10.
35. Harnad, S. 2008. "Validating Research Performance Metrics Against Peer Rankings," Ethics in Science and Environmental Politics (8:11), pp. 103-107.
36. Harvey, C., Morris, H., and Kelly, A. 2007. "Academic Journal Quality Guide: Journals Classified by Field and Rank," The Association of Business Schools.
37. Harzing, A.-W. 2011. Publish or Perish, Harzing.com (http://www.harzing. com/).
38. Hendrix, D. 2009. "Institutional Self-Citation Rates: A Three Year Study of Universities in the United States," Scientometrics (81:2), pp. 321-331.
39. Hirsch, J. 2005. "An Index to Quantify an Individual's Scientific Research Output," Proceedings of the National Academy of Sciences of the United States of America (102:46), pp. 16569-16572.
40. Iivari, J. 2008. "Expert Evaluation vs. Bibliometric Evaluation: Experiences from Finland," European Journal of Information Systems (17:2), pp. 169-173.
41. Judge, T. A., Hurst, C., and Simon, L. S. 2009. "Does it Pay to Be Smart, Attractive, or Confident (or All Three)? Relationships among General Mental Ability, Physical Attractiveness, Core Self-Evaluations, and Income," Journal of Applied Psychology (94:3), pp. 742-755.
42. Karuga, G. G., Lowry, P. B., and Richardson, V. J. 2007. "Assessing the Impact of Premier Information Systems Research over Time," Communications of AIS (2007:19), pp. 115-131.
43. Katerattanakul, P., and Han, B. 2003. "Are European IS Journals Under-Rated? An Answer Based on Citation Analysis," European Journal of Information Systems (12:1), pp. 60-71.
44. Konrad, A. M., and Pfeffer, J. 1990. "Do You Get What You Deserve? Factors Affecting the Relationship Between Productivity and Pay," Administrative Science Quarterly (35:2), pp. 258-285.
45. Kozar, K., Larsen, K., and Straub, D. 2006. "Leveling the Playing Field: A Comparative Analysis of Business School Journal Productivity," Communications of the AIS (17:23), pp. 524-538.
46. Lewis, B. R., Templeton, G. F., and Luo, X. 2007. "A Scientometric Investigation into the Validity of IS Journal Quality Measures," Journal of the Association for Information Systems (8:12), pp. 619-633.
47. Leydesdorff, L. 2008. "Caveats for the Use of Citation Indicators in Research and Journal Evaluations," Journal of the American Society for Information Science and Technology (59:2), pp. 278-287.
48. Li, E. Y. 2009. "Journal Self-Citation III: Exploring the Self- Citation Patterns in MIS Journals," Communications of the Association for Information Systems (25:1), pp. 21-32.
49. Long, R. G., Bowers, W. P., Barnett, T., and White, M. C. 1998. "Research Productivity of Graduates in Management: Effects of Academic Origin and Academic Affiliation," Academy of Management Journal (41:6), pp. 704-714.
50. Lowry, P. B., Romans, D., and Curtis, A. 2004. "Global Journal Prestige and Supporting Disciplines: A Scientometric Study of Information Systems Journals," Journal of the Association for Information Systems (5:2), pp. 29-80.
51. Lynch, J. G. 2012. "Business Journals Combat Coercive Citation," Science (335:6073), pp. 1169-a.
52. Maslach, C., and Leiter, M. P. 2008. "Early Predictors of Job Burnout and Engagement," Journal of Applied Psychology (93:3), pp. 498-512.
53. Mingers, J., and Harzing, A. 2007. "Ranking Journals in Business and Management: A Statistical Analysis of the Harzing Data Set," European Journal of Information Systems (16:4), pp. 303-316.
54. Mylonopoulos, N. A., and Theoharakis, V. 2001. "On Site: Global Perceptions of IS Journals," Communications of the ACM (44:9), pp. 29-33.
55. Nerur, S. P., Rasheed, A. A., and Natarajan, V. 2008. "The Intellectual Structure of the Strategic Management Field: An Author Co Citation Analysis," Strategic Management Journal (29:3), pp. 319-336.
56. Özbilgin, M. 2009. "From Journal Rankings to Making Sense of the World," The Academy of Management Learning and Education (8:1), pp. 113-121.
57. Peffers, K., and Ya, T. 2003. "Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the Journals," Journal of Information Technology Theory and Application (5:1), pp. 63-84.
58. Pfeffer, J., and Langton, N. 1993. "The Effect of Wage Dispersion on Satisfaction, Productivity, and Working Collaboratively: Evidence from College and University Faculty," Administrative Science Quarterly (38:3), pp. 382-407.
59. Polites, G. L., and Watson, R. T. 2009. "Using Social Network Analysis to Analyze Relationships among IS Journals," Journal of the Association for Information Systems (10:8), pp. 595-636.
60. Porta, S., Cruciiti, P., and Latora, V. 2006. "The Network Analysis of Urban Streets: A Primal Approach," Environment and Planning B: Planning and Design (33:5), pp. 705-725.
61. Rainer Jr., R. K., and Miller, M. D. 2005. "Examining Differences across Journal Rankings," Communications of the ACM (48:2), pp. 91-94.
62. Romano Jr., N. C. 2009. "Journal Self-Citation V: Coercive Journal Self-Citation-Manipulations to Increase Impact Factors May Do More Harm Than Good in the Long Run," Communications of the Association for Information Systems (25:1), pp. 41-56.
63. Sarkis, J. 2009. "Journal Self-Citation XVII: Editorial Self- Citation Requests-A Communications of the Association for Information Systems (25:1), pp. 141-148.
64. Saunders, C., Avison, D., Davis, G., Eindor, P., Galletta, D., Hirschheim, R., and Straub, D. 2007. "AIS Senior Scholars Forum Subcommittee on Journals" (http://home.aisnet.org/associations/7499/files/ Senior%20Scholars%20Letter.pdf; retrieved January 1, 2011).
65. Seibert, S. E., Crant, J. M., and Kraimer, M. L. 1999. "Proactive Personality and Career Success," Journal of Applied Psychology (84:3), pp. 416-427.
66. Sellers, S. L., Perry, R., Mathiesen, S. G., and Smith, T. 2004. "Evaluation of Social Work Journal Quality: Citation Versus Reputation Approaches," Journal of Social Work (40:1), pp. 143-160.
67. Sidiropoulos, A., Katsaros, D., and Manolopoulos, Y. 2007. "Generalized Hirsch h-Index for Disclosing Latent Facts in Citation Networks," Scientometrics (72:2), pp. 253-280.
68. Stephenson, K., and Zelen, M. 1989. "Rethinking Centrality: Methods and Examples," Social Networks (11:1), pp. 1-37.
69. Straub, D. 2006. "The Value of Scientometric Studies: An Introduction to a Debate on IS as a Reference Discipline," Journal of the Association for Information Systems (7:5), pp. 241-246.
70. Straub, D., and Anderson, C. 2009. "Journal Self-Citation VI: Forced Journal Self-Citation-Common, Appropriate, Ethical?," Communications of the Association for Information Systems (25:1), pp. 57-66.
71. Straub, D., and Anderson, C. 2010. "Journal Quality and Citations: Common Metrics and Considerations About Their Use," MIS Quarterly (34:1), pp. iii-xii.
72. Svensson, G., and Wood, G. 2006. "The Pareto Plus Syndrome in Top Marketing Journals: Research and Journal Criteria," European Business Review (18:6), pp. 457-467.
73. Trieschmann, J. S., Dennis, A. R., Northcraft, G. B., and Niemi, A. W. 2000. "Serving Multiple Constituencies in the Business School: MBA Program Versus Research Performance," Academy of Management Journal (43:6), pp. 1130-1141.
74. Trkman, P. 2009. "Journal Self-Citation XX: Citations and the Question of Fit," Communications of the Association for Information Systems (25:1), pp. 165-170.
75. Truex, D., Cuellar, M., and Takeda, H. 2009. "Assessing Scholarly Influence: Using the Hirsch Indices to Reframe the Discourse," Journal of the Association for Information Systems (10:7), pp. 560-594.
76. Valacich, J. S., Fuller, M. A., Schneider, C., and Dennis, A. R. 2006. "Publication Opportunities in Premier Business Outlets: How Level Is the Playing Field?," Information Systems Research (17:2), pp. 107-125.
77. van Dalen, H. P., and Henkens, K. 2001. "What Makes a Scientific Article Influential? The Case of Demographers," Scientometrics (50:3), pp. 455-482.
78. Walsh, J. P. 2011. "Presidential Address: Embracing the Sacred in Our Secular Scholarly World," Academy of Management Review (36:2), pp. 215-235.
79. Wilhite, A. W., and Fong, E. A. 2012. "Coercive Citation in Academic Publishing," Science (335:6068), pp. 542-543.
80. Willcocks, L., Whitley, E., and Avgerou, C. 2008. "The Ranking of Top IS Journals: A Perspective from the London School of Economics," European Journal of Information Systems (17:2), pp. 163-168.
81. Williamson, I. O., and Cable, D. M. 2003. "Predicting Early Career Research Productivity: The Case of Management Faculty," Journal of Organizational Behavior (24:1), pp. 25-44.
82. Wolff, H.-G., and Moser, K. 2009. "Effects of Networking on Career Success: A Longitudinal Study," Journal of Applied Psychology (94:1), pp. 196-206.
83. Zhang, C. T. 2009. "The e-Index, Complementing the h-Index for Excess Citations," PloS ONE (4:5): e5429.