The imprecise science of evaluating scholarly performance: Utilizing broad quality categories for an assessment of business and management journals

Evaluation Review

Volumn 30, Issue 4, 2006, Pages 505-532

  Lange, Thomas ^a,b  

^a AUCKLAND UNIVERSITY OF TECHNOLOGY   (New Zealand)

^b Department of Economics ^*

Author keywords

Compatible research quality; Government policy; Multisubject research in business and management; Nonparametric methods; Tenure and promotion practices

Indexed keywords

ARTICLE; COMMERCIAL PHENOMENA; EVALUATION; NEW ZEALAND; NONPARAMETRIC TEST; ORGANIZATION AND MANAGEMENT; PUBLICATION; RESEARCH; STANDARD;

COMMERCE; NEW ZEALAND; PERIODICALS; RESEARCH; STATISTICS, NONPARAMETRIC;

EID: 33745652108     PISSN: 0193841X     EISSN: 15523926     Source Type: Journal    
DOI: 10.1177/0193841X05284088     Document Type: Article

References (60)

1. Alexander, J., and R. Mabry. 1994. Relative significance of journals, authors and articles cited in financial research. Journal of Finance 49: 697-712.
2. Allen Consulting Group. 2004, November. International comparison of research quality in the New Zealand tertiary education system. Discussion paper commissioned by the New Zealand Tertiary Education Commission, Wellington.
3. Ballas, A., and V. Theoharakis. 2003. Exploring diversity in accounting through faculty journal perceptions. Contemporary Accounting Research 20 (4). 619-644.
4. Bavelas, J. 1978. The social psychology of citations. Canadian Psychological Review 19: 158-163.
5. Beed, C., and C. Beed. 1996. Measuring the quality of academic journals: The case of economics. Journal of Post-Keynesian Economics 18 (3). 369-396.
6. Blackburn, R. 1990. Organizational behavior: Whom do we talk to and who talks to us? Journal of Management 16: 279-305.
7. Bourke, P. 1997. Evaluating university research: The British research assessment exercise and Australian practice. Canberra, Australia: AGPS.
8. Brown, L. 2003. Ranking journals using social science research network downloads. Review of Quantitative Finance and Accounting 20 (3). 291-307.
9. Brown, L., and R. Huefner. 1994. The familiarity with and perceived quality of accounting journals: views of senior accounting faculty in leading U.S. MBA programs. Contemporary Accounting Research 11 (1). 223-250.
10. Chan, K., C. Chen, and T. Steiner. 2002. Production in the finance literature, institutional reputation and labor mobility in academia: A global perspective. Financial Management Winter: 131-156.
11. Coe, R., and I. Weinstock. 1969. Evaluating journal publications: Perceptions versus reality. AACSB Bulletin 1: 23-37.
12. Coe, R., and I. Weinstock. 1984. Evaluating the management journals: A second look. Academy of Management Journal 27 (3). 660-666.
13. Collin, S., U. Johansson, K. Svensson, and P. Ulvenblad. 1996. Market segmentation in scientific publications: Research patterns in American vs. European management journals. British Journal of Management 7 (2). 141-154.
14. Commonwealth of Australia. 2005. Research quality framework: Assessing the quality and impact of research in Australia - Issues paper. Canberra, Australia: Department of Education, Science and Training.
15. Conover, W. J. 2003. Practical nonparametric statistics. Wiley Series in Probability and Statistics: Applied Probability and Statistics Section. New York: Wiley.
16. Croom, D. 1970. Dangers in the use of the Science Citation Index. Nature 227: 1173-1173.
17. Cudd, M., and J. Morris. 1988. Bias in journal ratings. Financial Review 23 (1). 117-125.
18. Dunden, G., and L. Ellis. 1994. A method for identifying the most influential articles in an academic discipline. Atlantic Economic Journal 21 (4). 1-10.
19. Dunn, O. J. 1964. Multiple comparisons using rank sums. Technometrics 6: 241-252.
20. Exteijt, M., and J. Smith. 1990. The behavioural sciences and management: An evaluation of relevant journals. Journal of Management 16: 539-551.
21. Garfield, E. 1991. Social Science Citation Index. Vol. 17. Philadelphia: Philadelphia Institutes for Scientific Information.
22. Gomez-Meija, L., and D. Balkin. 1992. Determinants of faculty pay: An agency theory perspective. Academy of Management Journal 35 (5). 921-955.
23. Harzing, A. 2005. Journal quality list. 13th ed. Melbourne, Australia: Department of Management, University of Melbourne. Available from www.harzing.com.
24. Hasselback, J. 1996. The McGraw-Hill directory of management faculty 1995-1996. New York: McGraw-Hill.
25. Hasselback, J., and A. Reinstein. 1995. A proposal for measuring scholarly productivity of accounting faculty. Issues in Accounting Education 10 (2). 269-276.
26. Higher Education Authority. 2003. The programme for research in third level institutions: Transforming the Irish research landscape. Dublin, Ireland: Higher Education Authority.
27. Henderson, G., G. Ganesh, and P. Chandy. 1990. Across-discipline journal awareness and evaluation: Implications for the promotion and tenure process. Journalof Economics and Business 42: 325-351.
28. Holmstrom, B. 1979. Moral hazard and observability. Bell Journal of Economics 10: 74-91.
29. Holsapple, C., L. Johnson, I. Manakyan, and J. Tanner. 1993. A citation analysis of business computing research journals. Information and Management 25 (5). 231-244.
30. Hult, G., W. Neese, and R. Bashaw. 1997. Faculty perceptions of marketing journals. Journal of Marketing Education 19: 37-52.
31. Hustad, T. 1997. From the editor. Journal of Product Innovation Management 14 (3). 157-160.
32. Jobber, D., and P. Simpson. 1988. A citation analysis of selected marketing journals. International Journal of Research in Marketing 5 (2). 137-142.
33. Johnson, J., and P. Podsakoff. 1994. Journal influence in the field of management: An analysis using Salancik's index in a dependency network. Journal of Management 37: 1392-1407.
34. Kendall, M. G., and B. Babington-Smith. 1939. The problem of m rankings. Annals of Mathematical Statistics 10: 275-287.
35. Kruskal, W. H., and W. A. Wallis. 1952. Use of ranks on one-criterion variance analysis. Journal of the American Statistical Association 47: 583-621.
36. Lane, D. M. 2003. Hyper Stat online: An introductory statistics textbook. Houston, TX: Rice University Press.
37. Loft, A., A. Jorissen, and P. Walton. 2002. From newsletter to academic journal: Creating the European Accounting Review. European Accounting Review 11 (1). 43-75.
38. Lukka, K., and E. Kasanen. 1996. Is accounting a global or a local discipline? Evidence from major research journals. Accounting, Organizations and Society 21 (7/8). 755-773.
39. Lyall, C., A. Bruce, J. Firn, M. Firn, and J. Tait. 2004. Assessing end-use relevance of public sector research organisations. Research Policy 33: 73-87.
40. MacRoberts, M., and B. MacRoberts. 1989. Problems of citation analysis: A critical review. Journal of the American Society for Information Science 40 (5). 342-349.
41. May, K. 1967. Abuses of citation indexing. Science May: 890-890.
42. Organisation for Economic Co-operation and Development. 2002. Frascati manual: Proposed standard practice for surveys on research and experimental development. Paris: OECD: Organisation for Economic Co-operation and Development.
43. Panazzo, F. 1997. The making of the good academic accountant. Accounting, Organizations and Society 22 (5). 447-480.
44. Parnell, J. 1997. Assessing management journal quality: A methodological critique and empirical analysis. Mid-Atlantic Journal of Business 33 (1). 69-83.
45. Polonsky, M., G. Jones, and M. Kearsley. 1999. Accessibility: An alternative method of ranking marketing journals? Journal of Marketing Education 21 (3). 181-193.
46. Roberts, G. 2003. Review of research assessment. London: Higher Education Funding Council for England.
47. Schiermeier, Q. 2004. Grade expectations for German research institutes. Nature 18: 260-260.
48. Schmidt, R. C. 1997. Managing Delphi surveys using nonparametric statistical techniques. Decision Sciences 28 (3). 763-774.
49. Schofield, P. J. 2005, April. Simple statistical approaches to reliability and item analysis. Mimeo, University of Essex.
50. Seglen, P. 1994. Causal relationships between article citedness and journal impact. Journalof the American Society for Information Science 45 (1). 1-11.
51. Smith, J., and R. Dombrowski. 1998. An examination of the relationship between author-editor connections and subsequent citations of auditing research articles. Journal of Accounting Education 16 (3/4). 497-506.
52. Soteriou, A., G. Hadjincola, and K. Patsia. 1999. Assessing production and operations management related journals: The European perspective. Journal of Operations Management 17: 225-238.
53. Tertiary Education Commission. 2004. Performance based research fund: Evaluating research excellence - The 2003assessment. Wellington, New Zealand: Tertiary Education Commission.
54. Theoharakis, V., and A. Hirst. 2002. Perceptual differences of marketing journals: A worldwide perspective. Marketing Letters 13 (4). 389-402.
55. Tukey, J. W. 1994. The collected works of John W. Tukey VIII. Multiple comparisons:1948-1983. New York: Chapman and Hall.
56. Van Fleet, D., A. McWilliams, and D. Siegel. 2000. A theoretical and empirical analysis of journal rankings: The case of formal lists. Journal of Management 26 (5). 839-861.
57. Vastag, G., and F. Montabon. 2002. Journal characteristics, rankings and social acculturation in operations management. Omega - The International Journal of Management Science 30: 109-126.
58. Wallis, W. A. 1939. The correlation ratio for ranked data. Journal of the American Statistical Association 34: 533-538.
59. Woodward, A., and S. Henson. 1976. Citations to review serials. Journal of Documentation December: 290-293.
60. Zar, J. H. 1996. Biostatistical analysis. London: Prentice Hall International.