Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence

Journal of Informetrics

Volumn 5, Issue 4, 2011, Pages 629-648

  Serenko, Alexander ^a   Dohan, Michael ^b  

^a LAKEHEAD UNIVERSITY   (Canada)

^b MCMASTER UNIVERSITY   (Canada)

Author keywords

Academic journal; Artificial Intelligence; Citation impact; G index; Google Scholar; H index; Hc index; Journal ranking; Survey

Indexed keywords

EID: 80054042634     PISSN: 17511577     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.joi.2011.06.002     Document Type: Article

References (74)

1. Adler N., Harzing A.-W. When knowledge wins: Transcending the sense and nonsense of academic rankings. Academy of Management Learning & Education 2009, 8(1):72-95.
2. Bharati P., Tarasewich P. Global perceptions of journals publishing e-commerce research. Communications of the ACM 2002, 45(5):21-26.
3. Bonev I. Should we take Journal Impact Factors seriously?. ParalleMIC 2009, Available online at: http://www.parallemic.org/Reviews/Review016.html.
4. Bontis N., Serenko A. A follow-up ranking of academic journals. Journal of Knowledge Management 2009, 13(1):16-26.
5. Butler L. Identifying 'highly-rated' journals - An Australian case study. Scientometrics 2002, 53(2):207-227.
6. Calver M.C., Bradley J.S. Should we use the mean citations per paper to summarise a journal's impact or to rank journals in the same field?. Scientometrics 2009, 81(3):611-615.
7. Catling J.C., Mason V.L., Upton D. Quality is in the eye of the beholder? An evaluation of impact factors and perception of journal prestige in the UK. Scientometrics 2009, 81(2):333-345.
8. Cheng C.H., Holsapple C.W., Lee A. The impact of periodicals on expert systems research. IEEE Expert: Intelligent Systems and Their Applications 1994, 9(6):7-14.
9. Cheng C.H., Holsapple C.W., Lee A. Citation-based journal rankings for AI research: A business perspective. AI Magazine 1996, 17(2):87-97.
10. Coe R., Weinstock I. Evaluating the management journals: A second look. Academy of Management Journal 1984, 27(3):660-666.
11. Donohue J.M., Fox J.B. A multi-method evaluation of journals in the decision and management sciences by US academics. Omega 2000, 28(1):17-36.
12. DuBois F.L., Reeb D. Ranking the international business journals. Journal of International Business Studies 2000, 31(4):689-704.
13. Dul J., Karwowski W., Vinken J. Objective and subjective rankings of scientific journals in the field of ergonomics: 2004-2005. Human Factors and Ergonomics in Manufacturing 2005, 15(3):327-332.
14. Egghe L. Theory and practise of the g-index. Scientometrics 2006, 69(1):131-152.
15. Elkins M.R., Maher C.G., Herbert R.D., Moseley A.M., Sherrington C. Correlation between the Journal Impact Factor and three other journal citation indices. Scientometrics 2010, 85(1):81-93.
16. Etzkowitz H., Kemelgor C., Uzzi B. Athena unbound: The advancement of women in science and technology 2000, Cambridge University Press, Cambridge, UK.
17. Forgionne G.A., Kohli R., Jennings D. An AHP analysis of quality in AI and DSS journals. Omega 2002, 30(3):171-183.
18. Forgionne G.A., Kohlib R. A multiple criteria assessment of decision technology system journal quality. Information & Management 2001, 38(7):421-435.
19. Fox M.F. Gender, family characteristics, and publication productivity among scientists. Social Studies of Science 2005, 35(1):131-150.
20. Franceschet M. Journal influence factors. Journal of Informetrics 2010, 4(3):239-248.
21. Frandsen T.F. Journal self-citations - Analysing the JIF mechanism. Journal of Informetrics 2007, 1(1):47-58.
22. Gillenson M., Stafford T. Journal rankings 2008: A synthesis of studies. Proceedings of the 14th Americas Conference on Information Systems 2008.
23. Gillenson M.L., Stutz J.D. Academic issues in MIS: Journals and books. MIS Quarterly 1991, 15(4):447-452.
24. Goldstein H., Maier G. The use and valuation of journals in planning scholarship: Peer assessment versus impact factors. Journal of Planning Education and Research 2010, 30(1):66-75.
25. Gomez-Mejia L.R., Balkin D.B. Determinants of faculty pay: An agency theory perspective. The Academy of Management Journal 1992, 35(5):921-955.
26. Goodyear R.K., Brewer D.J., Gallagher K.S., Tracey T.J.G., Claiborn C.D., Lichtenberg J.W., Wampold B.E. The intellectual foundations of education: Core journals and their impacts on scholarship and practice. Educational Researcher 2009, 38(9):700-706.
27. Gordon M.D. Citation ranking versus subjective evaluation in the determination of journal hierarchies in the social sciences. Journal of the American Society for Information Science 1982, 33(1):55-57.
28. Haddow G., Genoni P. Citation analysis and peer ranking of Australian social science journals. Scientometrics 2010, 85:471-487.
29. Harzing A.-W., van der Wal R. Google Scholar as a new source for citation analysis. Ethics in Science and Environmental Politics 2008, 8(1):61-73.
30. Hirsch J.E. An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the United States of America 2005, 102(46):16569-16572.
31. Hodge D.R., Lacasse J.R. Evaluating journal quality: Is the h-index a better measure than impact factors?. Research on Social Work Practice 2010, 21(2):222-230.
32. Holsapple C.W., Johnson L.E., Manakyan H., Tanner J. An empirical assessment and categorization of journals relevant to DSS research. Decision Support Systems 1995, 14(4):359-367.
33. Jacques T.S., Sebire N.J. The impact of article titles on citation hits: An analysis of general and specialist medical journals. Journal of the Royal Society of Medicine 2010, 1(2):1-5.
34. Kao C., Lin H.-W., Chung S.-L., Tsai W.-C., Chiou J.-S., Chen Y.-L., Chen L.-H., Fang S.-C., Pao H.-L. Ranking Taiwanese management journals: A case study. Scientometrics 2008, 76(1):95-115.
35. Korevaar J.C., Moed H.F. Validation of bibliometric indicators in the field of mathematics. Scientometrics 1996, 37(1):117-130.
36. Lewison G. Researchers' and users' perceptions of the relative standing of biomedical papers in different journals. Scientometrics 2002, 53(2):229-240.
37. Lowry P.B., Humphreys S., Malwitz J., Nix J. A scientometric study of the perceived quality of business and technical communication journals. IEEE Transactions on Professional Communication 2007, 50(4):352-378.
38. Lowry P.B., Romans D., Curtis A. Global journal prestige and supporting disciplines: A scientometric study of information systems journals. Journal of the Association for Information Systems 2004, 5(2):29-77.
39. Maier G. Impact factors and peer judgment: The case of regional science journals. Scientometrics 2006, 69(3):651-667.
40. Manning L.M., Barrette J. Research performance in academe. Canadian Journal of Administrative Sciences 2005, 22(4):273-282.
41. McAllister P.R., Anderson R.C., Narin F. Comparison of peer and citation assessment of the influence of scientific journals. Journal of the American Society for Information Science 1980, 31(3):147-152.
42. McCarthy J. Inversion of functions defined by Turing Machines. Automata Studies 1956, 177-181. Princeton University Press, Princeton, NJ. C. Shannon, J. McCarthy (Eds.).
43. McCarthy J. Programs with common sense. Proceedings of the Teddington Conference on the Mechanization of Thought Processes 1958, 77-84.
44. Meho L.I., Yang K. Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology 2007, 58(13):2105-2125.
45. Mingers J., Harzing A.-W. Ranking journals in business and management: A statistical analysis of the Harzing data set. European Journal of Information Systems 2007, 16(4):303-316.
46. Mittal V., Feick L., Murshed F. Publish and prosper: The financial impact of publishing by marketing faculty. Marketing Science 2008, 27(3):430-442.
47. Mylonopoulos N.A., Theoharakis V. On site: Global perceptions of IS journals. Communications of the ACM 2001, 44(9):29-33.
48. Nederhof A.J., Luwel M., Moed H.F. Assessing the quality of scholarly journals in Linguistics: An alternative to citation-based journal impact factors. Scientometrics 2001, 51(1):241-265.
49. O'Brien J.P., Drnevich P.L., Crook T.R., Armstrong C.E. Does business school research add economic value for students?. Academy of Management Learning & Education 2010, 9(4):638-651.
50. Olson J.E. Top-25-business-school professors rate journals in operations management and related fields. Interfaces 2005, 35(4):323-338.
51. Peffers K., Ya T. Identifying and evaluating the universe of outlets for information systems research: Ranking the journals. Journal of Information Technology Theory and Application 2003, 5(1):63-84.
52. Peritz B.C., Bar-Ilan J. The sources used by bibliometrics-scientometrics as reflected in references. Scientometrics 2002, 54(2):269-284.
53. Rogers P.S., Campbell N., Louhiala-Salminen L., Rentz K., Suchan J. The impact of perceptions of journal quality on business and management communication academics. Journal of Business Communication 2007, 44(4):403-426.
54. Rousseau R. Temporal differences in self-citation rates of scientific journals. Scientometrics 1999, 44(3):521-531.
55. Rousseau S. Journal evaluation by environmental and resource economists: A survey. Scientometrics 2008, 77(2):223-233.
56. Saha S., Saint S., Christakis D.A. Impact factor: A valid measure of journal quality?. Journal of the Medical Library Association 2003, 91(1):42-46.
57. Schloegl C., Stock W.G. Impact and relevance of LIS journals: A scientometric analysis of international and German-language LIS journals - Citation analysis versus reader survey. Journal of the American Society for Information Science 2004, 55(13):1155-1168.
58. Seglen P.O. The skewness of science. Journal of the American Society for Information Science 1992, 43(9):628-638.
59. Seglen P.O. Why the impact factor of journals should not be used for evaluating research. British Medical Journal 1997, 314(7079):498-502.
60. Sellers S.L., Mathiesen S.G., Perry R., Smith T. Evaluation of social work journal quality: Citation versus reputation. Journal of Social Work Education 2004, 40(1):143-160.
61. Serenko A. The development of an AI journal ranking based on the revealed preference approach. Journal of Informetrics 2010, 4(4):447-459.
62. Serenko A., Bontis N. A citation-based ranking of the business ethics scholarly journals. International Journal of Business Governance and Ethics 2009, 4(4):390-399.
63. Serenko A., Bontis N. Global ranking of knowledge management and intellectual capital academic journals. Journal of Knowledge Management 2009, 13(1):4-15.
64. Serenko A., Bontis N. What's familiar is excellent: The impact of exposure effect on perceived journal quality. Journal of Informetrics 2011, 5(1):219-223.
65. Sevinc A. Manipulating impact factor: An unethical issue or an Editor's choice?. Swiss Medical Weekly 2004, 134(27):410.
66. Sidiropoulos A., Katsaros D., Manolopoulos Y. Generalized h-index for disclosing latent facts in citation networks. Scientometrics 2007, 72(2):253-280.
67. Sonderstrup-Andersen E.M., Sonderstrup-Andersen H.H.K. An investigation into diabetes researcher's perceptions of the Journal Impact Factor - reconsidering evaluating research. Scientometrics 2008, 76(2):391-406.
68. Starbuck W.H. How much better are the most-prestigious journals? The statistics of academic publication. Organization Science 2005, 16(2):180-200.
69. Suchan J. How academic organizational systems and culture undermine scholarship and quality research: A response to Ron Dulek. Journal of Business Communication 2008, 45(3):349-356.
70. Tahai A., Meyer M.J. A revealed preference study of management journals' direct influences. Strategic Management Journal 1999, 20(3):279-296.
71. Thomas P.R., Watkins D.S. Institutional research rankings via bibliometric analysis and direct peer review: A comparative case study with policy implications. Scientometrics 1998, 41(3):335-355.
72. Truex D., Cuellar M., Takeda H. Assessing scholarly influence: Using the Hirsch indices to reframe the discourse. Journal of the Association for Information Systems 2009, 10(7):560-594.
73. Vanclay J.K. Ranking forestry journals using the h-index. Journal of Informetrics 2008, 2:326-334.
74. Walstrom K.A., Hardgrave B.C. Forums for information systems scholars. Information & Management 2001, 39(2):117-124.