Predicting the impact of scientific concepts using full-text features

Journal of the Association for Information Science and Technology

Volumn 67, Issue 11, 2016, Pages 2684-2696

  McKeown, Kathy ^a   Daume, Hal ^b   Chaturvedi, Snigdha ^c   Paparrizos, John ^a   Thadani, Kapil ^a   Barrio, Pablo ^a   Biran, Or ^a   Bothe, Suvarna ^a   Collins, Michael ^a   Fleischmann, Kenneth R ^d   Gravano, Luis ^a   Jha, Rahul ^e   King, Ben ^f   McInerney, Kevin ^g   Moon, Taesun ^h   Neelakantan, Arvind ⁱ   O'Seaghdha, Diarmuid ^j   Radev, Dragomir ^e   Templeton, Clay ^d   Teufel, Simone ^j  

^a Columbia University ^*  (United States)

^b UNIVERSITY OF MARYLAND   (United States)

^c The University of Texas at Austin School of Information ^*  (United States)

^d UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^e UNIVERSITY OF MICHIGAN   (United States)

^f Ubiquiti Inc   (United States)

^g RUTGERS UNIVERSITY   (United States)

^h IBM T J WATSON RESEARCH CENTER   (United States)

ⁱ Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

^j UNIVERSITY OF CAMBRIDGE   (United Kingdom)

more..

Author keywords

machine learning; natural language processing; scientometrics

Indexed keywords

LEARNING ALGORITHMS; LEARNING SYSTEMS; METADATA; NATURAL LANGUAGE PROCESSING SYSTEMS; TIME SERIES ANALYSIS;

ACCURATE PREDICTION; LARGE SCALE EXPERIMENTS; LONG-TERM IMPACTS; PREDICTION TASKS; SCIENTIFIC LITERATURE; SCIENTOMETRICS; SENTENCE ANALYSIS; TECHNICAL TERMS;

FORECASTING;

EXTRACTION; HUMAN; HUMAN EXPERIMENT; MACHINE LEARNING; NATURAL LANGUAGE PROCESSING; PREDICTION; TIME SERIES ANALYSIS;

EID: 84984956897     PISSN: 23301635     EISSN: 23301643     Source Type: Journal    
DOI: 10.1002/asi.23612     Document Type: Article

References (96)

1. Acuna, D.E., Allesina, S., & Kording, K.P. (2012). Future impact: Predicting scientific success. Nature, 489(7415), 201–202.
2. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723.
3. Arbesman, S., & Christakis, N.A. (2011). Eurekometrics: Analyzing the nature of discovery. PLoS Computational Biology, 7(6), e1002072.
4. Athar, A. (2011). Sentiment analysis of citations using sentence structure-based features. In Proceedings of the ACL-11 Student Session. Portland, OR: Association for Computational Linguistics.
5. Bach, N., & Badaskar, S. (2007). A review of relation extraction. Literature Review for Language and Statistics II.
6. Bartneck, C., & Hu, J. (2009). Scientometric analysis of the CHI proceedings. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems.
7. Batagelj, V., & Cerinšek, M. (2013). On bibliographic networks. Scientometrics, 96(3), 845–864.
8. Bayer, A.E., & Folger, J. (1966). Some correlates of a citation measure of productivity in science. Sociology of Education, 39(4), 381–390.
9. Beel, J., & Gipp, B. (2009). Google Scholar's ranking algorithm: The impact of citation counts (an empirical study). In Proceedings of the 3rd International Conference on Research Challenges in Information Science (Rcis-09) (pp. 439–446). Fez, Morocco: Institute of Electrical and Electronics Engineers.
10. Birnholtz, J., Guha, S., Yuan, Y., Gay, G., & Heller, C. (2013). Cross-campus collaboration: A sceintometric and network case study of publication activity across two campuses of a single insitution. Journal of the American Society for Information Science and Technology, 64(1), 162–172.
11. Bonzi, S. (1982). Characteristics of a literature as predictors of relatedness between cited and citing works. Journal of the American Society for Information Science, 33(4), 208–216.
12. Bornmann, L., & Daniel, H.-D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45–80.
13. Bornmann, L., & Daniel, H.-D. (2009). The state of h index research. Is the h index the ideal way to measure research performance? EMBO Reports, 10(1), 2–6.
14. Box, G.E., & Cox, D.R. (1964). An analysis of transformations. Journal of the Royal Statistical Society: Series B (Methodological), 26(2), 211–252.
15. Boyack, K.W., Newman, D., Duhon, R.J., Klavans, R., Patek, M., Biberstine, J.R., & Börner, K. (2011). Clustering more than two million biomedical publications: Comparing the accuracies of nine text-based similarity approaches. PLoS ONE, 6(3), e18029.
16. Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57(8), 1060–1072.
17. Bui, Q.-C., Katrenko, S., & Sloot, P.M. (2011). A hybrid approach to extract protein–protein interactions. Bioinformatics (Oxford, England), 27(2), 259–265.
18. Burnham, K.P., & Anderson, D.R. (2002). Model selection and multimodel inference: A practical information-theoretic approach. New York: Springer Science & Business Media.
19. Burnham, K.P., & Anderson, D.R. (2004). Multimodel inference understanding aic and bic in model selection. Sociological Methods & Research, 33(2), 261–304.
20. Callaham, M., Wears, R.L., & Weber, E. (2002). Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. Journal of the American Medical Association, 287(21), 2847–2850.
21. Castillo, C., Donato, D., & Gionis, A. (2007). Estimating number of citations using author reputation. In Proceedings of the 14th International Symposium on String Processing and Information Retrieval (Spire-07) (pp. 107–117). Berlin: Springer-Verlag.
22. Chen, C. (2012). Predictive effects of structural variation on citation counts. Journal of the American Society for Information Science and Technology, 63(3), 431–449. Retrieved from http://doi.wiley.com/10.1002/asi.21694
23. Chubin, D.E., & Moitra, S.D. (1975). Content analysis of references: Adjunct or alternative to citation counting? Social Studies of Science, 5(4), 423–441.
24. Cole, S., & Cole, J.R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32(3), 391–403.
25. Ding, Y., Yan, E., Frazho, A., & Caverlee, J. (2009). PageRank for ranking authors in co-citation networks. Journal of the American Society for Information Science and Technology, 60(11), 2229–2243. Retrieved from http://doi.wiley.com/10.1002/asi.21171
26. Ding, Y., Song, M., Han, J., Yu, Q., Yan, E., Lin, L., & Chambers, T. (2013). Entitymetrics: Measuring the impact of entities. PLoS ONE, 8(8), e71416.
27. Dong, Y., Johnson, R.A., & Chawla, N.V. (2014). Will this paper increase your h-index? Scientific Impact Prediction. ArXiv e-prints.
28. Edge, D. (1979). Quantitative measures of communication in science: A critical review. History of Science, 17(36), 102–134.
29. Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13(4), e123.
30. Freeman, L.C. (1977). A set of measures of centrality based on betweenness. Sociometry, 40(1), 35–41. Retrieved from http://links.jstor.org/sici?sici=0038-0431\%28197703\%2940\%3A1\%3C35\%3AASOMOC\%3E2.0.CO\%3B2-H
31. Freeman, L.C. (1978). Centrality in social networks: Conceptual clarification. Social Networks, 3(1), 215–239.
32. Fu, L.D., & Aliferis, C. (2008). Models for predicting and explaining citation count of biomedical articles. In Proceedings of the AMIA Annual Symposium (pp. 222–226). Washington, DC: American Medical Informatics Association.
33. Fu, T.Z.J., Song, Q., & Chiu, D.M. (2013). The academic social network. Retrieved from http://arxiv.org/abs/1306.4623.
34. Funk, R., & Owen-Smith, J. (2012). A dynamic network approach to breakthrough innovation. Retrieved from http://arxiv.org/abs/1212.3559
35. Garfield, E. (2006). Citation indexes for science: A new dimension in documentation through association of ideas. International Journal of Epidemiology, 35(5), 1123–1127.
36. Garfield, E., & Malin, M.V. (1968). Can Nobel Prize winners be predicted? In Proceedings of the 135th Meeting of the American Association for the Advancement of Science. Dallas, TX: American Association for the Advancement of Science.
37. Gehrke, J., Ginsparg, P., & Kleinberg, J. (2003). Overview of the 2003 KDD Cup. ACM SIGKDD Explorations Newsletter, 5(2), 149–151.
38. Grueber, M., & Studt, T. (2012). Global R&D funding forecast. R&D Magazine, 16, 3–35.
39. Guha, S., Steinhardt, S., Ahmed, S., & Lagoze, C. (2013). Following bibliometric footprints: The ACM digital library and the evolution of computer science. Proceedings of the 13th Annual ACM/IEEE-CS Joint Conferenceo on Digital Libraries.
40. Gupta, S., & Manning, C.D. (2010). Identifying focus, techniques and domain of scientific papers. In Proceedings of the Nips-10 Workshop on Computational Social Science and the Wisdom of Crowds. Whistler, Canada: Neural Information Processing Systems (NIPS) Foundation.
41. Hall, D., Jurafsky, D., & Manning, C.D. (2008). Studying the history of ideas using topic models. In Processing (EMNLP-08) Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (Emnlp-08) (pp. 363–371). Honolulu, HI: Association for Computational Linguistics.
42. Havemann, F., & Larsen, B. (2014). Bibliometric indicators of young authors in astrophysics: Can later stars be predicted? ArXiv e-prints.
43. Hilborn, R.C. (2000). Chaos and nonlinear dynamics: An introduction for scientists and engineers. Oxford, UK: Oxford University Press.
44. Hirsch, J.E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences U S A, 102(46), 16569–16572.
45. Hirsch, J.E. (2007). Does the h index have predictive power? Proceedings of the National Academy of Sciences U S A, 104(49), 19193–19198.
46. Hönekopp, J., & Khan, J. (2012). Future publication success in science is better predicted by traditional measures than by the h index. Scientometrics, 90(3), 843–853.
47. Horn, D., Finholt, T., Birnholtz, J., Motwani, D., & Jayaraman, S. (2004). Six degrees of jonathan grudin: A social network analysis of the evolution and impact of cscw research. Proceedings of the 2004 ACM Conference on Computer Supported Cooperative Work (CSCW '04).
48. Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28(3–4), 321–377.
49. Ibáñez, A., Larrañaga, P., & Bielza, C. (2009). Predicting citation count of bioinformatics papers within four years of publication. Bioinformatics (Oxford, England), 25(24), 3303–3309.
50. Joachims, T. (1998). Text categorization with support vector machines: Learning with many relevant features. Berlin, Germany: Springer.
51. Kessler, M.M. (1965). Comparison of the results of bibliographic coupling and analytic subject indexing. American Documentation, 16(3), 223–233.
52. Knorr-Cetina, K. (1999). Epistemic cultures: How the sciences make knowledge. Cambridge, MA: Harvard University Press.
53. Kulkarni, A.V., Busse, J.W., & Shams, I. (2007). Characteristics associated with citation rate of the medical literature. PLoS ONE, 2(5), e403.
54. Lafferty, J., McCallum, A., & Pereira, F.C. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning (Icml-01) (pp. 282–289). Williamstown, MA: Morgan Kaufman.
55. Lane, J. (2010). Let's make science metrics more scientific. Nature, 464(7288), 488–489.
56. Lane, J., & Bertuzzi, S. (2011). Measuring the results of science investments. Science, 331(6018), 678–680.
57. Latour, B. (1988). Science in action: How to follow scientists and engineers through society. Cambridge, MA: Harvard University Press.
58. Laurance, W.F., Useche, D.C., Laurance, S.G., & Bradshaw, C.J. (2013). Predicting publication success for biologists. Bioscience, 63(10), 817–823.
59. Lokker, C., McKibbon, K., McKinlay, R.J., Wilczynski, N.L., & Haynes, R.B. (2008). Prediction of citation counts for clinical articles at two years using data available within three weeks of publication: Retrospective cohort study. British Medical Journal, 336(7645), 655–657.
60. Losiewicz, P., Oard, D.W., & Kostoff, R.N. (2000). Textual data mining to support science and technology management. Journal of Intelligent Information Systems, 15(2), 99–119.
61. Louis, A., & Nenkova, A. (2013). What makes writing great? First experiments on article quality prediction in the science journalism domain. Transactions of the Association for Computational Linguistics, 1, 341–352.
62. MacRoberts, M.H., & MacRoberts, B.R. (1984). The negational reference: Or the art of dissembling. Social Studies of Science, 14(1), 91–94.
63. Manjunatha, J.N., Sivaramakrishnan, K.R., Pandey, R.K., & Murthy, M.N. (2003). Citation prediction using time series approach KDD Cup 2003 (task 1). ACM SIGKDD Explorations Newsletter, 5(2), 152–153.
64. McCarty, C., Jawitz, J.W., Hopkins, A., & Goldman, A. (2013). Predicting author h-index using characteristics of the co-author network. Scientometrics, 96(2), 1–17.
65. Moravcsik, M.J., & Murugesan, P. (1975). Some results on the function and quality of citations. Social Studies of Science, 5(1), 86–92.
66. Myers, G. (1992). In this paper we report …—speech acts and scientific facts. Journal of Pragmatics, 17(4), 295–313.
67. Narin, F. (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity. Washington, DC: National Science Foundation.
68. Neelakantan, A., & Collins, M. (2014). Learning dictionaries for named entity recognition using minimal supervision. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics (Eacl-14). Gothenburg, Sweden: Association for Computational Linguistics.
69. Newman, M. (2003). Mixing patterns in networks. Physical Review E, 67(2), 026126/1–13.
70. Newman, M. (2010). Networks: An introduction. Oxford, UK: Oxford University Press.
71. Pan, R.K., Kaski, K., & Fortunato, S. (2012). World citation and collaboration networks: Uncovering the role of geography in science. Scientific Reports, 2, 902/1–9027.
72. Penner, O., Petersen, A.M., Pan, R.K., & Fortunato, S. (2013). The case for caution in predicting scientists' future impact. Physics Today, 66(4), 8–9.
73. Sarigöl, E., Pfitzner, R., Scholtes, I., Garas, A., & Schweitzer, F. (2014). Predicting scientific success based on coauthorship networks. ArXiv e-prints.
74. Schreiber, M. (2013). How relevant is the predictive power of the h index? A case study of the time-dependent Hirsch index. Journal of Informetrics, 7(2), 325–329.
75. Small, H. (1973). Co-citation in the scientific literature: A new measure of the relationship between two documents. Journal of the American Society for Information Science, 24(4), 265–269.
76. Small, H. (2011). Interpreting maps of science using citation context sentiments: A preliminary investigation. Scientometrics, 87(2), 373–388.
77. Spiegel-Rösing, I. (1977). Science studies: Bibliometric and content analysis. Social Studies of Science, 7(1), 97–113.
78. Sun, X., Kaur, J., Milojević, S., Flammini, A., & Menczer, F. (2013). Social dynamics of science. Scientific Reports, 3, 1069/1–10696.
79. Swales, J. (1990). Genre analysis: English in academic and research. In Chap. 7: Research articles in English. Cambridge, UK: Cambridge University Press.
80. Tan, C., & Lee, L. (2014). A corpus of sentence-level revisions in academic writing: A step towards understanding statement strength in communication. Proceedings of ACL.
81. Teräsvirta, T., Lin, C.F., & Granger, C.W. (1993). Power of the neural network linearity test. Journal of Time Series Analysis, 14(2), 209–220.
82. Teufel, S. (2010). The structure of scientific articles: Applications to citation indexing and summarization. Stanford, CA: CSLI Publications.
83. Traweek, S. (1992). Beamtimes and lifetimes: The world of high energy physicists. Cambridge, MA: Harvard University Press.
84. Tsai, C.-T., Kundu, G., & Roth, D. (2013). Concept-based analysis of scientific literature. Proceedings of the 22nd ACM International Conference on Conference on Information & Knowledge Management (pp. 1733–1738).
85. Velden, T., & Lagoze, C. (2013). The extraction of community structures from publication networks to support ethnographic observations of field differences in scientific communication. Journal of the American Society for Information Science and Technology, 64(12), 2405–2427.
86. Velden, T., Haque, A., & Lagoze, C. (2010). A new approach to analyzing patterns of collaboration in co-authorship networks: Mesoscopic analysis and interpretation. Scientometrics, 85(1), 219–242.
87. Viana, M.P., Amancio, D.R., & Costa, L.D.F. (2013). On time-varying collaboration networks. Journal of Informetrics, 7(2), 371–378.
88. Wang, S., Xie, S., Zhang, X., Li, Z., Yu, P.S., & Shu, X. (2014). Future influence ranking of scientific literature. ArXiv e-prints.
89. Wang, X., Smith, K., & Hyndman, R. (2006). Characteristic-based clustering for time series data. Data Mining and Knowledge Discovery, 13(3), 335–364.
90. Watts, D.J., & Strogatz, S.H. (1998). Collective dynamics of “small-world” networks. Nature, 393(6684), 409–410.
91. Wick, M.L., Kobren, A., & McCallum, A. (2013). Large-scale author coreference via hierarchical entity representations. In Proceedings of the ICML Workshop on Peer Reviewing and Publishing Models. Atlanta, GA: Morgan Kaufman.
92. Willinger, W., Paxson, V., & Taqqu, M.S. (1998). Self-similarity and heavy tails: Structural modeling of network traffic. In R.J. Adler, R.E. Feldman, & M.S. Taqqu (Eds.), A practical guide to heavy tails: Statistical techniques and applications (pp. 27–53). Boston, MA: Birkhäuser.
93. Yan, R., Tang, J., Liu, X., Shan, D., & Li, X. (2011). Citation count prediction: learning to estimate future citations for literature. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (Cikm-11) (pp. 1247–1252). Glasgow, UK: Association for Computing Machinery.
94. Yogatama, D., Heilman, M., O'Connor, B., Dyer, C., Routledge, B.R., & Smith, N.A. (2011). Predicting a scientific community's response to an article. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (Emnlp-11) (pp. 594–604). Edinburgh, UK: Association for Computational Linguistics.
95. Zhu, X., Turney, P., Lemire, D., & Vellino, A. (2015). Measuring academic influence: Not all citations are equal. Journal of the Association for Information Science and Technology, 66(2), 408–427. Retrieved from http://dx.doi.org/10.1002/asi.23179
96. Ziman, J.M. (1968). Public knowledge: An essay concerning the social dimension of science. Cambridge, UK: Cambridge University Press.