Text mining for market prediction: A systematic review

Expert Systems with Applications

Volumn 41, Issue 16, 2014, Pages 7653-7670

  Khadjeh Nassirtoussi, Arman ^a   Aghabozorgi, Saeed ^a   Ying Wah, Teh ^a   Ngo, David Chek Ling ^b  

^a UNIVERSITY OF MALAYA   (Malaysia)

^b SUNWAY UNIVERSITY   (Malaysia)

Author keywords

FOREX market prediction; News sentiment analysis; Online sentiment analysis; Social media text mining; Stock prediction based on news

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMMERCE; FORECASTING; SENTIMENT ANALYSIS; SOCIAL NETWORKING (ONLINE);

BEHAVIORAL ECONOMICS; COMPARATIVE ANALYSIS; FOREX MARKETS; GENERIC COMPONENTS; RESEARCH COMMUNITIES; SOCIAL MEDIA; STOCK PREDICTIONS; TECHNICAL FRAMEWORKS;

TEXT MINING;

EID: 84904349877     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2014.06.009     Document Type: Review

References (117)

1. M.H. Aghdam, N. Ghasem-Aghaee, and M.E. Basiri Text feature selection using ant colony optimization Expert Systems with Applications 36 2009 6843 6853
2. L. Anastasakis, and N. Mort Exchange rate forecasting using a combined parametric and nonparametric self-organising modelling approach Expert Systems with Applications 36 2009 12001 12011
3. M. Bahrepour, M.-R. Akbarzadeh-T, M. Yaghoobi, and M.-B. Naghibi-S An adaptive ordered fuzzy time series with application to FOREX Expert Systems with Applications 38 2011 475 485
4. A. Balahur, R. Steinberger, E.V.D. Goot, B. Pouliquen, and M. Kabadjov Opinion mining on newspaper quotations Proceedings of the 2009 IEEE/WIC/ACM international joint conference on web intelligence and intelligent agent technology 2009 IEEE Computer Society 523 526
5. T. Berka, and M. Vajteršic Parallel rare term vector replacement: Fast and effective dimensionality reduction for text Journal of Parallel and Distributed Computing 73 2013 341 351
6. E. Bikas, D. Jurevičiene, P. Dubinskas, and L. Novickyte Behavioural finance. The emergence and development trends Procedia - Social and Behavioral Sciences 82 2013 870 876
7. J. Bollen, and M. Huina Twitter mood as a stock market predictor Computer 44 2011 91 94
8. C.C. Burges A tutorial on support vector machines for pattern recognition Data Mining and Knowledge Discovery 2 1998 121 167
9. M. Butler, and V. Kešelj Financial forecasting using character n-gram analysis and readability scores of annual reports Y. Gao, N. Japkowicz, Advances in artificial intelligence 2009 Springer Berlin Heidelberg 39 51
10. E. Cambria, B. Schuller, X. Yunqing, and C. Havasi New avenues in opinion mining and sentiment analysis IEEE Intelligent Systems 28 2013 15 21
11. C.-C. Chang, and C.-J. Lin LIBSVM: A library for support vector machines ACM Transactions on Intelligent Systems and Technology 2 2011 1 27
12. A. Chatrath, H. Miao, S. Ramchander, and S. Villupuram Currency jumps, cojumps and the role of macro news Journal of International Money and Finance 40 2014 42 62
13. T. Chordia, A. Goyal, B.N. Lehmann, and G. Saar High-frequency trading Journal of Financial Markets 2013
14. T. Chordia, R. Roll, and A. Subrahmanyam Evidence on the speed of convergence to market efficiency Journal of Financial Economics 76 2005 271 292
15. T. Chordia, R. Roll, and A. Subrahmanyam Recent trends in trading activity and market quality Journal of Financial Economics 101 2011 243 263
16. M. Collins, and N. Duffy Convolution kernels for natural language 2001 MIT Press pp. 625-632
17. S.R. Das, and M.Y. Chen Yahoo! for Amazon: Sentiment extraction from small talk on the web Management Science 53 2007 1375 1388 (Pubitemid 351159590)
18. B. Desmet, and V. Hoste Emotion detection in suicide notes Expert Systems with Applications 40 2013 6351 6358
19. H. Drucker, C.J.C. Burges, L. Kaufman, A. Smola, and V. Vapnik Support vector regression machines Advances in Neural Information Processing Systems 1997 MIT Press 155 161
20. E. Duman, Y. Ekinci, and A. TanrIverdi Comparing alternative classifiers for database marketing: The case of imbalanced datasets Expert Systems with Applications 39 2012 48 53
21. A. Duric, and F. Song Feature selection for sentiment analysis based on content and syntax models Decision Support Systems 53 2012 704 711
22. C. Evans, K. Pappas, and F. Xhafa Utilizing artificial neural networks and genetic algorithms to build an algo-trading model for intra-day foreign exchange speculation Mathematical and Computer Modelling 58 2013 1249 1266
23. E.F. Fama Random walks in stock market prices Financial Analysts Journal 21 1965 55 59
24. E.F. Fama Efficient capital markets: A review of theory and empirical work The Journal of Finance 25 1970 383 417
25. R.-E. Fan, P.-H. Chen, and C.-J. Lin Working set selection using second order information for training support vector machines Journal of Machine Learning Research 6 2005 1889 1918 (Pubitemid 41798130)
26. M. Fasanghari, and G.A. Montazer Design and implementation of fuzzy expert system for Tehran stock exchange portfolio recommendation Expert Systems with Applications 37 2010 6138 6147
27. G. Feng, J. Guo, B.-Y. Jing, and L. Hao A Bayesian feature selection paradigm for text classification Information Processing and Management 48 2012 283 302
28. G. Friesen, and P.A. Weller Quantifying cognitive biases in analyst earnings forecasts Journal of Financial Markets 9 2006 333 365 (Pubitemid 44490352)
29. D. García, and B. Urošević Noise and aggregation of information in large markets Journal of Financial Markets 16 2013 526 549
30. J. Garcke, T. Gerstner, and M. Griebel Intraday foreign exchange rate forecasting using sparse grids J. Garcke, M. Griebel, Sparse grids and applications 2013 Springer Berlin Heidelberg 81 105
31. R. Ghazali, A.J. Hussain, and P. Liatsis Dynamic ridge polynomial neural network: Forecasting the univariate non-stationary and stationary trading signals Expert Systems with Applications 38 2011 3765 3776
32. M. Ghiassi, J. Skinner, and D. Zimbra Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network Expert Systems with Applications 40 2013 6266 6282
33. N. Gradojevic, and R. Gençay Fuzzy logic, trading uncertainty and technical trading Journal of Banking and Finance 37 2013 578 586
34. T.L. Griffiths, M. Steyvers, D.M. Blei, and J.B. Tenenbaum Integrating topics and syntax 2005 MIT Press pp. 537-544
35. S.S. Groth, and J. Muntermann An intraday market risk management approach based on textual analysis Decision Support Systems 50 2011 680 691
36. E. Haddi, X. Liu, and Y. Shi The role of text pre-processing in sentiment analysis Procedia Computer Science 17 2013 26 32
37. M. Hagenau, M. Liebmann, and D. Neumann Automated news reading: Stock price prediction based on financial news using context-capturing features Decision Support Systems 55 2013 685 697
38. J. Hasbrouck, and G. Saar Low-latency trading Journal of Financial Markets 2013
39. C. Hsinchun, and D. Zimbra AI and opinion mining IEEE Intelligent Systems 25 2010 74 80
40. S.-C. Huang, P.-J. Chuang, C.-F. Wu, and H.-J. Lai Chaos-based support vector regressions for exchange rate forecasting Expert Systems with Applications 37 2010 8590 8598
41. C.-J. Huang, J.-J. Liao, D.-X. Yang, T.-Y. Chang, and Y.-C. Luo Realization of a news dissemination agent based on weighted association rules and text mining techniques Expert Systems with Applications 37 2010 6409 6413
42. S. Jiang, G. Pang, M. Wu, and L. Kuang An improved K-nearest-neighbor algorithm for text categorization Expert Systems with Applications 39 2012 1503 1509
43. F. Jin, N. Self, P. Saraf, P. Butler, W. Wang, and N. Ramakrishnan Forex-foreteller: Currency trend modeling using news articles Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining 2013 ACM Chicago, Illinois, USA 1470 1473
44. Joachims, T. (1999). Making large-scale {SVM} learning practical. In B. Schölkopf, C. Burges & A. Smola (Eds.), (pp. 169-184). Cambridge, MA: MIT Press.
45. T. Joachims Learning to classify text using support vector machines - methods, theory, and algorithms 2002 Springer Kluwer
46. T. Joachims Text categorization with support vector machines: Learning with many relevant features C. Nédellec, C. Rouveirol, Machine learning: ECML-98 1998 Springer Berlin Heidelberg 137 142 Vol. 1398 (Pubitemid 128067178)
47. P.R. Kaltwasser Uncertainty about fundamentals and herding behavior in the FOREX market Physica A: Statistical Mechanics and its Applications 389 2010 1215 1222
48. H. Kanayama, and T. Nasukawa Textual demand analysis: Detection of users' wants and needs from opinions Proceedings of the 22nd international conference on computational linguistics Vol. 1 2008 Association for Computational Linguistics Manchester, United Kingdom 409 416
49. A. Khadjeh Nassirtoussi, T. Ying Wah, and D. Ngo Chek Ling A novel FOREX prediction methodology based on fundamental data African Journal of Business Management 5 2011 8322 8330
50. K. Kim, B.-S. Chung, Y. Choi, S. Lee, J.-Y. Jung, and J. Park Language independent semantic kernels for short-text classification Expert Systems with Applications 41 2014 735 743
51. J. Kleinnijenhuis, F. Schultz, D. Oegema, and W.H.van. Atteveldt Financial news and market panics in the age of high frequency trading algorithms Journalism 2013 14
52. E. Kontopoulos, C. Berberidis, T. Dergiades, and N. Bassiliades Ontology-based sentiment analysis of twitter posts Expert Systems with Applications 40 2013 4065 4074
53. D. Lewis Naive (Bayes) at forty: The independence assumption in information retrieval C. Nédellec, C. Rouveirol, Machine learning: ECML-98 1998 Springer Berlin Heidelberg 4 15
54. F. Li The information content of forward-looking statements in corporate filings - a naïve Bayesian machine learning approach Journal of Accounting Research 48 2010 1049 1102
55. W. Li, and H. Xu Text-based emotion classification using emotion cause extraction Expert Systems with Applications 41 2014 1742 1749
56. C.H. Li, J.C. Yang, and S.C. Park Text categorization algorithms using semantic approaches, corpus-based thesaurus and WordNet Expert Systems with Applications 39 2012 765 772
57. Y. Liu, H.T. Loh, and A. Sun Imbalanced text classification: A term weighting approach Expert Systems with Applications 36 2009 690 701
58. A.W. Lo Reconciling efficient markets with behavioral finance: The adaptive markets hypothesis Journal of Investment Consulting 2005
59. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins Text classification using string kernels Journal of Machine Learning Research 2 2002 419 444 (Pubitemid 135712568)
60. V. Loia, and S. Senatore A fuzzy-oriented sentic analysis to capture the human emotion in web-based content Knowledge-Based Systems 58 2014 75 85
61. Lugmayr, A.; & Gossen, G. (2012). Evaluation of methods and techniques for language based sentiment analysis for DAX 30 stock exchange - a first concept of a "LUGO" sentiment indicator. In A. Lugmayr, T. Risse, B. Stockleben, J. Kaario, B. Pogorelc & E. Serral Asensio (Eds.), SAME 2012-5th international workshop on semantic ambient media experience.
62. Q. Luo, E. Chen, and H. Xiong A semantic term weighting scheme for text categorization Expert Systems with Applications 38 2011 12708 12716
63. E. Lupiani-Ruiz, I. García-Manotas, R. Valencia-García, F. García-Sánchez, D. Castellanos-Nieves, and J.T. Fernández-Breis et al. Financial news semantic search engine Expert Systems with Applications 38 2011 15565 15572
64. S. Mabu, K. Hirasawa, M. Obayashi, and T. Kuremoto Enhanced decision making mechanism of rule-based genetic network programming for creating stock trading signals Expert Systems with Applications 40 2013 6311 6320
65. Mahajan, A.; Dey, L.; & Haque, S. M. (2008). Mining financial news for major events and their impacts on the market. In IEEE/WIC/ACM international conference on web intelligence and intelligent agent technology, 2008. WI-IAT '08 (Vol. 1, pp. 423-426).
66. D. Majumder Towards an efficient stock market: Empirical evidence from the Indian market Journal of Policy Modeling 35 2013 572 587
67. I. Maks, and P. Vossen A lexicon model for deep sentiment analysis and opinion mining applications Decision Support Systems 53 2012 680 688
68. G.A. Miller WordNet: A lexical database for English Communications of the ACM 38 1995 39 41
69. Mittermayer, M. A. (2004). Forecasting intraday stock price trends with text mining techniques. In Proceedings of the 37th annual Hawaii international conference on system sciences, 2004 (pp. 10).
70. R. Moraes, J.F. Valiati, and W.P. Gavião Neto Document-level sentiment classification: An empirical comparison between SVM and ANN Expert Systems with Applications 40 2013 621 633
71. M.M. Mostafa More than words: Social networks' text mining for consumer brand sentiments Expert Systems with Applications 40 2013 4241 4251
72. Nikfarjam, A.; Emadzadeh, E.; & Muthaiyah, S. (2010). Text mining approaches for stock market prediction. In The 2nd international conference on computer and automation engineering (ICCAE), 2010 (Vol. 4, pp. 256-260).
73. J. Ortigosa-Hernández, J.D. Rodríguez, L. Alzate, M. Lucania, I. Inza, and J.A. Lozano Approaching sentiment analysis by using semi-supervised learning of multi-dimensional classifiers Neurocomputing 92 2012 98 115
74. D. Peramunetilleke, and R.K. Wong Currency exchange rate forecasting from news headlines Australian Computer Science Communications 24 2002 131 139
75. V. Pestov Is the -NN classifier in high dimensions affected by the curse of dimensionality? Computers and Mathematics with Applications 65 2013 1427 1437
76. J.C. Platt Fast training of support vector machines using sequential minimal optimization Advances in kernel methods 1999 MIT Press 185 208
77. V. Potì, and A. Siddique What drives currency predictability? Journal of International Money and Finance 36 2013 86 106
78. B. Premanode, and C. Toumazou Improving prediction of exchange rates using differential EMD Expert Systems with Applications 40 2013 377 384
79. Pui Cheong Fung, G.; Xu Yu, J.; & Wai, L. (2003). Stock prediction: Integrating text mining approach using real-time news. In 2003 IEEE international conference on computational intelligence for financial engineering, 2003. Proceedings (pp. 395-402).
80. J.R. Quinlan C4.5: Programs for machine learning 1993 Morgan Kaufmann Publishers Inc
81. Rachlin, G.; Last, M.; Alberg, D.; & Kandel, A. (2007). ADMIRAL: A data mining based financial trading system. In IEEE symposium on computational intelligence and data mining, 2007. CIDM 2007 (pp. 720-725). (Pubitemid 47431511)
82. J.C. Reboredo, M.A. Rivera-Castro, J.G.V. Miranda, and R. García-Rubio How fast do stock prices adjust to market efficiency? Evidence from a detrended fluctuation analysis Physica A: Statistical Mechanics and its Applications 392 2013 1631 1637
83. C. Robertson, S. Geva, and R. Wolff What types of events provide the strongest evidence that the stock market is affected by company specific news? Proceedings of the fifth Australasian conference on data mining and analystics Vol. 61 2006 Australian Computer Society Inc Sydney, Australia 145 153
84. S. Salzberg On comparing classifiers: Pitfalls to avoid and a recommended approach Data Mining and Knowledge Discovery 1 1997 317 328
85. S. Sankaraguruswamy, J. Shen, and T. Yamada The relationship between the frequency of news release and the information asymmetry: The role of uninformed trading Journal of Banking and Finance 37 2013 4134 4143
86. R.P. Schumaker, and H. Chen Textual analysis of stock market prediction using breaking financial news: The AZFin text system ACM Transactions of Information Systems 27 2009 1 19
87. R.P. Schumaker, Y. Zhang, C.-N. Huang, and H. Chen Evaluating sentiment in financial news articles Decision Support Systems 2012
88. G. Sermpinis, J. Laws, A. Karathanasopoulos, and C.L. Dunis Forecasting and trading the EUR/USD exchange rate with gene expression and psi sigma neural networks Expert Systems with Applications 39 2012 8865 8877
89. K. Shi, J. He, H.-T. Liu, N.-T. Zhang, and W.-T. Song Efficient text classification method based on improved term reduction and term weighting The Journal of China Universities of Posts and Telecommunications 18 Supplement 1 2011 131 135
90. G. Sidorov, F. Velasquez, E. Stamatatos, A. Gelbukh, and L. Chanona-Hernández Syntactic n-grams as machine learning features for natural language processing Expert Systems with Applications 2013
91. L.A. Smales Order imbalance, market returns and macroeconomic news: Evidence from the Australian interest rate futures market Research in International Business and Finance 26 2012 410 427
92. Soni, A.; van Eck, N. J.; & Kaymak, U. (2007). Prediction of stock price movements based on concept map information. In IEEE symposium on computational intelligence in multicriteria decision making (pp. 205-211). (Pubitemid 47431782)
93. S. Tan, Y. Wang, and G. Wu Adapting centroid classifier for document categorization Expert Systems with Applications 38 2011 10264 10273
94. Ş. TaşcI, and T. Güngör Comparison of text feature selection policies and using an adaptive framework Expert Systems with Applications 40 2013 4871 4886
95. P.C. Tetlock Giving content to investor sentiment: The role of media in the stock market The Journal of Finance 62 2007 1139 1168
96. P.C. Tetlock, M. Saar-Tsechansky, and S. Macskassy More than words: Quantifying language to measure firms' fundamentals The Journal of Finance 63 2008 1437 1467
97. D. Thammasiri, D. Delen, P. Meesad, and N. Kasap A critical assessment of imbalanced class distribution problem: The case of predicting freshmen student attrition Expert Systems with Applications 41 2014 321 330
98. J.F. Tomer What is behavioral economics? The Journal of Socio-Economics 36 2007 463 479
99. C.-F. Tsai, W. Eberle, and C.-Y. Chu Genetic algorithms in feature and instance selection Knowledge-Based Systems 39 2013 240 247
100. A. Urquhart, and R. Hudson Efficient or adaptive markets? Evidence from major stock markets using very long run historic data International Review of Financial Analysis 28 2013 130 142
101. A.K. Uysal, and S. Gunal A novel probabilistic feature selection method for text classification Knowledge-Based Systems 36 2012 226 235
102. A.K. Uysal, and S. Gunal The impact of preprocessing on text classification Information Processing and Management 50 2014 104 112
103. B. Vanstone, and G. Finnie Enhancing stockmarket trading performance with ANNs Expert Systems with Applications 37 2010 6602 6610
104. Vu, T. T.; Chang, S.; Ha, Q. T.; & Collier, N. (2012). An experiment in integrating sentiment features for tech stock prediction in twitter. In Proceedings of the workshop on information extraction and entity analytics on social media data (pp. 23-38). Mumbai, India: The COLING 2012 Organizing Committee.
105. G.-J. Wang, C. Xie, and F. Han Multi-scale approximate entropy analysis of foreign exchange markets efficiency Systems Engineering Procedia 3 2012 201 208
106. Weiss, S. M.; Indurkhya, N.; & Zhang, T. (2010). Fundamentals of Predictive Text Mining.
107. A. Werner, and Z.F. Myrray Is all that talk just noise? The information content of internet stock message boards Journal of Finance 10 2004 1259 1294
108. T.P. Wisniewski, and B. Lambe The role of media in the credit crunch: The case of the banking sector Journal of Economic Behavior and Organization 85 2013 163 175
109. Witten, I. H.; & Frank, E. (2005). Data mining: Practical machine learning tools and techniques, (2nd ed.) Morgan Kaufmann series in data management systems: Morgan Kaufmann Publishers Inc.
110. Q. Wu, and S. Tan A two-stage framework for cross-domain sentiment classification Expert Systems with Applications 38 2011 14269 14275
111. Wuthrich, B.; Cho, V.; Leung, S.; Permunetilleke, D.; Sankaran, K.; & Zhang, J. (1998). Daily stock market forecast from textual web data. In IEEE international conference on systems, man, and cybernetics, 1998 (Vol. 3, pp. 2720-2725, Vol. 2723).
112. L. Yang, C. Li, Q. Ding, and L. Li Combining lexical and semantic features for short text classification Procedia Computer Science 22 2013 78 86
113. L. Yin, Y. Ge, K. Xiao, X. Wang, and X. Quan Feature selection for high-dimensional imbalanced data Neurocomputing 105 2013 3 11
114. Y. Yu, W. Duan, and Q. Cao The impact of social and conventional media on firm equity value: A sentiment analysis approach Decision Support Systems 2013
115. H. Yu, G.V. Nartea, C. Gan, and L.J. Yao Predictive ability and profitability of simple technical trading rules: Recent evidence from Southeast Asian stock markets International Review of Economics and Finance 25 2013 356 371
116. L.-C. Yu, J.-L. Wu, P.-C. Chang, and H.-S. Chu Using a contextual entropy model to expand emotion words and their intensity for the sentiment classification of stock market news Knowledge-Based Systems 2013
117. Y. Zhai, A. Hsu, and S.K. Halgamuge Combining news and technical indicators in daily stock price trends prediction Proceedings of the 4th international symposium on neural networks: advances in neural networks, Part III 2007 Springer-Verlag Nanjing, China 1087 1096