Making business predictions by combining human and machine intelligence in prediction markets

International Conference on Information Systems 2011, ICIS 2011

Volumn 4, Issue , 2011, Pages 2844-2859

  Nagar, Yiftach ^a   Malone, Thomas W ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Artificial intelligence; Business intelligence; Collective intelligence; Decision making makers; Decision support systems (DSS); Enterprise 2.0; Forecasting; Forecasts and implications; Intelligent systems; Prediction markets; Predictive modeling; Track 10. human computer interaction; Track 16. knowledge management and business intelligence; Track 18. e business

Indexed keywords

BUSINESS-INTELLIGENCE; COLLECTIVE-INTELLIGENCE; DECISION SUPPORT SYSTEM (DSS); EBUSINESS; ENTERPRISE 2.0; PREDICTION MARKETS; PREDICTIVE MODELING;

ARTIFICIAL INTELLIGENCE; COMMERCE; DECISION SUPPORT SYSTEMS; INFORMATION SYSTEMS; INTELLIGENT SYSTEMS; KNOWLEDGE MANAGEMENT; MANAGEMENT SCIENCE;

FORECASTING;

EID: 84884626454     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (76)

1. Armstrong, J.S. 1989. "Combining Forecasts: The End of the Beginning or the Beginning of the End?," International Journal of Forecasting (5), pp. 585-588.
2. Armstrong JS, 2001a. "Combining Forecasts," in Principles of Forecasting: A Handbook for Researchers and Practitioners, J.S. Armstrong (ed.). Kluwer Academic Publishers.
3. Armstrong, J.S. 2001b. "Judgmental Bootstrapping: Inferring Experts' Rules for Forecasting," in Principles of Forecasting: A Handbook for Researchers and Practitioners, J.S. Armstrong (ed.). Norwell, MA: Kluwer Academic Publishers.
4. Berg, J.E., Forsythe, R., Nelson, F., and Rietz, T.A. 2001. "Results from a Dozen Years of Election Futures Markets Research," Handbook of Experimental Economic Results), pp. 486-515.
5. Berg, J.E, Nelson F.D, and Rietz, T.A. 2008. "Prediction Market Accuracy in the Long Run," InternationalJournal ofForecasting (24:2), pp. 283-298.
6. Blattberg, R.C., and Hoch, S.J. 1990. "Database Models and Managerial Intuition: 50% Model+ 50% Manager," Management Science (36:8), pp. 887-899.
7. Bohorquez, J.C., Gourley, S., Dixon, A.R., Spagat M, and Johnson N.F,. 2009. "Common Ecology Quantifies Human Insurgency," Nature (462:7275), pp. 911-914.
8. Brier, G.W. 1950. "Verification of Forecasts Expressed in Terms of Probability," Monthly Weather Review (78:1), pp. 1-3.
9. Brown, R. 1986. Social Psychology, (2nd ed.). New York, NY: Free Press.
10. Bunn, D., and Wright, G. 1991. "Interaction of Judgemental and Statistical Forecasting Methods: Issues and Analysis," Management Science (37:5), May, pp. 501-518.
11. Chen K.-Y, and Plott C.R, 2002. "Information Aggregation Mechanisms: Concept, Design and Implementation for a Sales Forecasting Problem," California Institute of Technology, Division of the Humanities and Social Sciences, Working Paper 1131), March.
12. Chen, Y., Chu, C.-H., Mullen, T., and Pennock, DM, 2005. "Information Markets Vs. Opinion Pools: An Empirical Comparison," in: Proceedings of the 6th ACM conference on Electronic commerce Vancouver, BC, Canada: ACM, pp. 58-67.
13. Chen, Y., and Vaughan, J.W. 2010. "A New Understanding of Prediction Markets Via No-Regret Learning," 11th ACM conference on Electronic Commerce (EC '10), Cambridge, MA: ACM, pp. 189-198.
14. Clemen, R.T. 1989. "Combining Forecasts: A Review and Annotated Bibliography," InternationalJournal of Forecasting (5:4), pp. 559-583.
15. Cowgill, B., Wolfers, J., and Zitzewitz, E. 2008. "Using Prediction Markets to Track Information Flows: Evidence from Google," Dartmouth College)
16. Dawes, R.M., Faust D, and Meehl P.E, 1989. "Clinical Versus Actuarial Judgment," Science (243:4899), pp. 1668-1674.
17. Dawes R.M, and Kagan J, 1988. Rational Choice in an Uncertain World. Harcourt Brace Jovanovich San Diego.
18. DeLong, E.R, DeLong D.M, and Clarke-Pearson DL, 1988. "Comparing the Areas under Two or More Correlated Receiver Operating Characteristic Curves: A Nonparametric Approach," Biometrics (44:3),pp. 837-845.
19. Duin R, and Tax D, 2000. "Experiments with Classifier Combining Rules," First International Workshop on Multiple Classifier Systems (MCS 2000), Cagliari, Italy: Springer, pp. 16-29.
20. Einhorn, H.J. 1972. "Expert Measurement and Mechanical Combination," Organizational Behavior and Human Performance (7:1), pp. 86-106.
21. Gneiting, T., and Raftery, A.E. 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association (102:477), pp. 359-378.
22. Goel, S., Reeves, D.M., Watts, D.J., and Pennock, D.M. 2010. "Prediction without Markets," 11th ACM conference on Electronic commerce, Cambridge, MA: ACM, pp. 357-366.
23. Goldberg LR, 1970. "Man Versus Model of Man: A Rationale, Plus Some Evidence, for a Method of Improving on Clinical Inferences," Psychological Bulletin (73:6), pp. 422-432.
24. Good, I.J. 1952. "Rational Decisions," Journal of the Royal Statistical Society. Series B (Methodological) (14:1), pp. 107-114.
25. Grove, W.M., Zald, D.H., Lebow, B.S., Snitz BE, and Nelson C, 2000. "Clinical Versus Mechanical Prediction: A Meta Analysis," Psychological Assessment (12:1), pp. 19-30.
26. Hawkins J, and Blakeslee S, 2005. On Intelligence. Owl Books.
27. Hayek FA, 1945. "The Use of Knowledge in Society," The American Economic Review (35:4), pp. 519-530.
28. Hibon, M., and Evgeniou, T. 2005. "To Combine or Not to Combine: Selecting among Forecasts and Their Combinations," International Journal of Forecasting (21:1), pp. 15-24.
29. Ho, T.K., Hull JJ, and Srihari SN, 1994. "Decision Combination in Multiple Classifier Systems," IEEE Transactions on Pattern Analysis and Machine Intelligence (16:1), pp. 66-75.
30. Hopman, J. 2007. "Using Forecasting Markets to Manage Demand Risk," Intel Technology Journal (11), pp. 127-136.
31. Janis IL, 1972. Victims of Groupthink: A Psychological Study of Foreign-Policy Decisions and Fiascoes. Houghton Mifflin Boston.
32. Janis I.L, and Mann L, 1977. Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment. The Free Press New York.
33. Jolliffe IT, and Stephenson DB, 2003. Forecast Verification: A Practitioner's Guide in Atmospheric Science. Wiley.
34. Kahneman D, Slovic P, and Tversky A, 1982. Judgment under Uncertainty: Heuristics and Biases Cambridge University Press.
35. Kahneman, D., and Tversky, A. 1973. "On the Psychology of Prediction," Psychological review (80:4), pp. 237-251.
36. Kittler, J., Hatef, M., Duin, R.P.W., and Matas, J. 1998. "On Combining Classifiers," IEEE Transactions on Pattern Analysis and Machine Intelligence (20:3), pp. 226-239.
37. Kleinmuntz, B. 1990. "Why We Still Use Our Heads Instead of Formulas: Toward an Integrative Approach," Psychological Bulletin (107:3), pp. 296-310.
38. Kuncheva, L.I. 2002. "A Theoretical Study on Six Classifier Fusion Strategies," IEEE Transactions on pattern analysis and machine intelligence (24:2), February, pp. 281-286.
39. Lam, L. 2000. "Classifier Combinations: Implementations and Theoretical Issues," First International Workshop on Multiple Classifier Systems (MCS 2000), Cagliari, Italy: Springer, pp. 77-86.
40. Larrick, R.P., and Soil, J.B. 2006. "Intuitions About Combining Opinions: Misappreciation of the Averaging Principle," Management Science (52:1), p. 111.
41. Lichtenstein, S., Baruch, F., and Phillips, L.D. 1982. "Calibration of Probabilities: The State of the Art to 1980," in Judgment under Uncertainty: Heuristics and Biases, D. Kahneman, P. Slovic and A. Tversky (eds.). Cambridge University Press.
42. Littell RC, Milliken G.A, Stroup WW, and Wolfinger RD, 2006. Sas for Mixed Models, (2nd ed.). SAS Publishing.
43. Makridakis, S. 1989. "Why Combining Works?," International Journal of Forecasting (5:4), pp. 601-603.
44. Makridakis S, and Winkler RL, 1983. "Averages of Forecasts: Some Empirical Results," Management Science), pp. 987-996.
45. Malone, T.W. 2004. "Bringing the Market Inside," Harvard Business Review (82:4), pp. 106-114.
46. Mannes, A., Michael, M., Pate, A., Sliva, A., Subrahmanian, V.S., and Wilkenfeld, J. 2008. "Stochastic Opponent Modeling Agents: A Case Study with Hezbollah," in Social Computing, Behavioral Modeling, and Prediction, H. Liu, J.J. Salerno and M.J. Young (eds.). pp. 37-45.
47. Matteo, G., Stefano, R., and Iuris, C. 2004. "Beyond Data Warehousing: What's Next in Business Intelligence?," in: Proceedings of the 7th ACM international workshop on Data warehousing and OLAP. Washington, DC, USA: ACM.
48. Meehl P.E. 1954. Clinical Vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence. Minneapolis: University of Minnesota Press.
49. Murphy A.H. 1973. "A New Vector Partition of the Probability Score," Journal of Applied Meteorology (12:4), pp. 595-600.
50. Murphy, A.H., and Winkler, R. 1987. "A General Framework for Forecast Verification," Monthly Weather Review (115:7), pp. 1330-1338.
51. Murphy, A.H., and Winkler, R.L. 1977. "Reliability of Subjective Probability Forecasts of Precipitation and Temperature," Applied Statistics, pp. 41-47.
52. Negash, S. 2004. "Business Intelligence," The Communications of the Association for Information Systems (13:1), p. 54.
53. Ortner, G. 1997. "Forecasting Markets an Industrial Application." University of Technology Vienna.
54. Perols J., Chari K, and Agrawal M, 2009. "Information Market-Based Decision Fusion," Management Science (55:5), pp. 827-842.
55. Rabin, M. 1996. "Psychology and Economics," Journal of Economic Literature (36:1), March, pp. 11-46.
56. Schoonjans F, Zalata A, Depuydt, CE, and Comhaire FH, 1995. "Medcalc: A New Computer Program for Medical Statistics," Computer Methods and Programs in Biomedicine (48:3), pp. 257-262.
57. Sharpe, W.F. 1966. "Mutual Fund Performance," Journal of business (39:1), pp. 119-138.
58. Sharpe, W.F. 1994. "The Sharpe Ratio," Journal of portfolio management Fall, pp. 49-58.
59. Spann, M., and Skiera, B. 2009. "Sports Forecasting: A Comparison of the Forecast Accuracy of Prediction Markets, Betting Odds and Tipsters," Journal of Forecasting (28:1), pp. 55-72.
60. Stewart, T.R. 2001. "Improving Reliability of Judgmental Forecasts," in Principles of Forecasting: A Handbook for Researchers and Practitioners, J.S. Armstrong (ed.) Kluwer Academic Publishers, pp.81-106.
61. Suen, C.Y., and Lam, L. 2000. "Multiple Classifier Combination Methodologies for Different Output Levels," First International Workshop on Multiple Classifier Systems (MCS 2000), Cagliari, Italy: Springer, pp. 52-66.
62. Sunstein, C.R. 2005. "Group Judgments: Statistical Means, Deliberation, and Information Markets," New York University Law Review (80), p. 962.
63. Sunstein, C.R. 2006. Infotopia: How Many Minds Produce Knowledge. Oxford University Press, USA.
64. Surowiecki, J. 2004. The Wisdom of Crowds: Why the Many Are Smarter Than the Few and How Collective Wisdom Shapes Business, Economies, Societies and Nations. New York: Doubleday.
65. Swets JA, 1973. "The Relative Operating Characteristic in Psychology," Science (182:4116), pp. 990-1000.
66. Swets, JA. 1988. "Measuring the Accuracy of Diagnostic Systems," Science (240:4857), pp. 1285-1293.
67. Swets J.A, and Pickett RM, 1982. Evaluation of Diagnostic Systems: Methods from Signal Detection Theory. Academic Press.
68. Tarn K.Y, and Kiang MY, 1992. "Managerial Applications of Neural Networks: The Case of Bank Failure Predictions," Management Science (38:7), pp. 926-947.
69. Tetlock, P. 2005. Expert Political Judgment: How Good Is It? How Can We Know? Princeton UniversityPress.
70. Tziralis G, and Tatsiopoulos I, 2007. "Prediction Markets: An Extended Literature Review," Journal of Prediction Markets (1:1), pp. 75-91.
71. Winkler, R.L. 1969. "Scoring Rules and the Evaluation of Probability Assessors," Journal of the American Statistical Association (64:327), September, pp. 1073-1078.
72. Winkler RL, 1989. "Combining Forecasts: A Philosophical Basis and Some Current Issues," International Journal of Forecasting (5:4), pp. 605-609.
73. Wolfers, J., and Zitzewitz, E. 2004. "Prediction Markets," The Journal of Economic Perspectives (18:2), pp. 107-126.
74. Wolfers, J., and Zitzewitz, E. 2006. "Interpreting Prediction Market Prices as Probabilities," CEPR Discussion Paper No. 5676), May 2006.
75. Zuboff S, 1985. "Automate/Informate: The Two Faces of Intelligent Technology," Organizational Dynamics (14:2), pp. 5-18.
76. Zweig, M.H., and Campbell, G 1993. "Receiver-Operating Characteristic (Roc) Plots: A Fundamental Evaluation Tool in Clinical Medicine," Clinical Chemistry (39:4), pp. 561-577.