Mining big data to extract patterns and predict real-life outcomes

Psychological Methods

Volumn 21, Issue 4, 2016, Pages 493-506

  Kosinski, Michal ^a   Wang, Yilun ^b   Lakkaraju, Himabindu ^b   Leskovec, Jure ^b  

^a STANFORD UNIVERSITY   (United States)

^b Stanford University   (United States)

Author keywords

Big data; Computational social science; Digital footprints; Personality; R

Indexed keywords

DECOMPOSITION; EXTRACT; HUMAN; HUMAN EXPERIMENT; LEARNING; MINING; MODEL; PERSONALITY; SOCIOLOGY; DATA MINING; INFORMATION PROCESSING; INTERNET; PROCEDURES; THEORETICAL MODEL;

DATA MINING; DATASETS AS TOPIC; HUMANS; INTERNET; MODELS, THEORETICAL;

EID: 85004009893     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/met0000105     Document Type: Article

References (53)

1. Abdi, H. (2003). Factor rotations in factor analyses. In M. Lewis-Beck, A. E. Bryman, & T. F. Liao (Eds.), The SAGE encyclopedia of social science research methods (pp. 792-795). Thousand Oaks, CA: SAGE.
2. Asuncion, A., Welling, M., Smyth, P., & Teh, Y. W. (2009). On smoothing and inference for topic models. In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (pp. 27-34). Arlington, VA: AUAI Press.
3. Baglama, J., & Reichel, L. (2012). irlba: Fast partial SVD by implicitly-restarted Lanczos bidiagonalization (R package Version 2.0.0) [Computer Software]. Retrieved from https://cran.r-project.org/package=irlba
4. Barchard, K. A., & Williams, J. (2008). Practical advice for conducting ethical online experiments and questionnaires for United States psychologists. Behavior Research Methods, 40, 1111-1128. http://dx.doi.org/10.3758/BRM.40.4.1111
5. Bates, D., & Maechler, M. (2015). Matrix: Sparse and dense matrix classes and methods (R package Version 1.2-6) [Computer Software]. Retrieved from https://cran.r-project.org/package= Matrix
6. Bengio, I. G. Y., & Courville, A. (2016). Deep learning. Manuscript in preparation. Retrieved from http://www.deeplearningbook.org/
7. Bishop, C. M. (2006). Pattern recognition and machine learning. New York, NY: Springer.
8. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3, 993-1022. http://dx.doi.org/10.1162/jmlr.2003.3.4-5.993
9. Bruckman, A. (2002). Studying the amateur artist: A perspective on disguising data collected in human subjects research on the Internet. Ethics and Information Technology, 4, 217-231. http://dx.doi.org/10.1023/A:1021316409277
10. Buchanan, E., Aycock, J., Dexter, S., Dittrich, D., & Hvizdak, E. (2011). Computer science security research and human subjects: Emerging considerations for research ethics boards. Journal of Empirical Research on Human Research Ethics, 6, 71-83. http://dx.doi.org/10.1525/jer.2011.6.2.71
11. Canty, A., & Ripley, B. D. (2016). boot: Bootstrap R (S-Plus) functions (R package Version 1.3-18) [Computer Software]. Retrieved from https://cran.r-project.org/package=boot
12. Chang, J., & Blei, D. M. (2010). Hierarchical relational models for document networks. The Annals of Applied Statistics, 4, 124-150. http://dx.doi.org/10.1214/09-AOAS309
13. Danescu-Niculescu-Mizil, C., West, R., Jurafsky, D., Leskovec, J., & Potts, C. (2013). No country for old members: User lifecycle and linguistic change in online communities. Proceedings of the 22nd International Conference on World Wide Web (pp. 307-318). New York, NY: ACM. http://dx.doi.org/10.1145/2488388.2488416
14. Eichstaedt, J. C., Schwartz, H. A., Kern, M. L., Park, G., Labarthe, D. R., Merchant, R. M., Seligman, M. E. P. (2015). Psychological language on Twitter predicts county-level heart disease mortality. Psychological Science, 26, 159-169. http://dx.doi.org/10.1177/0956797614557867
15. Field, A., Miles, J., & Field, Z. (2012). Discovering Statistics Using R. Los Angeles, CA: Sage. http://dx.doi.org/10.1111/insr.12011_21
16. Goldberg, L. R. (1992). The development of markers for the Big-five factor structure. Psychological Assessment, 4, 26-42. http://dx.doi.org/10.1037/1040-3590.4.1.26
17. Goldberg, L. R. (2006). Doing it all bass-ackwards: The development of hierarchical factor structures from the top down. Journal of Research in Personality, 40, 347-358. http://dx.doi.org/10.1016/j.jrp.2006.01.001
18. Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96. http://dx.doi.org/10.1016/j.jrp.2005.08.007
19. Golub, G. H., & Reinsch, C. (1970). Singular value decomposition and least squares solutions. Numerische Mathematik, 14, 403-420. http://dx.doi.org/10.1007/BF02163027
20. Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America, 101, 5228-5235. http://dx.doi.org/10.1073/pnas.0307752101
21. Grůn, B., & Hornik, K. (2011). topicmodels: An R package for fitting topic models. Journal of Statistical Software, 40, 1-30. http://dx.doi.org/10.18637/jss.v040.i13
22. Guennebaud, G., & Jacob, B. (2010). Eigen v3 [Computer Software]. Retrieved from http://eigen.tuxfamily.org
23. Hall, T., & Flynn, V. (2001). Ethical issues in software engineering research: A survey of current practice. Empirical Software Engineering, 6, 305-317. http://dx.doi.org/10.1023/A:1011922615502
24. Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33, 61-135. http://dx.doi.org/10.1017/S0140525X0999152X
25. Herring, S. (1996). Linguistic and critical analysis of computer-mediated communication: Some ethical and scholarly considerations. The Information Society, 12, 153-168. http://dx.doi.org/10.1080/911232343
26. James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013). An introduction to statistical learning: with applications in R. New York, NY: Springer.
27. Jones, E., Oliphant, T., & Peterson, P. (2001). SciPy: Open source scientific tools for Python [Computer software]. Retrieved from http://www.scipy.org/
28. Kosinski, M., Matz, S. C., Gosling, S. D., Popov, V., & Stillwell, D. (2015). Facebook as a research tool for the social sciences: Opportunities, challenges, ethical considerations, and practical guidelines. American Psychologist, 70, 543-556. http://dx.doi.org/10.1037/a0039210
29. Kosinski, M., Stillwell, D., & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings ofthe National Academy ofSciences ofthe United States of America, 110. 5802-5805. http://dx.doi.org/10.1073/pnas.1218772110
30. Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings ofthe National Academy ofSciences ofthe United States of America, 111, 8788-8790. http://dx.doi.org/10.1073/pnas.1320040111
31. Kuhn, M. (2015). caret: Classification and Regression Training (R package Version 6.0-68) [Computer Software]. Retrieved from https://cran.r-project.org/package= caret
32. Lambiotte, R., & Kosinski, M. (2014). Tracking the digital footprints of personality. Proceedings ofthe Institute ofElectrical and Electronics Engineers, 102, 1934-1939. http://dx.doi.org/10.1109/JPROC.2014.2359054
33. Larsen, R. M. (2005). PROPACK 2.1. Retrieved from http://sun.stanford.edu/rmunk/PROPACK/
34. Lazer, D., Pentland, A., Adamic, L., Aral, S., Barabasi, A. L., Brewer, D., Alstyne, M. V. (2009). Life in the network: The coming age of computational social science. Science, 323, 721-723. http://dx.doi.org/10.1126/science.1167742
35. Leskovec, J., & Krevl, A. (2014). SNAP Datasets: Stanford Large Network Dataset Collection. Retrieved from http://snap.stanford.edu/data
36. Madsen, R. E., Hansen, L. K., & Winther, O. (2004). Singular value decomposition and principal component analysis. Retrieved from http://www2.imm.dtu.dk/pubdb/p.php?4000
37. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, aac4716. http://dx.doi.org/10.1126/science.aac4716
38. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal ofMachine Learning Research, 12, 2825-2830.
39. Phan, X.-H., & Nguyen, C.-T. (2007). A C/C+ + implementation of latent Dirichlet allocation. Retrieved from http://gibbslda.sourceforge.net/
40. R. Core Team. (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/
41. Schein, A. I., Popescul, A., Ungar, L. H., & Pennock, D. M. (2002). Methods and metrics for cold-start recommendations. In Proceedings of the 25th Annual International Conference on Research and Development in Information Retrieval (pp. 253-260). New York, NY: ACM. http://dx.doi.org/10.1145/564418.564421
42. Schultze, U., & Mason, R. O. (2012). Studying cyborgs: Re-examining Internet studies as human subjects research. Journal of Information Technology, 27, 301-312. http://dx.doi.org/10.1057/jit.2012.30
43. Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Ungar, L. H. (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLOS ONE, 8, e73791. http://dx.doi.org/10.1371/journal.pone.0073791
44. Sing, T., Sander, O., Beerenwinkel, N., & Lengauer, T. (2005). ROCR: Visualizing classifier performance in R. Bioinformatics, 21, 3940-3941. http://dx.doi.org/10.1093/bioinformatics/bti623
45. Solberg, L. (2010). Data mining on Facebook: A free space for researchers or an IRB nightmare? Journal of Law, Technology & Policy, 2, 311-433.
46. Steyvers, M. (2011). Matlab topic modeling toolbox 1.4. Retrieved from http://psiexp.ss.uci.edu/research/programs_data/toolbox.htm
47. Strang, G. (2016). Introduction to linear algebra, fifth edition. Wellesley, MA: Wellesley-Cambridge Press.
48. Tibshirani, R. (2011). Regression shrinkage and selection via the lasso: A retrospective. Journal ofthe Royal Statistical Society: Series B (Statistical Methodology), 73, 273-282. http://dx.doi.org/10.1111/j.1467-9868.2011.00771.x
49. Ugander, J., Karrer, B., Backstrom, L., & Marlow, C. (2011). The anatomy of the Facebook social graph. Computing Research Repository, 1-17. Retrieved from http://arxiv.org/abs/1111.4503
50. Wall, M. E., Rechtsteiner, A., & Rocha, L. M. (2003). Singular value decomposition and principal component analysis. In D. Berrar, W. Dubitzky, & M. Granzow (Eds.), A practical approach to microarray data analysis (pp. 91-109). Norwell, MA: Kluwer.
51. Wilson, R. E., Gosling, S. D., & Graham, L. T. (2012). A review of Facebook research in the social sciences. Perspectives on Psychological Science, 7, 203-220.
52. Youyou, W., Kosinski, M., & Stillwell, D. J. (2015). Computer-based personality judgements are more accurate than those made by humans. Proceedings ofthe National Academy ofSciences ofthe United States of America, 112, 1036-1040.
53. Zhang, L., Marron, J., Shen, H., & Zhu, Z. (2007). Singular value decomposition and its visualization. Journal ofComputational and Graphical Statistics, 16, 833-854.