Sparse factor analysis for learning and content analytics

Journal of Machine Learning Research

Volumn 15, Issue , 2014, Pages 1959-2008

  Lan, Andrew S ^a   Waters, Andrew E ^a   Studer, Christoph ^b   Baraniuk, Richard G ^a  

^a Rice University   (United States)

^b School of Operations Research and Industrial Engineering   (United States)

Author keywords

Bayesian latent factor analysis; Factor analysis; Personalized learning; Sparse logistic regression; Sparse probit regression

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; FACTOR ANALYSIS; MULTIVARIANT ANALYSIS;

BAYESIAN SOLUTION; DICTIONARY LEARNING; ESTIMATION PROBLEM; LATENT FACTOR ANALYSIS; LOGISTIC REGRESSIONS; MODEL AND ALGORITHMS; PERSONALIZED LEARNING; SPARSE PROBIT REGRESSION;

LEARNING SYSTEMS;

EID: 84904196866     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (85)

1. M. Aharon, M. Elad, and A. M. Bruckstein. K-SVD and its non-negative variant for dictionary design. In Proc. SPIE Conf. on Wavelets, volume 5914, pages 327-339, July 2005.
2. M. Aharon, M. Elad, and A. M. Bruckstein. K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54 (11):4311-4322, Dec. 2006. (Pubitemid 44637761)
3. Amazon Mechanical Turk, Sep. 2012. URL https://www.mturk.com/mturk/ welcome.
4. Y. Bachrach, T. P. Minka, J. Guiver, and T. Graepel. How to grade a test without knowing the answers - a Bayesian graphical model for adaptive crowdsourcing and aptitude testing. In Proc. 29th Intl. Conf. on Machine Learning, pages 1183-1190, June 2012.
5. F. B. Baker and S. H. Kim. Item Response Theory: Parameter Estimation Techniques. Marcel Dekker Inc., 2nd edition, 2004.
6. R. Baker and K. Yacef. The state of educational data mining in 2009: A review and future visions. Journal of Educational Data Mining, 1(1):3-17, Oct. 2009.
7. T. Barnes. The Q-matrix method: Mining student response data for knowledge. In Proc. AAAI Workshop Educational Data Mining, July 2005.
8. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Science, 2(1):183-202, Mar. 2009.
9. B. Beheshti, M. Desmarais, and R. Naceur. Methods to find the number of latent skills. In Proc. 5th Intl. Conf. on Educational Data Mining, pages 81-86, June 2012.
10. Y. Bergner, S. Droschler, G. Kortemeyer, S. Rayyan, D. Seaton, and D. Pritchard. Modelbased collaborative filtering analysis of student response data: Machine-learning item response theory. In Proc. 5th Intl. Conf. on Educational Data Mining, pages 95-102, June 2012.
11. J. Bolte, A. Daniilidis, and A. Lewis. The Łojasiewicz inequality for nonsmooth subanalytic functions with applications to subgradient dynamical systems. SIAM Journal on Optimization, 17(4):1205-1223, Dec. 2006. (Pubitemid 350121960)
12. P. T. Boufounos and R. G. Baraniuk. 1-bit compressive sensing. In Proc. Conf. on Information Science and Systems (CISS), Mar. 2008.
13. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
14. A. M. Bruckstein, M. Elad, and M. Zibulevsky. On the uniqueness of nonnegative sparse solutions to underdetermined systems of equations. IEEE Transactions on Information Theory, 54(11):4813-4820, Nov. 2008.
15. P. Brusilovsky and C. Peylo. Adaptive and intelligent web-based educational systems. Intl. Journal of Artificial Intelligence in Education, 13(2-4):159-172, Apr. 2003.
16. C. J. Butz, S. Hua, and R. B. Maguire. A web-based Bayesian intelligent tutoring system for computer programming. Web Intelligence and Agent Systems, 4(1):77-97, Nov. 2006.
17. H. Chang and Z. Ying. Nonlinear sequential designs for logistic item response theory models with applications to computerized adaptive tests. Annals of Statistics, 37(3):1466-1488, June 2009.
18. S. S. Chen, D. L. Donoho, and M. A. Saunders. Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20(1):33-61, Mar. 1998. (Pubitemid 128689501)
19. J. T. Chu. On bounds for the normal integral. IEEE Transactions on Signal Processing, 42 (1/2):263-265, June 1955.
20. M. Desmarais. Conditions for effectively deriving a Q-matrix from data with non-negative matrix factorization. In Proc. 4th Intl. Conf. on Educational Data Mining, pages 41-50, July 2011.
21. J. A. Dijksman and S. Khan. Khan Academy: the world's free virtual school. In APS Meeting Abstracts, page 14006, Mar. 2011.
22. ELEC 301, Rice University. Introduction to signals and systems, May 2011. URL http://dsp.rice.edu/courses/elec301.
23. A. Fischer. On sums of subanalytic functions. Preprint, 2008.
24. E. Fokoue. Stochastic determination of the intrinsic structure in Bayesian factor analysis. Technical report, Statistical and Applied Mathematical Sciences Institute, June 2004.
25. K. Fronczyk, A. E. Waters, M. Guindani, R. G. Baraniuk, and M. Vannucci. A Bayesian infinite factor model for learning and content analytics. Computational Statistics and Data Analysis, June 2013, submitted.
26. J. P. González-Brenes and J. Mostow. Dynamic cognitive tracing: Towards unified discovery of student and cognitive models. In Proc. 5th Intl. Conf. on Educational Data Mining, pages 49-56, June 2012.
27. I. Goodfellow, A. Courville, and Y. Bengio. Large-scale feature learning with spike-and-slab sparse coding. In Proc. 29th Intl. Conf. on Machine Learning, pages 1439-1446, July 2012.
28. A. Guisan, T. C. Edwards Jr, and T. Hastie. Generalized linear and generalized additive models in studies of species distributions: setting the scene. Ecological Modelling, 157 (2-3):89-100, Nov. 2002. (Pubitemid 35255408)
29. P. R. Hahn, C. M. Carvalho, and J. G. Scott. A sparse factor-analytic probit model for congressional voting patterns. Journal of the Royal Statistical Society, 61(4):619-635, Aug. 2012.
30. H. H. Harman. Modern Factor Analysis. The University of Chicago Press, 1976.
31. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, 2010.
32. J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1):5-53, Jan. 2004.
33. R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, 1991.
34. D. Hu. How Khan academy is using machine learning to assess student mastery. online: http://david-hu.com/, Nov. 2011.
35. H. Ishwaran and J. S. Rao. Spike and slab variable selection: frequentist and Bayesian strategies. Annals of Statistics, 33(2):730-773, Apr. 2005. (Pubitemid 41053721)
36. L. Jacques, J. N. Laska, P. T. Boufounos, and R. G. Baraniuk. Robust 1-bit compressive sensing via binary stable embeddings of sparse vectors. IEEE Transaction on Info Theory, 59(4), Apr. 2013.
37. Knewton. Knewton adaptive learning: Building the world's most powerful recommendation engine for education. online: http://www.knewton.com/adaptive- learning-white-paper/, June 2012.
38. K. R. Koedinger, J. R. Anderson, W. H. Hadley, and M. A. Mark. Intelligent tutoring goes to school in the big city. Intl. Journal of Artificial Intelligence in Education, 8(1):30-43, 1997.
39. 1-regularized logistic regression. Journal of Machine Learning Research, 8:1519-1555, 2007. (Pubitemid 47210273)
40. Y. Koren and J. Sill. OrdRec: an ordinal model for predicting personalized item rating distributions. In Proc. of the 5th ACM Conf. on Recommender Systems, pages 117-124, Oct. 2011.
41. Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, Aug. 2009.
42. S. G. Krantz and H. R. Parks. A Primer of Real Analytic Functions. Birkhauser, 2002.
43. G. A. Krudysz and J. H. McClellan. Collaborative system for signal processing education. In IEEE Intl. Conf. on Acoustics, Speech and Signal Processing, pages 2904-2907, May 2011.
44. G. A. Krudysz, J. S. Li, and J. H. McClellan. Web-based Bayesian tutoring system. In 12th Digital Signal Processing Workshop - 4th Signal Processing Education Workshop, pages 129-134, Sep. 2006. (Pubitemid 351245940)
45. A. S. Lan, C. Studer, A. E. Waters, and R. G. Baraniuk. Tag-aware ordinal sparse factor analysis for learning and content analytics. In Proc. 6th Intl. Conf. on Educational Data Mining, pages 90-97, July 2013a.
46. A. S. Lan, C. Studer, A. E. Waters, and R. G. Baraniuk. Joint topic modeling and factor analysis of textual information and graded response data. In Proc. 6th Intl. Conf. on Educational Data Mining, pages 324-325, July 2013b.
47. 1 regularized logistic regression. In Proc. National Conf. on Artificial Intelligence, volume 21, pages 401-408, 2006.
48. S. Lee, J. Z. Huang, and J. Hu. Sparse logistic principal components analysis for binary data. Annals of Applied Statistics, 4(3):1579-1601, Sept. 2010.
49. N. Li, W. W. Cohen, and K. R. Koedinger. A machine learning approach for automatic student model discovery. In Proc. 4th Intl. Conf. on Educational Data Mining, pages 31-40, July 2011.
50. J. M. Linacre. Understanding Rasch measurement: Estimation methods for Rasch measures. Journal of Outcome Measurement, 3(4):382-405, 1999.
51. G. Linden, B. Smith, and J. York. Amazon.com recommendations: Item-to-item collaborative filtering. IEEE Internet Computing, 7(1):76-80, Jan. 2003.
52. W. J. V. D. Linden and editors Glas, C. A. W. Computerized Adaptive Testing: Theory and Practice. Kluwer Academic Publishers, 2000.
53. F. M. Lord. Applications of Item Response Theory to Practical Testing Problems. Erlbaum Associates, 1980.
54. J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11:19-60, 2010.
55. R. P. McDonald. A basis for multidimensional item response theory. Applied Psychological Measurement, 247(2):99-114, June 2000.
56. J. Meng, J. Zhang, Y. Qi, Y. Chen, and Y. Huang. Uncovering transcriptional regulatory networks by sparse Bayesian factor model. EURASIP Journal on Advances in Signal Processing, 2010(3):1-18, Mar. 2010.
57. T. P. Minka. A comparison of numerical optimizers for logistic regression. Technical report, 2003. http://citeseerx.ist.psu.edu/viewdoc/ download?doi=10.1.1.85.7017 &rep=rep1&type=pdf.
58. 1 approaches for sparse unsupervised learning. In Proc. 29th Intl. Conf. on Machine Learning, pages 751-758, July 2012.
59. R. C. Murray, K. Van Lehn, and J. Mostow. Looking ahead to select tutorial actions: A decision-theoretic approach. Intl. Journal of Artificial Intelligence in Education, 14(3-4): 235-278, Dec. 2004.
60. Y. Nesterov. Gradient methods for minimizing composite objective function. Technical report, Université catholique de Louvain, Sep. 2007.
61. M. R. Norvick. The axioms and principal results of classical test theory. Journal of Mathematical Psychology, 3(1):1-18, Feb. 1966.
62. F. W. J. Olver, editor. NIST Handbook of Mathematical Functions. Cambridge University Press, 2010.
63. Z. A. Pardos and N. T. Heffernan. Modeling individualization in a Bayesian networks implementation of knowledge tracing. In User Modeling, Adaptation, and Personalization, volume 6075, pages 255-266. Springer, June 2010.
64. M. Y. Park and T. Hastie. Penalized logistic regression for detecting gene interactions. Biostatistics, 9(1):30-50, Jan. 2008. (Pubitemid 350274580)
65. Y. Plan and R. Vershynin. Robust 1-bit compressed sensing and sparse logistic regression: A convex programming approach. 59(1), Feb. 2012, submitted.
66. I. Pournara and L. Wernisch. Factor analysis for gene regulatory networks and transcription factor activity profiles. BMC Bioinformatics, 8(1):61, Feb. 2007.
67. J. Psotka, L. D. Massey, and editors Mutter, S. A. Intelligent Tutoring Systems: Lessons Learned. Lawrence Erlbaum Associates, 1988.
68. A. N. Rafferty, E. Brunskill, T. L. Griffiths, and P. Shafto. Faster teaching by POMDP planning. In Proc. 15th Intl. Conf. on Artificial Intelligence in Education, pages 280-287, June 2011.
69. G. Rasch. Probabilistic Models for Some Intelligence and Attainment Tests. MESA Press, 1993.
70. C. E. Rasmussen and C. K. I. Williams. Gaussian Process for Machine Learning. MIT Press, 2006.
71. M. D. Reckase. Multidimensional Item Response Theory. Springer Publishing Company, Incorporated, 1st edition, 2009.
72. C. Romero and S. Ventura. Educational data mining: A survey from 1995 to 2005. Expert Systems with Applications, 33(1):135-146, July 2007. (Pubitemid 44959924)
73. M. N. Schmidt, O. Winther, and L. K. Hansen. Bayesian non-negative matrix factorization. In Independent Component Analysis and Signal Separation, volume 5441, pages 540-547, Mar. 2009.
74. J. C. Stamper, T. Barnes, and M. Croy. Extracting student models for intelligent tutoring systems. In Proc. National Conf. on Artificial Intelligence, volume 22, pages 113-147, July 2007. STEMscopes. STEMscopes science education, Sep. 2012. URL http://stemscopes.com/.
75. C. Studer and R. G. Baraniuk. Dictionary learning from sparsely corrupted or compressed signals. In IEEE Intl. Conf. on Acoustics, Speech and Signal Processing, pages 3341-3344, Mar. 2012.
76. N. Thai-Nghe, L. Drumond, T. Horvath, and L. Schmidt-Thieme. Multi-relational factorization models for predicting student performance. KDD Workshop on Knowledge Discovery in Educational Data (KDDinED), Aug. 2011a.
77. N. Thai-Nghe, T. Horvath, and L. Schmidt-Thieme. Factorization models for forecasting student performance. In Proc. 4th Intl. Conf. on Educational Data Mining, pages 11-20, July 2011b.
78. N. A. Thompson. Item selection in computerized classification testing. Educational and Psychological Measurement, 69(5):778-793, Oct. 2009.
79. M. E. Tipping. Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1:211-244, 2001. (Pubitemid 33687203)
80. K. Van Lehn, C. Lynch, K. Schulze, J. A. Shapiro, R. Shelby, L. Taylor, D. Treacy, A. Weinstein, and M. Wintersgill. The Andes physics tutoring system: Lessons learned. Intl. Journal of Artificial Intelligence in Education, 15(3):147-204, Sep. 2005.
81. D. Vats, C. Studer, A. S. Lan, L. Carin, and R. G. Baraniuk. Test size reduction for concept estimation. In Proc. 6th Intl. Conf. on Educational Data Mining, pages 292-295, July 2013.
82. M. West. Bayesian factor regression models in the "large p, small n" paradigm. Bayesian Statistics, 7:723-732, Sep. 2003.
83. B. P. Woolf. Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning. Morgan Kaufman Publishers, 2008.
84. Y. Xu and W. Yin. A block coordinate descent method for multi-convex optimization with applications to nonnegative tensor factorization and completion. Technical report, Rice University CAAM, Sep. 2012.
85. L. Yao. BMIRT: Bayesian Multivariate Item Response Theory. CTb/McGraw-Hill, 2003.