Online PLSA: Batch updating techniques including out-of-vocabulary words

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 11, 2014, Pages 1953-1966

  Bassiou, Nikoletta K ^a   Kotropoulos, Constantine L ^a  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

Author keywords

Document clustering; document modeling; information retrieval; out of vocabulary (OOV) words; PLSA updating; probabilistic latent semantic analysis (PLSA); unsupervised learning.

Indexed keywords

UNSUPERVISED LEARNING;

DOCUMENT CLUSTERING; DOCUMENT MODEL; OUTOF-VOCABULARY WORDS (OOV); PLSA UPDATING; PROBABILISTIC LATENT SEMANTIC ANALYSIS;

INFORMATION RETRIEVAL;

EID: 84908090239     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2299806     Document Type: Article

References (45)

1. T. Hofmann, "Probabilistic latent semantic analysis," in Proc. Uncertainty Artif. Intell., 1999, pp. 286-296.
2. M. W. Berry, S. T. Dumais, and G. W. O'Brien, "Using linear algebra for intelligent information retrieval," Dept. Comput. Sci., Univ. Tennessee, Knoxville, TN, USA, Tech. Rep. UT-CS-94-270, 1994.
3. D. M. Blei, A. Y. Ng, and M. I. Jordan, "Latent Dirichlet allocation," J. Mach. Learn. Res., vol. 3, no. 5, pp. 993-1022, 2003.
4. M. Girolami and A. Kaban, "On an equivalence between PLSI and LDA," in Proc. 26th ACM SIGIR Conf. Res. Develop. Inf. Retr., Toronto, ON, Canada, 2003, pp. 433-434.
5. J. Blitzer, A. Globerson, and F. Pereira, "Distributed latent variable models of lexical co-occurrences," in Proc. 10th Int. Workshop Artif. Intell. Statist., 2005, pp. 1-8.
6. S. Wang, D. Schuurmans, F. Peng, and Y. Zhao, "Combining statistical language models via the latent maximum entropy principle," Mach. Learn., vol. 60, nos. 1-3, pp. 229-250, 2005.
7. P. Smaragdis, B. Raj, and M. Shashanka, "Missing data imputation for time-frequency representations of audio signals," J. Signal Process. Syst., vol. 65, no. 3, pp. 361-370, 2011.
8. P. Li, J. Cheng, Z. Li, and H. Lu, "Correlated PLSA for image clustering," in Advances in Multimedia Modeling, K.-T. Lee, W.-H. Tsai, H.-Y. Liao, T. Chen, J.-W. Hsieh, and C.-C. Tseng, Eds. Berlin, Germany: Springer-Verlag, 2011, pp. 307-316.
9. L. Cai and T. Hofmann, "Text categorization by boosting automatically extracted concepts," in Proc. 26th ACM SIGIR Conf. Res. Develop. Inf. Retr., Toronto, ON, Canada, 2003, pp. 182-189.
10. T. Hofmann, "Learning the similarity of documents: An informationgeometric approach to document retrieval and categorization," in Advances in Neural Information Processing Systems 12. Cambridge, MA, USA: MIT Press, 2000, pp. 914-920.
11. A. Vinokourov and M. Girolami, "A probabilistic framework for the hierarchic organisation and classification of document collections," J. Intell. Inf. Syst., vol. 18, nos. 2-3, pp. 153-172, 2002.
12. E. Gaussier, C. Goutte, K. Popat, and F. Chen, "A hierarchical model for clustering and categorising documents," in Proc. 24th BCS-IRSG ECIR Res., Glasgow, U.K., Mar. 2002, pp. 292-247.
13. Z. Zeng, S. Zhang, H. Li, W. Liang, and H. Zheng, "A novel approach to musical genre classification using probabilistic latent semantic analysis model," in Proc. IEEE ICME, Jul. 2009, pp. 486-489.
14. T. Brants, F. Chen, and I. Tsochantaridis, "Topic-based document segmentation with probabilistic latent semantic analysis," in Proc. 11th Int. Conf. Inf. Knowl. Manag., Washington, DC, USA, Nov. 2002, pp. 211-218.
15. R. Lienhart and M. Slaney, "PLSA on large scale image databases," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 18. Honolulu, HI, USA, Apr. 2007, pp. 1217-1220.
16. A. Bosch, A. Zisserman, and X. Muñoz, "Scene classification via PLSA," in Proc. 9th Eur. Conf. Comput. Vis., vol. 4. May 2006, pp. 517-530.
17. J. R. Bellegarda, "Fast update of latent semantic spaces using a linear transform framework," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 1. 2002, pp. 769-772.
18. G. W. O'Brien, "Information management tools for updating an SVDencoded indexing scheme," Comput. Sci. Dept., Univ. Tennessee, Knoxville, TN, USA, Tech. Rep. UT-CS-94-258, 1994.
19. J. E. B. Tougas, "Folding-up: A hybrid method for updating the partial singular value decomposition in latent semantic indexing," M.S. thesis, Dept. Comput. Sci., Dalhousie Univ., Halifax, NS, Canada, Dec. 2005.
20. M. Hoffman, D. Blei, and F. Bach, "Online learning for latent Dirichlet allocation," in Proc. Adv. NIPS, 2010, pp. 856-864.
21. K. Canini, L. Shi, and T. Griffiths, "Online inference of topics with latent Dirichlet allocation," in Proc. 12th Int. Conf. Artif. Intell. Statist., vol. 5. 2009, pp. 65-72.
22. A. Banerjee and S. Basu, "Topic models over text streams: A study of batch and online unsupervised learning," in Proc. 7th SIAM Int. Conf. Data Mining, 2007, pp. 437-442.
23. T. H. Brants, I. Tsochantaridis, T. Hofmann, and F. R. Chen, "Methods, apparatus, and program products for performing incremental probabilistic latent semantic analysis," U.S. Patent 20 060 112 128, May 25, 2006.
24. J. T. Chien and M. S. Wu, "Adaptive Bayesian latent semantic analysis," IEEE Trans. Audio, Speech, Lang. Process., vol. 16, no. 1, pp. 198-207, Jan. 2008.
25. T. C. Chou and M. C. Chen, "Using incremental PLSI for thresholdresilient online event analysis," IEEE Trans. Knowl. Data Eng., vol. 20, no. 3, pp. 289-299, Mar. 2008.
26. N. Bassiou and C. Kotropoulos, "RPLSA: A novel updating scheme for probabilistic latent semantic analysis," Comput. Speech Lang., vol. 25, no. 4, pp. 741-760, Oct. 2011.
27. R. M. Neal and G. E. Hinton, "A view of the EM algorithm that justifies incremental, sparse, and other variants," in Learning in Graphical Models, M. I. Jordan, Ed. Norwell, MA, USA: Kluwer, 1998, pp. 355-368.
28. A. Langville, C. Meyer, and R. Albright, "Initializations for the nonnegative matrix factorization," in Proc. 12th. ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2006, pp. 23-26.
29. T. Hofmann, "Probabilistic latent semantic indexing," in Proc. 22nd ACM SIGIR Conf. Res. Develop. Inf. Retr., 1999, pp. 50-57.
30. T. Hofmann and J. Puzicha, "Unsupervised learning from dyadic data," Int. Comput. Sci. Inst., Berkeley, CA, USA, Tech. Rep. TR-98-042, 1998.
31. T. Hofmann, "Unsupervised learning by probabilistic latent semantic analysis," Mach. Learn., vol. 42, nos. 1-2, pp. 177-196, Jan. 2001.
32. A. Dempster, N. Laird, and D. Rubin, "Maximum likelihood from incomplete data via the EM algorithm (with discussion)," J. R. Statist. Soc., Ser. B, vol. 39, no. 1, pp. 1-38, 1977.
33. D. Gildea and T. Hofmann, "Topic-based language models using EM," in Proc. 6th Eur. Conf. Speech Commun. Technol., 1999, pp. 2167-2170.
34. T. Brants, "Test data likelihood for PLSA models," Inf. Retr., vol. 8, no. 2, pp. 181-196, 2005.
35. H. Wu, D. Zhang, Y. Wang, and X. Cheng, "Incremental probabilistic latent semantic analysis for automatic question recommendation," in Proc. ACM Conf. Recommender Syst., Lausanne, Switzerland, Oct. 2008, pp. 99-106.
36. J. Xu, G. Ye, Y. Wang, G. Herman, B. Zhang, and J. Yang, "Incremental EM for probabilistic latent semantic analysis on human action recognition," in Proc. IEEE Int. Conf. Adv. Video Signal Based Surveill., Sep. 2009, pp. 55-60.
37. J. Xu, G. Ye, Y. Wang, W. Wang, and J. Yang, "Online learning for PLSA-based visual recognition," in Proc. 10th Asian Conf. Comput. Vis., 2011, pp. 95-108.
38. A. Hinneburg, H.-H. Gabriel, and A. Gohr, "Bayesian folding-in with Dirichlet kernels for PLSI," in Proc. 7th IEEE Int. Conf. Data Mining, Omaha, NE, USA, Oct. 2007, pp. 499-504.
39. E. Gaussier and C. Goutte, "Relation between PLSI and NMF and implications," in Proc. 28th ACM SIGIR Conf. Res. Develop. Inf. Retr., 2005, pp. 601-602.
40. H. Q. Ding, T. Li, and W. Peng, "NMF and PLSI: Equivalence and a hybrid algorithm," in Proc. 29th ACM SIGIR Conf. Res. Develop. Inf. Retr., 2006, pp. 641-642.
41. H. Ney, S. Martin, and F. Wessel, "Statistical language modeling using leaving-one-out," in Corpus-Based Methods in Language and Speech Processing, S. Young and G. Bloothooft, Eds. Dordrecht, The Netherlands: Kluwer, 1997, pp. 174-207.
42. M. F. Porter, "An algorithm for suffix stripping," Program, vol. 14, no. 3, pp. 130-137, Jul. 1980.
43. Z. He, A. Cichocki, S. Xie, and K. Choi, "Detecting the number of clusters in n-way probabilistic clustering," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 11, pp. 2006-2021, Nov. 2010.
44. M. Yan and K. Ye, "Determining the number of clusters using the weighted gap statistic," Biometrics, vol. 63, no. 4, pp. 1031-1037, Apr. 2007.
45. G. W. R. Tibshirani and T. Hastie, "Estimating the number of clusters in a data set via the gap statistic," J. R. Statist. Soc. (Ser. B), vol. 63, no. 2, pp. 411-423, 2001.