LSA and Information Retrieval: Getting Back to Basics

Handbook of Latent Semantic Analysis

Volumn , Issue , 2007, Pages 293-321

  Dumais, Susan T ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84890079459     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.4324/9780203936399-23     Document Type: Chapter

References (102)

1. Aclioptas, D., andMcSherry, F. (2001). Fast computation of low rankmatrix approximations. Proceedings of the 33rd Annual ACM Symposium on Theory of Computing, 611-618.
2. Ando, R. K., and Lee, L. (2001). Iterative residual rescaling: An analysis and generalization of LSI. Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 154-162.
3. Baker, F. B. (1962). Information retrieval based on latent class analysis. Journal of the ACM, 9, 512-521.
4. Bartell, B. T., Cottrell, G.W., andBelew, R.K. (1992). Latent semantic indexing is an optimal special case of multidimensional scaling. Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 161-167.
5. Bates, M. (1986). Subject access in online catalogs: A design model. Journal of the American Society for Information Science, 37, 357-376.
6. Bates, M. (1998, November).Howto use controlled vocabularies more effectively in online searching. Online, 45-56.
7. Berry, M.W,. and Young, P. G. (1995). Using latent semantic indexing for multilingual information retrieval. Computers and the Humanities, 29(6), 413-419.
8. Berry, M.W., Dumais, S. T., and O’Brien, G.W. (1995). Using linear algebra for intelligent information retrieval. SIAM: Review, 37(4), 573-595.
9. Bharat, K., and Henzinger, M. (1998). Improved algorithms for topic distillation in a hyperlinked environment. Proceedings of the 21st Annual InternationalACMSIGIR Conference on Research and Development in Information Retrieval, 104-111.
10. Borko, H., andBernick, M. D. (1963). Automatic document classification. Journal of the ACM, 10, 151-162.
11. Brin, S., andPage, L. (1998). Anatomy of a large-scale hypertextual web search engine. Proceedings of the 7th InternationalWorldWideWeb Conference/Computer Networks, 30(1-7), 107-117.
12. Caid, W. R., Dumais, S. T., andGallant, S. I. (1995). Learned vector space models for information retrieval. Information Processing and Management, 31(3), 419-429.
13. Carbonell, J., Yang, Y., Frederking, R., Brown, R. D., Geng, Y., and Lee, D. (1997). Translingual information retrieval: Acomparative evaluation. Proceedings of the 15th International Joint Conference on Artificial Intelligence, 323-345.
14. Chan, L. M. (1989). Inter-indexer consistency in subject cataloging. Information Technology and Libraries, 8(4), 349-358.
15. Chen, C-M., Stoffel, N., Post, M., Basu, C., Bassu, D., and Behrens, C. (2001). Telcordia LSI engine: Implementation and scalability issues. Proceedings of the 11th International Workshop on Research Issues in Data Engineering, 51-58.
16. Cristianini, N., Shawe-Taylor, J., and Lodhi, H. (2001). Latent semantic kernels. Proceedings of ICML-01, 18th International Conference on Machine Learning, 66-73.
17. Cohn, D., and Hofmann, T. (2001). The missing link: A probabilistic model of document content and hypertext connectivity. In T. K. Leen, T. G. Dietterich, and V. Tresp (Eds.), Advances in Neural Information Processing Systems (pp. 430-436). Cambridge, MA: MIT Press
18. Craswell, N., Hawking, D., andRobertson, S.E. (2001). Effective site findingusinganchortext information. Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 250-257.
19. Deerwester, S., Dumais, S. T., Furnas, G.W., Landauer, T. K., andHarshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41, 391-407.
20. de Kermadec, F. J. (2004). The fight against spam, part 2. O’Reilly Mac Developer. Retrieved August 25, 2006, from http://www.macdevcenter.com/pub/a/mac/2004/05/18/spam_pt2.html?page=1 and http://www.aple.com/lae/macosx/jaguar/mail.html
21. Ding, C.H. Q. (1999). Asimilarity-based probabilitymodel for latent semantic indexing. Proceedings of 22nd Annual International ACMSIGIR Conference on Research and Development in Information Retrieval, 59-65.
22. Dumais, S. T. (1991). Improving the retrieval of information from external sources. Behavior Research Methods, Instruments and Computers, 23(2), 229-236.
23. Dumais, S. T. (1995). Using LSI for information filtering: TREC-3 experiments. In D. Harman (Ed.), The Third Text REtrieval Conference (TREC-3) National Institute of Standards and Technology (Special Publication 500-225, pp. 219-230). Washington, DC: U.S. Government Printing Office.
24. Dumais, S. T. (2004). Latent semantic analysis. Annual Review of Information Science and Technology (ARIST), 38, 189-230.
25. Dumais, S. T., Furnas, G.W., Landauer, T. K., Deerwester, S., andHarshman, R. (1988). Using latent semantic analysis to improve information retrieval. Proceedings of SIGCHI Conference on Human Factors in Computing Systems, 281-285.
26. Dumais, S.T., Littman, M.L., andLandauer, T. K. (1998). Automatic cross-linguistic information retrieval using latent semantic analysis. In G. Grefenstette (Ed.), Cross language information retrieval (pp. 51-62). Boston: Kluwer Academix Publishers.
27. Dumais, S. T., and Nielsen, J. (1992). Automating the assignment of submitted manuscripts to reviewers. Proceedings of the 15th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 233-244.
28. Efron, M. (2002). Amended parallel analysis for optimal dimensionality estimation in latent semantic indexing (SLIS Tech. Rep. No. TR-2002-03). University of North Carolina, Chapel Hill.
29. Eiron, N., andMcCurley, K. S. (2003). Analysis of anchor texts for web search. Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 459-460.
30. Evans, D. A., Handerson, S.K., Monarch, I.A., Pereiro, J., Delon, L., andHersh, W.R. (1998). Mapping vocabularies using “latent semantics.” In G. Grefenstette (Ed.), Cross language information retrieval (pp. 63-80). Boston: Kluwer Academic Publishers.
31. Fidel, R. (1985). Individual variability in online searching behavior. Proceedings of the American Society for Information Science 48th Annual Meeting, 69-72.
32. Foltz, P.W., and Dumais, S. T. (1992). Personalized information delivery: An analysis of information filtering methods. Communications of the ACM, 35(12), 51-60.
33. Frieze, A., Kannan, R., and Vempala, S. (1998). Fast Monte-Carlo algorithms for finding low-rank approximations. Proceedings of the 39th Annual Symposium on Foundations of Computer Science, 370-378.
34. Furnas, G. (1985). Experience with an adaptive indexing scheme. Proceedings of the SIGCHI Conference on Human Factors in Computing, 131-135.
35. Furnas, G. W., Landauer, T. K., Gomez, L. M,. and Dumais, S. T. (1987). The vocabulary probleminhuman-computerinteraction. Communicationsof theACM, 30(11), 964-971.
36. Gao, J., and Zhang, J. (2003). Sparsification strategies in latent semantic indexing. (Tech. Rep. No. 368-03). University of Kentucky: Department of Computer Science.
37. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science, 178, 471-479.
38. Girolami, M. (2000). Document representations based on generative multivariate Bernoulli latent topic models. Proceedings of the BCS-Information Retrieval Specialist Group 22nd Annual Colloquium on Information Retrieval Research, 194-201.
39. Golub, G. H., and Van Loan, C. F. (1989). Matrix computations (2nd ed.). Baltimore: Johns Hopkins University Press.
40. Gong, Y. H., and Liu, X. (2001). Generic text summarization using relevance measure and latent semantic analysis. Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 19-25.
41. Gordon, M. D., and Dumais, S. T. (1998). Using latent semantic indexing for literature based discovery. Journal of the AmericanSociety for InformationScience, 49(8), 674-685.
42. Gordon, M. D., and Lindsay, R. K. (1996). Toward discovery support systems:Areplication, re-examination, and extension of Swanson’s work on literature based discovery of a connection between Raynaud’s and Fish Oil. Journal of the American Society for Information Science, 47(2), 116-128.
43. Griffiths, T. L., andSteyvers, M. (2002).Aprobabilistic approach to semantic representation. In W. D. Gray and C. D. Schunn (Eds.), Proceedings of 24th Annual Cognitive Science Conference, 381-386.
44. Harman, D. (1991). How effective is suffixing? Journal of the American Society for Information Science, 42(1), 7-15.
45. Heinrich, K., Berry, M., Dongarra, J. J., and Vadhiyar, S. (2003). The semantic conference organizer. In H. Bozdogan (Ed.), Statistical data mining and knowledge discovery (pp. 571-581). Boca Raton, FL: CRC Press.
46. Hull, D. A. (1996). Stemming algorithms: a case study for detailed evaluation. Journal of the American Society for Information Science, 47(1), 70-84.
47. Hofmann, T. (1999). Probabilistic latent semantic analysis. Proceedings of 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 50-57.
48. Hull, D. A. (1994). Improving text retrieval for the routing problem using latent semantic indexing. Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 282-290.
49. Hull, D. A., Pedersen, J. O., andSchütze, H. (1996). Method combination for document filtering. Proceedings of the 19th Annual InternationalACMSIGIR Conference on Research and Development in Information Retrieval, 279-288.
50. Husbands, P., Simon, H., and Ding, C. (2000, October). On the use of singular value decomposition for text retrieval. Proceedings of the First SIAMComputational Information Retrieval Workshop. Retrieved August 25, 2006, from http://www-library.lbl.gov/docs/LBNL/471/70/PDF/LBNL-47170.pdf
51. Isbell, C. L., andViola, P. (1998). Restructuring sparse high-dimensional data for effective retrieval. Advances in Neural Information Processing, NIPS11, 480-486.
52. Jardin, N., and van Rijsbergen, C. J. (1971). The use of hierarchical clustering in information retrieval. Information Storage and Retrieval, 7, 214-240.
53. Jiang, F., Kannan, R. Littman, M. L., andVempala, S. (1999, February). Efficient singular value decomposition via improved document sampling (Tech. Rep. No. CS-99-5). Duke University, Department of Computer Science, Durham, NC.
54. Jiang, F., and Littman, M. L. (2000). Approximate dimension equalization in vector-based information retrieval. Proceedings of the 17th International Conference on Machine Learning, 423-430.
55. Jiang, F., andLittman, M. L. (2001). Approximate dimension reduction at NTCIR. Proceedings of NTCIRWorkshop 2: Proceeding of the SecondNTCIRWorkshop on Research in Chinese and Japenese Text Retrieval and Text Summarization. Retrieved August 25, 2006, from http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings2/michael.pdf
56. Jing, Y., and Croft, W. B. (1994). An association thesaurus for information retrieval. Proceedings of RIAO’ 94, 146-160.
57. Karypis, G., andHan, E-H. (2000). Fast supervised dimensionality reduction algorithm with applications to document categorization and retrieval. Proceedings of the Ninth Annual International ACM SIGIR International Conference on Information and Knowledge Management, 12-19.
58. Kleinberg, J. (1998). Authoritative sources in a hyperlinked environment. Proceedings of the ACM-SIAM Symposium on Discrete Algorithms, 668-677.
59. Kolda, T. G., andO’Leary, D. P. (1998). Asemidiscrete matrix decomposition for latent semantic indexing in information retrieval.ACMTransactions on Information Systems (TOIS), 16(4), 322-346.
60. Koll, M. (1979). An approach to concept-based information. ACM SIGIR Forum, 13, 32-50.
61. Kontostathis, A., Pottenger, W. M., and Davison, B. D. (2004). Assessing the impact of sparsification on LSI performance. Proceedings of the 2004 Grace Hopper Celebra-tion of Women in Computing Conference. Retrieved August 25, 2006, from http://webpages.ursins.edu/akontostathis/KontostathisGH.pdf
62. Kurimo, M. (2000). Fast latent semantic indexing of spoken documents by using self-organizing maps. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP’2000, 2425-2428.
63. Lancaster, F. (1986). Vocabulary control for information retrieval (2nd ed.). Arlington, VA: Information Resources.
64. Landauer, T. K., and Dumais, S. T. (1997). Asolution to Plato’s problem: The latent semantic analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211-240.
65. Landauer, T. K., and Littman, M. L. (1990). Fully automatic cross-language document retrieval using latent semantic indexing. Proceedings of the Sixth Annual Conference on Electronic Text Research, 31-38.
66. Littman, M. L., and Jiang, F. (1998, June). A comparison of two corpus-based methods for translingual information retrieval (Tech. Rep. No. CS-1998-11). Department of Computer Science, Duke University.
67. Littman, M. L., Jiang, F., and Keim, G. A. (1998). Learning a language-independent representation for terms from a partially aligned corpus. Proceedings of the 15th International Conference on Machine Learning, 314-322.
68. Lochbaum, K., and Streeter, L. A. (1989). Comparing and combining the effectiveness of latent semantic indexing and the ordinary vector space model for information retrieval. Information Processing and Management, 25(6), 665-676.
69. Mori, T., Kokubu, T., and Tanaka, T. (2001). Cross-lingual information retrieval based on LSI with multiple word spaces. Proceedings of NTCIRWorkshop 2: Proceeding of the Second NTCIR Workshop on Research in Chinese and Japanese Text Retrieval and Text Summarization. Retrieved August 25, 2006, from http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings2/mori-ir.pdf
70. Nielsen, J., Phillips, V. L., and Dumais, S. T. (1992, August). Retrieving imperfectly recognized handwritten notes. Bellcore Technical Memorandum, TMARH-021781.
71. Oard, D.W., and Dorr, B. J. (1998). Evaluating cross-language text filtering effectiveness. In G. Grefenstette (Ed.), Cross language information retrieval (pp. 151-161).
72. Orengo, V. M., andHuyck, C. (2002, September). Portuguese-English experiments using latent semantic indexing. Proceedings of the Third Workshop of the Cross-Language Evaluation Forum, CLEF 2002, 147-154.
73. Ossorio, P. G. (1966). Classification space: A multivariate procedure for automatic document indexing and retrieval. Multivariate Behavior Research, 479-524.
74. Page, E. B. (1994). Computer grading of student prose using modern concepts and software. Journal of Experimental Education, 62, 127-142.
75. Papadimitriou, C. H., Raghavan, P., Tamaki, H., andVempala, S. (1995). Latent semantic indexing:Aprobabilistic analysis. Proceedings of the 17th Annual Symposium on Principles of Database Systems, 159-169.
76. Pinski, G., and Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory with application to the literature of physics. Information Processing and Management, 12(5), 297-312.
77. Reisner, P. (1966, August). Evaluation of a growing thesaurus (IBM Research Paper No. RC-1662). Yorktown Heights, NY: IBM Watson Research Center.
78. Rehder, B., Littman, M. L., Dumais, S. T., and Landauer, T. K. (1997). Automatic 3-language cross-language information retrieval with latent semantic indexing. NIST Special Publication 500-240: The SixthText Retrieval Conference (TREC-6), 233-240.
79. Robertson, S. E., and Soboroff, I. (2001). The TREC 2001 filtering track report. In E. Voorhees (Ed.), NIST Special Publication 500-250: The Tenth Text REtrieval Conference, 26-37.
80. Salton, G. (1968). Automatic information organization and retrieval. New York: McGraw-Hill.
81. Salton, G. (1969). Acomparison between manual and automatic indexing methods. American Documentation, 20(1), 61-71.
82. Salton, G., and Buckley, C. (1990). Improving retrieval by relevance feedback. Journal of the American Society for Information Science, 41(4), 288-297.
83. Salton, G. and McGill, M. (1983). Introduction to modern information retrieval. McGraw-Hill.
84. Schütze, H., Hull, D. A., andPedersen, J. (1995). Acomparison of classifiers and document representation for the routing problem. Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 229-237.
85. Schütze, H., and Silverstein, C. (1997). Acomparison of projections for efficient document clustering. Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 74-81.
86. Shen, D., Chen, Z., Yang, Q., Zeng, H-J., Zhang, B., Lu, Y., and Ma, W-Y. (2004). Text classification through summarization. Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 242-249.
87. Sheridan, P., and Ballerini, J. P. (1996). Experiments in multilingual information retrieval using the SPIDER system. Proceedings of the 19th Annual InternationalACM SIGIR Conference on Research and Development in Information Retrieval, 58-65.
88. Sparck Jones, K. (1971). Automatic keyword classification in information retrieval. London: Buttersworth.
89. Sparck Jones, K. (1972). Astatistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28, 11-21.
90. Srinivasan, P. (1992). Thesaurus construction. In W. B. Frakes and R. Baeza-Yates (Eds.), Information retrieval: Data structures and algorithms (pp. 161-218). Englewood Cliffs, NJ: Prentice-Hall.
91. Streeter, L. A., and Lochbaum, K. E. (1987). An expert/expert-locating system based on automatic representation of semantic structure. Proceedings of the Fourth Conference on Artificial Intelligence Applications, 345-350.
92. Steyvers, M., Smyth, P., and Griffiths, T. (2004). Probabilistic author-topic models for information discovery. Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 306-315.
93. Swanson, D. R. (1989). Online search for logically-related noninteractive medical literatures: A systematic trial-and-error strategy. Journal of the American Society for Information Science, 40(5), 356-358.
94. Tarr, D., and Borko, H. (1974). Factors influencing inter-indexer consistency. Proceedings of the American Society for Information Science 37th Annual Meeting, 11, 50-55.
95. Vinokourov, A., Shawe-Taylor, J., and Cristianini, N. (2002). Inferring a semantic representation of text via cross-language correlation analysis. Advances in Neural Information Processing Systems, 1473-1480.
96. Voorhees, E. (1994). Query expansion using lexical-semantic relations. Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 61-70.
97. Wiener, E., Pedersen, J. O., andWeigend, A. S. (1995). Aneural network approach to topic spotting. Proceedings of SDAIR-95, Fourth Annual Symposium on Document Analysis and Information Retrieval, 317-332.
98. Westerveld, T., Kraaij, W., and Hiemstra, D. (2002). Retrieving web pages using content, links, URLs and anchors. Proceedings of the Tenth Text REtrieval Conference (TREC-10), 663-672.
99. Wolf, P., and Raj, B. (2002). The MERL spoken query information retrieval system: A system for retrieving pertinent documents from a spoken query. IEEE International Conference on Multimedia and Expo (ICME’2002), 317-320.
100. Xu, J., andCroft, W.B. (1996). Query expansion using local and global document analysis. Proceedings of the 19th Annual InternationalACMSIGIR Conference on Research and Development in Information Retrieval, 4-11.
101. Zelikovitz, S., and Hirsh, H. (2001). Using LSI for text classification in the presence of background text. Proceedings of the 2005ACMCIKMInternational Conference on Information and Knowledge Management, 113-118.
102. Zha, H., and Simon, H. (1999). On updating problems in latent semantic indexing. SIAM Journal on Scientific Computing, 21(2), 782-791.