From babel to knowledge: Data mining large digital collections

D-Lib Magazine

Volumn 12, Issue 3, 2006, Pages 22-30

  Cohen, Daniel J ^a  

^a GEORGE MASON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33645073398     PISSN: 10829873     EISSN: 10829873     Source Type: Journal    
DOI: 10.1045/march2006-cohen     Document Type: Review

References (21)

1. Hosted at the Center for History and New Media at 〈http://chnm.gmu. edu/tools/syllabi/〉.
2. Google, Google Web API?, 〈http://www.google.com/apis/〉.
3. For an example of this analysis, see Daniel J. Cohen, "By the Book: Assessing the Place of Textbooks in U. S. Survey Courses," Journal of American History 91 (March 2005), 1405-1415.
4. For the origins of the method I use, see H. P. Luhn, "A Statistical Approach to Mechanized Encoding and Searching of Literary Information," IBM Journal of Research and Development, 1(4):309-317, October 1957;
5. Harold Borko and Myrna Bernick, "Automatic Document Classification." Journal of the ACM, 10(2): 151-162, April 1963.
6. It is worth comparing the Syllabus Finder with the Open Course Ware Finder at 〈http://opencontent.org/ocwfinder/〉. The latter always returns syllabi, but because it requires syllabi to conform to the OCW standard and format, it locates far fewer total syllabi.
7. Yahoo, Content Analysis Web Services: Term Extraction, 〈http://developer.yahoo.net/search/content/V1/termExtraction.html〈.
8. Daniel J. Cohen and Roy Rosenzweig, "No Computer Left Behind," Chronicle of Higher Education, 24 February 2006, B6-8.
9. National Institute of Standards and Technology, Text REtrieval Conference, 〈http://trec.nist.gov/〉.
10. For the latest on this conference, see NIST, "Overview of the TREC 2004 Question Answering Track," 〈http://trec. nist.gov/pubs/trec13/ papers/QA.OVERViEW.pdf〉.
11. For a similar methodology to that used by H-Bot, see Hristo Tanev, Milen Kouylekov, and Bernardo Magnini, "Combining Linguistic Processing and Web Mining for Question Answering: ITC-irst at TREC," 〈http://trec.nist. gov/pubs/trec13/papers/itc-irst-tanev.web.qa. pdf〉.
12. For some earlier foundations of this methodology, see Bernardo Magnini et al., "Is it the Right Answer? Exploiting Web Redundancy for Answer Validation," Association for Computational Linguistics 40th Anniversary Meeting (ACL-02), of Pennsylvania, Philadelphia, July 7 - 12;
13. Bernardo Magnini et al., "Comparing Statistical and Content-Based Techniques for Answer Validation on the Web," Proceedings of the VIII Convegno AI*IA, Siena - Italy, September 11-13.
14. Bernardo Magnini et al., "Mining Knowledge from Repeated Co-occurrences," TREC-11 Conference Notebook Papers, Gaithersburg, MD, November 19-22.
15. Available in beta at the Center for History and New Media at 〈http://chnm.gmu.edu/tools/h-bot/〉.
16. For the origins of this theory, see A.N. Kolmogorov, "Three approaches to the quantitative definition of information," Problems in Information Transmission, volume 1, number 1 (1965), pp. 1-7.
17. For its modern application, see Paul Vitanyi and Rudi Cilibrasi, "Automatic Meaning Discovery Using Google," at http://arxiv.org/abs/ cs/0412098, accessed on 30 April 2005.
18. For more on this topic, see Daniel J. Cohen and Roy Rosenzweig, "Web of Lies? Historical Knowledge on the Internet," First Monday 10 (December 2005), 〈http://firstmonday.org/issues/issue10_12/cohen〉.
19. At the Center for History and New Media we have recently launched a digital collection related to the hurricanes of 2005, which uses the Google Maps API to locate each photo, story, or multimedia file on the Gulf Coast, providing a new way of searching historical archives. Center for History and New Media, Hurricane Digital Memory Bank, 〈http://hurricanearchive.org/〉.
20. Open Content Alliance, 〈http://www.opencontentalliance.org/〉;
21. Google Book Search, 〈http://books. google.com/〉.