Relevance Ranking Metrics for Learning Objects

IEEE Transactions on Learning Technologies

Volumn 1, Issue 1, 2008, Pages 34-48

  Ochoa, Xavier ^a   Duval, Erik ^b  

^a ESCUELA SUPERIOR POLITÉCNICA DEL LITORAL   (Ecuador)

^b UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

learning object repository; Learning objects; metadata; RankNet; relevance ranking

Indexed keywords

CONTEXTUAL INFORMATIONS; CURRENT SITUATIONS; CURRENT STATUSES; EXPLICIT INFORMATIONS; LEARNING OBJECT REPOSITORY; LEARNING OBJECTS; METRICS; RANKING ORDERS; RANKNET; RELEVANCE RANKING; RELEVANCE RANKINGS; SITUATIONAL RELEVANCES;

EDUCATION; INFORMATION RETRIEVAL SYSTEMS; KETONES; LEARNING SYSTEMS; METADATA;

LEARNING ALGORITHMS;

EID: 70350130350     PISSN: 19391382     EISSN: None     Source Type: Journal    
DOI: 10.1109/TLT.2008.1     Document Type: Article

References (62)

1. R. McGreal, “Learning Objects: A Practical Definition,” Int'l J. Instructional Technology and Distance Learning, vol. 1, no. 9, p. 9, 2004
2. F. Neven and E. Duval, “Reusable Learning Objects: A Survey of LOM-Based Repositories,” Proc. 10th ACM Int'l Conf. Multimedia (MULTIMEDIA '02), pp. 291–294, 2002
3. E. Duval, K. Warkentyne, F. Haenni, E. Forte, K. Cardinaels, B. Verhoeven, R. Van Durm, K. Hendrikx, M.W. Forte, N. Ebel, and M. Macowicz, “The Ariadne Knowledge Pool System,” Comm. ACM, vol. 44, no. 5, pp. 72–78, 2001
4. J. Najjar, J. Klerkx, R. Vuorikari, and E. Duval, “Finding Appropriate Learning Objects: An Empirical Evaluation,” Proc. Ninth European Conf. Research and Advanced Technology for Digital Libraries (ECDL '05), A. Rauber, S. Christodoulakis, and A.M. Tjoa, eds., pp. 323–335, 2005
5. J. Najjar, S. Ternier, and E. Duval, “User Behavior in Learning Objects Repositories: An Empirical Analysis,” Proc. World Conf. Educational Multimedia, Hypermedia and Telecomm. (ED-MEDIA '04), L.C. Cantoni and C. McLoughlin, eds., pp. 4373–4378, 2004
6. L. Sokvitne, “An Evaluation of the Effectiveness of Current Dublin Core Metadata for Retrieval,” Proc. VALA (Libraries, Technology, and the Future) Biennial Conf., 2000
7. H. Chu and M. Rosenthal, “Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology,” Proc. 59th Ann. Meeting of the Am. Soc. Information Science, S. Hardin, ed., vol. 33, pp. 127–135, 1996
8. B. Simon, D. Massart, F. van Assche, S. Ternier, E. Duval, S. Brantner, D. Olmedilla, and Z. Miklos, “A Simple Query Interface for Interoperable Learning Repositories,” Proc. First Workshop Interoperability of Web-Based Educational Systems (WBES '05), N. Saito, D. Olmedilla, and B. Simon, eds., pp. 11–18, May 2005
9. H. Van de Sompel, M. Nelson, C. Lagoze, and S. Warner, “Resource Harvesting within the OAI-PMH Framework,” D-Lib Magazine, vol. 10, no. 12, pp. 1082–9873, 2004
10. K. Verbert, J. Jovanovic, D. Gasevic, and E. Duval, “Repurposing Learning Object Components,” Proc. Move to Meaningful Internet Systems 2005: OTM Workshops, R. Meersman, Z. Tari, and P. Herrero, eds., pp. 1169–1178, 2005
11. X. Ochoa, K. Cardinaels, M. Meire, and E. Duval, “Frameworks for the Automatic Indexation of Learning Management Systems Content into Learning Object Repositories,” Proc. World Conf. Educational Multimedia, Hypermedia, and Telecomm. (ED-MEDIA '05), P. Kommers and G. Richards, eds., pp. 1407–1414, June 2005
12. E. Duval, Policy and Innovation in Education—Quality Criteria, European Schoolnet, chapter LearnRank: The Real Quality Measure for Learning Materials, pp. 457–463, 2005
13. S. Kirsch, “Infoseek's Experiences Searching the Internet,” SIGIR Forum, vol. 32, no. 2, pp. 3–7, 1998
14. L. Page, S. Brin, R. Motwani, and T. Winograd, “The Pagerank Citation Ranking: Bringing Order to the Web,” technical report, Stanford Digital Library Technologies Project, 1998
15. X. Ochoa and E. Duval, “Use of Contextualized Attention Metadata for Ranking and Recommending Learning Objects,” Proc. First Int'l Workshop Contextualized Attention Metadata (CAMA '06), pp. 9–16, 2006
16. J. Najjar, M. Wolpers, and E. Duval, “Attention Metadata: Collection and Management,” Proc. 15th Int'l Conf. World Wide Web Workshop Logging Traces of Web Activity (WWW '06), C. Goble and M. Dahlin, eds., p. 4, 2006
17. J. Nesbit, K. Belfer, and J. Vargo, “A Convergent Participation Model for Evaluation of Learning Objects,” Canadian J. Learning and Technology, vol. 28, no. 3, pp. 105–120, 2002
18. R. Zemsky and W. Massy, “Thwarted Innovation: What Happened to e-Learning and Why,” technical report, Univ. of Pennsylvania and Thomson Corporation, 2004
19. J. Vargo, J.C. Nesbit, K. Belfer, and A. Archambault, “Learning Object Evaluation: Computer-Mediated Collaboration and Inter-Rater Reliability,” Int'l J. Computers and Applications, vol. 25, pp. 198–205, 2003
20. S. Weibel, “Border Crossings: Reflections on a Decade of Metadata Consensus Building,” D-Lib Magazine, vol. 11, nos. 7/8, p. 6, 2005
21. A. Agogino, “Visions for a Digital Library for Science, Mathematics, Engineering and Technology Education (SMETE),” Proc. Fourth ACM Digital Libraries Conf. (DL '99), pp. 205–206, 1999
22. G. Salton and C. Buckley, “Term-Weighting Approaches in Automatic Text Retrieval,” Information Processing and Management, vol. 24, no. 5, pp. 513–523, 1988
23. G. Salton and M. McGill, Introduction to Modern Information Retrieval. McGraw-Hill, 1986
24. R. Stata, K. Bharat, and F. Maghoul, “The Term Vector Database: Fast Access to Indexing Terms for Web Pages,” Computer Networks, vol. 33, nos. 1–6, pp. 247–255, 2000
25. V. Chellappa, “Content-Based Searching with Relevance Ranking for Learning Objects,” PhD dissertation, Univ. of Kansas, 2004
26. T. Joachims and F. Radlinski, “Search Engines that Learn from Implicit Feedback,” Computer, vol. 40, no. 8, pp. 34–40, 2007
27. T. Landauer, P. Foltz, and D. Laham, “An Introduction to Latent Semantic Analysis,” Discourse Processes, vol. 25, nos. 2–3, pp. 259–284, 1998
28. D. Olmedilla, “Realizing Interoperability of e-Learning Repositories,” PhD dissertation, Universidad Autónoma de Madrid, May 2007
29. E.L.-C. Law, T. Klobucar, and M. Pipan, “User Effect in Evaluating Personalized Information Retrieval Systems,” Proc. First European Conf. Technology Enhanced Learning (EC-TEL '06), W. Nejdl and K. Tochterman, eds., pp. 257–271, 2006
30. E. Hatcher and O. Gospodnetic, Lucene in Action (in Action Series). Manning, 2004
31. P. Dolog, N. Henze, W. Nejdl, and M. Sintek, “Personalization in Distributed e-Learning Environments,” Proc. 13th Int'l World Wide Web Conf. (WWW '04), M. Najork and C. Wills, eds., pp. 170–179, 2004
32. K. Sugiyama, K. Hatano, and M. Yoshikawa, “Adaptive Web Search Based on User Profile Constructed without Any Effort from Users,” Proc. 13th Int'l Conf. World Wide Web (WWW '04), pp. 675–684, 2004
33. L.M. Quiroga and J. Mostafa, “Empirical Evaluation of Explicit versus Implicit Acquisition of User Profiles in Information Filtering Systems,” Proc. Fourth ACM Conf. Digital Libraries (DL '99), N. Rowe and E.A. Fox, eds., pp. 238–239, 1999
34. A. Gulli and A. Signorini, “The Indexable Web Is More than 11.5 Billion Pages,” Proc. 14th Int'l Conf. World Wide Web (WWW '05), F. Douglis and P. Raghavan, eds., pp. 902–903, 2005
35. E. Garfield, “The Impact Factor,” Current Contents, vol. 25, no. 20, pp. 3–7, 1994
36. P. Borlund, “The Concept of Relevance in IR,” J. Am. Soc. Information Science and Technology, vol. 54, no. 10, pp. 913–925, May 2003
37. J. Broisin, P. Vidal, M. Meire, and E. Duval, “Bridging the Gap between Learning Management Systems and Learning Object Repositories: Exploiting Learning Context Information,” Proc. Advanced Industrial Conf. Telecomm./Service Assurance with Partial and Intermittent Resources Conf./E-Learning on Telecomm. Workshop (AICT-SAPIR-ELETE '05), pp. 478–483, 2005
38. P. Vandepitte, L. Van Rentergem, E. Duval, S. Ternier, and F. Neven, “Bridging an LCMS and an LMS: A Blackboard Building Block for the Ariadne Knowledge Pool System,” Proc. World Conf. Educational Multimedia, Hypermedia, and Telecomm. (ED-MEDIA '03), D.L.C. McNaught, ed., pp. 423–424, 2003
39. K. Verbert and E. Duval, “Evaluating the Alocom Approach for Scalable Content Repurposing,” Proc. Second European Conf. Technology Enhanced Learning (ECTEL '07), E. Duval, R. Klamma, and M. Wolpers, eds., vol. 4753, pp. 364–377, 2007
40. G. Linden, B. Smith, and J. York, “Amazon.com Recommendations: Item-to-Item Collaborative Filtering,” IEEE Internet Computing, vol. 7, no. 1, pp. 76–80, Jan. 2003
41. A. Pigeau, G. Raschia, M. Gelgon, N. Mouaddib, and R. Saint-Paul, “A Fuzzy Linguistic Summarization Technique for TV Recommender Systems,” Proc. IEEE Int'l Conf. Fuzzy Systems (FUZZ-IEEE '03), O. Nasraoui, H. Frigui, and J.M. Keller, eds., vol. 1, pp. 743–748, 2003
42. E.H. Chi, P. Pirolli, K. Chen, and J. Pitkow, “Using Information Scent to Model User Information Needs and Actions and the Web,” Proc. SIGCHI Conf. Human Factors in Computing Systems (CHI '01), J. Jacko and A. Sears, eds., pp. 490–497, 2001
43. A. Budanitsky and G. Hirst, “Semantic Distance in Wordnet: An Experimental, Application-Oriented Evaluation of Five Measures,” Proc. Workshop WordNet and Other Lexical Resources, Second Meeting of the North Am. Chapter of the Assoc. Computational Linguistics, pp. 29–34, 2001
44. G. Jeh and J. Widom, “Simrank: A Measure of Structural-Context Similarity,” Proc. Eighth ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '02), D. Hand, D. Keim, and R. Ng, eds., pp. 538–543, 2002
45. S. Downes, “Models for Sustainable Open Educational Resources,” Interdisciplinary J. Knowledge and Learning Objects, vol. 3, pp. 29–44, 2007
46. J.M. Kleinberg, “Authoritative Sources in a Hyperlinked Environment,” J. ACM, vol. 46, no. 5, pp. 604–632, 1999
47. A.M.A. Wasfi, “Collecting User Access Patterns for Building User Profiles and Collaborative Filtering,” Proc. Fourth Int'l Conf. Intelligent User Interfaces (IUI '99), M. Maybury, P. Szekely, and C. G. Thomas, eds., pp. 57–64, 1999
48. B. Mobasher, R. Cooley, and J. Srivastava, “Automatic Personalization Based on Web Usage Mining,” Comm. ACM, vol. 43, no. 8, pp. 142–151, 2000
49. E. Pampalk, T. Pohle, and G. Widmer, “Dynamic Playlist Generation Based on Skipping Behavior,” Proc. Sixth Int'l Conf. Music Information Retrieval (ISMIR '05), T. Crawford and M. Sandler, eds., pp. 634–637, 2005
50. O. Medelyan and I. Witten, “Thesaurus Based Automatic Keyphrase Indexing,” Proc. Sixth ACM/IEEE CS Joint Conf. Digital Libraries (JCDL '06), M.L. Nelson and C.C. Marshall, eds., pp. 296–297, 2006
51. M. Sicilia, E. Garcia, C. Pages, and J. Martinez, “Complete Metadata Records in Learning Object Repositories: Some Evidence and Requirements,” Int'l J. Learning Technology, vol. 1, no. 4, pp. 411–424, 2005
52. S. Upendra, “Social Information Filtering for Music Recommendation,” master's thesis, Massachusetts Inst. of Technology, http://citeseer.ist.psu.edu/shardanand94social.html, 1994
53. A. Aizawa, “An Information-Theoretic Perspective of TF-IDF Measures,” Information Processing and Management, vol. 39, no. 1, pp. 45–65, Jan. 2003
54. R. Kraft, F. Maghoul, and C.C. Chang, “Y!Q: Contextual Search at the Point of Inspiration,” Proc. 14th ACM Int'l Conf. Information and Knowledge Management (CIKM '05), F. Douglis and P. Raghavan, eds., pp. 816–823, 2005
55. Proc. Learning to Rank Workshop, Neural Information Processing Systems Conf. (NIPS), S. Agarwal, C. Cortes, and R. Herbrich, eds., 2005
56. R. Herbrich, T. Graepel, and K. Obermayer, Large Margin Rank Boundaries for Ordinal Regression, chapter Large Margin Rank Boundaries for Ordinal Regression, pp. 115–132, MIT Press, 2000
57. M. Richardson, A. Prakash, and E. Brill, “Beyond Pagerank: Machine Learning for Static Ranking,” Proc. 15th Int'l Conf. World Wide Web (WWW '06), C. Goble and M. Dahlin, eds., pp. 707–715, 2006
58. R. Yan and A. Hauptmann, “Efficient Margin-Based Rank Learning Algorithms for Information Retrieval,” Proc. Fifth Int'l Conf. Image and Video Retrieval (CIVR '06), W.-K. Leow, M.S. Lew, T.-S. Chua, W.-Y. Ma, L. Chaisorn, and E.M. Bakker, eds., pp. 113–122, 2006
59. V. Raykar, R. Duraiswami, and B. Krishnapuram, “A Fast Algorithm for Learning Large Scale Preference Relations,” Proc. 11th Int'l Conf. Artificial Intelligence and Statistics (AISTATS '07), M. Meila and X. Shen, eds., vol. 2, pp. 388–395, 2007
60. S. Carson, “Program Evaluation Findings Report MIT Opencourseware,” technical report, Massachusetts Inst. of Technology, 2005
61. R. Fagin, R. Kumar, M. Mahdian, D. Sivakumar, and E. Vee, “Comparing and Aggregating Rankings with Ties,” Proc. 23rd ACM SIGMOD-SIGACT-SIGART Symp. Principles of Database Systems (PODS '04), C. Beeri, ed., pp. 47–58, 2004
62. R. Kraft, C.C. Chang, F. Maghoul, and R. Kumar, “Searching with Context,” Proc. 15th Int'l Conf. World Wide Web (WWW '06), C. Groble and M. Dahlin, eds., pp. 477–486, 2006