Modeling human judgment of digital imagery for multimedia retrieval

IEEE Transactions on Multimedia

Volumn 9, Issue 5, 2007, Pages 967-974

  Volkmer, Timo ^a,b   Thom, James A ^b   Tahaghoghi, Seyed M M ^a,b  

^a IEEE   (Australia)

^b RMIT UNIVERSITY   (Australia)

Author keywords

Annotation; Latent class modeling

Indexed keywords

ANNOTATION QUALITY AND; LATENT CLASS MODELING; MANUAL ANNOTATION PROCESSES;

IMAGE RETRIEVAL; INFORMATION SYSTEMS; MATHEMATICAL MODELS; MULTIMEDIA SYSTEMS; ONLINE SEARCHING; SEMANTICS;

LEARNING SYSTEMS;

EID: 34547442343     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2007.900153     Document Type: Article

References (27)

1. C. G. M. Snoek and M. Worring, "Multimodal video indexing: A review of the state-of-the-art," Multimedia Tools Applicat., vol. 25, no. 1, pp. 5-35, Jan. 2005.
2. A. G. Hauptmann, R. Yan, W.-H. Lin, M. Christel, and H. Wactlar, "Can high-level concepts fill the semantic gap in video retrieval? A case study with broadcast news," IEEE Trans. Multimedia, to be published.
3. C. G. M. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring, "Adding semantics to detectors for video retrieval," IEEE Trans. Multimedia, to be published.
4. L. von Ahn and L. Dabbish, "Labeling images with, a computer game," in Proc. Conf. Human Factors in Computing Systems (CHI) 2004, Vienna, Austria, Apr. 24-29, 2004, pp. 319-326.
5. A. F. Smeaton, "Large scale evaluations of multimedia information retrieval: The TRECVid experience," in Proc. 4th Int. Conf. Image and Video Retrieval (CIVR) W. Leow, M. S. Lew, T. Chua, W. Ma, L. Chaisorn, and E. M. Bakker, Eds., vol. 3568, Singapore, Jul. 20-22, 2005, pp. 11-17.
6. C-Y. Lin, B. L. Tseng, and J. R. Smith, "Video collaborative annotation forum: Establishing ground-truth labels on large multimedia datasets," in Proc. TREC Video Retrieval Evaluation (TRECVID 2003), E. M. Voorhees and L. P. Buckland, Eds., Nov. 18-21, 2003 [Online]. Available: http://www-nlpir.nist.gov/projects/tvpubs/tvpapers03/ibm.final2.paper.pdf
7. M. R. Naphade, L. Kennedy, J. Render, S.-F. Chang, J. R. Smith, P. Over, and A. Hauptmann, A Light Scale Concept Ontology for Multimedia Understanding for TRECVID 2005 IBM T. J. Watson Research Center, Hawthorne, NY, Tech. Rep. RC23612, May 17, 2005 [Online], Available: http://www.domino.watson.ibm.com/ library/CyberDig.nsf/ papers/A33ABDB65967B53B852570070056B36F/$File/ rc23612.pdf
8. A. Amir, J. O. Argillander, M. S. Campbell, A. Haubold, G. Iyengar, S. Ebadollahi, F. Kang, M. R. Naphade, A. Natsev, J. R. Smith, J. Tešić, and T. Volkmer, "IBM research TRECVID-2005 video retrieval system," in Proc. TREC Video Retrieval Evaluation (TRECVID 2005), P. Over and T. Ianeva, Eds., Nov. 14-15, 2005 [Online], Available: http://www-nlpir.nist.gov/projects/tvpubs/tv5.papers/ibm.pdf
9. S.-F. Chang, W. Hsu, L. Kennedy, L. Xie, A. Yanagawa, E. Zavesky, and D.-Q. Zhang, "Columbia University TRECVID-2005 video search and high-level feature extraction," in Proc. TREC Video Retrieval Evaluation (TRECVID 2005), P. Over and T. Ianeva, Eds., Gaithersburg, MD, Nov. 14-15, 2005 [Online]. Available: http://www-nlpir.nist.gov/ projects/tvpubs/tv5.papers/ columbia.pdf
10. C. G. M. Snoek, M. Worring, J. C. van Gemert, J.-M. Geusebroek, and A. W. Smeulders, "The challenge problem, for automated detection of 10.1 semantic concepts in multimedia," in Proc. ACM Int. Conf. Multimedia 2006, K. Nahrstedt and M. Turk, Eds., Santa Barbara, CA, Oct. 23-27, 2006, pp. 421-430.
11. M. R. Naphade, J. R. Smith, J. Tešić, S.-F. Chang, W. Hsu, L. Kennedy, A. G. Hauptmann, and J. Curtis, "Large-scale concept ontology for multimedia," IEEE Multimedia, vol. 13, no. 3, pp. 86-91, Jul.-Sep. 2006.
12. T. Volkmer, J. R. Smith, A. Natsev, M. S. Campbell, and M. R. Naphade, "A web-based system for collaborative annotation of large image and video collections," in Proc. ACM Int. Conf. Multimedia 2005, H.-J. Zhang and T.-S. Chua, Eds., Singapore, Nov. 6-11, 2005, pp. 892-901.
13. P. F. Lazarsfeld and N. W. Henry, Latent Structure Analysis. New York: Houghton Mifflin New York, 1968.
14. J. Rost and R. Langeheine, "A guide through latent structure models for categorical data," in Applications of Latent Trait and Latent Class Models in the Social Sciences, J. Rost and R. Langeheine, Eds. New York: Waxman, 1997, ch. 1, pp. 13-37.
15. J. S. Uebersax and W. M. Grove, "Latent class analysis of diagnostic agreement," Statist. Med., vol. 9, pp. 559-572, May 1990.
16. J. K. Vermunt, "Log-Linear Models for Event Histories," Ph.D. dissertation, Department of Methodology and Statistics, Tilburg Univ., Amsterdam, The Netherlands, 1996.
17. C. Petersohn, "Fraunhofer HHI at TRECVID 2004: Shot boundary detection system," in Proc. TREC Video Retrieval Evaluation (TRECVID 2004), E. M. Voorhees and L. P. Buckland, Eds., Nov. 15-16, 2004 [Online]. Available: http://www-nlpir.nist.gov/projects/tvpubs/tvpapers04/fraunhofer.pdf
18. E. Cooke, P. Ferguson, G. Gaughan, C. Gurrin, G. J. F. Jones, H. L. Borgue, H. Lee, S. Marlow, K. McDonald, M. McHugh, N. Murphy, N. E. O'Connor, N. O'Hare, S. Rothwell, A. F. Smeaton, and P. Wilkins, "TRECVID 2004 experiments in Dublin City University," in Proc. TREC Video Retrieval Evaluation (TRECVID 2004), E. M. Voorhees and L. P. Buckland, Eds., Nov. 15-16, 2004 [Online], Available: http://www-nlpir.nist.gov/projects/tvpubs/ tvpapers04/dcu.pdf
19. J. K. Vermunt and J. Magidson, "Latent class analysis," Sage Encyclopedia Social Sci. Res. Methods, vol. 3, pp. 549-553, Dec. 2003.
20. H. O. Hartley, "Maximum likelihood estimation from, incomplete data," Biometrics, vol. 14, no. 2, pp. 174-194, Jun. 1958.
21. J. K. Vermunt, ℓEM: A General Program for the Analysis of Categorical Data (Users' Manual) Department of Methodology and Statistics, Tilburg Univ. Amsterdam, The Netherlands, 1997 [Online]. Available: http://www.uvt.nl/ faculteiten/fsw/organisatie/departementen/mto/software2.html
22. J. L. Fleiss, Statistical Methods for Rates and Proportions, 2nd ed. New York: Wiley, 1981, ch. 13, pp. 212-236.
23. J. Cohen, "A coefficient of agreement for nominal scales," Edu. Psychol. Meas., vol. 20, pp. 37-46, 1960.
24. B. DiEugenio and M. Glass, 'The kappa statistic: A second look," J. Comput. Linguist, vol. 30, no. 1, pp. 95-101, Mar. 2004.
25. J. S. Uebersax, "Validity inferences from interobserver agreement," Psychol. Bul, vol. 104, no. 3, pp. 405-416, Nov. 1988.
26. T. Volkmer, S. M. M. Tahaghoghi, and J. A. Thom, "Exploring human, judgement of digital imagery," in Proc. 30th Australasian Computer Science Conference (ACSC 2007), G. Dobbie, Ed., Ballarat, VIC, Australia, Jan. 9-Feb. 2 2007, vol. 62, no. 1, pp. 151-160.
27. R. R. Sokal and F. J. Rohlf, Biometry: The Principles and Practice of Statistics in Biological Research, 3rd ed. San Francisco, CA: Freeman, 1995, pp. 699-707.