TRECVID 2015 – An overview of the goals, tasks, data, evaluation mechanisms, and metrics

2015 TREC Video Retrieval Evaluation, TRECVID 2015

Volumn , Issue , 2015, Pages

  Over, Paul ^h   Awad, George ^a,b   Fiscus, Jon ^b   Michel, Martial ^b   Joy, David ^b   Smeaton, Alan F ^c   Kraaij, Wessel ^d,e   Quénot, Georges ^f   Ordelman, Roeland ^g   Aly, Robin ^g  

^a DAKOTA CONSULTING INC   (United States)

^b NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

^c Digital Enterprise Research Institute (DERI)   (Ireland)

^d TNO   (Netherlands)

^e RADBOUD UNIVERSITY NIJMEGEN   (Netherlands)

^f GRENOBLE INSTITUTE OF TECHNOLOGY   (France)

^g UNIVERSITY OF TWENTE   (Netherlands)

^h NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85021803260     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (18)

1. Aly, R., Eskevich, M., Ordelman, R., and Jones, G. J. F. (2013). Adapting Binary Information Retrieval Evaluation Metrics for Segment-based Retrieval Tasks. CoRR, abs/1312.1913.
2. Chatfield, K. and Zisserman, A. (2013). Visor: Towards on-the-fly large-scale object category retrieval. In Computer Vision–ACCV 2012, pages 432–446. Springer.
3. Eskevich, M., Magdy, W., and Jones, G. J. F. (2012). New Metrics for Meaningful Evaluation of Informally Structured Speech Retrieval. In Proceedings of ECIR 2012, pages 170–181, Barcelona, Spain.
4. Eyben, F., Weninger, F., Gross, F., and Schuller, B. (2013). Recent developments in openSMILE, the Munich open-source multimedia feature extractor. In Proceedings of ACM Multimedia 2013, pages 835–838, Barcelona, Spain. ACM.
5. Kamps, J., Pehcevski, J., Kazai, G., Lalmas, M., and Robertson, S. (2006). INEX 2007 evaluation measures. In Fuhr, N., Kamps, J., Lalmas, M., Trotman, A. (eds.) Focused access to XML Documents, pages 24–33. Springer.
6. Liu, B. and Oard, D. W. (2006). One-sided measures for evaluating ranked retrieval effectiveness with spontaneous conversational speech. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 673–674. ACM.
7. Manly, B. F. J. (1997). Randomization, Bootstrap, and Monte Carlo Methods in Biology. Chapman & Hall, London, UK, 2nd edition.
8. Ordelman, R. J., Eskevich, M., Aly, R., Huet, B., and Jones, G. J. (2015). Defining and evaluating video hyperlinking for navigating multimedia archives. In LIME 2015, 3rd International Workshop on Linked Media, co-located with WWW 2015, 18 May 2015, Florence, Italy, Florence, ITALIE.
9. Over, P., Ianeva, T., Kraaij, W., and Smeaton, A. F. (2006). TRECVID 2006 Overview. www-nlpir.nist.gov/projects/ tvpubs/tv6.papers/tv6overview.pdf.
10. Racca, D. N. and Jones, G. J. F. (2015). Evaluating Search and Hyperlinking: An Example of the Design, Test, Refine Cycle for Metric Development. In Proceedings of the MediaEval 2015 Workshop, Wurzen, Germany.
11. Rose, T., Fiscus, J., Over, P., Garofolo, J., and Michel, M. (2009). The TRECVid 2008 Event Detection Evaluation. In IEEE Workshop on Applications of Computer Vision (WACV). IEEE.
12. Strassel, S., Morris, A., Fiscus, J., Caruso, C., Lee, H., Over, P., Fiumara, J., Shaw, B., Antonishek, B., and Michel, M. (2012). Creating havic: Heterogeneous audio visual internet collection. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA).
13. Tommasi, T., Tuytelaars, T., and Caputo, B. (2014). A Testbed for Cross-Dataset Analysis. CoRR, abs/1402.5923.
14. TV15Notebook (2015). Trecvid 2015 workshop notebook. http: //www-nlpir.nist.gov/projects/tv2015/ active/tv15.workshop.notebook.
15. TV15Pubs (2015). Trecvid 2015 workshop proceeding. http://www-nlpir.nist. gov/projects/tvpubs/tv.pubs.15.org.html.
16. UKHO-CPNI (2007 (accessed June 30, 2009)). Imagery library for intelligent detection systems. http://scienceandresearch.homeoffice. gov.uk/hosdb/cctv-imaging-technology/ video-based-detection-systems/i-lids/.
17. Yilmaz, E. and Aslam, J. A. (2006). Estimating Average Precision with Incomplete and Imperfect Judgments. In Proceedings of the Fifteenth ACM International Conference on Information and Knowledge Management (CIKM), Arlington, VA, USA.
18. Yilmaz, E., Kanoulas, E., and Aslam, J. A. (2008). A simple and efficient sampling method for estimating AP and NDCG. In SIGIR’08: Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 603–610, New York, NY, USA. ACM.