Human-Machine CRFs for Identifying Bottlenecks in Scene Understanding

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 38, Issue 1, 2016, Pages 74-87

  Mottaghi, Roozbeh ^a   Fidler, Sanja ^c   Yuille, Alan ^b   Urtasun, Raquel ^c   Parikh, Devi ^d  

^a Allen Institute for Artificial Intelligence ^*  (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF TORONTO   (Canada)

^d State University   (United States)

Author keywords

Human Machine Hybrid; Object Detection; Scene Recognition; Scene Understanding; Semantic Segmentation

Indexed keywords

IMAGE SEGMENTATION; OBJECT RECOGNITION; SEMANTICS;

APPEARANCE BASED; CONDITIONAL RANDOM FIELD; CONTEXTUAL REASONING; HUMAN-MACHINE; RESEARCH EFFORTS; SCENE RECOGNITION; SCENE UNDERSTANDING; SEMANTIC SEGMENTATION;

OBJECT DETECTION;

ALGORITHM; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BRAIN COMPUTER INTERFACE; COMPUTER SIMULATION; FACTUAL DATABASE; HUMAN; PATTERN RECOGNITION; STATISTICS AND NUMERICAL DATA;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BRAIN-COMPUTER INTERFACES; COMPUTER SIMULATION; DATABASES, FACTUAL; HUMANS; PATTERN RECOGNITION, AUTOMATED; PATTERN RECOGNITION, VISUAL;

EID: 84961639809     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2015.2437377     Document Type: Article

References (44)

1. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan, "Object detection with discriminatively trained part based models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 9, pp. 1627-1645, Sep. 2010.
2. J. Xiao, J. Hays, K. Ehinger, A. Oliva, and A. Torralba, "Sun database: Large-scale scene recognition from abbey to zoo," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3485-3492.
3. A. Rabinovich, A. Vedaldi, C. Galleguillos, E. Wiewiora, and S. Belongie, "Objects in context," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
4. Y. Yang and D. Ramanan, "Articulated pose estimation using flexible mixtures of parts," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 1385-1392.
5. A. Torralba, "How many pixels make an image?" Vis. Neurosci., vol. 26, no. 1, pp. 123-131, 2009.
6. S. Gould, T. Gao, and D. Koller, "Region-based segmentation and object detection," in Proc. Adv. Neural Inf. Process. Syst., 2009, pp. 655-663.
7. J. Yao, S. Fidler, and R. Urtasun, "Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 702-709.
8. G. Heitz, S. Gould, A. Saxena, and D. Koller, "Cascaded classification models: Combining models for holistic scene understanding," in Proc. Adv. Neural Inf. Process. Syst., 2008, pp. 641-648.
9. C. Li, A. Kowdle, A. Saxena, and T. Chen, "Towards holistic scene understanding: Feedback enabled cascaded classification models," in Proc. Adv. Neural Inf. Process. Syst., 2010, pp. 1351-1359.
10. R. Mottaghi, X. Chen, X. Liu, N.-G. Cho, S.-W. Lee, S. Fidler, R. Urtasun, and A. Yuille, "The role of context for object detection and semantic segmentation in the wild," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2014, pp. 891-898.
11. H. G. Barrow and J. M. Tenenbaum, "Recovering intrinsic scene characteristics from images," Stanford Research Institute, Menlo Park, CA US, Tech. Rep. AI-TN-157, 1978.
12. D. Hoiem, A. A. Efros, and M. Hebert, "Closing the loop on scene interpretation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
13. V. Lempitsky, P. Kohli, C. Rother, and B. Sharp, "Image segmentation with a bounding box prior," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 277-284.
14. T. Brox, L. Bourdev, S. Maji, and J. Malik, "Object segmentation by alignment of poselet activations to image contours," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 2225-2232.
15. C. Gu, J. J. Lim, P. Arbelaez, and J. Malik, "Recognition using regions," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2009, pp. 1030-1037.
16. A. Torralba, K. P. Murphy, and W. T. Freeman, "Contextual models for object detection using boosted random fields," in Proc. Adv. Neural Inf. Process. Syst., 2005, pp. 1401-1408.
17. E. Sudderth, A. Torralba, W. T. Freeman, and A. Wilsky, "Learning hierarchical models of scenes, objects, and parts," in Proc. 10th IEEE Int. Conf. Comput. Vis., 2005, pp. 1331-1338.
18. S. Kumar and M. Hebert, "A hierarchical field framework for unified context-based classification," in Proc. 10th IEEE Int. Conf. Comput. Vis., 2005, pp. 1284-1291.
19. L. Ladicky, P. Sturgess, K. Alahari, C. Russell, and P. H. Torr, "What, where and how many? combining object detectors and crfs," in Proc. 11th Eur. Conf. Comput. Vis., 2010, pp. 424-437.
20. C. Wojek and B. Schiele, "A dynamic conditional random field model for joint labeling of object and scene classes," in Proc. Eur. Conf. Comput. Vis., 2008, vol. 4, pp. 733-747.
21. G. Cardinal, X. Boix, J. van de Weijer, A. D. Bagdanov, J. Serrat, and J. Gonzalez, "Harmony potentials for joint classification and segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 3280-3287.
22. J. Rivest and P. Cabanagh, "Localizing contours defined by more than one attribute," Vis. Res., vol. 36, no. 1, pp. 53-66, 1996.
23. C. C. Fowlkes, "Measuring the ecological validity of grouping and figure-ground cues," Thesis, UC Berkeley, 2005.
24. L. Fei-Fei, R. VanRullen, C. Koch, and P. Perona, "Rapid natural scene categorization in the near absence of attention," Proc. Nat. Acad. Sci. USA, vol. 99, no. 14, pp. 9596-9601, 2002.
25. T. Bachmann, "Identification of spatially quantized tachistoscopic images of faces: How many pixels does it take to carry identity?" Europ. J. Cogn. Psych., vol. 3, no. 1, pp. 87-103, 1991.
26. A. Oliva and P. G. Schyns, "Diagnostic colors mediate scene recognition," Cognitive Psychol., vol. 41, no. 2, pp. 176-210, 2000.
27. D. Parikh and C. L. Zitnick, "The role of features, algorithms and data in visual recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2010, pp. 2328-2335.
28. D. Parikh and C. Zitnick, "Finding the weakest link in person detectors," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 1425-1432.
29. D. Parikh, "Recognizing jumbled images: The role of local and global information in image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 519-526.
30. C. L. Zitnick and D. Parikh, "The role of image understanding in contour detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 622-629.
31. R. Mottaghi, S. Fidler, J. Yao, R. Urtasun, and D. Parikh, "Analyzing semantic segmentation using human-machine hybrid crfs," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2013, pp. 3143-3150.
32. L. Ladicky, C. Russell, P. H. S. Torr, and P. Kohli, "Associative hierarchical crfs for object class image segmentation," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 739-746.
33. A. Schwing, T. Hazan, M. Pollefeys, and R. Urtasun, "Distributed message passing for large scale graphical models," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2011, pp. 1833-1840.
34. T. Hazan and R. Urtasun, "A primal-dual message-passing algorithm for approximated large scale structured prediction," in Proc. Adv. Neural Inf. Process. Syst., 2010, pp. 838-846.
35. J. Shotton, J. Winn, C. Rother, and A. Criminisi, "Textonboost for image understanding: Multi-class object recognition and segmentation by jointly modeling appearance, shape and context," Int. J. Comput. Vis., vol. 81, no. 1, pp. 2-23, 2007.
36. T. Malisiewicz and A. A. Efros, "Improving spatial support for objects via multiple segmentations," in Proc. Brit. Mach. Vis. Conf., 2007, pp. 55.1-55.10.
37. J. Shotton, M. Johnson, and R. Cipolla, "Semantic texton forests for image categorization and segmentation," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-8.
38. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011.
39. P. Kohli, M. P. Kumar, and P. H. S. Torr, "p3 and beyond: Solving energies with higher order cliquess," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2007, pp. 1-8.
40. C. K. Chow and C. N. Liu, "Approximating discrete probability distributions with dependence trees," IEEE Trans. Inf. Theory, vol. IT-14, no. 3, p. 462-467, May 1968.
41. B. Hariharan, P. Arbelaez, L. Bourdev, S. Maji, and J. Malik, "Semantic contours from inverse detectors," in Proc. IEEE Int. Conf. Comput. Vis., 2011, pp. 991-998.
42. J. Carreira, R. Caseiroa, J. Batista, and C. Sminchisescu, "Semantic segmentation with second-order pooling," in Proc. 12th Eur. Conf. Comput. Vis., 2012, pp. 430-443.
43. X. Boix, J. M. Gonfaus, J. van de Weijer, A. D. Bagdanov, J. S. Gual, and J. Gonzlez, "Harmony potentials-Fusing global and local scale for semantic image segmentation," Int. J. Comput. Vis., vol. 91, no. 1, pp. 83-103, 2012.
44. M. Sun, B. soo Kim, P. Kohli, and S. Savarese, "Relating things and stuff via object property interactions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 7, pp. 1370-1383, Jul. 2014.