“What happens if…” learning to predict the effect of forces in images

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 9908 LNCS, Issue , 2016, Pages 269-285

  Mottaghi, Roozbeh ^a   Rastegari, Mohammad ^a   Gupta, Abhinav ^a,b   Farhadi, Ali ^a,c  

^a Allen Institute for Artificial Intelligence   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Forces; Motion estimation; Recurrent neural networks; Scene understanding

Indexed keywords

DEEP NEURAL NETWORKS; FORECASTING; LARGE DATASET; MOTION ESTIMATION;

EXPERIMENTAL EVALUATION; FORCES; LARGE-SCALE DATASET; NEURAL NETWORK MODEL; PHYSICAL MOVEMENTS; SCENE UNDERSTANDING; SEQUENTIAL DEPENDENCIES; SEQUENTIAL MOVEMENTS;

RECURRENT NEURAL NETWORKS;

EID: 84990038863     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-319-46493-0_17     Document Type: Conference Paper

References (44)

1. Battaglia, P., Hamrick, J., Tenenbaum, J.B.: Simulation as an engine of physical scene understanding. PNAS 110, 18327-18332 (2013)
2. Bhat, K.S., Seitz, S.M., Popović, J., Khosla, P.K.: Computing the physical parameters of rigid-body motion from video. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 551-565. Springer, Heidelberg (2002). doi:10.1007/3-540-47969-4-37
3. Brubaker, M.A., Sigal, L., Fleet, D.J.: Estimating contact dynamics. In: ICCV (2009)
4. Choi, W., Chao, Y.W., Pantofaru, C., Savarese, S.: Understanding indoor scenes using 3d geometric phrases. In: CVPR (2013)
5. Eigen, D., Puhrsch, C., Fergus, R.: Depth map prediction from a single image using a multi-scale deep network. In: NIPS (2014)
6. Everingham, M., Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (voc) challenge. IJCV 88, 303-338 (2010)
7. Fouhey, D.F., Zitnick, C.: Predicting object dynamics in scenes. In: CVPR (2014)
8. Fragkiadaki, K., Agrawal, P., Levine, S., Malik, J.: Learning predictive visual models of physics for playing billiards. In: ICLR (2016)
9. Gupta, A., Efros, A.A., Hebert, M.: Blocks world revisited: image understanding using qualitative geometry and mechanics. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 482-496. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15561-1-35
10. Hamrick, J., Battaglia, P., Tenenbaum, J.B.: Internal physics models guide probabilistic judgments about object dynamics. In: Annual Meeting of the Cognitive Science Society (2011)
11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)
12. Heitz, G., Gould, S., Saxena, A., Koller, D.: Cascaded classification models: Combining models for holistic scene understanding. In: NIPS (2008)
13. Jia, Z., Gallagher, A., Saxena, A., Chen, T.: 3d-based reasoning with blocks, support, and stability. In: CVPR (2013)
14. Jiang, Y., Lim, M., Zheng, C., Saxena, A.: Learning to place new objects in a scene. IJRR 31, 1021-1043 (2012)
15. Karpathy, A., Fei-Fei, L.: Deep visual-semantic alignments for generating image descriptions. In: CVPR (2015)
16. Kitani, K.M., Ziebart, B.D., Bagnell, J.A., Hebert, M.: Activity forecasting. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7575, pp. 201-214. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33765-9_15
17. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS (2012)
18. Le, Q.V., Jaitly, N., Hinton, G.E.: A simple way to initialize recurrent networks of rectified linear units. In: ArXiv (2015)
19. Levine, S., Finn, C., Darrell, T., Abbeel, P.: End-to-end training of deep visuomotor policies. In: ArXiv (2015)
20. Li, L.J., Socher, R., Fei-Fei, L.: Towards total scene understanding: Classification, annotation and segmentation in an automatic framework. In: CVPR (2009)
21. Lin, D., Fidler, S., Urtasun, R.: Holistic scene understanding for 3d object detection with rgbd cameras. In: ICCV (2013)
22. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740-755. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10602-1-48
23. Michalski, V., Memisevic, R., Konda, K.: Modeling deep temporal dependencies with recurrent grammar cells. In: NIPS (2014)
24. Mottaghi, R., Bagherinezhad, H., Rastegari, M., Farhadi, A.: Newtonian image understanding: Unfolding the dynamics of objects in static images. In: CVPR (2016)
25. Murphy, K., Torralba, A., Freeman, W.T.: Using the forest to see the trees: a graphical model relating features, objects, and scenes. In: NIPS (2003)
26. Oh, J., Guo, X., Lee, H., Lewis, R.L., Singh, S.P.: Action-conditional video prediction using deep networks in atari games. In: NIPS (2015)
27. Pintea, S.L., Gemert, J.C., Smeulders, A.W.M.: Déjá Vu: motion prediction in static. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 172-187. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10578-9_12
28. Ranzato, M., Szlam, A., Bruna, J., Mathieu, M., Collobert, R., Chopra, S.: Video (language) modeling: a baseline for generative models of natural videos. In: ArXiv (2014)
29. Rastegari, M., Keskin, C., Kohli, P., Izadi, S.: Computationally bounded retrieval. In: CVPR (2015)
30. Salzmann, M., Urtasun, R.: Physically-based motion models for 3d tracking: a convex formulation. In: ICCV (2011)
31. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746-760. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33715-4_54
32. Song, S., Lichtenberg, S.P., Xiao, J.: Sun rgb-d: A rgb-d scene understanding benchmark suite. In: CVPR (2015)
33. Sutskever, I., Hinton, G.E., Taylor, G.W.: The recurrent temporal restricted boltzmann machine. In: NIPS (2008)
34. Vinyals, O., Toshev, A., Bengio, S., Erhan, D.: Show and tell: A neural image caption generator. In: CVPR (2015)
35. Vondrak, M., Sigal, L., Jenkins, O.C.: Physical simulation for probabilistic motion tracking. In: CVPR (2008)
36. Walker, J., Gupta, A., Hebert, M.: Patch to the future: Unsupervised visual prediction. In: CVPR (2014)
37. Walker, J., Gupta, A., Hebert, M.: Dense optical flow prediction from a static image. In: ICCV (2015)
38. Wang, X., Fouhey, D.F., Gupta, A.: Designing deep networks for surface normal estimation. In: CVPR (2015)
39. Wu, J., Yildirim, I., Lim, J.J., Freeman, W.T., Tenenbaum, J.B.: Galileo: Perceiving physical object properties by integrating a physics engine with deep learning. In: NIPS (2015)
40. Yao, J., Fidler, S., Urtasun, R.: Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation. In: CVPR (2012)
41. Yuen, J., Torralba, A.: A data-driven approach for event prediction. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 707-720. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15552-9-51
42. Zhang, Y., Song, S., Tan, P., Xiao, J.: PanoContext: a whole-room 3D context model for panoramic scene understanding. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 668-686. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10599-4_43
43. Zheng, B., Zhao, Y., Yu, J.C., Ikeuchi, K., Zhu, S.C.: Beyond point clouds: Scene understanding by reasoning geometry and physics. In: CVPR (2013)
44. Zheng, B., Zhao, Y., Yu, J.C., Ikeuchi, K., Zhu, S.C.: Detecting potential falling objects by inferring human action and natural disturbance. In: ICRA (2014)