Recurrent Neural Networks for driver activity anticipation via sensory-fusion architecture

Proceedings - IEEE International Conference on Robotics and Automation

Volumn 2016-June, Issue , 2016, Pages 3118-3125

  Jain, Ashesh ^a,b   Singh, Avi ^a   Koppula, Hema S ^a,b   Soh, Shane ^b   Saxena, Ashutosh ^a,c  

^a Department of Molecular Biology and Genetics   (United States)

^b STANFORD UNIVERSITY   (United States)

^c Brain of Things Inc   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

NETWORK ARCHITECTURE; ROBOTICS;

LONG SHORT TERM MEMORY; MULTIPLE SENSORS; RECURRENT NEURAL NETWORK (RNNS); ROBOTICS APPLICATIONS; SENSORY FUSION; SEQUENCE PREDICTION; SPATIO-TEMPORAL REASONING; STATE OF THE ART;

RECURRENT NEURAL NETWORKS;

EID: 84977476000     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2016.7487478     Document Type: Conference Paper

References (41)

1. Bosch urban. http: //bit. ly/1feM3JM. Accessed: 2015-04-23.
2. A. Andreas, P. Lenz, and R. Urtasun. Are we ready for autonomous driving the kitti vision benchmark suite. In CVPR, 2012.
3. T. Baltrusaitis, P. Robinson, and L-P. Morency. Constrained local neural fields for robust facial landmark detection in the wild. In ICCV Workshop, 2013.
4. Y. Bengio and O. Delalleau. On the expressive power of deep architectures. In Algorithmic Learning Theory, pages 18-36, 2011.
5. Y. Bengio and O. Frasconi. An input output hmm architecture. NIPS, 1995.
6. M. Bennewitz, W. Burgard, G. Cielniak, and S. Thrun. Learning motion patterns of people for compliant robot motion. IJRR, 2005.
7. Y. N. Dauphin, H. de Vries, J. Chung, and Y. Bengio. Rmsprop and equilibrated adaptive learning rates for non-convex optimization. ArXiv: 1502. 04390, 2015.
8. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. In CVPR, 2015.
9. A. Doshi, B. Morris, and M. M. Trivedi. On-road prediction of driver's intent with multimodal sensory cues. IEEE Pervasive Computing, 2011.
10. A. Dragan and S. Srinivasa. Formalizing assistive teleoperation. In RSS, 2012.
11. K. Driggs-Campbell, V. Shia, and R. Bajcsy. Improved driver modeling for human-in-the-loop vehicular control. In ICRA, 2015.
12. Y. Du, W. Wang, and L. Wang. Hierarchical recurrent neural network for skeleton based action recognition. In CVPR, 2015.
13. L. Fletcher, N. Apostoloff, L. Petersson, and A. Zelinsky. Vision in and out of vehicles. IEEE IS, 18 (3), 2003.
14. L. Fletcher, G. Loy, N. Barnes, and A. Zelinsky. Correlating driver gaze with the road scene for driver assistance systems. RAS, 52 (1), 2005.
15. A. Hannun, C. Case, J. Casper, B. Catanzaro, G. Diamos, E. Elsen, R. Prenger, S. Satheesh, S. Sengupta, and A. Coates. Deepspeech: Scaling up end-to-end speech recognition. ArXiv: 1412. 5567, 2014.
16. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9 (8), 1997.
17. M. E. Jabon, J. N. Bailenson, E. Pontikakis, L. Takayama, and C. Nass. Facial expression analysis for predicting unsafe driving behavior. IEEE Pervasive Computing, (4), 2010.
18. A. Jain. Neuralmodels. https: //github. com/ asheshjain399/NeuralModels, 2015.
19. A. Jain, H. S. Koppula, B. Raghavan, S. Soh, and A. Saxena. Car that knows before you do: Anticipating maneuvers via learning temporal driving models. In ICCV, 2015.
20. K. M. Kitani, B. D. Ziebart, J. A. Bagnell, and M. Hebert. Activity forecasting. In Proc. ECCV. 2012.
21. H. Koppula and A. Saxena. Anticipating human activities using object affordances for reactive robotic response. In RSS, 2013.
22. H. Koppula, R. Gupta, and A. Saxena. Learning human activities and object affordances from rgb-d videos. IJRR, 32 (8), 2013.
23. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
24. M. Kuderer, H. Kretzschmar, C. Sprunk, and W. Burgard. Featurebased prediction of trajectories for socially compliant navigation. In RSS, 2012.
25. J. Mainprice and D. Berenson. Human-robot collaborative manipulation planning using early prediction of human motion. In IROS, 2013.
26. D. K. Misra, J. Sung, K. Lee, and A. Saxena. Tell me dave: Contextsensitive grounding of natural language to manipulation instructions. RSS, 2014.
27. B. Morris, A. Doshi, and M. Trivedi. Lane change intent prediction for driver assistance: On-road design and evaluation. In IEEE IVS, 2011.
28. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng. Multimodal deep learning. In ICML, 2011.
29. R. Pascanu, T. Mikolov, and Y. Bengio. On the difficulty of training recurrent neural networks. ArXiv: 1211. 5063, 2012.
30. T. Rueda-Domingo, P. Lardelli-Claret, J. Luna del Castillo, J. Jimenez-Moleon, M. Garcia-Martin, and A. Bueno-Cavanillas. The influence of passengers on the risk of the driver causing a car collision in spain: Analysis of collisions from 1990 to 1999. Accident Analysis & Prevention, 2004.
31. M. S. Ryoo. Human activity prediction: Early recognition of ongoing activities from streaming videos. In ICCV, 2011.
32. J. Shi and C. Tomasi. Good features to track. In CVPR, 1994.
33. V. Shia, Y. Gao, R. Vasudevan, K. D. Campbell, T. Lin, F. Borrelli, and R. Bajcsy. Semiautonomous vehicular control using driver modeling. IEEE Trans. on ITS, 15 (6), 2014.
34. J. Sung, S. H. Jin, and A. Saxena. Robobarista: Object partbased transfer of manipulation trajectories from crowd-sourcing in 3d pointclouds. In ISRR, 2015.
35. I. Sutskever, O. Vinyals, and Q. V. Le. Sequence to sequence learning with neural networks. In NIPS, 2014.
36. A. Tawari, S. Sivaraman, M. Trivedi, T. Shannon, and M. Tippelhofer. Looking-in and looking-out vision for urban intelligent assistance: Estimation of driver attentive state and dynamic surround for safe merging and braking. In IEEE IVS, 2014.
37. M. Trivedi, T. Gandhi, and J. McCall. Looking-in and looking-out of a vehicle: Computer-vision-based enhanced vehicle safety. IEEE Trans. on ITS, 8 (1), 2007.
38. R. Vasudevan, V. Shia, Y. Gao, R. Cervera-Navarro, R. Bajcsy, and F. Borrelli. Safe semi-autonomous control with enhanced driver modeling. In ACC, 2012.
39. H. Wang and C. Schmid. Action recognition with improved trajectories. In CVPR, 2013.
40. Z. Wang, K. Mülling, M. Deisenroth, H. Amor, D. Vogt, B. Schölkopf, and J. Peters. Probabilistic movement modeling for intention inference in human-robot interaction. IJRR, 2013.
41. B. D. Ziebart, N. Ratliff, G. Gallagher, C. Mertz, K. Peterson, J. A. Bagnell, M. Hebert, A. K. Dey, and S. Srinivasa. Planning-based prediction for pedestrians. In IROS, 2009.