A smart camera with gesture recognition and SLAM capabilities for mobile robots

International Journal of Electronics

Volumn 93, Issue 6, 2006, Pages 385-401

  Bonato, V ^a   Fernandes, M M ^b   Marques, E ^a  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

^b METHODIST UNIVERSITY OF PIRACICABA   (Brazil)

Author keywords

Gesture recognition; Mobile robot; Multi camera SLAM; Neural network; Smart camera; SoC

Indexed keywords

COMPUTATIONAL COMPLEXITY; GESTURE RECOGNITION; INFORMATION TECHNOLOGY; KNOWLEDGE BASED SYSTEMS; MICROPROCESSOR CHIPS; MOBILE ROBOTS; NEURAL NETWORKS;

FAULT TOLERANT MODULE (FTM); MULTI-CAMERA SLAM; SMART CAMERA; SYSTEM-ON-A-CHIP (SOC);

CAMERAS;

EID: 33744955675     PISSN: 00207217     EISSN: 13623060     Source Type: Journal    
DOI: 10.1080/00207210600565465     Document Type: Conference Paper

References (42)

1. Altera, "Nios development kit, stratix edition". Available online at: http://www.altera.com/ products/devkits/kit-index.html (accessed October 2005).
2. ARM, "AMBA multi-layer AHB and AHB-Lite". Available online at: http://www.arm.com/products/solutions/AMBAAHBandLite.html (accessed October 2005).
3. J. Austin, RAM-based neural network, London: World Scientific Publishing Co. Pte. Ltd., 1988.
4. V. Bonato, "A Simultaneous Localization and Mapping system using multicameras for mobile robots", PhD Dissertation Proposal, December, 2005, p. 107 (in Portuguese).
5. V. Bonato, A.K. Sanches, M.M. Fernandas, J.M.P. Cardoso, E.D.V. Simoes and E. Marques, "A real time gesture recognition system for mobile robots", in International Conference on Informatics in Control, Automation and Robotics (ICINCO), pp. 207-214, 2004.
6. S. Brown and Z. Vranesic, Fundamentals of Digital Logic with VHDL Design, 2nd Ed., Toronto: McGraw Hill, 2005.
7. P.W. ChiZhang, "A new method of color image segmentation based on intensity and hue clustering", in IEEE International Conference on Pattern Recognition (ICPR), pp. 3617-3620, 2000.
8. L.F. Costa and R.M.C. Junior, Shape Analysis and Classification, New York: Phillip A. Laplante, 2000.
9. R.E. Cummings, P. Pouliquen and A.M. Lewis, "A vision chip for color segmentation and pattern matching", EURASIP JASP, pp. 703-712, 2003.
10. A.J. Davison, "Real-time simultaneous localisation and mapping with a single camera", in Proceedings of the International Conference on Computer Vision, pp. 1403-1410, 2003.
11. R.C. Gonzalez and R.E. Woods, Digital Image Processing, New York: Addison-Wesley Company, 1992.
12. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd Ed., New York: Prentice Hall, 1999.
13. C. Hu, X. Wang, M.K. Mandal, M. Meng and D. Li, "Efficient face and gesture recognition techniques for robot control", in Proc. of the Canadian Conference on Electrical and Computer Engineering (CCECE), 2003.
14. S. Kallio, J. Kela and J. Mäntyjarvi, "Online gesture recognition system for mobile interaction", in IEEE International Conference on Systems, pp. 2070-2076, 2003.
15. E.R. Kalman, "A new approach to linear filtering and prediction problems", Journal of Basic Engineering, 82, pp. 35-45, 1960.
16. M. Kawamura, N. Kakita, T. Osaki, K. Sugahara and R. Konishi, "On the hardware realization of lip reading system", in SICE Annual Conference, 2003.
17. G.H. Kim, J.S. Kim and K.S. Hong, "Vision-based simultaneous localization and mapping with two cameras", in IEEE International Conference on Intelligent Robots and Systems, pp. 3401-3406, 2005.
18. M. Kolsch and M. Turk, "Robust hand detection", in Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 614-619, 2004.
19. M. Krips, T. Lammert and A. Kummert, "FPGA implementation of a neural network for a real-time hand tracking system", in Proceedings of the First IEEE International Workshop on Electronic Design, Test and Applications (DELTA), 2002.
20. C.H. Lin, T. Lv, W. Wolf and I.B. Ozer, "A peer-to-peer architecture for distributed real-time gesture recognition", in Proceedings of International Conference on Multimedia and Exhibition, pp. 57-60, 2004.
21. P. McCurry, F. Morgan and L. Kilmartin, "Xilinx FPGA implementation of an image classifier for object detection applications", in International Conference on Image Processing, pp. 346-349, 2001.
22. D. Nistér, O. Naroditsky and J.R. Bergen, "Visual odometry", in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 652-659, 2004.
23. B. Ozer, T. Lu and W. Wolf, "Design of a real-time gesture recognition system", IEEE Signal Processing Magazine, 57, 2005.
24. Philips, "The I2C-bus specification". Available online at: http://www.semiconductors.philips.com/acrobat/literature9398/39340011.pdf (accessed October 2005).
25. O. Rogalla, M. Ehrenmann, R. Zöllner, R. Becher and R. Dillmann, "Using gesture and speech control for command a robot assistant" in IEEE Int. Workshop on Robot and Human Interactive Communication, pp. 25-27, 2002.
26. A. Rowe, C. Rosenberg and I. Nourbakhsh, "A low cost embedded color vision system", in Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2002.
27. J.C. Russ, The image processing handbook, 2nd Ed., Boca Raton: CRC Press, 1995.
28. M. Sen, I. Corretjer, F. Haim, S. Saha, J. Schlessman, S. Bhattacharyya and W. Wolf, "Computer vision on FPGAs: design methodology and its application to gesture recognition", in Proceedings of the IEEE Workshop on Embedded Computer Vision, 2005.
29. L. Simsie, "Using programmable logic for embedded systems". Available online at: http://whitepapers.osdn.com/detail/RES/1074018954_768.html (accessed January 2005).
30. R. Smith, M. Self and P. Cheeseman, "Estimating uncertain spatial relationships in robotics", Autonomous robot vehicles, pp. 167-193, 1990.
31. R.L. Swenson, "A real-time high performance universal colour transformation hardware system", PhD thesis, University of Kent, Canterbury (2002).
32. S. Thompson and S. Kagami, "Revising stereo vision maps in particle filter based SLAM using localisation confidence and sample history", in 2nd International Conference on Autonomous Robots and Agents, pp. 218-223, 2004.
33. S. Thrun, S. Thayer, W. Whittaker, C. Baker, W. Burgard, D. Ferguson, D. H́ahnel, M. Montemerlo, A. Morris, Z. Omohundro, C. Reverte and W. Whittaker, "Autonomous exploration and mapping of abandoned mines", IEEE Robotics and Automation Magazine, pp. 79-91, 2004.
34. M. Turk, "Computer vision in the interface", Multimodal Interfaces that Flex, Adapt, and Persist, 47, pp. 60-64, 2004.
35. UNECE, "World robotics". Available online at: http://www.unece.org/press/pr2004/04sta_p01e.pdf (accessed September 2005).
36. S. Waldherr, R.A. Romero and S. Thrun, "A gesture based interface for human-robot interaction", Autonomous Robots, 9, pp. 151-173, 2000.
37. W. Wolf, B. Ozer and T. Lv, "Smart cameras as embedded systems", IEEE Computer Society, pp. 48-53, 2002.
38. D.F. Wolf and G. Sukhatme, "Online simultaneous localization and mapping in dynamic environments", in IEEE International Conference on Robotics and Automation, 2004, pp. 1301-1306.
39. J. Wolf, W. Burgard and H. Burkhardt, "Robust vision-based localization by combining an image-retrieval system with monte carlo localization", IEEE Transactions on Robotics, 21, pp. 208-216, 2005.
40. Y. Wu and T.S. Huang, "Vision-based gesture recognition: a review", in Proceedings of the International Gesture Workshop on Gesture-Based Communication in Human-Computer Interaction, pp. 103-115, 1999.
41. Xilinx, "Aerospace & Defense". Available online at: http://www.xilinx.com/esp/mil_aero/index.htm (accessed November 2004).
42. P. Zemcik, "Hardware acceleration of graphics and imaging algorithms using FPGAs", in Proceedings of the 18th Spring Conference on Computer graphics, pp. 25-32, 2002.