Design of an eye-in-hand sensing and servo control framework for harvesting robotics in dense vegetation

Biosystems Engineering

Volumn 146, Issue , 2016, Pages 71-84

  Barth, Ruud ^a,b   Hemming, Jochen ^b   van Henten, Eldert J ^b  

^a HARVARD UNIVERSITY   (United States)

^b WAGENINGEN UNIVERSITY   (Netherlands)

Author keywords

Framework; Harvest robots; ROS; SLAM; Visual servo control

Indexed keywords

COMPUTER PROGRAMMING; CROPS; END EFFECTORS; FRUITS; HARVESTING; MAPPING; MOTION CONTROL; PNEUMATIC CONTROL EQUIPMENT; ROBOTICS; VEGETATION;

AGRICULTURAL ROBOTICS; FRAMEWORK; GREENHOUSE CONDITIONS; LABORATORY CONDITIONS; SEPARATION OF CONCERNS; SIMULTANEOUS LOCALISATION AND MAPPINGS; SLAM; VISUAL SERVO CONTROL;

VISUAL SERVOING;

EID: 84983590187     PISSN: 15375110     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.biosystemseng.2015.12.001     Document Type: Article

References (45)

1. Bac, C.W., Improving obstacle awareness for robotic harvesting of sweet-pepper. (Ph.D. thesis), 2015, Wagenigen University & Research Centre, Wageningen.
2. Bac, C.W., Hemming, J., Van Henten, E.J., Robust pixel-based classification of obstacles for robotic harvesting of sweet-pepper. Computers and Electronics in Agriculture 96 (2013), 148–162.
3. Bac, C.W., Hemming, J., Van Henten, E.J., Stem localization of sweet-pepper plants using the support wire as a visual cue. Computers and Electronics in Agriculture 105 (2014), 111–120.
4. Bachiller, M., Cerrada, J., Cerrada, C., A modular scheme for controller design and performance evaluation in 3d visual servoing. Journal of Intelligent and Robotic Systems 36 (2003), 235–264.
5. Bachiller, M., Cerrada, C., Cerrada, J.A., Designing and building controllers for 3D visual servoing applications under a modular scheme. Industrial Robotics: Programming, simulation and applications, 2006, InTech.
6. Baeten, J., Donn, K., Boedrij, S., Beckers, W., Claesen, E., Autonomous fruit picking machine: a robotic apple harvester. Laugier, C., Siegwart, R., (eds.) Field and service robotics, 2008, Springer, Berlin Heidelberg, 531–539 volume 42 of Springer Tracts in Advanced Robotics.
7. Barth, R., Baur, J., Buschmann, T., Edan, Y., Hellstrom, T., Nguyen, T., et al. Using ros for agricultural robotics : design considerations and experiences. Proceeding of the International Conference on Robotics and associated High-technologies and Equipment for Agriculture and forestry (RHEA), 2014, 509–518.
8. Bohren, J., Cousins, S., The smach high-level executive. IEEE Robotics and Automation Magazine 17 (2010), 18–20.
9. Çalli, B., Active grasp synthesis for grasping unknown objects. (Ph.D. thesis), 2015, Delft University of Technology, Delft.
10. Çalli, B., Wisse, M., Jonker, P., Grasping of unknown objects via curvature maximization using active vision. Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on, 2011, 995–1001.
11. Chaumette, F., Potential problems of stability and convergence in image-based and position-based visual servoing. Kriegman, D., Hager, G., Morse, A., (eds.) The confluence of vision and control, 1998, Springer, London, 66–78 volume 237 of Lecture Notes in Control and Information Sciences.
12. Culjak, I., Abram, D., Pribanic, T., Dzapo, H., Cifrek, M., A brief introduction to opencv. MIPRO, 2012 Proceedings of the 35th International Convention, 2012, 1725–1730.
13. De-An, Z., Jidong, L., Wei, J., Ying, Z., Yu, C., Design and control of an apple harvesting robot. Biosystems Engineering 110 (2011), 112–122.
14. Dean, T.L., Wellman, M.P., Planning and control. The Morgan Kaufmann series in representation and reasoning. 1991, M. Kaufmann Publishers, Los Altos, Calif.
15. Durrant-Whyte, H., Bailey, T., Simultaneous localization and mapping: part i. Robotics Automation Magazine, IEEE 13 (2006), 99–110.
16. Engel, J., Schps, T., Cremers, D., Lsd-slam: large-scale direct monocular slam. Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T., (eds.) Computer Vision ECCV 2014, 2014, Springer International Publishing, 834–849 volume 8690 of Lecture Notes in Computer Science.
17. Espiau, B., Chaumette, F., Rives, P., A new approach to visual servoing in robotics. Robotics and Automation, IEEE Transactions on 8 (1992), 313–326.
18. Felix, J., Ortin, F., Aspect-oriented programming to improve modularity of object-oriented applications. Journal of Software, 9, 2014.
19. Fitzgerald, C., Developing baxter. Technologies for Practical Robot Applications (TePRA), 2013 IEEE International Conference on, 2013, 1–6.
20. Forssen, P.-E., Maximally stable colour regions for recognition and matching. Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on, 2007, 1–8.
21. Han, K.-S., Kim, S.-C., Lee, Y.-B., Kim, S.-C., Im, D.-H., Choi, H.-K., et al. Strawberry harvesting robot for bench-type cultivation. Biosystems Engineering 37 (2012), 65–74.
22. Hayashi, S., Shigematsu, K., Yamamoto, S., Kobayashi, K., Kohno, Y., Kamata, J., et al. Evaluation of a strawberry-harvesting robot in a field test. Biosystems Engineering 105 (2010), 160–171.
23. Hellstrom, T., Ringdahl, O., A software framework for agricultural and forestry robots. Industrial Robot: An International Journal 40 (2013), 20–26.
24. Hemming, J., Bac, C.W., Van Tuijl, B., Barth, R., Bontsema, J., Pekkeriet, E.J., et al. A robot for harvesting sweet-pepper in greenhouses. Proceedings of the International Conference of Agricultural Engineering, 2014.
25. Hemming, J., Ruizendaal, J., Hofstee, J.W., Van Henten, E.J., Fruit detectability analysis for different camera positions in sweet-pepper. Sensors 14 (2014), 6032–6044.
26. Hemming, J., Van Tuijl, B., Gauchel, W., Wais, E., Field test of differend end-effectors for robotic harvesting of sweet-pepper. Proceedings of the the 29th International Horticultural Congress, 2014.
27. Henten, E.V., Tuijl, B.V., Hemming, J., Kornet, J., Bontsema, J., Os, E.V., Field test of an autonomous cucumber picking robot. Biosystems Engineering 86 (2003), 305–313.
28. Hutchinson, S., Hager, G., Corke, P., A tutorial on visual servo control. Robotics and Automation, IEEE Transactions on 12 (1996), 651–670.
29. Jara, C.A., Pomares, J., Candelas, F.A., Torres, F., Control framework for Dexterous manipulation using dynamic visual servoing and tactile sensors' feedback. Sensors (Basel, Switzerland) 14 (2014), 1787–1804.
30. Kitamura, S., Oka, K., Recognition and cutting system of sweet pepper for picking robot in greenhouse horticulture. Mechatronics and Automation, 2005 IEEE International Conference, Vol. 4, 2005, 1807–1812.
31. Kröger, T., On-line trajectory generation in robotic systems volume 58 of Springer tracts in advanced robotics. 2010, Springer, Berlin, Heidelberg, Germany.
32. Mahony, R., Modular design of image based visual servo control for dynamic mechanical systems. Proceedings of International Symposium on robotics research, 2011, Springer, 1–16.
33. Marchand, E., Visp: a software environment for eye-in-hand visual servoing. Proceedings of the 1999 IEEE International Conference on Robotics and Automation, Vol. 4, 1999, 3224–3229.
34. Marchand, E., Spindler, F., Chaumette, F., Visp for visual servoing: a generic software platform with a wide class of robot control skills. Robotics Automation Magazine, IEEE 12 (2005), 40–52.
35. Mehta, S., Burks, T., Vision-based control of robotic manipulator for citrus harvesting. Computers and Electronics in Agriculture 102 (2014), 146–158.
36. Murphy, R.R., Introduction to AI Robotics. 1st ed., 2000, MIT Press, Cambridge, MA, USA.
37. MVTec Software GmbH, Halcon. 2015.
38. Nguyen, H., Ciocarlie, M., Hsiao, K., Kemp, C., Ros commander: flexible behavior creation for home robots. ICRA, 2013.
39. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T.B., Leibs, J., et al. ROS: an open-source robot operating system. ICRA Workshop on open source software, 2009.
40. Rusu, R., Cousins, S., 3d is here: Point cloud library (pcl). Robotics and Automation (ICRA), 2011 IEEE International Conference on, 2011, 1–4.
41. Song, Y., Glasbey, C., Horgan, G., Polder, G., Dieleman, J., van der Heijden, G., Automatic fruit recognition and counting from multiple images. Biosystems Engineering 118 (2014), 203–215.
42. Van Henten, E.J., Hemming, J., Van Tuijl, B., Kornet, J., Meuleman, J., Bontsema, J., et al. An autonomous robot for harvesting cucumbers in greenhouses. Autonomous Robots 13 (2002), 241–258.
43. Van Tuijl, B., Wais, E., Yael, E., Methodological design of an end-effector for a horticultural robot. Proceedings of 4th Israeli Conference on Robotics, 2013.
44. Wu, H., Lou, L., Chen, C.-C., Hirche, S., Kuhnlenz, K., A framework of networked visual servo control system with distributed computation. Control Automation Robotics Vision (ICARCV), 2010 11th International Conference on, 2010, 1466–1471.
45. Yazicioglu, A., Çalli, B., Unel, M., Image based visual servoing using algebraic curves applied to shape alignment. Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, 2009, 5444–5449.