Colour-agnostic shape-based 3D fruit detection for crop harvesting robots

Biosystems Engineering

Volumn 146, Issue , 2016, Pages 57-70

  Barnea, Ehud ^a   Mairon, Rotem ^a   Ben Shahar, Ohad ^a  

^a BEN GURION UNIVERSITY OF THE NEGEV   (Israel)

Author keywords

Agrobotics; Green sweet pepper; Highlights; RGB D; Shape; Symmetry

Indexed keywords

COLOR; COMPUTER VISION; CRYSTAL SYMMETRY;

AGRICULTURAL ROBOT; AGROBOTICS; COMPUTER VISION SYSTEM; HIGHLIGHTS; ILLUMINATION CONDITIONS; REFLECTIVE SYMMETRY; SHAPE; SWEET PEPPER;

FRUITS;

EID: 84958012823     PISSN: 15375110     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.biosystemseng.2016.01.013     Document Type: Article

References (62)

1. Adato, Y., Ben-Shahar, O., Specular flow and shape in one shot. British Machine Vision Conference, 2011, 1–11.
2. Adato, Y., Vasilyev, Y., Zickler, T., Ben-Shahar, O., Shape from specular flow. IEEE Transactions on Pattern Analysis and Machine Intelligence 32:11 (2010), 2054–2070.
3. Aldoma, A., Vincze, M., Blodow, N., Gossow, D., Gedikli, S., Rusu, R., et al. Cad-model recognition and 6DOF pose estimation using 3D cues. International Conference on Computer Vision Workshops (ICCV Workshops), 2011, 585–592.
4. Barnea, E., Ben-Shahar, O., Depth based object detection from partial pose estimation of symmetric objects. Proceedings of the European Conference on Computer Vision, 2014, 377–390.
5. Beckmann, P., Spizzichino, A., The scattering of electromagnetic waves from rough surfaces. 1963, Pergamon Press.
6. Bishop, C.M., Pattern recognition and machine learning. 2007, Springer-Verlag, New York.
7. Bo, L., Lai, K., Ren, X., Fox, D., Object recognition with hierarchical kernel descriptors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2011, 1729–1736.
8. Bradski, G., The opencv library. Dr. Dobb's Journal of Software Tools 25 (2000), 120–126.
9. Carson, C., Thomas, M., Belongie, S., Hellerstein, J., Malik, J., Blobworld: a system for region-based image indexing and retrieval. Visual information and information systems, 1999, Springer, 509–517.
10. Chang, C.-C., Lin, C.-J., LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2:27 (2011), 1–27.
11. Chi, Y., Ling, P., Fast fruit identification for robotic tomato picker. ASAE Annual International Meeting, 2004, American Society of Agricultural and Biological Engineers, 1.
12. Cristianini, N., Shawe-Taylor, J., An introduction to support vector machines and other kernel-based learning methods. 2000, Cambridge University Press, New York, NY, USA.
13. Dalal, N., Triggs, B., Histograms of oriented gradients for human detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2005, 886–893.
14. Do Carmo, M.P., Do Carmo, M.P., Differential geometry of curves and surfaces, Vol. 2, 1976, Prentice-hall, Englewood Cliffs.
15. Edan, Y., Rogozin, D., Flash, T., Miles, G., Robotic melon harvesting. IEEE Transactions on Robotics and Automation 16 (2000), 831–835.
16. Everingham, M., Van Gool, L., Williams, C., Winn, J., Zisserman, A., The Pascal visual object classes (voc) challenge. International Journal of Computer Vision 88 (2010), 303–338.
17. Felzenszwalb, P., McAllester, D., Ramanan, D., A discriminatively trained, multiscale, deformable part model. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1–8.
18. Hannan, M., Burks, T., Current developments in automated citrus harvesting. ASAE Annual International Meeting, 2004.
19. Harrell, R., Slaughter, D., Adsit, P., A fruit-tracking system for robotic harvesting. Machine Vision and Applications 2 (1989), 69–80.
20. Hinterstoisser, S., Holzer, S., Cagniart, C., Ilic, S., Konolige, K., Navab, N., et al. Multimodal templates for real-time detection of textureless objects in heavily cluttered scenes. Proceedings of the IEEE International Conference on Computer Vision, 2011, 858–865.
21. Janoch, A., Karayev, S., Jia, Y., Barron, J., Fritz, M., Saenko, K., et al. A category-level 3-D object dataset: putting the Kinect to work. Consumer depth cameras for computer vision, 2011, Springer, London, 141–165.
22. Jimenez, A., Ceres, R., Pons, J., A survey of computer vision methods for locating fruit on trees. Transactions of the ASAE-American Society of Agricultural Engineers 43:6 (2000), 1911–1920.
23. Jimenez, A., Ceres, R., Pons, J., A vision system based on a laser range-finder applied to robotic fruit harvesting. Machine Vision and Applications 11 (2000), 321–329.
24. Kapach, K., Barnea, E., Mairon, R., Edan, Y., Ben-Shahar, O., Computer vision for fruit harvesting robots – state of the art and challenges ahead. International Journal of Computer Vision and Robotics 3 (2012), 4–34.
25. Kim, B., Xu, S., Savarese, S., Accurate localization of 3D objects from RGB-D data using segmentation hypotheses. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2013, 886–893.
26. Lai, K., Bo, L., Ren, X., Fox, D., A large-scale hierarchical multi-view RGB-D object dataset. Proceedings of the IEEE International Conference on Robotics and Automation, 2011, 1817–1824.
27. Lai, K., Bo, L., Ren, X., Fox, D., Detection-based object labeling in 3D scenes. Proceedings of the IEEE International Conference on Robotics and Automation, 2012, 1330–1337.
28. Lee, S., Bajcsy, R., Detection of specularity using color and multiple views. Proceedings of the European Conference on Computer Vision, 1992, 99–114.
29. Lee, S., Liu, Y., Curved glide-reflection symmetry detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (2012), 266–278.
30. Levkowitz, H., Color theory and modeling for computer graphics, visualization, and multimedia applications, Vol. 402, 1997, Springer Science & Business Media. Springer, US.
31. Lin, Z., Davis, L., A pose-invariant descriptor for human detection and segmentation. Proceedings of the European Conference on Computer Vision, 2008, 423–436.
32. Lin, S., Li, Y., Kang, S., Tong, X., Shum, H., Diffuse-specular separation and depth recovery from image sequences. Computer Vision European Conference on Computer Vision, 2002, Springer, 210–224.
33. Lin, S., Shum, H., Separation of diffuse and specular reflection in color images. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, 2001, IEEE, I–341.
34. Loy, G., Eklundh, J., Detecting symmetry and symmetric constellations of features. Proceedings of the European Conference on Computer Vision, 2006, 508–521.
35. Mallick, S., Zickler, T., Belhumeur, P., Kriegman, D., Specularity removal in images and videos: a PDE approach. Proceedings of the European Conference on Computer Vision, 2006, Springer, 550–563.
36. Mallick, S., Zickler, T., Kriegman, D., Belhumeur, P., Beyond Lambert: reconstructing specular surfaces using color. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, 2005, 619–626.
37. Mitra, N., Pauly, M., Wand, M., Ceylan, D., Symmetry in 3D geometry: extraction and applications. Computer Graphics Forum 32:6 (2012), 1–23.
38. Monta, M., Namba, K., Three-dimensional sensing system for agricultural robots. Proceedings of the IEEE International Conference on Advanced Intelligent Mechatronics, Vol. 2, 2003, 1216–1221.
39. Nayar, S.K., Fang, X.-S., Boult, T., Separation of reflection components using color and polarization. International Journal of Computer Vision 21 (1997), 163–186.
40. Netz, A., Osadchy, M., Using specular highlights as pose invariant features for 2D-3D pose estimation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2011, IEEE, 721–728.
41. Osadchy, M., Jacobs, D., Ramamoorthi, R., Using specularities for recognition. Proceedings of the IEEE International Conference on Computer Vision, 2003, IEEE, 1512–1519.
42. Park, M., Lee, S., Chen, P., Kashyap, S., Butt, A., Liu, Y., Performance evaluation of state-of-the-art discrete symmetry detection algorithms. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2008, 1–8.
43. Park, J., Tou, J., Highlight separation and surface orientations for 3-D specular objects. Pattern recognition, Vol. 1, 1990, IEEE, 331–335.
44. Podolak, J., Shilane, P., Golovinskiy, A., Rusinkiewicz, S., Funkhouser, T., A planar-reflective symmetry transform for 3D shapes. ACM Transactions on Graphics 25 (2006), 549–559.
45. Raviv, D., Bronstein, A., Bronstein, M., Kimmel, R., Symmetries of non-rigid. Proceedings of the IEEE International Conference on Computer Vision, 2007, 1–7.
46. Redondo-Cabrera, C., López-Sastre, R., Acevedo-Rodriguez, J., Maldonado-Bascón, S., Surfing the point clouds: selective 3D spatial pyramids for category-level object recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2012, 3458–3465.
47. Rosen, J., Symmetry discovered – Concepts and applications in nature and science. 2011, Dover Publications.
48. Rusu, R.B., Cousins, S., 3D is here: Point Cloud Library (PCL). Proceedings of the IEEE International Conference on Robotics and Automation, 2011, 1–4.
49. Saenko, K., Karayev, S., Jia, Y., Shyr, A., Janoch, A., Long, J., et al. Practical 3-D object detection using category and instance-level appearance models. IEEE International Workshop on Intelligent Robots and Systems, 2011, 1817–1824.
50. Sato, Y., Ikeuchi, K., Temporal-color space analysis of reflection. Journal of the Optical Society of America A 11 (1994), 2990–3002.
51. Shen, H., Cai, Q., Simple and efficient method for specularity removal in an image. Applied Optics 48 (2009), 2711–2719.
52. Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., et al. Real-time human pose recognition in parts from single depth images. Communication of the ACM 56 (2013), 116–124.
53. Silberman, N., Fergus, R., Indoor scene segmentation using a structured light sensor. IEEE International Conference on Computer Vision Workshops (ICCV Workshops), 2011, 601–608.
54. Spinello, L., Arras, K., People detection in RGB-D data. IEEE International Workshop on Intelligent Robots and Systems, 2011, 3838–3843.
55. Tang, S., Wang, X., Lv, X., Han, T., Keller, J., He, Z., et al. Histogram of oriented normal vectors for object recognition with a depth sensor. Proceedings of the Asian Conference on Computer Vision, 2012, Springer Berlin Heidelberg, 525–538.
56. Tan, R., Ikeuchi, K., Reflection components decomposition of textured surfaces using linear basis functions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 1, 2005, IEEE, 125–131.
57. Tan, R., Ikeuchi, K., Separating reflection components of textured surfaces using a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (2005), 178–193.
58. Tan, P., Quan, L., Lin, S., Separation of highlight reflections on textured surfaces. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vol. 2, 2006, IEEE, 1855–1860.
59. Thrun, S., Wegbreit, B., Shape from symmetry. Proceedings of the IEEE International Conference on Computer Vision, 2005, 1824–1831.
60. Vasilyev, Y., Zickler, T., Gortler, S., Ben-Shahar, O., Shape from specular flow: is one flow enough?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2011, 2561–2568.
61. Wertheimer, M., Untersuchungen zur lehre von der gestalt. Psychologische Forschung 4 (1923), 301–350.
62. Zhao, P., Quan, L., Translation symmetry detection in a fronto-parallel view. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2011, 1009–1016.