On-tree mango fruit size estimation using RGB-D images

Sensors (Switzerland)

Volumn 17, Issue 12, 2017, Pages

  Wang, Zhenglin ^a   Walsh, Kerry B ^a   Verma, Brijesh ^a  

^a CENTRAL QUEENSLAND UNIVERSITY   (Australia)

Author keywords

Allometry; Fruit size; Machine vision; Precision fruiticulture; RGB D camera; Time of flight

Indexed keywords

CAMERAS; COLOR; COMPUTER VISION; COSTS; MEAN SQUARE ERROR; RANGE FINDERS; STEREO IMAGE PROCESSING; STEREO VISION;

ALLOMETRY; FRUIT SIZE; PRECISION FRUITICULTURE; RGB-D CAMERAS; TIME OF FLIGHT;

FRUITS;

COLOR; FRUIT; MANGIFERA; PLANT LEAF; TREE;

COLOR; FRUIT; MANGIFERA; PLANT LEAVES; TREES;

EID: 85046894339     PISSN: 14248220     EISSN: None     Source Type: Journal    
DOI: 10.3390/s17122738     Document Type: Article

References (48)

1. Jenni, S.; Stewart, K.A.; Bourgeois, G.; Cloutier, D.C. Predicting yield and time to maturity of muskmelons from weather and crop observations. J. Am. Soc. Hortic. Sci. 1998, 123, 195-201.
2. Jha, S.; Kingsly, A.; Chopra, S. Physical and mechanical properties of mango during growth and storage for determination of maturity. J. Food Eng. 2006, 72, 73-76. [CrossRef]
3. Mitchell, P. Pear fruit growth and the use of diameter to estimate fruit volume and weight. HortScience 1986, 21, 1003-1005.
4. Moreda, G.P.; Ortiz-Cañavate, J.; García-Ramos, F.J.; Ruiz-Altisent, M. Non-destructive technologies for fruit and vegetable size determination—A review. J. Food Eng. 2009, 92, 119-136. [CrossRef]
5. Jenni, S.; Stewart, K.A.; Bourgeois, G.; Cloutier, D.C. Nondestructive volume estimation for growth analysis of eastern-type muskmelon ovaries. HortScience 1996, 31, 1236.
6. Ngouajio, M.; Kirk, W.; Goldy, R. A simple model for rapid and nondestructive estimation of bell pepper fruit volume. HortScience 2003, 38, 509-511.
7. Léchaudel, M.; Joas, J. An overview of preharvest factors influencing mango fruit growth, quality and postharvest behaviour. Braz. J. Plant Physiol. 2007, 19, 287-298. [CrossRef]
8. Anderson, N.T.; Subedi, P.P.; Walsh, K.B. Manipulation of mango fruit dry matter content to improve eating quality. Sci. Hortic. 2017, 226, 316-321. [CrossRef]
9. Kernot, I.; Meurant, N.; Holmes, R.; MacLeod, N.; Fullelove, G.; Bally, I. Mango Information Kit; Queensland Department of Primary Industries: Brisbane, Australia, 1999.
10. Spreer, W.; Müller, J. Estimating the mass of mango fruit (Mangifera indica, cv. Chok Anan) from its geometric dimensions by optical measurement. Comput. Electron. Agric. 2011, 75, 125-131.
11. Yimyam, P.; Chalidabhongse, T.; Sirisomboon, P.; Boonmung, S. Physical Properties Analysis of Mango Using Computer Vision. In Proceedings of the 2005 International Conference on Control, Automation and Systems (ICCAS), Gyeonggi-Do, Korea, 2-5 June 2005.
12. Chalidabhongse, T.; Yimyam, P.; Sirisomboon, P. 2D/3D Vision-Based Mango’s Feature Extraction and Sorting. In Proceedings of the 9th International Conference on Control, Automation, Robotics and Vision, Singapore, 5-8 December 2006; pp. 1-6.
13. Gongal, A.; Amatya, S.; Karkee, M.; Zhang, Q.; Lewis, K. Sensors and systems for fruit detection and localization: A review. Comput. Electron. Agric. 2015, 116, 8-19. [CrossRef]
14. Payne, A.; Walsh, K. Machine vision in estimation of fruit crop yield. In Plant Image Analysis: Fundamentals and Applications; CRC Press: Boca Raton, FL, USA, 2014; pp. 329-374.
15. Qureshi, W.S.; Payne, A.; Walsh, K.B.; Linker, R.; Cohen, O.; Dailey, M.N. Machine vision for counting fruit on mango tree canopies. Precis. Agric. 2017, 18, 224-244. [CrossRef]
16. Stein, M.; Bargoti, S.; Underwood, J. Image Based Mango Fruit Detection, Localisation and Yield Estimation Using Multiple View Geometry. Sensors 2016, 16, 1915. [CrossRef] [PubMed]
17. Sa, I.; Ge, Z.; Dayoub, F.; Upcroft, B.; Perez, T.; McCool, C. DeepFruits: A Fruit Detection System Using Deep Neural Networks. Sensors 2016, 16, 1222. [CrossRef] [PubMed]
18. Payne, A.; Walsh, K.; Subedi, P.; Jarvis, D. Estimating mango crop yield using image analysis using fruit at ‘stone hardening’ stage and night time imaging. Comput. Electron. Agric. 2014, 100, 160-167. [CrossRef]
19. Payne, A.B.; Walsh, K.B.; Subedi, P.P.; Jarvis, D. Estimation of mango crop yield using image analysis—Segmentation method. Comput. Electron. Agric. 2013, 91, 57-64. [CrossRef]
20. Dalal, N.; Triggs, B. Histograms of Oriented Gradients for Human Detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA, 20-25 June 2005; IEEE: Piscataway, NJ, USA, 2005; pp. 886-893.
21. Cheng, H.; Damerow, L.; Sun, Y.; Blanke, M. Early Yield Prediction Using Image Analysis of Apple Fruit and Tree Canopy Features with Neural Networks. J. Imaging 2017, 3, 6. [CrossRef]
22. Regunathan, M.; Lee, W.S. Citrus Fruit Identification and Size Determination Using Machine Vision and Ultrasonic Sensors. In Proceedings of the 2005 ASAE Annual International Meeting, Tampa, FL, USA, 17-20 July 2005.
23. Seitz, S.M.; Curless, B.; Diebel, J.; Scharstein, D.; Szeliski, R. A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), New York, NY, USA, 17-22 June 2006; pp. 519-528.
24. Kytö, M.; Nuutinen, M.; Oittinen, P. Method for Measuring Stereo Camera Depth Accuracy Based on Stereoscopic Vision; IS&T/SPIE Electronic Imaging; International Society for Optics and Photonics: San Francisco, CA, USA, 2011.
25. Yang, Q. Stereo Matching Using Tree Filtering. IEEE Trans. Pattern Anal. Mach. Intell. 2015, 37, 834-846. [CrossRef] [PubMed]
26. Font, D.; Pallejà, T.; Tresanchez, M.; Runcan, D.; Moreno, J.; Martínez, D.; Teixidó, M.; Palacín, J. A Proposal for Automatic Fruit Harvesting by Combining a Low Cost Stereovision Camera and a Robotic Arm. Sensors 2014, 14, 11557-11579. [CrossRef] [PubMed]
27. Kazmi, W.; Foix, S.; Alenyà, G.; Andersen, H.J. Indoor and outdoor depth imaging of leaves with time-of-flight and stereo vision sensors: Analysis and comparison. ISPRS J. Photogramm. Remote Sens. 2014, 88, 128-146. [CrossRef]
28. Horaud, R.; Hansard, M.; Evangelidis, G.; Ménier, C. An overview of depth cameras and range scanners based on time-of-flight technologies. Mach. Vis. Appl. 2016, 27, 1005-1020. [CrossRef]
29. Feigin, M.; Bhandari, A.; Izadi, S.; Rhemann, C.; Schmidt, M.; Raskar, R. Resolving multipath interference in kinect: An inverse problem approach. IEEE Sens. J. 2015, 16, 3419-3427. [CrossRef]
30. Paulus, S.; Behmann, J.; Mahlein, A.-K.; Plümer, L.; Kuhlmann, H. Low-Cost 3D Systems Suitable Tools for Plant Phenotyping. Sensors 2014, 14, 3001-3018. [CrossRef] [PubMed]
31. Nguyen, T.T.; Vandevoorde, K.; Wouters, N.; Kayacan, E.; De Baerdemaeker, J.G.; Saeys, W. Detection of red and bicoloured apples on tree with an RGB-D camera. Biosyst. Eng. 2016, 146, 33-44. [CrossRef]
32. Andújar, D.; Ribeiro, A.; Fernández-Quintanilla, C.; Dorado, J. Using depth cameras to extract structural parameters to assess the growth state and yield of cauliflower crops. Comput. Electron. Agric. 2016, 122, 67-73. [CrossRef]
33. McCool, C.; Sa, I.; Dayoub, F.; Lehnert, C.; Perez, T.; Upcroft, B. Visual Detection of Occluded Crop: For Automated Harvesting. In Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, 16-21 May 2016; pp. 2506-2512.
34. Zhu, Q.; Yeh, M.-C.; Cheng, K.-T.; Avidan, S. Fast Human Detection Using a Cascade of Histograms of Oriented Gradients. In Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA, 17-22 June 2006; pp. 1491-1498.
35. Lachat, E.; Macher, H.; Landes, T.; Grussenmeyer, P. Assessment and calibration of a RGB-D camera (kinect v2 sensor) towards a potential use for close-range 3D modeling. Remote Sens. 2015, 7, 13070-13097. [CrossRef]
36. Pagliari, D.; Pinto, L. Calibration of kinect for xbox one and comparison between the two generations of Microsoft sensors. Sensors 2015, 15, 27569-27589. [CrossRef] [PubMed]
37. Fankhauser, P.; Bloesch, M.; Rodriguez, D.; Kaestner, R.; Hutter, M.; Siegwart, R. Kinect v2 for Mobile Robot Navigation: Evaluation and Modeling. In Proceedings of the International Conference on Advanced Robotics (ICAR), Istanbul, Turkey, 27-31 July 2015; pp. 388-394.
38. Sarbolandi, H.; Lefloch, D.; Kolb, A. Kinect range sensing: Structured-light versus Time-of-Flight Kinect. Comput. Vis. Image Underst. 2015, 139, 1-20. [CrossRef]
39. Almeida, L.; Vasconcelos, F.; Barreto, J.; Menezes, P.; Dias, J. On-line incremental 3D human body reconstruction for HMI or AR applications. In Field Robotics;World Scientific: Singapore, 2011; pp. 355-362. [CrossRef]
40. Otsu, N. A threshold selection method from gray-level histograms. Automatica 1975, 11, 23-27. [CrossRef]
41. Linker, R.; Cohen, O.; Naor, A. Determination of the number of green apples in RGB images recorded in orchards. Comput. Electron. Agric. 2012, 81, 45-57. [CrossRef]
42. Kilian, J. Simple image analysis by moments. In OpenCV Library Documentation. Available online: https://www.scribd.com/document/39759766/Simple-Image-Analysis-by-Moments (accessed on 27 November 2017).
43. Voss, K.; Suesse, H.; Neubauer, R. Moment-Based Invariant Fitting of Elliptical Segments. In Proceedings of the International Conference on Computer Analysis of Images and Patterns, Prague, Czech Republic, 6-8 September 1995; Springer: Berlin, Germany, 1995; pp. 562-567.
44. Rosin, P.L. Measuring Shape: Ellipticity, Rectangularity, and Triangularity. In Proceedings of the 15th International Conference on Pattern Recognition, Barcelona, Spain, 3-7 September 2000; IEEE: Piscataway, NJ, USA, 2000; pp. 952-955.
45. Stojmenovic, M.; Nayak, A. Direct Ellipse Fitting and Measuring Based on Shape Boundaries. In Proceedings of the Pacific-Rim Symposium on Image and Video Technology, Santiago de chile, Chile, 17-19 December 2007; Springer: Berlin, Germany, 2007; pp. 221-235.
46. Corti, A.; Giancola, S.; Mainetti, G.; Sala, R. A metrological characterization of the Kinect V2 time-of-flight camera. Robot. Auton. Syst. 2016, 75 Pt B, 584-594. [CrossRef]
47. Zude, M.; Pflanz, M.; Spinelli, L.; Dosche, C.; Torricelli, A. Non-destructive analysis of anthocyanins in cherries by means of Lambert-Beer and multivariate regression based on spectroscopy and scatter correction using time-resolved analysis. J. Food Eng. 2011, 103, 68-75. [CrossRef]
48. Sengupta, S.; Lee, W.S. Identification and determination of the number of immature green citrus fruit in a canopy under different ambient light conditions. Biosyst. Eng. 2014, 117, 51-61. [CrossRef]