study on cartesian-type harvesting robot for strawberry study on ...

STUDY ONCARTESIAN-TYPEHARVESTINGROBOT FORSTRAWBERRYMasateru Nagata, Yongjie Cui,Jasper G. Tallada, Taichi KobayashiUniversity of Miyazaki, JapanSeptember 10, 2006 Sapporo3 rd IFAC, Bio-robotics III September 9-10, 2006 Sapporo convention center Sapporo Japan, Univ. of Miyazaki M.Nagata

Introduction of Strawberry Production in JapanHarvestSorting and PackingGreenhouseProduction Yield: Current Problems:200 thousand tons/year • Harvesting by handin Japan (2001)• Hard working• Long working timeWorking Time3 rd IFAC, Bio-robotics III September 9-10, 2006 Sapporo convention center Sapporo Japan, Univ. of Miyazaki M.Nagata

1.Harvesting RobotResearch in Other InstitutionsEhime University (Arima et al.)(http://mama.agr.okayama-u.ac.jp/)2. Sorting RobotIshii Industry co.Ltd (Kondo et al.)(http://web.agr.ehime-u.ac.jp/)Tochigi Prefectual Agricultural Experiment Station(http://agrinet.pref.tochigi.jp/)

Development of Strawberry Harvest Robot1. Experimental Machinejiadbgh(a) Camera 1, (b) Camera 2, (c)X-axis, (d) Y-axis, (e) Z-axis, (f)S-axis, (g) Harvest hand, (h)Ridge, (i) wheel, (f) air tank.efcStartCamera 1 takes image ofharvest areaImage processing to locateripe fruits & order of collectionMove hand with camera 2 andtake detail imageEstimate ripeness, and locationand direction of peduncleNoAMove and stop robot atharvest areaRipenessmore than45%?YesMove hand to fruit peduncleposition and rotate tocorrect directionDescend hand and monitorfiber optic sensor signalFiber optic sensoractivated?YesStop descent of handAscend & move to beltconveyorNoCut peduncle & hold the fruitYesDrop the fruit & convey outAnother fruitin the area?NoAnotherharvest area?NoFinishAYes

Development of Strawberry Harvesting Robot2. Picking handStartMoves to the cut position of peduncle inhorizontalS axis is rotated to the direction corner ofpeduncleAir handDescends to the cut position of peduncleFiberSensorDetection ofpeduncleYesThe descent is stoppedNoSafetydevice10 mmScissorsFingerHolding and cutting of the peduncleTransports to the belt conveyerDischarge of FruitsFinish

Performance Tests1. Strawberry2. Set-up of RidgeVariety: Akihime100% , 80%, 60%, 40%,UnripeDegree of RipenessRidge modelSize of ridge model3L 2L L M SSizeThe strawberry annual hilltop growing conditions for the ‘Akihime’ variety were simulatedin the laboratory by bringing in three sets of planting materials and randomly arranging theleaves and fruits onto the experimental ridge. Each set had three harvesting areas.

Performance Tests3. Measured Quantities3.1 Fruit collection efficiency of the strawberry harvestingrobot for each operation stepharvested and missed fruits, misjudgement of peduncle orientation3.2 Operation time in seconds of the strawberry harvestingrobot for each harvest set and areadriving time, time for identifying the position of fruits from thewide-view image, the time for detail image processing andcollection of each fruit3.3 Proportion of ripe fruits and size of peduncle of theharvested fruits using the strawberry harvesting robotsize of fruits, length of peduncle and profile of fruits

Fruit collection efficiency of the strawberryharvesting robot for each operation stepSetNo. of Fruitsin SetRobotVisionJudgedRipeFruitsRobotVisionDetected PeduncleHandSensorDetectedPeduncleConve-yanceofFruitsOverallEfficiency(%)Unripe Ripe Count (Eff%) Count (Eff%) Count (Eff%) Count (Eff%)1 11 26 23 (88.46) 16 (69.57) 11 (68.75) 9 (81.82) 54.052 11 26 24 (92.31) 18 (75.00) 15 (83.33) 13 (86.67) 64.863 13 28 28 (100.00) 19 (67.86) 16 (84.21) 13 (81.25) 63.41Ave 93.59 70.81 78.76 83.24 60.7745°SuccessFailureExample of poorrecognition of PeduncleThe Reason:Due to unclear direction of peduncleetc.Holding and cutting of peduncleThe Reason:Due to too steep angle of peduncle etc.

Operation time in seconds of the strawberryharvesting robot for each harvest set and areaSet Harvest Fruit Movement to Camera 1 Fruit collection TotalArea count(pcs) Harvest area(s) Processing time(s) time(s) Time(s)1 6 8.0 4.5 106.3 118.81 2 9 7.9 3.1 141.6 152.53 7 8.0 2.6 94.2 104.91 7 8.0 3.9 105.7 117.62 2 8 8.0 2.9 123.8 134.73 5 8.0 2.6 80.0 90.61 9 7.9 3.5 118.5 129.83 2 11 8.1 4.0 169.9 182.03 10 8.3 2.3 155.6 166.2Ave 8 8.0 3.3 121.7 133.0

Proportion of ripe fruits and size of peduncle ofthe harvested fruits using the strawberryharvesting robotSetProportion ofSize of PeduncleRipe Fruits(45% andLength (mm)Diameter (mm)higher)Range Mean SD Range Mean SD1 100% 9.9 - 23.2 17.3 3.8 0.6 - 1.4 0.9 0.22 100% 11.0 - 23.5 16.8 3.8 0.7 - 1.4 1.0 0.23 100% 8.5 - 29.2 17.4 4.9 0.5 - 1.9 1.1 0.4Ave 100% 8.5 - 29.2 17.2 4.1 0.5 - 1.9 1.0 0.3Sizes of Fruit:Mass: 5.8~32.7 gLength: 15.7~61.7 mmExample of harvested fruit

Conclusions1. The <strong>study</strong> confirmed that the proposed imageprocessing method was able to detect position ofstrawberry fruits, evaluate their degree of ripeness andlocate the peduncle for harvesting.2. The location and direction of the peduncle wasautomatically obtained for cutting and Holding of fruitsusing the designed picking hand.3. The operation of the robot was about five times slowerthan the manual worker.4. Irregardless of the size of the fruit, the length of thepeduncle of harvested fruit was 17.2±4.2mm.5. It is better that the development of the robot should bedone side-by-side with the development of the culturesystem.

Fruit position for testVideo of picking