Beyond Trade-Off: An Optimized Binocular Stereo Vision Based Depth Estimation Algorithm for Designing Harvesting Robot in Orchards

Li Zhang, Qun Hao, Yefei Mao, Jianbin Su and Jie Cao ()
Additional contact information
Li Zhang: Key Laboratory of Biomimetic Robots and Systems, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
Qun Hao: Key Laboratory of Biomimetic Robots and Systems, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
Yefei Mao: China Academy of Aerospace Science and Innovation, Beijing 100176, China
Jianbin Su: Beijing Special Engineering Design and Research Institute, Beijing 101599, China
Jie Cao: Key Laboratory of Biomimetic Robots and Systems, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China

Agriculture, 2023, vol. 13, issue 6, 1-18

Abstract: Depth estimation is one of the bottleneck parts for harvesting robots to determine whether the operation of grasping or picking succeeds or not directly. This paper proposed a novel disparity completion method combined with bilateral filtering and pyramid fusion to improve the issues of incorrect outputs due to the missed or wrong matching when achieving 3D position from 2D images in open-world environments. Briefly, our proposed method has two significant advantages in general. Firstly, occlusion between leaves, branches, and fruits is a universal phenomenon in unstructured orchard environments, which results in the most depth estimation algorithms facing great challenges to obtain accurate outputs in these occluded regions. To alleviate these issues, unlike other research efforts that already exist, we optimized the semi-global matching algorithm to obtain high accuracy sparse values as an initial disparity map; then, an improved bilateral filtering algorithm is proposed to eliminate holes and discontinuous regions caused by occlusion to obtain precise and density disparity outputs. Secondly, due to taking the practical high-efficiency requirements of the automated harvesting robot in its working status into consideration, we attempted to merge multiple low-resolution bilateral filtering results through the pyramid fusion model which goes beyond the trade-off mechanism to improve the performance of both accuracy and time cost. Finally, a prototype harvesting robot was designed to conduct experiments at three kinds of different distances (0.6~0.75 m, 1~1.2 m, and 1.6~1.9 m). Experiment results showed that our proposed method achieved density disparity maps and eliminated holes and discontinuous defects in the disparity map effectively. The average absolute error of our proposed method is 3.2 mm, and the average relative error is 1.79%. In addition, the time cost is greatly reduced more than 90%. Comprehensive experimental results demonstrate that our proposed algorithm provides a potential possibility for designing harvesting.

Keywords: design harvesting robots; binocular stereo camera; depth estimation (search for similar items in EconPapers)
JEL-codes: Q1 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2077-0472/13/6/1117/pdf (application/pdf)
https://www.mdpi.com/2077-0472/13/6/1117/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jagris:v:13:y:2023:i:6:p:1117-:d:1155370

Access Statistics for this article

Agriculture is currently edited by Ms. Leda Xuan

More articles in Agriculture from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().