Real-time grasping strategies using event camera

Xiaoqian Huang (), Mohamad Halwani (), Rajkumar Muthusamy (), Abdulla Ayyad (), Dewald Swart (), Lakmal Seneviratne (), Dongming Gan () and Yahya Zweiri ()
Additional contact information
Xiaoqian Huang: Khalifa University
Mohamad Halwani: Khalifa University
Rajkumar Muthusamy: Dubai Future Labs
Abdulla Ayyad: Khalifa University
Dewald Swart: Strata Manufacturing PJSC
Lakmal Seneviratne: Khalifa University
Dongming Gan: Purdue University
Yahya Zweiri: Khalifa University

Journal of Intelligent Manufacturing, 2022, vol. 33, issue 2, No 12, 593-615

Abstract: Abstract Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. With advantages of microsecond-level sampling rate and no motion blur of event camera, the model-based and model-free approaches are developed for known and unknown objects’ grasping respectively. The event-based multi-view approach is used to localize the objects in the scene in the model-based approach, and then point cloud processing is utilized to cluster and register the objects. The proposed model-free approach, on the other hand, utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. Using a UR10 robot with an eye-in-hand neuromorphic camera and a Barrett hand gripper, the proposed approaches are experimentally validated with objects of different sizes. Furthermore, it demonstrates robustness and a significant advantage over grasping with a traditional frame-based camera in low-light conditions.

Keywords: Neuromorphic vision; Model-based grasping; Model-free grasping; Multi-object grasping; Event camera (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10845-021-01887-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joinma:v:33:y:2022:i:2:d:10.1007_s10845-021-01887-9

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10845

DOI: 10.1007/s10845-021-01887-9

Access Statistics for this article

Journal of Intelligent Manufacturing is currently edited by Andrew Kusiak

More articles in Journal of Intelligent Manufacturing from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().