A hierarchical 3D-motion learning framework for animal spontaneous behavior mapping

Kang Huang, Yaning Han, Ke Chen, Hongli Pan, Gaoyang Zhao, Wenling Yi, Xiaoxi Li, Siyuan Liu, Pengfei Wei () and Liping Wang ()
Additional contact information
Kang Huang: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Yaning Han: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Ke Chen: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Hongli Pan: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Gaoyang Zhao: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Wenling Yi: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Xiaoxi Li: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Siyuan Liu: Pennsylvania State University
Pengfei Wei: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences
Liping Wang: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences

Nature Communications, 2021, vol. 12, issue 1, 1-14

Abstract: Abstract Animal behavior usually has a hierarchical structure and dynamics. Therefore, to understand how the neural system coordinates with behaviors, neuroscientists need a quantitative description of the hierarchical dynamics of different behaviors. However, the recent end-to-end machine-learning-based methods for behavior analysis mostly focus on recognizing behavioral identities on a static timescale or based on limited observations. These approaches usually lose rich dynamic information on cross-scale behaviors. Here, inspired by the natural structure of animal behaviors, we address this challenge by proposing a parallel and multi-layered framework to learn the hierarchical dynamics and generate an objective metric to map the behavior into the feature space. In addition, we characterize the animal 3D kinematics with our low-cost and efficient multi-view 3D animal motion-capture system. Finally, we demonstrate that this framework can monitor spontaneous behavior and automatically identify the behavioral phenotypes of the transgenic animal disease model. The extensive experiment results suggest that our framework has a wide range of applications, including animal disease model phenotyping and the relationships modeling between the neural circuits and behavior.

Date: 2021
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.nature.com/articles/s41467-021-22970-y Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-22970-y

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-021-22970-y

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().