Music Control in an Interactive Conducting System Using Kinect

Yi-Shin Chen, Leng-Wee Toh and Yi-Lan Liu
Additional contact information
Yi-Shin Chen: Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu, Taiwan
Leng-Wee Toh: Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan
Yi-Lan Liu: Institute of Information Systems and Applications, National Tsing Hua University, Hsinchu, Taiwan

International Journal of Multimedia Data Engineering and Management (IJMDEM), 2013, vol. 4, issue 4, 35-57

Abstract: Music conducting is the art of directing musical ensembles with hand gestures to personalize and diversify a piece of music. Although the ability to successfully perform a musical piece demands intense training and coordination for the conductor and the orchestra, preparing a practice session is expensive and time-consuming. Hence, there is a genuine need for alternatives capable of providing adequate training for conductors at all skill levels. The potential use of virtual and augmented reality technology holds particular promise. The goal of this research is to examine the mechanics of music conducting and to develop a system capable of closely simulating the experience of conducting a piece of music. After extensive discussions with professional and nonprofessional conductors, in addition to wide-ranging research regarding music conducting materials, several key features of conducting were identified. A set of lightweight algorithms exploring these features was developed to enable tempo control, volume adjustment, and instrument emphasis, which are core components of conducting. Such a system would be a helpful training tool for students, an experiential tool allowing professional conductors and composers to shape music at a low cost, or an entertainment tool for nonprofessional music lovers. In this paper, we propose a real-time interactive conducting system using Microsoft Kinect. The proposed system overcomes the limitation of Kinect's design, which is generally designed for large body movements. In this system, delicate conducting signals can be correctly recognized without referencing any prior knowledge. Evaluation of the algorithms in real-world scenarios reveals promising results. The system was evaluated by conductors of all skill levels and provided a high level of accuracy and a low latency. Users of the final system expressed satisfaction with the virtual experience.

Date: 2013
References: Add references at CitEc
Citations:

Downloads: (external link)
http://services.igi-global.com/resolvedoi/resolve. ... 18/ijmdem.2013100103 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:igg:jmdem0:v:4:y:2013:i:4:p:35-57

Access Statistics for this article

International Journal of Multimedia Data Engineering and Management (IJMDEM) is currently edited by Chengcui Zhang

More articles in International Journal of Multimedia Data Engineering and Management (IJMDEM) from IGI Global
Bibliographic data for series maintained by Journal Editor ().