Deep Learning Based Hand Gesture Recognition and UAV Flight Controls

Bin Hu, Jiacun Wang

Research output: Contribution to journalArticlepeer-review

124 Scopus citations

Abstract

Dynamic hand gesture recognition is a desired alternative means for human-computer interactions. This paper presents a hand gesture recognition system that is designed for the control of flights of unmanned aerial vehicles (UAV). A data representation model that represents a dynamic gesture sequence by converting the 4-D spatiotemporal data to 2-D matrix and a 1-D array is introduced. To train the system to recognize designed gestures, skeleton data collected from a Leap Motion Controller are converted to two different data models. As many as 9 124 samples of the training dataset, 1 938 samples of the testing dataset are created to train and test the proposed three deep learning neural networks, which are a 2-layer fully connected neural network, a 5-layer fully connected neural network and an 8-layer convolutional neural network. The static testing results show that the 2-layer fully connected neural network achieves an average accuracy of 96.7% on scaled datasets and 12.3% on non-scaled datasets. The 5-layer fully connected neural network achieves an average accuracy of 98.0% on scaled datasets and 89.1% on non-scaled datasets. The 8-layer convolutional neural network achieves an average accuracy of 89.6% on scaled datasets and 96.9% on non-scaled datasets. Testing on a drone-kit simulator and a real drone shows that this system is feasible for drone flight controls.

Original languageEnglish
Pages (from-to)17-29
Number of pages13
JournalInternational Journal of Automation and Computing
Volume17
Issue number1
DOIs
StatePublished - 1 Feb 2020

Keywords

  • Deep learning
  • drones
  • hand gesture recognition
  • Leap Motion Controllers
  • neural networks

Fingerprint

Dive into the research topics of 'Deep Learning Based Hand Gesture Recognition and UAV Flight Controls'. Together they form a unique fingerprint.

Cite this