Deep learning based hand gesture recognition and UAV flight controls

Bin Hu, Jiacun Wang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

16 Scopus citations

Abstract

Dynamic hand gesture recognition is desired as an alternative means for human-computer interactions. This paper presents a hand gesture recognition system that is designed for the control of flights of unmanned aerial vehicles (UAV). To train the system to recognize designed gestures, skeleton data collected from a Leap Motion Controller are converted to two different data models. As many as 9124 samples of training dataset, 1938 samples of testing dataset are created to train and test the proposed three deep learning neural networks, which are a 2-layer fully connected neural network, a 5-layer fully connected neural network and an 8-layer convolutional neural network. The static testing results show that the 2-layer fully connected neural network achieves an average accuracy of 98.2% on normalized datasets and 11% on raw datasets. The 5-layer fully connected neural network achieves an average accuracy of 95.2% on normalized datasets and 45% on raw datasets. The 8-layers convolutional neural network achieves an average accuracy of 96.2% on normalized datasets and raw datasets. Testing on a drone-kit simulator and a real drone shows that this system is feasible for drone flight controls.

Original languageEnglish
Title of host publicationICAC 2018 - 2018 24th IEEE International Conference on Automation and Computing
Subtitle of host publicationImproving Productivity through Automation and Computing
EditorsXiandong Ma
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781862203426
DOIs
StatePublished - Sep 2018
Event24th IEEE International Conference on Automation and Computing, ICAC 2018 - Newcastle upon Tyne, United Kingdom
Duration: 6 Sep 20187 Sep 2018

Publication series

NameICAC 2018 - 2018 24th IEEE International Conference on Automation and Computing: Improving Productivity through Automation and Computing

Conference

Conference24th IEEE International Conference on Automation and Computing, ICAC 2018
Country/TerritoryUnited Kingdom
CityNewcastle upon Tyne
Period6/09/187/09/18

Keywords

  • Deep learning
  • Drones
  • Hand gesture recognition
  • Leap motion controllers
  • Neural networks

Fingerprint

Dive into the research topics of 'Deep learning based hand gesture recognition and UAV flight controls'. Together they form a unique fingerprint.

Cite this