Task oriented feature extraction for complex human activity recognition

Abstract

Human Activities Recognition (HAR) using mobile phone devices provides valuable contextaware information about the type of activities individuals perform within a time interval. HAR leverages sensory data available on today's sensor-rich, cheap, and portable mobile phones. It enables mobile phones to provide personalized support for many healthcare and well-being applications. It also has significant contributions to robotic, homeland security and smart environments. However, current recognition systems based on mobile phone sensors have observable issues in recognizing composite activities that occur concurrently or interleave i.e., complex activities, limiting their use in real-world applications. In those activities, the existence and variations of each activity as well as the order and length may vary. In this research, the issues of low recognition accuracy and high computing cost of complex human activities using mobile phone sensors are addressed. The composition and variations of human activity are examined as factors that impact the complexity of activity recognition. This research proposes to increase the quality of extracted features to increase the recognition accuracy with less resource consumption. It proposes extracting the wrist velocity as a feature for recognizing the performing arm’s complex activity. The wrist velocity feature is task oriented. Using the task-oriented wrist velocity feature will help to reduce recognition errors and therefore increase recognition accuracy. For this purpose, an extraction method for the wrist velocity feature is developed. In addition, the developed method is applied to recognize complex human activities using the Complex Activity Recognizer through Wrist velocity system (CARWV). Firstly, the extraction method begins by integrating the accelerometer and gyroscope data of the smartphone, which is placed on the upper arm and forearm. The integrated data is used to calculate the rotational angles of the upper arm and forearm. Then, the calculated rotational angles and lengths of the upper arm and forearm are used to calculate the position and the velocity of the wrist while performing the activity. Secondly, in the proposed recognition system (CARWV), the complex activity is broken into tasks that are represented by basic arm movements. The wrist velocity while performing the basic arm movements is extracted. The decision tree classifier is used to recognize the basic arm movements through the extracted feature. Then, the existing and order of recognized basic arm movements in the complex activity are used as features for recognizing the complex activity by measuring the similarity using the distance metric. The experiments demonstrate the validity of the task-oriented property of the extracted feature. The experiments also show increased recognition accuracy when using the proposed system up to 86% over performance for the state-of-the-art works, with 13 sec execution time and 31264 kb allocated memory in a notebook computer with Core i7 processor and 8GM memory. This study can facilitate future research in other fields where performance and limited resources are critical quality factors such as robotics and Wireless Sensor Networks (WSN).