Selective joint motion recognition using multi sensor for salat learning

Abstract

Over the past few years, there has been significant attention given on motion recognition in computer vision as it has a wide range of potential applications that can be further developed. Hence, a wide variety of algorithms and techniques has been proposed to develop human motion recognition systems for the benefit of the human. Salat—an essential ritual in Muslim daily life which helps them be good Muslims—is not solely about the spiritual act, but it also involves the physical movements in which it has to be done according to its code of conduct. The existing motion recognition proposed for computing applications for salat movement is unsuitable as the movement in salat must be performed in accordance to the rules and procedures stipulated, the accuracy and sequence. In addition, tracking all skeleton joints does not contribute equally toward activity recognition as well as it is also computationally intensive. The current salat recognition focuses on recognizing main movements and it does not cover the whole cycle of salat activity. Besides, using a wearable sensor is not natural in performing salat since the user needs to give absolute concentration during salat activity. The research conducted was based on the intersections of technological development and Muslim spiritual practices. This study has been developed utilizing dual-sensor cameras and a special sensor prayer mat that has the ability to cooperate in recognizing salat movement and identifying the error in the movement. With the current technology in depth cameras and software development kits, human joint information is available to locate the joint position. Only important joints with the significant movement were selected to be tracked to perform real-time motion recognition. This selective joint algorithm is computationally efficient and offers good recognition accuracy in real-time. Once the features have been constructed, the Hidden Markov Model classifier was utilized to train and test the algorithm. The algorithm was tested on a purposely built dataset of depth videos recorded using a Kinect camera. This motion recognition system was designed based on the salat activity to recognize the user movement and his error rate, which will later be compared with the traditional tutor-based methodology. Subsequently, an evaluation comprising 25 participants was conducted utilizing usability testing methods. The experiment was conducted to evaluate the success score of the user’s salat movement recognition and error rate. Besides, user experience and subjective satisfaction toward the proposed system have been considered to evaluate user acceptance. The results showed that the evaluation of the proposed system was significantly different from the traditional tutor-based method evaluation. Results indicated a significant difference (p < 0.05) in success score and user’s error rate between the proposed system and traditional tutor-based methodology. This study also depicted that the proposed motion recognition system had successfully recognized salat movement and evaluated user error in salat activity, offering an alternative salat learning methodology. This motion identification system appears to offer an alternate learning process in a variety of study domains, not just salat movement activity.