Real-time voice-controlled game interaction using convolutional neural networks

Abstract

Speech recognition has gained growing popularity due to its wide applications in almost every field, ranging from wake-word recognition, emotion recognition, command recognition, and interactive game. Recently, there is a growing interest in using voice in the gaming industry. Voice-controlled interaction made gaming much more accessible to a wider audience. However, the use of voice to control games requires real-time processing to avoid unwanted delay. This paper proposes speech command recognition using Convolutional Neural Networks (CNN) to control the popular snake game. First, the limited dataset for Up, Down, Left, Right speech commands was prepared for training, validation, and testing. Second, an optimum MFCC and CNN-based speech command recognition were proposed to recognize the four speech command. Results showed that our proposed algorithm could achieve high recognition accuracy of 96.5% and was able to detect all four commands. Finally, the proposed algorithm is integrated with a Python-based snake game.