A review on multimodal interaction in mixed reality environment

Abstract

Mixed Reality (MR) environment proposes to blend together virtual and real environments where both world coexist and can respond to each other. As MR has the understanding of real environment, it is possible for user to use their surrounding as a means to interact with the virtual content. The user interaction in Augmented Reality (AR) and Virtual Reality (VR) are widely explored however very less in MR. Interaction must be intuitive as possible to produce natural user interaction. Multimodal interaction is an advanced interaction techniques when involve the combination of more than one input modalities such as speech with gesture, gaze and gesture, and gaze with speech. The inputs or sensory however are not restricted to just two modalities. Therefore this paper presents a review on multimodal interaction in MR environments.