Universiti Teknologi Malaysia Institutional Repository

Multimodal fusion: Gesture and speech input in augmented reality environment

Ismail, Ajune Wanis and Sunar, Mohd. Shahrizal (2015) Multimodal fusion: Gesture and speech input in augmented reality environment. In: 4th International Neural Network Society Symposia Series on Computational Intelligence in Information Systems, INNS-CIIS 2014, 7 - 9 November 2014, Bandar Seri Begawan, Brunei.

Full text not available from this repository.

Official URL: http://dx.doi.org/10.1007/978-3-319-13153-5_24

Abstract

Augmented Reality (AR) has the capability to interact with the virtual objects and physical objects simultaneously since it combines the real world with virtual world seamlessly. However, most AR interface applies conventional Virtual Reality (VR) interaction techniques without modification. In this paper we explore the multimodal fusion for AR with speech and hand gesture input. Multimodal fusion enables users to interact with computers through various input modalities like speech, gesture, and eye gaze. At the first stage to propose the multimodal interaction, the input modalities are decided to be selected before be integrated in an interface. The paper presents several related works about to recap the multimodal approaches until it recently has been one of the research trends in AR. It presents the assorted existing works in multimodal for VR and AR. In AR, multimodal considers as the solution to improve the interaction between the virtual and physical entities. It is an ideal interaction technique for AR applications since AR supports interactions in real and virtual worlds in the real-time. This paper describes the recent studies in AR developments that appeal gesture and speech inputs. It looks into multimodal fusion and its developments, followed by the conclusion.This paper will give a guideline on multimodal fusion on how to integrate the gesture and speech inputs in AR environment.

Item Type:Conference or Workshop Item (Paper)
Uncontrolled Keywords:Augmented Reality, Gesture and Speech Input
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:Computing
ID Code:59385
Deposited By: Haliza Zainal
Deposited On:18 Jan 2017 01:50
Last Modified:10 Apr 2022 05:48

Repository Staff Only: item control page