Device-based manipulation technique with separated control structures for 3d object translation and rotation in handheld mobile ar

Abstract

3D object manipulation is one of the most critical tasks for handheld mobile Augmented Reality (AR), which can contribute towards its practical potential, especially for real-world assembly support. In this context, the study of techniques which are used to manipulate 3D objects is an important research area. This study has developed an improved device-based technique within handheld mobile AR interfaces, to solve the large-range 3D object rotation problem, as well as issues related to position and orientation deviations in manipulating 3D objects. We firstly enhanced the existing device-based 3D object rotation technique (named as HoldAR) with an innovative control structure (named as TiltAR) that utilizes the handheld mobile device tilting and skewing amplitudes to determine the rotation axes and directions of the 3D object. Whenever the device is tilted or skewed in a way that exceeds the threshold values of the amplitudes, the 3D object rotation will start continuously with a pre-defined angular speed per second, to prevent over-rotation of the handheld mobile device. This over-rotation is a common occurrence when using the existing technique to perform large-range 3D object rotations. The problem of over-rotation of the handheld mobile devices needs to be solved since it causes a 3D object registration error and a 3D object display issue, where the 3D object does not appear consistent within the user's range of view. Secondly, restructuring the existing device-based 3D object manipulation technique (named as DI) was done by separating the degrees of freedom (DOF) of the 3D object translation and rotation, to prevent deviations of the 3D object position and orientation, caused by the DOF integration that utilizes the same control structure, which is HoldAR, for both tasks. Next, an improved device-based manipulation technique (named as DS), with better performance on task completion time for 3D object manipulation within handheld mobile AR interfaces, was developed. A pilot test was carried out before other main tests to determine several pre-defined values designed in the control structure of TiltAR. A series of 3D manipulation tasks were designed and developed to benchmark DS (the proposed manipulation technique) with DI (the existing technique) on task completion time (s). Sixteen participants aged 19–24 years old were selected. Each participant had to complete twelve trials, which came to a total 192 trials per experiment for all the participants. Repeated measure analysis was used to analyze the data. The results obtained have statistically proven that DS markedly outpaced DI with significant shorter task completion times in all tasks consisting of different difficulty levels and rotation amounts. Based on the findings, an improved device-based 3D object manipulation technique has been successfully developed to address the insufficient functionalities of the current technique.