Balanced weighted unified discriminant and distribution alignment for open-view human action recognition

Abstract

Human action recognition (HAR) plays an increasingly important role in surveillance, robot learning, and human-computer interaction. However, there are many challenges and issues involved in achieving reliable and high-performance results. Among these challenges, view-invariant in an uncontrolled dataset where several cameras are placed at different locations received the most attention from researchers. One of the primary concerns for the uncontrolled dataset is the large difference between data distributions at the source (training) and target (testing) views. Such difference causes the data shift problem to occur and hence, decreases the performance of the HAR system. This issue has been explicitly discussed as an open-view HAR problem which aims to reduce the correlation between the source and the target views particularly when labelled data in unavailable in the target view. In addressing the issue, this thesis presents an unsupervised domain-adaptation model for the open-view HAR. Specifically, the proposed Balanced Weighted Unified Discriminant and Distribution Alignment (BW-UDDA) model has managed to handle datasets with significant variances across views. BW-UDDA balances and aligns marginal and conditional distribution features by projecting them into a low-dimensional subspace. This is to create more coordinated feature representations before feeding these features into an optimal classifier. Technically, BW-UDDA exploits two different unsupervised domain adaptation enhancement models, namely Balanced Weighted Joint Geometrical and Statistical Alignment (BW-JGSA) and Unified Discriminant and Distribution Alignment (UDDA). The BW-JGSA balances the marginal and conditional distributions in the nonparametric Maximum Mean Discrepancy (MMD) measurements on two disjointed embedded matrices. For the UDDA, two-dimensionality reduction techniques, namely linear discriminant analysis (LDA) and locality sensitivity discriminant analysis (LSDA), are incorporated to create features with global and local discriminant properties for the domain adaptation process. The enhancement models were evaluated on public image and digit datasets (Office, Caltech-256, USPS, MNIST and COIL20), while the BW-UDDA was assessed using the multi-camera action dataset (MCAD). Both enhancement models outperformed other state-of-the-art methods with average accuracies: 50.61% (object dataset) and 69.95% (digit dataset) for BW-JGSA, and 59.95% (object dataset) and 80.72% (digit dataset) for UDDA, respectively. BW-UDDA for open-view HAR was tested using two types of cross-view evaluations. The average accuracy of the first and second evaluations using the MCAD dataset outperformed the state-of-the-art with 13.38% and 61.45% higher accuracy, respectively. The BW-UDDA was also tested on a controlled multi-camera HAR dataset, the Inria Xmas Motion Acquisition Sequences (IXMAS), with an accuracy of 90.91% using the second type of cross-view evaluation. These results on MCAD and IXMAS confirmed the superiority of the proposed model for the open-view HAR.