New learning algorithm based on Hidden Markov Model (HMM) as stochastic modelling for pattern calssification

Abstract

This study investigates the use discriminative training methods of minimum classification error (MCE) to estimate the parameter of hidden Markov model (HMM). The conventional training of HMM is based on the maximum likelihood estimation (MLE) which aims to model the true probabilistic distribution of the data in term of maximizing the likelihood. This requires sufficient training data and correct choice of probabilistic models, which in reality hardly achievable. The insufficient training data and incorrect modeling assumption of HMM often yield an incorrect and unreliable model. Instead of learning the true distribution, the MCE based training targeted to minimizing the probability of error is used to obtain optimal Bayes classification. The central idea of MCE based training is to define a continuous, differentiable loss function to approximate the actual performance error rate. Gradient based optimization methods can be used to minimize this loss. In this study the first order online generalized probabilistic descent is used as optimization methods. The continuous density HMM is used as the classifier structure in the MCE framework. The MCE based training is evaluated on speaker-independent Malay isolated digit recognition. The MCE training achieves the classification accuracy of 96.4% compared to 96.1% of using MLE with small improvement rate of 0.31%. The small vocabulary is unable to reflect the performance comparison of the two methods, the MLE training given sufficient training data is sufficient to provide optimal classification accuracy. Future work will extend the evaluation on difficult classification task such as phoneme classification, to better access the discriminative ability of the both methods.