Universiti Teknologi Malaysia Institutional Repository

Spectral-based convolutional neural network without multiple spatial-frequency domain switchings

Ayat, Sayed Omid and Hani, Mohamed Khalil and Ab. Rahman, Ab. Al-Hadi and Abdellatef, Hamdan (2019) Spectral-based convolutional neural network without multiple spatial-frequency domain switchings. Neurocomputing, 364 . pp. 152-167. ISSN 0925-2312

Full text not available from this repository.

Official URL: http://dx.doi.org/10.1016/j.neucom.2019.06.094


Recent researches have shown that spectral representation provides a significant speed-up in the massive computation workload of convolution operations in the inference (feed-forward) algorithm of Convolutional Neural Networks (CNNs). This approach results in reducing the computational complexity of the classification task, which makes spectral-based CNN suitable for implementation on embedded platform that typically has constrained resources. However, a major challenge in this approach is that the mathematical formulation of a nonlinear activation function in spectral (frequency) domain is currently not available; hence, computation of the activation functions in each layer has to be performed in the spatial domain. This results in several spatial-frequency domain switchings that are computationally very costly, and as such, it would be advantageous to strictly stay in the frequency domain. Hence, in this work, a novel Spectral Rectified Linear Unit (SReLU) for the activation function is proposed, that makes it possible for the computations to remain in the frequency domain, and therefore avoids the multiple compute-intensive domain transformations. To further optimize the classification speed of the network, an efficient spectral-based CNN model is presented that uses only the lower frequency components by way of fusing the convolutional and sub-sampling layers. Additionally, we provide and utilize a frequency domain equivalent of the conventional batch normalization layer that results in improving the accuracy of the network. Experimental results indicate that the proposed spectral-based CNN model achieves up to 17.02 × and 3.45 × faster classification speed (without considerable accuracy loss) on AT&T face recognition and MNIST digit/fashion classification datasets, respectively, as compared to the equivalent models in the spatial domain, hence outperforming conventional approaches significantly.

Item Type:Article
Uncontrolled Keywords:Activation function, Frequency domain
Subjects:T Technology > TK Electrical engineering. Electronics Nuclear engineering
Divisions:Electrical Engineering
ID Code:87452
Deposited By: Widya Wahid
Deposited On:08 Nov 2020 11:59
Last Modified:08 Nov 2020 11:59

Repository Staff Only: item control page