Spectral domain convolutional neural network optimized for computational workload and memory access cost

Abstract

Conventional convolutional neural networks (CNNs), which are realized in the spatial domain, present a high computationalworkload and memory access cost (CMC). Spectral domain CNNs (SpCNNs) offer a computationally efficient approach for performing CNN training and inference. State-of-the-art SpCNNs propose activation functions (AFs) that are computationally costly or realize AFs in the spatial domain necessitating multiple and expensive spatial-spectral domain transformations. This work proposes a complex-valued AF for SpCNNs that transmits inputs unaltered or scaled depending on the activation area. This AF is computationally inexpensive and provides sufficient non-linear transformation that ensures high classification accuracy. This work also investigates the CMC of SpCNNs and its contributing components analytically and then proposes a methodology to optimize CMC, under three strategies, to enhance inference and training performance. The strategies involve the reduction of the output feature map (OFM) size, OFM depth, or both under an accuracy constraint to compute performance-optimized CNN inference and training. The proposed AF denoted as complex-valued leaky ReLU (CLReLU), was employed in a LeNet-5 SpCNN architecture and achieves an accuracy gain of up to 3% for MNIST and 8% for Fashion MNIST dataset, while providing up to 2.3 times higher throughput in inference, over state-of-the-art AFs applied to the same model. The proposed CMC reduction methodology was applied to LeNet-5 and AlexNet architectures. For instance, the optimal AlexNet model achieves up to 34 times higher throughput in inference, and up to 16 times greater energy efficiency in training, with a minor accuracy loss of 2%, as compared to related state-of-the-art work. The proposed AF and CMC reduction methodology helps develop an SpCNN model that provides faster and more energyefficient computation as well as high test accuracy.