Secure deep learning inference with intel software guard extension on intel ice lake-SP xeon processor

Abstract

In the current technology-driven era where the world is powered by the internet, artificial intelligence has never been more popular and so as the cloud computing or edge computing technology. With artificial intelligence being on an upward trend in the digitalized world, many industries have started to deploy artificial intelligence in their applications. Among many of the techniques used for artificial intelligence, deep learning is one of the most popular techniques deployed, thanks to its remarkable reputation that outperforms many of the other methodologies. Hence, deep learning inference has been actively used in numerous applications nowadays, even for those that are security critical. On top of that, with cloud computing and edge computing becoming more common now, many industries have also started the migration of their applications to the cloud or to the edge. All these raised a serious security concern that should be tackled with best effort. Throughout the years, many researchers have been active in exploring the solutions to secure a deep learning inference. Cryptographic primitives and trusted hardware are among the most popular methodologies proposed. Nevertheless, most of the efforts focused only on preserving the security of the input data and the deep learning model without providing any security to the application code or the inference forward pass. Since cryptographic primitives are known to incur a high performance overhead, this research proposed to secure a deep learning application via the trusted hardware approach, specifically Intel SGX on the recently launched 3rd Gen Intel® Xeon Scalable processor. An image classification application was used as an example for the case study in this research and performance evaluation was conducted mainly based on the performance impacts of SGX in terms of the time taken for a model to be loaded to the CPU, the number of inferences per second, the total application runtime as well as the parallel efficiency of the application. In this research, 5 different deep learning models had been tested across 3 different environments and the findings are all presented in this project report. Results showed that when the image classification application was placed within the SGX enclave, there was an overhead of up to 9X for the time taken to load a model to the CPU and as high as 13X for the overall application runtime. The number of inferences per second and the parallel efficiency of the application both suffered from a loss of up to 70% when being compared to the native environment. Nonetheless, this research has demonstrated that with the greatly expanded Intel SGX enclave size, it is feasible to secure the deep learning application with Intel SGX without any code partitioning despite the trade-off on the performance. The findings from this research could serve as a reference for those who wish to explore further on this piece and could also serve as one of the works that unleashed the potential of the first Intel Xeon Scalable Processor that comes with Intel SGX support.