Document summarization using transfer learning

Abstract

Document summarization refers to an automation method to shortening a document into a short and meaningful article. In computing, automatic summarization can basically split to two approaches where it can be done with classical method where the rank of the text is calculated and word is extracted with respect to the word’s rank. In the other hand, the task can be completed by using modern method, deep learning. In deep learning, it is slightly different that the programming is mainly prepare a model where it will learn to summarize the document. However, som pre-processing on the data is needed before it is fit into the deep learning model. All the detailed part will be discussed in this project. For this project, the sequence to sequence model will be used as the main computing unit. On top of that, word embedding layer will helps in the summarization by providing the knowledge of word’s relationship. By combining these two design, the deep learning model is able to differentiate the word with respect to the relationship. Of course, this project included some pre-processing where the data will pre-filtered and convert to data that recognized by the model. Similarly, the output will be converted back to word that understand by the human. At the end, the summarized output will be evaluate by BLUE, a benchmark for sentence similarity. As a result, the model can achieve loss as low as 0.8% and accuracy of 32%. Overall, the accuracy is capped by the size of the model. It is due to the reason that the model does not support high number of vocabulary. The design can be further improve by increasing the vocabulary size. However, the training process need to be completed by using a better hardware. In addition, the covered text is evaluated the BLEU value with respect to the expected output summary. Overall, a trainable model is designed. The model can be further improve by adding vocabulary size as well as increasing all the training set.