Swarm based features selection for text summarization

Abstract

The features are the main entries in text summarization. Treating all features equally causes poor summary generation. In this paper, we investigate the effect of the feature structure on the features selection using particle swarm optimization. The particle swarm optimization is trained using DUC 2002 data to learn the weight of each feature. The features used are different in terms of the structure, where some features were formed as combination of more than one feature while others as simple or individual feature. Therefore the determining of the effectiveness of each type of features could lead to mechanism to differentiate between the features having high importance and those having low importance. We assume that the combined features have higher priority of getting selection more than the simple features. In each iteration, the particle swarm optimization selects some features, then corresponding weights of those features are used to score the sentences and the top ranking sentences are selected as summary. The selected features of each best summary are used in calculation of the final features weights. The experimental results shown that the simple features are less effective than the combined features