Cost-effective model-based test case generation and prioritization for software product line

Abstract

In Software Product Line (SPL), testing is used to manage core assets that comprised variability and commonality in effective ways due to large sizes of products that continue to be developed. SPL testing requires a technique that is capable to manage SPL core assets. Model-based Testing (MBT) is a promising technique that offers automation and reusability in test cases generation. However, there are difficulties to ensure testing in MBT can achieve good test cases generation results based on cost (size of test suite, total execution time) and effectiveness (coverage criteria, fault detection rate) measures. This is due to lack of trade-off between cost and effectiveness in test cases generated in MBT for SPL. This study aims to increase quality of test cases based on cost and effectiveness by using generation and prioritization approaches for MBT in SPL. This study focuses on three parts to enhance quality of test cases. First, test model development based on traceability link. In order to improve test cases quality, this study focused on implementation of hybrid-based and hyper-heuristic based techniques to generate test cases. This is followed by Test Cases Prioritization (TCP) technique that is based on dissimilarity-based technique with string distance. These test cases generation and prioritization approaches are evaluated by using two benchmarks - one test object and one real object. The results are compared with other prominent approaches. The mapping approach showed 10.27% and 32.39% f-measure improvement against existing approach on e-shop object, respectively. For test cases generation using hybrid-based approach, the proposed approach outperformed existing approaches with 11.66% coverage, 17.78% average execution time, and 45.98% average size of test suite on vending machine object. The hyper-heuristic based approach NSGA-II-LHH outperformed other proposed low-level heuristic approaches with 12.00% improvement on coverage, 46.66% average execution time and 42.54% average size of test suite. Furthermore, evaluation of TCP approaches showed fault detection improvement of 21.60%, 10.40% and 12.20% and total execution time improvement of 48.00%, 22.70% and 31.80% in comparison with three existing approaches. The results revealed that proposed model transformations, test cases generation and prioritization approaches significantly improve cost and effectiveness measure in MBT for SPL.