Universiti Teknologi Malaysia Institutional Repository

An automated framework for software test oracle

Shahamiri, S. R. and Wan Kadir, Wan Mohd. Nasir and Ibrahim, Suhaimi and Mohd. Hashim, Siti Zaiton (2011) An automated framework for software test oracle. Information and Software Technology, 53 (7). pp. 774-788. ISSN 0950-5849

[img] PDF (Abstract)
74Kb

Official URL: http://dx.doi.org/10.1016/j.infsof.2011.02.006

Abstract

Context: One of the important issues of software testing is to provide an automated test oracle. Test oracles are reliable sources of how the software under test must operate. In particular, they are used to evaluate the actual results that produced by the software. However, in order to generate an automated test oracle, oracle challenges need to be addressed. These challenges are output-domain generation, input domain to output domain mapping, and a comparator to decide on the accuracy of the actual outputs. Objective: This paper proposes an automated test oracle framework to address all of these challenges. Method: I/O Relationship Analysis is used to generate the output domain automatically and Multi-Networks Oracles based on artificial neural networks are introduced to handle the second challenge. The last challenge is addressed using an automated comparator that adjusts the oracle precision by defining the comparison tolerance. The proposed approach was evaluated using an industry strength case study, which was injected with some faults. The quality of the proposed oracle was measured by assessing its accuracy, precision, misclassification error and practicality. Mutation testing was considered to provide the evaluation framework by implementing two different versions of the case study: a Golden Version and a Mutated Version. Furthermore, a comparative study between the existing automated oracles and the proposed one is provided based on which challenges they can automate. Results: Results indicate that the proposed approach automated the oracle generation process 97% in this experiment. Accuracy of the proposed oracle was up to 98.26%, and the oracle detected up to 97.7% of the injected faults. Conclusion: Consequently, the results of the study highlight the practicality of the proposed oracle in addition to the automation it offers.

Item Type:Article
Uncontrolled Keywords:artificial neural networks, I/O relationship analysis, mutation testing, software test oracle
Subjects:Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions:Computer Science and Information System (Formerly known)
ID Code:28669
Deposited By: Liza Porijo
Deposited On:12 Nov 2012 08:13
Last Modified:12 Nov 2012 08:13

Repository Staff Only: item control page