A comparative study on automated software test oracle methods

Abstract

Software testing has been used to find software faults in order to improve its quality. To verify the software behavior, testers require test oracle. Test oracle is a reliable source of expected software behavior that provides outputs for any input specified in the software specifications and a comparator to verify actual results. While test automation requires automated oracle support, oracle automation is considered as a challenging task. These challenges are from the automation required in expected output generation and results verification. This paper presents oracle activities and the challenges to prepare automated oracle. Then a comparative study of existing automated oracle and expected output generation methods is provided. Finally, a classification of these methods is suggested based on how these methods provide automated test oracle and the tool they used. The classification explains which oracle activities will be automated by the proposed approaches.