Applying alternative method to evaluate online problem-solving skill inventory (OPSI) using Rasch model analysis

Abstract

The aim of the study was to evaluate the extended version of the Problem-Solving Inventory (PSI) via an online learning setting known as the Online Problem-Solving Inventory (OPSI) through the lens of Rasch Model analysis. To date, there is no extended version of the PSI for online settings even though many researchers have used it, thus, this study validated the OPSI taking into account evidence based on Rasch model analysis. Firstly, the OPSI employed content validation by three experts. The result of the percent agreement showed that the overall percent agreement of the OPSI is 95.05% which consists of six constructs: impulsive style, reflective style, problem-solving confidence, avoidance style, monitoring, and planfulness. Next, eight measurement properties of Rasch model analysis were employed with the OPSI among 49 undergraduate students in Malaysia. The results revealed that 19 respondents needed to be dropped according to Rasch model analysis. Next, retained 30 respondents continued to perform item analysis using Rasch model. The results revealed that 11 items need to be dropped. Rasch model analysis provided strong evidence to show the 22 retained items in OPSI are fit, valid, and reliable to measure problem-solving skills in an online learning setting in an educational field.