Construct validity and reliability of higher order thinking skills in design and technology course: A Rasch measurement model approaches for pilot study

Abstract

This pilot study aims to produce empirical evidence of item validity and reliability of best practice instrument for Higher Order Thinking Skills (HOTS) for Design and Technology course. This pilot study instrument consists of 30 items and has been distributed to 150 respondents who are Design and Technology teachers of primary school in Muar and Batu Pahat district in the state of Johor. This instrument was developed to measure seven HOTS constructs namely 1) curriculum; 2) assessment; 3) pedagogy; 4) co-curriculum; 5) community and private sector support; 6) resources and 7) capacity building. The validity and reliability of this HOTS instrument was analyzed through the Rasch Measurement Model using Winstep V.3.69.1.11 software. Analysis was made on the aspects of reliability and separation of item-respondents, polarity item that measures the constructs, item fit as well as standard residual correlation values. The results of the analysis conducted found that there is no item that need to be dropped and all 30 items have been identified as suitable items to be use in measuring seven HOTS constructs. Findings from the study clearly show that the instruments developed have a high level of validity and reliability and are capable of being used in measuring Higher Order Thinking Skills in Design and Technology Course.