An instrument to measure information and communication technology user-skills ability for engineering learning

Abstract

Some of the most important skills for engineering education in today’s digital world are information and communication technology (ICT) user-skills. This research concerned two main issues regarding ICT user-skills of engineering students. The first issue was the lack of a reliable and valid instrument to measure ICT user-skills ability for engineering learning. The second issue was the lack of profile information on students’ existing ICT user-skills, such as what their ICT skills level were, how they acquired the skills, their conception of ICT user-skills, to what extent ICT user-skills support engineering learning, as well as the difficulties faced in acquiring those skills. This information would provide the basis for student ICT skills improvement strategies. Thus, this research sought to address these issues by developing an instrument to measure students’ ICT user-skills and subsequently establishing the ICT user-skills profile. This study adopted an across-stage mixed method design, combining quantitative and qualitative approaches. The research process comprised eight major phases: problem identification, literature review, determining problem statement and research objectives, instrument design and development, sample selection, data collection, data analysis, discussion and conclusion. Instrument development and validation were performed in five phases: determining what to measure, a review and assessment of major existing instruments, drafting a new instrument, getting expert reviews and student feedback, pilot testing the instrument, checking the internal consistency and refining the instrument, testing the modified instrument, and finally conducting the main study using a stratified random sample. Reliability and validity of the instrument were established using a Rasch model. Quantitative data analyses were performed using the PASW and WINSTEPS software. Thematic analysis of interview transcriptions was conducted to corroborate quantitative findings. The outcomes of this study were a new survey instrument to measure ICT user-skills within context of the study population, and a profile of engineering students’ ICT user-skills.