Determining Dimensionality and Model-Data Fit Statistics of the ICT Test Using the Rasch Model at Jigawa State University
DOI:
https://doi.org/10.66545/bdm40s93Keywords:
Dimensionality, fit statistics, rash analyses, ict testAbstract
This study evaluates the dimensionality and model-data fit statistics of an Information and Communication Technology (ICT) test administered at Jigawa state university using the Rasch model. Despite the test's widespread use, its psychometric properties had not been previously assessed. This cross-sectional study addresses this gap by employing Rasch analysis to evaluate the reliability and validity of the test items, using a sample of 600 undergraduate test scripts randomly selected from various departments. This approach ensures that the measurement instrument accurately reflects the underlying construct of ICT proficiency across a diverse student population. The results of the Principal Component Analysis indicate that the variance explained by the measures was 62.9%, confirming a strong primary dimension and supporting the one- dimension nature of the test. Additionally, model-data fit statistics shows that the mean values for both infit and outfit mean-square (MNSQ) were 1.00 and 1.03 logits, respectively, aligning with the expected values for the Rasch model. These findings confirm that the test items behave as expected, demonstrating strong psychometric properties in terms of validity and reliability. The findings reveal that the test is unidimensional and possesses
strong fit statistics, confirming its validity and reliability. These results provide valuable insights into the test's psychometric properties, validating its effectiveness in measuring ICT proficiency across different academic disciplines. Consequently, the study supports the continued use of the test while also offering guidance for potential improvements in its design and application within educational settings.
References
Adebisi, T. A., Oyeleke, O., & Adeleke, A. G. (2018). Digital literacy skills and MOOC participation among lecturers in a Nigerian private university. Library Philosophy and Practice, 1-18. https://core.ac.uk/download/pdf/188131266.pdf
Alsubaie, M. A., & Alquraini, A. A. (2022). Using the Rasch model to evaluate the quality of test items in a computer programming course. Journal of Educational Computing Research, 61(4), 419–433.
Adedokun-Shittu, N. A., & Shittu, A. J. K. (2015). Assessing the impacts of ICT deployment in teaching and learning in higher education: Using ICT impact assessment model. Journal of Applied Research in Higher Education, 7(2), 180-193. https://doi.org/10.1108/JARHE-02-2013-0012
Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.
Boone, W. J., Staver, J. R., & Yale, M. S. (2014). Rasch analysis in the human sciences. Springer.
Baghaei, P. (2008). The Rasch Model as a construct validation tool. Rasch Measurement Transactions, 22(1), 1145–1146.
Etikan, I., & Bala, K. (2017). Sampling and sampling methods. Biometrics & Biostatistics International Journal, 5(6), 00149. https://doi.org/10.15406/bbij.2017.05.00149.
Eakman, A. M. (2012). The measurement characteristics of the engagement in meaningful activities surveys in an age-diverse samples. The American Journal of Occupational Therapy, 66(2), 20–29. https://doi.org/10.5014/ajot.2012.001867.
Fisher, W.P. (2007). Rating scale instrument quality criteria. Rasch measurement transactions, 21(1), 1095. http://www.rasch.org/rmt/rmt211a.htm
Hamad, A. A. (2021). The Psychometric Properties of Measurement of the Mathematics Teachers' Professional Identity in Saudi Arabia (Unpublished Doctoral dissertation, USM). http://eprints.usm.my/52981/1/ALGHAMDI%20HAMAD%20ABDULLAH%20J%20
%20TESIS.pdf%20cut.pdf
Khalifa, B. S., & Liao, S. Y. (2021). Applying the Rasch model to evaluate the performance of test items in a mathematics assessment. International Journal of Assessment and Evaluation, 28(1), 1–15. DOI: 10.7763/IJSSH.2012.V2.104
Kline, P. (2016). A handbook of test development: Guide for professionals in education, psychology, and business. Sage Publications.
Lin, H. C., Wu, C. H., & Chen, S. Y. (2022). Using the Rasch model to develop and evaluate a reading comprehension test. Reading and Writing, 35(3), 537–555.
Levin, K. A. (2006). Study design III: Cross-sectional studies. Evidence-Based Dentistry, 7(1), 24-25. https://doi.org/10.1038/sj.ebd.6400375
Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328.
Linacre, J. M. (2021). Winsteps® Rasch measurement computer program user's guide. Winsteps.com.
Linacre, J. M. (2006). A user's guide to WINSTEPS/ MINISTEPS: A Rasch model computer programs. Chicago, USA: Winsteps.com.
Linacre, John M. (2007). A user's guide to WINSTEPS: Ministep Rasch Model Computer Programs. MESA Press.
Linacre, G. M. (2002). Optimizing rating scale category effectiveness. Journal of Applied Measurement, 3(1), 85-106. https://pubmed.ncbi.nlm.nih.gov/11997586/
Linacre, G. M. (2010). A User's Guide to winsteps. Program Manual 3.71.0.
Messick, S. (1995). Validity of psychological assessment: Validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist, 50(9), 741-749. https://doi.org/10.1037/0003-066X.50.9.741.
Olaniyi, O. A, & Ismaila, K. O. (2016). Information and communication technologies (ICTs) usage and household food security status of maize crop farmers in Kwara State, Nigeria. Library Philosophy and Practice, 1-20.
Olelewe, C. J., & Agomuo, E. E. (2016). Effects of B-learning and F2F learning environments on students' achievement in QBASIC programming. Computers & Education, 103, 76-86. https://doi.org/10.1016/j.compedu.2016.09.012.
Paek, I., & Wilson, M. (2018). A Rasch analysis of the Test of English as a Foreign Language (TOEFL). Language Testing, 35(2), 147-164.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Nielsen & Lydiche.
Reckase, D. (1979). Unifactor latent traits models applied to multifactor tests: results and implications. Journal of Educational Statistics, 4(3), 207–230. https://doi.org/10.3102/10769986004003207.
Smith, P. (2017). Validity theory and the Rasch model. Measurement: Interdisciplinary Research and Perspectives, 15(1), 34-46. https://doi.org/10.1080/15366367.2017.1285137
Wright, B. D., & Linacre, J. M. (1994). Reasonable mean-square fit values. Rasch Measurement Transactions, 8(3), 370.
Wright, B. D., & Masters, G. N. (2018). Rating scale analysis: Rasch measurement. SAGE Publications.
Wu, Y. W., & Chen, S. Y. (2020). Using the Rasch model to evaluate the performance of test items in a science assessment. Journal of Science, Education, and Technology, 29(2), 249–262.
Yang, P., et al. (2020). A Rasch analysis of the Chinese version of the Beck Depression Inventory II. Journal of Clinical Psychology, 76(1), 35-47.
Zwick, R., Thayer, D. T., & Lewis, C. (1999). An empirical Bayes approach to Mantel-Haenszel DIF analysis. Journal of Educational Measurement, 36(1), 1-28. https://doi.org/10.1111/j.1745-3984.1999.tb00543.x
Zhang, Y., & Wang, W. C. (2020). Applying the Rasch model to evaluate the quality of test items in a language proficiency test. Language Testing, 37(3), 349–366. DOI: 10.21831/cp.v38i1.22750
