Item Analysis of Formative Tests on Physics Problem Solving in the Area of Dynamic Electricity in High School

Candra Purnama; Kaharuddin Arafah; Suryadi Ishak

doi:10.20527/jipf.v10i2.18906

Item Analysis of Formative Tests on Physics Problem Solving in the Area of Dynamic Electricity in High School

Candra Purnama, Kaharuddin Arafah, Suryadi Ishak

Abstract

The implementation and development of formative test items often still face various challenges that impact the accuracy of evaluation results. This study aims to analyze the quality of formative test items on physics problem-solving, particularly in the topic of dynamic electricity, by applying the Item Response Theory (IRT) approach in senior high schools in Bulukumba Regency. The study focuses on four main aspects: validity, reliability, item difficulty, and item discrimination. This research employed a quantitative descriptive design with data consisting of test responses from 369 twelfth-grade students. The analysis was conducted in three stages: content validity testing using Aiken’s V to assess the alignment of items with the measured competencies; reliability testing using the Kuder–Richardson 20 (KR-20) coefficient; and item characteristic analysis based on the IRT approach to obtain indices of item difficulty and discrimination. The results indicate that all items demonstrated good content validity, with Aiken’s V values above 0.75. The reliability coefficient showed high internal consistency, indicating that the instrument is reliable as a measurement tool. Regarding item difficulty, the 15 test items ranged from easy to difficult, with the majority categorized as moderate (10 items), while a smaller number were classified as easy (2 items) and difficult (3 items). This distribution suggests that the test adequately represents students’ abilities, particularly at the average ability level. The discrimination index ranged from moderate to high, with five items categorized as high and ten items as moderate. No items fell into the low category, indicating that the test items effectively distinguish students based on their ability levels. Overall, the instrument is considered valid and reliable, with proportional levels of difficulty and good discrimination power, making it appropriate for measuring physics problem-solving competence, particularly in the topic of dynamic electricity. The implication of this study is the need for wider application of IRT-based item analysis in research on the development of educational instruments at the high school level.

Keywords

Formative assessment; IRT analysis; Problem solving

Full Text:

PDF

References

Aiken, L. R. (1985). Three coefficients for analyzing the reliability and validity of ratings, educational and psychological measurument. Educational and Psychological Measurement, 45(1), 131–142.

Alim, U. N., Syahrul, S., de Barros Cardoso, L. M. O., Ishak, S., & Asrifan, A. (2025). Analisis kualitas tes bahasa arab di indonesia: Studi systematic literature review tentang validitas, reliabilitas, tingkat kesukaran, dan daya beda. Pepatudzu: Media Pendidikan dan Sosial Kemasyarakatan, 21(2), 183-202.

Andrade, H. L., & Heritage, M. (2018). Using formative assessment to enhance learning , achievement , and academic self-regulation. ROutledge.

Andrade, H. L., Heritage, M., Cizek, G. J., Dick, W., Carey, L., Carey, J. O., Wilson, M., Madaus, G. F., College, B., Hill, C., Stufflebeam, D. L., Michigan, W., Finch, W. H., French, B. F., Ebel, R. L., Frisbie, D. A., Suen, H. K., Mehrens, W. A., Lehmann, I. J., … Denison, D. B. (1988). Classroom assessment techniques a handbook for college teachers.pdf (N. J. Salkind, Ed.; 1st & 2nd ed., p. 1120). Jossey-.

Ankomah, F., & Nugba, R. M. (2020). Validation of test construction skills inventory through the lens of item response theory (IRT). American Journal of Creative Education, 3(2), 86-100.

Azwar, S. (2016). Reliabilitas dan validitas (IV). Pustaka Pelajar.

Arikunto, S. (2019). Dasar-dasar evaluasi pendidikan (3rd ed.). Bumi Aksara.

Baharuddin, H., Riskawati, R., & Fitrawardani, F. (2025). Identifikasi kemampuan prosedural dan konseptual siswa dalam memecahkan masalah matematika dengan model dina. Jurnal Studi Guru dan Pembelajaran, 8(3), 2273-2282.

Baker, F. B. (2001). The basics of item response theory (2nd ed.). ERIC.

Baker, F. B., & Kim, S.-H. (2004). Item response theory parameter estimation techniques (second). Taylor & Francis Group. https://doi.org/10.4324/9781315391342-12

Beck, K. (2020). Ensuring content validity of psychological and educational tests – The role of experts. Frontline Learning Research, 8(6), 1–37. https://doi.org/10.14786/flr.v8i6.517

Demars, C. (2010). Item respon theory; Understanding statistics measurement (N. Beretvas, Ed.). Oxford university Press.

Ebel, R. L., & Frisbie, D. A. (1991). Essential of education (5th ed.). Prentice-Hall.

Embretson, S. E., & Reise, S. P. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates.

Elviana, E. (2020). Analisis butir soal evaluasi pembelajaran pendidikan agama islam menggunakan program. Jurnal Mudarrisuna, 10(2), 58–74.

Fitraynsyah, M. A., Hilmiyati, F., & Sultan Maulana Hasanuddin, U. (2024). Peran tingkat kesukaran dan daya pembeda dalam analisis butir tes: Kajian literatur untuk pendidikan menengah. Jrep :Jurnal Riset Dan Evaluasi Pendidikan, 1(4), 252–262.

Gompi, M., Bito, N., & Isa, D. R. (2022). Pengaruh model pembelajaran student facilitator and explaining terhadap hasil belajar sisiwa pada materi bentuk aljabar. Jurnal Cendekia : Jurnal Pendidikan Matematika, 6(3), 3287–3295.

https://doi.org/10.31004/cendekia.v6i3.1739

Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.

Hidayatullah, R., Safitri, M., & Suyanto, H. (2022). Analisis butir soal Bahasa Arab di MAN 2 Kota Padang. Jurnal Evaluasi Pembelajaran Islam, 13(1), 77–89.

Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and application. Springer Science Business Media.

Kreitchmann, R. S., de la Torre, J., Sorrel, M. A., Nájera, P., & Abad, F. J. (2023). Improving reliability estimation in cognitive diagnosis modeling. Behavior Research Methods, 55(7), 3446–3460. https://doi.org/10.3758/s13428-022-01967-5

Linacre, J. M. (1994). Sample size and item calibration stability. Rasch Measurement Transactions, 7(4), 328–329

Lord, F. M. (1980). Applications of item response theory to practical testing problems. Routledge.

Magnus, B. (2018). The Basics of Item Response Theory Using R. Measurement: Interdisciplinary Research and Perspectives, 16(3). https://doi.org/10.1080/15366367.2018.1462078

Mandinach, E. B., & Schildkamp, K. (2021). Misconceptions about Data-Based Decision Making in Education: An Exploration of the Literature. Studies in Educational Evaluation, 69, 100842

Miftahir, R., Andrina, M., Lestari, P. P., Indriyani, K. F., & Hidayanti, U. (2025). Analisis butir soal tes kemampuan siswa pada mata pelajaran manajemen perkantoran. DIAJAR: Jurnal Pendidikan Dan Pembelajaran, 4(3), 337–346. https://doi.org/10.54259/diajar.v4i3.4029

Nazharita, E., Hidayat, R., & Rahmatudin, J. (2022). Domain Geometri Siswa Smp Menggunakan Rasch. Pedagogy, 10(3), 989–1000.

Ningrum, D. O. S. (2014). Teknik guru dalam penyusunan soal, validitas, reliabilitas, kendala dan upaya dalam penyelesaian masalah penyusunan instrumen evaluasi pembelajaran ipa biologi smp se-kabupaten probolinggo. Skripsi Universitas Jember. https://repository.unej.ac.id/handle/123456789/19137

Nitko, A. J., & Brookhart, S. M. (2011). Educational Assessment of Students (P. A. Smith, Ed.; 6th ed.). Pearson.

Popham, W. J. (2017). Classroom assessment: What teachers need to know (8th ed.). Pearson. www.pearsoned.com/permissions/.

Rahman, A., Rusnayati, H., & Muslim, M. (2021). Analysis of the characteristics of higher order thinking skills (HOTS) test on momentum and impulse for senior high school student using item response theory. Jurnal Penelitian Fisika Dan Aplikasinya (JPFA), 11(2), 127-137

Reckase, M. D. (2009). Multidimensional item response theory (statistics for social and behavioral sciences). Springer. https://doi.org/10.1007/978-0-387-89976-3

Reise, S. P., & Moore, T. M. (2023). Item response theory. In APA handbook of research methods in psychology: Foundations, planning, measures, and psychometrics (2nd ed., Vol. 1, pp. 809–835). American Psychological Association.

https://doi.org/https://doi.org/10.1037/0000318-037

Retnawati, H. (2014). Teori respon butir dan penerapanya. Yogyakarta: Nuha Medika.

Saepuzaman, D., Istiyono, E., & Haryanto. (2022). Characteristics of fundamental physics higher-order thinking skills test using item response theory analysis. Pegem Journal of Education and Instruction, 12(4), 269-279. https://doi.org/10.47750/pegegog.12.04.28

Salkind, N. J., Frey, B. B., & Winter, R. J. (2020). Statistic for people who (think they) hate statistic (7th ed.). SAGE.

Syahriani, S. (2021). Analisis butir soal ujian sekolah mata pelajaran bahasa indonesia kelas vi sdn kota bengkulu. Jurnal Pembelajaran Dan Pengajaran Pendidikan Dasar, 5(1), 198–209. https://doi.org/10.33369/dikdas.v5i1.16221

Syafii, A., Haryanto, Ahmad, I. F., & Fauziah, A. (2021). Analysis of items with item response theory (IRT) approach on final assessment for Al-Quran Hadith subjects. Jurnal Pendidikan Agama Islam, 18(1), 167–194. https://doi.org/10.14421/jpai.2021.181-09

Wilson, M. (2005). Constructing measure: An item respon modeling approach. Larence Erbaum Associates.

DOI: https://doi.org/10.20527/jipf.v10i2.18906