Developing HOTS-Based Assessment Instrument for Science Learning in Junior High School: Validity, Reliability, and Item Analysis

Ismadi Sihombing, Ilham Adi Putra, Masringgit Marwiyah Nst, Festiyed Festiyed, Skunda Diliarosta, Fatni Mufit

Abstract


Despite growing policy demands for Higher Order Thinking Skills (HOTS) assessment in Indonesian science education, psychometrically validated instruments aligned with the Merdeka Curriculum remain scarce. This study aims to develop and validate a HOTS-based assessment instrument for eighth-grade junior high school (SMP) science students. Employing the Research and Development (R&D) approach with the 4-D model (Define, Design, Develop, Disseminate), the instrument was designed to comprehensively measure HOTS at cognitive levels C4 (Analyzing), C5 (Evaluating), and C6 (Creating) of the Revised Bloom’s Taxonomy, covering Basic Competency (BC) 3.8 (Respiratory System) and BC 3.9 (Photosynthesis) topics. The instrument consists of 40 two-tier multiple-choice (TTMC) items and 10 essay items. Content validity was assessed by three experts using Aiken’s V formula, yielding a mean V of 0.87 (highly valid). Reliability was estimated using the KR-20 formula in a large-group trial (n = 64), producing a coefficient of 0.82 (reliable), and Cronbach’s alpha of 0.79 for essay items. Item analysis encompassed difficulty index, discriminating power, and distractor effectiveness. Results showed that 34 of 40 TTMC items (85%) met all quality criteria simultaneously: moderate difficulty, good discriminating power, and effective distractors. The final instrument (34 TTMC + 10 essay items) is valid and reliable for measuring higher-order thinking skills among SMP students. The TTMC format provides added diagnostic value by revealing specific conceptual misconceptions in addition to measuring HOTS proficiency, aligning with the formative assessment emphasis of the Merdeka Curriculum.

Keywords


HOTS assessment; Instrument development; Item analysis; Junior high school; Two-tier multiple choice

Full Text:

PDF

References


Afifah, R. N., Oktaviya, U., Qoriroh, R., & Wahyuni, I. W. (2023). Analisis kemampuan berpikir kritis dalam menyelesaikan soal cerita berdasarkan kemampuan matematika siswa. Laplace : Jurnal Pendidikan Matematika, 6(1), 207–216. Https://Doi.Org/10.31537/Laplace.V6i1.1121

An, Y., Kaplan-Rakowski, R., Yang, J., Conan, J., Kinard, W., & Daughrity, L. A. (2021). Examining k-12 teachers’ feelings, experiences, and perspectives regarding online teaching during the early stage of the covid-19 pandemic. Educational Technology Research And Development, 69(5), 2589–2613. Https://Doi.Org/10.1007/S11423-021-10008-5

Anderson, L. W., & Krathwohl, D. R. (2021). A taxonomy for learning, teaching, and assessing: A revision of bloom’s taxonomy of educational objectives. Longman. Https://Doi.Org/Https://Doi.Org/10.51675/Jp.V3i1.128

Arifin, Z., & Retnawati, H. (2017). Pengembangan instrumen pengukur higher-order thinking skills matematika siswa SMA kelas X. PYTHAGORAS: Jurnal Pendidikan Matematika, 12(1), 98–108. https://doi.org/10.21831/pg.v12i1.14058

Asrafil, A., Retnawati, H., & Retnowati, E. (2020). The difficulties of students when solving hots problem and the description of students cognitive load after given worked example as a feedback. Journal Of Physics: Conference Series, 1511(1), 0–11. Https://Doi.Org/10.1088/1742-6596/1511/1/012092

Astuti, I. A. D., Bhakti, Y. B., & Prasetya, R. (2021). Four tier-magnetic diagnostic test (4t-Mdt): Instrumen evaluasi medan magnet untuk mengidentifikasi miskonsepsi siswa. Jipfri (Jurnal Inovasi Pendidikan Fisika Dan Riset Ilmiah), 5(2), 110–115. Https://Doi.Org/10.30599/Jipfri.V5i2.1205

Budi, B. Y., Agustina Dwi Astuti1, I., & Prasetya, R. (2022). Four-tier thermodynamics diagnostic test (4t-tdt) to identify student misconception. Kne Social Sciences, 106–116. Https://Doi.Org/10.18502/Kss.V7i14.11958

Dinni, H. N. (2018). Hots (High order thinking skills) dan kaitannya dengan kemampuan literasi matematika. Prisma, Prosiding Seminar Nasional Matematika, 1, 170–176.

Eliyasni, R., Kenedi, A. K., & Sayer, I. M. (2019). Blended learning and project based learning: the method to improve students’ higher order thinking skill (Hots). Jurnal Iqra’ : Kajian Ilmu Pendidikan, 4(2), 231–248. Https://Doi.Org/10.25217/Ji.V4i2.549

Fraenkel, J. R., Wallen, N. E., & Hyun, H. H. (2012). How to design and evaluate research in education (8th ed.). McGraw-Hill.

Fransiska, A., Prasetyo, E., & Jufriansah, A. (2021). Desain lkpd fisika terintegrasi hots untuk meningkatkan kemampuan berpikir kritis peserta didik. Jurnal Pendidikan Fisika Dan Teknologi, 7(2), 153–158. Https://Doi.Org/10.29303/Jpft.V7i2.3098

Hakiki, M., Fadli, R., Putra, Y. I., & Pertiwi, I. P. (2021). Pengembangan e-modul berorientasi higher order thinking skills (hots) pada pelajaran ilmu pengetahuan alam (ipa) kelas vi sekolah dasar. Jurnal Muara Pendidikan, 6(1), 50–57.

Https://Doi.Org/10.52060/Mp.V6i1.513

Haniah, A. R., Aman, A., & Setiawan, R. (2020). Integration of strengthening of character education and higher order thinking skills in history learning. Journal Of Education And Learning (Edulearn), 14(2), 183–190. Https://Doi.Org/10.11591/Edulearn.V14i2.15010

Haryanto, P. C., & Arty, I. S. (2019). The application of contextual teaching and learning in natural science to improve student’s hots and self-efficacy. Journal Of Physics: Conference Series, 1233(1). Https://Doi.Org/10.1088/1742-6596/1233/1/012106

Hidayah, N., Nurafiani, S., Handoko, A., Haka, N. B., Ningrum, A. R., & Hasanah, U. (2024). Analisis higher order thinking skills (hots) peserta didik melalui model pembelajaran search solve create and share (Sscs). Jurnal Riset Dan Inovasi Pembelajaran, 4(1), 183–192. Https://Doi.Org/10.51574/Jrip.V4i1.1234

Hua, W. Y., Wu, W. C., Nieh, H. P., & Chang, Y. H. (2024). Patterns of teachers’ responses to school bullying and their associations with training, self-efficacy, and age: A moderated mediation model. Behavioral Sciences And The Law, 42(6), 706–727. Https://Doi.Org/10.1002/Bsl.2692

Ichsan, I. Z., Sigit, D. V., Miarsyah, M., Ali, A., Arif, W. P., & Prayitno, T. A. (2019). Hots-Aep: Higher order thinking skills from elementary to master students in environmental learning. European Journal Of Educational Research, 8(4), 935–942.

Ichsan, I. Z., Sigit, D. V., Miarsyah, M., Ali, A., & Suwandi, T. (2020). Implementation supplementary book of green consumerism: improving students hots in environmental learning. European Journal Of Educational Research, 9(1), 227–237. Http://Www.Eu-Jer.Com

Kim, H. R., Zulnaidi, H., & Rahim, S. S. B. (2022). Development of higher-order thinking skills test instrument on quadratic equations (hots-qe) for secondary school students. Pegem Journal Of Education And Instruction, 13(1), 379–394.

Köksal, D., Ulum, Ö. G., & Yürük, N. (2023). Revised bloom’s taxonomy in reading texts in efl/esl settings. Acta Educationis Generalis, 13(1), 133–146. Https://Doi.Org/10.2478/Atd-2023-0007

Kurniawan, I., & Arnellis. (2019). Pengaruh penerapan model pembelajaran discovery learning terhadap kemampuan hots (high order thinkng skill) siswa kelas x mipa sman 3 padang. Jurnal Edukasi Dan Penelitian Matematika Hal, 8(4), 103–109.

Lestari, E., Cahyono, H., & Awaluddin, A. (2019). Penerapan model pembelajaran group investigation pada materi lingkaran untuk meningkatkan kemampuan berpikir kritis. Jurnal Math Educator Nusantara: Wahana Publikasi Karya Tulis Ilmiah Di Bidang Pendidikan Matematika, 5(2), 124–139.

Https://Doi.Org/10.29407/Jmen.V5i2.12814

Limatahu, I., Roini, C., & Limatahu, N. A. (2023). Ecoprint batik training for biology and craft teachers using local materials from ternate island. Transformasi: Jurnal Pengabdian Masyarakat, 19(2), 309–320. Https://Doi.Org/Https://Doi.Org/10.21009/Jtp.V26i1.45471

Marmaroti, P., & Galanopoulou, D. (2006). Pupils’ understanding of photosynthesis: A questionnaire for the simultaneous assessment of all aspects. International Journal of Science Education, 28(4), 383–403. https://doi.org/10.1080/09500690500277805

Mulyatna, F., Imswatama, A., & Rahmawati, N. D. (2021). Design Ethnic-math hots: mathematics higher order thinking skill questions based on culture and local wisdom. Malikussaleh Journal Of Mathematics Learning (Mjml), 4(1), 48–51.

Murni, D. (2013). Identifikasi miskonsepsi mahasiswa pada konsep substansi genetika menggunakan certainty of response index (CRI). Prosiding Semirata FMIPA Universitas Lampung, 1(1), 205–211.

Ningrum, A. P., Tindangen, M., & Rosifah, D. (2023, December). Upaya peningkatan kemampuan literasi peserta didik dengan menggunakan adaptasi model project base learning (pjbl), problem based learning (pbl), kooperatif, dan soal hots pada materi virus. Prosiding Seminar Nasional PPG Universitas Mulawarman, 4, 73-80.

OECD. (2023). PISA 2022 Results (Volume I): The state of learning and equity in education. OECD Publishing. https://doi.org/10.1787/53f23881-en

Permana, N. N., Setiani, A., & Nurcahyono, N. A. (2020). Analisis kemampuan penalaran adaptif siswa dalam menyelesaikan soal higher order thinking skills (hots). Jurnal Pengembangan Pembelajaran Matematika, 2(2), 51–60.

Pesman, H., & Eryılmaz, A. (2010). Development of a three-tier test to assess misconceptions about simple electric circuits. The Journal of Educational Research, 103(3), 208–222. https://doi.org/10.1080/00220670903383002

Purwasi, L. A., & Fitiyana, N. (2020). Peningkatan kemampuan berpikir tingkat tinggi siswa melalui pembelajaran matematika berbantuan lkpd berbasis hots. Jurnal Pendidikan Matematika (Judika Education), 3(2), 65–74. Https://Doi.Org/10.31539/Judika.V3i2.1594

Putri, R. I. I., Zulkardi, Z., & Riskanita, A. D. (2022). Students’ ability in solving PISA-like mathematics problems using the context of Jakabaring Sport City in terms of gender. Journal on Mathematics Education, 13(3), 511–526.

https://doi.org/10.22342/jme.v13i3.pp511-526

Rachmawati, D., Suharno, S., & Roemintoyo, R. (2023). The effects of learning design on learning activities based on higher order thinking skills in vocational high schools. Open Education Studies, 5(1), 20220202. Https://Doi.Org/10.1515/Edu-2022-0202

Ramadhan, S., Mardapi, D., Prasetyo, Z. K., & Utomo, H. B. (2019). The development of an instrument to measure the higher order thinking skill in physics. European Journal of Educational Research, 8(3), 743–751.

Rizki, C., & Setyarsih, W. (2022). Identifikasi miskonsepsi siswa dan penyebabnya pada materi elastisitas menggunakan three-tier diagnostic test. Inovasi Pendidikan Fisika, 11(3), 32–43. Https://Doi.Org/Https://Doi.Org/10.26740/Ipf.V11n3.P32-43

Rukmana, A. P., Mayasari, T., & Yusro, A. C. (2020). Pengembangan four-tier diagnostic test untuk mendeteksi miskonsepsi pada fisika sma. Snpf (Seminar Nasional Pendidikan Fisika).

Saputri, L., Maison, M., & Kurniawan, W. (2021). Pengembangan four-tier diagnostic test berbasis website untuk mengidentifikasi miskonsepsi pada materi suhu dan kalor. Jurnal Ilmiah Teknologi Informasi Asia, 15(1), 61. Https://Doi.Org/10.32815/Jitika.V15i1.563

Sholeh, M. I., & Olensia, Y. (2022). Strategi dan implementasi penyusunan soal hots kimia berbasis lesson study. Orbital: Jurnal Pendidikan Kimia, 6(1), 38–48. Https://Doi.Org/10.19109/Ojpk.V6i1.12169

Silaban, B., Surbakti, M. B., Josafat, I., & Silaban, A. (2024). Identifikasi miskonsepsi peserta didik sma melalui tes diagnostik four-tier pada hukum newton. Lensa : Jurnal Kependidikan Fisika, 12(2), 260–274.

Simorangkir, F. M. A., Sinaga, F. I. S. H., & H.S., D. W. S. (2021). The culture of numeric literacy with hots problems in mathematics learning. Journal Of Mathematics And Natural Sciences, 1(2), 52. Https://Doi.Org/10.24114/Jmns.V1i2.33218

Sipayung, K. T., Tampubolon, S., Napitupulu, F. D., & Tobing, F. L. (2021). Generic structure of descriptive text written english language learner and teachers. Linguistics And Culture Review, 5(S3), 1636–1650. Https://Doi.Org/10.21744/Lingcure.V5ns3.1950

Tajudin, N. M., & Chinnappan, M. (2016). The link between higher order thinking skills, representation and concepts in enhancing timss tasks. International Journal Of Instruction, 9(2), 199–214. Https://Doi.Org/10.12973/Iji.2016.9214a

Tambunan, H., & Naibaho, T. (2019). Performance of mathematics teachers to build students’ high order thinking skills (hots). Journal Of Education And Learning (Edulearn), 13(1), 111–117. Https://Doi.Org/10.11591/Edulearn.V13i1.11218

Tekkaya, C. (2002). Misconceptions as a barrier to understanding biology. Hacettepe Universitesi Egitim Fakultesi Dergisi, 23(23), 259–266.

Thiagarajan, S., Semmel, D. S., & Semmel, M. (1974). Instructional development for training teachers of exceptional children: A sourcebook. Center For Innovation In Teaching The Handicapped. Https://Doi.Org/10.1016/0022-4405(76)90066-2

Treagust, D. F. (1988). Development and use of diagnostic tests to evaluate students' misconceptions in science. International Journal of Science Education, 10(2), 159–169. https://doi.org/10.1080/0950069880100204

Tumanggor, F. L. L. (2023). Efektivitas model pembelajaran problem based learning (pbl) terhadap kemampuan berpikir tingkat tinggi (hots) siswa pada materi persamaan kuadrat kelas ix smp gajah mada t.a. 2022/2023. Universitas Hkbp Nommensen, 2007(2015).

Wardani, N., & Siregar, T. J. (2023). The development of hots (higher-other thinking skills) assessment based on numeration literacy in mathematics for junior high school. Indonesian Journal Of Science And Mathematics Education, 6(2), 238–252. Https://Doi.Org/10.24042/Ijsme.V6i2.17433

Wismayani, P. N. P., Sulistia Dewi, N. L. P. E., & Paramartha, A. A. G. Y. (2019). The reflection of hots in efl teachers’ summative assessment. Journal Of Education Research And Evaluation, 3(3), 127–133.

Witraguna, K. Y., Setiawati, G. A. D., Wahyuni, N. N. T., Jaya, I. K. M. A., & Mediani, N. K. A. A. (2024). Learning in the merdeka curriculum: elementary school teachers’ understanding of differentiated learning. International Journal Of Elementary Education, 8(1), 47–56. Https://Doi.Org/10.23887/Ijee.V8i1.69779

Zhou, T., & Zhang, W. (2022). Effectiveness study on online or blended language learning based on student achievement: A systematic review of empirical studies. Sustainability, 14(12), 7303. Https://Doi.Org/10.3390/Su14127303




DOI: https://doi.org/10.20527/jmscedu.v6i1.18792

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 International License.

 

Creative Commons License

Journal Of Mathematics Science and Computer Education is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.