Usability Evaluation of a School Library OPAC Using Heuristic Evaluation and User Testing

Authors

  • Faradina Faradina Indonesia
  • Taqwa Hariguna Indonesia
  • Fandy Setyo Utomo Indonesia
Pages Icon

DOI:

https://doi.org/10.63158/journalisi.v8i1.1528

Keywords:

Online Public Access Catalog (OPAC), Heuristic Evaluation, User Testing, System Usability Scale, Mental Models, Vocational High School Library

Abstract

This study evaluates the usability of the Online Public Access Catalog (OPAC) at SMK Negeri 1 Purwokerto to address the persistent gap between traditional library information architectures and the modern search behaviors of vocational students within the Kurikulum Merdeka ecosystem. The research aims to solve the problem of "mental model dissonance" that hinders independent information literacy among digital native learners. A hybrid evaluation approach was employed, integrating a Heuristic Evaluation by three experts with empirical User Testing involving students. The study utilized the Think-Aloud protocol and the System Usability Scale (SUS) to capture both performance and perception data. Result: The expert inspection identified 18 significant usability violations, primarily in library technical jargon (H2) and error prevention (H5). Empirical testing revealed a low average Task Success Rate (TSR) of 49.3% and a mean SUS score of 55.0, placing the system in the "Unacceptable" category. These figures confirm that current cataloging logic significantly obstructs retrieval efficiency. The originality of this research lies in the identification of specific dissonance points between vocational students' mental models and bibliographic metadata. It provides a strategic framework for interface restructuring through semantic simplification and department-based navigation, offering a practical model for developing user-centric "smart" library services in vocational education.

Downloads

Download data is not yet available.

References

[1] P. W. Jordan and Philips Corporate Design, Eds., Usability Evaluation in Industry: Based on the International Seminar Usability Evaluation in Industry that Was Held at Eindhoven, The Netherlands, on 14 and 15 September 1994. London: Taylor & Francis, 1996.

[2] S. B. Nyambaka and C. Mutwiri, "Usability of KOHA OPAC by undergraduate users for information retrieval with regard to usability testing in Technical University of Kenya Library," Int. J. Curr. Appl. Biol., vol. 7, no. 1, pp. 37–50, Mar. 2023, doi: 10.35942/ijcab.v7i1.302.

[3] A. Safitri, T. Avini, and Sugeng, "Eksplorasi desain user interface online public access catalog (OPAC) perpustakaan berbasis website dengan Figma," J. Komput. dan Teknol. Inf., vol. 3, no. 2, pp. 85–89, Jun. 2024, doi: 10.69916/jkbti.v3i2.132.

[4] B. Ghina, H. Ma’ruf, and J. Wong, "Analisis user experience terhadap website perpustakaan Universitas Gunadarma dengan metode heuristic evaluation," J. Teknol. Inform., vol. 18, 2019.

[5] O. Mardatillah and J. Prayudha, "The importance of critical thinking skills to vocational students: Strategies and benefits," Voc. Educ. Technol. J., vol. 5, no. 2, pp. 115–125, Apr. 2024, doi: 10.38038/vocatech.v5i2.136.

[6] A. G. Imana and Y. S. Nugroho, "UX (User Experience) evaluation of the Openlearning system at Universitas Muhammadiyah Surakarta using heuristic evaluation and usability testing," J. Tek. Inform. (JUTIF), vol. 4, no. 4, pp. 681–691, Aug. 2023, doi: 10.52436/1.jutif.2023.4.4.824.

[7] S. Beyer, Wie benutzerfreundlich sind OPACs?: Evaluation von WebOPACs mit Usability-Methoden, doctoral dissertation, Hochschule Hannover, 2005.

[8] M. Ruzegea, "The usability of OPAC interface features: The perspective of postgraduate students at International Islamic University Malaysia (IIUM)," Lib. Philos. Pract., 2012.

[9] M. L. Akanbi and Q. A. Bankole, "Perception and use of OPAC by users in academic libraries in Kwara State, Nigeria," Record Libr. J., accessed Feb. 11, 2026.

[10] M. R. Fachrezi, H. Aryanda, M. Farhan, D. Aqila, and A. D. Walidein, "Analisis usability aplikasi Perpustakaan Nasional melalui pendekatan heuristic evaluation berbasis human computer interaction," J. Teknol. Inf., vol. 6, no. 2, 2025.

[11] M. Adellia and F. Purwaningtyas, "Layanan cerdas di Perpustakaan UNPRI menggunakan sistem usability scale (SUS)," J. Pustaka Budaya, vol. 13, no. 1, pp. 26–41, 2026.

[12] T. F. Nisa’ and M. S. Arianto, "Evaluation of online public access catalog interface 'MyPustaka' in the Library of Muhammadiyah University of Yogyakarta (UMY)," PJIL, vol. 16, no. 1, pp. 72–85, Jun. 2025, doi: 10.20473/pjil.v16i1.71639.

[13] O. Vaherdi, H. M. Az-Zahra, and A. Syawli, "Evaluasi usability antarmuka pengguna pada aplikasi mobile E-Open Disdukcapil Kota Bekasi dengan metode usability testing dan heuristic evaluation," J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 9, no. 4, 2025.

[14] N. Afiah, L. Syahadiyanti, and C. A. Swastyastu, "Pengukuran tingkat kepuasan kinerja sistem informasi perpustakaan pada aplikasi OPAC menggunakan metode TAM dan EUCS," ZONAsi: J. Sist. Inform., vol. 6, no. 1, pp. 144–153, 2024, doi: 10.31849/zn.v6i1.18083.

[15] F. K. S. Dewi, T. A. P. Sidhi, and Y. C. Darmawan, "Analisis usability web SIATMA dengan metode heuristic evaluation dan system usability scale," J. Bibliotekas Inform., vol. 14, no. 02, pp. 87–96, Oct. 2023, doi: 10.24002/jbi.v14i02.5027.

[16] L. S. Green and K. Chassereau, "Modeling guided inquiry and school librarian instructional partnerships to pre-service teachers through digital video production," J. Educ. Libr. Inf. Sci., vol. 64, no. 2, pp. 185–205, Apr. 2023, doi: 10.3138/jelis-2022-0015.

[17] I. Rohmah, S. H. Wijoyo, and H. M. Az-Zahra, "Evaluasi website Institut Agama Islam Negeri (IAIN) Jember menggunakan metode Human Centered Design (HCD) dan System Usability Scale (SUS)," J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 9, pp. 3924–3933, 2021.

[18] M. Riedlinger and G. Thomas, "Usability testing your writing: Using the think-aloud protocol to get constructive reader feedback," in Teaching Science Students to Communicate: A Practical Guide, S. Rowland and L. Kuchel, Eds., Cham: Springer, 2023, pp. 207–212, doi: 10.1007/978-3-030-91628-2_25.

[19] S. Iqbal, N. Ikram, S. Imtiaz, and S. Imtiaz, "Maximizing coverage, reducing time: A usability evaluation method for web-based library systems," Sci. Rep., vol. 12, no. 1, p. 7285, May 2022, doi: 10.1038/s41598-022-11215-7.

[20] M. I. Burhan, A. K. Jaya, and L. Adira, "Bayesian intelligent tutoring system for vocational high schools," JIIT, pp. 1–13, Mar. 2024, doi: 10.51557/pt_jiit.v9i1.2415.

[21] Suparmin, Suharno, and Sukatiman, "Teachers' innovative strategies in integrating LMS for practical learning: A case study in vocational high schools," J. Educ. Res. (JERE), vol. 9, no. 3, pp. 452–462, Aug. 2025, doi: 10.23887/jere.v9i3.97819.

[22] M. A. F. Kamil, "User experience analysis of LinkedIn social media using usability metric for user experience (UMUX)," J. Inform. Eng. Educ. Technol., vol. 7, no. 2, pp. 78–82, Dec. 2023, doi: 10.26740/jieet.v7n2.p78-82.

[23] B. Klug, "An overview of the system usability scale in library website and system usability testing," Weave: J. Library User Exp., vol. 1, no. 6, Apr. 2017, doi: 10.3998/weave.12535642.0001.602.

[24] C. Jimenez, P. Lozada, and P. Rosas, "Usability heuristics: A systematic review," in 2016 IEEE 11th Colombian Computing Conference (CCC), Sep. 2016, pp. 1–8, doi: 10.1109/CCC.2016.7745955.

[25] C. Foltz, N. Schneider, B. Kausch, M. Wolf, C. Schlick, and H. Luczak, "Usability engineering," in Collaborative and Distributed Chemical Engineering. From Understanding to Substantial Design Process Support: Results of the IMPROVE Project, pp. 527–554, Berlin, Heidelberg: Springer, 2008.

[26] O. Alhadreti and P. Mayhew, "“Thinking about thinking aloud”: An investigation of think-aloud methods in usability testing," in Proceedings of the 30th International BCS Human Computer Interaction Conference, BCS Learning & Development, Jul. 2016, doi: 10.14236/ewic/HCI2016.101.

[27] Z. Soltani and N. J. Navimipour, "Customer relationship management mechanisms: A systematic review of the state-of-the-art literature and recommendations for future research," Comput. Hum. Behav., vol. 61, pp. 667–688, Aug. 2016, doi: 10.1016/j.chb.2016.03.008.

[28] A. Tavares, "The road less travelled: Exploring the reluctance of vocational students towards higher education in Portugal," Eur. J. Educ., Wiley Online Library, 2024, accessed Feb. 12, 2026.

Downloads

Published

2026-03-03

Issue

Vol. 8 No. 1 (2026): February

Section

Articles

License

Copyright (c) 2026 Journal of Information Systems and Informatics

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors Declaration

  1. The Authors certify that they have read, understood, and agreed to the Journal of Information Systems and Informatics (JournalISI) submission guidelines, policies, and submission declaration. The submission has been prepared using the provided template.
  2. The Authors certify that all authors have approved the publication of this manuscript and that there is no conflict of interest.
  3. The Authors confirm that the manuscript is their original work, has not received prior publication, is not under consideration for publication elsewhere, and has not been previously published.
  4. The Authors confirm that all authors listed on the title page have contributed significantly to the work, have read the manuscript, attest to the validity and legitimacy of the data and its interpretation, and agree to its submission.
  5. The Authors confirm that the manuscript is not copied from or plagiarized from any other published work.
  6. The Authors declare that the manuscript will not be submitted for publication in any other journal or magazine until a decision is made by the journal editors.
  7. If the manuscript is finally accepted for publication, the Authors confirm that they will either proceed with publication immediately or withdraw the manuscript in accordance with the journal’s withdrawal policies.
  8. The Authors agree that, upon publication of the manuscript in this journal, they transfer copyright or assign exclusive rights to the publisher, including commercial rights

How to Cite

[1]
F. Faradina, T. Hariguna, and F. S. Utomo, “Usability Evaluation of a School Library OPAC Using Heuristic Evaluation and User Testing”, journalisi, vol. 8, no. 1, pp. 1105–1123, Mar. 2026, doi: 10.63158/journalisi.v8i1.1528.