Exploring Generative AI Tools Frequency: Impacts on Attitude, Satisfaction, and Competency in Achieving Higher Education Learning Goals
DOI:
https://doi.org/10.35877/454RI.eduline2592Keywords:
ChatGPT, Student's attitudes, Satisfaction Levels, AI Competency, Learning GoalsAbstract
The urgency of using AI in the educational environment as a medium for optimizing student learning personalization in synchronous and asynchronous learning needs to be done to ensure that students experience improved attitudes, motivation, and learning satisfaction, which is reflected in student competencies. This study aimed to investigate the impact of ChatGPT use frequency on students' attitudes, satisfaction levels, and competence in higher education learning goals. The type of research employed was non-experimental quantitative research, specifically ex-post facto research. The sample size was 257 people, which was determined using the criteria of Issac and Michael (1971) and sampling distribution with a proportional random sampling technique. Data collection techniques that utilize questionnaires are effective in obtaining quantitative data. The PLS SEM analysis was used to determine the effect of exogenous latent variables on endogenous latent variables. The study revealed that reliability and satisfaction with ChatGPT positively impact learning outcomes, while the perceived impact on competence does not significantly enhance learning objectives. Increased usage frequency moderates these effects, diminishing the positive influence of perceptions, reliability, and satisfaction and shifting the impact on competence to negative and insignificant. These findings highlight the complexity of integrating ChatGPT into education, suggesting that initial positive perceptions may not be sustained with frequent use.
References
Ali, J., Shamsan, M., Hezam, T., & Mohammed, A. (2023). Impact of chatgpt on learning motivation. Journal of English Studies in Arabia Felix, 2(1), 41–49. https://doi.org/10.56540/jesaf.v2i1.51
Bai, S. (2023). Foreign language speaking anxiety among chinese english majors: causes, effects and strategies. Journal of Education Humanities and Social Sciences, 8, 2433–2438. https://doi.org/10.54097/ehss.v8i.5009
Belcher, B., & Halliwell, J. (2021). Conceptualizing the elements of research impact: towards semantic standards. Humanities and Social Sciences Communications, 8(1). https://doi.org/10.1057/s41599-021-00854-2
Busch, F., Hoffmann, L., Truhn, D., Ortiz-Prado, E., Makowski, M. R., Bressem, K. K., Adams, L. C., & Consortium, C. (2023). Medical students’ perceptions towards artificial intelligence in education and practice: A multinational, multicenter cross-sectional study. In medRxiv (p. 2023.12.09.23299744). https://doi.org/10.1101/2023.12.09.23299744
Chai, C. S., Rahmawati, Y., & Jong, M. S. Y. (2020). Indonesian science, mathematics, and engineering preservice teachers’ experiences in stem-tpack design-based learning. Sustainability (Switzerland), 12(21), 1–14. https://doi.org/10.3390/su12219050
Chan, K. S., & Zary, N. (2019). Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review. JMIR Medical Education, 5(1), e13930. https://doi.org/10.2196/13930
Cukurova, M., Luckin, R., & Kent, C. (2020). Impact of an Artificial Intelligence Research Frame on the Perceived Credibility of Educational Research Evidence. International Journal of Artificial Intelligence in Education, 30(2), 205–235. https://doi.org/10.1007/s40593-019-00188-w
Demir, K., & Güraksın, G. E. (2022). Determining middle school students’ perceptions of the concept of artificial intelligence: A metaphor analysis. Participatory Educational Research, 9(2), 297–312. https://doi.org/10.17275/per.22.41.9.2
Elder, H. (2012). An examination of Māori tamariki (child) and taiohi (adolescent) traumatic brain injury within a global cultural context. Australasian Psychiatry, 20(1), 20–23. https://doi.org/10.1177/1039856211430147
Fietta, V., Zecchinato, F., Stasi, B. Di, Polato, M., & Monaro, M. (2022). Dissociation Between Users’ Explicit and Implicit Attitudes Toward Artificial Intelligence: An Experimental Study. IEEE Transactions on Human-Machine Systems, 52(3), 481–489. https://doi.org/10.1109/THMS.2021.3125280
García-Martínez, I., Fernández-Batanero, J. M., Fernández-Cerero, J., & León, S. P. (2023). Analysing the Impact of Artificial Intelligence and Computational Sciences on Student Performance: Systematic Review and Meta-analysis. Journal of New Approaches in Educational Research, 12(1), 171–197. https://doi.org/10.7821/naer.2023.1.1240
Ghozali, I. nd. (2012). Fuad. Badan Penerbit Universitas Diponegoro.
Gilson, A., Safranek, C., Huang, T., Socrates, V., Chi, L., Taylor, A., & Chartash, D. (2023). How does chatgpt perform on the united states medical licensing examination? the implications of large language models for medical education and knowledge assessment. Jmir Medical Education, 9, 45312. https://doi.org/10.2196/45312
Gupta, K. P., & Bhaskar, P. (2020). Inhibiting and Motivating Factors Influencing Teachers’ Adoption of Ai-Based Teaching and Learning Solutions: Prioritization Using Analytic Hierarchy Process. Journal of Information Technology Education: Research, 19, 693–723. https://doi.org/10.28945/4640
Hair, J. F. (2017). A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM (J. F. Hair (ed.); Second). Sage.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate data analysis (6th ed.). Pearson Prentice Hall.
Hair, J. F., Risher, J. J., Sarstedt, M., & Ringle, C. M. (2019). When to use and how to report the results of PLS-SEM. European Business Review, 31(1), 2–24. https://doi.org/10.1108/EBR-11-2018-0203
Hair Jr., J. F., Matthews, L. M., Matthews, R. L., & Sarstedt, M. (2017). PLS-SEM or CB-SEM: updated guidelines on which method to use. International Journal of Multivariate Data Analysis, 1(2), 107. https://doi.org/10.1504/ijmda.2017.10008574
Hajam, K. B., & Gahir, S. (2024). Unveiling the Attitudes of University Students Toward Artificial Intelligence. Journal of Educational Technology Systems, 52(3), 335–345. https://doi.org/10.1177/00472395231225920
Han, Z., Battaglia, F., Udaiyar, A., Fooks, A., & Terlecky, S. (2023). An explorative assessment of chatgpt as an aid in medical education: use it with caution. https://doi.org/10.1101/2023.02.13.23285879
Hegarty, B., & Thompson, M. (2019). A teacher’s influence on student engagement: Using smartphones for creating vocational assessment ePortfolios. Journal of Information Technology Education: Research, 18, 113–159. https://doi.org/10.28945/4244
Huisman, M., Ranschaert, E., Parker, W., Mastrodicasa, D., Koci, M., Pinto de Santos, D., Coppola, F., Morozov, S., Zins, M., Bohyn, C., Koç, U., Wu, J., Veean, S., Fleischmann, D., Leiner, T., & Willemink, M. J. (2021). An international survey on AI in radiology in 1,041 radiologists and radiology residents part 1: fear of replacement, knowledge, and attitude. European Radiology, 31(9), 7058–7066. https://doi.org/10.1007/s00330-021-07781-5
Ibrahim, H., Liu, X., & Denniston, A. K. (2021). Reporting guidelines for artificial intelligence in healthcare research. Clinical and Experimental Ophthalmology, 49(5), 470–476. https://doi.org/10.1111/ceo.13943
Isaac, S., & Michael, W. B. (1971). Handbook in research and evaluation. In Behavior Therapy (Vol. 2, Issue 4). Knapp. https://doi.org/10.1016/s0005-7894(71)80129-6
Islam, M. A., Aldaihani, F. M. F., & Saatchi, S. G. (2023). Artificial intelligence adoption among human resource professionals: Does market turbulence play a role? Global Business and Organizational Excellence, 42(6), 59–74. https://doi.org/10.1002/joe.22226
Karan, B., & Angadi, G. R. (2023). Potential Risks of Artificial Intelligence Integration into School Education: A Systematic Review. Bulletin of Science, Technology and Society, 43(3–4), 67–85. https://doi.org/10.1177/02704676231224705
Kashive, N., Powale, L., & Kashive, K. (2021). Understanding user perception toward artificial intelligence (AI) enabled e-learning. International Journal of Information and Learning Technology, 38(1), 1–19. https://doi.org/10.1108/IJILT-05-2020-0090
Kerlinger, F. (1986). Foundations of {Behavioral} {Research}. Foundations of Behavioral Research, 3.
Ketchen, D. J. (2013). A Primer on Partial Least Squares Structural Equation Modeling. In Long Range Planning (Vol. 46, Issues 1–2, pp. 184–185). https://doi.org/10.1016/j.lrp.2013.01.002
Khairatun Hisan, U., & Miftahul Amri, M. (2022). Artificial Intelligence for Human Life: A Critical Opinion from Medical Bioethics Perspective – Part II. Journal of Public Health Sciences, 1(02), 112–130. https://doi.org/10.56741/jphs.v1i02.215
Kohnke, L., Moorhouse, B., & Zou, D. (2023). Chatgpt for language teaching and learning. Relc Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868
Li, G., Zarei, M. A., Alibakhshi, G., & Labbafi, A. (2024). Teachers and educators’ experiences and perceptions of artificial-powered interventions for autism groups. In BMC Psychology (Vol. 12, Issue 1). https://doi.org/10.1186/s40359-024-01664-2
McCoy, L. G., Nagaraj, S., Morgado, F., Harish, V., Das, S., & Celi, L. A. (2020). What do medical students actually need to know about artificial intelligence? Npj Digital Medicine, 3(1). https://doi.org/10.1038/s41746-020-0294-7
Muthmainnah, Ibna Seraj, P. M., & Oteir, I. (2022). Playing with AI to Investigate Human-Computer Interaction Technology and Improving Critical Thinking Skills to Pursue 21stCentury Age. Education Research International, 2022, 1–17. https://doi.org/10.1155/2022/6468995
Nazaretsky, T., Ariely, M., Cukurova, M., & Alexandron, G. (2022). Teachers’ trust in ai‐powered educational technology and a professional development program to improve it". British Journal of Educational Technology, 53(4), 914–931. https://doi.org/10.1111/bjet.13232
Niloy, A. C., Akter, S., Sultana, N., Sultana, J., & Rahman, S. I. U. (2024). Is Chatgpt a menace for creative writing ability? An experiment. Journal of Computer Assisted Learning, 40(2), 919–930. https://doi.org/10.1111/jcal.12929
Nunnally, B., & Bernstein, I. R. (1994). Psychometric Theory. Oxford University Press.
Oh, S., Kim, J. H., Choi, S. W., Lee, H. J., Hong, J., & Kwon, S. H. (2019). Physician confidence in artificial intelligence: An online mobile survey. Journal of Medical Internet Research, 21(3), 12422. https://doi.org/10.2196/12422
Raman, R., Mandal, S., Das, P., Kaur, T., JP, S., & Nedungadi, P. (2023). University students as early adopters of chatgpt: innovation diffusion study. https://doi.org/10.21203/rs.3.rs-2734142/v1
Raut, R. K., & Kumar, S. (2024). An integrated approach of TAM and TPB with financial literacy and perceived risk for influence on online trading intention. Digital Policy, Regulation and Governance , 26(2), 135–152. https://doi.org/10.1108/DPRG-07-2023-0101
Roy, R., Babakerkhell, M. D., Mukherjee, S., Pal, D., & Funilkul, S. (2022). Evaluating the Intention for the Adoption of Artificial Intelligence-Based Robots in the University to Educate the Students. IEEE Access, 10, 125666–125678. https://doi.org/10.1109/ACCESS.2022.3225555
Sadler, T. D., Mensah, F. M., & Tam, J. (2024). Artificial intelligence and the Journal of Research in Science Teaching. Journal of Research in Science Teaching, 61(4), 739–743. https://doi.org/10.1002/tea.21933
Salah, M., Alhalbusi, H., Abdelfattah, F., & Ismail, M. M. (2023). Chatting with ChatGPT: Investigating the Impact on Psychological Well-being and Self-esteem with a Focus on Harmful Stereotypes and Job Anxiety as Moderator. In Research Square. https://doi.org/10.21203/rs.3.rs-2610655/v1
Salas-Pilco, S. Z. (2020). The impact of AI and robotics on physical, social-emotional and intellectual learning outcomes: An integrated analytical framework. British Journal of Educational Technology, 51(5), 1808–1825. https://doi.org/10.1111/bjet.12984
Santosa, P. I. (2018). Metode Penelitian Kuantitatif Pengembangan Hipotesis dam Pengujiannya Menggunakan SmartPLS (1st ed.). ANDI. https://openlibrary.telkomuniversity.ac.id/pustaka/143392/metode-penelitian-kuantitatif-pengembangan-hipotesis-dan-pengujiannya-menggunakan-smartpls.html
Sarwar, S., Dent, A., Faust, K., Richer, M., Djuric, U., Van Ommeren, R., & Diamandis, P. (2019). Physician perspectives on integration of artificial intelligence into diagnostic pathology. Npj Digital Medicine, 2(1). https://doi.org/10.1038/s41746-019-0106-0
Skulimowski, A. M. J. (2021). Visions of a Future Research Workplace Arising from Recent Foresight Exercises (pp. 169–185). https://doi.org/10.1007/978-3-030-66262-2_11
Slimi, Z. (2021). The impact of AI implementation in higher education on educational process future: A systematic review. In ResearchSquare. https://doi.org/10.21203/rs.3.rs-1081043
Smith, K., & Hill, J. (2019). Defining the Nature of Blended Learning through Its Depiction in Current Research. Higher Education Research and Development, 38(2), 383–397. https://doi.org/10.1080/07294360.2018.1517732.
Stamate, A. N., Sauvé, G., & Denis, P. L. (2021). The rise of the machines and how they impact workers’ psychological health: An empirical study. Human Behavior and Emerging Technologies, 3(5), 942–955. https://doi.org/10.1002/hbe2.315
Suh, W., & Ahn, S. (2022). Development and Validation of a Scale Measuring Student Attitudes Toward Artificial Intelligence. SAGE Open, 12(2), 215824402211004. https://doi.org/10.1177/21582440221100463
Tedeschi, L. O. (2022). ASAS-NANP symposium: Mathematical modeling in animal nutrition: The progression of data analytics and artificial intelligence in support of sustainable development in animal science. Journal of Animal Science, 100(6). https://doi.org/10.1093/jas/skac111
Wang, X., Gong, Z., Wang, G., Jia, J., Xu, Y., Zhao, J., & Li, X. (2023). Chatgpt performs on the chinese national medical licensing examination. https://doi.org/10.21203/rs.3.rs-2584079/v1
Wang, Y., Shen, H., & Chen, T. (2023). Performance of chatgpt on the pharmacist licensing examination in taiwan. Journal of the Chinese Medical Association, 86(7), 653–658. https://doi.org/10.1097/jcma.0000000000000942
Wienrich, C., & Carolus, A. (2021). Development of an Instrument to Measure Conceptualizations and Competencies About Conversational Agents on the Example of Smart Speakers. Frontiers in Computer Science, 3. https://doi.org/10.3389/fcomp.2021.685277
Wu, T. T., Lee, H. Y., Li, P. H., Huang, C. N., & Huang, Y. M. (2024). Promoting Self-Regulation Progress and Knowledge Construction in Blended Learning via ChatGPT-Based Learning Aid. Journal of Educational Computing Research, 61(8), 3–31. https://doi.org/10.1177/07356331231191125
Xuan, X. (2019). Recent advances in direct current electrokinetic manipulation of particles for microfluidic applications. Electrophoresis, 40(18–19), 2484–2513. https://doi.org/10.1002/elps.201900048
Zerfass, A., Hagelstein, J., & Tench, R. (2020). Artificial intelligence in communication management: a cross-national study on adoption and knowledge, impact, challenges and risks. Journal of Communication Management, 24(4), 377–389. https://doi.org/10.1108/JCOM-10-2019-0137
Zhang, P. (2023). Taking advice from chatgpt. https://doi.org/10.31234/osf.io/b53vn
Zhang, W. (2023). Exploring the differences and evolution of college students’ computational thinking in programming learning through data analysis. Computer Applications in Engineering Education, 31(5), 1433–1446. https://doi.org/10.1002/cae.22654
Zhou, Y., & Kankanhalli, A. (2021). AI Regulation for Smart Cities: Challenges and Principles. In Public Administration and Information Technology (Vol. 37, pp. 101–118). https://doi.org/10.1007/978-3-030-61033-3_5
Downloads
Published
How to Cite
Issue
Section
