Examining the Relationship Between AI Chatbot Use Frequency and English Speaking Skills among the 2023 Cohort of English Education Students at UIN Datokarama Palu

Linda; Nur Asmawati; Ma'rifah Nurmala; Zul Aini Rengur

Authors

Linda Universitas Islam Negeri Datokarama Palu, Indonesia
Nur Asmawati Universitas Islam Negeri Datokarama Palu, Indonesia
Ma'rifah Nurmala Universitas Islam Negeri Datokarama Palu, Indonesia
Zul Aini Rengur Universitas Islam Negeri Datokarama Palu, Indonesia

Keywords:

Artificial Intelligence Chatbots, English Language Education, English Speaking Skills, Fluency, Pronunciation, Quantitative Correlational Study

Abstract

The rapid adoption of artificial intelligence (AI) chatbots in English language learning has created new opportunities for autonomous speaking practice; however, empirical evidence regarding whether the frequency of AI chatbot use is associated with learners’ speaking skills remains limited. This study aimed to examine the relationship between AI chatbot use frequency and the English speaking skills of the 2023 cohort students in the English Language Education Program at UIN Datokarama Palu. A quantitative correlational design was employed involving 15 students selected through purposive sampling based on predetermined inclusion criteria. Data were collected using a validated questionnaire to measure AI chatbot use frequency and a speaking test assessing two indicators, namely fluency and pronunciation. The data were analyzed using descriptive statistics, the Shapiro–Wilk normality test, the linearity test, and the Pearson Product Moment correlation test with the assistance of IBM SPSS. The findings indicated that the participants demonstrated relatively high AI chatbot use frequency and fairly good English speaking skills. However, the Pearson Product Moment correlation analysis revealed no statistically significant relationship between AI chatbot use frequency and English speaking skills (r = 0.010, p = 0.972). These findings suggest that frequent AI chatbot use alone does not necessarily improve students’ fluency and pronunciation. Therefore, AI chatbots should be integrated with structured pedagogical strategies and meaningful communicative activities to optimize their contribution to English speaking development in higher education.

Abstract views: 25 ,

PDF downloads: 10

Downloads

Download data is not yet available.

References

Andrea, P., Palencia, E., Patricia, K., Arteaga, A., Alfonso, T., & Cama, P. (2026). Artificial intelligence chatbots for inclusive pedagogical practices: A PRISMA systematic review and bibliometric mapping (2021–2025). Journal of Artificial Intelligence and Technology, 6(2), 1–10. https://doi.org/10.37965/jait.2026.1252

Astuti, E. S. (2019). Faktor-faktor yang mempengaruhi speaking performance mahasiswa jurusan pendidikan bahasa Inggris. Paradigma: Jurnal Filsafat, Sains, Teknologi, dan Sosial Budaya, 25(2), 27–33. https://doi.org/10.33503/paradigma.v25i2.543

Bahari, A., Han, F., & Strzelecki, A. (2025). Integrating CALL and AIALL for an interactive pedagogical model of language learning. Education and Information Technologies, 30, 14305–14333. https://doi.org/10.1007/s10639-025-13388-w

Brown, H. D. (2004). Language assessment: Principles and classroom practices. Pearson Education.

Caiza, G., Villafuerte, C., & Guanuche, A. (2025). Interactive application with virtual reality and artificial intelligence for improving pronunciation in English learning. Applied Sciences, 15(17), 1–18. https://doi.org/10.3390/app15179270

Du, J., & Kei, B. (2024). Transforming language education: A systematic review of AI-powered chatbots for English as a foreign language speaking practice. Computers and Education: Artificial Intelligence, 6, Article 100230. https://doi.org/10.1016/j.caeai.2024.100230

Education First. (2025). EF English Proficiency Index. https://www.ef.com/wwen/epi/

Han, J., & Lee, D. (2024). Research on the development of principles for designing elementary English speaking lessons using artificial intelligence chatbots. Humanities and Social Sciences Communications, 11(1), 1–15. https://doi.org/10.1057/s41599-024-02646-w

Harmer, J. (2007). How to teach English (2nd ed.). Pearson Education Limited.

Karataş, F., Abedi, F. Y., Ozek Gunyel, F., Karadeniz, D., & Kuzgun, Y. (2024). Incorporating AI in foreign language education: An investigation into ChatGPT's effect on foreign language learners. Education and Information Technologies, 29(15), 19343–19366. https://doi.org/10.1007/s10639-024-12574-6

Laun, M., & Wolff, F. (2025). Chatbots in education: Hype or help? A meta-analysis. Learning and Individual Differences, 119, Article 102646. https://doi.org/10.1016/j.lindif.2025.102646

Liu, J., Wumaier, A., Fan, C., & Guo, S. (2023). Automatic fluency assessment method for spontaneous speech without reference text. Electronics, 12(8). https://doi.org/10.3390/electronics12081775

Lo, C. K. (2023). What is the impact of ChatGPT on education? A rapid review of the literature. Education Sciences, 13(4). https://doi.org/10.3390/educsci13040410

Loewen, S., & Sato, M. (2018). Interaction and instructed second language acquisition. Language Teaching, 51(3), 285–329. https://doi.org/10.1017/S0261444818000125

Maulana Oscard Nasution, A. (2024). Exploring the role of AI chatbot in English language teaching and learning. Journal of English Language and Education, 9(5), 34–43. https://doi.org/10.31004/jele.v9i5.513

Mazhar, S. A., Anjum, R., Anwar, A. I., & Khan, A. A. (2021). Methods of data collection: A fundamental tool of research. Journal of Integrated Community Health, 10(1), 6–10. https://doi.org/10.24321/2319.9113.202101

Miao, F., Holmes, W., Huang, R., & Zhang, H. (2021). AI and education: Guidance for policy-makers (F. Miao, Ed.). United Nations Educational, Scientific and Cultural Organization (UNESCO). https://doi.org/10.54675/PCSP7350

Mojolou, D. N., Madli, F., Abdullah, A. S., Thomas, H., & Nayve, R. (2025). Transforming student engagement in higher education: The impact of artificial intelligence on learning involvement. ASEAN Artificial Intelligence Journal, 3(1), 58–65. https://karyailham.com.my/index.php/aaij/index

Mushaddiq, M., Noni, N., Abdullah, A., & Tahir, M. (2024). Exploring EFL students' creativity and engagement with AI chatbot in web-based English course. GLENS: Global English Insights Journal, 2(1), 19–27. https://doi.org/10.61220/glens.v2i1.591

Pannell, J., Partsch, F., & Fuller, N. (2017). The output hypothesis: From theory to practice. TESOL Working Paper Series, 15, 126-159.

Pebriani, Y., & Serasi, R. (2024). The correlation between AI (artificial intelligence) and students' speaking performance in the EFL classroom. Pendas: Jurnal Ilmiah Pendidikan Dasar, 9. https://doi.org/10.23969/jp.v9i3.18365

Prayogo, L., Agustin, R. K., & Puspita, B. (2025). Integrating artificial intelligence in English language learning: A systematic literature review. Journal of English Education, 5(2), 75–88. https://doi.org/10.30998/jedu.v5i2.14647

Rizal Bagus Syaifulloh, S. N. Y. (2025). AI chatbots in EFL: Synthesizing cognitive, psychological, and ethical implications. EDUTECH: Jurnal Inovasi Pendidikan Berbantuan Teknologi, 5(1), 113–119. https://doi.org/10.51878/edutech.v5i1.4701

Safitri, E. I., Hidayati, S., & Ciptaningrum, D. S. (2025). The impact of AI chatbots on English language learners' speaking proficiency: A systematic review. Journal of Research on English and Language Learning (J-REaLL), 6(2), 317–329. https://doi.org/10.33474/j-reall.v6i2.23866

Shehadeh, A. (2023). Review of Interaction, feedback and task research in second language learning: Methods and design (A. Mackey, 2020). Canadian Journal of Applied Linguistics, 26(1). https://doi.org/10.37213/cjal.2023.32147

Stoughton, A. M., & Kang, O. (2024). A systematic review of empirical mobile-assisted pronunciation studies through a perception–production lens. Languages, 9. https://doi.org/10.3390/languages9070251

Sugiyono. (2020). Metode penelitian kuantitatif, kualitatif, dan R&D. Alfabeta.

Wiboolyasarin, W., Wiboolyasarin, K., Tiranant, P., Boonyakitanont, P., & Jinowat, N. (2024). Designing chatbots in language classrooms: An empirical investigation from user learning experience. Smart Learning Environments, 11(1). https://doi.org/10.1186/s40561-024-00319-4