Journal of Academic Research for Humanities (JARH) is a double-blind, peer-reviewed, Open Free Access, online Multidisciplinary Research Journal
Skip to main navigation menu Skip to main content Skip to site footer

Vol. 6 No. 3 (2026)
International Journal of Academic Research for Humanities

Research Articles

Evaluating Cultural Balance and Representation in ChatGPT and DeepSeek-Generated EFL Materials

https://doi.org/10.67244/jar.bwo-researches.v6i3.a623

Published 06-08-2026

  • Idrees
    • ,
  • Liu Yongzhi

Idrees
Master’s Student, Applied Linguistics, College of Foreign Languages and Cultures, Chengdu University of Technology, Chengdu, Sichuan, China

Liu Yongzhi
Dean of College of Foreign Languages and Cultures, Chengdu University of Technology, Chengdu, Sichuan, China

Abstract

Generative artificial intelligence is increasingly used to create English as a Foreign Language (EFL) reading materials, yet fluent output may still simplify cultural groups. This study compared 24 classroom-oriented passages generated by ChatGPT GPT-5.6 and DeepSeek-V4 from 12 identical prompts submitted on 22 July 2026. Each prompt requested a 220–250-word intermediate-level passage representing Chinese, Pakistani, and mainstream English-speaking Western perspectives fairly. Directed qualitative content analysis examined inclusion, balance, intragroup variation, specificity, essentialization, the collectivist-individualist binary, evaluative hierarchy, stereotyping risk, intercultural sensitivity, and prompt compliance. ChatGPT produced 2,763 words and met the requested range in all 12 passages. DeepSeek produced 3,095 words and met the requested range in five passages. Both systems included the three perspectives and promoted respectful communication. ChatGPT used more qualifications and acknowledged internal variation more consistently. DeepSeek supplied more named cultural detail but relied more often on broad contrasts that framed Chinese and Pakistani contexts as collective and Western contexts as individualistic. The findings show that cultural inclusion does not ensure balanced representation. The proposed review criteria help teachers and curriculum developers evaluate variation, specificity, stereotyping risk, and classroom suitability before using AI-generated materials.

Keywords

Culture, representation, ChatGPT, DeepSeek, EFL

PDF Video Overview of the Paper

References

  1. Adilazuarda, M. F., Mukherjee, S., Lavania, P., Singh, S., Aji, A. F., O'Neill, J., Modi, A., & Choudhury, M. (2024). Towards measuring and modeling "culture" in LLMs: A survey. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (pp. 15763–15784). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.emnlp-main.882
  2. AlKhamissi, B., ElNokrashy, M., Alkhamissi, M., & Diab, M. (2024). Investigating cultural alignment of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 12404–12422). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.671
  3. Balouch, A., Barich, S. N., & Khaskheli, I. H. (2025). Intercultural communication competence: A comprehensive review of Oxford English for Undergraduates. International Journal of Academic Research for Humanities, 5(1), 33–42. https://jar.bwo-researches.com/index.php/jarh/article/view/545
  4. Bearman, M., Ryan, J., & Ajjawi, R. (2023). Discourses of artificial intelligence in higher education: A critical literature review. Higher Education, 86(2), 369–385. https://doi.org/10.1007/s10734-022-00937-2
  5. Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). Association for Computing Machinery. https://doi.org/10.1145/3442188.3445922
  6. Byram, M. (1997). Teaching and assessing intercultural communicative competence. Multilingual Matters.
  7. Cortazzi, M., & Jin, L. (1999). Cultural mirrors: Materials and methods in the EFL classroom. In E. Hinkel (Ed.), Culture in second language teaching and learning (pp. 196–219). Cambridge University Press.
  8. Godwin-Jones, R. (2024). Distributed agency in second language learning and teaching through generative AI. Language Learning & Technology, 28(2), 5–30. https://doi.org/10.64152/10125/73570
  9. Hsieh, H.-F., & Shannon, S. E. (2005). Three approaches to qualitative content analysis. Qualitative Health Research, 15(9), 1277–1288. https://doi.org/10.1177/1049732305276687
  10. Jeon, J., Lee, S., & Coronel-Molina, S. M. (2024). Rethinking AI: Bias in speech-recognition chatbots for ELT. ELT Journal, 78(4), 435–445. https://doi.org/10.1093/elt/ccae035
  11. Khosravi, H., Buckingham Shum, S., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable artificial intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. https://doi.org/10.1016/j.caeai.2022.100074
  12. Kohnke, L., Moorhouse, B. L., & Zou, D. (2023). ChatGPT for language teaching and learning. RELC Journal, 54(2), 537–550. https://doi.org/10.1177/00336882231162868
  13. Lashari, N. F., Guo, F., & Maganhar, A. A. (2026). Artificial intelligence and comprehensible input in second language acquisition: Evaluating AI language learning tools through Krashen's Input Hypothesis. International Journal of Academic Research for Humanities, 6(3), 13–25. https://doi.org/10.67244/jar.bwo-researches.v6i3.a609
  14. Lee, S., Jeon, J., McKinley, J., & Rose, H. (2025). Generative AI and English language teaching: A Global Englishes perspective. Annual Review of Applied Linguistics, 45, 85–108. https://doi.org/10.1017/S0267190525100184
  15. Lee, S., & Song, K.-S. (2024). Teachers' and students' perceptions of AI-generated concept explanations: Implications for integrating generative AI in computer science education. Computers and Education: Artificial Intelligence, 7, 100283. https://doi.org/10.1016/j.caeai.2024.100283
  16. Li, C., Chen, M., Wang, J., Sitaram, S., & Xie, X. (2024). CultureLLM: Incorporating cultural differences into large language models. Advances in Neural Information Processing Systems, 37, 84799–84838. https://doi.org/10.52202/079017-2693
  17. Lo, A. W. T. (2025). The educational affordances and challenges of generative AI in Global Englishes-oriented materials development and implementation: A critical ecological perspective. System, 130, 103610. https://doi.org/10.1016/j.system.2025.103610
  18. Ma, Q., Crosthwaite, P., Sun, D., & Zou, D. (2024). Exploring ChatGPT literacy in language education: A global perspective and comprehensive approach. Computers and Education: Artificial Intelligence, 7, 100278. https://doi.org/10.1016/j.caeai.2024.100278
  19. Minhas, H., & Salim, M. (2025). Constructing nature through artificial intelligence: An eco-semantic discourse study of Gemini and DeepSeek. International Journal of Academic Research for Humanities, 5(4), 100–113. https://jar.bwo-researches.com/index.php/jarh/article/view/586
  20. Moorhouse, B. L. (2024). Generative artificial intelligence and ELT. ELT Journal, 78(4), 378–392. https://doi.org/10.1093/elt/ccae032
  21. Moorhouse, B. L., & Wong, K. M. (2025). Generative
  22. artificial intelligence and language teaching. Cambridge University Press.
  23. https://doi.org/10.1017/9781009618823
  24. Myung, J., Lee, N., Zhou, Y., Jin, J., Putri, R. A., Antypas, D., Borkakoty, H., Kim, E., Perez-Almendros, C., Ayele, A. A., Gutierrez-Basulto, V., Ibanez-Garcia, Y., Lee, H., Muhammad, S. H., Park, K., Rzayev, A. S., White, N., Yimam, S. M., Pilehvar, M. T., ... Oh, A. (2024). BLEnD: A benchmark for LLMs on everyday knowledge in diverse cultures and languages. Advances in Neural Information Processing Systems, 37, 78104–78146. https://doi.org/10.52202/079017-2483
  25. Naous, T., Ryan, M. J., Ritter, A., & Xu, W. (2024). Having beer after prayer? Measuring cultural bias in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 16366–16393). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.862
  26. Naous, T., & Xu, W. (2025). On the origin of cultural biases in language models: From pre-training data to linguistic phenomena. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 6423–6443). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-long.326
  27. OECD. (2026). OECD digital education outlook 2026: Exploring effective uses of generative AI in education. OECD Publishing. https://doi.org/10.1787/062a7394-en
  28. Pan, Z., & Wang, Y. (2025). From technology-challenged teachers to empowered digitalized citizens: Exploring the profiles and antecedents of teacher AI literacy in the Chinese EFL context. European Journal of Education, 60(1), e70020. https://doi.org/10.1111/ejed.70020
  29. Papagiannidis, E., Mikalef, P., & Conboy, K. (2025). Responsible artificial intelligence governance: A review and research framework. The Journal of Strategic Information Systems, 34(2), 101885. https://doi.org/10.1016/j.jsis.2024.101885
  30. Rao, A. S., Yerukola, A., Shah, V., Reinecke, K., & Sap, M. (2025). NormAd: A framework for measuring the cultural adaptability of large language models. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (pp. 2373–2403). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.naacl-long.120
  31. Rose, H., McKinley, J., & Galloway, N. (2021). Global Englishes and language teaching: A review of pedagogical research. Language Teaching, 54(2), 157–189. https://doi.org/10.1017/S0261444820000518
  32. Shah, M. H. A., Ali, Z., & Shah, A. (2024). Challenging
  33. factors towards the effective use of ChatGPT in education in Province Sindh, Pakistan: Application of TAM model.
  34. International Journal of Academic Research for Humanities, 4(2), 86–94. https://jar.bwo-researches.com/index.php/jarh/article/view/450
  35. Shahid, M. M. (2025). AI matters: Debunking the myth that AI hinders the creative process. International Journal of Academic Research for Humanities, 5(2), 18–30. https://jar.bwo-researches.com/index.php/jarh/article/view/550
  36. Shi, W., Li, R., Zhang, Y., Ziems, C., Yu, S., Horesh, R., De Paula, R. A., & Yang, D. (2024). CultureBank: An online community-driven knowledge base towards culturally aware language technologies. In Findings of the Association for Computational Linguistics: EMNLP 2024 (pp. 4996–5025). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.findings-emnlp.288
  37. Tao, Y., Viberg, O., Baker, R. S., & Kizilcec, R. F. (2024). Cultural bias and cultural alignment of large language models. PNAS Nexus, 3(9), pgae346. https://doi.org/10.1093/pnasnexus/pgae346
  38. UNESCO. (2023). Guidance for generative AI in education and research. https://unesdoc.unesco.org/ark:/48223/pf0000386693
  39. UNESCO. (2024). AI competency framework for teachers. https://unesdoc.unesco.org/ark:/48223/pf0000391104
  40. Varsik, S., & Vosberg, L. (2024). The potential impact of artificial intelligence on equity and inclusion in education (OECD Artificial Intelligence Papers No. 23). OECD Publishing. https://doi.org/10.1787/15df715b-en
  41. Wang, W., Jiao, W., Huang, J., Dai, R., Huang, J.-T., Tu, Z., & Lyu, M. (2024). Not all countries celebrate Thanksgiving: On the cultural dominance in large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 6349–6384). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.345
  42. Zhang, H., Li, R., Chen, X., & Yan, F. (2024). Cultural representation in foreign language textbooks: A scoping review from 2012 to 2022. Linguistics and Education, 83, 101331. https://doi.org/10.1016/j.linged.2024.101331
  43. Zhou, N., Bamman, D., & Bleaman, I. L. (2025). Culture is not trivia: Sociocultural theory for cultural NLP. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 25869–25886). Association for Computational Linguistics. https://doi.org/10.18653/v1/2025.acl-long.1256