Speakers
Description
Recent advances in artificial intelligence have expanded the use of generative models such as ChatGPT in language assessment. However, there remains a relative lack of empirical research investigating whether these systems can produce consistent and reliable scores comparable to certified human examiners, particularly in contexts where English is increasingly positioned as a second language. This study aimed to examine the extent to which ChatGPT aligns with human examiners in scoring IELTS Writing Task 2 essays. A comparative quantitative design was employed. Ten authentic essays produced by Vietnamese IELTS candidates were analyzed. Each script was scored by certified examiners and independently evaluated by ChatGPT across three repeated runs under standardized prompting conditions. The analysis focused on four criteria: Task Response, Coherence and Cohesion, Lexical Resource, and Grammatical Range and Accuracy. Statistical procedures included descriptive statistics, Pearson correlation, intra-class correlation coefficients (ICC), and mean difference analysis. The results indicated that ChatGPT demonstrated a high level of internal consistency across repeated evaluations. Positive correlations with human scores suggested a broadly similar ranking tendency between the two raters. However, agreement remained partial, as ChatGPT did not consistently assign identical band scores. Notably, a criterion-specific bias was identified in Grammatical Range and Accuracy, where the model tended to under-score compared to human examiners. These findings suggest that ChatGPT can serve as a relatively consistent and accessible supplementary tool for formative assessment in ESL-oriented learning contexts; however, it is not yet capable of fully replicating human examiner judgment in high-stakes evaluation settings.
Keywords: ChatGPT, IELTS Writing Task 2, scoring consistency, human-AI agreement, EFL context
Biography
Vu Thi Hong Mai is an undergraduate student majoring in English Language Teaching at Hanoi Pedagogical University 2, Vietnam. Her academic interests center on language assessment, second language writing, and the application of artificial intelligence in English as a Foreign Language (EFL) and English as a Second Language (ESL) contexts. She is particularly interested in investigating the extent to which AI-driven tools such as ChatGPT, can support and potentially transform writing assessment practices.
Her current research examines the comparability between AI-generated and human evaluation in IELTS Writing Task 2, with a specific focus on scoring consistency, inter-rater reliability and criterion-specific bias. By adopting a quantitative comparative approach, her work seeks to contribute to the emerging discourse on AI-assisted language assessment and to critically evaluate its validity and pedagogical implications in high-stakes testing environments.
In addition to her research, she has gained practical teaching experience working with high school students and collaborating with international teachers in EFL and ESL instructional settings. This engagement has provided her with valuable insights into classroom-based assessment practices and learner needs, which in turn inform her research orientation. She is committed to bridging the gap between research and practice and aspires to pursue a career as an educator-researcher, with a focus on advancing innovative, evidence-based and technology-enhanced approaches to language teaching and assessment.
| Affiliate type | University |
|---|