VietTESOL International Convention 2025

Assessing EFL writing is one of the most challenging tasks in language assessment, particularly for novice raters. While performance testing has gained popularity, concerns remain about whether teachers can reliably apply rating scales. Rater severity and leniency are common issues, but most studies focus on in-service teachers. Few have explored pre-service teachers, especially in Vietnam, where research on this topic is scarce. This study addresses that gap by investigating rater severity among 10 pre-service teachers from a Vietnamese institution as they rated 20 student writing scripts. A quantitative approach was used, with analysis conducted via ACER ConQuest. Logit measures, fit statistics, discrimination indices, and point-biserial correlations were computed. Results showed that the rating scale functioned generally well in this context. While intra-rater reliability was high, inter-rater reliability was low, indicating inconsistent scoring among raters. The raters displayed a balanced mix of severity and leniency, and they could distinguish between score categories. However, their scores did not always align with the actual writing quality. These findings suggest the need for improved rater training. Pre-service teacher education should include practical rating activities, while institutions should prioritize training on scoring practices, rating scale use, and using software for analysis to identify rater errors and enhance rating consistency and accuracy.