Volume 1, Issue 2  November 2018, pp. 45-58          Download PDF

Articles
The impact of spelling errors on trained raters' scoring decisions

Ikkyu Choi https://orcid.org/0000-0002-8281-72111, Yeonsuk Cho https://orcid.org/0000-0002-4997-61062

1 Educational Testing Service, USA ichoi001@ets.org
2 Educational Testing Service, USA ycho@ets.org

DOI: https://doi.org/10.29140/lea.v1n2.61


Abstract

Second language (L2) writing assessments seldom allow a spellchecker and often have a time limit. Naturally, test takers often submit responses with spelling errors. However, little is known about whether and how spelling errors in test taker responses affect trained raters' scoring decisions. In this study, we investigated the impact of spelling errors on trained raters' holistic evaluation of response quality. We selected 148 responses to four L2 writing tasks, and created error-free versions of the responses by correcting spelling errors in the original responses. Both the original and corrected responses were randomly assigned to trained raters, who scored the responses according to the same holistic scoring rubrics. We compared the resulting scores of original and corrected responses to gauge the impact of spelling errors. We also examined whether the impact of spelling errors varied across different task types and spelling error characteristics. The results showed that the spelling error correction led to an average of more than a half score point increase. The score gains from the error correction varied across task types and the quantity of corrected errors. These findings have multiple implications, including suggestions for rater training and assessment development.



Copyright

© Ikkyu Choi, Yeonsuk Cho

CC BY-NC 4.0
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.


Suggested citation

Choi, I., & Cho, Y. (2018). The impact of spelling errors on trained raters' scoring decisions. Language Education & Assessment, 1(2), 45-58. https://doi.org/10.29140/lea.v1n2.61


Related Articles:

Interpersonal meaning of code-switching: An analysis of three TV series
Huabin Wang
Australian Journal of Applied Linguistics Published: 1 April, 2018, Volume 1(1), pp. 3-19. Download PDF

Revisiting the digital divide(s): Technology-enhanced English language practices at a university in Pakistan
Shaista Rashid, Una Cunningham, Kevin Watson, Jocelyn Howard
Australian Journal of Applied Linguistics Published: 31 August, 2018, Volume 1(2), pp. 64-87. Download PDF

Examining learners' further investment in Japanese learning through study abroad experiences
Fatmawati Djafri
Australian Journal of Applied Linguistics Published: 1 April, 2018, Volume 1(1), pp. 33-48. Download PDF

A day in the life: Mapping international students' language learning environments in multilingual Sydney
Phil Benson, Philip Chappell, Lynda Yates
Australian Journal of Applied Linguistics Published: 1 April, 2018, Volume 1(1), pp. 20-32. Download PDF

Designing a course in intercultural education
Adrian Holliday
Intercultural Communication Education Published: 11 August, 2018, Volume 1(1), pp. 4-11. Download PDF

Interculturality in a different light: Modesty towards democracy in education?
Yongjian Li, Fred Dervin
Intercultural Communication Education Published: 11 April, 2018, Volume 1(1), pp. 12-26. Download PDF

Revisiting communicative competence in the teaching and assessment of language for specific purposes
Shona Whyte
Language Education & Assessment Published: 30 April, 2019, Volume 2(1), pp. 1-19. Download PDF

Language learning and activation in and beyond the classroom
Julie Choi, David Nunan
Australian Journal of Applied Linguistics Published: 31 August, 2018, Volume 1(2), pp. 49-63. Download PDF

Then and now: Themes in language assessment research
Lia Plakans
Language Education & Assessment Published: 25 July, 2018, Volume 1(1), pp. 3-8. Download PDF

Beginning again
Glenn Stockwell
Australian Journal of Applied Linguistics Published: 1 April, 2018, Volume 1(1), pp. 1-2. Download PDF