An evaluation of automated writing assessment

Abstract

This paper describes research done to empirically study the efficacy of using Criterion, a commercial web-based writing evaluation platform, as a revision aid, especially within a class following the Process Writing pedagogy. This paper first describes the background and rationale for Criterion automated writing evaluation, the Process Writing approach, as well as the natural language processing techniques used by Criterion. In this study an essay was submitted to Criterion multiple times with variations in a single feature each time. By logging the results and comparing how Criterion scored each submission it is possible to determine what components of an essay influence Criterion’s scoring. The conclusions from these results suggest that Criterion alone is not an adequate revision aid for the developing writer.