Evaluating prompting and fine-tuning strategies for GPT-based grammatical error correction

Abstract

Evaluating Prompting and Fine-Tuning Strategies for GPT-Based Grammatical Error Correction. This study investigates how to align Large Language Models (LLMs) with the pedagogical goal of "minimal-edit" grammatical error correction (GEC) for L2 learners, contrasting with their default "fluency-first" behavior. We used a 2x2x2 Linear Mixed Models (LMM) analysis to compare prompting strategies against a task-specific fine-tuned GPT-4o model on 1,000 Korean EFL learner essays. We operationalized the "minimal-edit" goal using a metric profile: F0.5 (precision), GLEU (similarity to the original text), and Perplexity (fluency). The LMM revealed that fine-tuning was decisively superior. The fine-tuned model (FT-Complex) achieved desirable Perplexity and GLEU scores and the lowest total edit count. In contrast, the base GPT-4o, even with a complex pedagogical prompt, produced disfluent, over-corrected text, driven by a significant Model x Prompt interaction (high PPL). A second one-way LMM on fine-tuning sample sizes (N=0 to 2,617) revealed a clear trade-off: a small dataset (N=500) achieved the highest minimal-edit adherence (GLEU), while larger datasets (N=2,617) maximized precision (F0.5) and fluency. This study demonstrates that fine-tuning is an accessible and robust methodology for educators to create AI tools that align with specific pedagogical goals.