Generative AI-Assisted Feedback and Academic Writing Performance Among Undergraduate Students: A Systematic Review of Controlled Comparative Evidence

Abstract

Background: Generative artificial intelligence can provide immediate, scalable, and highly personalized feedback on student writing, but the educational value of such feedback depends on whether students use it to revise and learn rather than simply outsource composition. Evidence emerging in 2023 produced both positive and neutral controlled findings.
Objective: To synthesize high-quality evidence available through 31 December 2023 concerning the effect of generative AI-assisted feedback on academic writing performance, with particular attention to undergraduate and university language learners and to comparison with human or conventional instruction.
Methods: A structured systematic evidence synthesis was conducted using Q1 literature in educational technology, psychology, writing assessment, and computer-assisted language learning. Direct 2023 controlled studies of ChatGPT or GPT-4 were prioritized. Earlier automated writing evaluation research was incorporated as contextual evidence concerning feedback mechanisms. Because studies differed in intervention design, learner population, outcome instruments, and statistical reporting, no new pooled effect was calculated.
Results: Direct evidence was mixed. In a randomized three-month study of 50 Chinese bachelor's EFL students, ChatGPT-assisted instruction produced higher post-test writing performance than traditional instruction, with a large effect for overall writing (d=0.76) and large effects for organization (d=0.84) and language use (d=0.88). In contrast, a six-week quasi-experiment involving 48 university English-as-a-new-language students found similar writing gains with GPT-4 and human-tutor feedback; the group-by-time interaction was not statistically significant (F=3.094, p=0.085). A small controlled study of free ChatGPT-3.5 writing assistance in master's students likewise found no significant group effect on essay score (p=0.184). A 2023 meta-analysis of pre-generative-AI automated writing evaluation studies reported a large overall effect on writing quality (Hedges g=0.861), establishing that automated feedback can be educationally effective but not proving an equivalent GenAI effect.
Conclusion: Generative AI can improve academic writing when embedded in structured instruction, but direct evidence through 2023 does not support an assumption that AI feedback is inherently superior to human feedback. Benefits appear contingent on prompt and rubric design, student proficiency, critical engagement with feedback, iterative revision, and preservation of learner agency. The strongest pedagogical model is GenAI as a feedback scaffold rather than a substitute author.