EditLens: A New Model for Detecting and Quantifying AI Editing in Human-Written Text
By
horseradish
A weekday bagel. Dependable, satisfying, no fuss.
Summary
This paper introduces EditLens, a regression model that detects and quantifies the extent of AI editing in text, distinguishing between human-written, AI-generated, and AI-edited (mixed) text. The authors first use lightweight similarity metrics to measure AI editing magnitude, validated by human annotators, then train EditLens using these metrics as intermediate supervision. The model achieves state-of-the-art performance on binary (F1=94.7%) and ternary (F1=90.4%) classification tasks. A case study analyzes AI edits applied by Grammarly. The research has implications for authorship attribution, education, and policy, and the authors commit to publicly releasing their models and dataset.
Key quotes
· 5 pulledA significant proportion of queries to large language models ask them to edit user-provided text, rather than generate new text from scratch.
While previous work focuses on detecting fully AI-generated text, we demonstrate that AI-edited text is distinguishable from human-written and AI-generated text.
Our model achieves state-of-the-art performance on both binary (F1=94.7%) and ternary (F1=90.4%) classification tasks in distinguishing human, AI, and mixed writing.
Not only do we show that AI-edited text can be detected, but also that the degree of change made by AI to human writing can be detected, which has implications for authorship attribution, education, and policy.
To encourage further research, we commit to publicly releasing our models and dataset.
