How LLMs Distort Our Written Language

Executive Summary

LLMs are used by over a billion people globally, and the most frequent use case is to assist with writing. LLMs can provide a huge efficiency boost, but are they actually writing what we want? 

Many users recognize the "feel" of LLM prose, but few people realize the extent to which LLMs distort the meaning of writing. We find this across three datasets: a human user study, a dataset of human argumentative essays, and reviews from a top machine learning conference.

Main Findings 

• LLMs change the conclusions of writing, changing the stance as well as the argument type

• Human users report a paradox of preferences, being satisfied while reporting a statistically significant loss of voice and creativity

• LLMs introduce larger semantic shifts than human edits do, even when prompted only to introduce grammar edits

• These shifts apply even to our institutions: LLM reviews gave significantly different reasons for acceptance and rejection at the International Conference of Learning Representations (2026), a top AI conference, the 21% of peer reviews that were found to be AI-generated focused on significantly different scientific criteria.

Why should we care? 

As LLMs are integrated into society, these subtle changes in meaning could fundamentally alter politics, culture, science, and even the way we communicate with our friends and family. Our study focuses on argumentative writing, but our findings may generalize to many other forms of writing and communication as well. 

When LLMs revise human writing, they induce large homogenizing changes very unlike how people would have edited the same essay.
This figure shows a counterfactual analysis, comparing how LLMs edit a piece of text to how a person would. The upper left grey figure shows how people make edits to a first draft essay when they receive expert feedback. The first draft is shown as a light grey dot, with an arrow to the dark grey second draft, visualized using the MiniLM-L6 semantic embedding space and projected with PCA. The remaining panels show how an LLM edits the original human-written essay, when it is prompted with the expert feedback and a series of prompts. Even when the LLM is instructed to make minimal edits, we see that it induces large changes to all the essays, moving them in a consistent direction away from how humans write.

Above is an intuitive an example of how writing with an LLM alters the conclusions and removes the humans voice in essays from ArgRewrite-v2 dataset. 

We illustrate this more extensively in our results below. 

Heavy LLM users report that their essays do not reflect their own voice.

This presents a paradox of preferences where the user reports satisfaction, but report a significant loss in creativity and voice. 

RLHF optimizes for preferences, but this is not sufficient for maintaining creativity and semantics.

LLMs distort writing by shifting essays in a common semantic direction.

•  Essays written by humans in the control group are widely spread out throughout the embedding space, occupying a broad region that reflects the diversity of individual perspectives, writing styles, and argumentation. 

• Essays written by LLMs form a tight cluster in a region that is not occupied by any of the human-written essays. LLM revisions produce large semantic shifts strongly aligned in a common direction, and to a region of space not previously occupied by any human-written essay.

• This provides clear evidence that LLMs are shifting semantics in a way human editors do not. 

LLMs Alter the Conclusions of Written Language. 

LLM users wrote essays that were significantly more neutral and avoided taking a definitive stance on the question "Does money lead to happiness?".

This represents a fundamental shift in argument stance.

LLMs make substantially larger lexical changes than humans.

LLM edits  substantially alter the words used in an essay compared to human edits  

The unique lexical fingerprint of each writer is overwritten by the LLM's preferred vocabulary. 

LLMs Systematically Restructure Grammar Toward a Less Personal, Formal Style.
LLMs adopt a more formal style of writing, increasing the use of nouns and adjectives, and decreasing the use of pronouns, signifying a removal of first-person, experience-based argumentation toward impersonal language.

The Use of LLMs for Writing Increases Emotional Language.

 We see a substantial increase in emotional language across both negative and positive emotions when comparing LLM edits to human edits. 

Surprisingly, this occurs even when prompted to make minimal changes and with expert feedback. 

LLMs distort decisions affecting scientific institutions.
When LLMs are employed in the scientific review process, LLMs assign scores 10% higher than humans. Humans are 32% more likely to comment on the clarity as a strength, 58% more likely to comment on clarity as a weaknesses, and 32% more likely to comment on the relevance of research, while LLMs are 136% more likely to comment on reproducibility and 84% more likely to comment on scalability for both strengths and weaknesses. 

Differences in evaluation criteria between human and LLM reviews may impact decisions made about what scientific work is valid and incentivized.