The levers of political persuasion with conversational artificial intelligence

Editor’s summary

Many fear that we are on the precipice of unprecedented manipulation by large language models (LLMs), but techniques driving their persuasiveness are poorly understood. In the initial “pretrained” phase, LLMs may exhibit flawed reasoning. Their power unlocks during vital “posttraining,” when developers refine pretrained LLMs to sharpen their reasoning and align with users’ needs. Posttraining also enables LLMs to maintain logical, sophisticated conversations. Hackenburg et al. examined which techniques made diverse, conversational LLMs most persuasive across 707 British political issues (see the Perspective by Argyle). LLMs were most persuasive after posttraining, especially when prompted to use facts and evidence (information) to argue. However, information-dense LLMs produced the most inaccurate claims, raising concerns about the spread of misinformation during rollouts of future models. —Ekeoma Uzogara

Structured Abstract

INTRODUCTION

Rapid advances in artificial intelligence (AI) have sparked widespread concerns about its potential to influence human beliefs. One possibility is that conversational AI could be used to manipulate public opinion on political issues through interactive dialogue. Despite extensive speculation, however, fundamental questions about the actual mechanisms, or “levers,” responsible for driving advances in AI persuasiveness—e.g., computational power or sophisticated training techniques—remain largely unanswered. In this work, we systematically investigate these levers and chart the horizon of persuasiveness with conversational AI.

RATIONALE

We considered multiple factors that could enhance the persuasiveness of conversational AI: raw computational power (model scale), specialized post-training methods for persuasion, personalization to individual users, and instructed rhetorical strategies. Across three large-scale experiments with N = 76,977 responses (from 42,357 people), we deployed 19 large language models (LLMs) to persuade on 707 political issues while varying these factors independently. We also analyzed more than 466,000 AI-generated claims, examining the relationship between persuasiveness and truthfulness.

RESULTS

We found that the most powerful levers of AI persuasion were methods for post-training and rhetorical strategy (prompting), which increased persuasiveness by as much as 51 and 27%, respectively. These gains were often larger than those obtained from substantially increasing model scale. Personalizing arguments on the basis of user data had a comparatively small effect on persuasion. We observe that a primary mechanism driving AI persuasiveness was information density: Models were most persuasive when they packed their arguments with a high volume of factual claims. Notably, however, we documented a concerning trade-off between persuasion and accuracy: The same levers that made AI more persuasive—including persuasion post-training and information-focused prompting—also systematically caused the AI to produce information that was less factually accurate.

CONCLUSION

Our findings suggest that the persuasive power of current and near-future AI is likely to stem less from model scale or personalization and more from post-training and prompting techniques that mobilize an LLM’s ability to rapidly generate information during conversation. Further, we reveal a troubling trade-off: When AI systems are optimized for persuasion, they may increasingly deploy misleading or false information. This research provides an empirical foundation for policy-makers and technologists to anticipate and address the challenges of AI-driven persuasion, and it highlights the need for safeguards that balance AI’s legitimate uses in political discourse with protections against manipulation and misinformation.

Persuasiveness of conversational AI increases with model scale.
The persuasive impact in percentage points on the y axis is plotted against effective pretraining compute [floating-point operations (FLOPs)] on the x axis. Point estimates are persuasive effects of different AI models. Colored lines show trends for models that we uniformly chat-tuned for open-ended conversation (purple) versus those that were post-trained using heterogeneous, opaque methods by AI developers (green). pp, percentage points; CI, confidence interval.

Abstract

There are widespread fears that conversational artificial intelligence (AI) could soon exert unprecedented influence over human beliefs. In this work, in three large-scale experiments (N = 76,977 responses from 42,357 people), we deployed 19 large language models (LLMs)—including some post-trained explicitly for persuasion—to evaluate their persuasiveness on 707 political issues. We then checked the factual accuracy of 466,769 resulting LLM claims. We show that the persuasive power of current and near-future AI is likely to stem more from post-training and prompting methods—which boosted persuasiveness by as much as 51 and 27%, respectively—than from personalization or increasing model scale, which had smaller effects. We further show that these methods increased persuasion by exploiting LLMs’ ability to rapidly access and strategically deploy information and that, notably, where they increased AI persuasiveness, they also systematically decreased factual accuracy.

Access the full article

View all access options to continue reading this article.

Supplementary Materials

The PDF file includes:

Materials and Methods

Supplementary Text

Figs. S1 to S10

Tables S1 to S170

References (73–87)

Download
3.95 MB

Other Supplementary Material for this manuscript includes the following:

MDAR Reproducibility Checklist

Download
156.65 KB

Correction (4 February 2026): The authors have corrected the main text to clarify their descriptions of the sample size for this study. The distinction between the total number of responses versus the number of participants was originally reported only in the supplementary materials; this distinction is now clearly stated in the main text as well. Because some individuals participated in multiple experiments, references to “participant” numbers in four places (the Rationale section of the Research Article Summary, the Abstract in the main text, the third paragraph of the main text, and the first paragraph of the Discussion section of the main text) have been updated to distinguish between the total number of responses (76,977) and the number of individuals (42,357), rather than referring to 76,977 “participants.” The scientific conclusions of the article are unaffected by these changes.

References and Notes

F. Luciano, Hypersuasion – On AI’s Persuasive Power and How to Deal with It. Philos. Technol. 37, 64 (2024).

M. Burtell, T. Woodside, Artificial Influence: An Analysis Of AI-Driven Persuasion. arXiv:2303.08721 [cs.CY] (2023).

C. R. Jones, B. K. Bergen, Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models. arXiv:2412.17128 [cs.CL] (2024).

A. Rogiers, S. Noels, M. Buyl, T. De Bie, Persuasion with Large Language Models: A Survey. arXiv:2411.06837 [cs.CL] (2024).

S. El-Sayed, C. Akbulut, A. McCroskery, G. Keeling, Z. Kenton, Z. Jalan, N. Marchal, A. Manzini, T. Shevlane, S. Vallor, D. Susser, M. Franklin, S. Bridgers, H. Law, M. Rahtz, M. Shanahan, M. H. Tessler, A. Douillard, T. Everitt, S. Brown, A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI. arXiv:2404.15058 [cs.CY] (2024).

K. Grace, H. Stewart, J. F. Sandkühler, S. Thomas, B. Weinstein-Raun, J. Brauner, R. C. Korzekwa, Thousands of AI Authors on the Future of AI. arXiv:2401.02843 [cs.CY] (2025).

Y. Bengio, G. Hinton, A. Yao, D. Song, P. Abbeel, T. Darrell, Y. N. Harari, Y.-Q. Zhang, L. Xue, S. Shalev-Shwartz, G. Hadfield, J. Clune, T. Maharaj, F. Hutter, A. G. Baydin, S. McIlraith, Q. Gao, A. Acharya, D. Krueger, A. Dragan, P. Torr, S. Russell, D. Kahneman, J. Brauner, S. Mindermann, Managing extreme AI risks amid rapid progress. Science 384, 842–845 (2024).

J. A. Goldstein, G. Sastry, M. Musser, R. DiResta, M. Gentzel, K. Sedova, Generative Language Models and Automated Influence Operations: Emerging Threats and Potential Mitigations. arXiv:2301.04246 [cs.CY] (2023).

D. Broockman, J. Kalla, Durably reducing transphobia: A field experiment on door-to-door canvassing. Science 352, 220–224 (2016).

J. L. Kalla, A. S. Levine, D. E. Broockman, Personalizing Moral Reframing in Interpersonal Conversation: A Field Experiment. J. Polit. 84, 1239–1243 (2022).

J. L. Kalla, D. E. Broockman, Reducing Exclusionary Attitudes through Interpersonal Conversation: Evidence from Three Field Experiments. Am. Polit. Sci. Rev. 114, 410–425 (2020).

L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, R. Lowe, “ Training language models to follow instructions with human feedback” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Eds. (Curran Associates, Inc., 2022), pp. 27730–27744.

A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, V. Misra, “ Solving Quantitative Reasoning Problems with Language Models” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Eds. (Curran Associates, Inc., 2022), pp. 3843–3857.

J. Wei, X. Wang, D. Schuurmans, M. Bosma, B. Ichter, F. Xia, E. Chi, Q. V. Le, D. Zhou, “ Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” in Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, A. Oh, Eds. (Curran Associates, Inc., 2022), pp. 24824–24837.

J. A. Goldstein, J. Chao, S. Grossman, A. Stamos, M. Tomz, How persuasive is AI-generated propaganda? PNAS Nexus 3, pgae034 (2024).

M. Wack, C. Ehrett, D. Linvill, P. Warren, Generative propaganda: Evidence of AI’s impact from a state-backed disinformation campaign. PNAS Nexus 4, pgaf083 (2025).

F. Salvi, M. H. Ribeiro, R. Gallotti, R. West, On the conversational persuasiveness of GPT-4. Nat. Hum. Behav. 9, 1645–1653 (2025).

S. C. Matz, J. D. Teeny, S. S. Vaid, H. Peters, G. M. Harari, M. Cerf, The potential of generative AI for personalized persuasion at scale. Sci. Rep. 14, 4692 (2024).

A. Simchon, M. Edwards, S. Lewandowsky, The persuasive effects of political microtargeting in the age of generative artificial intelligence. PNAS Nexus 3, pgae035 (2024).

K. Hackenburg, H. Margetts, Evaluating the persuasive influence of political microtargeting with large language models. Proc. Natl. Acad. Sci. U.S.A. 121, e2403116121 (2024).

K. Hackenburg, B. M. Tappin, P. Röttger, S. A. Hale, J. Bright, H. Margetts, Scaling language model size yields diminishing returns for single-message political persuasion. Proc. Natl. Acad. Sci. U.S.A. 122, e2413443122 (2025).

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, D. Amodei, Scaling Laws for Neural Language Models. arXiv:2001.08361 [cs.LG] (2020).

N. Ding, Y. Chen, B. Xu, Y. Qin, Z. Zheng, S. Hu, Z. Liu, M. Sun, B. Zhou, Enhancing Chat Language Models by Scaling High-quality Instructional Conversations. arXiv:2305.14233 [cs.CL] (2023).

J. G. Voelkel, M. Feinberg, Morally Reframed Arguments Can Affect Support for Political Candidates. Soc. Psychol. Personal. Sci. 9, 917–924 (2017).

M. Feinberg, R. Willer, Moral reframing: A technique for effective and persuasive communication across political divides. Soc. Personal. Psychol. Compass 13, e12501 (2019).

M. C. Green, T. C. Brock, The role of transportation in the persuasiveness of public narratives. J. Pers. Soc. Psychol. 79, 701–721 (2000).

A. Hamby, D. Brinberg, K. Daniloski, Reflecting on the journey: Mechanisms in narrative persuasion. J. Consum. Psychol. 27, 11–22 (2017).

E. Santoro, D. E. Broockman, J. L. Kalla, R. Porat, Listen for a change? A longitudinal field experiment on listening’s potential to enhance persuasion. Proc. Natl. Acad. Sci. U.S.A. 122, e2421982122 (2025).

R. E. Petty, J. T. Cacioppo, “The Elaboration Likelihood Model of Persuasion” in Communication and Persuasion: Central and Peripheral Routes to Attitude Change, R. E. Petty, J. T. Cacioppo, Eds. (Springer, 1986), pp. 1–24.

A. Coppock, Persuasion in Parallel: How Information Changes Minds about Politics (Univ. Chicago Press, 2022).

T. H. Costello, G. Pennycook, D. G. Rand, Durably reducing conspiracy beliefs through dialogues with AI. Science 385, eadq1814 (2024).

P. Schoenegger, F. Salvi, J. Liu, X. Nan, R. Debnath, B. Fasolo, E. Leivada, G. Recchia, F. Günther, A. Zarifhonarvar, J. Kwon, Z. U. Islam, M. Dehnert, D. Y. H. Lee, M. G. Reinecke, D. G. Kamper, M. Kobaş, A. Sandford, J. Kgomo, L. Hewitt, S. Kapoor, K. Oktar, E. E. Kucuk, B. Feng, C. R. Jones, I. Gainsburg, S. Olschewski, N. Heinzelmann, F. Cruz, B. M. Tappin, T. Ma, P. S. Park, R. Onyonka, A. Hjorth, P. Slattery, Q. Zeng, L. Finke, I. Grossmann, A. Salatiello, E. Karger, Large Language Models Are More Persuasive Than Incentivized Human Persuaders. arXiv:2505.09662 [cs.CL] (2025).

D. M. Kahan, Ideology, motivated reasoning, and cognitive reflection. Judgm. Decis. Mak. 8, 407–424 (2013).

D. M. Kahan, “The Politically Motivated Reasoning Paradigm, Part 1: What Politically Motivated Reasoning Is and How to Measure It” in Emerging Trends in the Social and Behavioral Sciences, R. A. Scott, S. M. Kosslyn, Eds. (Wiley, 2016), pp. 1–16.

C. S. Taber, M. Lodge, Motivated Skepticism in the Evaluation of Political Beliefs. Am. J. Pol. Sci. 50, 755–769 (2006).

J. J. Van Bavel, A. Pereira, The Partisan Brain: An Identity-Based Model of Political Belief. Trends Cogn. Sci. 22, 213–224 (2018).

D. E. Broockman, J. L. Kalla, When and Why Are Campaigns’ Persuasive Effects Small? Evidence from the 2020 U.S. Presidential Election. Am. J. Pol. Sci. 67, 833–849 (2023).

B. M. Tappin, A. J. Berinsky, D. G. Rand, Partisans’ receptivity to persuasive messaging is undiminished by countervailing party leader cues. Nat. Hum. Behav. 7, 568–582 (2023).

J. R. Zaller, The Nature and Origins of Mass Opinion (Cambridge Univ. Press, 1992).

A. Coppock, S. J. Hill, L. Vavreck, The small effects of political advertising are small regardless of context, message, sender, or receiver: Evidence from 59 real-time randomized experiments. Sci. Adv. 6, eabc4046 (2020).

M. Hu, Cambridge Analytica’s black box. Big Data Soc. 7, 2053951720938091 (2020).

E. D. Hersh, B. F. Schaffner, Targeted Campaign Appeals and the Value of Ambiguity. J. Polit. 75, 520–534 (2013).

B. M. Tappin, C. Wittenberg, L. B. Hewitt, A. J. Berinsky, D. G. Rand, Quantifying the potential persuasive returns to political microtargeting. Proc. Natl. Acad. Sci. U.S.A. 120, e2216261120 (2023).

F. M. Simon, S. Altay, H. Mercier, Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harv. Kennedy Sch. Misinformation Rev. 4, 1–11 (2023).

D. C. Funder, D. J. Ozer, Evaluating Effect Size in Psychological Research: Sense and Nonsense. Adv. Methods Pract. Psychol. Sci. 2, 156–168 (2019).

L. B. Hewitt, B. M. Tappin, Rank-heterogeneous effects of political messages: Evidence from randomized survey experiments testing 59 video treatments. PsyArXiv (2022); https://doi.org/10.31234/osf.io/xk6t3.

L. P. Argyle, E. C. Busby, J. R. Gubler, A. Lyman, J. Olcott, J. Pond, D. Wingate, Testing theories of political persuasion using AI. Proc. Natl. Acad. Sci. U.S.A. 122, e2412815122 (2025).

S. Altay, M. Schwartz, A.-S. Hacquin, A. Allard, S. Blancke, H. Mercier, Scaling up interactive argumentation by providing counterarguments with a chatbot. Nat. Hum. Behav. 6, 579–592 (2022).

H. Mercier, Not Born Yesterday: The Science of Who We Trust and What We Believe (Princeton Univ. Press, 2020).

H. Mercier, D. Sperber, The Enigma of Reason (Harvard Univ. Press, 2018).

K. F. Pilz, J. Sanders, R. Rahman, L. Heim, Trends in AI Supercomputers. arXiv:2504.16026 [cs.CY] (2025).

H. Mercier, How Gullible are We? A Review of the Evidence from Psychology and Social Science. Rev. Gen. Psychol. 21, 103–122 (2017).

D. Sperber, F. Clément, C. Heintz, O. Mascaro, H. Mercier, G. Origgi, D. Wilson, Epistemic Vigilance. Mind Lang. 25, 359–393 (2010).

M. Prior, Post-Broadcast Democracy: How Media Choice Increases Inequality in Political Involvement and Polarizes Elections (Cambridge Univ. Press, 2007).

M. X. Delli Carpini, S. Keeter, What Americans Know about Politics and Why It Matters (Yale Univ. Press, 1996).

Z. Chen, J. Kalla, Q. Le, S. Nakamura-Sakai, J. Sekhon, R. Wang, A Framework to Assess the Persuasion Risks Large Language Model Chatbots Pose to Democratic Societies. arXiv:2505.00036 [cs.CL] (2025).

A. Coppock, Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach. Political Sci. Res. Methods 7, 613–628 (2019).

K. J. Mullinix, T. J. Leeper, J. N. Druckman, J. Freese, The Generalizability of Survey Experiments. J. Exp. Political Sci. 2, 109–138 (2015).

T. Yarkoni, The Generalizability Crisis. Behav. Brain Sci. 45, e1 (2022).

M. Rubin, J. Z. Li, F. Zimmerman, D. C. Ong, A. Goldenberg, A. Perry, Comparing the value of perceived human versus AI-generated empathy. Nat. Hum. Behav. 10.1038/s41562-025-02247-w (2025).

B. M. Tappin, Data and Analysis Code, OSF (2025).

M. N. Stagnaro, J. N. Druckman, A. J. Berinsky, A. A. Arechar, R. Willer, D. G. Rand, Representativeness and Response Validity Across Nine Opt-In Online Samples. PsyArXiv (2024); .

E. Peer, D. Rothschild, A. Gordon, Z. Evernden, E. Damer, Data quality of platforms and panels for online behavioral research. Behav. Res. Methods 54, 1643–1662 (2022).

P. Röttger, V. Hofmann, V. Pyatkin, M. Hinck, H. R. Kirk, H. Schütze, D. Hovy, Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models. arXiv:2402.16786 [cs.CL] (2024).

J. Wang, X. Hu, W. Hou, H. Chen, R. Zheng, Y. Wang, L. Yang, H. Huang, W. Ye, X. Geng, B. Jiao, Y. Zhang, X. Xie, On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective. arXiv:2302.12095 [cs.AI] (2023).

M. Sclar, Y. Choi, Y. Tsvetkov, A. Suhr, Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. arXiv:2310.11324 [cs.CL] (2024).

Y. Elazar, N. Kassner, S. Ravfogel, A. Ravichander, E. Hovy, H. Schütze, Y. Goldberg, Measuring and Improving Consistency in Pretrained Language Models. Trans. Assoc. Comput. Linguist. 9, 1012–1031 (2021).

R. B. Cialdini, C. A. Kallgren, R. R. Reno, “A Focus Theory of Normative Conduct: A Theoretical Refinement and Reevaluation of the Role of Norms in Human Behavior” in Advances in Experimental Social Psychology, vol. 24, M. P. Zanna, Ed. (Academic Press, 1991), pp. 201–234.

J. Blumenau, B. E. Lauderdale, The Variable Persuasiveness of Political Rhetoric. Am. J. Pol. Sci. 68, 255–270 (2024).

J. G. MacKinnon, H. White, Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties. J. Econom. 29, 305–325 (1985).

A. S. Gerber, D. P. Green, Field Experiments: Design, Analysis, and Interpretation (W. W. Norton, 2012).

L. Hewitt, D. Broockman, A. Coppock, B. M. Tappin, J. Slezak, V. Coffman, N. Lubin, M. Hamidian, How Experiments Help Campaigns Persuade Voters: Evidence from a Large Archive of Campaigns’ Own Experiments. Am. Polit. Sci. Rev. 118, 2021–2039 (2024).

P.-C. Bürkner, brms: An R Package for Bayesian Multilevel Models Using Stan. J. Stat. Softw. 80, 1–28 (2017).

A. Vehtari, A. Gelman, J. Gabry, Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC. Stat. Comput. 27, 1413–1432 (2017).