The Slant of Large Language Models: Evidence from a Persuasion Experiment (with Tiffany Kreutschy)

Large language models (LLMs) promise personalized persuasion at near-zero cost, but their output has a slant of its own. In an experiment, U.S. respondents viewed an LLM-generated post advocating a policy stance, randomly personalized to up to six demographics. Messages persuade on average. Personalization raises persuasion only when matching on political party, almost exclusively among Democrats. Benchmarked against Congress members' language, messages slant Democratic even when prompted to target Republicans. Paradoxically, personalization with Democratic-slanted LLMs can benefit Republican positions by finding language to persuade Democrats. Our findings speak to debates about persuasion and slant in the age of generative AI.