University of Kwazulu-Natal

Pdf Proling News Media For Factuality And Bias Using Llms And The Fact

Bonisiwe Shabane
-
pdf proling news media for factuality and bias using llms and the fact

Scientific Reports , Article number: (2025) Cite this article We are providing an unedited version of this manuscript to give early access to its findings. Before final publication, the manuscript will undergo further editing. Please note there may be errors present which affect the content, and all legal disclaimers apply. Mainstream media, with its broad reach, plays a central role in shaping public opinion and thus warrants close scrutiny. Subtle forms of media bias–such as selective fact presentation and tone–can meaningfully influence public attitudes, even when reporting remains factually accurate.

Although effects such as these have been widely studied by scholars of framing, much of the existing research focuses on specific topics and relies on manually constructed or pre-existing frames, limiting both scalability and... Here we introduce a novel framework that leverages large language models (LLMs) to generate synthetic news articles by systematically varying the selection and tone of the content while holding factual accuracy and other features... We evaluate the impact of these alternative framings in a large, pre-registered randomized experiment (N = 2,141), and find that selective presentation of accurate information can significantly shift individuals’ policy views and emotional responses... These effects are consistently stronger for negative than positive framings and are more pronounced among individuals who say they are less informed about the topic. Our findings demonstrate the persuasive power of subtle bias in mainstream news as well as the value of LLMs as tools for scalable, controlled investigations of media effects. All survey data, experiment materials, and analysis code required for replicating the results can be found in the project’s OSF page (OSF link: https://osf.io/9g7sq/?view_only=189a8b9b2f644433bdbfcd4fc5c63ffe).

Liedke, J. & Wang, L. News platform fact sheet. Pew Res. Center 15, 253 (2023).

People Also Search

Scientific Reports , Article Number: (2025) Cite This Article We

Scientific Reports , Article number: (2025) Cite this article We are providing an unedited version of this manuscript to give early access to its findings. Before final publication, the manuscript will undergo further editing. Please note there may be errors present which affect the content, and all legal disclaimers apply. Mainstream media, with its broad reach, plays a central role in shaping pu...

Although Effects Such As These Have Been Widely Studied By

Although effects such as these have been widely studied by scholars of framing, much of the existing research focuses on specific topics and relies on manually constructed or pre-existing frames, limiting both scalability and... Here we introduce a novel framework that leverages large language models (LLMs) to generate synthetic news articles by systematically varying the selection and tone of the...

Liedke, J. & Wang, L. News Platform Fact Sheet. Pew

Liedke, J. & Wang, L. News platform fact sheet. Pew Res. Center 15, 253 (2023).