University of Kwazulu-Natal

A Data Driven Approach To Supporting Fact Checking And Mitigating Mis

Bonisiwe Shabane
-
a data driven approach to supporting fact checking and mitigating mis

Misinformation and disinformation spread rapidly on social media, threatening public discourse, democratic processes, and social cohesion. One promising strategy to address these challenges is to evaluate the trustworthiness of entire domains (source websites) as a proxy for content credibility. This approach demands methods that are both scalable and data-driven. However, current solutions like NewsGuard and MBFC rely on expert assessments, cover only a limited number of domains, and often require paid subscriptions. These constraints limit their usefulness for large-scale research.This study introduces a machine-learning-based system designed to assess the quality and trustworthiness of websites. We propose a data-driven approach that leverages a large dataset of expert-rated domains to predict credibility scores for previously unseen domains using domain-level features.

Our supervised regression model achieves moderate performance, with a mean absolute error of 0.12. Using feature importance analysis, we found that PageRank-based features provided the greatest reduction in prediction error, confirming that link-based indicators play a central role in domain trustworthiness. This highlights the importance of highly referenced domains in reliable news dissemination. This approach can also help fact-checkers and social media platforms refine their credibility assessment strategies.The solution’s scalable design accommodates the continuously evolving nature of online content, ensuring that evaluations remain timely and relevant. The framework enables continuous assessment of thousands of domains with minimal manual effort. This capability allows stakeholders (social media platforms, media monitoring organizations, content moderators, and researchers) to allocate resources more efficiently, prioritize verification efforts, and reduce exposure to questionable sources.

Ultimately, this facilitates a more proactive and effective response to misinformation while also supporting robust public discourse and informed decision-making. A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions. The Evolution and Future of Fact-Checking in a Misinformation-Ridden World A decade ago, the fight against misinformation coalesced around a simple yet powerful idea: establish a global network of fact-checkers to debunk false claims and promote accurate information. This nascent movement, driven by organizations like the International Fact-Checking Network (IFCN), gained traction, earning the support of major platforms and becoming an integral part of public discourse.

However, the landscape shifted dramatically with the advent of the COVID-19 pandemic and the contentious 2020 US presidential election. These events thrust fact-checking into the global spotlight, simultaneously increasing its importance and exposing its vulnerabilities. While platforms like Meta initially invested heavily in these efforts, the underlying model struggled to keep pace with the evolving nature of online misinformation. The limitations of the traditional fact-checking model became increasingly apparent. While effective in debunking individual claims, it proved too slow and resource-intensive to combat the sheer volume and velocity of misinformation spreading online. Furthermore, fact-checkers faced increasing harassment and accusations of censorship, highlighting the precarious position they occupied in the online information ecosystem.

The reliance on platform funding also created a dependency that left fact-checking organizations vulnerable to shifts in platform priorities, a vulnerability starkly revealed by Meta’s recent decision to discontinue its third-party fact-checking program in... This move signaled a broader industry trend towards crowdsourced solutions like X’s (formerly Twitter’s) Community Notes, shifting the responsibility of verification away from professional journalists. While crowdsourcing offers scalability, it also presents significant challenges. Community-based systems are susceptible to manipulation, brigading, and the dominance of popular narratives, even if those narratives are factually incorrect. This raises serious concerns about the reliability of such systems, particularly when dealing with critical issues like public health, elections, and security. The expertise and methodological rigor of professional fact-checkers are essential components of a robust verification system, and cannot be easily replicated by crowdsourced opinions.

The focus must shift from simply gathering diverse perspectives to ensuring the accuracy and credibility of the information itself. The current model of fact-checking, largely reliant on manual processes and journalistic investigations, is ill-equipped to handle the speed and scale of today’s misinformation ecosystem. The rise of AI-generated content, sophisticated disinformation networks, and virality-driven platforms demands a more technologically advanced approach. Furthermore, the financial sustainability of fact-checking organizations remains a significant concern. The dependence on grant funding and short-term platform partnerships creates instability and hinders long-term planning. Meta’s decision to withdraw funding underscores the inherent fragility of this model and the need for a more sustainable approach.

Research output: Contribution to journal › Article › peer-review This study examined four fact checkers (Snopes, PolitiFact, Logically, and the Australian Associated Press FactCheck) using a data-driven approach. First, we scraped 22,349 fact-checking articles from Snopes and PolitiFact and compared their results and agreement on verdicts. Generally, the two fact checkers agreed with each other, with only one conflicting verdict among 749 matching claims after adjusting minor rating differences. Next, we assessed 1,820 fact-checking articles from Logically and the Australian Associated Press FactCheck and highlighted the differences in their fact-checking behaviors. Major events like the COVID-19 pandemic and the presidential election drove increased the frequency of fact-checking, with notable variations in ratings and authors across fact checkers.

Research output: Contribution to journal › Article › peer-review N1 - Publisher Copyright: © 2023, Harvard Kennedy School. All rights reserved. N2 - This study examined four fact checkers (Snopes, PolitiFact, Logically, and the Australian Associated Press FactCheck) using a data-driven approach. First, we scraped 22,349 fact-checking articles from Snopes and PolitiFact and compared their results and agreement on verdicts. Generally, the two fact checkers agreed with each other, with only one conflicting verdict among 749 matching claims after adjusting minor rating differences.

Next, we assessed 1,820 fact-checking articles from Logically and the Australian Associated Press FactCheck and highlighted the differences in their fact-checking behaviors. Major events like the COVID-19 pandemic and the presidential election drove increased the frequency of fact-checking, with notable variations in ratings and authors across fact checkers. This study examined four fact checkers (Snopes, PolitiFact, Logically, and the Australian Associated Press FactCheck) using a data-driven approach. First, we scraped 22,349 fact-checking articles from Snopes and PolitiFact and compared their results and agreement on verdicts. Generally, the two fact checkers agreed with each other, with only one conflicting verdict among 749 matching claims after adjusting minor rating differences. Next, we assessed 1,820 fact-checking articles from Logically and the Australian Associated Press FactCheck and highlighted the differences in their fact-checking behaviors.

Major events like the COVID-19 pandemic and the presidential election drove increased the frequency of fact-checking, with notable variations in ratings and authors across fact checkers. College of Information Sciences and Technology, The Pennsylvania State University, USA College of Information Sciences and Technology, The Pennsylvania State University, USA College of Information Sciences and Technology, The Pennsylvania State University, USA College of Information Sciences and Technology, The Pennsylvania State University, USA

People Also Search

Misinformation And Disinformation Spread Rapidly On Social Media, Threatening Public

Misinformation and disinformation spread rapidly on social media, threatening public discourse, democratic processes, and social cohesion. One promising strategy to address these challenges is to evaluate the trustworthiness of entire domains (source websites) as a proxy for content credibility. This approach demands methods that are both scalable and data-driven. However, current solutions like N...

Our Supervised Regression Model Achieves Moderate Performance, With A Mean

Our supervised regression model achieves moderate performance, with a mean absolute error of 0.12. Using feature importance analysis, we found that PageRank-based features provided the greatest reduction in prediction error, confirming that link-based indicators play a central role in domain trustworthiness. This highlights the importance of highly referenced domains in reliable news dissemination...

Ultimately, This Facilitates A More Proactive And Effective Response To

Ultimately, this facilitates a more proactive and effective response to misinformation while also supporting robust public discourse and informed decision-making. A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity.© Copyright 2025 IEEE - All rights reserved. Use of this web site signifies your...

However, The Landscape Shifted Dramatically With The Advent Of The

However, the landscape shifted dramatically with the advent of the COVID-19 pandemic and the contentious 2020 US presidential election. These events thrust fact-checking into the global spotlight, simultaneously increasing its importance and exposing its vulnerabilities. While platforms like Meta initially invested heavily in these efforts, the underlying model struggled to keep pace with the evol...

The Reliance On Platform Funding Also Created A Dependency That

The reliance on platform funding also created a dependency that left fact-checking organizations vulnerable to shifts in platform priorities, a vulnerability starkly revealed by Meta’s recent decision to discontinue its third-party fact-checking program in... This move signaled a broader industry trend towards crowdsourced solutions like X’s (formerly Twitter’s) Community Notes, shifting the respo...