Kevin Aslett
Computational Social Science

Threats Digital Technology Pose for Democracy

Evaluating Policies Designed to Mitigate Threats of Digital Technology

New Research Methods

Cracking Open the News Feed: Exploring What U.S. Facebook Users See and Share with Large-Scale Platform Data
(with Andrew Guess, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Published at Journal of Quantitative Description: Digital Media
Article

Do Your Own Research? Searching for Additional Information Online About Misinformation Increases Belief in Misinformation
(with William Godel, Zeve Sanderson, Nate Persily, Joshua Tucker, and Jonathan Nagler)
Published at Nature
Article

Framing Parkland: A Social Media Approach To Studying Issue Frames And Their Impact
(with Andreu Casas, Nora Webb Williams, John Wilkerson, and Wesley Zuidema)
Published at Policy-Studies Journal
Article

Abstract
In this study, we analyze for the first time newly available engagement data covering millions of web links shared on Facebook to describe how and by which categories of U.S. users different types of news are seen and shared on the platform. Using a combination of curated lists and supervised classifiers we focus on articles from low-credibility news publishers, credible news sources, purveyors of clickbait, and political news. Our results support recent findings that more fake news is shared by older users and there is a preference for ideologically congenial misinformation. We also find that fake news articles related to politics are more popular among older Americans than other types, while the youngest users share relatively more articles with clickbait headlines. Across the platform, however, articles from credible news sources are shared over 5 times more often and viewed over 7 times more often than articles from low-credibility sources.

Abstract
Considerable scholarly attention has been paid to understanding belief in online misinformation with a particular focus on social networks. However, the dominant role of search engines in the information environment remains underexplored, even though the use of online search to evaluate the veracity of information is a central component of media literacy interventions. Although conventional wisdom suggests that searching online when evaluating misinformation would reduce belief in it, there is little empirical evidence to evaluate this claim. Here, across five experiments, we present consistent evidence that online search to evaluate the truthfulness of false news articles actually increases the probability of believing them. To shed light on this relationship, we combine survey data with digital trace data collected using a custom browser extension. We find that the search effect is concentrated among individuals for whom search engines return lower-quality information. Our results indicate that those who search online to evaluate misinformation risk falling into data voids, or informational spaces in which there is corroborating evidence from low-quality sources. We also find consistent evidence that searching online to evaluate news increases belief in true news from low-quality sources, but inconsistent evidence that it increases belief in true news from mainstream sources.

Abstract
Agenda setting and issue framing research investigates how frames impact public attention, policy decisions, and political outcomes. Social media sites, such as Twitter, provide opportunities to study framing dynamics in an important area of political discourse. We present a method for identifying frames in tweets and measuring their effectiveness. We use topic modeling combined with manual validation to identify recurrent problem frames and topics in thousands of tweets by gun rights and gun control groups following the Marjory Stoneman Douglas High School in Parkland, Florida, shooting. We find that each side used Twitter to advance competing policy narratives. Gun rights groups' narratives implied that more gun restrictions were not the solution. Their most effective frame focused on officials’ failures to enforce existing laws. In contrast, gun control groups portrayed easy access to guns as the problem and emphasized the importance of mobilizing politically.

Testing The Effect of Information on Discerning the Veracity of News in Real-Time
(with William Godel, Zeve Sanderson, Nate Persily, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Published at Journal of Experimental Political Science
Article

News credibility labels have limited average effects on news diet quality and fail to reduce misperceptions
(with Andrew Guess, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Published at Science Advances
Article

An Automatic Framework to Continuously Monitor Multi-Platform Information Spread
(with Zhouhan Chen, Jen Rosiere Reynolds, Juliana Freire, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Published at the Proceedings of the Workshop on Misinformation Integrity in Social Networks 2021 co-located with the 30th Web Conference (TheWebConf 2021)
Article
Research Tool

Abstract
Despite broad adoption of digital media literacy interventions that provide online users with more information when consuming news, relatively little is known about the effect of this additional information on the discernment of news veracity in real-time. Gaining a comprehensive understanding of how information impacts discernment of news veracity has been hindered by challenges of external and ecological validity. Using a series of pre-registered experiments, we measure this effect in real-time. Access to the full article relative to solely the headline/lede and access to source information improves an individual's ability to correctly discern the veracity of news. We also find that encouraging individuals to search online increases belief in both false/misleading and true news. Taken together, we provide a generalizable method for measuring the effect of information on news discernment, as well as crucial evidence for practitioners developing strategies for improving the public's digital media literacy.

Abstract
As the primary arena for viral misinformation shifts toward transnational threats such as the Covid-19 pandemic, the search continues for scalable, lasting countermeasures compatible with principles of transparency and free expression. To advance scientific understanding and inform future interventions, we conducted a randomized field experiment evaluating the impact of source credibility labels embedded in users' social feeds and search results pages. By combining representative surveys and digital trace data from a subset of respondents, we provide a rare ecologically valid test of such an intervention on both attitudes and behavior. On average across the sample, we are unable to detect changes in real-world consumption of news from low-quality sources after three weeks. However, we present suggestive evidence of a substantively meaningful increase in news diet quality among the heaviest consumers of misinformation in our sample.

Abstract
Identifying and tracking the proliferation of misinformation, or fake news, poses unique challenges to academic researchers and online social networking platforms. Fake news increasingly traverses multiple platforms, posted on one platform and then re-shared on another, making it difficult to manually track the spread of individual messages. Also, the prevalence of fake news cannot be measured by a single indicator, but requires an ensemble of metrics that quantify information spread along multiple dimensions. To address these issues, we propose a framework called Information Tracer, that can (1) track the spread of news URLs over multiple platforms, (2) generate customizable metrics, and (3) enable investigators to compare, calibrate, and identify possible fake news stories. We implement a system that tracks URLs over Twitter, Facebook and Reddit and operationalize three impact indicators–Total Interaction, Breakout Scale and Coefficient of Traffic Manipulation–to quantify news spread patterns. We also demonstrate how our system can discover URLs whose spread pattern deviate from the norm, and be used to coordinate human fact-checking of news domains. Our framework provides a readily usable solution for researchers to trace information across multiple platforms, to experiment with new indicators, and to discover low-quality news URLs in near real-time.

How Language Shapes Belief in Misinformation: A Study Among Multilingual Speakers in Ukraine
(with Aaron Erlich, Jonathan Nagler, and Joshua Tucker)
Revise and Resubmit at Journal of Experimental Political Science
Working Paper

Moderating with the Mob: Evaluating the Efficacy of Real Time Crowdsourced Fact Checking
(with William Godel, Zeve Sanderson, Nate Persily, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Published at Journal of Online Trust and Safety
Article

When Conservatives See Red but Liberals Feel Blue: Why Labeler-Characteristic Bias Matters for Data Annotation
(with Andreu Casas, Nora Webb Williams, and John Wilkerson)
Working Paper

Abstract
Our cumulative knowledge about belief in misinformation and true news predominantly comes from surveying Americans about news (true and false) written in English from American media sources. However, the global media environment is complexly multilingual. Half of the global population uses two or more languages or dialects in their daily life and, therefore, likely consumes media, including misinformation, in multiple languages. As more multilingual speakers in a single media market consume news in different languages from different sources, it is imperative we develop a more comprehensive understanding of how news consumers perceive news in different languages. We test a new theory about the proficiency and source effect of language by randomly assigning respondents to evaluate false and true news stories in either their dominant or less proficient language (Russian or Ukrainian) in Ukraine in the period of time in which individuals are most likely to consume this news (directly after the publication of an article). Using this survey instrument we answer one main research questions: In areas in which most news is reported in two languages, are multilingual individuals less skeptical of misinformation produced in their less proficient language?

Abstract
Reducing the spread of false and misleading news remains a challenge for social media platforms, as the current strategy of using third-party fact-checkers lacks the capacity to address both the scale and speed of misinformation diffusion. Recent research on the “wisdom of the crowds” suggests one possible solution: aggregating the evaluations of groups of ordinary users to assess the veracity of online information. Using a pre-registered research design, we investigate the effectiveness of crowdsourced fact checking in real time. We select popular news stories in real time and have them evaluated by both ordinary individuals and professional fact checkers within 72 hours of publication. Our data consists of 21,531 individual evaluations across 135 articles published between November 2019 and June 2020. Although we find that machine learning based models (that use the crowd as input) perform significantly better than simple aggregation rules at identifying false news, our results suggest that neither approach can perform at the level of professional fact checkers.

Abstract
Human annotation of data, including text and image materials, is a bedrock of political science research. Yet we often overlook how the identities of our annotators may systematically affect their labels. We call the sensitivity of labels to annotator identity "labeler-characteristic bias" (LCB). We demonstrate the persistence and risks of LCB for downstream analyses in two examples, first with image data from the United States and second with text data from the Netherlands. In both examples we observe significant differences in annotations based on annotator gender and political identity. After laying out a general typology of annotator biases and their relationship to inter-rater reliability, we provide suggestions and solutions for how to handle LCB. The first step to addressing LCB is to recruit a diverse labeler corps and test for LCB. Where LCB is found, solutions are modeling subgroup effects or generating composite labels based on target population demographics.

Learning Media Quality from Facebook Data.
(with Tom Paskhalis, Cody Buntain, Zhanna Tereschenko, Jonathan Nagler, Richard Bonneau, and Joshua Tucker)
Working Paper

Measuring and mitigating misinformation at the scale of the social media ecosystem
(with Christ Tokita, William Godel, Zeve Sanderson, Nate Persily, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Working Paper

Automated Visual Clustering: A Technique for Image Corpus Exploration and Cost Reduction.
(with Andreu Casas, Nora Webb Williams, and John Wilkerson)
Working Paper

Abstract
The production, consumption, and dissemination of online news is of growing interest among scholars studying democracy, but much difficulty lies in the study of media quality in a comparative perspective. Many problems plague cross-country studies, but studies of media credibility are particularly susceptible to issues such as varying political environments, language barriers, cultural contexts, and differing media regulation. This study leverages an original dataset published by Facebook through the Social Science One initiative to study the prevalence of unreliable online news in 27 countries in Europe. We use a supervised model (trained on US data) to predict the credibility of a given news domain based on users’ feedback and behavior. We show that interactions with links to news websites on social media allow us to predict the credibility of news and a model that learns such relationships is portable across national contexts. Using this model we find an East-West divide between countries in Europe with higher proportion of unreliable news in former socialist countries, as well as in the UK. Furthermore, we find that more recently registered news domains and those registered outside of the country are more likely to be predicted to be the sources of unreliable information.

Abstract
In this paper we pair experimental survey data with observational Twitter data to create a more precise estimate of the impact of trending true and false news. We estimate the exposure of receptive users---that is, the users most likely to believe an article's content as predicted by their individual characteristics. Using this new approach, we find that millions of receptive users were potentially exposed to top-trending true and false news articles on social media. Importantly, we also find that the pattern of receptive user exposure differs between false news and true news: both true news and false news are seen by an ideologically diverse set of users, yet receptive users that are exposed to false news are far more concentrated on the conservative extreme of the ideological spectrum. Thus, efforts to infer the impact of misinformation by measuring total user exposure may not accurately capture the true impact of misinformation on social media. We extend this new method and conduct data-driven simulation to evaluate different interventions that social media platforms deploy to reduce the effect of misinformation. We find that interventions that are not deployed instantaneously are unlikely to reduce the exposure of receptive users to misinformation: most receptive users are exposed shortly after a URL is first shared and thus the effectiveness of interventions quickly dissipates with each hour delay in implementation.

Abstract
Compared to text and audio, images can be an especially effective form of political communication. It has become relatively easy to automatically label images for many features of interest (such as protests, famous people or facial expressions). As a result, scholars are increasingly using large-N image analysis to investigate contemporary political attitudes and behavior. We address three emerging needs of image scholarship. First, researchers may want to visually explore an image corpus to discern patterns before they begin assigning labels. Second, they may want to annotate images for the presence of complex theoretical mechanisms that cannot be easily assigned using existing automated methods. Third, they may be primarily interested in studying human annotation decisions. We demonstrate how unsupervised image clustering can help researchers address each of these needs when dealing with large unbalanced image corpora. We illustrate this using a corpus of images shared with the hashtag #FamiliesBelongTogether on Twitter.

Innovation and Populism
Working Paper

An Ecologically and Externally Valid Method for Assessing Belief in Popular Fake News
(with William Godel, Zeve Sanderson, Nate Persily, Joshua Tucker, Jonathan Nagler, and Richard Bonneau)
Working Paper

Abstract
Innovation has been partly blamed for the wave of populist success over the last fifteen years, but the empirical connection between innovation and populism remains relatively unclear. It is difficult to estimate the aggregate effect of innovation on support for populism in localities due to endogeneity issues. To solve for this problem I use exogenous destruction of railroad stations during World War 2 as an instrumental variable for innovation at the municipal-level in contemporary Poland. Drawing on an original geolocated data set of 496,617 patents and railroad destruction during the Second World War in Poland, I find that innovation reduces support for populist political candidates and parties. I also attempt to identify the causal pathway through which innovation affects support for populism using mediation analysis. This analysis finds that innovation may suppress support for populism by increasing the mobility of populations but not by increasing local human capital.

Abstract
Current research measuring the level of belief in fake news and the types of people who are more likely to believe it has done so by asking respondents to evaluate out-of-date headlines and ledes that they have chosen themselves. This method strays from how we measure and observe fake news consumption in the wild and could bias our understanding of belief in misinformation. To test if integrating advances in research on the consumption of fake news into survey instruments measuring belief in fake news changes our understanding of belief in fake news, we fielded three studies in which we repeatedly asked representative samples of Americans to evaluate popular full articles from non-credible and credible sources chosen by a pre-registered algorithm within 24-48 hours of their publication. By sourcing popular fake news articles without researcher selection and asking respondents to evaluate the full articles in the period news consumers are exposed to them, we find that, on average, false or misleading articles are rated as true 33.2% of the time; moreover, approximately 90% of individuals coded at least one false or misleading article as true when given a set of four false or misleading articles. Strikingly, these results are much higher than statistics reported in previous studies.