Measuring media impact: A brief history and analysis


Jamie Walsh, D.Phil student, Blavatnik School of Government, Oxford, Team member, Behavioral Science Unit (eMBeD), World Bank


Recent worries that social media partisanship is harming western democracy beg two important questions. First, what impact does the media actually have on how people think and behave? Second, if media does affect mindsets and behaviors, what are the practical implications for public policy?

Whether and how much media affects behavior is a longstanding controversy in the social sciences. During the first half of the 20th century the media was regarded as a source of substantial power. The father of modern journalism, Walter Lippmann, emphasized how indirectly we know the environment in which nevertheless we live.” He argued that the media is the foundational basis upon which we form our understanding of reality. The communications theorist Harold Lasswell similarly conceived of mass communication as a tool of immense power for the purposes of propaganda and social manipulation.  Fear of propaganda during the early-mid twentieth century led to concerns in the U.S. Congress that foreign entities might seek to broadcast subversive messages to its citizens. But by the mid-twentieth century empirical investigations were unable to find effects of the magnitude that Lipmann and Lasswell had claimed.

Measurement challenges

Rigorous measurement of media impact outside the lab is tough. The peculiarities of media demand creates selection bias—systematic differences between an “exposed” group being studied and the “unexposed” group against which it is being compared. As Joseph Klapper argued over 50 years ago, this is especially problematic in media research because people are drawn to information that confirms their prior beliefs and attitudes. Consider, for example, the following question: does the New York Times instantiate liberal views in its readership or do liberals choose to read the New York Times? Correlation by itself does not answer the question.

Researchers have found ways to overcome selection bias. Multi-wave and longitudinal survey data have become more common, enabling scholars to examine trends and investigate dynamic effects within countries. Some cross-country comparisons combine data on access to media sources (e.g., television, radio, newspaper) with survey data on social values.

These approaches reduce selection bias by examining behavioral changes within the same group across time. But the potential for bias due to other factors remains. Attitudinal or behavioral change may co-vary with exposure to media, especially in countries undergoing structural transformations. This obscures whether changes are due to media exposure or another contemporaneous social change (for example, an increase in income).

In some cases randomized field experiments (and lab-in-field experiments) have been used to measure the impact of exposure to media (for example, in Rwanda, Ethiopia, Nigeria, and Sudan). However, field experiments often have methodological shortcomings. Only in limited circumstances is it possible to ensure (or even check whether) individuals in the “treatment” group have interacted with individuals in the “control” group. This methodological problem, known as a spill-over effect, undermines the credibility of the counterfactual.

Quasi-experimental methods are also now also used. Natural variation in people’s exposure media is exploited to develop estimates of its causal impact. This empirical strategy only works under very specific conditions: the determinants of access to media must be unrelated to the outcomes of interest except through the mechanism of media exposure. By comparing statistically similar groups, some who arbitrarily have access to media and others who arbitrarily do not (for example, due to accidents of geographic or administrative history), researchers make causal inferences based on more realistic counterfactuals.

How media has affected societies

Despite these challenges, rigorous empirical methods have been employed to generate rich evidence on the effect of media on human behavior. Economists exploiting variation in exposure to Western television programming in East Berlin during the Cold War (see figure below) have found robust evidence that exposure to West German television resulted in lower fertility, higher aspirations, and preference for Western goods (though no impact on aggregate consumption levels) in East Germany. Moreover, people from areas that were more exposed to Western television were more likely to believe that effort, rather than luck, determined one’s success in life. In some cases, the media effects persisted for up to ten years after reunification.

Screen Shot 2018-03-23 at 12.15.22.pngWestern television stations’ signal strength in Eastern Germany (the German Democratic Republic) Source: Bönisch and Hyll (2015) Note: The figure shows the German Democratic Republic divided into administrative boundaries (Bezirksgrenzen). The areas shaded in black were dead spots for Western television reception.

Economists have used natural experiments in the United States to detect the effects of changes in the media landscape. Matthew Gentzkow has found that the growth of television viewership was associated with a fall in newspaper readership and voter turnout. He calculates that it was responsible for between a quarter to a half of the total fall in voter turnout from the 1950s through to the 1970s. Stefano DellaVigna and Ethan Kaplan (2007) took advantage of variation in access to Fox News Channel to estimate the impact of its coverage on support for Republicans vote share, calculating that it was responsible for persuading between 5-30 percent of non-Republican voters to switch to supporting George W. Bush in the 2000 election.

One of the most striking findings from research on media impact is just how pernicious its effects can be. During the Rwandan genocide, ethnic Hutu members of the Rwandan government led a radio-campaign that called for the extermination of the Tutsi ethnic minority. The effects were catastrophic. David Yanagizawa-Drott estimates that about 10 percent of the overall violence (the acts of 51,000 perpetrators) can be attributed to the station.

In Croatia, Stefano DellaVigna and his coauthors found that media exposure has significantly exacerbated national tensions. Some border communities have greater exposure to Serbian radio than others and community members listen to it for its entertainment content. In doing so, they are exposed to media that has an explicit and official mission to strengthen Serbian national identity. In areas that exposure to Serbian radio is high, the vote share for extremist nationalist political parties is greater and the probability of finding ethnically offensive graffiti is higher.

Media as a social remedy

While the media has prompted civic disengagement and incited violence, it can also be used for social and developmental purposes. A decade after the genocide in Rwanda, Betsy Paluck tested the impact of a radio soap opera aimed at reducing intergroup prejudice, violence, and trauma in Rwanda. The soap opera featured a “Romeo and Juliet” style love story centered on two fictional Rwandan communities. Although, the program did not change listeners’ personal beliefs, it shifted their perceptions of social norms and behaviors with respect to intermarriage, open dissent, trust, empathy, and cooperation.

As documented the World Development Report 2015, the use of entertainment media to promote social goals has proliferated over the last five decades. BBC Media Action is one such pioneer. Many efforts have been experimentally investigated. These find that rural villagers in Ethiopia exposed to films depicting entrepreneurs choose to invest more in their children, viewers of soap operas embedded with financial messaging become more financially literate, less likely to gamble, and more likely to use formal lending mechanisms. Exposure to television depicting agentic women in India and Brazil has promoted respect for women in India and reduced fertility rates from very high levels in Brazil.

Mechanisms of impact

The mechanisms by which media shapes behavior remains an unresolved debate. Social scientists in the intellectual traditions of social psychology and sociology emphasize the psychological dimensions of media impact: media communication sets agendas by priming cognitions and framing issues, and shapes how people see themselves and the world around them. Economists in the rational choice tradition model people as objective observers of the world. Media communications is nothing more than source of information.

The hermeneutical disagreement is undergirded by fundamental differences in belief about human nature. The rational actor in economic models is conceived as being stripped of social influence, with stable or even fixed preferences. In contrast, the individual in interpretative social sciences is conceived of as a product of society, imbuing meaning in situations on the basis of the social constructions available. A “second strand” of behavioral economics aims introduce the interpretive model into standard economics.

Media and behavioral public policy

The ethical implications of this dispute on policymaking are sizeable. If the media plays a purely informational role, then it simply causes people to update their stock of knowledge. Media effects are best conceived as a limited and benign set of impacts and its risk is isolated to the spread of “fake news.”  However, if the media plays a formative role not just in informing people of facts but in shaping the psychological basis on which people understand themselves, their neighbors, and their relationship to politics, then the stakes are much higher.

Further research is necessary to outline a normative framework for media use in policy. Recent events illustrate that media communications will play an increasingly central but ethically complex role in behavioral public policy in the years to come. Its ethical complexity demands serious engagement. If the media powerfully affects people’s sense of self-worth and dignity, interventions risk undermining individual agency. Conversely, they also have the capacity to expand how people see themselves, their world, and their possibility.