Gavin Adamson, Ryerson University
About the author
Gavin Adamson (firstname.lastname@example.org) is an assistant professor at the Ryerson School of Journalism. He’s interested in research about mental health news and how new reporting and distribution platforms are transforming the news media. He teaches introduction to online journalism and helps oversee the integrated digital-first newsroom for graduating students in the Bachelor and Masters of Journalism programs. He can also be reached @ga_adamson.
Both news media and its researchers aim to understand how online editorial content resonates with audiences, especially in conjunction with social media. The paper investigates that concern by analyzing a news domain, mental health, to show that data analysis of social media can be fruitful for measuring audience engagement in editorial content. Most surprisingly, we found statistically that social media audiences tend to share more frequently content in this realm that is not related to crime and violence. Our approach contributes to a growing understanding of what sort of editorial content the audience is more likely to engage with and redistribute, with hints at a reconsideration of editorial approaches.
Speaking on a panel about the state of multi-platform, multimedia journalism at the Canadian Association of Journalist conference in Ottawa in June, 2013, Marissa Nelson, a senior editorial executive at CBCNews.ca asked for more discussion and research about engagement (J-source, 2013). Management and editorial staff for news media are interested in engagement because of twinned concerns around building audience and business interests, in that pageviews and readership measurement is directly related to advertising revenue and opportunity (Grueskin et al, 2010).
Social media and audience
This project concentrates on Twitter because it is now a “normalized” element of journalism (Lasorsa et Al, 2011). Every major news organization, and most minor ones, own a Twitter account – sometimes several, for different sections and beats – and use the platform heavily to attempt to engage with readers.
Journalists use the medium as both a way to source of stories and also a way to distribute stories to the audience (Farhi, 2009, Armstrong et Al, 2010) and to drive page view analytics on websites. Twitter is Canada’s second- largest social medium after Facebook. Ipsos Canada estimated in 2011 that about 20 per cent of Canadians had an account while Quantcast estimates that more than 700,000 Canadian users access Twitter every month via mobile devices such as cell phones and tablets.
The ubiquity of the combined eyes and ears of all of Twitter’s users has been described as “ambient journalism” (Hermida, 2011), or an awareness system. Focused queries on social media, researchers suggest, can be a strong proxy for what the public cares about (Ripberger 2010).
In describing readership trends for online journalism, Mark Deuze (2010) defines a media economy in which readers consume news the same way that they consume all media and argues that “market logic” will determine what constitutes the news rather than “editorial logic” (2005). In other words, the audience will read what it likes and editors will tailor content to suit those interests. This is a result of (and results in) more editors considering page view analytics (i.e., the attention that stories receive from readers) when making decisions about what stories to produce and publish (Grueskin 2011). Editors are torn between which stories drive readers to their websites and what constitutes good public-service journalism. Already, we know that crime is the most commonly Tweeted topic by news organizations (Armstrong 2010) but at least one study about what news is shared suggests that might not be a successful audience-gathering strategy for newsroom Twitter accounts (Berger, 2011). And we know audience gathering is a going concern even apart from commercial interests. An ethnographic study of audience and user interaction in newsrooms suggests that professional reporters and editors are increasingly reliant on audience metrics as a supplement to news judgment (Anderson, 2011).
Mental health reporting
Studies by researchers into mental health journalism underline challenges with regards to sourcing and framing. The most recent study, Whitley’s retrospective study on Canadian newspaper articles from 2006-2010 showed that in 87 per cent of online, newspaper and video transcripts related to mental illness, no one with lived experience of a mental illness was quoted. In the same corpus of articles, 75 per cent did not quote a mental health care professional (Whitley, 2012 in press).
A discourse analysis of British journalism also identified the media’s inability to let psychiatric patients speak for themselves, and pointed to a preference for sources from the justice system and, less frequently, the healthcare system (Cross, 2010).
The framing of stories about mental health is similarly problematic. In cases in whcih mental health or illness is a subject in a news report, framing analysis asks, “what’s the context for the story?” Several studies – one of them a global review of the relevant literature – show that between 10 and 40 per cent of media reports about mental health have to do with violent crime (Whitley 2012, Francis et al 2001). A compelling Australian study detailed an over-emphasis on risk-based coverage around escaped forensic psychiatric patients from a local institution. The study resulted in a government-led review of media practices into this sort of coverage (Warwick 2004).
The concern about reporting around this subject is also a timely one, at least for the Canadian Broadcasting Corporation, which is working along with another stakeholder organization on a guide for reporting about mental health (Newswire, 2013).
Health researchers have documented the close relationship between stories reported by the media, the public understanding of health issues, and public policy (Soroka, 2007). For example, U.S. journalists have been shown to be consensus builders against unacceptable public health policy surrounding issues such as female genital mutilation in Africa (Wade, 2011). Researchers argue that the media have also helped to maintain a status quo of poor social and health policy for Australia’s Indigenous population (McCallum, 2011). Similarly, Canadian research on obesity coverage in the media has concluded that there is an interdependent relationship between “health sources and journalists that shapes the inception and mediation of obesity research and the translation of health research to the public” (Roy et al., 2007). Where mental health is concerned, stigma in public perception inevitably affects government policy (Corrigan et al., 2005). This stigma jeopardizes funding for mental healthcare research, drug treatment research, and infrastructure designed to cushion the social impact of mental illness, such as affordable or public housing (Smith and Giggs, 1988).
We approached these considerations with three research questions:
Research question 1
Would we be able to adequately track social media traffic for online news articles?
Among our concerns was whether our social media tool, Sysomos, would provide a full enough data set to be able to trace traffic related to our news set. Several researchers have noted that some social media data tools are missing random and untold data (Lewis et al, 2013). Sysomos describes in its marketing material on its website that it is one of a few companies with complete access to the so-called Twitter “firehose,” a complete archive of Twitter activity. Yet Sysomos limits historical access to the data to one year and despite advertisements, many media titles were not available in the traditional media set in its database. We had intended to survey audience and traffic related to titles such as the Winnipeg Free Press and either major Vancouver dailies, for example, and neither was available. Major east coast newsrooms were also not represented.
We thought that as long as there was no obvious bias in the missing data from the Twitter stream, and missing social media data was random and infrequent, then whatever search results we yielded should bear analysis.
Research question 2
In our online news research, would we find the same tendency towards a context of crime and violence as discovered by previous research teams studying the print medium? Online news distribution for many newsrooms often involves re-purposing of print content but there is a possibility that the introduction of strictly online reports could change the results.
We hypothesized that a study in a purely online context should yield roughly the same results as previous ones based on the print medium.
Research question 3
How would mental health news be distributed by Twitter users? Would they tend to amplify the identified challenges around this subject realm by over-distributing news related to crime and violence?
Social media traffic would have an amplifying effect on news distribution of crime-based studies in this subject realm. In other words, the distribution of news about violence and crime should spike around dramatic headlines, effectively heightening the challenges in this subject realm described by previous media researchers.
We used a simplified online content analysis method to answer our questions. We gathered news related to mental health during the three-month period to February 10, 2012 from Oct 10, 2011, manually coded it, then analyzed interaction with that news via Twitter as defined by tweets and retweets.
We used MAP, a commercially available social media-tracking platform owned by Marketwire that’s part of its Sysomos platform. The Ryerson University Faculty of Communication and Design had an academic license for it during the study period. Other free platforms, such as Topsy.com, would have provided most of the same data but Sysomos has some other features that could be useful and it is built to export data in csv formats, which facilitated the establishment of a dataset very quick once we had limited our search terms as below. It was also a convenient platform because, although its traditional media database was limited, it allowed for the search of that database and the Twitter data on the same platform,.
We tracked online news from The Globe and Mail, the Toronto Star and the Hamilton Spectator because they were available in the Sysomos data base as identified as “globeandmail.com,” “thestar.com” and “thespec.com.” At the time, the platform did not include major news titles in other provinces.
We identified news stories in the mental health news realm by doing exclusive Boolean searches through the Sysomos data base on, “mental health” and “mental illness.” At this point we dropped the Hamilton Spectator content because we felt there were too few results. To determine the incidence of sharing of online articles by Twitter users, we searched for each article title using the article title as a Boolean search term under the Sysomos social media tab, using the Twitter search button. News organizations tend to use their online headlines to distribute the URLs for their content. We recognized that Twitter users do not always practice news distribution in the same way. We knew we would be missing some data points but we could not conceive of a reason why limitation would not be evenly and randomly distributed among our headline set and therefore not a limitation that would affect the validity of our results. We left an open-end date for tracking each article’s life on the social networking platform assuming that some distribution of content may happen hours or days after the original tweet from the organization
Coding for violence and crime
Next, we applied a simple manual coding procedure to the articles we retrieved via Sysomos. This involved reading the article and its headline to identify whether each was related to crime or violence (YES or NO). A single researcher undertook that judgment. The coder read the content to make the determination. We took that approach on consideration of many studies around media and mental illness, one of them a global review of all research, and the latest, a study on Canadian news media (Whitley, 2013). In these studies, the determination of framing and context around crime is a key consideration (but not the only one) for understanding how the media cover the subject. For the purposes of this pilot study, it was the only judgment we made. For illustration, the table below illustrates the results for a portion of the table for “mental health” in the Toronto Star.
Over the course of three months, we were able to identify online and social media traffic related to 142 stories in the three media titles. Among those, 23 articles were omitted because Sysomos could locate tweets or narrow activity to a single news media source. The total tweets and retweets relating to those titles (N=2626). .
Our study corroborated previous ones that showed news media tend to frame mental health in the context of crime and violence at a rate of 39 per cent. Of our total of 119, 65 stories were coded relating to crime (fig. 1, below)
Our hypothesis was false. Overall, tweets and Twitter activity that were not related to the articles that were coded for crime and violence were distributed at a greater rate than those that were. In another manner of speaking, social media had an overall dampening effect on the news media’s tendency to cover mental health in the context of crime and violence because people were more likely to redistribute other news in the realm.
We applied t-tests on the data to determine the statistical significance of the observations. All tests were conducted with equal variances among groups assumed, and then repeated with that assumption relaxed. The conclusions below hold across all assumptions and versions of the tests.
Toronto star mental health: Number of tweets is higher for articles which do not feature crime/violence than for those who do feature them, at the 0.05 level of significance.
Toronto star mental illness: No significance; the number of tweets among the two types of articles is not significantly different.
G&M mental health: Number of tweets is higher for articles that do not feature crime/violence than for those who do feature them, at the 0.01 level of significance.
G&M mental illness: No significance; the number of tweets among the two types of articles is not significantly different.
A composite t-test of all the results showed that is statistically significant at the 0.01 level that articles which do not feature crime and violence do have more tweets than those that do feature crime/violence.
Some of the results confirm previous studies and others might have been predicted based on similar research. Whitley most recently found a 40 per cent incident rate of articles in this subject realm coded to the crime and violence. Of note, one previous Canadian study suggested that a single incident during a reporting period can affect results. Stuart (2004) notes in her study that involved intervening in the reporting process on reporting in Manitoba that one violent case dominated news coverage for the period. Similarly, the mass shooting in the Newtown, Connecticut happened on Dec. 14, 2012, during our data collection period. The coverage made its impact on Canadian media along with the concomitant speculation about the connection to mental illness. Part of the reason we theorized that social media would have an amplifying effect on this sort of coverage is that we suspected the impact of one or more major stories like this could reach an exponentially grown audience via Twitter. That did not happen. Instead, one the most viral stories in the subject realm was titled, “Exam rescheduling, early intervention can help students’ mental health.”
While it was surprising, initially, that social shares tended overall, to dampen the violent and criminal framing around mental health news, that result, too, had a precedent. Using an automated sentiment analysis function, a study of news in the New York Times’ ‘most emailed’ aggregator showed that “content is more likely to become viral the more positive it is” (Berger, 2011). The uses and gratifications for email and social media may not be the same because the platforms distribute content in different ways and that is one of the ways in which this research motivates further studies. Several major questions need answering:
- What criteria do non news-media users on Twitter use when they are making a judgment about whether to redistribute content from news media?
- What is the relationship between sharing and pageviews? Do readers tend to prefer to read a certain type of content while tending to distribute different content?
- What impact are stakeholder organizations like the Centre for Addiction and Mental Health, for example, having on the re-distribution of content?
- Can the attributes for greater sharing be narrowed further? For example do stories containing certain types of sources tend to be distributed more?
Finally, can any digital tools be developed to help journalists determine these sorts of questions to quickly make judgments, to help source story ideas and angles that are more likely to have social interest?
This dissemination of this research was made possible with the support of the Faculty of Communication and Design at Ryerson University. Part of the research was made possible with a SSHRC Insitutional Grant from Ryerson University. I would like to acknowledge the help of researchers Oghale Omole and Ishani Nath, and Ioana Moca, a research facilitator at the Ted Rogers School of Management at Ryerson University for statistical work.
Anderson, CW. “Between Creative and Quantified Audiences: Web Metrics and Changing Patterns of Newswork in Local US Newsrooms.” Journalism 12.5 (2011): 550-66. Print.
Armstrong, Cory L., and Fangfang Gao. “Now Tweet this.” Electronic News 4.4 (2010): 218-35. Print.
Blood, Warwick, and Kate Holland. “Risky News, Madness and Public Crisis.” Journalism 5.3 (2004): 323-42. Print.
Comscore. Twitter.Com Top 20 Global Markets. Web.
Corrigan, Patrick W., et al. “Newspaper Stories as Measures of Structural Stigma.” Psychiatric Services 56.5 (2005): 551-6. Print.
Deuze, Mark. “”Journalism, Citizenship and Digital Culture”.” Journalism and Citizenship. Ed. Zizi Papacharissi. New York: Routledge, 2009. Print.
Farhi, P. (2009). The Twitter explosion. American Journalism Review, 31, 26-31.
Francis, C., Pirkis, J., Dunt, D. & Blood, R.W. Mental Health and Illness in the Media: A Review of the Literature. Canberra: Commonwealth Department of Health and Ageing. (2001).
Grueskin, B., A. Seave, and L. Graves. “The Story so Far: What we Know about the Business of Digital Journalism.” (2011)Web.
Hermida, Alfred, et al. “Social Networks Transforming how Canadians Get the News.” Canadian Media Research Consortium SocialnewsPrint.
Hermida, Alfred. “Twittering the News.” Journalism Practice 4.3 (2010): 297-308. Print. Kitzinger, Jenny. “Researching Risk and the Media.” Health, Risk & Society 1.1 (1999): 55-69. Print.
J-source. (2013). Tweet/Shoot/Write – then prep for studio. Canadian Association of Journalists. Retrieved February 28, 2014, from http://live.caj.ca/Event/TweetShootWrite_–_then_prep_for_video/75806074
Lasorsa, Dominic, Seth Lewis, and Avery E. Holton. – NORMALIZING TWITTER. – Routledge Print.
Lewis, Seth C., Rodrigo Zamith, and Alfred Hermida. “Content Analysis in an Era of Big Data: A Hybrid Approach to Computational and Manual Methods.” Journal of Broadcasting & Electronic Media 57.1 (2013): 34-52. Print.
McCallum, Kerry. “Journalism and Indigenous Health Policy.” Australian Aboriginal Studies 2011 (2011): 21. Print.
Soroka, S. N. A Report to the Health Council of Canada: Canadian Perceptions of the Health Care System., 2007. Print.
Stuart, Heather. “Stigma and the Daily News: Evaluation of a Newspaper Intervention.” Canadian Journal of Psychiatry 48.10 (2003): 651-6. Print.
Smith, C. and Giggs, J. Location and Stigma: Contemporary Perspectives on Mental Health and Mental Health Care. 1988
Whitley, Rob and Berry, Sarah “Trends in Newspaper Coverage of Mental Illness in Canada: 2005-2010” Canadian Journal of Psychiatry In Press (2012)