Tag Archives: correlation is not causation

On Adam Lanza and That “School Shooting” Game

original

Much has been made of the supposed connections between Sandy Hook shooter Adam Lanza’s horrific crimes and his love of video games which, a state investigation revealed, not only resembled the gaming habits of every other American teenage boy, but weren’t all that focused on violent games; he was also particularly fond of Dance Dance Revolution.

One of the games that investigators say Lanza played is a controversial and rudimentary one called, simply, School Shooting. On its face, it’s easy to see why people would a) be offended by such a game, or Lanza’s interest in playing it, and b) why it seems like there would be some connection. However, Kotaku interviewed the game’s designer, and it reveals a point crucial to such investigations and connections-making: the game was rudimentary and barely playable.

Jacob [the designer] reached out to Kotaku because after Sedensky’s office released the report, no one knew what “School Shooting” was and some accounts seemed to take it seriously as a game or a game modification. We had never heard of it, and Sedensky’s office at the time told us it was “a very basic stand alone PC game.” Jacob wanted it known how trivial and amateurish it really was.

There’s some evidence that Lanza had a deep interest in other mass killings, although it’s been tough to tell whether he was truly fascinated by them or whether those claims have been trumped up by the same press who like to blame video games. However, if it’s true that Lanza was studying other such crimes, that could explain his interest in playing the “School Shooting” game. Who knows for certain.

However, the Kotaku piece is an important reminder that the mere presence of a video game — or any other artifact, really — in a teen killer’s room is not enough to create causation. Many kids buy books and never read them, or download games and play them once. It takes more detail than that to justify spending $10 million to study the effects of video games on youth crime.

How to read a social-science study


Photo by Flickr user Sociology at Work.

News outlets are chock-full of articles reporting some new study and what it says about health or climate change or violent video games. Reading these articles can sometimes be a little like playing that childhood game of telephone. By the time the message reaches you, it’s probably at least a little different from what the original person said. I’ve written for Poynter about how journalists can improve their coverage of research. Today I want to give everyday folks some pointers for reading and interpreting actual studies.

1. Get the actual study.
Articles usually mention the researchers involved, and the university, if there was one. Sometimes studies are locked away behind academic-journal paywalls, but if you do enough hunting you can find “author proofs” — non-final drafts — on the researchers’ web sites. Google is your friend, and you can’t assess a study if you don’t read it. I’m going to base this post off of one such proof, Christopher Ferguson and Cheryl Olson’s Video Game Violence Use Among ‘‘Vulnerable’’ Populations. Load it up and read along.

2. Consider whether the study is linked to any recent events.
Most of the studies I look at here are about kids and violent video games. Although they may not be explicit, it’s notable that these studies picked up in frequency after events like the Columbine High School massacre and attempts to blame video games like Doom. Be suspicious when the sensationalism comes first and the studies follow. Among other things, this gives researchers an agenda, whether they say so or not. One of the reasons I like Ferguson and Olson in general is that they come right out and say it:

Existing societal concerns about video games have intensified after the 1999 Columbine High School massacre (Ferguson 2013) and other well-publicized school shootings. The tragic 2012 Sandy Hook Elementary School murders in Newtown, Connecticut resurrected these debates amid reports that the 20-year-old shooter was an avid gamer (e.g., Henderson 2012). … Given … the recurring media focus on video games, researchers need to do more to answer the questions of greatest public concern regarding video games and any potential harm to youth.

3. Look at the demographics of the sample.
The sample means the group that is included in the study; demographics refers to their characteristics. Since Ferguson and Olson have said this is meant to apply to the question of whether violent video games harm youth, especially with respect to school shootings, you’d want the sample to resemble school shooters demographically; age 13 to 17, usually male, usually white, usually suburban. Often, the sample is full of college kids who are over 18. That’s less than ideal to begin with.

In Ferguson and Olson’s study here, we have a pretty good range of kids: 377 youth with a mean age of 13 (which means they were a variety of ages in that range). 182 had ADD symptoms, while 284 had symptoms of depression (yes, that means 89 of them had symptoms of both). The sample included 140 boys and 234 girls, as well as 3 who did not declare a gender. There were urban and suburban kids, though it doesn’t say how many. The kids were mostly white, but a significant number of the kids from the urban school were black.

4. Look at the size of the sample.
This one’s pretty easy. Bigger is always better. The more subjects you have, the more you’re getting away from anecdotal information and the more you’re getting into actual data. Now, a larger sample of extremely similar kids is only useful if they resemble the outside-world kids you’re trying to represent. More ideal is a larger sample of different kinds of kids. 377, the number in this study, isn’t a ton, but it’s more than the 100-200 many such studies have.

5. Look at the methodology.
Most sociological experiments are highly synthetic constructs, which are meant to single out and test one particular aspect of everyday life. In this case, Ferguson and Olson opted to put the kids through a survey or a questionnaire, which can be tricky because the results are only as good as the honesty of the kids. I personally think kids are reasonably reliable at answering certain kinds of questions (like how often they play certain kinds of video games, which was asked here) and not necessarily others (things they might not want adults to know, like sexual or drug experimentation). Still, it’s worth taking most self-report studies with a grain of salt, unless they’re corroborated with other data from the subjects’s lives.

(I’d also like to point out a detail from their survey that is relevant to this blog: Of all the kids they surveyed, only 6.1 percent had played no video games in the prior 6 months, and only 11 percent had played no violent video games. These are kids with a mean age of 13. In other words, most kids are playing them. Where’s the giant surge of violence and aggression?)

6. Look at what they say about the results.
This is especially true if you’re comparing your own read of a study to what was reported in the news. Here are the results from this study, truncated:

With the sample of children with clinically elevated depressive symptoms and regarding delinquent criminality as an outcome only stress and trait aggression were predictive of delinquent criminality. Neither exposure to video game violence nor the interaction between trait aggression and exposure to video game violence were predictive of delinquent outcomes.

In other words, among the kids with symptoms of depression, mood issues alone didn’t correlate with delinquency. Only if they were under stress and were aggressive did they have a tendency to be delinquent. Video games didn’t seem to play a role.

They go through other trait combinations and look at bullying as well as delinquency. Here’s one interesting angle to the study, which is one a lot of news outlets focused on:

Finally, with the sample once again of children with clinically elevated attention deficit symptoms and with regards to bullying behavior only trait aggression was predictive of bullying behaviors along with the interaction between trait aggression and exposure to violent games did approach significance suggesting that highly trait aggressive children who also played violent video games were less likely to engage in bullying behaviors. Exposure to video game violence was not a significant predictor of bullying behaviors.

Note that that result, while interesting, only “approached significance” mathematically, and they immediately point out that none of the findings in this section was statistically significant. In other words, it’s inconclusive.

7. Read the “discussion” section carefully.
This is where the researchers explain their findings and discuss how they think they should be interpreted. While journalists often overemphasize certain things (which makes for a more compelling news story), researchers usually de-emphasize. Here’s what they say about the above result:

[After saying they didn’t find video games were a factor n kids’ violent behavior.] The only exception was our finding that, for children with elevated attention deficit symptoms, trait aggression and video game violence interacted in such a way as to predict reduced bullying. This could be considered some small correlational evidence for a cathartic type effect, although we note it was for only one of four outcomes and small in effect size. Thus we caution against overinterpretation of this result.

8. Read the “limitations” section even more carefully.
This is where researchers will tell you the shortcomings of the methods they used, as well as the extent to which their findings are useful. They’ll usually suggest how future research can build upon what they’ve found, should anyone care to follow up with a larger or at least related study. This study’s limitations section is accompanied by a “word of caution” section, both of which are worth reading in full.

9. Check for bias.
Look at the researchers’ websites. Look at the titles of their papers, and see if you notice a trend. If they keep finding the same things over and over, there’s a chance they’re suffering from researcher bias. There’s a lot of this in the violent-video-game research world, which is one reason you see the same names popping up again and again (and finding the same things again and again).

10. Follow the money.
If research was conducted by an academic, chances are good that the money came from the university where they work. That could include public or private dollars. Unfortunately, researchers are pretty mum about where that money comes from or how much a given study cost. But at least sniff around and see what you can learn.

If you’re tired of reading about video games, this is a great breakdown of how the myth that eating breakfast helps you lose weight became so pervasive, and why it’s wrong. It also discusses the difference between association (correlation) and causation, which is important to understand when reading any sociological study, as well as how even the “gold standard” methods for research can be wrong, especially when they aren’t backed by further research.

So there you go. Now you know how to read a social-science study. Go read some more and have fun! Commenters: can you think of anything I’ve left out that you’d like to add?

After kid kills caregiver and CNN blames a violent video game, it’s time to do a little math


An 8-year-old shoots his elderly caregiver. And the police blame video games? Photo by Flickr user Whistling in the Dark.

CNN ran an article today about an 8-year-old Louisiana boy who was living with an elderly caregiver until he shot her in the back of the head Thursday, killing her. They didn’t waste much time before blaming video games. Here’s a quote from the local police:

“Although a motive for the shooting is unknown at this time investigators have learned that the juvenile suspect was playing a video game on the Play Station III ‘Grand Theft Auto IV’, a realistic game that has been associated with encouraging violence and awards points to players for killing people, just minutes before the homicide occurred.”

I have so many questions, I don’t know where to begin. Okay, that’s not entirely true. Here’s my first question:

1. Why did this little boy have access to a loaded gun?

It’s a question we’re not likely to get the answer to. Admittedly, it’s entirely possible that the boy was trained to use weapons responsibly, although part of that training involves teaching people never to point a loaded gun at another person.

2. Why are we still blaming Grand Theft Auto?

It’s worth remembering that the man most responsible for trying to create connections between this video game and youth violence, Jack Thompson was disbarred in part for his conduct in cases involving the video game. There’s no science connecting this game (or any game) to real-life violence. And let’s keep in mind that CNN and the sheriffs of Louisiana are not scientists.

3. Why don’t we trust kids to separate fact from fiction?

The kids I’ve talked to in my own research, whether they’re 8 or 12 or 18, recognized a very clear line between video-game violence and real violence. The same was true of those I interviewed for Wired in 2011. Once in a while, maybe, a kid can’t tell the difference; I remember seeing one of the early Superman movies in the theater, and my brother thinking he could jump off our play structure and fly. But he was also about 2 years old, much younger than this kid. You know who can’t tell the difference between fiction and reality so well these days? “Experts.” Oh, and Pat Robertson.

Speaking of which, I’m no expert on this kid’s life, but I’m going to go out on a limb and say if he’s living in a trailer park with a 90-year-old woman who isn’t a family member, and who leaves a loaded gun around, this kid might have bigger concerns than his video-game intake. I also find it really interesting that following this event he’s now living with his parents. I will grant that there are some kids who are better off not living with their parents, but when that’s the situation, that’s a pretty heavy thing for an 8-year-old kid to deal with on a daily basis.

I’ve already seen parents calling for the end of violent video games, but what would that solve? If this were a math equation, it would go something like:

kid + GTA + gun = fatality

Now let’s take one of those things out of the equation.

kid + GTA = fatality?

Not unless you can bludgeon someone to death with a game console.

Let’s try again:

kid + gun = fatality

Maybe. There’s still a missing factor here, that unknown something that actually made this kid kill.

We need to keep looking for it.

The video game? That isn’t it.

CDC gets $10M to study link between guns, gaming


Do violent games cause kids to go on shooting sprees? Congress intends to find out. Photo by Flickr user agitprop/Andrew Kitzmiller.

In January, I wrote a letter to President Barack Obama about his orders to Congress to give the CDC $10 million for more video game studies. Now, it seems like the gears on that plan are rolling. Here’s what CNET reported last week:

The CDC has asked the Institute of Medicine to put together a committee that will look at the influence of video games and other media on real-life violence. The IOM is part of the congressionally chartered and federally funded National Academy of Sciences. In a statement Wednesday the CDC said:

In more than 50 years of research, no study has focused on firearm violence as a specific outcome of violence in media. As a result, a direct relationship between violence in media and real-life firearm violence has not been established and will require additional research.

Interesting. So they’re going to try to tease out the “the characteristics of firearm violence; risk factors; interventions and strategies; gun safety technology; and the influence of video games and other media?” That’s kind of a first, actually.

Reading this, I thought, “Well, good luck with that.”

And then I thought: They might as well. A study this massive has the potential to find some real connections between fantasy violence and real gun violence, or it has the potential to resolve, once and for all, that there are no such connections. Could it actually show the true indicators that lead youths to commit mass shootings? Maybe that’s going too far, but perhaps it will put another nail in the coffin of the idea that video games, even violent ones, play any significant role in the process.

Looks like we’ll be waiting 2-3 years for the findings.

Leaders: don’t waste money on violent-game studies

Dear President Obama,

This month, you said two things: First, that you asked Congress to allocate $10 million to the Centers for Disease Control to study the “the relationship between video games, media images and violence.” Second, that “We won’t be able to stop every violent act, but if there is even one thing that we can do to prevent any of these events, we have a deep obligation, all of us, to try.”

That’s why I’m writing to you today. I’m not an avid player of video games. I don’t work in the game industry. I’m a journalist, writer and mom who has spent the past several years reading and writing about the relationship between kids, violent video games, and real-life violence. So far, what I’ve learned is that there isn’t one.

Yes, there are hundreds of studies, particularly from researchers Craig Anderson, Brad Bushman, and Doug Gentile, suggesting there may be a link between playing violent video games and short-term aggressive behavior immediately after switching off the game. But they haven’t been able to show that video games _cause_ that behavior, or that post-game aggression translates into violent acts later on. Some people are amped up after playing a particularly intense game of football, too, but we haven’t spent millions of dollars researching whether it makes kids bring guns to school.

If you dig deep into each of these researchers’ studies, they say as much.

There are other studies that reveal the positive influences of these games. For example, two studies from Ohio State University researchers David Ewoldsen and John Velez showed that when kids play violent games cooperatively – as many do – they come out of the games feeling pretty good. Canadian researcher Jayne Gackenbach has shown that playing violent video games can help soldiers overcome nightmares induced by the traumas of war, an outcome that seems like it could apply to other gamers trying to make sense of our violent world overall.

In Somalia, video-game-play is on the rise, and many parents are glad, because it’s keeping their kids off the dangerous streets. That’s also true at home: University of Texas at Arlington researcher Michael Ward found that in towns with more video-game retailers, juveniles commit fewer violent crimes – because they’re too busy playing to get into trouble.

By far the best text on the benefits of violent games and aggressive play for kids is Gerard Jones’ book “Killing Monsters.” I interviewed Jones in 2011 for a Wired.com article on why violent video games are good for teens, written at the time the Supreme Court voted against a ban on the sale of these games to minors. He said:

“For the world of adolescents, [reality has] mostly gotten more stressful and bleaker,” he said, citing the dire economy, stressed-out parents, the increasing demands of public education and two lengthy wars in the Middle East. “This is not a cheerful time to be coming of age in America. The need for escape, the need for fantasies of potency, and the need for a community of peers is greater than it’s been in a long time.” He has said, in other moments, that we cannot expect teens to accept forms of entertainment that have been sanitized of the violence they know exists around them every day.

However, one of the most important sources of information on the relationship between violent video games and young players is the players themselves. As a nation we have spent far too much time studying the supposed affects of games on gamers, and almost no time asking gamers questions about why they enjoy them. If you ask, they will tell you that they love the escape, the chance to explore violent ideas safely and without hurting anybody, the opportunity to play the hero, and much more. I interviewed and surveyed dozens of young gamers for a book I wrote for parents – a book that, given our current cultural climate, I believe parents need more than ever, but unfortunately has found almost no support in the publishing world.

So far, Congress has been smart, vetoing just about every bill that proposes a study of violent video games and young players. To start spending money now on such studies would be a tremendous waste of money that could be put to more productive use, such as providing more mental-health support for violent teens and their struggling families. If Congress does wind up putting money into video-game studies, however, please make sure those studies look at the potential benefits of violent games, not just our preconceived notions of harm, which hundreds of studies have already failed to support.

Thank you.

28 percent blame games for Sandy Hook. Sort of.


A memorial to Sandy Hook. Photo by Flickr user NorthEndWaterfront.com.

With all this talk of violent video games, it’s about time someone asked the real experts — random newspaper readers — whether games cause mass shootings like Sandy Hook. Thank goodness NJ.com did.

The poll headline asks, “Do you blame video games, movies for tragedies like Newtown shooting?” But the actual poll asked something different: “Will you limit violent content for your kids?”

28.4 percent said: “Yes. I returned video games on my kids’ holiday gift list and talked to them about violence.”

A sensible 57.7 percent said: “No. There is no link between entertainment and kids behavior.”

And another 14 percent said: “I don’t think movies are to blame, but I will try anything to end violence,” which is about like saying “yes,” considering that the end result is the same: people getting rid of video games because of the shooting, even though there’s no link between the two.

People are entitled to get rid of things in their own homes they think are harmful if they want to, but it’s too bad they’re going on misguided science and gut instincts, rather than actual facts.

Don’t even get me started on the drive to destroy violent games in Connecticut. Actually, do; I’ll have more on that plan later this week.

Violent games didn’t cause Sandy Hook shooting


Did Call of Duty make Adam Lanza kill? Not likely.

I don’t know if this seems fishy to anyone else, but over the weekend, politicians and the press began speculating that violent video games must have had something to do with the Sandy Hook Elementary shooting. For example, you had Connecticut’s own senator, Joe Lieberman, saying things like, “Very often these young men have an almost hypnotic involvement in some form of violence in our entertainment culture – particularly violent video games. And then they obtain guns and become not just troubled young men but mass murderers.”

That’s not the fishy part. Well, okay it is, but it gets fishier: a few days later, the UK’s oh-so-reputable Sun unearthed a plumber who swears that shooter Adam Lanza played Call of Duty for hours every day. I don’t even know where to start.

It’s hard to imagine how a plumber could have a good window into someone’s behavior over time, unless for some reason he lived in the Lanza home. So there’s that.

Let’s say, for the sake of argument, that Lanza did play the game. Then there’s the fact that more than 55 million people play Call of Duty. Sure, Anders Breivik also played Call of Duty. I bet both Anders Breivik and Adam Lanza also ate toast, or wore pants, or saw The Sound of Music. In other words, this is a pastime so common that it can’t be linked to any particular sort of behavior. All sorts of people play Call of Duty. It has wide, massive appeal. One or two of them is potentially going to go off the deep end in ways we couldn’t have predicted. Their gaming habits aren’t relevant.

This week, the Internet has been awash with writeups arguing that video games did — or didn’t — lend a hand in the Sandy Hook shooting. I’m not going to go through them exhaustively, but you can check them out on the Backward Messages Pinterest boards. I do want to call two pieces of news and commentary to your attention.

In the first, Sen. Jay Rockefeller (D-W.Va.) has introduced a bill to study the impact of violent video games on children. What a complete waste of taxpayer money. We’ve had dozens, even hundreds of studies — and even those that suggest a correlation between violent video games and aggression a) cannot prove that games lead to actual violence, b) only rarely show any verifiable link at all, and c) can’t prove whether it’s players’ need for an aggressive outlet which draw them to the games, rather than the games leading to aggression. Visit this blog’s video-games category to see articles on many of these studies.

In the second, the Washington Post looked at video games and gun violence in 10 countries and found, basically, “that countries where video games are popular also tend to be some of the world’s safest (probably because these countries are stable and developed, not because they have video games). And we also have learned, once again, that America’s rate of firearm-related homicides is extremely high for the developed world.”

A decade ago, studies showed that mass shooters tended to be kids who played video games less than average. Now that pretty much everyone plays a video game now and then — much more so than 10 or 20 years ago — it’s probably safe to say that these killers do play. But again, gaming is now so common that it’s akin to watching television or blockbuster movies; you just can’t say that engaging in it will lead to any specific outcome. And you can’t use one violent act to justify taking games away from the millions and millions of people who enjoy them safely.

In fact, it’s likely that Lanza enjoyed them safely, too. It’s likely that his gaming had nothing to do with his crime. It’s also likely that something in his mind went awry, and the fact that his mom trained him to shoot gunsnot the fact that he’d played a shooter video game — gave him the means to act on his brain’s break with reality.