Testing, Testing

Are Americans becoming more skeptical of scientific inquiry? Some are, according to the pundits. See, for example, Chystia Freeland’s article in the New York Times, “A Deep Faith in What’s Been Proved,” and Paul Krugman’s article in the same paper, “Republicans Against Science.”

Although there does appear to be a growing skepticism about the value of science to address problems such as global warming, there has long been a neglect of social science when it comes to evaluating programs designed to change people’s behavior in beneficial ways, such as those that try to get kids to avoid drugs and alcohol, teach parenting skills, and prevent adolescent behavior problems. Myriad programs that receive federal and state funding have never been adequately tested to see if they work. When they are tested, they are often found to be ineffective or even to do harm.

Consider the D.A.R.E. drug abuse resistance program, which is used in 75% of school districts in the United States and in more than 40 countries. D.A.R.E. lists among its sponsors the U. S. Drug Enforcement Administration, the U. S. Food and Drug Administration, the U. S. Department of State, all five branches of the U. S. military, and the White House Office of National Drug Control Policy. President Obama, like his predecessors, designated a day in April as National D.A.R.E. Day to commemorate the program.

There is only one problem: D.A.R.E. doesn’t work. Studies have repeatedly shown that kids who take part in the program are no less likely to smoke, drink, or abuse drugs than kids who do not. To their credit, D.A.R.E. officials revamped the program in 2009, and maybe this new version will do some good (it is currently being tested). But doesn’t it seem like putting the cart before the horse to sink millions of dollars into a program and implement it in 75% of our schools before we know whether it works?

Another example is Healthy Families America, which is a home visitation program designed to prevent child abuse in at-risk families. According to the Healthy Families America website, 54% of its funding came from federals sources, 38% from state governments, and 8% from local sources. But sadly, like the D.A.R.E. drug program, there is little evidence that it works. The program was been implement throughout the United States before it was adequately tested, and like D.A.R.E., it now has an institutional momentum that is hard to stop, despite the lack of hard evidence that it does any good.

I can hear my conservative friends now: “This just goes to show that we should slash funding further instead of wasting government money on failed programs.” But we can’t let problems like teen alcoholism and child abuse go unchecked, the human toll would be intolerable (not to mention the financial toll in terms of health care dollars). I have another suggestion: Let’s invest those funds in finding out which programs work and supporting the ones that do.

Programs such as D.A.R.E. and Healthy Families America are based on common-sense notions of what will work that turn out to be wrong. Meanwhile, there is an abundance of basic research in social psychology that examines the underpinnings of human cognition, emotion, and motivation, research that has led to successful interventions to reduce alcohol and drug use, prevent child abuse, narrow the achievement gap in education, and help people in many other ways. We know that these programs work because they have been put to the test by behavioral scientists in well-controlled experimental studies, in which participants were randomly assigned to get the intervention and or to a control condition that did not.

Many of these successful interventions, and the basic research that inspired them, were funded by federal research grants. And yet, funding for behavioral research is under attack. Members of Congress have tried to slash the budget of the National Science Foundation, and funding for behavioral research at the National Institutes of Health has all but dried up. Unless we invest in basic and applied research to find out what works to help kids stay away from drugs, reduce child abuse, and address a host of other societal problems, we will continue to waste tax payers’ money on programs that don’t work, and fail to discover new ways of addressing our most pressing problems.

[Originally posted on PsychologyToday.com]

Advertisements

Leave a comment

Filed under Uncategorized

How Can We Help When Johnny Can’t Read?

What is the best way to help children who have problems with reading comprehension? There are dozens of reading programs out there and you would think we would know the answer to this question by now. But the shocking thing is that until recently we didn’t know much at all about which ones really work. Educational researchers have been reluctant to conduct experiments in which children are randomly assigned to condition, to test the effectiveness of educational programs.  (Thomas Cook, for example, published an article with the title, “Why Have Educational Evaluators Chosen Not to Do Randomized Experiments?”)  Experiments are hard to do, but they are the gold standard in evaluating whether a program works. Fortunately, educational researchers are beginning to use this approach more.

Researchers in England, for example, studied fourth graders who had difficulties with reading comprehension. They randomly assigned the kids to receive one of two interventions over the course of 20 weeks. One program focused on text comprehension, teaching the kids strategies such as how to clarify unknown words and how to figure out the global meaning of a text. The other focused on spoken language; the tutor engaged the child in a dialogue that dealt with vocabulary, figurative language, and narratives. Other kids were randomly assigned to a control group that did not take part in either program.

Before reading any further, which of the two programs do you think worked the best?  If you are like me, you might guess that the first (text comprehension) program would be most effective, because after all, it attacks the core of the problem. But this is why we need to do experiments instead of relying on common sense (at least mine): The second (oral comprehension) program worked the best, leading to the biggest gains in reading comprehension, even 11 months after the program ended. Now that we know what works, this program can be widely applied.

Some readers might be a little queasy about including a control group of kids in this study who didn’t get any help with their reading difficulties. Is it ethical to keep these kids out of programs that might help them?  The tradeoff is that if don’t include control groups, we will never know which programs actually work, and risk exposing thousands of kids to interventions that are ineffective or worse, do more harm than good. This has in fact happened, not only with educational interventions, but programs designed to prevent alcohol and drug abuse, prevent teenage pregnancy, reduce violence, and so on (I discussed several such examples in my book Redirect). Note that this ethical dilemma is no different than the one faced by medical researchers who conduct experimental trials of the efficacy of new drugs.  Some patients are randomly assigned to a placebo group that gets no treatment, so that the researchers can find out whether the new drug really works. Why should we have different standards for social, psychological, and educational interventions?

(The study on reading comprehension was by Paula Clarke and colleagues, published in the August, 2010 issue of Psychological Science).

[Originally posted on PsychologyToday.com]

Leave a comment

Filed under Uncategorized

When Cameron Diaz Lights Up, Do Teenagers Start Smoking?

When a movie star lights up a cigarette in the movies, does it really make teenagers more likely to smoke?  And if so, what can we do about it?

This topic is timely because the Centers for Disease Control and Prevention just reported that smoking in movies is declining. Depictions of smoking in top-grossing films peaked in 2005, but since then have declined sharply.

Why does this matter?  Does the fact that Cameron Diaz smokes a cigarette in the movie Bad Teacher, or that Tom Hanks smokes a pipe in the movie Larry Crowne, really make viewers more likely to smoke?

Research suggests that it does.  The more the kids watch movies in which people smoke, the more likely they are to begin smoking themselves.  This is a correlational finding, of course; for all we know, kids who are likely to smoke in the first place are more likely to go to see movies in which the characters smoke.  But there is good research suggesting a cause-and-effect relationship.

In one study, for example, kids watched the movie Reality Bites, in which Winona Rider and Ethan Hawke smoke liberally.  Some of the kids were randomly assigned to see the original movie, whereas others were randomly assigned to see the movie with the smoking scenes edited out.  The results?  Those who saw the movie with the smoking scenes admired smokers more and indicated more of an intention to smoke than did the kids who saw the non-smoking version.  Teenagers, like everyone else, tell themselves stories about the world, and these stories are shaped by seeing what their peers do–and by what celebrities do.  Seeing cool people in the movies light up shapes teens’ stories in harmful ways.

There is a way around this.  There was another condition in the Reality Bites study, in which the researchers showed the kids an anti-smoking advertisement before they watched the movie.  In the anti-smoking ad, an attractive high school student starts smoking and as a result transforms into an unpopular kid.  The teens who watched this ad, and then saw Reality Bites, did not become more favorable toward smoking, even when they saw the version with the smoking scenes.  Fighting fire with fire (so to speak) worked:  Showing kids how smoking can make you unpopular acted as a prophylactic, making them immune to the Hollywood depiction of smoking.

Maybe Hollywood should take this to heart and depict smokers as people with wrinkled faces, hacking coughs, and yellowing teeth, abandoned by friends and family.  There isn’t much chance of this happening, of course, but parents could be proactive and try to shape their kids’ narratives about smoking early on, before their kids see it glamorized in the movies.

[Originally posted on PsychologyToday.com]

Leave a comment

Filed under Uncategorized

The U.S. Army tries Positive Psychology . . . but does it work?

Monday’s Washington Post had an article about a new U. S. Army program called Comprehensive Soldier Fitness, which is designed to “to make soldiers more psychologically ‘resilient’ amid the pressures of combat, repeated deployments, and family and financial crises.”

Martin Seligman, the father of Positive Psychology, helped design the program, which attempts to instill optimism and the ability to handle stress.  According to the Post, the program has come under fire by other psychologists, who have had two main criticisms:  First, that we don’t know whether the program works, and it is possible that it may do more harm than good.  Second, that it is a giant research project that violates established ethical guidelines, for example, by not obtaining the consent of the soldiers to participate.

I worry more about the first point more than the second.  In fact, I think that the Army and Seligman haven’t gone far enough in conducting rigorous research on whether the program works.  As far as I know, there is no control group that doesn’t get the intervention, making it impossible to tell for sure what its effects will be.  As I detail in my new book, Redirect:  The Surprising New Science of Psychological Change, many interventions designed to help people go untested, and when researchers do get around to testing them, the interventions are often found to be ineffective or to do more harm than good (think of the D.A.R.E. anti-drug program).  In the Army’s case, they could have conducted an experimental test in which volunteers were randomly assigned to take part in the program or not, to find out exactly what benefits (if any) the program has, while following established ethical guidelines for research.  Surely this would be better than subjecting a million soldiers to an untested program.

3 Comments

Filed under Uncategorized