Category Archives: Uncategorized

Is There a Crisis of False Negatives in Psychology?


The replication movement in psychology has had many positive effects, such as the discussion of how to avoid p-hacking and the emphasis on increased transparency, including posting data, detailed methods sections, and the results of unpublished studies on publicly available web sites. These practices will undoubtedly improve our science.

But something is seriously out of whack. Despite its benefits, the replication movement has had a polarizing effect. Whereas most of the researchers involved in the replication movement have the best interests of the field at heart and are well-intentioned, some seem bent on disproving other researchers’ results by failing to replicate. Whereas some researchers have embraced the movement and taken part in it, others are deeply suspicious and fear that ill-intentioned replicators will target them, fail to replicate their findings, and damage their reputations.

Why are many people afraid? One reason, I believe, is that there has been more emphasis on false positives than false negatives. When an effect fails to replicate, the spotlight of suspicion shines on the original study and the authors who conducted it. “False Positive Alert” flashes like a neon sign as the buzz spreads in the Tweetosphere and Blogworld. But why should we assume that a failure to replicate is “truer” than the original study? Shouldn’t the spotlight shine as brightly on the replicators, with a close examination of their research practices, in case they have obtained a false negative?

There are many reasons why a false negative could occur, including these:

  • Replications might be conducted by researchers who are inexperienced or lack expertise, either in general or in the particular area they are trying to replicate.
  • As has been well documented, researchers are human and can act in ways that make them more likely to confirm a hypothesis, resulting in p-hacking. But replicators are human too, and if their hypothesis is that an effect will not replicate, they too can act in ways that increase the likelihood of obtaining that outcome—a practice we might call p-squashing. For example, it would be relatively easy to take an independent variable that had a significant effect in the laboratory, translate it into an on-line study that delivers the manipulation in a much weaker fashion, and then run hundreds of participants, resulting in a null effect. Adding such a study to a meta-analysis could cancel out positive findings from several smaller studies because of its very large sample size, resulting in meta p-squashing.
  • As others have noted (e.g., Stroebe & Strack, 2013), a direct replication could fail because it was conducted in a different context or with a different population, and as a result did not manipulate the psychological construct in the same manner as did the original study.

Do I have evidence that many of the studies that have been done as part of the current replication movement have been plagued by the above problems? Well, not much, though I suggest that the evidence is equally weak that false positives are rampant. One might even argue that there is just as much evidence that we have a crisis of false negatives as we do a crisis of false positives.

This is important because both kinds of errors can have serious consequences. As many in the replication movement have argued, false positives can be costly to a field’s credibility and to subsequent researchers who spend valuable research time going down a blind alley. But false negatives can also be damaging, both to the reputation of the original researcher and the progression of science (see Fiedler, Kutzner, and Krueger, 2012, for an excellent discussion of this issue). Consequently, neither those who attempt replications nor the authors of original studies should stake out the moral high ground in this debate. We should all scrutinize replications with the same critical eye as we do original studies and not assume that a failure to duplicate a result means that the original finding was false. For example, if replications are submitted to a journal, they should undergo the same rigorous review process as any other submission.

There is another unintended effect of the replication movement, namely that it places too much emphasis on duplication and not enough on discovering new and interesting things about human behavior, which is, after all, why most of us got into the field in the first place. As noted by Jim Coan, the field has become preoccupied with prevention and error detection—negative psychology—at the expense of exploration and discovery. The biggest scientific advances are usually made by researchers who pursue unorthodox ideas, invent new methods, and take chances. Almost by definition, researchers who adopt this approach will produce findings that are less replicable than ones by researchers who conduct small extensions of established methodologies, at least at first, because the moderator variables and causal mechanisms of novel phenomena are not as well understood. I fear that in the current atmosphere, many researchers will gravitate to safe, easily replicable projects and away from novel, creative ones that may not be easily replicable at first but could lead to revolutionary advances.

For those interested in conducting replications, there might be a happy medium. For example, researchers all over the world have conducted replications of the same phenomenon as part of the “Many Labs” project. I suggest that we would learn more from this endeavor with a small twist: Ask all participating labs to add an interesting moderator variable of their choice to the design, with random assignment, in addition to performing a direct replication. This would nudge replicators into thinking deeply about the phenomenon they are trying to replicate and to make predictions about the underlying psychological processes, possibly leading to substantial advances in our understanding of the phenomenon under study—that is, to discovery as well as duplication.

In any polarized debate, common ground becomes obscured. It is thus worth remembering that all scientists agree on two things: We want our methods to be as sound as possible and we value novel, creative, groundbreaking findings. It would be unfortunate if the emphasis on one came at the expense of the other.

(Note: This post benefited greatly from comments by Jerry Clore, Dan Gilbert, and Brian Nosek—but by thanking them I do not mean to imply in the least that they agree with anything I have said.)



Filed under Uncategorized

Experiment Now

Should we do experiments to inform social policy? On the one side of this question are pundits who argue that psychology isn’t really a science and has little to contribute to society. Needless to say I think this view is wrong, as I argued in a recent op-ed piece in the Los Angeles Times. Others, from different quarters, have joined me in arguing for the importance of behavioral science.

The economist Richard Thaler, in a New York Times article, points to advances in Great Britain. David Cameron, the British prime minister, created a Behavioral Insights Team that is headed by a social psychologist (Thaler is an unpaid adviser to the team). The group has had some notable successes, such as finding ways to get more small business owners to pay their taxes on time and getting home owners to take advantage of a government energy savings program. How did they do it? In part by applying lessons from basic research in psychology to policy questions. Most importantly, though, they did their own experiments to find out what works the best. To increase tax collections, for example, they wrote different kinds of appeals, sent them to 140,000 business owners, and sat back and observed which appeals elicited the most payments (the particular letter a business owner got was randomly assigned, of course). As it happened, the appeal that worked the best was one that communicated social norms, telling business owners that a high percentage of businesses in their area paid their taxes on time. Government officials estimate that if this type of appeal were used throughout the country, annual tax revenues would increase by £30 million.

Jim Manzi, in his book Uncontrolled, is also a strong advocate of doing experiments. Manzi is a business CEO who stumbled on the idea that when it comes to understanding and predicting consumer behavior, all the analysts in the world, and all the fancy software, are not as good as an old-fashioned experiment. Many businesses have caught on– Capital One, Google, and Harrah’s Entertainment, for example, conduct thousands of experiments in which consumers are randomly assigned to receive one kind of appeal or another to see what works. Manzi quotes Gary Loveman, the CEO of Harrah’s, as saying that there are three things that will get you fired at his company: stealing, harassing someone, or failing to include a control group.

Manzi argues that we should learn from the business world and conduct experiments to see what works in all segments of society, including economics, education, politics, and criminology. He is well-versed in the history and philosophy of science and makes a persuasive case.

I hope that policy makers across the political spectrum take notice and come to the same conclusion as Thaler and Manzi. It is in everyone’s interest to find out what works the best to address social problems.

(Originally Posted on

Leave a comment

Filed under Uncategorized

Viva Behavioral Science

Gary Gutting, a professor of philosophy at the University of Notre Dame, questioned the value of the social sciences, in a piece on the Opinionator blog of the New York Times (“How Reliable are the Social Sciences?”).  He has been criticized, and rightly so, for his broadsided attack on entire disciplines.  See, for example, Jamil Zaki’s nice rebuttal.

The unfortunate thing about Gutting’s piece is that he actually made some reasonable points about how to study human behavior.  But, he doesn’t seem to recognize that a great deal of research is doing exactly what he suggests.  

What Gutting really objects to, it turns out, is the failure to use the experimental method to study people.  The reason that much social science research fails to produce precise predictions, he argues, is because “such predictions almost always require randomized controlled experiments, which are seldom possible when people are involved.”

He is sadly mistaken on this last point, having missing entire disciplines (such as mine, social psychology) that use the experimental method to study human behavior.  Nor does he mention the vast knowledge that has accrued through experimentation, including novel discoveries that have reduced human suffering.  Just a few examples: 

  • Various forms of psychotherapy (such as cognitive behavioral therapy) are effective at treating many psychological disorders, which we know from randomized clinical trials
  • Recent experimental work in schools shows that simple social psychological interventions can reduce the achievement gap by 40%
  • Getting high school students to do community service reduces teenage pregnancies and improves academic performance
  • A simple psychological intervention has been found to dramatically reduce child abuse

Each of these findings is based on research that used the experimental method, with random assignment to the “treatment” or control conditions.  I discuss them, and several other examples, in, Redirect:The Surprising New Science of Psychological Change.

One place in which I wholeheartedly agree with Gutting is that we need to “find ways of injecting more experimental data into government decisions.”  As he notes, social and educational policies have often been based on the flimsiest of evidence.  But this is not due to a grand failure of the social sciences, but rather to a failure by policy makers (and yes, some social scientists) to appreciate the value of a good experiment.  

But this is changing, as evidenced by the use of the experimental method to debunk some popular programs.  Examples of debunked programs are Critical Incident Stress Debriefing, an intervention used to prevent post-traumatic stress disorders in people who have witnessed horrific events; the D.A.R.E. anti-drug program, and Scared Straight programs designed to prevent at-risk teens from criminal behavior.  All three of these interventions have been shown, with solid experimental studies, to be ineffective or, in some cases, to increase the very behaviors they are trying to prevent.  And as a result, these programs have become less popular or have changed their methods.

The same is true of educational programs.  Gutting is right that too often, they have not been tested rigorously.  He is wrong that they can’t be tested with the experimental method.  They can be, and increasingly, they are.  See, for example, an experimental test of a teacher training program that successfully improved teacher quality and student performance, that Science Magazine, the premier journal in all of the sciences (hard or soft), saw fit to publish.    

Clearly, Gutting is not familiar with vast areas of psychological and educational research that do precisely what he suggests.  Too bad he didn’t read more widely in the disciplines he dismissed.

(Originally posted on

Leave a comment

Filed under Uncategorized

Affirm Yourself

When it comes to behavior change, social psychology rocks. Using basic principles discovered in the lab, researchers are designing all sorts of effective real-world interventions. Two new examples were published recently, one in which researchers succeeded in making people more open to discovering their medical risk factors, and another in which they helped people lose weight.

Both of these studies are examples of what I call “story-editing,” in which people are induced to revise the way they think about themselves in the social world. Changing people’s self-stories can have cascading, long-term effects, by bumping them out of a self-defeating cycle of thinking into a more positive, self-enhancing cycle. The story-editing approach has had remarkable success, helping to reduce child abuse, close the achievement gap in middle schools, reduce racial prejudice, and increase personal happiness.

The two new examples used a form of story editing called self-affirmation, whereby people are asked to think about an area of their lives that they value. Typically, people are given a list of topics, such as religion, athletics, musical ability, and relationships with other people, and asked to pick the one that is most important to them. Then, they are asked to write about a time when that area of their life was particularly important and why.  That’s all there is to it–a brief writing exercise that takes just a few minutes.

As mundane as this writing exercise sounds, completing it can have surprisingly powerful effects. It works by helping people deal with threats in their lives, such as the possibility that they have a serious physical ailment or are failing to reach an important goal (e.g., losing weight). People often deal with such threats by finding some way of deflecting them psychologically.  “I’m sure I’m fine,” we think, “no need to go to the doctor.”  Or, “So what if my jeans don’t fit, I haven’t gained THAT much weight.”  We rationalize in order to make ourselves feel better.

The self-affirmation exercise helps people deal with threats in a more effective way, by accepting them as real and taking steps to deal with them directly.  Consider a recent study by Jennifer Howell and James Shepperd.  Participants learned about a new disease that can have serious health consequences, and then were given an opportunity to find out what their lifetime risk of getting the disease was. Less than half of the participants (45%) chose to do so, adopting an avoidance strategy. After all, by avoiding the information, they could pretend that everything was just fine. But if people first completed the self-affirmation writing exercise, they were much more open to getting the feedback: 84% of them chose to find out what their lifetime risk of getting the disease was. Several other studies have found that self-affirmation promotes healthy behaviors.

Or, consider this example: In a study by Christine Logel and Geoffrey Cohen, college women were randomly assigned to do the self-affirmation exercise or to a control group that did not. All the women were weighed, sent home, and asked to return two-and-a-half months later. At the second session the women were weighed again, to see whether they had lost or gained weight since the first session.  As it happened, the women in the control condition had gained an average of 2.8 pounds, whereas the women in the self-affirmation condition had lost an average of 3.4 pounds.

All of the women had expressed dissatisfaction with their weight, suggesting that weight gain was an area of personal threat to them. Remarkably, lowering the heat on that threat just a little, by having women write about other aspects of their lives that were important to them, made them more successful in meeting their weight loss goals.

Self-affirmation exercises probably sound magically, or maybe even a little silly, like Al Franken’s character Stuart Smalley, who looks in the mirror and says to himself, “I’m good enough, I’m smart enough, and doggone it, people like me!”  And it’s true that the exact way in which they work is not entirely clear (lots of researchers are attempting to answer this question).  But the fact is that self-affirmation exercises have been proven to be effective in a number of domains. Whatever the exact mechanisms, they help people deal more effectively with challenges in their lives.

(Originally Posted on

Leave a comment

Filed under Uncategorized

We Are What We Do

Note: Each year, the website poses a question for scientists to address. The 2012 question is,  “What is Your Favorite Deep, Elegant, or Beautiful Explanation?” My answer is below; for the others, check out the Edge web site.

We Are What We Do

My favorite is the idea that people become what they do. This explanation of how people acquire attitudes and traits dates back to the philosopher Gilbert Ryle, but was formalized by the social psychologist Daryl Bem in his self-perception theory. People draw inferences about who they are, Bem suggested, by observing their own behavior.

Self-perception theory turns common wisdom on its head. People act the way they do because of their personality traits and attitudes, right? They return a lost wallet because they are honest, recycle their trash because they care about the environment, and pay $5 for a caramel brulée latte because they like expensive coffee drinks. While it is true that behavior emanates from people’s inner dispositions, Bem’s insight was to suggest that the reverse also holds. If we return a lost wallet, there is an upward tick on our honesty meter. After we drag the recycling bin to the curb, we infer that we really care about the environment. And after purchasing the latte, we assume that we are coffee connoisseurs.

Hundreds of experiments have confirmed the theory and shown when this self-inference process is most likely to operate (e.g., when people believe they freely chose to behave the way they did, and when they weren’t sure at the outset how they felt).

Self-perception theory is elegant in its simplicity. But it is also quite deep, with important implications for the nature of the human mind. Two other powerful ideas follow from it. The first is that we are strangers to ourselves. After all, if we knew our own minds, why would we need to guess what our preferences are from our behavior? If our minds were an open book, we would know exactly how honest we are and how much we like lattes. Instead, we often need to look to our behavior to figure out who we are. Self-perception theory thus anticipated the revolution in psychology in the study of human consciousness, a revolution that revealed the limits of introspection.

But it turns out that we don’t just use our behavior to reveal our dispositions—we infer dispositions that weren’t there before. Often, our behavior is shaped by subtle pressures around us, but we fail to recognize those pressures. As a result, we mistakenly believe that our behavior emanated from some inner disposition. Perhaps we aren’t particularly trustworthy and instead returned the wallet in order to impress the people around us. But, failing to realize that, we infer that we are squeaky clean honest. Maybe we recycle because the city has made it easy to do so (by giving us a bin and picking it up every Tuesday) and our spouse and neighbors would disapprove if we didn’t. Instead of recognizing those reasons, though, we assume that we should be nominated for the Green Neighbor of the Month Award. Countless studies have shown that people are quite susceptible to social influence, but rarely recognize the full extent of it, thereby misattributing their compliance to their true wishes and desires—the well-known fundamental attribution error.

Like all good psychological explanations, self-perception theory has practical uses. It is implicit in several versions of psychotherapy, in which clients are encouraged to change their behavior first, with the assumption that changes in their inner dispositions will follow. It has been used to prevent teenage pregnancies, by getting teens to do community service. The volunteer work triggers a change in their self-image, making them feel more a part of their community and less inclined to engage in risky behaviors. In short, we should all heed Kurt Vonnegut’s advice: “We are what we pretend to be, so we must be careful about what we pretend to be.”

Leave a comment

Filed under Uncategorized

Penn State, Abuse Victims, and the Power of Narrative

The scandal at Penn State (accusations of sexual predation by a coach, and alleged failures by many coaches and officials to report this abuse to the police) is an important reminder of the power of narratives. The way in which we frame events, and the stories we tell ourselves about them, determine how we feel, think, and act. Typically we think we are viewing the world objectively—“seeing it like it is”–without appreciating how much we are interpreting and spinning what we are seeing. That is why it can be so jarring to find out that our interpretation was wrong.

For many Penn State students and alumni, Joe Paterno was a godlike figure, a paragon of integrity and an icon of all that should be admired about college athletics. Then, overnight, that narrative crumbled, when Joe Paterno was fired for his alleged complicity in the scandal. Michael Weinreb, a sportswriter in New York, is a Penn State alum who grew up in State College. He was interviewed recently by a reporter from the radio show This American Life, to get his perspective on the scandal. What is striking is his sense of disbelief, an inability to abandon one narrative in favor of a very different one. “It doesn’t make any sense,” he said. It just doesn’t compute. . . it just goes against everything that you thought you understood, you know? . . . . I keep waking up every morning thinking that this is. . . that it’s a dream . . . it still doesn’t seem real.”

But obviously the people whose narratives we should be most concerned about are those of the victims. What does a 10-year-old think when a beloved mentor insists on having sex with him in a shower? It is easy to imagine that the world shifts on its axis for the victims and that he, too, wakes up every morning thinking (or wishing) that it was just a dream.

But things might be a little more complicated here. We shouldn’t assume that all children react in the same way or that we know how they will interpret abuse. Children have a different understanding of sex than adults, and research shows that at the time the abuse occurs, some children experience confusion more than an axis-changing view of who they are or their own sexuality—at least at first. Susan Clancy, in her book The Trauma Myth, acknowledges that sexual abuse has damaging, long-term effects on the victims. But she has a different view than many experts of how this happens. She interviewed more than 200 adults who were victims of child abuse, and found that “very few victims reported any fear, shock, force, or violence at the time the abuse occurred.” One reason for this is that most child abusers are not strangers who kidnap a child and forcibly rape him or her. Rather, 90% of child abusers are trusted family members or friends who establish loving relationships with victims. When this loving relationship becomes sexual, many children don’t know that they are being abused or believe that something horrible is happening—in large part because of the manipulative cleverness of their abusers, who have fostered a loving, trusting relationship with their victims. One of Clancy’s participants, for example, reported that when she was nine, an adult male family member induced her to have sexual contact several times, such as performing fellatio on him and penetrating her with his fingers. But, she reported, she did not find this particularly traumatic at the time: “I did not understand what sex was,” she said. “I did not understand what he was asking me to do. I did not know why he would want me to do that . . . I did not know anything, at all, about sexual matters. I supposed I had led a sheltered life. He said it was normal and I chose to believe him.”

Another one of Clancy’s participants reported that when he was at Bible camp as a young boy, a counselor accompanied him to the bathroom and explained that when boys urinate they often don’t clean themselves thoroughly, and that the best hygiene is for another person to clean the penis by taking it into his mouth. Several times that summer, the counselor “cleaned” the boy’s penis, and insisted that the boy clean his. As absurd as this sounds to us, it was a narrative that made sense to a nine-year-old boy; after all, it was a trusted, beloved counselor said that’s the way it was, so that’s the way it must be. When asked how he felt during these episodes, the victim reported that he wasn’t at all scared or traumatized. “Confused is a better word,” he said. This isn’t to say that victims find the abuse to be normal. Most knew that there was something wrong about it, often because their abuser told them not to tell anyone what they were doing. The point is that for most of the victims, the abuse was not horrible or traumatic at the time it occurred. Children have quite malleable narratives about sex, and a clever, predatory adult can shape them to his or her advantage.

As you might imagine, Clancy’s work triggered a firestorm of criticism. A legitimate concern is how representative her sample was. They were people who answered newspaper ads seeking research participants who had been abused, and the people who answered those ads might not be at all representative of all victims of child abuse. But some of the criticism was unfair, such as the accusation that she was minimizing the traumatic effects of abuse. Clancy agrees that child sexual abuse is a terrible thing with serious long-term negative consequences for many victims. Indeed, most of the victims she studied exhibited a host of psychological problems, and other studies confirm that children who were sexually abused are more likely than others to have psychological problems in adulthood. Further, no one denies that some children do experience abuse as traumatic at the time it occurs, especially when it involves violence. Nonetheless, many abuse victims seem to fit the pattern Clancy describes.

Thus the question is not whether child sexual abuse damages its victims, but rather how it does so. We have to square the fact that fact that for some victims, the abuse doesn’t seem very traumatic at the time it is happening with the fact that it can cause long-term psychological problems. Clancy’s claim is that the trouble comes later, when kids figure out what had happened to them and their narrative changes. Prior to this point, many victims believed that the perpetrator was a loving man or woman who had their best interests at heart. Now they realize that they were betrayed by a twisted person whose main goal was personal gratification. Good people are suddenly bad people, and that’s quite a narrative shift, especially when the perpetrator was your father, or a beloved uncle, or a famous coach at Penn State University. As one of Clancy’s victims put it, “My whole relationship, my memory, my past really shifted, from just ‘those things he did at night’ to complete betrayal. It was heartbreaking, really . . . I couldn’t stop crying. I was stupid enough to think he cared about me; I thought he was wonderful, a good person.”

In other words, for many victims, the world shifts on its axis not when the abuse occurs—because of the clever manipulation on the part of their abuser—but later, when they are old enough to figure out what happened to them. And, it is this later sense of betrayal, the belief that the world is not the safe, loving place that they thought it was, and that loving people can’t be trusted, that is so devastating. This is especially the case because of the natural tendency for many victims to blame themselves. Because abusers are so clever at making the child feel like a loving and willing partner, it is only natural for victims to later think they were somehow complicit in the abuse. “I was thinking that I had something to do with it; maybe it was my fault,” said one of Clancy’s participants. “I have been in therapy for this for a long time and I is still difficult for me to come to terms with what happened and to accept that it is not my fault,” said another. “I am filled with self-loathing that I allowed it, that I was pathetic enough to think that was love,” said a third.

In other words, many victims settle on the corrosive narrative that the abuse was their fault or that they somehow encouraged it. One reason for this is that they view their past through the lens of their current understanding of abuse and sexuality. “How could I have believed that lame story about hygiene and urination?” they probably think, forgetting how naïve and confused they were at the time. This self-blame fuels the shame and humiliation they feel, which leads them to hide the fact that they were abused, which makes them less likely to seek help (or accuse their abuser).

When Susan Clancy asked her participants what would make them feel better, one of the most common responses were, “I would like to know that this happened to other people” and, “I would like to know that it was not my fault.” If anything good has come of the Penn State scandal, it may be that the publicity has helped victims revise their narratives in this way. According to one report, the phones at sex abuse hotlines have been ringing off the hook since the Penn State story hit the news (and the scandal at Syracuse University that followed). Many of the callers are males reporting that were abused by an older man, who now, it seems, feel more comfortable coming forward, perhaps blaming themselves less than they did before.

Let us hope that they are coming to realize they were the victims of clever predators, that the abuse was not their fault, and that it does not define who they are. After all, of all the narratives that need changing from this scandal, it is the victim’s we should be concerned about the most.

Note: The quotes from Clancy’s book The Trauma Myth (2009, Basic Books) are from pages 22, 29, 34, 124, 129, 132-133, 136, and 145

(Originally posted on

Leave a comment

Filed under Uncategorized

Orphan Innovations

If someone has invented a successful, innovative, cost-effective social program, doesn’t it seem likely that it would spread quickly to other communities? Susan Evans and Peter Clarke have written a fascinating article detailing why many programs become “orphan innovations” that no one else adopts.

The authors describe a program started by a retired produce wholesaler in Los Angeles, who convinced distributers to donate slightly spoiled produce to food banks. Before long, poor families were receiving fresh fruits and vegetables that would have been dumped in landfills. Evans and Clarke took it upon themselves to make sure that this program was adopted in other cities, but ran into many roadblocks, such as skepticism from overworked local officials that the program would work. Eventually, through sheer determination, they succeeded: the program spread to dozens of communities. But it took 20 years of hard work, creativity, cajoling, and financial support.

Their conclusion? A social program cannot simply be transferred from one locale to another. Instead, it has to be customized at each new location. Unlike a fast food chain, that plops a carbon copy of a restaurant down in every community in American, social programs have to be adapted to the particular staff, clients, and ecology of each setting.

There are valuable lessons to be learned here for those of us interested in social psychological interventions that improve human welfare. There is a growing movement to translate social psychological theory into interventions that help people in the real world, including ones that help people recover from traumatic events, prevent child abuse, reduce adolescent behavior problems, and close the achievement gap in education (as I chronicle in my book Redirect). Critically, these interventions are being tested with well-controlled experiments, to see if they work. This is a huge advance over relying on common sense, which has led to the wide-spread adoption of programs that don’t work or do harm (see my earlier post, Testing, Testing).

But Evans and Clarke make a good point that we can’t stop there. Once we have developed a new intervention that works at one locale, it can take a great deal of effort, persuasion, and money to ensure that it spreads to other locations, instead of becoming an orphan innovation.

Although I agree with this important point, there is one thing about Evans and Clarke’s argument that makes me nervous. If programs can only spread through customization, how can we be sure that the new hybrids capture the essence of the one on which they were based? It is relatively easy to demonstrate that food banks are receiving free donations of fresh produce, but harder to show that a social psychological intervention is making people happier or reducing teen pregnancy. The only way to do so is to conduct experimental trials of the programs in each locale as they are implemented and customized. By so doing we can see if they are working, and may even discover ways of improving upon the original program, thereby spawning many effective sibling interventions.

[Originally posted on]

Leave a comment

Filed under Uncategorized