Nonprofit Innovators Can Learn From Failure as Well as Success

In recent months, sparked in no small part by the creation of the Obama administration’s Social Innovation Fund, much of the talk in the nonprofit world has focused on how to spread ideas that work.

In general, that is a positive development. But even as nonprofit leaders and government officials pinpoint the most effective programs in education, child welfare, and other fields, it is worth pausing for a moment to consider the reality that many innovations do not end up becoming the best ideas to put into widespread practice.

Indeed, after looking at a range of failed criminal-justice efforts—programs in which a bunch of smart, dedicated people got together to try something difficult—we have learned a lot about how to evaluate the effectiveness of an investment. Among the key lessons:

Failure is in the eye of the beholder. Few programs are utter failures—or absolute successes. Policymakers often have a pass-fail approach to evaluating social programs. The question they tend to ask is a fairly basic one: “Does this program work or not?” The truth is almost always more complicated.

We saw this most powerfully when we looked at Drug Abuse Resistance Education, or DARE, the program that sends police officers into schools to educate young people about the dangers of drugs. Unfortunately, numerous studies have documented that DARE has no impact on teenage substance abuse. Despite the negative findings, DARE is still operating in something like three out of four school districts in this country.

Why is this? We found that many school officials were well versed in the critique of DARE, and yet they still decided to maintain the program. In some cases, this was due to political pressure. But in many cases, local officials cited a range of other benefits from DARE, particularly improved communication with local police, as a reason to keep the program.

The DARE story tells us that it isn’t always easy to offer simple, thumbs-up or thumbs-down judgments about social programs. Another example of this truth comes from the work of our organization, the Center for Court Innovation.

A few years ago, the Center for Court Innovation helped to create an experimental “re-entry court” that provides parolees returning to New York’s Harlem neighborhood with enhanced services during the first six months of their parole term. A recent evaluation of the program found that the re-entry court had reduced recidivism—Harlem participants were less likely than a comparison group to be reconvicted. That’s the good news.

The bad news is that because of what is known as a “supervision effect,” which suggests that increased scrutiny of offenders will lead to the detection of additional misbehavior, more Harlem participants were going back to prison for technical violations of parole: missing an appointment, breaking curfew, flunking a drug test.

So is the Harlem program a success or a failure? The bottom line is that research doesn’t always provide easy answers.

Things fall apart. Just because something works once doesn’t mean that it will work forever.

The best example of this phenomenon is Operation Ceasefire, arguably one of the most important criminal-justice efforts of the past generation. The product of a collaboration among local criminal-justice agencies, street outreach workers, and scholars from Harvard University, Ceasefire has been credited with significantly reducing gang violence in Boston in the 1990s.

The only problem is that after experiencing success on a scale that few programs ever achieve, Ceasefire fell apart.

What went wrong? The reasons are complicated, but two stand out. The first is the challenge of collaborative work, particularly over the long haul. One of the program’s strengths was its ability to coordinate diverse organizations behind a single goal. But in the aftermath of Ceasefire’s success, bitter fights erupted among participants over credit and public attention.

The difficulty of managing all of the egos and interagency politics was compounded by the departure of several key players within the police department. In general, leadership transitions are crucial moments for programs. The importance of planning for succession, and building institutions and programs that do not rely entirely on heroic individuals, cannot be overstated.

Unrealistic expectations exert a seductive power. Perhaps the biggest challenge that policy reformers in any field must face is the mismatch between public expectations and what programs can reasonably be expected to deliver.

For example, individuals who find themselves enmeshed in the criminal-justice system bring a staggering array of problems with them, including addiction, mental illness, and histories of poverty and abuse. Helping these people get on the right track and avoid criminal behavior is no simple matter.

Experience tells us that this is an enormously difficult message to deliver to political officials and the public. Indeed, there is a powerful temptation to do the opposite: In the fierce fight for limited dollars, many programs end up over-promising what they can deliver. The costs of this are significant. When we fail to talk honestly about what reform efforts can realistically be expected to achieve, we run the risk of deepening public cynicism about government and nonprofit organizations.

A potential example of this comes from the federal Second Chance Act, a rare piece of bipartisan legislation, which was signed into law in 2008. The Second Chance Act authorizes federal grants to improve results for people returning from prisons and jails. All well and good—except that the law says that money must go to groups that have a goal of reducing recidivism by 50 percent.

Unfortunately, according to Joan Petersilia, a criminologist at Stanford University, even the best programs can’t reduce recidivism by more than 15 or 20 percent. “We just have to be more honest about that,” she said, adding, “We’ve assumed that we would not win public support with modest results.”

For those who control private and public purse strings, the ultimate lesson is simple. We can and should be aggressive in tackling the social problems that continue to plague us. We can and should hold grant seekers accountable for achieving positive results. But let’s be realistic about what we expect to achieve and acknowledge that there is no such thing as trial without error.