Contest Offers Charities Chance to Test Solutions to Social Problems

The randomized, controlled trial has emerged as the gold standard for nonprofits that seek to evaluate the impact of their work. But many charities complain that running such trials can be too costly.

The Coalition for Evidence-Based Policy is on a mission to show that they don’t have to be expensive. The key is designing tests that use data that have already being collected, such as statistics compiled by a government agency, says Jon Baron, the nonprofit organization’s president.

Low-cost trials that tap readily available data will increase the number of approaches that get evaluated, says Mr. Baron. That, he says, is critical because it’s hard to find strategies that truly make a dent in solving tough problems. He says most social programs, when subjected to rigorous evaluation, show little or no effect on the people they’re trying to help, even when initial tests or expert opinions were positive.

“Thomas Watson, who built IBM, once said that if you want to increase your success, double your failure rate,” he says. “This essentially offers a way to do that, to greatly accelerate the testing process so we’re not just talking about a few proven strategies.”

Using State Data

He points to the example of an evaluation of the Positive Parenting Program, also known as Triple P, which relied heavily on existing data.

In South Carolina, nine midsize counties chosen at random adopted the approach, while nine similar counties acted as the control group. Evaluators measured its impact by analyzing the rate of child abuse, hospital visits for maltreatment, and foster-care placements in the counties, data that the state had already collected. The study found that all three measures fell by 25 to 35 percent in the counties where Triple P was in place.

“There are hundreds of thousands of people in these counties,” he says. “If you had had to measure their outcomes the old-fashioned way, by tracking all of them down or choosing a random sample and administering an interview or a survey, the study would have been prohibitively expensive.”

With support from the Laura and John Arnold Foundation and the Annie E. Casey Foundation, the coalition is running a competition for low-cost randomized, controlled trials. It plans to select three proposed trials this year, and will provide up to $100,000 for each one. Go to: coalition4evidence.org.