How Foundations Approach Evaluation Varies Widely, Say Experts

More foundations today rely on evaluators to help them assess and improve their giving—and that’s not easy work. The job of chief evaluator is one of the most challenging in philanthropy, according to foundation officials and experts. Foundation leaders sometimes lack a true commitment to the work, program officers may view evaluators with discomfort, and money for evaluation seems to be relatively scarce, according to interviews and studies.

“It’s a brutally tough job,” says Phil Buchanan, president of the Center for Effective Philanthropy, which collects data on foundation performance. “Many times foundations aren’t operating with clear enough goals or coherent enough strategies that it’s even possible to assess.”

Even if foundation leaders are clear about what they want to achieve, they don’t always give evaluation a high priority. And some are frustrated by the work. While 68 percent of foundation executives surveyed last year by the Center for Effective Philanthropy said they thought philanthropies had gotten better at measuring their performance over the past decade, 65 percent reported that it’s a challenge to ensure that evaluations result in “meaningful insights.”

Managing Relationships

Grant makers struggle to find ways to structure evaluations so they will help shape programs and inform decisions.

A study for the Evaluation Roundtable, a network of foundation evaluators, seems to suggest that evaluators who report to a fund’s president, as opposed to program leaders, tend to believe they have bigger budgets, larger staffs, and greater impact.

But others wonder if that’s because foundations that structure the job that way start out with a bigger commitment to evaluation.

Managing relationships with program officers can also be tricky. Sometimes people who review grants have unrealistic expectations for what kinds of information can be collected. And too often they worry that an evaluation could result in a yes-or-no vote on their work, rather than a chance to refine or improve a program’s strategy.

“People have a lot of ego on the line—they’re putting their intellectual capital on the line—when they design programs,” says David Colby, vice president for research and evaluation at the Robert Wood Johnson Foundation. “Developing a culture where it’s clear we’re here to help learn and spread lessons can soften that.”

Without that culture, evaluators may be shut out.

“You don’t get invited to the right meetings, you don’t get the information until it’s too late, or you’re invited to meetings but you don’t get enough information to be effective or informed,” says Patricia Patrizi, a former evaluator at the Pew Charitable Trusts who leads the Evaluation Roundtable.

Slim Budgets

Then there’s the question of money. Although foundation-speak is often heavily doused in words like “outcome-oriented” and “strategic,” philanthropies don’t seem to be devoting much of their budgets to evaluation.

The survey of 31 foundations conducted in 2009 for the Evaluation Roundtable found they spent a median of 2.2 percent of their grants budget on evaluation.

Many funds—nearly 40 percent of those polled—said they spent less than 1 percent.

Longtime philanthropy watchers say, however, that some foundations are getting better at evaluation.

Fay Twersky, former director of impact, planning, and improvement at the Bill & Melinda Gates Foundation who is now a senior fellow at the William and Flora Hewlett Foundation, says philanthropies are getting a little smarter about integrating evaluation into their work and using it to learn from and make course corrections.

“I used to say these positions were the worst positions at foundations because you were either adversarial or irrelevant,” she says. “We’re moving beyond that, but we have a long way to go.”