Evaluation Process Proves Slow Going for Grant Maker

Anyone who wants to know how GiveWell — which says it operates the “world’s first completely

transparent charitable grant maker” — selects which charities should get its money need only look at the New York organization’s Web site.

The site lays out the process from beginning to end — how the group’s grant-making arm, the Clear Fund, selects the charities it invites to apply for grants, which criteria it uses to choose the finalists, and its candid (sometimes critical) assessments of the data those groups provide to prove their effectiveness.

The grant applications from the finalists are on public display, along with supporting documents like program evaluations, descriptions of clients, budgets, and staff biographies. The site even includes comments from charities about the grant-making process. (“A lot of work for not a lot of money,” an anonymous fund raiser griped when GiveWell started soliciting grant applications.)

By publishing such detailed information, GiveWell wants to make the results of its research available to other donors who might be trying to decide where to give.

Job Skills

GiveWell was created earlier this year by two young former hedge-fund workers, Holden Karnofsky and Elie Hassenfeld.

For its first round of assessments, Mr. Karnofsky, the executive director, and Mr. Hassenfeld, the program officer, wanted to find charities that could prove they helped low-income people in New York become self-supporting by giving them job skills. They looked for programs that helped people get jobs, especially relatively high-paying ones, that they probably would not have found otherwise.

They asked charities to document success by providing data on topics such as job placement and how long workers kept their jobs — and created a chart that compared the finalists on how well their clients fared, how many they served, and the total spent by the charity, along with notes to show whether their confidence in the numbers provided was “low,” “moderate,” or “high.”

The pair ended up recommending four charities out of 19 applicants, which they ranked in order of preference: Year Up New York City, St. Nicholas Neighborhood Preservation Corporation, Highbridge Community Life Center, and the Hope Program. One of the four will get the $25,000 grant.

They said their top choice, Year Up New York City — a one-year intensive training program designed to help young urban adults get corporate jobs — provides clients with jobs that pay “wages that are truly self-supporting” and that they would have had trouble finding otherwise.

Even so, Mr. Karnofsky and Mr. Hassenfeld were not entirely happy with the information the charity provided. “Unlike our other finalists, Year Up did not provide us with clear retention data (data on whether and for how long its graduates keep their jobs),” they wrote.

Cumbersome Effort

Lisette Nieves, Year Up’s executive director, said she appreciated GiveWell’s approach because her group believes in openness and accountability — and she thinks demand for more public information about charities is only going to grow. “This young generation was born to blog,” she says. “Young funders seem to be born to blog as well.”

But she said the application process — which allows charities to challenge GiveWell’s evaluation before it is posted on the Web — was quite time-consuming, and she worries that smaller organizations would have trouble accommodating it.

Year Up wrote a lengthy response to its evaluation on the GiveWell Web site, giving a fuller explanation of how the charity works and the way it measures results. “Long-term retention data is extremely difficult to track,” it wrote, “because our students move frequently, often change cell-phone numbers and e-mail addresses, and don’t always maintain regular communication with us.”

One charity that Give Well did not recommend in its draft evaluation, Vocational Foundation Inc., challenged the report’s accuracy — and the GiveWell Web site says it will be withheld “until we are in agreement on all statements of fact.”

VFI’s chief executive, Hector Batista, was angry when he discovered that GiveWell would publish negative information about his group, accusing it of failing to perform “due diligence.” “With all due respect, how are they experts in work-force development?” he asked.

Mr. Karnofsky concedes that his group’s report on the Vocational Fund was “flat-out wrong” on some facts — for example, misstating the length of its training program and misinterpreting some data about how selective the charity is in deciding which clients to help.

GiveWell now plans to recommend VFI to donors, and will rank it fifth, Mr. Karnofsky says. But he says he still has some questions about how much impact the charity is having — and contests the idea that he can’t make such judgments because he is not an “expert.”

Donors, he says, must decide between the recommendations of many conflicting experts, which means “they have to understand enough about the issues to make the best call, and that is what we seek to do.”

GiveWell, meanwhile, is close to publishing its evaluations of charities working in one of its other causes — saving lives in Africa.

While the group originally planned to post evaluations on charities working on five causes by the end of the year, that has proven to be unrealistic, Mr. Karnofsky says. It still plans to award $140,000 in grants in mid-December as planned, but the full evaluations of some of the applicants will have to follow.

The lesson? “We’re not going to take on five causes at a time in the future,” he says.

Mr. Karnofsky says he has also changed his mind about the value of site visits — which he previously eschewed — when he visited Year Up New York City after it won GiveWell’s top recommendation.

“Talking to the staff and students gave me a sense of what they think is most important about the program, which, on reflection, is a better place to start our investigation than with our own questions,” he wrote on the GiveWell Blog.

At the same time, he wrote, because they were working so hard to “sell” the program to him, “l’d much rather do a site visit as a spy than as a funder.”