Most Charities and Religious Groups Evaluate Programs, Report Says

October 31, 2002 | Read Time: 4 minutes

Slightly more than 85 percent of U.S. nonprofit groups and 72 percent of religious congregations

ALSO SEE:

TABLE: Measuring Performance: Challenges Charities and Religious Groups Face

TABLE: How Nonprofit Groups and Religious Congregations Evaluate Accomplishments

evaluate their programs, according to a new report.

But what those groups evaluate is very different: Secular groups are almost twice as likely as religious organizations to monitor the quality of their programs and what types of people they serve, a study of 900 organizations found. What’s more, 68 percent of nonprofit groups ask aid recipients or patrons to rate their satisfaction with services, but only about half that percentage of congregations do the same.

The report was issued this month by Independent Sector, a coalition of more than 700 charities, foundations, and corporate donors, in Washington. The report was based on data collected in 1998 and was underwritten by Atlantic Philanthropies, the Ford Foundation, the W.K. Kellogg Foundation, and the Lilly Endowment.

Religious organizations were more in line with their secular counterparts when it came to measuring what changes programs have brought about in the conditions of beneficiaries. A total of 65 percent of religious groups say they keep track of such changes and about 76 percent of nonreligious groups report the same.

Besides religion, budget size and the mission of the group also were linked to whether a charity examined itself. Organizations with budgets that exceed $1-million were more likely to measure their efforts. Arts organizations were most likely to perform regular evaluations, while human-services charities were the least likely.

Charities Cite Challenges

While Independent Sector’s report praises the fact that a majority of respondents performed some kind of evaluation, it says that more evaluations should focus on the quality of services provided, as opposed to the number of services provided.

The lack of attention given to the qualty of services means charities and their supporters will need “a considerable investment of time, money, and patience” to improve efforts to measure effectiveness, the report says.

But that may be hard because many of the charities that participated in the survey said it would be very difficult to develop ways to accurately measure their accomplishments.

About 76 percent of congregations and almost 60 percent of nonprofit groups said the results of some or all of their programs were not concrete enough to measure. Religious and environmental groups were the most likely to say that their efforts were intangible. Health organizations were the least likely to express that concern.

According to the report, a smaller but still significant percentage of groups — 20 percent of secular groups and 34 percent of congregations — said they lacked the knowledge to evaluate their efforts. About 35 percent of arts organizations described themselves this way, making them the most likely type of nonprofit group to say they were ignorant of evaluation methods. “Public and societal benefit” groups, such as civil-rights and advocacy groups, and research institutions, had the lowest percentage — 11 percent — of respondents saying they lacked knowledge of how to gauge their efforts. Organizations also mentioned other obstacles to quantifying accomplishments, including:

Limited capacity to collect and analyze performance information.
Difficulty keeping track of beneficiaries after they leave programs.
Lack of computers and computer software designed to track the results of charitable programs.

Social-Services Groups

In part because of the difficulties charities face in self-evaluation, a new report from the Urban Institute, a public-policy think tank in Washington, spells out ways human-service groups can measure their programs.

After studying the work of eight organizations near Baltimore and Washington, including a Big Brothers Big Sisters group and a Boys & Girls Club, the report recommends 19 ways organizations can better measure themselves.

Among the recommendations:

Improve computer technology and the computer skills of employees to better collect and analyze information on services.
Avoid using information about results to punish caseworkers whose efforts may not be effective in helping recipients of assistance.
Involve staff members in determining what information will be used to measure organizations’ effectiveness and how that information will be used.

The Aspen Institute and the David and Lucile Packard Foundation paid for the Urban Institute report.

Highlights of the Independent Sector report, Balancing the Scales: Measuring the Roles and Contributions of Nonprofit Organizations and Religious Congregations, are on the organization’s Web site at http://www.independentsector.org. The full report is available for $28 for Independent Sector members, $38 for nonmembers, from Independent Sector’s Publications Center, P.O. Box 343, Waldorf, Md. 20604-0343; (888) 860-8118; fax: (301) 843-0159.

The Urban Institute report, Making Use of Outcome Information for Improving Services: Recommendations for Nonprofit Organizations, is online at http://www.urban.org. It is available for $5 from the Urban Institute’s Publications Sales Office, 2100 M Street, N.W., Washington, D.C. 20037-1264; (877) 847-7377; fax: (202) 467-5775.