Unscientific surveys: taken too seriously

To the Editor:

The attention paid to the Council on Foundations’ grant makers’ salary reports and management surveys (“Foundation Employees Saw Salaries Rise by 5 Per Cent, New Survey Finds,” November 18) is baffling.

These are not “surveys” in the technical sense of providing information about a sample of respondents. They simply reflect information from those who choose to reply, with no effort made to normalize the data to philanthropy as a whole. Thus, they are of limited value for decision making and can, in many cases be misleading.

For example, are we to believe (as suggested in the table on “Median Foundation Salaries, by Position”) that program and administrative vice-presidents make more than C.E.O’s? Obviously, no. This is an aberration reflecting the fact that the vice-president title does not exist at the smaller foundations, which is likely to bring down the median C.E.O. salary.

Until such time that the Council on Foundations makes an effort to scientifically sample the field of philanthropy by size and location, reporting margins of error, we would probably all be better off without the data as presented. The size and location breakdowns in the fuller report suffer from the same lack of precision.

Stephen Viederman
President
Jessie Smith Noyes Foundation
New York

* * *

To the Editor:

I was a little disturbed to read your report on the recent Independent Sector conference in Los Angeles (“Non-Profit Officials Admit Weaknesses in Leadership,” November 4). I was at the conference and took part in the survey experiment. It was in no way a scientific poll with reliable results, nor would anyone involved make such a claim.

My personal belief is that flaws in the questioning led to a preponderance of mediocre scores. First, participants were asked to assess the sector as a whole as opposed to their own institutions. Very few of the conference participants had positions that would allow them to observe the sector as whole (as if the sector could be defined and treated as a single entity anyway).

The “sector” is a vast universe of diverse organizations performing functions across a complete spectrum of human endeavors. For any given characteristic (e.g., “leadership”), participants would naturally assume that some organizations do well and some poorly, thus resulting in an average score.

This leads to the second flaw, which is that “I don’t know” was never an option in the survey questions. Had this been included, I suspect we would have learned something valuable in the polling exercise — that there is far more ignorance in the independent sector than there is any coherent sense of “weakness.” There is no Dow Jones industrial average or “leading economic indicators” for the non-profit world, making the study and assessment of the sector the privilege of a few academics and not part of the daily awareness of your average social-service provider.

Anyway, I felt your coverage gave a false sense of legitimacy to a wacky experiment primarily designed to provoke conversation. It was simply a fun way to get people involved in a three-day conference that truly did provide many valuable insights into various aspects of the non-profit world. The sector remains unindicted until real research is performed.

Tony Silbert
Director of Foundations, Grants, and Research
University of Judaism
Los Angeles