Data Are ‘Problematic’ in Giving Report

To the Editor:

In November, we sent The Chronicle of Philanthropy’s editor a commentary explaining our grave misgivings about the data in the news organization’s “How America Gives” report (October 9), the methodology, and some of the prominent findings of the study as it appeared in print and on the interactive website. We are pleased that The Chronicle has revised its study in light of some of our reservations.

The gist of our concern was that using IRS ZIP-code data instead of IRS state or county data missed 20 percent of itemized charitable giving, most of which was concentrated at the upper end of the income distribution.

In the letter we suggested that the study’s findings regarding states and metropolitan areas be recalculated based on the IRS state and county data, as The Chronicle has now done.

As The Chronicle prepared its October study we were contacted for advice.

At that time we warned The Chronicle that using the ZIP-code data was problematic and gave an example of how ZIP codes did not add up to IRS state data.

We also offered, but were not engaged, to work with The Chronicle to make sure that its 2014 study did not result in the same problems as in its 2012 study based on ZIP codes.

We do think that once The Chronicle recognized the problems with its study, it should have alerted its readers to the problems and withdrawn the study from circulation while the findings were revised. Instead, even after deciding on revisions, The Chronicle sent out emails several times with banner links to the unrevised study.

As it circulates the revised findings, the paper needs to communicate to readers in a prominent manner just what revisions in methodology have been made, what storylines based on the previous study have changed, and which ones should now be deemed as more tentative.

This is especially important regarding the state, metropolitan, and income-based comparisons.

For example, the October study led to national and local headlines about rankings of geographical areas, the relative charitable giving of red and blue states, and the reduced relative charitable burden of the upper-income group.

In more than a decade’s work on charitable giving relative to income, we have found that adjusting income for geographic differences in tax burden and cost of living significantly modifies the ranking of states.

These adjustments reveal virtually equal relative charitable giving among red and blue states.

In contrast, The Chronicle’s 2008 rankings and recent 2012 rankings fill the knowledge space with the problematic finding that red states are far more charitably inclined.

While The Chronicle has now taken steps to improve the numerator (the amount of itemized charitable giving) in the ratios leading to its comparisons, it next needs to improve the denominator of its ratios (the amount of income) to account for cost of living and taxes. It must, therefore, communicate suitable caution even about its revised findings.

John Havens

Paul Schervish

Boston College Center on Wealth and Philanthropy

Editor’s note: We appreciate the advice from Mr. Schervish and Mr. Havens. We agree that using county-level data is more accurate and have revised the data on our site, added details about our methodology, and further explained the limitations of the available data. Adjusting the data to follow the advice from Mr. Schervish and Mr. Havens did not lead to any major changes in state or city rankings, and we have noted the small changes in each article on our site.

In producing the 2014 “How America Gives,” we followed advice from several scholars and changed the approach we took in 2012 to avoid cost-of-living adjustments. We also refrained from selling the data once we heard from Mr. Havens and Mr. Schervish, and everyone who purchased the previous version of the data was automatically furnished with the updates as soon as they were available. As Mr. Schervish and Mr. Havens know, there are many ways to look at IRS giving data. We stand by our approach, and will continue to refine our work in future years and push for better data to improve our ability to measure generosity.