To Make Program Evaluation More Affordable, Start With Existing Data

The Abdul Latif Jameel Poverty Action Lab, a network of more than 100 professors who work with nonprofits to evaluate their programs, has created a guide for using existing data to set a baseline for statistical comparisons.

Many researchers consider randomized, controlled trials to be the best way to measure the effects of nonprofit programs. In these experiments, the researcher randomly assigns some people to participate in particular programs or services and others to be in a control group. By analyzing the differences in outcomes between the two groups, researchers can determine what effects the services have.

But in order to identify any changes, they need to compare the outcomes against baseline data that shows the status of the study population at the start of the test. Collecting that information can be time-consuming and expensive. That’s where existing data sources come in. Government agencies and other groups regularly collect data from the public, such as medical statistics, school test scores, and crime records. The information is free and easy to access.

For example, to evaluate a youth-violence prevention program called Becoming a Man: Sports Edition, run by Youth Guidance and World Sport Chicago, researchers used state education and crime records as the baseline for their evaluation.

The Poverty Action Lab’s paper explains how the statistics were used as well as the costs and ethics involved in using that information.

The organization also created a catalog of data sets that nonprofits can use.