April 27, 2006 | Read Time: 9 minutes
TOOLS AND TRAINING
By Marilyn Dickey
For members of the Association of Fundraising Professionals, getting the results of the group’s survey of fund raisers’
salaries and benefits is one of the most valued perks of membership in the organization, says Joyce O’Brien, vice president for communications and marketing.
The questionnaire, which is sent to more than 4,000 fund raisers, breaks down the data on fund raisers’ salaries by such criteria as job title, location, the group’s budget, and the fund raiser’s years of experience.
But between printing, mailing, and waiting for replies, it used to take months for the association to conduct its survey.
Then, a few years ago, the international professional association, with headquarters in Alexandria, Va., started conducting its questionnaire online, using a Web site called WebSurveyor. Now the printing and mailing steps have been eliminated, and instead of it taking weeks to get replies, most responses arrive within 48 hours.
Doing the surveys online is an enormous time saver, says Ms. O’Brien, and the Web site is easy to use and versatile. Like other survey sites, WebSurveyor automatically tabulates the results, which can be downloaded to the user’s computer. “We can export the data, and we can create charts and graphs in it,” she says.
The association was initially reluctant to switch to an online method for something as important to its members as the salary and benefits survey, Ms. O’Brien says. “We were afraid it would change the response rates,” she says. “We were pleased that it didn’t get worse. It was pretty comparable.”
In addition, polling online is cheaper: While its pen-and-paper surveys used to cost the organization about $30,000 each to conduct, now it pays WebSurveyor less than $5,000 a year for unlimited surveys.
For many nonprofit groups, online surveys have taken the place of more costly and time-consuming print projects. The online questionnaires are created in a Web browser and don’t require downloading software. All the survey creator has to do is type in the questions, send an e-mail alert to the people to be surveyed, and wait for the replies.
Many sites offer free technical support and survey samples to use as guides. Nonprofit groups can often obtain online survey services at a discount, and many sites even allow users to do small surveys free. (See “Resources and Tips for Conducting Online Surveys.”)
Cutting Costs
Many sites charge not by the survey but by a period of time, allowing unlimited polls to be conducted during that period. Calculating the financial value of signing up for an online-survey service depends not just on how many questionnaires an organization will send in a certain time frame, but also on what features a survey site offers.
Leah Eskenazi, a project manager at the Family Caregiver Alliance, in San Francisco, learned about online surveys when she was asked to participate in one being conducted by the Gerontological Society of America.
In 2002, she started creating her own online customer-satisfaction surveys, using a site called Zoomerang. For example, she says, “we have an information line so caregivers anywhere in the United States can call us. We survey those folks and ask if the information we provided was helpful.”
For Family Caregiver Alliance, online surveys saved not only time but also money, she says: “Doing a paper survey is expensive. You have to create the survey, print it, mail it, and send out a postage-paid return envelope if you want to get a good percentage back.”
Internet surveys have other advantages too, she says, including the ability for participants to see rolling results as the surveys are completed.
When participants know they will be able to see how the results stand as soon as they fill out the survey, it increases participation, says David Fetterman, director of evaluation at Stanford University’s School of Medicine, who frequently conducts surveys online and helps nonprofit clients conduct questionnaires. “People are curious: What do people think about these topics?” he says.
Online Glitches
However, for a group like the Family Caregiver Alliance, polling clients online carries some disadvantages. Online surveys, notes Ms. Eskenazi, are “only helpful if people being surveyed use a computer.” Many people who use the alliance’s services live in rural areas, where Internet access is limited. Other clients have access to computers but have frustratingly slow dial-up connections, so they prefer paper questionnaires.
“You definitely need to have alternative mechanisms,” she says. For people who prefer paper copies, she prints questionnaires from the survey site and mails them.
Many online surveys present other limitations, says Katrin Verclas, acting managing director of Innovation Funders Network, a group in San Francisco that helps its fellow nonprofit organizations make better use of technology. Most Internet survey tools, she says, are not accessible to people with certain disabilities. For example, people with visual impairments often rely on screen readers, which use voice technology to make written words audible.
After conducting one survey that asked nonprofit groups how they find and purchase software, the organization got complaints from charities whose members use screen readers and were unable to take the survey. Ms. Verclas then called several other sites to find out if they had encountered the same problem, and found that most did: “They said that this is on their radar, but it would require quite a bit of engineering.”
Before choosing a survey site, it is best to check what features each offers. WebSurveyor is accessible for people using screen readers, according to Yegor Kuznetsov, the company’s public-relations manager. Zoomerang makes available translators who, for a price, can translate a survey into any of 42 languages and, if necessary, translate the replies.
Furthermore, check what formats are available for presenting questions: yes or no, multiple choice, short essays, or more complex options. Some questions branch off into different paths, notes Ms. Verclas, so they present alternate sets of questions depending on whether the answer to a particular question is yes or no.
Priming Participation
The response rate for online questionnaires varies, says Phil Orr, senior vice president for community and investor relations at the United Way of Metropolitan Nashville. His group uses online questionnaires for internal employee-satisfaction surveys as well as for surveys of the charities it supports, donors, and volunteers. Recently, it conducted an online survey to get feedback on a tool it uses to rate the performance of the charities that apply for funds.
“Our response rate was probably about the same or maybe a little higher than paper surveys,” Mr. Orr says.
Ms. Verclas’s most recent previous employer, Aspiration, a nonprofit group in Amherst, Mass., that helps charities use technology better, now uses online questionnaires to find out whether the group’s seminars were useful to participants. Before conducting its polls online, the organization relied on paper questionnaires after a daylong seminar. It would reserve the last 10 minutes of a session so participants could fill out the survey. With that captive audience, nearly everyone complied. But now that it uses online surveys, which participants fill out after they leave the seminar — sometimes days later — about 50 to 70 percent of people fill out the surveys. However, says Ms. Verclas, the responses are more thorough and thoughtful.
“If people are sitting there, they do it immediately,” she says. “A time delay drops the response rate even if you remind them and prod them. But people tell you a few more stories, and the stories are richer.”
For example, she says, once people get back to work, they are more apt to realize how they will apply what they learned in the seminar, and many will share that information on the survey.
Spreading the Word
In 2003, the Foundation for Ichthyosis and Related Skin Types, in Lansdale, Pa., was in the midst of updating a strategic plan and wanted to get members’ opinions about the direction the organization should take for the next five years. The group supports research and offers programs for people with a disorder characterized by scaly, itchy skin.
To invite participation in a survey, the foundation sent e-mail messages that included the Internet address of the questionnaire, says Jean Pickford, the organization’s executive director. And in case anybody was missed by that method, an announcement was posted in the group’s newsletter, too. “We had over 350 responses,” she says, “and that gave us some good insight into what they were interested in and areas we needed to strengthen.” Reaching out also showed members that the organization valued their opinions, she adds.
But it is possible to conduct a survey without having e-mail addresses for the target group, as Ms. Verclas’s experience has shown.
For Aspiration’s software survey, the group contacted organizations that managed electronic discussion boards. “We targeted networks that have a lot of members, and they put it in their newsletters and on their Web sites,” she says. “We reached out to associations of NGO’s and nonprofits for them to contact their membership — varied organizations such as arts and museum associations, health-care networks, dot-org Webmasters, technology providers to nonprofits, techies in human-service organizations, etc.”
Because of the nature of her efforts to get the word out, she doesn’t know how many people knew about the survey — and, due to the method by which it was conducted, she could make no claims to its statistical accuracy. But she was pleased to receive about 800 responses.
Crafting Questions
Even the most useful and efficient survey sites, however, cannot write pollsters’ questions for them. Survey sites often include sample questionnaires, says Ms. Eskenazi, of Family Caregiver Alliance, “but I would encourage people to meet with someone about how to craft the questions, too, so they’ll get the information they want. It needs to be customized.
“If you don’t put together a good survey, you won’t have good information,” she says. “Nothing is more frustrating than getting the results and realizing they’re not what you were looking for.”
Ms. Eskenazi, who has had training in program evaluation, sends a draft of each survey she creates to 10 of the organization’s staff members and clients for vetting, then makes modifications based on their feedback before sending the final questionnaire to the target audience.
To get a better response rate, she recommends keeping it short: “Eleven questions is probably maximum.”
Survey sites do tabulations and cross-tabulations, but some complex surveys may need more work once the results are in, says Ms. O’Brien. “We still send out the results to a statistician to have some of the analysis work done. But WebSurveyor gets us three-fourths of the way there.”
Even after finding a survey site that suits a nonprofit organization’s needs, it is worth checking different sites from time to time to see what features they have added.
“They’re always updating,” says Ms. Eskenazi. “They’ll survey us and say, ‘What do we need to do differently?’ And then they get back to you with regular updates.”
Does your organization do online surveys? Tell how you gather data on your supporters or clients in the Tools and Training online forum.
