Response Rates and Random Sampling

Q: We are conducting a communication audit. What is the right number of surveys to send out and what percentage of the surveys we send out should we expect to be returned?

Alice McCormack

A: Dear Alice:

The answer to your question about how many surveys to send out is very complex. It sounds as if you’re considering sending out surveys to a sample of employees rather than all of them.

First of all, the number of surveys to send out depends on how many employees you have, which you don’t mention. If you have a relatively small number of employees, you might need to send out surveys to everyone. If you have over several thousand employees in total, you would need only 500-600 completed surveys to have fairly reliable results for your population AS A WHOLE, assuming the responders accurately reflect the demographics of the entire group.

However, most companies also want to be able to compare various organizational subgroups against each other (locations, business units, etc.). This typically requires a much larger number of responses so that you have a sufficient proportion of each subgroup participating. Also, smaller subgroups may need a larger proportion of the group responding for statistical reliability than larger subgroups.

Determining the number of people to send surveys to is something that really needs to be determined by a statistician who is provided with information about the size of your employee group and subgroups. If you don’t do this carefully, some executive with some statistics background will invalidate all the results of your survey when you’re done.

The second part of your question, though, relates to those 500-600 COMPLETED surveys I mentioned. If you typically get a 50% response rate on surveys, then you would send out 1,000. However, a great many factors will affect your response rate.

The response rate on communication surveys I’ve done for clients varies from 20% to over 80%. A lot of it has to do with:

• The length of the survey. The longer it is, the lower the response rate.
• Demographics questions. If there are too many of them, or if they are on the first page of the survey, the response rate plummets.
• Previous experience. If a company has administered many surveys and never reported back results or made changes based on the surveys, the response rate will go down with each new survey.
• Management support. If senior management lets middle management know that they really want to see the results and want to see good participation in all units, managers make sure employees are given some time to complete it. Otherwise, they give people grief about “not working” while they’re completing surveys.
• If there is a reward for the location or the department with the best response rate, or if every location with at least a minimum return rate receives a reward, that makes the biggest difference. Then peer pressure gets results. For example, getting an extra day off around a three-day weekend for each “winning” location.
• How and where it is administered. Paper surveys sent to the home will have a lower response rate than those distributed individually at work. (However, you need to be aware that some employee groups don’t have the physical environment at work that provides so much as a writing surface.) Of course, group administration in small meetings will get the best rate. Electronic surveys (Web, email or phone) tend to have responses come in more quickly. Most responders do it right away; with paper, many delay until closer to the deadline. However, with some electronic administration methods, people are more likely to feel that they could be identified individually. If you’re asking communication questions of a sensitive nature, such as about supervisors’ communication skills, you might get a lower response rate electronically than on paper, which is perceived as providing more anonymity.

I’m sorry for such a long answer, but this is a very complicated issue!

Angela D. Sinickas