The following article appeared in
Best Practice Measurement Strategies, May 2001
Melcrum Publishing, Ltd., London,

.

Unraveling the mysteries of sampling

By Angela D. Sinickas, ABC

 Some of the most common questions about surveys are: "How many surveys do I need to distribute?" "What's the right sample size?" "What's a good response rate?"

The number of surveys to send out depends on how many employees you have and what rate of response you are likely to achieve. If you have a relatively small number of employees, you might need to send out surveys to everyone. If you have over several thousand employees, you would need only 500-600 completed surveys to have fairly reliable results for your population as a whole, assuming the respondents accurately reflect the demographics of the entire group. So, if you expect to have a 100% response rate, you would mail out surveys to a random sample of 600. More realistically, if you typically have a response rate of 50%, you'd need to survey 1,200 people (600 divided by 0.5).

Complication #1:

Most organizations also want statistical validity in comparing organizational subgroups against each other (locations, business units, etc.). This typically requires a much larger number of responses than 600 so that you have a sufficient proportion of each subgroup participating. For example, if you have two locations of several thousand employees each, you'll need 600 responses for each location. Also, smaller subgroups may need a larger proportion of the group responding for statistical reliability than larger subgroups.

Complication #2:

You need to achieve a good response rate or you'll wish you had sent out more surveys. The response rate on voluntary surveys generally varies from 20% to over 80%. Where your own response rate will fall depends on a number of factors:

• The length of the survey. The longer it is, the lower the response rate.
• Demographics questions. If there are too many of them, or if they are at the beginning of the survey, the response rate plummets.
• Previous experience. If a company has administered many surveys and never reported back results or made changes based on the surveys, the response rate will continue to go down with each new survey.
• Management support. If senior management lets middle management know that they really want to see the results and want to see good participation in all units, managers make sure employees are given some time to complete it. Otherwise, they give people grief about "not working" while they're completing surveys.
• Incentives. If there is a reward for the location or the department with the best response rate, or if every location with at least a threshold return rate receives a reward, that makes a big difference. Then peer pressure gets results. For example, getting an extra day off around a three-day weekend for each "winning" location.
• How and where the survey is administered. Paper surveys sent to the home will have a lower response rate than those distributed individually at work. (However, you need to be aware that some employee groups don't have the physical environment at work that provides so much as a writing surface.) Of course, group administration in small meetings will get the best rate. Electronic surveys (Web, e-mail or phone) tend to have responses come in more quickly. With paper, many delay until closer to the deadline. However, with some electronic administration methods, people are more likely to feel that they could be identified individually, which reduces return rates. If you're asking questions of a sensitive nature, such as about supervisors' communication skills or intention to stay with the company, you might get a lower response rate electronically than on paper.

Deciding a sample size is best done with the help of a statistician who knows not only how to sample employees, but also to pick a random sample. (Hint: This does not involve throwing darts at a list of names!). If you don't do this step carefully, some executive with even a little background in statistics may invalidate all the results of your survey when you're done.

Angela Sinickas, ABC, is president of Sinickas Communications, Inc., a communication consultancy specializing in helping corporations achieve business results through targeted diagnostics and practical solutions. You can visit her new website, CommToolbox.com, to see the automated planning, measurement, and benchmarking tools she has developed based on her manual, How to Measure Your Communication Programs.