The following article appeared in
Strategic Communication Management, December/January 1998
Melcrum Publishing Ltd, London

© 2000 Angela D. Sinickas. All rights reserved.

.

How to Measure How Well Supervisors Are Communicating With Their Employees

By Angela D. Sinickas, ABC

Question:  How can I measure how well supervisors are communicating with their employees?

First, be clear on how you will use results of your measurements to help you decide if "soft,"  qualitative findings from focus groups will provide enough information or whether you need to get "hard," quantitative data. For example, if you want to track improvement over time or compare different supervisors, you will need the kind of statistically reliable results available only from a survey.

Even if you need hard numbers, initial focus groups with supervisors and employees will help you ask the right questions and pick the right measurement method. For example, focus groups might discover that the problem isn't communication skills, but that supervisors haven't been given the content to communicate.  This can be measured by asking supervisors themselves how well informed they feel about various topics they are expected to communicate.  Another thing that can be measured is tracking how frequently various departments have staff meetings.  It's possible to have supervisors who know how to conduct meetings, but who rarely do so. This can be tracked by asking employees how frequently staff meetings are available to them and how frequent they should be conducted.

Second, determine the purpose of your information gathering. If the goal is to assess the general level of supervisory communication in an organization, you can include questions on an employee survey administered to a random sample of employees and use that information to prioritize communication training content.  If you want to assess the communication effectiveness of individual supervisors, you would instead need to conduct a survey of all employees to obtain enough responses to tell how well supervisors in different organizational units are doing. Better yet, surveys can be administered in work groups to assess the skills of the group's own manager. This is known as 360-degree feedback on each supervisor from peers, subordinates and bosses, which is often used as part of the performance appraisal process. In addition, these individual results could be aggregated to get a picture of the organization's overall supervisory communication skills.

Additional questions that should be answered if you are measuring supervisors' communication skills include:

Will the evaluation measure skills independently of each other or in relation to each other (a relative assessment)? An independent evaluation lets you know how good "good" is and how bad "bad" is &endash; and how much different measures improve after corrections have been made. Identify various communication skills. Either ask how strongly employees agree or disagree that their own supervisor uses each skill well, or ask how frequently the supervisor demonstrates each skill, from "almost all of the time" to "never."

Using a relative evaluation, respondents rank-order the skills listed on the survey from most to least effectively used by their managers.  So, if 10 skills are listed, even a strong communicator will have one skill ranked at the bottom, one at the top, and eight in the middle.  Similarly, an ineffective communicator will receive a top ranking for some aspect of communication skills.  The relative evaluation is not intended for comparing supervisors against each other  or tracking improvement over time. It is most useful when you intend for supervisors to discuss their own evaluations with their subordinates. The results are still disturbing enough to encourage change, yet not threatening enough to prevent employees from filling out the forms.

Will supervisors and managers be asked to evaluate themselves as well as be evaluated by their subordinates?  Comparing the results of how well managers think they communicate with how their subordinates rate them often highlights very interesting and surprising discrepancies.

Are all skills equally important?  You can also ask employees and supervisors their perceptions of the relative importance of various supervisor communication skills for doing their jobs well. Different skills may be more important in some parts of your organization or for different types of jobs. The priorities for training will then be the skills that are rated as very important, yet are not frequently or well demonstrated.  In addition, a statistical technique called regression analysis can help determine the relative importance of various communication behaviors.

How will the results be fed back to employees and supervisors? Typically, the results of a broad survey will be fed back to all managers at once, and then to all employees. However, if part of the purpose of the measurement is to improve communication within work groups, it can be even more useful to feed back results in a "cascade" from top management to individual work groups. First the president would share his or her results in a  meeting with the vice presidents. They would also discuss what actions will be taken to improve weaknesses and make more use of the strengths.  Then these vice presidents' direct reports feed back the results to their staffs.  When the assessment cascades downward, each new group of evaluators feels safe in being candid because they have already experienced the benefits of providing this type of information to their own bosses in a safe environment.

Will any part of managers' compensation be based on communication skills?  For people to change behavior, it helps to have an incentive.  The skills survey scores can be used as an objective assessment during a manager's performance review for skills like teamwork, leadership, etc.  If so, you will need to use a survey and be sure to obtain enough responses for statistical reliability. For instance, if many managers have small work groups, you will probably need to have each direct report return a survey. Exhibit one and two show some examples of the difference in how you would structure a questionnaire asking for either independent evaluations or relative evaluations. Remember that the independent evaluation is more effective for tracking improvements over time either for individuals or for the organization as a whole:

Exhibit One: Independent Evaluation Questionnaire
                                      

Independent Evaluation	All or Almost All of the Time	Most of the Time	Some of the Time	Hardly Ever	Never
1.  My supervisor keeps me informed about things that are important.	1	2	3	4	5
2.  My supervisor provides clear direction and priorities for the department.	1	2	3	4	5
3.  My supervisor really listens when I speak to him or her.	1	2	3	4	5
4.  My supervisor communicates well with me one-on-one.	1	2	3	4	5

 

Exhibit Two: Relative Evaluation Questionnaire

Communication Skills	My Supervisor's Rank Order 1 = Most Effective Skill 4 = Least Effective Skill	Importance for Supervisors to Have This Skill 1 = Least Important 4 = Most Important
1.  Keeping me informed about things that are important	_____	_____
2.  Providing clear direction and priorities for the department	_____	_____
3.  Really listening when I speak to him or her	_____
4. Communicating well with me one-on-one	_____	_____

Angela Sinickas, ABC, is president of Sinickas Communications, Inc., a communication consultancy specializing in helping corporations achieve business results through targeted diagnostics and practical solutions. You can visit her new website, CommToolbox.com, to see the automated planning, measurement, and benchmarking tools she has developed based on her manual, How to Measure Your Communication Programs.