By Angela Sinickas,

Founder and President, Sinickas Communications

Angela Sinickas is a pioneer in the field of organizational communication measurement. Her company’s recent clients include Merck, Nordstrom, 3M, Raytheon, ExxonMobil, and Lockheed Martin. Ms. Sinickas’s work assessing communication quality has been cited in Harvard Business Review and Investor’s Business Daily as well as several other publications.

 The biggest challenge for auditors is to make sure that they’re measuring the right things. All too often communicators measure only their outputs—the messages and channels they’re producing—without connecting them to the outcomes of using these outputs.

For valid and reliable information, you need to use a quantitative approach. In many cases, that means using a survey, but only if you’re careful in selecting a large enough, and truly random, sample and being sure that the completed surveys are returned in the same proportions. There are also many other quantitative techniques that can be used in an audit. For example, a content analysis can quantify to what extent the communications you send out are aligned with the goals of your organization. You can also quantitatively track changes in your audience’s behaviors due to your communications—either over time or using a pilot/control group study to isolate the impact of communication versus other possible change agents.

I have had situations where clients did not want to “hear” what our research had uncovered. Executives are all too often sheltered from the reality of their organizations by middle managers who don’t want to be the bearers of bad news. Even when executives conduct meetings with employees, the employees won’t always be candid if their supervisors are also in the meeting. Focus group findings are more often challenged than survey results, and with good reason. There are too many variables in how the focus group participants are selected and how the facilitators conduct the sessions. Plus, few organizations involve enough participants to constitute a valid sample for projecting findings to the entire group. I did have one executive who refused to believe some negative results from a survey administered to all employees, in spite of having a very large response rate. And I’ve known executives who give more credence to lots of direct quotes from focus groups than to survey results.

           There’s no reason an organization can’t do an audit without outside help, as long as they have appropriate resources available inside. For example, if they’re doing a survey, they should find someone in HR, Finance or Marketing who can help them in determining sample sizes. They should use the experience of someone in Organizational Development or Market Research to ensure the questions they’re asking and the response scales they’re using will provide usable data. Most important, for any type of qualitative research, the facilitators must not be the individuals who are responsible for creating the communications that are being audited, for a number of reasons that affect the quality of the interaction in the sessions. However, if the communication department has recently hired someone who has no vested interest in the existing communication program, that person could conduct the qualitative research pretty well, assuming he or she had some training in the process. Otherwise, I’d recommend using someone from HR or Market Research to conduct interviews and focus groups more objectively.

The downside of doing it all yourself is that you will make mistakes and it will take longer because you’re inventing everything yourself. The half-way solution is to do as much as you can in-house, but use outside resources judiciously, perhaps for peer review.

If you choose to hire someone from the outside, choose an auditor who has had lots of experience. Talk to his or her previous clients. Ask to look at samples of the types of reports they have presented to other clients. Determine if the reports would be useful to you or if they are organized and written in a way that only a researcher would find useful. Try to find an auditor who knows not only research, but has also worked in communication. Preferably, find one who has spent some time working within organizations, not just in consulting firms. You’ll find the recommendations the auditor comes up with will be more practical and usable.