The following article appeared in
Strategic Communication Measurement, December/January 2003
Melcrum Publishing, Ltd, London, UK

.

Pitfalls, Problems...and Possibilities in Measurement

By Angela D. Sinickas, ABC

This month's issue of SCM marks the fifth anniversary of Angela Sinickas' columns on measurement for three different Melcrum publications. We asked her to use this opportunity to take stock of how far communicators have come on measurement, and how far we have yet to go.

What's right with communication measurement these days is that a lot more communicators are conducting research. What's wrong is that we're not always measuring the right things, analyzing what we've learned, or doing much about it.

Measuring the right things

Too many communicators are still predominantly measuring output, without looking at communication's effectiveness or outcomes. Sending out twice the number of news releases is not an improvement if the media get irritated with the lack of meaty subject matter. Obtaining twice the number of clips among readers of publications geared to our target audiences, and having the articles be accurate and mostly positive, would be a significant improvement.

Sometimes communicators' measures are dictated by others. Perhaps the Finance Department is demanding external benchmarking on staffing and budgets, without asking how effectively the people and money are being used. Or maybe the communication metrics are created to fit into a corporate scorecard, even though they're quantifying the wrong things or they don't change often enough to be tracked monthly like the other scorecard numbers.

Sometimes communicators have embraced new technologies for measurement ("free" software like Zoomerang and Quask) without thinking about what is important to measure or if the limited formats for structuring questions and analyzing data are appropriate.

Analyzing what we learn

Many communicators are overwhelmed by the volume of raw data they're gathering. They don't know where to begin in assessing what they have. Website and intranet usage reports provide a perfect example. Communicators should be identifying what they want visitors to do on their sites and tracking those specific measures from the reports. Communicators then need to make changes to their Websites to drive the behavior they want, and track their usage numbers to see if they have changed.

Communicators need to think about the meaning of the data they're collecting. Numbers are not intrinsically good or bad. They only have meaning in context. When communicators look at their data, they should be looking for blips, variations and gaps from what might be expected or from what occurred in the past. They need to ask why those numbers are the way they are.

These answers can come from looking at additional data, such as demographic variations or focus group results. Other times, communicators might find explanations in the observable corporate environment. This approach to thoughtful interpretation leads them to make better decisions about what to change in the communications they're measuring.

An example of what to look for

To illustrate all these points, Figure One illustrates research conducted about the 33 articles I've written for three Melcrum publications during the last five years. The measures in the left column are listed in increasing order of importance. The "Initial Reactions" column offers first guesses at the possible meaning of the numbers. The "Analysis" column takes the conclusions further to identify the measure as not relevant, to assess whether the result is due to something real or something random, and to develop action steps based on the findings.

Taking action

The point of this example is to show how a logical and creative approach to analysis can lead to action. Decisions I made, based on the analysis, range from what I should do before submitting my column, to how I should work with the editors, to an idea for writing a new book.

So when you finish this issue of SCM, take a look at what you're measuring. Are you measuring enough of the right things? Are you focusing on the key measures rather than letting the rest of the numbers overwhelm you? Do you understand what's creating your current results? Do you know what actions you'll need to take to change the results in the future? When you can answer all these questions, then you'll know you're getting full value from the measurements you're conducting.

Efficiency Measures	Initial Reactions	Analysis .
Average number of words per column of 913, with a steady decline from a high of 1,113 per column in 1998, to a low of 801 per column in 2002, which is a 28% decrease!	Have I learned to be more concise? Maybe the editors don't like me as much as they used to.	No significance; different publications had different amount of space for the column based on fonts and page margins.
Total of 30,127 words written.	Surprised at how much I've written and how many hours of time that represents; I should be getting paid for this.	With this large number of words written, it may be time to edit the columns into a book and look for a publisher!
Effectiveness Measures	Initial Reactions	Analysis .
Reading grade-level measurements show that it takes from 11 to 12 years of formal education to understand the writing in my columns.	Seems to be on target since most corporate communicators have at least that much education, although research shows people prefer to read technical material at two grade levels below their capacity.	My original drafts tend to be written at a grade 12 and stay at that level after editing by SCM editors. A different editor for two previous Melcrum publications brought the grade level of my columns down to an even easier-to-read score of 11.3; I should do a readability check on my drafts and simplify them before submission.
Best-read columns were on reporting focus group findings and measuring supervisory effectiveness (measured by how many of my Website visitors opened electronic reprints of columns).	The decision to click on a column is based solely on its headline; I need to make sure the main headlines are direct and specific since the reader can't see any explanatory subheads, as in the original printed piece.	Need to factor in if these favorite columns were selected due to their placement near the top of the reprints screen, not their subject matter; analysis showed these columns actually required significant scrolling to reach, so these findings must be significant. Since these best-read columns were published in 2000 and 1998, it may be time to write about these topics again.
In a survey of my 2001 clients, 63% said they were aware of my columns.	Need to find ways to increase awareness of the columns among more people.	Awareness is great, but what impact did the column have on readers?
Outcome Measures	Initial Reactions	Analysis .
In a survey of my 2001 clients, 70% of those who were aware of my columns said it had influenced their decisions to hire my firm.	The columns were the second greatest predictor of a decision to hire us.	I should continue to send reprints to people who are not yet SCM subscribers to build credibility with a greater number of business prospects.
Informal research showed that the column on "Measuring the brand internally" led to a new international client asking me for a proposal for developing an internal communication strategy, infrastructure and measurement program.	The timing just happened to be right; they needed a consultant and they noticed my name. Further probing showed the thought process reflected in the column led them to read more writings on my Web site, which strongly affected their decision to hire us.	I should continue to write these columns for free since the value of just this one project is more than the billable value of the estimated time spent writing all 33 columns in the last five years; in fact, I'll have an 89% return on investment after covering my time. That's not bad!

Angela Sinickas, ABC, is president of Sinickas Communications, Inc., a communication consultancy specializing in helping corporations achieve business results through targeted diagnostics and practical solutions. You can visit her new website, CommToolbox.com, to see the automated planning, measurement, and benchmarking tools she has developed based on her manual, How to Measure Your Communication Programs.