The following article appeared in Journal of Employee Communication Management, Sept/Oct 2000 Ragan Communications, Chicago
	Benchmarking: Ugly Truths and Unpredictable Outcomes By Angela D. Sinickas, ABC .
	Benchmarking is orderly and scientific, although benchmarking projects are anything but. Veteran consultant Angela Sinickas takes us on several benchmarking roller coasters she's ridden in pursuit of accurate research and measurement, and shares some of the rewards of a job well done. Imagine having a critically important project due with an inflexible deadline. A communication department's future may depend on management's reaction to the results of the project. No sweat, you do that all the time, right? Now imagine that in order to complete this project, you must convince dozens of very busy people you don't know to take from two to eight hours of their own time to gather and share highly confidential information&emdash;perhaps lots of information&emdash;with their competitors in a very detailed way. You can't pay them. And by the way, even after some of them agree to help, some of your volunteers have to abandon you midway through the data collection process for reasons no one could have anticipated. So goes your typical benchmarking project: high stakes, no control, surprises around every corner and an inexorably fixed deadline. JECM asked me to walk readers through a benchmarking project, and to share some of the behind-the-scenes stories of benchmarking gone right, and gone wrong. So, here they are, complete with tales of terror, moments of madness and even some back-room horse-trading. First things first: What does and doesn't work in getting people to participate? You don't have a benchmarking project until you've convinced several companies to participate.  Here are some of the rationales we used to convince companies to join the study (and some of their reactions): "You'll get to see a copy of all the detailed results (without company names identified) for free." ("That's worth putting in my time.") "You'll get to ask them questions during the site visit." ("I would like that.") "My client's president is on your board." ("So why should I care?") "My client's president or senior VP used to work at your company and considers your communication department world class." (Reaction depends on if they knew him and liked him.) "I know you're going through an acquisition; the results of this study could be helpful in negotiating with your new parent company." ("If I take time to help create this study, I won't get my work done, then I risk losing this job for sure.") "If you do this for my client, I'll give you an equal amount of free consulting time in exchange." ("Now you're talking!"). Even with these compelling arguments, most companies decline to participate.  For one client, I contacted 23 companies and finally convinced six to participate; the client lined up the other two from personal contacts. Here are the reasons the non-participants opted out of a study: "We're preparing for a big conference/meeting about the same time as the site visit." "We're up to our eyeballs in things for this fall. Can you call back in a few months?" (5) "I'm new to the position/company, but would be willing to participate in a few months." (2) "We're already participating in another study." (2) "We don't benchmark with competitors" (2) "I'm very busy with a new CEO right now (1) I'm not sure why six of the other companies said no because they never returned a single email or voicemail.  Then, my client rejected five companies who were interested in participating because they were either too small or because another company in the same industry had already accepted our invitation. How are our herding instincts? Your participating companies may go through major changes mid-project, which affects the availability of your contacts. One benchmarking study seemed particularly beset with crises. Here are some examples: A large and well-known management consulting firm was assessing one of the companies participating in a study of ours. Initially, my contact saw this study as a way to provide some objective comparisons with which to leaven their ultimate recommendations. However, before we even finished gathering the data, she found herself in a management meeting where the consulting firm said point-blank that the company didn't need an in-house communication function&emdash;in fact, the management consulting firm would be happy to provide those services on an outsourced basis. (She did win the battle, by the way. Her own measurable results over the years had convinced management that financial return communication was bringing them financial returns.) Two participating companies were going through mergers at the time, one of them twice&emdash;as a potential buyer and a potential target. When we hadn't heard back from our main contact from this second company for a 10-day period, we were making plans to replace his organization in the study. Very late one evening, the contact called, apologized for not returning our calls, and gave us all the information we needed. He apologized in advance for not being available for the next 10 days or so because he was undergoing elective surgery. I owe him a large debt. A fourth company also had to say no after keeping us hoping for over a month. When I heard this news, I was overseas at a conference and feeling a bit helpless. I had made plans to visit a long-time friend. When I arrived there, it occurred to me that her company, although based in Europe, would be a perfect match for the benchmarking study. Within minutes, we had a potential replacement. My client approved the change and we were back on track. Sometimes, you begin gathering data that you eventually can't use and need to replace. In one case, a contact found out after providing us the data that a predecessor had been fired for having shared potentially sensitive information in another benchmarking study. While none of the questions we had asked fit into that category, we had to respect his fears for potential consequences. In another case, a contact provided the data on her department, and then discovered her employer had a policy against participating in benchmarking. Why do I share all these haps and mishaps? Because this is how benchmarking projects always go -- awkwardly, and in fits and starts. And, because there's typically a relatively short period of time to collect, collate and interpret the data, the key is to start lining up the participating companies as early as possible, even before the questions are finalized, for several reasons: Your best contacts may be on vacation or out of the country on business for the first week or two. They return and are excited about the project, but they have to get their bosses' approval. The boss, however, is now taking his two-week vacation. It's now a busy season by the time the boss returns, or the person who has access to the data is on vacation, or it's budgeting time, or the global managers' meeting is coming up -- and they have to regretfully decline. Gathering the data takes time. The initial interview may be only the start of four or five follow-up calls while you're put in touch with specialists throughout the company for the level of detail you need. You have absolutely no leverage to hold the participants to deadlines. You have only their good will going in your favor as it is. Other unexpected events you should expect Near the end of each of these projects, I promised myself never to accept another benchmarking study. Strange things happen while you're gathering data. I kept missing scheduled calls with one contact. I thought one company in the Southeast was on Eastern time while they were actually far enough west that they just spilled over into the Central zone. I had interviews scheduled with two different companies in one client's industry, which they had approved. While we knew they were owned by the same parent company, we didn't know until the day of the second interview that both companies used the exactly same approach to communication -- the same executive oversaw the function at both places. We had to scramble to find another competitor to participate. During one study, I spent way too much time with a phone stuck between my ear and my shoulder while typing interviews verbatim. One morning near the end of the project, I gently turned over on my pillow for a last few minutes' snooze. As I did, I pulled a muscle in one shoulder. I thought it was just a pinched nerve and kept working, albeit with a pronounced droop. My doctor, upon feeling the knotted muscle, prescribed muscle relaxants and enforced bed rest -- which I complied with one-and-a-half painful days later when the report was finished. When I joined a conference call to discuss the results with the client, I did so from my bedroom. (I have since invested in a telephone headset!) With another client, timing issues beyond anyone's control nearly did me in. I had originally budgeted the project to have a lower-billing-rate associate help to gather and collate much of the data. Unfortunately, the benchmarking study and a survey report for the same client was happening at the same time, even though they were originally planned for different months. My colleague worked on the survey and I did more of the benchmarking study than expected -- or budgeted. We were pressed for time, we had to replace some participants at the last minute and I knew I couldn't recover the added costs of my billing rate. I broke out in hives, something I'd never experienced before in my life. Fortunately, the nature of my job allowed me to work unseen from home until it cleared up. Benchmarking: its rewards are unpredictable, too Some of the people I spoke with at comparator companies later became clients. When they didn't have answers to some of the questions we asked, they realized they needed better ways of gathering their own communication measurements. On the benchmarking study where we made site visits, my clients became my friends. After all those days and nights on the road together, we learned a lot about each other, not just all those companies. With another client, we won an IABC Gold Quill for the project. And finally, the best reward of all. I worked all night before one benchmarking study was due to recheck all the calculations. The client had originally been including a number of overhead costs in their budget figures that none of the other companies had included. Papers covered my dining room table. The laptop was getting overheated. I was about an hour away from being finished as dawn began creeping in through my dining room window and my four-year-old nephew quietly crept down the stairs in his jammies. I had not seen much of him for most of his life because I lived too far away, but he and my sister had arrived at our house for a visit late the night before. "Whatcha doin', Annie Angie?" he asked. My initial reaction was that I wished he had slept in just a little bit longer. I was about to ask him to go watch TV or go back to bed. I had promised the report to the client first thing in the morning. "Working on a report, Billy." "Can I help?" And then it hit me. Here was a precious moment to get to know my first nephew. If the report arrived first thing or second thing in the morning, it probably wouldn't make much difference to anyone but me and my own expectations for delivering on time. So, dear client, the final calculations for your report were made by tiny little fingers on a Mickey Mouse-shaped calculator and typed into the report excruciatingly slowly, but with great pride. And every single number was perfect. Now Billy joins me in the office of Sinickas Communications, Inc. every Friday afternoon after pre-school. He has his own folders and his own assignments. He's a whiz at the computer already. And it all began because of a benchmarking study. He used to want to drive a garbage truck. © 2000 Angela D. Sinickas, All rights reserved Angela Sinickas, ABC, is president of Sinickas Communications, Inc., a communication consultancy specializing in helping corporations achieve business results through targeted diagnostics and practical solutions. You can visit her new website, CommToolbox.com, to see the automated planning, measurement, and benchmarking tools she has developed based on her manual, How to Measure Your Communication Programs.