The following article appeared in
Strategic Communication Management, April/May 1998
Melcrum Publishing Ltd, London

© 2000 Angela D. Sinickas. All rights reserved.

.

Alternative ways to Measure the Effectiveness of Your Intranet Sites

By Angela D. Sinickas, ABC

Q:  I've been counting hits on my intranet site, but are there any other ways of measuring the effectiveness of electronic communication channels?

When you measure hits on inter/intranet sites, you are measuring overall volume of usage -- how many times parts of your site have been opened. However, hits don't distinguish between the opening of an entire page or a single illustration.

There are many additional ways of measuring usage. However, measuring the "userability" of a site is just as important in order to improve usage numbers.  But the first place any communicator should start when measuring the effectiveness of electronic communications is to identify the original objectives for putting something on-line. Conducting some baseline audience research upfront to make sure your electronic solutions will be as effective as possible and then measuring afterward to see if the intended objectives are being met.

Measuring Outcomes Against Objectives

Some companies have no strategy for using electronic communications. They simply put everything they've done on paper onto a computer network. However, it helps to be clear on why you want to put different types of information on your electronic sites so you know what to measure.  You can also publish those objectives as part of your site and invite audience comments as part of a dialogue on shaping the site to best meet their needs.

The following list suggests possible objectives for putting different kinds of information on-line and examples of outcomes that could be tracked against those objectives.

• Improve timeliness:  count the number of days it used to take to provide updated policy manuals, phone directories, forms, etc. versus the time it takes to update them on-line.
• Improve productivity:  make intranet access available only to some groups of employees but not others in similar jobs but in different locations. Measure the productivity of both groups before and after the intranet becomes available to one group to see what difference it makes.
• Increase participation:  track how many employees applied for internal job transfers before and after job postings became available on-line.
• Increase profitable sales:  compare how effective your internet Web site is versus other marketing techniques on identifying sales leads, percentage of leads that turn into sales, time lag between lead identification to a sale, cost per lead and cost per sale.

Other types of outcomes that could be tracked include cost reduction, error reduction and wider sharing of intellectual capital.

Measuring Usage

A number of software packages are available that track how visitors access your site, providing far more useful information than just how many hits you're receiving.  Some of the types of measures available include:

• The most and least requested pages;
• The pages most frequently used as a point of entry to and exit from your site;
• The most common pathways visitors follow through your site;
• The most frequently downloaded files;
• Activity levels by day of week and by hour of day;
• Most frequently used browsers;
• Most frequently used operating systems;
• Number of user sessions;
• Average length of user sessions.

These kinds of information help you manage your site and make decisions about where to focus your resources.

Exhibits One, Two and Three are adapted from measurements of a U.S. financial services company's human resources intranet site, which can also be accessed from employees' home computers. Among the findings this company learned from usage statistics were:

• 15% of user sessions occurred during the weekends;
• Nearly 45% of user sessions took place outside of normal working hours;
• As many users were coming in through a combination of America Online and CompuServe as were coming through the company's intranet directly;
• The average user session lasted just over seven minutes, which meant that employees were not losing that much productive time when accessing the site at work.

Measuring Usability

In designing and refining your electronic sites, you should consider testing how user-friendly they are. This can be done before the site becomes available and as you add more pages to your site.

One approach is to develop a list of the most likely types of information your audience will want to find on your site. Then develop a "quiz" that asks people to find the right answers to questions by accessing the site.

Select a group of individuals who are a representative cross-section of the people you will expect to use your site. As they complete the quiz, you can track how long it takes them to find each answer and the pathways they use in trying to find the information. You can also debrief them afterwards in a focus group discussion. These usability tests will provide you with invaluable advice in fine-tuning your site, which may also improve your usage measures.

Other Measures

Remember that electronic communication shares many characteristics with print communication.  Anything that can be measured about publications may be adapted to inter/intranets, such as reading grade level tests of the writing, content analysis and adapted Starch Tests of aided and unaided recall (see last month's issue of Strategic Communication Management).

You can also build in one or two multiple-choice questions at the end of various pages on your inter/intranet sites asking your visitors to evaluate some aspect of their usefulness.  Questions can be answered by clicking the mouse on "radio buttons." These readership survey questions might include:

• How interesting or useful the information was;
• Appropriateness of the length;
• Ease of reading;
• Use of graphics.

Software for this type of measurement can provide real-time tabulations on how visitors are responding to questions.

Angela Sinickas, ABC, is president of Sinickas Communications, Inc., a communication consultancy specializing in helping corporations achieve business results through targeted diagnostics and practical solutions. You can visit her new website, CommToolbox.com, to see the automated planning, measurement, and benchmarking tools she has developed based on her manual, How to Measure Your Communication Programs.