Angela Sinickas: A practical guide to measuring the effectiveness of executive communication

Twenty early birds took advantage of this pre-conference session presented by Angela Sinickas. She’s president of her own communication consultancy and the author of How to Measure your communications programs and a regular columnist for Strategic Communication Management Magazine. She’s a pioneer in the field of organizational communication measurement and has been measuring the effectiveness of communications since 1981.

This was a great session for corporate speechwriters, communications managers and internal communications staff who have faced skeptical managers with one eye on their function as a potential ‘cost center’ which practices the squishy art of wordsmithing and does little to contribute to the bottom line.

It’s all very well to say speechwriting is an art and the measure of good speechwriting is a satisfied client. But what do you say when your executives tire of the road—talking to yet another group on yet another topic—and ask if it’s really worth their time … or worth spending all that money on speechwriters?

Angela promised to outline simple research techniques that can give ammunition for anyone who needs to show executives the business value of staying in front of all those key stakeholders. The class brochure made some pretty impressive claims, including:

A quick poll at the start of the class showed that we wanted answers to a variety of familiar questions including how to meet requests for ROI measurement; how to filter a multiplicity of speaking requests; how to prove worth and value of exec comms; how to make what we do seem more valuable; quantify performance; gather the right metrics; measure the impact of new media and more.

Planning and Measurement

Angela began by advising us what NOT to measure:

Better to measure:

When measuring communications objectives move down this list for better results:

1. Activities: Just measuring the number of speeches
2. Attitudes & audience perceptions
3. Behavior changes – actions taken
4. Financial impact on organization goals

Just by asking these behavior change questions can help you clarify the value of a speech; should the executive even do it? These questions will highlight if there’s a call to action included anywhere in the speech outline.

Follow a linear path of reasoning to step up your appeal to the left-brained engineers and scientists we often support. It helps them see how the holistically right-brained speechwriter arrived at the messages we script for them:

1. Identify stakeholder groups
2. Identify their “stake�
&#160 a. Why are they a stakeholder?
&#160 b. What group are they part of?
3. Identify their current behavior and ideal behavior to reach goal
&#160 a. Are they customers already?
&#160 b. Are they customers of the competition?
&#160 c. Have they ever been customers?
4. What knowledge affects their current behavior, where do they get this knowledge?
&#160 a. What messages and basic facts affect current knowledge? What do their heads need to know?
&#160 b. What current channels deliver facts in best way?
5. What attitudes affect current behavior and where are these attitudes reinforced?
&#160 a. What attitudes affect messages? Above and beyond ‘facts’ what do their hearts tell them?
&#160 b. What channels affect attitudes: compelling face-to-face interaction, perhaps emotive video?
6. What knowledge and attitudes affect ideal behavior, which sources are preferred?
&#160 a. Same as for 4a/b and 5a/b for Ideal future
&#160 b. What would it take for you to change this behavior?
7. What research will help answer these questions?
&#160 a. This does not have to be formal research.

Angela then had use divide into small groups and experiment with the technique. The feedback given afterwards was:

Post-event surveys can gauge how an audience’s knowledge, attitudes and behavior has changed.

Measurement can be made along the axis of:

1. Satisfaction – “Do you like me?â€?
2. Efficiency – time and budget, in context
3. Effectiveness – do messages achieve objectives?
4. Outcomes – change in objective reality, not just opinions.

And across:

1. The activities of the communications event
2. Audience Perceptions
3. Changes in audience behavior
4. Financial impact

Tools to use

A simple trick is to turn on the Flesch-Kincaid Grade Level test for readability in Microsoft Word under [Spelling and Grammar] [Options] [Show readability statistics] and make sure you are writing at a grade level sufficiently clear enough for members of your audience to understand. (This blog entry scores a 51, right around where Time Magazine sits.)

Speechwriters can also measure number of times concepts like ‘Brand’ are used in an annual count of speeches or articles. Simply do a word count for key brand attributes, examine the totals in a given period of time as a percent of total communciations for that period.

Interviews and focus groups can help ID messages that people want to hear. Quantify on a scale of 1-5.

Conduct baseline surveys before a speech – allow respondents to see what others think.

Knowledge quizzes should always include a ‘Don’t know’ response.
Take pressure off them feeling dumb in the way you phrase questions.
Do this at the event or online via web. Online surveys can let them click after they respond to see others’ results.

Use the well-respected Starch test from the field of advertising to measure the effectiveness of different forms of communication:

Soon after speech ask if they:

to different segments of the presentation.

Measuring different channels

Measure pure volume – how many speeches are given at different times of year, measure percentages each quarter. Ask: Are these at best time for our industry and business?

Measure content pf speech against things like revenue. This is more sophisticated than just measuring content in terms of brand attributes.

Measure preferred and ideal sources of information.

Outcomes

Use demographic questions to see which variety of executive communications the audience has been exposed to. Then look at levels of favorable results and compare and contrast those who’ve been exposed to specific communications.

Measure percentage differences of those exposed to different forms of communications.