But, having recently endured my own personal measurement start-up ordeal (see “How I Slept My Way to Measurement Success“), it occurred to me that there are a number of pitfalls and opportunities for disappointment in the process. I thought back over many beginning measurement programs, and wrote down the most common unrealistic expectations. Here they are.
Expectation #1: You expect to have a “measurement number” that will justify your budget and answer all questions.
Reality: First, measurement isn’t a single number. Your CFO doesn’t go into the Board of Directors meeting with one number. So why would communications—made up of such myriad of different components—think it can demonstrate results with one number?
Second, measurement is a long-term process. It will take you time to identify changes and trends. Your manufacturing vice president doesn’t report on the productivity for today, he or she reports on changes in productivity month-to-month, quarter-to-quarter, and year-to-year. That’s what you’ll be doing, too.
Expectation #2: You’ve just gone through an incredibly complicated screening process to find the right vendor, and now you expect everything will be simple and easy.
Reality: Like any research project, there’s a lot of data involved. And, as we know, data is never simple, no matter your program or tool. You expect to sign the purchase order and have your metrics magically appear. It doesn’t work that way. You are going to have to put in lots more time if you want to get useful data.
Expectation #3: You’ve spent all this money, so you expect your data will be 100 percent perfect from day one.
Reality: Expect glitches in the data, especially to start. Computers, like children, take a while to learn the rules. For the first few months, you are going to be sorting out search terms and media outlet specifics, and you and your computer(s) are going to make some mistakes.
At first you will probably define search terms that are too broad, and you will be inundated with mentions that have nothing to do with your organization(s). This is what is called a learning opportunity.
For instance, in the world of social media, you’ll be amazed at what people think your name is. Just ask the folks at CarMax (the car-selling company) and CarMex (the lip balm.) Or SAS, the analytic software company, (not the airline or the Semester at Sea folks). Then you narrow your search and the next thing you know the product management types will be screaming because you missed something.
It’s probably going to take at least three months to get it right. You can shorten this cycle if you chose to work with a very clearly defined list of media outlets and specific URLs. That doesn’t mean, for instance, asking for “Business Week blog.” In that case, you’ll discover there are something like 170 of them. Which one did you have in mind?
Expectation #4: You expect results right away.
Unless you know how to do a Vulcan mind meld, you will need to spend time with your measurement vendor’s staff to clearly define the parameters of your program. You will need to define subjects, messages, campaigns, and any other details you will be tracking.
If you give your vendor the wrong messages, incorrect competitors, or out-of-date product names, they will not know it until you discover it and tell them. If it’s on your website, they will assume it to be current. And if it’s not on your website and you haven’t told them about it, how will they know it exists?
Expectation #5: You expect your first report to explain everything.
Reality: Your first report will be confusing. No matter how clear the program seems to be, your first report is going to throw you for a loop. There will be charts that use language you may not understand. There will be charts that look weird. Until you see it visually on the screen, you probably won’t understand what you’ve ordered. That’s the nature of data. And remember, some of your early data will probably not be what you expected.
People will question your data and methodology. Like any expectant mother you will need to push (back) hard. It will hurt. Odds are, you’re going to have work at it to get what you want from your measurement. Change is painful but in the end, you’ll have a beautiful product. Congratulations.