Statistics and Lies

Let’s play a guessing game.

I want you to guess the identity of the man described below:

• This world-famous man was born in England, in 1948
• He’s very wealthy but has no “job”
• He loves international travel, expensive cars, fine wines.
• He lives on a massive estate near London
• He is now on his second marriage.

Time to guess—who is he?

After hearing the list of clues, most people would guess that the man in question is Prince Charles. However, most people would be wrong.

While it’s true that all the facts above do indeed describe the future king of England, they also fit the description of someone else: Ozzy Osbourne. Beware of conclusions based only on statistics.

Going Beyond the Statistics

NPS isn’t designed to reduce your members to mere statistics. Sure, the scoring system helps categorize members to make follow-up activities easier and more meaningful. But you still have to make the connections to get the real benefit. Each member has their own story to tell. Just like Ozzy and Prince Charles—up close, the difference is easy to spot.

Dig Deeper to Find the Truth

It’s important to recognize that the effort to measure and understand the member experience is not an expense, but rather it’s an investment in creating more profitable members. A better Mx will create happier members which can have a big impact on your bottom line.This step is the core element of information gathering with NPS, or any system for that matter. By delving deeper and exploring the “deeper why” of their answers, you will begin to uncover the root causes of member dissatisfaction within each category.

Step 1: Start With a Hypothesis

What makes a member give you a 10 vs an 8? What is the most common reason for a member to give you a “detractor” score? Before we begin digging in to find out, you should list out a few ideas of why you think members become promoters or detractors.

Whether it’s just a “hunch”, or feedback from branch staff, or the results of a previous survey, it’s helpful to have some ideas of why members become promoters or detractors to help focus your analysis.

For the quick start method, we are going to use what’s called Stated Driver Analysis. This is a fancy way of saying “talk to the members, or ask them why.

Step 2: Read comments and categorize

The NPS score helps you “pre-segment” the responses. Use the Detractor/Passive/Promoter designations and group your answers accordingly. Next, read all the answers to the “open ended” why question.

Read them with an eye for repeating themes (service issues, pricing & fees, e-services, convenience, etc.). Next, categorize responses by theme, using a tagging system like livesurvey, or whatever system you currently use to gather Voice of the Member Feedback. If you’re not using a system, put all your responses into Microsoft Excel and add a column titled “Subject”.

Step 3: Interview using the 5 whys

This is the core element of information gathering with NPS, or any system for that matter. By delving deeper and exploring the “deeper why” of their answers, you will begin to uncover the root causes of member dissatisfaction within each category.

Choose a group of members (the NPS gurus suggest you start with Detractors) and contact some of them to get more details about why they answered the way they did. This step is the core element of information gathering with NPS, or any system for that matter. By delving deeper and exploring the “deeper why” of their answers, you will begin to uncover the root causes of member dissatisfaction within each category.

Some tips on digging deeper:
• While it’s important to have pre-selected questions to start with, the “interviewer” will need to learn to listen to members carefully and base their follow-up questions on issues brought up by the member.
• Be considerate of your member’s time and try to keep follow-up calls as short as possible
• Thank members for their time and promise to inform them of any changes brought about by their input.

Step 4: Review your results and start with small changes that can have the most impact.

Whether you’re an aging heavy metal icon or the Prince of Wales, small changes can have a big impact—especially over time. In other words, by making a series of small improvements to existing processes, systems, or service methods over time – in response to uncovered member wants and needs – you’ll end up making big improvements to your member’s experience.