The Most Frustrating Things About NPS Scores

After enticing someone to take a survey, you want to make sure they finish it—and that they provide good data. Yet, the smallest things can derail even the best surveys.

So, beyond preventing survey fatigue, your credit union has another task:

Combatting unclear or ambiguous questions.

What Does an Ambiguous Survey Question Look Like?

Ambiguous survey questions lead to interesting interpretations by the survey takers. Two people who read the same question and had the same experience may answer differently because a word or phrase throws them off.

Unfortunately, ambiguous survey questions aren’t as easy to spot as simply looking for a particular word, phrase, or sentence construction. You have to look a little bit closer. Here are some ambiguous question examples. (The questions may not look ambiguous at first. But there are many assumptions that we in the industry make—those assumptions aren’t always made by our members.)

What is your primary financial institution?

It’s an easy question that most of us would feel comfortable answering. However, it uses what could be considered jargon with financial institution. Some people may not understand the term—they may not realize that you’re a financial institution!

Also, PFI is a complicated issue in and of itself. Is it where most of their money is deposited? That could include investment accounts in fintechs such as Robinhood and Acorns.

Is it the issuer of their most commonly used card? That could be Capital One, Amex, or Discover.

How about where they look first for loans or funding opportunities? Maybe it’s your credit union, or maybe it’s the dealership selling the car—or the payday loan shop down the street.

If you want accurate information, you’ll need to ask very specific questions!

How likely are you to pay with a credit card?

Most in the industry now agree that “paying by card” doesn’t mean exactly what it used to mean. The line between card payments and mobile/online payments is blurry at best.

When you save your payment information with an online merchant (such as Amazon), your payment is still a card payment. Subscription payments are also card payments. But will the survey respondent think of them that way?

Also, the term “pay” seems a little vague. A person may be more likely to pay for a flight with a credit card, but more likely to buy Doritos and Gatorade with a debit card.

So, the answer to that question depends—and that changes the way people interpret and respond to it.

What About Standard Survey Metrics?

Sadly, even standard survey metrics and questions can be open to interpretation. Consider the classic relationship NPS question:

On a scale from 1–10, how likely are you to recommend Fancy Credit Union to a friend or colleague?

The truth is that most people don’t understand how NPS surveys work. Many, many, many people will answer realistically, as though they think the survey asks if they have regularly have conversations about their bank or credit union.

Almost nobody does.

So, they’ll answer “1” on that scale of 1–10, because they assume that they’re not going to walk up to their friend and say, “hey, you know what I recommend? This credit union.” Because they won’t. Why would they? That would be weird.

But they’re not seeing the question from the right angle. The real question is more like:

“If someone asked a local Facebook group which financial institution they should join, and if you felt so inclined to answer, would you recommend your credit union, stay silent, or warn them off?”

Of course, people don’t realize that’s what the question is really asking, so they answer their own interpretation of it. And their interpretation is wrong. Consequently, your precious, industry-standard metric gets messed up.

Is that enough reason to distrust the NPS system? A little bit, yes, it is.

But there’s a better lesson to learn than that.