Sometimes, a credit union asks us why their NPS is so low. Or why it’s so high. Or why their response rates are down. Or why so many people abandon their survey minutes after starting it.
Almost always, the answer is related to how the survey was administered. That low (or high) member satisfaction score might be related to sampling bias. The response rates might be due to when you send surveys and how you word them.
And it doesn’t stop there. There’s a chance that you’re not surveying the right people, too. Read on to learn more about this type of sampling bias!
A Quick Review of Sampling Bias
What is sampling bias? It’s the tendency for results to get skewed or inaccurate because of who was surveyed, how they were surveyed, or how their results were analyzed.
The classic credit union example is with NPS or member satisfaction scores. Here are a few ways that sampling bias may affect your credit union’s NPS:
- Uninvolved, indirect members may lower your NPS;
- Members in collections or whose accounts are in bad standing may lower your NPS;
- Younger members who prefer digital-first banking may decrease your NPS;
- Older members who don’t bank digital-first may increase your NPS;
- Members with multiple accounts and products may increase your NPS; and
- Members who’ve had accounts for a longer time may increase your NPS.
Those examples are just the tip of the surface for sampling bias. There are many other ways to introduce sampling bias by asking the wrong people.
But sometimes results aren’t “skewed” as much as they are “hard to find.” Here’s what sampling bias looks like when it introduces noise.
The Wrong People to Survey
Who are the wrong people to survey? It depends on what kind of information you need.
But when your credit union needs member feedback about a specific product or service, the wrong people might be obvious. Don’t survey:
- Members who are ineligible for the product or service;
- Members who are unlikely to need or use the product or service; and
- Members whose feedback might be similarly unhelpful for any reason.
For example, if you want to offer a premium credit card, you might not want to ask members with poor credit, member with delinquent loans, or members who use alternative payment strategies like BNPL.
Similarly, if you’re worried about interest rates on loans, ask the right people. Ask people who are currently shopping for a loan or who are both eligible and likely to shop—don’t ask the retired baby boomer, because they’re unlikely to provide useful feedback.
Reducing Noise in Survey Results
Surveying the wrong people can skew the results. But even if they don’t skew the results much, they can water down the results that you actually care about.
Imagine getting 100 survey responses, but half of them are from people whose feedback won’t help. Responses from the wrong members make it harder to find the information you’re looking for.
The best way to reduce noise in survey results is to survey the right people. You must know who cares about which things. Then, you much ensure that your surveys are crafted towards those people. Finally, you must send your survey to only those people. Leave the rest out of it!
Surveys at Your Credit Union
Are you ready to begin sending surveys at your credit union? LiveSurvey is a digital survey platform built specifically for credit unions.
LiveSurvey allows credit unions to send automated surveys after specific events, manage ongoing NPS drip campaigns, and get you started with several out-of-the-box surveys.
