How to win arguments when data trumps intuition
Making decisions can be a daunting task, especially when data and intuition seem to be at odds. In today's world, data is king, and it's easy to get lost in the numbers. However, it's important to remember that intuition and experience can be just as valuable in guiding decision making. In this post, we'll explore how to win arguments when data trumps intuition.
I was recently faced with a dilemma. A project I’ve sunk the past three months into leading was not proving successful in production testing based on our success metrics, despite the fact our early qualitative feedback and competitive analysis validated our approach.
We made a series of strategic moves to modify the module and tweak characteristics to try to move the needle and we came within a 1% delta. Sadly, it was 1% in favour of the control, so technically we’d exhausted our options and failed. But this week I’m shipping that feature… why? Because my confidence in our customer’s desires, our intuition and prior experience leads me to believe it’s the right decision.
On customers
First and foremost, it's important to listen to the customer. Customers can often provide valuable insights into their wants and needs. By gathering feedback from customers, decision makers can gain a better understanding of what their target audience is looking for. For example, a company may conduct surveys or focus groups to gather feedback on a new product or service. This feedback can then be used to inform decision making and ensure that the product or service meets the needs of the target audience.
Steve Jobs famously declared that people didn’t know what they wanted until they saw it.
Some people say give the customers what they want, but that's not my approach. Our job is to figure out what they're going to want before they do. I think Henry Ford once said, 'If I'd ask customers what they wanted, they would've told me a faster horse.' People don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page.
That’s certainly true sometimes, especially when you’re breaking new ground, but most of us work for companies where there’s a lot of known knowns. When your data conflicts with what other established products are doing, rather than assuming you got it wrong and need to rethink your approach, instead focus on stress testing your experiment design. There’s a high chance the delta between data and intuition is driven by poor experiment design and issues in customer allocation or the analysis.
On intuition
Trusting your intuition is also an important factor in decision making. Intuition can be a result of subconscious processing of data points. While it may not always be easy to explain why you feel a certain way, it's important to trust your gut instincts if you can source validation for those feelings.
For example, my intuition led me to make a call on how best to present an interface for adding instant bank transfers when I worked at Moneybox. We did qualitative testing, but truthfully when it comes to finance, people will tell you what they think you want to hear. It’s a sensitive subject and testing to see if your UI will help them deposit large values is almost impossible in that environment. Everyone is like “oh yeah I tap here, I’ll deposit £1,000, then go to my bank… etc”. So what do you do?
Well I trusted my intuition and we shipped the feature via feature flags on a progressive release from 10-100% over the course of a month whilst monitoring deposits. And you know what, it was a huge success. We saw the largest singular deposits in our history. Boom.
On experience
Drawing on previous experience is a powerful tool that can help individuals make informed decisions. By reflecting on past successes and failures, individuals can gain valuable insights into what works and what doesn't. This knowledge can be used to inform decisions about how to launch a new product or feature. For example, if a particular approach was successful in the past, it may be worth considering using a similar approach for a new project.
In addition to drawing on personal experience, competitive analysis can also be a valuable tool in decision making. By analysing the competition, individuals can gain insights into what has worked well for others in the industry. This can help inform decisions about how to approach a new project or product launch. For instance, if a well-established player in the industry has had success with a particular approach, it may be worth considering mimicking that approach and relying on post-release analysis to determine success.
It's important to note that while mimicking the competition can be a useful strategy, it's not always the best approach. It's important to consider the unique strengths and weaknesses of your own organisation and tailor your approach accordingly. By combining insights from previous experience and competitive analysis, you can make informed decisions that are more likely to lead to success.
When it comes to decision making, it's important to consider multiple factors. While data is important, it's not the only factor to consider. By listening to customers, trusting intuition, and drawing on previous experience and competitive analysis, decision makers can make informed decisions that lead to greater success. So the next time you're faced with a decision, remember to consider all the factors at play, don't be afraid to trust your instincts, and fight for your customers if you truly believe it’s the right thing to do.