This article is based on a discussion by practioners of lean startup principles and I think it clarifies a very useful point i.e. how much should we listen to our customers ?
I remember having the same (and at that point of time inconclusive) debate around customer development process with a friend who was very inspired by the Signal vs Noise blog by the 37 Signals team.
Here's the discussion ~
Mark ~ "Hi, I just read this article (snippet below) and it has changed my opinion slightly (only slightly) on how much emphasis we should place on listening to our customers."
I'm working with a company that at one point had a product that was not only best in its class, but also technically far ahead of its competition. It created a better way of offering its service, and customers loved it and paid for it.
Then it made a fatal mistake. It asked its customers what features they wanted to see in the product, and they delivered on those features. Unfortunately for this company, its competitors didn't ask customers what they wanted. Instead, they had a vision of ways that business could be done differently and, as a result, better. Customers didn't really see the value or need until they saw the new product. When they tried it, they loved it.
So what did "my" company do when it saw what its competitor had done? It repeated its mistake and once again asked its customers what they wanted in the product. Of course the customer responded with the features that they now loved from the other product.
William ~ "Yeah, I think customers are an excellent source of hypotheses. Sometimes direct ones, when they say, "We want X." More often indirect ones, when we observe our customers and say, "You know what they could use..."
But either way, they're just hypotheses. I think the way to stay ahead of competitors is by being great at quickly and cheaply testing lots of hypotheses, and then being great on following up on the small wins to create big ones."
John ~ "Perhaps, sometimes the problem is listening to the customers solutions to the problems, rather than listening to the pain points and then testing the different possible solutions.
In the end, you have to come back to testing if that one customer who saw X as a solution, accurately represented the behavior/ expectations of the wider customer base.
Without testing that hypothesis, you would not know.
To have one customer say I want X and to then go off and build it, is only one step away from having the idea that customers want X and then going off and building it."
Spike ~ "The answer is in the article itself, in the very small bit you didn't quote:
Your customers can tell you the things that are broken and how they want to be made happy. Listen to them. Make them happy. But don't rely on them to create the future road map for your product or service. That's your job."
Roger ~ "Totally agree with you here, Spike. Customer development and problem interviews are of fundamental importance for startups. But what you ask, and what you listen for, makes all the difference. Users don't know what they want.
If they do know what they want, what they want isn't necessarily what will solve their problems. (They aren't experts on design, but they may be experts on their own situation and challenges.)
If you ask a hypothetical question, you'll probably get a worthless response.
If you pointedly ask a prospect if she is experiencing a problem, you often won't get a reliable or helpful response.
It seems many in the lean startup community disagree with me, but I believe open-ended questions yield the most reliable, new, and actionable information. Shift the conversation to, and listen for, the situation and problems. Get beyond what customers say they want."
Spike ~ "Well sometimes the community can be wrong :) . In all seriousness, I don't know if you're coming from that angle or not, but there's a growing acceptance of Design Thinking in the Lean Startup community and open questions are the bread and butter of the empathy stage.
That said it all depends on the details, I don't think open questions or open ended discovery are either good or bad, they are a tool like any other and sometimes you need to drive the conversation to validate specific hypothesis and open ended questions might just not get you there. I think that's why you're finding resistance and honestly I agree as much with that as I agree with what you said."
Jaana ~ "A great thread. Must read for product startups.
“If I had asked people what they wanted, they would have said faster horses.” (-Ford)
When I interview customers, I just let them speak and try to understand their pains and world-views. Especially the latter. When they say something that makes me go 'WTF?!?!', I know I have found something valuable.
Sometimes people have big pains that could have simple solutions, but they just can't see the solutions themselves. And that's where it's easy to build products and education that have very high value.
You'll have to trust people to know their pains the best, but you should also trust your own expertise to find the solutions. If people would know a perfect solution for their painful problem, they would have solved it already.
And when you interview people, and they haven't tried to solve the pain at least somehow already, that's a great sign that maybe the pain you are targeting isn't big enough.
To be able to sell the product, you must have a real customer pain. But you'll also need to have a competitive edge, something unique that makes you better than "that another product", and that's what you get from understanding your customers. Today we can't really compete with features - competitors can copy those in 2 weeks."
Mark ~ "Hi, can someone explain what 'open end questions' are or what 'open end discovery' is please, thanks in advance."
Tristan ~ "Hi Mark,
A close ended question is one where there are a limited number of possible answers. e.g. Which is your favorite color between the choices red and blue?
Closed questions generally aren't good in customer discovery. You miss the opportunity for someone to answer, "YELLOW!"
An open ended question then is one without a fixed answer. e.g. "Why do you like yellow?"
The discovery phase is when you are trying to discover:
Who is the customer?
What is their problem?
When do they have it?
Why do they have it?
The Validation phase is where you try to validate if your product solves that problem.