The art of interviewing in Product Discovery
Interviews are a key component of Product Discovery. Still, many PMs (Product Managers) don’t handle them properly and miss critical insights, or worse, collect corrupted insights that will ruin the Product strategy. Most of them can’t be blamed : they come from Engineering or Sales backgrounds where interviewing is not part of their courses.
First, let’s clarify what the goal of a Discovery interview is: deeply understanding a user’s context. Not finding solutions, neither surfacing user’s problems (even if both of them can pop up during the interview).
The aim for PMs is to put themselves in the user’s shoes : what do they experience ? How do they feel ? Which improvement in their life do they try to achieve ? What are the substitutes they use ?
Interviewing requires a huge amount of humility. PMs must first put aside their ego because let’s be honest with you dear PMs, we don’t care about your brillant assumptions, only do we care about our the user’s context.
People are obsessed with biases when talking about interviewing but I don’t think they are the key to conducting impactful interviews even if they matter. I will try to present another approach based on other frameworks like NVC (Non Violent Communication) or NLP (Neuro Linguistic Programming) coming from the coaching field.
Key takeaways of this article:
- Accept there is no objective reality
- Free the words
- Listen actively
- Hunt down language shortcuts
- Ask for behaviors rather than opinions
- Go up to the Need level
- Mitigate only one biais (confirmation)
I tried to provide as much tangible examples through transcripts to help the reader understand.
1/ The problem : there is no “objective” reality
In the 19th century, Kant said : “We see the world and things not as they are but as we are”. So, it’s been a while since we know that no matter how hard we try, we will never be totally objective when conducting Discovery interviews because we alter the reality we see and hear.
NLP in the 70’s developed a strong framework to explain the transformations our brain operates on the reality we experience.
- Perception : our senses alter what we see. An extreme exemple is a color-blind person unable to distinguish between colors… But you can also think of these pictures that you can see in two different ways (eg. the chin of a young lady being the nose of an old woman)
- Processing : our beliefs impact the way our brain turns a signal into a memory in our brain by giving a certain meaning to it (ex. a Coca Cola can will be described by some people as a “red cylinder” while others will talk of “symbol of capitalism”)
- Rendering : when rendering the information to others, mainly through language, human language will again “transform” the informations. Inuit people have 16 words to describe “snow”. So if we talk to them about “snow”, they will just say “Ok, but what kind of snow ?”
This transformation happens twice : first between what the users did and what they said they did. Then between what the users told to the Interviewer and what the interviewer will say about what the users said (they did)…
Let’s have a look at a real exemple outside the world of Product Management. The User is a member in a golf club who says he has been robbed (one of his golf clubs). He complains to the Interviewer, the club owner. A few weeks later, the User finds his club: in fact, he wasn’t robbed, he just lost his club.
Even if in the first place he genuinely believed he was robbed, when he meets the club owner and is asked about wether or not he had news about his club, he keeps saying he was robbed. Why ? Because he is ashamed to admit that in reality he just lost his club…
The golf owner is now convinced that there are thefts in his club. Since he was already very suspicious by nature because he got robbed himself several times, he amplifies the story within the other club members creating a sense of unsafety when in fact… nothing was stolen !
Why is it such a big issue for Product Management ? If you get a wrong sense of a user’s context, needs and problems, you will be misled for the rest of your product lifecycle (Strategy and Delivery). No matter how well you deliver the solution, it won’t work.
We will now see how to deal with this immutable truth…
2/ Free the words
What is the point of interviewing a user if you don’t give them the chance to fully express what they think ? PMs spend a huge amount of time finding users, scheduling interviews, shaping a questionnaire, yet they often jump too fast in the interview. This results into the user providing the interviewer with poor insights.
Why ? Mainly for two reasons :
- User won’t be “effective” if no framing
- User won’t be honest if no relation
First, frame the interview to engage the user and then establish a relation to build trust.
As an interviewer, you want to be thankful of the time participants afford you. The user is the VIP, not you, so don’t talk about yourself except if you are asked. You can talk about your role, especially to give context to the interview. Show them how their words can be important to you and your team. Explain your role, most users are not accustomed to being interviewed. Stick to simple words, no jargon like “Product Manager”, “Discovery”, “User Research”. To avoid any courtesy bias, make it clear that your role is to hear what hurts, potentially that you are not the one who created the product. Take your time. Synchronize (verbally but also gestures)
Hi dear user ! Thank you so much for your time, I know you are very busy, so I am grateful you could make it for this interview. I work on MyProductName, trying to improve it by understanding what users like you need. I need 30 minutes of your time to better figure out what you experience when using our product (or performing a given task you PM are interested in). Most of the time I will be listening to you. I might ask you some questions : there are no right answers, just share your experience ! (if the product exists, you can add : I did not conceive MyProductName, so feel free to raise concerns, this is my job to collect this kind of feedback !
If you fail creating this trustful relationship, users won’t be honest when dealing with issues where shame or money are at stake. So you will miss the real insight. Let me give you an example.
The User, a friend of mine, let’s call him Paula, asked me to introduce her to another friend (let’s call him Jean, the Interviewer) who rents his secondary house to relatives.
They had a chat about the house and at the end Paula asked for the price. Two weeks later, Jean called back Paula to know if she was still interested in renting the house. Paula said that finally, he found the house was too far from the beach.
In fact, Paula had told me that she found the price was overvalued but she was not able to confess it to Jean. So Jean was left with the insight that his house was too far from the sea, which was wrong, while still ignoring that the price was too high.
What could Jean have said to surface the real insight ?
Paula, I get your point about the location. As I am not familiar with the renting market in the area, do you have any other feedback about my rental that could help me rent it more easily ? It could be about price, the condition of the house or whatever ?
You could say that by asking so directly the question, we influence the user. But by asking, we don’t put too much pressure on the user, rather we help them to free their words.
3/ Listen actively
How can you not listen actively ? Mainly by interrupting the user, wether to show them that you knew what they were about to say, or by trying to convince them (that your product idea is a good one). As you can see, ego is the enemy that lies behind these behaviors. By doing so, you prevent the user from going deep into their memory to reveal what they really have in mind.
So how to listen actively to a user ? You goal is to try to understand what the other person wants to say, not convince them. There are two critical practices to actively listen and make deep insights surface :
- Leaving blanks
- Rephrasing
Leaving blanks can make you feel uncomfortable at first because we are not used to it. But it allows the user to access deeper memories or thoughts. Where your user is looking at is a great sign wether you should leave a blank or not… Should he look up / side / down, it indicates that the user is processing memories in his mind (visual / auditive / kinesthetic). You should NOT interrupt him then ! When their eyes are back in contact with yours, you can start talking again.
Rephrasing might also sound awkward. You can feel like you are a parrot because a good rephrasing is accurate (think about synchronization). By rephrasing their exact words, the user will feel truly listened and it will mirror their words and usually will make them rethink about what they just said, going one step further. Sometimes you might want to rephrase in your own words, be sure then to start your sentence with “If I got what you said, what you mean is [your rephrasing], is it correct ?”
See the difference between these two interviews where we try to understand why a given User does not use Google Maps reviews to choose a restaurant (at least that’s what he says…).
First version (the “bad” one) :
User : I never use Google Maps for restaurant reviews.
Interviewer : What do you use instead ? [focused on the solution, did not try to understand the context]
User : The Fooding. It’s a great app with very good restaurants.
Interviewer : What do you like best in the fooding app ? îInterviewer probably has in mind something like “that we could do for our own app…”]
User : The fact that reviews are written by professionals…
Interviewer : [interrupting] Do you like the fact that there is only one review per restaurant ?[the User never said anything like that, it’s probably interviewer’s mind and he thinks the User is like him]
User : Yes I like it [in fact the user probably got influenced, perhaps he does but the way we got this information is not valid]
At the end of this first version of the interview, the interviewer is left with the impression that the User does not use Google Maps for restaurant reviews but only the Fooding.
Second version (the “good” one) :
User : I never use Google Maps for restaurant reviews.
Interviewer : [rephrasing] You never use Google Maps for restaurant reviews ?
User is thinking [interviewer can know that from the look of the User that went up and did not move]. Interviewer is leaving a blank.
User [after 10 seconds that seem very long to the interviewer but not to the User] : In fact I did, the other day.
Interviewer : [rephrasing] You did the other day ?
User is thinking [again, his eyes are going up].
Another blank from the Interviewer
User : Yes, when for lunch time, I needed to find a restaurant in a place I didn’t know between my office and the office of the person I was having lunch with…
Blank
User is thinking (always the look).
User (his look coming back onto the Interviewer) : So I went on Google Maps to find a restaurant. And I read the reviews. In fact, I frequently use Google Maps reviews for lunch time… I didn’t even realize how often I used it for restaurant reviews !!
Interviewer : [rephrasing] You use google maps for lunch time ?
User : Absolutely [User is feeling listened to]. But for dinner, I use the Fooding app…
In this version of the interview, the interviewer collected way more insights only by leaving blanks and rephrasing ! The User did bring the answers by his own, simply because he was left the time to go deep in his mind.
4/ Hunt down language shortcuts
In our every day talks, our brain operates multiple shortcuts when rendering information through language. Otherwise, it would be too much of a cognitive load. In the context of User Research, these shortcuts might prevent you from gaining access to key insights. If you don’t challenge users’s language shortcuts, you may only surface users’s journey and drivers. You will certainly get low level insights. I like the Meta Model in the NLP framework that highlights three main types of shortcuts:
- Generalization : they are the easiest ones to spot. Words like “never”, “always” are generally a good sign. You can challenge them by asking something like : “Always ? Really ? It never happened differently, even once ?”
- Deletion : these ones are quite easy to spot as well. They are expressions that are very vague like “It was not so good”, “It was ok”, “I feel better”. You can question them by asking for more context : “What makes you feel better ? Compared to when ?”
- Distorsion : they are a bit more difficult to spot. Distortion occurs when something is mistaken for that which it is not. For instance “The hiring manager never replied to me, I will never get the job”. You can question them by asking “what demonstrates that you won’t get the job in the fact that the hiring manager did not answer ?” Indeed, the hiring manager might just be expecting the final results of another candidate or is about to close a more urgent recruitment…
Try to find the language shortcuts in the following examples and try to question them by not jumping too fast on the Interviewer part. These quotes are taken from some interviews I made with Head Hunters for my new project WILL.
User : The HeadHunter job changed a lot… It’s harder. I chose it to be independent, autonomous because before that I was a TAM (Talent Acquisition Manager) in a company. I felt as being under the control of the company…
Interviewer : [spotting a deletion, trying to question to get more helpful insights] “It’s harder” ? What makes it harder ? Compared to when ?
User : [thinking, he needs to make an effort to precise his own deletion]. Compared to three years ago when I started… I feel like I lack consideration from the candidates. [The interviewer got a far better insight].
Interviewer [rephrasing] : You lack consideration from the candidates ?
User : It’s very difficult to get in touch with candidates, to get an answer from them… They are hunted down by so many people. They don’t care about our messages because they don’t reply to them. It is such a pain to just be heard, I feel tired…
Interviewer : What makes you think that because they don’t answer your messages, they don’t care about them ? [this was clearly a distorsion].
User : [thinking] Well, you’re right… They might just lack time to answer my messages because most candidates I hunt receive a lot of messages… Or perhaps they just don’t have anything to say if they are not interested. After all, they hardly know me, I am the one reaching out without any notice… In fact, I realize that there was not a single candidate that asked me not to reach them out, even if they don’t answer…
Interviewer : Not a single candidate ? Never ? [this is a generalization, questioning it might bring some precious insights about the exceptions].
User : [again, thinking, searching for exceptions]. In fact very rarely some do. But most of the time they don’t, because as one of them told me once, it’s a way for them to assess their “attractiveness” and not miss the perfect match, should it ever show up. I guess it’s kind of a FOMO thing (Fear Of Missing Out)
The Interviewer got a very precious insight in the last paragraph about some needs of the candidates :
- Assess their attractiveness on the labour market
- Monitor outstanding opportunities even if they don’t feel like moving
5/ Ask for behaviors rather than opinions
I am often asked if “closed questions” are a problem. My answer is that if they relate to a behavior, they are ok. But if they relate to an opinion, I have concerns. It turns out that closed questions are not the problem but asking about opinions is. People can’t lie (at least less easily) about their behaviors whereas opinions do not convey any real value per se. So every time you can, ask about the behavior that lie behind an opinion to get proper insights.
Let’s take a real example. Imagine that you are conducting interviews to discover wether or not there is a need for ordering ahead one’s food in takeaway restaurants (which is a solution).
Let’s see how asking for opinions can be counter productive :
Interviewer : How do you feel about lining at lunch time ? [note that this is an open question]
User : Bad. I hate lining. [Of course, who the hell likes lining ? But when he really lines, does he feel so upset ??]
Interviewer : What if you could pre-order your meal to skip the line ?
User : That would be amazing ! [by jumping directly to the solution, we totally skipped the context]
Interviewer : What would be the most important thing for you in such a pre-ordering app ?
User is thinking
User : [still thinking, the question has an important cognitive load]. To be able to pay online. [In fact, it’s almost a random answer. if you would ask the same question 2 days later, he would probably answer a totally different answer, like “meal still hot”. Recency bias].
You can see that the User gave us an insight that probably has no or little value since only based on an opinion.
Let’s explore a better way to address this interview based on behaviors :
Interviewer : Can you tell me about your lunch time at work ? How does it happen ? For instance, when did you last go to lunch with colleagues for a take away ?
User : Yesterday.
Interviewer : Where did you go ?
User : We went at Elgi (nice pasta and salad bars in Paris).
Interviewer : How did you end up choosing this restaurant ?
User : the food is gorgeous. The price to pay is to wait a bit but yesterday we had time so we took advantage of it.
Interviewer : You said the price to pay is to wait [rephrasing], what do you do while lining with your colleagues?
User : We just chat. We don’t have that many opportunities to chat during the day.
Interviewer : And what about the day before ? You did not go for take away. What did you do instead ?
User : We were in a hurry so we ordered delivery.
We collected in this case many insights that have much more value because they are based on behaviors : it seems that users don’t really care to line (“we chat, we don’t have so many opportunities”) and that if they really are in a hurry, they would order a delivery meal and not even leave the office…
6/ Go up to the needs level
I talked about the Product Pyramid in a previous article so I will shortly reintroduce it : we generally talk about solutions when having product discussion, when it should be the conclusion. So your aim as an interviewer is to climb up the pyramid : the solution you are talking about is solving which problem ? That relates to which need ? That appears in which context ?
Let’s see the following discussions about an interviewer conducting discovery research about a gym.
Interviewer : why did you decide to go to the Gym ?
User : I want to lose weight [Solution]
Interviewer : what made you want to lose weight ?
User : I need to run faster [Need] ! Weight [Problem] is definitely slowing me dow…
Interviewer : You need to run faster ? Are you preparing a run ?
User : Yes, a marathon, my first one !
Going to the gym was a solution to lose weight (problem) so that to be able to run faster (need) during the marathon (context).
But the User could have had a health issue because he was too fat. Or just wanted to lose weight to look fine on the beach next summer…
So it is critical that the interviewer be able to level up to the “Context” level of the pyramid to get valuable insight.
7/ Mitigate only one biais (confirmation)
There are tons of articles about biases. There are also dozens of biases. If you want to know more, this article from Buster Benson is one of the most useful. I attached below his cognitive bias map to give you a hint about the number of biases, don’t feel overwhelmed !
To me, it is irrelevant to try to learn them all in order to fight them. Just be aware they exist, and have a look at them from time to time to improve your self-awareness. I think the most dangerous bias in Discovery is the confirmation one.
Confirmation bias ruins your Discovery effort mainly for two reasons :
- It makes you influence the user to hear what you want
- It prevents you from hearing what you dont want
I really like this video to demonstrate that Confirmation bias can make us blind about a basketball team making passes (watch it until the end, even if you know about it).
I have another example that always strikes me when I recall it. It was the time I was CPO at ManoMano. ManoMano’s COO organized a nice talk with Zalando’s CPO. At the end, we both wrote up an executive summary of our notes. I suggested to exchange our executive summary before sending a compiled version to the rest of the C-suite. I couldn’t believe it when I read the COO version : none of her three bullet points matched mine ! We heard exactly the same discussion but she had her confirmation bias, I had mine, and it resulted in two totally different executive summaries ! So two different product roadmaps if you translate this experiment in a Discovery process… At least we had a great discussion to converge on which elements we should emphasize in our joint version.
Let’s shift to the most important : how to mitigate this bias (again, avoiding it is impossible) ?
There are no magic solutions, what works best is :
- Pairing : interview by pair, one as observer, the other as interviewer. It will bring two different perspectives. Try to pair with “non similar” people to maximize perspectives
- Awareness : before interviewing, try to become aware of your biases. What do you want to hear ? (because you always want to hear for something special). What do you think the User will tell you ? By becoming aware of these elements, you give you a chance to mitigate them strongly.
I would like to thank Axel Sooriah who spent (quite a long) time to proof read this article, especially the English language !
If you enjoyed reading this article, feel free to subscribe to my Newsletter (every quarter). It talks about Product, Business, HR, Management… (mix of French and English articles). If you want to have a look at the kind of articles I write, feel free !
If you want to learn more about Discovery, I created a 5 sessions training that keeps improving after every session’s feedback. The average mark is 9,5 out of 10, dozens of respondents coming from top French scale ups.
