6 common User Research pitfalls and how to avoid them
Once executives in a company finally realize the importance of being user-centric (read this article if you want to make them aware), excitement can be really strong. They might ask their employees to start doing extensive User Research right away without providing them with the appropriate training. Time goes by and first results could turn out to be disappointing, undermining the hype around user-centricity in the company. Why? User Research is difficult, way more than what non familiar executives might think. It’s not only about sending your people onto the field to chat with your users, even if better than nothing. You need expertise in the company to coach people to avoid most frequent pitfalls that we listed below:
1/ Asking your users for solutions
2/ Believing what your users say
3/ Forgetting about the context
4/ Looking for confirmation
5/ Focusing on your product subpart journey
6/ Dealing with lookalike users
1/ Asking your users for solutions
Imagine you were asked to develop an “innovative” product (which does not sound very user-centric). What would you do? Go onto the field, tell potential users about your idea and ask them for their best solution? Henry Ford stated once : “If I had asked my customers what they wanted, they would have answered faster horses”. Asking for solution is dangerous…
Let’s have a look at an example taken from the book “Cracking the PM interview” from Gayle Laakmann McDowell and Jackie Bavaro. When Oxo, a kitchen ustensil company asked customers what was wrong with their measuring cup, they talked about the cup breaking when they dropped it or its having a slippery handle. But when they watched people use the measuring cup, they saw people pour, then bend down to read the measurements, then pour, then bend, then pour, then bend. Nobody asked to be able to read the measurements while pouring, but Oxo was able to see the need. They now sell a measuring cup with the measurements at an angle so you can see the lines while pouring.
So what is the problem with asking for solutions? Knowing what one needs requires a great deal of awareness (what triggered this behavior?), thinking (which need was I trying to fill?) and perspective (what are the solutions at my disposal to solve this need?). This is why finding the right solution is a job, and why you shouldn’t expect it comes to light only through an informal discussion with your users!
How to avoid this pitfall : Don’t ask your users for what they think they need, rather observe, and infer from how users behave what they really need, even if your users themselves are not aware of it.
2/ Believing what your users say
Think about the last time you interviewed users about your existing product. Did you ask questions like “What do you think of the new interface?” or “Do you like this design?”? Then be aware that more often than you think, your users lie. Most of the time unconsciously!
Let me share with you a compelling example that gave me Rémi Guyot (BlaBlaCar CPO) from the time he worked at Paypal. They conducted regular surveys to know on which website did users last pay with Paypal. 8% of them answered Amazon. This is a great insight… except Paypal is not available on Amazon!
Why the hell would your users lie to you? Mainly because of psychological biases! Some common biases that prevent users from being honest with you (I will detail them in a future article with tangible examples) are courtesy bias (not wanting to hurt you with negative feedbacks), social desirability (tendency to answer what would be seen as great by the others rather than what you really think is correct), familiarity heuristic (overweighting in your answers elements you are already familiar with), misinformation effect (recall of episodic memories becomes less accurate because of post-event information)..
How to avoid this pitfall: again, observe rather than ask! Watch what people actually do and don’t believe what people say they do. Make them comfortable giving negative feedbacks (“my job is to collect negative feedbacks, I am paid for that”).
3/ Forgetting about context
Guerilla testing is a popular format of usability testing. It consists in having some users from your neighborhood (eg. the Starbucks around the corner, the metro station…) perform a quick test on your product. Even if better than nothing, Guerilla testing usually fails to capture a very important criteria for User Research: context (the two others being User and Goal).
Let me give you a tangible example taken from my time at ManoMano, a DIY (Do It Yourself) marketplace. We were planning to launch an improved experience for painting category (enhanced color pick-up widget, surface calculator, samples…). We held a Guerilla test in a mall from La Défense. To what extent were these users “good” testers ? Indeed, there is a wild gap between a user faking to buy a paint can on his way to work and a user wanting to refurbish a room he lives in… The way he will make up his buying decision, the questions that will arise will certainly be very different and much more complex!
What could have been done better in this example? Probably get users with a more tangible context. How? Why not going to DIY stores? Once there, why not going to the painting department? Chances you meet users wanting to paint a room for real are very high.
How to avoid this pitfall: always try to bring as much context as possible when doing user research by selecting users that are close to your product area. To find them, always think of possible substitutes to your products. Prefer interacting with less users but better qualified.
4/ Looking for confirmations
Remember last time you did some field research. You probably already had a solution in mind that you wanted to test, didn’t you? Especially if you are the one doing both the research and the design. Unfortunately, it might have led you to the wrong conclusion.
I know very well this pitfall: when I started my first company, an app to order food ahead in restaurants and pick it up without lining, I had my idea in mind before starting my User Research. And every thing I did (interviews, surveys, observation) had a unique purpose: confirm that lining in a restaurant was a pain. I was always able to finally get people say that, because the way I asked my questions and ran the discussion influenced them to tell me that exact insight. I also overweighted some experiments I discovered on the field, like a restaurant called Tante Juliette which built its own pre-ordering website without challenging the potential drawbacks (always the same customers ordering, the restaurant owner pushing hard the solution among his clients which would not happen with a platform…). I came to the conclusion that lining was a huge pain and my solution strongly needed. When we launched, we had at best 3 orders per day in our top restaurants. We finally discovered that the real pain of restaurant “users” was the price, which I had not (wanted to) seen in the first UR iteration.
How could I jump so quickly on wrong conclusions? Two biases affected me : confirmation and generalization biases. The first one will make you overweight users’ insights that confirm your hypothesis and underweight the one that go against. As regards to generalization, you are keen to overestimate the frequency of isolated patterns that confirmed your hypothesis.
How to avoid this pitfall: Ideally try to have someone different to conduct User Research from the one designing the product. If not possible, be aware of your bias when performing your research. At every interview, keep asking yourself “is it an insight I want to get or an insight the user naturally gave me ?”.
5/ Focusing on your product’s sub-journey
How much do you focus only on your product journey, especially when trying to optimize an existing product? Your product journey is usually a sub-part of a more global user journey. Even if natural, narrowing your research only on your product sub-journey might make you miss several opportunities for your users.
For instance, if you sell paper tissues, your product sub-journey would be the moment when your user blows his/her nose. Let’s focus on a specific target: kids. By focusing only on your sub-journey, you could think of several improvements like flavoring tissues with cool perfumes (eg. cotton candy flavor). But if you take a broader view, the real journey around your tissue does not end once the nose has been blown. More probably for kids, they keep the tissue in their pocket until their trousers end up in the washing machine… And if it happened to you, you know how terrible it is to recover all the small pieces of the paper tissue… So maybe the greatest feature to add to your product would be “resistant to washing machine” (which Kleenex actually did, I found it brilliant).
How to avoid this pitfall: don’t be focused only on your product, understand how and why your users ended up there and what will happen to them after.
6/ Dealing with lookalike users
Think about last times you tested a new feature or showcased a new prototype. How often did it happen with people “like you”, same age, same background, same company, same culture or same group of friends? Some of your users can often be opposite to yourself. Not including them in your research process will definitely make you miss key insights.
I will share with you the story of Josette. She was our host during a product offsite we had at ManoMano in the countryside. Josette was retired, not very comfortable with internet and e-commerce. So quite the opposite of the whole product team (young urban digital natives). After the offsite, she gave it a try to ManoMano. She was first very pleased but she called me back a few months later to tell me she was extremely disappointed by the quality of the products ManoMano sold. She held ManoMano as responsible, which was quite logic from her perspective. I tried to explain her that ManoMano was only aggregating sellers on its platform but had little leverage on the products sold (even if ManoMano is a closed marketplace and perform quite a hard selection). The marketplace concept, that was for the product team as familiar as the air we breath, was totally inexplicable for her… And probably for many others of our users.
How to avoid this pitfall: when doing research, try to interact with “edge case users” like children, old people… It will make frictions pop up more clearly (while requiring less interviews) and will give you a different perspective on how your product can bu used.
If you liked this article, that makes me happy because it takes time to write! Feel free to clap (always rewarding), share or register to my personal newsletter
