How to ask the right questions in the right way: Insights from 5 senior UX leaders

If you’re conducting user research, you need to learn how to ask the right questions in the right way.

But what are the right questions? Why are they so important?

The right questions do not influence, lead, or coax a test participant. Instead, they reveal truthful opinions and behaviors without bias. This is important because the insights from a biased test won’t lead to a true user-focused product.

To uncover how to ask user research questions correctly, we sought the advice of a variety of senior research leaders on how they frame their user research. These leaders shared how they surface the most useful and valuable insights.

Check your own bias

Vivianne Castillo, Salesforce, Senior Design Researcher

I think the hardest part about asking the right questions in the right way is being able to clearly identify any bias and internal/external influences you are bringing into the research project.

Some examples of this include (but aren’t limited to):

• Belief bias (e.g. religious, political, socioeconomic, etc.)
• Confirmation bias.
• Hindsight bias (e.g. seeing past actions by people more predictable than what they actually are; the “I knew it all along!” effect).
• Crafting leading questions in order to please certain stakeholders.

Knowing this research project might influence your next promotion.

Lately, before I begin crafting a research plan, I’ve been jotting down (yes, with pen and paper) what biases/influences I am bringing into the questions I want to ask. I ask myself questions like, “How do I view people who always desire luxury?” or “What do I believe about the leaders of Brand X versus Brand Y?”

After crafting my questions, I always run them by another co-worker or two, because I know that I’m human and am not immune to blind spots.

I find that this practice goes beyond helping me gut-check my questions: it increases buy-in and collaboration with my team and helps me notice patterns in how I ask questions, which influences how I approach future projects.

Takeaways

• Identify any bias and internal/external influences you are bringing to a research project
• Ask yourself tough and honest questions
• Write them down, then run them past co-workers to identify any blind spots

Flow and form

Bradley Miller, Honeywell, UX Research Manager

You’ve definitely hit on some of the biggest challenges UX Researchers face. Given my background as a trial attorney, I’ll focus on the topic of ensuring you’re asking the right questions in the right way.

What are the right questions? 

As UX Researchers, we need to focus on understanding user behavior. That means observing what users actually do, and attempting to understand the thought processes behind why users do or don’t take certain actions. In order to do this, first focus your questions around tasks. What are the major tasks a typical user would need to complete?

Of those, which ones are the most critical decision points for your team or business to understand? For those latter tasks, be sure to ask the user follow-up questions, such as why they made the choice they did.

What’s the right way to ask questions? 

There are two components to asking questions the right way – flow and form.

Flow refers to the order of your questions. An ideal flow follows the user’s natural path through the experience. It’s important to follow, rather than lead, as much as possible. Only by following can you observe a user’s natural decision making.

The art to doing this is to avoid dictating or suggesting a specific path whenever possible. Instead, if you’re trying to test or observe a specific portion of an experience, ask the user to imagine she is trying to complete a task which would naturally include the thing you want to observe.

Form is just as important as flow. UX Researchers should always use direct, open-ended questions, and avoid leading questions unless absolutely necessary. What is a leading question? A leading question is any question that suggests an answer. By contrast, an open-ended question does not suggest an answer.

For example, the question “Did you find this experience delightful?” suggests the participant should either agree or disagree with your statement; you’ve therefore influenced her answer by limiting and characterizing how she should respond. A better question would be, “How was the experience?” This question is entirely open-ended: the participant is free to answer in her own words.

For further reading, check out this comparison of open-ended vs closed-ended questions, or explore some leading questions examples that show you what to avoid. Allowing participants to answer in their own words will give you a higher fidelity insight into how they characterize the experience.

Takeaways

• Focus your questions around the tasks a typical user would need to complete
• Which of these are the most critical decision points for your business to understand?
• Follow the user’s natural path through the experience – try not to lead
• Avoid dictating or suggesting a specific path whenever possible
• Use direct, open-ended questions – avoid leading questions unless absolutely necessary

Tell me about…

Nikki Anderson, From A to B, Senior UX Researcher

There are a few ways to ensure you ask questions in the most unbiased and least leading way possible.

As researchers, we are constantly trying to understand the why behind someone’s thoughts, emotions and behaviors. The method I teach most of my students, which is the easiest to remember, is the TEDW method.

The TEDW method promotes conversations over interviews, by ensuring you are asking open-ended questions. TEDW stands for: “Tell me about…?” Or “Explain what you mean by…?” Or “Describe…?” Or “Walk me through…?”

Stating questions in this way ensures you are not leading the participant down a path only you care about, and allows them to answer beyond yes or no, which gets you closer to the why.

My second piece of advice is to combine the TEDW method with always asking about the past. The best way to predict future behavior is through past behavior, so asking a participant how they have done something in the past is the best way to figure out how they would do it in the future.

Takeaways

• Use a conversational style, instead of interviewing users – you can do this by asking open-ended questions
• Open-ended questions such as “Tell me about your experience?” allows users to answer beyond simple yes or no responses
• Asking participants how they have done something in the past to figure out how they would do it in the future

Start at the end

Angie Amon, Mailchimp, UX Research Manager

I start every project at the end – what do you need to solve for with this research? What decisions do you need to be able to make when it’s completed? It is critical that everyone involved is crystal clear on those things.

Having this info allows you to decide on the right methodology and then from there, the specific questions that need to be part of the research.

Whether you’re talking about qualitative questions or questions in a survey, avoiding any bias is critical. Questions need to be as neutral as possible – no using inflammatory words or phrases that might push a respondent in one direction or the other.

Here’s an extreme ‘don’t’ example: “How horrible is this website?” Consider instead a more neutral approach: “Which of the following words best fits how you feel about this website?”

I always suggest running your questionnaire or discussion guide by a neutral party who isn’t invested in the outcome to make sure that they understand the questions. There’s no confusion or compound meaning in a question. If you have an open-text question, don’t ask it in such a way that a yes or no answer will suffice because that’s what you’ll end up with.

I don’t know that you can absolutely ensure that you are always asking the right question in the right way but having a clear vision of what you need from the research is a great starting point. Ask yourself and those who need to take action from the results, “If you get a majority of people answering this question a certain way, will it be clear to you what to do or what not to do with the information?”

Takeaways

• First figure out what the research is trying to solve – make sure everyone is clear on this
• Questions need to be as neutral as possible – no using inflammatory words or phrases
• Running your questions by a neutral party first
• Be aware of your tone – even open-ended question can be asked in a leading way

Go down the rabbit hole

Samantha Alaimo, GrubHub, Senior UX Researcher

Understand why we are asking the questions that were proposed.

Start with broad questions and then narrow in on the particulars.

Think of your users as the experts, you are just there to learn from them.

Follow the users down their rabbit holes if their answers go in a different direction than you expected.

Takeaways

• Start with broad questions, then narrow down the scope
• Think of users as the experts – you’re learning from them
• Follow your users wherever they go