Consider these variables before you choose a UX research method

Do any of these statements sound familiar?

• “We have a new product idea; let’s run a focus group to find out what people will think!”
• “We’re not sure if people understand how to navigate our site. We should set up a moderated usability test.”
• “Let’s do a 5-month-long longitudinal study to learn how people really interact with our product.”

If so, you’re not alone. Many UX researchers get pressure from stakeholders to conduct a particular type of research in order to answer questions. The focus is on UX research deliverables only. The problem is that sometimes the recommended research method simply isn’t the most efficient way to get those answers.

UX research isn’t just about deliverables; it’s also about strategy. To produce the most helpful results, you’ll need to determine your research objective and then choose a method that supports it. As you evaluate your options, consider these different dimensions of research to help you arrive at the right decision.

Quantitative vs. qualitative research

As you conduct your research, you’ll probably be using a combination of quantitative and qualitative methods to answer your questions. Quantitative research methods rely on using large sample sizes to establish trends and conclusions. Qualitative research, on the other hand, is appropriate for getting a more in-depth, contextual understanding of why those trends occur. Some objectives are best suited for one type of research over the other. However, if you’re looking get a full picture of your user experience, you need to understand both what’s happening and why. If you only have quantitative data, you may be missing out on key insights that could make all the difference in your understanding of the user experience—and make sure you’re solving a problem that actually needs solving. And if you only do qualitative research, you won’t be able to tell whether your findings are representative of a larger population. That’s why the best research strategies incorporate both approaches.

Why you really need both

While you might be tempted to lean heavily on one type of research over the other, having a balance of both quantitative and qualitative data is crucial. Here’s why:

• From numbers to narrative: Quantitative data might tell you that a new feature isn’t catching on as expected, but it doesn’t tell you why. That’s where qualitative research comes in. By following up with user interviews or a few observational studies, you can start to see the story behind the stats.

• From detailed scenes to big scenes: Conversely, if you only rely on qualitative data, you might get rich, detailed insights, but how do you know these findings are truly representative of your entire user base? That’s where quantitative data helps. It ensures that the voices you’re hearing are not just loud but also widespread.

Integrating for deeper insights

The best UX research strategies cleverly weave together quantitative and qualitative research. Start with quantitative methods to spot the trends and identify where the problem areas might be. Then, bring in qualitative research to dig into those areas more deeply and understand the user behaviors and motivations behind the numbers.

For instance, if your quantitative data points to a low completion rate on a specific user journey, qualitative methods like moderated usability tests can help you watch what’s tripping users up and hear directly from them about what’s not working.

By keeping both quantitative and qualitative tools in your toolbox, you ensure a well-rounded understanding of what’s happening and why. This combo not only gives you the complete picture of your user experience but also makes sure you’re focusing your efforts on improvements that your users will truly benefit from. This integrated approach is your best bet for making research-driven decisions that can really move the needle on your project.

Attitudinal vs. behavioral research

Though sometimes misconstrued as being the same thing, attitudinal and behavioral research are not synonymous. However, as with quantitative and qualitative research, the two can be useful when assessed concurrently.

What people say is often different from what they do.

Understanding attitudinal research

Attitudinal research dives into the opinions, feelings, and stated preferences of your users. It’s about getting to the heart of what users believe or feel about an experience before they even interact with it. This type of research typically involves direct questions like, "What do you think about this feature?" or "How would you rate your overall satisfaction with our service?" It can be conducted through surveys, interviews, and even focus groups, aiming to gather users' declared attitudes or expectations.

For example, you might use attitudinal research to understand why users prefer one design over another or to gauge initial reactions to a product concept. This can be incredibly useful for capturing the emotional and psychological responses that might not be evident from behavior alone.

Exploring behavioral research

In contrast, behavioral research is all about the actions—what users actually do when they interact with your product. This research type tracks real activities rather than opinions, using methods like usability testing, clickstream analysis, and heat maps to observe how users navigate a system, what features they use the most, and where they encounter problems.

Behavioral data is invaluable because it shows you the reality of user interaction, which can often differ significantly from what users say they will do. For instance, a user might claim they find a feature essential, but behavioral research could reveal that they rarely or never use it.

Why both are crucial

When you look at attitudinal and behavioral research together, you start to see a full picture emerge. Attitudinal data gives you insight into the 'why'—the motivations, preferences, and feelings of users. Behavioral data shows you the 'what'—what they are actually doing with your product.

This dual approach can help identify discrepancies between user expectations and their actual behavior. For example, if users express high satisfaction in a survey but seldom use the product in real life, this could indicate issues with usability that weren’t apparent in their attitudinal feedback.

Integrating attitudinal and behavioral insights

The most successful UX research strategies blend attitudinal and behavioral data to refine product design and functionality. Start by gathering attitudinal insights to understand user expectations and perceptions. Then, observe their actual behavior to see if it aligns with those expectations. Where you find gaps, you have a powerful indicator of where the product might be failing to meet needs or where new opportunities lie.

For instance, if attitudinal research shows that users expect a feature to work a certain way, but behavioral research shows they struggle to use it as intended, you might need to revisit your design or provide clearer guidance.

By aligning what users say with what they do, you can create more user-centric products that not only meet but exceed user expectations, ensuring that your UX design is both appealing and functional. This cohesive approach ensures you’re not just solving for perceived issues but addressing real-world challenges that affect user satisfaction and retention.

Stages of the product life cycle

The type of research you do will depend on the stage of development of the product you’re interested in. Each stage of development has different research objectives—and different questions that need to be answered.

Before development: Exploring the unknown

Goal: Exploration

At this nascent stage, your primary objective is to explore and understand the landscape:

• Questions: What do users need? What solutions are they currently using? What’s working well in those solutions, and what gaps are evident? Where is our starting point?
• Methods: This stage is about gathering as much information as possible to inform the initial direction of your product. Techniques include:
  • Interviews and 1:1 user interviews: Direct conversations with potential users help uncover needs and frustrations with current solutions.
  • Focus groups: Bringing together groups of potential users to discuss needs, desires, and experiences provides diverse perspectives.
  • Longitudinal studies: Observing users over time can reveal deeper insights into their behaviors and needs that may not be apparent in shorter studies.
  • Competitive analysis: Assessing what competitors are doing can help identify industry benchmarks and opportunities for differentiation.
  • Benchmark studies: Establishing a baseline for how current products are performing to understand where improvements can be made.

During development: Validating and refining

Goal: Evaluation & validation

As your product begins to take shape, the focus shifts to evaluation:

• Questions: Are we on the right track? Does the product look and feel right? Do users understand how to use it? Can they find what they’re looking for?
• Methods: This phase involves more interactive and testing-oriented methods to refine the product based on real user interactions:
  • Card sorting and tree testing: These methods help validate information architecture and navigational structures.
  • Moderated and unmoderated usability testing: Testing how users interact with your product to identify usability issues.
  • Preference testing: Asking users to choose between different options helps determine which elements they prefer.
  • Prototype testing: Allows users to interact with prototypes to validate design concepts and functionality before full development.

Live design: Measuring and optimizing

Goal: Measurement

Once the product is live, the emphasis is on optimization and ongoing improvement:

• Questions: How are we performing? What can we optimize? Does the actual user experience meet users’ expectations?
• Methods: Continuous and iterative testing becomes key:
  • Moderated and unmoderated usability testing: Further testing with real users to refine and enhance the user experience.
  • A/B or multivariate testing: Testing different versions of elements to see which performs better in real-world scenarios.
  • Benchmark studies and longitudinal studies: Revisiting earlier benchmarks and observing changes over time can highlight shifts in user behavior and satisfaction.
  • Diary studies: Gathering ongoing user feedback through diaries provides insights into the user experience over longer periods.
  • Multichannel and omnichannel studies: Understanding user interactions across different platforms and channels to ensure a cohesive user experience.

Context and location

Many UX research teams use office space in-house to conduct in-person studies, which can be great for longer usability tests, interviews, and focus groups. But not all research needs to be conducted in person to be useful! Many methods can be done remotely, which means that your pool of potential study participants isn’t limited to a location. You can often conduct more research sessions in a shorter amount of time (and with less recruiting hassle) if you do remote research.

Remote testing can save time and take the hassle out of your research.

In-the-wild research (research that happens in the participant’s environment) can give you more realistic insights than lab research. Since participants aren’t in an artificial and unfamiliar setting, they’re more likely to give honest reactions and unbiased opinions. You also gather contextual findings about how your product is used naturally. Plus, some research just can’t be done in a lab. If you’re trying to answer questions about how users interact with a product in a specific context (using a fitness app at the gym, for example), then you’ll need to make sure your research happens in that location.

We call that destination-based testing. For research that isn’t dependent on a location, study participants can be in their homes, offices, or anywhere that is convenient.

Finally, there are times when you might need to observe something other than a digital experience on a computer or mobile device. For example, maybe you’re researching how customers find their way around a theme park, install a thermostat at home, or choose which brand of laundry detergent to buy at the store. These studies are called beyond-the-device tests, whereas traditional app and website studies conducted on computers and mobile devices are simply device tests.

Choosing a UX research method

Once you’ve zeroed in on the strategic requirements for your research, you can narrow down your choices of methods. A one-size-fits-all approach is likely to limit the usefulness of your research, so make sure to consider all of your options when it’s time to pick a method. If it turns out the best method is something out of the ordinary, then talk with your stakeholders about why you’re recommending it. Share what results you can expect—and how they support the research objective.

Why flexibility matters in method selection

Selecting the right UX research method isn't just about sticking to the tried and true; it's about matching the method to the specific questions you need to answer. This means that sometimes the best approach might be something a bit unconventional or out of the ordinary. For instance, if traditional usability testing isn't yielding the depth of insight you need, consider employing diary studies or contextual inquiries that can provide a more nuanced view of user behavior over time or in their natural environments.

Communicating your choice to stakeholders

Once you've chosen a method, especially if it's a less conventional one, it's important to communicate effectively with your stakeholders about why this approach is recommended. This conversation should cover:

• The rationale: Clearly explain why this method is the most suitable for achieving the research objectives. Detail how it aligns better with the project goals compared to other methods.
• Expected outcomes: Discuss what results you anticipate from using this method and how these will contribute to understanding and solving the UX challenges at hand.

Leveraging results to support your research objectives

When it comes time to share your findings, the way you present them can significantly impact their reception and utilization:

• Highlight relevance: Make sure to connect your findings directly to the research objectives and overall project goals. This helps stakeholders see the value of your work and how it contributes to the broader business or product strategy.
• Share actionable insights: Focus on delivering insights that are actionable. It's not just about what users do or don't like, but about what steps can be taken to enhance the user experience based on these insights.
• Visual and narrative clarity: Use clear visuals and narratives to make your results compelling and understandable. Whether it’s charts, videos, or quotes, the right presentation can make a big difference in how your findings are perceived and acted upon.

By carefully choosing your UX research methods and communicating effectively about them, you ensure that your work not only generates valuable insights but also resonates with and is supported by your stakeholders. This careful planning and execution help turn UX research into a pivotal tool for driving product success.