Remote research, faster: 10 strategies to make unmoderated testing even more efficient

In order to keep pace with the changing attitudes, needs, and behaviors of people adapting to the COVID-19 pandemic, you’re required to gather insights faster and more frequently than usual. To help with this, we pulled together a list of our top ten remote research strategies to help you uncover valuable insights more quickly while conducting remote unmoderated tests on the UserTesting platform.

Taken together, these strategies will help you pinpoint aha moments so that you can identify the trends and themes you want to understand by watching videos more strategically.  

10 strategies for faster unmoderated testing

1. Use structured test plans 

Instead of relying on verbal response questions and listening to entire video sessions for all participants, consider using a combination of tasks, multiple-choice questions, and rating scale questions to structure participants’ time. A little extra time spent upfront planning and structuring the test will help you pinpoint moments you care about quickly—without having to watch a lot of videos to find that needle in a haystack. How this might look:

Set participant expectations

By providing an expectation for how long you want people to spend on a task, you’ll set the expectation for how they should spend their time and control the amount of time you need to devote to watching videos.

Task: Spend about 2 minutes reviewing the content on this page. Remember to talk about what you’re thinking.  When you’re done, move on to the next task.

Use rating scale questions

Use rating scale questions to get a quick sense of how people feel, create counts, and pinpoint outlier participants. When you prompt people to speak to their rating, you can then watch an automatic highlight reel to provide more context around the ratings.

Rating scale: On a scale of 1-7 where 1 is ‘Not at all understandable’ and 7 is ‘Very understandable,’ please rate how easy it was for you to understand the meaning of the text. Remember to describe your rating as you make a selection. 

Use multiple-choice questions

These questions enable you to create counts as well and help you understand how participants think or feel about a concept. In addition, you can create an automatic highlight reel to listen to all of the feedback, back-to-back.

In another example, you might consider adding a multiple-choice question that is structured like a quiz question that enables you to gauge whether participants understood the text—again, without having to watch all of the videos.

Multiple choice (single select): How likely would you be to change your behavior based on what you read:

• I would be unlikely to change my behavior because….
• I would change my behavior because….
• I would be very likely to change my behavior because….
• I am not sure I would change my behavior because….

2. Use automatic highlight reels

Now that you have a highly structured test, you can use automatic highlight reels to easily watch all of your participants do the same task or answer the same question back-to-back. The advantage of using this strategy is that themes surface very quickly. 

Typically, it’ll take between 5-15 minutes to watch all participants complete a certain task, where the average time each participant took to complete the entire test was 15 minutes. This also means that you don’t need to watch all participants complete the entire test to get quick answers.

3. Understand and frame the story you want to tell

Make sure you’re very clear about your research objective and only review the data points and videos from tasks that help you answer that question. Typically, this means that you’ll want to select 1-3 objectives; the more you have, the longer the test will need to be to cover all of them.

Consider framing out your report first so that all you need to do is plug in your findings. This has the added benefit of forcing you to recheck the tasks and questions to make sure they capture the findings you want to share.

4. Capture a pulse

A limited set of study objectives doesn’t mean that you have to limit the test to only capture the information related to your current research objectives. On the contrary, consider collecting more information than you need. For example, ask a few questions about needs, behaviors, and perspectives, so you can go back and use the information later for tracking and trending purposes. 

In this way, you can create a data set over a series of studies that enables you to get a pulse on what people are thinking, feeling, and wanting over time. This can be an efficient way to gather information that you can use to inform persona research rather than conducting a larger scale study all at once.

5. Use the metrics tab 

Use the visualizations available in the platform to pinpoint specific videos and moments of interest that exemplify what is normal, show why specific participants behaved in unexpected ways, or help you to pinpoint the ‘why’ that you want to focus on.

6. Use the Excel download

When you download the Excel spreadsheet, you’ll bring together all of the text data that’s generated from a study. It’s faster to read text than watch videos. In addition, you can create charts and graphs based on the counts you’ve created from metrics or multiple-choice questions. Transcripts are also available in the download. These can be used to pull quick quotes for use in your report.

7. Use the minimum number of participants

To develop a sense of themes or trends, you get good data testing 3-5 participants per segment. If you find that you need more information, you can always add more later. By starting with as few as three participants and watching their entire videos, you’ll be able to get a sense for what a larger group might tell you. You can use the “long and deep” (watch the entirety of a smaller number of videos for studies with a smaller number of participants) or “short and broad” (watch single tasks or groups of tasks across all participants).

8. Develop and use templates

The UserTesting Professional Services team has developed a number of templates that are available both within the platform and via our help page. They cover a wide variety of objectives, from discovery to validation. Additional templates have been developed that focus on COVID-19. Using these templates as a starting point to develop your own Saved Test Plans that can save you a substantial amount of time as well as ensure that you start with a robust approach.

9. Time your test launch

Think through your availability and work day as you plan when you want to launch tests. We find that launching at the end of the work day means that you can start the following day over a cup of coffee watching videos that spark ideas or next steps that you can share with your team as they also start their days. 

Other teams that are working on quick iterations can launch tests first thing in the morning, iterate prototypes in the middle of the day, and launch the next test immediately to get through 2-3 rounds of testing in a day. 

Alternatively, schedule launch for a time that people are naturally thinking about or doing the activity that you want to capture. This might be over the weekend, during a special incentive period, or during a holiday.

10. Watch videos at an accelerated rate

The video player allows you to speed up the rate of playback up to 2.8x. We find that most people tend to be comfortable with watching videos at 1.5 to 1.8x. This enables you to discern what people are saying as well as watch and compare what they are doing.

Bonus: take a balanced approach that works for you

While structure can provide you with speed to insights across a typical interaction, another potential strategy to consider is asking a small number of questions, around 3-5, that are more exploratory in nature or a single, very short task. 

This approach enables you to get quick answers, but mirror actual behaviors more realistically. Balancing these approaches and integrating them into your workflow can help you to make decisions based on user feedback more frequently.