Moderated vs. unmoderated usability testing: the pros and cons

Being customer-centric isn’t just a competitive advantage these days, it’s a basic necessity. In addition to making use of all the essential usability testing tools available, it's important to have a human-centered mindset at an organizational level. One way teams can validate their efforts is through qualitative studies like moderated and unmoderated usability testing. While terms like unmoderated and moderated usability testing can sound intimidating, we’re here to help break it down with everything you need to know—and how you can figure out which one is best for your needs.

One question that we at UserTesting often hear from customers is which is better: moderated or unmoderated usability testing? The answer, as you might’ve guessed, is that it depends. Like any research strategy, there are differing benefits for each approach, making them more or less suitable depending on what you’re studying and your ultimate research goals.

Moderated vs. unmoderated tests, what’s the difference?

Before we dig into the pros and cons of each approach, it’s important to understand how the two strategies differ.

Moderated usability testing

With moderated usability testing, a real person will be there to help facilitate (i.e., moderate) the test. The moderator will work directly with the test contributor, guiding them through the study and answering questions if the contributor encounters any challenges while completing their tasks. What’s great about moderated studies is that they can be conducted either remotely or in-person, you can ask (or answer) questions in real time, and you can open the space for dynamic discussions.

However, one of the drawbacks with moderated testing is that it can require a bit more planning upfront since you’ll need to coordinate schedules with contributors and have them commit to the set date and time. It’ll also usually require a designated, quiet location, where you can interact with your contributor, free of interruptions.

Here at UserTesting, we refer to remote, moderated tests as Live Conversations. They can be as short as 30 minutes, or as long as 120. Healthcare software company CoverMyMeds came to UserTesting to figure out how to best prioritize patient needs during the prescription authorization process. After conducting weekly Live Conversation interviews, the organization was able to better address pain points, reduce costs, and highlight the importance of UX. This customer success story proved how beneficial Live Conversations can be, and validated moderated tests as the right choice.

Unmoderated usability testing

Unmoderated usability testing is just like it sounds. It’s not monitored or guided, so there’s no one else present during the study except the contributor. The contributor completes tasks and answers questions at their own pace, on their own time, at a time and location of their choosing. Pretty nice, right? Unmoderated testing tends to be less time-consuming and more flexible than moderated usability tests, as contributors can complete their tests on their own time without any disruption to your daily workflow.

Now that we’ve established the differences between the two, let’s dive into how and when to best apply each strategy based on your research needs.

When to use moderated usability testing

It helps to think of moderated user testing as an interview or a real-time conversation that you’re having with a contributor or customer. Everything that you’d do to prepare for that type of interaction is what you’ll need to consider for a moderated study. At UserTesting, we especially recommend them for early development stages use cases, including prototype tests, competitive analysis, usability tests, and discovery interviews.

Pros

Moderated testing works best when you need a high level of interaction between you and your contributor. For example, if you want to study a prototype with limited functionality, or a complicated process or concept, moderated testing provides you with the interaction you’d need to guide a contributor through the study. It's also an excellent way to conduct interviews, understand the customer journey, and discover pain points.

Moderated user testing is also a great way to observe body language and pick up on subtle behaviors and responses. It enables you to probe contributors for more information if they seem stuck or confused, and minimizes the risk of a contributor speeding through the tasks or questions. This type of test allows you to develop a rapport and have a natural conversation with your customers. This helps establish trust and leads to candid feedback that might not be possible with other qualitative research methods.

Ideal if you don’t have a lot of time to spare, moderated user testing is more time-efficient than traditional interviews or focus groups—which may require weeks for both recruiting and scheduling, and months to receive and compile results. With UserTesting’s Live Conversation test, you’ll only need one business day lead time. Even better to know, our test results can return to our customers in as quick as minutes or hours after launching.

Cons

Because of the additional time and resources, moderated testing does cost more than unmoderated testing. However, you can reduce some of that cost by conducting remote moderated studies, rather than in-person. This gives you the opportunity to connect with contributors all around the world and reduces the need to block out dedicated time and space for onsite interviews. Who doesn’t love getting some hours back in their day?

When to use unmoderated usability testing

Unmoderated testing is best for validating concepts and designs quickly with a diverse group of contributors. This type of testing works great if you have specific questions that you need answered, need a large sample size, require feedback quickly, or want to observe a contributor interacting in their natural environment.

Pros

The beauty of remote unmoderated usability testing is that it can be done anytime, anywhere, and you typically have actionable feedback within a day if not sooner. Because a moderator isn’t needed, the cost is typically much lower than moderated tests, which can enable you to run more tests with a wider variety of contributors. Additionally, if you conduct a test and later (understandably) decide that you need to rework the tasks or questions, sending out a new round of remote unmoderated tests would be more cost- and time-friendly than if you were to redo moderated usability tests. If you prioritize flexibility, unmoderated usability tests may be the way to go.

Cons

Since unmoderated tests are completely unsupervised, they require a fixed set of questions and tasks for contributors to complete on their own. If contributors run into issues, spend less time than asked of, face technical difficulties, or don’t understand the tasks or questions, you unfortunately won’t be able to step in and guide them. For this reason, when making tests, our customers are encouraged to over-communicate, avoid making assumptions, and ask the contributor to verbalize their expectations if they experience any technical issues.

You know why user testing is important, and at the end of the day, the choice of which test is best is up to you and your team. Now you’re better equipped to make an informed decision.

Related reading: Usability testing examples