This article will introduce you to the basics of Unmoderated Usability Testing. We will explain why you should conduct such testing, how to prepare tasks, or how to recruit the right participants.
Unmoderated usability testing is definitely on the rise. The fact that it’s cheaper, faster, and more accessible than in moderated or in-person research makes it popular. Nowadays, it is sometimes even the only option for UX teams to conduct user research. Let us take a look at what unmoderated usability testing is.
What is an Unmoderated Usability Study?
An unmoderated usability study is where respondents are not guided by a moderator. The test is prepared beforehand, and respondents complete it on their own. With an unmoderated usability study, all you have to do is to prepare the study and recruit people to complete it. The advantage of this study is that the number of participants is unlimited.
What is Unmoderated Usability Testing?
Usability testing is the process of testing designs with users, used for collecting information about user experience – how difficult or easy the product is to use. Because the experience of the user is subjective and intangible, it relies on observing how users interact with the product as a way to learn about their experience.
It is the process of testing designs with users, used for collecting information about user experience – how difficult or easy the product is to use. Because the experience of the user is subjective and intangible, it relies on observing how users interact with the product as a way to learn about their experience.
According to the to NNgroup “User Experience encompasses all aspects of the end-users interaction with the company, its services, and its products.”
It is a great way to test and collect valuable feedback on websites or other products. It is frequently used as it is useful throughout a product’s whole lifecycle and provides a lot of insights into a user’s behavior. Unmoderated Usability testing is conducted with the help of online usability testing tools.
As participants complete the study on their own, no supervisor is needed. As a result, with unmoderated testing, you can collect much more information in a much shorter time at a lower cost.
You are currently reading one of our 101s. This blog introduces you to the basics of Unmoderated Usability Testing. If you are looking for more complete resources, check out our Complete Guide to Usability Testing.
What is Website Testing?
Website Testing is a tool by UX research platform UXtweak for conducting unmoderated usability tests that also integrates techniques of Session Recording. This gives you the ability to conduct user tests online and then replay user interactions like a video. Allowing you to pinpoint the exact place and time when users had issues completing their objectives.
What can you find out with Unmoderated Testing?
Unmoderated Testing can help you find answers to questions such as:
- Can people use the website/app for important use cases?
- How do users interact with the website/app? What are they thinking when using the website/app?
- How do users tackle their objectives?
- Is there something that keeps users from fulfilling their objectives?
- Do they take a completely different way than they should have or you expected? Or do they take the correct path but get lost and confused along the way?
Thanks to Unmoderated Testing, you will find out answers to these questions and discover any usability and user experience issues. Knowing how users interact and use your website, will allow you to optimize and improve it.
When should you use Unmoderated Testing?
The great advantage of Unmoderated Testing is that it is very flexible – meaning that can be used pretty much anytime during the development or design process. You can use Website Testing to test simple HTML prototypes as well as full of live websites.
All that you need to make it work is a working piece of HTML that’s hosted somewhere public. That way, Website Testing can link to your website, web app, or prototype via URL.
Usability testing using Unmoderated Testing is versatile and scalable. You can set up a simple Unmoderated Testing study with minimum effort in a few minutes, or you can run studies with tens of tasks that cover main use cases, or you can achieve that with just a single task. Tests can be conducted with only a few people, or you can collect results from hundreds of respondents. With Unmoderated Testing, you can create studies that suit your needs.
Preparing Unmoderated Testing
During the preparations of your Unmoderated Testing you should consider asking yourself the following:
What are you going to test?
Obviously, you should start by defining the scope of your usability testing. There will be situations that call for testing all the main use cases of your website or app because you just launched a full redesign to find out what works and what doesn’t. At the same time, all you need might be more focused testing of a few scenarios that cover the new functionality.
Who should be the respondents?
There should not be any user testing without considering your users’ identity. If you already have a website or an app, you’re probably well aware of whom your target group is. You can use any of your existing personas, real customer data, or even data from surveys and other sources of feedback you have.
Information and the more of it you have is about your user is useful in many ways. Apart from the obvious (to be able to recruit respondents who represent your audience), it will help you prepare the right use cases and formulate relatable tasks that represent the user’s natural way of thinking. So, for these reasons – keep your users in mind when you start planning usability testing.
How to write tasks for Unmoderated testing
At its core, the Unmoderated Testing study consists of several tasks that the respondents are asked to complete to successfully finish the study. By preparing your tasks carefully, you will increase the chance that your user study will provide good results. Well-defined tasks lead the respondents to try to complete them as naturally as they can.
When preparing keep in mind that you need to:
- Write tasks that cover your objectives
- Write realistic tasks
- Write actionable tasks
- Write tasks without giving away the answer
Recruiting respondents for your test
Once your study setup is completed and you want to launch the test, now you need to recruit respondents. Recruiting respondents for an Unmoderated Testing study is easy. Simply share a link to the study with your respondents. You can use social media like Facebook, Twitter, and LinkedIn to get it to your target or send the link directly to your testers. Don’t forget your own channels like newsletter subscribers list – they can provide a great source of respondents.
Of course, there is more to recruiting respondents than sending out a link. It is important to prepare a recruitment strategy as a good one will make the difference between learning a lot of valuable information to improve your website to learning nothing at all – wasting a bunch of time and money. Also, if you want to save time and energy on searching for participants, we suggest you to use our User Panel.
Questions that should help you with recruiting respondents:
- Whom do I want to recruit?
Your respondent sample needs to be representative of the end-user so you can learn about actual user behavior. When using the Website Testing Tool by UXtweak this is very easy as you can make use of – UXtweak Recruiting Widget which will allow you to recruit respondents directly from your website.
- How many people do I want to recruit?
According to NNGroup 5 respondents are the optimal number of respondents. With five respondents, you find out most of the main problems in the tested scenarios. Although you might want to gauge the statistical significance of metrics in your study so to get narrower confidence intervals, we recommend at least 20 respondents.
- What information do I want to communicate to the respondents and how?
This will depend on who your respondents are and how you recruited them, but the golden rule is that the respondents should be always well informed. A well-informed respondent has more value for you than someone who has no clue of what’s going on.
Information we recommend sharing with your respondents before testing:
- How long does it take – e.g. 20 minutes
- What is expected of them – e.g. completing tasks on the e-commerce website of a fashion company
- What is the point of the test – e.g. to make our website easier to use
- Why would users want to participate? – you should always keep in mind that respondents are not always willing to give you their time for free, especially if the test takes more than a few minutes.
Therefore it is an excellent idea to offer respondents some incentive for participating in the study (could be monetary or something, like a chance to win in competition or a different benefit). Always make sure that they’re well informed about the reward and how they receive it.
After a successful study, you obtained a lot of data. But you still need to analyze it to make something useful out of it.
The overview gives you all the information that you need for checking on the state of study. The first thing in the summary is the dates when your study was first created when it was launched, along with the number of respondents who participated in it. Check the donut chart to see how many respondents completed the study and whether there were any quitters.
Time Taken is a number that tells you how much time the respondents spent doing the study. Checking whether Time Taken matches the expected completion time is a good way to keep tabs on any unexpected issues so they can be addressed quickly.
The Success score is the percentage of tasks that the respondents completed by submitting the correct page as the answer.
The overview contains a breakdown of various attributes of your users such as:
- Top Locations
- Device Breakdown
- OS Breakdown
- Browser Breakdown
- Screen Resolution Breakdown
In the list of respondents, you’ll find information about all respondents who participated in your study. It’s here where you can replay the completion of tasks, filter your respondents, and select which ones should be included in the analysis. If you want to learn more, check out Interpreting WebsiteTesting results in our Definitive Guide to Website Testing.
Pressing Play on the task will open up the session player, showing you exactly how the interaction between the tester and the website/app went down. You will be seeing what the respondent was while they were trying to complete the task. You can control the session player like any video player. If you read our Session Recording guide, you know that even though it looks and feels like a video, it is a re-interpretation of how the respondent interacted with your website/app.
We also created a complete guide to Setting up a Website Testing study for you to better understand the whole process.
Unmoderated usability testing is a simple way to get a lot of information about how your users behave. The Website Testing tool gives you the option for recruiting the right respondents that fit your needs easily – by sending them a link or using the recruiter widget.
All results are analyzed after the end of the study, giving you detailed information about tasks, answers to questionnaires, respondents, etc. This helps you to understand the issues on your website and to fix them quickly, easily, and in a budget-friendly way. Try it yourself – register to UXtweak!