The complete guide to usability testing

Usability testing is a proven method to evaluate your product with real people. In this complete guide to usability testing, we share everything you need to know to run usability tests and get actionable insights to create better user experiences.

Chapter 1

What is usability testing?

Let’s start right away with a definition of usability testing. Usability testing is the process of testing your product with real people by getting them to complete a list of tasks while observing and noting their interactions. The goal of conducting usability testing is to understand if your design is usable and intuitive enough for users to accomplish their goals. 

However, the benefits of usability testing extend beyond testing functionality and usability. More than that, it helps you really grasp the use cases of your product, understand your audience, and build a good product.

Behzod Sirjani, founder of Yet Another Studio and former Head of Research & Analytics Operations at Slack, explains that "Good usability testing takes a holistic look at the potential audience that you can serve and the ranges of intended and unintended uses of a product to understand how you're enabling people to do what they want with your product while preventing misuse."

Start usability testing now

Maze lets you run in-depth usability tests on prototypes imported directly from Figma, InVision, Marvel, or Sketch. Sign up for free.

A brief history of usability 

Before we dive into usability testing specifics, let's define what usability means in the context of the human-centered approach to design. Human-centered design is a creative approach to problem-solving, established and developed by IDEO, which places the user front and center in the design process. 

The approach involves various UX research methods and techniques for developing products, such as ideating, rapid prototyping, the jobs-to-be-done framework, and more. Usability testing is one of those methods and a key principle for building user-centric products.

The modern approach to usability testing started in the ’80s as work and personal computers became more prevalent. At that point, most people had very limited exposure to computers, so getting familiar with new technology was usually a very long and frustrating process. 

To make sure computers were usable and intuitive, programmers needed to understand users' thought processes for completing a task. In 1981, two researchers–Karl Ericsson and Herbert Simon–published an article called Verbal Reports as Data, which popularized the “think aloud” method in which the participant literally speaks their thought process out loud. 

More than a decade later, the ISO 9241 definition of usability—published in 1998—defined usability as “the effectiveness, efficiency and satisfaction with which specified users achieve specified goals in particular environments.”

These developments meant that instead of guessing how a product is used by people, you could observe them complete tasks, learn, and improve usability problems in real-time. That’s where usability testing comes in. It gives you a direct look into what the participant is thinking and how they use your product. 

The benefits of usability testing

Have you ever noticed on cooking shows–when possible–chefs have others taste components of a dish while they’re still cooking? Maybe it’s a sauce or some other ingredient. Their counterpart might say, “needs more salt” or “a little too spicy.” What they’re doing is running a very small-scale user test. 

When you’re too close to something–like a chef may be to a dish–it’s very easy to miss things. We tend to overestimate how likely someone else is to share in our opinion. The psychological term for the phenomenon is the false-consensus effect. 

The only way to combat false-consensus effects when designing is with usability testing. This allows you to get another point of view, and most importantly, a perspective from the actual users who’ll be using your product in real life.

Usability testing is also relevant for long-term customer success. A broader perspective means you can design better for your audience, and make sure more people can be successful with your product. Forbes reports that companies investing in UX have a lower customer acquisition cost, lower support cost, and higher customer retention rates. 

To give a more concrete example, Shopify reports that one of the most common reasons a customer abandons a shopping cart is due to a complicated check-out process. When you employ usability testing during the design process, you can find and fix usability issues and complex flows before you launch your product, and ensure the user experience is up to standard.

When should you do usability testing? 

There are multiple times when you and your team should run usability tests during the design process. In most cases, it’s something you should do more than once, and early and often in the process to inform your design decisions.

Here are some general guidelines you can follow when deciding when to do usability tests and creating your test plan.

Before you start designing 

If you want to start a redesign, chances are there’s something about your current site that’s not working. You can run a usability test before you start any redesign to discover users’ pain points and bottlenecks with the current design. If you have that data as a jumping-off point, you will be able to benchmark your new design against the existing one to make sure it’s a better version.

Additionally, if you’re designing a new product from scratch, you can do usability testing with competitor products to observe how people use similar tools and services in real life and understand their mental models when using products like yours. The test results can be a benchmark for your UX design when the time comes to test it.

Once you have a wireframe or prototype

With the initial research phase completed, you can start in on your new design. It’s important to test your designs as you go through the iterative design process. You can start testing as soon as you have your initial ideas sketched in a low-fidelity prototype or even a paper mock-up. 

This type of early testing is called formative testing, and it helps you test and validate ideas as you move through the process, so you don’t get to the end without any user feedback. 

For this stage, you might want to do card sorting or tree testing to test your website or app's information architecture. Generally, if you’re starting from scratch, card sorting is recommended first to understand how your target audience groups and associates information. That, followed by a round of tree testing, ensures that the content on your website or app is easy to find, and the web pages follow an intuitive flow.

Product tip ✨

 If you have a Figma, InVision, Sketch, or Marvel prototype ready, you can import it into Maze and start testing your design with users immediately.

Prior to the launch of the product

Once you’re at the end of the development process, you’ll probably have a fully-sketched high-fidelity prototype. That’s where summative testing comes in. Summative testing is a usability evaluation you do at the final stage to prevent any major issues and test if the end-users can complete tasks with the user interface. 

When running summative testing, you should aim for a large sample size to get statistically significant results from a large pool of test participants. Usually, the test results can be benchmarked against any data you collected throughout the testing process, so ease of use and user satisfaction can also be measured before launch.

At regular intervals after launch

Just as products aren’t static, neither are user bases. Since they’ll also continue to change and evolve, it’s good to run test sessions every so often to make sure everything is still working as intended. Consider regularly checking after launch to make sure your product stays in tip-top shape, and your actual users aren't experiencing any issues. 

5 best practices for usability testing

This section was written by Behzod Sirjani, founder of Yet Another Studio and former Head of Research & Analytics Operations at Slack.

There are lots of things to keep in mind when you are usability testing. That said, here are some of the most important ones:

1. Get consent.  You should get consent at two separate points during usability testing from your test participants. First, at the beginning of the test and then at the end. Here's why. At the beginning of the test, you should ask for their consent to record the test and the results. Then at the end of the test, you should ask for their consent to keep the test results and use the data. It's important to ask for consent twice because a participant only has a loose idea of what they are consenting to in the beginning. 

2. Be inclusive. You should be intentional about the people you want to participate in your usability test, and you should make sure to include people who may have different perspectives on your product. This includes people of different abilities, demographics, usage behaviors, market segments, etc. Being intentional and inclusive in your testing audience will help you build better products because you will learn from people who are different than you and who see and do things that you do not. 

3. Run a pilot test. Before you ask for any of your customers or potential customers to spend their time giving you feedback, make sure to pilot your usability test with someone inside your organization, ideally someone not on your team. Among other things, this allows you to identify aspects of the study or product that are obvious to you, but that may be missing from your study experience. 

4. Establish evaluation criteria. You should know what success looks like for your product testing and have clear ways to identify if and when people were (not) successful. Bonus points if you have easy ways to keep track of that in your notes. 

5. Be mindful of length. Your usability test should be only as long as it needs to be for you to have confidence in your results. People do not exist to give you product feedback, and very few people want to spend all day evaluating the usability of your product. If you worry that your test is too long, pilot it and gather feedback. It’s much better to run multiple separate tests than to gather feedback from participants who are cognitively exhausted because you’re asking too much of them.

In the next chapters, we'll learn more about the different usability testing methods and about the different types of usability testing, such as in-person user testing versus remote usability testing, or unmoderated versus moderated sessions.