3 ways to use questions in usability testing

August 20, 2019

Let’s be honest: we all know that good design is good business. Over the past decades, design rose to occupy an essential role in business success. Design-centric organizations such as Google and Airbnb are among the highest valued and well-performing companies in the world.

At start-up companies, too: Design is a competitive advantage that makes the product stand out, attracts new users, and retains customers. In the Future of Design in Start-ups Survey, 87% of respondents believed that design is important, and that design led to higher consumer engagement, more sales, and improved retention.

A mark of good design is a product that’s easy to use, efficient at helping you achieve your goals, and provides a great user experience. Design practitioners use various research and testing methodologies, e.g., user interviews, prototyping, surveys, to learn about users and create solutions that prioritize and improve their experience.

“Focus on the user and all else will follow.” - Google

One such method is usability testing, which involves getting real people to use your website, mobile app or product and recording their interactions and feedback. The value of usability testing lies in its effectiveness to reveal users’ difficulties and pain points when completing specified tasks with the product.

In addition to measuring task metrics, you can get even more insights by asking questions during usability testing to learn more about your users' experience.

Let’s now look at three ways you can include questions in usability testing to gather feedback that helps you build stellar product experiences.

types of question to use in usability testing

1. Questions to ask before testing

Before you give users tasks to complete, you can collect data about background, experience, and demographics. Even if you've already screened users for usability testing and selected representative participants, it can also be useful to gather this information to correlate the data from task activities with responses.

Background Questions
These type of questions inquire into the previous experiences of users, how familiar they are with similar products, and what solutions they used before (if any).

Examples of background questions:

  • How experienced/comfortable are you with using an application to do [X action]?
  • What type of product do you use to do [X action]?
  • Have you used any products in the [X industry] before?
  • How often do you use [X type of product] to do [X action]?

The information you collect with background questions can help you better understand your users, and whether your product addresses their needs. You can discover details about habits and preferences that are relevant for the product you're testing.

Demographic Questions
Demographic questions can help you learn valuable information about your target audience. These details can be useful to map out the different types of user personas that your product addresses.

Asking for demographic information at this stage can reveal patterns in how different user groups perform tasks and the way each of these groups uses your product. You can then use this information to improve your design, making it easy to use and intuitive for all users.

Age, location, education, gender, marital status, family, employment, etc., are some examples of demographics you can collect from users.

2. Questions to ask during testing

During testing, the goal is to try to understand why users did or didn’t do a particular action when attempting to complete the tasks you prepared. The answers can help you improve your designs by getting to the root of an issue users are experiencing.

Examples of questions you can ask during testing:

  • How was your experience completing this task?
  • What did you think of the interface?
  • How was the language used on this page?

With in-person testing, you can ask these type of questions after users attempt to complete every task. With Maze, you can add questions before or after each task in your test.

When asking users questions about their experience, it’s important to keep in mind the Aesthetic-Usability effect. This effect notes that users perceive aesthetically pleasing design as more usable. You can notice this by looking at what users do versus what they say.

Compare the data you record from the tasks to the answers given by users. If the data reveals a user has a hard time completing a task, but gives a positive remark, the Aesthetic Usability effect might have influenced their experience.

3. Questions to ask after testing

After each usability test ends, you can ask post-testing questions to gauge the user’s overall experience and collect their feedback.

Examples of questions you can ask after testing:

  • What did you think of this app/website?
  • What did you like the most/least about this product? Why?
  • How would you describe your experience with this product?
  • Would you use such a product to do [X activity] in real life?
  • What features would you like this product to introduce?

These types of questions will help you gather user opinions and attitudes about your product. In addition, you can introduce a survey like the System Usability Scale (SUS) to evaluate the usability of your product. The survey includes a 10-item questionnaire (view template here) with five option responses from ‘Strongly Disagree’ to ‘Strongly Agree.’

Whether you're using a survey or simply asking users about their experience at the end of the test, it's important to thank them for their time and feedback. You can take this opportunity to ask for final remarks users have that didn't relate to any of the previous questions.

The art of asking questions in usability testing

The answers you will get from usability testing will depend on how you phrase and pose your questions. Let's now look at some tips to keep in mind when asking usability testing questions:

  • Use simple and straightforward language without being relative or confusing. Avoid jargon or internal language that only you or your team might know.
  • Avoid misleading questions. For instance, don't ask respondents how positive or negative their experience was. This already leads them toward an answer. Instead, ask them to describe their experience and to share what they thought about the product in their own words.
  • Keep open-ended questions to a minimum. In surveys in particular, open-ended questions take longer to answer. Substitute open-ended questions with Yes/No or Multiple Choice questions when suitable (e.g. when recording the age bracket). Respondents can select a choice from a pre-written list, which makes it easier to answer.

The fine art of asking questions in usability testing means using concise and clear language, and making your questions easy to understand and answer. In the end, you can only collect valuable information by asking good and well-formed questions. Make sure to test your questions with colleagues or friends before you survey respondents.

Maze makes user testing a breeze by allowing you to complement the tasks in your test with questions, ratings, and surveys. The insights you accumulate from usability testing can help you to make informed design iterations or validate a design concept with users–so it's well worth your time and investment.