The Ultimate Guide to UX Research
In this chapter, we zoom in on card sorting to learn how this UX research technique works, and share a few practical tips on how to run an effective card sorting session.
Product tip ✨
Maze allows you to run card sorting tests to discover how people understand and categorize information and optimize information architecture. Get started for free.
Card sorting is a UX research method that helps you discover how people understand and categorize information. In a card sorting study, participants group ideas or information written on cards into categories in a way that makes sense to them.
“Card sorting is a great exercise to understand your users’ mental models and a helpful way to question your biases and validate the information with real people.”
Using the insights gathered from the sort results, you can:
While card sorting shares the same goal of content categorization as tree testing, each method approaches this from a different perspective and is used at various stages during the research process.
When you plan a new website and IA, involving users in the process as soon as you can is essential. Card sorting is useful in this respect, as it helps you to understand how users group similar information on your website or in your app. On the other hand, tree testing is meant to help you test the current website’s navigation or the categories identified during a card sorting session.
If you want to test how your current IA performs, you can start with a tree test to identify user experience issues and collect data benchmarks for subsequent comparisons. If the tree test results indicate most users find it hard to navigate your website or a part of it—and you decide to redesign it—a card sorting session makes sense next.
Conduct a card sorting session to understand how your users organize the content in a way that makes sense to them, independent of how your current website is organized.
Then, after you have redesigned the website based on the data collected during card sorting, you can do a new tree test with the redefined architecture to make sure it performs better than the previous one.
In the next chapter, we go over how to run a tree testing session in detail. Ultimately, it’s helpful to combine both UX research methods when working on your information architecture (IA) to create an intuitive and easy to use website.
“All card sorting types complement each other. Choosing the right type of card sorting comes down to the objective of your project.”
There are three types of card sorting you can choose from:
Depending on your objectives, you can also choose from different card sorting formats:
When it comes to selecting the right type of card sorting, Vaida Pakulyte, UX Researcher and Designer at Electrolux, explains:
Open card sorting is a generative research exercise, rather than an evaluative one. In this card sorting type, participants organize topics into categories that make sense to them, thus generating new ideas and category names.
“An open sort exercise is good at the beginning of the project because it helps you to understand how people naturally categorize information.”
The results from an open card sorting test are helpful in understanding how a target audience structures information, identifies potential bottlenecks, and better label categories and sub-categories.
In a closed card sorting session, you present participants with a pre-selected set of categories and ask them to prioritize and sort cards as it makes sense to them. This type of card sorting is usually used to test if category labels are clear.
You can also use a closed card sort as a follow-up to an open card sort to analyze if the categories identified in the first round make sense to most users.
Hybrid card sorting is a type of card sorting that combines open and closed card sorting methods. Participants can group a set of cards into predefined categories, but they can also create new categories.
You can use the hybrid card sorting technique when you already have certain categories established, but you need input into how the remaining ones should be labeled.
"I use a hybrid card sorting exercise to map all the content of the website. First, I ask users and stakeholders to group the content into categories, and then I get the participants to name or create their own categories."
When improving a live website, the hybrid method might be most suitable. First, you can conduct a hybrid card sorting exercise to see how participants label and sort your website’s structure. You can then follow that up with a closed card sorting session that will help you validate the website’s structure and test if people can find the relevant information. Vaida explains:
"Once the initial structure is set, then you can test it a second time with closed card sorting or a tree testing exercise.”
Moderated card sorting involves including a moderator who will debrief the participants and further probe them by asking questions. This helps give you qualitative insights to understand the rationale behind the grouping.
On the other hand, unmoderated card sorting requires participants to organize content into groups on their own. These sessions can be quicker and easier to organize as they don’t require facilitators, and you can use online card sorting tools such as Maze to conduct remote card tests and collect insights.
In digital card sorting, you can use online tools and card sorting software to simulate the card sorting drag and drop activity of adding cards into groups.
This method is generally easier because the tool will do the heavy lifting of analyzing the results and revealing which items were the most commonly grouped. It’s also much faster because you can get a sufficient amount of data in less time and identify patterns quicker.
Conversely, paper card sorting is a more traditional technique where you write down the topics on physical cards and ask participants to group them on a large workspace, making the process more flexible for the participant.
Identifying the card sorting type that’s best for you comes down to the type of project you’re conducting, your user research goals, and available resources.
This section is supported by insights from Vaida Pakulyte, UX Researcher and Designer at Electrolux
As with any type of research, card sorting requires some planning. In this section, we go through a step-by-step process on how to run a card sorting session.
Before delving into any card sorting sessions, it’s important to define your objectives and research goals. This includes writing down your assumptions and your stakeholders’ to know what to look for during analysis. During the sessions, keep an open mind to new perspectives and ideas from the participants, even if they contradict your assumptions.
“Card sorting enables you to structure your content from the user’s point of view, as opposed to your own or that of your company. So a good card sorting session will always lead you to question your assumptions.”
Carefully curate and prioritize your topic hierarchy based on your research objectives. That means identifying the topic you’ll be testing (e.g. the settings options in your app) and writing down all the ideas that need to go into that.
“It’s important to use card sorting for the right reasons and at the right time in a project. Most importantly, a card sort should be used when you’re looking forward to really getting a deep insight into what matters to your user, and discovering how they prioritize things.”
After you select your topic, it’s time to write the ideas on the cards as words or phrases. For example, suppose you’re trying to group the information that should go into the Setting page of an app. In that case, the cards should include words that represent the different options available, such as About me, Add a credit card, Reset your password, etc.
To make it straightforward for the participants, write one concept per card and ensure the cards are in a random order to keep sorting bias in check.
If you’re running a closed card sorting session, you’ll also have to create the categories that participants will group cards into. In the example above, examples of cards could be Account information, Payment information, etc.
Make sure to include an introduction to the session that welcomes participants to the test and lets them know the goal. Here’s an example:
“Hi, welcome to our card sorting exercise. Your contribution will be used to help organize the content of our website. All you have to do is organize topics into groups as it makes sense to you.”
Finally, prepare the instructions to let participants know what they have to do to complete the exercise. When you’re using a testing tool such as Maze, these instructions are usually incorporated into the platform’s interface and displayed to participants automatically.
However, if you’re running a moderated session, you'll have to prepare these in advance. Here’s some examples of card sorting tasks:
As we saw above, there are different card sorting types—choosing the one that best fits your research project will depend on your resources, budget, and goals.
When choosing the card sorting type you want to run, you’ll have to select about whether you want to do an in-person or remote session, moderated or unmoderated one, and whether a closed or open card sorting session will suit your project better.
Vaida Pakulyte, UX Researcher and Designer at Electrolux, recommends doing a dry-run before running a card-sorting test. Practicing the session with a colleague or a friend will ensure everything makes sense and iron out any potential issues.
It’s important to plan the dry-run in a way that mimics the actual card sort from the way you come up with the instructions and card phrasing to selecting a remote or in-person session. You can also run these sessions with a sample from your actual test participants. This will help you find gaps in your instructions and get real insights.
When the testing session starts, the participants will sort the cards into the categories that make the most sense to them. “Make sure there is enough space either digitally or physically to have all the cards visible so that people can get an overview easily,” Vaida says.
In a moderated session, ask users to think out loud while they are doing an exercise to understand the thinking behind their decisions.
After the card sorting session ends, running follow-up interviews will help you dig deeper into the participants’ mental models and what they thought the cards and groupings represented. This will uncover useful information that will help you come up with actionable items.
“Sometimes I come up with a few questions right after card sorting to ask people about the reasoning they did. Once the sorter is finished, I just take a few minutes to talk through the reasoning behind the positions that they’ve placed the cards in, and take notes. So this is more like a light interview containing a few questions.”
Some questions to guide your interview are:
When analyzing card sort results, start by taking an overall look at the results as a whole. Look at the big picture and try to find common patterns in how the actual cards have been sorted and the category names have been given by participants.
In the results, does anything jump out as surprising? Are there similarities or differences between participant sorts? It’s very important to focus on the big perspective before looking at the nitty-gritty details such as comments, wording, and quantitative data. Ultimately, this analysis will help you understand the information that will be most useful for users to see in a particular group or category.