Card Sorting: Improving Information Architecture

In this chapter, we zoom in on card sorting to learn how this UX research technique works, and share a few practical tips on how to run an effective card sorting session. 

Card sorting is a UX research method that helps you discover how people understand and categorize information. In a card sorting study, participants group ideas or information written on cards into categories in a way that makes sense to them. 

Using the insights gathered from the sort results, you can:

• Create an information architecture (IA) for your website or homepage that is intuitive and easy to navigate
• Learn how people understand different concepts or ideas
• Understand where users expect certain content to be found
• Get inspiration for labeling and grouping content.

While card sorting shares the same goal of content categorization as tree testing, each method approaches this from a different perspective and is used at various stages during the research process. 

When you plan a new website and IA, involving users in the process as soon as you can is essential. Card sorting is useful in this respect, as it helps you to understand how users group similar information on your website or in your app. On the other hand, tree testing is meant to help you test the current website’s navigation or the categories identified during a card sorting session. 

If you want to test how your current IA performs, you can start with a tree test to identify user experience issues and collect data benchmarks for subsequent comparisons. If the tree test results indicate most users find it hard to navigate your website or a part of it—and you decide to redesign it—a card sorting session makes sense next.

Conduct a card sorting session to understand how your users organize the content in a way that makes sense to them, independent of how your current website is organized. 

Then, after you have redesigned the website based on the data collected during card sorting, you can do a new tree test with the redefined architecture to make sure it performs better than the previous one.

In the next chapter, we go over how to run a tree testing session in detail. Ultimately, it’s helpful to combine both UX research methods when working on your information architecture (IA) to create an intuitive and easy to use website.

There are three types of card sorting you can choose from:

• Open card sorting: participants sort the cards by groups and then name the groups themselves
• Closed card sorting: a moderator gives participants the defined categories to sort the cards into
• Hybrid card sorting: some categories have already been defined for participants to sort the cards into, but participants also can create their own categories

Depending on your objectives, you can also choose from different card sorting formats:

• Moderated vs. unmoderated card sorting: A researcher can conduct these (moderated), or users work through the session on their own (unmoderated)
• Paper vs. digital card sorting: you can either write the topics on physical index cards (paper) or type them onto cards in a simulated environment (digital).

When it comes to selecting the right type of card sorting, Vaida Pakulyte, UX Researcher and Designer at Electrolux, explains:

Open card sorting

Open card sorting is a generative research exercise, rather than an evaluative one. In this card sorting type, participants organize topics into categories that make sense to them, thus generating new ideas and category names.

The results from an open card sorting test are helpful in understanding how a target audience structures information, identifies potential bottlenecks, and better label categories and sub-categories. 

Closed card sorting

In a closed card sorting session, you present participants with a pre-selected set of categories and ask them to prioritize and sort cards as it makes sense to them. This type of card sorting is usually used to test if category labels are clear.
You can also use a closed card sort as a follow-up to an open card sort to analyze if the categories identified in the first round make sense to most users.

Hybrid card sorting

Hybrid card sorting is a type of card sorting that combines open and closed card sorting methods. Participants can group a set of cards into predefined categories, but they can also create new categories.
You can use the hybrid card sorting technique when you already have certain categories established, but you need input into how the remaining ones should be labeled.

When improving a live website, the hybrid method might be most suitable. First, you can conduct a hybrid card sorting exercise to see how participants label and sort your website’s structure. You can then follow that up with a closed card sorting session that will help you validate the website’s structure and test if people can find the relevant information. Vaida explains:

Moderated vs. unmoderated card sorting

Moderated card sorting involves including a moderator who will debrief the participants and further probe them by asking questions. This helps give you qualitative insights to understand the rationale behind the grouping. 

On the other hand, unmoderated card sorting requires participants to organize content into groups on their own. These sessions can be quicker and easier to organize as they don’t require facilitators, and you can use online card sorting tools such as Maze to conduct remote card tests and collect insights.

Digital vs. paper card sorting 

In digital card sorting, you can use online tools and card sorting software to simulate the card sorting drag and drop activity of adding cards into groups.
This method is generally easier because the tool will do the heavy lifting of analyzing the results and revealing which items were the most commonly grouped. It’s also much faster because you can get a sufficient amount of data in less time and identify patterns quicker.

Conversely, paper card sorting is a more traditional technique where you write down the topics on physical cards and ask participants to group them on a large workspace, making the process more flexible for the participant. 

Identifying the card sorting type that’s best for you comes down to the type of project you’re conducting, your user research goals, and available resources.

As with any type of research, card sorting requires some planning. In this section, we go through a step-by-step process on how to run a card sorting session.

Step 1: Document your assumptions

Before delving into any card sorting sessions, it’s important to define your objectives and research goals. This includes writing down your assumptions and your stakeholders’ to know what to look for during analysis. During the sessions, keep an open mind to new perspectives and ideas from the participants, even if they contradict your assumptions.

Step 2: Choose a topic based on your research goals

Carefully curate and prioritize your topic hierarchy based on your research objectives. That means identifying the topic you’ll be testing (e.g. the settings options in your app) and writing down all the ideas that need to go into that.

Step 3: Prepare the card sorting test

After you select your topic, it’s time to write the ideas on the cards as words or phrases. For example, suppose you’re trying to group the information that should go into the Setting page of an app. In that case, the cards should include words that represent the different options available, such as About me, Add a credit card, Reset your password, etc. 

To make it straightforward for the participants, write one concept per card and ensure the cards are in a random order to keep sorting bias in check.

If you’re running a closed card sorting session, you’ll also have to create the categories that participants will group cards into. In the example above, examples of cards could be Account information, Payment information, etc. 

Make sure to include an introduction to the session that welcomes participants to the test and lets them know the goal. Here’s an example:

“Hi, welcome to our card sorting exercise. Your contribution will be used to help organize the content of our website. All you have to do is organize topics into groups as it makes sense to you.”

Finally, prepare the instructions to let participants know what they have to do to complete the exercise. When you’re using a testing tool such as Maze, these instructions are usually incorporated into the platform’s interface and displayed to participants automatically.
However, if you’re running a moderated session, you'll have to prepare these in advance. Here’s some examples of card sorting tasks:

• Take a quick look at the list of items to the left. We'd like you to sort them into groups that make sense to you. There is no right or wrong answer. Just do what comes naturally.
• Drag an item from the left into this area to create your first group.

Step 4: Choose your card sorting type

As we saw above, there are different card sorting types—choosing the one that best fits your research project will depend on your resources, budget, and goals. 

When choosing the card sorting type you want to run, you’ll have to select about whether you want to do an in-person or remote session, moderated or unmoderated one, and whether a closed or open card sorting session will suit your project better.

Step 5: Run a dry test before doing a real session

Vaida Pakulyte, UX Researcher and Designer at Electrolux, recommends doing a dry-run before running a card-sorting test. Practicing the session with a colleague or a friend will ensure everything makes sense and iron out any potential issues. 

It’s important to plan the dry-run in a way that mimics the actual card sort from the way you come up with the instructions and card phrasing to selecting a remote or in-person session. You can also run these sessions with a sample from your actual test participants. This will help you find gaps in your instructions and get real insights.

Step 6: Launch the study 

When the testing session starts, the participants will sort the cards into the categories that make the most sense to them. “Make sure there is enough space either digitally or physically to have all the cards visible so that people can get an overview easily,” Vaida says.

Step 7: Hold follow-up interviews for additional insights

After the card sorting session ends, running follow-up interviews will help you dig deeper into the participants’ mental models and what they thought the cards and groupings represented. This will uncover useful information that will help you come up with actionable items.

Some questions to guide your interview are:

• “How easy or difficult did you find this card sorting exercise?”
• “Which cards, if any, were most difficult to sort? Why?”
• “Which cards, if any, were difficult to understand?”
• “What groups, if any, were difficult to name? What groups, if any, were easy to name?”

Step 8: Analyze the results

When analyzing card sort results, start by taking an overall look at the results as a whole. Look at the big picture and try to find common patterns in how the actual cards have been sorted and the category names have been given by participants. 

In the results, does anything jump out as surprising? Are there similarities or differences between participant sorts? It’s very important to focus on the big perspective before looking at the nitty-gritty details such as comments, wording, and quantitative data. Ultimately, this analysis will help you understand the information that will be most useful for users to see in a particular group or category.