What is a card sort?
Card sorts are a generative research technique we use in early-stage user research. We give participants a selection of pages from a website and ask them to organise the information into groups that make sense to them. We typically conduct a card sort with at least 15 people. Results from this many participants helps us find patterns across data and ultimately generate potential information architectures (IA). Once we’ve designed an IA we then validate and iterate on it, usually using a tree test and/or a prototype.
Running a card sort in Optimal Workshop
We use Optimal Workshop (OW) for card sorts. We can quickly deploy a small card sort study using this tool. It also comes with a suite of analysis tools to help you find patterns amongst the resulting data. If you keep the card numbers reasonable – 30 cards or fewer – it's quite easy to quickly deploy an unmoderated card sort and let participants complete the study independently in their own time.
Doing more with a card sort
Optimal Workshop allows for three types of card sorts:
- closed card sort – participants group information into pre-defined categories that are given to them
- open card sort – participants organise information into categories that make sense to them and create their own labels for each category
- hybrid card sort – participants organise information into pre-defined categories and also optionally create additional categories and labels as they need
But what if you want to get more out of your card sort?
During our intranet alpha we wanted to know how participants label and group intranet content. But we also wanted to know:
- what information do we have that staff really don’t need?
- what information do staff need that we aren’t currently providing?
- who is the information for?
There's no easy or obvious way to build on a card sort in OW and get this information.
Running a card sort in Miro
Before the pandemic if we wanted to run a more elaborate card sort, we'd schedule a face-to-face session and use paper cards. To replicate this experience in a remote session we looked to Miro, a collaborative online whiteboard application.
We started by looking into accessibility. We learned that neither Miro nor OW were screen reader accessible. Because of this we screened out participants who relied on a screen reader or keyboard. We will include these users in subsequent research.
We also knew from other workshops that there was a learning curve using Miro, and that not everyone got along with the tool. There were a few things we did to mitigate against this:
- we created a mini Miro tutorial that explained how to complete study tasks in Miro
- we set aside 10 minutes of the study for participants to familiarise themselves with Miro
- we piloted our Miro card sort with a colleague that had really struggled to use Miro.
From the pilot we learned that if a participant used a mouse it was much easier for them to navigate the board. As a part of our recruitment messaging, we strongly recommended participants had a mouse ready to use for the session.
Additional benefits of using Miro
Besides being able to add additional sorting tasks to our study there were a few other benefits of using Miro. We found we could:
Randomise the cards we presented users
To do this we uploaded our list of cards into this online randomising tool. We saved the random order into a spreadsheet. Then we used the bulk upload feature within Miro to create the study cards in one step.
Set up the Miro board for each card sort at the start of the study
We created a separate board for research materials and notes. For each participant we created a frame with the randomised cards and activities on this board. Then on the board used for the actual sessions we simply swapped in a prepared frame. This also allowed us to use the same Miro board for every research session and avoid confusion.
Reveal activities as the session progressed
We placed each activity in a different frame. So we didn’t overwhelm or distract participants we only revealed each activity as we went along. To do this we used the hide frame functionality.
Better save the card sort
With a paper card sort we used to take photographs of the card sort. This takes time and it’s easy to accidently lose track of a card when cards are placed one on top of the other. A Miro card sort retains the card sort in 3 dimensions eliminating some of the problems that can occur with taking photos of paper card sorts. We were also able to copy the entire completed card sort and use the copy for analysis, leaving a record of the final card sort as the participant had created it.
Screenshots – a final piece of advice
In pre-pandemic days we would take photographs of the card sorts as they developed or between different activities. There’s value in having an observer take a screenshot at the card sort at key moments. Make a note of these times in the test script, and if this is uncommon mention it to notetakers who may not have had time to read the script beforehand. Screenshots allow you to see how participants had initially grouped things before a subsequent task, where they may re-group and move cards around. This also saves time going through video recordings trying to understand which group a card might have initially been placed in.
3 comments
Comment by Hannah posted on
This is a really interesting case study, thanks!
Did you run these sessions 1:1? How did you synthesise the card sorts if so (and how long did that take)?
Cheers.
Comment by Helen Calderon posted on
Hi Hannah,
Yes, we ran 6 face-to-face card sorts one on one. We also ran an unmoderated card sort alongside it. That had the same cards but with fewer tasks and was completed by 11 people.
The synthesis could probably fill another blog post! I started by putting the cards and participant labels in a big spreadsheet. I also used Miro to look at the moderated card sorts and unmoderated card sort reports together. We used a separate spreadsheet to create possible site structures or trees.
First I tried to find the groups of content, then I looked at possible and common labels for groups. We had data on which content could be streamlined and which content was missing so I also reviewed that. I created a couple of trees from the data. I then met up with the lead content designer and we combined the research findings with their expertise to come up with one tree that we then tested. I want to say it took 1-2 days.
Helen
Comment by jero posted on
Hello
Thank you for sharing your experience, it is very interesting.
I look for doing also cardsorting with Miro because we want to use some visual pieces/components.
I wonder how to collect and synthesise, could you please share some screenshots (just the structure could be fine), I find very interesting understand how you solve it
Thank you in advance