The User Research team were contacted by colleagues within the design team at Shelter after they had read one of our blogs: User testing: creating our own user panel. This is a panel of now 300+ Essex residents who have agreed to help us with user research and testing. Our user researcher Narelle set up a meeting with the team at Shelter, to talk about the experiences with our user research panel. The Shelter team intrigued to know more had heaps of interesting questions:
- How often do you engage with the panel, and how?
- How do you decide when to use the panel?
- How is the panel curated and managed?
We’ve summarised the discussion we had with Shelter about these points below.
How often do you engage with the panel, and how?
When we started the panel, the aim was to engage with the panel every six weeks or so, but this hasn’t always been easy! Sometimes we might engage with the panel a little more often, and sometimes a little less often, depending on the types of projects we are working on. We are fortunate that our user research team is small: we check with each other to ensure that we aren’t drawing on too much of the panel’s time, but also, that we are keeping them engaged.
We have a few rules for engaging with the panel:
- Give feedback. We aim to feedback what we have learnt to the panel. We do this via short emails after we have analysed data, to share specific findings from projects they have helped with, and via a newsletter outlining broader work of the service transformation team. This is our way of demonstrating how their contribution has had an impact and how thankful we are for their help.
- Keep it short. We try to keep most of our asks of the panel ‘short and sweet’. We aim for each research or testing piece to take the panel no more than 7 minutes to complete, as we know this is less onerous for our residents. We say in the email subject how long the piece will take, so that the participants can mentally prepare themselves for the task ahead and battle it at an appropriate time.
- Keep it fresh. We also try to keep the research fun and interesting—we’ve found that the panel enjoy card sorting, tree-tests and click-tests much more than they do regularly surveys! We also try to reduce any unnecessary instructions/ details about the research: we share enough to understand what we are asking for, but no more.
How do you decide when to use the panel?
We use the panel when we have short and snappy research tasks, that anyone from Essex would have insights into. The panel is particularly useful when we want to undertake quick card sorts, tree tests or click tests, to see how our design ideas are working quickly with a small audience.
It’s not always appropriate to use our panel. For example, we would not use our panel when:
- We need to engage with a very specific group of residents, for example, young people who use youth services, who we know would not be reflected in our panel.
- We want our responses to be reflective of a whole population—for example, a resident survey that would be used to plan and manage services across the whole county.
- When we are planning to gain a large amount of information from respondents— If we do call on our panel for larger research pieces, we leave participation very much in their hands—we would use the panel as a screener (to identify those relevant to our research brief) and then ask them if they would be happy to be involved in a longer research piece.
How is the panel curated and managed?
Doing this well can be really hard! It takes time to manage the panel and keep it up to date. Here are some ways we try to manage this:
- We have a simple, secure Excel file for managing our panel. This file is only accessible to the user research team. This generally meets our needs, but we are looking into better systems to manage our panel, such as a more sophisticated database platform.
- We store only key information about our panel members, information that helps us decide which panel members to call on for certain research pieces. For example, we store information on whether the panel member has ever worked for Essex County Council, and in what field. Where appropriate, we would choose not to invite these members to a research piece: they may know too much about the content area to best reflect true resident experience.
- We provide the option for residents to ‘opt-out’ of the panel, each and every time we contact them. Any time a resident opts-out, they are thanked for their time and instantly removed from the panel.
Leave a comment