I'm Chris Babajide and I'm a digital analyst in the Service Design team. It's my job to help ensure we use measurement in a meaningful way. In this post I'll talk about how we developed a measurement and KPI framework to track the efficacy of recent content updates and so that all sites can be measured in a meaningful way against a consistent baseline.
We've recently been updating the Information Advice and Guidance content for Adult Social Care on the Living Well website. Now some of the content is live, we need to monitor its performance. We need to know which pages require further tweaking and whether the improvements have made a positive difference.
To do this we've set up a measurement framework: basically a short document which translates desired user behaviours into measurable metrics and dimensions.
How to measure success
Living Well is mainly a content site. It makes sense that anything we measure would be around content engagement: how many people are reading the material, for how long, that sort of thing. There's many different things we can measure so its important the measurements map to our objectives.
We initially started a matrix to map our objectives to means of measuring them. There were 3 broad themes:
- Provide engaging content: users find the website easy to navigate and easy to read
- Provide effective content: users get what they need from the website, it works for them
- Act as a useful repository of content: the website is an information resource which people link to and from
Each of these we gave goals to, and each goal had several KPIs, for instance, pages per session or number of views.
Choosing some of these KPIs can be counterintuitive. For some websites, having users spend a long time on a page would be considered successful. If the objective is for users to be able to find information quickly, then a long time on page may be an indicator that the user is struggling to find the required information and the pages need to be reworked.
Implementation
After working out the KPIs we add code to the pages that helps us see what people are clicking on or how long they spend on the page or how far they scroll down it.
Bumps along the road
We can change most of the content on Living Well via its content management system, but much of the code that is relevant to analytics is held by a third party, which has slowed us down. What we’ve learnt is we need to have the admin rights and where we have commissioned sites we need to make sure our ownership of the analytics code is built into that commission.
How long to wait
After implementation, an important next step is to agree the format and frequency of reporting. After an update to pages, we generally wait a period of time before looking at user behaviour. For the Living Well pages, we decided that 8 weeks would be an appropriate period of time.
After the results...
We're coming up to this period for some of the Living Well updates to Blue Badge; Charging for Care, Carers and Dementia content. Once we’ve got the numbers, we’ll analyse them and translate them into actions, asking did we get it right? Has it reduced phone calls and traffic to contact us pages etc. We can then, in the agile way of working, iterate accordingly.
Leave a comment