We have just reached a new milestone on GOV.UK Verify - our 100th round of usability testing.
100 rounds of usability testing is certainly a lot. Here are some numbers to put it into context:
- 600 users
- 600 hours in the lab
- 500 hours of analysis
- 200 hours of presenting results and prioritising issues
- 30,000 sticky notes
And that’s not all: those 100 rounds are in addition to a range of other research we do: large scale remote usability testing, contextual research in users’ homes and job centres; customer support queries and feedback; analytics; A/B testing; accessibility testing.
Why so much research?
Traditionally, some companies have often only conducted one round of usability testing on a product or service to fix issues before launch. While some research is better than no research, I would argue that there is limited value in testing a product once, “fixing” the issues, and launching it. How do you know that the solutions you have come up with have addressed the issues you observed, unless you test it again? And what if you discover a major problem for users but it's too late for you to fix it?
At GDS, we believe that continuous iteration of research is the key to building services that meet user needs.
Our Service Design Manual states that user research should be done in every iteration of every phase - starting in discovery and continuing through live.
Through an iterative design and testing process, we can constantly assess our service to understand if it meets user needs, test new design ideas, identify new issues, and check if any previous issues have been solved.
This diagram shows the iterative user research process that we follow in our work. It also demonstrates that, once issues are solved we can A/B test to quantify whether these solutions work at scale, or push the solutions into the live service.
This quantity of research does come with some challenges...
Doing 100 rounds of usability testing means that we have a really detailed understanding of our user needs, and a lot of insight into what works and what doesn’t work in terms of designing the GOV.UK Verify journey.
However, it also comes with a number of challenges:
- Clear documentation and effective indexing is essential so that it’s possible to understand what design solutions have been explored in the past and what was learned; understand why particular design decisions have been made; go back and explore issues that may not have been in focus at the time
- Staff changes over the 100 rounds mean that knowledge can be lost (making the documentation particularly critical)
- Keeping the wider team engaged when, sometimes, what we are doing is only incremental. For this, it’s essential to share the longer term goals of what we are working towards so they can understand the context
- Having the courage to go back to design ideas that may not have worked in the past to see if attitudes may have changed, for example attitudes to social media.
What has changed as a result of research?
Here’s what GOV.UK Verify looked like in December 2012, before usability testing started:
And here’s what it looks like now:
100 rounds is just the beginning…
GOV.UK Verify is live, but we are not finished doing research. We will continue to iterate our design and do continual testing, and keep working to make the service as simple and straightforward as possible for users.
For regular updates on GOV.UK Verify’s development, subscribe to the blog.