Case Study: Collecting user feedback on alpha and beta versions of GOV.UK
About the project:
GOV.UK is the recently launched replacement for the Directgov and Businesslink websites, developed by the Government Digital Service (GDS). Developed in the open, the alpha and beta versions of GOV.UK were released to the public with the express intention of gathering as much feedback from actual site users as possible.
High quality feedback was vital to the development of GOV.UK because we wanted to engage users directly in the design of the user experience of the site. As well reporting problems, we wanted a forum where users could ask questions and make suggestions for improvement. It had to be low-cost, easy to set-up, reliable and user-friendly.
We chose a customer service forum, GetSatisfaction. The product works pretty much ‘out of the box’, configuration was via a web interface and the monthly cost was $19. It also provides a feedback ‘widget’ or button, which you embed in your own site and links users to the GetSatisfaction forum.
After the beta release of GOV.UK in January 2012, up to 22 GDS staff responded to users’ feedback (usually spending no more than 30 minutes a day). The forum was closely monitored in the immediate aftermath of the alpha and beta releases and there were three people who were administrators of the service. In the busy periods after releases there was one person monitoring the forum full-time.
During the alpha and beta phases, 1,045 different topics were posted in the forum by just over a thousand individual users. The topics ranged from bug reports and detailed technical queries to suggestions for new tools, features and content. Users of the forum came via the feedback pages on GOV.UK and we gleaned a large amount of valuable data and insight.
Many of the forums posted in GetSatisfaction resulted in tangible changes to GOV.UK, particularly around search and the layout of pages. Of particular benefit were regular bug reports detailing inaccuracies with ‘Find my nearest’ searches on both the alpha and beta versions of GOV.UK.
Testimonials & lessons learned:
GetSatisfaction performed well as one avenue for users to give detailed feedback during the development of GOV.UK. It was possible to have detailed discussions with users out in the open and some of the ideas received had a direct influence on the direction of GOV.UK.
GetSatisfaction and services like it, e.g. UserVoice, are valuable tools for user engagement, debate and collecting detailed user feedback during product development. However, this is not an appropriate channel if you anticipate you will collect any sensitive or personal data, note GDS is not using it on the production version of GOV.UK.
The GetSatisfaction service provides a management console that filters topics by type, i.e. questions, ideas, problems and praise from which we were able to compile summary reports. This was useful to get a snapshot of sentiment around the product and a sense of which precise usability and technical problems users encountered.
As a cautionary note, bear in mind GetSatisfaction was just one avenue of communication among many for our users. We also had a dedicated email for feedback, Twitter, the GDS blog and other tools to help us manage customer contact. Some users complained about having to log in to the forum, either by creating a GetSatisfaction account or using a Facebook or Twitter log in. We should have made alternative feedback mechanisms more obvious to those people who were uncomfortable about the log-in.
Like other social media, with a product like GetSatisfaction, the more time and effort you put in to engage users, the richer the output. If you respond quickly to user queries, the more likely people are to post. For the alpha release, the development team was small so everyone was active on the GetSatisfaction forum and the results were excellent. On the beta release, while still valuable, because there were fewer people directly answering user queries, more useful feedback came via email.
On balance, GetSatisfaction proved valuable as part of the product engagement supporting the development of GOV.UK and we would certainly consider using it, or a similar service again if the need arises.