GOV.UK Verify is a federation of many systems, including those of the 8 different certified companies who verify users’ identities, and a growing number of government services.
The main parts of GOV.UK Verify are: the hub (which allows communication between the user, the certified company, and the service on GOV.UK); the certified companies’ service (which verifies your identity and confirms it to government); and the government services which use GOV.UK Verify. There’s also the Document Checking Service, which allows the certified company to check digitally that your given information matches a valid record.
In this post I’m going to explain some of the tools and systems that support these two important components, as well as those that we operate that help the federation to run smoothly and securely. This is also something that is considered under Point 6 of the Digital by Default Service Standard assessment for taking a service live, which says:
Evaluate what tools and systems will be used to build, host, operate and measure the service, and how to procure them.
The codebase for GOV.UK Verify is mainly java microservices at the backend. At the frontend we use Jade as our templating language. However, we’re currently rebuilding the frontend of the service in Ruby on Rails, as we move towards going live, as part of our overall plan to open up more of the service, including starting to publish source code and work in the open. We’ll talk more about opening up code soon.
Open source tools
To build and run GOV.UK Verify, like most digital operations, we use a large number of open source tools, frameworks and libraries, as well as paid for cloud services, some of which I will explain here.
The GOV.UK Verify hub does not store or log user’s personal data. However, we do log events for monitoring and audit purposes. For these we use Mongodb and Elasticsearch. We find that these offer easy ways to horizontally scale, and allow us to easily carry out searches against these logs.
Building and deploying
The GOV.UK Verify technical team is able to release code into production several times a day, if necessary. In order to do that reliably and repeatably, we use a lot of automation.
Our build pipeline uses Jenkins, a popular open source continuous integration tool, to support building, testing, deploying and automation. When a build is created a suite of automated tests runs, to ensure the build is viable. We also run nightly performance tests to ensure that our builds will not introduce performance issues. We use Gatling to run these tests. We have a gradle gatling plugin in the open that is used by a few other projects outside of our programme.
To deploy our builds we use Jenkins, a popular open source automation server, and Fabric. Fabric gives us flexibility to manage how we deploy our code they way we want to. For managing infrastructure provisioning we use Puppet and ansible, which are open source tools that help our web operations engineers automate the many repetitive tasks they regularly perform.
We realised early on that we needed to do a lot of work to manage our repositories as artefacts through our build pipeline. For this, we make use of aptly, which is an open source tool which helps us keep control of changes in our environments.
We manage the majority of our code in an internal GitHub repository. However, we plan to increase our current usage of the public GDS repository in the coming months.
Logging and monitoring
We’re using sensu and blinken to provide our team with alerting, and we use pingdom for lightweight external monitoring to help us spot problems outside the hub service. We use pagerduty as part of our 24/7 monitoring solution. We’re currently trialling dashing, a alternative framework for building dashboards.
Analytics and performance
To gather performance data for the service we create our own logging. During the early days of private beta, we started measuring performance using our own logging system, but we now make use of Piwik, an open source web analytics system, which we host ourselves. We chose this so that we could avoid sending data to third party web analytics companies.
The GOV.UK Verify hub, and the Document Checking Service are hosted on an Infrastructure as a Service platform, using virtual servers, which allow us to create and remove infrastructure on-the-fly. This allows us to scale relatively quickly, and makes it easier to make changes. We migrated infrastructure providers previously on the project as we moved from alpha to private beta.
Public Key Infrastructure
We also run a PKI to enable secure communication between the entities in the GOV.UK Verify federation, for example, between government services and the GOV.UK Verify hub.
Communicating and planning
Making communication simple and effective within the programme is essential. Like most Agile teams, we believe in manifesting what is currently happening on the walls in our workplace. But we also use digital tools. For quick communication, we use a variety of slack channels. We do most of our planning in Mingle. We’ve also used Pivotal tracker in the past, as well as Trello for short term small projects and discoveries.
While the majority of support in GOV.UK Verify is handled by the certified companies, we have a small team that provides support to the users who contact us through the link on the GOV.UK Verify hub. For managing this and our own support tickets we currently use Zendesk. We’ve also used RT in the past.
I hope this post has been useful to others building platforms, we will continue to evaluate our technology and tool choices as we grow over the coming months and years. Let us know what tools you find most helpful in the comments below.
Subscribe to the blog for more on how GOV.UK Verify works.