Our latest Transform Academy saw Director of DevOps and Quality Engineering, Deepak Selvaraj, in conversation with David Rutter-Close, Principal Test Engineer at the Department for Education (DfE), talking about how they did just that for the department.
This was most definitely not a ‘one size fits all’ approach but judging by the comments and contributions from attendees on the day, sharing the journey helped shine a light on what’s possible and also fostered a further sense of community within the audience.
Each year the DfE handles very sensitive data from 25,000 schools and approximately 8 million pupils, with the additional complexity of 150 different data sources. For over 15 years the department has outsourced the collection, storage and matching of this data to a 3rd party, spending approximately £5 million a year.
In 2021, they decided it was time to bring this process in house. Cue the introduction of Transform to build a data automation solution for the newly created Learner Data Service (LDS) programme, a solution that went on to win the European Software Testing Awards for Most Innovative Project in 2022.
Online events can be tough when it comes to audience participation and to get the conversation started Deepak and David asked the audience to add their 3 main challenges via slido.com when thinking about building an effective QE team in a data centric environment. There were a whole range of answers and humongous data quantity, process chaos and upskilling were all great shouts. Number one, though, was having the right data to test. Getting accurate test data that’s as similar as possible to production data is key to making sure the testing is good enough.
David moved on to explain to the audience the challenges that DfE and Transform faced and the 5-staged approach they adopted.
Automation Experts in Data: ETL testing and front-end testing are different disciplines – massive data sets undergoing huge transformation require skill sets that take years to learn properly and QA Engineers needed to be upskilled to support automation practices.
Lack of a centralised framework: DfE didn’t have a centralised framework and where they did have some automation, programming languages, tool sets and approaches were very random. This resulted from each project being treated as a separate business – they had different suppliers and people weren’t working together. When they started building a unified framework, they had to decide which programming language to use. A decision was made to use the one developers used, because they quickly needed their help, especially in the early days.
Agility and Autonomy: agile working can be particularly difficult in the data space, because data automation is not a one sprint job. Frequent requirement changes and multiple iterations of code changes led to longer delivery cycles and long, manual test cycles impacted agility.
Environment Provisioning: inconsistent and manual data provisioning impacted scheduling; with massive data sets come many variations especially when we’re talking about almost every child in the UK. These variations are in there and can be difficult to test. The automated solution Transform and DfE developed was key to solving the problem of spinning up extremely large data sets automatically.
Collaboration, Process & Governance: it’s crucial to bring the community together and facilitate discussions. As is documentation – tracking and keeping records of all the processes so people onboarded later in the project can be brought up to speed more quickly.
Before this project, David explained that DfE didn’t have automation in the data space. Data was viewed much like how the internet used to be, still in its infancy but growing at pace. Now however, the department has implemented a number of QE strategies - from introducing agile led shift-left test approach to increased speed delivery with early automation - and can demonstrate the transformational effects of these.
He also mentioned a recent discussion with a tester about how much manual work was still needed on data sets. The answer was almost nothing.
Culturally there were a number of challenges they encountered while building the solution. For example, moving from two agile teams to ten in just two months and over Christmas: testers are not too hard to find but finding ETL testers who could also do front-end testing was way more difficult. They made the decision to move outside IR35, creating new roles (including Test Coordinator and Lead Technical Tester), drafting a resource mapping sheet to identify both existing people and gaps that needed to be filled.
Finally, David talked metrics. Automation saved many hours of testing and, after the initial investment, DfE is now able to implement the same framework in other teams. Speaking of success, a highlight for the team involved was the Learner Data Service programme winning the Most Innovative Project category at the European Software Testing Awards in 2022!
Wrapping up the session, Deepak talked through the two main areas of focus for this partnership going forward: performance testing and a dedicated chatbot for testing. And that was that. The session was over. Sixty(ish) minutes of knowledge sharing with an opportunity for the audience to contribute and ask questions. Exactly what we hope for at a Transform Academy.
If you’re interested in chatbots, we’re hosting a meetup on May 18th in our London office where we’ll share a demo of how the DfE one works. Please get in touch with us at talent@TransformUK.com if you’d like to come along.
If you’d like to re watch the full recording of the session, just drop us a note and we’ll be more than happy to send you a link.
If our work with DfE has sparked your interest in how we can support your organisation in the Quality Engineering space or you’d like to be informed about future Transform Academies, please do get in touch at transformation@TransformUK.com for both queries.
