As the world is becoming more data-driven, attitudes are changing towards data. Recognition for the importance of data architecture is becoming more prevalent in the private sector, and government departments too are recognising the need for more modern data architecture.
A more modern data architecture will provide the governance, technical assurance and strategic guidance for the management of data and metadata for the full lifecycle of that data – from acquisition through to provisioning and the ultimate archiving or purging of the data. The diagram above shows just some of the complex dependencies which must be managed during development of new data architecture capabilities.
Like many large organisations which have been around for a while we have a lot of legacy systems which use older technology or bespoke solutions to hold our data. These systems are often difficult to map out, connect with and test. It’s important for any new government data architects to understand enough of these systems and what other services they relate to, so that we can properly determine what our options are. This journey is currently underway in most government departments. In many cases, we are building new systems which must still interact with those legacy systems to connect with the data, which after all is at the heart of this process.
Building connections
There are many ways to build these connections. For example, many government departments are looking to use technology such as data lakes as central repositories, to remove these data silos and exploit their data more effectively. Some departments are also looking at how artificial intelligence and machine learning could help us, but for these initiatives to be successful it is crucial for us to improve our data quality and integrity, and a key challenge faced across government is to find the most efficient and cost-effective ways to do this.
However, this is just one of myriad challenges faced by an organisation going through a data architecture transformation of this scale. Others include:
- recruitment and retention of expertise in data architecture, to guide us as we move away from many legacy systems and siloed data towards integrated and centrally-stored data platforms
- maintaining any legacy systems which are still necessary in some form, and deciding how long this should last
- this is a big culture change for many departments, so it is important to ensure we have the correct governance in place and that we will not be creating the same silos again
- establishing proper data strategies
- fitting data architecture alongside other architecture disciplines across the government professions structure
Exciting times
These are very exciting times for data architecture with many important transformations happening across government. I hear a lot of phrases and buzzwords such as ‘one version of the truth’ and ‘open data’ but to truly understand what they mean and the work involved in developing and implementing this kind of change will require us to embrace new ways of working as we develop solutions to the questions above.
Some departments are further along their transformation journeys than others, and collaboration initiatives like the ongoing Government Data Architecture workshops and the establishment of a community of interest will help us all to share and learn from each other’s experiences.
Whilst we still have a long way to go before we have mature data architecture capabilities within all government departments, I feel we are heading in the right direction. With the data we create and hold, we have a huge opportunity to improve our services – and data architecture is a fundamental platform for us to achieve our potential.
Those are my thoughts, but what do you think? Leave a comment below.
Leave a comment