Licence: Creative Commons Attribution
Who needs Data Architecture?
Anniversaries are a time of reflection, so it’s natural to look to the past before we start thinking about the future. For me that means thinking about the last year working as the Data Architecture lead here in Defra, and what we have achieved in that time.
It doesn’t seem like that long ago that I joined the Defra Data Transformation Programme, with a view to setting up a Data Architecture Framework to establish getting some of the core fundamentals right to enable a data-driven Defra. At that point data architecture was all new to Defra: while there was a well-established community of Enterprise, Technical, Security and Solution Architects, the term “Data Architect” was not in common use. In fact, it was seen as something that could be covered by other architects.
Times have changed
Fast forward a year, and things couldn’t be more different. Now I am regularly contacted by project leads asking whether they need a data architect, if I could supply one, or how they could recruit one. It’s great to see the value of Data Architecture being recognised in Defra, as it should be in any data-driven organisation.
Why data architecture?
In spite of all this, I am still regularly asked: what is data architecture? Who needs it? And, of course, does data architecture work in an agile environment?
In response I always say that almost everyone needs a data architect for almost every project imaginable. Whenever there is a question that needs answering, data architecture has a pivotal role in finding that answer and implementing it in a meaningful way. This is particularly true when working on transformation projects.
According to the government’s Digital, Data and Technology Profession guidelines, a data architect sets the vision for an organisation’s use of data, through data design, to meet business needs. Together with others in technical and business design roles, data architects define data strategies and principles, and ensure systems operate in line with established data policies.
To put this in context, here are some examples of what we do in transformational projects here at Defra:
Data Models: This is commonly associated with Data Architects, but contrary to popular opinion they aren’t something we spend four months defining whilst sat alone in a dark room with a team of eager developers wanting to push out wire frames. They can be produced iteratively, ahead of sprints, to guide those lovely front ends. Collaboration is essential.
Data Analysis: As architects, we can provide the most pertinent facts and edge cases for each project, and we can spot things that are hidden within a database where business users can’t see them. Assumptions are made on data, but more often than not, these assumptions don’t tell the full story. We need evidence, and that is where data architects come in.
Data Migration: If you believe what some people say, this is one of the easiest tasks in the world of data. But we know this isn’t the case, and we spend a considerable amount of time thinking and fact-finding to ensure migration is done properly.
Data Quality: We can help to understand what the business really needs from its data by profiling it and ensuring it fits with the appropriate data standards.
Reference Data: I could write a whole blog post on this (and I probably will…) because this is so important in our work. As data architects, we work to understand how reference data is used, and where it can best be sourced and supplied from.
Archiving, retention and storage: We work closely with Security and Knowledge and Information teams to understand policies and requirements in order to advise on the best policies for keeping, or disposing of, data.
Reporting and Analytics: And finally, we know that data is no use unless it is being used and properly understood. So we work with the business to design data warehouses and data marts for optimal operation.
We have to work with others if we want the best results
Data Architecture isn’t an isolated activity done by purists in ivory towers. It’s a collaborative effort with other architects, developers, the business, users and delivery managers, and it works best when we’re embedded in delivery teams. At times we can be seen as pedantic by those who want a quick win, but we’re there to make sure data is treated as it should be - at the forefront of design and retaining its value, wherever it sits, and whoever looks after it.
So who needs Data Architecture? I think all organisations do, and certainly anyone working on transforming data or digital services. If you’re still not convinced – or just want to talk further - please feel free to contact me and I will be happy to talk about how our Data Architecture Framework and ways of working can help you.
Leave a comment