You may think that the task of teasing out the exact nature of the data and processes within a company is a boring job. Not a bit of it! It is half way between archaeology and forensic science, and I have spent some of the happiest years of my life doing it, mainly as a Data Architect. I’ve done small businesses all the way up to government departments and multi-nationals.
Is the job necessary? I disagree with Yourdon when he says that the detailed understanding of the current systems in place within an organization is unnecessary. Having been reared on SSADM, and used various other methodologies, I’ve experienced cases where a detailed understanding of the current systems has saved an IT initiative from oblivion. Often, the model that you bring into life is the first time that the senior management of an organization has possessed a ‘helicopter –height’ panorama of the flow and transformation of data through the organization. Often, if you take care over the presentation of the model, and communicate it well, it sparks all sorts of ideas to improve the way that the organization works; ideas that often don’t require technology to implement.
The first thing about establishing what really goes on in a company is that
- No one person or group in the organization understands the whole general picture.
- If you’ve got it right, then two data architects working independently will produce a very similar model. They’re uncovering something real.
- Some people who you interview will deliberately try to mislead you
- A few people on the organization are so deluded as to hold entirely erratic views on what is actually going on in a company.
- The senior management generally have only a limited grasp of what is going on in an organization. (I’ve known one or two remarkable exceptions to this)
- The biggest struggle is to understand the broad overview. The greatest mistakes; the ones that cause enormous IT initiatives to fail, come directly from failing to get an accurate consensus of what a company is actually doing before you start your initiative.
Why bother with the long-drawn-out business of getting the data and process architecture of an enterprise right? Often, the data architects are seen as pedantic, doddery old veterans put safely out to grass where they can do no harm. When companies look for cuts in IT expenditure, the Data Architect is usually seen as an easy low-flying turkey to pick off. Nothing stops working when they are given the mantelpiece clock. Likewise, nothing bad happens for a while after you leap off the top of a tall cliff. Data Architects are important. The reasons why include
- If companies can maintain an overall understanding of their data and processes, then ‘Silo-ization’ is prevented. (Silo-ization’ refers to the IT syndrome of having several apparently effective applications that cannot inter-communicate because the data models of each application have entirely different assumptions)
- The data model, if properly drawn out and presented, actually helps the organization to understand what it is doing well-enough to take more intelligent decisions and understand the pressure-points.
- With a good corporate data model, fraudulent financial practices are easier to detect
- When IT projects are started, the vital stages of business and system analysis can kick off much faster as the preliminary work will already be done.
- Staff training and induction is much easier and less costly once you have a well-written and tested data model, because the data and process model is couched in business terms that are familiar to the staff of the company, and years of knowledge can be gleaned in a short time from the model.
- Data ‘warehousing’ in all its many brands and flavors becomes much easier because there is a general agreement on the constitution of every business object. I once worked for a bank where everyone knew what a customer was, but it was only apparent that here was no common consensus when time came to report on the output from two different IT projects.
Maintaining a data model is surely the most important function an IT department can perform. In any place I’ve worked that has managed the job of maintaining such a thing, life for everyone has seemed, somehow magically, more simple and the reason for this hasn’t always been obvious. In fact, whenever I’ve poured through reports of why IT projects have failed, it has always struck me that nobody has mentioned ‘lack of an enterprise-wide data model’, as a good reason for failure. Maybe the task and its end-results just seem so esoteric that it is difficult to see the connection between that and all those little C#/LINQ projects all over the place. Maybe, as well, people have been asking the wrong question. It should be ‘Why do IT projects succeed?’. I’m sure ‘Having a good enterprise-wide view of your data systems’ must be up there.
(p.s. Two years ago, I expanded on this theme, in a rather more humorous vein, in The Data Dialog over on Simple Talk)