SQL Clone
SQLServerCentral is supported by Redgate
Log in  ::  Register  ::  Not logged in

The Joy of Data Modelling.

You may think that the task of teasing out the exact nature of the data and processes within a company is a boring job. Not a bit of it! It is half way between archaeology and forensic science, and I have spent some of the happiest years of my life doing it, mainly as a Data Architect. I’ve done small businesses all the way up to government departments and  multi-nationals. 

Is the job necessary? I disagree with Yourdon when he says that the detailed understanding of the current systems in place within an organization is unnecessary.  Having been reared on SSADM, and used various other methodologies, I’ve experienced cases where a detailed understanding of the current systems has saved an IT initiative from oblivion.  Often, the model that you bring into life is the first time that the senior management of an organization has possessed a ‘helicopter –height’ panorama of the flow and transformation of data through the organization.  Often, if you take care over the presentation of the model, and communicate it well, it sparks all sorts of ideas to improve the way that the organization works; ideas that often don’t require technology to implement.

The first thing about establishing what really goes on in a company is that

  • No one person or group in the organization understands the whole general picture.
  • If you’ve got it right, then two data architects working independently will produce a very similar model. They’re uncovering something real.
  • Some people who you interview will deliberately try to  mislead you
  • A few people on the organization are so deluded as to hold entirely erratic views on what is actually going on in a company.
  • The senior management generally have only a limited grasp of what is going on in an organization. (I’ve known one or two remarkable exceptions to this)
  • The biggest struggle is to understand the broad overview. The greatest mistakes; the ones that cause enormous IT initiatives to fail, come  directly from failing to get an accurate consensus of what a company is actually doing before you start your initiative.

Why bother with the long-drawn-out business of getting the data and process architecture of an enterprise right?  Often, the data architects are seen as pedantic, doddery old veterans put safely out to grass where they can do no harm. When companies look for cuts in IT expenditure, the Data Architect is usually seen as an easy low-flying turkey to pick off. Nothing stops working when they are given the mantelpiece clock. Likewise, nothing bad happens for a while after you leap off the top of a tall cliff.  Data Architects are important. The reasons why include

  • If companies can maintain an overall understanding of their data and processes, then ‘Silo-ization’ is prevented. (Silo-ization’ refers to the IT syndrome of having several apparently effective applications that cannot inter-communicate because  the data models of each application have entirely different assumptions)
  • The data model, if properly drawn out and presented, actually helps the organization to understand what it is doing well-enough to take more intelligent decisions and understand the pressure-points.
  • With a good corporate data model, fraudulent financial practices are easier to detect
  • When IT projects are started, the vital stages of business and system analysis can kick off much faster as the preliminary work will already be done.
  • Staff training and induction is much easier and less costly once you have a well-written and tested data model, because the data and process model is couched in business terms that are familiar to the staff of the company, and years of knowledge can be gleaned in a short time from the model.
  • Data ‘warehousing’ in all its many brands and flavors becomes much easier because there is a general agreement on the constitution of every business object.  I once worked for a bank where everyone knew what a customer was,  but it was only apparent that here was no common consensus  when time came to report on the output from two different IT projects.

Maintaining a data model is surely the most important function an IT department can perform. In any place I’ve worked that has managed the job of maintaining such a thing, life for everyone has seemed, somehow magically, more simple and the reason for this hasn’t always been obvious. In fact, whenever I’ve poured through reports of why IT projects have failed, it has always struck me that nobody has mentioned ‘lack of an enterprise-wide data model’, as a good reason for failure. Maybe the task and its end-results just seem so esoteric that it is difficult to see the connection between that and all those little C#/LINQ projects all over the place.  Maybe, as well, people have been asking the wrong question. It should be ‘Why do IT projects succeed?’. I’m sure ‘Having a good enterprise-wide view of your data systems’ must be up there.

(p.s. Two years ago, I expanded on this theme, in a rather more humorous vein, in  The Data Dialog over on Simple Talk)


Posted by Steve Jones on 11 August 2009

I tend to agree with this. It doesn't appear the good architecture or modeling provide many benefits in a single project or on a day-to-day basis, but they seem to increase the overall efficiency of all your work.

Posted by jcrawf02 on 12 August 2009

"Likewise, nothing bad happens for a while after you leap off the top of a tall cliff."

Hysterical. No idea what the rest of the article is about. Kidding, but who cares? The quote takes the field...

Seriously though, good Phought, particularly found the first set of bullet points to be spot on. The important part is figuring out which of your associates fall into which group.

Posted by Nigel Ainscoe on 13 August 2009

That's absolutely spot on. Once you know where all the data is, and why it's where it is, so much more of what's going on in the enterprise starts to become clear.

Posted by hamishmccreight on 16 August 2009

Been in IT for about 20 years and found that most people don't get this...this being that if you cannot walk through your database tables designed on paper (post-its are good !!!) OR a printed diagram and readily extract your data and satisfy all the potential data transactions...then no UI will ever compensate. UI is only to set and get data. As I say, a lot of people don't get this.

Posted by h_deering on 17 August 2009

Your article is right on.  Before my team starts any project, we map out a preliminary data model.  Maintaining is still a challenge, however, starting with a data model reaps tremendous value to the structure and future maintenance of the project.  Great articles.  I would like to see more articles exposing the huge benefits of data modeling.

Posted by michael.rosquist on 17 August 2009

Very good article. However, I find the notion of information model missing. It is very good to know where the data is and how it looks, but absolutely essential how to interpret this data as information.

Example: What does status in (G,F,N,U, S) for an insurance means information-whise?

Posted by Phil Factor on 17 August 2009

Re: Information model

Yes, apologies for not making the distinction. I was trying to keep the issues simple, and covering several models (and methodologies) under the one umbrella term. I agree entirely that a lot can go wrong at the 'information end' when trying to reconcile data! It might be a good time for someone to pull together the different approaches and methodologies into a full-length article that would help the non-specialist to appreciate why this work is so important.

Posted by austondeville on 17 August 2009

YES! Model it and Print it and Post it!

If you can't print it then draw it on $1 poster paper with a majic marker.

Posted by cs_troyk on 17 August 2009

Excellent summary of the importance of data modeling -- I'm bookmarking this as a "must-read" for management-types.

If you're not already familiar with Simsion's research and writing on this subject, you may be interested in his website and books (not sure if he still actively travels for seminars): http://www.simsion.com.au/ (see "Data Modeling" section).

Leave a Comment

Please register or log in to leave a comment.