Click here to monitor SSC
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Problems In Building a Data Warehouse


Problems In Building a Data Warehouse

Author
Message
Loner
Loner
SSCrazy
SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)

Group: General Forum Members
Points: 2918 Visits: 3331
Comments posted here are about the content posted at http://www.sqlservercentral.com/columnists/jchan/2832.asp
Richard Gardner-291039
Richard Gardner-291039
SSC Veteran
SSC Veteran (231 reputation)SSC Veteran (231 reputation)SSC Veteran (231 reputation)SSC Veteran (231 reputation)SSC Veteran (231 reputation)SSC Veteran (231 reputation)SSC Veteran (231 reputation)SSC Veteran (231 reputation)

Group: General Forum Members
Points: 231 Visits: 421

You got that right.

I've built a data warehouse (after a fashion). It's brilliant. I can predict stock costs for a year to within 1.8% of actuals.

Unfortunately the users don't really care.

As I see it the problem is largely the 80:20 rule. Top management are really only interested in the first 80% of the implementation. Once your sales orders are running through the system and you're controlling inventory OK (ish) they just don't care. So the onus for the last 20% is between IT and the business units themselves.

What I've found is that actually the people I am trying to work with to resolve data quality issues just don't care. They spend two hours a day fiddling with spreadsheets and take great pride in their accomplishments. These spreadsheets are used for top management reporting, and despite the fact that they aren't accurate another lesson I've learned is that top management DON'T ACTUALLY CARE because information at this level isn't reconciled to anything by anyone except me.

The amount of bull information flowing around our organisation is frightening, but as long as it is delivered people think they've done their job.

So when I give the user a report reconciling stock reports with sales (which show the flaws in our stock and shop floor reporting procedures) the user just ignores it.

The fact I can prove a 10% deviation between our production, sales, stock take and finance systems is irrelevant because nobody wants to know.

So, in reply to your article, here's what I learned :

1) Very few users can be trusted to tell you what their information requirements are because they see it as infringing on their job, which they perceive to be wasting hours creating bull spreadsheets.

2) It is extremely difficult for IT to resolve data problems themselves, if you try and take on this function (as I did) you end up doing everyone else's job for them, I actually did the stock take myself for a couple of months.

3) ANY manual spreadsheet of any size is AT LEAST 10% inaccurate, but is never checked. Manual errors tend to be small but cumulative.

4) The nature of computerised errors is different - they tend to be generally very accurate but miss the odd bit of info off. This is very easy for users to spot, in my case my reports are 95% accurate but as the users spot the errors quickly they don't trust them.

5) It may just be my users (they are pretty dumb) but they refuse to download a 95% correct report into a spreadsheet and correct it, they would rather create a manual spreadsheet every day. Despite the fact that it is inaccurate and takes 90 minutes longer.

I suspect they just like looking busy.

I give up. I am buying a goat farm in the Hebrides.


RonKyle
RonKyle
Ten Centuries
Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)Ten Centuries (1.3K reputation)

Group: General Forum Members
Points: 1290 Visits: 3344
This article doesn't even begin to address what IT staff, the presumed readership of the article, can do to address the situation presented. Nothing here about determining measures, level of granularity, etc to help management and IT get on the same page. All we see is the problem. We all know that there's often a problem. What we need are recommended solutions and/or courses of action.



Frances L
Frances L
Old Hand
Old Hand (357 reputation)Old Hand (357 reputation)Old Hand (357 reputation)Old Hand (357 reputation)Old Hand (357 reputation)Old Hand (357 reputation)Old Hand (357 reputation)Old Hand (357 reputation)

Group: General Forum Members
Points: 357 Visits: 299
the title is Problems In Building a Data Warehouse.
Mike Dillon
Mike Dillon
SSC Rookie
SSC Rookie (28 reputation)SSC Rookie (28 reputation)SSC Rookie (28 reputation)SSC Rookie (28 reputation)SSC Rookie (28 reputation)SSC Rookie (28 reputation)SSC Rookie (28 reputation)SSC Rookie (28 reputation)

Group: General Forum Members
Points: 28 Visits: 38
Good article on some of the things DBA's should look out for when management approaches them. I think the main point here is that often, DBA's don't fully understand what data warehousing entails. As a result they often find themselves in this position of letting management set false expectations without being able to educate them as to the steps and participation required from the business side.
joseph dolan
joseph dolan
Forum Newbie
Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)Forum Newbie (3 reputation)

Group: General Forum Members
Points: 3 Visits: 35

the most important word was "incrementally".

start small. simple procedure to keep a modest snowflake going (sales, custs, prods, time ... 4 tables). do NOT jump right into Analysis Serv. and all that.

then, start showing some users something. you will sometimes amaze them with just the basics. i find that a nice little ad hoc qry tool that they can pound on the snowflake with reaps great rewards. little by little the light bulbs start going on. then you can begin the great adventure

and about the 80/20 rule...it's about ROI. 20% of the effort gets 80% of the reward. so do some little things first. just to get users to open their eyes.

and i don't mean to belittle them. but they have so few tools at their disposal in most companies outside of the ERP.


David Lu
David Lu
SSC-Enthusiastic
SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)SSC-Enthusiastic (192 reputation)

Group: General Forum Members
Points: 192 Visits: 135

Hello Janet Wong:

This is my first time to read your articles. However, after reading it, I begin to read all your previous publishings. They are full with passion from reality and I can not agree with any more....

Keep posting your great work!

thanks

David





DavidSimpson
DavidSimpson
SSC Eights!
SSC Eights! (982 reputation)SSC Eights! (982 reputation)SSC Eights! (982 reputation)SSC Eights! (982 reputation)SSC Eights! (982 reputation)SSC Eights! (982 reputation)SSC Eights! (982 reputation)SSC Eights! (982 reputation)

Group: General Forum Members
Points: 982 Visits: 1074

Very good, to-the-point article. I'm working on a DW project now and it's good to be reminded of some of the issues we will be facing or already have faced.

One thing to point out is that a DW project is reverse of a normal project. Basically we need to deliver something first, then go back and figure out all of the requirements... very incremental process as Joseph pointed out. This is because the users will not know their requirements when you start... they may think they do but once you deliver some data for them to work with they will always change their minds.

David





Me Da Man
Me Da Man
Grasshopper
Grasshopper (24 reputation)Grasshopper (24 reputation)Grasshopper (24 reputation)Grasshopper (24 reputation)Grasshopper (24 reputation)Grasshopper (24 reputation)Grasshopper (24 reputation)Grasshopper (24 reputation)

Group: General Forum Members
Points: 24 Visits: 6

Richard's reponse is kind of funny to me. My organization is the exact opposite. EVERYONE except IT is constantly worrying about data inaccuracy and trying to remove it. For anyone inside IT, unless the final revenue number is wrong, they are loath to change things. It's gotten so bad that our finance department has to build the data warehouse. Yes, I know it is crazy to have a full fledge developer in the IT department, but sometimes you need to do crazy things to get people's attention. (FYI, we are working on plans for moving the development into IT.)

Anyways, Richard, let me know if you are looking for a new job in the next month or two and are interested in taking on a significant role in a DW project. Our company needs IT people with your mindset!


Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum

































































































































































SQLServerCentral


Search