SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 


Managing Your IT Data


Managing Your IT Data

Author
Message
Steve Jones
Steve Jones
SSC Guru
SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)

Group: Administrators
Points: 545540 Visits: 20703
Comments posted to this topic are about the item Managing Your IT Data

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Andy Leonard
Andy Leonard
SSCertifiable
SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)SSCertifiable (7.5K reputation)

Group: General Forum Members
Points: 7547 Visits: 1127
Enterprise metadata is important. Unfortunately, you learn that (or you will) when something goes wrong. I wrote about using SSIS to collect enterprise metadata at http://sqlblog.com/blogs/andy_leonard/archive/2008/08/28/ssis-design-pattern-collect-enterprise-sql-server-database-metadata-with-ssis.aspx. That article was prompted by an email from someone doing what I've needed to do before: detect changes in the source system schema before it broke the ETL.

On storing data in flat files: I've heard the argument that all data is stored in flat files.

An anecdote: My first database application used Access (and I'm admitting that here for all to see). But Access wasn't fast enough to lookup and return the data I needed. I needed screens to "pop." So I wrote my own data store using ini files. I created numeric indices using the keys of key/value pairs.

It flew.

:{> Andy

Andy Leonard, Chief Data Engineer, Enterprise Data & Analytics
agatti
agatti
Mr or Mrs. 500
Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)Mr or Mrs. 500 (514 reputation)

Group: General Forum Members
Points: 514 Visits: 241
We use LogClarity, by LogFidelity, to do the same thing. It gathers and stores raw log data, including SQL server traces, but also separately stores the data after it has been correlated and cleaned up. It has reporting and alerts and the data are stored in SQL Server. http://www.logfidelity.com/



Carla Wilson-484785
Carla Wilson-484785
SSCarpal Tunnel
SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)SSCarpal Tunnel (4.2K reputation)

Group: General Forum Members
Points: 4222 Visits: 1951
Andy Leonard (1/20/2009)
Enterprise metadata is important. Unfortunately, you learn that (or you will) when something goes wrong. I wrote about using SSIS to collect enterprise metadata at http://sqlblog.com/blogs/andy_leonard/archive/2008/08/28/ssis-design-pattern-collect-enterprise-sql-server-database-metadata-with-ssis.aspx. That article was prompted by an email from someone doing what I've needed to do before: detect changes in the source system schema before it broke the ETL.

:{> Andy

Andy, I don't know if it is just my computer, but a lot of the images in your article seem to be broken.
Dan Guzman - Not the MVP
Dan Guzman - Not the MVP
SSCrazy
SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)SSCrazy (2.9K reputation)

Group: General Forum Members
Points: 2919 Visits: 779
It's just you. Images are fine for me.
catwalker63
catwalker63
Valued Member
Valued Member (52 reputation)Valued Member (52 reputation)Valued Member (52 reputation)Valued Member (52 reputation)Valued Member (52 reputation)Valued Member (52 reputation)Valued Member (52 reputation)Valued Member (52 reputation)

Group: General Forum Members
Points: 52 Visits: 43
Carla,

I'm not having any trouble with Andy's images either.
Steve Jones
Steve Jones
SSC Guru
SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)SSC Guru (545K reputation)

Group: Administrators
Points: 545540 Visits: 20703
Working fine in the UK.

Follow me on Twitter: @way0utwest
Forum Etiquette: How to post data/code on a forum to get the best help
My Blog: www.voiceofthedba.com
Wayne Evans-440401
Wayne Evans-440401
Right there with Babe
Right there with Babe (731 reputation)Right there with Babe (731 reputation)Right there with Babe (731 reputation)Right there with Babe (731 reputation)Right there with Babe (731 reputation)Right there with Babe (731 reputation)Right there with Babe (731 reputation)Right there with Babe (731 reputation)

Group: General Forum Members
Points: 731 Visits: 274
Andy

might it be some browsers can't handle multiple heights and widths?

from the bottom image:

<img style="WIDTH:608px;HEIGHT:1614px;" height="1614" src="http://vsteamsystemcentral.com/images/ext/EntDBMetadata_31.png" width="608">

Chrome, IE9, FF for windows handle it OK (UK too)



Wayne

Did you get access denied? Great the security works.

Ross McMicken
Ross McMicken
Hall of Fame
Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)Hall of Fame (3.5K reputation)

Group: General Forum Members
Points: 3459 Visits: 2281
I know some folks using Splunk for log analysis on enterprise internet connections that generate 150GB+ of log data per day. So far, they are pretty happy with the tool, but the learning curve to full usability is steep. A limited use version is available for free download, so it's pretty easy to play with.
j.a.c
j.a.c
SSCommitted
SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)SSCommitted (1.9K reputation)

Group: General Forum Members
Points: 1857 Visits: 4538
Splunk is an excellent tool for mining tons of data (our systems log hundreds of Gb per day) and for correlating events across different devices/apps/log sources. While Splunk can make useful reports and dashboards it is not a BI platform. Now if only I can convince my superiors of this fact...
Go


Permissions

You can't post new topics.
You can't post topic replies.
You can't post new polls.
You can't post replies to polls.
You can't edit your own topics.
You can't delete your own topics.
You can't edit other topics.
You can't delete other topics.
You can't edit your own posts.
You can't edit other posts.
You can't delete your own posts.
You can't delete other posts.
You can't post events.
You can't edit your own events.
You can't edit other events.
You can't delete your own events.
You can't delete other events.
You can't send private messages.
You can't send emails.
You can read topics.
You can't vote in polls.
You can't upload attachments.
You can download attachments.
You can't post HTML code.
You can't edit HTML code.
You can't post IFCode.
You can't post JavaScript.
You can post emoticons.
You can't post or upload images.

Select a forum








































































































































































SQLServerCentral


Search