SQL Clone
SQLServerCentral is supported by Redgate
 
Log in  ::  Register  ::  Not logged in
 
 
 

Common Data Challenges

By Steve Jones,

Working with data has proven to be a challenge for most of my career. It's been fun, and certainly fulfilling, but there are constant challenges involved. Let's take away the hardware and admin challenges of keeping systems running, backed up, and performing well. I'm thinking today of the struggles of just data.

There is a short article that talks about three common data issues, and these are some of the same ones I've struggled with for most of my career. Are these challenges any different than they were 20 or 30 years ago? I'm not sure, and I was working with databases and software nearly 30 years ago.

ETL is a constant challenge, even today with tools like SSIS and Biml that make it much easier to build flows that migrate data from one database to another. ETL is such a challenge that many people make a very comfortable living helping organizations meet their every changing needs to move and prepare information for end users.

The other challenges noted in the article are getting a complete data picture because of missing data and not trusting or believing in data. The latter hasn't been as much of a problem for me.  I might describe it differently as more often we aren't sure what weight to place on certain data. The world is messy, and often we collect data that we think might be valuable only to realize later that it doesn't mean what we thought or our our hypothesis was incorrect in the first place.

I think the challenges are part of what makes this work interesting. Our employers and clients might view the effort and time involved as frustrating, and I wish I had solutions to make our process quicker and smoother. Actually, I do think that advances like SSIS have made things quicker, but the world has grown more complex. We deal with more data from more systems, in the still chaotic, messy formats of the world.

What are your challenges with data today? Are they getting better, worse, or still the same as they always were?

 
Total article views: 54 | Views in the last 30 days: 54
 
Related Articles
ARTICLE

Disaster in the Real World - Hurricane

Hurricane Charlie had a pretty substantial impact on residents and businesses in Central Florida. An...

ARTICLE

A Virtual Database World

Steve Jones talks about virtualization with databases, but not in the way you might think.

BLOG

Summiting that Technical Challenge

Conquering Challenges I really like the topic of the TSQLTuesday party today. Today we have bee...

ARTICLE

T-SQL Challenge #1

Do you want to improve your T-SQL skills? MVP Jacob Sebastian runs regular challenges to get you to ...

ARTICLE

Thinking Time

It's important to have time to think, but quite often we don't find the time. Steve Jones comments o...

Tags
editorial    
 
Contribute