Click here to monitor SSC
SQLServerCentral is supported by Red Gate Software Ltd.
 
Log in  ::  Register  ::  Not logged in
 
 
 

Archiving

By Steve Jones,

Not many database systems get designed with a well thought out archiving plan at the beginning of their lifecycle. In most cases that's fine since few databases seem to gather enough data to require archiving, and if there is extra time early in the life cycle of an application, it's probably better spent tuning queries for efficiency anyway.

However when archiving is implemented, it can't be done so in a cumbersome way. That almost always leads to some sort of data hoarding by individuals, which often leads to an application being built on Excel rather than SQL Server. This article talks about the problem of data access from a storage point of view, but it could easily apply to databases. If we require some special function, or intervention by the DBA to allow access to archived data, it's quite possible that users will take matters into their own hands.

As SQL Server has grown, we have ended up with a number of features that should make the archival of data much easier. Partitioned views and tables in many cases can allow an application to function with understanding that the underlying data is stored in multiple locations, and potentially even multiple systems. It does take a little more administrative work to seamlessly implement these features, but it's a skill that DBAs should be able to make work smoothly.

The problem of large data sets will continue to grow over time, and it's a problem I suspect most DBAs will face at some point in their career. Take some time to learn about partitioning and how you can both manage data more efficiently over time, as well as improve performance if you find access patterns vary for different sets of data.

It's also a good reason to ensure that your applications don't ever expect to pull back an entire table for any reason. Working with a few rows of data at a time is prudent when you have thousands of rows and essential when you have billions.

Steve Jones


The Voice of the DBA Podcasts

Everyday Jones

The podcast feeds are available at sqlservercentral.mevio.com. Comments are definitely appreciated and wanted, and you can get feeds from there. Overall RSS Feed: or now on iTunes!

Today's podcast features music by Everyday Jones. No relation, but I stumbled on to them and really like the music. Support this great duo at www.everydayjones.com.

You can also follow Steve Jones on Twitter:

Total article views: 402 | Views in the last 30 days: 7
 
Related Articles
FORUM

Archive database

Archive database

FORUM

Archive/Split a large SQL database and should be accessiable

Able to access both the present data and Archived/split database

FORUM

Data Archiving in replicated database

Data Archiving in replicated database

FORUM

Database Archiving Tool

Database Archiving Tool

FORUM

Reporting on archive database

Report data from active and archive database

Tags
archiving    
editorial    
 
Contribute

Join the most active online SQL Server Community

SQL knowledge, delivered daily, free:

Email address:  

You make SSC a better place

As a member of SQLServerCentral, you get free access to loads of fresh content: thousands of articles and SQL scripts, a library of free eBooks, a weekly database news roundup, a great Q & A platform… And it’s our huge, buzzing community of SQL Server Professionals that makes it such a success.

Join us!

Steve Jones
Editor, SQLServerCentral.com

Already a member? Jump in:

Email address:   Password:   Remember me: Forgotten your password?
Steve Jones