October 26, 2008 at 2:07 pm
Comments posted to this topic are about the item On-Call Duties
October 27, 2008 at 5:26 am
An excellent article and thank you for sharing it with us.
It would be great if you could also make available the Excel document that you describe.
Cheers,
October 27, 2008 at 6:57 am
Excellent article...Thanks
KU
October 27, 2008 at 7:26 am
I would suggest taking it another step. It may require some dreaded overtime to get in place but is well worth it. We are dba's not secretaries or monkeys.
Automate it. Then the data can be stored into a repository automatically and you can use ssrs or excell or whatever to view your trends and histories.
October 27, 2008 at 9:00 am
Robert Hermsen (10/27/2008)
I would suggest taking it another step. It may require some dreaded overtime to get in place but is well worth it. We are dba's not secretaries or monkeys.Automate it. Then the data can be stored into a repository automatically and you can use ssrs or excell or whatever to view your trends and histories.
i agree. I love to automate things, and this is one that dreadfully needs it.
thanks for the comment and reminder to us all to automate.
October 27, 2008 at 9:57 am
Excellent article and particularly the point related to Checking on the Backups.
A lot of times DBAs or System Admins fail to check on the backup status only to find out they have to restore at a point way back in time which leads to unnecessary inconveniences at all levels of an organization.
The only point I would like to add is that a DBA should not only check for successful backups but also periodically perform a restore using a backup set in order to ensure that the backups are indeed intact - furthermore this also helps being compliant when it's time for the company's audit!
October 27, 2008 at 10:46 am
To me these are things that are daily checks, things you need to make sure are available if you get called when you're on-call. Issues should be documented and also raised as some sort of turnover for the next person to be aware of.
My goal has always been to automate these things, but also to proactively manage the systems, looking to catch things before they fail so that you're not getting called!
October 27, 2008 at 11:21 am
Steve Jones - Editor (10/27/2008)
To me these are things that are daily checks, things you need to make sure are available if you get called when you're on-call. Issues should be documented and also raised as some sort of turnover for the next person to be aware of.My goal has always been to automate these things, but also to proactively manage the systems, looking to catch things before they fail so that you're not getting called!
I can see automating some of the items i have detailed. But you are correct, some manual intervention still needs too occur, to ensure that your systems are in proper working order.
Something that can gather and store the historical data would be nice. Automating some of the data gathering would be nice. But having a human DBA look at it, still seems like a necessity.
Another DBA here just went on call this week, and thought to grab snapshots with SQL Compare as part of his on0call duties. Nice addition. If something goes wonky during the week, we have a snapshot to compare with...
October 27, 2008 at 11:59 am
tjaybelt (10/27/2008)
Steve Jones - Editor (10/27/2008)
To me these are things that are daily checks, things you need to make sure are available if you get called when you're on-call. Issues should be documented and also raised as some sort of turnover for the next person to be aware of.My goal has always been to automate these things, but also to proactively manage the systems, looking to catch things before they fail so that you're not getting called!
I can see automating some of the items i have detailed. But you are correct, some manual intervention still needs too occur, to ensure that your systems are in proper working order.
Something that can gather and store the historical data would be nice. Automating some of the data gathering would be nice. But having a human DBA look at it, still seems like a necessity.
Another DBA here just went on call this week, and thought to grab snapshots with SQL Compare as part of his on0call duties. Nice addition. If something goes wonky during the week, we have a snapshot to compare with...
On the same vein of automating, one useful tool is an automated restore of your backups on another server.
That assumes you have another server(s) available - in our case we have a customer log-shipping of LitetSpeed backups to off-site DR servers. This is configured to keep those off-site DB servers 1 hour behind the live servers (which CAN be invaluable when a junior DBA says ... "oooops!!" when asked to do an update). In conjunction with MOM reporting set to alert (via SMS and E-mail) on failures, we have automated, alerting knwoeldge of the viability of our backups.
Needless to say, several of the other obviously automatable tasks (like the space available etc.) are also configured to send MOM alerts on threshold values (typically at 80%, 90 %, 95% and 99% full - so no server should run out of space without at least 4 alerts).
This does add an additional check on the "daily checks" list however - make sure MOM is up and working 😀
October 27, 2008 at 12:06 pm
a benefit to storing the data is that in lieu of alerting on 80-90 % drive utilization you can alert on % changed. It allows you to catch when suddenly your drive with 3% monthly growth suddenly changes to 15 % quickly shortening your window.
October 27, 2008 at 12:26 pm
Robert Hermsen (10/27/2008)
a benefit to storing the data is that in lieu of alerting on 80-90 % drive utilization you can alert on % changed. It allows you to catch when suddenly your drive with 3% monthly growth suddenly changes to 15 % quickly shortening your window.
completely agree. The customer I am at makes use of Quest's Capacity Manager, so we get a GUI on top of that data, but a homegrown version can be as good (or better, as it can be built into any alerting system, whereas Cap Man is outside the alerting structure we have here - we can't see a referenece to "alert sent" for example).
Whether using Cap Man, or a similar tool, or a home grown tool - the importance of predicting, and then measuring, data growth is often under-estimated.
October 27, 2008 at 8:59 pm
The duties seem to be DBA routine/regular tasks. It is nice to list them out and have the problems/changes tracked and documented. A very good and detailed list.
Just curious, if the DBA is not on-call, what do they do in your shop? As my experience, on-call is to face and fix whatever problem comes after the business hours.
October 28, 2008 at 1:55 am
Vivien Xing (10/27/2008)
The duties seem to be DBA routine/regular tasks. It is nice to list them out and have the problems/changes tracked and documented. A very good and detailed list.Just curious, if the DBA is not on-call, what do they do in your shop? As my experience, on-call is to face and fix whatever problem comes after the business hours.
at our shop, we have a series of tasks dolled out to each of us. when not on-call, that dba gets to work on projects or tasks that actually matter.
October 28, 2008 at 10:42 am
A pretty good article. However things related are more of day-to-day activates that a DBA performs or has performed by automated tasks.
For going on-call I would suggest something more like some of the task I have to perform to prepare for going on-call:
- reviewing the previous weeks DBA callouts
- reviewing the previous weeks turnover minutes for:
- production support
- backups
- network
- resetting the DBA 'hotline' call forwarding to you phone
- changing your cell phone ringing/paging profiles
- daily changing of your work phone to call-forward to your cell phone during off hours
- for Blackberry users
- changing profiles
- changing mail folders forwarded
- attending daily/weekly turnover meeting
- help desk notification for automated tracking systems
RegardsRudy KomacsarSenior Database Administrator"Ave Caesar! - Morituri te salutamus."
October 28, 2008 at 11:12 pm
Excellent article ...:)
Viewing 15 posts - 1 through 14 (of 14 total)
You must be logged in to reply to this topic. Login to reply